Skip to content
This repository has been archived by the owner on Oct 9, 2023. It is now read-only.

Rename Preprocess to InputTransform #951

Merged
merged 22 commits into from
Nov 9, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).

### Changed

- Changed `Preprocess` to `InputTransform` ([#951](https://github.com/PyTorchLightning/lightning-flash/pull/951))

- Changed classes named `*Serializer` and properties / variables named `serializer` to be `*Output` and `output` respectively ([#927](https://github.com/PyTorchLightning/lightning-flash/pull/927))

- Changed `Postprocess` to `OutputTransform` ([#942](https://github.com/PyTorchLightning/lightning-flash/pull/942))
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -227,7 +227,7 @@ model = ImageClassifier(backbone="resnet18", num_classes=2, optimizer="Adam", lr

Flash includes some simple augmentations for each task by default, however, you will often want to override these and control your own augmentation recipe.
To this end, Flash supports custom transformations backed by our powerful data pipeline.
The transform requires to be passed as a dictionary of transforms where the keys are the [hook's name](https://lightning-flash.readthedocs.io/en/latest/api/generated/flash.core.data.process.Preprocess.html?highlight=Preprocess).
The transform requires to be passed as a dictionary of transforms where the keys are the [hook's name](https://lightning-flash.readthedocs.io/en/latest/api/generated/flash.core.data.io.input_transform.InputTransform.html?highlight=InputTransform).
This enable transforms to be applied per sample or per batch either on or off device.
It is important to note that data are being processed as a dictionary for all tasks (typically containing `input`, `target`, and `metadata`),
Therefore, you can use [`ApplyToKeys`](https://lightning-flash.readthedocs.io/en/latest/api/generated/flash.core.data.transforms.ApplyToKeys.html#flash.core.data.transforms.ApplyToKeys) utility to apply the transform to a specific key.
Expand Down
8 changes: 4 additions & 4 deletions docs/extensions/autodatasources.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,17 +53,17 @@ def run(self):

data_module = getattr(importlib.import_module(data_module_path), data_module_name)

class PatchedPreprocess(data_module.preprocess_cls):
class PatchedInputTransform(data_module.input_transform_cls):
"""TODO: This is a hack to prevent default transforms form being created"""

@staticmethod
def _resolve_transforms(_):
return None

preprocess = PatchedPreprocess()
input_transform = PatchedInputTransform()
data_sources = {
data_source: preprocess.data_source_of_name(data_source)
for data_source in preprocess.available_data_sources()
data_source: input_transform.data_source_of_name(data_source)
for data_source in input_transform.available_data_sources()
}

ENVIRONMENT.get_template("base.rst")
Expand Down
4 changes: 2 additions & 2 deletions docs/source/api/audio.rst
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ ______________
:template: classtemplate.rst

~classification.data.AudioClassificationData
~classification.data.AudioClassificationPreprocess
~classification.data.AudioClassificationInputTransform

Speech Recognition
__________________
Expand All @@ -31,7 +31,7 @@ __________________
~speech_recognition.data.SpeechRecognitionData
~speech_recognition.model.SpeechRecognition

speech_recognition.data.SpeechRecognitionPreprocess
speech_recognition.data.SpeechRecognitionInputTransform
speech_recognition.data.SpeechRecognitionBackboneState
speech_recognition.data.SpeechRecognitionOutputTransform
speech_recognition.data.SpeechRecognitionCSVDataSource
Expand Down
7 changes: 4 additions & 3 deletions docs/source/api/data.rst
Original file line number Diff line number Diff line change
Expand Up @@ -118,12 +118,13 @@ _______________________
:nosignatures:
:template: classtemplate.rst

~flash.core.data.process.BasePreprocess
~flash.core.data.process.DefaultPreprocess
~flash.core.data.io.input_transform.BaseInputTransform
~flash.core.data.io.input_transform.DefaultInputTransform
~flash.core.data.process.DeserializerMapping
~flash.core.data.process.Deserializer
~flash.core.data.io.output_transform.OutputTransform
~flash.core.data.process.Preprocess
~flash.core.data.io.input_transform.InputTransform
~flash.core.data.process.Serializer

flash.core.data.properties
__________________________
Expand Down
2 changes: 1 addition & 1 deletion docs/source/api/flash.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ flash
~flash.core.data.data_source.DataSource
~flash.core.data.data_module.DataModule
~flash.core.data.callback.FlashCallback
~flash.core.data.process.Preprocess
~flash.core.data.io.output_transform.OutputTransform
~flash.core.data.io.output.Output
~flash.core.data.io.input_transform.InputTransform
~flash.core.model.Task
~flash.core.trainer.Trainer
2 changes: 1 addition & 1 deletion docs/source/api/graph.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ ______________
~classification.model.GraphClassifier
~classification.data.GraphClassificationData

classification.data.GraphClassificationPreprocess
classification.data.GraphClassificationInputTransform

flash.graph.data
________________
Expand Down
12 changes: 6 additions & 6 deletions docs/source/api/image.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ ______________

~classification.model.ImageClassifier
~classification.data.ImageClassificationData
~classification.data.ImageClassificationPreprocess
~classification.data.ImageClassificationInputTransform

classification.data.MatplotlibVisualization

Expand All @@ -44,8 +44,8 @@ ________________

detection.data.FiftyOneParser
detection.data.ObjectDetectionFiftyOneDataSource
detection.data.ObjectDetectionPreprocess
detection.output.FiftyOneDetectionLabels
detection.data.ObjectDetectionInputTransform

Keypoint Detection
__________________
Expand All @@ -58,7 +58,7 @@ __________________
~keypoint_detection.model.KeypointDetector
~keypoint_detection.data.KeypointDetectionData

keypoint_detection.data.KeypointDetectionPreprocess
keypoint_detection.data.KeypointDetectionInputTransform

Instance Segmentation
_____________________
Expand All @@ -71,7 +71,7 @@ _____________________
~instance_segmentation.model.InstanceSegmentation
~instance_segmentation.data.InstanceSegmentationData

instance_segmentation.data.InstanceSegmentationPreprocess
instance_segmentation.data.InstanceSegmentationInputTransform

Embedding
_________
Expand All @@ -93,7 +93,7 @@ ____________

~segmentation.model.SemanticSegmentation
~segmentation.data.SemanticSegmentationData
~segmentation.data.SemanticSegmentationPreprocess
~segmentation.data.SemanticSegmentationInputTransform

segmentation.data.SegmentationMatplotlibVisualization
segmentation.data.SemanticSegmentationNumpyDataSource
Expand Down Expand Up @@ -123,7 +123,7 @@ ______________

~style_transfer.model.StyleTransfer
~style_transfer.data.StyleTransferData
~style_transfer.data.StyleTransferPreprocess
~style_transfer.data.StyleTransferInputTransform

.. autosummary::
:toctree: generated/
Expand Down
4 changes: 2 additions & 2 deletions docs/source/api/pointcloud.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ ____________
~segmentation.model.PointCloudSegmentation
~segmentation.data.PointCloudSegmentationData

segmentation.data.PointCloudSegmentationPreprocess
segmentation.data.PointCloudSegmentationInputTransform
segmentation.data.PointCloudSegmentationFoldersDataSource
segmentation.data.PointCloudSegmentationDatasetDataSource

Expand All @@ -35,6 +35,6 @@ ________________
~detection.model.PointCloudObjectDetector
~detection.data.PointCloudObjectDetectorData

detection.data.PointCloudObjectDetectorPreprocess
detection.data.PointCloudObjectDetectorInputTransform
detection.data.PointCloudObjectDetectorFoldersDataSource
detection.data.PointCloudObjectDetectorDatasetDataSource
4 changes: 2 additions & 2 deletions docs/source/api/tabular.rst
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ ___________
~forecasting.model.TabularForecaster
~forecasting.data.TabularForecastingData

forecasting.data.TabularForecastingPreprocess
forecasting.data.TabularForecastingInputTransform
forecasting.data.TabularForecastingDataFrameDataSource
forecasting.data.TimeSeriesDataSetParametersState

Expand All @@ -58,5 +58,5 @@ __________________
~data.TabularDataFrameDataSource
~data.TabularCSVDataSource
~data.TabularDeserializer
~data.TabularPreprocess
~data.TabularOutputTransform
~data.TabularInputTransform
10 changes: 5 additions & 5 deletions docs/source/api/text.rst
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ ______________
~classification.data.TextClassificationData

classification.data.TextClassificationOutputTransform
classification.data.TextClassificationPreprocess
classification.data.TextClassificationInputTransform
classification.data.TextDeserializer
classification.data.TextDataSource
classification.data.TextCSVDataSource
Expand Down Expand Up @@ -49,7 +49,7 @@ __________________
question_answering.data.QuestionAnsweringFileDataSource
question_answering.data.QuestionAnsweringJSONDataSource
question_answering.data.QuestionAnsweringOutputTransform
question_answering.data.QuestionAnsweringPreprocess
question_answering.data.QuestionAnsweringInputTransform
question_answering.data.SQuADDataSource


Expand All @@ -64,7 +64,7 @@ _____________
~seq2seq.summarization.model.SummarizationTask
~seq2seq.summarization.data.SummarizationData

seq2seq.summarization.data.SummarizationPreprocess
seq2seq.summarization.data.SummarizationInputTransform

Translation
___________
Expand All @@ -77,7 +77,7 @@ ___________
~seq2seq.translation.model.TranslationTask
~seq2seq.translation.data.TranslationData

seq2seq.translation.data.TranslationPreprocess
seq2seq.translation.data.TranslationInputTransform

General Seq2Seq
_______________
Expand All @@ -97,5 +97,5 @@ _______________
seq2seq.core.data.Seq2SeqFileDataSource
seq2seq.core.data.Seq2SeqJSONDataSource
seq2seq.core.data.Seq2SeqOutputTransform
seq2seq.core.data.Seq2SeqPreprocess
seq2seq.core.data.Seq2SeqInputTransform
seq2seq.core.data.Seq2SeqSentencesDataSource
2 changes: 1 addition & 1 deletion docs/source/api/video.rst
Original file line number Diff line number Diff line change
Expand Up @@ -23,5 +23,5 @@ ______________
classification.data.BaseVideoClassification
classification.data.VideoClassificationFiftyOneDataSource
classification.data.VideoClassificationPathsDataSource
classification.data.VideoClassificationPreprocess
classification.data.VideoClassificationInputTransform
classification.model.VideoClassifierFinetuning
Loading