Preprocessing module is very confusing

❓ Questions and Help

There seems to be a bit of code smell cropping up with the #770 issue related to to_onnx and how example inputs are used. I have a few questions.

What is your question?

1. How does the DataPipeline class relate to the preprocess module? The documentation for this class is very sparse.
2. Why is it the responsibility of the Preprocessor to instantiate the data source when from_data_source("something custom") is called to create a data module? This flow is very unintuitive.
3. ApplyToKeys operates on dictionaries, and example_input_array can be a tensor, tuple, or dict. Where and how is this handled? For example in to_onnx or summarize the example_input_array tensor is passed through the model, which works fine on just the model, but the preprocessing expects a dict for the most part since it is highly encouraged in the docs to use that function as everything in flash is supposed to be a dict until right before inference. This results in all kinds of strange errors.

Code

The fix for 3 seems to be to do this on my custom Task

    def _apply_batch_transfer_handler(self, batch: Any, device: Optional[torch.device] = None, dataloader_idx: Optional[int] = None) -> Any:
        if isinstance(batch, torch.Tensor):
            return super()._apply_batch_transfer_handler(batch={DefaultDataKeys.INPUT: batch}, device=device, dataloader_idx=dataloader_idx)[DefaultDataKeys.INPUT]
        else:
            return super()._apply_batch_transfer_handler(batch, device=device, dataloader_idx=dataloader_idx)

But I have no idea what kinds of side effects this would have and it feels like a strange fix to a fundamental problem.

What have you tried?

This issue sprang up when I implemented a custom preprocess class and spiraled from there. to_onnx and summarize (by extension, training) are broken due to this bug. The ImageClassificationPreprocess won’t work for me because my outputs are continuous,

What’s your environment?

• OS: All
• Packaging: conda/pip
• Version lightning 1.4.7, flash 0.5.0, bolts 0.4.0

[dlangerm]

#816

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.