Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[DataPipe] DataPipe Deprecation Tracker

See original GitHub issue

We have a number of DataPipes that are being deprecated. Our general policy is that we first mark the DataPipe as deprecated with a warning, and wait at least one release cycle (~3 months) before removing it. Note that some DataPipes will be removed from the PyTorch Core library but will remain in TorchData, and some others are renamed.

`IterDataPipe` Tracker

Status Types: Deprecated - marked as deprecated with a warning Removed - removed from repository

Name	Functional API	Module	Deprecation Date	Status	Earliest Removal Version
BucketBatcher	NA	Core	Sep 30th, 2021	Removed (moved to TorchData)
HTTPReader	NA	Core	Sep 30th, 2021	Removed (moved to TorchData)
LineReader	NA	Core	Sep 30th, 2021	Removed (moved to TorchData)
TarArchiveReader	NA	Core	Sep 30th, 2021	Removed (moved to TorchData)
ZipArchiveReader	NA	Core	Sep 30th, 2021	Removed (moved to TorchData)
FileLoader	NA	Core	Jan 5th, 2022	Deprecated (use FileOpener)	1.13 (Sept 2022)
FileLoader	NA	Data	Jan 5th, 2022	Removed (use FileOpener)
IoPathFileLoader	load_file_by_iopath	Data	Jan 5th, 2022	Removed (use IoPathFileOpener)
RoutedDecoder	routed_decode	Core	Jan 10th, 2022	Deprecated	1.13 (Sept 2022)
TarArchiveReader	read_from_tar	Data	Feb 22th, 2022	Deprecated (use TarArchiveLoader)	0.5 (Sept 2022)
XzFileReader	read_from_xz	Data	Feb 22th, 2022	Deprecated (use XzFileLoader)	0.5 (Sept 2022)
ZipArchiveReader	read_from_zip	Data	Feb 22th, 2022	Deprecated (use ZipArchiveLoader)	0.5 (Sept 2022)
Filter	filter	Core	1.12	Deprecated argument (drop_empty_batches)	1.14 (Nov 2022)
FSSpecFileOpener	open_files_by_fsspec	Core	0.4	`open_file_by_fsspec` is deprecated	0.6 (Nov 2022)
IoPathFileOpener	open_files_by_fsspec	Core	0.4	`open_file_by_iopath` is deprecated	0.6 (Nov 2022)

`MapDataPipe` Tracker

Nothing for now

cc: @ejguan @VitalyFedyunin @NivekT

Issue Analytics

State:
Created 2 years ago
Reactions:1
Comments:10 (6 by maintainers)

Top GitHub Comments

1reaction

ejguancommented, Mar 1, 2022

For TarArchiveReader, should we add a deprecation warning in main branch as 0.3.0 branch cut has been finished.

0reactions

ejguancommented, Nov 23, 2022

Okay. What is the preferred mechanism to decode images? Ideally I think it should be done in batches if performance is needed

It depends on if your decode_fn supports batched decoding in high performance (multithreading). Otherwise, I think it’s going to be similar to do decoding per image.

Top Results From Across the Web

Datapipe Completes Transition to 100% Green-Powered Data ...

But cloud service provider Datapipe yesterday announced that it had completed a year-long project to get all three of its U.S. data centers ......

AWS Data Pipeline FAQs - Managed ETL Service

Q: How is AWS Data Pipeline different from Amazon Simple Workflow Service? While both services provide execution tracking, handling retries and exceptions, and ......

com.northconcepts.datapipeline.job (Data Pipeline JavaDoc)

Package com.northconcepts.datapipeline.job. Running, tracking, and managing data transfer pipelines. See: Description. Interface Summary ...

Is aws datapipeline service being deprecated? - Stack Overflow

It is not being deprecated however it is now maintenance mode meaning active development has stopped and it will no longer support new ......

Data pipeline | Administering Jira applications Data Center ...

Older schema versions will be marked as 'deprecated', and may be removed ... to <base-url>/rest/datapipeline/1.0/config/optout and pass the ...