Legacy interface for prototype datasets?

The prototype datasets change the interface in two ways:

1. The input parameters are a little different in some cases. For example in the current API the MNIST dataset would be instantiated with datasets.MNIST(..., train=True) whereas now it looks like datasets.load("mnist", split="train").
2. The output is completely different. Before we returned a tuple (sometimes of varying length) whereas now we always return a dictionary. Furthermore, before we used PIL images and numpy arrays as return types, whereas now we always use tensor subclasses.

To lower the burden to move to the new style datasets a little, we could have a legacy: bool = False keyword argument on datasets.load(). For all datasets that have a legacy variant, we could simply implement two functions that map the input and output. If a dataset doesn’t support a legacy variant, we could simply error out.

cc @pmeier @bjuncek

[datumbox]

Yeah I think this is a critical detail that needs to be highlighted here. If what we get from the dataset is different, the user still needs to make a bunch of changes on their code to handle this. It’s not just the X part of the training data pair; it might be necessary handling the Y as well.

Given the above, I’m not sure if writing extra code to make things look different is worth it, given that the APIs need to be handled differently (due to their return types). Instead, one could argue that we should be putting enough good features in the new API, solving common user issues to make the migration worth it.

I would love to hear what others think on this. @NicolasHug @prabhat00155 @fmassa?

[NicolasHug]

Overall I share @datumbox points.

I don’t think it’s worth the hassle and extra complexity of maintaining 2 sets of API. If anything, I feel like this would actually hurt the migration, because some users would just do half of it instead of doing it all. and if/when we remove support for the legacy, they would have to apply a second set of change.

Instead, one could argue that we should be putting enough good features in the new API, solving common user issues to make the migration worth it

I was going to comment something along these lines before reading it