For how long does the meta batch data loader iterate?

Usually in python iterators stop when the StopIteration exception is raised.

But I saw that the length of the data loader is a strange number (where I expected infinity since it’s usually up to the user how many episodes they want to do, which usually corresponds to a batch of tasks sampled).

So when does the data loader stop?

Code I am referencing is from the example: https://github.com/tristandeleu/pytorch-meta/blob/master/examples/maml-higher/train.py

    # Training loop
    with tqdm(dataloader, total=args.num_batches) as pbar:
        for batch_idx, batch in enumerate(pbar):
            model.zero_grad()

is args.num_batches the same as the number of episodes?

the weird size I mentioned:

        print(len(dataloader)) # prints 446400

[tristandeleu]

Like any vanilla PyTorch dataloader, the dataloader has size len(dataset) // batch_size, where len(dataset) is the total number of tasks (C(4112, 5) for 5-way Omniglot). That 446400 is indeed surprising, because when I tried print(len(dataloader)) in the maml-higher example, I get 610069224856650 which looks reasonable.

However since the dataloader is a combinatorially large, it is not recommended to loop over the whole dataloader (and reaching the StopIteration exception as you mention). That’s why you have the args.num_batches argument in the example, which loops over args.num_batches batches only (it breaks here).

I am closing this issue, because the example is working as intended. Feel free to re-open it if you still get len(dataloader) == 446400.

[tristandeleu]

The random hash trick I was talking about would be at the level of the Task (see also here for the parent class), so this would require rewriting the Sinusoid/SinusoidTask. I don’t think there is an easy way to monkeypatch the existing dataset to add the random hash.

If having seed=None does the trick for you that’s good! I originally thought what you wanted to do was orthogonal to not having a fixed seed, which would have required the random hash trick.