How do I have torchmeta use different data examples for each meta-batch when looping through the entire dataloader?

I want to loop through a meta-dataloader multiple times (which I’ve been able to do easily, especially in regression tasks) but when I loop the second time I want the data examples to be different. Will torchmeta do that automatically?

e.g. if I have a regression problem with 200 functions and my meta-batch size is 200 then I’d get all the 200 functions each time I get 1 batch of the meta-data loader. e.g.

print('-- start analysis --')
print(f'number of workers = {args.num_workers}')
print(f'--> args.meta_batch_size = {args.meta_batch_size_eval}')
print(f'--> args.iters = {args.iters}')
print(f'--> args.nb_inner_train_steps = {args.nb_inner_train_steps}')

print(meta_dataloader.batch_size)
# looping through the data set multiple times with different examples: https://github.com/tristandeleu/pytorch-meta/issues/112
with tqdm(range(args.iters)) as pbar:
    it = 0
    while it < args.iters:
        for batch_idx, batch in enumerate(meta_dataloader):
            # batch has a batch of tasks e.g. 200 regression functions or
            print(f'it = {it}')
            # print(batch['train'][0].size(0))
            # print(batch['train'][1].size(0))
            spt_x, spt_y, qry_x, qry_y = process_meta_batch(args, batch)
            print(spt_x.mean())
            print(qry_x.mean())

I want that each time it increases by 1 that I get 200 tasks but each has different examples.

[tristandeleu]

Having the same data when sampling a task multiple times, even in regression problems, is a design choice in Torchmeta for reproducibility which I explained in #69, so that is not a bug. You are right, the random hash trick (which should be applied to the Task object, and not the dataloader) should not help in your case, because you want a fixed set of functions, but different samples for each function at every iterations (as opposed to different sets of functions).

There is no way of doing that out of the box in Torchmeta, and you would need to create your own dataset to have this features. Taking the example of Sinusoid, the simplest fix would be to not pass np_random when getting the task here: https://github.com/tristandeleu/pytorch-meta/blob/389e35ef9aa812f07ce50a3f3bd253c4efb9765c/torchmeta/toy/sinusoid.py#L84-L86 (remove np_random, or set np_random=None).

[brando90]

Having the same data when sampling a task multiple times, even in regression problems, is a design choice in Torchmeta for reproducibility which I explained in #69, so that is not a bug. You are right, the random hash trick (which should be applied to the Task object, and not the dataloader) should not help in your case, because you want a fixed set of functions, but different samples for each function at every iterations (as opposed to different sets of functions).

There is no way of doing that out of the box in Torchmeta, and you would need to create your own dataset to have this features. Taking the example of Sinusoid, the simplest fix would be to not pass np_random when getting the task here: https://github.com/tristandeleu/pytorch-meta/blob/389e35ef9aa812f07ce50a3f3bd253c4efb9765c/torchmeta/toy/sinusoid.py#L84-L86

(remove np_random, or set np_random=None).

cool, will see how this applies to my data set. Thanks for the reply!

Will reply (and hopefully close the issue once I have it working for me and let you know).