Measure the model performance

Hi,

Thanks for the simpler implementation of MAML.

As per the MAML paper At the end of meta-training, new tasks are sampled fromp(T),and meta-performance is measured by the model’s perfor-mance after learning fromKsamples. Generally, tasksused for meta-testing are held out during meta-training.

Anybody has tried fine-tuning the model with few number (0 to 10) of samples for a new class which was not there in the training dataset and measured the performance?

Is that part of the code already available in this repository?

Thank you, KK

[zzpustc]

Tasks in testing phase have the same structures as those in training phase. So, in testing phase, query_set should be generated as what in training does. The optimal theta will adapt on support data in testing tasks and the performance will be measured on query data in testing tasks.

Just treat the task as data in common supervised learning. The structure of the ‘data’ should keep consistent.

@HongduanTian Hi, have you ever tried saving the model, and load model weights in an extra evaluate code? I have tried, I got ~47% accuracy when training with this repo(miniImagenet_train.py), but when I tried the operation mentioned above, I can only get ~44% accuracy. Can you provide some insights to me? Many thx!

[kk2491]

@HongduanTian Yes, also it is happening every 500 episodes. Did you use original code from here. Does that have model performance part ?

Thank you, KK