Inconsistency in defaults?

I trained dino with default settings (the vit_small arch), and tried to run the video_generation.py script to look at the results. This gave the following error:

Take key teacher in provided checkpoint dict
Traceback (most recent call last):
  File "/usr/src/app/video_generation.py", line 377, in <module>
    vg = VideoGenerator(args)
  File "/usr/src/app/video_generation.py", line 46, in __init__
    self.model = self.__load_model()
  File "/usr/src/app/video_generation.py", line 263, in __load_model
    msg = model.load_state_dict(state_dict, strict=False)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1070, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for VisionTransformer:
        size mismatch for pos_embed: copying a param with shape torch.Size([1, 197, 384]) from checkpoint, the shape in current model is torch.Size([1, 785, 384]).
        size mismatch for patch_embed.proj.weight: copying a param with shape torch.Size([384, 3, 16, 16]) from checkpoint, the shape in current model is torch.Size([384, 3, 8, 8]).

I think this is caused by training defaulting to patch_size of 16, while video generation defaults to 8. Adding --patch_size 16 to the video_generation command line seems to have fixed it.

(No big deal, of course, but thought I might as well report it.)

[mathildecaron31]

Thanks @ketil-malde However having left Meta I no longer can accept pull request.

[ketil-malde]

Just merge this patch? https://github.com/ketil-malde/dino/commit/ce2b20bb3e89c528f1ad256ee64465d027667b50

This is pretty trivial, but if I should do something to facilitate it (put it on a separate branch, create a pull request, whatever), just give me instructions, and I’m happy to oblige.