Fine-tuning to other datasets using the same self-supervised paradigm

Thanks for providing such an awesome repository.

I am trying to transfer the learning ability of DINO to other datasets so that an excellent k-NN classifier can be learnt without heavy human annotations. Currently, I find the full checkpoint of ViT-S/16 does not include information of DINO head, which is important for fine-tuning in my experiments.

Is it possible to open-source the DINO head? Could you offer some suggestions for fine-tuning?

[mathildecaron31]

Hi @kaleidoscopical The head is present in the full ckpt links: https://github.com/facebookresearch/dino#pretrained-models

You can start from these weights by downloading the full checkpoint and renaming it as checkpoint.pth into your experiment repository. For example for ViT-S/16:

step1: create experiment repo mkdir ssl_finetuning

step2: download full pretrained checkpoint and move it into your experiment repo wget https://dl.fbaipublicfiles.com/dino/dino_deitsmall16_pretrain/dino_deitsmall16_pretrain_full_checkpoint.pth ssl_finetuning/checkpoint.pth

step3: launch dino training, this will start from checkpoint.pth located in output_dir python -m torch.distributed.launch --nproc_per_node=8 main_dino.py --arch vit_small --data_path /path/to/imagenet/train --output_dir ssl_finetuning

Hope that helps.

[kaleidoscopical]

I wish to fine-tune my data in a self-supervised manner, so the head is necessary.