facebook

/

dinov2-base

like 77

Hi,

DINOv2 is supported by the official notebook for image classification as well as the scripts: https://github.com/huggingface/transformers/tree/main/examples/pytorch/image-classification (which showcase how to fine-tune the model using either the Trainer API or using the Accelerate library)

I have satellite image data, and I want to improve feature extraction through fine-tuning. Since I don't have labels for the images, training with image classification is not possible. Is it correct to fine-tune using only the loss function in this case? If so, what would the loss function be?

In case you want to improve feature extraction and have no labels, I would recommend self-supervised learning.

For that I'd recommend using the original repository: https://github.com/facebookresearch/dinov2/tree/main. You can then, after training, convert the model to the Transformers API using the conversion script: https://github.com/huggingface/transformers/blob/main/src/transformers/models/dinov2/convert_dinov2_to_hf.py

Thank you, i will try

Hi @nielsr

I have a gray scale image set of about 7000 images without labels but highly domain specific.
Since 7000 images is likely too small for self supervised training from scratch using DINOv2 I would like to test whether I can fine tune a pretrained DINOv2 net with my images using DINOv2 SSL.
Is there a way I can do that with the hugging face implementation?

I also looked at the original implementation.
However, as stated here, further training using SSL does not seem to be supported out of the box:
https://github.com/facebookresearch/dinov2/issues/339
While the developers say further SSL training could in principle be done, I have not found the parts in the code to adjust for doing it manually, unfortunately.
Would you have a suggestion / hint regarding this problem?
Any help would be greatly appreciated!
Thank you for your time and the great hf implementation! :-)

https://github.com/csaroff/dinov2/tree/main/sky @Threepwood