vit-base-v1-eval-epoch-maxgrad-decay-cosine

This model is a fine-tuned version of google/vit-base-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 1.2420
• Accuracy: 0.7032

Model description

Detects the 14 highest mountains in the world

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
0.0001	0.9903	51	1.0182	0.7898
0.0027	2.0	103	1.4837	0.6688
0.0076	2.9903	154	1.2528	0.7420
0.0001	4.0	206	1.2986	0.7325
0.0007	4.9903	257	1.2049	0.7261
0.0001	6.0	309	1.1404	0.7707
0.0	6.9903	360	1.1531	0.7675
0.0	8.0	412	1.1605	0.7643
0.0	8.9903	463	1.1647	0.7643
0.0	10.0	515	1.1668	0.7675
0.0	10.9903	566	1.1690	0.7707
0.0	12.0	618	1.1702	0.7739
0.0	12.9903	669	1.1707	0.7739
0.0	14.0	721	1.1711	0.7739
0.0	14.8544	765	1.1710	0.7739

Framework versions

• Transformers 4.46.3
• Pytorch 2.5.1+cu121
• Datasets 3.1.0
• Tokenizers 0.20.3