beit-base-patch16-224-dmae-va-U5-42D

This model is a fine-tuned version of microsoft/beit-base-patch16-224 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.6193
• Accuracy: 0.7

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0004
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 42

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.9	7	0.8832	0.6167
1.2709	1.94	15	0.8194	0.7
0.9705	2.97	23	0.7715	0.6
0.7033	4.0	31	1.1243	0.6333
0.7033	4.9	38	1.2102	0.55
0.7367	5.94	46	1.0123	0.65
0.5424	6.97	54	1.1218	0.6667
0.3998	8.0	62	1.3493	0.6
0.3998	8.9	69	1.3213	0.5833
0.3478	9.94	77	1.2704	0.6167
0.3467	10.97	85	1.4684	0.6167
0.2805	12.0	93	1.1832	0.7167
0.2496	12.9	100	1.3645	0.6333
0.2496	13.94	108	1.3561	0.6667
0.1855	14.97	116	1.6048	0.6167
0.2162	16.0	124	1.3662	0.6833
0.1833	16.9	131	1.6070	0.6667
0.1833	17.94	139	1.8448	0.6
0.2022	18.97	147	1.3397	0.65
0.1587	20.0	155	1.5272	0.65
0.1991	20.9	162	1.4667	0.65
0.1639	21.94	170	1.3853	0.65
0.1639	22.97	178	1.8195	0.6667
0.1364	24.0	186	1.7032	0.6167
0.1513	24.9	193	1.6734	0.65
0.1197	25.94	201	1.8673	0.6
0.1197	26.97	209	1.9885	0.6
0.1366	28.0	217	1.7329	0.6667
0.0982	28.9	224	1.6177	0.7333
0.0983	29.94	232	1.6226	0.7167
0.1131	30.97	240	1.5967	0.7
0.1131	32.0	248	1.6310	0.7
0.0694	32.9	255	1.6673	0.7333
0.0745	33.94	263	1.6257	0.7
0.058	34.97	271	1.6017	0.7167
0.058	36.0	279	1.6812	0.7
0.055	36.9	286	1.6524	0.7
0.0569	37.94	294	1.6193	0.7

Framework versions

• Transformers 4.38.2
• Pytorch 2.2.1+cu121
• Datasets 2.19.0
• Tokenizers 0.15.2