BEiT-RD-DA

This model is a fine-tuned version of microsoft/beit-base-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 1.9617
• Accuracy: 0.6655

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
1.4123	1.0	96	1.4099	0.4927
0.9503	2.0	192	1.8852	0.4927
0.8284	3.0	288	2.1702	0.5073
0.7677	4.0	384	2.0408	0.5345
0.788	5.0	480	2.7991	0.5127
0.5822	6.0	576	2.0951	0.5636
0.5172	7.0	672	2.5977	0.5364
0.4615	8.0	768	2.0968	0.58
0.3672	9.0	864	2.8535	0.5436
0.379	10.0	960	2.9515	0.5382
0.3301	11.0	1056	2.7200	0.5582
0.2786	12.0	1152	1.9000	0.6273
0.2746	13.0	1248	3.1768	0.5364
0.2298	14.0	1344	3.1003	0.5527
0.2013	15.0	1440	2.3441	0.6182
0.2225	16.0	1536	3.0214	0.5709
0.2229	17.0	1632	2.0676	0.6164
0.2024	18.0	1728	2.6478	0.5673
0.1401	19.0	1824	2.8952	0.5636
0.1984	20.0	1920	2.3083	0.6145
0.1788	21.0	2016	3.7702	0.52
0.1907	22.0	2112	1.9617	0.6655
0.1113	23.0	2208	2.6546	0.5964
0.1293	24.0	2304	2.6427	0.6036
0.1354	25.0	2400	3.4105	0.5527
0.1447	26.0	2496	2.5460	0.6127
0.0995	27.0	2592	2.9865	0.5855
0.1369	28.0	2688	3.5281	0.5545
0.1238	29.0	2784	2.8161	0.6018
0.1256	30.0	2880	3.4917	0.5491
0.1064	31.0	2976	3.0659	0.58
0.1333	32.0	3072	3.5972	0.5473
0.1134	33.0	3168	3.6116	0.54
0.0831	34.0	3264	3.5308	0.5509
0.1035	35.0	3360	3.4789	0.5582
0.0957	36.0	3456	3.6358	0.5509
0.0764	37.0	3552	3.3639	0.5709
0.072	38.0	3648	3.5639	0.5564
0.0727	39.0	3744	3.5193	0.5582
0.0619	40.0	3840	3.5836	0.5582

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu118
• Datasets 2.16.1
• Tokenizers 0.15.0