beit-base-patch16-224-RD

This model is a fine-tuned version of microsoft/beit-base-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.3771
• Accuracy: 0.8673

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
1.4986	0.99	40	1.4512	0.4945
1.0553	1.99	80	0.9355	0.7473
0.7972	2.98	120	0.7250	0.7436
0.7156	4.0	161	0.5845	0.7582
0.6723	4.99	201	0.5509	0.8036
0.5942	5.99	241	0.5018	0.8218
0.6223	6.98	281	0.4993	0.8218
0.5731	8.0	322	0.4590	0.8291
0.5583	8.99	362	0.4878	0.8
0.5784	9.99	402	0.4485	0.8455
0.4968	10.98	442	0.4305	0.8345
0.5324	12.0	483	0.4737	0.8345
0.4629	12.99	523	0.4253	0.8436
0.4398	13.99	563	0.4184	0.8473
0.4575	14.98	603	0.3929	0.8564
0.4554	16.0	644	0.4282	0.8491
0.4646	16.99	684	0.4363	0.8236
0.4535	17.99	724	0.4337	0.8455
0.3823	18.98	764	0.3771	0.8673
0.4584	20.0	805	0.3966	0.8564
0.4103	20.99	845	0.4001	0.8491
0.3659	21.99	885	0.3948	0.8582
0.3241	22.98	925	0.4007	0.8582
0.3575	24.0	966	0.4328	0.8327
0.3411	24.99	1006	0.3990	0.8564
0.3829	25.99	1046	0.4011	0.8636
0.2855	26.98	1086	0.3859	0.8655
0.254	28.0	1127	0.4196	0.8673
0.2937	28.99	1167	0.4340	0.8618
0.258	29.99	1207	0.4387	0.8509
0.2735	30.98	1247	0.4097	0.8655
0.2674	32.0	1288	0.4183	0.8527
0.2547	32.99	1328	0.4217	0.8636
0.2109	33.99	1368	0.4240	0.8527
0.2248	34.98	1408	0.4250	0.86
0.2397	36.0	1449	0.4431	0.8582
0.1823	36.99	1489	0.4442	0.8582
0.1834	37.99	1529	0.4362	0.8618
0.1864	38.98	1569	0.4338	0.8545
0.1779	39.75	1600	0.4332	0.8582

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu118
• Datasets 2.16.1
• Tokenizers 0.15.0