git-base-bdd100k

This model is a fine-tuned version of microsoft/git-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4069
• Wer Score: 1.7469

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 35
• eval_batch_size: 35
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 70
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer Score
10.5843	1.0	3	9.3667	6.8626
9.2401	2.0	6	8.8728	6.9596
8.7545	3.0	9	8.2995	3.9543
8.2253	4.0	12	7.8495	6.1327
7.815	5.0	15	7.4921	6.4292
7.4785	6.0	18	7.1823	5.6943
7.1794	7.0	21	6.8969	6.7444
6.9018	8.0	24	6.6233	7.2815
6.633	9.0	27	6.3577	6.2946
6.3716	10.0	30	6.0982	3.9794
6.1145	11.0	33	5.8410	1.7244
5.8587	12.0	36	5.5841	1.3375
5.6036	13.0	39	5.3301	1.2846
5.3508	14.0	42	5.0789	1.2795
5.1004	15.0	45	4.8313	1.2645
4.8522	16.0	48	4.5858	1.3498
4.607	17.0	51	4.3436	1.3841
4.363	18.0	54	4.1048	1.3598
4.1205	19.0	57	3.8677	1.3941
3.8822	20.0	60	3.6347	1.4838
3.6467	21.0	63	3.4047	1.5396
3.4146	22.0	66	3.1813	1.5850
3.1857	23.0	69	2.9609	1.5897
2.9594	24.0	72	2.7443	1.7628
2.7404	25.0	75	2.5347	1.7511
2.5251	26.0	78	2.3310	1.8832
2.3179	27.0	81	2.1342	1.8400
2.1174	28.0	84	1.9478	1.8548
1.9222	29.0	87	1.7691	1.8977
1.7389	30.0	90	1.6009	1.9543
1.5638	31.0	93	1.4450	2.1109
1.4006	32.0	96	1.3019	2.0713
1.2486	33.0	99	1.1684	2.2235
1.1087	34.0	102	1.0515	1.9969
0.9815	35.0	105	0.9437	2.1112
0.8688	36.0	108	0.8525	2.1831
0.766	37.0	111	0.7711	2.0917
0.6799	38.0	114	0.7014	2.2617
0.6043	39.0	117	0.6455	2.3133
0.5357	40.0	120	0.5936	1.9220
0.4756	41.0	123	0.5531	2.4083
0.424	42.0	126	0.5178	2.1998
0.3795	43.0	129	0.4884	2.2793
0.3425	44.0	132	0.4695	2.2614
0.3132	45.0	135	0.4543	2.1388
0.2835	46.0	138	0.4319	2.2199
0.2573	47.0	141	0.4201	1.9774
0.2339	48.0	144	0.4066	2.4571
0.2144	49.0	147	0.4033	1.7444
0.1966	50.0	150	0.3948	2.3091
0.1811	51.0	153	0.3861	1.9247
0.1687	52.0	156	0.3846	2.1204
0.1549	53.0	159	0.3833	2.0151
0.1441	54.0	162	0.3762	1.8776
0.1339	55.0	165	0.3786	1.8495
0.1231	56.0	168	0.3776	2.0407
0.1123	57.0	171	0.3757	2.0056
0.1042	58.0	174	0.3772	1.7985
0.0957	59.0	177	0.3776	1.9055
0.0881	60.0	180	0.3764	1.8409
0.0801	61.0	183	0.3811	1.9128
0.0743	62.0	186	0.3796	1.6321
0.0693	63.0	189	0.3778	1.7338
0.0634	64.0	192	0.3818	1.8191
0.0594	65.0	195	0.3834	1.7001
0.0543	66.0	198	0.3770	1.7305
0.0506	67.0	201	0.3835	1.7450
0.0466	68.0	204	0.3867	1.6380
0.0447	69.0	207	0.3875	1.7717
0.0413	70.0	210	0.3879	1.7280
0.0395	71.0	213	0.3899	1.6834
0.0366	72.0	216	0.3897	1.8994
0.0352	73.0	219	0.3913	1.8119
0.0326	74.0	222	0.3931	1.7511
0.0314	75.0	225	0.3925	1.7907
0.0298	76.0	228	0.3966	1.7662
0.0287	77.0	231	0.3940	1.7327
0.0272	78.0	234	0.3963	1.7745
0.0267	79.0	237	0.3995	1.8200
0.0253	80.0	240	0.3991	1.7899
0.0249	81.0	243	0.4008	1.7910
0.0237	82.0	246	0.3997	1.8565
0.0231	83.0	249	0.4015	1.7687
0.0225	84.0	252	0.4015	1.7093
0.0217	85.0	255	0.4012	1.7667
0.0212	86.0	258	0.4024	1.7938
0.0205	87.0	261	0.4037	1.7821
0.0203	88.0	264	0.4041	1.8085
0.02	89.0	267	0.4041	1.8275
0.0193	90.0	270	0.4051	1.7912
0.0192	91.0	273	0.4059	1.7508
0.0188	92.0	276	0.4059	1.7355
0.0185	93.0	279	0.4055	1.7322
0.0183	94.0	282	0.4055	1.7255
0.0181	95.0	285	0.4060	1.7302
0.0177	96.0	288	0.4065	1.7414
0.0174	97.0	291	0.4067	1.7483
0.0177	98.0	294	0.4069	1.7464
0.0174	99.0	297	0.4069	1.7489
0.0175	100.0	300	0.4069	1.7469

Framework versions

• Transformers 4.45.2
• Pytorch 2.5.0+cu124
• Datasets 3.0.2
• Tokenizers 0.20.1