bengali_qa_model_AGGRO_bert_base_uncased

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1386
• Exact Match: 95.2857
• F1 Score: 96.3846

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 3407
• gradient_accumulation_steps: 16
• total_train_batch_size: 64
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• training_steps: 150

Training results

Training Loss	Epoch	Step	Validation Loss	Exact Match	F1 Score
6.2014	0.0053	1	6.1979	0.0	7.0506
6.1881	0.0107	2	6.1825	0.0	7.3879
6.2139	0.0160	3	6.1522	0.0	8.2521
6.1951	0.0214	4	6.1075	0.0	10.5119
6.1083	0.0267	5	6.0489	0.4511	15.3179
6.0835	0.0321	6	5.9767	2.1053	24.1412
6.0129	0.0374	7	5.8920	3.6842	32.7230
5.9993	0.0428	8	5.7951	6.5414	40.7170
5.8684	0.0481	9	5.6866	9.9248	46.7951
5.7231	0.0535	10	5.5651	13.8346	51.5365
5.6534	0.0588	11	5.4313	18.4211	55.1575
5.5657	0.0641	12	5.2856	23.2331	58.0711
5.5148	0.0695	13	5.1299	26.6165	60.2489
5.3065	0.0748	14	4.9627	29.7744	62.0035
5.1358	0.0802	15	4.7850	33.7594	64.1128
4.9002	0.0855	16	4.5951	36.9173	66.1340
4.8232	0.0909	17	4.4087	39.7744	67.8590
4.604	0.0962	18	4.2247	43.1579	69.4792
4.5291	0.1016	19	4.0460	47.1429	71.8867
4.3711	0.1069	20	3.8705	50.2256	73.7469
4.2404	0.1123	21	3.6984	52.6316	74.5106
4.1769	0.1176	22	3.5303	54.9624	75.5802
3.8464	0.1230	23	3.3656	56.4662	76.4567
3.8178	0.1283	24	3.2085	57.9699	77.2230
3.6047	0.1336	25	3.0549	60.3008	78.0660
3.4466	0.1390	26	2.9061	61.8797	78.4774
3.3154	0.1443	27	2.7639	63.8346	79.6681
3.3505	0.1497	28	2.6242	66.1654	80.4722
3.0315	0.1550	29	2.4883	67.3684	81.0899
2.8796	0.1604	30	2.3571	68.9474	82.0043
2.9183	0.1657	31	2.2325	70.8271	82.8722
2.4212	0.1711	32	2.1128	71.7293	83.2822
2.283	0.1764	33	1.9975	72.2556	83.7327
2.2454	0.1818	34	1.8851	73.0075	84.1584
2.1467	0.1871	35	1.7746	74.2105	84.9934
2.1079	0.1924	36	1.6643	76.0150	86.0725
1.7214	0.1978	37	1.5538	77.6692	87.2392
2.0057	0.2031	38	1.4412	78.1203	87.5816
1.8565	0.2085	39	1.3280	79.5489	88.3833
1.8383	0.2138	40	1.2168	80.4511	89.0872
1.6629	0.2192	41	1.1118	81.0526	89.6248
1.3915	0.2245	42	1.0174	81.6541	89.5453
1.3512	0.2299	43	0.9330	82.3308	89.6999
1.1958	0.2352	44	0.8608	82.4812	89.8430
1.0808	0.2406	45	0.8000	82.7820	89.7950
1.0992	0.2459	46	0.7469	83.0075	89.8987
0.8846	0.2513	47	0.6994	83.3083	90.0405
0.9159	0.2566	48	0.6537	84.5113	90.4492
0.7867	0.2619	49	0.6108	85.1128	90.7277
0.849	0.2673	50	0.5724	86.2406	90.9660
0.7173	0.2726	51	0.5379	86.7669	91.2355
0.8123	0.2780	52	0.5058	87.3684	91.5345
0.6065	0.2833	53	0.4770	87.3684	91.4719
0.5135	0.2887	54	0.4495	87.6692	91.4269
0.809	0.2940	55	0.4236	88.3459	91.6852
0.5281	0.2994	56	0.3990	88.6466	91.8734
0.5029	0.3047	57	0.3780	89.2481	92.0989
0.5069	0.3101	58	0.3593	89.6241	92.2735
0.4163	0.3154	59	0.3425	90.1504	92.6192
0.4271	0.3207	60	0.3275	90.8271	92.8536
0.385	0.3261	61	0.3087	91.3534	93.0501
0.365	0.3314	62	0.2891	91.5038	93.1793
0.3785	0.3368	63	0.2729	91.8797	93.3364
0.1751	0.3421	64	0.2598	92.2556	93.5858
0.3697	0.3475	65	0.2499	92.7068	93.9267
0.4384	0.3528	66	0.2413	92.9323	94.1523
0.2969	0.3582	67	0.2327	93.0075	94.2276
0.3493	0.3635	68	0.2251	93.3083	94.4743
0.1457	0.3689	69	0.2182	93.4586	94.5879
0.6827	0.3742	70	0.2096	93.6842	94.7584
0.4472	0.3796	71	0.2018	93.7594	94.8085
0.1831	0.3849	72	0.1945	93.6842	94.6772
0.2381	0.3902	73	0.1891	93.7594	94.6731
0.1892	0.3956	74	0.1830	93.8346	94.7656

Framework versions

• Transformers 4.46.3
• Pytorch 2.4.0
• Datasets 3.1.0
• Tokenizers 0.20.3