bert-30M-uncased-classification-fqa-100e

This model is a fine-tuned version of vietgpt/bert-30M-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0654
• Accuracy: 0.9897

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	110	5.2087	0.0051
No log	2.0	220	5.2019	0.0
No log	3.0	330	5.1778	0.0
No log	4.0	440	5.0728	0.0154
5.1354	5.0	550	4.7829	0.0974
5.1354	6.0	660	4.4895	0.1385
5.1354	7.0	770	4.1981	0.2923
5.1354	8.0	880	3.9328	0.4359
5.1354	9.0	990	3.6937	0.5641
4.2477	10.0	1100	3.4751	0.6308
4.2477	11.0	1210	3.2537	0.7026
4.2477	12.0	1320	3.0355	0.7590
4.2477	13.0	1430	2.8367	0.7846
3.1067	14.0	1540	2.6431	0.8103
3.1067	15.0	1650	2.4553	0.8462
3.1067	16.0	1760	2.2817	0.8718
3.1067	17.0	1870	2.1120	0.8821
3.1067	18.0	1980	1.9496	0.8974
2.1912	19.0	2090	1.7956	0.9128
2.1912	20.0	2200	1.6507	0.9179
2.1912	21.0	2310	1.5192	0.9282
2.1912	22.0	2420	1.3942	0.9333
1.48	23.0	2530	1.2758	0.9436
1.48	24.0	2640	1.1671	0.9538
1.48	25.0	2750	1.0670	0.9590
1.48	26.0	2860	0.9741	0.9590
1.48	27.0	2970	0.8877	0.9590
0.9627	28.0	3080	0.8078	0.9641
0.9627	29.0	3190	0.7388	0.9641
0.9627	30.0	3300	0.6762	0.9692
0.9627	31.0	3410	0.6123	0.9744
0.6161	32.0	3520	0.5591	0.9744
0.6161	33.0	3630	0.5129	0.9744
0.6161	34.0	3740	0.4734	0.9744
0.6161	35.0	3850	0.4341	0.9692
0.6161	36.0	3960	0.3932	0.9846
0.3834	37.0	4070	0.3645	0.9795
0.3834	38.0	4180	0.3398	0.9744
0.3834	39.0	4290	0.3128	0.9846
0.3834	40.0	4400	0.2884	0.9795
0.2414	41.0	4510	0.2659	0.9846
0.2414	42.0	4620	0.2488	0.9846
0.2414	43.0	4730	0.2286	0.9897
0.2414	44.0	4840	0.2145	0.9897
0.2414	45.0	4950	0.2014	0.9949
0.1522	46.0	5060	0.1913	0.9897
0.1522	47.0	5170	0.1808	0.9897
0.1522	48.0	5280	0.1697	0.9897
0.1522	49.0	5390	0.1604	0.9846
0.0981	50.0	5500	0.1551	0.9846
0.0981	51.0	5610	0.1448	0.9897
0.0981	52.0	5720	0.1390	0.9949
0.0981	53.0	5830	0.1348	0.9846
0.0981	54.0	5940	0.1268	0.9897
0.0679	55.0	6050	0.1206	0.9897
0.0679	56.0	6160	0.1178	0.9897
0.0679	57.0	6270	0.1162	0.9897
0.0679	58.0	6380	0.1126	0.9897
0.0679	59.0	6490	0.1086	0.9897
0.0504	60.0	6600	0.1053	0.9897
0.0504	61.0	6710	0.1020	0.9897
0.0504	62.0	6820	0.0995	0.9897
0.0504	63.0	6930	0.0973	0.9897
0.0397	64.0	7040	0.0968	0.9897
0.0397	65.0	7150	0.0921	0.9949
0.0397	66.0	7260	0.0900	0.9897
0.0397	67.0	7370	0.0886	0.9949
0.0397	68.0	7480	0.0865	0.9949
0.0321	69.0	7590	0.0854	0.9949
0.0321	70.0	7700	0.0849	0.9949
0.0321	71.0	7810	0.0835	0.9897
0.0321	72.0	7920	0.0822	0.9949
0.0269	73.0	8030	0.0805	0.9897
0.0269	74.0	8140	0.0789	0.9949
0.0269	75.0	8250	0.0781	0.9897
0.0269	76.0	8360	0.0773	0.9897
0.0269	77.0	8470	0.0760	0.9897
0.0229	78.0	8580	0.0749	0.9949
0.0229	79.0	8690	0.0745	0.9897
0.0229	80.0	8800	0.0736	0.9897
0.0229	81.0	8910	0.0728	0.9949
0.0202	82.0	9020	0.0717	0.9949
0.0202	83.0	9130	0.0711	0.9949
0.0202	84.0	9240	0.0711	0.9949
0.0202	85.0	9350	0.0710	0.9897
0.0202	86.0	9460	0.0704	0.9897
0.018	87.0	9570	0.0687	0.9897
0.018	88.0	9680	0.0685	0.9897
0.018	89.0	9790	0.0677	0.9897
0.018	90.0	9900	0.0675	0.9897
0.0165	91.0	10010	0.0669	0.9897
0.0165	92.0	10120	0.0673	0.9897
0.0165	93.0	10230	0.0664	0.9897
0.0165	94.0	10340	0.0658	0.9897
0.0165	95.0	10450	0.0657	0.9897
0.0156	96.0	10560	0.0657	0.9897
0.0156	97.0	10670	0.0656	0.9897
0.0156	98.0	10780	0.0655	0.9897
0.0156	99.0	10890	0.0654	0.9897
0.0149	100.0	11000	0.0654	0.9897

Framework versions

• Transformers 4.37.1
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.1