bert-30M-uncased-classification-fqa

This model is a fine-tuned version of vietgpt/bert-30M-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.8275
• Accuracy: 0.9538

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	110	5.2109	0.0051
No log	2.0	220	5.2109	0.0
No log	3.0	330	5.1842	0.0
No log	4.0	440	5.0942	0.0256
5.141	5.0	550	4.8190	0.0615
5.141	6.0	660	4.5678	0.1128
5.141	7.0	770	4.3158	0.2205
5.141	8.0	880	4.0876	0.3333
5.141	9.0	990	3.8774	0.4410
4.3509	10.0	1100	3.6849	0.5179
4.3509	11.0	1210	3.5020	0.6103
4.3509	12.0	1320	3.3304	0.6718
4.3509	13.0	1430	3.1628	0.7026
3.3612	14.0	1540	3.0022	0.7692
3.3612	15.0	1650	2.8528	0.7897
3.3612	16.0	1760	2.7042	0.8051
3.3612	17.0	1870	2.5607	0.8205
3.3612	18.0	1980	2.4300	0.8359
2.5735	19.0	2090	2.3086	0.8462
2.5735	20.0	2200	2.1888	0.8462
2.5735	21.0	2310	2.0762	0.8667
2.5735	22.0	2420	1.9736	0.8718
1.9651	23.0	2530	1.8741	0.8718
1.9651	24.0	2640	1.7797	0.9026
1.9651	25.0	2750	1.6888	0.9077
1.9651	26.0	2860	1.6092	0.9128
1.9651	27.0	2970	1.5296	0.9128
1.5059	28.0	3080	1.4617	0.9179
1.5059	29.0	3190	1.3951	0.9179
1.5059	30.0	3300	1.3362	0.9179
1.5059	31.0	3410	1.2752	0.9179
1.1862	32.0	3520	1.2268	0.9179
1.1862	33.0	3630	1.1743	0.9179
1.1862	34.0	3740	1.1333	0.9231
1.1862	35.0	3850	1.0918	0.9282
1.1862	36.0	3960	1.0535	0.9385
0.9586	37.0	4070	1.0206	0.9385
0.9586	38.0	4180	0.9899	0.9333
0.9586	39.0	4290	0.9638	0.9385
0.9586	40.0	4400	0.9418	0.9436
0.8104	41.0	4510	0.9163	0.9436
0.8104	42.0	4620	0.8983	0.9436
0.8104	43.0	4730	0.8811	0.9487
0.8104	44.0	4840	0.8657	0.9538
0.8104	45.0	4950	0.8551	0.9538
0.7164	46.0	5060	0.8448	0.9538
0.7164	47.0	5170	0.8384	0.9487
0.7164	48.0	5280	0.8326	0.9538
0.7164	49.0	5390	0.8289	0.9538
0.6757	50.0	5500	0.8275	0.9538

Framework versions

• Transformers 4.37.1
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.1