PhoBert_Lexical_15-10

This model is a fine-tuned version of vinai/phobert-base-v2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0211
• Accuracy: 0.9945
• F1: 0.9945

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
No log	0.1996	200	0.3681	0.8393	0.8397
No log	0.3992	400	0.3333	0.8558	0.8563
No log	0.5988	600	0.3128	0.8681	0.8684
No log	0.7984	800	0.3062	0.8698	0.8703
No log	0.9980	1000	0.2766	0.8822	0.8827
0.3857	1.1976	1200	0.2619	0.8896	0.8900
0.3857	1.3972	1400	0.2524	0.8939	0.8943
0.3857	1.5968	1600	0.2386	0.8997	0.9000
0.3857	1.7964	1800	0.2422	0.9013	0.9013
0.3857	1.9960	2000	0.2229	0.9073	0.9077
0.2899	2.1956	2200	0.2121	0.9135	0.9137
0.2899	2.3952	2400	0.2311	0.9028	0.9032
0.2899	2.5948	2600	0.2070	0.9163	0.9166
0.2899	2.7944	2800	0.1931	0.9224	0.9226
0.2899	2.9940	3000	0.1894	0.9236	0.9238
0.248	3.1936	3200	0.1797	0.9285	0.9287
0.248	3.3932	3400	0.1714	0.9330	0.9331
0.248	3.5928	3600	0.1628	0.9369	0.9371
0.248	3.7924	3800	0.1642	0.9353	0.9355
0.248	3.9920	4000	0.1541	0.9403	0.9405
0.2167	4.1916	4200	0.1430	0.9467	0.9468
0.2167	4.3912	4400	0.1510	0.9425	0.9427
0.2167	4.5908	4600	0.1509	0.9433	0.9435
0.2167	4.7904	4800	0.1323	0.9514	0.9515
0.2167	4.9900	5000	0.1248	0.9559	0.9559
0.1901	5.1896	5200	0.1159	0.9577	0.9577
0.1901	5.3892	5400	0.1123	0.9595	0.9595
0.1901	5.5888	5600	0.1128	0.9590	0.9591
0.1901	5.7884	5800	0.1016	0.9636	0.9636
0.1901	5.9880	6000	0.1080	0.9628	0.9629
0.1659	6.1876	6200	0.0910	0.9679	0.9679
0.1659	6.3872	6400	0.0939	0.9674	0.9675
0.1659	6.5868	6600	0.0948	0.9665	0.9665
0.1659	6.7864	6800	0.0846	0.9711	0.9711
0.1659	6.9860	7000	0.0866	0.9691	0.9691
0.1451	7.1856	7200	0.0778	0.9733	0.9734
0.1451	7.3852	7400	0.0823	0.9707	0.9708
0.1451	7.5848	7600	0.0702	0.9771	0.9772
0.1451	7.7844	7800	0.0684	0.9776	0.9776
0.1451	7.9840	8000	0.0695	0.9765	0.9766
0.128	8.1836	8200	0.0635	0.9793	0.9793
0.128	8.3832	8400	0.0586	0.9805	0.9805
0.128	8.5828	8600	0.0698	0.9747	0.9748
0.128	8.7824	8800	0.0570	0.9815	0.9815
0.128	8.9820	9000	0.0514	0.9849	0.9849
0.1113	9.1816	9200	0.0514	0.9836	0.9836
0.1113	9.3812	9400	0.0470	0.9862	0.9862
0.1113	9.5808	9600	0.0480	0.9850	0.9850
0.1113	9.7804	9800	0.0482	0.9849	0.9849
0.1113	9.9800	10000	0.0423	0.9872	0.9872
0.0969	10.1796	10200	0.0410	0.9871	0.9871
0.0969	10.3792	10400	0.0374	0.9887	0.9887
0.0969	10.5788	10600	0.0372	0.9884	0.9884
0.0969	10.7784	10800	0.0350	0.9903	0.9903
0.0969	10.9780	11000	0.0386	0.9878	0.9878
0.0864	11.1776	11200	0.0327	0.9908	0.9908
0.0864	11.3772	11400	0.0305	0.9915	0.9916
0.0864	11.5768	11600	0.0304	0.9916	0.9916
0.0864	11.7764	11800	0.0336	0.9895	0.9895
0.0864	11.9760	12000	0.0276	0.9925	0.9925
0.0734	12.1756	12200	0.0265	0.9929	0.9929
0.0734	12.3752	12400	0.0275	0.9924	0.9924
0.0734	12.5749	12600	0.0265	0.9928	0.9928
0.0734	12.7745	12800	0.0256	0.9927	0.9928
0.0734	12.9741	13000	0.0238	0.9940	0.9940
0.0665	13.1737	13200	0.0251	0.9929	0.9929
0.0665	13.3733	13400	0.0242	0.9934	0.9934
0.0665	13.5729	13600	0.0228	0.9942	0.9942
0.0665	13.7725	13800	0.0232	0.9938	0.9938
0.0665	13.9721	14000	0.0228	0.9940	0.9940
0.0623	14.1717	14200	0.0215	0.9944	0.9944
0.0623	14.3713	14400	0.0217	0.9943	0.9943
0.0623	14.5709	14600	0.0213	0.9944	0.9944
0.0623	14.7705	14800	0.0213	0.9944	0.9944
0.0623	14.9701	15000	0.0211	0.9945	0.9945

Framework versions

• Transformers 4.45.2
• Pytorch 2.1.2
• Datasets 2.20.0
• Tokenizers 0.20.1