pholarge

This model is a fine-tuned version of vinai/phobert-large on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2752
• F1: 0.9361

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 24
• eval_batch_size: 24
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 40.0

Training results

Training Loss	Epoch	Step	Validation Loss	F1
0.3094	1.0	1033	0.2295	0.9327
0.2346	2.0	2066	0.2651	0.9280
0.2359	3.0	3099	0.2357	0.9338
0.1734	4.0	4132	0.2720	0.9277
0.1551	5.0	5165	0.2940	0.9350
0.1282	6.0	6198	0.2752	0.9361
0.1208	7.0	7231	0.3137	0.9347
0.1074	8.0	8264	0.2967	0.9355
0.2719	9.0	9297	0.8269	0.6410
0.8456	10.0	10330	0.8220	0.6410
0.84	11.0	11363	0.8212	0.6410
0.841	12.0	12396	0.8207	0.6410
0.8383	13.0	13429	0.8198	0.6410
0.8371	14.0	14462	0.8276	0.6410
0.8486	15.0	15495	0.8244	0.6410
0.844	16.0	16528	0.8391	0.6410
0.837	17.0	17561	0.8235	0.6410
0.8438	18.0	18594	0.8247	0.6410
0.8418	19.0	19627	0.8237	0.6410
0.8384	20.0	20660	0.8199	0.6410
0.8387	21.0	21693	0.8226	0.6410
0.8478	22.0	22726	0.8205	0.6410
0.8364	23.0	23759	0.8259	0.6410
0.8325	24.0	24792	0.8245	0.6410
0.8289	25.0	25825	0.8248	0.6410
0.8251	26.0	26858	0.8247	0.6410
0.824	27.0	27891	0.8214	0.6410
0.8197	28.0	28924	0.8282	0.6410
0.8241	29.0	29957	0.8340	0.6410
0.8285	30.0	30990	0.8360	0.6410
0.8169	31.0	32023	0.8401	0.6410
0.811	32.0	33056	0.8534	0.6410
0.8056	33.0	34089	0.8690	0.6410
0.8023	34.0	35122	0.8640	0.6410
0.8146	35.0	36155	0.8704	0.6410
0.8079	36.0	37188	0.8959	0.6410
0.8081	37.0	38221	0.8802	0.6410
0.8059	38.0	39254	0.8901	0.6410
0.8045	39.0	40287	0.8882	0.6410
0.8024	40.0	41320	0.8918	0.6410

Framework versions

• Transformers 4.48.0.dev0
• Pytorch 2.1.0+cu121
• Datasets 3.1.0
• Tokenizers 0.21.0