bartpho-syllable-base-ceg-pair-lexnorm-wiki20k

This model is a fine-tuned version of vinai/bartpho-syllable-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0408
• Rouge1: 84.3157
• Rouge2: 77.3019
• Rougel: 83.8705
• Rougelsum: 83.8558
• Gen Len: 15.0248

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
0.1329	1.0	625	0.0553	80.5353	71.5638	80.1035	80.1039	15.0790
0.0424	2.0	1250	0.0423	82.2298	74.0725	81.7495	81.7617	14.9752
0.0253	3.0	1875	0.0423	82.9736	75.0608	82.4352	82.4329	15.0762
0.0167	4.0	2500	0.0396	83.3918	75.9135	82.8826	82.8858	15.0610
0.0116	5.0	3125	0.0415	83.4002	76.079	82.8948	82.9143	15.0714
0.0082	6.0	3750	0.0403	83.7144	76.3254	83.2485	83.2335	15.0238
0.0061	7.0	4375	0.0408	83.6799	76.3557	83.2511	83.265	15.0324
0.0047	8.0	5000	0.0415	84.1614	77.0332	83.6674	83.6623	15.0124
0.0038	9.0	5625	0.0404	84.2497	77.2485	83.7718	83.7791	15.0352
0.0032	10.0	6250	0.0408	84.3157	77.3019	83.8705	83.8558	15.0248

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2
• Datasets 2.15.0
• Tokenizers 0.15.0