mbart-en-id-smaller-indo-amr-generation-fted-with-prefix

This model was trained from scratch on the data dataset. It achieves the following results on the evaluation set:

• Loss: 2.3974
• Bleu: 13.717
• Gen Len: 36.5221

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-07
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 12
• total_train_batch_size: 24
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: polynomial
• lr_scheduler_warmup_steps: 200
• num_epochs: 16.0
• label_smoothing_factor: 0.1

Training results

Training Loss	Epoch	Step	Bleu	Gen Len	Validation Loss
3.0219	0.9999	3869	0.0741	114.8177	2.9798
2.8978	2.0	7739	0.0747	113.0081	2.8610
2.8109	2.9999	11608	0.0795	111.475	2.7648
2.7623	4.0	15478	0.1685	105.7747	2.6956
2.7116	4.9999	19347	0.5081	92.4187	2.6404
2.6331	5.9999	23214	1.6991	66.9245	2.5961
2.5716	7.0	27084	5.2201	46.1405	2.5611
2.5943	7.9999	30953	8.0263	40.7538	2.5300
2.5622	9.0	34823	10.2353	38.2607	2.5050
2.537	9.9999	38692	11.3364	36.0732	2.4840
2.5345	11.0	42562	12.1716	36.4367	2.4645
2.4706	11.9999	46428	2.4479	12.51	37.4146
2.4558	13.0	50298	2.4330	12.8144	37.2979
2.4125	13.9999	54167	2.4199	13.0772	37.0436
2.4053	15.0	58037	2.4081	13.5764	36.1492
2.439	15.9994	61904	2.3974	13.717	36.5221

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1