mt5-small-finetuned-summary

This model is a fine-tuned version of google/mt5-small on the xlsum dataset. It achieves the following results on the evaluation set:

• Loss: 2.1466
• Rouge1: 21.6196
• Rouge2: 7.7979
• Rougel: 17.5683
• Rougelsum: 17.6757

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
3.9629	1.0	2170	2.3132	20.7316	7.3431	16.9157	17.0149
2.9704	2.0	4340	2.2299	21.3148	7.8529	17.2968	17.3873
2.8026	3.0	6510	2.2092	21.3313	7.8526	17.3679	17.4773
2.7054	4.0	8680	2.1876	21.6909	7.9841	17.6881	17.7839
2.6453	5.0	10850	2.1743	21.7372	7.7546	17.6551	17.7575
2.5925	6.0	13020	2.1602	21.5715	7.7879	17.5943	17.6994
2.5619	7.0	15190	2.1482	21.5888	7.8789	17.6519	17.752
2.5415	8.0	17360	2.1466	21.6196	7.7979	17.5683	17.6757

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.15.0
• Tokenizers 0.15.0