t5-small-finetuned-newssum

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 3.0180
• Rouge1: 28.4281
• Rouge2: 11.7926
• Rougel: 26.0068
• Rougelsum: 25.971
• Gen Len: 15.3115

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.002
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 6
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	16	2.7920	28.3868	12.9442	25.9886	26.0546	14.5492
No log	2.0	32	2.7583	27.0411	10.9079	24.7416	24.6482	15.2213
No log	3.0	48	2.7845	27.7028	11.5886	25.5631	25.6304	15.2787
No log	4.0	64	2.8982	27.9576	12.5207	25.7261	25.6778	15.623
No log	5.0	80	2.9824	27.9748	11.3803	25.4079	25.3393	15.3689
No log	6.0	96	3.0180	28.4281	11.7926	26.0068	25.971	15.3115

Framework versions

• Transformers 4.35.0
• Pytorch 2.1.2+cu118
• Datasets 2.11.0
• Tokenizers 0.14.1