t5-base-v2

This model is a fine-tuned version of google-t5/t5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.1406
• Rouge1: 26.0593
• Rouge2: 8.3440
• Rougel: 18.7623
• Rougelsum: 18.7644

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 7
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
2.3035	0.6	200	1.9551	26.5313	8.5854	18.5427	18.5364
2.235	1.19	400	2.0039	26.6457	8.8987	18.9702	18.9660
2.3213	1.79	600	2.0703	26.3265	8.6686	18.8473	18.8465
2.3909	2.39	800	2.1191	25.8424	8.3369	18.6260	18.6392
2.4381	2.99	1000	2.1406	26.0593	8.3440	18.7623	18.7644

Framework versions

• PEFT 0.10.0
• Transformers 4.38.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2