t5-small-hardaDerailKP

This model is a fine-tuned version of t5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.1390
• Rouge1: 51.5439
• Rouge2: 41.2421
• Rougel: 51.4764
• Rougelsum: 51.5006
• Gen Len: 6.3538

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
1.2197	1.0	6157	1.1987	51.2268	39.9596	51.1923	51.1914	6.7607
0.9954	2.0	12314	1.1706	50.8022	39.6403	50.7374	50.6872	6.3795
0.9489	3.0	18471	1.1442	52.3931	42.1802	52.3291	52.2775	6.3484
0.8887	4.0	24628	1.1390	51.5439	41.2421	51.4764	51.5006	6.3538
0.8414	5.0	30785	1.1799	51.9563	41.1814	51.8804	51.8698	6.7852
0.753	6.0	36942	1.1829	52.4688	41.3965	52.3511	52.3868	6.6134
0.7471	7.0	43099	1.1995	51.3549	40.6927	51.2323	51.2653	6.6271
0.7327	8.0	49256	1.2001	51.5724	40.8948	51.4687	51.4899	6.6366

Framework versions

• Transformers 4.39.3
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2