text-summary-training

This model is a fine-tuned version of google/flan-t5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 28.4764

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss
40.8156	0.06	80	41.3782
38.5844	0.13	160	39.1979
36.9094	0.19	240	37.3007
35.5187	0.26	320	35.7000
34.4125	0.32	400	34.3162
33.3875	0.39	480	33.1880
32.6469	0.45	560	32.2870
31.9062	0.51	640	31.5437
31.3094	0.58	720	30.9870
31.0562	0.64	800	30.5852
30.7172	0.71	880	30.2761
30.5016	0.77	960	30.0183
30.3547	0.84	1040	29.7996
30.2609	0.9	1120	29.5926
30.0734	0.97	1200	29.4338
30.1172	1.03	1280	29.2835
29.8297	1.09	1360	29.1521
29.7688	1.16	1440	29.0546
29.6641	1.22	1520	28.9620
29.6781	1.29	1600	28.8916
29.7469	1.35	1680	28.8338
29.5297	1.42	1760	28.7799
29.5125	1.48	1840	28.7278
29.5609	1.54	1920	28.6842
29.3547	1.61	2000	28.6606
29.5156	1.67	2080	28.6338
29.3359	1.74	2160	28.6084
29.3609	1.8	2240	28.5887
29.4531	1.87	2320	28.5683
29.5172	1.93	2400	28.5577
29.3516	2.0	2480	28.5433
29.5656	2.06	2560	28.5335
29.325	2.12	2640	28.5257
29.4469	2.19	2720	28.5187
29.2875	2.25	2800	28.5158
29.3562	2.32	2880	28.5074
29.4391	2.38	2960	28.5053
29.4484	2.45	3040	28.5039
29.475	2.51	3120	28.4989
29.3891	2.57	3200	28.4940
29.3094	2.64	3280	28.4926
29.3672	2.7	3360	28.4905
29.2906	2.77	3440	28.4898
29.2906	2.83	3520	28.4842
29.4031	2.9	3600	28.4870
29.3	2.96	3680	28.4813
29.2766	3.02	3760	28.4764
29.1906	3.09	3840	28.4764
29.3141	3.15	3920	28.4778
29.4516	3.22	4000	28.4764

Framework versions

• Transformers 4.38.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2