finetuned-baseline-phase-0.0

This model is a fine-tuned version of valhalla/t5-small-e2e-qg on the None dataset. It achieves the following results on the evaluation set:

• Loss: 4.0205

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 16
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss
5.2173	0.66	5	4.8296
4.8604	1.32	10	4.5708
4.6755	1.98	15	4.4653
4.6046	2.64	20	4.4006
4.5457	3.31	25	4.3465
4.502	3.97	30	4.2920
4.4677	4.63	35	4.2398
4.3849	5.29	40	4.2034
4.3815	5.95	45	4.1794
4.3412	6.61	50	4.1628
4.3026	7.27	55	4.1417
4.3104	7.93	60	4.1198
4.2791	8.6	65	4.1001
4.2523	9.26	70	4.0855
4.235	9.92	75	4.0724
4.2201	10.58	80	4.0610
4.1716	11.24	85	4.0534
4.2005	11.9	90	4.0489
4.1902	12.56	95	4.0450
4.1632	13.22	100	4.0399
4.1467	13.88	105	4.0349
4.1347	14.55	110	4.0310
4.1606	15.21	115	4.0277
4.1425	15.87	120	4.0255
4.1289	16.53	125	4.0235
4.126	17.19	130	4.0218
4.1551	17.85	135	4.0209
4.1567	18.51	140	4.0205

Framework versions

• Transformers 4.34.1
• Pytorch 2.1.0+cu118
• Datasets 2.14.6
• Tokenizers 0.14.1