genz_model2

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.1282
• Bleu: 40.1672
• Gen Len: 15.25

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
No log	1.0	107	1.9410	28.2848	15.4509
No log	2.0	214	1.7415	32.3881	15.3645
No log	3.0	321	1.6506	32.8796	15.5374
No log	4.0	428	1.5856	33.1982	15.5748
1.9676	5.0	535	1.5352	34.3335	15.4556
1.9676	6.0	642	1.4929	34.962	15.5187
1.9676	7.0	749	1.4595	35.459	15.535
1.9676	8.0	856	1.4316	35.6253	15.5421
1.9676	9.0	963	1.4066	35.9011	15.4953
1.5695	10.0	1070	1.3838	36.5102	15.4907
1.5695	11.0	1177	1.3608	36.2464	15.5631
1.5695	12.0	1284	1.3410	36.3368	15.5748
1.5695	13.0	1391	1.3238	37.2607	15.493
1.5695	14.0	1498	1.3092	36.9306	15.5234
1.4322	15.0	1605	1.2943	37.2516	15.5701
1.4322	16.0	1712	1.2812	37.9106	15.4696
1.4322	17.0	1819	1.2694	38.0468	15.4907
1.4322	18.0	1926	1.2559	38.0982	15.4836
1.3384	19.0	2033	1.2455	38.5418	15.4556
1.3384	20.0	2140	1.2375	38.2567	15.4463
1.3384	21.0	2247	1.2285	38.3496	15.3972
1.3384	22.0	2354	1.2182	38.6696	15.4393
1.3384	23.0	2461	1.2092	38.6524	15.4182
1.2646	24.0	2568	1.2013	38.5694	15.4346
1.2646	25.0	2675	1.1947	38.8347	15.4065
1.2646	26.0	2782	1.1893	38.7466	15.3738
1.2646	27.0	2889	1.1840	38.8294	15.3855
1.2646	28.0	2996	1.1795	38.8043	15.3738
1.2144	29.0	3103	1.1722	38.9285	15.3995
1.2144	30.0	3210	1.1691	39.1174	15.3435
1.2144	31.0	3317	1.1646	39.2841	15.3341
1.2144	32.0	3424	1.1612	39.1613	15.2687
1.1741	33.0	3531	1.1581	39.2741	15.2921
1.1741	34.0	3638	1.1528	39.3863	15.3014
1.1741	35.0	3745	1.1501	39.5385	15.264
1.1741	36.0	3852	1.1465	39.7548	15.2897
1.1741	37.0	3959	1.1448	39.8433	15.25
1.1518	38.0	4066	1.1415	39.8777	15.2243
1.1518	39.0	4173	1.1398	40.0676	15.2453
1.1518	40.0	4280	1.1384	40.0178	15.2033
1.1518	41.0	4387	1.1348	39.8617	15.278
1.1518	42.0	4494	1.1336	39.9387	15.2664
1.1216	43.0	4601	1.1322	40.1468	15.257
1.1216	44.0	4708	1.1314	40.0534	15.257
1.1216	45.0	4815	1.1305	40.1604	15.257
1.1216	46.0	4922	1.1297	40.1344	15.2523
1.112	47.0	5029	1.1290	40.1921	15.2617
1.112	48.0	5136	1.1285	40.2545	15.25
1.112	49.0	5243	1.1283	40.1672	15.25
1.112	50.0	5350	1.1282	40.1672	15.25

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu118
• Datasets 2.14.3
• Tokenizers 0.13.3