finetuned-baseline-phase-1

This model is a fine-tuned version of valhalla/t5-small-e2e-qg on the None dataset. It achieves the following results on the evaluation set:

• Loss: 3.1073

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 16
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss
7.3947	0.14	5	6.5866
6.1276	0.29	10	5.0631
4.8984	0.43	15	4.1654
4.4942	0.57	20	3.9987
4.2374	0.72	25	3.7471
3.9935	0.86	30	3.6307
3.8155	1.01	35	3.5470
3.7181	1.15	40	3.4950
3.6391	1.29	45	3.4587
3.6432	1.44	50	3.4328
3.5728	1.58	55	3.4103
3.6185	1.72	60	3.3889
3.5931	1.87	65	3.3722
3.5249	2.01	70	3.3605
3.595	2.15	75	3.3459
3.5795	2.3	80	3.3356
3.4731	2.44	85	3.3281
3.4917	2.59	90	3.3216
3.4628	2.73	95	3.3140
3.4421	2.87	100	3.3065
3.4528	3.02	105	3.2972
3.4554	3.16	110	3.2884
3.4619	3.3	115	3.2827
3.4654	3.45	120	3.2778
3.3787	3.59	125	3.2735
3.3945	3.73	130	3.2690
3.458	3.88	135	3.2647
3.4034	4.02	140	3.2569
3.4042	4.17	145	3.2499
3.4147	4.31	150	3.2463
3.4611	4.45	155	3.2423
3.3803	4.6	160	3.2392
3.3861	4.74	165	3.2364
3.3503	4.88	170	3.2335
3.4182	5.03	175	3.2299
3.356	5.17	180	3.2286
3.3826	5.31	185	3.2260
3.3368	5.46	190	3.2221
3.3739	5.6	195	3.2160
3.4032	5.75	200	3.2112
3.3825	5.89	205	3.2075
3.3381	6.03	210	3.2055
3.3162	6.18	215	3.2033
3.2946	6.32	220	3.1988
3.3505	6.46	225	3.1944
3.3643	6.61	230	3.1921
3.336	6.75	235	3.1904
3.374	6.89	240	3.1905
3.3148	7.04	245	3.1859
3.3649	7.18	250	3.1829
3.2273	7.32	255	3.1835
3.305	7.47	260	3.1821
3.3225	7.61	265	3.1795
3.3526	7.76	270	3.1757
3.3127	7.9	275	3.1746
3.3137	8.04	280	3.1766
3.2641	8.19	285	3.1739
3.2587	8.33	290	3.1683
3.2954	8.47	295	3.1669
3.3443	8.62	300	3.1682
3.2783	8.76	305	3.1641
3.2698	8.9	310	3.1597
3.3021	9.05	315	3.1577
3.3145	9.19	320	3.1578
3.2308	9.34	325	3.1589
3.2509	9.48	330	3.1574
3.2615	9.62	335	3.1544
3.2387	9.77	340	3.1521
3.2738	9.91	345	3.1501
3.2565	10.05	350	3.1494
3.2863	10.2	355	3.1495
3.1892	10.34	360	3.1496
3.2688	10.48	365	3.1460
3.2417	10.63	370	3.1441
3.3144	10.77	375	3.1421
3.292	10.92	380	3.1390
3.2722	11.06	385	3.1372
3.2685	11.2	390	3.1368
3.2317	11.35	395	3.1367
3.2512	11.49	400	3.1390
3.2268	11.63	405	3.1400
3.2148	11.78	410	3.1386
3.2577	11.92	415	3.1368
3.2406	12.06	420	3.1344
3.2415	12.21	425	3.1343
3.2433	12.35	430	3.1348
3.2126	12.5	435	3.1324
3.2706	12.64	440	3.1295
3.189	12.78	445	3.1267
3.2343	12.93	450	3.1253
3.1968	13.07	455	3.1247
3.242	13.21	460	3.1255
3.2193	13.36	465	3.1259
3.2464	13.5	470	3.1254
3.2374	13.64	475	3.1241
3.2849	13.79	480	3.1217
3.2263	13.93	485	3.1203
3.2702	14.08	490	3.1187
3.3134	14.22	495	3.1177
3.1861	14.36	500	3.1176
3.2232	14.51	505	3.1180
3.1825	14.65	510	3.1180
3.2067	14.79	515	3.1178
3.1963	14.94	520	3.1165
3.2425	15.08	525	3.1153
3.1739	15.22	530	3.1150
3.1967	15.37	535	3.1152
3.2015	15.51	540	3.1156
3.1911	15.66	545	3.1156
3.2413	15.8	550	3.1146
3.2284	15.94	555	3.1138
3.2534	16.09	560	3.1128
3.2333	16.23	565	3.1118
3.1774	16.37	570	3.1117
3.1782	16.52	575	3.1118
3.1897	16.66	580	3.1123
3.197	16.8	585	3.1119
3.2257	16.95	590	3.1107
3.1869	17.09	595	3.1100
3.1515	17.24	600	3.1096
3.2433	17.38	605	3.1096
3.241	17.52	610	3.1089
3.2323	17.67	615	3.1090
3.1672	17.81	620	3.1088
3.1555	17.95	625	3.1087
3.2066	18.1	630	3.1087
3.1844	18.24	635	3.1087
3.2146	18.38	640	3.1086
3.2339	18.53	645	3.1083
3.2031	18.67	650	3.1080
3.1772	18.82	655	3.1078
3.1573	18.96	660	3.1076
3.2879	19.1	665	3.1074
3.2407	19.25	670	3.1073
3.1676	19.39	675	3.1073
3.2272	19.53	680	3.1073

Framework versions

• Transformers 4.34.1
• Pytorch 2.1.0+cu118
• Datasets 2.14.6
• Tokenizers 0.14.1