lesson-summarization

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.5713

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 200

Training results

Training Loss	Epoch	Step	Validation Loss
2.9037	3.12	200	2.2456
2.5914	6.25	400	2.1498
2.393	9.38	600	2.1002
2.2409	12.5	800	2.0754
2.1515	15.62	1000	2.0683
2.0633	18.75	1200	2.0541
1.9418	21.88	1400	2.0603
1.837	25.0	1600	2.0788
1.7715	28.12	1800	2.0754
1.6957	31.25	2000	2.0815
1.6079	34.38	2200	2.0940
1.5947	37.5	2400	2.1094
1.4603	40.62	2600	2.1147
1.4621	43.75	2800	2.1354
1.4021	46.88	3000	2.1519
1.3394	50.0	3200	2.1670
1.2866	53.12	3400	2.1921
1.2681	56.25	3600	2.2045
1.1866	59.38	3800	2.2194
1.2098	62.5	4000	2.2302
1.1386	65.62	4200	2.2400
1.0853	68.75	4400	2.2634
1.0888	71.88	4600	2.2810
1.0408	75.0	4800	2.2909
1.0309	78.12	5000	2.3059
0.9523	81.25	5200	2.3249
0.9671	84.38	5400	2.3333
0.9413	87.5	5600	2.3543
0.9127	90.62	5800	2.3636
0.9095	93.75	6000	2.3676
0.8952	96.88	6200	2.3756
0.857	100.0	6400	2.3878
0.8474	103.12	6600	2.4148
0.8215	106.25	6800	2.4231
0.8172	109.38	7000	2.4243
0.7761	112.5	7200	2.4489
0.7737	115.62	7400	2.4718
0.7476	118.75	7600	2.4614
0.7345	121.88	7800	2.4705
0.7426	125.0	8000	2.4740
0.7151	128.12	8200	2.4833
0.7191	131.25	8400	2.4786
0.6818	134.38	8600	2.4882
0.6862	137.5	8800	2.4938
0.6929	140.62	9000	2.4977
0.6494	143.75	9200	2.5195
0.6689	146.88	9400	2.5185
0.6492	150.0	9600	2.5259
0.6384	153.12	9800	2.5259
0.6435	156.25	10000	2.5287
0.6251	159.38	10200	2.5284
0.6295	162.5	10400	2.5398
0.6324	165.62	10600	2.5442
0.6252	168.75	10800	2.5481
0.6108	171.88	11000	2.5455
0.6034	175.0	11200	2.5502
0.5969	178.12	11400	2.5601
0.5949	181.25	11600	2.5617
0.6183	184.38	11800	2.5679
0.5805	187.5	12000	2.5687
0.6032	190.62	12200	2.5708
0.5955	193.75	12400	2.5709
0.5961	196.88	12600	2.5713
0.5914	200.0	12800	2.5713

Framework versions

• Transformers 4.31.0
• Pytorch 1.13.1
• Datasets 2.12.0
• Tokenizers 0.13.3