patent-summarization-t5-base-2022-09-20

This model is a fine-tuned version of t5-base on the farleyknight/big_patent_5_percent dataset. It achieves the following results on the evaluation set:

• Loss: 1.9975
• Rouge1: 36.0843
• Rouge2: 12.1856
• Rougel: 25.8099
• Rougelsum: 30.1664
• Gen Len: 118.3137

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1.0

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
2.2811	0.08	5000	2.1767	18.5624	6.8795	15.5361	16.6836	19.0
2.2551	0.17	10000	2.1327	19.077	6.8512	15.79	17.086	19.0
2.2818	0.25	15000	2.1029	18.8637	6.9233	15.7341	16.9717	19.0
2.1952	0.33	20000	2.0805	18.962	7.1157	15.8297	17.0333	19.0
2.157	0.41	25000	2.0641	19.1418	7.315	16.05	17.2551	19.0
2.1775	0.5	30000	2.0452	19.2387	7.3193	16.0852	17.3563	19.0
2.1376	0.58	35000	2.0308	19.291	7.363	16.1243	17.4151	19.0
2.1853	0.66	40000	2.0207	19.2808	7.4671	16.1593	17.3836	19.0
2.1416	0.75	45000	2.0113	19.0414	7.3335	15.9747	17.1899	19.0
2.1245	0.83	50000	2.0055	19.1445	7.3715	16.0166	17.2621	19.0
2.133	0.91	55000	1.9997	19.3033	7.4821	16.1413	17.3949	19.0
2.1191	0.99	60000	1.9973	19.4044	7.5483	16.2429	17.488	19.0

Framework versions

• Transformers 4.23.0.dev0
• Pytorch 1.12.0
• Datasets 2.4.0
• Tokenizers 0.12.1