fine-tuned-bart-20-epochs-wang-lab

This model is a fine-tuned version of facebook/bart-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.1462
• Rouge1: 0.2876
• Rouge2: 0.1104
• Rougel: 0.2587
• Rougelsum: 0.2583
• Gen Len: 15.32

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	301	0.8236	0.2393	0.0872	0.2103	0.2098	15.1
2.6644	2.0	602	0.7800	0.2486	0.0882	0.219	0.2187	14.24
2.6644	3.0	903	0.7623	0.3152	0.131	0.2914	0.2901	15.83
0.6713	4.0	1204	0.7802	0.2909	0.104	0.2577	0.2577	14.4
0.4641	5.0	1505	0.8159	0.2986	0.1058	0.2629	0.2606	14.71
0.4641	6.0	1806	0.8451	0.3212	0.1374	0.2892	0.2892	15.3
0.2986	7.0	2107	0.8913	0.2965	0.115	0.2724	0.2728	15.25
0.2986	8.0	2408	0.9194	0.2686	0.1036	0.2395	0.2389	15.07
0.2025	9.0	2709	0.9674	0.283	0.1077	0.2549	0.2535	15.38
0.1397	10.0	3010	0.9848	0.2805	0.1127	0.2484	0.2475	15.99
0.1397	11.0	3311	1.0356	0.2943	0.1158	0.2568	0.2586	15.32
0.0922	12.0	3612	1.0481	0.3291	0.1211	0.297	0.2999	15.39
0.0922	13.0	3913	1.0846	0.2861	0.1074	0.2473	0.2482	15.04
0.0618	14.0	4214	1.0941	0.2929	0.103	0.2511	0.2505	15.34
0.042	15.0	4515	1.1076	0.2639	0.1111	0.2349	0.2328	15.11
0.042	16.0	4816	1.1180	0.2825	0.1125	0.2465	0.2452	15.08
0.03	17.0	5117	1.1310	0.2924	0.1073	0.2527	0.2528	15.47
0.03	18.0	5418	1.1407	0.2823	0.1017	0.2491	0.2471	15.1
0.0204	19.0	5719	1.1445	0.2952	0.1142	0.2635	0.264	15.13
0.0153	20.0	6020	1.1462	0.2876	0.1104	0.2587	0.2583	15.32

Framework versions

• Transformers 4.36.2
• Pytorch 1.12.1+cu113
• Datasets 2.15.0
• Tokenizers 0.15.0