hubert2BertMusic200

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.4016
• Rouge1: 29.7694
• Rouge2: 6.8881
• Rougel: 20.8382
• Rougelsum: 20.7475
• Gen Len: 54.68

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-06
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
1.5312	1.0	1361	1.4350	29.7078	7.6167	21.0154	20.9599	57.96
1.479	2.0	2722	1.4311	29.5523	7.5127	20.8746	20.7455	55.77
1.4774	3.0	4083	1.4296	29.4948	7.41	20.6514	20.531	55.12
1.4663	4.0	5444	1.4239	28.8275	6.8898	20.1906	20.1528	54.0
1.5195	5.0	6805	1.4219	31.4265	7.8481	21.8264	21.7471	53.33
1.5115	6.0	8166	1.4177	29.8254	6.8757	21.0269	20.9813	53.45
1.5424	7.0	9527	1.4154	29.9847	7.0621	20.9479	20.9081	54.73
1.5635	8.0	10888	1.4131	29.8807	7.1097	20.9508	20.902	53.83
1.6138	9.0	12249	1.4113	29.1418	6.725	20.4919	20.404	55.46
1.676	10.0	13610	1.4094	29.4633	6.7466	20.5997	20.5457	54.8
1.6447	11.0	14971	1.4087	30.1765	6.7892	20.9128	20.8485	54.75
1.6683	12.0	16332	1.4074	29.7832	6.7904	20.6381	20.5824	54.86
1.6927	13.0	17693	1.4056	29.5094	6.6848	20.5682	20.4713	53.62
1.6567	14.0	19054	1.4040	29.674	6.7272	20.7709	20.7017	52.78
1.672	15.0	20415	1.4032	29.447	6.6842	20.707	20.6147	53.81
1.6468	16.0	21776	1.4032	30.1311	7.1838	21.0813	21.0099	54.62
1.6661	17.0	23137	1.4022	30.1715	7.1566	21.0716	20.9984	54.95
1.668	18.0	24498	1.4020	30.1766	7.1429	21.1461	21.0611	54.56
1.6448	19.0	25859	1.4016	29.5354	6.7505	20.5517	20.4719	54.34
1.6647	20.0	27220	1.4016	29.7694	6.8881	20.8382	20.7475	54.68

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu117
• Datasets 2.14.2
• Tokenizers 0.13.3