guma/distilgpt2-finetuned-shakespeare

This model is a fine-tuned version of distilgpt2 on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 3.1769
• Validation Loss: 3.5116
• Epoch: 19

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 2e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}
• training_precision: float32

Training results

Train Loss	Validation Loss	Epoch
4.2136	3.8303	0
3.8997	3.6993	1
3.7790	3.6344	2
3.7061	3.5923	3
3.6476	3.5653	4
3.6003	3.5513	5
3.5578	3.5360	6
3.5204	3.5277	7
3.4843	3.5171	8
3.4514	3.5117	9
3.4194	3.5048	10
3.3903	3.5040	11
3.3627	3.5006	12
3.3332	3.5006	13
3.3052	3.5019	14
3.2772	3.5051	15
3.2514	3.5043	16
3.2249	3.5026	17
3.2019	3.5129	18
3.1769	3.5116	19

Framework versions

• Transformers 4.22.2
• TensorFlow 2.8.2
• Datasets 2.5.2
• Tokenizers 0.12.1