2c6-d6-reward

This model is a fine-tuned version of gpt2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4515
• Accuracy: 0.7904

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1.41e-05
• train_batch_size: 256
• eval_batch_size: 256
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: constant_with_warmup
• lr_scheduler_warmup_steps: 20
• num_epochs: 3.0

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
1.1379	0.2151	20	1.1470	0.4924
0.7698	0.4301	40	0.7288	0.5486
0.6967	0.6452	60	0.6538	0.6201
0.6147	0.8602	80	0.5742	0.7028
0.5642	1.0753	100	0.5263	0.7341
0.5461	1.2903	120	0.5121	0.7430
0.53	1.5054	140	0.4923	0.7606
0.5129	1.7204	160	0.4896	0.7598
0.5043	1.9355	180	0.4769	0.7647
0.4631	2.1505	200	0.4694	0.7703
0.4671	2.3656	220	0.4633	0.7807
0.4514	2.5806	240	0.4599	0.7775
0.454	2.7957	260	0.4515	0.7904

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.0+cu121
• Datasets 2.19.2
• Tokenizers 0.19.1