js-fake-bach-epochs20

This model is a fine-tuned version of gpt2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5973
• Accuracy: 0.0033

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0006058454513356471
• train_batch_size: 16
• eval_batch_size: 32
• seed: 1
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.01
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
1.2427	1.2550	315	0.8253	0.0007
0.8106	2.5100	630	0.7777	0.0021
0.7663	3.7649	945	0.7449	0.0017
0.7263	5.0199	1260	0.6997	0.0027
0.689	6.2749	1575	0.6683	0.0018
0.6524	7.5299	1890	0.6396	0.0008
0.6158	8.7849	2205	0.6139	0.0021
0.5807	10.0398	2520	0.5981	0.0010
0.5437	11.2948	2835	0.5848	0.0030
0.5109	12.5498	3150	0.5841	0.0026
0.4781	13.8048	3465	0.5799	0.0028
0.4453	15.0598	3780	0.5867	0.0034
0.4169	16.3147	4095	0.5915	0.0034
0.3972	17.5697	4410	0.5968	0.0034
0.3847	18.8247	4725	0.5973	0.0033

Framework versions

• Transformers 4.47.1
• Pytorch 2.5.1+cu121
• Datasets 3.2.0
• Tokenizers 0.21.0