run-llama

This model is a fine-tuned version of meta-llama/Meta-Llama-3.1-8B on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.4304
  • Accuracy: 0.7917

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 32
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 64
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • training_steps: 200

Training results

Training Loss Epoch Step Accuracy Validation Loss
0.6426 0.1130 10 0.5625 0.6485
0.6074 0.2260 20 0.6319 0.5821
0.4355 0.3390 30 0.7153 0.4967
0.5957 0.4520 40 0.7569 0.4683
0.3984 0.5650 50 0.7431 0.4616
0.4795 0.9836 60 0.7361 0.4514
0.3154 1.1475 70 0.7292 0.4393
0.3086 1.3115 80 0.75 0.4339
0.499 1.4754 90 0.7569 0.4281
0.2642 1.6393 100 0.7847 0.4281
0.2173 1.8033 110 0.7708 0.4200
0.3281 1.9672 120 0.7847 0.4151
0.2329 3.4211 130 0.7847 0.4172
0.2891 3.6842 140 0.7847 0.4202
0.3906 3.9474 150 0.7708 0.4285
0.2373 4.1026 160 0.4338 0.7778
0.2144 4.3590 170 0.4312 0.7847
0.2715 4.6154 180 0.4300 0.7847
0.1855 4.8718 190 0.4327 0.7917
0.3506 5.1282 200 0.4304 0.7917

Framework versions

  • PEFT 0.12.0
  • Transformers 4.44.0
  • Pytorch 2.4.0+cu121
  • Datasets 2.20.0
  • Tokenizers 0.19.1
Downloads last month
1
Inference API
Unable to determine this model’s pipeline type. Check the docs .

Model tree for Baidicoot/run-llama

Adapter
(135)
this model