mc_cot_16 / README.md
brettbbb's picture
End of training
af89261
metadata
license: llama2
base_model: lmsys/vicuna-7b-v1.5
tags:
  - generated_from_trainer
model-index:
  - name: mc_cot_16
    results: []

mc_cot_16

This model is a fine-tuned version of lmsys/vicuna-7b-v1.5 on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 3.0303

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 4
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 5
  • num_epochs: 20
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss
2.1538 1.0 4 2.1552
1.7905 2.0 8 1.9012
1.4229 3.0 12 1.7686
1.3834 4.0 16 1.7070
1.4421 5.0 20 1.7308
0.9003 6.0 24 1.7646
0.7013 7.0 28 1.9070
0.6291 8.0 32 2.0078
0.3314 9.0 36 2.2682
0.1554 10.0 40 2.3624
0.0814 11.0 44 2.6523
0.0499 12.0 48 2.7565
0.0216 13.0 52 2.8505
0.0197 14.0 56 2.9170
0.0174 15.0 60 2.9433
0.0174 16.0 64 2.9683
0.0145 17.0 68 2.9966
0.013 18.0 72 3.0193
0.0151 19.0 76 3.0277
0.0145 20.0 80 3.0303

Framework versions

  • Transformers 4.36.0.dev0
  • Pytorch 2.1.0+cu121
  • Datasets 2.13.1
  • Tokenizers 0.14.1