Hugging Face's logo

just1nseo
/
zephyr-dpop-qlora-uf-ours-5e-7-epoch1

PEFT
TensorBoard
Safetensors
mistral
alignment-handbook
trl
dpo
Generated from Trainer
4-bit precision
bitsandbytes
Model card Files Metrics Community
zephyr-dpop-qlora-uf-ours-5e-7-epoch1 / README.md
just1nseo's picture
just1nseo
End of training
63fc752 verified
preview code
|
raw
history blame
3.3 kB
metadata
base_model: alignment-handbook/zephyr-7b-sft-full
datasets:
  - generation/UF
library_name: peft
license: apache-2.0
tags:
  - alignment-handbook
  - trl
  - dpo
  - generated_from_trainer
model-index:
  - name: zephyr-dpop-qlora-uf-ours-5e-7-epoch1
    results: []

zephyr-dpop-qlora-uf-ours-5e-7-epoch1

This model is a fine-tuned version of alignment-handbook/zephyr-7b-sft-full on the generation/UF dataset. It achieves the following results on the evaluation set:

  • Loss: 0.7024
  • Positive Losses: 0.1617
  • Dpo Losses: 0.6844
  • Rewards/chosen: 0.0533
  • Rewards/rejected: 0.0348
  • Rewards/accuracies: 0.5940
  • Rewards/margins: 0.0185
  • Rewards/margins Max: 0.1082
  • Rewards/margins Min: -0.0600
  • Rewards/margins Std: 0.0561
  • Logps/rejected: -255.0996
  • Logps/chosen: -279.2664
  • Logits/rejected: -2.7455
  • Logits/chosen: -2.7839

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-07
  • train_batch_size: 4
  • eval_batch_size: 8
  • seed: 42
  • distributed_type: multi-GPU
  • num_devices: 2
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 16
  • total_eval_batch_size: 16
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 1

Training results

Training Loss Epoch Step Validation Loss Positive Losses Dpo Losses Rewards/chosen Rewards/rejected Rewards/accuracies Rewards/margins Rewards/margins Max Rewards/margins Min Rewards/margins Std Logps/rejected Logps/chosen Logits/rejected Logits/chosen
0.6834 0.28 100 0.6932 0.0200 0.6905 0.0248 0.0194 0.5930 0.0054 0.0341 -0.0197 0.0178 -256.6423 -282.1181 -2.7657 -2.8044
0.6629 0.56 200 0.6977 0.1042 0.6860 0.0485 0.0335 0.5980 0.0149 0.0881 -0.0489 0.0456 -255.2263 -279.7464 -2.7492 -2.7879
0.6479 0.85 300 0.7024 0.1592 0.6845 0.0528 0.0346 0.5940 0.0182 0.1072 -0.0591 0.0555 -255.1217 -279.3168 -2.7478 -2.7859

Framework versions

  • PEFT 0.7.1
  • Transformers 4.39.0.dev0
  • Pytorch 2.1.2+cu121
  • Datasets 2.14.6
  • Tokenizers 0.15.2