just1nseo's picture
End of training
8c404c5 verified
|
raw
history blame
4.16 kB
metadata
base_model: alignment-handbook/zephyr-7b-sft-full
datasets:
  - generation/UF6k
library_name: peft
license: apache-2.0
tags:
  - alignment-handbook
  - trl
  - dpo
  - generated_from_trainer
model-index:
  - name: zephyr-dpop-qlora-uf-oursuf6k-5e-7
    results: []

zephyr-dpop-qlora-uf-oursuf6k-5e-7

This model is a fine-tuned version of alignment-handbook/zephyr-7b-sft-full on the generation/UF6k dataset. It achieves the following results on the evaluation set:

  • Loss: 0.6903
  • Positive Losses: 0.0369
  • Dpo Losses: 0.6851
  • Rewards/chosen: 0.0601
  • Rewards/rejected: 0.0433
  • Rewards/accuracies: 0.6090
  • Rewards/margins: 0.0168
  • Rewards/margins Max: 0.0933
  • Rewards/margins Min: -0.0510
  • Rewards/margins Std: 0.0482
  • Logps/rejected: -254.2451
  • Logps/chosen: -278.5801
  • Logits/rejected: -2.7575
  • Logits/chosen: -2.7964

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-07
  • train_batch_size: 4
  • eval_batch_size: 8
  • seed: 42
  • distributed_type: multi-GPU
  • num_devices: 2
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 16
  • total_eval_batch_size: 16
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 1

Training results

Training Loss Epoch Step Validation Loss Positive Losses Dpo Losses Rewards/chosen Rewards/rejected Rewards/accuracies Rewards/margins Rewards/margins Max Rewards/margins Min Rewards/margins Std Logps/rejected Logps/chosen Logits/rejected Logits/chosen
0.6918 0.15 100 0.6930 0.0047 0.6922 0.0135 0.0117 0.6000 0.0019 0.0130 -0.0081 0.0070 -257.4135 -283.2392 -2.7718 -2.8105
0.6867 0.29 200 0.6917 0.0124 0.6900 0.0299 0.0235 0.6100 0.0063 0.0373 -0.0214 0.0194 -256.2250 -281.6053 -2.7690 -2.8075
0.6825 0.44 300 0.6909 0.0225 0.6879 0.0441 0.0333 0.625 0.0108 0.0619 -0.0341 0.0319 -255.2539 -280.1855 -2.7634 -2.8021
0.6808 0.58 400 0.6905 0.0314 0.6861 0.0543 0.0397 0.6170 0.0147 0.0808 -0.0436 0.0415 -254.6122 -279.1606 -2.7608 -2.7994
0.6769 0.73 500 0.6902 0.0330 0.6855 0.0585 0.0426 0.6120 0.0160 0.0885 -0.0487 0.0458 -254.3216 -278.7400 -2.7587 -2.7975
0.67 0.88 600 0.6903 0.0373 0.6851 0.0599 0.0432 0.6160 0.0167 0.0930 -0.0511 0.0481 -254.2623 -278.6050 -2.7602 -2.7988

Framework versions

  • PEFT 0.7.1
  • Transformers 4.39.0.dev0
  • Pytorch 2.1.2+cu121
  • Datasets 2.14.6
  • Tokenizers 0.15.2