Edit model card

zephyr-7b-gpo-u4-i1

This model is a fine-tuned version of DUAL-GPO/zephyr-7b-gpo-update3-i0 on the HuggingFaceH4/ultrafeedback_binarized dataset. It achieves the following results on the evaluation set:

  • Loss: 0.0595
  • Rewards/chosen: -0.1134
  • Rewards/rejected: -0.1090
  • Rewards/accuracies: 0.4030
  • Rewards/margins: -0.0044
  • Logps/rejected: -276.7458
  • Logps/chosen: -289.3788
  • Logits/rejected: -1.8514
  • Logits/chosen: -2.0125

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-06
  • train_batch_size: 2
  • eval_batch_size: 2
  • seed: 42
  • distributed_type: multi-GPU
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 4
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 5

Training results

Training Loss Epoch Step Validation Loss Rewards/chosen Rewards/rejected Rewards/accuracies Rewards/margins Logps/rejected Logps/chosen Logits/rejected Logits/chosen
0.3803 0.4 100 0.0537 0.0 0.0 0.0 0.0 -254.9398 -266.6976 -1.8067 -1.9618
0.2725 0.8 200 0.0549 -0.0209 -0.0219 0.4410 0.0010 -259.3280 -270.8865 -1.8361 -1.9947
0.3013 1.2 300 0.0532 -0.1669 -0.1811 0.4675 0.0141 -291.1523 -300.0851 -1.8278 -1.9902
0.3433 1.6 400 0.0523 -0.1720 -0.1893 0.4780 0.0173 -292.8069 -301.0948 -1.8287 -1.9909
0.3606 2.0 500 0.0623 -0.1514 -0.1465 0.4050 -0.0048 -284.2478 -296.9682 -1.8491 -2.0112
0.3038 2.4 600 0.0616 -0.1610 -0.1582 0.4090 -0.0029 -286.5705 -298.9020 -1.8490 -2.0113
0.3161 2.8 700 0.0613 -0.1640 -0.1619 0.4125 -0.0021 -287.3163 -299.4932 -1.8473 -2.0096
0.3852 3.2 800 0.0574 -0.1342 -0.1354 0.4260 0.0012 -282.0106 -293.5319 -1.8537 -2.0157
0.3359 3.6 900 0.0595 -0.1131 -0.1086 0.4005 -0.0044 -276.6672 -289.3095 -1.8507 -2.0116
0.3701 4.0 1000 0.0596 -0.1134 -0.1090 0.4000 -0.0044 -276.7309 -289.3763 -1.8513 -2.0123
0.4025 4.4 1100 0.0596 -0.1134 -0.1088 0.4030 -0.0045 -276.7074 -289.3722 -1.8516 -2.0127
0.3754 4.8 1200 0.0595 -0.1136 -0.1091 0.4010 -0.0044 -276.7694 -289.4114 -1.8515 -2.0125

Framework versions

  • PEFT 0.7.1
  • Transformers 4.36.2
  • Pytorch 2.1.2+cu121
  • Datasets 2.14.6
  • Tokenizers 0.15.2
Downloads last month
3
Inference API
Unable to determine this model’s pipeline type. Check the docs .

Model tree for DUAL-GPO/zephyr-7b-gpo-u4-i1

Adapter
(1170)
this model

Dataset used to train DUAL-GPO/zephyr-7b-gpo-u4-i1