Update README.md

README.md

@@ -4,13 +4,14 @@ datasets:
 - berkeley-nest/Nectar
 language:
 - en
+base_model: openchat/openchat_3.5
 ---
 This is a model released for our paper: [REBEL: Reinforcement Learning via Regressing Relative Rewards](https://arxiv.org/abs/2404.16767). 
 
-# REBEL-Llama-3
+# REBEL-OpenChat-3.5
 
 This model is developed with REBEL based on [OpenChat-3.5](https://huggingface.co/openchat/openchat_3.5) with [Starling-RM-7B-alpha](https://huggingface.co/berkeley-nest/Starling-RM-7B-alpha) as the reward model and [Nectar](https://huggingface.co/datasets/berkeley-nest/Nectar) dataset.
-The training code is available at https://github.com/ZhaolinGao/REBEL.
+The training code is available at https://github.com/ZhaolinGao/REBEL. We collect online generations during each iteration with a batch size of 32.
 
 ### Links to Other Model
 
@@ -18,27 +19,22 @@ The training code is available at https://github.com/ZhaolinGao/REBEL.
 
 [REBEL-Llama-3-epoch_2](https://huggingface.co/Cornell-AGI/REBEL-Llama-3-epoch_2)
 
-### AlpacaEval 2.0 Evaluations
+[REBEL-Llama-3-Armo-iter_1](https://huggingface.co/Cornell-AGI/REBEL-Llama-3-Armo-iter_1)
 
-| Model | AlpacaEval 2.0<br>LC Win Rate | AlpacaEval 2.0<br>Win Rate |
-| :--------: | :--------: |  :--------: |
-| REBEL-OpenChat-3.5| 17.3 | 12.8 |
-| REBEL-Llama-3 | 30.1 | 32.6 |
-| REBEL-Llama-3-epoch_2| 31.33 | 34.22 |
+[REBEL-Llama-3-Armo-iter_2](https://huggingface.co/Cornell-AGI/REBEL-Llama-3-Armo-iter_2)
 
-### MT-Bench Evaluations
+[REBEL-Llama-3-Armo-iter_3](https://huggingface.co/Cornell-AGI/REBEL-Llama-3-Armo-iter_3)
 
-| Model | MT-Bench<br>1st Turn | MT-Bench<br>2nd Turn | MT-Bench<br>Average |
-| :--------: | :--------: |  :--------: |  :--------: |
-| REBEL-OpenChat-3.5 | 8.54 | 7.58 | 8.06 |
-| REBEL-Llama-3 | 8.63 | 7.69 | 8.16 |
+### Evaluations
 
-### Open LLM Leaderboard Evaluations
-
-| Model | MMLU<br>(5-shot) | GSM8K<br>(5-shot) | Arc<br>(25-shot) | Winogrande<br>(5-shot) | TruthfulQA<br>(0-shot) | HellaSway<br>(10-shot) | Average
-| :--------: | :--------: |  :--------: |  :--------: | :--------: | :--------: |  :--------: |  :--------: |
-| REBEL-OpenChat-3.5 | 63.7 | 68.8 | 64.3 | 80.4 | 48.2 | 85.0 | 68.4 |
-| REBEL-Llama-3 | 65.8 | 75.6 | 61.7 | 75.8 | 51.7 | 78.8 | 68.2 |
+| Model | AlpacaEval 2.0<br>LC Win Rate | AlpacaEval 2.0<br>Win Rate | MT-Bench<br>Average | MMLU<br>(5-shot) | GSM8K<br>(5-shot) |
+| :--------: | :--------: |   :--------: | :--------: |  :--------: | :--------: |
+| REBEL-OpenChat-3.5| 17.3 | 12.8 | 8.06 | 63.7 | 68.8 |
+| REBEL-Llama-3 | 30.1 | 32.6 | 8.16 | 65.8 | 75.6 |
+| REBEL-Llama-3-epoch_2| 31.3 | 34.2 | 7.83 | 65.4 | 75.4 |
+| REBEL-Llama-3-Armo-iter_1| 48.3 | 41.8 | 8.13 | 66.3 | 75.8 |
+| REBEL-Llama-3-Armo-iter_2| 50.0 | 48.5 | 8.07 | 65.9 | 75.4 |
+| REBEL-Llama-3-Armo-iter_3| 49.7 | 48.1 | 8.01 | 66.0 | 75.7 |
 
 ## Citation
 Please cite our paper if you use this model in your own work:
@@ -51,11 +47,4 @@ Please cite our paper if you use this model in your own work:
       archivePrefix={arXiv},
       primaryClass={cs.LG}
 }
-```
-
-
-
-
-
-
-
+```