Update README.md

README.md

@@ -14,6 +14,7 @@ datasets:
 
 Based on [Llama-3 8B Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct).
 
+
 ChatML prompt format:
 ```
 <|im_start|>system
@@ -49,4 +50,18 @@ v2:
 - dataset code revision d0d123dd221e10bb2a3383bcb1c6e4efe1b4a28a
 - wandb [link](https://wandb.ai/ilyagusev/huggingface/runs/r6u5juyk)
 - 5 datasets: ru_turbo_saiga, ru_sharegpt_cleaned, oasst1_ru_main_branch, gpt_roleplay_realm, ru_instruct_gpt4
-- Datasets merging script: [create_short_chat_set.py](https://github.com/IlyaGusev/rulm/blob/d0d123dd221e10bb2a3383bcb1c6e4efe1b4a28a/self_instruct/src/data_processing/create_short_chat_set.py)
+- Datasets merging script: [create_short_chat_set.py](https://github.com/IlyaGusev/rulm/blob/d0d123dd221e10bb2a3383bcb1c6e4efe1b4a28a/self_instruct/src/data_processing/create_short_chat_set.py)
+
+
+# Evaluation
+
+* Dataset: https://github.com/IlyaGusev/rulm/blob/master/self_instruct/data/tasks.jsonl
+* Framework: https://github.com/tatsu-lab/alpaca_eval
+* Evaluator: alpaca_eval_cot_gpt4_turbo_fn
+
+| model | length_controlled_winrate | win_rate | standard_error | avg_length |
+|-----|-----|-----|-----|-----|
+|chatgpt_4_turbo | 76.04 | 90.00 |1.46 | 1270 |
+|chatgpt_3_5_turbo | 50.00 | 50.00 | 0.00 | 536 |
+|saiga_llama3_8b | 33.07 | 48.19 | 2.45 | 1166 |
+saiga_mistral_7b | 23.38 | 35.99 | 2.34 | 949 |