| Quantization made by Richard Erkhov. | |
| [Github](https://github.com/RichardErkhov) | |
| [Discord](https://discord.gg/pvy7H8DZMG) | |
| [Request more models](https://github.com/RichardErkhov/quant_request) | |
| Llama3-8B-SuperNova-Spectrum-Hermes-DPO - GGUF | |
| - Model creator: https://huggingface.co/yuvraj17/ | |
| - Original model: https://huggingface.co/yuvraj17/Llama3-8B-SuperNova-Spectrum-Hermes-DPO/ | |
| | Name | Quant method | Size | | |
| | ---- | ---- | ---- | | |
| | [Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q2_K.gguf](https://huggingface.co/RichardErkhov/yuvraj17_-_Llama3-8B-SuperNova-Spectrum-Hermes-DPO-gguf/blob/main/Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q2_K.gguf) | Q2_K | 2.96GB | | |
| | [Llama3-8B-SuperNova-Spectrum-Hermes-DPO.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/yuvraj17_-_Llama3-8B-SuperNova-Spectrum-Hermes-DPO-gguf/blob/main/Llama3-8B-SuperNova-Spectrum-Hermes-DPO.IQ3_XS.gguf) | IQ3_XS | 3.28GB | | |
| | [Llama3-8B-SuperNova-Spectrum-Hermes-DPO.IQ3_S.gguf](https://huggingface.co/RichardErkhov/yuvraj17_-_Llama3-8B-SuperNova-Spectrum-Hermes-DPO-gguf/blob/main/Llama3-8B-SuperNova-Spectrum-Hermes-DPO.IQ3_S.gguf) | IQ3_S | 3.43GB | | |
| | [Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/yuvraj17_-_Llama3-8B-SuperNova-Spectrum-Hermes-DPO-gguf/blob/main/Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q3_K_S.gguf) | Q3_K_S | 3.41GB | | |
| | [Llama3-8B-SuperNova-Spectrum-Hermes-DPO.IQ3_M.gguf](https://huggingface.co/RichardErkhov/yuvraj17_-_Llama3-8B-SuperNova-Spectrum-Hermes-DPO-gguf/blob/main/Llama3-8B-SuperNova-Spectrum-Hermes-DPO.IQ3_M.gguf) | IQ3_M | 3.52GB | | |
| | [Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q3_K.gguf](https://huggingface.co/RichardErkhov/yuvraj17_-_Llama3-8B-SuperNova-Spectrum-Hermes-DPO-gguf/blob/main/Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q3_K.gguf) | Q3_K | 3.74GB | | |
| | [Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/yuvraj17_-_Llama3-8B-SuperNova-Spectrum-Hermes-DPO-gguf/blob/main/Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q3_K_M.gguf) | Q3_K_M | 3.74GB | | |
| | [Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/yuvraj17_-_Llama3-8B-SuperNova-Spectrum-Hermes-DPO-gguf/blob/main/Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q3_K_L.gguf) | Q3_K_L | 4.03GB | | |
| | [Llama3-8B-SuperNova-Spectrum-Hermes-DPO.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/yuvraj17_-_Llama3-8B-SuperNova-Spectrum-Hermes-DPO-gguf/blob/main/Llama3-8B-SuperNova-Spectrum-Hermes-DPO.IQ4_XS.gguf) | IQ4_XS | 4.18GB | | |
| | [Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q4_0.gguf](https://huggingface.co/RichardErkhov/yuvraj17_-_Llama3-8B-SuperNova-Spectrum-Hermes-DPO-gguf/blob/main/Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q4_0.gguf) | Q4_0 | 4.34GB | | |
| | [Llama3-8B-SuperNova-Spectrum-Hermes-DPO.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/yuvraj17_-_Llama3-8B-SuperNova-Spectrum-Hermes-DPO-gguf/blob/main/Llama3-8B-SuperNova-Spectrum-Hermes-DPO.IQ4_NL.gguf) | IQ4_NL | 4.38GB | | |
| | [Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/yuvraj17_-_Llama3-8B-SuperNova-Spectrum-Hermes-DPO-gguf/blob/main/Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q4_K_S.gguf) | Q4_K_S | 4.37GB | | |
| | [Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q4_K.gguf](https://huggingface.co/RichardErkhov/yuvraj17_-_Llama3-8B-SuperNova-Spectrum-Hermes-DPO-gguf/blob/main/Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q4_K.gguf) | Q4_K | 4.58GB | | |
| | [Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/yuvraj17_-_Llama3-8B-SuperNova-Spectrum-Hermes-DPO-gguf/blob/main/Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q4_K_M.gguf) | Q4_K_M | 4.58GB | | |
| | [Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q4_1.gguf](https://huggingface.co/RichardErkhov/yuvraj17_-_Llama3-8B-SuperNova-Spectrum-Hermes-DPO-gguf/blob/main/Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q4_1.gguf) | Q4_1 | 4.78GB | | |
| | [Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q5_0.gguf](https://huggingface.co/RichardErkhov/yuvraj17_-_Llama3-8B-SuperNova-Spectrum-Hermes-DPO-gguf/blob/main/Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q5_0.gguf) | Q5_0 | 5.21GB | | |
| | [Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/yuvraj17_-_Llama3-8B-SuperNova-Spectrum-Hermes-DPO-gguf/blob/main/Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q5_K_S.gguf) | Q5_K_S | 5.21GB | | |
| | [Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q5_K.gguf](https://huggingface.co/RichardErkhov/yuvraj17_-_Llama3-8B-SuperNova-Spectrum-Hermes-DPO-gguf/blob/main/Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q5_K.gguf) | Q5_K | 5.34GB | | |
| | [Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/yuvraj17_-_Llama3-8B-SuperNova-Spectrum-Hermes-DPO-gguf/blob/main/Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q5_K_M.gguf) | Q5_K_M | 5.34GB | | |
| | [Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q5_1.gguf](https://huggingface.co/RichardErkhov/yuvraj17_-_Llama3-8B-SuperNova-Spectrum-Hermes-DPO-gguf/blob/main/Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q5_1.gguf) | Q5_1 | 5.65GB | | |
| | [Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q6_K.gguf](https://huggingface.co/RichardErkhov/yuvraj17_-_Llama3-8B-SuperNova-Spectrum-Hermes-DPO-gguf/blob/main/Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q6_K.gguf) | Q6_K | 6.14GB | | |
| | [Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q8_0.gguf](https://huggingface.co/RichardErkhov/yuvraj17_-_Llama3-8B-SuperNova-Spectrum-Hermes-DPO-gguf/blob/main/Llama3-8B-SuperNova-Spectrum-Hermes-DPO.Q8_0.gguf) | Q8_0 | 7.95GB | | |
| Original model description: | |
| --- | |
| language: | |
| - en | |
| license: apache-2.0 | |
| library_name: transformers | |
| tags: | |
| - dpo | |
| - rlhf | |
| - trl | |
| pipeline_tag: text-generation | |
| model-index: | |
| - name: Llama3-8B-SuperNova-Spectrum-Hermes-DPO | |
| results: | |
| - task: | |
| type: text-generation | |
| name: Text Generation | |
| dataset: | |
| name: IFEval (0-Shot) | |
| type: HuggingFaceH4/ifeval | |
| args: | |
| num_few_shot: 0 | |
| metrics: | |
| - type: inst_level_strict_acc and prompt_level_strict_acc | |
| value: 46.91 | |
| name: strict accuracy | |
| source: | |
| url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=yuvraj17/Llama3-8B-SuperNova-Spectrum-Hermes-DPO | |
| name: Open LLM Leaderboard | |
| - task: | |
| type: text-generation | |
| name: Text Generation | |
| dataset: | |
| name: BBH (3-Shot) | |
| type: BBH | |
| args: | |
| num_few_shot: 3 | |
| metrics: | |
| - type: acc_norm | |
| value: 21.24 | |
| name: normalized accuracy | |
| source: | |
| url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=yuvraj17/Llama3-8B-SuperNova-Spectrum-Hermes-DPO | |
| name: Open LLM Leaderboard | |
| - task: | |
| type: text-generation | |
| name: Text Generation | |
| dataset: | |
| name: MATH Lvl 5 (4-Shot) | |
| type: hendrycks/competition_math | |
| args: | |
| num_few_shot: 4 | |
| metrics: | |
| - type: exact_match | |
| value: 5.14 | |
| name: exact match | |
| source: | |
| url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=yuvraj17/Llama3-8B-SuperNova-Spectrum-Hermes-DPO | |
| name: Open LLM Leaderboard | |
| - task: | |
| type: text-generation | |
| name: Text Generation | |
| dataset: | |
| name: GPQA (0-shot) | |
| type: Idavidrein/gpqa | |
| args: | |
| num_few_shot: 0 | |
| metrics: | |
| - type: acc_norm | |
| value: 6.94 | |
| name: acc_norm | |
| source: | |
| url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=yuvraj17/Llama3-8B-SuperNova-Spectrum-Hermes-DPO | |
| name: Open LLM Leaderboard | |
| - task: | |
| type: text-generation | |
| name: Text Generation | |
| dataset: | |
| name: MuSR (0-shot) | |
| type: TAUR-Lab/MuSR | |
| args: | |
| num_few_shot: 0 | |
| metrics: | |
| - type: acc_norm | |
| value: 9.62 | |
| name: acc_norm | |
| source: | |
| url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=yuvraj17/Llama3-8B-SuperNova-Spectrum-Hermes-DPO | |
| name: Open LLM Leaderboard | |
| - task: | |
| type: text-generation | |
| name: Text Generation | |
| dataset: | |
| name: MMLU-PRO (5-shot) | |
| type: TIGER-Lab/MMLU-Pro | |
| config: main | |
| split: test | |
| args: | |
| num_few_shot: 5 | |
| metrics: | |
| - type: acc | |
| value: 18.16 | |
| name: accuracy | |
| source: | |
| url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=yuvraj17/Llama3-8B-SuperNova-Spectrum-Hermes-DPO | |
| name: Open LLM Leaderboard | |
| --- | |
| # Llama3-8B-SuperNova-Spectrum-Hermes-DPO | |
| This model is a **DPO fine-tuned** version of my `DARE_TIES` merged Model [`yuvraj17/Llama3-8B-SuperNova-Spectrum-dare_ties`](https://huggingface.co/yuvraj17/Llama3-8B-SuperNova-Spectrum-dare_ties) on the [yuvraj17/chatml-OpenHermes2.5-dpo-binarized-alpha-2k](https://huggingface.co/datasets/yuvraj17/chatml-OpenHermes2.5-dpo-binarized-alpha-2k) dataset. | |
| ## DPO (Direct Preference Optimization): | |
| Direct Preference Optimization (DPO) is a fine-tuning technique that focuses on aligning a model's responses with human preferences or ranking data without requiring reinforcement learning steps, like in RLHF. | |
| <figure> | |
| <img src="https://cdn-uploads.huggingface.co/production/uploads/66137d95e8d2cda230ddcea6/kHcU5dkcSVqxEIWt_GRUB.png" width="1000" height="768"> | |
| <figcaption> DPO vs RLHF <a href="//arxiv.org/abs/2305.18290">Reference</a> </figcaption> | |
| </figure> | |
| ## Training: | |
| - Trained on **1x A40s (48GB VRAM)** using the [HuggingFace TRL](https://huggingface.co/docs/trl/index). | |
| - **QLoRA**(`4-bit precision`) for 1 epoch | |
| ``` | |
| # LoRA configuration | |
| peft_config = LoraConfig( | |
| r=32, | |
| lora_alpha=16, | |
| lora_dropout=0.05, | |
| bias="none", | |
| task_type="CAUSAL_LM", | |
| target_modules=['k_proj', 'gate_proj', 'v_proj', 'up_proj', 'q_proj', 'o_proj', 'down_proj'] | |
| ) | |
| ``` | |
| ### Training Params | |
| The following hyperparameters were used during training: | |
| - learning_rate: 5e-05 | |
| - beta=0.1 | |
| - num_devices: 1 | |
| - gradient_accumulation_steps: 4 | |
| - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 | |
| - lr_scheduler_type: cosine | |
| - lr_scheduler_warmup_steps: 100 | |
| - num_epochs: 1 | |
| ### Training Time = **1:57:00** hours | |
| ### Weight & Biases Report | |
| [Report-Link](https://api.wandb.ai/links/my-sft-team/d211juao) | |
| ## 💻 Usage | |
| ```python | |
| !pip install -qU transformers accelerate | |
| from transformers import AutoTokenizer | |
| import transformers | |
| import torch | |
| model = "yuvraj17/Llama3-8B-SuperNova-Spectrum-Hermes-DPO" | |
| messages = [{"role": "user", "content": "What is a large language model?"}] | |
| tokenizer = AutoTokenizer.from_pretrained(model) | |
| prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) | |
| pipeline = transformers.pipeline( | |
| "text-generation", | |
| model=model, | |
| torch_dtype=torch.float16, | |
| device_map="auto", | |
| ) | |
| outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95) | |
| print(outputs[0]["generated_text"]) | |
| ``` | |
| ## 🏆 Evaluation Scores | |
| Coming Soon | |
| # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard) | |
| Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_yuvraj17__Llama3-8B-SuperNova-Spectrum-Hermes-DPO) | |
| | Metric |Value| | |
| |-------------------|----:| | |
| |Avg. |18.00| | |
| |IFEval (0-Shot) |46.91| | |
| |BBH (3-Shot) |21.24| | |
| |MATH Lvl 5 (4-Shot)| 5.14| | |
| |GPQA (0-shot) | 6.94| | |
| |MuSR (0-shot) | 9.62| | |
| |MMLU-PRO (5-shot) |18.16| | |