|
--- |
|
library_name: transformers |
|
license: llama2 |
|
datasets: |
|
- aqua_rat |
|
- microsoft/orca-math-word-problems-200k |
|
- m-a-p/CodeFeedback-Filtered-Instruction |
|
- anon8231489123/ShareGPT_Vicuna_unfiltered |
|
--- |
|
|
|
## Description |
|
This repo contains GGUF format model files for Llama-3-Smaug-8B. |
|
|
|
## Files Provided |
|
| Name | Quant | Bits | File Size | Remark | |
|
| -------------------------- | ----- | ---- | --------- | -------------------------------- | |
|
| llama-3-smaug-8b.Q2_K.gguf | Q2_K | 2 | 3.18 GB | 2.96G, +3.5199 ppl @ Llama-3-8B | |
|
| llama-3-smaug-8b.Q3_K.gguf | Q3_K | 3 | 4.02 GB | 3.74G, +0.6569 ppl @ Llama-3-8B | |
|
| llama-3-smaug-8b.Q4_0.gguf | Q4_0 | 4 | 4.66 GB | 4.34G, +0.4685 ppl @ Llama-3-8B | |
|
| llama-3-smaug-8b.Q4_K.gguf | Q4_K | 4 | 4.92 GB | 4.58G, +0.1754 ppl @ Llama-3-8B | |
|
| llama-3-smaug-8b.Q5_K.gguf | Q5_K | 5 | 5.73 GB | 5.33G, +0.0569 ppl @ Llama-3-8B | |
|
| llama-3-smaug-8b.Q6_K.gguf | Q6_K | 6 | 6.60 GB | 6.14G, +0.0217 ppl @ Llama-3-8B | |
|
| llama-3-smaug-8b.Q8_0.gguf | Q8_0 | 8 | 8.54 GB | 7.96G, +0.0026 ppl @ Llama-3-8B | |
|
|
|
## Parameters |
|
| path | type | architecture | rope_theta | sliding_win | max_pos_embed | |
|
| ------------------------- | ----- | ---------------- | ---------- | ----------- | ------------- | |
|
| abacusai/Llama-3-Smaug-8B | llama | LlamaForCausalLM | 500000.0 | null | 8192 | |
|
|
|
## Benchmarks |
|
 |
|
|
|
# Original Model Card |
|
|
|
# Llama-3-Smaug-8B |
|
|
|
### Built with Meta Llama 3 |
|
|
|
|
|
 |
|
|
|
This model was built using the Smaug recipe for improving performance on real world multi-turn conversations applied to |
|
[meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct). |
|
|
|
### Model Description |
|
|
|
- **Developed by:** [Abacus.AI](https://abacus.ai) |
|
- **License:** https://llama.meta.com/llama3/license/ |
|
- **Finetuned from model:** [meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct). |
|
|
|
|
|
## Evaluation |
|
|
|
### MT-Bench |
|
|
|
``` |
|
########## First turn ########## |
|
score |
|
model turn |
|
Llama-3-Smaug-8B 1 8.77500 |
|
Meta-Llama-3-8B-Instruct 1 8.31250 |
|
########## Second turn ########## |
|
score |
|
model turn |
|
Meta-Llama-3-8B-Instruct 2 7.8875 |
|
Llama-3-Smaug-8B 2 7.8875 |
|
########## Average ########## |
|
score |
|
model |
|
Llama-3-Smaug-8B 8.331250 |
|
Meta-Llama-3-8B-Instruct 8.10 |
|
``` |
|
|
|
| Model | First turn | Second Turn | Average | |
|
| :---- | ---------: | ----------: | ------: | |
|
| Llama-3-Smaug-8B | 8.78 | 7.89 | 8.33 | |
|
| Llama-3-8B-Instruct | 8.31 | 7.89 | 8.10 | |
|
|
|
This version of Smaug uses new techniques and new data compared to [Smaug-72B](https://huggingface.co/abacusai/Smaug-72B-v0.1), and more information will be released later on. For now, see the previous Smaug paper: https://arxiv.org/abs/2402.13228. |
