YAML Metadata
Warning:
empty or missing yaml metadata in repo card
(https://huggingface.co/docs/hub/model-cards#model-card-metadata)
Quantization made by Richard Erkhov.
MiniChat-1.5-3B - GGUF
- Model creator: https://huggingface.co/GeneZC/
- Original model: https://huggingface.co/GeneZC/MiniChat-1.5-3B/
Name | Quant method | Size |
---|---|---|
MiniChat-1.5-3B.Q2_K.gguf | Q2_K | 1.09GB |
MiniChat-1.5-3B.IQ3_XS.gguf | IQ3_XS | 1.21GB |
MiniChat-1.5-3B.IQ3_S.gguf | IQ3_S | 1.27GB |
MiniChat-1.5-3B.Q3_K_S.gguf | Q3_K_S | 1.27GB |
MiniChat-1.5-3B.IQ3_M.gguf | IQ3_M | 1.33GB |
MiniChat-1.5-3B.Q3_K.gguf | Q3_K | 1.4GB |
MiniChat-1.5-3B.Q3_K_M.gguf | Q3_K_M | 1.4GB |
MiniChat-1.5-3B.Q3_K_L.gguf | Q3_K_L | 1.52GB |
MiniChat-1.5-3B.IQ4_XS.gguf | IQ4_XS | 1.55GB |
MiniChat-1.5-3B.Q4_0.gguf | Q4_0 | 1.62GB |
MiniChat-1.5-3B.IQ4_NL.gguf | IQ4_NL | 1.63GB |
MiniChat-1.5-3B.Q4_K_S.gguf | Q4_K_S | 1.63GB |
MiniChat-1.5-3B.Q4_K.gguf | Q4_K | 1.72GB |
MiniChat-1.5-3B.Q4_K_M.gguf | Q4_K_M | 1.72GB |
MiniChat-1.5-3B.Q4_1.gguf | Q4_1 | 1.79GB |
MiniChat-1.5-3B.Q5_0.gguf | Q5_0 | 1.95GB |
MiniChat-1.5-3B.Q5_K_S.gguf | Q5_K_S | 1.95GB |
MiniChat-1.5-3B.Q5_K.gguf | Q5_K | 2.01GB |
MiniChat-1.5-3B.Q5_K_M.gguf | Q5_K_M | 2.01GB |
MiniChat-1.5-3B.Q5_1.gguf | Q5_1 | 2.12GB |
MiniChat-1.5-3B.Q6_K.gguf | Q6_K | 2.31GB |
MiniChat-1.5-3B.Q8_0.gguf | Q8_0 | 2.99GB |
Original model description:
language:
- en
- zh
license: apache-2.0
library_name: transformers
widget:
- text: [|User|] Hi 👋 [|Assistant|]
model-index:
- name: MiniChat-1.5-3B
results:
- task:
type: text-generation
name: Text Generation
dataset:
name: AI2 Reasoning Challenge (25-Shot)
type: ai2_arc
config: ARC-Challenge
split: test
args:
num_few_shot: 25
metrics:
- type: acc_norm
value: 46.5
name: normalized accuracy
source:
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=GeneZC/MiniChat-1.5-3B
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: HellaSwag (10-Shot)
type: hellaswag
split: validation
args:
num_few_shot: 10
metrics:
- type: acc_norm
value: 68.28
name: normalized accuracy
source:
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=GeneZC/MiniChat-1.5-3B
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: MMLU (5-Shot)
type: cais/mmlu
config: all
split: test
args:
num_few_shot: 5
metrics:
- type: acc
value: 46.67
name: accuracy
source:
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=GeneZC/MiniChat-1.5-3B
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: TruthfulQA (0-shot)
type: truthful_qa
config: multiple_choice
split: validation
args:
num_few_shot: 0
metrics:
- type: mc2
value: 50.71
source:
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=GeneZC/MiniChat-1.5-3B
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: Winogrande (5-shot)
type: winogrande
config: winogrande_xl
split: validation
args:
num_few_shot: 5
metrics:
- type: acc
value: 65.04
name: accuracy
source:
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=GeneZC/MiniChat-1.5-3B
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: GSM8k (5-shot)
type: gsm8k
config: main
split: test
args:
num_few_shot: 5
metrics:
- type: acc
value: 24.18
name: accuracy
source:
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=GeneZC/MiniChat-1.5-3B
name: Open LLM Leaderboard
MiniChat-1.5-3B
📑 arXiv | 👻 GitHub | 🤗 HuggingFace-MiniMA | 🤗 HuggingFace-MiniChat | 🤗 HuggingFace-MiniChat-1.5 | 🤖 ModelScope-MiniMA | 🤖 ModelScope-MiniChat
🆕 Updates from MiniChat-3B:
❗ Must comply with LICENSE of LLaMA2 since it is derived from LLaMA2.
A language model distilled and finetuned from an adapted version of LLaMA2-7B following "Towards the Law of Capacity Gap in Distilling Language Models".
Outperforming a wide range of 3B competitors in GPT4 evaluation and even competing with several 7B chat models.
The following is an example code snippet to use MiniChat-3B:
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from conversation import get_default_conv_template
# MiniChat
tokenizer = AutoTokenizer.from_pretrained("GeneZC/MiniChat-3B", use_fast=False)
# GPU.
model = AutoModelForCausalLM.from_pretrained("GeneZC/MiniChat-3B", use_cache=True, device_map="auto", torch_dtype=torch.float16).eval()
# CPU.
# model = AutoModelForCausalLM.from_pretrained("GeneZC/MiniChat-3B", use_cache=True, device_map="cpu", torch_dtype=torch.float16).eval()
conv = get_default_conv_template("minichat")
question = "Implement a program to find the common elements in two arrays without using any extra data structures."
conv.append_message(conv.roles[0], question)
conv.append_message(conv.roles[1], None)
prompt = conv.get_prompt()
input_ids = tokenizer([prompt]).input_ids
output_ids = model.generate(
torch.as_tensor(input_ids).cuda(),
do_sample=True,
temperature=0.7,
max_new_tokens=1024,
)
output_ids = output_ids[0][len(input_ids[0]):]
output = tokenizer.decode(output_ids, skip_special_tokens=True).strip()
# output: "def common_elements(arr1, arr2):\n if len(arr1) == 0:\n return []\n if len(arr2) == 0:\n return arr1\n\n common_elements = []\n for element in arr1:\n if element in arr2:\n common_elements.append(element)\n\n return common_elements"
# Multiturn conversation could be realized by continuously appending questions to `conv`.
Bibtex
@article{zhang2023law,
title={Towards the Law of Capacity Gap in Distilling Language Models},
author={Zhang, Chen and Song, Dawei and Ye, Zheyu and Gao, Yan},
year={2023},
url={https://arxiv.org/abs/2311.07052}
}
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
Metric | Value |
---|---|
Avg. | 50.23 |
AI2 Reasoning Challenge (25-Shot) | 46.50 |
HellaSwag (10-Shot) | 68.28 |
MMLU (5-Shot) | 46.67 |
TruthfulQA (0-shot) | 50.71 |
Winogrande (5-shot) | 65.04 |
GSM8k (5-shot) | 24.18 |
- Downloads last month
- 23