OPEA/Meta-Llama-3.1-70B-Instruct-int4-sym-inc
Updated
•
1
•
1
wenhua cheng
wenhuach
AI & ML interests
Model Compression, CV
Recent Activity
new activity
26 days ago
kaitchup/Qwen2.5-72B-Instruct-AutoRound-GPTQ-4bit:how to run this model
reacted
to
their
post
with 🚀
about 2 months ago
Check out [DeepSeek-R1 INT2 model(https://huggingface.co/OPEA/DeepSeek-R1-int2-mixed-sym-inc). This 200GB DeepSeek-R1 model shows only about a 2% drop in MMLU, though it's quite slow due to kernel issue.
| | BF16 | INT2-mixed |
| ------------- | ------ | ---------- |
| mmlu | 0.8514 | 0.8302 |
| hellaswag | 0.6935 | 0.6657 |
| winogrande | 0.7932 | 0.7940 |
| arc_challenge | 0.6212 | 0.6084 |
posted
an
update
about 2 months ago
Check out [DeepSeek-R1 INT2 model(https://huggingface.co/OPEA/DeepSeek-R1-int2-mixed-sym-inc). This 200GB DeepSeek-R1 model shows only about a 2% drop in MMLU, though it's quite slow due to kernel issue.
| | BF16 | INT2-mixed |
| ------------- | ------ | ---------- |
| mmlu | 0.8514 | 0.8302 |
| hellaswag | 0.6935 | 0.6657 |
| winogrande | 0.7932 | 0.7940 |
| arc_challenge | 0.6212 | 0.6084 |
Organizations
wenhuach's activity
liked
a
model
5 months ago
liked
4
models
8 months ago
LnL-AI/TinyLlama-1.1B-intermediate-step-1341k-3T-autoround-lm_head-symFalse
Text Generation
•
Updated
•
63
•
1
kaitchup/Meta-Llama-3.1-8B-Instruct-autoround-gptq-4bit-asym
Text Generation
•
Updated
•
35
•
1
kaitchup/Meta-Llama-3.1-8B-Instruct-autoround-gptq-4bit-sym
Text Generation
•
Updated
•
1
•
1
Intel/Qwen2-57B-A14B-Instruct-int4-inc
Updated
•
1
•
1
liked
a
model
10 months ago
liked
5
models
11 months ago
Intel/Qwen2-7B-int4-inc
Text Generation
•
Updated
•
25
•
6
Qwen/Qwen-72B-Chat
Text Generation
•
Updated
•
1.33k
•
154
Intel/phi-2-int4-inc
Text Generation
•
Updated
•
19
•
3
Intel/Mistral-7B-v0.1-int4-inc-lmhead
Text Generation
•
Updated
•
2
•
1
cstr/Spaetzle-v60-7b-int4-inc
Text Generation
•
Updated
•
3
•
2
liked
a
Space
12 months ago
liked
a
model
about 1 year ago
liked
a
dataset
about 1 year ago