MaziyarPanahi
commited on
Commit
β’
5992b45
1
Parent(s):
a3d4a67
Update README.md (#1)
Browse files- Update README.md (8799a0c3641110e3e09427bc273e49a046e27d30)
README.md
CHANGED
@@ -1,6 +1,5 @@
|
|
1 |
---
|
2 |
tags:
|
3 |
-
- finetuned
|
4 |
- quantized
|
5 |
- 4-bit
|
6 |
- AWQ
|
@@ -25,8 +24,27 @@ inference: false
|
|
25 |
model_creator: v2ray
|
26 |
pipeline_tag: text-generation
|
27 |
quantized_by: MaziyarPanahi
|
|
|
|
|
|
|
|
|
|
|
|
|
28 |
---
|
29 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
30 |
[MaziyarPanahi/Mixtral-8x22B-v0.1-AWQ](https://huggingface.co/MaziyarPanahi/Mixtral-8x22B-v0.1-AWQ) is a quantized (AWQ) version of [v2ray/Mixtral-8x22B-v0.1](https://huggingface.co/v2ray/Mixtral-8x22B-v0.1)
|
31 |
|
32 |
## How to use
|
@@ -46,26 +64,57 @@ model_id = "MaziyarPanahi/Mixtral-8x22B-v0.1-AWQ"
|
|
46 |
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
47 |
model = AutoModelForCausalLM.from_pretrained(model_id).to(0)
|
48 |
|
49 |
-
text = "
|
50 |
inputs = tokenizer(text, return_tensors="pt").to(0)
|
51 |
|
52 |
out = model.generate(**inputs, max_new_tokens=300)
|
53 |
print(tokenizer.decode(out[0], skip_special_tokens=True))
|
54 |
```
|
55 |
|
56 |
-
Results:
|
57 |
-
```
|
58 |
-
User:
|
59 |
-
Hello can you provide me with top-3 cool places to visit in Paris?
|
60 |
-
|
61 |
-
Assistant:
|
62 |
-
Absolutely, here are my top-3 recommendations for must-see places in Paris:
|
63 |
-
|
64 |
-
1. The Eiffel Tower: An icon of Paris, this wrought-iron lattice tower is a global cultural icon of France and is among the most recognizable structures in the world. Climbing up to the top offers breathtaking views of the city.
|
65 |
|
66 |
-
|
67 |
|
68 |
-
|
|
|
|
|
69 |
|
70 |
-
|
71 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
tags:
|
|
|
3 |
- quantized
|
4 |
- 4-bit
|
5 |
- AWQ
|
|
|
24 |
model_creator: v2ray
|
25 |
pipeline_tag: text-generation
|
26 |
quantized_by: MaziyarPanahi
|
27 |
+
language:
|
28 |
+
- en
|
29 |
+
- es
|
30 |
+
- de
|
31 |
+
- it
|
32 |
+
- fr
|
33 |
---
|
34 |
+
<img src="./mixtral-8x22b.jpeg" width="600" />
|
35 |
+
|
36 |
+
# Mixtral-8x22B-v0.1-AWQ
|
37 |
+
|
38 |
+
On April 10th, [@MistralAI](https://huggingface.co/mistralai) released a model named "Mixtral 8x22B," an 176B MoE via magnet link (torrent):
|
39 |
+
|
40 |
+
- 176B MoE with ~40B active
|
41 |
+
- Context length of 65k tokens
|
42 |
+
- The base model can be fine-tuned
|
43 |
+
- Requires ~260GB VRAM in fp16, 73GB in int4
|
44 |
+
- Licensed under Apache 2.0, according to their Discord
|
45 |
+
- Available on @huggingface (community)
|
46 |
+
- Utilizes a tokenizer similar to previous models
|
47 |
+
|
48 |
[MaziyarPanahi/Mixtral-8x22B-v0.1-AWQ](https://huggingface.co/MaziyarPanahi/Mixtral-8x22B-v0.1-AWQ) is a quantized (AWQ) version of [v2ray/Mixtral-8x22B-v0.1](https://huggingface.co/v2ray/Mixtral-8x22B-v0.1)
|
49 |
|
50 |
## How to use
|
|
|
64 |
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
65 |
model = AutoModelForCausalLM.from_pretrained(model_id).to(0)
|
66 |
|
67 |
+
text = "Hello can you provide me with top-3 cool places to visit in Paris?"
|
68 |
inputs = tokenizer(text, return_tensors="pt").to(0)
|
69 |
|
70 |
out = model.generate(**inputs, max_new_tokens=300)
|
71 |
print(tokenizer.decode(out[0], skip_special_tokens=True))
|
72 |
```
|
73 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
74 |
|
75 |
+
## Credit
|
76 |
|
77 |
+
- [MistralAI](https://huggingface.co/mistralai) for opening the weights
|
78 |
+
- [v2ray](https://huggingface.co/v2ray/) for downloading, converting, and sharing it with the community [Mixtral-8x22B-v0.1](https://huggingface.co/v2ray/Mixtral-8x22B-v0.1)
|
79 |
+
- [philschmid](https://huggingface.co/philschmid) for the photo he shared on his Twitter
|
80 |
|
81 |
+
βββββ
|
82 |
+
ββββββββββββββββββ
|
83 |
+
βββββββββββββββββββββββββββββββ
|
84 |
+
ββββββββββββββββββββββββββββββββββ
|
85 |
+
ββββββββββββββ ββββββββββββββββββββββββββββββββββ
|
86 |
+
βββββββββββββββββββββββββββββ βββββββββββββββββββββββββββββββββββ
|
87 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
88 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
89 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
90 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
91 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
92 |
+
ββββββββββββββββββββββββββββββββοΏ½οΏ½οΏ½ββββββββββββββββββββββββββββββββββββββββ
|
93 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
94 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
95 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
96 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
97 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
98 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
99 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
100 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
101 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
102 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
103 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
104 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
105 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
106 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
107 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
108 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
109 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
110 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
111 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
112 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
113 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
114 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
115 |
+
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ βββββββ
|
116 |
+
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ βββ
|
117 |
+
ββββββββββββββββββββββββββββββββ ββββββββββββββββββ
|
118 |
+
ββββββββββββββββββββββββββββ
|
119 |
+
βββββββββββββββββ
|
120 |
+
βββββ
|