nintwentydo
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -16,14 +16,13 @@ license: apache-2.0
|
|
16 |
library_name: vllm
|
17 |
base_model:
|
18 |
- mistral-community/pixtral-12b
|
19 |
-
- mgoin/pixtral-12b
|
20 |
- mistralai/Pixtral-12B-2409
|
21 |
base_model_relation: quantized
|
22 |
---
|
23 |
|
24 |
# Pixtral-12B-2409: 2:4 sparse
|
25 |
|
26 |
-
2:4 sparse version of [mistral-community/pixtral-12b](https://huggingface.co/
|
27 |
|
28 |
Example VLLM usage
|
29 |
```
|
|
|
16 |
library_name: vllm
|
17 |
base_model:
|
18 |
- mistral-community/pixtral-12b
|
|
|
19 |
- mistralai/Pixtral-12B-2409
|
20 |
base_model_relation: quantized
|
21 |
---
|
22 |
|
23 |
# Pixtral-12B-2409: 2:4 sparse
|
24 |
|
25 |
+
2:4 sparse version of [mistral-community/pixtral-12b](https://huggingface.co/mistral-community/pixtral-12b) using [kylesayrs/gptq-partition branch of LLM Compressor](https://github.com/vllm-project/llm-compressor/tree/kylesayrs/gptq-partition) for optimised inference on VLLM.
|
26 |
|
27 |
Example VLLM usage
|
28 |
```
|