bartowski commited on
Commit
5c28e7a
1 Parent(s): 0338a68

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -21,9 +21,9 @@ Original model: https://huggingface.co/abacusai/Smaug-Mixtral-v0.1
21
  | ------ | ---- | ------------ | ---- | ---- | ---- | ----------- |
22
  | [6_5](https://huggingface.co/bartowski/Smaug-Mixtral-v0.1-exl2/tree/6_5) | 6.5 | 8.0 | 38.9 GB | 40.4 GB | 42.4 GB | Near unquantized performance at vastly reduced size, **recommended (if you can run it..)**. |
23
  | [4_25](https://huggingface.co/bartowski/Smaug-Mixtral-v0.1-exl2/tree/4_25) | 4.25 | 6.0 | 25.9 GB | 27.4 GB | 29.4 GB | GPTQ equivalent bits per weight, slightly higher quality. |
24
- | [3_75](https://huggingface.co/bartowski/Smaug-Mixtral-v0.1-exl2/tree/3_75) | 3.5 | 6.0 | 23.0 GB | 24.5 GB | 26.5 GB | Lower quality, only use if you have to. |
25
- | [3_5](https://huggingface.co/bartowski/Smaug-Mixtral-v0.1-exl2/tree/3_5) | 3.5 | 6.0 | 21.5 GB | 23.0 GB | 25.0 GB | Lower quality, only use if you have to. |
26
- | [3_0](https://huggingface.co/bartowski/Smaug-Mixtral-v0.1-exl2/tree/3_0) | 3.0 | 6.0 | 18.9 GB | 20.4 GB | 22.4 GB | Very low quality, usable with 16gb of VRAM. |
27
 
28
  ## Download instructions
29
 
 
21
  | ------ | ---- | ------------ | ---- | ---- | ---- | ----------- |
22
  | [6_5](https://huggingface.co/bartowski/Smaug-Mixtral-v0.1-exl2/tree/6_5) | 6.5 | 8.0 | 38.9 GB | 40.4 GB | 42.4 GB | Near unquantized performance at vastly reduced size, **recommended (if you can run it..)**. |
23
  | [4_25](https://huggingface.co/bartowski/Smaug-Mixtral-v0.1-exl2/tree/4_25) | 4.25 | 6.0 | 25.9 GB | 27.4 GB | 29.4 GB | GPTQ equivalent bits per weight, slightly higher quality. |
24
+ | [3_75](https://huggingface.co/bartowski/Smaug-Mixtral-v0.1-exl2/tree/3_75) | 3.5 | 6.0 | 23.0 GB | 24.5 GB | 26.5 GB | Lower quality, but pretty usable. Good for 4k context on 24GB. |
25
+ | [3_5](https://huggingface.co/bartowski/Smaug-Mixtral-v0.1-exl2/tree/3_5) | 3.5 | 6.0 | 21.5 GB | 23.0 GB | 25.0 GB | Lower quality, only use if you need more context on 24GB. |
26
+ | [3_0](https://huggingface.co/bartowski/Smaug-Mixtral-v0.1-exl2/tree/3_0) | 3.0 | 6.0 | 18.9 GB | 20.4 GB | 22.4 GB | Very low quality, pushes context to max but likely unusable. |
27
 
28
  ## Download instructions
29