Update README.md
Browse files
README.md
CHANGED
@@ -34,7 +34,7 @@ Unless you are able to use the latest Triton GPTQ-for-LLaMa code, please use `me
|
|
34 |
* Works with text-generation-webui one-click-installers
|
35 |
* Works on Windows
|
36 |
* Will require ~40GB of VRAM, meaning you'll need an A100 or 2 x 24GB cards.
|
37 |
-
* I haven't yet tested how much VRAM is required exactly so it's possible it won't run on an A100
|
38 |
* Parameters: Groupsize = 128g. No act-order.
|
39 |
* Command used to create the GPTQ:
|
40 |
```
|
@@ -56,7 +56,7 @@ Unless you are able to use the latest Triton GPTQ-for-LLaMa code, please use `me
|
|
56 |
* Only works with the latest Triton branch of GPTQ-for-LLaMa
|
57 |
* **Does not** work with text-generation-webui one-click-installers
|
58 |
* **Does not** work on Windows
|
59 |
-
* Should require less VRAM than the 128g file, so hopefully will run in an A100 40GB
|
60 |
* I haven't yet tested how much VRAM is required exactly
|
61 |
* Parameters: Groupsize = 1024g. act-order.
|
62 |
* Offers the benefits of act-order, but at a higher groupsize to reduce VRAM requirements
|
|
|
34 |
* Works with text-generation-webui one-click-installers
|
35 |
* Works on Windows
|
36 |
* Will require ~40GB of VRAM, meaning you'll need an A100 or 2 x 24GB cards.
|
37 |
+
* I haven't yet tested how much VRAM is required exactly so it's possible it won't run on an A100 40GB
|
38 |
* Parameters: Groupsize = 128g. No act-order.
|
39 |
* Command used to create the GPTQ:
|
40 |
```
|
|
|
56 |
* Only works with the latest Triton branch of GPTQ-for-LLaMa
|
57 |
* **Does not** work with text-generation-webui one-click-installers
|
58 |
* **Does not** work on Windows
|
59 |
+
* Should require less VRAM than the 128g file, so hopefully it will run in an A100 40GB
|
60 |
* I haven't yet tested how much VRAM is required exactly
|
61 |
* Parameters: Groupsize = 1024g. act-order.
|
62 |
* Offers the benefits of act-order, but at a higher groupsize to reduce VRAM requirements
|