Update README.md
Browse files
README.md
CHANGED
|
@@ -61,34 +61,34 @@ Best results in **bold**.
|
|
| 61 |
- If this model produces answers with weird characters, it means you are not using the correct version of qwopqwop200/GPTQ-for-LLaMa as mentioned below.
|
| 62 |
- If this model produces answers that are out of topic or if it talks to itself, it means you are not using the correct checkout 508de42 of qwopqwop200/GPTQ-for-LLaMa as mentioned below.
|
| 63 |
|
| 64 |
-
|
| 65 |
```
|
| 66 |
git clone https://github.com/oobabooga/text-generation-webui
|
| 67 |
cd text-generation-webui
|
|
|
|
|
|
|
| 68 |
pip install -r requirements.txt
|
| 69 |
|
| 70 |
mkdir repositories
|
| 71 |
cd repositories
|
| 72 |
-
git clone https://github.com/qwopqwop200/GPTQ-for-LLaMa.git -b cuda
|
| 73 |
cd GPTQ-for-LLaMa
|
|
|
|
| 74 |
pip install -r requirements.txt
|
| 75 |
-
python setup_cuda.py install
|
| 76 |
```
|
| 77 |
|
| 78 |
-
|
| 79 |
```
|
| 80 |
git clone https://github.com/oobabooga/text-generation-webui
|
| 81 |
cd text-generation-webui
|
| 82 |
-
git fetch origin pull/1229/head:triton # This is the version that supports Triton - https://github.com/oobabooga/text-generation-webui/pull/1229
|
| 83 |
-
git checkout triton
|
| 84 |
pip install -r requirements.txt
|
| 85 |
|
| 86 |
mkdir repositories
|
| 87 |
cd repositories
|
| 88 |
-
git clone https://github.com/qwopqwop200/GPTQ-for-LLaMa.git # -
|
| 89 |
cd GPTQ-for-LLaMa
|
| 90 |
-
git checkout 508de42 # Before qwopqwop200 broke everything... - https://github.com/qwopqwop200/GPTQ-for-LLaMa/issues/183
|
| 91 |
pip install -r requirements.txt
|
|
|
|
| 92 |
```
|
| 93 |
|
| 94 |
<br>
|
|
|
|
| 61 |
- If this model produces answers with weird characters, it means you are not using the correct version of qwopqwop200/GPTQ-for-LLaMa as mentioned below.
|
| 62 |
- If this model produces answers that are out of topic or if it talks to itself, it means you are not using the correct checkout 508de42 of qwopqwop200/GPTQ-for-LLaMa as mentioned below.
|
| 63 |
|
| 64 |
+
RECOMMENDED - Triton (Fast tokens/s) - Works on Windows with WSL (what I've used) or Linux:
|
| 65 |
```
|
| 66 |
git clone https://github.com/oobabooga/text-generation-webui
|
| 67 |
cd text-generation-webui
|
| 68 |
+
git fetch origin pull/1229/head:triton # This is the version that supports Triton - https://github.com/oobabooga/text-generation-webui/pull/1229
|
| 69 |
+
git checkout triton
|
| 70 |
pip install -r requirements.txt
|
| 71 |
|
| 72 |
mkdir repositories
|
| 73 |
cd repositories
|
| 74 |
+
git clone https://github.com/qwopqwop200/GPTQ-for-LLaMa.git # -b cuda
|
| 75 |
cd GPTQ-for-LLaMa
|
| 76 |
+
git checkout 508de42 # Before qwopqwop200 broke everything... - https://github.com/qwopqwop200/GPTQ-for-LLaMa/issues/183
|
| 77 |
pip install -r requirements.txt
|
|
|
|
| 78 |
```
|
| 79 |
|
| 80 |
+
DISCOURAGED - Cuda (Slow tokens/s) and output issues https://github.com/qwopqwop200/GPTQ-for-LLaMa/issues/128:
|
| 81 |
```
|
| 82 |
git clone https://github.com/oobabooga/text-generation-webui
|
| 83 |
cd text-generation-webui
|
|
|
|
|
|
|
| 84 |
pip install -r requirements.txt
|
| 85 |
|
| 86 |
mkdir repositories
|
| 87 |
cd repositories
|
| 88 |
+
git clone https://github.com/qwopqwop200/GPTQ-for-LLaMa.git -b cuda # Make sure you obtain the qwopqwop200 version, not the oobabooga one! (because "act-order: yes")
|
| 89 |
cd GPTQ-for-LLaMa
|
|
|
|
| 90 |
pip install -r requirements.txt
|
| 91 |
+
python setup_cuda.py install
|
| 92 |
```
|
| 93 |
|
| 94 |
<br>
|