llama-cpp-python not working
Browse files
README.md
CHANGED
|
@@ -8,7 +8,7 @@ datasets:
|
|
| 8 |
|
| 9 |
- join these model parts with `cat airoboros-180b-2.2.1-Q4_K_M.gguf* > airoboros-180b-2.2.1-Q4_K_M.gguf`
|
| 10 |
|
| 11 |
-
- These files were tested on 0eb332a and 207b519 on linux.
|
| 12 |
- The model was converted with https://github.com/ggerganov/llama.cpp/pull/3864, with this change
|
| 13 |
|
| 14 |
```
|
|
|
|
| 8 |
|
| 9 |
- join these model parts with `cat airoboros-180b-2.2.1-Q4_K_M.gguf* > airoboros-180b-2.2.1-Q4_K_M.gguf`
|
| 10 |
|
| 11 |
+
- These files were tested on 0eb332a and 207b519 in llama.cpp on linux, as well as kobold.cpp (faae84e). This does not seem to work with llama-cpp-python: `ERROR: byte not found in vocab: '` A user reported successfully launching Q5_K_M with mac gpu in llama.cpp.
|
| 12 |
- The model was converted with https://github.com/ggerganov/llama.cpp/pull/3864, with this change
|
| 13 |
|
| 14 |
```
|