dreamerdeo
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -90,7 +90,7 @@ pip install -r requirements.txt
|
|
90 |
./main -ngl 40 -m ggml-model-Q4_K_M.gguf -p "<|im_start|>question\nCara memanggang ikan?\n<|im_start|>answer\n" --temp 0.7 --repeat_penalty 1.1 -n 400 -e
|
91 |
```
|
92 |
|
93 |
-
> Change `-ngl
|
94 |
|
95 |
### How to run with `llama-cpp-python`
|
96 |
|
|
|
90 |
./main -ngl 40 -m ggml-model-Q4_K_M.gguf -p "<|im_start|>question\nCara memanggang ikan?\n<|im_start|>answer\n" --temp 0.7 --repeat_penalty 1.1 -n 400 -e
|
91 |
```
|
92 |
|
93 |
+
> Change `-ngl 40` to the number of layers to offload to GPU. Remove it if you don't have GPU acceleration.
|
94 |
|
95 |
### How to run with `llama-cpp-python`
|
96 |
|