Update README.md
Browse files
README.md
CHANGED
@@ -48,7 +48,7 @@ for the downloaded image which mounts the folder we crated before:<br> <br>
|
|
48 |
-v ./llama.cpp_in_Docker:/llama.cpp \
|
49 |
-t basic-python /bin/bash`<br> <br>(you may have to adjust the path to your local folder)
|
50 |
5. back in the <u>Docker Desktop</u>, open the "Terminal" tab of the started container and enter the
|
51 |
-
following commands:<br
|
52 |
```
|
53 |
apt update
|
54 |
apt-get install software-properties-common -y
|
@@ -64,13 +64,13 @@ choose "Edit file"
|
|
64 |
change `ifneq` to `ifeq`
|
65 |
8. save your change using the disk icon in the upper right corner of the editor pane and open the "Terminal"
|
66 |
tab again
|
67 |
-
9. now enter the following commands:<br
|
68 |
```
|
69 |
make
|
70 |
python3 -m pip install -r requirements.txt
|
71 |
python3 convert.py ../LLaMA-2-7B-32K
|
72 |
```
|
73 |
-
10. you are now ready to run the actual quantization, e.g., using<br
|
74 |
```
|
75 |
./quantize ../LLaMA-2-7B-32K/ggml-model-f16.gguf \
|
76 |
../LLaMA-2-7B-32K/LLaMA-2-7B-32K-Q4_0.gguf Q4_0
|
|
|
48 |
-v ./llama.cpp_in_Docker:/llama.cpp \
|
49 |
-t basic-python /bin/bash`<br> <br>(you may have to adjust the path to your local folder)
|
50 |
5. back in the <u>Docker Desktop</u>, open the "Terminal" tab of the started container and enter the
|
51 |
+
following commands:<br>
|
52 |
```
|
53 |
apt update
|
54 |
apt-get install software-properties-common -y
|
|
|
64 |
change `ifneq` to `ifeq`
|
65 |
8. save your change using the disk icon in the upper right corner of the editor pane and open the "Terminal"
|
66 |
tab again
|
67 |
+
9. now enter the following commands:<br>
|
68 |
```
|
69 |
make
|
70 |
python3 -m pip install -r requirements.txt
|
71 |
python3 convert.py ../LLaMA-2-7B-32K
|
72 |
```
|
73 |
+
10. you are now ready to run the actual quantization, e.g., using<br>
|
74 |
```
|
75 |
./quantize ../LLaMA-2-7B-32K/ggml-model-f16.gguf \
|
76 |
../LLaMA-2-7B-32K/LLaMA-2-7B-32K-Q4_0.gguf Q4_0
|