Update README.md
Browse files
README.md
CHANGED
@@ -48,7 +48,8 @@ for the downloaded image which mounts the folder we crated before:<br> <br>
|
|
48 |
-v ./llama.cpp_in_Docker:/llama.cpp \
|
49 |
-t basic-python /bin/bash`<br> <br>(you may have to adjust the path to your local folder)
|
50 |
5. back in the <u>Docker Desktop</u>, open the "Terminal" tab of the started container and enter the
|
51 |
-
following commands:<br> <br
|
|
|
52 |
apt update
|
53 |
apt-get install software-properties-common -y
|
54 |
apt-get update
|
@@ -63,12 +64,14 @@ choose "Edit file"
|
|
63 |
change `ifneq` to `ifeq`
|
64 |
8. save your change using the disk icon in the upper right corner of the editor pane and open the "Terminal"
|
65 |
tab again
|
66 |
-
9. now enter the following commands:<br> <br
|
|
|
67 |
make
|
68 |
python3 -m pip install -r requirements.txt
|
69 |
python3 convert.py ../LLaMA-2-7B-32K
|
70 |
```
|
71 |
-
10. you are now ready to run the actual quantization, e.g., using<br> <br
|
|
|
72 |
./quantize ../LLaMA-2-7B-32K/ggml-model-f16.gguf \
|
73 |
../LLaMA-2-7B-32K/LLaMA-2-7B-32K-Q4_0.gguf Q4_0
|
74 |
```
|
|
|
48 |
-v ./llama.cpp_in_Docker:/llama.cpp \
|
49 |
-t basic-python /bin/bash`<br> <br>(you may have to adjust the path to your local folder)
|
50 |
5. back in the <u>Docker Desktop</u>, open the "Terminal" tab of the started container and enter the
|
51 |
+
following commands:<br> <br>
|
52 |
+
```
|
53 |
apt update
|
54 |
apt-get install software-properties-common -y
|
55 |
apt-get update
|
|
|
64 |
change `ifneq` to `ifeq`
|
65 |
8. save your change using the disk icon in the upper right corner of the editor pane and open the "Terminal"
|
66 |
tab again
|
67 |
+
9. now enter the following commands:<br> <br>
|
68 |
+
```
|
69 |
make
|
70 |
python3 -m pip install -r requirements.txt
|
71 |
python3 convert.py ../LLaMA-2-7B-32K
|
72 |
```
|
73 |
+
10. you are now ready to run the actual quantization, e.g., using<br> <br>
|
74 |
+
```
|
75 |
./quantize ../LLaMA-2-7B-32K/ggml-model-f16.gguf \
|
76 |
../LLaMA-2-7B-32K/LLaMA-2-7B-32K-Q4_0.gguf Q4_0
|
77 |
```
|