Update README.md
Browse files
README.md
CHANGED
@@ -102,11 +102,11 @@ python3 convert.py ../LLaMA-2-7B-32K
|
|
102 |
../LLaMA-2-7B-32K/LLaMA-2-7B-32K-Q4_0.gguf Q4_0
|
103 |
```
|
104 |
11. run any quantizations you need and stop the container when finished (the container will automatically
|
105 |
-
be deleted
|
106 |
12. the `basic-python` image may also be deleted (manually) unless you plan to use it again in the near future
|
107 |
|
108 |
You are now free to move the quanitization results to where you need them and run inferences with context
|
109 |
-
lengths up to 32K (depending on the amount of memory you will have available - long contexts need
|
110 |
lot of RAM)
|
111 |
|
112 |
## License ##
|
|
|
102 |
../LLaMA-2-7B-32K/LLaMA-2-7B-32K-Q4_0.gguf Q4_0
|
103 |
```
|
104 |
11. run any quantizations you need and stop the container when finished (the container will automatically
|
105 |
+
be deleted but the generated files will remain available on your host computer)
|
106 |
12. the `basic-python` image may also be deleted (manually) unless you plan to use it again in the near future
|
107 |
|
108 |
You are now free to move the quanitization results to where you need them and run inferences with context
|
109 |
+
lengths up to 32K (depending on the amount of memory you will have available - long contexts need a
|
110 |
lot of RAM)
|
111 |
|
112 |
## License ##
|