rozek commited on
Commit
ed9c6c4
1 Parent(s): 6a4410d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +25 -1
README.md CHANGED
@@ -25,9 +25,33 @@ However, llama.cpp requires quantized files in the new GGUF format - that's wher
25
  it contains a few quantizations of the original weights from Together's fined-tuned model (as indicated by
26
  the file names)
27
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
28
  Concerning the license(s):
29
 
30
- * the [orignal model](https://ai.meta.com/llama/) (from Meta AI) was released under a rather [permittive
31
  license](https://ai.meta.com/llama/license/)
32
  * the fine tuned model from Together Computer uses the
33
  [same license](https://huggingface.co/togethercomputer/LLaMA-2-7B-32K/blob/main/README.md)
 
25
  it contains a few quantizations of the original weights from Together's fined-tuned model (as indicated by
26
  the file names)
27
 
28
+ ## How the Quantization was done ##
29
+
30
+ Since the author does not want arbitrary Python stuff loitering on his computer, the quatization was done
31
+ using [Docker](https://www.docker.com/).
32
+
33
+ Assuming that you have the [Docker Desktop](https://www.docker.com/products/docker-desktop/) installed on
34
+ your system and also have a basic knowledge of how it is used, you mayx just follow the instructions shown
35
+ below in order to generate your own quantizations:
36
+
37
+ > Nota bene: you will need 30+x GB of free disk space, at least - depending on your quantization
38
+
39
+ 1. create a new folder called `llama.cpp_in_Docker`<br>this folder will later be mounted into the Docker
40
+ container and store the quantization results
41
+ 2. download the weights for the fine-tuned LLaMA-2 model from
42
+ [Hugging Face](https://huggingface.co/togethercomputer/LLaMA-2-7B-32K) into a subfolder of `llama.cpp_in_Docker`
43
+ (let's call the new folder `LLaMA-2-7B-32K`)
44
+ 3. within the Docker Desktop, download search for and download a `basic-python` image - just use one of the
45
+ most popular ones
46
+ 4. from a terminal session on your host computer (i.e., not a Docker container!), start a new container for the
47
+ downloaded image which mounts the folder we crated before:<br>&nbsp;<br>``
48
+ ...
49
+
50
+ ## License ##
51
+
52
  Concerning the license(s):
53
 
54
+ * the [original model](https://ai.meta.com/llama/) (from Meta AI) was released under a rather [permittive
55
  license](https://ai.meta.com/llama/license/)
56
  * the fine tuned model from Together Computer uses the
57
  [same license](https://huggingface.co/togethercomputer/LLaMA-2-7B-32K/blob/main/README.md)