dpfried commited on
Commit
24f56a4
1 Parent(s): 2955601

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -35,13 +35,13 @@ pip install git+https://github.com/huggingface/transformers
35
 
36
  See [https://github.com/dpfried/incoder](https://github.com/dpfried/incoder) for example code.
37
 
38
- This 6B model comes in two versions: with weights in full-precision (float32) (branch `main`) and weights in half-precision (float16) (branch `float16`). The versions can be loaded as follows:
39
 
40
- - Full-precision (float32): This should be used if you are fine-tuning the model (note: this will take a lot of GPU memory, probably multiple GPUs, and we have not tried training the model in `transformers` --- it was trained in Fairseq)
41
 
42
  `model = AutoModelForCausalLM.from_pretrained("facebook/incoder-6B")`
43
 
44
- - Half-precision (float16): This can be used if you are only doing inference (i.e. generating from the model). It will use less GPU memory, and less RAM when loading the model. With this version it should be able to perform inference on a 16 GB GPU (with a batch size of 1, to sequence lengths of at least 256).
45
 
46
  `model = AutoModelForCausalLM.from_pretrained("facebook/incoder-6B", revision="float16", torch_dtype=torch.float16, low_cpu_mem_usage=True)`
47
 
 
35
 
36
  See [https://github.com/dpfried/incoder](https://github.com/dpfried/incoder) for example code.
37
 
38
+ This 6B model comes in two versions: with weights in full-precision (float32, stored on branch `main`) and weights in half-precision (float16, stored on branch `float16`). The versions can be loaded as follows:
39
 
40
+ *Full-precision* (float32): This should be used if you are fine-tuning the model (note: this will take a lot of GPU memory, probably multiple GPUs, and we have not tried training the model in `transformers` --- it was trained in Fairseq). Load with:
41
 
42
  `model = AutoModelForCausalLM.from_pretrained("facebook/incoder-6B")`
43
 
44
+ *Half-precision* (float16): This can be used if you are only doing inference (i.e. generating from the model). It will use less GPU memory, and less RAM when loading the model. With this version it should be able to perform inference on a 16 GB GPU (with a batch size of 1, to sequence lengths of at least 256). Load with:
45
 
46
  `model = AutoModelForCausalLM.from_pretrained("facebook/incoder-6B", revision="float16", torch_dtype=torch.float16, low_cpu_mem_usage=True)`
47