Update README.md

README.md

@@ -15,6 +15,12 @@ This is a version of `hivemind/gpt-j-6B-8bit` for low-RAM loading, i.e., free Co
 
 > **NOTE:** PRIOR to loading the model, you need to "patch" it to be compatible with loading 8bit weights etc. See the original model card above for details on how to do this.
 
+install `transformers` and `accelerate` if needed:
+```sh
+pip install transformers accelerate
+```
+Patch the model, load using `device_map="auto"`:
+
 ```python
 import transformers 
 from transformers import AutoTokenizer
@@ -27,7 +33,8 @@ tokenizer = AutoTokenizer.from_pretrained("ethzanalytics/gpt-j-6B-8bit-sharded")
 
 model = GPTJForCausalLM.from_pretrained(
  "ethzanalytics/gpt-j-6B-8bit-sharded",
- low_cpu_mem_usage=True,
- max_shard_size=f"1000MB",
+ device_map="auto",
 )
-```
+```
+
+Take a look at the notebook for details.