Abhinav Kulkarni
commited on
Commit
•
ebf7f99
1
Parent(s):
2dae9a9
Updated README
Browse files
README.md
CHANGED
@@ -27,7 +27,7 @@ Please refer to the AWQ quantization license ([link](https://github.com/llm-awq/
|
|
27 |
|
28 |
## CUDA Version
|
29 |
|
30 |
-
This model was successfully tested on CUDA driver v530.30.02 and runtime v11.7 with Python v3.10.11. Please note that AWQ requires NVIDIA GPUs with compute capability of
|
31 |
|
32 |
For Docker users, the `nvcr.io/nvidia/pytorch:23.06-py3` image is runtime v12.1 but otherwise the same as the configuration above and has also been verified to work.
|
33 |
|
@@ -88,7 +88,7 @@ output = model.generate(
|
|
88 |
repetition_penalty=1.1,
|
89 |
eos_token_id=tokenizer.eos_token_id
|
90 |
)
|
91 |
-
print(tokenizer.decode(output[0], skip_special_tokens=True))
|
92 |
```
|
93 |
|
94 |
## Evaluation
|
|
|
27 |
|
28 |
## CUDA Version
|
29 |
|
30 |
+
This model was successfully tested on CUDA driver v530.30.02 and runtime v11.7 with Python v3.10.11. Please note that AWQ requires NVIDIA GPUs with compute capability of `8.0` or higher.
|
31 |
|
32 |
For Docker users, the `nvcr.io/nvidia/pytorch:23.06-py3` image is runtime v12.1 but otherwise the same as the configuration above and has also been verified to work.
|
33 |
|
|
|
88 |
repetition_penalty=1.1,
|
89 |
eos_token_id=tokenizer.eos_token_id
|
90 |
)
|
91 |
+
# print(tokenizer.decode(output[0], skip_special_tokens=True))
|
92 |
```
|
93 |
|
94 |
## Evaluation
|