Pedram Rostami
commited on
Commit
•
06ff501
1
Parent(s):
e004aaf
Update README.md
Browse files
README.md
CHANGED
@@ -92,7 +92,7 @@ model = LlamaForCausalLM.from_pretrained(
|
|
92 |
)
|
93 |
```
|
94 |
|
95 |
-
Alternatively, you can quantize the model in 4-bit (`
|
96 |
|
97 |
```python
|
98 |
from transformers import BitsAndBytesConfig
|
@@ -115,7 +115,7 @@ model = LlamaForCausalLM.from_pretrained(
|
|
115 |
| :----------------------------------------------------------------: | :--------------------------------------------------------------: | :------------------------------------------------------------------------: | :------------------------------------------------------------------------: | :--------: | :--------: |
|
116 |
| <span style="font-variant:small-caps;">PersianMind</span> (`BF16`) | 73.9 | 83.61 | 79.44 | 13.7G | 25.35 |
|
117 |
| <span style="font-variant:small-caps;">PersianMind</span> (`INT8`) | 73.7 | 82.32 | 78.61 | 7.2G | 11.36 |
|
118 |
-
| <span style="font-variant:small-caps;">PersianMind</span> (`
|
119 |
|
120 |
We evaluated quantized models in various tasks against the original model.
|
121 |
Specifically, we evaluated all models using the reading comprehension multiple-choice
|
|
|
92 |
)
|
93 |
```
|
94 |
|
95 |
+
Alternatively, you can quantize the model in 4-bit (`NormalFloat4`) with the following code.
|
96 |
|
97 |
```python
|
98 |
from transformers import BitsAndBytesConfig
|
|
|
115 |
| :----------------------------------------------------------------: | :--------------------------------------------------------------: | :------------------------------------------------------------------------: | :------------------------------------------------------------------------: | :--------: | :--------: |
|
116 |
| <span style="font-variant:small-caps;">PersianMind</span> (`BF16`) | 73.9 | 83.61 | 79.44 | 13.7G | 25.35 |
|
117 |
| <span style="font-variant:small-caps;">PersianMind</span> (`INT8`) | 73.7 | 82.32 | 78.61 | 7.2G | 11.36 |
|
118 |
+
| <span style="font-variant:small-caps;">PersianMind</span> (`NormalFloat4`) | 70.2 | 82.07 | 80.36 | 3.9G | 24.36 |
|
119 |
|
120 |
We evaluated quantized models in various tasks against the original model.
|
121 |
Specifically, we evaluated all models using the reading comprehension multiple-choice
|