Pedram Rostami commited on
Commit
06ff501
1 Parent(s): e004aaf

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -92,7 +92,7 @@ model = LlamaForCausalLM.from_pretrained(
92
  )
93
  ```
94
 
95
- Alternatively, you can quantize the model in 4-bit (`INT4`) with the following code.
96
 
97
  ```python
98
  from transformers import BitsAndBytesConfig
@@ -115,7 +115,7 @@ model = LlamaForCausalLM.from_pretrained(
115
  | :----------------------------------------------------------------: | :--------------------------------------------------------------: | :------------------------------------------------------------------------: | :------------------------------------------------------------------------: | :--------: | :--------: |
116
  | <span style="font-variant:small-caps;">PersianMind</span> (`BF16`) | 73.9 | 83.61 | 79.44 | 13.7G | 25.35 |
117
  | <span style="font-variant:small-caps;">PersianMind</span> (`INT8`) | 73.7 | 82.32 | 78.61 | 7.2G | 11.36 |
118
- | <span style="font-variant:small-caps;">PersianMind</span> (`INT4`) | 70.2 | 82.07 | 80.36 | 3.9G | 24.36 |
119
 
120
  We evaluated quantized models in various tasks against the original model.
121
  Specifically, we evaluated all models using the reading comprehension multiple-choice
 
92
  )
93
  ```
94
 
95
+ Alternatively, you can quantize the model in 4-bit (`NormalFloat4`) with the following code.
96
 
97
  ```python
98
  from transformers import BitsAndBytesConfig
 
115
  | :----------------------------------------------------------------: | :--------------------------------------------------------------: | :------------------------------------------------------------------------: | :------------------------------------------------------------------------: | :--------: | :--------: |
116
  | <span style="font-variant:small-caps;">PersianMind</span> (`BF16`) | 73.9 | 83.61 | 79.44 | 13.7G | 25.35 |
117
  | <span style="font-variant:small-caps;">PersianMind</span> (`INT8`) | 73.7 | 82.32 | 78.61 | 7.2G | 11.36 |
118
+ | <span style="font-variant:small-caps;">PersianMind</span> (`NormalFloat4`) | 70.2 | 82.07 | 80.36 | 3.9G | 24.36 |
119
 
120
  We evaluated quantized models in various tasks against the original model.
121
  Specifically, we evaluated all models using the reading comprehension multiple-choice