Update README.md
Browse files
README.md
CHANGED
@@ -121,7 +121,7 @@ We evaluated quantized models in various tasks against the original model.
|
|
121 |
Specifically, we evaluated all models using the reading comprehension multiple-choice
|
122 |
question-answering benchmark of [<span style="font-variant:small-caps;">Belebele</span>](https://github.com/facebookresearch/belebele) (Persian subset) and reported the accuracy of each model.
|
123 |
Additionally, we evaluated our models for Persian-to-English and English-to-Persian translation tasks.
|
124 |
-
For this, we utilized the Persian-English subset of the [Flores
|
125 |
reported our results using the <span style="font-variant:small-caps;">Comet</span> metric.
|
126 |
Furthermore, we calculated the average number of generated tokens per second by each model during running the translation tasks.
|
127 |
To understand resource efficiency, we measured the memory usage of each model by employing the `get_memory_footprint()` function.
|
|
|
121 |
Specifically, we evaluated all models using the reading comprehension multiple-choice
|
122 |
question-answering benchmark of [<span style="font-variant:small-caps;">Belebele</span>](https://github.com/facebookresearch/belebele) (Persian subset) and reported the accuracy of each model.
|
123 |
Additionally, we evaluated our models for Persian-to-English and English-to-Persian translation tasks.
|
124 |
+
For this, we utilized the Persian-English subset of the [<span style="font-variant:small-caps;">Flores</span>-200](https://github.com/facebookresearch/flores/tree/main/flores200) dataset and
|
125 |
reported our results using the <span style="font-variant:small-caps;">Comet</span> metric.
|
126 |
Furthermore, we calculated the average number of generated tokens per second by each model during running the translation tasks.
|
127 |
To understand resource efficiency, we measured the memory usage of each model by employing the `get_memory_footprint()` function.
|