Commit
·
3158261
1
Parent(s):
50fa40e
Update README.md
Browse files
README.md
CHANGED
@@ -38,7 +38,7 @@ This multilingual model can perform natural language inference (NLI) on 100 lang
|
|
38 |
zero-shot classification. The underlying model was pre-trained by Baidu, based on Meta's RoBERTa (pre-trained on the
|
39 |
[CC100 multilingual dataset](https://huggingface.co/datasets/cc100). It was then fine-tuned on the [XNLI dataset](https://huggingface.co/datasets/xnli),
|
40 |
which contains hypothesis-premise pairs from 15 languages, as well as the English [MNLI dataset](https://huggingface.co/datasets/multi_nli).
|
41 |
-
The model was introduced by Baidu in [this paper](https://arxiv.org/pdf/2012.15674.pdf).
|
42 |
|
43 |
If you are looking for a much faster (but less performant) model, you can
|
44 |
try [multilingual-MiniLMv2-L6-mnli-xnli](https://huggingface.co/MoritzLaurer/multilingual-MiniLMv2-L6-mnli-xnli).
|
@@ -114,10 +114,10 @@ other than English, the authors have most likely made a mistake during testing s
|
|
114 |
shows a multilingual average performance of more than a few points above 80% on XNLI
|
115 |
(see [here](https://arxiv.org/pdf/2111.09543.pdf) or [here](https://arxiv.org/pdf/1911.02116.pdf)).
|
116 |
|
117 |
-
|Datasets|mnli_m|mnli_mm|ar|bg|de|el|en|es|fr|hi|ru|sw|th|tr|ur|vi|zh|
|
118 |
| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
119 |
-
|Accuracy|0.881|0.878|0.818|0.853|0.84|0.837|0.882|0.855|0.849|0.799|0.83|0.751|0.809|0.818|0.76|0.826|0.799|
|
120 |
-
|Inference text/sec (A100, batch=120)|783.0|774.0|1487.0|1396.0|1430.0|1206.0|1623.0|1482.0|1291.0|1302.0|1366.0|1484.0|1500.0|1609.0|1344.0|1403.0|1302.0|
|
121 |
|
122 |
|
123 |
## Limitations and bias
|
|
|
38 |
zero-shot classification. The underlying model was pre-trained by Baidu, based on Meta's RoBERTa (pre-trained on the
|
39 |
[CC100 multilingual dataset](https://huggingface.co/datasets/cc100). It was then fine-tuned on the [XNLI dataset](https://huggingface.co/datasets/xnli),
|
40 |
which contains hypothesis-premise pairs from 15 languages, as well as the English [MNLI dataset](https://huggingface.co/datasets/multi_nli).
|
41 |
+
The model was introduced by Baidu in [this paper](https://arxiv.org/pdf/2012.15674.pdf). The model outperforms RoBERTa models of equal size.
|
42 |
|
43 |
If you are looking for a much faster (but less performant) model, you can
|
44 |
try [multilingual-MiniLMv2-L6-mnli-xnli](https://huggingface.co/MoritzLaurer/multilingual-MiniLMv2-L6-mnli-xnli).
|
|
|
114 |
shows a multilingual average performance of more than a few points above 80% on XNLI
|
115 |
(see [here](https://arxiv.org/pdf/2111.09543.pdf) or [here](https://arxiv.org/pdf/1911.02116.pdf)).
|
116 |
|
117 |
+
|Datasets|avg_xnli|mnli_m|mnli_mm|ar|bg|de|el|en|es|fr|hi|ru|sw|th|tr|ur|vi|zh|
|
118 |
| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
119 |
+
|Accuracy|0.822|0.881|0.878|0.818|0.853|0.84|0.837|0.882|0.855|0.849|0.799|0.83|0.751|0.809|0.818|0.76|0.826|0.799|
|
120 |
+
|Inference text/sec (A100, batch=120)|1415.0|783.0|774.0|1487.0|1396.0|1430.0|1206.0|1623.0|1482.0|1291.0|1302.0|1366.0|1484.0|1500.0|1609.0|1344.0|1403.0|1302.0|
|
121 |
|
122 |
|
123 |
## Limitations and bias
|