RVN commited on
Commit
1abf532
1 Parent(s): a244173

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -9
README.md CHANGED
@@ -30,17 +30,17 @@ For training, we used all Icelandic data that was present in the monolingual Ice
30
 
31
  # Benchmark performance
32
 
33
- We tested the performance of **XLMR-MaCoCu-is** on benchmarks of XPOS, UPOS and NER. For UPOS and XPOS, we used the data from the [Universal Dependencies](https://universaldependencies.org/) project. For NER, we used the data from the MIM-GOLD-NER data set. We compare performance to the strong multi-lingual models XLMR-base and XLMR-large, but also the monolingual [IceBERT](https://huggingface.co/vesteinn/IceBERT) model. For details regarding the fine-tuning procedure you can checkout our [Github](https://github.com/macocu/LanguageModels).
34
 
35
- Scores are averages of three runs. We use the same hyperparameter settings for all models.
36
 
37
- | | **UPOS** | **UPOS** | **XPOS** | **XPOS** | **NER** | **NER** |
38
- |--------------------|:--------:|:--------:|:--------:|:--------:|---------|----------|
39
- | | **Dev** | **Test** | **Dev** | **Test** | **Dev** | **Test** |
40
- | **XLM-R-base** | 96.8 | 96.5 | 94.6 | 94.3 | 85.3 | 89.7 |
41
- | **XLM-R-large** | 97.0 | 96.7 | 94.9 | 94.7 | 88.5 | 91.7 |
42
- | **IceBERT** | 96.4 | 96.0 | 94.0 | 93.7 | 83.8 | 89.7 |
43
- | **XLMR-MaCoCu-is** | **97.3** | **97.0** | **95.4** | **95.1** | **90.8** | **93.2** |
44
 
45
  # Acknowledgements
46
 
 
30
 
31
  # Benchmark performance
32
 
33
+ We tested the performance of **XLMR-MaCoCu-is** on benchmarks of XPOS, UPOS, NER and COPA. For UPOS and XPOS, we used the data from the [Universal Dependencies](https://universaldependencies.org/) project. For NER, we used the data from the MIM-GOLD-NER data set. For COPA, we automatically translated the English data set by using Google Translate. For details please see our [Github repo](https://github.com/RikVN/COPA). We compare performance to the strong multi-lingual models XLMR-base and XLMR-large, but also the monolingual [IceBERT](https://huggingface.co/vesteinn/IceBERT) model. For details regarding the XPOS/UPOS/NER fine-tuning procedure you can checkout our [Github](https://github.com/macocu/LanguageModels).
34
 
35
+ Scores are averages of three runs, except for COPA, for which we use 10 runs. We use the same hyperparameter settings for all models.
36
 
37
+ | | **UPOS** | **UPOS** | **XPOS** | **XPOS** | **NER** | **NER** | **COPA** |
38
+ |--------------------|:--------:|:--------:|:--------:|:--------:|---------|----------| ----------|
39
+ | | **Dev** | **Test** | **Dev** | **Test** | **Dev** | **Test** | **Test** |
40
+ | **XLM-R-base** | 96.8 | 96.5 | 94.6 | 94.3 | 85.3 | 89.7 | 55.2 |
41
+ | **XLM-R-large** | 97.0 | 96.7 | 94.9 | 94.7 | 88.5 | 91.7 | 54.3 |
42
+ | **IceBERT** | 96.4 | 96.0 | 94.0 | 93.7 | 83.8 | 89.7 | 54.6 |
43
+ | **XLMR-MaCoCu-is** | **97.3** | **97.0** | **95.4** | **95.1** | **90.8** | **93.2** | **59.6** |
44
 
45
  # Acknowledgements
46