josecannete
commited on
Commit
•
d1299df
1
Parent(s):
9067a0f
Update README.md
Browse files
README.md
CHANGED
@@ -1,5 +1,3 @@
|
|
1 |
-
** **This is work in progress** **
|
2 |
-
|
3 |
# BETO: Spanish BERT
|
4 |
|
5 |
BETO is a [BERT model](https://github.com/google-research/bert) trained on a [big Spanish corpus](https://github.com/josecannete/spanish-corpora). BETO is of size similar to a BERT-Base and was trained with the Whole Word Masking technique. Below you find Tensorflow and Pytorch checkpoints for the uncased and cased versions, as well as some results for Spanish benchmarks comparing BETO with [Multilingual BERT](https://github.com/google-research/bert/blob/master/multilingual.md) as well as other (not BERT-based) models.
|
@@ -57,6 +55,11 @@ To cite this resource in a publication please use the following:
|
|
57 |
}
|
58 |
```
|
59 |
|
|
|
|
|
|
|
|
|
|
|
60 |
## References
|
61 |
|
62 |
* [1] [Original Multilingual BERT](https://github.com/google-research/bert/blob/master/multilingual.md)
|
@@ -66,4 +69,4 @@ To cite this resource in a publication please use the following:
|
|
66 |
* [5] [XLM (MLM+TLM)](https://arxiv.org/pdf/1901.07291.pdf)
|
67 |
* [6] [UDPipe on "75 Languages, 1 Model: Parsing Universal Dependencies Universally"](https://arxiv.org/pdf/1904.02099.pdf)
|
68 |
* [7] [Multilingual BERT on "Sequence Tagging with Contextual and Non-Contextual Subword Representations: A Multilingual Evaluation"](https://arxiv.org/pdf/1906.01569.pdf)
|
69 |
-
* [8] [Multilingual BERT on "PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identification"](https://arxiv.org/abs/1908.11828)
|
|
|
|
|
|
|
1 |
# BETO: Spanish BERT
|
2 |
|
3 |
BETO is a [BERT model](https://github.com/google-research/bert) trained on a [big Spanish corpus](https://github.com/josecannete/spanish-corpora). BETO is of size similar to a BERT-Base and was trained with the Whole Word Masking technique. Below you find Tensorflow and Pytorch checkpoints for the uncased and cased versions, as well as some results for Spanish benchmarks comparing BETO with [Multilingual BERT](https://github.com/google-research/bert/blob/master/multilingual.md) as well as other (not BERT-based) models.
|
|
|
55 |
}
|
56 |
```
|
57 |
|
58 |
+
|
59 |
+
## License Disclaimer
|
60 |
+
The license CC BY 4.0 best describes our intentions for our work. However we are not sure that all the datasets used to train BETO have licenses compatible with CC BY 4.0 (specially for commercial use). Please use at your own discretion and verify that the licenses of the original text resources match your needs.
|
61 |
+
|
62 |
+
|
63 |
## References
|
64 |
|
65 |
* [1] [Original Multilingual BERT](https://github.com/google-research/bert/blob/master/multilingual.md)
|
|
|
69 |
* [5] [XLM (MLM+TLM)](https://arxiv.org/pdf/1901.07291.pdf)
|
70 |
* [6] [UDPipe on "75 Languages, 1 Model: Parsing Universal Dependencies Universally"](https://arxiv.org/pdf/1904.02099.pdf)
|
71 |
* [7] [Multilingual BERT on "Sequence Tagging with Contextual and Non-Contextual Subword Representations: A Multilingual Evaluation"](https://arxiv.org/pdf/1906.01569.pdf)
|
72 |
+
* [8] [Multilingual BERT on "PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identification"](https://arxiv.org/abs/1908.11828)
|