dumitrescustefan
commited on
Commit
•
b8a3e83
1
Parent(s):
4db9018
Update README.md
Browse files
README.md
CHANGED
@@ -9,7 +9,7 @@ This is a pretrained [mt5x](https://github.com/google-research/multilingual-t5)
|
|
9 |
|
10 |
Training was performed with the span corruption task on a clean 80GB Romanian text corpus for 4M total steps with these [scripts](https://github.com/dumitrescustefan/t5x_models), starting from the 1M public mt5x-base checkpoint. The model was trained with an encoder sequence length of 512 and a decoder sequence length of 256; it has the same mt5x vocabulary as the 1M multilingual checkpoint.
|
11 |
|
12 |
-
|
13 |
|
14 |
### How to load an mt5x model
|
15 |
|
|
|
9 |
|
10 |
Training was performed with the span corruption task on a clean 80GB Romanian text corpus for 4M total steps with these [scripts](https://github.com/dumitrescustefan/t5x_models), starting from the 1M public mt5x-base checkpoint. The model was trained with an encoder sequence length of 512 and a decoder sequence length of 256; it has the same mt5x vocabulary as the 1M multilingual checkpoint.
|
11 |
|
12 |
+
**!! IMPORTANT !!** This model was pretrained on the span corruption MLM task, meaning this model is **not usable** in any downstream task **without finetuning** first!
|
13 |
|
14 |
### How to load an mt5x model
|
15 |
|