Update README.md
Browse files
README.md
CHANGED
|
@@ -116,7 +116,7 @@ output = model(encoded_input)
|
|
| 116 |
This bias will also affect all fine-tuned versions of this model.--->
|
| 117 |
|
| 118 |
Training data
|
| 119 |
-
The
|
| 120 |
|
| 121 |
#### Training procedure
|
| 122 |
|
|
|
|
| 116 |
This bias will also affect all fine-tuned versions of this model.--->
|
| 117 |
|
| 118 |
Training data
|
| 119 |
+
The MathBERT model was pretrained on pre-k to HS math curriculum (engageNY, Utah Math, Illustrative Math), college math books from openculture.com as well as graduate level math from arxiv math paper abstracts. There is about 100M tokens got pretrained on.
|
| 120 |
|
| 121 |
#### Training procedure
|
| 122 |
|