hyunwoongko
commited on
Commit
·
f7a396b
1
Parent(s):
eae0ac9
Update README.md
Browse files
README.md
CHANGED
@@ -58,7 +58,7 @@ Furthermore, in order to avoid the model memorizing and generating personally id
|
|
58 |
* `<|tell|>` : phone number
|
59 |
|
60 |
## Training procedure
|
61 |
-
Polyglot-Ko-5.8B was trained for
|
62 |
|
63 |
## How to use
|
64 |
|
|
|
58 |
* `<|tell|>` : phone number
|
59 |
|
60 |
## Training procedure
|
61 |
+
Polyglot-Ko-5.8B was trained for 172 billion tokens over 320,000 steps on 256 A100 GPUs with the [GPT-NeoX framework](https://github.com/EleutherAI/gpt-neox). It was trained as an autoregressive language model, using cross-entropy loss to maximize the likelihood of predicting the next token.
|
62 |
|
63 |
## How to use
|
64 |
|