hyunwoongko commited on
Commit
f7a396b
·
1 Parent(s): eae0ac9

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -58,7 +58,7 @@ Furthermore, in order to avoid the model memorizing and generating personally id
58
  * `<|tell|>` : phone number
59
 
60
  ## Training procedure
61
- Polyglot-Ko-5.8B was trained for 219 billion tokens over 105,000 steps on 256 A100 GPUs with the [GPT-NeoX framework](https://github.com/EleutherAI/gpt-neox). It was trained as an autoregressive language model, using cross-entropy loss to maximize the likelihood of predicting the next token.
62
 
63
  ## How to use
64
 
 
58
  * `<|tell|>` : phone number
59
 
60
  ## Training procedure
61
+ Polyglot-Ko-5.8B was trained for 172 billion tokens over 320,000 steps on 256 A100 GPUs with the [GPT-NeoX framework](https://github.com/EleutherAI/gpt-neox). It was trained as an autoregressive language model, using cross-entropy loss to maximize the likelihood of predicting the next token.
62
 
63
  ## How to use
64