Saving weights and log at step 420000

README.md

@@ -30,7 +30,7 @@ Tokenizer:
 Training details:
 
 * Trained for 70K steps (batch size 64) to ppl 27 on mc4 nl tiny 1 epoch
-* Trained for 300K steps (batch size 16) to ppl 20.4 on mc4 nl full
+* Trained for 420K steps (batch size 16) to ppl 19.0 on mc4 nl full
 * Training continuing
 * Block size: 512
 * Optimizer: adafactor

flax_model.msgpack

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b4b4e9bbfc621b6f5f40d464b9a457e710c116168f3175beec1363fd2a531006
+oid sha256:748e3d61a600defeb512048593d4f85161ba408f30ecb0bf5d21a7b4dc9cc9df
 size 5262314590

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0ab4fce27a9dee2817a712b540727128d366cc77783f654308348bbfafd61d0c
+oid sha256:34af1f8a14a2b1db2c796487ede034c347be73b67c7a0069667f284284014d12
 size 5363100545

runs/events.out.tfevents.1641156371.t1v-n-2f64d7c8-w-0.13342.0.v2

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:76f565900752295f8ea814d3b2d748344955071342c221463fbf4b7f6e21a37c
-size 46284021
+oid sha256:87e080048d898a981df47a0956825f5be43a98e3e22d799e89c3160cbeba1f39
+size 63363863