new data

logs/main_log.txt

@@ -106523,3 +106523,57 @@ time (ms)
 time (ms)
  iteration     2516/  292968 | consumed samples:      5152768 | consumed tokens:    666812416 | elapsed time per iteration (ms): 126860.5 | learning rate: 1.000E-04 | global batch size:  2048 | lm loss: 3.729679E+00 | loss scale: 131072.0 | grad norm: 70559.533 | num zeros: 0.0 | curriculum seqlen:   200 | number of skipped iterations:   0 | number of nan iterations:   0 |
 time (ms)
+ iteration     2517/  292968 | consumed samples:      5154816 | consumed tokens:    667222016 | elapsed time per iteration (ms): 131649.4 | learning rate: 1.000E-04 | global batch size:  2048 | lm loss: 3.677740E+00 | loss scale: 131072.0 | grad norm: 56023.485 | num zeros: 0.0 | curriculum seqlen:   200 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)
+ iteration     2518/  292968 | consumed samples:      5156864 | consumed tokens:    667631616 | elapsed time per iteration (ms): 133698.0 | learning rate: 1.000E-04 | global batch size:  2048 | lm loss: 3.696238E+00 | loss scale: 131072.0 | grad norm: 57083.392 | num zeros: 0.0 | curriculum seqlen:   200 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)
+ iteration     2519/  292968 | consumed samples:      5158912 | consumed tokens:    668041216 | elapsed time per iteration (ms): 133130.7 | learning rate: 1.000E-04 | global batch size:  2048 | lm loss: 3.712332E+00 | loss scale: 131072.0 | grad norm: 66522.220 | num zeros: 0.0 | curriculum seqlen:   200 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)
+ iteration     2520/  292968 | consumed samples:      5160960 | consumed tokens:    668450816 | elapsed time per iteration (ms): 132776.6 | learning rate: 1.000E-04 | global batch size:  2048 | lm loss: 3.699082E+00 | loss scale: 131072.0 | grad norm: 52981.553 | num zeros: 0.0 | curriculum seqlen:   200 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)
+ iteration     2521/  292968 | consumed samples:      5163008 | consumed tokens:    668860416 | elapsed time per iteration (ms): 133609.4 | learning rate: 1.000E-04 | global batch size:  2048 | lm loss: 3.677561E+00 | loss scale: 131072.0 | grad norm: 49201.207 | num zeros: 0.0 | curriculum seqlen:   200 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)
+ iteration     2522/  292968 | consumed samples:      5165056 | consumed tokens:    669270016 | elapsed time per iteration (ms): 134264.5 | learning rate: 1.000E-04 | global batch size:  2048 | lm loss: 3.699126E+00 | loss scale: 131072.0 | grad norm: 38187.609 | num zeros: 0.0 | curriculum seqlen:   200 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)
+ iteration     2523/  292968 | consumed samples:      5167104 | consumed tokens:    669679616 | elapsed time per iteration (ms): 133050.7 | learning rate: 1.000E-04 | global batch size:  2048 | lm loss: 3.711785E+00 | loss scale: 131072.0 | grad norm: 50523.507 | num zeros: 0.0 | curriculum seqlen:   200 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)
+ iteration     2524/  292968 | consumed samples:      5169152 | consumed tokens:    670089216 | elapsed time per iteration (ms): 129836.0 | learning rate: 1.000E-04 | global batch size:  2048 | lm loss: 3.695742E+00 | loss scale: 131072.0 | grad norm: 54330.129 | num zeros: 0.0 | curriculum seqlen:   200 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)
+ iteration     2525/  292968 | consumed samples:      5171200 | consumed tokens:    670498816 | elapsed time per iteration (ms): 136356.6 | learning rate: 1.000E-04 | global batch size:  2048 | lm loss: 3.732819E+00 | loss scale: 131072.0 | grad norm: 39968.544 | num zeros: 0.0 | curriculum seqlen:   200 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)
+ iteration     2526/  292968 | consumed samples:      5173248 | consumed tokens:    670908416 | elapsed time per iteration (ms): 134571.0 | learning rate: 1.000E-04 | global batch size:  2048 | lm loss: 3.712886E+00 | loss scale: 131072.0 | grad norm: 51363.977 | num zeros: 0.0 | curriculum seqlen:   200 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)
+ iteration     2527/  292968 | consumed samples:      5175296 | consumed tokens:    671318016 | elapsed time per iteration (ms): 132047.8 | learning rate: 1.000E-04 | global batch size:  2048 | lm loss: 3.695562E+00 | loss scale: 131072.0 | grad norm: 51765.676 | num zeros: 0.0 | curriculum seqlen:   200 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)
+ iteration     2528/  292968 | consumed samples:      5177344 | consumed tokens:    671727616 | elapsed time per iteration (ms): 134158.6 | learning rate: 1.000E-04 | global batch size:  2048 | lm loss: 3.707468E+00 | loss scale: 131072.0 | grad norm: 54323.308 | num zeros: 0.0 | curriculum seqlen:   200 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)
+ iteration     2529/  292968 | consumed samples:      5179392 | consumed tokens:    672137216 | elapsed time per iteration (ms): 131022.2 | learning rate: 1.000E-04 | global batch size:  2048 | lm loss: 3.695577E+00 | loss scale: 131072.0 | grad norm: 41546.541 | num zeros: 0.0 | curriculum seqlen:   200 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)
+ iteration     2530/  292968 | consumed samples:      5181440 | consumed tokens:    672546816 | elapsed time per iteration (ms): 137329.4 | learning rate: 1.000E-04 | global batch size:  2048 | lm loss: 3.701566E+00 | loss scale: 131072.0 | grad norm: 42285.909 | num zeros: 0.0 | curriculum seqlen:   200 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)
+ iteration     2531/  292968 | consumed samples:      5183488 | consumed tokens:    672956416 | elapsed time per iteration (ms): 135951.6 | learning rate: 1.000E-04 | global batch size:  2048 | lm loss: 3.682348E+00 | loss scale: 131072.0 | grad norm: 55894.421 | num zeros: 0.0 | curriculum seqlen:   200 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)
+ iteration     2532/  292968 | consumed samples:      5185536 | consumed tokens:    673366016 | elapsed time per iteration (ms): 134684.0 | learning rate: 1.000E-04 | global batch size:  2048 | lm loss: 3.719293E+00 | loss scale: 131072.0 | grad norm: 64429.092 | num zeros: 0.0 | curriculum seqlen:   200 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)
+ iteration     2533/  292968 | consumed samples:      5187584 | consumed tokens:    673775616 | elapsed time per iteration (ms): 139215.6 | learning rate: 1.000E-04 | global batch size:  2048 | lm loss: 3.718337E+00 | loss scale: 131072.0 | grad norm: 49058.682 | num zeros: 0.0 | curriculum seqlen:   200 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)
+ iteration     2534/  292968 | consumed samples:      5189632 | consumed tokens:    674185216 | elapsed time per iteration (ms): 140178.2 | learning rate: 1.000E-04 | global batch size:  2048 | lm loss: 3.713611E+00 | loss scale: 131072.0 | grad norm: 66713.209 | num zeros: 0.0 | curriculum seqlen:   200 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)
+ iteration     2535/  292968 | consumed samples:      5191680 | consumed tokens:    674594816 | elapsed time per iteration (ms): 137068.0 | learning rate: 1.000E-04 | global batch size:  2048 | lm loss: 3.720226E+00 | loss scale: 131072.0 | grad norm: 70072.153 | num zeros: 0.0 | curriculum seqlen:   200 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)
+ iteration     2536/  292968 | consumed samples:      5193728 | consumed tokens:    675004416 | elapsed time per iteration (ms): 133750.0 | learning rate: 1.000E-04 | global batch size:  2048 | lm loss: 3.686594E+00 | loss scale: 131072.0 | grad norm: 47463.962 | num zeros: 0.0 | curriculum seqlen:   200 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)
+ iteration     2537/  292968 | consumed samples:      5195776 | consumed tokens:    675414016 | elapsed time per iteration (ms): 134502.6 | learning rate: 1.000E-04 | global batch size:  2048 | lm loss: 3.718798E+00 | loss scale: 131072.0 | grad norm: 75553.129 | num zeros: 0.0 | curriculum seqlen:   200 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)
+ iteration     2538/  292968 | consumed samples:      5197824 | consumed tokens:    675823616 | elapsed time per iteration (ms): 132873.7 | learning rate: 1.000E-04 | global batch size:  2048 | lm loss: 3.676980E+00 | loss scale: 131072.0 | grad norm: 72938.459 | num zeros: 0.0 | curriculum seqlen:   200 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)
+ iteration     2539/  292968 | consumed samples:      5199872 | consumed tokens:    676233216 | elapsed time per iteration (ms): 136842.6 | learning rate: 1.000E-04 | global batch size:  2048 | lm loss: 3.690705E+00 | loss scale: 131072.0 | grad norm: 63805.103 | num zeros: 0.0 | curriculum seqlen:   200 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)
+ iteration     2540/  292968 | consumed samples:      5201920 | consumed tokens:    676642816 | elapsed time per iteration (ms): 138332.7 | learning rate: 1.000E-04 | global batch size:  2048 | lm loss: 3.699246E+00 | loss scale: 131072.0 | grad norm: 60131.574 | num zeros: 0.0 | curriculum seqlen:   200 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)
+ iteration     2541/  292968 | consumed samples:      5203968 | consumed tokens:    677052416 | elapsed time per iteration (ms): 137209.6 | learning rate: 1.000E-04 | global batch size:  2048 | lm loss: 3.687837E+00 | loss scale: 131072.0 | grad norm: 57555.686 | num zeros: 0.0 | curriculum seqlen:   200 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)
+ iteration     2542/  292968 | consumed samples:      5206016 | consumed tokens:    677462016 | elapsed time per iteration (ms): 135834.7 | learning rate: 1.000E-04 | global batch size:  2048 | lm loss: 3.714507E+00 | loss scale: 131072.0 | grad norm: 56971.731 | num zeros: 0.0 | curriculum seqlen:   200 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)
+ iteration     2543/  292968 | consumed samples:      5208064 | consumed tokens:    677871616 | elapsed time per iteration (ms): 133073.7 | learning rate: 1.000E-04 | global batch size:  2048 | lm loss: 3.731538E+00 | loss scale: 131072.0 | grad norm: 53881.397 | num zeros: 0.0 | curriculum seqlen:   200 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)