Use bfloat16 for core module.
Use parallel attention and MLP in ViViT.

Epoch 001/400
Train		loss: 121388096.00	correlation: 0.0109
Validation	loss: 200734000.00	correlation: 0.0221
Elapse: 522.91s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 002/400
Train		loss: 98318960.00	correlation: 0.0308
Validation	loss: 199955104.00	correlation: 0.0339
Elapse: 525.64s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 003/400
Train		loss: 97097640.00	correlation: 0.0413
Validation	loss: 199270752.00	correlation: 0.0396
Elapse: 531.95s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 004/400
Train		loss: 96457752.00	correlation: 0.0479
Validation	loss: 198723456.00	correlation: 0.0421
Elapse: 536.86s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 005/400
Train		loss: 95920056.00	correlation: 0.0541
Validation	loss: 197835248.00	correlation: 0.0461
Elapse: 538.89s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 006/400
Train		loss: 95131240.00	correlation: 0.0621
Validation	loss: 196873360.00	correlation: 0.0519
Elapse: 542.34s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 007/400
Train		loss: 94226616.00	correlation: 0.0710
Validation	loss: 195726576.00	correlation: 0.0578
Elapse: 541.43s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 008/400
Train		loss: 93506680.00	correlation: 0.0783
Validation	loss: 194520016.00	correlation: 0.0654
Elapse: 541.56s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 009/400
Train		loss: 92727504.00	correlation: 0.0871
Validation	loss: 193465776.00	correlation: 0.0716
Elapse: 543.31s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 010/400
Train		loss: 91843728.00	correlation: 0.0966
Validation	loss: 192176672.00	correlation: 0.0791
Elapse: 541.68s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 011/400
Train		loss: 90936304.00	correlation: 0.1063
Validation	loss: 190695968.00	correlation: 0.0928
Elapse: 540.29s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 012/400
Train		loss: 90051912.00	correlation: 0.1153
Validation	loss: 189604784.00	correlation: 0.1011
Elapse: 542.29s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 013/400
Train		loss: 89180568.00	correlation: 0.1245
Validation	loss: 188031648.00	correlation: 0.1112
Elapse: 542.62s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 014/400
Train		loss: 88307792.00	correlation: 0.1337
Validation	loss: 186159440.00	correlation: 0.1226
Elapse: 540.81s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 015/400
Train		loss: 87278960.00	correlation: 0.1436
Validation	loss: 184683440.00	correlation: 0.1313
Elapse: 543.63s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 016/400
Train		loss: 86609264.00	correlation: 0.1505
Validation	loss: 183679264.00	correlation: 0.1374
Elapse: 544.07s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 017/400
Train		loss: 85933704.00	correlation: 0.1573
Validation	loss: 182858960.00	correlation: 0.1427
Elapse: 543.60s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 018/400
Train		loss: 85316224.00	correlation: 0.1634
Validation	loss: 181758608.00	correlation: 0.1502
Elapse: 543.09s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 019/400
Train		loss: 84800144.00	correlation: 0.1686
Validation	loss: 181267248.00	correlation: 0.1538
Elapse: 545.10s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 020/400
Train		loss: 84276656.00	correlation: 0.1737
Validation	loss: 180435344.00	correlation: 0.1578
Elapse: 545.34s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 021/400
Train		loss: 83717728.00	correlation: 0.1793
Validation	loss: 179796752.00	correlation: 0.1639
Elapse: 545.93s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 022/400
Train		loss: 83306056.00	correlation: 0.1836
Validation	loss: 178887952.00	correlation: 0.1689
Elapse: 545.45s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 023/400
Train		loss: 82787680.00	correlation: 0.1887
Validation	loss: 177999440.00	correlation: 0.1749
Elapse: 545.47s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 024/400
Train		loss: 82335960.00	correlation: 0.1933
Validation	loss: 177650256.00	correlation: 0.1774
Elapse: 545.75s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 025/400
Train		loss: 82013392.00	correlation: 0.1962
Validation	loss: 177572752.00	correlation: 0.1786
Elapse: 545.69s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 026/400
Train		loss: 81675472.00	correlation: 0.1996
Validation	loss: 176845120.00	correlation: 0.1841
Elapse: 545.58s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 027/400
Train		loss: 81393008.00	correlation: 0.2025
Validation	loss: 176056032.00	correlation: 0.1886
Elapse: 545.76s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 028/400
Train		loss: 80972248.00	correlation: 0.2065
Validation	loss: 175799008.00	correlation: 0.1898
Elapse: 545.76s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 029/400
Train		loss: 80749744.00	correlation: 0.2086
Validation	loss: 175376608.00	correlation: 0.1918
Elapse: 546.06s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 030/400
Train		loss: 80453840.00	correlation: 0.2117
Validation	loss: 174830400.00	correlation: 0.1967
Elapse: 546.10s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 031/400
Train		loss: 80186704.00	correlation: 0.2145
Validation	loss: 174322544.00	correlation: 0.1984
Elapse: 545.45s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 032/400
Train		loss: 79856432.00	correlation: 0.2176
Validation	loss: 174104672.00	correlation: 0.2003
Elapse: 545.77s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 033/400
Train		loss: 79589576.00	correlation: 0.2200
Validation	loss: 173607248.00	correlation: 0.2031
Elapse: 545.80s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 034/400
Train		loss: 79380120.00	correlation: 0.2221
Validation	loss: 173319472.00	correlation: 0.2055
Elapse: 545.70s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 035/400
Train		loss: 79158640.00	correlation: 0.2243
Validation	loss: 172901472.00	correlation: 0.2089
Elapse: 545.61s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 036/400
Train		loss: 78927200.00	correlation: 0.2266
Validation	loss: 172636560.00	correlation: 0.2097
Elapse: 546.53s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 037/400
Train		loss: 78682176.00	correlation: 0.2288
Validation	loss: 172407184.00	correlation: 0.2118
Elapse: 545.83s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 038/400
Train		loss: 78582504.00	correlation: 0.2299
Validation	loss: 172238496.00	correlation: 0.2137
Elapse: 542.71s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 039/400
Train		loss: 78397632.00	correlation: 0.2314
Validation	loss: 171969648.00	correlation: 0.2147
Elapse: 545.53s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 040/400
Train		loss: 78234600.00	correlation: 0.2330
Validation	loss: 171758000.00	correlation: 0.2170
Elapse: 545.77s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 041/400
Train		loss: 78047656.00	correlation: 0.2349
Validation	loss: 171651168.00	correlation: 0.2175
Elapse: 545.10s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 042/400
Train		loss: 77868808.00	correlation: 0.2362
Validation	loss: 171537312.00	correlation: 0.2173
Elapse: 545.58s

Epoch 043/400
Train		loss: 77839128.00	correlation: 0.2370
Validation	loss: 171300016.00	correlation: 0.2185
Elapse: 546.30s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 044/400
Train		loss: 77629288.00	correlation: 0.2390
Validation	loss: 171374944.00	correlation: 0.2191
Elapse: 546.42s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 045/400
Train		loss: 77550776.00	correlation: 0.2396
Validation	loss: 171207344.00	correlation: 0.2194
Elapse: 545.96s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 046/400
Train		loss: 77437640.00	correlation: 0.2408
Validation	loss: 171176240.00	correlation: 0.2198
Elapse: 546.38s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 047/400
Train		loss: 77397712.00	correlation: 0.2412
Validation	loss: 170808800.00	correlation: 0.2224
Elapse: 545.84s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 048/400
Train		loss: 77307432.00	correlation: 0.2422
Validation	loss: 170735520.00	correlation: 0.2233
Elapse: 546.33s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 049/400
Train		loss: 77119432.00	correlation: 0.2439
Validation	loss: 170442368.00	correlation: 0.2245
Elapse: 545.45s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 050/400
Train		loss: 76988832.00	correlation: 0.2449
Validation	loss: 170672960.00	correlation: 0.2232
Elapse: 544.34s

Epoch 051/400
Train		loss: 76992464.00	correlation: 0.2450
Validation	loss: 170562112.00	correlation: 0.2244
Elapse: 545.56s

Epoch 052/400
Train		loss: 76925112.00	correlation: 0.2452
Validation	loss: 170404016.00	correlation: 0.2248
Elapse: 545.72s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 053/400
Train		loss: 76844464.00	correlation: 0.2460
Validation	loss: 170137952.00	correlation: 0.2261
Elapse: 545.92s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 054/400
Train		loss: 76812920.00	correlation: 0.2469
Validation	loss: 170023120.00	correlation: 0.2259
Elapse: 545.56s

Epoch 055/400
Train		loss: 76702536.00	correlation: 0.2475
Validation	loss: 170054384.00	correlation: 0.2263
Elapse: 544.95s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 056/400
Train		loss: 76612752.00	correlation: 0.2485
Validation	loss: 170032864.00	correlation: 0.2272
Elapse: 545.49s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 057/400
Train		loss: 76617696.00	correlation: 0.2485
Validation	loss: 169630816.00	correlation: 0.2293
Elapse: 545.90s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 058/400
Train		loss: 76465880.00	correlation: 0.2499
Validation	loss: 169825600.00	correlation: 0.2283
Elapse: 545.37s

Epoch 059/400
Train		loss: 76456912.00	correlation: 0.2499
Validation	loss: 169766688.00	correlation: 0.2284
Elapse: 545.61s

Epoch 060/400
Train		loss: 76426872.00	correlation: 0.2501
Validation	loss: 169651072.00	correlation: 0.2298
Elapse: 545.66s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 061/400
Train		loss: 76272176.00	correlation: 0.2515
Validation	loss: 169694896.00	correlation: 0.2290
Elapse: 544.26s

Epoch 062/400
Train		loss: 76381040.00	correlation: 0.2506
Validation	loss: 169396208.00	correlation: 0.2316
Elapse: 544.91s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 063/400
Train		loss: 76206744.00	correlation: 0.2523
Validation	loss: 169683888.00	correlation: 0.2297
Elapse: 544.69s

Epoch 064/400
Train		loss: 76167792.00	correlation: 0.2527
Validation	loss: 169194928.00	correlation: 0.2329
Elapse: 545.55s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 065/400
Train		loss: 76127384.00	correlation: 0.2528
Validation	loss: 169763808.00	correlation: 0.2287
Elapse: 545.54s

Epoch 066/400
Train		loss: 76104336.00	correlation: 0.2531
Validation	loss: 169165280.00	correlation: 0.2327
Elapse: 545.65s

Epoch 067/400
Train		loss: 76021512.00	correlation: 0.2539
Validation	loss: 169219872.00	correlation: 0.2317
Elapse: 545.30s

Epoch 068/400
Train		loss: 76148160.00	correlation: 0.2528
Validation	loss: 169344656.00	correlation: 0.2314
Elapse: 545.59s

Epoch 069/400
Train		loss: 75939984.00	correlation: 0.2545
Validation	loss: 169043376.00	correlation: 0.2339
Elapse: 545.18s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 070/400
Train		loss: 75822872.00	correlation: 0.2557
Validation	loss: 169167760.00	correlation: 0.2328
Elapse: 544.88s

Epoch 071/400
Train		loss: 75873072.00	correlation: 0.2551
Validation	loss: 169123712.00	correlation: 0.2336
Elapse: 545.60s

Epoch 072/400
Train		loss: 75867984.00	correlation: 0.2552
Validation	loss: 168947232.00	correlation: 0.2344
Elapse: 545.61s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 073/400
Train		loss: 75840912.00	correlation: 0.2555
Validation	loss: 169026784.00	correlation: 0.2348
Elapse: 545.60s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 074/400
Train		loss: 75855384.00	correlation: 0.2555
Validation	loss: 168842592.00	correlation: 0.2341
Elapse: 545.99s

Epoch 075/400
Train		loss: 75737744.00	correlation: 0.2565
Validation	loss: 168971312.00	correlation: 0.2336
Elapse: 545.40s

Epoch 076/400
Train		loss: 75735656.00	correlation: 0.2565
Validation	loss: 168829920.00	correlation: 0.2338
Elapse: 546.00s

Epoch 077/400
Train		loss: 75607888.00	correlation: 0.2576
Validation	loss: 168935968.00	correlation: 0.2349
Elapse: 545.47s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 078/400
Train		loss: 75679128.00	correlation: 0.2571
Validation	loss: 168987280.00	correlation: 0.2328
Elapse: 545.11s

Epoch 079/400
Train		loss: 75621152.00	correlation: 0.2574
Validation	loss: 168822512.00	correlation: 0.2347
Elapse: 545.59s

Epoch 080/400
Train		loss: 75483728.00	correlation: 0.2590
Validation	loss: 168774416.00	correlation: 0.2347
Elapse: 545.47s

Epoch 081/400
Train		loss: 75531304.00	correlation: 0.2586
Validation	loss: 169126304.00	correlation: 0.2331
Elapse: 545.29s

Epoch 082/400
Train		loss: 75558736.00	correlation: 0.2583
Validation	loss: 168670160.00	correlation: 0.2354
Elapse: 545.72s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 083/400
Train		loss: 75590176.00	correlation: 0.2579
Validation	loss: 169091472.00	correlation: 0.2330
Elapse: 545.76s

Epoch 084/400
Train		loss: 75406416.00	correlation: 0.2595
Validation	loss: 169084192.00	correlation: 0.2329
Elapse: 545.78s

Epoch 085/400
Train		loss: 75376912.00	correlation: 0.2599
Validation	loss: 168861952.00	correlation: 0.2341
Elapse: 545.89s

Epoch 086/400
Train		loss: 75349048.00	correlation: 0.2604
Validation	loss: 168882304.00	correlation: 0.2349
Elapse: 545.73s

Epoch 087/400
Train		loss: 75302984.00	correlation: 0.2607
Validation	loss: 169006784.00	correlation: 0.2333
Elapse: 545.26s

Loaded checkpoint from epoch 82 (correlation: 0.2354).

Reduce learning rate of core to 1.4400e-03 (num. reduce: 1).
Reduce learning rate of readouts to 1.0800e-03 (num. reduce: 1).
Reduce learning rate of shifters to 1.0800e-03 (num. reduce: 1).

Epoch 088/400
Train		loss: 73472912.00	correlation: 0.2760
Validation	loss: 166967968.00	correlation: 0.2466
Elapse: 546.46s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 089/400
Train		loss: 72834528.00	correlation: 0.2820
Validation	loss: 166834160.00	correlation: 0.2485
Elapse: 546.50s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 090/400
Train		loss: 72607408.00	correlation: 0.2844
Validation	loss: 166764992.00	correlation: 0.2490
Elapse: 546.18s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 091/400
Train		loss: 72571872.00	correlation: 0.2846
Validation	loss: 166824368.00	correlation: 0.2476
Elapse: 546.25s

Epoch 092/400
Train		loss: 72491600.00	correlation: 0.2854
Validation	loss: 166791872.00	correlation: 0.2474
Elapse: 545.90s

Epoch 093/400
Train		loss: 72479560.00	correlation: 0.2856
Validation	loss: 166900064.00	correlation: 0.2473
Elapse: 546.88s

Epoch 094/400
Train		loss: 72391640.00	correlation: 0.2867
Validation	loss: 166895872.00	correlation: 0.2471
Elapse: 546.67s

Epoch 095/400
Train		loss: 72406800.00	correlation: 0.2865
Validation	loss: 166785824.00	correlation: 0.2477
Elapse: 546.76s

Loaded checkpoint from epoch 90 (correlation: 0.2490).

Reduce learning rate of core to 4.3200e-04 (num. reduce: 1).
Reduce learning rate of readouts to 3.2400e-04 (num. reduce: 1).
Reduce learning rate of shifters to 3.2400e-04 (num. reduce: 1).

Epoch 096/400
Train		loss: 71820392.00	correlation: 0.2914
Validation	loss: 166357664.00	correlation: 0.2510
Elapse: 546.29s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 097/400
Train		loss: 71646128.00	correlation: 0.2926
Validation	loss: 166358480.00	correlation: 0.2512
Elapse: 546.63s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 098/400
Train		loss: 71469944.00	correlation: 0.2943
Validation	loss: 166261168.00	correlation: 0.2520
Elapse: 547.33s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 099/400
Train		loss: 71457032.00	correlation: 0.2945
Validation	loss: 166242256.00	correlation: 0.2517
Elapse: 547.22s

Epoch 100/400
Train		loss: 71381520.00	correlation: 0.2953
Validation	loss: 166269072.00	correlation: 0.2516
Elapse: 546.93s

Epoch 101/400
Train		loss: 71365904.00	correlation: 0.2954
Validation	loss: 166306016.00	correlation: 0.2512
Elapse: 547.14s

Epoch 102/400
Train		loss: 71313408.00	correlation: 0.2956
Validation	loss: 166202720.00	correlation: 0.2520
Elapse: 547.18s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 103/400
Train		loss: 71246520.00	correlation: 0.2963
Validation	loss: 166201664.00	correlation: 0.2519
Elapse: 546.59s

Epoch 104/400
Train		loss: 71237544.00	correlation: 0.2967
Validation	loss: 166225280.00	correlation: 0.2516
Elapse: 546.47s

Epoch 105/400
Train		loss: 71175248.00	correlation: 0.2973
Validation	loss: 166234304.00	correlation: 0.2515
Elapse: 546.73s

Epoch 106/400
Train		loss: 71139096.00	correlation: 0.2976
Validation	loss: 166179840.00	correlation: 0.2518
Elapse: 547.27s

Epoch 107/400
Train		loss: 71152664.00	correlation: 0.2972
Validation	loss: 166277328.00	correlation: 0.2509
Elapse: 547.28s

Loaded checkpoint from epoch 102 (correlation: 0.2520).

Reduce learning rate of core to 1.2960e-04 (num. reduce: 1).
Reduce learning rate of readouts to 9.7200e-05 (num. reduce: 1).
Reduce learning rate of shifters to 9.7200e-05 (num. reduce: 1).

Epoch 108/400
Train		loss: 70975272.00	correlation: 0.2990
Validation	loss: 166102416.00	correlation: 0.2526
Elapse: 547.38s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 109/400
Train		loss: 70927400.00	correlation: 0.2990
Validation	loss: 166090848.00	correlation: 0.2526
Elapse: 547.10s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 110/400
Train		loss: 70928176.00	correlation: 0.2993
Validation	loss: 166087520.00	correlation: 0.2526
Elapse: 547.24s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 111/400
Train		loss: 70948912.00	correlation: 0.2989
Validation	loss: 166067424.00	correlation: 0.2526
Elapse: 547.30s

Epoch 112/400
Train		loss: 70843856.00	correlation: 0.2999
Validation	loss: 166089904.00	correlation: 0.2527
Elapse: 547.44s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 113/400
Train		loss: 70930864.00	correlation: 0.2991
Validation	loss: 166048192.00	correlation: 0.2527
Elapse: 547.22s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 114/400
Train		loss: 70888224.00	correlation: 0.2998
Validation	loss: 166076336.00	correlation: 0.2525
Elapse: 547.07s

Epoch 115/400
Train		loss: 70807400.00	correlation: 0.3003
Validation	loss: 166078992.00	correlation: 0.2524
Elapse: 547.34s

Epoch 116/400
Train		loss: 70784824.00	correlation: 0.3006
Validation	loss: 166036512.00	correlation: 0.2528
Elapse: 547.33s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 117/400
Train		loss: 70760888.00	correlation: 0.3007
Validation	loss: 166109792.00	correlation: 0.2523
Elapse: 547.36s

Epoch 118/400
Train		loss: 70793224.00	correlation: 0.3003
Validation	loss: 166033088.00	correlation: 0.2527
Elapse: 547.66s

Epoch 119/400
Train		loss: 70801416.00	correlation: 0.3002
Validation	loss: 166062960.00	correlation: 0.2529
Elapse: 547.68s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 120/400
Train		loss: 70748944.00	correlation: 0.3009
Validation	loss: 166063152.00	correlation: 0.2524
Elapse: 547.18s

Epoch 121/400
Train		loss: 70736464.00	correlation: 0.3010
Validation	loss: 166058976.00	correlation: 0.2528
Elapse: 547.53s

Epoch 122/400
Train		loss: 70653008.00	correlation: 0.3019
Validation	loss: 166041920.00	correlation: 0.2527
Elapse: 547.68s

Epoch 123/400
Train		loss: 70723656.00	correlation: 0.3011
Validation	loss: 166053344.00	correlation: 0.2525
Elapse: 547.37s

Epoch 124/400
Train		loss: 70689936.00	correlation: 0.3016
Validation	loss: 166053472.00	correlation: 0.2526
Elapse: 547.93s

Loaded checkpoint from epoch 119 (correlation: 0.2529).

Reduce learning rate of core to 3.8880e-05 (num. reduce: 1).
Reduce learning rate of readouts to 2.9160e-05 (num. reduce: 1).
Reduce learning rate of shifters to 2.9160e-05 (num. reduce: 1).

Epoch 125/400
Train		loss: 70604440.00	correlation: 0.3023
Validation	loss: 166025312.00	correlation: 0.2529
Elapse: 547.63s

Epoch 126/400
Train		loss: 70608160.00	correlation: 0.3021
Validation	loss: 166010512.00	correlation: 0.2530
Elapse: 547.36s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 127/400
Train		loss: 70626448.00	correlation: 0.3020
Validation	loss: 166023648.00	correlation: 0.2528
Elapse: 547.13s

Epoch 128/400
Train		loss: 70612096.00	correlation: 0.3018
Validation	loss: 166023216.00	correlation: 0.2529
Elapse: 546.99s

Epoch 129/400
Train		loss: 70635640.00	correlation: 0.3020
Validation	loss: 166036208.00	correlation: 0.2528
Elapse: 547.23s

Epoch 130/400
Train		loss: 70666920.00	correlation: 0.3015
Validation	loss: 166035808.00	correlation: 0.2527
Elapse: 546.99s

Epoch 131/400
Train		loss: 70564592.00	correlation: 0.3028
Validation	loss: 166005856.00	correlation: 0.2528
Elapse: 547.30s

Loaded checkpoint from epoch 126 (correlation: 0.2530).

Reduce learning rate of core to 1.1664e-05 (num. reduce: 1).
Reduce learning rate of readouts to 8.7480e-06 (num. reduce: 1).
Reduce learning rate of shifters to 8.7480e-06 (num. reduce: 1).

Epoch 132/400
Train		loss: 70603848.00	correlation: 0.3022
Validation	loss: 166004944.00	correlation: 0.2530
Elapse: 547.32s

Checkpoint saved to /home/storage/runs/vivit_ensemble/009/ckpt/model_state.pt.

Epoch 133/400
Train		loss: 70533872.00	correlation: 0.3031
Validation	loss: 166019360.00	correlation: 0.2529
Elapse: 547.50s

Epoch 134/400
Train		loss: 70635248.00	correlation: 0.3019
Validation	loss: 166021888.00	correlation: 0.2529
Elapse: 547.34s

Epoch 135/400
Train		loss: 70620176.00	correlation: 0.3020
Validation	loss: 166009888.00	correlation: 0.2529
Elapse: 546.38s

Epoch 136/400
Train		loss: 70572368.00	correlation: 0.3023
Validation	loss: 166012144.00	correlation: 0.2529
Elapse: 547.10s

Epoch 137/400
Train		loss: 70658640.00	correlation: 0.3016
Validation	loss: 166018080.00	correlation: 0.2529
Elapse: 546.90s

Loaded checkpoint from epoch 132 (correlation: 0.2530).

Reduce learning rate of core to 3.4992e-06 (num. reduce: 1).
Reduce learning rate of readouts to 2.6244e-06 (num. reduce: 1).
Reduce learning rate of shifters to 2.6244e-06 (num. reduce: 1).

Epoch 138/400
Train		loss: 70545520.00	correlation: 0.3026
Validation	loss: 166011936.00	correlation: 0.2530
Elapse: 546.72s

Epoch 139/400
Train		loss: 70622800.00	correlation: 0.3018
Validation	loss: 166013088.00	correlation: 0.2529
Elapse: 547.01s

Epoch 140/400
Train		loss: 70572936.00	correlation: 0.3025
Validation	loss: 166013696.00	correlation: 0.2529
Elapse: 547.38s

Epoch 141/400
Train		loss: 70624016.00	correlation: 0.3020
Validation	loss: 166012496.00	correlation: 0.2530
Elapse: 546.98s

Epoch 142/400
Train		loss: 70651392.00	correlation: 0.3018
Validation	loss: 166017120.00	correlation: 0.2529
Elapse: 547.57s

Loaded checkpoint from epoch 132 (correlation: 0.2530).

Reduce learning rate of core to 1.0498e-06 (num. reduce: 2).
Reduce learning rate of readouts to 7.8732e-07 (num. reduce: 2).
Reduce learning rate of shifters to 7.8732e-07 (num. reduce: 2).

Epoch 143/400
Train		loss: 70616608.00	correlation: 0.3018
Validation	loss: 166009488.00	correlation: 0.2530
Elapse: 547.46s

Epoch 144/400
Train		loss: 70557224.00	correlation: 0.3027
Validation	loss: 166010240.00	correlation: 0.2530
Elapse: 547.42s

Epoch 145/400
Train		loss: 70617064.00	correlation: 0.3020
Validation	loss: 166011824.00	correlation: 0.2530
Elapse: 547.48s

Epoch 146/400
Train		loss: 70601720.00	correlation: 0.3022
Validation	loss: 166010880.00	correlation: 0.2530
Elapse: 547.26s

Epoch 147/400
Train		loss: 70616768.00	correlation: 0.3021
Validation	loss: 166012944.00	correlation: 0.2530
Elapse: 547.43s

Model has not improved after 2 LR reductions.

Loaded checkpoint from epoch 132 (correlation: 0.2530).


ValidationA: 0.2508	B: 0.2804	C: 0.2750	D: 0.2393	E: 0.2390	F: 0.2346	G: 0.2573	H: 0.2379	I: 0.2555	J: 0.2599	average: 0.2530

Results saved to /home/storage/runs/vivit_ensemble/009.