[2024-08-31 21:28:47,239][__main__][INFO] - Using GPU for training. Main device : cuda:2
[2024-08-31 21:28:47,239][__main__][INFO] - Using multiple GPUs : [2, 3, 1, 0]
[2024-08-31 21:28:54,509][__main__][INFO] - PSF shape : torch.Size([1, 380, 507, 3])
[2024-08-31 21:28:54,533][__main__][INFO] - PSF min : 0.0
[2024-08-31 21:28:54,543][__main__][INFO] - PSF max : 0.04469018802046776
[2024-08-31 21:28:54,543][__main__][INFO] - PSF dtype : torch.float32
[2024-08-31 21:28:54,567][__main__][INFO] - PSF norm : 1.000011682510376
[2024-08-31 21:28:54,568][__main__][INFO] - Reconstruction a few images with ADMM...
[2024-08-31 21:28:55,158][__main__][INFO] - Cropped shape :  (123, 123, 3)
[2024-08-31 21:28:55,374][__main__][INFO] - Cropped shape :  (123, 123, 3)
[2024-08-31 21:28:57,488][__main__][INFO] - Train test size : 21250
[2024-08-31 21:28:57,489][__main__][INFO] - Test test size : 3750
[2024-08-31 21:28:57,590][__main__][INFO] - Training model with 8102934 parameters
[2024-08-31 21:28:57,590][__main__][INFO] - -- Pre-process model with 4051744 parameters
[2024-08-31 21:28:57,591][__main__][INFO] - -- Post-process model with 4051168 parameters
[2024-08-31 21:28:57,591][__main__][INFO] - Setup time : 0.10225033760070801 s
[2024-08-31 21:28:57,591][__main__][INFO] - PSF shape : torch.Size([1, 380, 507, 3])
[2024-08-31 21:28:57,591][__main__][INFO] - Results saved in /root/FORKS/LenslessPiCamStefan/outputs/2024-08-31/21-28-42
[2024-08-31 21:37:11,087][__main__][INFO] - Epoch 0 with learning rate [0.0, 0.0]
[2024-08-31 23:17:05,837][__main__][INFO] - loss : 0.9336680045308261
[2024-08-31 23:25:18,014][__main__][INFO] - Epoch 1 with learning rate [0.0001, 0.0001]
[2024-09-01 01:05:03,839][__main__][INFO] - loss : 0.5193481093832436
[2024-09-01 01:13:11,614][__main__][INFO] - Epoch 2 with learning rate [9.957224306869053e-05, 9.957224306869053e-05]
[2024-09-01 02:52:36,264][__main__][INFO] - loss : 0.4395406529568206
[2024-09-01 03:00:40,241][__main__][INFO] - Epoch 3 with learning rate [9.829629131445342e-05, 9.829629131445342e-05]
[2024-09-01 04:40:00,249][__main__][INFO] - loss : 0.4004595386719447
[2024-09-01 04:48:08,593][__main__][INFO] - Epoch 4 with learning rate [9.619397662556435e-05, 9.619397662556435e-05]
[2024-09-01 06:27:44,762][__main__][INFO] - loss : 0.377622020103244
[2024-09-01 06:35:50,807][__main__][INFO] - Epoch 5 with learning rate [9.330127018922194e-05, 9.330127018922194e-05]
[2024-09-01 08:15:24,734][__main__][INFO] - loss : 0.3627228374795772
[2024-09-01 08:23:31,445][__main__][INFO] - Epoch 6 with learning rate [8.966766701456177e-05, 8.966766701456177e-05]
[2024-09-01 10:03:01,500][__main__][INFO] - loss : 0.3507881691546487
[2024-09-01 10:11:09,387][__main__][INFO] - Epoch 7 with learning rate [8.535533905932738e-05, 8.535533905932738e-05]
[2024-09-01 11:50:41,731][__main__][INFO] - loss : 0.34087594506959457
[2024-09-01 11:58:48,140][__main__][INFO] - Epoch 8 with learning rate [8.043807145043604e-05, 8.043807145043604e-05]
[2024-09-01 13:38:18,274][__main__][INFO] - loss : 0.3323160061336123
[2024-09-01 13:46:27,896][__main__][INFO] - Epoch 9 with learning rate [7.500000000000001e-05, 7.500000000000001e-05]
[2024-09-01 15:25:55,020][__main__][INFO] - loss : 0.3239407126633833
[2024-09-01 15:34:00,359][__main__][INFO] - Epoch 10 with learning rate [6.91341716182545e-05, 6.91341716182545e-05]
[2024-09-01 17:13:23,234][__main__][INFO] - loss : 0.31631824282206134
[2024-09-01 17:21:30,130][__main__][INFO] - Epoch 11 with learning rate [6.294095225512603e-05, 6.294095225512603e-05]
[2024-09-01 19:01:08,250][__main__][INFO] - loss : 0.3089538212468032
[2024-09-01 19:09:14,506][__main__][INFO] - Epoch 12 with learning rate [5.6526309611002594e-05, 5.6526309611002594e-05]
[2024-09-01 20:48:51,958][__main__][INFO] - loss : 0.30188819992941557
[2024-09-01 20:56:57,377][__main__][INFO] - Epoch 13 with learning rate [5e-05, 5e-05]
[2024-09-01 22:36:25,458][__main__][INFO] - loss : 0.29506130460436014
[2024-09-01 22:44:31,246][__main__][INFO] - Epoch 14 with learning rate [4.347369038899744e-05, 4.347369038899744e-05]
[2024-09-02 00:24:00,971][__main__][INFO] - loss : 0.28844775429457237
[2024-09-02 00:32:05,929][__main__][INFO] - Epoch 15 with learning rate [3.705904774487396e-05, 3.705904774487396e-05]
[2024-09-02 02:11:43,561][__main__][INFO] - loss : 0.282020980271719
[2024-09-02 02:19:53,581][__main__][INFO] - Epoch 16 with learning rate [3.086582838174551e-05, 3.086582838174551e-05]
[2024-09-02 03:59:26,406][__main__][INFO] - loss : 0.2761205089180888
[2024-09-02 04:07:32,841][__main__][INFO] - Epoch 17 with learning rate [2.500000000000001e-05, 2.500000000000001e-05]
[2024-09-02 05:47:10,054][__main__][INFO] - loss : 0.2705010529877475
[2024-09-02 05:55:18,268][__main__][INFO] - Epoch 18 with learning rate [1.9561928549563968e-05, 1.9561928549563968e-05]
[2024-09-02 07:34:49,388][__main__][INFO] - loss : 0.26540775840540054
[2024-09-02 07:42:53,843][__main__][INFO] - Epoch 19 with learning rate [1.4644660940672627e-05, 1.4644660940672627e-05]
[2024-09-02 09:22:19,223][__main__][INFO] - loss : 0.26100382957948903
[2024-09-02 09:30:24,011][__main__][INFO] - Epoch 20 with learning rate [1.0332332985438248e-05, 1.0332332985438248e-05]
[2024-09-02 11:10:36,051][__main__][INFO] - loss : 0.2572166878788258
[2024-09-02 11:18:40,116][__main__][INFO] - Epoch 21 with learning rate [6.698729810778065e-06, 6.698729810778065e-06]
[2024-09-02 12:58:04,815][__main__][INFO] - loss : 0.25426806968617616
[2024-09-02 13:06:09,277][__main__][INFO] - Epoch 22 with learning rate [3.8060233744356633e-06, 3.8060233744356633e-06]
[2024-09-02 14:45:35,634][__main__][INFO] - loss : 0.25200684457518174
[2024-09-02 14:53:39,392][__main__][INFO] - Epoch 23 with learning rate [1.70370868554659e-06, 1.70370868554659e-06]
[2024-09-02 16:33:03,454][__main__][INFO] - loss : 0.2504931665394634
[2024-09-02 16:41:07,901][__main__][INFO] - Epoch 24 with learning rate [4.277569313094809e-07, 4.277569313094809e-07]
[2024-09-02 18:20:27,417][__main__][INFO] - loss : 0.2496207069339849
[2024-09-02 18:28:30,769][__main__][INFO] - Train time [hour] : 44.99214575827122 h
[2024-09-02 18:28:30,769][__main__][INFO] - Results saved in /root/FORKS/LenslessPiCamStefan/outputs/2024-08-31/21-28-42