diff --git "a/sf_log.txt" "b/sf_log.txt" --- "a/sf_log.txt" +++ "b/sf_log.txt" @@ -1,50 +1,50 @@ -[2024-09-04 14:16:33,353][00226] Saving configuration to /content/train_dir/default_experiment/config.json... -[2024-09-04 14:16:33,356][00226] Rollout worker 0 uses device cpu -[2024-09-04 14:16:33,357][00226] Rollout worker 1 uses device cpu -[2024-09-04 14:16:33,359][00226] Rollout worker 2 uses device cpu -[2024-09-04 14:16:33,361][00226] Rollout worker 3 uses device cpu -[2024-09-04 14:16:33,363][00226] Rollout worker 4 uses device cpu -[2024-09-04 14:16:33,365][00226] Rollout worker 5 uses device cpu -[2024-09-04 14:16:33,366][00226] Rollout worker 6 uses device cpu -[2024-09-04 14:16:33,367][00226] Rollout worker 7 uses device cpu -[2024-09-04 14:16:33,526][00226] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2024-09-04 14:16:33,527][00226] InferenceWorker_p0-w0: min num requests: 2 -[2024-09-04 14:16:33,560][00226] Starting all processes... -[2024-09-04 14:16:33,562][00226] Starting process learner_proc0 -[2024-09-04 14:16:34,260][00226] Starting all processes... -[2024-09-04 14:16:34,280][00226] Starting process inference_proc0-0 -[2024-09-04 14:16:34,292][00226] Starting process rollout_proc4 -[2024-09-04 14:16:34,289][00226] Starting process rollout_proc1 -[2024-09-04 14:16:34,289][00226] Starting process rollout_proc2 -[2024-09-04 14:16:34,291][00226] Starting process rollout_proc3 -[2024-09-04 14:16:34,292][00226] Starting process rollout_proc5 -[2024-09-04 14:16:34,292][00226] Starting process rollout_proc6 -[2024-09-04 14:16:34,292][00226] Starting process rollout_proc7 -[2024-09-04 14:16:34,285][00226] Starting process rollout_proc0 -[2024-09-04 14:16:50,953][02746] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2024-09-04 14:16:50,953][02746] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 -[2024-09-04 14:16:51,073][02746] Num visible devices: 1 -[2024-09-04 14:16:51,357][02733] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2024-09-04 14:16:51,366][02733] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 -[2024-09-04 14:16:51,456][02748] Worker 1 uses CPU cores [1] -[2024-09-04 14:16:51,500][02750] Worker 3 uses CPU cores [1] -[2024-09-04 14:16:51,538][02733] Num visible devices: 1 -[2024-09-04 14:16:51,616][02733] Starting seed is not provided -[2024-09-04 14:16:51,618][02733] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2024-09-04 14:16:51,618][02733] Initializing actor-critic model on device cuda:0 -[2024-09-04 14:16:51,619][02733] RunningMeanStd input shape: (3, 72, 128) -[2024-09-04 14:16:51,623][02733] RunningMeanStd input shape: (1,) -[2024-09-04 14:16:51,770][02733] ConvEncoder: input_channels=3 -[2024-09-04 14:16:52,047][02752] Worker 5 uses CPU cores [1] -[2024-09-04 14:16:52,191][02747] Worker 4 uses CPU cores [0] -[2024-09-04 14:16:52,201][02753] Worker 7 uses CPU cores [1] -[2024-09-04 14:16:52,206][02749] Worker 2 uses CPU cores [0] -[2024-09-04 14:16:52,262][02751] Worker 6 uses CPU cores [0] -[2024-09-04 14:16:52,289][02754] Worker 0 uses CPU cores [0] -[2024-09-04 14:16:52,355][02733] Conv encoder output size: 512 -[2024-09-04 14:16:52,356][02733] Policy head output size: 512 -[2024-09-04 14:16:52,414][02733] Created Actor Critic model with architecture: -[2024-09-04 14:16:52,414][02733] ActorCriticSharedWeights( +[2024-09-05 08:53:10,744][00556] Saving configuration to /content/train_dir/default_experiment/config.json... +[2024-09-05 08:53:10,748][00556] Rollout worker 0 uses device cpu +[2024-09-05 08:53:10,751][00556] Rollout worker 1 uses device cpu +[2024-09-05 08:53:10,754][00556] Rollout worker 2 uses device cpu +[2024-09-05 08:53:10,758][00556] Rollout worker 3 uses device cpu +[2024-09-05 08:53:10,764][00556] Rollout worker 4 uses device cpu +[2024-09-05 08:53:10,768][00556] Rollout worker 5 uses device cpu +[2024-09-05 08:53:10,770][00556] Rollout worker 6 uses device cpu +[2024-09-05 08:53:10,773][00556] Rollout worker 7 uses device cpu +[2024-09-05 08:53:11,017][00556] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2024-09-05 08:53:11,026][00556] InferenceWorker_p0-w0: min num requests: 2 +[2024-09-05 08:53:11,074][00556] Starting all processes... +[2024-09-05 08:53:11,079][00556] Starting process learner_proc0 +[2024-09-05 08:53:12,271][00556] Starting all processes... +[2024-09-05 08:53:12,347][00556] Starting process inference_proc0-0 +[2024-09-05 08:53:12,348][00556] Starting process rollout_proc0 +[2024-09-05 08:53:12,348][00556] Starting process rollout_proc1 +[2024-09-05 08:53:12,353][00556] Starting process rollout_proc2 +[2024-09-05 08:53:12,358][00556] Starting process rollout_proc3 +[2024-09-05 08:53:12,358][00556] Starting process rollout_proc4 +[2024-09-05 08:53:12,358][00556] Starting process rollout_proc5 +[2024-09-05 08:53:12,358][00556] Starting process rollout_proc6 +[2024-09-05 08:53:12,376][00556] Starting process rollout_proc7 +[2024-09-05 08:53:29,018][02575] Worker 2 uses CPU cores [0] +[2024-09-05 08:53:29,287][02573] Worker 0 uses CPU cores [0] +[2024-09-05 08:53:29,529][02559] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2024-09-05 08:53:29,534][02559] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 +[2024-09-05 08:53:29,589][02559] Num visible devices: 1 +[2024-09-05 08:53:29,621][02559] Starting seed is not provided +[2024-09-05 08:53:29,622][02559] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2024-09-05 08:53:29,623][02559] Initializing actor-critic model on device cuda:0 +[2024-09-05 08:53:29,624][02559] RunningMeanStd input shape: (3, 72, 128) +[2024-09-05 08:53:29,627][02559] RunningMeanStd input shape: (1,) +[2024-09-05 08:53:29,675][02578] Worker 6 uses CPU cores [0] +[2024-09-05 08:53:29,725][02559] ConvEncoder: input_channels=3 +[2024-09-05 08:53:29,748][02579] Worker 5 uses CPU cores [1] +[2024-09-05 08:53:29,777][02577] Worker 4 uses CPU cores [0] +[2024-09-05 08:53:29,849][02574] Worker 1 uses CPU cores [1] +[2024-09-05 08:53:29,863][02572] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2024-09-05 08:53:29,864][02572] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 +[2024-09-05 08:53:29,887][02572] Num visible devices: 1 +[2024-09-05 08:53:29,912][02580] Worker 7 uses CPU cores [1] +[2024-09-05 08:53:29,936][02576] Worker 3 uses CPU cores [1] +[2024-09-05 08:53:30,040][02559] Conv encoder output size: 512 +[2024-09-05 08:53:30,040][02559] Policy head output size: 512 +[2024-09-05 08:53:30,099][02559] Created Actor Critic model with architecture: +[2024-09-05 08:53:30,099][02559] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( @@ -85,1058 +85,1046 @@ (distribution_linear): Linear(in_features=512, out_features=5, bias=True) ) ) -[2024-09-04 14:16:52,895][02733] Using optimizer -[2024-09-04 14:16:53,518][00226] Heartbeat connected on Batcher_0 -[2024-09-04 14:16:53,526][00226] Heartbeat connected on InferenceWorker_p0-w0 -[2024-09-04 14:16:53,535][00226] Heartbeat connected on RolloutWorker_w0 -[2024-09-04 14:16:53,539][00226] Heartbeat connected on RolloutWorker_w1 -[2024-09-04 14:16:53,542][00226] Heartbeat connected on RolloutWorker_w2 -[2024-09-04 14:16:53,546][00226] Heartbeat connected on RolloutWorker_w3 -[2024-09-04 14:16:53,550][00226] Heartbeat connected on RolloutWorker_w4 -[2024-09-04 14:16:53,553][00226] Heartbeat connected on RolloutWorker_w5 -[2024-09-04 14:16:53,557][00226] Heartbeat connected on RolloutWorker_w6 -[2024-09-04 14:16:53,560][00226] Heartbeat connected on RolloutWorker_w7 -[2024-09-04 14:16:53,792][02733] No checkpoints found -[2024-09-04 14:16:53,793][02733] Did not load from checkpoint, starting from scratch! -[2024-09-04 14:16:53,794][02733] Initialized policy 0 weights for model version 0 -[2024-09-04 14:16:53,799][02733] LearnerWorker_p0 finished initialization! -[2024-09-04 14:16:53,801][02733] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2024-09-04 14:16:53,800][00226] Heartbeat connected on LearnerWorker_p0 -[2024-09-04 14:16:53,997][02746] RunningMeanStd input shape: (3, 72, 128) -[2024-09-04 14:16:53,998][02746] RunningMeanStd input shape: (1,) -[2024-09-04 14:16:54,012][02746] ConvEncoder: input_channels=3 -[2024-09-04 14:16:54,116][02746] Conv encoder output size: 512 -[2024-09-04 14:16:54,117][02746] Policy head output size: 512 -[2024-09-04 14:16:54,169][00226] Inference worker 0-0 is ready! -[2024-09-04 14:16:54,170][00226] All inference workers are ready! Signal rollout workers to start! -[2024-09-04 14:16:54,362][02747] Doom resolution: 160x120, resize resolution: (128, 72) -[2024-09-04 14:16:54,364][02749] Doom resolution: 160x120, resize resolution: (128, 72) -[2024-09-04 14:16:54,366][02751] Doom resolution: 160x120, resize resolution: (128, 72) -[2024-09-04 14:16:54,367][02754] Doom resolution: 160x120, resize resolution: (128, 72) -[2024-09-04 14:16:54,412][02750] Doom resolution: 160x120, resize resolution: (128, 72) -[2024-09-04 14:16:54,418][02752] Doom resolution: 160x120, resize resolution: (128, 72) -[2024-09-04 14:16:54,422][02748] Doom resolution: 160x120, resize resolution: (128, 72) -[2024-09-04 14:16:54,423][02753] Doom resolution: 160x120, resize resolution: (128, 72) -[2024-09-04 14:16:55,741][02754] Decorrelating experience for 0 frames... -[2024-09-04 14:16:55,739][02751] Decorrelating experience for 0 frames... -[2024-09-04 14:16:55,741][02749] Decorrelating experience for 0 frames... -[2024-09-04 14:16:55,756][02748] Decorrelating experience for 0 frames... -[2024-09-04 14:16:55,759][02752] Decorrelating experience for 0 frames... -[2024-09-04 14:16:55,764][02750] Decorrelating experience for 0 frames... -[2024-09-04 14:16:56,492][02753] Decorrelating experience for 0 frames... -[2024-09-04 14:16:56,498][02748] Decorrelating experience for 32 frames... -[2024-09-04 14:16:56,903][02747] Decorrelating experience for 0 frames... -[2024-09-04 14:16:56,968][02754] Decorrelating experience for 32 frames... -[2024-09-04 14:16:57,062][02749] Decorrelating experience for 32 frames... -[2024-09-04 14:16:57,135][02748] Decorrelating experience for 64 frames... -[2024-09-04 14:16:57,889][02751] Decorrelating experience for 32 frames... -[2024-09-04 14:16:58,039][02753] Decorrelating experience for 32 frames... -[2024-09-04 14:16:58,198][02747] Decorrelating experience for 32 frames... -[2024-09-04 14:16:58,258][02752] Decorrelating experience for 32 frames... -[2024-09-04 14:16:58,393][00226] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) -[2024-09-04 14:16:58,552][02754] Decorrelating experience for 64 frames... -[2024-09-04 14:16:58,569][02748] Decorrelating experience for 96 frames... -[2024-09-04 14:16:59,245][02750] Decorrelating experience for 32 frames... -[2024-09-04 14:16:59,522][02753] Decorrelating experience for 64 frames... -[2024-09-04 14:16:59,800][02749] Decorrelating experience for 64 frames... -[2024-09-04 14:17:00,204][02747] Decorrelating experience for 64 frames... -[2024-09-04 14:17:00,336][02751] Decorrelating experience for 64 frames... -[2024-09-04 14:17:00,429][02754] Decorrelating experience for 96 frames... -[2024-09-04 14:17:01,416][02752] Decorrelating experience for 64 frames... -[2024-09-04 14:17:01,463][02749] Decorrelating experience for 96 frames... -[2024-09-04 14:17:01,562][02753] Decorrelating experience for 96 frames... -[2024-09-04 14:17:01,834][02750] Decorrelating experience for 64 frames... -[2024-09-04 14:17:02,068][02747] Decorrelating experience for 96 frames... -[2024-09-04 14:17:03,393][00226] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 224.4. Samples: 1122. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) -[2024-09-04 14:17:03,395][00226] Avg episode reward: [(0, '2.458')] -[2024-09-04 14:17:05,586][02752] Decorrelating experience for 96 frames... -[2024-09-04 14:17:06,509][02751] Decorrelating experience for 96 frames... -[2024-09-04 14:17:06,565][02750] Decorrelating experience for 96 frames... -[2024-09-04 14:17:08,395][00226] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 207.6. Samples: 2076. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) -[2024-09-04 14:17:08,399][00226] Avg episode reward: [(0, '2.919')] -[2024-09-04 14:17:08,796][02733] Signal inference workers to stop experience collection... -[2024-09-04 14:17:08,830][02746] InferenceWorker_p0-w0: stopping experience collection -[2024-09-04 14:17:10,981][02733] Signal inference workers to resume experience collection... -[2024-09-04 14:17:10,983][02746] InferenceWorker_p0-w0: resuming experience collection -[2024-09-04 14:17:13,393][00226] Fps is (10 sec: 1638.4, 60 sec: 1092.3, 300 sec: 1092.3). Total num frames: 16384. Throughput: 0: 256.9. Samples: 3854. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) -[2024-09-04 14:17:13,398][00226] Avg episode reward: [(0, '3.279')] -[2024-09-04 14:17:18,393][00226] Fps is (10 sec: 3687.2, 60 sec: 1843.2, 300 sec: 1843.2). Total num frames: 36864. Throughput: 0: 519.2. Samples: 10384. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-09-04 14:17:18,397][00226] Avg episode reward: [(0, '3.845')] -[2024-09-04 14:17:18,430][02746] Updated weights for policy 0, policy_version 10 (0.0030) -[2024-09-04 14:17:23,395][00226] Fps is (10 sec: 3685.7, 60 sec: 2129.8, 300 sec: 2129.8). Total num frames: 53248. Throughput: 0: 507.2. Samples: 12682. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-09-04 14:17:23,398][00226] Avg episode reward: [(0, '4.176')] -[2024-09-04 14:17:28,393][00226] Fps is (10 sec: 2867.2, 60 sec: 2184.5, 300 sec: 2184.5). Total num frames: 65536. Throughput: 0: 551.1. Samples: 16534. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-09-04 14:17:28,400][00226] Avg episode reward: [(0, '4.322')] -[2024-09-04 14:17:31,092][02746] Updated weights for policy 0, policy_version 20 (0.0026) -[2024-09-04 14:17:33,393][00226] Fps is (10 sec: 3687.1, 60 sec: 2574.6, 300 sec: 2574.6). Total num frames: 90112. Throughput: 0: 663.1. Samples: 23208. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-09-04 14:17:33,395][00226] Avg episode reward: [(0, '4.348')] -[2024-09-04 14:17:38,398][00226] Fps is (10 sec: 4503.6, 60 sec: 2764.5, 300 sec: 2764.5). Total num frames: 110592. Throughput: 0: 663.5. Samples: 26544. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:17:38,400][00226] Avg episode reward: [(0, '4.474')] -[2024-09-04 14:17:38,409][02733] Saving new best policy, reward=4.474! -[2024-09-04 14:17:42,945][02746] Updated weights for policy 0, policy_version 30 (0.0023) -[2024-09-04 14:17:43,394][00226] Fps is (10 sec: 3276.5, 60 sec: 2730.6, 300 sec: 2730.6). Total num frames: 122880. Throughput: 0: 685.6. Samples: 30854. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-09-04 14:17:43,399][00226] Avg episode reward: [(0, '4.496')] -[2024-09-04 14:17:43,402][02733] Saving new best policy, reward=4.496! -[2024-09-04 14:17:48,393][00226] Fps is (10 sec: 3278.2, 60 sec: 2867.2, 300 sec: 2867.2). Total num frames: 143360. Throughput: 0: 787.8. Samples: 36572. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:17:48,396][00226] Avg episode reward: [(0, '4.282')] -[2024-09-04 14:17:52,762][02746] Updated weights for policy 0, policy_version 40 (0.0026) -[2024-09-04 14:17:53,393][00226] Fps is (10 sec: 4096.4, 60 sec: 2978.9, 300 sec: 2978.9). Total num frames: 163840. Throughput: 0: 841.6. Samples: 39944. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-09-04 14:17:53,395][00226] Avg episode reward: [(0, '4.364')] -[2024-09-04 14:17:58,393][00226] Fps is (10 sec: 3686.4, 60 sec: 3003.7, 300 sec: 3003.7). Total num frames: 180224. Throughput: 0: 916.2. Samples: 45084. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:17:58,395][00226] Avg episode reward: [(0, '4.523')] -[2024-09-04 14:17:58,403][02733] Saving new best policy, reward=4.523! -[2024-09-04 14:18:03,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3276.8, 300 sec: 3024.7). Total num frames: 196608. Throughput: 0: 880.7. Samples: 50014. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-09-04 14:18:03,399][00226] Avg episode reward: [(0, '4.566')] -[2024-09-04 14:18:03,402][02733] Saving new best policy, reward=4.566! -[2024-09-04 14:18:04,821][02746] Updated weights for policy 0, policy_version 50 (0.0033) -[2024-09-04 14:18:08,393][00226] Fps is (10 sec: 3686.4, 60 sec: 3618.3, 300 sec: 3101.3). Total num frames: 217088. Throughput: 0: 902.6. Samples: 53296. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:18:08,399][00226] Avg episode reward: [(0, '4.349')] -[2024-09-04 14:18:13,393][00226] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3167.6). Total num frames: 237568. Throughput: 0: 954.4. Samples: 59482. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:18:13,399][00226] Avg episode reward: [(0, '4.486')] -[2024-09-04 14:18:15,994][02746] Updated weights for policy 0, policy_version 60 (0.0029) -[2024-09-04 14:18:18,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3123.2). Total num frames: 249856. Throughput: 0: 897.6. Samples: 63600. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-09-04 14:18:18,400][00226] Avg episode reward: [(0, '4.473')] -[2024-09-04 14:18:23,393][00226] Fps is (10 sec: 3686.4, 60 sec: 3686.5, 300 sec: 3228.6). Total num frames: 274432. Throughput: 0: 898.1. Samples: 66956. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:18:23,399][00226] Avg episode reward: [(0, '4.642')] -[2024-09-04 14:18:23,404][02733] Saving new best policy, reward=4.642! -[2024-09-04 14:18:26,129][02746] Updated weights for policy 0, policy_version 70 (0.0039) -[2024-09-04 14:18:28,393][00226] Fps is (10 sec: 4505.5, 60 sec: 3822.9, 300 sec: 3276.8). Total num frames: 294912. Throughput: 0: 945.5. Samples: 73400. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:18:28,396][00226] Avg episode reward: [(0, '4.620')] -[2024-09-04 14:18:28,410][02733] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000072_294912.pth... -[2024-09-04 14:18:33,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3233.7). Total num frames: 307200. Throughput: 0: 913.0. Samples: 77658. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:18:33,397][00226] Avg episode reward: [(0, '4.603')] -[2024-09-04 14:18:38,212][02746] Updated weights for policy 0, policy_version 80 (0.0027) -[2024-09-04 14:18:38,393][00226] Fps is (10 sec: 3276.9, 60 sec: 3618.4, 300 sec: 3276.8). Total num frames: 327680. Throughput: 0: 895.3. Samples: 80234. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:18:38,399][00226] Avg episode reward: [(0, '4.454')] -[2024-09-04 14:18:43,393][00226] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3315.8). Total num frames: 348160. Throughput: 0: 931.2. Samples: 86990. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-09-04 14:18:43,395][00226] Avg episode reward: [(0, '4.330')] -[2024-09-04 14:18:48,393][00226] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3314.0). Total num frames: 364544. Throughput: 0: 937.8. Samples: 92216. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-09-04 14:18:48,399][00226] Avg episode reward: [(0, '4.393')] -[2024-09-04 14:18:48,776][02746] Updated weights for policy 0, policy_version 90 (0.0027) -[2024-09-04 14:18:53,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3312.4). Total num frames: 380928. Throughput: 0: 909.2. Samples: 94212. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-09-04 14:18:53,395][00226] Avg episode reward: [(0, '4.432')] -[2024-09-04 14:18:58,393][00226] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3345.1). Total num frames: 401408. Throughput: 0: 911.2. Samples: 100486. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:18:58,396][00226] Avg episode reward: [(0, '4.719')] -[2024-09-04 14:18:58,460][02733] Saving new best policy, reward=4.719! -[2024-09-04 14:18:59,433][02746] Updated weights for policy 0, policy_version 100 (0.0015) -[2024-09-04 14:19:03,394][00226] Fps is (10 sec: 4095.7, 60 sec: 3754.6, 300 sec: 3375.1). Total num frames: 421888. Throughput: 0: 956.9. Samples: 106662. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:19:03,397][00226] Avg episode reward: [(0, '4.658')] -[2024-09-04 14:19:08,393][00226] Fps is (10 sec: 3276.7, 60 sec: 3618.1, 300 sec: 3339.8). Total num frames: 434176. Throughput: 0: 926.3. Samples: 108638. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:19:08,401][00226] Avg episode reward: [(0, '4.584')] -[2024-09-04 14:19:11,365][02746] Updated weights for policy 0, policy_version 110 (0.0033) -[2024-09-04 14:19:13,393][00226] Fps is (10 sec: 3686.7, 60 sec: 3686.4, 300 sec: 3398.2). Total num frames: 458752. Throughput: 0: 905.6. Samples: 114152. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:19:13,395][00226] Avg episode reward: [(0, '4.560')] -[2024-09-04 14:19:18,393][00226] Fps is (10 sec: 4505.6, 60 sec: 3822.9, 300 sec: 3423.1). Total num frames: 479232. Throughput: 0: 955.2. Samples: 120644. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:19:18,399][00226] Avg episode reward: [(0, '4.542')] -[2024-09-04 14:19:21,953][02746] Updated weights for policy 0, policy_version 120 (0.0024) -[2024-09-04 14:19:23,393][00226] Fps is (10 sec: 3276.7, 60 sec: 3618.1, 300 sec: 3389.8). Total num frames: 491520. Throughput: 0: 949.4. Samples: 122958. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:19:23,400][00226] Avg episode reward: [(0, '4.524')] -[2024-09-04 14:19:28,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3413.3). Total num frames: 512000. Throughput: 0: 894.9. Samples: 127260. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:19:28,399][00226] Avg episode reward: [(0, '4.587')] -[2024-09-04 14:19:33,022][02746] Updated weights for policy 0, policy_version 130 (0.0028) -[2024-09-04 14:19:33,393][00226] Fps is (10 sec: 4096.1, 60 sec: 3754.7, 300 sec: 3435.4). Total num frames: 532480. Throughput: 0: 925.7. Samples: 133872. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-09-04 14:19:33,396][00226] Avg episode reward: [(0, '4.734')] -[2024-09-04 14:19:33,403][02733] Saving new best policy, reward=4.734! -[2024-09-04 14:19:38,393][00226] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3430.4). Total num frames: 548864. Throughput: 0: 954.4. Samples: 137158. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-09-04 14:19:38,400][00226] Avg episode reward: [(0, '4.700')] -[2024-09-04 14:19:43,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3425.7). Total num frames: 565248. Throughput: 0: 904.7. Samples: 141196. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-09-04 14:19:43,399][00226] Avg episode reward: [(0, '4.640')] -[2024-09-04 14:19:45,065][02746] Updated weights for policy 0, policy_version 140 (0.0037) -[2024-09-04 14:19:48,393][00226] Fps is (10 sec: 3686.3, 60 sec: 3686.4, 300 sec: 3445.5). Total num frames: 585728. Throughput: 0: 905.8. Samples: 147424. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:19:48,398][00226] Avg episode reward: [(0, '4.532')] -[2024-09-04 14:19:53,394][00226] Fps is (10 sec: 4505.2, 60 sec: 3822.9, 300 sec: 3487.4). Total num frames: 610304. Throughput: 0: 935.0. Samples: 150714. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-09-04 14:19:53,400][00226] Avg episode reward: [(0, '4.426')] -[2024-09-04 14:19:54,942][02746] Updated weights for policy 0, policy_version 150 (0.0034) -[2024-09-04 14:19:58,394][00226] Fps is (10 sec: 3686.2, 60 sec: 3686.4, 300 sec: 3458.8). Total num frames: 622592. Throughput: 0: 917.9. Samples: 155456. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-09-04 14:19:58,401][00226] Avg episode reward: [(0, '4.440')] -[2024-09-04 14:20:03,393][00226] Fps is (10 sec: 2867.5, 60 sec: 3618.2, 300 sec: 3453.9). Total num frames: 638976. Throughput: 0: 891.5. Samples: 160760. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-09-04 14:20:03,398][00226] Avg episode reward: [(0, '4.390')] -[2024-09-04 14:20:06,420][02746] Updated weights for policy 0, policy_version 160 (0.0014) -[2024-09-04 14:20:08,393][00226] Fps is (10 sec: 4096.3, 60 sec: 3822.9, 300 sec: 3492.4). Total num frames: 663552. Throughput: 0: 913.7. Samples: 164074. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-09-04 14:20:08,398][00226] Avg episode reward: [(0, '4.593')] -[2024-09-04 14:20:13,396][00226] Fps is (10 sec: 4094.7, 60 sec: 3686.2, 300 sec: 3486.8). Total num frames: 679936. Throughput: 0: 946.0. Samples: 169834. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-09-04 14:20:13,399][00226] Avg episode reward: [(0, '4.516')] -[2024-09-04 14:20:18,393][00226] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3461.1). Total num frames: 692224. Throughput: 0: 897.6. Samples: 174266. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-09-04 14:20:18,399][00226] Avg episode reward: [(0, '4.633')] -[2024-09-04 14:20:18,422][02746] Updated weights for policy 0, policy_version 170 (0.0029) -[2024-09-04 14:20:23,393][00226] Fps is (10 sec: 3687.6, 60 sec: 3754.7, 300 sec: 3496.6). Total num frames: 716800. Throughput: 0: 900.5. Samples: 177680. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-09-04 14:20:23,396][00226] Avg episode reward: [(0, '4.458')] -[2024-09-04 14:20:28,145][02746] Updated weights for policy 0, policy_version 180 (0.0034) -[2024-09-04 14:20:28,393][00226] Fps is (10 sec: 4505.5, 60 sec: 3754.7, 300 sec: 3510.9). Total num frames: 737280. Throughput: 0: 955.2. Samples: 184178. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:20:28,397][00226] Avg episode reward: [(0, '4.399')] -[2024-09-04 14:20:28,411][02733] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000180_737280.pth... -[2024-09-04 14:20:33,394][00226] Fps is (10 sec: 3276.6, 60 sec: 3618.1, 300 sec: 3486.4). Total num frames: 749568. Throughput: 0: 903.0. Samples: 188060. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:20:33,399][00226] Avg episode reward: [(0, '4.520')] -[2024-09-04 14:20:38,393][00226] Fps is (10 sec: 3276.9, 60 sec: 3686.4, 300 sec: 3500.2). Total num frames: 770048. Throughput: 0: 894.1. Samples: 190946. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:20:38,399][00226] Avg episode reward: [(0, '4.584')] -[2024-09-04 14:20:39,954][02746] Updated weights for policy 0, policy_version 190 (0.0043) -[2024-09-04 14:20:43,393][00226] Fps is (10 sec: 4096.3, 60 sec: 3754.7, 300 sec: 3513.5). Total num frames: 790528. Throughput: 0: 937.4. Samples: 197638. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:20:43,400][00226] Avg episode reward: [(0, '4.702')] -[2024-09-04 14:20:48,394][00226] Fps is (10 sec: 3686.2, 60 sec: 3686.4, 300 sec: 3508.3). Total num frames: 806912. Throughput: 0: 927.4. Samples: 202494. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:20:48,401][00226] Avg episode reward: [(0, '4.605')] -[2024-09-04 14:20:52,220][02746] Updated weights for policy 0, policy_version 200 (0.0041) -[2024-09-04 14:20:53,393][00226] Fps is (10 sec: 3276.7, 60 sec: 3549.9, 300 sec: 3503.4). Total num frames: 823296. Throughput: 0: 898.9. Samples: 204524. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:20:53,399][00226] Avg episode reward: [(0, '4.713')] -[2024-09-04 14:20:58,393][00226] Fps is (10 sec: 3686.6, 60 sec: 3686.4, 300 sec: 3515.7). Total num frames: 843776. Throughput: 0: 915.3. Samples: 211020. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:20:58,397][00226] Avg episode reward: [(0, '4.733')] -[2024-09-04 14:21:01,383][02746] Updated weights for policy 0, policy_version 210 (0.0023) -[2024-09-04 14:21:03,393][00226] Fps is (10 sec: 4096.2, 60 sec: 3754.7, 300 sec: 3527.6). Total num frames: 864256. Throughput: 0: 946.0. Samples: 216834. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:21:03,399][00226] Avg episode reward: [(0, '4.589')] -[2024-09-04 14:21:08,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3506.2). Total num frames: 876544. Throughput: 0: 913.1. Samples: 218768. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2024-09-04 14:21:08,402][00226] Avg episode reward: [(0, '4.778')] -[2024-09-04 14:21:08,412][02733] Saving new best policy, reward=4.778! -[2024-09-04 14:21:13,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3618.3, 300 sec: 3517.7). Total num frames: 897024. Throughput: 0: 894.6. Samples: 224436. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2024-09-04 14:21:13,399][00226] Avg episode reward: [(0, '4.687')] -[2024-09-04 14:21:13,419][02746] Updated weights for policy 0, policy_version 220 (0.0029) -[2024-09-04 14:21:18,394][00226] Fps is (10 sec: 4505.2, 60 sec: 3822.9, 300 sec: 3544.6). Total num frames: 921600. Throughput: 0: 955.1. Samples: 231038. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:21:18,397][00226] Avg episode reward: [(0, '4.699')] -[2024-09-04 14:21:23,393][00226] Fps is (10 sec: 3686.3, 60 sec: 3618.1, 300 sec: 3524.1). Total num frames: 933888. Throughput: 0: 936.1. Samples: 233072. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:21:23,396][00226] Avg episode reward: [(0, '4.660')] -[2024-09-04 14:21:25,753][02746] Updated weights for policy 0, policy_version 230 (0.0023) -[2024-09-04 14:21:28,403][00226] Fps is (10 sec: 2864.6, 60 sec: 3549.3, 300 sec: 3519.4). Total num frames: 950272. Throughput: 0: 888.2. Samples: 237614. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:21:28,405][00226] Avg episode reward: [(0, '4.759')] -[2024-09-04 14:21:33,393][00226] Fps is (10 sec: 4096.1, 60 sec: 3754.7, 300 sec: 3544.9). Total num frames: 974848. Throughput: 0: 927.7. Samples: 244238. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:21:33,395][00226] Avg episode reward: [(0, '4.665')] -[2024-09-04 14:21:35,030][02746] Updated weights for policy 0, policy_version 240 (0.0013) -[2024-09-04 14:21:38,394][00226] Fps is (10 sec: 4099.7, 60 sec: 3686.3, 300 sec: 3540.1). Total num frames: 991232. Throughput: 0: 950.6. Samples: 247302. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-09-04 14:21:38,397][00226] Avg episode reward: [(0, '4.554')] -[2024-09-04 14:21:43,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3535.5). Total num frames: 1007616. Throughput: 0: 895.7. Samples: 251326. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:21:43,397][00226] Avg episode reward: [(0, '4.636')] -[2024-09-04 14:21:47,188][02746] Updated weights for policy 0, policy_version 250 (0.0047) -[2024-09-04 14:21:48,393][00226] Fps is (10 sec: 3686.8, 60 sec: 3686.4, 300 sec: 3545.2). Total num frames: 1028096. Throughput: 0: 905.9. Samples: 257600. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:21:48,399][00226] Avg episode reward: [(0, '4.670')] -[2024-09-04 14:21:53,394][00226] Fps is (10 sec: 4095.6, 60 sec: 3754.6, 300 sec: 3554.5). Total num frames: 1048576. Throughput: 0: 935.2. Samples: 260852. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:21:53,399][00226] Avg episode reward: [(0, '4.666')] -[2024-09-04 14:21:58,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3596.1). Total num frames: 1060864. Throughput: 0: 910.5. Samples: 265410. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:21:58,399][00226] Avg episode reward: [(0, '4.551')] -[2024-09-04 14:21:59,285][02746] Updated weights for policy 0, policy_version 260 (0.0035) -[2024-09-04 14:22:03,393][00226] Fps is (10 sec: 3277.1, 60 sec: 3618.1, 300 sec: 3665.6). Total num frames: 1081344. Throughput: 0: 884.5. Samples: 270838. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:22:03,395][00226] Avg episode reward: [(0, '4.571')] -[2024-09-04 14:22:08,393][00226] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3679.5). Total num frames: 1101824. Throughput: 0: 911.4. Samples: 274084. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:22:08,397][00226] Avg episode reward: [(0, '4.836')] -[2024-09-04 14:22:08,405][02733] Saving new best policy, reward=4.836! -[2024-09-04 14:22:08,911][02746] Updated weights for policy 0, policy_version 270 (0.0018) -[2024-09-04 14:22:13,393][00226] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3665.6). Total num frames: 1118208. Throughput: 0: 934.1. Samples: 279640. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:22:13,397][00226] Avg episode reward: [(0, '4.918')] -[2024-09-04 14:22:13,399][02733] Saving new best policy, reward=4.918! -[2024-09-04 14:22:18,398][00226] Fps is (10 sec: 3275.1, 60 sec: 3549.6, 300 sec: 3665.5). Total num frames: 1134592. Throughput: 0: 887.9. Samples: 284200. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:22:18,401][00226] Avg episode reward: [(0, '5.019')] -[2024-09-04 14:22:18,411][02733] Saving new best policy, reward=5.019! -[2024-09-04 14:22:21,155][02746] Updated weights for policy 0, policy_version 280 (0.0027) -[2024-09-04 14:22:23,393][00226] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3693.3). Total num frames: 1155072. Throughput: 0: 891.2. Samples: 287406. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:22:23,397][00226] Avg episode reward: [(0, '4.877')] -[2024-09-04 14:22:28,394][00226] Fps is (10 sec: 4097.6, 60 sec: 3755.2, 300 sec: 3679.4). Total num frames: 1175552. Throughput: 0: 943.5. Samples: 293784. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:22:28,399][00226] Avg episode reward: [(0, '4.956')] -[2024-09-04 14:22:28,409][02733] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000287_1175552.pth... -[2024-09-04 14:22:28,573][02733] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000072_294912.pth -[2024-09-04 14:22:32,737][02746] Updated weights for policy 0, policy_version 290 (0.0037) -[2024-09-04 14:22:33,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3651.7). Total num frames: 1187840. Throughput: 0: 890.4. Samples: 297666. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2024-09-04 14:22:33,397][00226] Avg episode reward: [(0, '4.979')] -[2024-09-04 14:22:38,393][00226] Fps is (10 sec: 3277.2, 60 sec: 3618.2, 300 sec: 3679.5). Total num frames: 1208320. Throughput: 0: 882.2. Samples: 300552. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:22:38,398][00226] Avg episode reward: [(0, '5.138')] -[2024-09-04 14:22:38,407][02733] Saving new best policy, reward=5.138! -[2024-09-04 14:22:42,776][02746] Updated weights for policy 0, policy_version 300 (0.0032) -[2024-09-04 14:22:43,393][00226] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3679.5). Total num frames: 1228800. Throughput: 0: 926.8. Samples: 307116. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:22:43,402][00226] Avg episode reward: [(0, '5.061')] -[2024-09-04 14:22:48,393][00226] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3665.6). Total num frames: 1245184. Throughput: 0: 913.2. Samples: 311934. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:22:48,398][00226] Avg episode reward: [(0, '5.083')] -[2024-09-04 14:22:53,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3665.6). Total num frames: 1261568. Throughput: 0: 886.4. Samples: 313972. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-09-04 14:22:53,398][00226] Avg episode reward: [(0, '5.241')] -[2024-09-04 14:22:53,401][02733] Saving new best policy, reward=5.241! -[2024-09-04 14:22:54,962][02746] Updated weights for policy 0, policy_version 310 (0.0029) -[2024-09-04 14:22:58,394][00226] Fps is (10 sec: 3685.9, 60 sec: 3686.3, 300 sec: 3679.4). Total num frames: 1282048. Throughput: 0: 908.0. Samples: 320500. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:22:58,401][00226] Avg episode reward: [(0, '5.388')] -[2024-09-04 14:22:58,417][02733] Saving new best policy, reward=5.388! -[2024-09-04 14:23:03,396][00226] Fps is (10 sec: 4095.0, 60 sec: 3686.3, 300 sec: 3679.4). Total num frames: 1302528. Throughput: 0: 932.6. Samples: 326166. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:23:03,402][00226] Avg episode reward: [(0, '5.270')] -[2024-09-04 14:23:06,395][02746] Updated weights for policy 0, policy_version 320 (0.0038) -[2024-09-04 14:23:08,393][00226] Fps is (10 sec: 3277.2, 60 sec: 3549.9, 300 sec: 3651.7). Total num frames: 1314816. Throughput: 0: 905.3. Samples: 328146. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2024-09-04 14:23:08,395][00226] Avg episode reward: [(0, '5.284')] -[2024-09-04 14:23:13,396][00226] Fps is (10 sec: 3276.5, 60 sec: 3617.9, 300 sec: 3679.4). Total num frames: 1335296. Throughput: 0: 894.1. Samples: 334022. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2024-09-04 14:23:13,399][00226] Avg episode reward: [(0, '5.411')] -[2024-09-04 14:23:13,434][02733] Saving new best policy, reward=5.411! -[2024-09-04 14:23:16,282][02746] Updated weights for policy 0, policy_version 330 (0.0030) -[2024-09-04 14:23:18,393][00226] Fps is (10 sec: 4505.7, 60 sec: 3755.0, 300 sec: 3679.5). Total num frames: 1359872. Throughput: 0: 954.7. Samples: 340628. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:23:18,399][00226] Avg episode reward: [(0, '5.443')] -[2024-09-04 14:23:18,422][02733] Saving new best policy, reward=5.443! -[2024-09-04 14:23:23,398][00226] Fps is (10 sec: 3685.7, 60 sec: 3617.8, 300 sec: 3651.6). Total num frames: 1372160. Throughput: 0: 934.1. Samples: 342592. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) -[2024-09-04 14:23:23,401][00226] Avg episode reward: [(0, '5.543')] -[2024-09-04 14:23:23,405][02733] Saving new best policy, reward=5.543! -[2024-09-04 14:23:28,393][00226] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3665.6). Total num frames: 1388544. Throughput: 0: 892.6. Samples: 347284. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-09-04 14:23:28,403][00226] Avg episode reward: [(0, '5.597')] -[2024-09-04 14:23:28,415][02733] Saving new best policy, reward=5.597! -[2024-09-04 14:23:28,676][02746] Updated weights for policy 0, policy_version 340 (0.0037) -[2024-09-04 14:23:33,393][00226] Fps is (10 sec: 4098.2, 60 sec: 3754.7, 300 sec: 3679.5). Total num frames: 1413120. Throughput: 0: 931.6. Samples: 353858. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:23:33,399][00226] Avg episode reward: [(0, '5.944')] -[2024-09-04 14:23:33,404][02733] Saving new best policy, reward=5.944! -[2024-09-04 14:23:38,393][00226] Fps is (10 sec: 4095.9, 60 sec: 3686.4, 300 sec: 3665.6). Total num frames: 1429504. Throughput: 0: 948.8. Samples: 356668. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:23:38,398][00226] Avg episode reward: [(0, '6.219')] -[2024-09-04 14:23:38,412][02733] Saving new best policy, reward=6.219! -[2024-09-04 14:23:39,782][02746] Updated weights for policy 0, policy_version 350 (0.0024) -[2024-09-04 14:23:43,393][00226] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3651.7). Total num frames: 1441792. Throughput: 0: 893.9. Samples: 360726. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:23:43,396][00226] Avg episode reward: [(0, '5.915')] -[2024-09-04 14:23:48,393][00226] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3679.5). Total num frames: 1466368. Throughput: 0: 913.4. Samples: 367266. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2024-09-04 14:23:48,396][00226] Avg episode reward: [(0, '6.035')] -[2024-09-04 14:23:49,861][02746] Updated weights for policy 0, policy_version 360 (0.0026) -[2024-09-04 14:23:53,393][00226] Fps is (10 sec: 4505.6, 60 sec: 3754.7, 300 sec: 3679.5). Total num frames: 1486848. Throughput: 0: 943.0. Samples: 370580. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-09-04 14:23:53,396][00226] Avg episode reward: [(0, '6.477')] -[2024-09-04 14:23:53,399][02733] Saving new best policy, reward=6.477! -[2024-09-04 14:23:58,393][00226] Fps is (10 sec: 3276.9, 60 sec: 3618.2, 300 sec: 3651.7). Total num frames: 1499136. Throughput: 0: 911.3. Samples: 375026. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-09-04 14:23:58,399][00226] Avg episode reward: [(0, '6.794')] -[2024-09-04 14:23:58,407][02733] Saving new best policy, reward=6.794! -[2024-09-04 14:24:02,062][02746] Updated weights for policy 0, policy_version 370 (0.0041) -[2024-09-04 14:24:03,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3618.3, 300 sec: 3679.5). Total num frames: 1519616. Throughput: 0: 889.3. Samples: 380646. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-09-04 14:24:03,399][00226] Avg episode reward: [(0, '6.730')] -[2024-09-04 14:24:08,395][00226] Fps is (10 sec: 4095.1, 60 sec: 3754.5, 300 sec: 3665.5). Total num frames: 1540096. Throughput: 0: 919.1. Samples: 383950. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:24:08,402][00226] Avg episode reward: [(0, '6.351')] -[2024-09-04 14:24:12,548][02746] Updated weights for policy 0, policy_version 380 (0.0040) -[2024-09-04 14:24:13,395][00226] Fps is (10 sec: 3685.9, 60 sec: 3686.5, 300 sec: 3651.7). Total num frames: 1556480. Throughput: 0: 935.9. Samples: 389400. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-09-04 14:24:13,403][00226] Avg episode reward: [(0, '6.514')] -[2024-09-04 14:24:18,393][00226] Fps is (10 sec: 3277.5, 60 sec: 3549.9, 300 sec: 3665.6). Total num frames: 1572864. Throughput: 0: 895.1. Samples: 394138. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-09-04 14:24:18,400][00226] Avg episode reward: [(0, '6.197')] -[2024-09-04 14:24:23,288][02746] Updated weights for policy 0, policy_version 390 (0.0046) -[2024-09-04 14:24:23,393][00226] Fps is (10 sec: 4096.6, 60 sec: 3755.0, 300 sec: 3679.5). Total num frames: 1597440. Throughput: 0: 908.9. Samples: 397566. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-09-04 14:24:23,396][00226] Avg episode reward: [(0, '6.364')] -[2024-09-04 14:24:28,393][00226] Fps is (10 sec: 4095.9, 60 sec: 3754.7, 300 sec: 3665.6). Total num frames: 1613824. Throughput: 0: 958.6. Samples: 403864. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-09-04 14:24:28,396][00226] Avg episode reward: [(0, '6.586')] -[2024-09-04 14:24:28,410][02733] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000394_1613824.pth... -[2024-09-04 14:24:28,592][02733] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000180_737280.pth -[2024-09-04 14:24:33,393][00226] Fps is (10 sec: 2867.1, 60 sec: 3549.9, 300 sec: 3651.7). Total num frames: 1626112. Throughput: 0: 900.8. Samples: 407800. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:24:33,399][00226] Avg episode reward: [(0, '7.044')] -[2024-09-04 14:24:33,401][02733] Saving new best policy, reward=7.044! -[2024-09-04 14:24:35,698][02746] Updated weights for policy 0, policy_version 400 (0.0020) -[2024-09-04 14:24:38,393][00226] Fps is (10 sec: 3686.5, 60 sec: 3686.4, 300 sec: 3679.5). Total num frames: 1650688. Throughput: 0: 894.5. Samples: 410832. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2024-09-04 14:24:38,396][00226] Avg episode reward: [(0, '6.878')] -[2024-09-04 14:24:43,393][00226] Fps is (10 sec: 4505.7, 60 sec: 3822.9, 300 sec: 3679.5). Total num frames: 1671168. Throughput: 0: 945.0. Samples: 417550. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:24:43,400][00226] Avg episode reward: [(0, '7.007')] -[2024-09-04 14:24:45,360][02746] Updated weights for policy 0, policy_version 410 (0.0017) -[2024-09-04 14:24:48,394][00226] Fps is (10 sec: 3276.7, 60 sec: 3618.1, 300 sec: 3637.8). Total num frames: 1683456. Throughput: 0: 924.4. Samples: 422246. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:24:48,396][00226] Avg episode reward: [(0, '6.910')] -[2024-09-04 14:24:53,396][00226] Fps is (10 sec: 3276.0, 60 sec: 3618.0, 300 sec: 3665.6). Total num frames: 1703936. Throughput: 0: 899.5. Samples: 424426. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:24:53,402][00226] Avg episode reward: [(0, '6.967')] -[2024-09-04 14:24:56,676][02746] Updated weights for policy 0, policy_version 420 (0.0023) -[2024-09-04 14:24:58,393][00226] Fps is (10 sec: 4096.2, 60 sec: 3754.7, 300 sec: 3679.5). Total num frames: 1724416. Throughput: 0: 925.1. Samples: 431026. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:24:58,401][00226] Avg episode reward: [(0, '6.508')] -[2024-09-04 14:25:03,393][00226] Fps is (10 sec: 4096.9, 60 sec: 3754.7, 300 sec: 3665.6). Total num frames: 1744896. Throughput: 0: 947.2. Samples: 436764. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-09-04 14:25:03,400][00226] Avg episode reward: [(0, '6.362')] -[2024-09-04 14:25:08,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3618.3, 300 sec: 3651.7). Total num frames: 1757184. Throughput: 0: 915.1. Samples: 438744. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-09-04 14:25:08,399][00226] Avg episode reward: [(0, '6.516')] -[2024-09-04 14:25:08,627][02746] Updated weights for policy 0, policy_version 430 (0.0021) -[2024-09-04 14:25:13,398][00226] Fps is (10 sec: 3275.4, 60 sec: 3686.2, 300 sec: 3679.4). Total num frames: 1777664. Throughput: 0: 910.1. Samples: 444822. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-09-04 14:25:13,404][00226] Avg episode reward: [(0, '7.500')] -[2024-09-04 14:25:13,410][02733] Saving new best policy, reward=7.500! -[2024-09-04 14:25:18,125][02746] Updated weights for policy 0, policy_version 440 (0.0024) -[2024-09-04 14:25:18,393][00226] Fps is (10 sec: 4505.7, 60 sec: 3822.9, 300 sec: 3679.5). Total num frames: 1802240. Throughput: 0: 965.8. Samples: 451262. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2024-09-04 14:25:18,398][00226] Avg episode reward: [(0, '8.295')] -[2024-09-04 14:25:18,412][02733] Saving new best policy, reward=8.295! -[2024-09-04 14:25:23,393][00226] Fps is (10 sec: 3688.0, 60 sec: 3618.1, 300 sec: 3651.7). Total num frames: 1814528. Throughput: 0: 941.8. Samples: 453212. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:25:23,402][00226] Avg episode reward: [(0, '8.632')] -[2024-09-04 14:25:23,409][02733] Saving new best policy, reward=8.632! -[2024-09-04 14:25:28,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3686.4, 300 sec: 3679.5). Total num frames: 1835008. Throughput: 0: 905.5. Samples: 458296. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:25:28,395][00226] Avg episode reward: [(0, '8.512')] -[2024-09-04 14:25:30,134][02746] Updated weights for policy 0, policy_version 450 (0.0022) -[2024-09-04 14:25:33,393][00226] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3679.5). Total num frames: 1855488. Throughput: 0: 947.9. Samples: 464900. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:25:33,396][00226] Avg episode reward: [(0, '8.158')] -[2024-09-04 14:25:38,393][00226] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3665.6). Total num frames: 1871872. Throughput: 0: 959.1. Samples: 467584. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:25:38,403][00226] Avg episode reward: [(0, '8.527')] -[2024-09-04 14:25:41,978][02746] Updated weights for policy 0, policy_version 460 (0.0032) -[2024-09-04 14:25:43,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3665.6). Total num frames: 1888256. Throughput: 0: 906.6. Samples: 471822. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:25:43,402][00226] Avg episode reward: [(0, '8.600')] -[2024-09-04 14:25:48,393][00226] Fps is (10 sec: 3686.3, 60 sec: 3754.7, 300 sec: 3679.5). Total num frames: 1908736. Throughput: 0: 927.7. Samples: 478510. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:25:48,397][00226] Avg episode reward: [(0, '9.461')] -[2024-09-04 14:25:48,406][02733] Saving new best policy, reward=9.461! -[2024-09-04 14:25:51,300][02746] Updated weights for policy 0, policy_version 470 (0.0024) -[2024-09-04 14:25:53,393][00226] Fps is (10 sec: 4096.0, 60 sec: 3754.8, 300 sec: 3679.5). Total num frames: 1929216. Throughput: 0: 957.1. Samples: 481814. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2024-09-04 14:25:53,395][00226] Avg episode reward: [(0, '9.314')] -[2024-09-04 14:25:58,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3651.7). Total num frames: 1941504. Throughput: 0: 915.1. Samples: 485996. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:25:58,396][00226] Avg episode reward: [(0, '9.560')] -[2024-09-04 14:25:58,406][02733] Saving new best policy, reward=9.560! -[2024-09-04 14:26:03,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3679.5). Total num frames: 1961984. Throughput: 0: 901.3. Samples: 491820. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2024-09-04 14:26:03,397][00226] Avg episode reward: [(0, '9.752')] -[2024-09-04 14:26:03,416][02733] Saving new best policy, reward=9.752! -[2024-09-04 14:26:03,419][02746] Updated weights for policy 0, policy_version 480 (0.0020) -[2024-09-04 14:26:08,393][00226] Fps is (10 sec: 4505.6, 60 sec: 3822.9, 300 sec: 3693.3). Total num frames: 1986560. Throughput: 0: 930.1. Samples: 495066. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:26:08,396][00226] Avg episode reward: [(0, '10.861')] -[2024-09-04 14:26:08,404][02733] Saving new best policy, reward=10.861! -[2024-09-04 14:26:13,393][00226] Fps is (10 sec: 3686.3, 60 sec: 3686.7, 300 sec: 3651.7). Total num frames: 1998848. Throughput: 0: 932.2. Samples: 500246. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-09-04 14:26:13,399][00226] Avg episode reward: [(0, '10.602')] -[2024-09-04 14:26:15,174][02746] Updated weights for policy 0, policy_version 490 (0.0034) -[2024-09-04 14:26:18,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3679.5). Total num frames: 2019328. Throughput: 0: 899.8. Samples: 505392. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:26:18,402][00226] Avg episode reward: [(0, '11.075')] -[2024-09-04 14:26:18,417][02733] Saving new best policy, reward=11.075! -[2024-09-04 14:26:23,393][00226] Fps is (10 sec: 4096.1, 60 sec: 3754.7, 300 sec: 3693.5). Total num frames: 2039808. Throughput: 0: 911.5. Samples: 508600. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:26:23,402][00226] Avg episode reward: [(0, '10.690')] -[2024-09-04 14:26:24,755][02746] Updated weights for policy 0, policy_version 500 (0.0015) -[2024-09-04 14:26:28,393][00226] Fps is (10 sec: 3686.3, 60 sec: 3686.4, 300 sec: 3665.6). Total num frames: 2056192. Throughput: 0: 953.0. Samples: 514708. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-09-04 14:26:28,401][00226] Avg episode reward: [(0, '10.657')] -[2024-09-04 14:26:28,413][02733] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000502_2056192.pth... -[2024-09-04 14:26:28,583][02733] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000287_1175552.pth -[2024-09-04 14:26:33,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3665.6). Total num frames: 2072576. Throughput: 0: 893.7. Samples: 518728. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:26:33,395][00226] Avg episode reward: [(0, '10.648')] -[2024-09-04 14:26:37,017][02746] Updated weights for policy 0, policy_version 510 (0.0019) -[2024-09-04 14:26:38,393][00226] Fps is (10 sec: 3686.5, 60 sec: 3686.4, 300 sec: 3679.5). Total num frames: 2093056. Throughput: 0: 892.8. Samples: 521988. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-09-04 14:26:38,397][00226] Avg episode reward: [(0, '9.365')] -[2024-09-04 14:26:43,393][00226] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3679.5). Total num frames: 2113536. Throughput: 0: 949.8. Samples: 528738. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:26:43,400][00226] Avg episode reward: [(0, '9.457')] -[2024-09-04 14:26:47,996][02746] Updated weights for policy 0, policy_version 520 (0.0036) -[2024-09-04 14:26:48,393][00226] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3665.6). Total num frames: 2129920. Throughput: 0: 915.4. Samples: 533012. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-09-04 14:26:48,398][00226] Avg episode reward: [(0, '9.239')] -[2024-09-04 14:26:53,398][00226] Fps is (10 sec: 3684.5, 60 sec: 3686.1, 300 sec: 3693.3). Total num frames: 2150400. Throughput: 0: 902.4. Samples: 535678. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:26:53,405][00226] Avg episode reward: [(0, '10.288')] -[2024-09-04 14:26:58,000][02746] Updated weights for policy 0, policy_version 530 (0.0039) -[2024-09-04 14:26:58,396][00226] Fps is (10 sec: 4094.7, 60 sec: 3822.7, 300 sec: 3693.3). Total num frames: 2170880. Throughput: 0: 936.5. Samples: 542390. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:26:58,399][00226] Avg episode reward: [(0, '11.123')] -[2024-09-04 14:26:58,408][02733] Saving new best policy, reward=11.123! -[2024-09-04 14:27:03,393][00226] Fps is (10 sec: 3688.3, 60 sec: 3754.7, 300 sec: 3679.5). Total num frames: 2187264. Throughput: 0: 932.8. Samples: 547370. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-09-04 14:27:03,400][00226] Avg episode reward: [(0, '11.246')] -[2024-09-04 14:27:03,405][02733] Saving new best policy, reward=11.246! -[2024-09-04 14:27:08,393][00226] Fps is (10 sec: 3277.8, 60 sec: 3618.1, 300 sec: 3679.5). Total num frames: 2203648. Throughput: 0: 905.5. Samples: 549346. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-09-04 14:27:08,397][00226] Avg episode reward: [(0, '12.101')] -[2024-09-04 14:27:08,408][02733] Saving new best policy, reward=12.101! -[2024-09-04 14:27:10,197][02746] Updated weights for policy 0, policy_version 540 (0.0027) -[2024-09-04 14:27:13,393][00226] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3693.4). Total num frames: 2224128. Throughput: 0: 910.9. Samples: 555698. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:27:13,396][00226] Avg episode reward: [(0, '11.934')] -[2024-09-04 14:27:18,396][00226] Fps is (10 sec: 4094.7, 60 sec: 3754.5, 300 sec: 3693.3). Total num frames: 2244608. Throughput: 0: 956.7. Samples: 561784. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:27:18,401][00226] Avg episode reward: [(0, '12.652')] -[2024-09-04 14:27:18,417][02733] Saving new best policy, reward=12.652! -[2024-09-04 14:27:21,297][02746] Updated weights for policy 0, policy_version 550 (0.0018) -[2024-09-04 14:27:23,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3665.6). Total num frames: 2256896. Throughput: 0: 928.4. Samples: 563768. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-09-04 14:27:23,401][00226] Avg episode reward: [(0, '13.175')] -[2024-09-04 14:27:23,403][02733] Saving new best policy, reward=13.175! -[2024-09-04 14:27:28,393][00226] Fps is (10 sec: 3277.9, 60 sec: 3686.4, 300 sec: 3693.3). Total num frames: 2277376. Throughput: 0: 900.6. Samples: 569266. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:27:28,396][00226] Avg episode reward: [(0, '13.036')] -[2024-09-04 14:27:31,565][02746] Updated weights for policy 0, policy_version 560 (0.0046) -[2024-09-04 14:27:33,393][00226] Fps is (10 sec: 4505.6, 60 sec: 3822.9, 300 sec: 3707.2). Total num frames: 2301952. Throughput: 0: 952.8. Samples: 575888. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:27:33,399][00226] Avg episode reward: [(0, '13.351')] -[2024-09-04 14:27:33,404][02733] Saving new best policy, reward=13.351! -[2024-09-04 14:27:38,393][00226] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3679.5). Total num frames: 2314240. Throughput: 0: 940.6. Samples: 577998. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:27:38,398][00226] Avg episode reward: [(0, '13.268')] -[2024-09-04 14:27:43,393][00226] Fps is (10 sec: 2867.2, 60 sec: 3618.1, 300 sec: 3679.5). Total num frames: 2330624. Throughput: 0: 895.2. Samples: 582670. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:27:43,399][00226] Avg episode reward: [(0, '14.210')] -[2024-09-04 14:27:43,402][02733] Saving new best policy, reward=14.210! -[2024-09-04 14:27:43,739][02746] Updated weights for policy 0, policy_version 570 (0.0028) -[2024-09-04 14:27:48,393][00226] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3707.2). Total num frames: 2355200. Throughput: 0: 931.8. Samples: 589300. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:27:48,398][00226] Avg episode reward: [(0, '14.100')] -[2024-09-04 14:27:53,394][00226] Fps is (10 sec: 4095.5, 60 sec: 3686.6, 300 sec: 3693.3). Total num frames: 2371584. Throughput: 0: 956.0. Samples: 592366. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:27:53,400][00226] Avg episode reward: [(0, '14.125')] -[2024-09-04 14:27:54,499][02746] Updated weights for policy 0, policy_version 580 (0.0030) -[2024-09-04 14:27:58,393][00226] Fps is (10 sec: 2867.1, 60 sec: 3550.0, 300 sec: 3665.6). Total num frames: 2383872. Throughput: 0: 903.1. Samples: 596338. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:27:58,400][00226] Avg episode reward: [(0, '14.876')] -[2024-09-04 14:27:58,408][02733] Saving new best policy, reward=14.876! -[2024-09-04 14:28:03,393][00226] Fps is (10 sec: 3686.9, 60 sec: 3686.4, 300 sec: 3707.2). Total num frames: 2408448. Throughput: 0: 907.4. Samples: 602614. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:28:03,399][00226] Avg episode reward: [(0, '14.185')] -[2024-09-04 14:28:05,096][02746] Updated weights for policy 0, policy_version 590 (0.0021) -[2024-09-04 14:28:08,393][00226] Fps is (10 sec: 4505.8, 60 sec: 3754.7, 300 sec: 3707.3). Total num frames: 2428928. Throughput: 0: 935.5. Samples: 605864. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:28:08,398][00226] Avg episode reward: [(0, '16.260')] -[2024-09-04 14:28:08,408][02733] Saving new best policy, reward=16.260! -[2024-09-04 14:28:13,394][00226] Fps is (10 sec: 3276.4, 60 sec: 3618.1, 300 sec: 3665.6). Total num frames: 2441216. Throughput: 0: 915.7. Samples: 610474. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-09-04 14:28:13,403][00226] Avg episode reward: [(0, '16.077')] -[2024-09-04 14:28:17,201][02746] Updated weights for policy 0, policy_version 600 (0.0041) -[2024-09-04 14:28:18,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3618.3, 300 sec: 3693.4). Total num frames: 2461696. Throughput: 0: 894.8. Samples: 616156. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:28:18,400][00226] Avg episode reward: [(0, '17.216')] -[2024-09-04 14:28:18,410][02733] Saving new best policy, reward=17.216! -[2024-09-04 14:28:23,393][00226] Fps is (10 sec: 4096.5, 60 sec: 3754.7, 300 sec: 3707.2). Total num frames: 2482176. Throughput: 0: 921.1. Samples: 619448. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:28:23,399][00226] Avg episode reward: [(0, '17.285')] -[2024-09-04 14:28:23,402][02733] Saving new best policy, reward=17.285! -[2024-09-04 14:28:27,596][02746] Updated weights for policy 0, policy_version 610 (0.0028) -[2024-09-04 14:28:28,394][00226] Fps is (10 sec: 3686.1, 60 sec: 3686.3, 300 sec: 3679.4). Total num frames: 2498560. Throughput: 0: 937.3. Samples: 624850. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-09-04 14:28:28,399][00226] Avg episode reward: [(0, '16.903')] -[2024-09-04 14:28:28,424][02733] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000610_2498560.pth... -[2024-09-04 14:28:28,602][02733] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000394_1613824.pth -[2024-09-04 14:28:33,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3679.5). Total num frames: 2514944. Throughput: 0: 890.4. Samples: 629370. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-09-04 14:28:33,399][00226] Avg episode reward: [(0, '16.618')] -[2024-09-04 14:28:38,393][00226] Fps is (10 sec: 3686.7, 60 sec: 3686.4, 300 sec: 3707.2). Total num frames: 2535424. Throughput: 0: 896.4. Samples: 632702. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-09-04 14:28:38,396][00226] Avg episode reward: [(0, '15.906')] -[2024-09-04 14:28:38,584][02746] Updated weights for policy 0, policy_version 620 (0.0049) -[2024-09-04 14:28:43,393][00226] Fps is (10 sec: 4095.9, 60 sec: 3754.6, 300 sec: 3693.3). Total num frames: 2555904. Throughput: 0: 955.0. Samples: 639314. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:28:43,396][00226] Avg episode reward: [(0, '16.702')] -[2024-09-04 14:28:48,395][00226] Fps is (10 sec: 3276.3, 60 sec: 3549.8, 300 sec: 3665.6). Total num frames: 2568192. Throughput: 0: 902.6. Samples: 643232. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:28:48,400][00226] Avg episode reward: [(0, '16.590')] -[2024-09-04 14:28:50,625][02746] Updated weights for policy 0, policy_version 630 (0.0031) -[2024-09-04 14:28:53,393][00226] Fps is (10 sec: 3686.5, 60 sec: 3686.5, 300 sec: 3707.2). Total num frames: 2592768. Throughput: 0: 899.4. Samples: 646338. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:28:53,397][00226] Avg episode reward: [(0, '17.655')] -[2024-09-04 14:28:53,402][02733] Saving new best policy, reward=17.655! -[2024-09-04 14:28:58,393][00226] Fps is (10 sec: 4506.3, 60 sec: 3823.0, 300 sec: 3707.2). Total num frames: 2613248. Throughput: 0: 943.6. Samples: 652936. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:28:58,401][00226] Avg episode reward: [(0, '17.554')] -[2024-09-04 14:29:00,624][02746] Updated weights for policy 0, policy_version 640 (0.0026) -[2024-09-04 14:29:03,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3679.5). Total num frames: 2625536. Throughput: 0: 919.6. Samples: 657538. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-09-04 14:29:03,399][00226] Avg episode reward: [(0, '17.459')] -[2024-09-04 14:29:08,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3693.4). Total num frames: 2646016. Throughput: 0: 894.9. Samples: 659718. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-09-04 14:29:08,397][00226] Avg episode reward: [(0, '17.369')] -[2024-09-04 14:29:11,941][02746] Updated weights for policy 0, policy_version 650 (0.0033) -[2024-09-04 14:29:13,393][00226] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3707.2). Total num frames: 2666496. Throughput: 0: 924.6. Samples: 666456. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-09-04 14:29:13,396][00226] Avg episode reward: [(0, '16.993')] -[2024-09-04 14:29:18,393][00226] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3679.5). Total num frames: 2682880. Throughput: 0: 948.2. Samples: 672040. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:29:18,396][00226] Avg episode reward: [(0, '16.714')] -[2024-09-04 14:29:23,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3679.5). Total num frames: 2699264. Throughput: 0: 917.9. Samples: 674006. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-09-04 14:29:23,396][00226] Avg episode reward: [(0, '18.105')] -[2024-09-04 14:29:23,401][02733] Saving new best policy, reward=18.105! -[2024-09-04 14:29:23,987][02746] Updated weights for policy 0, policy_version 660 (0.0018) -[2024-09-04 14:29:28,393][00226] Fps is (10 sec: 3686.4, 60 sec: 3686.5, 300 sec: 3707.2). Total num frames: 2719744. Throughput: 0: 906.5. Samples: 680108. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:29:28,395][00226] Avg episode reward: [(0, '21.723')] -[2024-09-04 14:29:28,404][02733] Saving new best policy, reward=21.723! -[2024-09-04 14:29:33,393][00226] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3693.3). Total num frames: 2740224. Throughput: 0: 956.3. Samples: 686262. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:29:33,400][00226] Avg episode reward: [(0, '21.988')] -[2024-09-04 14:29:33,402][02733] Saving new best policy, reward=21.988! -[2024-09-04 14:29:33,779][02746] Updated weights for policy 0, policy_version 670 (0.0018) -[2024-09-04 14:29:38,396][00226] Fps is (10 sec: 3275.8, 60 sec: 3617.9, 300 sec: 3665.5). Total num frames: 2752512. Throughput: 0: 928.2. Samples: 688108. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:29:38,399][00226] Avg episode reward: [(0, '21.729')] -[2024-09-04 14:29:43,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3693.3). Total num frames: 2772992. Throughput: 0: 898.1. Samples: 693352. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-09-04 14:29:43,399][00226] Avg episode reward: [(0, '21.463')] -[2024-09-04 14:29:45,469][02746] Updated weights for policy 0, policy_version 680 (0.0039) -[2024-09-04 14:29:48,393][00226] Fps is (10 sec: 4507.0, 60 sec: 3823.0, 300 sec: 3707.3). Total num frames: 2797568. Throughput: 0: 941.4. Samples: 699902. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:29:48,396][00226] Avg episode reward: [(0, '20.284')] -[2024-09-04 14:29:53,394][00226] Fps is (10 sec: 3686.2, 60 sec: 3618.1, 300 sec: 3679.5). Total num frames: 2809856. Throughput: 0: 949.9. Samples: 702462. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:29:53,399][00226] Avg episode reward: [(0, '18.593')] -[2024-09-04 14:29:57,435][02746] Updated weights for policy 0, policy_version 690 (0.0022) -[2024-09-04 14:29:58,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3679.5). Total num frames: 2830336. Throughput: 0: 896.3. Samples: 706788. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:29:58,396][00226] Avg episode reward: [(0, '19.173')] -[2024-09-04 14:30:03,393][00226] Fps is (10 sec: 4096.2, 60 sec: 3754.7, 300 sec: 3707.2). Total num frames: 2850816. Throughput: 0: 917.6. Samples: 713330. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-09-04 14:30:03,396][00226] Avg episode reward: [(0, '20.767')] -[2024-09-04 14:30:06,945][02746] Updated weights for policy 0, policy_version 700 (0.0024) -[2024-09-04 14:30:08,393][00226] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3693.4). Total num frames: 2867200. Throughput: 0: 946.8. Samples: 716614. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-09-04 14:30:08,397][00226] Avg episode reward: [(0, '21.662')] -[2024-09-04 14:30:13,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3665.6). Total num frames: 2883584. Throughput: 0: 902.6. Samples: 720724. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-09-04 14:30:13,398][00226] Avg episode reward: [(0, '23.310')] -[2024-09-04 14:30:13,400][02733] Saving new best policy, reward=23.310! -[2024-09-04 14:30:18,393][00226] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3693.3). Total num frames: 2904064. Throughput: 0: 900.0. Samples: 726760. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-09-04 14:30:18,395][00226] Avg episode reward: [(0, '23.273')] -[2024-09-04 14:30:18,856][02746] Updated weights for policy 0, policy_version 710 (0.0029) -[2024-09-04 14:30:23,395][00226] Fps is (10 sec: 4095.1, 60 sec: 3754.5, 300 sec: 3693.3). Total num frames: 2924544. Throughput: 0: 933.8. Samples: 730130. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-09-04 14:30:23,399][00226] Avg episode reward: [(0, '25.370')] -[2024-09-04 14:30:23,404][02733] Saving new best policy, reward=25.370! -[2024-09-04 14:30:28,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3665.6). Total num frames: 2936832. Throughput: 0: 926.4. Samples: 735040. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2024-09-04 14:30:28,396][00226] Avg episode reward: [(0, '22.862')] -[2024-09-04 14:30:28,408][02733] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000717_2936832.pth... -[2024-09-04 14:30:28,598][02733] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000502_2056192.pth -[2024-09-04 14:30:32,404][02746] Updated weights for policy 0, policy_version 720 (0.0030) -[2024-09-04 14:30:33,393][00226] Fps is (10 sec: 2458.2, 60 sec: 3481.6, 300 sec: 3651.7). Total num frames: 2949120. Throughput: 0: 854.7. Samples: 738364. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2024-09-04 14:30:33,396][00226] Avg episode reward: [(0, '22.564')] -[2024-09-04 14:30:38,393][00226] Fps is (10 sec: 3276.9, 60 sec: 3618.3, 300 sec: 3665.6). Total num frames: 2969600. Throughput: 0: 848.0. Samples: 740622. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:30:38,395][00226] Avg episode reward: [(0, '19.962')] -[2024-09-04 14:30:43,393][00226] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3651.7). Total num frames: 2985984. Throughput: 0: 882.8. Samples: 746512. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:30:43,398][00226] Avg episode reward: [(0, '19.687')] -[2024-09-04 14:30:44,520][02746] Updated weights for policy 0, policy_version 730 (0.0024) -[2024-09-04 14:30:48,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3637.8). Total num frames: 3002368. Throughput: 0: 837.8. Samples: 751032. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:30:48,399][00226] Avg episode reward: [(0, '17.931')] -[2024-09-04 14:30:53,393][00226] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3665.6). Total num frames: 3022848. Throughput: 0: 838.3. Samples: 754336. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-09-04 14:30:53,395][00226] Avg episode reward: [(0, '17.739')] -[2024-09-04 14:30:54,820][02746] Updated weights for policy 0, policy_version 740 (0.0033) -[2024-09-04 14:30:58,393][00226] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3665.6). Total num frames: 3043328. Throughput: 0: 894.8. Samples: 760992. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:30:58,398][00226] Avg episode reward: [(0, '18.879')] -[2024-09-04 14:31:03,395][00226] Fps is (10 sec: 3276.0, 60 sec: 3413.2, 300 sec: 3623.9). Total num frames: 3055616. Throughput: 0: 848.6. Samples: 764948. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-09-04 14:31:03,398][00226] Avg episode reward: [(0, '20.423')] -[2024-09-04 14:31:06,889][02746] Updated weights for policy 0, policy_version 750 (0.0042) -[2024-09-04 14:31:08,393][00226] Fps is (10 sec: 3276.7, 60 sec: 3481.6, 300 sec: 3651.7). Total num frames: 3076096. Throughput: 0: 839.1. Samples: 767886. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:31:08,396][00226] Avg episode reward: [(0, '20.313')] -[2024-09-04 14:31:13,393][00226] Fps is (10 sec: 4506.5, 60 sec: 3618.1, 300 sec: 3665.6). Total num frames: 3100672. Throughput: 0: 879.7. Samples: 774626. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-09-04 14:31:13,396][00226] Avg episode reward: [(0, '22.449')] -[2024-09-04 14:31:17,408][02746] Updated weights for policy 0, policy_version 760 (0.0025) -[2024-09-04 14:31:18,393][00226] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3637.8). Total num frames: 3112960. Throughput: 0: 912.8. Samples: 779438. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:31:18,398][00226] Avg episode reward: [(0, '21.581')] -[2024-09-04 14:31:23,393][00226] Fps is (10 sec: 3276.9, 60 sec: 3481.7, 300 sec: 3651.7). Total num frames: 3133440. Throughput: 0: 909.3. Samples: 781540. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-09-04 14:31:23,399][00226] Avg episode reward: [(0, '21.632')] -[2024-09-04 14:31:27,978][02746] Updated weights for policy 0, policy_version 770 (0.0029) -[2024-09-04 14:31:28,393][00226] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 3665.6). Total num frames: 3153920. Throughput: 0: 926.9. Samples: 788222. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-09-04 14:31:28,401][00226] Avg episode reward: [(0, '22.403')] -[2024-09-04 14:31:33,394][00226] Fps is (10 sec: 4095.7, 60 sec: 3754.6, 300 sec: 3665.6). Total num frames: 3174400. Throughput: 0: 955.9. Samples: 794050. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:31:33,402][00226] Avg episode reward: [(0, '22.642')] -[2024-09-04 14:31:38,397][00226] Fps is (10 sec: 3275.4, 60 sec: 3617.9, 300 sec: 3637.8). Total num frames: 3186688. Throughput: 0: 925.0. Samples: 795964. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:31:38,400][00226] Avg episode reward: [(0, '22.464')] -[2024-09-04 14:31:40,005][02746] Updated weights for policy 0, policy_version 780 (0.0019) -[2024-09-04 14:31:43,393][00226] Fps is (10 sec: 3277.0, 60 sec: 3686.4, 300 sec: 3651.7). Total num frames: 3207168. Throughput: 0: 896.1. Samples: 801316. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:31:43,395][00226] Avg episode reward: [(0, '22.513')] -[2024-09-04 14:31:48,393][00226] Fps is (10 sec: 4097.7, 60 sec: 3754.7, 300 sec: 3651.8). Total num frames: 3227648. Throughput: 0: 956.0. Samples: 807966. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-09-04 14:31:48,399][00226] Avg episode reward: [(0, '24.762')] -[2024-09-04 14:31:50,346][02746] Updated weights for policy 0, policy_version 790 (0.0047) -[2024-09-04 14:31:53,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3624.0). Total num frames: 3239936. Throughput: 0: 935.6. Samples: 809990. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:31:53,396][00226] Avg episode reward: [(0, '23.826')] -[2024-09-04 14:31:58,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3637.8). Total num frames: 3260416. Throughput: 0: 894.7. Samples: 814888. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:31:58,401][00226] Avg episode reward: [(0, '23.933')] -[2024-09-04 14:32:01,525][02746] Updated weights for policy 0, policy_version 800 (0.0028) -[2024-09-04 14:32:03,394][00226] Fps is (10 sec: 4095.7, 60 sec: 3754.8, 300 sec: 3651.7). Total num frames: 3280896. Throughput: 0: 934.9. Samples: 821510. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:32:03,396][00226] Avg episode reward: [(0, '23.824')] -[2024-09-04 14:32:08,394][00226] Fps is (10 sec: 3686.2, 60 sec: 3686.4, 300 sec: 3637.8). Total num frames: 3297280. Throughput: 0: 950.8. Samples: 824328. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:32:08,399][00226] Avg episode reward: [(0, '23.919')] -[2024-09-04 14:32:13,393][00226] Fps is (10 sec: 3277.0, 60 sec: 3549.9, 300 sec: 3624.0). Total num frames: 3313664. Throughput: 0: 893.2. Samples: 828416. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:32:13,400][00226] Avg episode reward: [(0, '22.521')] -[2024-09-04 14:32:13,541][02746] Updated weights for policy 0, policy_version 810 (0.0038) -[2024-09-04 14:32:18,393][00226] Fps is (10 sec: 4096.3, 60 sec: 3754.7, 300 sec: 3665.6). Total num frames: 3338240. Throughput: 0: 913.5. Samples: 835158. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-09-04 14:32:18,395][00226] Avg episode reward: [(0, '22.825')] -[2024-09-04 14:32:23,103][02746] Updated weights for policy 0, policy_version 820 (0.0025) -[2024-09-04 14:32:23,396][00226] Fps is (10 sec: 4504.1, 60 sec: 3754.5, 300 sec: 3665.5). Total num frames: 3358720. Throughput: 0: 945.5. Samples: 838510. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:32:23,399][00226] Avg episode reward: [(0, '21.877')] -[2024-09-04 14:32:28,397][00226] Fps is (10 sec: 3275.4, 60 sec: 3617.9, 300 sec: 3623.9). Total num frames: 3371008. Throughput: 0: 922.0. Samples: 842812. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-09-04 14:32:28,403][00226] Avg episode reward: [(0, '22.175')] -[2024-09-04 14:32:28,418][02733] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000823_3371008.pth... -[2024-09-04 14:32:28,602][02733] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000610_2498560.pth -[2024-09-04 14:32:33,393][00226] Fps is (10 sec: 3277.9, 60 sec: 3618.2, 300 sec: 3651.7). Total num frames: 3391488. Throughput: 0: 902.4. Samples: 848576. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:32:33,398][00226] Avg episode reward: [(0, '23.581')] -[2024-09-04 14:32:34,936][02746] Updated weights for policy 0, policy_version 830 (0.0023) -[2024-09-04 14:32:38,393][00226] Fps is (10 sec: 4097.7, 60 sec: 3754.9, 300 sec: 3665.6). Total num frames: 3411968. Throughput: 0: 932.1. Samples: 851934. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:32:38,397][00226] Avg episode reward: [(0, '22.130')] -[2024-09-04 14:32:43,393][00226] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3637.8). Total num frames: 3428352. Throughput: 0: 938.1. Samples: 857104. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:32:43,396][00226] Avg episode reward: [(0, '22.662')] -[2024-09-04 14:32:46,893][02746] Updated weights for policy 0, policy_version 840 (0.0020) -[2024-09-04 14:32:48,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3637.8). Total num frames: 3444736. Throughput: 0: 903.6. Samples: 862170. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:32:48,396][00226] Avg episode reward: [(0, '22.621')] -[2024-09-04 14:32:53,393][00226] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3679.5). Total num frames: 3469312. Throughput: 0: 914.9. Samples: 865496. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:32:53,395][00226] Avg episode reward: [(0, '23.381')] -[2024-09-04 14:32:56,128][02746] Updated weights for policy 0, policy_version 850 (0.0036) -[2024-09-04 14:32:58,393][00226] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3651.7). Total num frames: 3485696. Throughput: 0: 962.0. Samples: 871706. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:32:58,399][00226] Avg episode reward: [(0, '21.814')] -[2024-09-04 14:33:03,393][00226] Fps is (10 sec: 2867.2, 60 sec: 3618.2, 300 sec: 3623.9). Total num frames: 3497984. Throughput: 0: 902.0. Samples: 875750. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:33:03,396][00226] Avg episode reward: [(0, '21.887')] -[2024-09-04 14:33:08,153][02746] Updated weights for policy 0, policy_version 860 (0.0026) -[2024-09-04 14:33:08,393][00226] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3665.6). Total num frames: 3522560. Throughput: 0: 901.5. Samples: 879074. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:33:08,401][00226] Avg episode reward: [(0, '22.422')] -[2024-09-04 14:33:13,393][00226] Fps is (10 sec: 4505.6, 60 sec: 3822.9, 300 sec: 3665.6). Total num frames: 3543040. Throughput: 0: 955.1. Samples: 885788. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:33:13,402][00226] Avg episode reward: [(0, '22.841')] -[2024-09-04 14:33:18,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3637.8). Total num frames: 3555328. Throughput: 0: 924.8. Samples: 890190. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:33:18,399][00226] Avg episode reward: [(0, '23.048')] -[2024-09-04 14:33:20,079][02746] Updated weights for policy 0, policy_version 870 (0.0036) -[2024-09-04 14:33:23,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3618.3, 300 sec: 3651.7). Total num frames: 3575808. Throughput: 0: 905.9. Samples: 892698. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:33:23,396][00226] Avg episode reward: [(0, '23.276')] -[2024-09-04 14:33:28,393][00226] Fps is (10 sec: 4505.6, 60 sec: 3823.2, 300 sec: 3679.5). Total num frames: 3600384. Throughput: 0: 943.0. Samples: 899540. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-09-04 14:33:28,395][00226] Avg episode reward: [(0, '24.094')] -[2024-09-04 14:33:29,129][02746] Updated weights for policy 0, policy_version 880 (0.0025) -[2024-09-04 14:33:33,398][00226] Fps is (10 sec: 4094.2, 60 sec: 3754.4, 300 sec: 3665.5). Total num frames: 3616768. Throughput: 0: 947.0. Samples: 904790. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-09-04 14:33:33,400][00226] Avg episode reward: [(0, '23.939')] -[2024-09-04 14:33:38,393][00226] Fps is (10 sec: 2867.2, 60 sec: 3618.1, 300 sec: 3637.8). Total num frames: 3629056. Throughput: 0: 918.4. Samples: 906822. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-09-04 14:33:38,395][00226] Avg episode reward: [(0, '22.874')] -[2024-09-04 14:33:41,163][02746] Updated weights for policy 0, policy_version 890 (0.0025) -[2024-09-04 14:33:43,393][00226] Fps is (10 sec: 3688.0, 60 sec: 3754.7, 300 sec: 3679.5). Total num frames: 3653632. Throughput: 0: 919.3. Samples: 913074. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:33:43,396][00226] Avg episode reward: [(0, '23.447')] -[2024-09-04 14:33:48,393][00226] Fps is (10 sec: 4505.6, 60 sec: 3822.9, 300 sec: 3665.6). Total num frames: 3674112. Throughput: 0: 969.9. Samples: 919396. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:33:48,399][00226] Avg episode reward: [(0, '23.314')] -[2024-09-04 14:33:52,278][02746] Updated weights for policy 0, policy_version 900 (0.0018) -[2024-09-04 14:33:53,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3637.8). Total num frames: 3686400. Throughput: 0: 939.6. Samples: 921358. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:33:53,401][00226] Avg episode reward: [(0, '22.438')] -[2024-09-04 14:33:58,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3686.4, 300 sec: 3665.6). Total num frames: 3706880. Throughput: 0: 912.6. Samples: 926856. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:33:58,400][00226] Avg episode reward: [(0, '22.197')] -[2024-09-04 14:34:02,113][02746] Updated weights for policy 0, policy_version 910 (0.0022) -[2024-09-04 14:34:03,393][00226] Fps is (10 sec: 4505.6, 60 sec: 3891.2, 300 sec: 3679.5). Total num frames: 3731456. Throughput: 0: 962.4. Samples: 933496. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:34:03,401][00226] Avg episode reward: [(0, '23.298')] -[2024-09-04 14:34:08,393][00226] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3651.7). Total num frames: 3743744. Throughput: 0: 958.8. Samples: 935842. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:34:08,399][00226] Avg episode reward: [(0, '23.250')] -[2024-09-04 14:34:13,394][00226] Fps is (10 sec: 3276.7, 60 sec: 3686.4, 300 sec: 3665.6). Total num frames: 3764224. Throughput: 0: 908.8. Samples: 940436. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:34:13,400][00226] Avg episode reward: [(0, '22.878')] -[2024-09-04 14:34:14,291][02746] Updated weights for policy 0, policy_version 920 (0.0034) -[2024-09-04 14:34:18,393][00226] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3679.5). Total num frames: 3784704. Throughput: 0: 940.8. Samples: 947120. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:34:18,396][00226] Avg episode reward: [(0, '23.141')] -[2024-09-04 14:34:23,393][00226] Fps is (10 sec: 3686.5, 60 sec: 3754.7, 300 sec: 3665.6). Total num frames: 3801088. Throughput: 0: 967.9. Samples: 950376. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2024-09-04 14:34:23,396][00226] Avg episode reward: [(0, '23.948')] -[2024-09-04 14:34:25,026][02746] Updated weights for policy 0, policy_version 930 (0.0027) -[2024-09-04 14:34:28,393][00226] Fps is (10 sec: 3276.7, 60 sec: 3618.1, 300 sec: 3651.7). Total num frames: 3817472. Throughput: 0: 918.0. Samples: 954382. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-09-04 14:34:28,399][00226] Avg episode reward: [(0, '24.981')] -[2024-09-04 14:34:28,410][02733] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000932_3817472.pth... -[2024-09-04 14:34:28,532][02733] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000717_2936832.pth -[2024-09-04 14:34:33,393][00226] Fps is (10 sec: 3686.4, 60 sec: 3686.7, 300 sec: 3679.5). Total num frames: 3837952. Throughput: 0: 918.4. Samples: 960726. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:34:33,396][00226] Avg episode reward: [(0, '24.091')] -[2024-09-04 14:34:35,514][02746] Updated weights for policy 0, policy_version 940 (0.0032) -[2024-09-04 14:34:38,395][00226] Fps is (10 sec: 4095.2, 60 sec: 3822.8, 300 sec: 3679.4). Total num frames: 3858432. Throughput: 0: 946.2. Samples: 963938. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:34:38,399][00226] Avg episode reward: [(0, '24.568')] -[2024-09-04 14:34:43,393][00226] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3651.7). Total num frames: 3874816. Throughput: 0: 928.2. Samples: 968624. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-09-04 14:34:43,400][00226] Avg episode reward: [(0, '24.829')] -[2024-09-04 14:34:47,447][02746] Updated weights for policy 0, policy_version 950 (0.0029) -[2024-09-04 14:34:48,393][00226] Fps is (10 sec: 3687.2, 60 sec: 3686.4, 300 sec: 3679.5). Total num frames: 3895296. Throughput: 0: 905.5. Samples: 974244. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:34:48,401][00226] Avg episode reward: [(0, '26.581')] -[2024-09-04 14:34:48,412][02733] Saving new best policy, reward=26.581! -[2024-09-04 14:34:53,393][00226] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3679.5). Total num frames: 3915776. Throughput: 0: 924.6. Samples: 977450. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:34:53,402][00226] Avg episode reward: [(0, '24.849')] -[2024-09-04 14:34:58,074][02746] Updated weights for policy 0, policy_version 960 (0.0039) -[2024-09-04 14:34:58,393][00226] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3665.6). Total num frames: 3932160. Throughput: 0: 947.3. Samples: 983062. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-09-04 14:34:58,400][00226] Avg episode reward: [(0, '24.778')] -[2024-09-04 14:35:03,393][00226] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3665.6). Total num frames: 3948544. Throughput: 0: 902.6. Samples: 987738. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-09-04 14:35:03,401][00226] Avg episode reward: [(0, '25.443')] -[2024-09-04 14:35:08,393][00226] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3679.5). Total num frames: 3969024. Throughput: 0: 901.8. Samples: 990958. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-09-04 14:35:08,396][00226] Avg episode reward: [(0, '24.398')] -[2024-09-04 14:35:08,719][02746] Updated weights for policy 0, policy_version 970 (0.0037) -[2024-09-04 14:35:13,394][00226] Fps is (10 sec: 4095.5, 60 sec: 3754.6, 300 sec: 3679.4). Total num frames: 3989504. Throughput: 0: 958.7. Samples: 997524. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-09-04 14:35:13,397][00226] Avg episode reward: [(0, '24.312')] -[2024-09-04 14:35:18,396][00226] Fps is (10 sec: 3276.0, 60 sec: 3618.0, 300 sec: 3651.7). Total num frames: 4001792. Throughput: 0: 905.8. Samples: 1001490. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-09-04 14:35:18,397][00226] Avg episode reward: [(0, '23.501')] -[2024-09-04 14:35:18,784][02733] Stopping Batcher_0... -[2024-09-04 14:35:18,785][02733] Loop batcher_evt_loop terminating... -[2024-09-04 14:35:18,787][02733] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... -[2024-09-04 14:35:18,785][00226] Component Batcher_0 stopped! -[2024-09-04 14:35:18,848][02746] Weights refcount: 2 0 -[2024-09-04 14:35:18,853][00226] Component InferenceWorker_p0-w0 stopped! -[2024-09-04 14:35:18,858][02746] Stopping InferenceWorker_p0-w0... -[2024-09-04 14:35:18,858][02746] Loop inference_proc0-0_evt_loop terminating... -[2024-09-04 14:35:18,928][02733] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000823_3371008.pth -[2024-09-04 14:35:18,946][02733] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... -[2024-09-04 14:35:19,127][00226] Component LearnerWorker_p0 stopped! -[2024-09-04 14:35:19,135][02733] Stopping LearnerWorker_p0... -[2024-09-04 14:35:19,136][02733] Loop learner_proc0_evt_loop terminating... -[2024-09-04 14:35:19,174][00226] Component RolloutWorker_w3 stopped! -[2024-09-04 14:35:19,181][02750] Stopping RolloutWorker_w3... -[2024-09-04 14:35:19,196][02750] Loop rollout_proc3_evt_loop terminating... -[2024-09-04 14:35:19,214][00226] Component RolloutWorker_w1 stopped! -[2024-09-04 14:35:19,221][02748] Stopping RolloutWorker_w1... -[2024-09-04 14:35:19,222][02748] Loop rollout_proc1_evt_loop terminating... -[2024-09-04 14:35:19,242][00226] Component RolloutWorker_w7 stopped! -[2024-09-04 14:35:19,248][02753] Stopping RolloutWorker_w7... -[2024-09-04 14:35:19,250][02753] Loop rollout_proc7_evt_loop terminating... -[2024-09-04 14:35:19,256][00226] Component RolloutWorker_w5 stopped! -[2024-09-04 14:35:19,262][02752] Stopping RolloutWorker_w5... -[2024-09-04 14:35:19,263][02752] Loop rollout_proc5_evt_loop terminating... -[2024-09-04 14:35:19,296][02754] Stopping RolloutWorker_w0... -[2024-09-04 14:35:19,296][00226] Component RolloutWorker_w0 stopped! -[2024-09-04 14:35:19,307][02749] Stopping RolloutWorker_w2... -[2024-09-04 14:35:19,307][00226] Component RolloutWorker_w2 stopped! -[2024-09-04 14:35:19,296][02754] Loop rollout_proc0_evt_loop terminating... -[2024-09-04 14:35:19,308][02749] Loop rollout_proc2_evt_loop terminating... -[2024-09-04 14:35:19,330][00226] Component RolloutWorker_w6 stopped! -[2024-09-04 14:35:19,336][02751] Stopping RolloutWorker_w6... -[2024-09-04 14:35:19,336][02751] Loop rollout_proc6_evt_loop terminating... -[2024-09-04 14:35:19,352][00226] Component RolloutWorker_w4 stopped! -[2024-09-04 14:35:19,358][00226] Waiting for process learner_proc0 to stop... -[2024-09-04 14:35:19,361][02747] Stopping RolloutWorker_w4... -[2024-09-04 14:35:19,362][02747] Loop rollout_proc4_evt_loop terminating... -[2024-09-04 14:35:20,763][00226] Waiting for process inference_proc0-0 to join... -[2024-09-04 14:35:20,773][00226] Waiting for process rollout_proc0 to join... -[2024-09-04 14:35:22,788][00226] Waiting for process rollout_proc1 to join... -[2024-09-04 14:35:22,791][00226] Waiting for process rollout_proc2 to join... -[2024-09-04 14:35:22,797][00226] Waiting for process rollout_proc3 to join... -[2024-09-04 14:35:22,801][00226] Waiting for process rollout_proc4 to join... -[2024-09-04 14:35:22,805][00226] Waiting for process rollout_proc5 to join... -[2024-09-04 14:35:22,808][00226] Waiting for process rollout_proc6 to join... -[2024-09-04 14:35:22,811][00226] Waiting for process rollout_proc7 to join... -[2024-09-04 14:35:22,814][00226] Batcher 0 profile tree view: -batching: 27.6191, releasing_batches: 0.0333 -[2024-09-04 14:35:22,816][00226] InferenceWorker_p0-w0 profile tree view: -wait_policy: 0.0021 - wait_policy_total: 424.8422 -update_model: 9.6260 - weight_update: 0.0034 -one_step: 0.0033 - handle_policy_step: 624.4399 - deserialize: 15.8167, stack: 3.3955, obs_to_device_normalize: 125.3660, forward: 334.9877, send_messages: 29.9518 - prepare_outputs: 85.0549 - to_cpu: 48.8722 -[2024-09-04 14:35:22,818][00226] Learner 0 profile tree view: -misc: 0.0069, prepare_batch: 14.2060 -train: 76.2658 - epoch_init: 0.0057, minibatch_init: 0.0064, losses_postprocess: 0.5895, kl_divergence: 0.6413, after_optimizer: 34.8555 - calculate_losses: 27.6681 - losses_init: 0.0035, forward_head: 1.3844, bptt_initial: 18.7944, tail: 1.1849, advantages_returns: 0.2757, losses: 3.7814 - bptt: 1.9570 - bptt_forward_core: 1.8674 - update: 11.8121 - clip: 0.9048 -[2024-09-04 14:35:22,822][00226] RolloutWorker_w0 profile tree view: -wait_for_trajectories: 0.2879, enqueue_policy_requests: 106.4764, env_step: 856.8877, overhead: 15.9705, complete_rollouts: 7.6942 -save_policy_outputs: 23.0430 - split_output_tensors: 8.9945 -[2024-09-04 14:35:22,824][00226] RolloutWorker_w7 profile tree view: -wait_for_trajectories: 0.3581, enqueue_policy_requests: 107.5445, env_step: 857.6214, overhead: 15.7869, complete_rollouts: 6.5798 -save_policy_outputs: 22.0498 - split_output_tensors: 8.9251 -[2024-09-04 14:35:22,826][00226] Loop Runner_EvtLoop terminating... -[2024-09-04 14:35:22,828][00226] Runner profile tree view: -main_loop: 1129.2680 -[2024-09-04 14:35:22,829][00226] Collected {0: 4005888}, FPS: 3547.3 -[2024-09-04 14:35:23,113][00226] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json -[2024-09-04 14:35:23,115][00226] Overriding arg 'num_workers' with value 1 passed from command line -[2024-09-04 14:35:23,117][00226] Adding new argument 'no_render'=True that is not in the saved config file! -[2024-09-04 14:35:23,119][00226] Adding new argument 'save_video'=True that is not in the saved config file! -[2024-09-04 14:35:23,121][00226] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! -[2024-09-04 14:35:23,123][00226] Adding new argument 'video_name'=None that is not in the saved config file! -[2024-09-04 14:35:23,124][00226] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! -[2024-09-04 14:35:23,126][00226] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! -[2024-09-04 14:35:23,127][00226] Adding new argument 'push_to_hub'=False that is not in the saved config file! -[2024-09-04 14:35:23,128][00226] Adding new argument 'hf_repository'=None that is not in the saved config file! -[2024-09-04 14:35:23,129][00226] Adding new argument 'policy_index'=0 that is not in the saved config file! -[2024-09-04 14:35:23,130][00226] Adding new argument 'eval_deterministic'=False that is not in the saved config file! -[2024-09-04 14:35:23,131][00226] Adding new argument 'train_script'=None that is not in the saved config file! -[2024-09-04 14:35:23,132][00226] Adding new argument 'enjoy_script'=None that is not in the saved config file! -[2024-09-04 14:35:23,133][00226] Using frameskip 1 and render_action_repeat=4 for evaluation -[2024-09-04 14:35:23,166][00226] Doom resolution: 160x120, resize resolution: (128, 72) -[2024-09-04 14:35:23,169][00226] RunningMeanStd input shape: (3, 72, 128) -[2024-09-04 14:35:23,172][00226] RunningMeanStd input shape: (1,) -[2024-09-04 14:35:23,188][00226] ConvEncoder: input_channels=3 -[2024-09-04 14:35:23,293][00226] Conv encoder output size: 512 -[2024-09-04 14:35:23,295][00226] Policy head output size: 512 -[2024-09-04 14:35:23,584][00226] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... -[2024-09-04 14:35:24,391][00226] Num frames 100... -[2024-09-04 14:35:24,521][00226] Num frames 200... -[2024-09-04 14:35:24,644][00226] Num frames 300... -[2024-09-04 14:35:24,767][00226] Num frames 400... -[2024-09-04 14:35:24,887][00226] Num frames 500... -[2024-09-04 14:35:25,007][00226] Num frames 600... -[2024-09-04 14:35:25,135][00226] Num frames 700... -[2024-09-04 14:35:25,252][00226] Num frames 800... -[2024-09-04 14:35:25,383][00226] Num frames 900... -[2024-09-04 14:35:25,507][00226] Num frames 1000... -[2024-09-04 14:35:25,629][00226] Num frames 1100... -[2024-09-04 14:35:25,749][00226] Num frames 1200... -[2024-09-04 14:35:25,871][00226] Num frames 1300... -[2024-09-04 14:35:25,995][00226] Num frames 1400... -[2024-09-04 14:35:26,122][00226] Num frames 1500... -[2024-09-04 14:35:26,202][00226] Avg episode rewards: #0: 34.130, true rewards: #0: 15.130 -[2024-09-04 14:35:26,204][00226] Avg episode reward: 34.130, avg true_objective: 15.130 -[2024-09-04 14:35:26,312][00226] Num frames 1600... -[2024-09-04 14:35:26,440][00226] Num frames 1700... -[2024-09-04 14:35:26,557][00226] Num frames 1800... -[2024-09-04 14:35:26,732][00226] Avg episode rewards: #0: 20.485, true rewards: #0: 9.485 -[2024-09-04 14:35:26,734][00226] Avg episode reward: 20.485, avg true_objective: 9.485 -[2024-09-04 14:35:26,741][00226] Num frames 1900... -[2024-09-04 14:35:26,860][00226] Num frames 2000... -[2024-09-04 14:35:26,979][00226] Num frames 2100... -[2024-09-04 14:35:27,095][00226] Num frames 2200... -[2024-09-04 14:35:27,225][00226] Num frames 2300... -[2024-09-04 14:35:27,349][00226] Num frames 2400... -[2024-09-04 14:35:27,470][00226] Num frames 2500... -[2024-09-04 14:35:27,610][00226] Avg episode rewards: #0: 17.563, true rewards: #0: 8.563 -[2024-09-04 14:35:27,612][00226] Avg episode reward: 17.563, avg true_objective: 8.563 -[2024-09-04 14:35:27,651][00226] Num frames 2600... -[2024-09-04 14:35:27,770][00226] Num frames 2700... -[2024-09-04 14:35:27,889][00226] Num frames 2800... -[2024-09-04 14:35:28,009][00226] Num frames 2900... -[2024-09-04 14:35:28,127][00226] Num frames 3000... -[2024-09-04 14:35:28,258][00226] Num frames 3100... -[2024-09-04 14:35:28,390][00226] Num frames 3200... -[2024-09-04 14:35:28,543][00226] Num frames 3300... -[2024-09-04 14:35:28,716][00226] Num frames 3400... -[2024-09-04 14:35:28,887][00226] Num frames 3500... -[2024-09-04 14:35:29,060][00226] Num frames 3600... -[2024-09-04 14:35:29,237][00226] Num frames 3700... -[2024-09-04 14:35:29,418][00226] Num frames 3800... -[2024-09-04 14:35:29,582][00226] Num frames 3900... -[2024-09-04 14:35:29,760][00226] Num frames 4000... -[2024-09-04 14:35:29,936][00226] Num frames 4100... -[2024-09-04 14:35:30,091][00226] Avg episode rewards: #0: 22.145, true rewards: #0: 10.395 -[2024-09-04 14:35:30,094][00226] Avg episode reward: 22.145, avg true_objective: 10.395 -[2024-09-04 14:35:30,175][00226] Num frames 4200... -[2024-09-04 14:35:30,362][00226] Num frames 4300... -[2024-09-04 14:35:30,542][00226] Num frames 4400... -[2024-09-04 14:35:30,719][00226] Num frames 4500... -[2024-09-04 14:35:30,894][00226] Num frames 4600... -[2024-09-04 14:35:31,026][00226] Num frames 4700... -[2024-09-04 14:35:31,149][00226] Num frames 4800... -[2024-09-04 14:35:31,271][00226] Num frames 4900... -[2024-09-04 14:35:31,404][00226] Num frames 5000... -[2024-09-04 14:35:31,529][00226] Avg episode rewards: #0: 20.908, true rewards: #0: 10.108 -[2024-09-04 14:35:31,531][00226] Avg episode reward: 20.908, avg true_objective: 10.108 -[2024-09-04 14:35:31,590][00226] Num frames 5100... -[2024-09-04 14:35:31,713][00226] Num frames 5200... -[2024-09-04 14:35:31,832][00226] Num frames 5300... -[2024-09-04 14:35:31,952][00226] Num frames 5400... -[2024-09-04 14:35:32,077][00226] Num frames 5500... -[2024-09-04 14:35:32,200][00226] Num frames 5600... -[2024-09-04 14:35:32,378][00226] Avg episode rewards: #0: 19.980, true rewards: #0: 9.480 -[2024-09-04 14:35:32,379][00226] Avg episode reward: 19.980, avg true_objective: 9.480 -[2024-09-04 14:35:32,398][00226] Num frames 5700... -[2024-09-04 14:35:32,518][00226] Num frames 5800... -[2024-09-04 14:35:32,640][00226] Num frames 5900... -[2024-09-04 14:35:32,762][00226] Num frames 6000... -[2024-09-04 14:35:32,882][00226] Num frames 6100... -[2024-09-04 14:35:33,002][00226] Num frames 6200... -[2024-09-04 14:35:33,123][00226] Num frames 6300... -[2024-09-04 14:35:33,245][00226] Num frames 6400... -[2024-09-04 14:35:33,387][00226] Num frames 6500... -[2024-09-04 14:35:33,508][00226] Num frames 6600... -[2024-09-04 14:35:33,629][00226] Num frames 6700... -[2024-09-04 14:35:33,752][00226] Num frames 6800... -[2024-09-04 14:35:33,878][00226] Num frames 6900... -[2024-09-04 14:35:33,979][00226] Avg episode rewards: #0: 21.337, true rewards: #0: 9.909 -[2024-09-04 14:35:33,981][00226] Avg episode reward: 21.337, avg true_objective: 9.909 -[2024-09-04 14:35:34,076][00226] Num frames 7000... -[2024-09-04 14:35:34,210][00226] Num frames 7100... -[2024-09-04 14:35:34,336][00226] Num frames 7200... -[2024-09-04 14:35:34,480][00226] Num frames 7300... -[2024-09-04 14:35:34,605][00226] Num frames 7400... -[2024-09-04 14:35:34,729][00226] Num frames 7500... -[2024-09-04 14:35:34,849][00226] Num frames 7600... -[2024-09-04 14:35:34,971][00226] Num frames 7700... -[2024-09-04 14:35:35,094][00226] Num frames 7800... -[2024-09-04 14:35:35,216][00226] Num frames 7900... -[2024-09-04 14:35:35,347][00226] Num frames 8000... -[2024-09-04 14:35:35,479][00226] Num frames 8100... -[2024-09-04 14:35:35,603][00226] Num frames 8200... -[2024-09-04 14:35:35,728][00226] Num frames 8300... -[2024-09-04 14:35:35,853][00226] Num frames 8400... -[2024-09-04 14:35:35,974][00226] Num frames 8500... -[2024-09-04 14:35:36,096][00226] Num frames 8600... -[2024-09-04 14:35:36,219][00226] Num frames 8700... -[2024-09-04 14:35:36,349][00226] Num frames 8800... -[2024-09-04 14:35:36,520][00226] Avg episode rewards: #0: 24.860, true rewards: #0: 11.110 -[2024-09-04 14:35:36,522][00226] Avg episode reward: 24.860, avg true_objective: 11.110 -[2024-09-04 14:35:36,542][00226] Num frames 8900... -[2024-09-04 14:35:36,662][00226] Num frames 9000... -[2024-09-04 14:35:36,783][00226] Num frames 9100... -[2024-09-04 14:35:36,902][00226] Num frames 9200... -[2024-09-04 14:35:37,022][00226] Num frames 9300... -[2024-09-04 14:35:37,148][00226] Num frames 9400... -[2024-09-04 14:35:37,326][00226] Avg episode rewards: #0: 23.218, true rewards: #0: 10.551 -[2024-09-04 14:35:37,327][00226] Avg episode reward: 23.218, avg true_objective: 10.551 -[2024-09-04 14:35:37,337][00226] Num frames 9500... -[2024-09-04 14:35:37,469][00226] Num frames 9600... -[2024-09-04 14:35:37,592][00226] Num frames 9700... -[2024-09-04 14:35:37,712][00226] Num frames 9800... -[2024-09-04 14:35:37,835][00226] Num frames 9900... -[2024-09-04 14:35:37,958][00226] Num frames 10000... -[2024-09-04 14:35:38,080][00226] Num frames 10100... -[2024-09-04 14:35:38,202][00226] Num frames 10200... -[2024-09-04 14:35:38,331][00226] Num frames 10300... -[2024-09-04 14:35:38,454][00226] Num frames 10400... -[2024-09-04 14:35:38,586][00226] Num frames 10500... -[2024-09-04 14:35:38,708][00226] Num frames 10600... -[2024-09-04 14:35:38,830][00226] Num frames 10700... -[2024-09-04 14:35:38,949][00226] Num frames 10800... -[2024-09-04 14:35:39,070][00226] Num frames 10900... -[2024-09-04 14:35:39,192][00226] Num frames 11000... -[2024-09-04 14:35:39,315][00226] Num frames 11100... -[2024-09-04 14:35:39,442][00226] Num frames 11200... -[2024-09-04 14:35:39,574][00226] Num frames 11300... -[2024-09-04 14:35:39,696][00226] Num frames 11400... -[2024-09-04 14:35:39,822][00226] Num frames 11500... -[2024-09-04 14:35:39,995][00226] Avg episode rewards: #0: 26.196, true rewards: #0: 11.596 -[2024-09-04 14:35:39,998][00226] Avg episode reward: 26.196, avg true_objective: 11.596 -[2024-09-04 14:36:45,310][00226] Replay video saved to /content/train_dir/default_experiment/replay.mp4! -[2024-09-04 14:38:27,064][00226] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json -[2024-09-04 14:38:27,067][00226] Overriding arg 'num_workers' with value 1 passed from command line -[2024-09-04 14:38:27,069][00226] Adding new argument 'no_render'=True that is not in the saved config file! -[2024-09-04 14:38:27,071][00226] Adding new argument 'save_video'=True that is not in the saved config file! -[2024-09-04 14:38:27,072][00226] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! -[2024-09-04 14:38:27,073][00226] Adding new argument 'video_name'=None that is not in the saved config file! -[2024-09-04 14:38:27,076][00226] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! -[2024-09-04 14:38:27,077][00226] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! -[2024-09-04 14:38:27,078][00226] Adding new argument 'push_to_hub'=True that is not in the saved config file! -[2024-09-04 14:38:27,079][00226] Adding new argument 'hf_repository'='neeldevenshah/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! -[2024-09-04 14:38:27,080][00226] Adding new argument 'policy_index'=0 that is not in the saved config file! -[2024-09-04 14:38:27,082][00226] Adding new argument 'eval_deterministic'=False that is not in the saved config file! -[2024-09-04 14:38:27,083][00226] Adding new argument 'train_script'=None that is not in the saved config file! -[2024-09-04 14:38:27,084][00226] Adding new argument 'enjoy_script'=None that is not in the saved config file! -[2024-09-04 14:38:27,089][00226] Using frameskip 1 and render_action_repeat=4 for evaluation -[2024-09-04 14:38:27,115][00226] RunningMeanStd input shape: (3, 72, 128) -[2024-09-04 14:38:27,118][00226] RunningMeanStd input shape: (1,) -[2024-09-04 14:38:27,131][00226] ConvEncoder: input_channels=3 -[2024-09-04 14:38:27,168][00226] Conv encoder output size: 512 -[2024-09-04 14:38:27,169][00226] Policy head output size: 512 -[2024-09-04 14:38:27,189][00226] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... -[2024-09-04 14:38:27,630][00226] Num frames 100... -[2024-09-04 14:38:27,757][00226] Num frames 200... -[2024-09-04 14:38:27,876][00226] Num frames 300... -[2024-09-04 14:38:27,945][00226] Avg episode rewards: #0: 4.090, true rewards: #0: 3.090 -[2024-09-04 14:38:27,947][00226] Avg episode reward: 4.090, avg true_objective: 3.090 -[2024-09-04 14:38:28,061][00226] Num frames 400... -[2024-09-04 14:38:28,181][00226] Num frames 500... -[2024-09-04 14:38:28,299][00226] Num frames 600... -[2024-09-04 14:38:28,428][00226] Num frames 700... -[2024-09-04 14:38:28,549][00226] Num frames 800... -[2024-09-04 14:38:28,672][00226] Num frames 900... -[2024-09-04 14:38:28,800][00226] Num frames 1000... -[2024-09-04 14:38:28,921][00226] Num frames 1100... -[2024-09-04 14:38:29,039][00226] Num frames 1200... -[2024-09-04 14:38:29,164][00226] Num frames 1300... -[2024-09-04 14:38:29,288][00226] Num frames 1400... -[2024-09-04 14:38:29,448][00226] Avg episode rewards: #0: 16.405, true rewards: #0: 7.405 -[2024-09-04 14:38:29,449][00226] Avg episode reward: 16.405, avg true_objective: 7.405 -[2024-09-04 14:38:29,476][00226] Num frames 1500... -[2024-09-04 14:38:29,617][00226] Num frames 1600... -[2024-09-04 14:38:29,789][00226] Num frames 1700... -[2024-09-04 14:38:29,967][00226] Num frames 1800... -[2024-09-04 14:38:30,133][00226] Num frames 1900... -[2024-09-04 14:38:30,314][00226] Num frames 2000... -[2024-09-04 14:38:30,491][00226] Num frames 2100... -[2024-09-04 14:38:30,658][00226] Num frames 2200... -[2024-09-04 14:38:30,827][00226] Num frames 2300... -[2024-09-04 14:38:31,004][00226] Num frames 2400... -[2024-09-04 14:38:31,177][00226] Num frames 2500... -[2024-09-04 14:38:31,366][00226] Num frames 2600... -[2024-09-04 14:38:31,545][00226] Num frames 2700... -[2024-09-04 14:38:31,717][00226] Num frames 2800... -[2024-09-04 14:38:31,910][00226] Num frames 2900... -[2024-09-04 14:38:32,019][00226] Avg episode rewards: #0: 24.753, true rewards: #0: 9.753 -[2024-09-04 14:38:32,021][00226] Avg episode reward: 24.753, avg true_objective: 9.753 -[2024-09-04 14:38:32,118][00226] Num frames 3000... -[2024-09-04 14:38:32,238][00226] Num frames 3100... -[2024-09-04 14:38:32,369][00226] Num frames 3200... -[2024-09-04 14:38:32,490][00226] Num frames 3300... -[2024-09-04 14:38:32,612][00226] Num frames 3400... -[2024-09-04 14:38:32,732][00226] Num frames 3500... -[2024-09-04 14:38:32,855][00226] Num frames 3600... -[2024-09-04 14:38:32,984][00226] Num frames 3700... -[2024-09-04 14:38:33,104][00226] Num frames 3800... -[2024-09-04 14:38:33,208][00226] Avg episode rewards: #0: 23.848, true rewards: #0: 9.597 -[2024-09-04 14:38:33,209][00226] Avg episode reward: 23.848, avg true_objective: 9.597 -[2024-09-04 14:38:33,285][00226] Num frames 3900... -[2024-09-04 14:38:33,417][00226] Num frames 4000... -[2024-09-04 14:38:33,539][00226] Num frames 4100... -[2024-09-04 14:38:33,659][00226] Num frames 4200... -[2024-09-04 14:38:33,822][00226] Avg episode rewards: #0: 20.974, true rewards: #0: 8.574 -[2024-09-04 14:38:33,823][00226] Avg episode reward: 20.974, avg true_objective: 8.574 -[2024-09-04 14:38:33,842][00226] Num frames 4300... -[2024-09-04 14:38:33,973][00226] Num frames 4400... -[2024-09-04 14:38:34,095][00226] Num frames 4500... -[2024-09-04 14:38:34,220][00226] Num frames 4600... -[2024-09-04 14:38:34,345][00226] Num frames 4700... -[2024-09-04 14:38:34,474][00226] Num frames 4800... -[2024-09-04 14:38:34,596][00226] Num frames 4900... -[2024-09-04 14:38:34,720][00226] Num frames 5000... -[2024-09-04 14:38:34,874][00226] Num frames 5100... -[2024-09-04 14:38:34,953][00226] Avg episode rewards: #0: 20.365, true rewards: #0: 8.532 -[2024-09-04 14:38:34,954][00226] Avg episode reward: 20.365, avg true_objective: 8.532 -[2024-09-04 14:38:35,065][00226] Num frames 5200... -[2024-09-04 14:38:35,190][00226] Num frames 5300... -[2024-09-04 14:38:35,319][00226] Num frames 5400... -[2024-09-04 14:38:35,443][00226] Num frames 5500... -[2024-09-04 14:38:35,567][00226] Num frames 5600... -[2024-09-04 14:38:35,687][00226] Num frames 5700... -[2024-09-04 14:38:35,809][00226] Num frames 5800... -[2024-09-04 14:38:35,928][00226] Num frames 5900... -[2024-09-04 14:38:36,064][00226] Num frames 6000... -[2024-09-04 14:38:36,186][00226] Num frames 6100... -[2024-09-04 14:38:36,306][00226] Num frames 6200... -[2024-09-04 14:38:36,428][00226] Num frames 6300... -[2024-09-04 14:38:36,551][00226] Num frames 6400... -[2024-09-04 14:38:36,676][00226] Num frames 6500... -[2024-09-04 14:38:36,800][00226] Num frames 6600... -[2024-09-04 14:38:36,921][00226] Num frames 6700... -[2024-09-04 14:38:37,056][00226] Num frames 6800... -[2024-09-04 14:38:37,178][00226] Num frames 6900... -[2024-09-04 14:38:37,303][00226] Num frames 7000... -[2024-09-04 14:38:37,431][00226] Num frames 7100... -[2024-09-04 14:38:37,553][00226] Num frames 7200... -[2024-09-04 14:38:37,632][00226] Avg episode rewards: #0: 25.313, true rewards: #0: 10.313 -[2024-09-04 14:38:37,634][00226] Avg episode reward: 25.313, avg true_objective: 10.313 -[2024-09-04 14:38:37,732][00226] Num frames 7300... -[2024-09-04 14:38:37,853][00226] Num frames 7400... -[2024-09-04 14:38:37,974][00226] Num frames 7500... -[2024-09-04 14:38:38,100][00226] Num frames 7600... -[2024-09-04 14:38:38,220][00226] Num frames 7700... -[2024-09-04 14:38:38,351][00226] Num frames 7800... -[2024-09-04 14:38:38,508][00226] Avg episode rewards: #0: 24.109, true rewards: #0: 9.859 -[2024-09-04 14:38:38,510][00226] Avg episode reward: 24.109, avg true_objective: 9.859 -[2024-09-04 14:38:38,529][00226] Num frames 7900... -[2024-09-04 14:38:38,649][00226] Num frames 8000... -[2024-09-04 14:38:38,771][00226] Num frames 8100... -[2024-09-04 14:38:38,889][00226] Num frames 8200... -[2024-09-04 14:38:39,009][00226] Num frames 8300... -[2024-09-04 14:38:39,136][00226] Num frames 8400... -[2024-09-04 14:38:39,260][00226] Num frames 8500... -[2024-09-04 14:38:39,387][00226] Num frames 8600... -[2024-09-04 14:38:39,510][00226] Num frames 8700... -[2024-09-04 14:38:39,634][00226] Num frames 8800... -[2024-09-04 14:38:39,760][00226] Num frames 8900... -[2024-09-04 14:38:39,882][00226] Num frames 9000... -[2024-09-04 14:38:40,003][00226] Num frames 9100... -[2024-09-04 14:38:40,139][00226] Num frames 9200... -[2024-09-04 14:38:40,260][00226] Num frames 9300... -[2024-09-04 14:38:40,392][00226] Num frames 9400... -[2024-09-04 14:38:40,513][00226] Num frames 9500... -[2024-09-04 14:38:40,649][00226] Num frames 9600... -[2024-09-04 14:38:40,761][00226] Avg episode rewards: #0: 25.830, true rewards: #0: 10.719 -[2024-09-04 14:38:40,763][00226] Avg episode reward: 25.830, avg true_objective: 10.719 -[2024-09-04 14:38:40,829][00226] Num frames 9700... -[2024-09-04 14:38:40,948][00226] Num frames 9800... -[2024-09-04 14:38:41,068][00226] Num frames 9900... -[2024-09-04 14:38:41,199][00226] Num frames 10000... -[2024-09-04 14:38:41,325][00226] Num frames 10100... -[2024-09-04 14:38:41,451][00226] Num frames 10200... -[2024-09-04 14:38:41,572][00226] Num frames 10300... -[2024-09-04 14:38:41,694][00226] Avg episode rewards: #0: 24.650, true rewards: #0: 10.350 -[2024-09-04 14:38:41,698][00226] Avg episode reward: 24.650, avg true_objective: 10.350 -[2024-09-04 14:39:42,606][00226] Replay video saved to /content/train_dir/default_experiment/replay.mp4! +[2024-09-05 08:53:30,405][02559] Using optimizer +[2024-09-05 08:53:30,995][00556] Heartbeat connected on Batcher_0 +[2024-09-05 08:53:31,018][00556] Heartbeat connected on InferenceWorker_p0-w0 +[2024-09-05 08:53:31,036][00556] Heartbeat connected on RolloutWorker_w0 +[2024-09-05 08:53:31,042][00556] Heartbeat connected on RolloutWorker_w1 +[2024-09-05 08:53:31,046][00556] Heartbeat connected on RolloutWorker_w2 +[2024-09-05 08:53:31,051][00556] Heartbeat connected on RolloutWorker_w3 +[2024-09-05 08:53:31,056][00556] Heartbeat connected on RolloutWorker_w4 +[2024-09-05 08:53:31,061][00556] Heartbeat connected on RolloutWorker_w5 +[2024-09-05 08:53:31,073][00556] Heartbeat connected on RolloutWorker_w7 +[2024-09-05 08:53:31,074][00556] Heartbeat connected on RolloutWorker_w6 +[2024-09-05 08:53:31,117][02559] No checkpoints found +[2024-09-05 08:53:31,117][02559] Did not load from checkpoint, starting from scratch! +[2024-09-05 08:53:31,118][02559] Initialized policy 0 weights for model version 0 +[2024-09-05 08:53:31,124][02559] LearnerWorker_p0 finished initialization! +[2024-09-05 08:53:31,125][02559] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2024-09-05 08:53:31,125][00556] Heartbeat connected on LearnerWorker_p0 +[2024-09-05 08:53:31,222][02572] RunningMeanStd input shape: (3, 72, 128) +[2024-09-05 08:53:31,223][02572] RunningMeanStd input shape: (1,) +[2024-09-05 08:53:31,245][02572] ConvEncoder: input_channels=3 +[2024-09-05 08:53:31,355][02572] Conv encoder output size: 512 +[2024-09-05 08:53:31,355][02572] Policy head output size: 512 +[2024-09-05 08:53:31,408][00556] Inference worker 0-0 is ready! +[2024-09-05 08:53:31,409][00556] All inference workers are ready! Signal rollout workers to start! +[2024-09-05 08:53:31,624][02579] Doom resolution: 160x120, resize resolution: (128, 72) +[2024-09-05 08:53:31,626][02576] Doom resolution: 160x120, resize resolution: (128, 72) +[2024-09-05 08:53:31,628][02580] Doom resolution: 160x120, resize resolution: (128, 72) +[2024-09-05 08:53:31,631][02574] Doom resolution: 160x120, resize resolution: (128, 72) +[2024-09-05 08:53:31,623][02577] Doom resolution: 160x120, resize resolution: (128, 72) +[2024-09-05 08:53:31,628][02573] Doom resolution: 160x120, resize resolution: (128, 72) +[2024-09-05 08:53:31,632][02578] Doom resolution: 160x120, resize resolution: (128, 72) +[2024-09-05 08:53:31,643][02575] Doom resolution: 160x120, resize resolution: (128, 72) +[2024-09-05 08:53:31,952][00556] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) +[2024-09-05 08:53:33,104][02575] Decorrelating experience for 0 frames... +[2024-09-05 08:53:33,105][02573] Decorrelating experience for 0 frames... +[2024-09-05 08:53:33,107][02577] Decorrelating experience for 0 frames... +[2024-09-05 08:53:33,112][02579] Decorrelating experience for 0 frames... +[2024-09-05 08:53:33,114][02576] Decorrelating experience for 0 frames... +[2024-09-05 08:53:33,116][02580] Decorrelating experience for 0 frames... +[2024-09-05 08:53:34,507][02575] Decorrelating experience for 32 frames... +[2024-09-05 08:53:34,516][02577] Decorrelating experience for 32 frames... +[2024-09-05 08:53:34,526][02573] Decorrelating experience for 32 frames... +[2024-09-05 08:53:35,033][02580] Decorrelating experience for 32 frames... +[2024-09-05 08:53:35,038][02576] Decorrelating experience for 32 frames... +[2024-09-05 08:53:35,049][02579] Decorrelating experience for 32 frames... +[2024-09-05 08:53:36,952][00556] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) +[2024-09-05 08:53:37,369][02578] Decorrelating experience for 0 frames... +[2024-09-05 08:53:37,756][02575] Decorrelating experience for 64 frames... +[2024-09-05 08:53:37,766][02573] Decorrelating experience for 64 frames... +[2024-09-05 08:53:37,783][02577] Decorrelating experience for 64 frames... +[2024-09-05 08:53:37,917][02574] Decorrelating experience for 0 frames... +[2024-09-05 08:53:38,322][02576] Decorrelating experience for 64 frames... +[2024-09-05 08:53:38,324][02580] Decorrelating experience for 64 frames... +[2024-09-05 08:53:38,324][02579] Decorrelating experience for 64 frames... +[2024-09-05 08:53:38,956][02574] Decorrelating experience for 32 frames... +[2024-09-05 08:53:39,862][02574] Decorrelating experience for 64 frames... +[2024-09-05 08:53:40,106][02578] Decorrelating experience for 32 frames... +[2024-09-05 08:53:40,288][02577] Decorrelating experience for 96 frames... +[2024-09-05 08:53:40,432][02575] Decorrelating experience for 96 frames... +[2024-09-05 08:53:40,460][02573] Decorrelating experience for 96 frames... +[2024-09-05 08:53:41,303][02578] Decorrelating experience for 64 frames... +[2024-09-05 08:53:41,724][02578] Decorrelating experience for 96 frames... +[2024-09-05 08:53:41,880][02574] Decorrelating experience for 96 frames... +[2024-09-05 08:53:41,952][00556] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) +[2024-09-05 08:53:42,060][02579] Decorrelating experience for 96 frames... +[2024-09-05 08:53:42,221][02576] Decorrelating experience for 96 frames... +[2024-09-05 08:53:42,574][02580] Decorrelating experience for 96 frames... +[2024-09-05 08:53:46,009][02559] Signal inference workers to stop experience collection... +[2024-09-05 08:53:46,038][02572] InferenceWorker_p0-w0: stopping experience collection +[2024-09-05 08:53:46,952][00556] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 156.9. Samples: 2354. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) +[2024-09-05 08:53:46,955][00556] Avg episode reward: [(0, '1.943')] +[2024-09-05 08:53:50,604][02559] Signal inference workers to resume experience collection... +[2024-09-05 08:53:50,606][02572] InferenceWorker_p0-w0: resuming experience collection +[2024-09-05 08:53:51,955][00556] Fps is (10 sec: 819.0, 60 sec: 409.5, 300 sec: 409.5). Total num frames: 8192. Throughput: 0: 117.7. Samples: 2354. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) +[2024-09-05 08:53:51,958][00556] Avg episode reward: [(0, '2.574')] +[2024-09-05 08:53:56,953][00556] Fps is (10 sec: 2047.7, 60 sec: 819.1, 300 sec: 819.1). Total num frames: 20480. Throughput: 0: 218.0. Samples: 5450. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 08:53:56,956][00556] Avg episode reward: [(0, '3.669')] +[2024-09-05 08:54:01,476][02572] Updated weights for policy 0, policy_version 10 (0.0036) +[2024-09-05 08:54:01,952][00556] Fps is (10 sec: 3277.7, 60 sec: 1365.3, 300 sec: 1365.3). Total num frames: 40960. Throughput: 0: 359.8. Samples: 10794. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 08:54:01,958][00556] Avg episode reward: [(0, '4.167')] +[2024-09-05 08:54:06,952][00556] Fps is (10 sec: 4506.1, 60 sec: 1872.4, 300 sec: 1872.4). Total num frames: 65536. Throughput: 0: 404.2. Samples: 14148. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-09-05 08:54:06,955][00556] Avg episode reward: [(0, '4.348')] +[2024-09-05 08:54:11,957][00556] Fps is (10 sec: 3684.4, 60 sec: 1945.3, 300 sec: 1945.3). Total num frames: 77824. Throughput: 0: 488.8. Samples: 19554. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) +[2024-09-05 08:54:11,960][00556] Avg episode reward: [(0, '4.364')] +[2024-09-05 08:54:12,410][02572] Updated weights for policy 0, policy_version 20 (0.0036) +[2024-09-05 08:54:16,952][00556] Fps is (10 sec: 2867.3, 60 sec: 2093.5, 300 sec: 2093.5). Total num frames: 94208. Throughput: 0: 530.3. Samples: 23864. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) +[2024-09-05 08:54:16,957][00556] Avg episode reward: [(0, '4.370')] +[2024-09-05 08:54:21,952][00556] Fps is (10 sec: 3688.3, 60 sec: 2293.8, 300 sec: 2293.8). Total num frames: 114688. Throughput: 0: 605.3. Samples: 27238. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-09-05 08:54:21,959][00556] Avg episode reward: [(0, '4.396')] +[2024-09-05 08:54:21,966][02559] Saving new best policy, reward=4.396! +[2024-09-05 08:54:23,067][02572] Updated weights for policy 0, policy_version 30 (0.0028) +[2024-09-05 08:54:26,952][00556] Fps is (10 sec: 4096.1, 60 sec: 2457.6, 300 sec: 2457.6). Total num frames: 135168. Throughput: 0: 754.7. Samples: 33962. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-09-05 08:54:26,956][00556] Avg episode reward: [(0, '4.516')] +[2024-09-05 08:54:26,964][02559] Saving new best policy, reward=4.480! +[2024-09-05 08:54:31,952][00556] Fps is (10 sec: 3686.4, 60 sec: 2525.9, 300 sec: 2525.9). Total num frames: 151552. Throughput: 0: 796.0. Samples: 38174. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 08:54:31,956][00556] Avg episode reward: [(0, '4.449')] +[2024-09-05 08:54:35,372][02572] Updated weights for policy 0, policy_version 40 (0.0048) +[2024-09-05 08:54:36,952][00556] Fps is (10 sec: 3276.8, 60 sec: 2798.9, 300 sec: 2583.6). Total num frames: 167936. Throughput: 0: 847.7. Samples: 40500. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-09-05 08:54:36,958][00556] Avg episode reward: [(0, '4.489')] +[2024-09-05 08:54:36,960][02559] Saving new best policy, reward=4.489! +[2024-09-05 08:54:41,952][00556] Fps is (10 sec: 3686.3, 60 sec: 3140.3, 300 sec: 2691.6). Total num frames: 188416. Throughput: 0: 918.5. Samples: 46780. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 08:54:41,955][00556] Avg episode reward: [(0, '4.491')] +[2024-09-05 08:54:41,964][02559] Saving new best policy, reward=4.491! +[2024-09-05 08:54:45,663][02572] Updated weights for policy 0, policy_version 50 (0.0039) +[2024-09-05 08:54:46,952][00556] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 2730.7). Total num frames: 204800. Throughput: 0: 923.9. Samples: 52370. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 08:54:46,954][00556] Avg episode reward: [(0, '4.392')] +[2024-09-05 08:54:51,952][00556] Fps is (10 sec: 3276.9, 60 sec: 3550.0, 300 sec: 2764.8). Total num frames: 221184. Throughput: 0: 895.9. Samples: 54462. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-09-05 08:54:51,955][00556] Avg episode reward: [(0, '4.388')] +[2024-09-05 08:54:56,952][00556] Fps is (10 sec: 3686.3, 60 sec: 3686.5, 300 sec: 2843.1). Total num frames: 241664. Throughput: 0: 908.2. Samples: 60420. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 08:54:56,955][00556] Avg episode reward: [(0, '4.398')] +[2024-09-05 08:54:57,249][02572] Updated weights for policy 0, policy_version 60 (0.0023) +[2024-09-05 08:55:01,952][00556] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 2912.7). Total num frames: 262144. Throughput: 0: 948.9. Samples: 66564. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-09-05 08:55:01,958][00556] Avg episode reward: [(0, '4.391')] +[2024-09-05 08:55:01,968][02559] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000064_262144.pth... +[2024-09-05 08:55:06,957][00556] Fps is (10 sec: 3684.5, 60 sec: 3549.6, 300 sec: 2931.7). Total num frames: 278528. Throughput: 0: 918.4. Samples: 68570. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-09-05 08:55:06,961][00556] Avg episode reward: [(0, '4.315')] +[2024-09-05 08:55:09,404][02572] Updated weights for policy 0, policy_version 70 (0.0022) +[2024-09-05 08:55:11,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3618.5, 300 sec: 2949.1). Total num frames: 294912. Throughput: 0: 875.2. Samples: 73346. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-09-05 08:55:11,956][00556] Avg episode reward: [(0, '4.475')] +[2024-09-05 08:55:16,952][00556] Fps is (10 sec: 4098.3, 60 sec: 3754.7, 300 sec: 3042.7). Total num frames: 319488. Throughput: 0: 934.5. Samples: 80226. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-09-05 08:55:16,954][00556] Avg episode reward: [(0, '4.484')] +[2024-09-05 08:55:18,464][02572] Updated weights for policy 0, policy_version 80 (0.0021) +[2024-09-05 08:55:21,952][00556] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3053.4). Total num frames: 335872. Throughput: 0: 956.0. Samples: 83518. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 08:55:21,957][00556] Avg episode reward: [(0, '4.532')] +[2024-09-05 08:55:21,966][02559] Saving new best policy, reward=4.532! +[2024-09-05 08:55:26,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3063.1). Total num frames: 352256. Throughput: 0: 906.7. Samples: 87580. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 08:55:26,959][00556] Avg episode reward: [(0, '4.366')] +[2024-09-05 08:55:30,364][02572] Updated weights for policy 0, policy_version 90 (0.0023) +[2024-09-05 08:55:31,952][00556] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3106.1). Total num frames: 372736. Throughput: 0: 923.0. Samples: 93906. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 08:55:31,958][00556] Avg episode reward: [(0, '4.297')] +[2024-09-05 08:55:36,954][00556] Fps is (10 sec: 4505.6, 60 sec: 3822.9, 300 sec: 3178.5). Total num frames: 397312. Throughput: 0: 951.7. Samples: 97290. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-09-05 08:55:36,958][00556] Avg episode reward: [(0, '4.458')] +[2024-09-05 08:55:41,306][02572] Updated weights for policy 0, policy_version 100 (0.0022) +[2024-09-05 08:55:41,952][00556] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3150.8). Total num frames: 409600. Throughput: 0: 933.6. Samples: 102432. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 08:55:41,954][00556] Avg episode reward: [(0, '4.403')] +[2024-09-05 08:55:46,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3185.8). Total num frames: 430080. Throughput: 0: 910.8. Samples: 107552. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-09-05 08:55:46,958][00556] Avg episode reward: [(0, '4.432')] +[2024-09-05 08:55:51,956][00556] Fps is (10 sec: 3684.7, 60 sec: 3754.4, 300 sec: 3188.9). Total num frames: 446464. Throughput: 0: 942.2. Samples: 110968. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 08:55:51,960][00556] Avg episode reward: [(0, '4.526')] +[2024-09-05 08:55:52,053][02572] Updated weights for policy 0, policy_version 110 (0.0031) +[2024-09-05 08:55:56,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3686.4, 300 sec: 3192.1). Total num frames: 462848. Throughput: 0: 935.2. Samples: 115432. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 08:55:56,955][00556] Avg episode reward: [(0, '4.412')] +[2024-09-05 08:56:01,957][00556] Fps is (10 sec: 2867.2, 60 sec: 3549.6, 300 sec: 3167.5). Total num frames: 475136. Throughput: 0: 862.4. Samples: 119040. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-09-05 08:56:01,959][00556] Avg episode reward: [(0, '4.470')] +[2024-09-05 08:56:06,419][02572] Updated weights for policy 0, policy_version 120 (0.0060) +[2024-09-05 08:56:06,952][00556] Fps is (10 sec: 2867.2, 60 sec: 3550.2, 300 sec: 3171.1). Total num frames: 491520. Throughput: 0: 839.8. Samples: 121310. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 08:56:06,959][00556] Avg episode reward: [(0, '4.585')] +[2024-09-05 08:56:06,964][02559] Saving new best policy, reward=4.585! +[2024-09-05 08:56:11,952][00556] Fps is (10 sec: 4097.8, 60 sec: 3686.4, 300 sec: 3225.6). Total num frames: 516096. Throughput: 0: 896.5. Samples: 127924. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-09-05 08:56:11,959][00556] Avg episode reward: [(0, '4.587')] +[2024-09-05 08:56:11,968][02559] Saving new best policy, reward=4.587! +[2024-09-05 08:56:16,540][02572] Updated weights for policy 0, policy_version 130 (0.0021) +[2024-09-05 08:56:16,952][00556] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3227.2). Total num frames: 532480. Throughput: 0: 882.6. Samples: 133622. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-09-05 08:56:16,954][00556] Avg episode reward: [(0, '4.587')] +[2024-09-05 08:56:21,952][00556] Fps is (10 sec: 2867.3, 60 sec: 3481.6, 300 sec: 3204.5). Total num frames: 544768. Throughput: 0: 853.4. Samples: 135694. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 08:56:21,958][00556] Avg episode reward: [(0, '4.576')] +[2024-09-05 08:56:26,952][00556] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3253.4). Total num frames: 569344. Throughput: 0: 869.4. Samples: 141556. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 08:56:26,961][00556] Avg episode reward: [(0, '4.439')] +[2024-09-05 08:56:27,656][02572] Updated weights for policy 0, policy_version 140 (0.0025) +[2024-09-05 08:56:31,955][00556] Fps is (10 sec: 4504.0, 60 sec: 3617.9, 300 sec: 3276.7). Total num frames: 589824. Throughput: 0: 904.5. Samples: 148260. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 08:56:31,958][00556] Avg episode reward: [(0, '4.341')] +[2024-09-05 08:56:36,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3254.7). Total num frames: 602112. Throughput: 0: 876.0. Samples: 150384. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 08:56:36,958][00556] Avg episode reward: [(0, '4.360')] +[2024-09-05 08:56:39,747][02572] Updated weights for policy 0, policy_version 150 (0.0040) +[2024-09-05 08:56:41,952][00556] Fps is (10 sec: 3277.9, 60 sec: 3549.9, 300 sec: 3276.8). Total num frames: 622592. Throughput: 0: 877.2. Samples: 154906. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 08:56:41,958][00556] Avg episode reward: [(0, '4.385')] +[2024-09-05 08:56:46,952][00556] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3297.8). Total num frames: 643072. Throughput: 0: 946.3. Samples: 161620. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 08:56:46,956][00556] Avg episode reward: [(0, '4.524')] +[2024-09-05 08:56:48,843][02572] Updated weights for policy 0, policy_version 160 (0.0021) +[2024-09-05 08:56:51,952][00556] Fps is (10 sec: 4096.0, 60 sec: 3618.4, 300 sec: 3317.8). Total num frames: 663552. Throughput: 0: 968.2. Samples: 164880. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 08:56:51,955][00556] Avg episode reward: [(0, '4.675')] +[2024-09-05 08:56:51,981][02559] Saving new best policy, reward=4.675! +[2024-09-05 08:56:56,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3296.8). Total num frames: 675840. Throughput: 0: 910.7. Samples: 168904. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 08:56:56,955][00556] Avg episode reward: [(0, '4.592')] +[2024-09-05 08:57:00,994][02572] Updated weights for policy 0, policy_version 170 (0.0019) +[2024-09-05 08:57:01,952][00556] Fps is (10 sec: 3686.3, 60 sec: 3754.9, 300 sec: 3335.3). Total num frames: 700416. Throughput: 0: 922.6. Samples: 175140. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 08:57:01,955][00556] Avg episode reward: [(0, '4.742')] +[2024-09-05 08:57:01,966][02559] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000171_700416.pth... +[2024-09-05 08:57:02,094][02559] Saving new best policy, reward=4.742! +[2024-09-05 08:57:06,952][00556] Fps is (10 sec: 4505.6, 60 sec: 3822.9, 300 sec: 3353.0). Total num frames: 720896. Throughput: 0: 947.5. Samples: 178330. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 08:57:06,963][00556] Avg episode reward: [(0, '4.563')] +[2024-09-05 08:57:11,952][00556] Fps is (10 sec: 3276.9, 60 sec: 3618.1, 300 sec: 3332.7). Total num frames: 733184. Throughput: 0: 931.1. Samples: 183454. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-09-05 08:57:11,959][00556] Avg episode reward: [(0, '4.692')] +[2024-09-05 08:57:12,261][02572] Updated weights for policy 0, policy_version 180 (0.0021) +[2024-09-05 08:57:16,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3686.4, 300 sec: 3349.6). Total num frames: 753664. Throughput: 0: 893.9. Samples: 188482. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-09-05 08:57:16,955][00556] Avg episode reward: [(0, '4.620')] +[2024-09-05 08:57:21,952][00556] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3365.8). Total num frames: 774144. Throughput: 0: 925.1. Samples: 192012. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-09-05 08:57:21,955][00556] Avg episode reward: [(0, '4.714')] +[2024-09-05 08:57:22,052][02572] Updated weights for policy 0, policy_version 190 (0.0036) +[2024-09-05 08:57:26,952][00556] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3381.4). Total num frames: 794624. Throughput: 0: 970.7. Samples: 198588. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-09-05 08:57:26,959][00556] Avg episode reward: [(0, '4.770')] +[2024-09-05 08:57:26,962][02559] Saving new best policy, reward=4.770! +[2024-09-05 08:57:31,954][00556] Fps is (10 sec: 3276.1, 60 sec: 3618.2, 300 sec: 3362.1). Total num frames: 806912. Throughput: 0: 915.6. Samples: 202826. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-09-05 08:57:31,957][00556] Avg episode reward: [(0, '4.740')] +[2024-09-05 08:57:33,993][02572] Updated weights for policy 0, policy_version 200 (0.0035) +[2024-09-05 08:57:36,952][00556] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3393.8). Total num frames: 831488. Throughput: 0: 913.2. Samples: 205974. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 08:57:36,954][00556] Avg episode reward: [(0, '4.396')] +[2024-09-05 08:57:41,952][00556] Fps is (10 sec: 4916.0, 60 sec: 3891.2, 300 sec: 3424.3). Total num frames: 856064. Throughput: 0: 978.5. Samples: 212936. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 08:57:41,956][00556] Avg episode reward: [(0, '4.385')] +[2024-09-05 08:57:43,251][02572] Updated weights for policy 0, policy_version 210 (0.0023) +[2024-09-05 08:57:46,952][00556] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3405.3). Total num frames: 868352. Throughput: 0: 946.0. Samples: 217710. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-09-05 08:57:46,954][00556] Avg episode reward: [(0, '4.545')] +[2024-09-05 08:57:51,952][00556] Fps is (10 sec: 2867.3, 60 sec: 3686.4, 300 sec: 3402.8). Total num frames: 884736. Throughput: 0: 923.6. Samples: 219890. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-09-05 08:57:51,958][00556] Avg episode reward: [(0, '4.463')] +[2024-09-05 08:57:54,725][02572] Updated weights for policy 0, policy_version 220 (0.0031) +[2024-09-05 08:57:56,954][00556] Fps is (10 sec: 4095.1, 60 sec: 3891.1, 300 sec: 3431.3). Total num frames: 909312. Throughput: 0: 964.6. Samples: 226864. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-09-05 08:57:56,961][00556] Avg episode reward: [(0, '4.525')] +[2024-09-05 08:58:01,952][00556] Fps is (10 sec: 4505.6, 60 sec: 3822.9, 300 sec: 3443.7). Total num frames: 929792. Throughput: 0: 989.7. Samples: 233018. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 08:58:01,957][00556] Avg episode reward: [(0, '4.519')] +[2024-09-05 08:58:05,896][02572] Updated weights for policy 0, policy_version 230 (0.0025) +[2024-09-05 08:58:06,953][00556] Fps is (10 sec: 3277.1, 60 sec: 3686.3, 300 sec: 3425.7). Total num frames: 942080. Throughput: 0: 958.1. Samples: 235130. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 08:58:06,956][00556] Avg episode reward: [(0, '4.549')] +[2024-09-05 08:58:11,952][00556] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3452.3). Total num frames: 966656. Throughput: 0: 939.8. Samples: 240878. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 08:58:11,955][00556] Avg episode reward: [(0, '4.488')] +[2024-09-05 08:58:15,295][02572] Updated weights for policy 0, policy_version 240 (0.0030) +[2024-09-05 08:58:16,952][00556] Fps is (10 sec: 4915.7, 60 sec: 3959.4, 300 sec: 3478.0). Total num frames: 991232. Throughput: 0: 995.7. Samples: 247630. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 08:58:16,958][00556] Avg episode reward: [(0, '4.585')] +[2024-09-05 08:58:21,952][00556] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3460.4). Total num frames: 1003520. Throughput: 0: 980.7. Samples: 250104. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-09-05 08:58:21,954][00556] Avg episode reward: [(0, '4.596')] +[2024-09-05 08:58:26,878][02572] Updated weights for policy 0, policy_version 250 (0.0026) +[2024-09-05 08:58:26,952][00556] Fps is (10 sec: 3277.0, 60 sec: 3822.9, 300 sec: 3471.2). Total num frames: 1024000. Throughput: 0: 932.1. Samples: 254882. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-09-05 08:58:26,958][00556] Avg episode reward: [(0, '4.695')] +[2024-09-05 08:58:31,952][00556] Fps is (10 sec: 4096.0, 60 sec: 3959.6, 300 sec: 3540.6). Total num frames: 1044480. Throughput: 0: 983.9. Samples: 261984. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-09-05 08:58:31,959][00556] Avg episode reward: [(0, '4.680')] +[2024-09-05 08:58:36,053][02572] Updated weights for policy 0, policy_version 260 (0.0023) +[2024-09-05 08:58:36,952][00556] Fps is (10 sec: 4095.9, 60 sec: 3891.2, 300 sec: 3610.0). Total num frames: 1064960. Throughput: 0: 1014.8. Samples: 265554. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-09-05 08:58:36,955][00556] Avg episode reward: [(0, '4.601')] +[2024-09-05 08:58:41,952][00556] Fps is (10 sec: 3686.5, 60 sec: 3754.7, 300 sec: 3665.6). Total num frames: 1081344. Throughput: 0: 954.7. Samples: 269822. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-09-05 08:58:41,954][00556] Avg episode reward: [(0, '4.539')] +[2024-09-05 08:58:46,952][00556] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3707.3). Total num frames: 1101824. Throughput: 0: 957.4. Samples: 276100. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-09-05 08:58:46,955][00556] Avg episode reward: [(0, '4.750')] +[2024-09-05 08:58:47,228][02572] Updated weights for policy 0, policy_version 270 (0.0042) +[2024-09-05 08:58:51,952][00556] Fps is (10 sec: 4505.5, 60 sec: 4027.7, 300 sec: 3748.9). Total num frames: 1126400. Throughput: 0: 987.6. Samples: 279572. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-09-05 08:58:51,955][00556] Avg episode reward: [(0, '5.012')] +[2024-09-05 08:58:51,965][02559] Saving new best policy, reward=5.012! +[2024-09-05 08:58:56,952][00556] Fps is (10 sec: 3686.4, 60 sec: 3823.1, 300 sec: 3721.1). Total num frames: 1138688. Throughput: 0: 980.5. Samples: 285002. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-09-05 08:58:56,954][00556] Avg episode reward: [(0, '5.118')] +[2024-09-05 08:58:57,014][02559] Saving new best policy, reward=5.118! +[2024-09-05 08:58:58,601][02572] Updated weights for policy 0, policy_version 280 (0.0034) +[2024-09-05 08:59:01,952][00556] Fps is (10 sec: 3276.9, 60 sec: 3822.9, 300 sec: 3707.2). Total num frames: 1159168. Throughput: 0: 944.3. Samples: 290124. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 08:59:01,954][00556] Avg episode reward: [(0, '5.154')] +[2024-09-05 08:59:01,964][02559] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000283_1159168.pth... +[2024-09-05 08:59:02,094][02559] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000064_262144.pth +[2024-09-05 08:59:02,110][02559] Saving new best policy, reward=5.154! +[2024-09-05 08:59:06,952][00556] Fps is (10 sec: 4096.0, 60 sec: 3959.6, 300 sec: 3735.1). Total num frames: 1179648. Throughput: 0: 961.7. Samples: 293382. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 08:59:06,955][00556] Avg episode reward: [(0, '5.208')] +[2024-09-05 08:59:07,051][02559] Saving new best policy, reward=5.208! +[2024-09-05 08:59:07,984][02572] Updated weights for policy 0, policy_version 290 (0.0031) +[2024-09-05 08:59:11,952][00556] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3748.9). Total num frames: 1200128. Throughput: 0: 1002.4. Samples: 299988. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-09-05 08:59:11,956][00556] Avg episode reward: [(0, '5.210')] +[2024-09-05 08:59:11,975][02559] Saving new best policy, reward=5.210! +[2024-09-05 08:59:16,957][00556] Fps is (10 sec: 3684.7, 60 sec: 3754.4, 300 sec: 3734.9). Total num frames: 1216512. Throughput: 0: 939.0. Samples: 304244. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-09-05 08:59:16,959][00556] Avg episode reward: [(0, '5.657')] +[2024-09-05 08:59:16,962][02559] Saving new best policy, reward=5.657! +[2024-09-05 08:59:19,836][02572] Updated weights for policy 0, policy_version 300 (0.0026) +[2024-09-05 08:59:21,952][00556] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3735.0). Total num frames: 1236992. Throughput: 0: 927.3. Samples: 307284. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-09-05 08:59:21,956][00556] Avg episode reward: [(0, '5.778')] +[2024-09-05 08:59:21,966][02559] Saving new best policy, reward=5.778! +[2024-09-05 08:59:26,952][00556] Fps is (10 sec: 4097.9, 60 sec: 3891.2, 300 sec: 3748.9). Total num frames: 1257472. Throughput: 0: 983.5. Samples: 314080. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 08:59:26,954][00556] Avg episode reward: [(0, '5.262')] +[2024-09-05 08:59:29,677][02572] Updated weights for policy 0, policy_version 310 (0.0034) +[2024-09-05 08:59:31,952][00556] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3748.9). Total num frames: 1273856. Throughput: 0: 958.2. Samples: 319220. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 08:59:31,959][00556] Avg episode reward: [(0, '5.243')] +[2024-09-05 08:59:36,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3735.0). Total num frames: 1290240. Throughput: 0: 927.0. Samples: 321288. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 08:59:36,954][00556] Avg episode reward: [(0, '5.225')] +[2024-09-05 08:59:40,707][02572] Updated weights for policy 0, policy_version 320 (0.0035) +[2024-09-05 08:59:41,952][00556] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3762.8). Total num frames: 1314816. Throughput: 0: 955.8. Samples: 328014. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 08:59:41,959][00556] Avg episode reward: [(0, '5.492')] +[2024-09-05 08:59:46,954][00556] Fps is (10 sec: 4505.6, 60 sec: 3891.2, 300 sec: 3776.7). Total num frames: 1335296. Throughput: 0: 976.9. Samples: 334086. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 08:59:46,957][00556] Avg episode reward: [(0, '5.472')] +[2024-09-05 08:59:51,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3686.4, 300 sec: 3748.9). Total num frames: 1347584. Throughput: 0: 950.0. Samples: 336132. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-09-05 08:59:51,959][00556] Avg episode reward: [(0, '5.338')] +[2024-09-05 08:59:52,838][02572] Updated weights for policy 0, policy_version 330 (0.0021) +[2024-09-05 08:59:56,952][00556] Fps is (10 sec: 2867.2, 60 sec: 3754.7, 300 sec: 3735.0). Total num frames: 1363968. Throughput: 0: 903.8. Samples: 340660. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-09-05 08:59:56,959][00556] Avg episode reward: [(0, '5.482')] +[2024-09-05 09:00:01,952][00556] Fps is (10 sec: 2867.2, 60 sec: 3618.1, 300 sec: 3721.2). Total num frames: 1376256. Throughput: 0: 904.1. Samples: 344924. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-09-05 09:00:01,960][00556] Avg episode reward: [(0, '5.405')] +[2024-09-05 09:00:06,243][02572] Updated weights for policy 0, policy_version 340 (0.0029) +[2024-09-05 09:00:06,952][00556] Fps is (10 sec: 2867.1, 60 sec: 3549.9, 300 sec: 3721.1). Total num frames: 1392640. Throughput: 0: 895.3. Samples: 347572. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-09-05 09:00:06,960][00556] Avg episode reward: [(0, '5.463')] +[2024-09-05 09:00:11,952][00556] Fps is (10 sec: 3276.7, 60 sec: 3481.6, 300 sec: 3693.3). Total num frames: 1409024. Throughput: 0: 839.7. Samples: 351868. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-09-05 09:00:11,956][00556] Avg episode reward: [(0, '5.734')] +[2024-09-05 09:00:16,692][02572] Updated weights for policy 0, policy_version 350 (0.0025) +[2024-09-05 09:00:16,952][00556] Fps is (10 sec: 4096.1, 60 sec: 3618.4, 300 sec: 3721.1). Total num frames: 1433600. Throughput: 0: 876.7. Samples: 358670. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-09-05 09:00:16,957][00556] Avg episode reward: [(0, '5.920')] +[2024-09-05 09:00:16,962][02559] Saving new best policy, reward=5.920! +[2024-09-05 09:00:21,952][00556] Fps is (10 sec: 4505.7, 60 sec: 3618.1, 300 sec: 3735.0). Total num frames: 1454080. Throughput: 0: 906.8. Samples: 362094. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:00:21,955][00556] Avg episode reward: [(0, '5.709')] +[2024-09-05 09:00:26,954][00556] Fps is (10 sec: 3276.0, 60 sec: 3481.5, 300 sec: 3707.2). Total num frames: 1466368. Throughput: 0: 858.9. Samples: 366668. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:00:26,957][00556] Avg episode reward: [(0, '5.779')] +[2024-09-05 09:00:28,467][02572] Updated weights for policy 0, policy_version 360 (0.0023) +[2024-09-05 09:00:31,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3693.3). Total num frames: 1486848. Throughput: 0: 857.0. Samples: 372652. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:00:31,954][00556] Avg episode reward: [(0, '6.184')] +[2024-09-05 09:00:31,967][02559] Saving new best policy, reward=6.184! +[2024-09-05 09:00:36,952][00556] Fps is (10 sec: 4506.7, 60 sec: 3686.4, 300 sec: 3735.0). Total num frames: 1511424. Throughput: 0: 885.4. Samples: 375976. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:00:36,957][00556] Avg episode reward: [(0, '6.284')] +[2024-09-05 09:00:36,960][02559] Saving new best policy, reward=6.284! +[2024-09-05 09:00:37,472][02572] Updated weights for policy 0, policy_version 370 (0.0038) +[2024-09-05 09:00:41,952][00556] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3721.1). Total num frames: 1527808. Throughput: 0: 910.5. Samples: 381632. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-09-05 09:00:41,958][00556] Avg episode reward: [(0, '6.069')] +[2024-09-05 09:00:46,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3721.2). Total num frames: 1544192. Throughput: 0: 921.9. Samples: 386408. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:00:46,960][00556] Avg episode reward: [(0, '5.675')] +[2024-09-05 09:00:49,353][02572] Updated weights for policy 0, policy_version 380 (0.0023) +[2024-09-05 09:00:51,952][00556] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3735.0). Total num frames: 1564672. Throughput: 0: 938.8. Samples: 389820. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:00:51,958][00556] Avg episode reward: [(0, '6.108')] +[2024-09-05 09:00:56,957][00556] Fps is (10 sec: 4093.8, 60 sec: 3686.1, 300 sec: 3762.8). Total num frames: 1585152. Throughput: 0: 993.8. Samples: 396592. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:00:56,963][00556] Avg episode reward: [(0, '6.781')] +[2024-09-05 09:00:56,972][02559] Saving new best policy, reward=6.781! +[2024-09-05 09:01:00,208][02572] Updated weights for policy 0, policy_version 390 (0.0040) +[2024-09-05 09:01:01,953][00556] Fps is (10 sec: 3685.8, 60 sec: 3754.6, 300 sec: 3762.7). Total num frames: 1601536. Throughput: 0: 933.3. Samples: 400670. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-09-05 09:01:01,956][00556] Avg episode reward: [(0, '6.826')] +[2024-09-05 09:01:01,977][02559] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000391_1601536.pth... +[2024-09-05 09:01:02,141][02559] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000171_700416.pth +[2024-09-05 09:01:02,163][02559] Saving new best policy, reward=6.826! +[2024-09-05 09:01:06,952][00556] Fps is (10 sec: 3688.4, 60 sec: 3822.9, 300 sec: 3748.9). Total num frames: 1622016. Throughput: 0: 917.7. Samples: 403390. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) +[2024-09-05 09:01:06,957][00556] Avg episode reward: [(0, '7.158')] +[2024-09-05 09:01:06,962][02559] Saving new best policy, reward=7.158! +[2024-09-05 09:01:10,205][02572] Updated weights for policy 0, policy_version 400 (0.0035) +[2024-09-05 09:01:11,952][00556] Fps is (10 sec: 4096.6, 60 sec: 3891.2, 300 sec: 3762.8). Total num frames: 1642496. Throughput: 0: 968.6. Samples: 410254. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-09-05 09:01:11,957][00556] Avg episode reward: [(0, '7.466')] +[2024-09-05 09:01:12,051][02559] Saving new best policy, reward=7.466! +[2024-09-05 09:01:16,952][00556] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3776.7). Total num frames: 1658880. Throughput: 0: 951.7. Samples: 415478. Policy #0 lag: (min: 0.0, avg: 0.3, max: 2.0) +[2024-09-05 09:01:16,959][00556] Avg episode reward: [(0, '7.832')] +[2024-09-05 09:01:16,965][02559] Saving new best policy, reward=7.832! +[2024-09-05 09:01:21,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3686.4, 300 sec: 3748.9). Total num frames: 1675264. Throughput: 0: 925.4. Samples: 417618. Policy #0 lag: (min: 0.0, avg: 0.3, max: 2.0) +[2024-09-05 09:01:21,956][00556] Avg episode reward: [(0, '7.690')] +[2024-09-05 09:01:22,019][02572] Updated weights for policy 0, policy_version 410 (0.0043) +[2024-09-05 09:01:26,952][00556] Fps is (10 sec: 4095.8, 60 sec: 3891.3, 300 sec: 3762.8). Total num frames: 1699840. Throughput: 0: 945.5. Samples: 424182. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) +[2024-09-05 09:01:26,955][00556] Avg episode reward: [(0, '7.430')] +[2024-09-05 09:01:31,101][02572] Updated weights for policy 0, policy_version 420 (0.0016) +[2024-09-05 09:01:31,952][00556] Fps is (10 sec: 4505.6, 60 sec: 3891.2, 300 sec: 3790.5). Total num frames: 1720320. Throughput: 0: 982.9. Samples: 430640. Policy #0 lag: (min: 0.0, avg: 0.3, max: 2.0) +[2024-09-05 09:01:31,955][00556] Avg episode reward: [(0, '7.436')] +[2024-09-05 09:01:36,952][00556] Fps is (10 sec: 3277.0, 60 sec: 3686.4, 300 sec: 3762.8). Total num frames: 1732608. Throughput: 0: 953.9. Samples: 432746. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) +[2024-09-05 09:01:36,955][00556] Avg episode reward: [(0, '7.520')] +[2024-09-05 09:01:41,952][00556] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3776.7). Total num frames: 1757184. Throughput: 0: 927.0. Samples: 438302. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-09-05 09:01:41,955][00556] Avg episode reward: [(0, '7.651')] +[2024-09-05 09:01:42,579][02572] Updated weights for policy 0, policy_version 430 (0.0028) +[2024-09-05 09:01:46,952][00556] Fps is (10 sec: 4505.6, 60 sec: 3891.2, 300 sec: 3776.7). Total num frames: 1777664. Throughput: 0: 988.5. Samples: 445150. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-09-05 09:01:46,954][00556] Avg episode reward: [(0, '7.644')] +[2024-09-05 09:01:51,956][00556] Fps is (10 sec: 3684.9, 60 sec: 3822.7, 300 sec: 3790.5). Total num frames: 1794048. Throughput: 0: 987.6. Samples: 447834. Policy #0 lag: (min: 0.0, avg: 0.3, max: 2.0) +[2024-09-05 09:01:51,965][00556] Avg episode reward: [(0, '8.112')] +[2024-09-05 09:01:51,974][02559] Saving new best policy, reward=8.112! +[2024-09-05 09:01:53,878][02572] Updated weights for policy 0, policy_version 440 (0.0063) +[2024-09-05 09:01:56,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3755.0, 300 sec: 3762.8). Total num frames: 1810432. Throughput: 0: 936.1. Samples: 452378. Policy #0 lag: (min: 0.0, avg: 0.3, max: 2.0) +[2024-09-05 09:01:56,954][00556] Avg episode reward: [(0, '8.305')] +[2024-09-05 09:01:56,960][02559] Saving new best policy, reward=8.305! +[2024-09-05 09:02:01,952][00556] Fps is (10 sec: 4097.7, 60 sec: 3891.3, 300 sec: 3776.7). Total num frames: 1835008. Throughput: 0: 975.0. Samples: 459352. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:02:01,954][00556] Avg episode reward: [(0, '9.176')] +[2024-09-05 09:02:01,968][02559] Saving new best policy, reward=9.176! +[2024-09-05 09:02:03,319][02572] Updated weights for policy 0, policy_version 450 (0.0032) +[2024-09-05 09:02:06,955][00556] Fps is (10 sec: 4505.6, 60 sec: 3891.2, 300 sec: 3804.4). Total num frames: 1855488. Throughput: 0: 1003.6. Samples: 462780. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:02:06,962][00556] Avg episode reward: [(0, '10.076')] +[2024-09-05 09:02:06,968][02559] Saving new best policy, reward=10.076! +[2024-09-05 09:02:11,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3776.7). Total num frames: 1867776. Throughput: 0: 953.9. Samples: 467108. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:02:11,954][00556] Avg episode reward: [(0, '10.471')] +[2024-09-05 09:02:11,967][02559] Saving new best policy, reward=10.471! +[2024-09-05 09:02:14,901][02572] Updated weights for policy 0, policy_version 460 (0.0047) +[2024-09-05 09:02:16,952][00556] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3790.5). Total num frames: 1892352. Throughput: 0: 946.5. Samples: 473234. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:02:16,955][00556] Avg episode reward: [(0, '10.517')] +[2024-09-05 09:02:16,962][02559] Saving new best policy, reward=10.517! +[2024-09-05 09:02:21,955][00556] Fps is (10 sec: 4504.0, 60 sec: 3959.2, 300 sec: 3790.5). Total num frames: 1912832. Throughput: 0: 974.9. Samples: 476620. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:02:21,958][00556] Avg episode reward: [(0, '11.655')] +[2024-09-05 09:02:21,973][02559] Saving new best policy, reward=11.655! +[2024-09-05 09:02:25,339][02572] Updated weights for policy 0, policy_version 470 (0.0032) +[2024-09-05 09:02:26,952][00556] Fps is (10 sec: 3686.4, 60 sec: 3823.0, 300 sec: 3804.4). Total num frames: 1929216. Throughput: 0: 969.5. Samples: 481928. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:02:26,954][00556] Avg episode reward: [(0, '12.074')] +[2024-09-05 09:02:26,959][02559] Saving new best policy, reward=12.074! +[2024-09-05 09:02:31,952][00556] Fps is (10 sec: 3277.9, 60 sec: 3754.7, 300 sec: 3776.6). Total num frames: 1945600. Throughput: 0: 934.3. Samples: 487196. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:02:31,955][00556] Avg episode reward: [(0, '13.274')] +[2024-09-05 09:02:31,967][02559] Saving new best policy, reward=13.274! +[2024-09-05 09:02:35,708][02572] Updated weights for policy 0, policy_version 480 (0.0036) +[2024-09-05 09:02:36,952][00556] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3776.7). Total num frames: 1970176. Throughput: 0: 951.1. Samples: 490630. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:02:36,954][00556] Avg episode reward: [(0, '13.427')] +[2024-09-05 09:02:36,957][02559] Saving new best policy, reward=13.427! +[2024-09-05 09:02:41,952][00556] Fps is (10 sec: 4096.1, 60 sec: 3822.9, 300 sec: 3790.5). Total num frames: 1986560. Throughput: 0: 986.8. Samples: 496784. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:02:41,957][00556] Avg episode reward: [(0, '13.273')] +[2024-09-05 09:02:46,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3790.5). Total num frames: 2002944. Throughput: 0: 926.9. Samples: 501062. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:02:46,954][00556] Avg episode reward: [(0, '13.703')] +[2024-09-05 09:02:46,960][02559] Saving new best policy, reward=13.703! +[2024-09-05 09:02:47,654][02572] Updated weights for policy 0, policy_version 490 (0.0031) +[2024-09-05 09:02:51,952][00556] Fps is (10 sec: 3686.4, 60 sec: 3823.2, 300 sec: 3776.7). Total num frames: 2023424. Throughput: 0: 924.5. Samples: 504384. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-09-05 09:02:51,954][00556] Avg episode reward: [(0, '12.837')] +[2024-09-05 09:02:56,554][02572] Updated weights for policy 0, policy_version 500 (0.0014) +[2024-09-05 09:02:56,952][00556] Fps is (10 sec: 4505.5, 60 sec: 3959.5, 300 sec: 3790.5). Total num frames: 2048000. Throughput: 0: 985.4. Samples: 511452. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:02:56,954][00556] Avg episode reward: [(0, '13.359')] +[2024-09-05 09:03:01,952][00556] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3790.6). Total num frames: 2060288. Throughput: 0: 950.6. Samples: 516012. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:03:01,954][00556] Avg episode reward: [(0, '13.143')] +[2024-09-05 09:03:01,967][02559] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000503_2060288.pth... +[2024-09-05 09:03:02,139][02559] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000283_1159168.pth +[2024-09-05 09:03:06,952][00556] Fps is (10 sec: 3276.9, 60 sec: 3754.7, 300 sec: 3776.7). Total num frames: 2080768. Throughput: 0: 935.3. Samples: 518704. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-09-05 09:03:06,963][00556] Avg episode reward: [(0, '13.856')] +[2024-09-05 09:03:06,972][02559] Saving new best policy, reward=13.856! +[2024-09-05 09:03:07,990][02572] Updated weights for policy 0, policy_version 510 (0.0042) +[2024-09-05 09:03:11,952][00556] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 3776.7). Total num frames: 2105344. Throughput: 0: 970.0. Samples: 525578. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-09-05 09:03:11,955][00556] Avg episode reward: [(0, '14.154')] +[2024-09-05 09:03:11,967][02559] Saving new best policy, reward=14.154! +[2024-09-05 09:03:16,952][00556] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3790.5). Total num frames: 2121728. Throughput: 0: 973.4. Samples: 531000. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:03:16,960][00556] Avg episode reward: [(0, '15.343')] +[2024-09-05 09:03:16,961][02559] Saving new best policy, reward=15.343! +[2024-09-05 09:03:19,420][02572] Updated weights for policy 0, policy_version 520 (0.0025) +[2024-09-05 09:03:21,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3754.9, 300 sec: 3776.6). Total num frames: 2138112. Throughput: 0: 944.2. Samples: 533118. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:03:21,961][00556] Avg episode reward: [(0, '15.865')] +[2024-09-05 09:03:21,969][02559] Saving new best policy, reward=15.865! +[2024-09-05 09:03:26,952][00556] Fps is (10 sec: 4095.9, 60 sec: 3891.2, 300 sec: 3790.5). Total num frames: 2162688. Throughput: 0: 950.7. Samples: 539566. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-09-05 09:03:26,961][00556] Avg episode reward: [(0, '16.982')] +[2024-09-05 09:03:26,964][02559] Saving new best policy, reward=16.982! +[2024-09-05 09:03:28,607][02572] Updated weights for policy 0, policy_version 530 (0.0034) +[2024-09-05 09:03:31,952][00556] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 3790.5). Total num frames: 2183168. Throughput: 0: 1002.1. Samples: 546156. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:03:31,960][00556] Avg episode reward: [(0, '15.947')] +[2024-09-05 09:03:36,952][00556] Fps is (10 sec: 3276.9, 60 sec: 3754.7, 300 sec: 3776.6). Total num frames: 2195456. Throughput: 0: 975.4. Samples: 548278. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:03:36,958][00556] Avg episode reward: [(0, '14.973')] +[2024-09-05 09:03:40,112][02572] Updated weights for policy 0, policy_version 540 (0.0031) +[2024-09-05 09:03:41,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3776.7). Total num frames: 2215936. Throughput: 0: 941.7. Samples: 553830. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:03:41,954][00556] Avg episode reward: [(0, '14.413')] +[2024-09-05 09:03:46,952][00556] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 3776.7). Total num frames: 2240512. Throughput: 0: 995.5. Samples: 560810. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:03:46,956][00556] Avg episode reward: [(0, '14.327')] +[2024-09-05 09:03:49,916][02572] Updated weights for policy 0, policy_version 550 (0.0037) +[2024-09-05 09:03:51,952][00556] Fps is (10 sec: 4095.8, 60 sec: 3891.2, 300 sec: 3790.5). Total num frames: 2256896. Throughput: 0: 994.9. Samples: 563476. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:03:51,959][00556] Avg episode reward: [(0, '15.265')] +[2024-09-05 09:03:56,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3776.7). Total num frames: 2273280. Throughput: 0: 943.1. Samples: 568018. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:03:56,955][00556] Avg episode reward: [(0, '16.985')] +[2024-09-05 09:03:56,957][02559] Saving new best policy, reward=16.985! +[2024-09-05 09:04:01,955][00556] Fps is (10 sec: 3275.9, 60 sec: 3822.7, 300 sec: 3762.7). Total num frames: 2289664. Throughput: 0: 939.7. Samples: 573290. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:04:01,957][00556] Avg episode reward: [(0, '16.565')] +[2024-09-05 09:04:02,489][02572] Updated weights for policy 0, policy_version 560 (0.0042) +[2024-09-05 09:04:06,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3748.9). Total num frames: 2306048. Throughput: 0: 940.0. Samples: 575416. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-09-05 09:04:06,954][00556] Avg episode reward: [(0, '16.400')] +[2024-09-05 09:04:11,952][00556] Fps is (10 sec: 2868.0, 60 sec: 3549.9, 300 sec: 3735.1). Total num frames: 2318336. Throughput: 0: 888.5. Samples: 579548. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:04:11,955][00556] Avg episode reward: [(0, '15.898')] +[2024-09-05 09:04:14,977][02572] Updated weights for policy 0, policy_version 570 (0.0029) +[2024-09-05 09:04:16,952][00556] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3748.9). Total num frames: 2342912. Throughput: 0: 881.2. Samples: 585810. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:04:16,954][00556] Avg episode reward: [(0, '14.277')] +[2024-09-05 09:04:21,952][00556] Fps is (10 sec: 4505.6, 60 sec: 3754.7, 300 sec: 3748.9). Total num frames: 2363392. Throughput: 0: 909.5. Samples: 589206. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:04:21,954][00556] Avg episode reward: [(0, '14.302')] +[2024-09-05 09:04:24,997][02572] Updated weights for policy 0, policy_version 580 (0.0059) +[2024-09-05 09:04:26,952][00556] Fps is (10 sec: 3686.4, 60 sec: 3618.2, 300 sec: 3748.9). Total num frames: 2379776. Throughput: 0: 908.1. Samples: 594696. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-09-05 09:04:26,959][00556] Avg episode reward: [(0, '15.926')] +[2024-09-05 09:04:31,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3748.9). Total num frames: 2396160. Throughput: 0: 870.8. Samples: 599994. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-09-05 09:04:31,956][00556] Avg episode reward: [(0, '17.115')] +[2024-09-05 09:04:31,977][02559] Saving new best policy, reward=17.115! +[2024-09-05 09:04:35,527][02572] Updated weights for policy 0, policy_version 590 (0.0030) +[2024-09-05 09:04:36,952][00556] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3748.9). Total num frames: 2420736. Throughput: 0: 886.4. Samples: 603364. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-09-05 09:04:36,957][00556] Avg episode reward: [(0, '18.489')] +[2024-09-05 09:04:36,961][02559] Saving new best policy, reward=18.489! +[2024-09-05 09:04:41,952][00556] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3735.0). Total num frames: 2437120. Throughput: 0: 927.1. Samples: 609738. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:04:41,954][00556] Avg episode reward: [(0, '19.600')] +[2024-09-05 09:04:41,969][02559] Saving new best policy, reward=19.600! +[2024-09-05 09:04:46,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3748.9). Total num frames: 2453504. Throughput: 0: 904.6. Samples: 613996. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:04:46,957][00556] Avg episode reward: [(0, '19.410')] +[2024-09-05 09:04:47,461][02572] Updated weights for policy 0, policy_version 600 (0.0025) +[2024-09-05 09:04:51,952][00556] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3776.6). Total num frames: 2478080. Throughput: 0: 932.3. Samples: 617370. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:04:51,955][00556] Avg episode reward: [(0, '18.704')] +[2024-09-05 09:04:56,122][02572] Updated weights for policy 0, policy_version 610 (0.0029) +[2024-09-05 09:04:56,954][00556] Fps is (10 sec: 4504.5, 60 sec: 3754.5, 300 sec: 3804.4). Total num frames: 2498560. Throughput: 0: 996.5. Samples: 624394. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:04:56,957][00556] Avg episode reward: [(0, '18.206')] +[2024-09-05 09:05:01,959][00556] Fps is (10 sec: 3683.8, 60 sec: 3754.4, 300 sec: 3804.3). Total num frames: 2514944. Throughput: 0: 961.0. Samples: 629062. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-09-05 09:05:01,966][00556] Avg episode reward: [(0, '19.347')] +[2024-09-05 09:05:01,979][02559] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000614_2514944.pth... +[2024-09-05 09:05:02,165][02559] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000391_1601536.pth +[2024-09-05 09:05:06,952][00556] Fps is (10 sec: 3687.3, 60 sec: 3822.9, 300 sec: 3818.3). Total num frames: 2535424. Throughput: 0: 942.5. Samples: 631618. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-09-05 09:05:06,957][00556] Avg episode reward: [(0, '18.625')] +[2024-09-05 09:05:07,616][02572] Updated weights for policy 0, policy_version 620 (0.0048) +[2024-09-05 09:05:11,952][00556] Fps is (10 sec: 4099.0, 60 sec: 3959.5, 300 sec: 3804.4). Total num frames: 2555904. Throughput: 0: 977.4. Samples: 638680. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-09-05 09:05:11,957][00556] Avg episode reward: [(0, '18.854')] +[2024-09-05 09:05:16,952][00556] Fps is (10 sec: 4096.1, 60 sec: 3891.2, 300 sec: 3804.4). Total num frames: 2576384. Throughput: 0: 982.8. Samples: 644218. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-09-05 09:05:16,958][00556] Avg episode reward: [(0, '18.050')] +[2024-09-05 09:05:18,526][02572] Updated weights for policy 0, policy_version 630 (0.0013) +[2024-09-05 09:05:21,952][00556] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3818.3). Total num frames: 2592768. Throughput: 0: 952.6. Samples: 646230. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-09-05 09:05:21,957][00556] Avg episode reward: [(0, '17.419')] +[2024-09-05 09:05:26,952][00556] Fps is (10 sec: 3686.3, 60 sec: 3891.2, 300 sec: 3818.3). Total num frames: 2613248. Throughput: 0: 951.6. Samples: 652558. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-09-05 09:05:26,958][00556] Avg episode reward: [(0, '17.627')] +[2024-09-05 09:05:28,272][02572] Updated weights for policy 0, policy_version 640 (0.0025) +[2024-09-05 09:05:31,952][00556] Fps is (10 sec: 4095.8, 60 sec: 3959.4, 300 sec: 3804.4). Total num frames: 2633728. Throughput: 0: 1000.1. Samples: 659002. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-09-05 09:05:31,956][00556] Avg episode reward: [(0, '16.934')] +[2024-09-05 09:05:36,954][00556] Fps is (10 sec: 3276.1, 60 sec: 3754.5, 300 sec: 3790.5). Total num frames: 2646016. Throughput: 0: 971.5. Samples: 661090. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-09-05 09:05:36,959][00556] Avg episode reward: [(0, '16.916')] +[2024-09-05 09:05:40,083][02572] Updated weights for policy 0, policy_version 650 (0.0059) +[2024-09-05 09:05:41,952][00556] Fps is (10 sec: 3686.6, 60 sec: 3891.2, 300 sec: 3818.3). Total num frames: 2670592. Throughput: 0: 936.6. Samples: 666540. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:05:41,954][00556] Avg episode reward: [(0, '17.948')] +[2024-09-05 09:05:46,952][00556] Fps is (10 sec: 4506.6, 60 sec: 3959.5, 300 sec: 3818.3). Total num frames: 2691072. Throughput: 0: 987.7. Samples: 673502. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-09-05 09:05:46,959][00556] Avg episode reward: [(0, '18.330')] +[2024-09-05 09:05:49,757][02572] Updated weights for policy 0, policy_version 660 (0.0029) +[2024-09-05 09:05:51,952][00556] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3804.5). Total num frames: 2707456. Throughput: 0: 987.9. Samples: 676074. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-09-05 09:05:51,955][00556] Avg episode reward: [(0, '18.467')] +[2024-09-05 09:05:56,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3754.8, 300 sec: 3804.4). Total num frames: 2723840. Throughput: 0: 927.3. Samples: 680408. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-09-05 09:05:56,959][00556] Avg episode reward: [(0, '19.145')] +[2024-09-05 09:06:00,976][02572] Updated weights for policy 0, policy_version 670 (0.0020) +[2024-09-05 09:06:01,952][00556] Fps is (10 sec: 4096.0, 60 sec: 3891.7, 300 sec: 3818.3). Total num frames: 2748416. Throughput: 0: 954.3. Samples: 687160. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:06:01,959][00556] Avg episode reward: [(0, '18.104')] +[2024-09-05 09:06:06,958][00556] Fps is (10 sec: 4093.6, 60 sec: 3822.6, 300 sec: 3804.3). Total num frames: 2764800. Throughput: 0: 987.0. Samples: 690650. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:06:06,960][00556] Avg episode reward: [(0, '17.727')] +[2024-09-05 09:06:11,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3804.4). Total num frames: 2781184. Throughput: 0: 946.1. Samples: 695132. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:06:11,959][00556] Avg episode reward: [(0, '18.652')] +[2024-09-05 09:06:12,860][02572] Updated weights for policy 0, policy_version 680 (0.0020) +[2024-09-05 09:06:16,952][00556] Fps is (10 sec: 3688.5, 60 sec: 3754.7, 300 sec: 3818.3). Total num frames: 2801664. Throughput: 0: 935.4. Samples: 701096. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:06:16,959][00556] Avg episode reward: [(0, '18.219')] +[2024-09-05 09:06:21,936][02572] Updated weights for policy 0, policy_version 690 (0.0024) +[2024-09-05 09:06:21,952][00556] Fps is (10 sec: 4505.6, 60 sec: 3891.2, 300 sec: 3818.3). Total num frames: 2826240. Throughput: 0: 960.4. Samples: 704304. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:06:21,959][00556] Avg episode reward: [(0, '17.381')] +[2024-09-05 09:06:26,957][00556] Fps is (10 sec: 3684.4, 60 sec: 3754.3, 300 sec: 3790.5). Total num frames: 2838528. Throughput: 0: 964.2. Samples: 709936. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:06:26,968][00556] Avg episode reward: [(0, '18.590')] +[2024-09-05 09:06:31,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3818.3). Total num frames: 2859008. Throughput: 0: 919.0. Samples: 714858. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:06:31,955][00556] Avg episode reward: [(0, '18.294')] +[2024-09-05 09:06:33,528][02572] Updated weights for policy 0, policy_version 700 (0.0030) +[2024-09-05 09:06:36,952][00556] Fps is (10 sec: 4098.3, 60 sec: 3891.3, 300 sec: 3804.4). Total num frames: 2879488. Throughput: 0: 939.3. Samples: 718344. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:06:36,955][00556] Avg episode reward: [(0, '18.017')] +[2024-09-05 09:06:41,952][00556] Fps is (10 sec: 4095.8, 60 sec: 3822.9, 300 sec: 3804.4). Total num frames: 2899968. Throughput: 0: 992.9. Samples: 725090. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-09-05 09:06:41,956][00556] Avg episode reward: [(0, '19.009')] +[2024-09-05 09:06:44,148][02572] Updated weights for policy 0, policy_version 710 (0.0023) +[2024-09-05 09:06:46,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3686.4, 300 sec: 3790.6). Total num frames: 2912256. Throughput: 0: 936.5. Samples: 729302. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:06:46,954][00556] Avg episode reward: [(0, '19.771')] +[2024-09-05 09:06:46,961][02559] Saving new best policy, reward=19.771! +[2024-09-05 09:06:51,952][00556] Fps is (10 sec: 3686.6, 60 sec: 3822.9, 300 sec: 3818.3). Total num frames: 2936832. Throughput: 0: 925.4. Samples: 732288. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:06:51,958][00556] Avg episode reward: [(0, '21.126')] +[2024-09-05 09:06:51,969][02559] Saving new best policy, reward=21.126! +[2024-09-05 09:06:54,447][02572] Updated weights for policy 0, policy_version 720 (0.0018) +[2024-09-05 09:06:56,952][00556] Fps is (10 sec: 4505.6, 60 sec: 3891.2, 300 sec: 3804.4). Total num frames: 2957312. Throughput: 0: 973.4. Samples: 738936. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:06:56,959][00556] Avg episode reward: [(0, '21.435')] +[2024-09-05 09:06:56,962][02559] Saving new best policy, reward=21.435! +[2024-09-05 09:07:01,953][00556] Fps is (10 sec: 3685.9, 60 sec: 3754.6, 300 sec: 3790.5). Total num frames: 2973696. Throughput: 0: 949.7. Samples: 743834. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-09-05 09:07:01,958][00556] Avg episode reward: [(0, '20.289')] +[2024-09-05 09:07:01,972][02559] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000726_2973696.pth... +[2024-09-05 09:07:02,165][02559] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000503_2060288.pth +[2024-09-05 09:07:06,243][02572] Updated weights for policy 0, policy_version 730 (0.0016) +[2024-09-05 09:07:06,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3755.0, 300 sec: 3804.4). Total num frames: 2990080. Throughput: 0: 925.8. Samples: 745966. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:07:06,955][00556] Avg episode reward: [(0, '19.439')] +[2024-09-05 09:07:11,952][00556] Fps is (10 sec: 4096.5, 60 sec: 3891.2, 300 sec: 3804.4). Total num frames: 3014656. Throughput: 0: 954.9. Samples: 752902. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:07:11,958][00556] Avg episode reward: [(0, '20.353')] +[2024-09-05 09:07:15,630][02572] Updated weights for policy 0, policy_version 740 (0.0027) +[2024-09-05 09:07:16,952][00556] Fps is (10 sec: 4095.8, 60 sec: 3822.9, 300 sec: 3790.6). Total num frames: 3031040. Throughput: 0: 977.7. Samples: 758856. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:07:16,960][00556] Avg episode reward: [(0, '19.309')] +[2024-09-05 09:07:21,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3686.4, 300 sec: 3790.5). Total num frames: 3047424. Throughput: 0: 945.6. Samples: 760896. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-09-05 09:07:21,958][00556] Avg episode reward: [(0, '20.002')] +[2024-09-05 09:07:26,737][02572] Updated weights for policy 0, policy_version 750 (0.0017) +[2024-09-05 09:07:26,952][00556] Fps is (10 sec: 4096.2, 60 sec: 3891.6, 300 sec: 3818.3). Total num frames: 3072000. Throughput: 0: 930.9. Samples: 766982. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:07:26,956][00556] Avg episode reward: [(0, '20.515')] +[2024-09-05 09:07:31,952][00556] Fps is (10 sec: 4505.6, 60 sec: 3891.2, 300 sec: 3804.4). Total num frames: 3092480. Throughput: 0: 992.8. Samples: 773980. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:07:31,956][00556] Avg episode reward: [(0, '21.027')] +[2024-09-05 09:07:36,952][00556] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3804.4). Total num frames: 3108864. Throughput: 0: 972.6. Samples: 776056. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:07:36,955][00556] Avg episode reward: [(0, '21.848')] +[2024-09-05 09:07:36,962][02559] Saving new best policy, reward=21.848! +[2024-09-05 09:07:38,387][02572] Updated weights for policy 0, policy_version 760 (0.0040) +[2024-09-05 09:07:41,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3804.4). Total num frames: 3125248. Throughput: 0: 936.3. Samples: 781070. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-09-05 09:07:41,958][00556] Avg episode reward: [(0, '22.179')] +[2024-09-05 09:07:42,007][02559] Saving new best policy, reward=22.179! +[2024-09-05 09:07:46,952][00556] Fps is (10 sec: 4095.9, 60 sec: 3959.5, 300 sec: 3818.3). Total num frames: 3149824. Throughput: 0: 979.6. Samples: 787914. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:07:46,956][00556] Avg episode reward: [(0, '22.185')] +[2024-09-05 09:07:46,959][02559] Saving new best policy, reward=22.185! +[2024-09-05 09:07:47,535][02572] Updated weights for policy 0, policy_version 770 (0.0035) +[2024-09-05 09:07:51,952][00556] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3790.5). Total num frames: 3166208. Throughput: 0: 995.9. Samples: 790780. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:07:51,954][00556] Avg episode reward: [(0, '22.584')] +[2024-09-05 09:07:51,974][02559] Saving new best policy, reward=22.584! +[2024-09-05 09:07:56,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3804.4). Total num frames: 3182592. Throughput: 0: 933.0. Samples: 794888. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-09-05 09:07:56,957][00556] Avg episode reward: [(0, '23.748')] +[2024-09-05 09:07:56,963][02559] Saving new best policy, reward=23.748! +[2024-09-05 09:07:59,492][02572] Updated weights for policy 0, policy_version 780 (0.0047) +[2024-09-05 09:08:01,952][00556] Fps is (10 sec: 3686.4, 60 sec: 3823.0, 300 sec: 3804.4). Total num frames: 3203072. Throughput: 0: 946.5. Samples: 801446. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:08:01,958][00556] Avg episode reward: [(0, '23.655')] +[2024-09-05 09:08:06,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3762.8). Total num frames: 3215360. Throughput: 0: 945.8. Samples: 803458. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:08:06,961][00556] Avg episode reward: [(0, '23.804')] +[2024-09-05 09:08:06,963][02559] Saving new best policy, reward=23.804! +[2024-09-05 09:08:11,956][00556] Fps is (10 sec: 2456.5, 60 sec: 3549.6, 300 sec: 3748.8). Total num frames: 3227648. Throughput: 0: 889.5. Samples: 807012. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:08:11,959][00556] Avg episode reward: [(0, '23.020')] +[2024-09-05 09:08:13,993][02572] Updated weights for policy 0, policy_version 790 (0.0023) +[2024-09-05 09:08:16,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3618.2, 300 sec: 3762.8). Total num frames: 3248128. Throughput: 0: 853.1. Samples: 812370. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:08:16,957][00556] Avg episode reward: [(0, '23.317')] +[2024-09-05 09:08:21,952][00556] Fps is (10 sec: 4097.8, 60 sec: 3686.4, 300 sec: 3748.9). Total num frames: 3268608. Throughput: 0: 883.7. Samples: 815824. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:08:21,961][00556] Avg episode reward: [(0, '20.902')] +[2024-09-05 09:08:22,919][02572] Updated weights for policy 0, policy_version 800 (0.0040) +[2024-09-05 09:08:26,956][00556] Fps is (10 sec: 4094.2, 60 sec: 3617.9, 300 sec: 3748.8). Total num frames: 3289088. Throughput: 0: 914.2. Samples: 822212. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:08:26,958][00556] Avg episode reward: [(0, '19.727')] +[2024-09-05 09:08:31,952][00556] Fps is (10 sec: 3686.3, 60 sec: 3549.8, 300 sec: 3762.8). Total num frames: 3305472. Throughput: 0: 856.4. Samples: 826450. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-09-05 09:08:31,955][00556] Avg episode reward: [(0, '18.975')] +[2024-09-05 09:08:34,614][02572] Updated weights for policy 0, policy_version 810 (0.0056) +[2024-09-05 09:08:36,952][00556] Fps is (10 sec: 3688.0, 60 sec: 3618.1, 300 sec: 3762.8). Total num frames: 3325952. Throughput: 0: 870.2. Samples: 829938. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-09-05 09:08:36,960][00556] Avg episode reward: [(0, '19.685')] +[2024-09-05 09:08:41,952][00556] Fps is (10 sec: 4505.7, 60 sec: 3754.7, 300 sec: 3762.8). Total num frames: 3350528. Throughput: 0: 933.8. Samples: 836908. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-09-05 09:08:41,959][00556] Avg episode reward: [(0, '19.820')] +[2024-09-05 09:08:44,610][02572] Updated weights for policy 0, policy_version 820 (0.0023) +[2024-09-05 09:08:46,952][00556] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3748.9). Total num frames: 3362816. Throughput: 0: 892.7. Samples: 841618. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:08:46,955][00556] Avg episode reward: [(0, '19.537')] +[2024-09-05 09:08:51,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3762.8). Total num frames: 3383296. Throughput: 0: 907.2. Samples: 844282. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:08:51,960][00556] Avg episode reward: [(0, '20.711')] +[2024-09-05 09:08:54,929][02572] Updated weights for policy 0, policy_version 830 (0.0025) +[2024-09-05 09:08:56,952][00556] Fps is (10 sec: 4505.7, 60 sec: 3754.7, 300 sec: 3790.6). Total num frames: 3407872. Throughput: 0: 980.1. Samples: 851110. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:08:56,960][00556] Avg episode reward: [(0, '20.926')] +[2024-09-05 09:09:01,952][00556] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3790.5). Total num frames: 3424256. Throughput: 0: 984.7. Samples: 856680. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:09:01,958][00556] Avg episode reward: [(0, '19.977')] +[2024-09-05 09:09:01,968][02559] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000836_3424256.pth... +[2024-09-05 09:09:02,109][02559] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000614_2514944.pth +[2024-09-05 09:09:06,616][02572] Updated weights for policy 0, policy_version 840 (0.0041) +[2024-09-05 09:09:06,956][00556] Fps is (10 sec: 3275.3, 60 sec: 3754.4, 300 sec: 3804.4). Total num frames: 3440640. Throughput: 0: 953.8. Samples: 858748. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:09:06,959][00556] Avg episode reward: [(0, '20.138')] +[2024-09-05 09:09:11,952][00556] Fps is (10 sec: 4096.0, 60 sec: 3959.8, 300 sec: 3804.4). Total num frames: 3465216. Throughput: 0: 955.1. Samples: 865188. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:09:11,959][00556] Avg episode reward: [(0, '20.356')] +[2024-09-05 09:09:15,463][02572] Updated weights for policy 0, policy_version 850 (0.0034) +[2024-09-05 09:09:16,952][00556] Fps is (10 sec: 4507.5, 60 sec: 3959.5, 300 sec: 3804.4). Total num frames: 3485696. Throughput: 0: 1006.8. Samples: 871754. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:09:16,958][00556] Avg episode reward: [(0, '21.021')] +[2024-09-05 09:09:21,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3790.5). Total num frames: 3497984. Throughput: 0: 975.6. Samples: 873842. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-09-05 09:09:21,958][00556] Avg episode reward: [(0, '20.843')] +[2024-09-05 09:09:26,952][00556] Fps is (10 sec: 3276.7, 60 sec: 3823.2, 300 sec: 3804.4). Total num frames: 3518464. Throughput: 0: 939.9. Samples: 879202. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:09:26,955][00556] Avg episode reward: [(0, '20.252')] +[2024-09-05 09:09:27,019][02572] Updated weights for policy 0, policy_version 860 (0.0020) +[2024-09-05 09:09:31,952][00556] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 3804.4). Total num frames: 3543040. Throughput: 0: 994.0. Samples: 886348. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-09-05 09:09:31,955][00556] Avg episode reward: [(0, '19.490')] +[2024-09-05 09:09:36,953][00556] Fps is (10 sec: 4095.5, 60 sec: 3891.1, 300 sec: 3804.4). Total num frames: 3559424. Throughput: 0: 994.2. Samples: 889024. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-09-05 09:09:36,956][00556] Avg episode reward: [(0, '18.917')] +[2024-09-05 09:09:37,635][02572] Updated weights for policy 0, policy_version 870 (0.0038) +[2024-09-05 09:09:41,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3804.4). Total num frames: 3575808. Throughput: 0: 945.1. Samples: 893640. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-09-05 09:09:41,954][00556] Avg episode reward: [(0, '19.306')] +[2024-09-05 09:09:46,952][00556] Fps is (10 sec: 4096.6, 60 sec: 3959.5, 300 sec: 3804.4). Total num frames: 3600384. Throughput: 0: 977.9. Samples: 900686. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-09-05 09:09:46,954][00556] Avg episode reward: [(0, '19.264')] +[2024-09-05 09:09:47,171][02572] Updated weights for policy 0, policy_version 880 (0.0038) +[2024-09-05 09:09:51,952][00556] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 3804.5). Total num frames: 3620864. Throughput: 0: 1010.9. Samples: 904234. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-09-05 09:09:51,954][00556] Avg episode reward: [(0, '19.217')] +[2024-09-05 09:09:56,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3790.6). Total num frames: 3633152. Throughput: 0: 965.9. Samples: 908652. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-09-05 09:09:56,955][00556] Avg episode reward: [(0, '19.173')] +[2024-09-05 09:09:58,834][02572] Updated weights for policy 0, policy_version 890 (0.0041) +[2024-09-05 09:10:01,952][00556] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3804.4). Total num frames: 3657728. Throughput: 0: 958.0. Samples: 914864. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-09-05 09:10:01,955][00556] Avg episode reward: [(0, '19.644')] +[2024-09-05 09:10:06,952][00556] Fps is (10 sec: 4915.1, 60 sec: 4028.0, 300 sec: 3818.3). Total num frames: 3682304. Throughput: 0: 987.8. Samples: 918294. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-09-05 09:10:06,954][00556] Avg episode reward: [(0, '19.645')] +[2024-09-05 09:10:08,126][02572] Updated weights for policy 0, policy_version 900 (0.0031) +[2024-09-05 09:10:11,952][00556] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3790.5). Total num frames: 3694592. Throughput: 0: 990.4. Samples: 923768. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:10:11,954][00556] Avg episode reward: [(0, '19.657')] +[2024-09-05 09:10:16,952][00556] Fps is (10 sec: 3276.9, 60 sec: 3822.9, 300 sec: 3804.4). Total num frames: 3715072. Throughput: 0: 949.2. Samples: 929062. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-09-05 09:10:16,954][00556] Avg episode reward: [(0, '20.141')] +[2024-09-05 09:10:19,372][02572] Updated weights for policy 0, policy_version 910 (0.0025) +[2024-09-05 09:10:21,952][00556] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3804.4). Total num frames: 3735552. Throughput: 0: 966.2. Samples: 932502. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:10:21,955][00556] Avg episode reward: [(0, '21.187')] +[2024-09-05 09:10:26,952][00556] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3804.4). Total num frames: 3756032. Throughput: 0: 1003.2. Samples: 938784. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-09-05 09:10:26,956][00556] Avg episode reward: [(0, '21.733')] +[2024-09-05 09:10:30,726][02572] Updated weights for policy 0, policy_version 920 (0.0035) +[2024-09-05 09:10:31,952][00556] Fps is (10 sec: 3686.3, 60 sec: 3822.9, 300 sec: 3818.3). Total num frames: 3772416. Throughput: 0: 943.1. Samples: 943126. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-09-05 09:10:31,958][00556] Avg episode reward: [(0, '21.713')] +[2024-09-05 09:10:36,952][00556] Fps is (10 sec: 3686.4, 60 sec: 3891.3, 300 sec: 3804.4). Total num frames: 3792896. Throughput: 0: 942.7. Samples: 946656. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:10:36,959][00556] Avg episode reward: [(0, '21.005')] +[2024-09-05 09:10:39,900][02572] Updated weights for policy 0, policy_version 930 (0.0026) +[2024-09-05 09:10:41,952][00556] Fps is (10 sec: 4505.8, 60 sec: 4027.7, 300 sec: 3818.3). Total num frames: 3817472. Throughput: 0: 1000.1. Samples: 953658. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-09-05 09:10:41,958][00556] Avg episode reward: [(0, '21.026')] +[2024-09-05 09:10:46,952][00556] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3804.4). Total num frames: 3829760. Throughput: 0: 965.3. Samples: 958304. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:10:46,957][00556] Avg episode reward: [(0, '21.165')] +[2024-09-05 09:10:51,499][02572] Updated weights for policy 0, policy_version 940 (0.0032) +[2024-09-05 09:10:51,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3818.3). Total num frames: 3850240. Throughput: 0: 946.9. Samples: 960906. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:10:51,954][00556] Avg episode reward: [(0, '21.622')] +[2024-09-05 09:10:56,952][00556] Fps is (10 sec: 4505.5, 60 sec: 4027.7, 300 sec: 3818.3). Total num frames: 3874816. Throughput: 0: 976.9. Samples: 967728. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) +[2024-09-05 09:10:56,959][00556] Avg episode reward: [(0, '22.062')] +[2024-09-05 09:11:01,373][02572] Updated weights for policy 0, policy_version 950 (0.0028) +[2024-09-05 09:11:01,954][00556] Fps is (10 sec: 4095.2, 60 sec: 3891.1, 300 sec: 3818.4). Total num frames: 3891200. Throughput: 0: 983.8. Samples: 973334. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:11:01,956][00556] Avg episode reward: [(0, '22.559')] +[2024-09-05 09:11:01,974][02559] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000950_3891200.pth... +[2024-09-05 09:11:02,141][02559] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000726_2973696.pth +[2024-09-05 09:11:06,952][00556] Fps is (10 sec: 3276.9, 60 sec: 3754.7, 300 sec: 3818.3). Total num frames: 3907584. Throughput: 0: 953.9. Samples: 975428. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:11:06,960][00556] Avg episode reward: [(0, '22.057')] +[2024-09-05 09:11:11,872][02572] Updated weights for policy 0, policy_version 960 (0.0030) +[2024-09-05 09:11:11,952][00556] Fps is (10 sec: 4096.8, 60 sec: 3959.5, 300 sec: 3832.2). Total num frames: 3932160. Throughput: 0: 960.3. Samples: 981996. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-09-05 09:11:11,958][00556] Avg episode reward: [(0, '22.809')] +[2024-09-05 09:11:16,952][00556] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 3818.3). Total num frames: 3952640. Throughput: 0: 1010.2. Samples: 988584. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-09-05 09:11:16,964][00556] Avg episode reward: [(0, '22.974')] +[2024-09-05 09:11:21,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3818.4). Total num frames: 3964928. Throughput: 0: 979.0. Samples: 990712. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-09-05 09:11:21,959][00556] Avg episode reward: [(0, '23.266')] +[2024-09-05 09:11:23,649][02572] Updated weights for policy 0, policy_version 970 (0.0022) +[2024-09-05 09:11:26,952][00556] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3818.3). Total num frames: 3985408. Throughput: 0: 940.7. Samples: 995988. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-09-05 09:11:26,954][00556] Avg episode reward: [(0, '23.837')] +[2024-09-05 09:11:26,962][02559] Saving new best policy, reward=23.837! +[2024-09-05 09:11:30,899][02559] Stopping Batcher_0... +[2024-09-05 09:11:30,899][02559] Loop batcher_evt_loop terminating... +[2024-09-05 09:11:30,899][00556] Component Batcher_0 stopped! +[2024-09-05 09:11:30,901][02559] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... +[2024-09-05 09:11:30,964][02572] Weights refcount: 2 0 +[2024-09-05 09:11:30,970][02572] Stopping InferenceWorker_p0-w0... +[2024-09-05 09:11:30,970][00556] Component InferenceWorker_p0-w0 stopped! +[2024-09-05 09:11:30,971][02572] Loop inference_proc0-0_evt_loop terminating... +[2024-09-05 09:11:31,090][02559] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000836_3424256.pth +[2024-09-05 09:11:31,110][02559] Saving new best policy, reward=24.234! +[2024-09-05 09:11:31,274][02559] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... +[2024-09-05 09:11:31,289][00556] Component RolloutWorker_w6 stopped! +[2024-09-05 09:11:31,296][02578] Stopping RolloutWorker_w6... +[2024-09-05 09:11:31,297][02578] Loop rollout_proc6_evt_loop terminating... +[2024-09-05 09:11:31,307][00556] Component RolloutWorker_w4 stopped! +[2024-09-05 09:11:31,313][02577] Stopping RolloutWorker_w4... +[2024-09-05 09:11:31,315][02577] Loop rollout_proc4_evt_loop terminating... +[2024-09-05 09:11:31,321][00556] Component RolloutWorker_w0 stopped! +[2024-09-05 09:11:31,327][02573] Stopping RolloutWorker_w0... +[2024-09-05 09:11:31,329][02573] Loop rollout_proc0_evt_loop terminating... +[2024-09-05 09:11:31,336][00556] Component RolloutWorker_w2 stopped! +[2024-09-05 09:11:31,342][02575] Stopping RolloutWorker_w2... +[2024-09-05 09:11:31,344][02575] Loop rollout_proc2_evt_loop terminating... +[2024-09-05 09:11:31,407][02580] Stopping RolloutWorker_w7... +[2024-09-05 09:11:31,407][02580] Loop rollout_proc7_evt_loop terminating... +[2024-09-05 09:11:31,411][00556] Component RolloutWorker_w7 stopped! +[2024-09-05 09:11:31,444][02574] Stopping RolloutWorker_w1... +[2024-09-05 09:11:31,447][00556] Component RolloutWorker_w1 stopped! +[2024-09-05 09:11:31,445][02574] Loop rollout_proc1_evt_loop terminating... +[2024-09-05 09:11:31,470][02559] Stopping LearnerWorker_p0... +[2024-09-05 09:11:31,471][02559] Loop learner_proc0_evt_loop terminating... +[2024-09-05 09:11:31,473][00556] Component LearnerWorker_p0 stopped! +[2024-09-05 09:11:31,502][02579] Stopping RolloutWorker_w5... +[2024-09-05 09:11:31,502][00556] Component RolloutWorker_w5 stopped! +[2024-09-05 09:11:31,503][02579] Loop rollout_proc5_evt_loop terminating... +[2024-09-05 09:11:31,574][02576] Stopping RolloutWorker_w3... +[2024-09-05 09:11:31,574][00556] Component RolloutWorker_w3 stopped! +[2024-09-05 09:11:31,578][00556] Waiting for process learner_proc0 to stop... +[2024-09-05 09:11:31,579][02576] Loop rollout_proc3_evt_loop terminating... +[2024-09-05 09:11:33,034][00556] Waiting for process inference_proc0-0 to join... +[2024-09-05 09:11:33,039][00556] Waiting for process rollout_proc0 to join... +[2024-09-05 09:11:35,627][00556] Waiting for process rollout_proc1 to join... +[2024-09-05 09:11:35,871][00556] Waiting for process rollout_proc2 to join... +[2024-09-05 09:11:35,874][00556] Waiting for process rollout_proc3 to join... +[2024-09-05 09:11:35,879][00556] Waiting for process rollout_proc4 to join... +[2024-09-05 09:11:35,881][00556] Waiting for process rollout_proc5 to join... +[2024-09-05 09:11:35,882][00556] Waiting for process rollout_proc6 to join... +[2024-09-05 09:11:35,883][00556] Waiting for process rollout_proc7 to join... +[2024-09-05 09:11:35,886][00556] Batcher 0 profile tree view: +batching: 28.7009, releasing_batches: 0.0292 +[2024-09-05 09:11:35,889][00556] InferenceWorker_p0-w0 profile tree view: +wait_policy: 0.0001 + wait_policy_total: 386.3923 +update_model: 9.3406 + weight_update: 0.0025 +one_step: 0.0123 + handle_policy_step: 634.4478 + deserialize: 15.2177, stack: 3.3282, obs_to_device_normalize: 129.3269, forward: 336.0333, send_messages: 30.4733 + prepare_outputs: 88.4075 + to_cpu: 51.2483 +[2024-09-05 09:11:35,890][00556] Learner 0 profile tree view: +misc: 0.0075, prepare_batch: 14.5544 +train: 75.3765 + epoch_init: 0.0110, minibatch_init: 0.0067, losses_postprocess: 0.6628, kl_divergence: 0.7560, after_optimizer: 33.5400 + calculate_losses: 27.0137 + losses_init: 0.0072, forward_head: 1.3314, bptt_initial: 18.0916, tail: 1.1112, advantages_returns: 0.2629, losses: 3.7827 + bptt: 2.1088 + bptt_forward_core: 2.0185 + update: 12.7220 + clip: 0.9137 +[2024-09-05 09:11:35,892][00556] RolloutWorker_w0 profile tree view: +wait_for_trajectories: 0.3611, enqueue_policy_requests: 93.7444, env_step: 835.0048, overhead: 13.9005, complete_rollouts: 7.6595 +save_policy_outputs: 21.8211 + split_output_tensors: 8.8036 +[2024-09-05 09:11:35,893][00556] RolloutWorker_w7 profile tree view: +wait_for_trajectories: 0.3465, enqueue_policy_requests: 96.0715, env_step: 834.0358, overhead: 13.6210, complete_rollouts: 6.8607 +save_policy_outputs: 20.8596 + split_output_tensors: 8.4140 +[2024-09-05 09:11:35,896][00556] Loop Runner_EvtLoop terminating... +[2024-09-05 09:11:35,898][00556] Runner profile tree view: +main_loop: 1104.8237 +[2024-09-05 09:11:35,899][00556] Collected {0: 4005888}, FPS: 3625.8 +[2024-09-05 09:11:36,254][00556] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json +[2024-09-05 09:11:36,255][00556] Overriding arg 'num_workers' with value 1 passed from command line +[2024-09-05 09:11:36,259][00556] Adding new argument 'no_render'=True that is not in the saved config file! +[2024-09-05 09:11:36,261][00556] Adding new argument 'save_video'=True that is not in the saved config file! +[2024-09-05 09:11:36,263][00556] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! +[2024-09-05 09:11:36,265][00556] Adding new argument 'video_name'=None that is not in the saved config file! +[2024-09-05 09:11:36,267][00556] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! +[2024-09-05 09:11:36,269][00556] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! +[2024-09-05 09:11:36,270][00556] Adding new argument 'push_to_hub'=False that is not in the saved config file! +[2024-09-05 09:11:36,271][00556] Adding new argument 'hf_repository'=None that is not in the saved config file! +[2024-09-05 09:11:36,273][00556] Adding new argument 'policy_index'=0 that is not in the saved config file! +[2024-09-05 09:11:36,274][00556] Adding new argument 'eval_deterministic'=False that is not in the saved config file! +[2024-09-05 09:11:36,275][00556] Adding new argument 'train_script'=None that is not in the saved config file! +[2024-09-05 09:11:36,277][00556] Adding new argument 'enjoy_script'=None that is not in the saved config file! +[2024-09-05 09:11:36,278][00556] Using frameskip 1 and render_action_repeat=4 for evaluation +[2024-09-05 09:11:36,318][00556] Doom resolution: 160x120, resize resolution: (128, 72) +[2024-09-05 09:11:36,322][00556] RunningMeanStd input shape: (3, 72, 128) +[2024-09-05 09:11:36,325][00556] RunningMeanStd input shape: (1,) +[2024-09-05 09:11:36,341][00556] ConvEncoder: input_channels=3 +[2024-09-05 09:11:36,456][00556] Conv encoder output size: 512 +[2024-09-05 09:11:36,459][00556] Policy head output size: 512 +[2024-09-05 09:11:36,639][00556] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... +[2024-09-05 09:11:37,498][00556] Num frames 100... +[2024-09-05 09:11:37,626][00556] Num frames 200... +[2024-09-05 09:11:37,751][00556] Num frames 300... +[2024-09-05 09:11:37,873][00556] Num frames 400... +[2024-09-05 09:11:37,996][00556] Num frames 500... +[2024-09-05 09:11:38,117][00556] Num frames 600... +[2024-09-05 09:11:38,242][00556] Num frames 700... +[2024-09-05 09:11:38,367][00556] Num frames 800... +[2024-09-05 09:11:38,501][00556] Num frames 900... +[2024-09-05 09:11:38,625][00556] Num frames 1000... +[2024-09-05 09:11:38,713][00556] Avg episode rewards: #0: 24.240, true rewards: #0: 10.240 +[2024-09-05 09:11:38,715][00556] Avg episode reward: 24.240, avg true_objective: 10.240 +[2024-09-05 09:11:38,806][00556] Num frames 1100... +[2024-09-05 09:11:38,928][00556] Num frames 1200... +[2024-09-05 09:11:39,054][00556] Num frames 1300... +[2024-09-05 09:11:39,175][00556] Num frames 1400... +[2024-09-05 09:11:39,293][00556] Num frames 1500... +[2024-09-05 09:11:39,414][00556] Num frames 1600... +[2024-09-05 09:11:39,508][00556] Avg episode rewards: #0: 17.660, true rewards: #0: 8.160 +[2024-09-05 09:11:39,509][00556] Avg episode reward: 17.660, avg true_objective: 8.160 +[2024-09-05 09:11:39,600][00556] Num frames 1700... +[2024-09-05 09:11:39,731][00556] Num frames 1800... +[2024-09-05 09:11:39,852][00556] Num frames 1900... +[2024-09-05 09:11:39,976][00556] Num frames 2000... +[2024-09-05 09:11:40,103][00556] Num frames 2100... +[2024-09-05 09:11:40,226][00556] Num frames 2200... +[2024-09-05 09:11:40,352][00556] Num frames 2300... +[2024-09-05 09:11:40,475][00556] Num frames 2400... +[2024-09-05 09:11:40,609][00556] Num frames 2500... +[2024-09-05 09:11:40,739][00556] Num frames 2600... +[2024-09-05 09:11:40,861][00556] Num frames 2700... +[2024-09-05 09:11:40,984][00556] Num frames 2800... +[2024-09-05 09:11:41,105][00556] Num frames 2900... +[2024-09-05 09:11:41,227][00556] Num frames 3000... +[2024-09-05 09:11:41,348][00556] Num frames 3100... +[2024-09-05 09:11:41,471][00556] Num frames 3200... +[2024-09-05 09:11:41,610][00556] Avg episode rewards: #0: 24.547, true rewards: #0: 10.880 +[2024-09-05 09:11:41,612][00556] Avg episode reward: 24.547, avg true_objective: 10.880 +[2024-09-05 09:11:41,663][00556] Num frames 3300... +[2024-09-05 09:11:41,792][00556] Num frames 3400... +[2024-09-05 09:11:41,913][00556] Num frames 3500... +[2024-09-05 09:11:42,037][00556] Num frames 3600... +[2024-09-05 09:11:42,171][00556] Num frames 3700... +[2024-09-05 09:11:42,294][00556] Num frames 3800... +[2024-09-05 09:11:42,419][00556] Num frames 3900... +[2024-09-05 09:11:42,541][00556] Num frames 4000... +[2024-09-05 09:11:42,678][00556] Num frames 4100... +[2024-09-05 09:11:42,800][00556] Num frames 4200... +[2024-09-05 09:11:42,921][00556] Num frames 4300... +[2024-09-05 09:11:43,046][00556] Num frames 4400... +[2024-09-05 09:11:43,171][00556] Num frames 4500... +[2024-09-05 09:11:43,294][00556] Num frames 4600... +[2024-09-05 09:11:43,419][00556] Num frames 4700... +[2024-09-05 09:11:43,547][00556] Num frames 4800... +[2024-09-05 09:11:43,686][00556] Num frames 4900... +[2024-09-05 09:11:43,808][00556] Num frames 5000... +[2024-09-05 09:11:43,931][00556] Num frames 5100... +[2024-09-05 09:11:44,053][00556] Num frames 5200... +[2024-09-05 09:11:44,128][00556] Avg episode rewards: #0: 32.040, true rewards: #0: 13.040 +[2024-09-05 09:11:44,130][00556] Avg episode reward: 32.040, avg true_objective: 13.040 +[2024-09-05 09:11:44,232][00556] Num frames 5300... +[2024-09-05 09:11:44,357][00556] Num frames 5400... +[2024-09-05 09:11:44,483][00556] Num frames 5500... +[2024-09-05 09:11:44,610][00556] Num frames 5600... +[2024-09-05 09:11:44,749][00556] Num frames 5700... +[2024-09-05 09:11:44,886][00556] Avg episode rewards: #0: 28.336, true rewards: #0: 11.536 +[2024-09-05 09:11:44,887][00556] Avg episode reward: 28.336, avg true_objective: 11.536 +[2024-09-05 09:11:44,929][00556] Num frames 5800... +[2024-09-05 09:11:45,050][00556] Num frames 5900... +[2024-09-05 09:11:45,175][00556] Num frames 6000... +[2024-09-05 09:11:45,296][00556] Num frames 6100... +[2024-09-05 09:11:45,417][00556] Num frames 6200... +[2024-09-05 09:11:45,538][00556] Num frames 6300... +[2024-09-05 09:11:45,674][00556] Num frames 6400... +[2024-09-05 09:11:45,826][00556] Avg episode rewards: #0: 25.953, true rewards: #0: 10.787 +[2024-09-05 09:11:45,827][00556] Avg episode reward: 25.953, avg true_objective: 10.787 +[2024-09-05 09:11:45,864][00556] Num frames 6500... +[2024-09-05 09:11:45,987][00556] Num frames 6600... +[2024-09-05 09:11:46,107][00556] Num frames 6700... +[2024-09-05 09:11:46,254][00556] Num frames 6800... +[2024-09-05 09:11:46,434][00556] Num frames 6900... +[2024-09-05 09:11:46,613][00556] Num frames 7000... +[2024-09-05 09:11:46,791][00556] Num frames 7100... +[2024-09-05 09:11:46,966][00556] Num frames 7200... +[2024-09-05 09:11:47,137][00556] Num frames 7300... +[2024-09-05 09:11:47,303][00556] Num frames 7400... +[2024-09-05 09:11:47,517][00556] Avg episode rewards: #0: 25.566, true rewards: #0: 10.709 +[2024-09-05 09:11:47,518][00556] Avg episode reward: 25.566, avg true_objective: 10.709 +[2024-09-05 09:11:47,528][00556] Num frames 7500... +[2024-09-05 09:11:47,704][00556] Num frames 7600... +[2024-09-05 09:11:47,884][00556] Num frames 7700... +[2024-09-05 09:11:48,061][00556] Num frames 7800... +[2024-09-05 09:11:48,239][00556] Num frames 7900... +[2024-09-05 09:11:48,417][00556] Num frames 8000... +[2024-09-05 09:11:48,602][00556] Num frames 8100... +[2024-09-05 09:11:48,732][00556] Num frames 8200... +[2024-09-05 09:11:48,866][00556] Num frames 8300... +[2024-09-05 09:11:48,993][00556] Num frames 8400... +[2024-09-05 09:11:49,117][00556] Num frames 8500... +[2024-09-05 09:11:49,242][00556] Num frames 8600... +[2024-09-05 09:11:49,366][00556] Num frames 8700... +[2024-09-05 09:11:49,493][00556] Num frames 8800... +[2024-09-05 09:11:49,621][00556] Num frames 8900... +[2024-09-05 09:11:49,752][00556] Num frames 9000... +[2024-09-05 09:11:49,884][00556] Num frames 9100... +[2024-09-05 09:11:50,011][00556] Num frames 9200... +[2024-09-05 09:11:50,133][00556] Num frames 9300... +[2024-09-05 09:11:50,256][00556] Num frames 9400... +[2024-09-05 09:11:50,413][00556] Avg episode rewards: #0: 28.600, true rewards: #0: 11.850 +[2024-09-05 09:11:50,414][00556] Avg episode reward: 28.600, avg true_objective: 11.850 +[2024-09-05 09:11:50,442][00556] Num frames 9500... +[2024-09-05 09:11:50,568][00556] Num frames 9600... +[2024-09-05 09:11:50,699][00556] Num frames 9700... +[2024-09-05 09:11:50,822][00556] Num frames 9800... +[2024-09-05 09:11:50,953][00556] Num frames 9900... +[2024-09-05 09:11:51,079][00556] Num frames 10000... +[2024-09-05 09:11:51,202][00556] Num frames 10100... +[2024-09-05 09:11:51,324][00556] Num frames 10200... +[2024-09-05 09:11:51,448][00556] Num frames 10300... +[2024-09-05 09:11:51,572][00556] Num frames 10400... +[2024-09-05 09:11:51,710][00556] Num frames 10500... +[2024-09-05 09:11:51,834][00556] Num frames 10600... +[2024-09-05 09:11:51,969][00556] Num frames 10700... +[2024-09-05 09:11:52,099][00556] Num frames 10800... +[2024-09-05 09:11:52,228][00556] Num frames 10900... +[2024-09-05 09:11:52,357][00556] Num frames 11000... +[2024-09-05 09:11:52,482][00556] Num frames 11100... +[2024-09-05 09:11:52,607][00556] Num frames 11200... +[2024-09-05 09:11:52,740][00556] Num frames 11300... +[2024-09-05 09:11:52,889][00556] Num frames 11400... +[2024-09-05 09:11:53,043][00556] Num frames 11500... +[2024-09-05 09:11:53,197][00556] Avg episode rewards: #0: 31.755, true rewards: #0: 12.867 +[2024-09-05 09:11:53,198][00556] Avg episode reward: 31.755, avg true_objective: 12.867 +[2024-09-05 09:11:53,226][00556] Num frames 11600... +[2024-09-05 09:11:53,348][00556] Num frames 11700... +[2024-09-05 09:11:53,474][00556] Num frames 11800... +[2024-09-05 09:11:53,599][00556] Num frames 11900... +[2024-09-05 09:11:53,737][00556] Num frames 12000... +[2024-09-05 09:11:53,858][00556] Num frames 12100... +[2024-09-05 09:11:54,029][00556] Avg episode rewards: #0: 29.488, true rewards: #0: 12.188 +[2024-09-05 09:11:54,030][00556] Avg episode reward: 29.488, avg true_objective: 12.188 +[2024-09-05 09:13:13,015][00556] Replay video saved to /content/train_dir/default_experiment/replay.mp4! +[2024-09-05 09:18:45,792][00556] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json +[2024-09-05 09:18:45,794][00556] Overriding arg 'num_workers' with value 1 passed from command line +[2024-09-05 09:18:45,796][00556] Adding new argument 'no_render'=True that is not in the saved config file! +[2024-09-05 09:18:45,798][00556] Adding new argument 'save_video'=True that is not in the saved config file! +[2024-09-05 09:18:45,800][00556] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! +[2024-09-05 09:18:45,802][00556] Adding new argument 'video_name'=None that is not in the saved config file! +[2024-09-05 09:18:45,803][00556] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! +[2024-09-05 09:18:45,805][00556] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! +[2024-09-05 09:18:45,806][00556] Adding new argument 'push_to_hub'=True that is not in the saved config file! +[2024-09-05 09:18:45,807][00556] Adding new argument 'hf_repository'='neeldevenshah/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! +[2024-09-05 09:18:45,808][00556] Adding new argument 'policy_index'=0 that is not in the saved config file! +[2024-09-05 09:18:45,809][00556] Adding new argument 'eval_deterministic'=False that is not in the saved config file! +[2024-09-05 09:18:45,810][00556] Adding new argument 'train_script'=None that is not in the saved config file! +[2024-09-05 09:18:45,811][00556] Adding new argument 'enjoy_script'=None that is not in the saved config file! +[2024-09-05 09:18:45,812][00556] Using frameskip 1 and render_action_repeat=4 for evaluation +[2024-09-05 09:18:45,842][00556] RunningMeanStd input shape: (3, 72, 128) +[2024-09-05 09:18:45,843][00556] RunningMeanStd input shape: (1,) +[2024-09-05 09:18:45,856][00556] ConvEncoder: input_channels=3 +[2024-09-05 09:18:45,894][00556] Conv encoder output size: 512 +[2024-09-05 09:18:45,895][00556] Policy head output size: 512 +[2024-09-05 09:18:45,914][00556] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... +[2024-09-05 09:18:46,357][00556] Num frames 100... +[2024-09-05 09:18:46,482][00556] Num frames 200... +[2024-09-05 09:18:46,603][00556] Num frames 300... +[2024-09-05 09:18:46,726][00556] Avg episode rewards: #0: 4.520, true rewards: #0: 3.520 +[2024-09-05 09:18:46,728][00556] Avg episode reward: 4.520, avg true_objective: 3.520 +[2024-09-05 09:18:46,799][00556] Num frames 400... +[2024-09-05 09:18:46,926][00556] Num frames 500... +[2024-09-05 09:18:47,066][00556] Num frames 600... +[2024-09-05 09:18:47,188][00556] Num frames 700... +[2024-09-05 09:18:47,311][00556] Num frames 800... +[2024-09-05 09:18:47,440][00556] Num frames 900... +[2024-09-05 09:18:47,568][00556] Num frames 1000... +[2024-09-05 09:18:47,696][00556] Num frames 1100... +[2024-09-05 09:18:47,818][00556] Num frames 1200... +[2024-09-05 09:18:47,941][00556] Num frames 1300... +[2024-09-05 09:18:48,081][00556] Num frames 1400... +[2024-09-05 09:18:48,203][00556] Num frames 1500... +[2024-09-05 09:18:48,326][00556] Num frames 1600... +[2024-09-05 09:18:48,459][00556] Num frames 1700... +[2024-09-05 09:18:48,585][00556] Num frames 1800... +[2024-09-05 09:18:48,718][00556] Num frames 1900... +[2024-09-05 09:18:48,798][00556] Avg episode rewards: #0: 24.600, true rewards: #0: 9.600 +[2024-09-05 09:18:48,799][00556] Avg episode reward: 24.600, avg true_objective: 9.600 +[2024-09-05 09:18:48,903][00556] Num frames 2000... +[2024-09-05 09:18:49,027][00556] Num frames 2100... +[2024-09-05 09:18:49,154][00556] Num frames 2200... +[2024-09-05 09:18:49,286][00556] Num frames 2300... +[2024-09-05 09:18:49,412][00556] Num frames 2400... +[2024-09-05 09:18:49,538][00556] Num frames 2500... +[2024-09-05 09:18:49,662][00556] Num frames 2600... +[2024-09-05 09:18:49,824][00556] Num frames 2700... +[2024-09-05 09:18:50,004][00556] Num frames 2800... +[2024-09-05 09:18:50,186][00556] Num frames 2900... +[2024-09-05 09:18:50,360][00556] Num frames 3000... +[2024-09-05 09:18:50,536][00556] Num frames 3100... +[2024-09-05 09:18:50,706][00556] Num frames 3200... +[2024-09-05 09:18:50,876][00556] Num frames 3300... +[2024-09-05 09:18:51,053][00556] Num frames 3400... +[2024-09-05 09:18:51,237][00556] Num frames 3500... +[2024-09-05 09:18:51,410][00556] Num frames 3600... +[2024-09-05 09:18:51,593][00556] Num frames 3700... +[2024-09-05 09:18:51,783][00556] Num frames 3800... +[2024-09-05 09:18:51,964][00556] Num frames 3900... +[2024-09-05 09:18:52,044][00556] Avg episode rewards: #0: 33.373, true rewards: #0: 13.040 +[2024-09-05 09:18:52,046][00556] Avg episode reward: 33.373, avg true_objective: 13.040 +[2024-09-05 09:18:52,231][00556] Num frames 4000... +[2024-09-05 09:18:52,385][00556] Num frames 4100... +[2024-09-05 09:18:52,509][00556] Num frames 4200... +[2024-09-05 09:18:52,663][00556] Num frames 4300... +[2024-09-05 09:18:52,802][00556] Num frames 4400... +[2024-09-05 09:18:52,925][00556] Num frames 4500... +[2024-09-05 09:18:53,052][00556] Num frames 4600... +[2024-09-05 09:18:53,211][00556] Avg episode rewards: #0: 28.700, true rewards: #0: 11.700 +[2024-09-05 09:18:53,212][00556] Avg episode reward: 28.700, avg true_objective: 11.700 +[2024-09-05 09:18:53,243][00556] Num frames 4700... +[2024-09-05 09:18:53,395][00556] Num frames 4800... +[2024-09-05 09:18:53,519][00556] Num frames 4900... +[2024-09-05 09:18:53,647][00556] Num frames 5000... +[2024-09-05 09:18:53,784][00556] Num frames 5100... +[2024-09-05 09:18:53,905][00556] Num frames 5200... +[2024-09-05 09:18:54,033][00556] Num frames 5300... +[2024-09-05 09:18:54,189][00556] Avg episode rewards: #0: 25.168, true rewards: #0: 10.768 +[2024-09-05 09:18:54,190][00556] Avg episode reward: 25.168, avg true_objective: 10.768 +[2024-09-05 09:18:54,214][00556] Num frames 5400... +[2024-09-05 09:18:54,347][00556] Num frames 5500... +[2024-09-05 09:18:54,473][00556] Num frames 5600... +[2024-09-05 09:18:54,599][00556] Num frames 5700... +[2024-09-05 09:18:54,728][00556] Num frames 5800... +[2024-09-05 09:18:54,851][00556] Num frames 5900... +[2024-09-05 09:18:54,975][00556] Num frames 6000... +[2024-09-05 09:18:55,102][00556] Avg episode rewards: #0: 23.426, true rewards: #0: 10.093 +[2024-09-05 09:18:55,104][00556] Avg episode reward: 23.426, avg true_objective: 10.093 +[2024-09-05 09:18:55,161][00556] Num frames 6100... +[2024-09-05 09:18:55,289][00556] Num frames 6200... +[2024-09-05 09:18:55,432][00556] Num frames 6300... +[2024-09-05 09:18:55,559][00556] Num frames 6400... +[2024-09-05 09:18:55,665][00556] Avg episode rewards: #0: 21.343, true rewards: #0: 9.200 +[2024-09-05 09:18:55,667][00556] Avg episode reward: 21.343, avg true_objective: 9.200 +[2024-09-05 09:18:55,745][00556] Num frames 6500... +[2024-09-05 09:18:55,868][00556] Num frames 6600... +[2024-09-05 09:18:55,996][00556] Num frames 6700... +[2024-09-05 09:18:56,127][00556] Num frames 6800... +[2024-09-05 09:18:56,287][00556] Num frames 6900... +[2024-09-05 09:18:56,423][00556] Num frames 7000... +[2024-09-05 09:18:56,551][00556] Num frames 7100... +[2024-09-05 09:18:56,690][00556] Num frames 7200... +[2024-09-05 09:18:56,816][00556] Num frames 7300... +[2024-09-05 09:18:56,914][00556] Avg episode rewards: #0: 20.670, true rewards: #0: 9.170 +[2024-09-05 09:18:56,915][00556] Avg episode reward: 20.670, avg true_objective: 9.170 +[2024-09-05 09:18:56,999][00556] Num frames 7400... +[2024-09-05 09:18:57,126][00556] Num frames 7500... +[2024-09-05 09:18:57,251][00556] Num frames 7600... +[2024-09-05 09:18:57,385][00556] Num frames 7700... +[2024-09-05 09:18:57,513][00556] Num frames 7800... +[2024-09-05 09:18:57,642][00556] Num frames 7900... +[2024-09-05 09:18:57,783][00556] Num frames 8000... +[2024-09-05 09:18:57,903][00556] Num frames 8100... +[2024-09-05 09:18:58,027][00556] Num frames 8200... +[2024-09-05 09:18:58,150][00556] Num frames 8300... +[2024-09-05 09:18:58,277][00556] Num frames 8400... +[2024-09-05 09:18:58,363][00556] Avg episode rewards: #0: 20.693, true rewards: #0: 9.360 +[2024-09-05 09:18:58,366][00556] Avg episode reward: 20.693, avg true_objective: 9.360 +[2024-09-05 09:18:58,477][00556] Num frames 8500... +[2024-09-05 09:18:58,604][00556] Num frames 8600... +[2024-09-05 09:18:58,734][00556] Num frames 8700... +[2024-09-05 09:18:58,859][00556] Num frames 8800... +[2024-09-05 09:18:58,982][00556] Num frames 8900... +[2024-09-05 09:18:59,106][00556] Num frames 9000... +[2024-09-05 09:18:59,172][00556] Avg episode rewards: #0: 19.908, true rewards: #0: 9.008 +[2024-09-05 09:18:59,173][00556] Avg episode reward: 19.908, avg true_objective: 9.008 +[2024-09-05 09:19:58,814][00556] Replay video saved to /content/train_dir/default_experiment/replay.mp4!