diff --git "a/sf_log.txt" "b/sf_log.txt" --- "a/sf_log.txt" +++ "b/sf_log.txt" @@ -1,44 +1,50 @@ -[2025-05-25 11:09:42,915][00035] Saving configuration to /kaggle/working/train_dir/default_experiment/config.json... -[2025-05-25 11:09:42,917][00035] Rollout worker 0 uses device cpu -[2025-05-25 11:09:42,917][00035] Rollout worker 1 uses device cpu -[2025-05-25 11:09:42,918][00035] Rollout worker 2 uses device cpu -[2025-05-25 11:09:42,919][00035] Rollout worker 3 uses device cpu -[2025-05-25 11:09:42,920][00035] Rollout worker 4 uses device cpu -[2025-05-25 11:09:42,921][00035] Rollout worker 5 uses device cpu -[2025-05-25 11:09:42,921][00035] Rollout worker 6 uses device cpu -[2025-05-25 11:09:42,922][00035] Rollout worker 7 uses device cpu -[2025-05-25 11:09:43,047][00035] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2025-05-25 11:09:43,048][00035] InferenceWorker_p0-w0: min num requests: 2 -[2025-05-25 11:09:43,092][00035] Starting all processes... -[2025-05-25 11:09:43,092][00035] Starting process learner_proc0 -[2025-05-25 11:09:43,184][00035] Starting all processes... -[2025-05-25 11:09:43,195][00035] Starting process inference_proc0-0 -[2025-05-25 11:09:43,195][00035] Starting process rollout_proc0 -[2025-05-25 11:09:43,195][00035] Starting process rollout_proc1 -[2025-05-25 11:09:43,196][00035] Starting process rollout_proc2 -[2025-05-25 11:09:43,197][00035] Starting process rollout_proc3 -[2025-05-25 11:09:43,197][00035] Starting process rollout_proc4 -[2025-05-25 11:09:43,197][00035] Starting process rollout_proc5 -[2025-05-25 11:09:43,198][00035] Starting process rollout_proc6 -[2025-05-25 11:09:43,199][00035] Starting process rollout_proc7 -[2025-05-25 11:09:50,891][01047] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2025-05-25 11:09:50,891][01047] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 -[2025-05-25 11:09:50,957][01047] Num visible devices: 1 -[2025-05-25 11:09:50,964][01047] Starting seed is not provided -[2025-05-25 11:09:50,965][01047] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2025-05-25 11:09:50,965][01047] Initializing actor-critic model on device cuda:0 -[2025-05-25 11:09:50,966][01047] RunningMeanStd input shape: (3, 72, 128) -[2025-05-25 11:09:50,972][01047] RunningMeanStd input shape: (1,) -[2025-05-25 11:09:51,032][01047] ConvEncoder: input_channels=3 -[2025-05-25 11:09:51,371][01061] Worker 2 uses CPU cores [2] -[2025-05-25 11:09:51,651][01060] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2025-05-25 11:09:51,652][01060] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 -[2025-05-25 11:09:51,669][01047] Conv encoder output size: 512 -[2025-05-25 11:09:51,670][01047] Policy head output size: 512 -[2025-05-25 11:09:51,723][01060] Num visible devices: 1 -[2025-05-25 11:09:51,826][01068] Worker 6 uses CPU cores [2] -[2025-05-25 11:09:51,837][01047] Created Actor Critic model with architecture: -[2025-05-25 11:09:51,837][01047] ActorCriticSharedWeights( +[2025-05-25 11:42:26,634][04028] Saving configuration to /content/train_dir/default_experiment/config.json... +[2025-05-25 11:42:26,637][04028] Rollout worker 0 uses device cpu +[2025-05-25 11:42:26,638][04028] Rollout worker 1 uses device cpu +[2025-05-25 11:42:26,639][04028] Rollout worker 2 uses device cpu +[2025-05-25 11:42:26,640][04028] Rollout worker 3 uses device cpu +[2025-05-25 11:42:26,641][04028] Rollout worker 4 uses device cpu +[2025-05-25 11:42:26,642][04028] Rollout worker 5 uses device cpu +[2025-05-25 11:42:26,642][04028] Rollout worker 6 uses device cpu +[2025-05-25 11:42:26,643][04028] Rollout worker 7 uses device cpu +[2025-05-25 11:42:26,779][04028] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2025-05-25 11:42:26,780][04028] InferenceWorker_p0-w0: min num requests: 2 +[2025-05-25 11:42:26,810][04028] Starting all processes... +[2025-05-25 11:42:26,811][04028] Starting process learner_proc0 +[2025-05-25 11:42:26,864][04028] Starting all processes... +[2025-05-25 11:42:26,872][04028] Starting process inference_proc0-0 +[2025-05-25 11:42:26,873][04028] Starting process rollout_proc0 +[2025-05-25 11:42:26,873][04028] Starting process rollout_proc1 +[2025-05-25 11:42:26,873][04028] Starting process rollout_proc2 +[2025-05-25 11:42:26,873][04028] Starting process rollout_proc3 +[2025-05-25 11:42:26,873][04028] Starting process rollout_proc4 +[2025-05-25 11:42:26,873][04028] Starting process rollout_proc5 +[2025-05-25 11:42:26,873][04028] Starting process rollout_proc6 +[2025-05-25 11:42:26,873][04028] Starting process rollout_proc7 +[2025-05-25 11:42:43,569][04219] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2025-05-25 11:42:43,576][04219] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 +[2025-05-25 11:42:43,703][04219] Num visible devices: 1 +[2025-05-25 11:42:43,718][04219] Starting seed is not provided +[2025-05-25 11:42:43,719][04219] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2025-05-25 11:42:43,720][04219] Initializing actor-critic model on device cuda:0 +[2025-05-25 11:42:43,721][04219] RunningMeanStd input shape: (3, 72, 128) +[2025-05-25 11:42:43,739][04219] RunningMeanStd input shape: (1,) +[2025-05-25 11:42:43,887][04240] Worker 5 uses CPU cores [1] +[2025-05-25 11:42:43,916][04219] ConvEncoder: input_channels=3 +[2025-05-25 11:42:44,231][04235] Worker 2 uses CPU cores [0] +[2025-05-25 11:42:44,602][04232] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2025-05-25 11:42:44,606][04232] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 +[2025-05-25 11:42:44,634][04234] Worker 1 uses CPU cores [1] +[2025-05-25 11:42:44,698][04232] Num visible devices: 1 +[2025-05-25 11:42:44,739][04236] Worker 3 uses CPU cores [1] +[2025-05-25 11:42:44,830][04237] Worker 6 uses CPU cores [0] +[2025-05-25 11:42:44,847][04239] Worker 4 uses CPU cores [0] +[2025-05-25 11:42:44,877][04238] Worker 7 uses CPU cores [1] +[2025-05-25 11:42:44,881][04233] Worker 0 uses CPU cores [0] +[2025-05-25 11:42:44,918][04219] Conv encoder output size: 512 +[2025-05-25 11:42:44,919][04219] Policy head output size: 512 +[2025-05-25 11:42:44,982][04219] Created Actor Critic model with architecture: +[2025-05-25 11:42:44,982][04219] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( @@ -79,608 +85,992 @@ (distribution_linear): Linear(in_features=512, out_features=5, bias=True) ) ) -[2025-05-25 11:09:51,839][01067] Worker 7 uses CPU cores [3] -[2025-05-25 11:09:51,850][01066] Worker 5 uses CPU cores [1] -[2025-05-25 11:09:51,850][01063] Worker 0 uses CPU cores [0] -[2025-05-25 11:09:51,868][01064] Worker 4 uses CPU cores [0] -[2025-05-25 11:09:51,926][01065] Worker 3 uses CPU cores [3] -[2025-05-25 11:09:51,948][01062] Worker 1 uses CPU cores [1] -[2025-05-25 11:09:52,115][01047] Using optimizer -[2025-05-25 11:09:54,770][01047] No checkpoints found -[2025-05-25 11:09:54,770][01047] Did not load from checkpoint, starting from scratch! -[2025-05-25 11:09:54,770][01047] Initialized policy 0 weights for model version 0 -[2025-05-25 11:09:54,772][01047] LearnerWorker_p0 finished initialization! -[2025-05-25 11:09:54,773][01047] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2025-05-25 11:09:54,891][01060] RunningMeanStd input shape: (3, 72, 128) -[2025-05-25 11:09:54,892][01060] RunningMeanStd input shape: (1,) -[2025-05-25 11:09:54,903][01060] ConvEncoder: input_channels=3 -[2025-05-25 11:09:55,016][01060] Conv encoder output size: 512 -[2025-05-25 11:09:55,017][01060] Policy head output size: 512 -[2025-05-25 11:09:55,074][00035] Inference worker 0-0 is ready! -[2025-05-25 11:09:55,075][00035] All inference workers are ready! Signal rollout workers to start! -[2025-05-25 11:09:55,197][01061] Doom resolution: 160x120, resize resolution: (128, 72) -[2025-05-25 11:09:55,199][01064] Doom resolution: 160x120, resize resolution: (128, 72) -[2025-05-25 11:09:55,200][01068] Doom resolution: 160x120, resize resolution: (128, 72) -[2025-05-25 11:09:55,199][01065] Doom resolution: 160x120, resize resolution: (128, 72) -[2025-05-25 11:09:55,200][01063] Doom resolution: 160x120, resize resolution: (128, 72) -[2025-05-25 11:09:55,202][01066] Doom resolution: 160x120, resize resolution: (128, 72) -[2025-05-25 11:09:55,203][01067] Doom resolution: 160x120, resize resolution: (128, 72) -[2025-05-25 11:09:55,204][01062] Doom resolution: 160x120, resize resolution: (128, 72) -[2025-05-25 11:09:55,805][01065] Decorrelating experience for 0 frames... -[2025-05-25 11:09:55,805][01061] Decorrelating experience for 0 frames... -[2025-05-25 11:09:55,805][01063] Decorrelating experience for 0 frames... -[2025-05-25 11:09:56,078][01061] Decorrelating experience for 32 frames... -[2025-05-25 11:09:56,186][01062] Decorrelating experience for 0 frames... -[2025-05-25 11:09:56,183][01066] Decorrelating experience for 0 frames... -[2025-05-25 11:09:56,255][01065] Decorrelating experience for 32 frames... -[2025-05-25 11:09:56,493][01061] Decorrelating experience for 64 frames... -[2025-05-25 11:09:56,744][01064] Decorrelating experience for 0 frames... -[2025-05-25 11:09:56,796][01062] Decorrelating experience for 32 frames... -[2025-05-25 11:09:56,795][01066] Decorrelating experience for 32 frames... -[2025-05-25 11:09:56,984][01061] Decorrelating experience for 96 frames... -[2025-05-25 11:09:57,301][01065] Decorrelating experience for 64 frames... -[2025-05-25 11:09:57,553][01064] Decorrelating experience for 32 frames... -[2025-05-25 11:09:57,673][01063] Decorrelating experience for 32 frames... -[2025-05-25 11:09:57,833][01066] Decorrelating experience for 64 frames... -[2025-05-25 11:09:57,836][01062] Decorrelating experience for 64 frames... -[2025-05-25 11:09:58,315][01067] Decorrelating experience for 0 frames... -[2025-05-25 11:09:58,325][00035] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 60. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) -[2025-05-25 11:09:58,327][00035] Avg episode reward: [(0, '1.280')] -[2025-05-25 11:09:58,403][01062] Decorrelating experience for 96 frames... -[2025-05-25 11:09:58,487][01065] Decorrelating experience for 96 frames... -[2025-05-25 11:09:58,645][01064] Decorrelating experience for 64 frames... -[2025-05-25 11:09:58,761][01063] Decorrelating experience for 64 frames... -[2025-05-25 11:09:59,169][01067] Decorrelating experience for 32 frames... -[2025-05-25 11:09:59,194][01068] Decorrelating experience for 0 frames... -[2025-05-25 11:09:59,245][01064] Decorrelating experience for 96 frames... -[2025-05-25 11:10:00,207][01068] Decorrelating experience for 32 frames... -[2025-05-25 11:10:00,235][01066] Decorrelating experience for 96 frames... -[2025-05-25 11:10:00,298][01067] Decorrelating experience for 64 frames... -[2025-05-25 11:10:00,661][01047] Signal inference workers to stop experience collection... -[2025-05-25 11:10:00,669][01060] InferenceWorker_p0-w0: stopping experience collection -[2025-05-25 11:10:00,926][01067] Decorrelating experience for 96 frames... -[2025-05-25 11:10:01,019][01068] Decorrelating experience for 64 frames... -[2025-05-25 11:10:01,062][01063] Decorrelating experience for 96 frames... -[2025-05-25 11:10:01,385][01068] Decorrelating experience for 96 frames... -[2025-05-25 11:10:02,639][01047] Signal inference workers to resume experience collection... -[2025-05-25 11:10:02,640][01060] InferenceWorker_p0-w0: resuming experience collection -[2025-05-25 11:10:03,036][00035] Heartbeat connected on Batcher_0 -[2025-05-25 11:10:03,041][00035] Heartbeat connected on LearnerWorker_p0 -[2025-05-25 11:10:03,051][00035] Heartbeat connected on InferenceWorker_p0-w0 -[2025-05-25 11:10:03,057][00035] Heartbeat connected on RolloutWorker_w0 -[2025-05-25 11:10:03,067][00035] Heartbeat connected on RolloutWorker_w1 -[2025-05-25 11:10:03,072][00035] Heartbeat connected on RolloutWorker_w3 -[2025-05-25 11:10:03,074][00035] Heartbeat connected on RolloutWorker_w2 -[2025-05-25 11:10:03,076][00035] Heartbeat connected on RolloutWorker_w4 -[2025-05-25 11:10:03,081][00035] Heartbeat connected on RolloutWorker_w5 -[2025-05-25 11:10:03,086][00035] Heartbeat connected on RolloutWorker_w6 -[2025-05-25 11:10:03,104][00035] Heartbeat connected on RolloutWorker_w7 -[2025-05-25 11:10:03,325][00035] Fps is (10 sec: 1638.4, 60 sec: 1638.4, 300 sec: 1638.4). Total num frames: 8192. Throughput: 0: 452.4. Samples: 2322. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) -[2025-05-25 11:10:03,327][00035] Avg episode reward: [(0, '3.049')] -[2025-05-25 11:10:07,426][01060] Updated weights for policy 0, policy_version 10 (0.0100) -[2025-05-25 11:10:08,325][00035] Fps is (10 sec: 4505.6, 60 sec: 4505.6, 300 sec: 4505.6). Total num frames: 45056. Throughput: 0: 1020.2. Samples: 10262. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:10:08,326][00035] Avg episode reward: [(0, '4.305')] -[2025-05-25 11:10:11,699][01060] Updated weights for policy 0, policy_version 20 (0.0016) -[2025-05-25 11:10:13,325][00035] Fps is (10 sec: 8601.6, 60 sec: 6280.5, 300 sec: 6280.5). Total num frames: 94208. Throughput: 0: 1624.4. Samples: 24426. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:10:13,327][00035] Avg episode reward: [(0, '4.254')] -[2025-05-25 11:10:16,062][01060] Updated weights for policy 0, policy_version 30 (0.0019) -[2025-05-25 11:10:18,326][00035] Fps is (10 sec: 9829.9, 60 sec: 7167.8, 300 sec: 7167.8). Total num frames: 143360. Throughput: 0: 1569.2. Samples: 31444. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:10:18,328][00035] Avg episode reward: [(0, '4.510')] -[2025-05-25 11:10:18,336][01047] Saving new best policy, reward=4.510! -[2025-05-25 11:10:20,320][01060] Updated weights for policy 0, policy_version 40 (0.0017) -[2025-05-25 11:10:23,325][00035] Fps is (10 sec: 9830.5, 60 sec: 7700.5, 300 sec: 7700.5). Total num frames: 192512. Throughput: 0: 1828.0. Samples: 45760. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:10:23,328][00035] Avg episode reward: [(0, '4.294')] -[2025-05-25 11:10:24,718][01060] Updated weights for policy 0, policy_version 50 (0.0015) -[2025-05-25 11:10:28,325][00035] Fps is (10 sec: 9421.2, 60 sec: 7918.9, 300 sec: 7918.9). Total num frames: 237568. Throughput: 0: 1991.2. Samples: 59796. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:10:28,326][00035] Avg episode reward: [(0, '4.298')] -[2025-05-25 11:10:28,975][01060] Updated weights for policy 0, policy_version 60 (0.0017) -[2025-05-25 11:10:33,325][00035] Fps is (10 sec: 9011.2, 60 sec: 8075.0, 300 sec: 8075.0). Total num frames: 282624. Throughput: 0: 1911.6. Samples: 66966. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) -[2025-05-25 11:10:33,327][00035] Avg episode reward: [(0, '4.416')] -[2025-05-25 11:10:33,391][01060] Updated weights for policy 0, policy_version 70 (0.0015) -[2025-05-25 11:10:38,325][00035] Fps is (10 sec: 8601.7, 60 sec: 8089.6, 300 sec: 8089.6). Total num frames: 323584. Throughput: 0: 1988.3. Samples: 79590. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:10:38,327][00035] Avg episode reward: [(0, '4.345')] -[2025-05-25 11:10:38,347][01060] Updated weights for policy 0, policy_version 80 (0.0014) -[2025-05-25 11:10:42,635][01060] Updated weights for policy 0, policy_version 90 (0.0014) -[2025-05-25 11:10:43,325][00035] Fps is (10 sec: 9011.2, 60 sec: 8283.0, 300 sec: 8283.0). Total num frames: 372736. Throughput: 0: 2085.2. Samples: 93896. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:10:43,328][00035] Avg episode reward: [(0, '4.276')] -[2025-05-25 11:10:46,987][01060] Updated weights for policy 0, policy_version 100 (0.0017) -[2025-05-25 11:10:48,325][00035] Fps is (10 sec: 9830.3, 60 sec: 8437.8, 300 sec: 8437.8). Total num frames: 421888. Throughput: 0: 2192.2. Samples: 100970. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:10:48,327][00035] Avg episode reward: [(0, '4.723')] -[2025-05-25 11:10:48,340][01047] Saving new best policy, reward=4.723! -[2025-05-25 11:10:51,327][01060] Updated weights for policy 0, policy_version 110 (0.0017) -[2025-05-25 11:10:53,326][00035] Fps is (10 sec: 9419.7, 60 sec: 8489.7, 300 sec: 8489.7). Total num frames: 466944. Throughput: 0: 2330.9. Samples: 115154. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:10:53,328][00035] Avg episode reward: [(0, '4.337')] -[2025-05-25 11:10:55,634][01060] Updated weights for policy 0, policy_version 120 (0.0018) -[2025-05-25 11:10:58,325][00035] Fps is (10 sec: 9420.8, 60 sec: 8601.6, 300 sec: 8601.6). Total num frames: 516096. Throughput: 0: 2331.9. Samples: 129360. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2025-05-25 11:10:58,327][00035] Avg episode reward: [(0, '4.619')] -[2025-05-25 11:10:59,914][01060] Updated weights for policy 0, policy_version 130 (0.0019) -[2025-05-25 11:11:03,325][00035] Fps is (10 sec: 9421.9, 60 sec: 9216.0, 300 sec: 8633.1). Total num frames: 561152. Throughput: 0: 2334.4. Samples: 136490. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:11:03,327][00035] Avg episode reward: [(0, '4.545')] -[2025-05-25 11:11:04,325][01060] Updated weights for policy 0, policy_version 140 (0.0017) -[2025-05-25 11:11:08,325][00035] Fps is (10 sec: 9011.2, 60 sec: 9352.5, 300 sec: 8660.1). Total num frames: 606208. Throughput: 0: 2322.6. Samples: 150278. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:11:08,329][00035] Avg episode reward: [(0, '4.622')] -[2025-05-25 11:11:09,245][01060] Updated weights for policy 0, policy_version 150 (0.0016) -[2025-05-25 11:11:13,325][00035] Fps is (10 sec: 9011.3, 60 sec: 9284.3, 300 sec: 8683.5). Total num frames: 651264. Throughput: 0: 2303.1. Samples: 163436. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:11:13,326][00035] Avg episode reward: [(0, '4.569')] -[2025-05-25 11:11:13,604][01060] Updated weights for policy 0, policy_version 160 (0.0016) -[2025-05-25 11:11:17,920][01060] Updated weights for policy 0, policy_version 170 (0.0016) -[2025-05-25 11:11:18,326][00035] Fps is (10 sec: 9010.2, 60 sec: 9215.9, 300 sec: 8703.9). Total num frames: 696320. Throughput: 0: 2298.6. Samples: 170404. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:11:18,327][00035] Avg episode reward: [(0, '4.691')] -[2025-05-25 11:11:22,236][01060] Updated weights for policy 0, policy_version 180 (0.0014) -[2025-05-25 11:11:23,325][00035] Fps is (10 sec: 9420.7, 60 sec: 9216.0, 300 sec: 8770.3). Total num frames: 745472. Throughput: 0: 2336.2. Samples: 184720. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:11:23,327][00035] Avg episode reward: [(0, '4.675')] -[2025-05-25 11:11:26,530][01060] Updated weights for policy 0, policy_version 190 (0.0015) -[2025-05-25 11:11:28,325][00035] Fps is (10 sec: 9831.5, 60 sec: 9284.3, 300 sec: 8829.2). Total num frames: 794624. Throughput: 0: 2336.0. Samples: 199014. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:11:28,327][00035] Avg episode reward: [(0, '4.634')] -[2025-05-25 11:11:30,816][01060] Updated weights for policy 0, policy_version 200 (0.0015) -[2025-05-25 11:11:33,325][00035] Fps is (10 sec: 9420.8, 60 sec: 9284.3, 300 sec: 8838.7). Total num frames: 839680. Throughput: 0: 2336.2. Samples: 206098. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:11:33,329][00035] Avg episode reward: [(0, '4.466')] -[2025-05-25 11:11:35,222][01060] Updated weights for policy 0, policy_version 210 (0.0017) -[2025-05-25 11:11:38,325][00035] Fps is (10 sec: 9420.7, 60 sec: 9420.8, 300 sec: 8888.3). Total num frames: 888832. Throughput: 0: 2332.1. Samples: 220098. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:11:38,327][00035] Avg episode reward: [(0, '4.307')] -[2025-05-25 11:11:38,338][01047] Saving /kaggle/working/train_dir/default_experiment/checkpoint_p0/checkpoint_000000217_888832.pth... -[2025-05-25 11:11:39,629][01060] Updated weights for policy 0, policy_version 220 (0.0014) -[2025-05-25 11:11:43,325][00035] Fps is (10 sec: 9011.2, 60 sec: 9284.3, 300 sec: 8855.2). Total num frames: 929792. Throughput: 0: 2298.3. Samples: 232782. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:11:43,327][00035] Avg episode reward: [(0, '4.415')] -[2025-05-25 11:11:44,595][01060] Updated weights for policy 0, policy_version 230 (0.0014) -[2025-05-25 11:11:48,325][00035] Fps is (10 sec: 8601.6, 60 sec: 9216.0, 300 sec: 8862.3). Total num frames: 974848. Throughput: 0: 2295.1. Samples: 239770. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:11:48,326][00035] Avg episode reward: [(0, '4.504')] -[2025-05-25 11:11:48,853][01060] Updated weights for policy 0, policy_version 240 (0.0015) -[2025-05-25 11:11:53,236][01060] Updated weights for policy 0, policy_version 250 (0.0017) -[2025-05-25 11:11:53,325][00035] Fps is (10 sec: 9420.8, 60 sec: 9284.4, 300 sec: 8904.4). Total num frames: 1024000. Throughput: 0: 2307.2. Samples: 254104. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:11:53,327][00035] Avg episode reward: [(0, '4.490')] -[2025-05-25 11:11:57,483][01060] Updated weights for policy 0, policy_version 260 (0.0017) -[2025-05-25 11:11:58,325][00035] Fps is (10 sec: 9420.8, 60 sec: 9216.0, 300 sec: 8908.8). Total num frames: 1069056. Throughput: 0: 2331.1. Samples: 268338. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) -[2025-05-25 11:11:58,327][00035] Avg episode reward: [(0, '4.684')] -[2025-05-25 11:12:01,749][01060] Updated weights for policy 0, policy_version 270 (0.0016) -[2025-05-25 11:12:03,325][00035] Fps is (10 sec: 9420.9, 60 sec: 9284.3, 300 sec: 8945.7). Total num frames: 1118208. Throughput: 0: 2336.9. Samples: 275562. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:12:03,326][00035] Avg episode reward: [(0, '4.461')] -[2025-05-25 11:12:06,045][01060] Updated weights for policy 0, policy_version 280 (0.0020) -[2025-05-25 11:12:08,325][00035] Fps is (10 sec: 9830.4, 60 sec: 9352.5, 300 sec: 8979.7). Total num frames: 1167360. Throughput: 0: 2335.9. Samples: 289836. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:12:08,326][00035] Avg episode reward: [(0, '4.818')] -[2025-05-25 11:12:08,338][01047] Saving new best policy, reward=4.818! -[2025-05-25 11:12:10,364][01060] Updated weights for policy 0, policy_version 290 (0.0017) -[2025-05-25 11:12:13,325][00035] Fps is (10 sec: 9420.8, 60 sec: 9352.5, 300 sec: 8980.9). Total num frames: 1212416. Throughput: 0: 2317.7. Samples: 303310. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2025-05-25 11:12:13,326][00035] Avg episode reward: [(0, '4.870')] -[2025-05-25 11:12:13,331][01047] Saving new best policy, reward=4.870! -[2025-05-25 11:12:15,405][01060] Updated weights for policy 0, policy_version 300 (0.0015) -[2025-05-25 11:12:18,325][00035] Fps is (10 sec: 8601.6, 60 sec: 9284.4, 300 sec: 8952.7). Total num frames: 1253376. Throughput: 0: 2298.6. Samples: 309534. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:12:18,328][00035] Avg episode reward: [(0, '4.492')] -[2025-05-25 11:12:19,643][01060] Updated weights for policy 0, policy_version 310 (0.0015) -[2025-05-25 11:12:23,325][00035] Fps is (10 sec: 9011.2, 60 sec: 9284.3, 300 sec: 8983.0). Total num frames: 1302528. Throughput: 0: 2308.3. Samples: 323972. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:12:23,328][00035] Avg episode reward: [(0, '4.823')] -[2025-05-25 11:12:24,014][01060] Updated weights for policy 0, policy_version 320 (0.0019) -[2025-05-25 11:12:28,326][00035] Fps is (10 sec: 9420.4, 60 sec: 9215.9, 300 sec: 8983.9). Total num frames: 1347584. Throughput: 0: 2337.3. Samples: 337960. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:12:28,327][00035] Avg episode reward: [(0, '4.739')] -[2025-05-25 11:12:28,347][01060] Updated weights for policy 0, policy_version 330 (0.0018) -[2025-05-25 11:12:32,703][01060] Updated weights for policy 0, policy_version 340 (0.0017) -[2025-05-25 11:12:33,326][00035] Fps is (10 sec: 9420.3, 60 sec: 9284.2, 300 sec: 9011.2). Total num frames: 1396736. Throughput: 0: 2339.3. Samples: 345040. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:12:33,327][00035] Avg episode reward: [(0, '4.669')] -[2025-05-25 11:12:37,103][01060] Updated weights for policy 0, policy_version 350 (0.0014) -[2025-05-25 11:12:38,325][00035] Fps is (10 sec: 9421.2, 60 sec: 9216.0, 300 sec: 9011.2). Total num frames: 1441792. Throughput: 0: 2332.7. Samples: 359074. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:12:38,326][00035] Avg episode reward: [(0, '4.659')] -[2025-05-25 11:12:41,364][01060] Updated weights for policy 0, policy_version 360 (0.0018) -[2025-05-25 11:12:43,325][00035] Fps is (10 sec: 9421.3, 60 sec: 9352.5, 300 sec: 9036.0). Total num frames: 1490944. Throughput: 0: 2331.3. Samples: 373246. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2025-05-25 11:12:43,328][00035] Avg episode reward: [(0, '4.626')] -[2025-05-25 11:12:46,160][01060] Updated weights for policy 0, policy_version 370 (0.0017) -[2025-05-25 11:12:48,325][00035] Fps is (10 sec: 9011.2, 60 sec: 9284.3, 300 sec: 9011.2). Total num frames: 1531904. Throughput: 0: 2304.9. Samples: 379282. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:12:48,327][00035] Avg episode reward: [(0, '4.745')] -[2025-05-25 11:12:50,760][01060] Updated weights for policy 0, policy_version 380 (0.0018) -[2025-05-25 11:12:53,325][00035] Fps is (10 sec: 8601.6, 60 sec: 9216.0, 300 sec: 9011.2). Total num frames: 1576960. Throughput: 0: 2291.3. Samples: 392946. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) -[2025-05-25 11:12:53,327][00035] Avg episode reward: [(0, '4.803')] -[2025-05-25 11:12:55,118][01060] Updated weights for policy 0, policy_version 390 (0.0017) -[2025-05-25 11:12:58,325][00035] Fps is (10 sec: 9420.7, 60 sec: 9284.3, 300 sec: 9034.0). Total num frames: 1626112. Throughput: 0: 2303.5. Samples: 406966. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:12:58,327][00035] Avg episode reward: [(0, '4.808')] -[2025-05-25 11:12:59,559][01060] Updated weights for policy 0, policy_version 400 (0.0022) -[2025-05-25 11:13:03,325][00035] Fps is (10 sec: 9420.6, 60 sec: 9216.0, 300 sec: 9033.3). Total num frames: 1671168. Throughput: 0: 2320.5. Samples: 413958. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:13:03,327][00035] Avg episode reward: [(0, '4.546')] -[2025-05-25 11:13:03,906][01060] Updated weights for policy 0, policy_version 410 (0.0017) -[2025-05-25 11:13:08,223][01060] Updated weights for policy 0, policy_version 420 (0.0015) -[2025-05-25 11:13:08,325][00035] Fps is (10 sec: 9420.8, 60 sec: 9216.0, 300 sec: 9054.3). Total num frames: 1720320. Throughput: 0: 2311.5. Samples: 427988. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2025-05-25 11:13:08,327][00035] Avg episode reward: [(0, '4.971')] -[2025-05-25 11:13:08,338][01047] Saving new best policy, reward=4.971! -[2025-05-25 11:13:12,572][01060] Updated weights for policy 0, policy_version 430 (0.0018) -[2025-05-25 11:13:13,325][00035] Fps is (10 sec: 9421.0, 60 sec: 9216.0, 300 sec: 9053.2). Total num frames: 1765376. Throughput: 0: 2315.3. Samples: 442146. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2025-05-25 11:13:13,327][00035] Avg episode reward: [(0, '4.909')] -[2025-05-25 11:13:16,950][01060] Updated weights for policy 0, policy_version 440 (0.0014) -[2025-05-25 11:13:18,325][00035] Fps is (10 sec: 9011.2, 60 sec: 9284.3, 300 sec: 9052.2). Total num frames: 1810432. Throughput: 0: 2314.3. Samples: 449184. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:13:18,327][00035] Avg episode reward: [(0, '4.971')] -[2025-05-25 11:13:21,857][01060] Updated weights for policy 0, policy_version 450 (0.0013) -[2025-05-25 11:13:23,325][00035] Fps is (10 sec: 9011.2, 60 sec: 9216.0, 300 sec: 9051.2). Total num frames: 1855488. Throughput: 0: 2287.2. Samples: 461998. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:13:23,326][00035] Avg episode reward: [(0, '4.872')] -[2025-05-25 11:13:26,248][01060] Updated weights for policy 0, policy_version 460 (0.0018) -[2025-05-25 11:13:28,325][00035] Fps is (10 sec: 9011.0, 60 sec: 9216.0, 300 sec: 9050.2). Total num frames: 1900544. Throughput: 0: 2285.6. Samples: 476098. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:13:28,327][00035] Avg episode reward: [(0, '4.930')] -[2025-05-25 11:13:30,604][01060] Updated weights for policy 0, policy_version 470 (0.0015) -[2025-05-25 11:13:33,325][00035] Fps is (10 sec: 9420.8, 60 sec: 9216.1, 300 sec: 9068.4). Total num frames: 1949696. Throughput: 0: 2305.4. Samples: 483026. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:13:33,327][00035] Avg episode reward: [(0, '4.919')] -[2025-05-25 11:13:35,151][01060] Updated weights for policy 0, policy_version 480 (0.0014) -[2025-05-25 11:13:38,326][00035] Fps is (10 sec: 9420.1, 60 sec: 9215.8, 300 sec: 9067.0). Total num frames: 1994752. Throughput: 0: 2309.5. Samples: 496876. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:13:38,329][00035] Avg episode reward: [(0, '4.709')] -[2025-05-25 11:13:38,341][01047] Saving /kaggle/working/train_dir/default_experiment/checkpoint_p0/checkpoint_000000487_1994752.pth... -[2025-05-25 11:13:39,490][01060] Updated weights for policy 0, policy_version 490 (0.0015) -[2025-05-25 11:13:43,325][00035] Fps is (10 sec: 9011.2, 60 sec: 9147.7, 300 sec: 9065.8). Total num frames: 2039808. Throughput: 0: 2309.6. Samples: 510900. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:13:43,326][00035] Avg episode reward: [(0, '5.060')] -[2025-05-25 11:13:43,376][01047] Saving new best policy, reward=5.060! -[2025-05-25 11:13:43,903][01060] Updated weights for policy 0, policy_version 500 (0.0020) -[2025-05-25 11:13:48,248][01060] Updated weights for policy 0, policy_version 510 (0.0017) -[2025-05-25 11:13:48,325][00035] Fps is (10 sec: 9421.5, 60 sec: 9284.2, 300 sec: 9082.4). Total num frames: 2088960. Throughput: 0: 2307.4. Samples: 517790. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:13:48,327][00035] Avg episode reward: [(0, '5.235')] -[2025-05-25 11:13:48,338][01047] Saving new best policy, reward=5.235! -[2025-05-25 11:13:53,230][01060] Updated weights for policy 0, policy_version 520 (0.0016) -[2025-05-25 11:13:53,325][00035] Fps is (10 sec: 9011.2, 60 sec: 9216.0, 300 sec: 9063.5). Total num frames: 2129920. Throughput: 0: 2277.9. Samples: 530494. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:13:53,326][00035] Avg episode reward: [(0, '4.951')] -[2025-05-25 11:13:57,558][01060] Updated weights for policy 0, policy_version 530 (0.0018) -[2025-05-25 11:13:58,325][00035] Fps is (10 sec: 8601.8, 60 sec: 9147.7, 300 sec: 9062.4). Total num frames: 2174976. Throughput: 0: 2278.1. Samples: 544660. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:13:58,327][00035] Avg episode reward: [(0, '5.032')] -[2025-05-25 11:14:01,877][01060] Updated weights for policy 0, policy_version 540 (0.0016) -[2025-05-25 11:14:03,325][00035] Fps is (10 sec: 9420.6, 60 sec: 9216.0, 300 sec: 9078.1). Total num frames: 2224128. Throughput: 0: 2281.6. Samples: 551856. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:14:03,327][00035] Avg episode reward: [(0, '5.191')] -[2025-05-25 11:14:06,245][01060] Updated weights for policy 0, policy_version 550 (0.0014) -[2025-05-25 11:14:08,325][00035] Fps is (10 sec: 9420.8, 60 sec: 9147.7, 300 sec: 9076.7). Total num frames: 2269184. Throughput: 0: 2308.7. Samples: 565888. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:14:08,327][00035] Avg episode reward: [(0, '4.991')] -[2025-05-25 11:14:10,587][01060] Updated weights for policy 0, policy_version 560 (0.0014) -[2025-05-25 11:14:13,325][00035] Fps is (10 sec: 9420.8, 60 sec: 9216.0, 300 sec: 9091.5). Total num frames: 2318336. Throughput: 0: 2310.0. Samples: 580046. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:14:13,327][00035] Avg episode reward: [(0, '5.251')] -[2025-05-25 11:14:13,332][01047] Saving new best policy, reward=5.251! -[2025-05-25 11:14:15,049][01060] Updated weights for policy 0, policy_version 570 (0.0016) -[2025-05-25 11:14:18,325][00035] Fps is (10 sec: 9420.8, 60 sec: 9216.0, 300 sec: 9090.0). Total num frames: 2363392. Throughput: 0: 2306.0. Samples: 586798. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:14:18,327][00035] Avg episode reward: [(0, '4.570')] -[2025-05-25 11:14:19,385][01060] Updated weights for policy 0, policy_version 580 (0.0018) -[2025-05-25 11:14:23,326][00035] Fps is (10 sec: 9011.0, 60 sec: 9215.9, 300 sec: 9088.5). Total num frames: 2408448. Throughput: 0: 2312.6. Samples: 600940. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2025-05-25 11:14:23,327][00035] Avg episode reward: [(0, '4.864')] -[2025-05-25 11:14:24,252][01060] Updated weights for policy 0, policy_version 590 (0.0016) -[2025-05-25 11:14:28,325][00035] Fps is (10 sec: 9011.2, 60 sec: 9216.0, 300 sec: 9087.1). Total num frames: 2453504. Throughput: 0: 2281.2. Samples: 613554. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:14:28,327][00035] Avg episode reward: [(0, '4.893')] -[2025-05-25 11:14:28,782][01060] Updated weights for policy 0, policy_version 600 (0.0016) -[2025-05-25 11:14:33,046][01060] Updated weights for policy 0, policy_version 610 (0.0017) -[2025-05-25 11:14:33,325][00035] Fps is (10 sec: 9011.6, 60 sec: 9147.7, 300 sec: 9085.7). Total num frames: 2498560. Throughput: 0: 2285.9. Samples: 620656. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:14:33,326][00035] Avg episode reward: [(0, '4.584')] -[2025-05-25 11:14:37,540][01060] Updated weights for policy 0, policy_version 620 (0.0014) -[2025-05-25 11:14:38,325][00035] Fps is (10 sec: 9011.1, 60 sec: 9147.9, 300 sec: 9084.3). Total num frames: 2543616. Throughput: 0: 2311.4. Samples: 634508. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2025-05-25 11:14:38,328][00035] Avg episode reward: [(0, '4.673')] -[2025-05-25 11:14:41,807][01060] Updated weights for policy 0, policy_version 630 (0.0016) -[2025-05-25 11:14:43,325][00035] Fps is (10 sec: 9420.8, 60 sec: 9216.0, 300 sec: 9097.4). Total num frames: 2592768. Throughput: 0: 2315.9. Samples: 648874. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:14:43,326][00035] Avg episode reward: [(0, '4.641')] -[2025-05-25 11:14:46,083][01060] Updated weights for policy 0, policy_version 640 (0.0016) -[2025-05-25 11:14:48,325][00035] Fps is (10 sec: 9830.5, 60 sec: 9216.0, 300 sec: 9110.1). Total num frames: 2641920. Throughput: 0: 2312.9. Samples: 655934. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:14:48,330][00035] Avg episode reward: [(0, '5.311')] -[2025-05-25 11:14:48,340][01047] Saving new best policy, reward=5.311! -[2025-05-25 11:14:50,425][01060] Updated weights for policy 0, policy_version 650 (0.0016) -[2025-05-25 11:14:53,325][00035] Fps is (10 sec: 9420.8, 60 sec: 9284.3, 300 sec: 9108.4). Total num frames: 2686976. Throughput: 0: 2319.4. Samples: 670262. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:14:53,328][00035] Avg episode reward: [(0, '4.891')] -[2025-05-25 11:14:54,765][01060] Updated weights for policy 0, policy_version 660 (0.0016) -[2025-05-25 11:14:58,325][00035] Fps is (10 sec: 8601.6, 60 sec: 9216.0, 300 sec: 9219.5). Total num frames: 2727936. Throughput: 0: 2281.0. Samples: 682690. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2025-05-25 11:14:58,327][00035] Avg episode reward: [(0, '4.978')] -[2025-05-25 11:14:59,766][01060] Updated weights for policy 0, policy_version 670 (0.0015) -[2025-05-25 11:15:03,325][00035] Fps is (10 sec: 9011.2, 60 sec: 9216.0, 300 sec: 9261.1). Total num frames: 2777088. Throughput: 0: 2289.7. Samples: 689836. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:15:03,327][00035] Avg episode reward: [(0, '4.763')] -[2025-05-25 11:15:04,161][01060] Updated weights for policy 0, policy_version 680 (0.0020) -[2025-05-25 11:15:08,325][00035] Fps is (10 sec: 9420.8, 60 sec: 9216.0, 300 sec: 9247.2). Total num frames: 2822144. Throughput: 0: 2290.4. Samples: 704008. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:15:08,326][00035] Avg episode reward: [(0, '4.814')] -[2025-05-25 11:15:08,424][01060] Updated weights for policy 0, policy_version 690 (0.0016) -[2025-05-25 11:15:12,722][01060] Updated weights for policy 0, policy_version 700 (0.0016) -[2025-05-25 11:15:13,325][00035] Fps is (10 sec: 9420.6, 60 sec: 9216.0, 300 sec: 9247.2). Total num frames: 2871296. Throughput: 0: 2327.4. Samples: 718286. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) -[2025-05-25 11:15:13,327][00035] Avg episode reward: [(0, '5.006')] -[2025-05-25 11:15:17,098][01060] Updated weights for policy 0, policy_version 710 (0.0020) -[2025-05-25 11:15:18,325][00035] Fps is (10 sec: 9420.8, 60 sec: 9216.0, 300 sec: 9233.4). Total num frames: 2916352. Throughput: 0: 2326.5. Samples: 725350. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:15:18,327][00035] Avg episode reward: [(0, '5.181')] -[2025-05-25 11:15:21,292][01060] Updated weights for policy 0, policy_version 720 (0.0017) -[2025-05-25 11:15:23,325][00035] Fps is (10 sec: 9421.0, 60 sec: 9284.3, 300 sec: 9247.2). Total num frames: 2965504. Throughput: 0: 2340.7. Samples: 739838. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:15:23,327][00035] Avg episode reward: [(0, '5.979')] -[2025-05-25 11:15:23,329][01047] Saving new best policy, reward=5.979! -[2025-05-25 11:15:25,628][01060] Updated weights for policy 0, policy_version 730 (0.0017) -[2025-05-25 11:15:28,325][00035] Fps is (10 sec: 9420.8, 60 sec: 9284.3, 300 sec: 9247.2). Total num frames: 3010560. Throughput: 0: 2330.8. Samples: 753762. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:15:28,327][00035] Avg episode reward: [(0, '4.867')] -[2025-05-25 11:15:30,499][01060] Updated weights for policy 0, policy_version 740 (0.0015) -[2025-05-25 11:15:33,325][00035] Fps is (10 sec: 9011.3, 60 sec: 9284.3, 300 sec: 9261.1). Total num frames: 3055616. Throughput: 0: 2306.3. Samples: 759716. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2025-05-25 11:15:33,327][00035] Avg episode reward: [(0, '5.079')] -[2025-05-25 11:15:34,851][01060] Updated weights for policy 0, policy_version 750 (0.0015) -[2025-05-25 11:15:38,325][00035] Fps is (10 sec: 9011.1, 60 sec: 9284.3, 300 sec: 9247.2). Total num frames: 3100672. Throughput: 0: 2304.2. Samples: 773952. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:15:38,326][00035] Avg episode reward: [(0, '5.331')] -[2025-05-25 11:15:38,345][01047] Saving /kaggle/working/train_dir/default_experiment/checkpoint_p0/checkpoint_000000758_3104768.pth... -[2025-05-25 11:15:38,428][01047] Removing /kaggle/working/train_dir/default_experiment/checkpoint_p0/checkpoint_000000217_888832.pth -[2025-05-25 11:15:39,263][01060] Updated weights for policy 0, policy_version 760 (0.0016) -[2025-05-25 11:15:43,325][00035] Fps is (10 sec: 9420.8, 60 sec: 9284.3, 300 sec: 9247.2). Total num frames: 3149824. Throughput: 0: 2340.7. Samples: 788020. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:15:43,327][00035] Avg episode reward: [(0, '5.236')] -[2025-05-25 11:15:43,545][01060] Updated weights for policy 0, policy_version 770 (0.0017) -[2025-05-25 11:15:47,943][01060] Updated weights for policy 0, policy_version 780 (0.0017) -[2025-05-25 11:15:48,325][00035] Fps is (10 sec: 9420.8, 60 sec: 9216.0, 300 sec: 9247.3). Total num frames: 3194880. Throughput: 0: 2337.9. Samples: 795042. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:15:48,327][00035] Avg episode reward: [(0, '5.114')] -[2025-05-25 11:15:52,181][01060] Updated weights for policy 0, policy_version 790 (0.0014) -[2025-05-25 11:15:53,325][00035] Fps is (10 sec: 9420.8, 60 sec: 9284.3, 300 sec: 9247.2). Total num frames: 3244032. Throughput: 0: 2339.5. Samples: 809286. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2025-05-25 11:15:53,330][00035] Avg episode reward: [(0, '4.960')] -[2025-05-25 11:15:56,550][01060] Updated weights for policy 0, policy_version 800 (0.0016) -[2025-05-25 11:15:58,325][00035] Fps is (10 sec: 9830.5, 60 sec: 9420.8, 300 sec: 9261.1). Total num frames: 3293184. Throughput: 0: 2336.0. Samples: 823404. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:15:58,326][00035] Avg episode reward: [(0, '5.070')] -[2025-05-25 11:16:01,138][01060] Updated weights for policy 0, policy_version 810 (0.0018) -[2025-05-25 11:16:03,325][00035] Fps is (10 sec: 9011.3, 60 sec: 9284.3, 300 sec: 9247.2). Total num frames: 3334144. Throughput: 0: 2324.6. Samples: 829956. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:16:03,328][00035] Avg episode reward: [(0, '5.166')] -[2025-05-25 11:16:05,923][01060] Updated weights for policy 0, policy_version 820 (0.0015) -[2025-05-25 11:16:08,325][00035] Fps is (10 sec: 8601.6, 60 sec: 9284.3, 300 sec: 9247.2). Total num frames: 3379200. Throughput: 0: 2292.6. Samples: 843004. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2025-05-25 11:16:08,329][00035] Avg episode reward: [(0, '4.748')] -[2025-05-25 11:16:10,243][01060] Updated weights for policy 0, policy_version 830 (0.0017) -[2025-05-25 11:16:13,325][00035] Fps is (10 sec: 9011.2, 60 sec: 9216.0, 300 sec: 9247.3). Total num frames: 3424256. Throughput: 0: 2299.3. Samples: 857232. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:16:13,326][00035] Avg episode reward: [(0, '4.711')] -[2025-05-25 11:16:14,749][01060] Updated weights for policy 0, policy_version 840 (0.0015) -[2025-05-25 11:16:18,325][00035] Fps is (10 sec: 9420.7, 60 sec: 9284.3, 300 sec: 9247.2). Total num frames: 3473408. Throughput: 0: 2320.2. Samples: 864126. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) -[2025-05-25 11:16:18,327][00035] Avg episode reward: [(0, '5.047')] -[2025-05-25 11:16:19,039][01060] Updated weights for policy 0, policy_version 850 (0.0015) -[2025-05-25 11:16:23,312][01060] Updated weights for policy 0, policy_version 860 (0.0016) -[2025-05-25 11:16:23,325][00035] Fps is (10 sec: 9830.4, 60 sec: 9284.3, 300 sec: 9247.2). Total num frames: 3522560. Throughput: 0: 2322.8. Samples: 878476. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:16:23,328][00035] Avg episode reward: [(0, '4.728')] -[2025-05-25 11:16:27,638][01060] Updated weights for policy 0, policy_version 870 (0.0018) -[2025-05-25 11:16:28,325][00035] Fps is (10 sec: 9420.8, 60 sec: 9284.3, 300 sec: 9247.2). Total num frames: 3567616. Throughput: 0: 2324.4. Samples: 892620. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:16:28,327][00035] Avg episode reward: [(0, '4.699')] -[2025-05-25 11:16:31,846][01060] Updated weights for policy 0, policy_version 880 (0.0019) -[2025-05-25 11:16:33,325][00035] Fps is (10 sec: 9420.7, 60 sec: 9352.5, 300 sec: 9247.2). Total num frames: 3616768. Throughput: 0: 2331.6. Samples: 899962. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:16:33,328][00035] Avg episode reward: [(0, '4.872')] -[2025-05-25 11:16:36,735][01060] Updated weights for policy 0, policy_version 890 (0.0016) -[2025-05-25 11:16:38,325][00035] Fps is (10 sec: 9011.2, 60 sec: 9284.3, 300 sec: 9247.2). Total num frames: 3657728. Throughput: 0: 2299.2. Samples: 912750. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:16:38,327][00035] Avg episode reward: [(0, '4.904')] -[2025-05-25 11:16:41,142][01060] Updated weights for policy 0, policy_version 900 (0.0019) -[2025-05-25 11:16:43,325][00035] Fps is (10 sec: 9011.2, 60 sec: 9284.3, 300 sec: 9261.1). Total num frames: 3706880. Throughput: 0: 2300.6. Samples: 926932. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:16:43,326][00035] Avg episode reward: [(0, '4.862')] -[2025-05-25 11:16:45,449][01060] Updated weights for policy 0, policy_version 910 (0.0015) -[2025-05-25 11:16:48,325][00035] Fps is (10 sec: 9420.6, 60 sec: 9284.3, 300 sec: 9247.2). Total num frames: 3751936. Throughput: 0: 2309.8. Samples: 933898. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2025-05-25 11:16:48,327][00035] Avg episode reward: [(0, '4.784')] -[2025-05-25 11:16:49,832][01060] Updated weights for policy 0, policy_version 920 (0.0014) -[2025-05-25 11:16:53,325][00035] Fps is (10 sec: 9420.8, 60 sec: 9284.3, 300 sec: 9261.1). Total num frames: 3801088. Throughput: 0: 2334.8. Samples: 948070. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:16:53,327][00035] Avg episode reward: [(0, '4.952')] -[2025-05-25 11:16:54,202][01060] Updated weights for policy 0, policy_version 930 (0.0016) -[2025-05-25 11:16:58,325][00035] Fps is (10 sec: 9420.9, 60 sec: 9216.0, 300 sec: 9247.2). Total num frames: 3846144. Throughput: 0: 2335.3. Samples: 962320. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:16:58,326][00035] Avg episode reward: [(0, '4.933')] -[2025-05-25 11:16:58,474][01060] Updated weights for policy 0, policy_version 940 (0.0017) -[2025-05-25 11:17:02,731][01060] Updated weights for policy 0, policy_version 950 (0.0018) -[2025-05-25 11:17:03,325][00035] Fps is (10 sec: 9420.8, 60 sec: 9352.5, 300 sec: 9247.2). Total num frames: 3895296. Throughput: 0: 2343.0. Samples: 969562. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2025-05-25 11:17:03,327][00035] Avg episode reward: [(0, '4.647')] -[2025-05-25 11:17:07,592][01060] Updated weights for policy 0, policy_version 960 (0.0014) -[2025-05-25 11:17:08,326][00035] Fps is (10 sec: 9010.6, 60 sec: 9284.2, 300 sec: 9233.3). Total num frames: 3936256. Throughput: 0: 2319.7. Samples: 982866. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2025-05-25 11:17:08,327][00035] Avg episode reward: [(0, '5.051')] -[2025-05-25 11:17:11,906][01060] Updated weights for policy 0, policy_version 970 (0.0015) -[2025-05-25 11:17:13,326][00035] Fps is (10 sec: 9010.5, 60 sec: 9352.4, 300 sec: 9261.1). Total num frames: 3985408. Throughput: 0: 2317.5. Samples: 996910. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) -[2025-05-25 11:17:13,329][00035] Avg episode reward: [(0, '4.935')] -[2025-05-25 11:17:15,292][01047] Stopping Batcher_0... -[2025-05-25 11:17:15,292][01047] Loop batcher_evt_loop terminating... -[2025-05-25 11:17:15,293][01047] Saving /kaggle/working/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... -[2025-05-25 11:17:15,292][00035] Component Batcher_0 stopped! -[2025-05-25 11:17:15,324][01060] Weights refcount: 2 0 -[2025-05-25 11:17:15,326][01060] Stopping InferenceWorker_p0-w0... -[2025-05-25 11:17:15,327][01060] Loop inference_proc0-0_evt_loop terminating... -[2025-05-25 11:17:15,327][00035] Component InferenceWorker_p0-w0 stopped! -[2025-05-25 11:17:15,382][01047] Removing /kaggle/working/train_dir/default_experiment/checkpoint_p0/checkpoint_000000487_1994752.pth -[2025-05-25 11:17:15,396][00035] Component RolloutWorker_w6 stopped! -[2025-05-25 11:17:15,395][01068] Stopping RolloutWorker_w6... -[2025-05-25 11:17:15,400][01068] Loop rollout_proc6_evt_loop terminating... -[2025-05-25 11:17:15,401][01047] Saving /kaggle/working/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... -[2025-05-25 11:17:15,402][01067] Stopping RolloutWorker_w7... -[2025-05-25 11:17:15,402][01067] Loop rollout_proc7_evt_loop terminating... -[2025-05-25 11:17:15,404][00035] Component RolloutWorker_w7 stopped! -[2025-05-25 11:17:15,409][01061] Stopping RolloutWorker_w2... -[2025-05-25 11:17:15,410][00035] Component RolloutWorker_w2 stopped! -[2025-05-25 11:17:15,411][00035] Component RolloutWorker_w3 stopped! -[2025-05-25 11:17:15,411][01065] Stopping RolloutWorker_w3... -[2025-05-25 11:17:15,413][01065] Loop rollout_proc3_evt_loop terminating... -[2025-05-25 11:17:15,413][01061] Loop rollout_proc2_evt_loop terminating... -[2025-05-25 11:17:15,525][00035] Component LearnerWorker_p0 stopped! -[2025-05-25 11:17:15,527][01047] Stopping LearnerWorker_p0... -[2025-05-25 11:17:15,527][01047] Loop learner_proc0_evt_loop terminating... -[2025-05-25 11:17:15,563][00035] Component RolloutWorker_w0 stopped! -[2025-05-25 11:17:15,565][01062] Stopping RolloutWorker_w1... -[2025-05-25 11:17:15,566][01062] Loop rollout_proc1_evt_loop terminating... -[2025-05-25 11:17:15,566][00035] Component RolloutWorker_w1 stopped! -[2025-05-25 11:17:15,565][01063] Stopping RolloutWorker_w0... -[2025-05-25 11:17:15,569][01063] Loop rollout_proc0_evt_loop terminating... -[2025-05-25 11:17:15,601][00035] Component RolloutWorker_w4 stopped! -[2025-05-25 11:17:15,601][01064] Stopping RolloutWorker_w4... -[2025-05-25 11:17:15,603][01064] Loop rollout_proc4_evt_loop terminating... -[2025-05-25 11:17:15,625][01066] Stopping RolloutWorker_w5... -[2025-05-25 11:17:15,626][01066] Loop rollout_proc5_evt_loop terminating... -[2025-05-25 11:17:15,628][00035] Component RolloutWorker_w5 stopped! -[2025-05-25 11:17:15,629][00035] Waiting for process learner_proc0 to stop... -[2025-05-25 11:17:17,018][00035] Waiting for process inference_proc0-0 to join... -[2025-05-25 11:17:17,021][00035] Waiting for process rollout_proc0 to join... -[2025-05-25 11:17:17,358][00035] Waiting for process rollout_proc1 to join... -[2025-05-25 11:17:17,555][00035] Waiting for process rollout_proc2 to join... -[2025-05-25 11:17:17,557][00035] Waiting for process rollout_proc3 to join... -[2025-05-25 11:17:17,558][00035] Waiting for process rollout_proc4 to join... -[2025-05-25 11:17:17,559][00035] Waiting for process rollout_proc5 to join... -[2025-05-25 11:17:17,560][00035] Waiting for process rollout_proc6 to join... -[2025-05-25 11:17:17,561][00035] Waiting for process rollout_proc7 to join... -[2025-05-25 11:17:17,562][00035] Batcher 0 profile tree view: -batching: 21.5743, releasing_batches: 0.0246 -[2025-05-25 11:17:17,563][00035] InferenceWorker_p0-w0 profile tree view: +[2025-05-25 11:42:45,370][04219] Using optimizer +[2025-05-25 11:42:46,772][04028] Heartbeat connected on Batcher_0 +[2025-05-25 11:42:46,780][04028] Heartbeat connected on InferenceWorker_p0-w0 +[2025-05-25 11:42:46,786][04028] Heartbeat connected on RolloutWorker_w0 +[2025-05-25 11:42:46,790][04028] Heartbeat connected on RolloutWorker_w1 +[2025-05-25 11:42:46,793][04028] Heartbeat connected on RolloutWorker_w2 +[2025-05-25 11:42:46,799][04028] Heartbeat connected on RolloutWorker_w4 +[2025-05-25 11:42:46,801][04028] Heartbeat connected on RolloutWorker_w3 +[2025-05-25 11:42:46,802][04028] Heartbeat connected on RolloutWorker_w5 +[2025-05-25 11:42:46,807][04028] Heartbeat connected on RolloutWorker_w6 +[2025-05-25 11:42:46,812][04028] Heartbeat connected on RolloutWorker_w7 +[2025-05-25 11:42:49,720][04219] No checkpoints found +[2025-05-25 11:42:49,720][04219] Did not load from checkpoint, starting from scratch! +[2025-05-25 11:42:49,720][04219] Initialized policy 0 weights for model version 0 +[2025-05-25 11:42:49,723][04219] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2025-05-25 11:42:49,730][04219] LearnerWorker_p0 finished initialization! +[2025-05-25 11:42:49,731][04028] Heartbeat connected on LearnerWorker_p0 +[2025-05-25 11:42:49,939][04232] RunningMeanStd input shape: (3, 72, 128) +[2025-05-25 11:42:49,940][04232] RunningMeanStd input shape: (1,) +[2025-05-25 11:42:49,952][04232] ConvEncoder: input_channels=3 +[2025-05-25 11:42:50,052][04232] Conv encoder output size: 512 +[2025-05-25 11:42:50,053][04232] Policy head output size: 512 +[2025-05-25 11:42:50,089][04028] Inference worker 0-0 is ready! +[2025-05-25 11:42:50,090][04028] All inference workers are ready! Signal rollout workers to start! +[2025-05-25 11:42:50,350][04240] Doom resolution: 160x120, resize resolution: (128, 72) +[2025-05-25 11:42:50,346][04234] Doom resolution: 160x120, resize resolution: (128, 72) +[2025-05-25 11:42:50,360][04238] Doom resolution: 160x120, resize resolution: (128, 72) +[2025-05-25 11:42:50,359][04239] Doom resolution: 160x120, resize resolution: (128, 72) +[2025-05-25 11:42:50,367][04237] Doom resolution: 160x120, resize resolution: (128, 72) +[2025-05-25 11:42:50,371][04233] Doom resolution: 160x120, resize resolution: (128, 72) +[2025-05-25 11:42:50,390][04236] Doom resolution: 160x120, resize resolution: (128, 72) +[2025-05-25 11:42:50,390][04235] Doom resolution: 160x120, resize resolution: (128, 72) +[2025-05-25 11:42:51,830][04238] Decorrelating experience for 0 frames... +[2025-05-25 11:42:51,832][04240] Decorrelating experience for 0 frames... +[2025-05-25 11:42:51,830][04234] Decorrelating experience for 0 frames... +[2025-05-25 11:42:51,834][04235] Decorrelating experience for 0 frames... +[2025-05-25 11:42:51,832][04239] Decorrelating experience for 0 frames... +[2025-05-25 11:42:51,834][04237] Decorrelating experience for 0 frames... +[2025-05-25 11:42:52,613][04028] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) +[2025-05-25 11:42:52,895][04233] Decorrelating experience for 0 frames... +[2025-05-25 11:42:52,979][04234] Decorrelating experience for 32 frames... +[2025-05-25 11:42:52,981][04238] Decorrelating experience for 32 frames... +[2025-05-25 11:42:52,978][04240] Decorrelating experience for 32 frames... +[2025-05-25 11:42:52,983][04237] Decorrelating experience for 32 frames... +[2025-05-25 11:42:52,988][04235] Decorrelating experience for 32 frames... +[2025-05-25 11:42:54,080][04238] Decorrelating experience for 64 frames... +[2025-05-25 11:42:54,082][04240] Decorrelating experience for 64 frames... +[2025-05-25 11:42:54,392][04233] Decorrelating experience for 32 frames... +[2025-05-25 11:42:54,582][04239] Decorrelating experience for 32 frames... +[2025-05-25 11:42:54,794][04234] Decorrelating experience for 64 frames... +[2025-05-25 11:42:54,965][04237] Decorrelating experience for 64 frames... +[2025-05-25 11:42:54,968][04235] Decorrelating experience for 64 frames... +[2025-05-25 11:42:55,546][04236] Decorrelating experience for 0 frames... +[2025-05-25 11:42:55,879][04238] Decorrelating experience for 96 frames... +[2025-05-25 11:42:55,936][04233] Decorrelating experience for 64 frames... +[2025-05-25 11:42:56,273][04240] Decorrelating experience for 96 frames... +[2025-05-25 11:42:56,387][04235] Decorrelating experience for 96 frames... +[2025-05-25 11:42:57,626][04028] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) +[2025-05-25 11:42:57,825][04236] Decorrelating experience for 32 frames... +[2025-05-25 11:42:58,681][04234] Decorrelating experience for 96 frames... +[2025-05-25 11:42:59,048][04237] Decorrelating experience for 96 frames... +[2025-05-25 11:42:59,274][04233] Decorrelating experience for 96 frames... +[2025-05-25 11:43:02,277][04236] Decorrelating experience for 64 frames... +[2025-05-25 11:43:02,508][04239] Decorrelating experience for 64 frames... +[2025-05-25 11:43:02,613][04028] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 198.6. Samples: 1986. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) +[2025-05-25 11:43:02,617][04028] Avg episode reward: [(0, '2.336')] +[2025-05-25 11:43:03,011][04219] Signal inference workers to stop experience collection... +[2025-05-25 11:43:03,053][04232] InferenceWorker_p0-w0: stopping experience collection +[2025-05-25 11:43:04,056][04236] Decorrelating experience for 96 frames... +[2025-05-25 11:43:04,441][04239] Decorrelating experience for 96 frames... +[2025-05-25 11:43:04,621][04219] Signal inference workers to resume experience collection... +[2025-05-25 11:43:04,622][04232] InferenceWorker_p0-w0: resuming experience collection +[2025-05-25 11:43:07,613][04028] Fps is (10 sec: 2050.6, 60 sec: 1365.3, 300 sec: 1365.3). Total num frames: 20480. Throughput: 0: 189.3. Samples: 2840. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:43:07,614][04028] Avg episode reward: [(0, '3.452')] +[2025-05-25 11:43:12,614][04028] Fps is (10 sec: 3686.2, 60 sec: 1843.2, 300 sec: 1843.2). Total num frames: 36864. Throughput: 0: 470.6. Samples: 9412. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2025-05-25 11:43:12,618][04028] Avg episode reward: [(0, '3.907')] +[2025-05-25 11:43:13,069][04232] Updated weights for policy 0, policy_version 10 (0.0095) +[2025-05-25 11:43:17,613][04028] Fps is (10 sec: 3686.4, 60 sec: 2293.8, 300 sec: 2293.8). Total num frames: 57344. Throughput: 0: 594.1. Samples: 14852. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:43:17,618][04028] Avg episode reward: [(0, '4.578')] +[2025-05-25 11:43:22,273][04232] Updated weights for policy 0, policy_version 20 (0.0015) +[2025-05-25 11:43:22,613][04028] Fps is (10 sec: 4505.8, 60 sec: 2730.7, 300 sec: 2730.7). Total num frames: 81920. Throughput: 0: 614.9. Samples: 18446. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:43:22,617][04028] Avg episode reward: [(0, '4.436')] +[2025-05-25 11:43:27,613][04028] Fps is (10 sec: 4095.9, 60 sec: 2808.7, 300 sec: 2808.7). Total num frames: 98304. Throughput: 0: 708.2. Samples: 24788. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2025-05-25 11:43:27,615][04028] Avg episode reward: [(0, '4.263')] +[2025-05-25 11:43:32,614][04028] Fps is (10 sec: 3686.3, 60 sec: 2969.6, 300 sec: 2969.6). Total num frames: 118784. Throughput: 0: 752.9. Samples: 30116. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:43:32,615][04028] Avg episode reward: [(0, '4.378')] +[2025-05-25 11:43:32,624][04219] Saving new best policy, reward=4.378! +[2025-05-25 11:43:33,270][04232] Updated weights for policy 0, policy_version 30 (0.0034) +[2025-05-25 11:43:37,613][04028] Fps is (10 sec: 4096.1, 60 sec: 3094.8, 300 sec: 3094.8). Total num frames: 139264. Throughput: 0: 739.6. Samples: 33284. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:43:37,614][04028] Avg episode reward: [(0, '4.368')] +[2025-05-25 11:43:42,613][04028] Fps is (10 sec: 3686.5, 60 sec: 3113.0, 300 sec: 3113.0). Total num frames: 155648. Throughput: 0: 873.1. Samples: 39280. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) +[2025-05-25 11:43:42,617][04028] Avg episode reward: [(0, '4.297')] +[2025-05-25 11:43:44,226][04232] Updated weights for policy 0, policy_version 40 (0.0016) +[2025-05-25 11:43:47,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3202.3, 300 sec: 3202.3). Total num frames: 176128. Throughput: 0: 947.4. Samples: 44618. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) +[2025-05-25 11:43:47,618][04028] Avg episode reward: [(0, '4.368')] +[2025-05-25 11:43:52,613][04028] Fps is (10 sec: 4505.5, 60 sec: 3345.0, 300 sec: 3345.0). Total num frames: 200704. Throughput: 0: 1003.5. Samples: 47998. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:43:52,618][04028] Avg episode reward: [(0, '4.412')] +[2025-05-25 11:43:52,624][04219] Saving new best policy, reward=4.412! +[2025-05-25 11:43:53,435][04232] Updated weights for policy 0, policy_version 50 (0.0022) +[2025-05-25 11:43:57,613][04028] Fps is (10 sec: 3686.3, 60 sec: 3550.6, 300 sec: 3276.8). Total num frames: 212992. Throughput: 0: 983.7. Samples: 53678. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:43:57,619][04028] Avg episode reward: [(0, '4.313')] +[2025-05-25 11:44:02,613][04028] Fps is (10 sec: 3276.9, 60 sec: 3891.2, 300 sec: 3335.3). Total num frames: 233472. Throughput: 0: 987.4. Samples: 59286. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2025-05-25 11:44:02,618][04028] Avg episode reward: [(0, '4.455')] +[2025-05-25 11:44:02,638][04219] Saving new best policy, reward=4.455! +[2025-05-25 11:44:04,448][04232] Updated weights for policy 0, policy_version 60 (0.0021) +[2025-05-25 11:44:07,613][04028] Fps is (10 sec: 4505.8, 60 sec: 3959.5, 300 sec: 3440.6). Total num frames: 258048. Throughput: 0: 978.9. Samples: 62496. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2025-05-25 11:44:07,616][04028] Avg episode reward: [(0, '4.554')] +[2025-05-25 11:44:07,618][04219] Saving new best policy, reward=4.554! +[2025-05-25 11:44:12,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3379.2). Total num frames: 270336. Throughput: 0: 957.7. Samples: 67884. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:44:12,617][04028] Avg episode reward: [(0, '4.545')] +[2025-05-25 11:44:15,817][04232] Updated weights for policy 0, policy_version 70 (0.0012) +[2025-05-25 11:44:17,613][04028] Fps is (10 sec: 3276.8, 60 sec: 3891.2, 300 sec: 3421.4). Total num frames: 290816. Throughput: 0: 967.0. Samples: 73630. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:44:17,615][04028] Avg episode reward: [(0, '4.443')] +[2025-05-25 11:44:22,613][04028] Fps is (10 sec: 4505.5, 60 sec: 3891.2, 300 sec: 3504.4). Total num frames: 315392. Throughput: 0: 972.2. Samples: 77034. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:44:22,615][04028] Avg episode reward: [(0, '4.382')] +[2025-05-25 11:44:22,622][04219] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000077_315392.pth... +[2025-05-25 11:44:26,445][04232] Updated weights for policy 0, policy_version 80 (0.0020) +[2025-05-25 11:44:27,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3823.0, 300 sec: 3449.3). Total num frames: 327680. Throughput: 0: 954.6. Samples: 82236. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:44:27,616][04028] Avg episode reward: [(0, '4.574')] +[2025-05-25 11:44:27,619][04219] Saving new best policy, reward=4.574! +[2025-05-25 11:44:32,616][04028] Fps is (10 sec: 3685.4, 60 sec: 3891.0, 300 sec: 3522.5). Total num frames: 352256. Throughput: 0: 968.0. Samples: 88182. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:44:32,623][04028] Avg episode reward: [(0, '4.523')] +[2025-05-25 11:44:36,354][04232] Updated weights for policy 0, policy_version 90 (0.0021) +[2025-05-25 11:44:37,614][04028] Fps is (10 sec: 4505.2, 60 sec: 3891.1, 300 sec: 3549.8). Total num frames: 372736. Throughput: 0: 963.9. Samples: 91374. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:44:37,618][04028] Avg episode reward: [(0, '4.542')] +[2025-05-25 11:44:42,613][04028] Fps is (10 sec: 3277.7, 60 sec: 3822.9, 300 sec: 3500.2). Total num frames: 385024. Throughput: 0: 952.7. Samples: 96548. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:44:42,621][04028] Avg episode reward: [(0, '4.706')] +[2025-05-25 11:44:42,630][04219] Saving new best policy, reward=4.706! +[2025-05-25 11:44:47,345][04232] Updated weights for policy 0, policy_version 100 (0.0018) +[2025-05-25 11:44:47,613][04028] Fps is (10 sec: 3686.7, 60 sec: 3891.2, 300 sec: 3561.7). Total num frames: 409600. Throughput: 0: 963.1. Samples: 102626. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:44:47,614][04028] Avg episode reward: [(0, '4.518')] +[2025-05-25 11:44:52,613][04028] Fps is (10 sec: 4505.6, 60 sec: 3822.9, 300 sec: 3584.0). Total num frames: 430080. Throughput: 0: 966.7. Samples: 105996. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:44:52,616][04028] Avg episode reward: [(0, '4.442')] +[2025-05-25 11:44:57,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3571.7). Total num frames: 446464. Throughput: 0: 958.0. Samples: 110992. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2025-05-25 11:44:57,618][04028] Avg episode reward: [(0, '4.430')] +[2025-05-25 11:44:58,411][04232] Updated weights for policy 0, policy_version 110 (0.0014) +[2025-05-25 11:45:02,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3591.9). Total num frames: 466944. Throughput: 0: 969.4. Samples: 117252. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2025-05-25 11:45:02,618][04028] Avg episode reward: [(0, '4.373')] +[2025-05-25 11:45:07,613][04028] Fps is (10 sec: 4095.9, 60 sec: 3822.9, 300 sec: 3610.5). Total num frames: 487424. Throughput: 0: 969.0. Samples: 120638. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:45:07,615][04028] Avg episode reward: [(0, '4.317')] +[2025-05-25 11:45:07,909][04232] Updated weights for policy 0, policy_version 120 (0.0015) +[2025-05-25 11:45:12,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3598.6). Total num frames: 503808. Throughput: 0: 960.5. Samples: 125458. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2025-05-25 11:45:12,618][04028] Avg episode reward: [(0, '4.602')] +[2025-05-25 11:45:17,613][04028] Fps is (10 sec: 3686.5, 60 sec: 3891.2, 300 sec: 3615.8). Total num frames: 524288. Throughput: 0: 969.8. Samples: 131818. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2025-05-25 11:45:17,621][04028] Avg episode reward: [(0, '4.642')] +[2025-05-25 11:45:18,861][04232] Updated weights for policy 0, policy_version 130 (0.0013) +[2025-05-25 11:45:22,613][04028] Fps is (10 sec: 4095.9, 60 sec: 3822.9, 300 sec: 3631.8). Total num frames: 544768. Throughput: 0: 974.4. Samples: 135222. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:45:22,623][04028] Avg episode reward: [(0, '4.647')] +[2025-05-25 11:45:27,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3620.3). Total num frames: 561152. Throughput: 0: 963.2. Samples: 139894. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2025-05-25 11:45:27,618][04028] Avg episode reward: [(0, '4.704')] +[2025-05-25 11:45:29,829][04232] Updated weights for policy 0, policy_version 140 (0.0017) +[2025-05-25 11:45:32,613][04028] Fps is (10 sec: 4096.1, 60 sec: 3891.4, 300 sec: 3660.8). Total num frames: 585728. Throughput: 0: 976.0. Samples: 146548. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2025-05-25 11:45:32,615][04028] Avg episode reward: [(0, '4.500')] +[2025-05-25 11:45:37,613][04028] Fps is (10 sec: 4505.6, 60 sec: 3891.2, 300 sec: 3674.0). Total num frames: 606208. Throughput: 0: 977.2. Samples: 149972. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:45:37,614][04028] Avg episode reward: [(0, '4.477')] +[2025-05-25 11:45:40,646][04232] Updated weights for policy 0, policy_version 150 (0.0016) +[2025-05-25 11:45:42,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 3662.3). Total num frames: 622592. Throughput: 0: 966.4. Samples: 154482. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:45:42,615][04028] Avg episode reward: [(0, '4.567')] +[2025-05-25 11:45:47,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3674.7). Total num frames: 643072. Throughput: 0: 979.6. Samples: 161336. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2025-05-25 11:45:47,614][04028] Avg episode reward: [(0, '4.519')] +[2025-05-25 11:45:49,925][04232] Updated weights for policy 0, policy_version 160 (0.0015) +[2025-05-25 11:45:52,619][04028] Fps is (10 sec: 4093.5, 60 sec: 3890.8, 300 sec: 3686.3). Total num frames: 663552. Throughput: 0: 978.3. Samples: 164668. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2025-05-25 11:45:52,621][04028] Avg episode reward: [(0, '4.434')] +[2025-05-25 11:45:57,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3675.3). Total num frames: 679936. Throughput: 0: 976.1. Samples: 169384. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:45:57,618][04028] Avg episode reward: [(0, '4.460')] +[2025-05-25 11:46:00,842][04232] Updated weights for policy 0, policy_version 170 (0.0028) +[2025-05-25 11:46:02,613][04028] Fps is (10 sec: 4098.6, 60 sec: 3959.5, 300 sec: 3708.0). Total num frames: 704512. Throughput: 0: 984.0. Samples: 176096. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:46:02,626][04028] Avg episode reward: [(0, '4.525')] +[2025-05-25 11:46:07,613][04028] Fps is (10 sec: 4095.9, 60 sec: 3891.2, 300 sec: 3696.9). Total num frames: 720896. Throughput: 0: 983.4. Samples: 179476. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:46:07,615][04028] Avg episode reward: [(0, '4.574')] +[2025-05-25 11:46:11,938][04232] Updated weights for policy 0, policy_version 180 (0.0026) +[2025-05-25 11:46:12,613][04028] Fps is (10 sec: 3276.8, 60 sec: 3891.2, 300 sec: 3686.4). Total num frames: 737280. Throughput: 0: 979.1. Samples: 183952. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:46:12,617][04028] Avg episode reward: [(0, '4.585')] +[2025-05-25 11:46:17,613][04028] Fps is (10 sec: 4096.1, 60 sec: 3959.5, 300 sec: 3716.4). Total num frames: 761856. Throughput: 0: 982.2. Samples: 190748. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:46:17,618][04028] Avg episode reward: [(0, '4.867')] +[2025-05-25 11:46:17,622][04219] Saving new best policy, reward=4.867! +[2025-05-25 11:46:21,871][04232] Updated weights for policy 0, policy_version 190 (0.0016) +[2025-05-25 11:46:22,613][04028] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3705.9). Total num frames: 778240. Throughput: 0: 980.1. Samples: 194078. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2025-05-25 11:46:22,615][04028] Avg episode reward: [(0, '5.001')] +[2025-05-25 11:46:22,627][04219] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000190_778240.pth... +[2025-05-25 11:46:22,771][04219] Saving new best policy, reward=5.001! +[2025-05-25 11:46:27,613][04028] Fps is (10 sec: 3276.8, 60 sec: 3891.2, 300 sec: 3695.9). Total num frames: 794624. Throughput: 0: 980.6. Samples: 198608. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2025-05-25 11:46:27,614][04028] Avg episode reward: [(0, '5.089')] +[2025-05-25 11:46:27,627][04219] Saving new best policy, reward=5.089! +[2025-05-25 11:46:31,807][04232] Updated weights for policy 0, policy_version 200 (0.0027) +[2025-05-25 11:46:32,613][04028] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3723.6). Total num frames: 819200. Throughput: 0: 984.9. Samples: 205656. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2025-05-25 11:46:32,617][04028] Avg episode reward: [(0, '4.849')] +[2025-05-25 11:46:37,613][04028] Fps is (10 sec: 4505.6, 60 sec: 3891.2, 300 sec: 3731.9). Total num frames: 839680. Throughput: 0: 988.6. Samples: 209148. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:46:37,617][04028] Avg episode reward: [(0, '4.834')] +[2025-05-25 11:46:42,484][04232] Updated weights for policy 0, policy_version 210 (0.0012) +[2025-05-25 11:46:42,613][04028] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3739.8). Total num frames: 860160. Throughput: 0: 992.1. Samples: 214030. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2025-05-25 11:46:42,614][04028] Avg episode reward: [(0, '4.839')] +[2025-05-25 11:46:47,617][04028] Fps is (10 sec: 4094.2, 60 sec: 3959.2, 300 sec: 3747.3). Total num frames: 880640. Throughput: 0: 1001.7. Samples: 221178. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:46:47,619][04028] Avg episode reward: [(0, '4.739')] +[2025-05-25 11:46:51,789][04232] Updated weights for policy 0, policy_version 220 (0.0015) +[2025-05-25 11:46:52,613][04028] Fps is (10 sec: 4096.0, 60 sec: 3959.9, 300 sec: 3754.7). Total num frames: 901120. Throughput: 0: 1002.9. Samples: 224608. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) +[2025-05-25 11:46:52,616][04028] Avg episode reward: [(0, '4.790')] +[2025-05-25 11:46:57,613][04028] Fps is (10 sec: 4097.8, 60 sec: 4027.7, 300 sec: 3761.6). Total num frames: 921600. Throughput: 0: 1020.0. Samples: 229850. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:46:57,616][04028] Avg episode reward: [(0, '4.899')] +[2025-05-25 11:47:01,245][04232] Updated weights for policy 0, policy_version 230 (0.0018) +[2025-05-25 11:47:02,613][04028] Fps is (10 sec: 4505.6, 60 sec: 4027.7, 300 sec: 3784.7). Total num frames: 946176. Throughput: 0: 1029.2. Samples: 237062. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:47:02,618][04028] Avg episode reward: [(0, '4.788')] +[2025-05-25 11:47:07,613][04028] Fps is (10 sec: 4096.0, 60 sec: 4027.8, 300 sec: 3774.7). Total num frames: 962560. Throughput: 0: 1024.5. Samples: 240182. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:47:07,615][04028] Avg episode reward: [(0, '4.741')] +[2025-05-25 11:47:11,854][04232] Updated weights for policy 0, policy_version 240 (0.0016) +[2025-05-25 11:47:12,613][04028] Fps is (10 sec: 3686.4, 60 sec: 4096.0, 300 sec: 3780.9). Total num frames: 983040. Throughput: 0: 1042.8. Samples: 245534. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:47:12,617][04028] Avg episode reward: [(0, '4.842')] +[2025-05-25 11:47:17,613][04028] Fps is (10 sec: 4505.6, 60 sec: 4096.0, 300 sec: 3802.3). Total num frames: 1007616. Throughput: 0: 1047.0. Samples: 252772. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:47:17,617][04028] Avg episode reward: [(0, '4.876')] +[2025-05-25 11:47:21,245][04232] Updated weights for policy 0, policy_version 250 (0.0034) +[2025-05-25 11:47:22,613][04028] Fps is (10 sec: 4096.1, 60 sec: 4096.0, 300 sec: 3792.6). Total num frames: 1024000. Throughput: 0: 1035.3. Samples: 255738. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:47:22,614][04028] Avg episode reward: [(0, '4.902')] +[2025-05-25 11:47:27,613][04028] Fps is (10 sec: 4096.0, 60 sec: 4232.5, 300 sec: 3813.0). Total num frames: 1048576. Throughput: 0: 1056.2. Samples: 261560. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:47:27,614][04028] Avg episode reward: [(0, '5.271')] +[2025-05-25 11:47:27,618][04219] Saving new best policy, reward=5.271! +[2025-05-25 11:47:30,577][04232] Updated weights for policy 0, policy_version 260 (0.0026) +[2025-05-25 11:47:32,613][04028] Fps is (10 sec: 4915.2, 60 sec: 4232.5, 300 sec: 3832.7). Total num frames: 1073152. Throughput: 0: 1056.9. Samples: 268732. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:47:32,615][04028] Avg episode reward: [(0, '4.930')] +[2025-05-25 11:47:37,613][04028] Fps is (10 sec: 4095.9, 60 sec: 4164.3, 300 sec: 3822.9). Total num frames: 1089536. Throughput: 0: 1037.2. Samples: 271280. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2025-05-25 11:47:37,618][04028] Avg episode reward: [(0, '4.923')] +[2025-05-25 11:47:40,912][04232] Updated weights for policy 0, policy_version 270 (0.0020) +[2025-05-25 11:47:42,613][04028] Fps is (10 sec: 3686.4, 60 sec: 4164.3, 300 sec: 3827.6). Total num frames: 1110016. Throughput: 0: 1052.8. Samples: 277226. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:47:42,616][04028] Avg episode reward: [(0, '5.132')] +[2025-05-25 11:47:47,613][04028] Fps is (10 sec: 4505.7, 60 sec: 4232.8, 300 sec: 3846.1). Total num frames: 1134592. Throughput: 0: 1054.2. Samples: 284502. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:47:47,616][04028] Avg episode reward: [(0, '5.711')] +[2025-05-25 11:47:47,696][04219] Saving new best policy, reward=5.711! +[2025-05-25 11:47:50,610][04232] Updated weights for policy 0, policy_version 280 (0.0021) +[2025-05-25 11:47:52,613][04028] Fps is (10 sec: 4096.0, 60 sec: 4164.3, 300 sec: 3901.8). Total num frames: 1150976. Throughput: 0: 1037.5. Samples: 286870. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2025-05-25 11:47:52,617][04028] Avg episode reward: [(0, '5.860')] +[2025-05-25 11:47:52,625][04219] Saving new best policy, reward=5.860! +[2025-05-25 11:47:57,613][04028] Fps is (10 sec: 4096.0, 60 sec: 4232.5, 300 sec: 3984.9). Total num frames: 1175552. Throughput: 0: 1049.9. Samples: 292778. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2025-05-25 11:47:57,617][04028] Avg episode reward: [(0, '5.770')] +[2025-05-25 11:48:00,381][04232] Updated weights for policy 0, policy_version 290 (0.0012) +[2025-05-25 11:48:02,613][04028] Fps is (10 sec: 4505.5, 60 sec: 4164.3, 300 sec: 3984.9). Total num frames: 1196032. Throughput: 0: 1037.1. Samples: 299444. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2025-05-25 11:48:02,615][04028] Avg episode reward: [(0, '6.137')] +[2025-05-25 11:48:02,623][04219] Saving new best policy, reward=6.137! +[2025-05-25 11:48:07,613][04028] Fps is (10 sec: 3276.8, 60 sec: 4096.0, 300 sec: 3971.0). Total num frames: 1208320. Throughput: 0: 1015.0. Samples: 301414. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:48:07,617][04028] Avg episode reward: [(0, '6.022')] +[2025-05-25 11:48:11,328][04232] Updated weights for policy 0, policy_version 300 (0.0025) +[2025-05-25 11:48:12,620][04028] Fps is (10 sec: 3683.8, 60 sec: 4163.8, 300 sec: 3984.8). Total num frames: 1232896. Throughput: 0: 1022.6. Samples: 307586. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:48:12,623][04028] Avg episode reward: [(0, '5.822')] +[2025-05-25 11:48:17,613][04028] Fps is (10 sec: 4505.5, 60 sec: 4096.0, 300 sec: 3971.0). Total num frames: 1253376. Throughput: 0: 1002.6. Samples: 313850. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2025-05-25 11:48:17,618][04028] Avg episode reward: [(0, '6.436')] +[2025-05-25 11:48:17,621][04219] Saving new best policy, reward=6.436! +[2025-05-25 11:48:22,149][04232] Updated weights for policy 0, policy_version 310 (0.0018) +[2025-05-25 11:48:22,613][04028] Fps is (10 sec: 3689.1, 60 sec: 4096.0, 300 sec: 3971.0). Total num frames: 1269760. Throughput: 0: 990.8. Samples: 315866. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2025-05-25 11:48:22,615][04028] Avg episode reward: [(0, '6.742')] +[2025-05-25 11:48:22,620][04219] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000310_1269760.pth... +[2025-05-25 11:48:22,741][04219] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000077_315392.pth +[2025-05-25 11:48:22,753][04219] Saving new best policy, reward=6.742! +[2025-05-25 11:48:27,613][04028] Fps is (10 sec: 3686.5, 60 sec: 4027.7, 300 sec: 3971.0). Total num frames: 1290240. Throughput: 0: 1000.0. Samples: 322228. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:48:27,618][04028] Avg episode reward: [(0, '6.881')] +[2025-05-25 11:48:27,629][04219] Saving new best policy, reward=6.881! +[2025-05-25 11:48:31,896][04232] Updated weights for policy 0, policy_version 320 (0.0016) +[2025-05-25 11:48:32,615][04028] Fps is (10 sec: 4095.3, 60 sec: 3959.3, 300 sec: 3971.0). Total num frames: 1310720. Throughput: 0: 973.1. Samples: 328292. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:48:32,616][04028] Avg episode reward: [(0, '6.953')] +[2025-05-25 11:48:32,625][04219] Saving new best policy, reward=6.953! +[2025-05-25 11:48:37,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 3971.0). Total num frames: 1327104. Throughput: 0: 965.9. Samples: 330334. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2025-05-25 11:48:37,617][04028] Avg episode reward: [(0, '7.059')] +[2025-05-25 11:48:37,621][04219] Saving new best policy, reward=7.059! +[2025-05-25 11:48:42,551][04232] Updated weights for policy 0, policy_version 330 (0.0015) +[2025-05-25 11:48:42,613][04028] Fps is (10 sec: 4096.7, 60 sec: 4027.7, 300 sec: 3984.9). Total num frames: 1351680. Throughput: 0: 980.8. Samples: 336914. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:48:42,618][04028] Avg episode reward: [(0, '7.648')] +[2025-05-25 11:48:42,627][04219] Saving new best policy, reward=7.648! +[2025-05-25 11:48:47,614][04028] Fps is (10 sec: 4095.5, 60 sec: 3891.1, 300 sec: 3957.1). Total num frames: 1368064. Throughput: 0: 963.6. Samples: 342806. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:48:47,616][04028] Avg episode reward: [(0, '7.719')] +[2025-05-25 11:48:47,617][04219] Saving new best policy, reward=7.719! +[2025-05-25 11:48:52,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 3984.9). Total num frames: 1388544. Throughput: 0: 969.6. Samples: 345046. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2025-05-25 11:48:52,616][04028] Avg episode reward: [(0, '7.574')] +[2025-05-25 11:48:53,364][04232] Updated weights for policy 0, policy_version 340 (0.0023) +[2025-05-25 11:48:57,613][04028] Fps is (10 sec: 4096.5, 60 sec: 3891.2, 300 sec: 3984.9). Total num frames: 1409024. Throughput: 0: 984.1. Samples: 351864. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2025-05-25 11:48:57,617][04028] Avg episode reward: [(0, '6.985')] +[2025-05-25 11:49:02,615][04028] Fps is (10 sec: 3685.7, 60 sec: 3822.8, 300 sec: 3957.1). Total num frames: 1425408. Throughput: 0: 969.3. Samples: 357472. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2025-05-25 11:49:02,619][04028] Avg episode reward: [(0, '6.958')] +[2025-05-25 11:49:04,060][04232] Updated weights for policy 0, policy_version 350 (0.0017) +[2025-05-25 11:49:07,613][04028] Fps is (10 sec: 4096.0, 60 sec: 4027.7, 300 sec: 3998.8). Total num frames: 1449984. Throughput: 0: 982.8. Samples: 360090. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:49:07,617][04028] Avg episode reward: [(0, '7.038')] +[2025-05-25 11:49:12,613][04028] Fps is (10 sec: 4506.4, 60 sec: 3960.0, 300 sec: 3998.8). Total num frames: 1470464. Throughput: 0: 991.0. Samples: 366824. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2025-05-25 11:49:12,618][04028] Avg episode reward: [(0, '6.616')] +[2025-05-25 11:49:13,218][04232] Updated weights for policy 0, policy_version 360 (0.0022) +[2025-05-25 11:49:17,616][04028] Fps is (10 sec: 3685.1, 60 sec: 3891.0, 300 sec: 3971.0). Total num frames: 1486848. Throughput: 0: 972.5. Samples: 372058. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:49:17,618][04028] Avg episode reward: [(0, '6.853')] +[2025-05-25 11:49:22,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 3998.8). Total num frames: 1507328. Throughput: 0: 991.8. Samples: 374966. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2025-05-25 11:49:22,615][04028] Avg episode reward: [(0, '7.530')] +[2025-05-25 11:49:24,098][04232] Updated weights for policy 0, policy_version 370 (0.0015) +[2025-05-25 11:49:27,613][04028] Fps is (10 sec: 4507.1, 60 sec: 4027.7, 300 sec: 3998.8). Total num frames: 1531904. Throughput: 0: 996.5. Samples: 381756. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:49:27,614][04028] Avg episode reward: [(0, '8.314')] +[2025-05-25 11:49:27,615][04219] Saving new best policy, reward=8.314! +[2025-05-25 11:49:32,613][04028] Fps is (10 sec: 3686.3, 60 sec: 3891.3, 300 sec: 3971.0). Total num frames: 1544192. Throughput: 0: 977.2. Samples: 386778. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2025-05-25 11:49:32,615][04028] Avg episode reward: [(0, '8.416')] +[2025-05-25 11:49:32,624][04219] Saving new best policy, reward=8.416! +[2025-05-25 11:49:34,929][04232] Updated weights for policy 0, policy_version 380 (0.0022) +[2025-05-25 11:49:37,613][04028] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 4012.7). Total num frames: 1568768. Throughput: 0: 996.8. Samples: 389900. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:49:37,619][04028] Avg episode reward: [(0, '8.217')] +[2025-05-25 11:49:42,615][04028] Fps is (10 sec: 4504.7, 60 sec: 3959.3, 300 sec: 3998.8). Total num frames: 1589248. Throughput: 0: 995.0. Samples: 396640. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:49:42,617][04028] Avg episode reward: [(0, '8.084')] +[2025-05-25 11:49:44,991][04232] Updated weights for policy 0, policy_version 390 (0.0019) +[2025-05-25 11:49:47,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3959.6, 300 sec: 3984.9). Total num frames: 1605632. Throughput: 0: 978.2. Samples: 401488. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:49:47,614][04028] Avg episode reward: [(0, '7.761')] +[2025-05-25 11:49:52,613][04028] Fps is (10 sec: 3687.2, 60 sec: 3959.5, 300 sec: 3998.8). Total num frames: 1626112. Throughput: 0: 996.5. Samples: 404932. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:49:52,618][04028] Avg episode reward: [(0, '8.324')] +[2025-05-25 11:49:54,561][04232] Updated weights for policy 0, policy_version 400 (0.0019) +[2025-05-25 11:49:57,613][04028] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3998.8). Total num frames: 1646592. Throughput: 0: 998.4. Samples: 411750. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2025-05-25 11:49:57,614][04028] Avg episode reward: [(0, '8.583')] +[2025-05-25 11:49:57,617][04219] Saving new best policy, reward=8.583! +[2025-05-25 11:50:02,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3959.6, 300 sec: 3984.9). Total num frames: 1662976. Throughput: 0: 988.7. Samples: 416546. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2025-05-25 11:50:02,619][04028] Avg episode reward: [(0, '8.421')] +[2025-05-25 11:50:05,536][04232] Updated weights for policy 0, policy_version 410 (0.0021) +[2025-05-25 11:50:07,613][04028] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 4012.7). Total num frames: 1687552. Throughput: 0: 1000.7. Samples: 419996. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:50:07,619][04028] Avg episode reward: [(0, '8.386')] +[2025-05-25 11:50:12,613][04028] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 4012.7). Total num frames: 1708032. Throughput: 0: 997.6. Samples: 426648. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:50:12,615][04028] Avg episode reward: [(0, '8.811')] +[2025-05-25 11:50:12,620][04219] Saving new best policy, reward=8.811! +[2025-05-25 11:50:16,402][04232] Updated weights for policy 0, policy_version 420 (0.0014) +[2025-05-25 11:50:17,613][04028] Fps is (10 sec: 3686.3, 60 sec: 3959.7, 300 sec: 3998.8). Total num frames: 1724416. Throughput: 0: 993.2. Samples: 431474. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:50:17,615][04028] Avg episode reward: [(0, '9.224')] +[2025-05-25 11:50:17,623][04219] Saving new best policy, reward=9.224! +[2025-05-25 11:50:22,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 4012.7). Total num frames: 1744896. Throughput: 0: 996.4. Samples: 434740. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:50:22,615][04028] Avg episode reward: [(0, '8.317')] +[2025-05-25 11:50:22,620][04219] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000426_1744896.pth... +[2025-05-25 11:50:22,744][04219] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000190_778240.pth +[2025-05-25 11:50:25,723][04232] Updated weights for policy 0, policy_version 430 (0.0025) +[2025-05-25 11:50:27,614][04028] Fps is (10 sec: 4095.7, 60 sec: 3891.1, 300 sec: 3998.8). Total num frames: 1765376. Throughput: 0: 988.5. Samples: 441122. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:50:27,619][04028] Avg episode reward: [(0, '8.733')] +[2025-05-25 11:50:32,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 3984.9). Total num frames: 1781760. Throughput: 0: 993.4. Samples: 446190. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:50:32,619][04028] Avg episode reward: [(0, '8.769')] +[2025-05-25 11:50:36,469][04232] Updated weights for policy 0, policy_version 440 (0.0014) +[2025-05-25 11:50:37,613][04028] Fps is (10 sec: 4096.4, 60 sec: 3959.5, 300 sec: 4012.7). Total num frames: 1806336. Throughput: 0: 992.6. Samples: 449598. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:50:37,618][04028] Avg episode reward: [(0, '9.565')] +[2025-05-25 11:50:37,623][04219] Saving new best policy, reward=9.565! +[2025-05-25 11:50:42,617][04028] Fps is (10 sec: 4094.4, 60 sec: 3891.1, 300 sec: 3998.8). Total num frames: 1822720. Throughput: 0: 973.2. Samples: 455550. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2025-05-25 11:50:42,619][04028] Avg episode reward: [(0, '9.335')] +[2025-05-25 11:50:47,385][04232] Updated weights for policy 0, policy_version 450 (0.0013) +[2025-05-25 11:50:47,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 3998.9). Total num frames: 1843200. Throughput: 0: 988.6. Samples: 461032. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:50:47,614][04028] Avg episode reward: [(0, '9.991')] +[2025-05-25 11:50:47,620][04219] Saving new best policy, reward=9.991! +[2025-05-25 11:50:52,613][04028] Fps is (10 sec: 4097.6, 60 sec: 3959.5, 300 sec: 4012.7). Total num frames: 1863680. Throughput: 0: 985.7. Samples: 464354. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:50:52,625][04028] Avg episode reward: [(0, '10.093')] +[2025-05-25 11:50:52,635][04219] Saving new best policy, reward=10.093! +[2025-05-25 11:50:57,613][04028] Fps is (10 sec: 3686.3, 60 sec: 3891.2, 300 sec: 3984.9). Total num frames: 1880064. Throughput: 0: 966.1. Samples: 470122. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:50:57,615][04028] Avg episode reward: [(0, '10.742')] +[2025-05-25 11:50:57,621][04219] Saving new best policy, reward=10.742! +[2025-05-25 11:50:58,495][04232] Updated weights for policy 0, policy_version 460 (0.0018) +[2025-05-25 11:51:02,613][04028] Fps is (10 sec: 3686.5, 60 sec: 3959.5, 300 sec: 3998.8). Total num frames: 1900544. Throughput: 0: 986.3. Samples: 475856. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2025-05-25 11:51:02,615][04028] Avg episode reward: [(0, '11.197')] +[2025-05-25 11:51:02,620][04219] Saving new best policy, reward=11.197! +[2025-05-25 11:51:07,514][04232] Updated weights for policy 0, policy_version 470 (0.0014) +[2025-05-25 11:51:07,613][04028] Fps is (10 sec: 4505.8, 60 sec: 3959.5, 300 sec: 4026.6). Total num frames: 1925120. Throughput: 0: 987.6. Samples: 479182. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2025-05-25 11:51:07,614][04028] Avg episode reward: [(0, '11.619')] +[2025-05-25 11:51:07,617][04219] Saving new best policy, reward=11.619! +[2025-05-25 11:51:12,613][04028] Fps is (10 sec: 3686.5, 60 sec: 3822.9, 300 sec: 3984.9). Total num frames: 1937408. Throughput: 0: 967.1. Samples: 484642. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:51:12,615][04028] Avg episode reward: [(0, '11.062')] +[2025-05-25 11:51:17,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 4012.7). Total num frames: 1961984. Throughput: 0: 986.6. Samples: 490588. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:51:17,614][04028] Avg episode reward: [(0, '11.399')] +[2025-05-25 11:51:18,534][04232] Updated weights for policy 0, policy_version 480 (0.0017) +[2025-05-25 11:51:22,613][04028] Fps is (10 sec: 4505.5, 60 sec: 3959.5, 300 sec: 4026.6). Total num frames: 1982464. Throughput: 0: 987.8. Samples: 494050. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:51:22,619][04028] Avg episode reward: [(0, '11.212')] +[2025-05-25 11:51:27,613][04028] Fps is (10 sec: 3686.3, 60 sec: 3891.2, 300 sec: 3998.8). Total num frames: 1998848. Throughput: 0: 972.2. Samples: 499294. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) +[2025-05-25 11:51:27,619][04028] Avg episode reward: [(0, '12.153')] +[2025-05-25 11:51:27,623][04219] Saving new best policy, reward=12.153! +[2025-05-25 11:51:29,315][04232] Updated weights for policy 0, policy_version 490 (0.0017) +[2025-05-25 11:51:32,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 3998.8). Total num frames: 2019328. Throughput: 0: 988.6. Samples: 505518. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:51:32,619][04028] Avg episode reward: [(0, '11.715')] +[2025-05-25 11:51:37,613][04028] Fps is (10 sec: 4505.7, 60 sec: 3959.5, 300 sec: 4012.7). Total num frames: 2043904. Throughput: 0: 990.5. Samples: 508926. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:51:37,615][04028] Avg episode reward: [(0, '12.789')] +[2025-05-25 11:51:37,619][04219] Saving new best policy, reward=12.789! +[2025-05-25 11:51:38,972][04232] Updated weights for policy 0, policy_version 500 (0.0020) +[2025-05-25 11:51:42,613][04028] Fps is (10 sec: 3686.3, 60 sec: 3891.4, 300 sec: 3985.0). Total num frames: 2056192. Throughput: 0: 971.9. Samples: 513856. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2025-05-25 11:51:42,618][04028] Avg episode reward: [(0, '13.283')] +[2025-05-25 11:51:42,625][04219] Saving new best policy, reward=13.283! +[2025-05-25 11:51:47,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 3998.8). Total num frames: 2080768. Throughput: 0: 985.6. Samples: 520208. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2025-05-25 11:51:47,614][04028] Avg episode reward: [(0, '13.210')] +[2025-05-25 11:51:49,369][04232] Updated weights for policy 0, policy_version 510 (0.0022) +[2025-05-25 11:51:52,613][04028] Fps is (10 sec: 4505.7, 60 sec: 3959.5, 300 sec: 3998.8). Total num frames: 2101248. Throughput: 0: 986.3. Samples: 523564. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2025-05-25 11:51:52,617][04028] Avg episode reward: [(0, '13.287')] +[2025-05-25 11:51:52,625][04219] Saving new best policy, reward=13.287! +[2025-05-25 11:51:57,613][04028] Fps is (10 sec: 3686.3, 60 sec: 3959.5, 300 sec: 3971.0). Total num frames: 2117632. Throughput: 0: 970.9. Samples: 528332. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:51:57,615][04028] Avg episode reward: [(0, '12.539')] +[2025-05-25 11:52:00,240][04232] Updated weights for policy 0, policy_version 520 (0.0018) +[2025-05-25 11:52:02,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 3984.9). Total num frames: 2138112. Throughput: 0: 988.6. Samples: 535076. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2025-05-25 11:52:02,615][04028] Avg episode reward: [(0, '12.582')] +[2025-05-25 11:52:07,614][04028] Fps is (10 sec: 4096.1, 60 sec: 3891.2, 300 sec: 3984.9). Total num frames: 2158592. Throughput: 0: 987.1. Samples: 538468. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:52:07,616][04028] Avg episode reward: [(0, '12.032')] +[2025-05-25 11:52:11,137][04232] Updated weights for policy 0, policy_version 530 (0.0017) +[2025-05-25 11:52:12,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 3957.2). Total num frames: 2174976. Throughput: 0: 975.1. Samples: 543174. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2025-05-25 11:52:12,618][04028] Avg episode reward: [(0, '12.640')] +[2025-05-25 11:52:17,613][04028] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3984.9). Total num frames: 2199552. Throughput: 0: 987.1. Samples: 549938. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2025-05-25 11:52:17,614][04028] Avg episode reward: [(0, '13.216')] +[2025-05-25 11:52:20,218][04232] Updated weights for policy 0, policy_version 540 (0.0031) +[2025-05-25 11:52:22,613][04028] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3957.1). Total num frames: 2215936. Throughput: 0: 988.2. Samples: 553396. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2025-05-25 11:52:22,618][04028] Avg episode reward: [(0, '13.691')] +[2025-05-25 11:52:22,631][04219] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000541_2215936.pth... +[2025-05-25 11:52:22,801][04219] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000310_1269760.pth +[2025-05-25 11:52:22,819][04219] Saving new best policy, reward=13.691! +[2025-05-25 11:52:27,613][04028] Fps is (10 sec: 3276.8, 60 sec: 3891.2, 300 sec: 3929.4). Total num frames: 2232320. Throughput: 0: 981.5. Samples: 558024. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2025-05-25 11:52:27,620][04028] Avg episode reward: [(0, '14.232')] +[2025-05-25 11:52:27,642][04219] Saving new best policy, reward=14.232! +[2025-05-25 11:52:31,401][04232] Updated weights for policy 0, policy_version 550 (0.0029) +[2025-05-25 11:52:32,613][04028] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3957.2). Total num frames: 2256896. Throughput: 0: 990.3. Samples: 564772. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2025-05-25 11:52:32,618][04028] Avg episode reward: [(0, '14.832')] +[2025-05-25 11:52:32,626][04219] Saving new best policy, reward=14.832! +[2025-05-25 11:52:37,613][04028] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3943.3). Total num frames: 2273280. Throughput: 0: 988.4. Samples: 568042. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2025-05-25 11:52:37,618][04028] Avg episode reward: [(0, '14.653')] +[2025-05-25 11:52:42,281][04232] Updated weights for policy 0, policy_version 560 (0.0039) +[2025-05-25 11:52:42,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 3929.4). Total num frames: 2293760. Throughput: 0: 984.5. Samples: 572634. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2025-05-25 11:52:42,614][04028] Avg episode reward: [(0, '14.902')] +[2025-05-25 11:52:42,628][04219] Saving new best policy, reward=14.902! +[2025-05-25 11:52:47,613][04028] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3943.3). Total num frames: 2314240. Throughput: 0: 983.2. Samples: 579320. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2025-05-25 11:52:47,618][04028] Avg episode reward: [(0, '15.011')] +[2025-05-25 11:52:47,621][04219] Saving new best policy, reward=15.011! +[2025-05-25 11:52:52,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3915.5). Total num frames: 2330624. Throughput: 0: 977.6. Samples: 582462. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2025-05-25 11:52:52,617][04028] Avg episode reward: [(0, '15.688')] +[2025-05-25 11:52:52,624][04219] Saving new best policy, reward=15.688! +[2025-05-25 11:52:52,945][04232] Updated weights for policy 0, policy_version 570 (0.0017) +[2025-05-25 11:52:57,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3915.5). Total num frames: 2351104. Throughput: 0: 983.3. Samples: 587422. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2025-05-25 11:52:57,614][04028] Avg episode reward: [(0, '14.825')] +[2025-05-25 11:53:02,409][04232] Updated weights for policy 0, policy_version 580 (0.0022) +[2025-05-25 11:53:02,613][04028] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 3957.2). Total num frames: 2375680. Throughput: 0: 984.0. Samples: 594220. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2025-05-25 11:53:02,616][04028] Avg episode reward: [(0, '15.404')] +[2025-05-25 11:53:07,613][04028] Fps is (10 sec: 4095.8, 60 sec: 3891.2, 300 sec: 3929.5). Total num frames: 2392064. Throughput: 0: 971.9. Samples: 597132. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:53:07,620][04028] Avg episode reward: [(0, '14.219')] +[2025-05-25 11:53:12,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 3929.4). Total num frames: 2412544. Throughput: 0: 983.3. Samples: 602274. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:53:12,619][04028] Avg episode reward: [(0, '14.329')] +[2025-05-25 11:53:13,232][04232] Updated weights for policy 0, policy_version 590 (0.0025) +[2025-05-25 11:53:17,613][04028] Fps is (10 sec: 4096.1, 60 sec: 3891.2, 300 sec: 3943.3). Total num frames: 2433024. Throughput: 0: 984.7. Samples: 609084. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2025-05-25 11:53:17,614][04028] Avg episode reward: [(0, '14.444')] +[2025-05-25 11:53:22,614][04028] Fps is (10 sec: 3686.1, 60 sec: 3891.2, 300 sec: 3929.4). Total num frames: 2449408. Throughput: 0: 969.3. Samples: 611662. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2025-05-25 11:53:22,615][04028] Avg episode reward: [(0, '15.435')] +[2025-05-25 11:53:24,022][04232] Updated weights for policy 0, policy_version 600 (0.0020) +[2025-05-25 11:53:27,616][04028] Fps is (10 sec: 3685.3, 60 sec: 3959.3, 300 sec: 3929.4). Total num frames: 2469888. Throughput: 0: 991.2. Samples: 617240. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:53:27,622][04028] Avg episode reward: [(0, '16.740')] +[2025-05-25 11:53:27,626][04219] Saving new best policy, reward=16.740! +[2025-05-25 11:53:32,613][04028] Fps is (10 sec: 4506.0, 60 sec: 3959.5, 300 sec: 3957.2). Total num frames: 2494464. Throughput: 0: 992.9. Samples: 624000. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:53:32,615][04028] Avg episode reward: [(0, '17.020')] +[2025-05-25 11:53:32,632][04219] Saving new best policy, reward=17.020! +[2025-05-25 11:53:33,728][04232] Updated weights for policy 0, policy_version 610 (0.0012) +[2025-05-25 11:53:37,613][04028] Fps is (10 sec: 3687.5, 60 sec: 3891.2, 300 sec: 3915.5). Total num frames: 2506752. Throughput: 0: 972.6. Samples: 626230. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2025-05-25 11:53:37,619][04028] Avg episode reward: [(0, '17.369')] +[2025-05-25 11:53:37,665][04219] Saving new best policy, reward=17.369! +[2025-05-25 11:53:42,613][04028] Fps is (10 sec: 3686.3, 60 sec: 3959.5, 300 sec: 3943.3). Total num frames: 2531328. Throughput: 0: 988.8. Samples: 631916. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:53:42,615][04028] Avg episode reward: [(0, '16.397')] +[2025-05-25 11:53:44,224][04232] Updated weights for policy 0, policy_version 620 (0.0025) +[2025-05-25 11:53:47,613][04028] Fps is (10 sec: 4505.7, 60 sec: 3959.5, 300 sec: 3943.3). Total num frames: 2551808. Throughput: 0: 988.4. Samples: 638696. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2025-05-25 11:53:47,614][04028] Avg episode reward: [(0, '15.721')] +[2025-05-25 11:53:52,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 3929.4). Total num frames: 2568192. Throughput: 0: 969.0. Samples: 640738. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:53:52,615][04028] Avg episode reward: [(0, '16.235')] +[2025-05-25 11:53:55,023][04232] Updated weights for policy 0, policy_version 630 (0.0021) +[2025-05-25 11:53:57,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 3943.3). Total num frames: 2588672. Throughput: 0: 993.0. Samples: 646960. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:53:57,618][04028] Avg episode reward: [(0, '15.227')] +[2025-05-25 11:54:02,613][04028] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3929.4). Total num frames: 2609152. Throughput: 0: 986.3. Samples: 653468. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2025-05-25 11:54:02,616][04028] Avg episode reward: [(0, '17.023')] +[2025-05-25 11:54:05,726][04232] Updated weights for policy 0, policy_version 640 (0.0021) +[2025-05-25 11:54:07,613][04028] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3929.4). Total num frames: 2629632. Throughput: 0: 975.4. Samples: 655552. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:54:07,614][04028] Avg episode reward: [(0, '16.683')] +[2025-05-25 11:54:12,613][04028] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3943.3). Total num frames: 2650112. Throughput: 0: 994.4. Samples: 661986. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2025-05-25 11:54:12,624][04028] Avg episode reward: [(0, '17.001')] +[2025-05-25 11:54:14,817][04232] Updated weights for policy 0, policy_version 650 (0.0023) +[2025-05-25 11:54:17,613][04028] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3943.3). Total num frames: 2670592. Throughput: 0: 980.4. Samples: 668120. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:54:17,616][04028] Avg episode reward: [(0, '16.987')] +[2025-05-25 11:54:22,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 3915.5). Total num frames: 2686976. Throughput: 0: 976.3. Samples: 670162. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:54:22,616][04028] Avg episode reward: [(0, '15.471')] +[2025-05-25 11:54:22,623][04219] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000656_2686976.pth... +[2025-05-25 11:54:22,737][04219] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000426_1744896.pth +[2025-05-25 11:54:25,681][04232] Updated weights for policy 0, policy_version 660 (0.0017) +[2025-05-25 11:54:27,613][04028] Fps is (10 sec: 4096.0, 60 sec: 4027.9, 300 sec: 3957.2). Total num frames: 2711552. Throughput: 0: 999.7. Samples: 676902. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:54:27,614][04028] Avg episode reward: [(0, '16.166')] +[2025-05-25 11:54:32,615][04028] Fps is (10 sec: 4095.2, 60 sec: 3891.1, 300 sec: 3929.4). Total num frames: 2727936. Throughput: 0: 976.8. Samples: 682654. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:54:32,619][04028] Avg episode reward: [(0, '15.546')] +[2025-05-25 11:54:36,521][04232] Updated weights for policy 0, policy_version 670 (0.0021) +[2025-05-25 11:54:37,613][04028] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 3929.4). Total num frames: 2748416. Throughput: 0: 985.0. Samples: 685064. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:54:37,617][04028] Avg episode reward: [(0, '15.066')] +[2025-05-25 11:54:42,620][04028] Fps is (10 sec: 4093.9, 60 sec: 3959.0, 300 sec: 3943.2). Total num frames: 2768896. Throughput: 0: 996.8. Samples: 691824. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2025-05-25 11:54:42,622][04028] Avg episode reward: [(0, '15.204')] +[2025-05-25 11:54:46,031][04232] Updated weights for policy 0, policy_version 680 (0.0020) +[2025-05-25 11:54:47,614][04028] Fps is (10 sec: 4095.4, 60 sec: 3959.4, 300 sec: 3943.3). Total num frames: 2789376. Throughput: 0: 976.3. Samples: 697404. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2025-05-25 11:54:47,616][04028] Avg episode reward: [(0, '15.938')] +[2025-05-25 11:54:52,613][04028] Fps is (10 sec: 3689.1, 60 sec: 3959.5, 300 sec: 3929.4). Total num frames: 2805760. Throughput: 0: 989.2. Samples: 700068. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:54:52,618][04028] Avg episode reward: [(0, '15.234')] +[2025-05-25 11:54:56,262][04232] Updated weights for policy 0, policy_version 690 (0.0015) +[2025-05-25 11:54:57,613][04028] Fps is (10 sec: 4096.5, 60 sec: 4027.7, 300 sec: 3957.2). Total num frames: 2830336. Throughput: 0: 1000.4. Samples: 707004. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2025-05-25 11:54:57,618][04028] Avg episode reward: [(0, '16.848')] +[2025-05-25 11:55:02,613][04028] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3929.4). Total num frames: 2846720. Throughput: 0: 986.4. Samples: 712506. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:55:02,619][04028] Avg episode reward: [(0, '17.428')] +[2025-05-25 11:55:02,629][04219] Saving new best policy, reward=17.428! +[2025-05-25 11:55:06,524][04232] Updated weights for policy 0, policy_version 700 (0.0028) +[2025-05-25 11:55:07,613][04028] Fps is (10 sec: 4096.0, 60 sec: 4027.7, 300 sec: 3943.3). Total num frames: 2871296. Throughput: 0: 1010.8. Samples: 715646. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:55:07,620][04028] Avg episode reward: [(0, '19.037')] +[2025-05-25 11:55:07,623][04219] Saving new best policy, reward=19.037! +[2025-05-25 11:55:12,613][04028] Fps is (10 sec: 4505.5, 60 sec: 4027.7, 300 sec: 3957.2). Total num frames: 2891776. Throughput: 0: 1021.2. Samples: 722856. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2025-05-25 11:55:12,615][04028] Avg episode reward: [(0, '19.920')] +[2025-05-25 11:55:12,688][04219] Saving new best policy, reward=19.920! +[2025-05-25 11:55:16,650][04232] Updated weights for policy 0, policy_version 710 (0.0015) +[2025-05-25 11:55:17,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 3943.3). Total num frames: 2908160. Throughput: 0: 1004.5. Samples: 727854. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2025-05-25 11:55:17,614][04028] Avg episode reward: [(0, '19.791')] +[2025-05-25 11:55:22,613][04028] Fps is (10 sec: 4096.1, 60 sec: 4096.0, 300 sec: 3957.2). Total num frames: 2932736. Throughput: 0: 1028.9. Samples: 731364. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:55:22,619][04028] Avg episode reward: [(0, '20.571')] +[2025-05-25 11:55:22,633][04219] Saving new best policy, reward=20.571! +[2025-05-25 11:55:25,462][04232] Updated weights for policy 0, policy_version 720 (0.0028) +[2025-05-25 11:55:27,613][04028] Fps is (10 sec: 4915.0, 60 sec: 4096.0, 300 sec: 3984.9). Total num frames: 2957312. Throughput: 0: 1038.8. Samples: 738564. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2025-05-25 11:55:27,615][04028] Avg episode reward: [(0, '18.526')] +[2025-05-25 11:55:32,613][04028] Fps is (10 sec: 4096.0, 60 sec: 4096.1, 300 sec: 3957.2). Total num frames: 2973696. Throughput: 0: 1030.7. Samples: 743784. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2025-05-25 11:55:32,617][04028] Avg episode reward: [(0, '18.832')] +[2025-05-25 11:55:35,631][04232] Updated weights for policy 0, policy_version 730 (0.0018) +[2025-05-25 11:55:37,613][04028] Fps is (10 sec: 4096.1, 60 sec: 4164.3, 300 sec: 3985.0). Total num frames: 2998272. Throughput: 0: 1051.7. Samples: 747396. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2025-05-25 11:55:37,614][04028] Avg episode reward: [(0, '19.408')] +[2025-05-25 11:55:42,613][04028] Fps is (10 sec: 4505.6, 60 sec: 4164.8, 300 sec: 3984.9). Total num frames: 3018752. Throughput: 0: 1057.7. Samples: 754602. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2025-05-25 11:55:42,615][04028] Avg episode reward: [(0, '17.481')] +[2025-05-25 11:55:46,096][04232] Updated weights for policy 0, policy_version 740 (0.0013) +[2025-05-25 11:55:47,613][04028] Fps is (10 sec: 3686.4, 60 sec: 4096.1, 300 sec: 3971.0). Total num frames: 3035136. Throughput: 0: 1047.2. Samples: 759630. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:55:47,619][04028] Avg episode reward: [(0, '18.542')] +[2025-05-25 11:55:52,613][04028] Fps is (10 sec: 4096.0, 60 sec: 4232.5, 300 sec: 3998.8). Total num frames: 3059712. Throughput: 0: 1058.0. Samples: 763256. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2025-05-25 11:55:52,620][04028] Avg episode reward: [(0, '18.708')] +[2025-05-25 11:55:54,447][04232] Updated weights for policy 0, policy_version 750 (0.0021) +[2025-05-25 11:55:57,613][04028] Fps is (10 sec: 4505.6, 60 sec: 4164.3, 300 sec: 3998.8). Total num frames: 3080192. Throughput: 0: 1058.6. Samples: 770492. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2025-05-25 11:55:57,614][04028] Avg episode reward: [(0, '18.288')] +[2025-05-25 11:56:02,613][04028] Fps is (10 sec: 4096.0, 60 sec: 4232.5, 300 sec: 3984.9). Total num frames: 3100672. Throughput: 0: 1062.9. Samples: 775684. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:56:02,618][04028] Avg episode reward: [(0, '18.659')] +[2025-05-25 11:56:04,657][04232] Updated weights for policy 0, policy_version 760 (0.0018) +[2025-05-25 11:56:07,613][04028] Fps is (10 sec: 4505.6, 60 sec: 4232.5, 300 sec: 4026.6). Total num frames: 3125248. Throughput: 0: 1065.9. Samples: 779330. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2025-05-25 11:56:07,614][04028] Avg episode reward: [(0, '19.530')] +[2025-05-25 11:56:12,613][04028] Fps is (10 sec: 4505.6, 60 sec: 4232.6, 300 sec: 4012.7). Total num frames: 3145728. Throughput: 0: 1059.9. Samples: 786258. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:56:12,617][04028] Avg episode reward: [(0, '19.388')] +[2025-05-25 11:56:14,893][04232] Updated weights for policy 0, policy_version 770 (0.0028) +[2025-05-25 11:56:17,615][04028] Fps is (10 sec: 4095.1, 60 sec: 4300.6, 300 sec: 4012.7). Total num frames: 3166208. Throughput: 0: 1065.8. Samples: 791746. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:56:17,617][04028] Avg episode reward: [(0, '19.156')] +[2025-05-25 11:56:22,613][04028] Fps is (10 sec: 4505.6, 60 sec: 4300.8, 300 sec: 4040.5). Total num frames: 3190784. Throughput: 0: 1065.6. Samples: 795350. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2025-05-25 11:56:22,619][04028] Avg episode reward: [(0, '19.686')] +[2025-05-25 11:56:22,628][04219] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000779_3190784.pth... +[2025-05-25 11:56:22,744][04219] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000541_2215936.pth +[2025-05-25 11:56:23,391][04232] Updated weights for policy 0, policy_version 780 (0.0028) +[2025-05-25 11:56:27,614][04028] Fps is (10 sec: 4096.3, 60 sec: 4164.2, 300 sec: 4026.6). Total num frames: 3207168. Throughput: 0: 1050.6. Samples: 801880. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2025-05-25 11:56:27,618][04028] Avg episode reward: [(0, '20.881')] +[2025-05-25 11:56:27,622][04219] Saving new best policy, reward=20.881! +[2025-05-25 11:56:32,613][04028] Fps is (10 sec: 3686.4, 60 sec: 4232.5, 300 sec: 4012.7). Total num frames: 3227648. Throughput: 0: 1067.3. Samples: 807658. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:56:32,618][04028] Avg episode reward: [(0, '20.458')] +[2025-05-25 11:56:33,631][04232] Updated weights for policy 0, policy_version 790 (0.0027) +[2025-05-25 11:56:37,613][04028] Fps is (10 sec: 4506.2, 60 sec: 4232.5, 300 sec: 4054.4). Total num frames: 3252224. Throughput: 0: 1066.9. Samples: 811268. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:56:37,614][04028] Avg episode reward: [(0, '20.871')] +[2025-05-25 11:56:42,613][04028] Fps is (10 sec: 4096.0, 60 sec: 4164.3, 300 sec: 4026.6). Total num frames: 3268608. Throughput: 0: 1041.3. Samples: 817350. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:56:42,621][04028] Avg episode reward: [(0, '20.965')] +[2025-05-25 11:56:42,628][04219] Saving new best policy, reward=20.965! +[2025-05-25 11:56:44,162][04232] Updated weights for policy 0, policy_version 800 (0.0038) +[2025-05-25 11:56:47,613][04028] Fps is (10 sec: 3686.4, 60 sec: 4232.5, 300 sec: 4026.6). Total num frames: 3289088. Throughput: 0: 1058.2. Samples: 823302. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:56:47,619][04028] Avg episode reward: [(0, '21.206')] +[2025-05-25 11:56:47,652][04219] Saving new best policy, reward=21.206! +[2025-05-25 11:56:52,613][04028] Fps is (10 sec: 4505.6, 60 sec: 4232.5, 300 sec: 4054.3). Total num frames: 3313664. Throughput: 0: 1056.2. Samples: 826858. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:56:52,618][04028] Avg episode reward: [(0, '19.817')] +[2025-05-25 11:56:52,847][04232] Updated weights for policy 0, policy_version 810 (0.0020) +[2025-05-25 11:56:57,613][04028] Fps is (10 sec: 4096.0, 60 sec: 4164.3, 300 sec: 4040.5). Total num frames: 3330048. Throughput: 0: 1031.5. Samples: 832676. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:56:57,618][04028] Avg episode reward: [(0, '19.903')] +[2025-05-25 11:57:02,614][04028] Fps is (10 sec: 4095.9, 60 sec: 4232.5, 300 sec: 4054.3). Total num frames: 3354624. Throughput: 0: 1051.0. Samples: 839038. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:57:02,619][04028] Avg episode reward: [(0, '21.075')] +[2025-05-25 11:57:03,225][04232] Updated weights for policy 0, policy_version 820 (0.0017) +[2025-05-25 11:57:07,613][04028] Fps is (10 sec: 4915.2, 60 sec: 4232.5, 300 sec: 4082.1). Total num frames: 3379200. Throughput: 0: 1050.1. Samples: 842604. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:57:07,618][04028] Avg episode reward: [(0, '21.092')] +[2025-05-25 11:57:12,613][04028] Fps is (10 sec: 4096.0, 60 sec: 4164.2, 300 sec: 4054.3). Total num frames: 3395584. Throughput: 0: 1031.0. Samples: 848276. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) +[2025-05-25 11:57:12,616][04028] Avg episode reward: [(0, '20.504')] +[2025-05-25 11:57:13,458][04232] Updated weights for policy 0, policy_version 830 (0.0021) +[2025-05-25 11:57:17,613][04028] Fps is (10 sec: 3686.4, 60 sec: 4164.4, 300 sec: 4068.2). Total num frames: 3416064. Throughput: 0: 1043.4. Samples: 854612. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:57:17,619][04028] Avg episode reward: [(0, '20.358')] +[2025-05-25 11:57:22,613][04028] Fps is (10 sec: 4096.1, 60 sec: 4096.0, 300 sec: 4082.1). Total num frames: 3436544. Throughput: 0: 1039.6. Samples: 858048. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:57:22,617][04028] Avg episode reward: [(0, '21.221')] +[2025-05-25 11:57:22,634][04219] Saving new best policy, reward=21.221! +[2025-05-25 11:57:22,639][04232] Updated weights for policy 0, policy_version 840 (0.0012) +[2025-05-25 11:57:27,613][04028] Fps is (10 sec: 3686.4, 60 sec: 4096.1, 300 sec: 4054.3). Total num frames: 3452928. Throughput: 0: 1012.0. Samples: 862890. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:57:27,614][04028] Avg episode reward: [(0, '22.162')] +[2025-05-25 11:57:27,628][04219] Saving new best policy, reward=22.162! +[2025-05-25 11:57:32,613][04028] Fps is (10 sec: 4096.0, 60 sec: 4164.3, 300 sec: 4082.1). Total num frames: 3477504. Throughput: 0: 1025.1. Samples: 869430. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2025-05-25 11:57:32,614][04028] Avg episode reward: [(0, '21.919')] +[2025-05-25 11:57:33,450][04232] Updated weights for policy 0, policy_version 850 (0.0012) +[2025-05-25 11:57:37,617][04028] Fps is (10 sec: 4503.8, 60 sec: 4095.7, 300 sec: 4082.1). Total num frames: 3497984. Throughput: 0: 1022.0. Samples: 872850. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:57:37,621][04028] Avg episode reward: [(0, '21.891')] +[2025-05-25 11:57:42,613][04028] Fps is (10 sec: 3686.4, 60 sec: 4096.0, 300 sec: 4068.2). Total num frames: 3514368. Throughput: 0: 995.7. Samples: 877482. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2025-05-25 11:57:42,618][04028] Avg episode reward: [(0, '21.114')] +[2025-05-25 11:57:44,456][04232] Updated weights for policy 0, policy_version 860 (0.0021) +[2025-05-25 11:57:47,613][04028] Fps is (10 sec: 3687.7, 60 sec: 4096.0, 300 sec: 4082.1). Total num frames: 3534848. Throughput: 0: 1004.8. Samples: 884252. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2025-05-25 11:57:47,619][04028] Avg episode reward: [(0, '19.034')] +[2025-05-25 11:57:52,615][04028] Fps is (10 sec: 4095.1, 60 sec: 4027.6, 300 sec: 4082.1). Total num frames: 3555328. Throughput: 0: 1001.8. Samples: 887686. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:57:52,621][04028] Avg episode reward: [(0, '18.600')] +[2025-05-25 11:57:55,227][04232] Updated weights for policy 0, policy_version 870 (0.0027) +[2025-05-25 11:57:57,614][04028] Fps is (10 sec: 3686.1, 60 sec: 4027.7, 300 sec: 4054.3). Total num frames: 3571712. Throughput: 0: 980.9. Samples: 892418. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2025-05-25 11:57:57,621][04028] Avg episode reward: [(0, '19.614')] +[2025-05-25 11:58:02,613][04028] Fps is (10 sec: 4096.9, 60 sec: 4027.8, 300 sec: 4082.1). Total num frames: 3596288. Throughput: 0: 995.2. Samples: 899398. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:58:02,614][04028] Avg episode reward: [(0, '20.119')] +[2025-05-25 11:58:04,232][04232] Updated weights for policy 0, policy_version 880 (0.0012) +[2025-05-25 11:58:07,613][04028] Fps is (10 sec: 4096.5, 60 sec: 3891.2, 300 sec: 4068.2). Total num frames: 3612672. Throughput: 0: 992.4. Samples: 902706. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2025-05-25 11:58:07,616][04028] Avg episode reward: [(0, '20.402')] +[2025-05-25 11:58:12,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 4068.2). Total num frames: 3633152. Throughput: 0: 992.7. Samples: 907562. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2025-05-25 11:58:12,619][04028] Avg episode reward: [(0, '21.279')] +[2025-05-25 11:58:14,975][04232] Updated weights for policy 0, policy_version 890 (0.0017) +[2025-05-25 11:58:17,613][04028] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 4082.1). Total num frames: 3653632. Throughput: 0: 995.5. Samples: 914228. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:58:17,616][04028] Avg episode reward: [(0, '19.305')] +[2025-05-25 11:58:22,619][04028] Fps is (10 sec: 3684.1, 60 sec: 3890.8, 300 sec: 4068.2). Total num frames: 3670016. Throughput: 0: 987.6. Samples: 917294. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:58:22,621][04028] Avg episode reward: [(0, '18.808')] +[2025-05-25 11:58:22,629][04219] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000896_3670016.pth... +[2025-05-25 11:58:22,790][04219] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000656_2686976.pth +[2025-05-25 11:58:25,862][04232] Updated weights for policy 0, policy_version 900 (0.0022) +[2025-05-25 11:58:27,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 4054.3). Total num frames: 3690496. Throughput: 0: 998.2. Samples: 922402. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:58:27,615][04028] Avg episode reward: [(0, '17.714')] +[2025-05-25 11:58:32,613][04028] Fps is (10 sec: 4508.4, 60 sec: 3959.5, 300 sec: 4096.0). Total num frames: 3715072. Throughput: 0: 998.8. Samples: 929198. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:58:32,615][04028] Avg episode reward: [(0, '17.723')] +[2025-05-25 11:58:35,942][04232] Updated weights for policy 0, policy_version 910 (0.0014) +[2025-05-25 11:58:37,615][04028] Fps is (10 sec: 4095.1, 60 sec: 3891.3, 300 sec: 4068.2). Total num frames: 3731456. Throughput: 0: 983.4. Samples: 931938. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) +[2025-05-25 11:58:37,619][04028] Avg episode reward: [(0, '17.621')] +[2025-05-25 11:58:42,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 4068.2). Total num frames: 3751936. Throughput: 0: 999.1. Samples: 937376. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2025-05-25 11:58:42,614][04028] Avg episode reward: [(0, '19.292')] +[2025-05-25 11:58:45,932][04232] Updated weights for policy 0, policy_version 920 (0.0019) +[2025-05-25 11:58:47,634][04028] Fps is (10 sec: 4088.5, 60 sec: 3958.1, 300 sec: 4081.8). Total num frames: 3772416. Throughput: 0: 991.0. Samples: 944014. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:58:47,635][04028] Avg episode reward: [(0, '19.666')] +[2025-05-25 11:58:52,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3891.3, 300 sec: 4068.2). Total num frames: 3788800. Throughput: 0: 972.7. Samples: 946478. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:58:52,616][04028] Avg episode reward: [(0, '20.411')] +[2025-05-25 11:58:56,720][04232] Updated weights for policy 0, policy_version 930 (0.0015) +[2025-05-25 11:58:57,613][04028] Fps is (10 sec: 3694.0, 60 sec: 3959.5, 300 sec: 4068.2). Total num frames: 3809280. Throughput: 0: 992.8. Samples: 952236. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) +[2025-05-25 11:58:57,620][04028] Avg episode reward: [(0, '21.512')] +[2025-05-25 11:59:02,613][04028] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 4082.1). Total num frames: 3833856. Throughput: 0: 997.7. Samples: 959124. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:59:02,618][04028] Avg episode reward: [(0, '20.614')] +[2025-05-25 11:59:07,435][04232] Updated weights for policy 0, policy_version 940 (0.0020) +[2025-05-25 11:59:07,613][04028] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 4068.2). Total num frames: 3850240. Throughput: 0: 976.1. Samples: 961212. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:59:07,618][04028] Avg episode reward: [(0, '19.897')] +[2025-05-25 11:59:12,613][04028] Fps is (10 sec: 3686.3, 60 sec: 3959.5, 300 sec: 4068.2). Total num frames: 3870720. Throughput: 0: 1000.1. Samples: 967408. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:59:12,619][04028] Avg episode reward: [(0, '18.298')] +[2025-05-25 11:59:16,387][04232] Updated weights for policy 0, policy_version 950 (0.0016) +[2025-05-25 11:59:17,613][04028] Fps is (10 sec: 4095.9, 60 sec: 3959.5, 300 sec: 4082.1). Total num frames: 3891200. Throughput: 0: 991.0. Samples: 973794. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:59:17,615][04028] Avg episode reward: [(0, '17.235')] +[2025-05-25 11:59:22,613][04028] Fps is (10 sec: 3686.5, 60 sec: 3959.9, 300 sec: 4054.3). Total num frames: 3907584. Throughput: 0: 975.0. Samples: 975810. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2025-05-25 11:59:22,614][04028] Avg episode reward: [(0, '17.317')] +[2025-05-25 11:59:27,291][04232] Updated weights for policy 0, policy_version 960 (0.0022) +[2025-05-25 11:59:27,613][04028] Fps is (10 sec: 4096.1, 60 sec: 4027.7, 300 sec: 4082.1). Total num frames: 3932160. Throughput: 0: 998.1. Samples: 982290. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2025-05-25 11:59:27,615][04028] Avg episode reward: [(0, '18.341')] +[2025-05-25 11:59:32,613][04028] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 4082.1). Total num frames: 3952640. Throughput: 0: 988.8. Samples: 988488. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2025-05-25 11:59:32,617][04028] Avg episode reward: [(0, '19.201')] +[2025-05-25 11:59:37,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3959.6, 300 sec: 4068.3). Total num frames: 3969024. Throughput: 0: 979.0. Samples: 990532. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2025-05-25 11:59:37,614][04028] Avg episode reward: [(0, '20.641')] +[2025-05-25 11:59:38,181][04232] Updated weights for policy 0, policy_version 970 (0.0025) +[2025-05-25 11:59:42,613][04028] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 4068.2). Total num frames: 3989504. Throughput: 0: 1001.6. Samples: 997306. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2025-05-25 11:59:42,614][04028] Avg episode reward: [(0, '20.992')] +[2025-05-25 11:59:45,654][04219] Stopping Batcher_0... +[2025-05-25 11:59:45,659][04219] Loop batcher_evt_loop terminating... +[2025-05-25 11:59:45,655][04219] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... +[2025-05-25 11:59:45,666][04028] Component Batcher_0 stopped! +[2025-05-25 11:59:45,780][04232] Weights refcount: 2 0 +[2025-05-25 11:59:45,788][04232] Stopping InferenceWorker_p0-w0... +[2025-05-25 11:59:45,789][04232] Loop inference_proc0-0_evt_loop terminating... +[2025-05-25 11:59:45,790][04028] Component InferenceWorker_p0-w0 stopped! +[2025-05-25 11:59:45,827][04219] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000779_3190784.pth +[2025-05-25 11:59:45,851][04219] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... +[2025-05-25 11:59:46,046][04219] Stopping LearnerWorker_p0... +[2025-05-25 11:59:46,048][04219] Loop learner_proc0_evt_loop terminating... +[2025-05-25 11:59:46,052][04028] Component LearnerWorker_p0 stopped! +[2025-05-25 11:59:46,139][04028] Component RolloutWorker_w7 stopped! +[2025-05-25 11:59:46,143][04238] Stopping RolloutWorker_w7... +[2025-05-25 11:59:46,150][04238] Loop rollout_proc7_evt_loop terminating... +[2025-05-25 11:59:46,184][04028] Component RolloutWorker_w3 stopped! +[2025-05-25 11:59:46,187][04236] Stopping RolloutWorker_w3... +[2025-05-25 11:59:46,188][04236] Loop rollout_proc3_evt_loop terminating... +[2025-05-25 11:59:46,212][04028] Component RolloutWorker_w1 stopped! +[2025-05-25 11:59:46,214][04234] Stopping RolloutWorker_w1... +[2025-05-25 11:59:46,214][04234] Loop rollout_proc1_evt_loop terminating... +[2025-05-25 11:59:46,223][04028] Component RolloutWorker_w5 stopped! +[2025-05-25 11:59:46,231][04240] Stopping RolloutWorker_w5... +[2025-05-25 11:59:46,231][04240] Loop rollout_proc5_evt_loop terminating... +[2025-05-25 11:59:46,418][04235] Stopping RolloutWorker_w2... +[2025-05-25 11:59:46,419][04235] Loop rollout_proc2_evt_loop terminating... +[2025-05-25 11:59:46,419][04028] Component RolloutWorker_w2 stopped! +[2025-05-25 11:59:46,431][04028] Component RolloutWorker_w0 stopped! +[2025-05-25 11:59:46,442][04233] Stopping RolloutWorker_w0... +[2025-05-25 11:59:46,442][04233] Loop rollout_proc0_evt_loop terminating... +[2025-05-25 11:59:46,475][04028] Component RolloutWorker_w4 stopped! +[2025-05-25 11:59:46,476][04239] Stopping RolloutWorker_w4... +[2025-05-25 11:59:46,480][04239] Loop rollout_proc4_evt_loop terminating... +[2025-05-25 11:59:46,512][04028] Component RolloutWorker_w6 stopped! +[2025-05-25 11:59:46,513][04028] Waiting for process learner_proc0 to stop... +[2025-05-25 11:59:46,514][04237] Stopping RolloutWorker_w6... +[2025-05-25 11:59:46,531][04237] Loop rollout_proc6_evt_loop terminating... +[2025-05-25 11:59:48,707][04028] Waiting for process inference_proc0-0 to join... +[2025-05-25 11:59:48,709][04028] Waiting for process rollout_proc0 to join... +[2025-05-25 11:59:51,197][04028] Waiting for process rollout_proc1 to join... +[2025-05-25 11:59:51,199][04028] Waiting for process rollout_proc2 to join... +[2025-05-25 11:59:51,201][04028] Waiting for process rollout_proc3 to join... +[2025-05-25 11:59:51,203][04028] Waiting for process rollout_proc4 to join... +[2025-05-25 11:59:51,206][04028] Waiting for process rollout_proc5 to join... +[2025-05-25 11:59:51,207][04028] Waiting for process rollout_proc6 to join... +[2025-05-25 11:59:51,211][04028] Waiting for process rollout_proc7 to join... +[2025-05-25 11:59:51,213][04028] Batcher 0 profile tree view: +batching: 25.8661, releasing_batches: 0.0286 +[2025-05-25 11:59:51,215][04028] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0001 - wait_policy_total: 13.4992 -update_model: 6.2238 + wait_policy_total: 416.8903 +update_model: 8.0970 weight_update: 0.0015 -one_step: 0.0046 - handle_policy_step: 399.8286 - deserialize: 11.6803, stack: 2.5635, obs_to_device_normalize: 98.9273, forward: 196.3114, send_messages: 20.6973 - prepare_outputs: 53.6470 - to_cpu: 34.9520 -[2025-05-25 11:17:17,564][00035] Learner 0 profile tree view: -misc: 0.0036, prepare_batch: 12.0117 -train: 51.2261 - epoch_init: 0.0044, minibatch_init: 0.0069, losses_postprocess: 0.5186, kl_divergence: 0.5606, after_optimizer: 22.0477 - calculate_losses: 17.7847 - losses_init: 0.0034, forward_head: 1.0293, bptt_initial: 12.4695, tail: 0.7633, advantages_returns: 0.2058, losses: 1.6964 - bptt: 1.4446 - bptt_forward_core: 1.3814 - update: 9.9105 - clip: 0.9112 -[2025-05-25 11:17:17,565][00035] RolloutWorker_w0 profile tree view: -wait_for_trajectories: 0.1511, enqueue_policy_requests: 7.6643, env_step: 316.0414, overhead: 6.3073, complete_rollouts: 1.0488 -save_policy_outputs: 8.7204 - split_output_tensors: 3.3696 -[2025-05-25 11:17:17,565][00035] RolloutWorker_w7 profile tree view: -wait_for_trajectories: 0.1530, enqueue_policy_requests: 7.7979, env_step: 314.2266, overhead: 6.5045, complete_rollouts: 0.9601 -save_policy_outputs: 8.9357 - split_output_tensors: 3.3884 -[2025-05-25 11:17:17,566][00035] Loop Runner_EvtLoop terminating... -[2025-05-25 11:17:17,567][00035] Runner profile tree view: -main_loop: 454.4760 -[2025-05-25 11:17:17,568][00035] Collected {0: 4005888}, FPS: 8814.3 -[2025-05-25 11:17:17,820][00035] Loading existing experiment configuration from /kaggle/working/train_dir/default_experiment/config.json -[2025-05-25 11:17:17,821][00035] Overriding arg 'num_workers' with value 1 passed from command line -[2025-05-25 11:17:17,822][00035] Adding new argument 'no_render'=True that is not in the saved config file! -[2025-05-25 11:17:17,823][00035] Adding new argument 'save_video'=True that is not in the saved config file! -[2025-05-25 11:17:17,824][00035] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! -[2025-05-25 11:17:17,825][00035] Adding new argument 'video_name'=None that is not in the saved config file! -[2025-05-25 11:17:17,825][00035] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! -[2025-05-25 11:17:17,826][00035] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! -[2025-05-25 11:17:17,827][00035] Adding new argument 'push_to_hub'=False that is not in the saved config file! -[2025-05-25 11:17:17,828][00035] Adding new argument 'hf_repository'=None that is not in the saved config file! -[2025-05-25 11:17:17,829][00035] Adding new argument 'policy_index'=0 that is not in the saved config file! -[2025-05-25 11:17:17,829][00035] Adding new argument 'eval_deterministic'=False that is not in the saved config file! -[2025-05-25 11:17:17,830][00035] Adding new argument 'train_script'=None that is not in the saved config file! -[2025-05-25 11:17:17,830][00035] Adding new argument 'enjoy_script'=None that is not in the saved config file! -[2025-05-25 11:17:17,831][00035] Using frameskip 1 and render_action_repeat=4 for evaluation -[2025-05-25 11:17:17,860][00035] Doom resolution: 160x120, resize resolution: (128, 72) -[2025-05-25 11:17:17,862][00035] RunningMeanStd input shape: (3, 72, 128) -[2025-05-25 11:17:17,864][00035] RunningMeanStd input shape: (1,) -[2025-05-25 11:17:17,879][00035] ConvEncoder: input_channels=3 -[2025-05-25 11:17:17,986][00035] Conv encoder output size: 512 -[2025-05-25 11:17:17,987][00035] Policy head output size: 512 -[2025-05-25 11:17:18,985][00035] Num frames 100... -[2025-05-25 11:17:19,099][00035] Num frames 200... -[2025-05-25 11:17:19,212][00035] Num frames 300... -[2025-05-25 11:17:19,365][00035] Avg episode rewards: #0: 3.840, true rewards: #0: 3.840 -[2025-05-25 11:17:19,366][00035] Avg episode reward: 3.840, avg true_objective: 3.840 -[2025-05-25 11:17:19,387][00035] Num frames 400... -[2025-05-25 11:17:19,499][00035] Num frames 500... -[2025-05-25 11:17:19,608][00035] Num frames 600... -[2025-05-25 11:17:19,716][00035] Num frames 700... -[2025-05-25 11:17:19,775][00035] Avg episode rewards: #0: 3.520, true rewards: #0: 3.520 -[2025-05-25 11:17:19,776][00035] Avg episode reward: 3.520, avg true_objective: 3.520 -[2025-05-25 11:17:19,883][00035] Num frames 800... -[2025-05-25 11:17:20,003][00035] Num frames 900... -[2025-05-25 11:17:20,125][00035] Num frames 1000... -[2025-05-25 11:17:20,282][00035] Avg episode rewards: #0: 3.627, true rewards: #0: 3.627 -[2025-05-25 11:17:20,283][00035] Avg episode reward: 3.627, avg true_objective: 3.627 -[2025-05-25 11:17:20,300][00035] Num frames 1100... -[2025-05-25 11:17:20,418][00035] Num frames 1200... -[2025-05-25 11:17:20,531][00035] Num frames 1300... -[2025-05-25 11:17:20,641][00035] Num frames 1400... -[2025-05-25 11:17:20,777][00035] Avg episode rewards: #0: 3.680, true rewards: #0: 3.680 -[2025-05-25 11:17:20,777][00035] Avg episode reward: 3.680, avg true_objective: 3.680 -[2025-05-25 11:17:20,810][00035] Num frames 1500... -[2025-05-25 11:17:20,921][00035] Num frames 1600... -[2025-05-25 11:17:21,031][00035] Num frames 1700... -[2025-05-25 11:17:21,141][00035] Num frames 1800... -[2025-05-25 11:17:21,257][00035] Avg episode rewards: #0: 3.712, true rewards: #0: 3.712 -[2025-05-25 11:17:21,257][00035] Avg episode reward: 3.712, avg true_objective: 3.712 -[2025-05-25 11:17:21,309][00035] Num frames 1900... -[2025-05-25 11:17:21,425][00035] Num frames 2000... -[2025-05-25 11:17:21,537][00035] Num frames 2100... -[2025-05-25 11:17:21,653][00035] Num frames 2200... -[2025-05-25 11:17:21,770][00035] Num frames 2300... -[2025-05-25 11:17:21,829][00035] Avg episode rewards: #0: 4.007, true rewards: #0: 3.840 -[2025-05-25 11:17:21,830][00035] Avg episode reward: 4.007, avg true_objective: 3.840 -[2025-05-25 11:17:21,944][00035] Num frames 2400... -[2025-05-25 11:17:22,062][00035] Num frames 2500... -[2025-05-25 11:17:22,171][00035] Num frames 2600... -[2025-05-25 11:17:22,283][00035] Num frames 2700... -[2025-05-25 11:17:22,427][00035] Avg episode rewards: #0: 4.406, true rewards: #0: 3.977 -[2025-05-25 11:17:22,428][00035] Avg episode reward: 4.406, avg true_objective: 3.977 -[2025-05-25 11:17:22,447][00035] Num frames 2800... -[2025-05-25 11:17:22,562][00035] Num frames 2900... -[2025-05-25 11:17:22,675][00035] Num frames 3000... -[2025-05-25 11:17:22,788][00035] Num frames 3100... -[2025-05-25 11:17:22,920][00035] Avg episode rewards: #0: 4.335, true rewards: #0: 3.960 -[2025-05-25 11:17:22,921][00035] Avg episode reward: 4.335, avg true_objective: 3.960 -[2025-05-25 11:17:22,959][00035] Num frames 3200... -[2025-05-25 11:17:23,076][00035] Num frames 3300... -[2025-05-25 11:17:23,195][00035] Num frames 3400... -[2025-05-25 11:17:23,310][00035] Num frames 3500... -[2025-05-25 11:17:23,427][00035] Avg episode rewards: #0: 4.280, true rewards: #0: 3.947 -[2025-05-25 11:17:23,428][00035] Avg episode reward: 4.280, avg true_objective: 3.947 -[2025-05-25 11:17:23,486][00035] Num frames 3600... -[2025-05-25 11:17:23,622][00035] Num frames 3700... -[2025-05-25 11:17:23,737][00035] Num frames 3800... -[2025-05-25 11:17:23,849][00035] Num frames 3900... -[2025-05-25 11:17:23,961][00035] Num frames 4000... -[2025-05-25 11:17:24,013][00035] Avg episode rewards: #0: 4.400, true rewards: #0: 4.000 -[2025-05-25 11:17:24,014][00035] Avg episode reward: 4.400, avg true_objective: 4.000 -[2025-05-25 11:17:34,266][00035] Replay video saved to /kaggle/working/train_dir/default_experiment/replay.mp4! -[2025-05-25 11:20:23,684][00035] Loading existing experiment configuration from /kaggle/working/train_dir/default_experiment/config.json -[2025-05-25 11:20:23,685][00035] Overriding arg 'num_workers' with value 1 passed from command line -[2025-05-25 11:20:23,685][00035] Adding new argument 'no_render'=True that is not in the saved config file! -[2025-05-25 11:20:23,686][00035] Adding new argument 'save_video'=True that is not in the saved config file! -[2025-05-25 11:20:23,687][00035] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! -[2025-05-25 11:20:23,687][00035] Adding new argument 'video_name'=None that is not in the saved config file! -[2025-05-25 11:20:23,689][00035] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! -[2025-05-25 11:20:23,689][00035] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! -[2025-05-25 11:20:23,690][00035] Adding new argument 'push_to_hub'=True that is not in the saved config file! -[2025-05-25 11:20:23,691][00035] Adding new argument 'hf_repository'='wowthecoder/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! -[2025-05-25 11:20:23,691][00035] Adding new argument 'policy_index'=0 that is not in the saved config file! -[2025-05-25 11:20:23,692][00035] Adding new argument 'eval_deterministic'=False that is not in the saved config file! -[2025-05-25 11:20:23,693][00035] Adding new argument 'train_script'=None that is not in the saved config file! -[2025-05-25 11:20:23,694][00035] Adding new argument 'enjoy_script'=None that is not in the saved config file! -[2025-05-25 11:20:23,695][00035] Using frameskip 1 and render_action_repeat=4 for evaluation -[2025-05-25 11:20:23,719][00035] RunningMeanStd input shape: (3, 72, 128) -[2025-05-25 11:20:23,720][00035] RunningMeanStd input shape: (1,) -[2025-05-25 11:20:23,731][00035] ConvEncoder: input_channels=3 -[2025-05-25 11:20:23,763][00035] Conv encoder output size: 512 -[2025-05-25 11:20:23,764][00035] Policy head output size: 512 -[2025-05-25 11:20:24,219][00035] Num frames 100... -[2025-05-25 11:20:24,328][00035] Num frames 200... -[2025-05-25 11:20:24,437][00035] Num frames 300... -[2025-05-25 11:20:24,583][00035] Avg episode rewards: #0: 3.840, true rewards: #0: 3.840 -[2025-05-25 11:20:24,584][00035] Avg episode reward: 3.840, avg true_objective: 3.840 -[2025-05-25 11:20:24,604][00035] Num frames 400... -[2025-05-25 11:20:24,713][00035] Num frames 500... -[2025-05-25 11:20:24,823][00035] Num frames 600... -[2025-05-25 11:20:24,946][00035] Num frames 700... -[2025-05-25 11:20:25,081][00035] Avg episode rewards: #0: 3.840, true rewards: #0: 3.840 -[2025-05-25 11:20:25,082][00035] Avg episode reward: 3.840, avg true_objective: 3.840 -[2025-05-25 11:20:25,119][00035] Num frames 800... -[2025-05-25 11:20:25,224][00035] Num frames 900... -[2025-05-25 11:20:25,330][00035] Num frames 1000... -[2025-05-25 11:20:25,436][00035] Num frames 1100... -[2025-05-25 11:20:25,545][00035] Avg episode rewards: #0: 3.840, true rewards: #0: 3.840 -[2025-05-25 11:20:25,545][00035] Avg episode reward: 3.840, avg true_objective: 3.840 -[2025-05-25 11:20:25,598][00035] Num frames 1200... -[2025-05-25 11:20:25,709][00035] Num frames 1300... -[2025-05-25 11:20:25,826][00035] Num frames 1400... -[2025-05-25 11:20:25,950][00035] Num frames 1500... -[2025-05-25 11:20:26,052][00035] Avg episode rewards: #0: 3.840, true rewards: #0: 3.840 -[2025-05-25 11:20:26,052][00035] Avg episode reward: 3.840, avg true_objective: 3.840 -[2025-05-25 11:20:26,133][00035] Num frames 1600... -[2025-05-25 11:20:26,239][00035] Num frames 1700... -[2025-05-25 11:20:26,344][00035] Num frames 1800... -[2025-05-25 11:20:26,454][00035] Num frames 1900... -[2025-05-25 11:20:26,565][00035] Avg episode rewards: #0: 4.104, true rewards: #0: 3.904 -[2025-05-25 11:20:26,566][00035] Avg episode reward: 4.104, avg true_objective: 3.904 -[2025-05-25 11:20:26,620][00035] Num frames 2000... -[2025-05-25 11:20:26,736][00035] Num frames 2100... -[2025-05-25 11:20:26,852][00035] Num frames 2200... -[2025-05-25 11:20:26,964][00035] Num frames 2300... -[2025-05-25 11:20:27,078][00035] Num frames 2400... -[2025-05-25 11:20:27,241][00035] Avg episode rewards: #0: 4.660, true rewards: #0: 4.160 -[2025-05-25 11:20:27,242][00035] Avg episode reward: 4.660, avg true_objective: 4.160 -[2025-05-25 11:20:27,248][00035] Num frames 2500... -[2025-05-25 11:20:27,366][00035] Num frames 2600... -[2025-05-25 11:20:27,481][00035] Num frames 2700... -[2025-05-25 11:20:27,598][00035] Num frames 2800... -[2025-05-25 11:20:27,744][00035] Avg episode rewards: #0: 4.543, true rewards: #0: 4.114 -[2025-05-25 11:20:27,744][00035] Avg episode reward: 4.543, avg true_objective: 4.114 -[2025-05-25 11:20:27,770][00035] Num frames 2900... -[2025-05-25 11:20:27,890][00035] Num frames 3000... -[2025-05-25 11:20:28,004][00035] Num frames 3100... -[2025-05-25 11:20:28,118][00035] Num frames 3200... -[2025-05-25 11:20:28,244][00035] Avg episode rewards: #0: 4.455, true rewards: #0: 4.080 -[2025-05-25 11:20:28,245][00035] Avg episode reward: 4.455, avg true_objective: 4.080 -[2025-05-25 11:20:28,289][00035] Num frames 3300... -[2025-05-25 11:20:28,407][00035] Num frames 3400... -[2025-05-25 11:20:28,524][00035] Num frames 3500... -[2025-05-25 11:20:28,636][00035] Num frames 3600... -[2025-05-25 11:20:28,722][00035] Avg episode rewards: #0: 4.364, true rewards: #0: 4.031 -[2025-05-25 11:20:28,723][00035] Avg episode reward: 4.364, avg true_objective: 4.031 -[2025-05-25 11:20:28,810][00035] Num frames 3700... -[2025-05-25 11:20:28,919][00035] Num frames 3800... -[2025-05-25 11:20:29,027][00035] Num frames 3900... -[2025-05-25 11:20:29,134][00035] Num frames 4000... -[2025-05-25 11:20:29,201][00035] Avg episode rewards: #0: 4.312, true rewards: #0: 4.012 -[2025-05-25 11:20:29,202][00035] Avg episode reward: 4.312, avg true_objective: 4.012 -[2025-05-25 11:20:39,527][00035] Replay video saved to /kaggle/working/train_dir/default_experiment/replay.mp4! +one_step: 0.0184 + handle_policy_step: 554.0371 + deserialize: 13.2066, stack: 3.0619, obs_to_device_normalize: 117.8002, forward: 285.8711, send_messages: 26.2649 + prepare_outputs: 84.4751 + to_cpu: 51.8477 +[2025-05-25 11:59:51,216][04028] Learner 0 profile tree view: +misc: 0.0036, prepare_batch: 12.3742 +train: 72.7860 + epoch_init: 0.0100, minibatch_init: 0.0058, losses_postprocess: 0.7466, kl_divergence: 0.6800, after_optimizer: 33.5673 + calculate_losses: 25.6316 + losses_init: 0.0033, forward_head: 1.3885, bptt_initial: 16.8819, tail: 1.0992, advantages_returns: 0.2975, losses: 3.7248 + bptt: 1.9870 + bptt_forward_core: 1.9110 + update: 11.5785 + clip: 1.0119 +[2025-05-25 11:59:51,217][04028] RolloutWorker_w0 profile tree view: +wait_for_trajectories: 0.2933, enqueue_policy_requests: 89.1367, env_step: 808.8613, overhead: 19.6574, complete_rollouts: 7.6504 +save_policy_outputs: 16.5366 + split_output_tensors: 7.8996 +[2025-05-25 11:59:51,218][04028] RolloutWorker_w7 profile tree view: +wait_for_trajectories: 0.2729, enqueue_policy_requests: 99.4620, env_step: 800.0832, overhead: 20.0151, complete_rollouts: 5.4136 +save_policy_outputs: 16.2777 + split_output_tensors: 7.8891 +[2025-05-25 11:59:51,220][04028] Loop Runner_EvtLoop terminating... +[2025-05-25 11:59:51,221][04028] Runner profile tree view: +main_loop: 1044.4106 +[2025-05-25 11:59:51,222][04028] Collected {0: 4005888}, FPS: 3835.5 +[2025-05-25 12:00:22,264][04028] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json +[2025-05-25 12:00:22,265][04028] Overriding arg 'num_workers' with value 1 passed from command line +[2025-05-25 12:00:22,266][04028] Adding new argument 'no_render'=True that is not in the saved config file! +[2025-05-25 12:00:22,266][04028] Adding new argument 'save_video'=True that is not in the saved config file! +[2025-05-25 12:00:22,267][04028] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! +[2025-05-25 12:00:22,267][04028] Adding new argument 'video_name'=None that is not in the saved config file! +[2025-05-25 12:00:22,268][04028] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! +[2025-05-25 12:00:22,269][04028] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! +[2025-05-25 12:00:22,270][04028] Adding new argument 'push_to_hub'=False that is not in the saved config file! +[2025-05-25 12:00:22,270][04028] Adding new argument 'hf_repository'=None that is not in the saved config file! +[2025-05-25 12:00:22,271][04028] Adding new argument 'policy_index'=0 that is not in the saved config file! +[2025-05-25 12:00:22,272][04028] Adding new argument 'eval_deterministic'=False that is not in the saved config file! +[2025-05-25 12:00:22,272][04028] Adding new argument 'train_script'=None that is not in the saved config file! +[2025-05-25 12:00:22,277][04028] Adding new argument 'enjoy_script'=None that is not in the saved config file! +[2025-05-25 12:00:22,278][04028] Using frameskip 1 and render_action_repeat=4 for evaluation +[2025-05-25 12:00:22,319][04028] Doom resolution: 160x120, resize resolution: (128, 72) +[2025-05-25 12:00:22,322][04028] RunningMeanStd input shape: (3, 72, 128) +[2025-05-25 12:00:22,324][04028] RunningMeanStd input shape: (1,) +[2025-05-25 12:00:22,342][04028] ConvEncoder: input_channels=3 +[2025-05-25 12:00:22,501][04028] Conv encoder output size: 512 +[2025-05-25 12:00:22,504][04028] Policy head output size: 512 +[2025-05-25 12:00:23,629][04028] Num frames 100... +[2025-05-25 12:00:23,762][04028] Num frames 200... +[2025-05-25 12:00:23,898][04028] Num frames 300... +[2025-05-25 12:00:24,022][04028] Num frames 400... +[2025-05-25 12:00:24,146][04028] Num frames 500... +[2025-05-25 12:00:24,273][04028] Num frames 600... +[2025-05-25 12:00:24,394][04028] Num frames 700... +[2025-05-25 12:00:24,520][04028] Num frames 800... +[2025-05-25 12:00:24,577][04028] Avg episode rewards: #0: 15.010, true rewards: #0: 8.010 +[2025-05-25 12:00:24,578][04028] Avg episode reward: 15.010, avg true_objective: 8.010 +[2025-05-25 12:00:24,700][04028] Num frames 900... +[2025-05-25 12:00:24,834][04028] Num frames 1000... +[2025-05-25 12:00:24,964][04028] Num frames 1100... +[2025-05-25 12:00:25,089][04028] Num frames 1200... +[2025-05-25 12:00:25,216][04028] Num frames 1300... +[2025-05-25 12:00:25,336][04028] Num frames 1400... +[2025-05-25 12:00:25,459][04028] Num frames 1500... +[2025-05-25 12:00:25,582][04028] Num frames 1600... +[2025-05-25 12:00:25,706][04028] Num frames 1700... +[2025-05-25 12:00:25,830][04028] Num frames 1800... +[2025-05-25 12:00:25,965][04028] Num frames 1900... +[2025-05-25 12:00:26,089][04028] Num frames 2000... +[2025-05-25 12:00:26,219][04028] Num frames 2100... +[2025-05-25 12:00:26,344][04028] Num frames 2200... +[2025-05-25 12:00:26,472][04028] Num frames 2300... +[2025-05-25 12:00:26,596][04028] Num frames 2400... +[2025-05-25 12:00:26,723][04028] Num frames 2500... +[2025-05-25 12:00:26,851][04028] Num frames 2600... +[2025-05-25 12:00:26,987][04028] Num frames 2700... +[2025-05-25 12:00:27,111][04028] Num frames 2800... +[2025-05-25 12:00:27,238][04028] Num frames 2900... +[2025-05-25 12:00:27,295][04028] Avg episode rewards: #0: 36.005, true rewards: #0: 14.505 +[2025-05-25 12:00:27,296][04028] Avg episode reward: 36.005, avg true_objective: 14.505 +[2025-05-25 12:00:27,413][04028] Num frames 3000... +[2025-05-25 12:00:27,537][04028] Num frames 3100... +[2025-05-25 12:00:27,660][04028] Num frames 3200... +[2025-05-25 12:00:27,783][04028] Num frames 3300... +[2025-05-25 12:00:27,907][04028] Num frames 3400... +[2025-05-25 12:00:28,038][04028] Num frames 3500... +[2025-05-25 12:00:28,164][04028] Num frames 3600... +[2025-05-25 12:00:28,266][04028] Avg episode rewards: #0: 29.123, true rewards: #0: 12.123 +[2025-05-25 12:00:28,267][04028] Avg episode reward: 29.123, avg true_objective: 12.123 +[2025-05-25 12:00:28,347][04028] Num frames 3700... +[2025-05-25 12:00:28,472][04028] Num frames 3800... +[2025-05-25 12:00:28,596][04028] Num frames 3900... +[2025-05-25 12:00:28,721][04028] Num frames 4000... +[2025-05-25 12:00:28,847][04028] Num frames 4100... +[2025-05-25 12:00:28,988][04028] Num frames 4200... +[2025-05-25 12:00:29,118][04028] Num frames 4300... +[2025-05-25 12:00:29,243][04028] Num frames 4400... +[2025-05-25 12:00:29,368][04028] Num frames 4500... +[2025-05-25 12:00:29,493][04028] Num frames 4600... +[2025-05-25 12:00:29,617][04028] Num frames 4700... +[2025-05-25 12:00:29,747][04028] Num frames 4800... +[2025-05-25 12:00:29,874][04028] Num frames 4900... +[2025-05-25 12:00:30,006][04028] Num frames 5000... +[2025-05-25 12:00:30,135][04028] Num frames 5100... +[2025-05-25 12:00:30,260][04028] Num frames 5200... +[2025-05-25 12:00:30,386][04028] Num frames 5300... +[2025-05-25 12:00:30,449][04028] Avg episode rewards: #0: 32.512, true rewards: #0: 13.263 +[2025-05-25 12:00:30,450][04028] Avg episode reward: 32.512, avg true_objective: 13.263 +[2025-05-25 12:00:30,569][04028] Num frames 5400... +[2025-05-25 12:00:30,692][04028] Num frames 5500... +[2025-05-25 12:00:30,817][04028] Num frames 5600... +[2025-05-25 12:00:30,942][04028] Avg episode rewards: #0: 27.314, true rewards: #0: 11.314 +[2025-05-25 12:00:30,943][04028] Avg episode reward: 27.314, avg true_objective: 11.314 +[2025-05-25 12:00:30,999][04028] Num frames 5700... +[2025-05-25 12:00:31,134][04028] Num frames 5800... +[2025-05-25 12:00:31,261][04028] Num frames 5900... +[2025-05-25 12:00:31,383][04028] Num frames 6000... +[2025-05-25 12:00:31,509][04028] Num frames 6100... +[2025-05-25 12:00:31,611][04028] Avg episode rewards: #0: 24.061, true rewards: #0: 10.228 +[2025-05-25 12:00:31,612][04028] Avg episode reward: 24.061, avg true_objective: 10.228 +[2025-05-25 12:00:31,689][04028] Num frames 6200... +[2025-05-25 12:00:31,816][04028] Num frames 6300... +[2025-05-25 12:00:31,940][04028] Num frames 6400... +[2025-05-25 12:00:32,072][04028] Num frames 6500... +[2025-05-25 12:00:32,197][04028] Num frames 6600... +[2025-05-25 12:00:32,324][04028] Num frames 6700... +[2025-05-25 12:00:32,447][04028] Num frames 6800... +[2025-05-25 12:00:32,572][04028] Num frames 6900... +[2025-05-25 12:00:32,695][04028] Num frames 7000... +[2025-05-25 12:00:32,820][04028] Num frames 7100... +[2025-05-25 12:00:32,913][04028] Avg episode rewards: #0: 23.613, true rewards: #0: 10.184 +[2025-05-25 12:00:32,914][04028] Avg episode reward: 23.613, avg true_objective: 10.184 +[2025-05-25 12:00:33,000][04028] Num frames 7200... +[2025-05-25 12:00:33,158][04028] Num frames 7300... +[2025-05-25 12:00:33,339][04028] Num frames 7400... +[2025-05-25 12:00:33,508][04028] Num frames 7500... +[2025-05-25 12:00:34,045][04028] Num frames 7600... +[2025-05-25 12:00:34,242][04028] Num frames 7700... +[2025-05-25 12:00:34,411][04028] Num frames 7800... +[2025-05-25 12:00:34,753][04028] Num frames 7900... +[2025-05-25 12:00:35,104][04028] Num frames 8000... +[2025-05-25 12:00:35,300][04028] Num frames 8100... +[2025-05-25 12:00:35,485][04028] Num frames 8200... +[2025-05-25 12:00:35,619][04028] Num frames 8300... +[2025-05-25 12:00:35,742][04028] Num frames 8400... +[2025-05-25 12:00:35,871][04028] Num frames 8500... +[2025-05-25 12:00:35,998][04028] Num frames 8600... +[2025-05-25 12:00:36,126][04028] Num frames 8700... +[2025-05-25 12:00:36,261][04028] Num frames 8800... +[2025-05-25 12:00:36,388][04028] Num frames 8900... +[2025-05-25 12:00:36,516][04028] Num frames 9000... +[2025-05-25 12:00:36,640][04028] Num frames 9100... +[2025-05-25 12:00:36,766][04028] Num frames 9200... +[2025-05-25 12:00:36,861][04028] Avg episode rewards: #0: 27.786, true rewards: #0: 11.536 +[2025-05-25 12:00:36,861][04028] Avg episode reward: 27.786, avg true_objective: 11.536 +[2025-05-25 12:00:36,949][04028] Num frames 9300... +[2025-05-25 12:00:37,077][04028] Num frames 9400... +[2025-05-25 12:00:37,201][04028] Num frames 9500... +[2025-05-25 12:00:37,336][04028] Num frames 9600... +[2025-05-25 12:00:37,460][04028] Num frames 9700... +[2025-05-25 12:00:37,586][04028] Num frames 9800... +[2025-05-25 12:00:37,648][04028] Avg episode rewards: #0: 25.783, true rewards: #0: 10.894 +[2025-05-25 12:00:37,649][04028] Avg episode reward: 25.783, avg true_objective: 10.894 +[2025-05-25 12:00:37,766][04028] Num frames 9900... +[2025-05-25 12:00:37,892][04028] Num frames 10000... +[2025-05-25 12:00:38,015][04028] Num frames 10100... +[2025-05-25 12:00:38,141][04028] Num frames 10200... +[2025-05-25 12:00:38,265][04028] Num frames 10300... +[2025-05-25 12:00:38,402][04028] Num frames 10400... +[2025-05-25 12:00:38,527][04028] Num frames 10500... +[2025-05-25 12:00:38,635][04028] Avg episode rewards: #0: 25.041, true rewards: #0: 10.541 +[2025-05-25 12:00:38,636][04028] Avg episode reward: 25.041, avg true_objective: 10.541 +[2025-05-25 12:01:39,338][04028] Replay video saved to /content/train_dir/default_experiment/replay.mp4! +[2025-05-25 12:02:40,173][04028] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json +[2025-05-25 12:02:40,174][04028] Overriding arg 'num_workers' with value 1 passed from command line +[2025-05-25 12:02:40,174][04028] Adding new argument 'no_render'=True that is not in the saved config file! +[2025-05-25 12:02:40,175][04028] Adding new argument 'save_video'=True that is not in the saved config file! +[2025-05-25 12:02:40,176][04028] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! +[2025-05-25 12:02:40,177][04028] Adding new argument 'video_name'=None that is not in the saved config file! +[2025-05-25 12:02:40,178][04028] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! +[2025-05-25 12:02:40,179][04028] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! +[2025-05-25 12:02:40,179][04028] Adding new argument 'push_to_hub'=True that is not in the saved config file! +[2025-05-25 12:02:40,180][04028] Adding new argument 'hf_repository'='wowthecoder/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! +[2025-05-25 12:02:40,181][04028] Adding new argument 'policy_index'=0 that is not in the saved config file! +[2025-05-25 12:02:40,182][04028] Adding new argument 'eval_deterministic'=False that is not in the saved config file! +[2025-05-25 12:02:40,182][04028] Adding new argument 'train_script'=None that is not in the saved config file! +[2025-05-25 12:02:40,183][04028] Adding new argument 'enjoy_script'=None that is not in the saved config file! +[2025-05-25 12:02:40,184][04028] Using frameskip 1 and render_action_repeat=4 for evaluation +[2025-05-25 12:02:40,215][04028] RunningMeanStd input shape: (3, 72, 128) +[2025-05-25 12:02:40,216][04028] RunningMeanStd input shape: (1,) +[2025-05-25 12:02:40,228][04028] ConvEncoder: input_channels=3 +[2025-05-25 12:02:40,261][04028] Conv encoder output size: 512 +[2025-05-25 12:02:40,261][04028] Policy head output size: 512 +[2025-05-25 12:02:40,722][04028] Num frames 100... +[2025-05-25 12:02:40,848][04028] Num frames 200... +[2025-05-25 12:02:40,975][04028] Num frames 300... +[2025-05-25 12:02:41,103][04028] Num frames 400... +[2025-05-25 12:02:41,230][04028] Num frames 500... +[2025-05-25 12:02:41,354][04028] Num frames 600... +[2025-05-25 12:02:41,476][04028] Num frames 700... +[2025-05-25 12:02:41,608][04028] Num frames 800... +[2025-05-25 12:02:41,731][04028] Num frames 900... +[2025-05-25 12:02:41,820][04028] Avg episode rewards: #0: 16.280, true rewards: #0: 9.280 +[2025-05-25 12:02:41,821][04028] Avg episode reward: 16.280, avg true_objective: 9.280 +[2025-05-25 12:02:41,912][04028] Num frames 1000... +[2025-05-25 12:02:42,040][04028] Num frames 1100... +[2025-05-25 12:02:42,162][04028] Num frames 1200... +[2025-05-25 12:02:42,288][04028] Num frames 1300... +[2025-05-25 12:02:42,412][04028] Num frames 1400... +[2025-05-25 12:02:42,536][04028] Num frames 1500... +[2025-05-25 12:02:42,672][04028] Num frames 1600... +[2025-05-25 12:02:42,798][04028] Num frames 1700... +[2025-05-25 12:02:42,921][04028] Num frames 1800... +[2025-05-25 12:02:43,048][04028] Num frames 1900... +[2025-05-25 12:02:43,184][04028] Num frames 2000... +[2025-05-25 12:02:43,338][04028] Avg episode rewards: #0: 21.900, true rewards: #0: 10.400 +[2025-05-25 12:02:43,339][04028] Avg episode reward: 21.900, avg true_objective: 10.400 +[2025-05-25 12:02:43,366][04028] Num frames 2100... +[2025-05-25 12:02:43,491][04028] Num frames 2200... +[2025-05-25 12:02:43,626][04028] Num frames 2300... +[2025-05-25 12:02:43,749][04028] Num frames 2400... +[2025-05-25 12:02:43,872][04028] Num frames 2500... +[2025-05-25 12:02:43,995][04028] Num frames 2600... +[2025-05-25 12:02:44,124][04028] Num frames 2700... +[2025-05-25 12:02:44,302][04028] Avg episode rewards: #0: 20.330, true rewards: #0: 9.330 +[2025-05-25 12:02:44,303][04028] Avg episode reward: 20.330, avg true_objective: 9.330 +[2025-05-25 12:02:44,306][04028] Num frames 2800... +[2025-05-25 12:02:44,429][04028] Num frames 2900... +[2025-05-25 12:02:44,549][04028] Num frames 3000... +[2025-05-25 12:02:44,683][04028] Num frames 3100... +[2025-05-25 12:02:44,805][04028] Num frames 3200... +[2025-05-25 12:02:44,927][04028] Num frames 3300... +[2025-05-25 12:02:45,051][04028] Num frames 3400... +[2025-05-25 12:02:45,214][04028] Num frames 3500... +[2025-05-25 12:02:45,336][04028] Num frames 3600... +[2025-05-25 12:02:45,456][04028] Num frames 3700... +[2025-05-25 12:02:45,544][04028] Avg episode rewards: #0: 19.318, true rewards: #0: 9.317 +[2025-05-25 12:02:45,545][04028] Avg episode reward: 19.318, avg true_objective: 9.317 +[2025-05-25 12:02:45,638][04028] Num frames 3800... +[2025-05-25 12:02:45,771][04028] Num frames 3900... +[2025-05-25 12:02:45,896][04028] Num frames 4000... +[2025-05-25 12:02:46,021][04028] Num frames 4100... +[2025-05-25 12:02:46,149][04028] Num frames 4200... +[2025-05-25 12:02:46,277][04028] Num frames 4300... +[2025-05-25 12:02:46,401][04028] Num frames 4400... +[2025-05-25 12:02:46,526][04028] Num frames 4500... +[2025-05-25 12:02:46,647][04028] Num frames 4600... +[2025-05-25 12:02:46,830][04028] Num frames 4700... +[2025-05-25 12:02:47,005][04028] Num frames 4800... +[2025-05-25 12:02:47,186][04028] Num frames 4900... +[2025-05-25 12:02:47,317][04028] Avg episode rewards: #0: 20.286, true rewards: #0: 9.886 +[2025-05-25 12:02:47,320][04028] Avg episode reward: 20.286, avg true_objective: 9.886 +[2025-05-25 12:02:47,424][04028] Num frames 5000... +[2025-05-25 12:02:47,591][04028] Num frames 5100... +[2025-05-25 12:02:47,766][04028] Num frames 5200... +[2025-05-25 12:02:47,940][04028] Num frames 5300... +[2025-05-25 12:02:48,112][04028] Num frames 5400... +[2025-05-25 12:02:48,289][04028] Num frames 5500... +[2025-05-25 12:02:48,383][04028] Avg episode rewards: #0: 18.532, true rewards: #0: 9.198 +[2025-05-25 12:02:48,384][04028] Avg episode reward: 18.532, avg true_objective: 9.198 +[2025-05-25 12:02:48,526][04028] Num frames 5600... +[2025-05-25 12:02:48,701][04028] Num frames 5700... +[2025-05-25 12:02:48,874][04028] Num frames 5800... +[2025-05-25 12:02:48,994][04028] Num frames 5900... +[2025-05-25 12:02:49,120][04028] Num frames 6000... +[2025-05-25 12:02:49,292][04028] Avg episode rewards: #0: 17.136, true rewards: #0: 8.707 +[2025-05-25 12:02:49,293][04028] Avg episode reward: 17.136, avg true_objective: 8.707 +[2025-05-25 12:02:49,301][04028] Num frames 6100... +[2025-05-25 12:02:49,422][04028] Num frames 6200... +[2025-05-25 12:02:49,545][04028] Num frames 6300... +[2025-05-25 12:02:49,666][04028] Num frames 6400... +[2025-05-25 12:02:49,788][04028] Num frames 6500... +[2025-05-25 12:02:49,924][04028] Num frames 6600... +[2025-05-25 12:02:50,049][04028] Num frames 6700... +[2025-05-25 12:02:50,179][04028] Num frames 6800... +[2025-05-25 12:02:50,305][04028] Avg episode rewards: #0: 16.818, true rewards: #0: 8.567 +[2025-05-25 12:02:50,306][04028] Avg episode reward: 16.818, avg true_objective: 8.567 +[2025-05-25 12:02:50,364][04028] Num frames 6900... +[2025-05-25 12:02:50,485][04028] Num frames 7000... +[2025-05-25 12:02:50,609][04028] Num frames 7100... +[2025-05-25 12:02:50,731][04028] Num frames 7200... +[2025-05-25 12:02:50,855][04028] Num frames 7300... +[2025-05-25 12:02:50,989][04028] Num frames 7400... +[2025-05-25 12:02:51,114][04028] Num frames 7500... +[2025-05-25 12:02:51,242][04028] Num frames 7600... +[2025-05-25 12:02:51,325][04028] Avg episode rewards: #0: 16.580, true rewards: #0: 8.469 +[2025-05-25 12:02:51,326][04028] Avg episode reward: 16.580, avg true_objective: 8.469 +[2025-05-25 12:02:51,423][04028] Num frames 7700... +[2025-05-25 12:02:51,548][04028] Num frames 7800... +[2025-05-25 12:02:51,669][04028] Num frames 7900... +[2025-05-25 12:02:51,792][04028] Num frames 8000... +[2025-05-25 12:02:51,921][04028] Num frames 8100... +[2025-05-25 12:02:52,050][04028] Num frames 8200... +[2025-05-25 12:02:52,178][04028] Num frames 8300... +[2025-05-25 12:02:52,304][04028] Num frames 8400... +[2025-05-25 12:02:52,445][04028] Avg episode rewards: #0: 16.671, true rewards: #0: 8.471 +[2025-05-25 12:02:52,446][04028] Avg episode reward: 16.671, avg true_objective: 8.471 +[2025-05-25 12:03:39,426][04028] Replay video saved to /content/train_dir/default_experiment/replay.mp4!