[2024-09-30 00:25:00,956][1148693] Saving configuration to /home/luyang/workspace/rl/train_dir/default_experiment/config.json... [2024-09-30 00:25:00,961][1148693] Rollout worker 0 uses device cpu [2024-09-30 00:25:00,961][1148693] Rollout worker 1 uses device cpu [2024-09-30 00:25:00,961][1148693] Rollout worker 2 uses device cpu [2024-09-30 00:25:00,961][1148693] Rollout worker 3 uses device cpu [2024-09-30 00:25:00,961][1148693] Rollout worker 4 uses device cpu [2024-09-30 00:25:00,961][1148693] Rollout worker 5 uses device cpu [2024-09-30 00:25:00,961][1148693] Rollout worker 6 uses device cpu [2024-09-30 00:25:00,962][1148693] Rollout worker 7 uses device cpu [2024-09-30 00:25:01,008][1148693] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-09-30 00:25:01,008][1148693] InferenceWorker_p0-w0: min num requests: 2 [2024-09-30 00:25:01,042][1148693] Starting all processes... [2024-09-30 00:25:01,042][1148693] Starting process learner_proc0 [2024-09-30 00:25:02,676][1148693] Starting all processes... [2024-09-30 00:25:02,680][1148981] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-09-30 00:25:02,680][1148981] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2024-09-30 00:25:02,680][1148693] Starting process inference_proc0-0 [2024-09-30 00:25:02,680][1148693] Starting process rollout_proc0 [2024-09-30 00:25:02,681][1148693] Starting process rollout_proc1 [2024-09-30 00:25:02,681][1148693] Starting process rollout_proc2 [2024-09-30 00:25:02,681][1148693] Starting process rollout_proc3 [2024-09-30 00:25:02,681][1148693] Starting process rollout_proc4 [2024-09-30 00:25:02,681][1148693] Starting process rollout_proc5 [2024-09-30 00:25:02,686][1148693] Starting process rollout_proc6 [2024-09-30 00:25:02,686][1148693] Starting process rollout_proc7 [2024-09-30 00:25:02,712][1148981] Num visible devices: 1 [2024-09-30 00:25:02,719][1148981] Starting seed is not provided [2024-09-30 00:25:02,719][1148981] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-09-30 00:25:02,719][1148981] Initializing actor-critic model on device cuda:0 [2024-09-30 00:25:02,719][1148981] RunningMeanStd input shape: (3, 72, 128) [2024-09-30 00:25:02,720][1148981] RunningMeanStd input shape: (1,) [2024-09-30 00:25:02,729][1148981] ConvEncoder: input_channels=3 [2024-09-30 00:25:02,801][1148981] Conv encoder output size: 512 [2024-09-30 00:25:02,801][1148981] Policy head output size: 512 [2024-09-30 00:25:02,812][1148981] Created Actor Critic model with architecture: [2024-09-30 00:25:02,813][1148981] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): VizdoomEncoder( (basic_encoder): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ELU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ELU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ELU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ELU) ) ) ) ) (core): ModelCoreRNN( (core): GRU(512, 512) ) (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=5, bias=True) ) ) [2024-09-30 00:25:02,951][1148981] Using optimizer [2024-09-30 00:25:03,366][1148693] Keyboard interrupt detected in the event loop EvtLoop [Runner_EvtLoop, process=main process 1148693], exiting... [2024-09-30 00:25:03,366][1148693] Runner profile tree view: main_loop: 2.3244 [2024-09-30 00:25:03,367][1148693] Collected {}, FPS: 0.0 [2024-09-30 00:25:03,367][1148981] Stopping Batcher_0... [2024-09-30 00:25:03,368][1148981] Loop batcher_evt_loop terminating... [2024-09-30 00:25:03,637][1148981] No checkpoints found [2024-09-30 00:25:03,637][1148981] Did not load from checkpoint, starting from scratch! [2024-09-30 00:25:03,637][1148981] Initialized policy 0 weights for model version 0 [2024-09-30 00:25:03,639][1148981] LearnerWorker_p0 finished initialization! [2024-09-30 00:25:03,640][1148981] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth... [2024-09-30 00:25:03,662][1148981] Stopping LearnerWorker_p0... [2024-09-30 00:25:03,662][1148981] Loop learner_proc0_evt_loop terminating... [2024-09-30 00:26:16,204][1149865] Saving configuration to /home/luyang/workspace/rl/train_dir/default_experiment/config.json... [2024-09-30 00:26:16,209][1149865] Rollout worker 0 uses device cpu [2024-09-30 00:26:16,209][1149865] Rollout worker 1 uses device cpu [2024-09-30 00:26:16,209][1149865] Rollout worker 2 uses device cpu [2024-09-30 00:26:16,209][1149865] Rollout worker 3 uses device cpu [2024-09-30 00:26:16,209][1149865] Rollout worker 4 uses device cpu [2024-09-30 00:26:16,209][1149865] Rollout worker 5 uses device cpu [2024-09-30 00:26:16,209][1149865] Rollout worker 6 uses device cpu [2024-09-30 00:26:16,209][1149865] Rollout worker 7 uses device cpu [2024-09-30 00:26:16,252][1149865] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-09-30 00:26:16,252][1149865] InferenceWorker_p0-w0: min num requests: 2 [2024-09-30 00:26:16,286][1149865] Starting all processes... [2024-09-30 00:26:16,286][1149865] Starting process learner_proc0 [2024-09-30 00:26:17,897][1149865] Starting all processes... [2024-09-30 00:26:17,901][1150061] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-09-30 00:26:17,901][1150061] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2024-09-30 00:26:17,901][1149865] Starting process inference_proc0-0 [2024-09-30 00:26:17,901][1149865] Starting process rollout_proc0 [2024-09-30 00:26:17,901][1149865] Starting process rollout_proc1 [2024-09-30 00:26:17,902][1149865] Starting process rollout_proc2 [2024-09-30 00:26:17,902][1149865] Starting process rollout_proc3 [2024-09-30 00:26:17,902][1149865] Starting process rollout_proc4 [2024-09-30 00:26:17,902][1149865] Starting process rollout_proc5 [2024-09-30 00:26:17,902][1149865] Starting process rollout_proc6 [2024-09-30 00:26:17,903][1149865] Starting process rollout_proc7 [2024-09-30 00:26:17,953][1150061] Num visible devices: 1 [2024-09-30 00:26:17,959][1150061] Starting seed is not provided [2024-09-30 00:26:17,959][1150061] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-09-30 00:26:17,959][1150061] Initializing actor-critic model on device cuda:0 [2024-09-30 00:26:17,959][1150061] RunningMeanStd input shape: (3, 72, 128) [2024-09-30 00:26:17,960][1150061] RunningMeanStd input shape: (1,) [2024-09-30 00:26:17,968][1150061] ConvEncoder: input_channels=3 [2024-09-30 00:26:18,041][1150061] Conv encoder output size: 512 [2024-09-30 00:26:18,041][1150061] Policy head output size: 512 [2024-09-30 00:26:18,052][1150061] Created Actor Critic model with architecture: [2024-09-30 00:26:18,052][1150061] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): VizdoomEncoder( (basic_encoder): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ELU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ELU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ELU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ELU) ) ) ) ) (core): ModelCoreRNN( (core): GRU(512, 512) ) (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=5, bias=True) ) ) [2024-09-30 00:26:18,183][1150061] Using optimizer [2024-09-30 00:26:18,816][1150061] Loading state from checkpoint /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth... [2024-09-30 00:26:18,828][1150061] Loading model from checkpoint [2024-09-30 00:26:18,829][1150061] Loaded experiment state at self.train_step=0, self.env_steps=0 [2024-09-30 00:26:18,829][1150061] Initialized policy 0 weights for model version 0 [2024-09-30 00:26:18,831][1150061] LearnerWorker_p0 finished initialization! [2024-09-30 00:26:18,831][1150061] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-09-30 00:26:19,422][1150142] Worker 3 uses CPU cores [36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47] [2024-09-30 00:26:19,449][1150140] Worker 7 uses CPU cores [84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95] [2024-09-30 00:26:19,451][1150144] Worker 0 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] [2024-09-30 00:26:19,456][1150145] Worker 6 uses CPU cores [72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83] [2024-09-30 00:26:19,456][1150137] Worker 5 uses CPU cores [60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71] [2024-09-30 00:26:19,462][1150141] Worker 4 uses CPU cores [48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59] [2024-09-30 00:26:19,465][1150143] Worker 1 uses CPU cores [12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23] [2024-09-30 00:26:19,466][1150138] Worker 2 uses CPU cores [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35] [2024-09-30 00:26:19,483][1150139] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-09-30 00:26:19,484][1150139] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 [2024-09-30 00:26:19,545][1150139] Num visible devices: 1 [2024-09-30 00:26:19,557][1149865] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2024-09-30 00:26:19,639][1150139] RunningMeanStd input shape: (3, 72, 128) [2024-09-30 00:26:19,640][1150139] RunningMeanStd input shape: (1,) [2024-09-30 00:26:19,648][1150139] ConvEncoder: input_channels=3 [2024-09-30 00:26:19,720][1150139] Conv encoder output size: 512 [2024-09-30 00:26:19,720][1150139] Policy head output size: 512 [2024-09-30 00:26:19,751][1149865] Inference worker 0-0 is ready! [2024-09-30 00:26:19,751][1149865] All inference workers are ready! Signal rollout workers to start! [2024-09-30 00:26:19,776][1150144] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:26:19,776][1150141] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:26:19,777][1150142] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:26:19,777][1150138] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:26:19,777][1150145] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:26:19,781][1150140] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:26:19,785][1150137] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:26:19,791][1150143] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:26:20,015][1150141] Decorrelating experience for 0 frames... [2024-09-30 00:26:20,019][1150142] Decorrelating experience for 0 frames... [2024-09-30 00:26:20,020][1150145] Decorrelating experience for 0 frames... [2024-09-30 00:26:20,020][1150138] Decorrelating experience for 0 frames... [2024-09-30 00:26:20,021][1150140] Decorrelating experience for 0 frames... [2024-09-30 00:26:20,028][1150137] Decorrelating experience for 0 frames... [2024-09-30 00:26:20,226][1150141] Decorrelating experience for 32 frames... [2024-09-30 00:26:20,233][1150142] Decorrelating experience for 32 frames... [2024-09-30 00:26:20,233][1150145] Decorrelating experience for 32 frames... [2024-09-30 00:26:20,239][1150137] Decorrelating experience for 32 frames... [2024-09-30 00:26:20,271][1150143] Decorrelating experience for 0 frames... [2024-09-30 00:26:20,481][1150143] Decorrelating experience for 32 frames... [2024-09-30 00:26:20,496][1150145] Decorrelating experience for 64 frames... [2024-09-30 00:26:20,508][1150142] Decorrelating experience for 64 frames... [2024-09-30 00:26:20,739][1150141] Decorrelating experience for 64 frames... [2024-09-30 00:26:20,745][1150142] Decorrelating experience for 96 frames... [2024-09-30 00:26:20,759][1150137] Decorrelating experience for 64 frames... [2024-09-30 00:26:20,987][1150141] Decorrelating experience for 96 frames... [2024-09-30 00:26:20,991][1150137] Decorrelating experience for 96 frames... [2024-09-30 00:26:20,993][1150143] Decorrelating experience for 64 frames... [2024-09-30 00:26:21,227][1150143] Decorrelating experience for 96 frames... [2024-09-30 00:26:21,234][1150145] Decorrelating experience for 96 frames... [2024-09-30 00:26:21,489][1150138] Decorrelating experience for 32 frames... [2024-09-30 00:26:21,652][1150061] Signal inference workers to stop experience collection... [2024-09-30 00:26:21,655][1150139] InferenceWorker_p0-w0: stopping experience collection [2024-09-30 00:26:21,743][1150140] Decorrelating experience for 32 frames... [2024-09-30 00:26:21,758][1150138] Decorrelating experience for 64 frames... [2024-09-30 00:26:21,995][1150138] Decorrelating experience for 96 frames... [2024-09-30 00:26:22,002][1150140] Decorrelating experience for 64 frames... [2024-09-30 00:26:22,237][1150140] Decorrelating experience for 96 frames... [2024-09-30 00:26:22,624][1150061] Signal inference workers to resume experience collection... [2024-09-30 00:26:22,624][1150139] InferenceWorker_p0-w0: resuming experience collection [2024-09-30 00:26:23,854][1150139] Updated weights for policy 0, policy_version 10 (0.0128) [2024-09-30 00:26:24,557][1149865] Fps is (10 sec: 12288.1, 60 sec: 12288.1, 300 sec: 12288.1). Total num frames: 61440. Throughput: 0: 484.0. Samples: 2420. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2024-09-30 00:26:24,557][1149865] Avg episode reward: [(0, '4.453')] [2024-09-30 00:26:25,059][1150139] Updated weights for policy 0, policy_version 20 (0.0006) [2024-09-30 00:26:26,154][1150139] Updated weights for policy 0, policy_version 30 (0.0006) [2024-09-30 00:26:27,304][1150139] Updated weights for policy 0, policy_version 40 (0.0006) [2024-09-30 00:26:28,424][1150139] Updated weights for policy 0, policy_version 50 (0.0006) [2024-09-30 00:26:29,557][1149865] Fps is (10 sec: 24166.4, 60 sec: 24166.4, 300 sec: 24166.4). Total num frames: 241664. Throughput: 0: 5481.2. Samples: 54812. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2024-09-30 00:26:29,557][1149865] Avg episode reward: [(0, '4.420')] [2024-09-30 00:26:29,562][1150061] Saving new best policy, reward=4.420! [2024-09-30 00:26:29,562][1150139] Updated weights for policy 0, policy_version 60 (0.0005) [2024-09-30 00:26:30,697][1150139] Updated weights for policy 0, policy_version 70 (0.0005) [2024-09-30 00:26:31,899][1150139] Updated weights for policy 0, policy_version 80 (0.0005) [2024-09-30 00:26:33,041][1150139] Updated weights for policy 0, policy_version 90 (0.0005) [2024-09-30 00:26:34,164][1150139] Updated weights for policy 0, policy_version 100 (0.0006) [2024-09-30 00:26:34,557][1149865] Fps is (10 sec: 36044.6, 60 sec: 28125.8, 300 sec: 28125.8). Total num frames: 421888. Throughput: 0: 5422.5. Samples: 81338. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) [2024-09-30 00:26:34,557][1149865] Avg episode reward: [(0, '4.360')] [2024-09-30 00:26:35,313][1150139] Updated weights for policy 0, policy_version 110 (0.0006) [2024-09-30 00:26:36,243][1149865] Heartbeat connected on Batcher_0 [2024-09-30 00:26:36,247][1149865] Heartbeat connected on LearnerWorker_p0 [2024-09-30 00:26:36,254][1149865] Heartbeat connected on InferenceWorker_p0-w0 [2024-09-30 00:26:36,261][1149865] Heartbeat connected on RolloutWorker_w1 [2024-09-30 00:26:36,265][1149865] Heartbeat connected on RolloutWorker_w2 [2024-09-30 00:26:36,270][1149865] Heartbeat connected on RolloutWorker_w3 [2024-09-30 00:26:36,273][1149865] Heartbeat connected on RolloutWorker_w4 [2024-09-30 00:26:36,278][1149865] Heartbeat connected on RolloutWorker_w5 [2024-09-30 00:26:36,283][1149865] Heartbeat connected on RolloutWorker_w6 [2024-09-30 00:26:36,286][1149865] Heartbeat connected on RolloutWorker_w7 [2024-09-30 00:26:36,388][1150139] Updated weights for policy 0, policy_version 120 (0.0006) [2024-09-30 00:26:37,485][1150139] Updated weights for policy 0, policy_version 130 (0.0005) [2024-09-30 00:26:38,623][1150139] Updated weights for policy 0, policy_version 140 (0.0005) [2024-09-30 00:26:39,557][1149865] Fps is (10 sec: 36454.4, 60 sec: 30310.4, 300 sec: 30310.4). Total num frames: 606208. Throughput: 0: 6816.1. Samples: 136322. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2024-09-30 00:26:39,557][1149865] Avg episode reward: [(0, '4.362')] [2024-09-30 00:26:39,793][1150139] Updated weights for policy 0, policy_version 150 (0.0005) [2024-09-30 00:26:40,942][1150139] Updated weights for policy 0, policy_version 160 (0.0006) [2024-09-30 00:26:42,120][1150139] Updated weights for policy 0, policy_version 170 (0.0006) [2024-09-30 00:26:43,255][1150139] Updated weights for policy 0, policy_version 180 (0.0006) [2024-09-30 00:26:44,363][1150139] Updated weights for policy 0, policy_version 190 (0.0006) [2024-09-30 00:26:44,557][1149865] Fps is (10 sec: 36044.9, 60 sec: 31293.4, 300 sec: 31293.4). Total num frames: 782336. Throughput: 0: 7583.6. Samples: 189590. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2024-09-30 00:26:44,557][1149865] Avg episode reward: [(0, '4.718')] [2024-09-30 00:26:44,560][1150061] Saving new best policy, reward=4.718! [2024-09-30 00:26:45,498][1150139] Updated weights for policy 0, policy_version 200 (0.0006) [2024-09-30 00:26:46,658][1150139] Updated weights for policy 0, policy_version 210 (0.0005) [2024-09-30 00:26:47,749][1150139] Updated weights for policy 0, policy_version 220 (0.0005) [2024-09-30 00:26:48,879][1150139] Updated weights for policy 0, policy_version 230 (0.0006) [2024-09-30 00:26:49,557][1149865] Fps is (10 sec: 36044.5, 60 sec: 32221.8, 300 sec: 32221.8). Total num frames: 966656. Throughput: 0: 7214.5. Samples: 216436. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-09-30 00:26:49,558][1149865] Avg episode reward: [(0, '4.984')] [2024-09-30 00:26:49,558][1150061] Saving new best policy, reward=4.984! [2024-09-30 00:26:49,996][1150139] Updated weights for policy 0, policy_version 240 (0.0006) [2024-09-30 00:26:51,078][1150139] Updated weights for policy 0, policy_version 250 (0.0005) [2024-09-30 00:26:52,165][1150139] Updated weights for policy 0, policy_version 260 (0.0005) [2024-09-30 00:26:53,360][1150139] Updated weights for policy 0, policy_version 270 (0.0006) [2024-09-30 00:26:54,557][1149865] Fps is (10 sec: 36044.3, 60 sec: 32650.8, 300 sec: 32650.8). Total num frames: 1142784. Throughput: 0: 7775.5. Samples: 272144. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2024-09-30 00:26:54,558][1149865] Avg episode reward: [(0, '6.825')] [2024-09-30 00:26:54,561][1150061] Saving new best policy, reward=6.825! [2024-09-30 00:26:54,627][1150139] Updated weights for policy 0, policy_version 280 (0.0006) [2024-09-30 00:26:55,804][1150139] Updated weights for policy 0, policy_version 290 (0.0006) [2024-09-30 00:26:56,921][1150139] Updated weights for policy 0, policy_version 300 (0.0006) [2024-09-30 00:26:58,061][1150139] Updated weights for policy 0, policy_version 310 (0.0006) [2024-09-30 00:26:59,306][1150139] Updated weights for policy 0, policy_version 320 (0.0006) [2024-09-30 00:26:59,557][1149865] Fps is (10 sec: 35225.8, 60 sec: 32972.8, 300 sec: 32972.8). Total num frames: 1318912. Throughput: 0: 8094.2. Samples: 323770. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-09-30 00:26:59,557][1149865] Avg episode reward: [(0, '7.969')] [2024-09-30 00:26:59,558][1150061] Saving new best policy, reward=7.969! [2024-09-30 00:27:00,379][1150139] Updated weights for policy 0, policy_version 330 (0.0006) [2024-09-30 00:27:01,493][1150139] Updated weights for policy 0, policy_version 340 (0.0006) [2024-09-30 00:27:02,648][1150139] Updated weights for policy 0, policy_version 350 (0.0005) [2024-09-30 00:27:03,846][1150139] Updated weights for policy 0, policy_version 360 (0.0006) [2024-09-30 00:27:04,557][1149865] Fps is (10 sec: 35635.7, 60 sec: 33314.1, 300 sec: 33314.1). Total num frames: 1499136. Throughput: 0: 7803.8. Samples: 351172. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2024-09-30 00:27:04,557][1149865] Avg episode reward: [(0, '9.395')] [2024-09-30 00:27:04,560][1150061] Saving new best policy, reward=9.395! [2024-09-30 00:27:04,958][1150139] Updated weights for policy 0, policy_version 370 (0.0005) [2024-09-30 00:27:06,158][1150139] Updated weights for policy 0, policy_version 380 (0.0006) [2024-09-30 00:27:07,341][1150139] Updated weights for policy 0, policy_version 390 (0.0006) [2024-09-30 00:27:08,471][1150139] Updated weights for policy 0, policy_version 400 (0.0006) [2024-09-30 00:27:09,550][1150139] Updated weights for policy 0, policy_version 410 (0.0005) [2024-09-30 00:27:09,557][1149865] Fps is (10 sec: 36045.1, 60 sec: 33587.2, 300 sec: 33587.2). Total num frames: 1679360. Throughput: 0: 8916.3. Samples: 403652. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-09-30 00:27:09,557][1149865] Avg episode reward: [(0, '10.451')] [2024-09-30 00:27:09,558][1150061] Saving new best policy, reward=10.451! [2024-09-30 00:27:10,625][1150139] Updated weights for policy 0, policy_version 420 (0.0005) [2024-09-30 00:27:11,753][1150139] Updated weights for policy 0, policy_version 430 (0.0006) [2024-09-30 00:27:12,906][1150139] Updated weights for policy 0, policy_version 440 (0.0005) [2024-09-30 00:27:14,043][1150139] Updated weights for policy 0, policy_version 450 (0.0005) [2024-09-30 00:27:14,557][1149865] Fps is (10 sec: 36044.8, 60 sec: 33810.6, 300 sec: 33810.6). Total num frames: 1859584. Throughput: 0: 8974.4. Samples: 458662. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2024-09-30 00:27:14,557][1149865] Avg episode reward: [(0, '13.438')] [2024-09-30 00:27:14,560][1150061] Saving new best policy, reward=13.438! [2024-09-30 00:27:15,159][1150139] Updated weights for policy 0, policy_version 460 (0.0006) [2024-09-30 00:27:16,224][1150139] Updated weights for policy 0, policy_version 470 (0.0006) [2024-09-30 00:27:17,339][1150139] Updated weights for policy 0, policy_version 480 (0.0006) [2024-09-30 00:27:18,411][1150139] Updated weights for policy 0, policy_version 490 (0.0006) [2024-09-30 00:27:19,490][1150139] Updated weights for policy 0, policy_version 500 (0.0006) [2024-09-30 00:27:19,557][1149865] Fps is (10 sec: 36863.4, 60 sec: 34133.3, 300 sec: 34133.3). Total num frames: 2048000. Throughput: 0: 9008.8. Samples: 486736. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-09-30 00:27:19,558][1149865] Avg episode reward: [(0, '15.719')] [2024-09-30 00:27:19,558][1150061] Saving new best policy, reward=15.719! [2024-09-30 00:27:20,560][1150139] Updated weights for policy 0, policy_version 510 (0.0006) [2024-09-30 00:27:21,675][1150139] Updated weights for policy 0, policy_version 520 (0.0006) [2024-09-30 00:27:22,733][1150139] Updated weights for policy 0, policy_version 530 (0.0006) [2024-09-30 00:27:23,821][1150139] Updated weights for policy 0, policy_version 540 (0.0006) [2024-09-30 00:27:24,557][1149865] Fps is (10 sec: 37683.2, 60 sec: 36249.6, 300 sec: 34406.4). Total num frames: 2236416. Throughput: 0: 9047.6. Samples: 543462. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) [2024-09-30 00:27:24,557][1149865] Avg episode reward: [(0, '18.072')] [2024-09-30 00:27:24,569][1150061] Saving new best policy, reward=18.072! [2024-09-30 00:27:24,893][1150139] Updated weights for policy 0, policy_version 550 (0.0006) [2024-09-30 00:27:25,971][1150139] Updated weights for policy 0, policy_version 560 (0.0005) [2024-09-30 00:27:27,037][1150139] Updated weights for policy 0, policy_version 570 (0.0006) [2024-09-30 00:27:28,155][1150139] Updated weights for policy 0, policy_version 580 (0.0005) [2024-09-30 00:27:29,272][1150139] Updated weights for policy 0, policy_version 590 (0.0006) [2024-09-30 00:27:29,557][1149865] Fps is (10 sec: 37683.7, 60 sec: 36386.1, 300 sec: 34640.5). Total num frames: 2424832. Throughput: 0: 9118.1. Samples: 599906. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2024-09-30 00:27:29,557][1149865] Avg episode reward: [(0, '20.999')] [2024-09-30 00:27:29,558][1150061] Saving new best policy, reward=20.999! [2024-09-30 00:27:30,323][1150139] Updated weights for policy 0, policy_version 600 (0.0006) [2024-09-30 00:27:31,407][1150139] Updated weights for policy 0, policy_version 610 (0.0006) [2024-09-30 00:27:32,467][1150139] Updated weights for policy 0, policy_version 620 (0.0006) [2024-09-30 00:27:33,593][1150139] Updated weights for policy 0, policy_version 630 (0.0006) [2024-09-30 00:27:34,557][1149865] Fps is (10 sec: 37683.3, 60 sec: 36522.7, 300 sec: 34843.3). Total num frames: 2613248. Throughput: 0: 9160.0. Samples: 628636. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2024-09-30 00:27:34,557][1149865] Avg episode reward: [(0, '20.389')] [2024-09-30 00:27:34,683][1150139] Updated weights for policy 0, policy_version 640 (0.0005) [2024-09-30 00:27:35,755][1150139] Updated weights for policy 0, policy_version 650 (0.0006) [2024-09-30 00:27:36,830][1150139] Updated weights for policy 0, policy_version 660 (0.0006) [2024-09-30 00:27:37,890][1150139] Updated weights for policy 0, policy_version 670 (0.0006) [2024-09-30 00:27:38,971][1150139] Updated weights for policy 0, policy_version 680 (0.0006) [2024-09-30 00:27:39,557][1149865] Fps is (10 sec: 38092.7, 60 sec: 36659.2, 300 sec: 35072.0). Total num frames: 2805760. Throughput: 0: 9181.5. Samples: 685308. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2024-09-30 00:27:39,557][1149865] Avg episode reward: [(0, '19.388')] [2024-09-30 00:27:40,031][1150139] Updated weights for policy 0, policy_version 690 (0.0006) [2024-09-30 00:27:41,100][1150139] Updated weights for policy 0, policy_version 700 (0.0005) [2024-09-30 00:27:42,153][1150139] Updated weights for policy 0, policy_version 710 (0.0006) [2024-09-30 00:27:43,230][1150139] Updated weights for policy 0, policy_version 720 (0.0006) [2024-09-30 00:27:44,312][1150139] Updated weights for policy 0, policy_version 730 (0.0006) [2024-09-30 00:27:44,557][1149865] Fps is (10 sec: 38502.4, 60 sec: 36932.3, 300 sec: 35273.8). Total num frames: 2998272. Throughput: 0: 9311.9. Samples: 742806. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2024-09-30 00:27:44,557][1149865] Avg episode reward: [(0, '22.197')] [2024-09-30 00:27:44,560][1150061] Saving new best policy, reward=22.197! [2024-09-30 00:27:45,376][1150139] Updated weights for policy 0, policy_version 740 (0.0006) [2024-09-30 00:27:46,438][1150139] Updated weights for policy 0, policy_version 750 (0.0006) [2024-09-30 00:27:47,515][1150139] Updated weights for policy 0, policy_version 760 (0.0005) [2024-09-30 00:27:48,572][1150139] Updated weights for policy 0, policy_version 770 (0.0006) [2024-09-30 00:27:49,557][1149865] Fps is (10 sec: 38502.4, 60 sec: 37068.9, 300 sec: 35453.2). Total num frames: 3190784. Throughput: 0: 9343.1. Samples: 771610. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) [2024-09-30 00:27:49,557][1149865] Avg episode reward: [(0, '21.389')] [2024-09-30 00:27:49,645][1150139] Updated weights for policy 0, policy_version 780 (0.0006) [2024-09-30 00:27:50,716][1150139] Updated weights for policy 0, policy_version 790 (0.0005) [2024-09-30 00:27:51,800][1150139] Updated weights for policy 0, policy_version 800 (0.0006) [2024-09-30 00:27:52,845][1150139] Updated weights for policy 0, policy_version 810 (0.0006) [2024-09-30 00:27:53,913][1150139] Updated weights for policy 0, policy_version 820 (0.0005) [2024-09-30 00:27:54,557][1149865] Fps is (10 sec: 38092.7, 60 sec: 37273.7, 300 sec: 35570.5). Total num frames: 3379200. Throughput: 0: 9454.0. Samples: 829084. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-09-30 00:27:54,557][1149865] Avg episode reward: [(0, '25.173')] [2024-09-30 00:27:54,560][1150061] Saving new best policy, reward=25.173! [2024-09-30 00:27:54,982][1150139] Updated weights for policy 0, policy_version 830 (0.0006) [2024-09-30 00:27:56,101][1150139] Updated weights for policy 0, policy_version 840 (0.0006) [2024-09-30 00:27:57,180][1150139] Updated weights for policy 0, policy_version 850 (0.0005) [2024-09-30 00:27:58,258][1150139] Updated weights for policy 0, policy_version 860 (0.0006) [2024-09-30 00:27:59,351][1150139] Updated weights for policy 0, policy_version 870 (0.0005) [2024-09-30 00:27:59,557][1149865] Fps is (10 sec: 38092.9, 60 sec: 37546.7, 300 sec: 35717.1). Total num frames: 3571712. Throughput: 0: 9494.8. Samples: 885926. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2024-09-30 00:27:59,557][1149865] Avg episode reward: [(0, '21.345')] [2024-09-30 00:28:00,433][1150139] Updated weights for policy 0, policy_version 880 (0.0006) [2024-09-30 00:28:01,499][1150139] Updated weights for policy 0, policy_version 890 (0.0006) [2024-09-30 00:28:02,611][1150139] Updated weights for policy 0, policy_version 900 (0.0006) [2024-09-30 00:28:03,681][1150139] Updated weights for policy 0, policy_version 910 (0.0006) [2024-09-30 00:28:04,557][1149865] Fps is (10 sec: 38092.8, 60 sec: 37683.2, 300 sec: 35810.7). Total num frames: 3760128. Throughput: 0: 9499.9. Samples: 914232. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-09-30 00:28:04,557][1149865] Avg episode reward: [(0, '25.023')] [2024-09-30 00:28:04,751][1150139] Updated weights for policy 0, policy_version 920 (0.0006) [2024-09-30 00:28:05,834][1150139] Updated weights for policy 0, policy_version 930 (0.0006) [2024-09-30 00:28:07,053][1150139] Updated weights for policy 0, policy_version 940 (0.0006) [2024-09-30 00:28:08,307][1150139] Updated weights for policy 0, policy_version 950 (0.0006) [2024-09-30 00:28:09,418][1150139] Updated weights for policy 0, policy_version 960 (0.0006) [2024-09-30 00:28:09,557][1149865] Fps is (10 sec: 36453.7, 60 sec: 37614.8, 300 sec: 35784.1). Total num frames: 3936256. Throughput: 0: 9452.1. Samples: 968808. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-09-30 00:28:09,558][1149865] Avg episode reward: [(0, '26.893')] [2024-09-30 00:28:09,558][1150061] Saving new best policy, reward=26.893! [2024-09-30 00:28:10,576][1150139] Updated weights for policy 0, policy_version 970 (0.0006) [2024-09-30 00:28:11,497][1149865] Component Batcher_0 stopped! [2024-09-30 00:28:11,497][1150061] Stopping Batcher_0... [2024-09-30 00:28:11,497][1149865] Component RolloutWorker_w0 process died already! Don't wait for it. [2024-09-30 00:28:11,497][1150061] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... [2024-09-30 00:28:11,498][1150061] Loop batcher_evt_loop terminating... [2024-09-30 00:28:11,513][1150139] Weights refcount: 2 0 [2024-09-30 00:28:11,514][1150139] Stopping InferenceWorker_p0-w0... [2024-09-30 00:28:11,514][1150139] Loop inference_proc0-0_evt_loop terminating... [2024-09-30 00:28:11,514][1149865] Component InferenceWorker_p0-w0 stopped! [2024-09-30 00:28:11,527][1150138] Stopping RolloutWorker_w2... [2024-09-30 00:28:11,527][1149865] Component RolloutWorker_w2 stopped! [2024-09-30 00:28:11,528][1150138] Loop rollout_proc2_evt_loop terminating... [2024-09-30 00:28:11,530][1150142] Stopping RolloutWorker_w3... [2024-09-30 00:28:11,530][1149865] Component RolloutWorker_w3 stopped! [2024-09-30 00:28:11,530][1150142] Loop rollout_proc3_evt_loop terminating... [2024-09-30 00:28:11,531][1149865] Component RolloutWorker_w5 stopped! [2024-09-30 00:28:11,531][1150137] Stopping RolloutWorker_w5... [2024-09-30 00:28:11,531][1149865] Component RolloutWorker_w6 stopped! [2024-09-30 00:28:11,531][1150145] Stopping RolloutWorker_w6... [2024-09-30 00:28:11,531][1150137] Loop rollout_proc5_evt_loop terminating... [2024-09-30 00:28:11,531][1150145] Loop rollout_proc6_evt_loop terminating... [2024-09-30 00:28:11,532][1149865] Component RolloutWorker_w1 stopped! [2024-09-30 00:28:11,532][1150143] Stopping RolloutWorker_w1... [2024-09-30 00:28:11,533][1150143] Loop rollout_proc1_evt_loop terminating... [2024-09-30 00:28:11,533][1149865] Component RolloutWorker_w4 stopped! [2024-09-30 00:28:11,533][1150141] Stopping RolloutWorker_w4... [2024-09-30 00:28:11,533][1150141] Loop rollout_proc4_evt_loop terminating... [2024-09-30 00:28:11,536][1149865] Component RolloutWorker_w7 stopped! [2024-09-30 00:28:11,536][1150140] Stopping RolloutWorker_w7... [2024-09-30 00:28:11,536][1150140] Loop rollout_proc7_evt_loop terminating... [2024-09-30 00:28:11,548][1150061] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... [2024-09-30 00:28:11,677][1150061] Stopping LearnerWorker_p0... [2024-09-30 00:28:11,677][1150061] Loop learner_proc0_evt_loop terminating... [2024-09-30 00:28:11,677][1149865] Component LearnerWorker_p0 stopped! [2024-09-30 00:28:11,678][1149865] Waiting for process learner_proc0 to stop... [2024-09-30 00:28:12,213][1149865] Waiting for process inference_proc0-0 to join... [2024-09-30 00:28:12,214][1149865] Waiting for process rollout_proc0 to join... [2024-09-30 00:28:12,214][1149865] Waiting for process rollout_proc1 to join... [2024-09-30 00:28:12,214][1149865] Waiting for process rollout_proc2 to join... [2024-09-30 00:28:12,214][1149865] Waiting for process rollout_proc3 to join... [2024-09-30 00:28:12,214][1149865] Waiting for process rollout_proc4 to join... [2024-09-30 00:28:12,215][1149865] Waiting for process rollout_proc5 to join... [2024-09-30 00:28:12,215][1149865] Waiting for process rollout_proc6 to join... [2024-09-30 00:28:12,215][1149865] Waiting for process rollout_proc7 to join... [2024-09-30 00:28:12,215][1149865] Batcher 0 profile tree view: batching: 8.1702, releasing_batches: 0.0148 [2024-09-30 00:28:12,215][1149865] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0000 wait_policy_total: 2.2430 update_model: 1.6718 weight_update: 0.0006 one_step: 0.0013 handle_policy_step: 101.6105 deserialize: 4.2527, stack: 0.5251, obs_to_device_normalize: 21.3149, forward: 52.1177, send_messages: 6.7725 prepare_outputs: 11.8901 to_cpu: 6.4354 [2024-09-30 00:28:12,216][1149865] Learner 0 profile tree view: misc: 0.0031, prepare_batch: 4.0428 train: 10.3860 epoch_init: 0.0033, minibatch_init: 0.0037, losses_postprocess: 0.1662, kl_divergence: 0.2113, after_optimizer: 0.8304 calculate_losses: 4.6270 losses_init: 0.0020, forward_head: 0.3762, bptt_initial: 2.3802, tail: 0.3318, advantages_returns: 0.0873, losses: 0.6229 bptt: 0.7209 bptt_forward_core: 0.6909 update: 4.3204 clip: 0.4495 [2024-09-30 00:28:12,216][1149865] RolloutWorker_w7 profile tree view: wait_for_trajectories: 0.0819, enqueue_policy_requests: 4.5546, env_step: 67.0989, overhead: 3.2408, complete_rollouts: 0.1226 save_policy_outputs: 5.6070 split_output_tensors: 1.8787 [2024-09-30 00:28:12,216][1149865] Loop Runner_EvtLoop terminating... [2024-09-30 00:28:12,216][1149865] Runner profile tree view: main_loop: 115.9303 [2024-09-30 00:28:12,216][1149865] Collected {0: 4005888}, FPS: 34554.3 [2024-09-30 00:28:12,419][1149865] Loading existing experiment configuration from /home/luyang/workspace/rl/train_dir/default_experiment/config.json [2024-09-30 00:28:12,419][1149865] Overriding arg 'num_workers' with value 1 passed from command line [2024-09-30 00:28:12,420][1149865] Adding new argument 'no_render'=True that is not in the saved config file! [2024-09-30 00:28:12,420][1149865] Adding new argument 'save_video'=True that is not in the saved config file! [2024-09-30 00:28:12,420][1149865] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2024-09-30 00:28:12,420][1149865] Adding new argument 'video_name'=None that is not in the saved config file! [2024-09-30 00:28:12,420][1149865] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! [2024-09-30 00:28:12,420][1149865] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2024-09-30 00:28:12,420][1149865] Adding new argument 'push_to_hub'=True that is not in the saved config file! [2024-09-30 00:28:12,420][1149865] Adding new argument 'hf_repository'='esperesa/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! [2024-09-30 00:28:12,420][1149865] Adding new argument 'policy_index'=0 that is not in the saved config file! [2024-09-30 00:28:12,420][1149865] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2024-09-30 00:28:12,420][1149865] Adding new argument 'train_script'=None that is not in the saved config file! [2024-09-30 00:28:12,420][1149865] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2024-09-30 00:28:12,420][1149865] Using frameskip 1 and render_action_repeat=4 for evaluation [2024-09-30 00:28:12,441][1149865] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:28:12,443][1149865] RunningMeanStd input shape: (3, 72, 128) [2024-09-30 00:28:12,443][1149865] RunningMeanStd input shape: (1,) [2024-09-30 00:28:12,452][1149865] ConvEncoder: input_channels=3 [2024-09-30 00:28:12,522][1149865] Conv encoder output size: 512 [2024-09-30 00:28:12,522][1149865] Policy head output size: 512 [2024-09-30 00:28:12,681][1149865] Loading state from checkpoint /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... [2024-09-30 00:28:13,271][1149865] Num frames 100... [2024-09-30 00:28:13,350][1149865] Num frames 200... [2024-09-30 00:28:13,427][1149865] Num frames 300... [2024-09-30 00:28:13,503][1149865] Num frames 400... [2024-09-30 00:28:13,581][1149865] Num frames 500... [2024-09-30 00:28:13,659][1149865] Num frames 600... [2024-09-30 00:28:13,738][1149865] Num frames 700... [2024-09-30 00:28:13,815][1149865] Num frames 800... [2024-09-30 00:28:13,894][1149865] Num frames 900... [2024-09-30 00:28:13,973][1149865] Num frames 1000... [2024-09-30 00:28:14,052][1149865] Num frames 1100... [2024-09-30 00:28:14,132][1149865] Num frames 1200... [2024-09-30 00:28:14,208][1149865] Num frames 1300... [2024-09-30 00:28:14,285][1149865] Num frames 1400... [2024-09-30 00:28:14,364][1149865] Num frames 1500... [2024-09-30 00:28:14,445][1149865] Avg episode rewards: #0: 34.360, true rewards: #0: 15.360 [2024-09-30 00:28:14,445][1149865] Avg episode reward: 34.360, avg true_objective: 15.360 [2024-09-30 00:28:14,497][1149865] Num frames 1600... [2024-09-30 00:28:14,575][1149865] Num frames 1700... [2024-09-30 00:28:14,654][1149865] Num frames 1800... [2024-09-30 00:28:14,733][1149865] Num frames 1900... [2024-09-30 00:28:14,811][1149865] Num frames 2000... [2024-09-30 00:28:14,891][1149865] Num frames 2100... [2024-09-30 00:28:14,967][1149865] Num frames 2200... [2024-09-30 00:28:15,045][1149865] Num frames 2300... [2024-09-30 00:28:15,125][1149865] Num frames 2400... [2024-09-30 00:28:15,204][1149865] Num frames 2500... [2024-09-30 00:28:15,283][1149865] Num frames 2600... [2024-09-30 00:28:15,363][1149865] Num frames 2700... [2024-09-30 00:28:15,443][1149865] Num frames 2800... [2024-09-30 00:28:15,523][1149865] Num frames 2900... [2024-09-30 00:28:15,602][1149865] Num frames 3000... [2024-09-30 00:28:15,681][1149865] Num frames 3100... [2024-09-30 00:28:15,758][1149865] Num frames 3200... [2024-09-30 00:28:15,834][1149865] Num frames 3300... [2024-09-30 00:28:15,958][1149865] Avg episode rewards: #0: 40.959, true rewards: #0: 16.960 [2024-09-30 00:28:15,958][1149865] Avg episode reward: 40.959, avg true_objective: 16.960 [2024-09-30 00:28:15,966][1149865] Num frames 3400... [2024-09-30 00:28:16,049][1149865] Num frames 3500... [2024-09-30 00:28:16,127][1149865] Num frames 3600... [2024-09-30 00:28:16,205][1149865] Num frames 3700... [2024-09-30 00:28:16,284][1149865] Num frames 3800... [2024-09-30 00:28:16,363][1149865] Num frames 3900... [2024-09-30 00:28:16,443][1149865] Num frames 4000... [2024-09-30 00:28:16,521][1149865] Num frames 4100... [2024-09-30 00:28:16,598][1149865] Num frames 4200... [2024-09-30 00:28:16,675][1149865] Num frames 4300... [2024-09-30 00:28:16,754][1149865] Num frames 4400... [2024-09-30 00:28:16,832][1149865] Num frames 4500... [2024-09-30 00:28:16,919][1149865] Avg episode rewards: #0: 36.480, true rewards: #0: 15.147 [2024-09-30 00:28:16,919][1149865] Avg episode reward: 36.480, avg true_objective: 15.147 [2024-09-30 00:28:16,968][1149865] Num frames 4600... [2024-09-30 00:28:17,047][1149865] Num frames 4700... [2024-09-30 00:28:17,126][1149865] Num frames 4800... [2024-09-30 00:28:17,206][1149865] Num frames 4900... [2024-09-30 00:28:17,282][1149865] Num frames 5000... [2024-09-30 00:28:17,358][1149865] Num frames 5100... [2024-09-30 00:28:17,470][1149865] Avg episode rewards: #0: 31.192, true rewards: #0: 12.942 [2024-09-30 00:28:17,471][1149865] Avg episode reward: 31.192, avg true_objective: 12.942 [2024-09-30 00:28:17,490][1149865] Num frames 5200... [2024-09-30 00:28:17,569][1149865] Num frames 5300... [2024-09-30 00:28:17,649][1149865] Num frames 5400... [2024-09-30 00:28:17,729][1149865] Num frames 5500... [2024-09-30 00:28:17,807][1149865] Num frames 5600... [2024-09-30 00:28:17,887][1149865] Num frames 5700... [2024-09-30 00:28:17,966][1149865] Num frames 5800... [2024-09-30 00:28:18,043][1149865] Num frames 5900... [2024-09-30 00:28:18,120][1149865] Num frames 6000... [2024-09-30 00:28:18,196][1149865] Num frames 6100... [2024-09-30 00:28:18,274][1149865] Num frames 6200... [2024-09-30 00:28:18,352][1149865] Num frames 6300... [2024-09-30 00:28:18,477][1149865] Avg episode rewards: #0: 30.786, true rewards: #0: 12.786 [2024-09-30 00:28:18,477][1149865] Avg episode reward: 30.786, avg true_objective: 12.786 [2024-09-30 00:28:18,484][1149865] Num frames 6400... [2024-09-30 00:28:18,564][1149865] Num frames 6500... [2024-09-30 00:28:18,643][1149865] Num frames 6600... [2024-09-30 00:28:18,723][1149865] Num frames 6700... [2024-09-30 00:28:18,803][1149865] Num frames 6800... [2024-09-30 00:28:18,879][1149865] Num frames 6900... [2024-09-30 00:28:18,957][1149865] Num frames 7000... [2024-09-30 00:28:19,033][1149865] Num frames 7100... [2024-09-30 00:28:19,110][1149865] Num frames 7200... [2024-09-30 00:28:19,190][1149865] Num frames 7300... [2024-09-30 00:28:19,271][1149865] Num frames 7400... [2024-09-30 00:28:19,353][1149865] Num frames 7500... [2024-09-30 00:28:19,443][1149865] Num frames 7600... [2024-09-30 00:28:19,538][1149865] Num frames 7700... [2024-09-30 00:28:19,630][1149865] Num frames 7800... [2024-09-30 00:28:19,721][1149865] Num frames 7900... [2024-09-30 00:28:19,817][1149865] Num frames 8000... [2024-09-30 00:28:19,910][1149865] Num frames 8100... [2024-09-30 00:28:20,014][1149865] Avg episode rewards: #0: 33.588, true rewards: #0: 13.588 [2024-09-30 00:28:20,015][1149865] Avg episode reward: 33.588, avg true_objective: 13.588 [2024-09-30 00:28:20,062][1149865] Num frames 8200... [2024-09-30 00:28:20,156][1149865] Num frames 8300... [2024-09-30 00:28:20,246][1149865] Num frames 8400... [2024-09-30 00:28:20,341][1149865] Num frames 8500... [2024-09-30 00:28:20,433][1149865] Num frames 8600... [2024-09-30 00:28:20,523][1149865] Num frames 8700... [2024-09-30 00:28:20,616][1149865] Num frames 8800... [2024-09-30 00:28:20,707][1149865] Num frames 8900... [2024-09-30 00:28:20,800][1149865] Num frames 9000... [2024-09-30 00:28:20,892][1149865] Num frames 9100... [2024-09-30 00:28:20,986][1149865] Num frames 9200... [2024-09-30 00:28:21,079][1149865] Num frames 9300... [2024-09-30 00:28:21,172][1149865] Num frames 9400... [2024-09-30 00:28:21,264][1149865] Num frames 9500... [2024-09-30 00:28:21,356][1149865] Num frames 9600... [2024-09-30 00:28:21,451][1149865] Num frames 9700... [2024-09-30 00:28:21,526][1149865] Avg episode rewards: #0: 34.030, true rewards: #0: 13.887 [2024-09-30 00:28:21,526][1149865] Avg episode reward: 34.030, avg true_objective: 13.887 [2024-09-30 00:28:21,591][1149865] Num frames 9800... [2024-09-30 00:28:21,672][1149865] Num frames 9900... [2024-09-30 00:28:21,757][1149865] Num frames 10000... [2024-09-30 00:28:21,850][1149865] Num frames 10100... [2024-09-30 00:28:21,945][1149865] Num frames 10200... [2024-09-30 00:28:22,035][1149865] Num frames 10300... [2024-09-30 00:28:22,128][1149865] Num frames 10400... [2024-09-30 00:28:22,220][1149865] Num frames 10500... [2024-09-30 00:28:22,312][1149865] Num frames 10600... [2024-09-30 00:28:22,393][1149865] Num frames 10700... [2024-09-30 00:28:22,474][1149865] Num frames 10800... [2024-09-30 00:28:22,537][1149865] Avg episode rewards: #0: 32.886, true rewards: #0: 13.511 [2024-09-30 00:28:22,537][1149865] Avg episode reward: 32.886, avg true_objective: 13.511 [2024-09-30 00:28:22,621][1149865] Num frames 10900... [2024-09-30 00:28:22,714][1149865] Num frames 11000... [2024-09-30 00:28:22,806][1149865] Num frames 11100... [2024-09-30 00:28:22,898][1149865] Num frames 11200... [2024-09-30 00:28:22,990][1149865] Num frames 11300... [2024-09-30 00:28:23,082][1149865] Num frames 11400... [2024-09-30 00:28:23,165][1149865] Num frames 11500... [2024-09-30 00:28:23,247][1149865] Num frames 11600... [2024-09-30 00:28:23,338][1149865] Num frames 11700... [2024-09-30 00:28:23,432][1149865] Num frames 11800... [2024-09-30 00:28:23,522][1149865] Num frames 11900... [2024-09-30 00:28:23,616][1149865] Num frames 12000... [2024-09-30 00:28:23,730][1149865] Num frames 12100... [2024-09-30 00:28:23,823][1149865] Num frames 12200... [2024-09-30 00:28:23,904][1149865] Num frames 12300... [2024-09-30 00:28:23,983][1149865] Num frames 12400... [2024-09-30 00:28:24,063][1149865] Num frames 12500... [2024-09-30 00:28:24,150][1149865] Num frames 12600... [2024-09-30 00:28:24,230][1149865] Num frames 12700... [2024-09-30 00:28:24,306][1149865] Num frames 12800... [2024-09-30 00:28:24,390][1149865] Avg episode rewards: #0: 35.268, true rewards: #0: 14.268 [2024-09-30 00:28:24,391][1149865] Avg episode reward: 35.268, avg true_objective: 14.268 [2024-09-30 00:28:24,438][1149865] Num frames 12900... [2024-09-30 00:28:24,516][1149865] Num frames 13000... [2024-09-30 00:28:24,594][1149865] Num frames 13100... [2024-09-30 00:28:24,675][1149865] Num frames 13200... [2024-09-30 00:28:24,755][1149865] Num frames 13300... [2024-09-30 00:28:24,833][1149865] Num frames 13400... [2024-09-30 00:28:24,946][1149865] Avg episode rewards: #0: 33.076, true rewards: #0: 13.476 [2024-09-30 00:28:24,946][1149865] Avg episode reward: 33.076, avg true_objective: 13.476 [2024-09-30 00:28:42,313][1149865] Replay video saved to /home/luyang/workspace/rl/train_dir/default_experiment/replay.mp4! [2024-09-30 00:29:17,655][1149865] The model has been pushed to https://huggingface.co/esperesa/rl_course_vizdoom_health_gathering_supreme [2024-09-30 00:33:37,627][1153456] Saving configuration to /home/luyang/workspace/rl/train_dir/default_experiment/config.json... [2024-09-30 00:33:37,631][1153456] Rollout worker 0 uses device cpu [2024-09-30 00:33:37,631][1153456] Rollout worker 1 uses device cpu [2024-09-30 00:33:37,631][1153456] Rollout worker 2 uses device cpu [2024-09-30 00:33:37,631][1153456] Rollout worker 3 uses device cpu [2024-09-30 00:33:37,632][1153456] Rollout worker 4 uses device cpu [2024-09-30 00:33:37,632][1153456] Rollout worker 5 uses device cpu [2024-09-30 00:33:37,632][1153456] Rollout worker 6 uses device cpu [2024-09-30 00:33:37,632][1153456] Rollout worker 7 uses device cpu [2024-09-30 00:33:37,632][1153456] Rollout worker 8 uses device cpu [2024-09-30 00:33:37,632][1153456] Rollout worker 9 uses device cpu [2024-09-30 00:33:37,632][1153456] Rollout worker 10 uses device cpu [2024-09-30 00:33:37,632][1153456] Rollout worker 11 uses device cpu [2024-09-30 00:33:37,632][1153456] Rollout worker 12 uses device cpu [2024-09-30 00:33:37,632][1153456] Rollout worker 13 uses device cpu [2024-09-30 00:33:37,632][1153456] Rollout worker 14 uses device cpu [2024-09-30 00:33:37,633][1153456] Rollout worker 15 uses device cpu [2024-09-30 00:33:37,744][1153456] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-09-30 00:33:37,745][1153456] InferenceWorker_p0-w0: min num requests: 5 [2024-09-30 00:33:37,828][1153456] Starting all processes... [2024-09-30 00:33:37,828][1153456] Starting process learner_proc0 [2024-09-30 00:33:39,432][1153456] Starting all processes... [2024-09-30 00:33:39,436][1153683] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-09-30 00:33:39,437][1153683] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2024-09-30 00:33:39,438][1153456] Starting process inference_proc0-0 [2024-09-30 00:33:39,438][1153456] Starting process rollout_proc0 [2024-09-30 00:33:39,438][1153456] Starting process rollout_proc1 [2024-09-30 00:33:39,441][1153456] Starting process rollout_proc2 [2024-09-30 00:33:39,441][1153456] Starting process rollout_proc3 [2024-09-30 00:33:39,444][1153456] Starting process rollout_proc4 [2024-09-30 00:33:39,444][1153456] Starting process rollout_proc5 [2024-09-30 00:33:39,444][1153456] Starting process rollout_proc6 [2024-09-30 00:33:39,444][1153456] Starting process rollout_proc7 [2024-09-30 00:33:39,447][1153456] Starting process rollout_proc8 [2024-09-30 00:33:39,454][1153456] Starting process rollout_proc9 [2024-09-30 00:33:39,454][1153456] Starting process rollout_proc10 [2024-09-30 00:33:39,455][1153456] Starting process rollout_proc11 [2024-09-30 00:33:39,455][1153456] Starting process rollout_proc12 [2024-09-30 00:33:39,458][1153456] Starting process rollout_proc13 [2024-09-30 00:33:39,460][1153456] Starting process rollout_proc14 [2024-09-30 00:33:39,478][1153683] Num visible devices: 1 [2024-09-30 00:33:39,484][1153683] Starting seed is not provided [2024-09-30 00:33:39,484][1153683] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-09-30 00:33:39,485][1153683] Initializing actor-critic model on device cuda:0 [2024-09-30 00:33:39,485][1153683] RunningMeanStd input shape: (3, 72, 128) [2024-09-30 00:33:39,485][1153683] RunningMeanStd input shape: (1,) [2024-09-30 00:33:39,494][1153683] ConvEncoder: input_channels=3 [2024-09-30 00:33:39,566][1153683] Conv encoder output size: 512 [2024-09-30 00:33:39,566][1153683] Policy head output size: 512 [2024-09-30 00:33:39,578][1153683] Created Actor Critic model with architecture: [2024-09-30 00:33:39,578][1153683] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): VizdoomEncoder( (basic_encoder): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ELU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ELU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ELU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ELU) ) ) ) ) (core): ModelCoreRNN( (core): GRU(512, 512) ) (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=5, bias=True) ) ) [2024-09-30 00:33:39,783][1153683] Using optimizer [2024-09-30 00:33:40,533][1153683] Loading state from checkpoint /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... [2024-09-30 00:33:40,555][1153683] Loading model from checkpoint [2024-09-30 00:33:40,556][1153683] Loaded experiment state at self.train_step=978, self.env_steps=4005888 [2024-09-30 00:33:40,556][1153683] Initialized policy 0 weights for model version 978 [2024-09-30 00:33:40,558][1153683] LearnerWorker_p0 finished initialization! [2024-09-30 00:33:40,558][1153683] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-09-30 00:33:41,097][1153456] Starting process rollout_proc15 [2024-09-30 00:33:41,101][1153813] Worker 7 uses CPU cores [42, 43, 44, 45, 46, 47] [2024-09-30 00:33:41,103][1153807] Worker 1 uses CPU cores [6, 7, 8, 9, 10, 11] [2024-09-30 00:33:41,104][1153814] Worker 8 uses CPU cores [48, 49, 50, 51, 52, 53] [2024-09-30 00:33:41,118][1153809] Worker 2 uses CPU cores [12, 13, 14, 15, 16, 17] [2024-09-30 00:33:41,128][1153812] Worker 5 uses CPU cores [30, 31, 32, 33, 34, 35] [2024-09-30 00:33:41,140][1153808] Worker 3 uses CPU cores [18, 19, 20, 21, 22, 23] [2024-09-30 00:33:41,157][1153816] Worker 10 uses CPU cores [60, 61, 62, 63, 64, 65] [2024-09-30 00:33:41,178][1153881] Worker 14 uses CPU cores [84, 85, 86, 87, 88, 89] [2024-09-30 00:33:41,182][1153882] Worker 11 uses CPU cores [66, 67, 68, 69, 70, 71] [2024-09-30 00:33:41,188][1153806] Worker 0 uses CPU cores [0, 1, 2, 3, 4, 5] [2024-09-30 00:33:41,197][1153880] Worker 13 uses CPU cores [78, 79, 80, 81, 82, 83] [2024-09-30 00:33:41,206][1153810] Worker 4 uses CPU cores [24, 25, 26, 27, 28, 29] [2024-09-30 00:33:41,206][1153883] Worker 12 uses CPU cores [72, 73, 74, 75, 76, 77] [2024-09-30 00:33:41,210][1153805] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-09-30 00:33:41,210][1153805] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 [2024-09-30 00:33:41,223][1153811] Worker 6 uses CPU cores [36, 37, 38, 39, 40, 41] [2024-09-30 00:33:41,240][1153815] Worker 9 uses CPU cores [54, 55, 56, 57, 58, 59] [2024-09-30 00:33:41,305][1153805] Num visible devices: 1 [2024-09-30 00:33:41,407][1153805] RunningMeanStd input shape: (3, 72, 128) [2024-09-30 00:33:41,408][1153805] RunningMeanStd input shape: (1,) [2024-09-30 00:33:41,416][1153805] ConvEncoder: input_channels=3 [2024-09-30 00:33:41,488][1153805] Conv encoder output size: 512 [2024-09-30 00:33:41,488][1153805] Policy head output size: 512 [2024-09-30 00:33:42,519][1153456] Inference worker 0-0 is ready! [2024-09-30 00:33:42,519][1153456] All inference workers are ready! Signal rollout workers to start! [2024-09-30 00:33:42,520][1153456] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 4005888. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2024-09-30 00:33:42,521][1154909] Worker 15 uses CPU cores [90, 91, 92, 93, 94, 95] [2024-09-30 00:33:42,544][1153882] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:33:42,545][1153808] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:33:42,546][1153806] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:33:42,546][1153810] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:33:42,550][1153814] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:33:42,551][1153883] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:33:42,552][1153813] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:33:42,554][1153811] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:33:42,555][1153881] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:33:42,565][1153807] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:33:42,565][1153815] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:33:42,567][1153809] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:33:42,567][1153816] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:33:42,570][1153880] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:33:42,570][1153812] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:33:42,640][1154909] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:33:42,822][1153882] Decorrelating experience for 0 frames... [2024-09-30 00:33:42,823][1153808] Decorrelating experience for 0 frames... [2024-09-30 00:33:42,827][1153810] Decorrelating experience for 0 frames... [2024-09-30 00:33:42,827][1153883] Decorrelating experience for 0 frames... [2024-09-30 00:33:42,829][1153814] Decorrelating experience for 0 frames... [2024-09-30 00:33:42,833][1153811] Decorrelating experience for 0 frames... [2024-09-30 00:33:42,842][1153815] Decorrelating experience for 0 frames... [2024-09-30 00:33:42,846][1153809] Decorrelating experience for 0 frames... [2024-09-30 00:33:43,038][1153882] Decorrelating experience for 32 frames... [2024-09-30 00:33:43,040][1153813] Decorrelating experience for 0 frames... [2024-09-30 00:33:43,042][1153814] Decorrelating experience for 32 frames... [2024-09-30 00:33:43,043][1153883] Decorrelating experience for 32 frames... [2024-09-30 00:33:43,057][1154909] Decorrelating experience for 0 frames... [2024-09-30 00:33:43,059][1153809] Decorrelating experience for 32 frames... [2024-09-30 00:33:43,079][1153881] Decorrelating experience for 0 frames... [2024-09-30 00:33:43,256][1153815] Decorrelating experience for 32 frames... [2024-09-30 00:33:43,267][1153882] Decorrelating experience for 64 frames... [2024-09-30 00:33:43,272][1154909] Decorrelating experience for 32 frames... [2024-09-30 00:33:43,274][1153806] Decorrelating experience for 0 frames... [2024-09-30 00:33:43,286][1153809] Decorrelating experience for 64 frames... [2024-09-30 00:33:43,295][1153807] Decorrelating experience for 0 frames... [2024-09-30 00:33:43,296][1153881] Decorrelating experience for 32 frames... [2024-09-30 00:33:43,296][1153811] Decorrelating experience for 32 frames... [2024-09-30 00:33:43,297][1153808] Decorrelating experience for 32 frames... [2024-09-30 00:33:43,478][1153815] Decorrelating experience for 64 frames... [2024-09-30 00:33:43,496][1153810] Decorrelating experience for 32 frames... [2024-09-30 00:33:43,498][1154909] Decorrelating experience for 64 frames... [2024-09-30 00:33:43,524][1153808] Decorrelating experience for 64 frames... [2024-09-30 00:33:43,524][1153809] Decorrelating experience for 96 frames... [2024-09-30 00:33:43,526][1153814] Decorrelating experience for 64 frames... [2024-09-30 00:33:43,528][1153883] Decorrelating experience for 64 frames... [2024-09-30 00:33:43,533][1153806] Decorrelating experience for 32 frames... [2024-09-30 00:33:43,550][1153816] Decorrelating experience for 0 frames... [2024-09-30 00:33:43,704][1153880] Decorrelating experience for 0 frames... [2024-09-30 00:33:43,711][1153882] Decorrelating experience for 96 frames... [2024-09-30 00:33:43,714][1153815] Decorrelating experience for 96 frames... [2024-09-30 00:33:43,757][1153806] Decorrelating experience for 64 frames... [2024-09-30 00:33:43,759][1153808] Decorrelating experience for 96 frames... [2024-09-30 00:33:43,773][1153881] Decorrelating experience for 64 frames... [2024-09-30 00:33:43,773][1153810] Decorrelating experience for 64 frames... [2024-09-30 00:33:43,915][1153880] Decorrelating experience for 32 frames... [2024-09-30 00:33:43,924][1153811] Decorrelating experience for 64 frames... [2024-09-30 00:33:43,935][1154909] Decorrelating experience for 96 frames... [2024-09-30 00:33:43,960][1153807] Decorrelating experience for 32 frames... [2024-09-30 00:33:44,012][1153881] Decorrelating experience for 96 frames... [2024-09-30 00:33:44,012][1153810] Decorrelating experience for 96 frames... [2024-09-30 00:33:44,013][1153813] Decorrelating experience for 32 frames... [2024-09-30 00:33:44,017][1153816] Decorrelating experience for 32 frames... [2024-09-30 00:33:44,021][1153815] Decorrelating experience for 128 frames... [2024-09-30 00:33:44,143][1153806] Decorrelating experience for 96 frames... [2024-09-30 00:33:44,158][1153812] Decorrelating experience for 0 frames... [2024-09-30 00:33:44,165][1153883] Decorrelating experience for 96 frames... [2024-09-30 00:33:44,234][1153809] Decorrelating experience for 128 frames... [2024-09-30 00:33:44,235][1153807] Decorrelating experience for 64 frames... [2024-09-30 00:33:44,240][1153816] Decorrelating experience for 64 frames... [2024-09-30 00:33:44,242][1153813] Decorrelating experience for 64 frames... [2024-09-30 00:33:44,266][1153808] Decorrelating experience for 128 frames... [2024-09-30 00:33:44,344][1153810] Decorrelating experience for 128 frames... [2024-09-30 00:33:44,396][1154909] Decorrelating experience for 128 frames... [2024-09-30 00:33:44,472][1153881] Decorrelating experience for 128 frames... [2024-09-30 00:33:44,480][1153813] Decorrelating experience for 96 frames... [2024-09-30 00:33:44,488][1153815] Decorrelating experience for 160 frames... [2024-09-30 00:33:44,488][1153816] Decorrelating experience for 96 frames... [2024-09-30 00:33:44,495][1153880] Decorrelating experience for 64 frames... [2024-09-30 00:33:44,567][1153812] Decorrelating experience for 32 frames... [2024-09-30 00:33:44,609][1153811] Decorrelating experience for 96 frames... [2024-09-30 00:33:44,701][1153806] Decorrelating experience for 128 frames... [2024-09-30 00:33:44,704][1154909] Decorrelating experience for 160 frames... [2024-09-30 00:33:44,730][1153880] Decorrelating experience for 96 frames... [2024-09-30 00:33:44,738][1153883] Decorrelating experience for 128 frames... [2024-09-30 00:33:44,738][1153881] Decorrelating experience for 160 frames... [2024-09-30 00:33:44,792][1153816] Decorrelating experience for 128 frames... [2024-09-30 00:33:44,793][1153812] Decorrelating experience for 64 frames... [2024-09-30 00:33:44,804][1153810] Decorrelating experience for 160 frames... [2024-09-30 00:33:44,819][1153814] Decorrelating experience for 96 frames... [2024-09-30 00:33:44,919][1153882] Decorrelating experience for 128 frames... [2024-09-30 00:33:44,958][1153806] Decorrelating experience for 160 frames... [2024-09-30 00:33:44,990][1153813] Decorrelating experience for 128 frames... [2024-09-30 00:33:45,006][1153883] Decorrelating experience for 160 frames... [2024-09-30 00:33:45,028][1153812] Decorrelating experience for 96 frames... [2024-09-30 00:33:45,034][1153880] Decorrelating experience for 128 frames... [2024-09-30 00:33:45,075][1153815] Decorrelating experience for 192 frames... [2024-09-30 00:33:45,131][1153808] Decorrelating experience for 160 frames... [2024-09-30 00:33:45,168][1153814] Decorrelating experience for 128 frames... [2024-09-30 00:33:45,178][1153882] Decorrelating experience for 160 frames... [2024-09-30 00:33:45,228][1153806] Decorrelating experience for 192 frames... [2024-09-30 00:33:45,247][1153813] Decorrelating experience for 160 frames... [2024-09-30 00:33:45,271][1153883] Decorrelating experience for 192 frames... [2024-09-30 00:33:45,326][1153807] Decorrelating experience for 96 frames... [2024-09-30 00:33:45,328][1153812] Decorrelating experience for 128 frames... [2024-09-30 00:33:45,362][1153809] Decorrelating experience for 160 frames... [2024-09-30 00:33:45,401][1153816] Decorrelating experience for 160 frames... [2024-09-30 00:33:45,426][1153814] Decorrelating experience for 160 frames... [2024-09-30 00:33:45,497][1153880] Decorrelating experience for 160 frames... [2024-09-30 00:33:45,526][1153806] Decorrelating experience for 224 frames... [2024-09-30 00:33:45,554][1153808] Decorrelating experience for 192 frames... [2024-09-30 00:33:45,584][1154909] Decorrelating experience for 192 frames... [2024-09-30 00:33:45,592][1153810] Decorrelating experience for 192 frames... [2024-09-30 00:33:45,609][1153812] Decorrelating experience for 160 frames... [2024-09-30 00:33:45,614][1153815] Decorrelating experience for 224 frames... [2024-09-30 00:33:45,624][1153807] Decorrelating experience for 128 frames... [2024-09-30 00:33:45,684][1153813] Decorrelating experience for 192 frames... [2024-09-30 00:33:45,773][1153882] Decorrelating experience for 192 frames... [2024-09-30 00:33:45,807][1153814] Decorrelating experience for 192 frames... [2024-09-30 00:33:45,856][1153808] Decorrelating experience for 224 frames... [2024-09-30 00:33:45,856][1153883] Decorrelating experience for 224 frames... [2024-09-30 00:33:45,879][1153810] Decorrelating experience for 224 frames... [2024-09-30 00:33:45,885][1154909] Decorrelating experience for 224 frames... [2024-09-30 00:33:45,894][1153807] Decorrelating experience for 160 frames... [2024-09-30 00:33:45,931][1153880] Decorrelating experience for 192 frames... [2024-09-30 00:33:45,962][1153813] Decorrelating experience for 224 frames... [2024-09-30 00:33:45,987][1153456] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 4005888. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2024-09-30 00:33:46,018][1153811] Decorrelating experience for 128 frames... [2024-09-30 00:33:46,056][1153882] Decorrelating experience for 224 frames... [2024-09-30 00:33:46,068][1153881] Decorrelating experience for 192 frames... [2024-09-30 00:33:46,129][1153812] Decorrelating experience for 192 frames... [2024-09-30 00:33:46,207][1153880] Decorrelating experience for 224 frames... [2024-09-30 00:33:46,239][1153809] Decorrelating experience for 192 frames... [2024-09-30 00:33:46,277][1153811] Decorrelating experience for 160 frames... [2024-09-30 00:33:46,324][1153807] Decorrelating experience for 192 frames... [2024-09-30 00:33:46,350][1153683] Signal inference workers to stop experience collection... [2024-09-30 00:33:46,354][1153805] InferenceWorker_p0-w0: stopping experience collection [2024-09-30 00:33:46,413][1153812] Decorrelating experience for 224 frames... [2024-09-30 00:33:46,456][1153881] Decorrelating experience for 224 frames... [2024-09-30 00:33:46,488][1153816] Decorrelating experience for 192 frames... [2024-09-30 00:33:46,546][1153809] Decorrelating experience for 224 frames... [2024-09-30 00:33:46,551][1153811] Decorrelating experience for 192 frames... [2024-09-30 00:33:46,711][1153807] Decorrelating experience for 224 frames... [2024-09-30 00:33:46,743][1153814] Decorrelating experience for 224 frames... [2024-09-30 00:33:46,789][1153816] Decorrelating experience for 224 frames... [2024-09-30 00:33:46,978][1153811] Decorrelating experience for 224 frames... [2024-09-30 00:33:47,381][1153683] Signal inference workers to resume experience collection... [2024-09-30 00:33:47,382][1153805] InferenceWorker_p0-w0: resuming experience collection [2024-09-30 00:33:48,488][1153805] Updated weights for policy 0, policy_version 989 (0.0124) [2024-09-30 00:33:49,140][1153805] Updated weights for policy 0, policy_version 999 (0.0006) [2024-09-30 00:33:49,805][1153805] Updated weights for policy 0, policy_version 1009 (0.0006) [2024-09-30 00:33:50,347][1153805] Updated weights for policy 0, policy_version 1019 (0.0006) [2024-09-30 00:33:50,892][1153805] Updated weights for policy 0, policy_version 1029 (0.0006) [2024-09-30 00:33:50,987][1153456] Fps is (10 sec: 25156.3, 60 sec: 25156.3, 300 sec: 25156.3). Total num frames: 4218880. Throughput: 0: 5607.8. Samples: 47480. Policy #0 lag: (min: 0.0, avg: 2.5, max: 5.0) [2024-09-30 00:33:50,987][1153456] Avg episode reward: [(0, '20.825')] [2024-09-30 00:33:51,478][1153805] Updated weights for policy 0, policy_version 1039 (0.0006) [2024-09-30 00:33:52,018][1153805] Updated weights for policy 0, policy_version 1049 (0.0006) [2024-09-30 00:33:52,549][1153805] Updated weights for policy 0, policy_version 1059 (0.0006) [2024-09-30 00:33:53,078][1153805] Updated weights for policy 0, policy_version 1069 (0.0006) [2024-09-30 00:33:53,599][1153805] Updated weights for policy 0, policy_version 1079 (0.0006) [2024-09-30 00:33:54,116][1153805] Updated weights for policy 0, policy_version 1089 (0.0006) [2024-09-30 00:33:54,662][1153805] Updated weights for policy 0, policy_version 1099 (0.0006) [2024-09-30 00:33:55,159][1153805] Updated weights for policy 0, policy_version 1109 (0.0006) [2024-09-30 00:33:55,702][1153805] Updated weights for policy 0, policy_version 1119 (0.0006) [2024-09-30 00:33:55,987][1153456] Fps is (10 sec: 59801.2, 60 sec: 44406.8, 300 sec: 44406.8). Total num frames: 4603904. Throughput: 0: 7749.4. Samples: 104360. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 00:33:55,987][1153456] Avg episode reward: [(0, '24.186')] [2024-09-30 00:33:56,193][1153805] Updated weights for policy 0, policy_version 1129 (0.0006) [2024-09-30 00:33:56,744][1153805] Updated weights for policy 0, policy_version 1139 (0.0006) [2024-09-30 00:33:57,289][1153805] Updated weights for policy 0, policy_version 1149 (0.0006) [2024-09-30 00:33:57,735][1153456] Heartbeat connected on Batcher_0 [2024-09-30 00:33:57,739][1153456] Heartbeat connected on LearnerWorker_p0 [2024-09-30 00:33:57,747][1153456] Heartbeat connected on InferenceWorker_p0-w0 [2024-09-30 00:33:57,751][1153456] Heartbeat connected on RolloutWorker_w0 [2024-09-30 00:33:57,755][1153456] Heartbeat connected on RolloutWorker_w1 [2024-09-30 00:33:57,757][1153456] Heartbeat connected on RolloutWorker_w2 [2024-09-30 00:33:57,769][1153456] Heartbeat connected on RolloutWorker_w3 [2024-09-30 00:33:57,770][1153456] Heartbeat connected on RolloutWorker_w5 [2024-09-30 00:33:57,772][1153456] Heartbeat connected on RolloutWorker_w4 [2024-09-30 00:33:57,780][1153456] Heartbeat connected on RolloutWorker_w6 [2024-09-30 00:33:57,784][1153456] Heartbeat connected on RolloutWorker_w7 [2024-09-30 00:33:57,786][1153456] Heartbeat connected on RolloutWorker_w8 [2024-09-30 00:33:57,789][1153456] Heartbeat connected on RolloutWorker_w9 [2024-09-30 00:33:57,792][1153456] Heartbeat connected on RolloutWorker_w10 [2024-09-30 00:33:57,801][1153456] Heartbeat connected on RolloutWorker_w11 [2024-09-30 00:33:57,803][1153456] Heartbeat connected on RolloutWorker_w12 [2024-09-30 00:33:57,803][1153456] Heartbeat connected on RolloutWorker_w13 [2024-09-30 00:33:57,817][1153805] Updated weights for policy 0, policy_version 1159 (0.0006) [2024-09-30 00:33:57,823][1153456] Heartbeat connected on RolloutWorker_w14 [2024-09-30 00:33:57,831][1153456] Heartbeat connected on RolloutWorker_w15 [2024-09-30 00:33:58,309][1153805] Updated weights for policy 0, policy_version 1169 (0.0006) [2024-09-30 00:33:58,804][1153805] Updated weights for policy 0, policy_version 1179 (0.0006) [2024-09-30 00:33:59,290][1153805] Updated weights for policy 0, policy_version 1189 (0.0006) [2024-09-30 00:33:59,795][1153805] Updated weights for policy 0, policy_version 1199 (0.0006) [2024-09-30 00:34:00,308][1153805] Updated weights for policy 0, policy_version 1209 (0.0006) [2024-09-30 00:34:00,813][1153805] Updated weights for policy 0, policy_version 1219 (0.0006) [2024-09-30 00:34:00,987][1153456] Fps is (10 sec: 78643.4, 60 sec: 54120.2, 300 sec: 54120.2). Total num frames: 5005312. Throughput: 0: 12071.2. Samples: 222916. Policy #0 lag: (min: 1.0, avg: 2.4, max: 6.0) [2024-09-30 00:34:00,987][1153456] Avg episode reward: [(0, '30.110')] [2024-09-30 00:34:00,988][1153683] Saving new best policy, reward=30.110! [2024-09-30 00:34:01,305][1153805] Updated weights for policy 0, policy_version 1229 (0.0006) [2024-09-30 00:34:01,811][1153805] Updated weights for policy 0, policy_version 1239 (0.0006) [2024-09-30 00:34:02,345][1153805] Updated weights for policy 0, policy_version 1249 (0.0006) [2024-09-30 00:34:02,857][1153805] Updated weights for policy 0, policy_version 1259 (0.0006) [2024-09-30 00:34:03,366][1153805] Updated weights for policy 0, policy_version 1269 (0.0006) [2024-09-30 00:34:03,888][1153805] Updated weights for policy 0, policy_version 1279 (0.0006) [2024-09-30 00:34:04,411][1153805] Updated weights for policy 0, policy_version 1289 (0.0006) [2024-09-30 00:34:04,905][1153805] Updated weights for policy 0, policy_version 1299 (0.0006) [2024-09-30 00:34:05,396][1153805] Updated weights for policy 0, policy_version 1309 (0.0006) [2024-09-30 00:34:05,887][1153805] Updated weights for policy 0, policy_version 1319 (0.0006) [2024-09-30 00:34:05,987][1153456] Fps is (10 sec: 80691.1, 60 sec: 59868.8, 300 sec: 59868.8). Total num frames: 5410816. Throughput: 0: 14672.0. Samples: 344304. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 00:34:05,987][1153456] Avg episode reward: [(0, '26.263')] [2024-09-30 00:34:06,367][1153805] Updated weights for policy 0, policy_version 1329 (0.0006) [2024-09-30 00:34:06,902][1153805] Updated weights for policy 0, policy_version 1339 (0.0006) [2024-09-30 00:34:07,398][1153805] Updated weights for policy 0, policy_version 1349 (0.0006) [2024-09-30 00:34:07,857][1153805] Updated weights for policy 0, policy_version 1359 (0.0006) [2024-09-30 00:34:08,393][1153805] Updated weights for policy 0, policy_version 1369 (0.0006) [2024-09-30 00:34:08,923][1153805] Updated weights for policy 0, policy_version 1379 (0.0006) [2024-09-30 00:34:09,422][1153805] Updated weights for policy 0, policy_version 1389 (0.0006) [2024-09-30 00:34:09,921][1153805] Updated weights for policy 0, policy_version 1399 (0.0006) [2024-09-30 00:34:10,424][1153805] Updated weights for policy 0, policy_version 1409 (0.0006) [2024-09-30 00:34:10,932][1153805] Updated weights for policy 0, policy_version 1419 (0.0006) [2024-09-30 00:34:10,987][1153456] Fps is (10 sec: 81100.9, 60 sec: 63598.2, 300 sec: 63598.2). Total num frames: 5816320. Throughput: 0: 14244.4. Samples: 405492. Policy #0 lag: (min: 0.0, avg: 2.5, max: 7.0) [2024-09-30 00:34:10,987][1153456] Avg episode reward: [(0, '24.072')] [2024-09-30 00:34:11,427][1153805] Updated weights for policy 0, policy_version 1429 (0.0006) [2024-09-30 00:34:11,997][1153805] Updated weights for policy 0, policy_version 1439 (0.0006) [2024-09-30 00:34:12,604][1153805] Updated weights for policy 0, policy_version 1449 (0.0006) [2024-09-30 00:34:13,189][1153805] Updated weights for policy 0, policy_version 1459 (0.0006) [2024-09-30 00:34:13,760][1153805] Updated weights for policy 0, policy_version 1469 (0.0006) [2024-09-30 00:34:14,325][1153805] Updated weights for policy 0, policy_version 1479 (0.0006) [2024-09-30 00:34:14,867][1153805] Updated weights for policy 0, policy_version 1489 (0.0006) [2024-09-30 00:34:15,422][1153805] Updated weights for policy 0, policy_version 1499 (0.0006) [2024-09-30 00:34:15,925][1153805] Updated weights for policy 0, policy_version 1509 (0.0006) [2024-09-30 00:34:15,987][1153456] Fps is (10 sec: 77415.0, 60 sec: 65111.6, 300 sec: 65111.6). Total num frames: 6184960. Throughput: 0: 15510.1. Samples: 519072. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 00:34:15,987][1153456] Avg episode reward: [(0, '27.426')] [2024-09-30 00:34:16,430][1153805] Updated weights for policy 0, policy_version 1519 (0.0006) [2024-09-30 00:34:16,937][1153805] Updated weights for policy 0, policy_version 1529 (0.0006) [2024-09-30 00:34:17,445][1153805] Updated weights for policy 0, policy_version 1539 (0.0006) [2024-09-30 00:34:17,474][1153683] Signal inference workers to stop experience collection... (50 times) [2024-09-30 00:34:17,475][1153683] Signal inference workers to resume experience collection... (50 times) [2024-09-30 00:34:17,479][1153805] InferenceWorker_p0-w0: stopping experience collection (50 times) [2024-09-30 00:34:17,479][1153805] InferenceWorker_p0-w0: resuming experience collection (50 times) [2024-09-30 00:34:17,909][1153805] Updated weights for policy 0, policy_version 1549 (0.0006) [2024-09-30 00:34:18,408][1153805] Updated weights for policy 0, policy_version 1559 (0.0006) [2024-09-30 00:34:18,914][1153805] Updated weights for policy 0, policy_version 1569 (0.0006) [2024-09-30 00:34:19,420][1153805] Updated weights for policy 0, policy_version 1579 (0.0006) [2024-09-30 00:34:19,933][1153805] Updated weights for policy 0, policy_version 1589 (0.0006) [2024-09-30 00:34:20,433][1153805] Updated weights for policy 0, policy_version 1599 (0.0005) [2024-09-30 00:34:20,874][1153805] Updated weights for policy 0, policy_version 1609 (0.0006) [2024-09-30 00:34:20,987][1153456] Fps is (10 sec: 77824.6, 60 sec: 67296.5, 300 sec: 67296.5). Total num frames: 6594560. Throughput: 0: 16654.0. Samples: 640624. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 00:34:20,987][1153456] Avg episode reward: [(0, '26.127')] [2024-09-30 00:34:21,376][1153805] Updated weights for policy 0, policy_version 1619 (0.0006) [2024-09-30 00:34:21,913][1153805] Updated weights for policy 0, policy_version 1629 (0.0006) [2024-09-30 00:34:22,441][1153805] Updated weights for policy 0, policy_version 1639 (0.0005) [2024-09-30 00:34:22,968][1153805] Updated weights for policy 0, policy_version 1649 (0.0006) [2024-09-30 00:34:23,482][1153805] Updated weights for policy 0, policy_version 1659 (0.0006) [2024-09-30 00:34:23,975][1153805] Updated weights for policy 0, policy_version 1669 (0.0006) [2024-09-30 00:34:24,474][1153805] Updated weights for policy 0, policy_version 1679 (0.0006) [2024-09-30 00:34:25,003][1153805] Updated weights for policy 0, policy_version 1689 (0.0006) [2024-09-30 00:34:25,527][1153805] Updated weights for policy 0, policy_version 1699 (0.0006) [2024-09-30 00:34:25,987][1153456] Fps is (10 sec: 81099.9, 60 sec: 68789.9, 300 sec: 68789.9). Total num frames: 6995968. Throughput: 0: 16121.9. Samples: 700768. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 00:34:25,987][1153456] Avg episode reward: [(0, '27.073')] [2024-09-30 00:34:26,023][1153805] Updated weights for policy 0, policy_version 1709 (0.0006) [2024-09-30 00:34:26,528][1153805] Updated weights for policy 0, policy_version 1719 (0.0006) [2024-09-30 00:34:27,034][1153805] Updated weights for policy 0, policy_version 1729 (0.0006) [2024-09-30 00:34:27,559][1153805] Updated weights for policy 0, policy_version 1739 (0.0006) [2024-09-30 00:34:28,048][1153805] Updated weights for policy 0, policy_version 1749 (0.0006) [2024-09-30 00:34:28,561][1153805] Updated weights for policy 0, policy_version 1759 (0.0006) [2024-09-30 00:34:29,053][1153805] Updated weights for policy 0, policy_version 1769 (0.0006) [2024-09-30 00:34:29,700][1153805] Updated weights for policy 0, policy_version 1779 (0.0006) [2024-09-30 00:34:30,335][1153805] Updated weights for policy 0, policy_version 1789 (0.0006) [2024-09-30 00:34:30,926][1153805] Updated weights for policy 0, policy_version 1799 (0.0006) [2024-09-30 00:34:30,987][1153456] Fps is (10 sec: 77822.5, 60 sec: 69468.4, 300 sec: 69468.4). Total num frames: 7372800. Throughput: 0: 18231.6. Samples: 820424. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 00:34:30,987][1153456] Avg episode reward: [(0, '25.497')] [2024-09-30 00:34:31,523][1153805] Updated weights for policy 0, policy_version 1809 (0.0006) [2024-09-30 00:34:32,164][1153805] Updated weights for policy 0, policy_version 1819 (0.0006) [2024-09-30 00:34:32,809][1153805] Updated weights for policy 0, policy_version 1829 (0.0006) [2024-09-30 00:34:33,413][1153805] Updated weights for policy 0, policy_version 1839 (0.0006) [2024-09-30 00:34:34,008][1153805] Updated weights for policy 0, policy_version 1849 (0.0006) [2024-09-30 00:34:34,647][1153805] Updated weights for policy 0, policy_version 1859 (0.0006) [2024-09-30 00:34:35,275][1153805] Updated weights for policy 0, policy_version 1869 (0.0006) [2024-09-30 00:34:35,822][1153805] Updated weights for policy 0, policy_version 1879 (0.0007) [2024-09-30 00:34:35,987][1153456] Fps is (10 sec: 70861.3, 60 sec: 69177.4, 300 sec: 69177.4). Total num frames: 7704576. Throughput: 0: 19378.1. Samples: 919492. Policy #0 lag: (min: 0.0, avg: 3.0, max: 6.0) [2024-09-30 00:34:35,987][1153456] Avg episode reward: [(0, '27.382')] [2024-09-30 00:34:36,437][1153805] Updated weights for policy 0, policy_version 1889 (0.0006) [2024-09-30 00:34:36,983][1153805] Updated weights for policy 0, policy_version 1899 (0.0006) [2024-09-30 00:34:37,522][1153805] Updated weights for policy 0, policy_version 1909 (0.0006) [2024-09-30 00:34:38,151][1153805] Updated weights for policy 0, policy_version 1919 (0.0006) [2024-09-30 00:34:38,763][1153805] Updated weights for policy 0, policy_version 1929 (0.0006) [2024-09-30 00:34:39,293][1153805] Updated weights for policy 0, policy_version 1939 (0.0007) [2024-09-30 00:34:39,914][1153805] Updated weights for policy 0, policy_version 1949 (0.0006) [2024-09-30 00:34:40,439][1153805] Updated weights for policy 0, policy_version 1959 (0.0006) [2024-09-30 00:34:40,987][1153456] Fps is (10 sec: 68812.7, 60 sec: 69356.2, 300 sec: 69356.2). Total num frames: 8060928. Throughput: 0: 19301.7. Samples: 972936. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 00:34:40,987][1153456] Avg episode reward: [(0, '27.449')] [2024-09-30 00:34:41,028][1153805] Updated weights for policy 0, policy_version 1969 (0.0006) [2024-09-30 00:34:41,523][1153805] Updated weights for policy 0, policy_version 1979 (0.0006) [2024-09-30 00:34:42,083][1153805] Updated weights for policy 0, policy_version 1989 (0.0006) [2024-09-30 00:34:42,584][1153805] Updated weights for policy 0, policy_version 1999 (0.0006) [2024-09-30 00:34:43,064][1153805] Updated weights for policy 0, policy_version 2009 (0.0007) [2024-09-30 00:34:43,554][1153805] Updated weights for policy 0, policy_version 2019 (0.0006) [2024-09-30 00:34:44,043][1153805] Updated weights for policy 0, policy_version 2029 (0.0006) [2024-09-30 00:34:44,531][1153805] Updated weights for policy 0, policy_version 2039 (0.0006) [2024-09-30 00:34:45,030][1153805] Updated weights for policy 0, policy_version 2049 (0.0006) [2024-09-30 00:34:45,524][1153805] Updated weights for policy 0, policy_version 2059 (0.0006) [2024-09-30 00:34:45,987][1153456] Fps is (10 sec: 76594.4, 60 sec: 74410.5, 300 sec: 70346.0). Total num frames: 8470528. Throughput: 0: 19239.1. Samples: 1088680. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 00:34:45,987][1153456] Avg episode reward: [(0, '31.135')] [2024-09-30 00:34:46,000][1153683] Saving new best policy, reward=31.135! [2024-09-30 00:34:46,001][1153805] Updated weights for policy 0, policy_version 2069 (0.0006) [2024-09-30 00:34:46,586][1153805] Updated weights for policy 0, policy_version 2079 (0.0006) [2024-09-30 00:34:47,190][1153805] Updated weights for policy 0, policy_version 2089 (0.0006) [2024-09-30 00:34:47,788][1153805] Updated weights for policy 0, policy_version 2099 (0.0006) [2024-09-30 00:34:48,430][1153805] Updated weights for policy 0, policy_version 2109 (0.0006) [2024-09-30 00:34:49,037][1153805] Updated weights for policy 0, policy_version 2119 (0.0006) [2024-09-30 00:34:49,642][1153805] Updated weights for policy 0, policy_version 2129 (0.0006) [2024-09-30 00:34:50,271][1153805] Updated weights for policy 0, policy_version 2139 (0.0006) [2024-09-30 00:34:50,444][1153683] Signal inference workers to stop experience collection... (100 times) [2024-09-30 00:34:50,445][1153683] Signal inference workers to resume experience collection... (100 times) [2024-09-30 00:34:50,448][1153805] InferenceWorker_p0-w0: stopping experience collection (100 times) [2024-09-30 00:34:50,450][1153805] InferenceWorker_p0-w0: resuming experience collection (100 times) [2024-09-30 00:34:50,837][1153805] Updated weights for policy 0, policy_version 2149 (0.0006) [2024-09-30 00:34:50,987][1153456] Fps is (10 sec: 75366.2, 60 sec: 76595.0, 300 sec: 70234.0). Total num frames: 8814592. Throughput: 0: 18930.5. Samples: 1196180. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 00:34:50,987][1153456] Avg episode reward: [(0, '26.156')] [2024-09-30 00:34:51,431][1153805] Updated weights for policy 0, policy_version 2159 (0.0006) [2024-09-30 00:34:52,020][1153805] Updated weights for policy 0, policy_version 2169 (0.0006) [2024-09-30 00:34:52,617][1153805] Updated weights for policy 0, policy_version 2179 (0.0007) [2024-09-30 00:34:53,186][1153805] Updated weights for policy 0, policy_version 2189 (0.0006) [2024-09-30 00:34:53,758][1153805] Updated weights for policy 0, policy_version 2199 (0.0006) [2024-09-30 00:34:54,387][1153805] Updated weights for policy 0, policy_version 2209 (0.0006) [2024-09-30 00:34:54,943][1153805] Updated weights for policy 0, policy_version 2219 (0.0006) [2024-09-30 00:34:55,535][1153805] Updated weights for policy 0, policy_version 2229 (0.0006) [2024-09-30 00:34:55,987][1153456] Fps is (10 sec: 68813.5, 60 sec: 75912.6, 300 sec: 70137.4). Total num frames: 9158656. Throughput: 0: 18726.7. Samples: 1248192. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 00:34:55,987][1153456] Avg episode reward: [(0, '29.746')] [2024-09-30 00:34:56,118][1153805] Updated weights for policy 0, policy_version 2239 (0.0006) [2024-09-30 00:34:56,694][1153805] Updated weights for policy 0, policy_version 2249 (0.0006) [2024-09-30 00:34:57,301][1153805] Updated weights for policy 0, policy_version 2259 (0.0007) [2024-09-30 00:34:57,805][1153805] Updated weights for policy 0, policy_version 2269 (0.0006) [2024-09-30 00:34:58,374][1153805] Updated weights for policy 0, policy_version 2279 (0.0006) [2024-09-30 00:34:58,906][1153805] Updated weights for policy 0, policy_version 2289 (0.0006) [2024-09-30 00:34:59,442][1153805] Updated weights for policy 0, policy_version 2299 (0.0006) [2024-09-30 00:34:59,969][1153805] Updated weights for policy 0, policy_version 2309 (0.0006) [2024-09-30 00:35:00,474][1153805] Updated weights for policy 0, policy_version 2319 (0.0006) [2024-09-30 00:35:00,987][1153456] Fps is (10 sec: 72090.6, 60 sec: 75502.9, 300 sec: 70470.6). Total num frames: 9535488. Throughput: 0: 18617.4. Samples: 1356856. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 00:35:00,987][1153456] Avg episode reward: [(0, '31.327')] [2024-09-30 00:35:00,988][1153683] Saving new best policy, reward=31.327! [2024-09-30 00:35:01,059][1153805] Updated weights for policy 0, policy_version 2329 (0.0006) [2024-09-30 00:35:01,541][1153805] Updated weights for policy 0, policy_version 2339 (0.0006) [2024-09-30 00:35:02,014][1153805] Updated weights for policy 0, policy_version 2349 (0.0006) [2024-09-30 00:35:02,509][1153805] Updated weights for policy 0, policy_version 2359 (0.0006) [2024-09-30 00:35:03,003][1153805] Updated weights for policy 0, policy_version 2369 (0.0006) [2024-09-30 00:35:03,490][1153805] Updated weights for policy 0, policy_version 2379 (0.0006) [2024-09-30 00:35:03,986][1153805] Updated weights for policy 0, policy_version 2389 (0.0006) [2024-09-30 00:35:04,468][1153805] Updated weights for policy 0, policy_version 2399 (0.0006) [2024-09-30 00:35:04,947][1153805] Updated weights for policy 0, policy_version 2409 (0.0006) [2024-09-30 00:35:05,456][1153805] Updated weights for policy 0, policy_version 2419 (0.0006) [2024-09-30 00:35:05,890][1153805] Updated weights for policy 0, policy_version 2429 (0.0006) [2024-09-30 00:35:05,987][1153456] Fps is (10 sec: 79463.2, 60 sec: 75707.9, 300 sec: 71254.7). Total num frames: 9953280. Throughput: 0: 18651.0. Samples: 1479920. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 00:35:05,987][1153456] Avg episode reward: [(0, '29.677')] [2024-09-30 00:35:06,386][1153805] Updated weights for policy 0, policy_version 2439 (0.0006) [2024-09-30 00:35:06,873][1153805] Updated weights for policy 0, policy_version 2449 (0.0006) [2024-09-30 00:35:07,354][1153805] Updated weights for policy 0, policy_version 2459 (0.0006) [2024-09-30 00:35:07,831][1153805] Updated weights for policy 0, policy_version 2469 (0.0006) [2024-09-30 00:35:08,356][1153805] Updated weights for policy 0, policy_version 2479 (0.0006) [2024-09-30 00:35:08,867][1153805] Updated weights for policy 0, policy_version 2489 (0.0006) [2024-09-30 00:35:09,352][1153805] Updated weights for policy 0, policy_version 2499 (0.0006) [2024-09-30 00:35:09,840][1153805] Updated weights for policy 0, policy_version 2509 (0.0006) [2024-09-30 00:35:10,285][1153805] Updated weights for policy 0, policy_version 2519 (0.0006) [2024-09-30 00:35:10,741][1153805] Updated weights for policy 0, policy_version 2529 (0.0006) [2024-09-30 00:35:10,987][1153456] Fps is (10 sec: 84378.4, 60 sec: 76049.2, 300 sec: 72042.7). Total num frames: 10379264. Throughput: 0: 18705.0. Samples: 1542492. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 00:35:10,987][1153456] Avg episode reward: [(0, '27.577')] [2024-09-30 00:35:11,230][1153805] Updated weights for policy 0, policy_version 2539 (0.0006) [2024-09-30 00:35:11,721][1153805] Updated weights for policy 0, policy_version 2549 (0.0006) [2024-09-30 00:35:12,216][1153805] Updated weights for policy 0, policy_version 2559 (0.0006) [2024-09-30 00:35:12,704][1153805] Updated weights for policy 0, policy_version 2569 (0.0006) [2024-09-30 00:35:13,169][1153805] Updated weights for policy 0, policy_version 2579 (0.0006) [2024-09-30 00:35:13,664][1153805] Updated weights for policy 0, policy_version 2589 (0.0006) [2024-09-30 00:35:14,143][1153805] Updated weights for policy 0, policy_version 2599 (0.0006) [2024-09-30 00:35:14,638][1153805] Updated weights for policy 0, policy_version 2609 (0.0006) [2024-09-30 00:35:15,138][1153805] Updated weights for policy 0, policy_version 2619 (0.0006) [2024-09-30 00:35:15,632][1153805] Updated weights for policy 0, policy_version 2629 (0.0006) [2024-09-30 00:35:15,987][1153456] Fps is (10 sec: 84376.8, 60 sec: 76868.2, 300 sec: 72658.7). Total num frames: 10797056. Throughput: 0: 18878.2. Samples: 1669940. Policy #0 lag: (min: 0.0, avg: 2.5, max: 7.0) [2024-09-30 00:35:15,987][1153456] Avg episode reward: [(0, '27.694')] [2024-09-30 00:35:16,124][1153805] Updated weights for policy 0, policy_version 2639 (0.0006) [2024-09-30 00:35:16,624][1153805] Updated weights for policy 0, policy_version 2649 (0.0006) [2024-09-30 00:35:17,085][1153805] Updated weights for policy 0, policy_version 2659 (0.0006) [2024-09-30 00:35:17,585][1153805] Updated weights for policy 0, policy_version 2669 (0.0006) [2024-09-30 00:35:18,084][1153805] Updated weights for policy 0, policy_version 2679 (0.0006) [2024-09-30 00:35:18,552][1153805] Updated weights for policy 0, policy_version 2689 (0.0006) [2024-09-30 00:35:19,049][1153805] Updated weights for policy 0, policy_version 2699 (0.0006) [2024-09-30 00:35:19,542][1153805] Updated weights for policy 0, policy_version 2709 (0.0006) [2024-09-30 00:35:20,017][1153805] Updated weights for policy 0, policy_version 2719 (0.0006) [2024-09-30 00:35:20,512][1153805] Updated weights for policy 0, policy_version 2729 (0.0006) [2024-09-30 00:35:20,987][1153456] Fps is (10 sec: 83558.1, 60 sec: 77004.7, 300 sec: 73212.2). Total num frames: 11214848. Throughput: 0: 19474.6. Samples: 1795848. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 00:35:20,987][1153456] Avg episode reward: [(0, '29.085')] [2024-09-30 00:35:20,997][1153805] Updated weights for policy 0, policy_version 2739 (0.0006) [2024-09-30 00:35:21,452][1153805] Updated weights for policy 0, policy_version 2749 (0.0006) [2024-09-30 00:35:21,950][1153805] Updated weights for policy 0, policy_version 2759 (0.0006) [2024-09-30 00:35:22,446][1153805] Updated weights for policy 0, policy_version 2769 (0.0006) [2024-09-30 00:35:22,925][1153805] Updated weights for policy 0, policy_version 2779 (0.0006) [2024-09-30 00:35:23,377][1153805] Updated weights for policy 0, policy_version 2789 (0.0006) [2024-09-30 00:35:23,859][1153805] Updated weights for policy 0, policy_version 2799 (0.0006) [2024-09-30 00:35:24,310][1153805] Updated weights for policy 0, policy_version 2809 (0.0006) [2024-09-30 00:35:24,789][1153805] Updated weights for policy 0, policy_version 2819 (0.0006) [2024-09-30 00:35:25,289][1153805] Updated weights for policy 0, policy_version 2829 (0.0006) [2024-09-30 00:35:25,785][1153805] Updated weights for policy 0, policy_version 2839 (0.0006) [2024-09-30 00:35:25,987][1153456] Fps is (10 sec: 84786.5, 60 sec: 77482.6, 300 sec: 73830.8). Total num frames: 11644928. Throughput: 0: 19711.9. Samples: 1859972. Policy #0 lag: (min: 0.0, avg: 2.2, max: 4.0) [2024-09-30 00:35:25,987][1153456] Avg episode reward: [(0, '30.366')] [2024-09-30 00:35:26,243][1153805] Updated weights for policy 0, policy_version 2849 (0.0006) [2024-09-30 00:35:26,732][1153805] Updated weights for policy 0, policy_version 2859 (0.0006) [2024-09-30 00:35:27,237][1153805] Updated weights for policy 0, policy_version 2869 (0.0006) [2024-09-30 00:35:27,736][1153805] Updated weights for policy 0, policy_version 2879 (0.0006) [2024-09-30 00:35:28,219][1153805] Updated weights for policy 0, policy_version 2889 (0.0006) [2024-09-30 00:35:28,734][1153805] Updated weights for policy 0, policy_version 2899 (0.0006) [2024-09-30 00:35:29,198][1153805] Updated weights for policy 0, policy_version 2909 (0.0006) [2024-09-30 00:35:29,572][1153683] Signal inference workers to stop experience collection... (150 times) [2024-09-30 00:35:29,573][1153683] Signal inference workers to resume experience collection... (150 times) [2024-09-30 00:35:29,579][1153805] InferenceWorker_p0-w0: stopping experience collection (150 times) [2024-09-30 00:35:29,579][1153805] InferenceWorker_p0-w0: resuming experience collection (150 times) [2024-09-30 00:35:29,690][1153805] Updated weights for policy 0, policy_version 2919 (0.0006) [2024-09-30 00:35:30,178][1153805] Updated weights for policy 0, policy_version 2929 (0.0006) [2024-09-30 00:35:30,663][1153805] Updated weights for policy 0, policy_version 2939 (0.0006) [2024-09-30 00:35:30,987][1153456] Fps is (10 sec: 84785.6, 60 sec: 78165.3, 300 sec: 74279.2). Total num frames: 12062720. Throughput: 0: 19949.9. Samples: 1986428. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 00:35:30,987][1153456] Avg episode reward: [(0, '28.907')] [2024-09-30 00:35:31,146][1153805] Updated weights for policy 0, policy_version 2949 (0.0006) [2024-09-30 00:35:31,642][1153805] Updated weights for policy 0, policy_version 2959 (0.0006) [2024-09-30 00:35:32,128][1153805] Updated weights for policy 0, policy_version 2969 (0.0006) [2024-09-30 00:35:32,610][1153805] Updated weights for policy 0, policy_version 2979 (0.0006) [2024-09-30 00:35:33,108][1153805] Updated weights for policy 0, policy_version 2989 (0.0006) [2024-09-30 00:35:33,600][1153805] Updated weights for policy 0, policy_version 2999 (0.0006) [2024-09-30 00:35:34,079][1153805] Updated weights for policy 0, policy_version 3009 (0.0005) [2024-09-30 00:35:34,540][1153805] Updated weights for policy 0, policy_version 3019 (0.0006) [2024-09-30 00:35:34,991][1153805] Updated weights for policy 0, policy_version 3029 (0.0006) [2024-09-30 00:35:35,483][1153805] Updated weights for policy 0, policy_version 3039 (0.0006) [2024-09-30 00:35:35,982][1153805] Updated weights for policy 0, policy_version 3049 (0.0006) [2024-09-30 00:35:35,987][1153456] Fps is (10 sec: 84376.8, 60 sec: 79735.2, 300 sec: 74760.3). Total num frames: 12488704. Throughput: 0: 20386.2. Samples: 2113560. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 00:35:35,987][1153456] Avg episode reward: [(0, '30.962')] [2024-09-30 00:35:35,994][1153683] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000003049_12488704.pth... [2024-09-30 00:35:36,041][1153683] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth [2024-09-30 00:35:36,477][1153805] Updated weights for policy 0, policy_version 3059 (0.0006) [2024-09-30 00:35:36,961][1153805] Updated weights for policy 0, policy_version 3069 (0.0006) [2024-09-30 00:35:37,449][1153805] Updated weights for policy 0, policy_version 3079 (0.0006) [2024-09-30 00:35:37,970][1153805] Updated weights for policy 0, policy_version 3089 (0.0006) [2024-09-30 00:35:38,472][1153805] Updated weights for policy 0, policy_version 3099 (0.0006) [2024-09-30 00:35:38,981][1153805] Updated weights for policy 0, policy_version 3109 (0.0006) [2024-09-30 00:35:39,484][1153805] Updated weights for policy 0, policy_version 3119 (0.0006) [2024-09-30 00:35:39,999][1153805] Updated weights for policy 0, policy_version 3129 (0.0006) [2024-09-30 00:35:40,501][1153805] Updated weights for policy 0, policy_version 3139 (0.0006) [2024-09-30 00:35:40,987][1153456] Fps is (10 sec: 83149.7, 60 sec: 80554.8, 300 sec: 75028.0). Total num frames: 12894208. Throughput: 0: 20605.2. Samples: 2175424. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 00:35:40,987][1153456] Avg episode reward: [(0, '29.171')] [2024-09-30 00:35:41,017][1153805] Updated weights for policy 0, policy_version 3149 (0.0006) [2024-09-30 00:35:41,519][1153805] Updated weights for policy 0, policy_version 3159 (0.0006) [2024-09-30 00:35:42,026][1153805] Updated weights for policy 0, policy_version 3169 (0.0006) [2024-09-30 00:35:42,545][1153805] Updated weights for policy 0, policy_version 3179 (0.0006) [2024-09-30 00:35:43,055][1153805] Updated weights for policy 0, policy_version 3189 (0.0006) [2024-09-30 00:35:43,559][1153805] Updated weights for policy 0, policy_version 3199 (0.0006) [2024-09-30 00:35:44,059][1153805] Updated weights for policy 0, policy_version 3209 (0.0006) [2024-09-30 00:35:44,590][1153805] Updated weights for policy 0, policy_version 3219 (0.0006) [2024-09-30 00:35:45,092][1153805] Updated weights for policy 0, policy_version 3229 (0.0006) [2024-09-30 00:35:45,591][1153805] Updated weights for policy 0, policy_version 3239 (0.0006) [2024-09-30 00:35:45,987][1153456] Fps is (10 sec: 81101.8, 60 sec: 80486.5, 300 sec: 75273.9). Total num frames: 13299712. Throughput: 0: 20868.2. Samples: 2295924. Policy #0 lag: (min: 0.0, avg: 1.5, max: 6.0) [2024-09-30 00:35:45,987][1153456] Avg episode reward: [(0, '32.326')] [2024-09-30 00:35:45,993][1153683] Saving new best policy, reward=32.326! [2024-09-30 00:35:46,083][1153805] Updated weights for policy 0, policy_version 3249 (0.0006) [2024-09-30 00:35:46,528][1153805] Updated weights for policy 0, policy_version 3259 (0.0006) [2024-09-30 00:35:47,059][1153805] Updated weights for policy 0, policy_version 3269 (0.0006) [2024-09-30 00:35:47,599][1153805] Updated weights for policy 0, policy_version 3279 (0.0006) [2024-09-30 00:35:48,021][1153805] Updated weights for policy 0, policy_version 3289 (0.0006) [2024-09-30 00:35:48,518][1153805] Updated weights for policy 0, policy_version 3299 (0.0006) [2024-09-30 00:35:49,019][1153805] Updated weights for policy 0, policy_version 3309 (0.0006) [2024-09-30 00:35:49,513][1153805] Updated weights for policy 0, policy_version 3319 (0.0006) [2024-09-30 00:35:50,010][1153805] Updated weights for policy 0, policy_version 3329 (0.0006) [2024-09-30 00:35:50,507][1153805] Updated weights for policy 0, policy_version 3339 (0.0006) [2024-09-30 00:35:50,987][1153456] Fps is (10 sec: 81510.6, 60 sec: 81578.9, 300 sec: 75532.6). Total num frames: 13709312. Throughput: 0: 20888.1. Samples: 2419888. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 00:35:50,987][1153456] Avg episode reward: [(0, '30.049')] [2024-09-30 00:35:51,006][1153805] Updated weights for policy 0, policy_version 3349 (0.0006) [2024-09-30 00:35:51,498][1153805] Updated weights for policy 0, policy_version 3359 (0.0006) [2024-09-30 00:35:51,998][1153805] Updated weights for policy 0, policy_version 3369 (0.0006) [2024-09-30 00:35:52,492][1153805] Updated weights for policy 0, policy_version 3379 (0.0006) [2024-09-30 00:35:52,990][1153805] Updated weights for policy 0, policy_version 3389 (0.0006) [2024-09-30 00:35:53,493][1153805] Updated weights for policy 0, policy_version 3399 (0.0006) [2024-09-30 00:35:54,000][1153805] Updated weights for policy 0, policy_version 3409 (0.0006) [2024-09-30 00:35:54,477][1153805] Updated weights for policy 0, policy_version 3419 (0.0006) [2024-09-30 00:35:54,996][1153805] Updated weights for policy 0, policy_version 3429 (0.0006) [2024-09-30 00:35:54,999][1153683] Signal inference workers to stop experience collection... (200 times) [2024-09-30 00:35:55,000][1153683] Signal inference workers to resume experience collection... (200 times) [2024-09-30 00:35:55,005][1153805] InferenceWorker_p0-w0: stopping experience collection (200 times) [2024-09-30 00:35:55,005][1153805] InferenceWorker_p0-w0: resuming experience collection (200 times) [2024-09-30 00:35:55,468][1153805] Updated weights for policy 0, policy_version 3439 (0.0006) [2024-09-30 00:35:55,971][1153805] Updated weights for policy 0, policy_version 3449 (0.0006) [2024-09-30 00:35:55,987][1153456] Fps is (10 sec: 82739.3, 60 sec: 82807.4, 300 sec: 75833.2). Total num frames: 14127104. Throughput: 0: 20878.1. Samples: 2482008. Policy #0 lag: (min: 0.0, avg: 1.5, max: 4.0) [2024-09-30 00:35:55,987][1153456] Avg episode reward: [(0, '30.073')] [2024-09-30 00:35:56,474][1153805] Updated weights for policy 0, policy_version 3459 (0.0006) [2024-09-30 00:35:56,968][1153805] Updated weights for policy 0, policy_version 3469 (0.0006) [2024-09-30 00:35:57,460][1153805] Updated weights for policy 0, policy_version 3479 (0.0006) [2024-09-30 00:35:57,956][1153805] Updated weights for policy 0, policy_version 3489 (0.0006) [2024-09-30 00:35:58,456][1153805] Updated weights for policy 0, policy_version 3499 (0.0006) [2024-09-30 00:35:58,951][1153805] Updated weights for policy 0, policy_version 3509 (0.0006) [2024-09-30 00:35:59,425][1153805] Updated weights for policy 0, policy_version 3519 (0.0006) [2024-09-30 00:35:59,942][1153805] Updated weights for policy 0, policy_version 3529 (0.0006) [2024-09-30 00:36:00,445][1153805] Updated weights for policy 0, policy_version 3539 (0.0006) [2024-09-30 00:36:00,868][1153805] Updated weights for policy 0, policy_version 3549 (0.0006) [2024-09-30 00:36:00,987][1153456] Fps is (10 sec: 83558.3, 60 sec: 83490.1, 300 sec: 76112.2). Total num frames: 14544896. Throughput: 0: 20815.6. Samples: 2606640. Policy #0 lag: (min: 0.0, avg: 2.8, max: 7.0) [2024-09-30 00:36:00,987][1153456] Avg episode reward: [(0, '29.898')] [2024-09-30 00:36:01,375][1153805] Updated weights for policy 0, policy_version 3559 (0.0006) [2024-09-30 00:36:01,870][1153805] Updated weights for policy 0, policy_version 3569 (0.0006) [2024-09-30 00:36:02,406][1153805] Updated weights for policy 0, policy_version 3579 (0.0006) [2024-09-30 00:36:02,918][1153805] Updated weights for policy 0, policy_version 3589 (0.0006) [2024-09-30 00:36:03,430][1153805] Updated weights for policy 0, policy_version 3599 (0.0006) [2024-09-30 00:36:03,939][1153805] Updated weights for policy 0, policy_version 3609 (0.0006) [2024-09-30 00:36:04,431][1153805] Updated weights for policy 0, policy_version 3619 (0.0006) [2024-09-30 00:36:04,919][1153805] Updated weights for policy 0, policy_version 3629 (0.0006) [2024-09-30 00:36:05,408][1153805] Updated weights for policy 0, policy_version 3639 (0.0006) [2024-09-30 00:36:05,896][1153805] Updated weights for policy 0, policy_version 3649 (0.0006) [2024-09-30 00:36:05,987][1153456] Fps is (10 sec: 82329.6, 60 sec: 83285.2, 300 sec: 76286.0). Total num frames: 14950400. Throughput: 0: 20749.6. Samples: 2729580. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 00:36:05,987][1153456] Avg episode reward: [(0, '28.929')] [2024-09-30 00:36:06,379][1153805] Updated weights for policy 0, policy_version 3659 (0.0006) [2024-09-30 00:36:06,854][1153805] Updated weights for policy 0, policy_version 3669 (0.0006) [2024-09-30 00:36:07,330][1153805] Updated weights for policy 0, policy_version 3679 (0.0006) [2024-09-30 00:36:07,826][1153805] Updated weights for policy 0, policy_version 3689 (0.0006) [2024-09-30 00:36:08,324][1153805] Updated weights for policy 0, policy_version 3699 (0.0006) [2024-09-30 00:36:08,795][1153805] Updated weights for policy 0, policy_version 3709 (0.0006) [2024-09-30 00:36:09,262][1153805] Updated weights for policy 0, policy_version 3719 (0.0006) [2024-09-30 00:36:09,783][1153805] Updated weights for policy 0, policy_version 3729 (0.0006) [2024-09-30 00:36:10,356][1153805] Updated weights for policy 0, policy_version 3739 (0.0006) [2024-09-30 00:36:10,949][1153683] Signal inference workers to stop experience collection... (250 times) [2024-09-30 00:36:10,953][1153805] InferenceWorker_p0-w0: stopping experience collection (250 times) [2024-09-30 00:36:10,961][1153683] Signal inference workers to resume experience collection... (250 times) [2024-09-30 00:36:10,962][1153805] InferenceWorker_p0-w0: resuming experience collection (250 times) [2024-09-30 00:36:10,963][1153805] Updated weights for policy 0, policy_version 3749 (0.0007) [2024-09-30 00:36:10,987][1153456] Fps is (10 sec: 81510.3, 60 sec: 83012.1, 300 sec: 76475.8). Total num frames: 15360000. Throughput: 0: 20739.1. Samples: 2793232. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 00:36:10,987][1153456] Avg episode reward: [(0, '26.908')] [2024-09-30 00:36:11,545][1153805] Updated weights for policy 0, policy_version 3759 (0.0006) [2024-09-30 00:36:12,099][1153805] Updated weights for policy 0, policy_version 3769 (0.0006) [2024-09-30 00:36:12,712][1153805] Updated weights for policy 0, policy_version 3779 (0.0006) [2024-09-30 00:36:13,288][1153805] Updated weights for policy 0, policy_version 3789 (0.0006) [2024-09-30 00:36:13,856][1153805] Updated weights for policy 0, policy_version 3799 (0.0006) [2024-09-30 00:36:14,503][1153805] Updated weights for policy 0, policy_version 3809 (0.0006) [2024-09-30 00:36:15,056][1153805] Updated weights for policy 0, policy_version 3819 (0.0006) [2024-09-30 00:36:15,624][1153805] Updated weights for policy 0, policy_version 3829 (0.0006) [2024-09-30 00:36:15,987][1153456] Fps is (10 sec: 75776.2, 60 sec: 81851.7, 300 sec: 76252.8). Total num frames: 15708160. Throughput: 0: 20355.5. Samples: 2902424. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 00:36:15,987][1153456] Avg episode reward: [(0, '26.893')] [2024-09-30 00:36:16,198][1153805] Updated weights for policy 0, policy_version 3839 (0.0006) [2024-09-30 00:36:16,738][1153805] Updated weights for policy 0, policy_version 3849 (0.0006) [2024-09-30 00:36:17,265][1153805] Updated weights for policy 0, policy_version 3859 (0.0006) [2024-09-30 00:36:17,784][1153805] Updated weights for policy 0, policy_version 3869 (0.0006) [2024-09-30 00:36:18,296][1153805] Updated weights for policy 0, policy_version 3879 (0.0006) [2024-09-30 00:36:18,841][1153805] Updated weights for policy 0, policy_version 3889 (0.0006) [2024-09-30 00:36:19,381][1153805] Updated weights for policy 0, policy_version 3899 (0.0006) [2024-09-30 00:36:19,926][1153805] Updated weights for policy 0, policy_version 3909 (0.0006) [2024-09-30 00:36:20,497][1153805] Updated weights for policy 0, policy_version 3919 (0.0006) [2024-09-30 00:36:20,987][1153456] Fps is (10 sec: 72908.8, 60 sec: 81237.2, 300 sec: 76250.7). Total num frames: 16089088. Throughput: 0: 20031.3. Samples: 3014964. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 00:36:20,987][1153456] Avg episode reward: [(0, '31.297')] [2024-09-30 00:36:21,040][1153805] Updated weights for policy 0, policy_version 3929 (0.0006) [2024-09-30 00:36:21,560][1153805] Updated weights for policy 0, policy_version 3939 (0.0006) [2024-09-30 00:36:22,089][1153805] Updated weights for policy 0, policy_version 3949 (0.0006) [2024-09-30 00:36:22,606][1153805] Updated weights for policy 0, policy_version 3959 (0.0006) [2024-09-30 00:36:23,124][1153805] Updated weights for policy 0, policy_version 3969 (0.0006) [2024-09-30 00:36:23,631][1153805] Updated weights for policy 0, policy_version 3979 (0.0006) [2024-09-30 00:36:24,165][1153805] Updated weights for policy 0, policy_version 3989 (0.0006) [2024-09-30 00:36:24,669][1153805] Updated weights for policy 0, policy_version 3999 (0.0006) [2024-09-30 00:36:25,176][1153805] Updated weights for policy 0, policy_version 4009 (0.0006) [2024-09-30 00:36:25,678][1153805] Updated weights for policy 0, policy_version 4019 (0.0006) [2024-09-30 00:36:25,987][1153456] Fps is (10 sec: 77414.2, 60 sec: 80623.0, 300 sec: 76323.9). Total num frames: 16482304. Throughput: 0: 19945.3. Samples: 3072964. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 00:36:25,987][1153456] Avg episode reward: [(0, '28.992')] [2024-09-30 00:36:26,193][1153805] Updated weights for policy 0, policy_version 4029 (0.0006) [2024-09-30 00:36:26,701][1153805] Updated weights for policy 0, policy_version 4039 (0.0006) [2024-09-30 00:36:27,200][1153805] Updated weights for policy 0, policy_version 4049 (0.0006) [2024-09-30 00:36:27,756][1153805] Updated weights for policy 0, policy_version 4059 (0.0006) [2024-09-30 00:36:28,252][1153805] Updated weights for policy 0, policy_version 4069 (0.0006) [2024-09-30 00:36:28,802][1153805] Updated weights for policy 0, policy_version 4079 (0.0006) [2024-09-30 00:36:29,340][1153805] Updated weights for policy 0, policy_version 4089 (0.0006) [2024-09-30 00:36:29,857][1153805] Updated weights for policy 0, policy_version 4099 (0.0006) [2024-09-30 00:36:30,377][1153805] Updated weights for policy 0, policy_version 4109 (0.0006) [2024-09-30 00:36:30,878][1153805] Updated weights for policy 0, policy_version 4119 (0.0006) [2024-09-30 00:36:30,987][1153456] Fps is (10 sec: 78642.5, 60 sec: 80213.4, 300 sec: 76392.7). Total num frames: 16875520. Throughput: 0: 19912.1. Samples: 3191968. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 00:36:30,987][1153456] Avg episode reward: [(0, '32.306')] [2024-09-30 00:36:31,392][1153805] Updated weights for policy 0, policy_version 4129 (0.0006) [2024-09-30 00:36:31,912][1153805] Updated weights for policy 0, policy_version 4139 (0.0006) [2024-09-30 00:36:32,416][1153805] Updated weights for policy 0, policy_version 4149 (0.0006) [2024-09-30 00:36:32,913][1153805] Updated weights for policy 0, policy_version 4159 (0.0006) [2024-09-30 00:36:33,424][1153805] Updated weights for policy 0, policy_version 4169 (0.0006) [2024-09-30 00:36:33,972][1153805] Updated weights for policy 0, policy_version 4179 (0.0006) [2024-09-30 00:36:34,497][1153805] Updated weights for policy 0, policy_version 4189 (0.0006) [2024-09-30 00:36:35,013][1153805] Updated weights for policy 0, policy_version 4199 (0.0006) [2024-09-30 00:36:35,537][1153805] Updated weights for policy 0, policy_version 4209 (0.0006) [2024-09-30 00:36:35,987][1153456] Fps is (10 sec: 79052.9, 60 sec: 79735.7, 300 sec: 76481.2). Total num frames: 17272832. Throughput: 0: 19802.7. Samples: 3311008. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 00:36:35,987][1153456] Avg episode reward: [(0, '28.748')] [2024-09-30 00:36:36,058][1153805] Updated weights for policy 0, policy_version 4219 (0.0006) [2024-09-30 00:36:36,601][1153805] Updated weights for policy 0, policy_version 4229 (0.0006) [2024-09-30 00:36:37,143][1153805] Updated weights for policy 0, policy_version 4239 (0.0006) [2024-09-30 00:36:37,660][1153805] Updated weights for policy 0, policy_version 4249 (0.0006) [2024-09-30 00:36:38,201][1153805] Updated weights for policy 0, policy_version 4259 (0.0006) [2024-09-30 00:36:38,733][1153805] Updated weights for policy 0, policy_version 4269 (0.0006) [2024-09-30 00:36:39,255][1153805] Updated weights for policy 0, policy_version 4279 (0.0007) [2024-09-30 00:36:39,849][1153805] Updated weights for policy 0, policy_version 4289 (0.0006) [2024-09-30 00:36:40,361][1153805] Updated weights for policy 0, policy_version 4299 (0.0006) [2024-09-30 00:36:40,931][1153805] Updated weights for policy 0, policy_version 4309 (0.0006) [2024-09-30 00:36:40,987][1153456] Fps is (10 sec: 77824.2, 60 sec: 79325.8, 300 sec: 76472.9). Total num frames: 17653760. Throughput: 0: 19700.2. Samples: 3368520. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 00:36:40,987][1153456] Avg episode reward: [(0, '32.397')] [2024-09-30 00:36:40,988][1153683] Saving new best policy, reward=32.397! [2024-09-30 00:36:41,473][1153805] Updated weights for policy 0, policy_version 4319 (0.0006) [2024-09-30 00:36:41,971][1153805] Updated weights for policy 0, policy_version 4329 (0.0006) [2024-09-30 00:36:42,475][1153805] Updated weights for policy 0, policy_version 4339 (0.0006) [2024-09-30 00:36:42,975][1153805] Updated weights for policy 0, policy_version 4349 (0.0006) [2024-09-30 00:36:43,513][1153805] Updated weights for policy 0, policy_version 4359 (0.0006) [2024-09-30 00:36:44,114][1153805] Updated weights for policy 0, policy_version 4369 (0.0006) [2024-09-30 00:36:44,624][1153805] Updated weights for policy 0, policy_version 4379 (0.0006) [2024-09-30 00:36:45,224][1153805] Updated weights for policy 0, policy_version 4389 (0.0006) [2024-09-30 00:36:45,814][1153805] Updated weights for policy 0, policy_version 4399 (0.0006) [2024-09-30 00:36:45,987][1153456] Fps is (10 sec: 75775.2, 60 sec: 78847.9, 300 sec: 76442.7). Total num frames: 18030592. Throughput: 0: 19471.0. Samples: 3482836. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 00:36:45,987][1153456] Avg episode reward: [(0, '30.083')] [2024-09-30 00:36:46,425][1153805] Updated weights for policy 0, policy_version 4409 (0.0006) [2024-09-30 00:36:46,966][1153805] Updated weights for policy 0, policy_version 4419 (0.0006) [2024-09-30 00:36:47,568][1153805] Updated weights for policy 0, policy_version 4429 (0.0006) [2024-09-30 00:36:48,186][1153805] Updated weights for policy 0, policy_version 4439 (0.0006) [2024-09-30 00:36:48,789][1153805] Updated weights for policy 0, policy_version 4449 (0.0006) [2024-09-30 00:36:49,403][1153805] Updated weights for policy 0, policy_version 4459 (0.0006) [2024-09-30 00:36:50,084][1153805] Updated weights for policy 0, policy_version 4469 (0.0006) [2024-09-30 00:36:50,708][1153805] Updated weights for policy 0, policy_version 4479 (0.0006) [2024-09-30 00:36:50,987][1153456] Fps is (10 sec: 70861.2, 60 sec: 77550.9, 300 sec: 76175.1). Total num frames: 18362368. Throughput: 0: 19006.6. Samples: 3584876. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 00:36:50,987][1153456] Avg episode reward: [(0, '31.676')] [2024-09-30 00:36:51,338][1153805] Updated weights for policy 0, policy_version 4489 (0.0006) [2024-09-30 00:36:51,652][1153683] Signal inference workers to stop experience collection... (300 times) [2024-09-30 00:36:51,653][1153683] Signal inference workers to resume experience collection... (300 times) [2024-09-30 00:36:51,658][1153805] InferenceWorker_p0-w0: stopping experience collection (300 times) [2024-09-30 00:36:51,658][1153805] InferenceWorker_p0-w0: resuming experience collection (300 times) [2024-09-30 00:36:51,935][1153805] Updated weights for policy 0, policy_version 4499 (0.0007) [2024-09-30 00:36:52,592][1153805] Updated weights for policy 0, policy_version 4509 (0.0007) [2024-09-30 00:36:53,218][1153805] Updated weights for policy 0, policy_version 4519 (0.0006) [2024-09-30 00:36:53,857][1153805] Updated weights for policy 0, policy_version 4529 (0.0006) [2024-09-30 00:36:54,505][1153805] Updated weights for policy 0, policy_version 4539 (0.0006) [2024-09-30 00:36:55,103][1153805] Updated weights for policy 0, policy_version 4549 (0.0006) [2024-09-30 00:36:55,714][1153805] Updated weights for policy 0, policy_version 4559 (0.0006) [2024-09-30 00:36:55,987][1153456] Fps is (10 sec: 65945.6, 60 sec: 76048.9, 300 sec: 75900.1). Total num frames: 18690048. Throughput: 0: 18664.3. Samples: 3633128. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 00:36:55,987][1153456] Avg episode reward: [(0, '31.825')] [2024-09-30 00:36:56,368][1153805] Updated weights for policy 0, policy_version 4569 (0.0005) [2024-09-30 00:36:57,015][1153805] Updated weights for policy 0, policy_version 4579 (0.0006) [2024-09-30 00:36:57,622][1153805] Updated weights for policy 0, policy_version 4589 (0.0006) [2024-09-30 00:36:58,234][1153805] Updated weights for policy 0, policy_version 4599 (0.0006) [2024-09-30 00:36:58,833][1153805] Updated weights for policy 0, policy_version 4609 (0.0006) [2024-09-30 00:36:59,473][1153805] Updated weights for policy 0, policy_version 4619 (0.0006) [2024-09-30 00:37:00,103][1153805] Updated weights for policy 0, policy_version 4629 (0.0006) [2024-09-30 00:37:00,712][1153805] Updated weights for policy 0, policy_version 4639 (0.0006) [2024-09-30 00:37:00,987][1153456] Fps is (10 sec: 65536.0, 60 sec: 74547.2, 300 sec: 75639.1). Total num frames: 19017728. Throughput: 0: 18433.4. Samples: 3731928. Policy #0 lag: (min: 1.0, avg: 3.1, max: 6.0) [2024-09-30 00:37:00,987][1153456] Avg episode reward: [(0, '33.036')] [2024-09-30 00:37:01,002][1153683] Saving new best policy, reward=33.036! [2024-09-30 00:37:01,285][1153805] Updated weights for policy 0, policy_version 4649 (0.0006) [2024-09-30 00:37:01,881][1153805] Updated weights for policy 0, policy_version 4659 (0.0006) [2024-09-30 00:37:02,478][1153805] Updated weights for policy 0, policy_version 4669 (0.0006) [2024-09-30 00:37:03,069][1153805] Updated weights for policy 0, policy_version 4679 (0.0006) [2024-09-30 00:37:03,661][1153805] Updated weights for policy 0, policy_version 4689 (0.0006) [2024-09-30 00:37:04,245][1153805] Updated weights for policy 0, policy_version 4699 (0.0006) [2024-09-30 00:37:04,843][1153805] Updated weights for policy 0, policy_version 4709 (0.0006) [2024-09-30 00:37:05,405][1153805] Updated weights for policy 0, policy_version 4719 (0.0006) [2024-09-30 00:37:05,938][1153805] Updated weights for policy 0, policy_version 4729 (0.0006) [2024-09-30 00:37:05,987][1153456] Fps is (10 sec: 67994.1, 60 sec: 73659.7, 300 sec: 75511.6). Total num frames: 19369984. Throughput: 0: 18232.3. Samples: 3835416. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 00:37:05,987][1153456] Avg episode reward: [(0, '35.498')] [2024-09-30 00:37:05,993][1153683] Saving new best policy, reward=35.498! [2024-09-30 00:37:06,456][1153805] Updated weights for policy 0, policy_version 4739 (0.0006) [2024-09-30 00:37:06,976][1153805] Updated weights for policy 0, policy_version 4749 (0.0006) [2024-09-30 00:37:07,533][1153805] Updated weights for policy 0, policy_version 4759 (0.0006) [2024-09-30 00:37:08,044][1153805] Updated weights for policy 0, policy_version 4769 (0.0006) [2024-09-30 00:37:08,580][1153805] Updated weights for policy 0, policy_version 4779 (0.0006) [2024-09-30 00:37:09,180][1153805] Updated weights for policy 0, policy_version 4789 (0.0006) [2024-09-30 00:37:09,702][1153805] Updated weights for policy 0, policy_version 4799 (0.0006) [2024-09-30 00:37:10,238][1153805] Updated weights for policy 0, policy_version 4809 (0.0006) [2024-09-30 00:37:10,736][1153805] Updated weights for policy 0, policy_version 4819 (0.0006) [2024-09-30 00:37:10,987][1153456] Fps is (10 sec: 73728.7, 60 sec: 73250.2, 300 sec: 75547.4). Total num frames: 19755008. Throughput: 0: 18227.1. Samples: 3893184. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 00:37:10,987][1153456] Avg episode reward: [(0, '34.137')] [2024-09-30 00:37:11,235][1153805] Updated weights for policy 0, policy_version 4829 (0.0006) [2024-09-30 00:37:11,747][1153805] Updated weights for policy 0, policy_version 4839 (0.0006) [2024-09-30 00:37:12,291][1153805] Updated weights for policy 0, policy_version 4849 (0.0006) [2024-09-30 00:37:12,854][1153805] Updated weights for policy 0, policy_version 4859 (0.0006) [2024-09-30 00:37:13,396][1153805] Updated weights for policy 0, policy_version 4869 (0.0006) [2024-09-30 00:37:13,934][1153805] Updated weights for policy 0, policy_version 4879 (0.0006) [2024-09-30 00:37:14,487][1153805] Updated weights for policy 0, policy_version 4889 (0.0006) [2024-09-30 00:37:15,023][1153805] Updated weights for policy 0, policy_version 4899 (0.0006) [2024-09-30 00:37:15,574][1153805] Updated weights for policy 0, policy_version 4909 (0.0006) [2024-09-30 00:37:15,987][1153456] Fps is (10 sec: 76595.0, 60 sec: 73796.2, 300 sec: 75562.3). Total num frames: 20135936. Throughput: 0: 18132.6. Samples: 4007936. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 00:37:15,987][1153456] Avg episode reward: [(0, '31.965')] [2024-09-30 00:37:16,103][1153805] Updated weights for policy 0, policy_version 4919 (0.0006) [2024-09-30 00:37:16,646][1153805] Updated weights for policy 0, policy_version 4929 (0.0006) [2024-09-30 00:37:17,197][1153805] Updated weights for policy 0, policy_version 4939 (0.0006) [2024-09-30 00:37:17,753][1153805] Updated weights for policy 0, policy_version 4949 (0.0006) [2024-09-30 00:37:18,290][1153805] Updated weights for policy 0, policy_version 4959 (0.0006) [2024-09-30 00:37:18,847][1153805] Updated weights for policy 0, policy_version 4969 (0.0006) [2024-09-30 00:37:19,377][1153805] Updated weights for policy 0, policy_version 4979 (0.0006) [2024-09-30 00:37:19,945][1153805] Updated weights for policy 0, policy_version 4989 (0.0006) [2024-09-30 00:37:20,483][1153805] Updated weights for policy 0, policy_version 4999 (0.0006) [2024-09-30 00:37:20,987][1153456] Fps is (10 sec: 75774.5, 60 sec: 73727.8, 300 sec: 75557.8). Total num frames: 20512768. Throughput: 0: 17993.9. Samples: 4120736. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 00:37:20,987][1153456] Avg episode reward: [(0, '31.147')] [2024-09-30 00:37:21,069][1153805] Updated weights for policy 0, policy_version 5009 (0.0006) [2024-09-30 00:37:21,564][1153805] Updated weights for policy 0, policy_version 5019 (0.0006) [2024-09-30 00:37:22,124][1153805] Updated weights for policy 0, policy_version 5029 (0.0006) [2024-09-30 00:37:22,665][1153805] Updated weights for policy 0, policy_version 5039 (0.0006) [2024-09-30 00:37:23,191][1153805] Updated weights for policy 0, policy_version 5049 (0.0006) [2024-09-30 00:37:23,748][1153805] Updated weights for policy 0, policy_version 5059 (0.0006) [2024-09-30 00:37:24,304][1153805] Updated weights for policy 0, policy_version 5069 (0.0006) [2024-09-30 00:37:24,852][1153805] Updated weights for policy 0, policy_version 5079 (0.0006) [2024-09-30 00:37:25,379][1153805] Updated weights for policy 0, policy_version 5089 (0.0006) [2024-09-30 00:37:25,920][1153805] Updated weights for policy 0, policy_version 5099 (0.0006) [2024-09-30 00:37:25,987][1153456] Fps is (10 sec: 74955.9, 60 sec: 73386.5, 300 sec: 75535.2). Total num frames: 20885504. Throughput: 0: 17969.4. Samples: 4177144. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 00:37:25,987][1153456] Avg episode reward: [(0, '30.199')] [2024-09-30 00:37:26,485][1153683] Signal inference workers to stop experience collection... (350 times) [2024-09-30 00:37:26,488][1153805] InferenceWorker_p0-w0: stopping experience collection (350 times) [2024-09-30 00:37:26,492][1153683] Signal inference workers to resume experience collection... (350 times) [2024-09-30 00:37:26,492][1153805] InferenceWorker_p0-w0: resuming experience collection (350 times) [2024-09-30 00:37:26,494][1153805] Updated weights for policy 0, policy_version 5109 (0.0006) [2024-09-30 00:37:27,004][1153805] Updated weights for policy 0, policy_version 5119 (0.0006) [2024-09-30 00:37:27,520][1153805] Updated weights for policy 0, policy_version 5129 (0.0006) [2024-09-30 00:37:28,076][1153805] Updated weights for policy 0, policy_version 5139 (0.0006) [2024-09-30 00:37:28,632][1153805] Updated weights for policy 0, policy_version 5149 (0.0006) [2024-09-30 00:37:29,138][1153805] Updated weights for policy 0, policy_version 5159 (0.0006) [2024-09-30 00:37:29,663][1153805] Updated weights for policy 0, policy_version 5169 (0.0006) [2024-09-30 00:37:30,201][1153805] Updated weights for policy 0, policy_version 5179 (0.0006) [2024-09-30 00:37:30,744][1153805] Updated weights for policy 0, policy_version 5189 (0.0006) [2024-09-30 00:37:30,987][1153456] Fps is (10 sec: 75776.8, 60 sec: 73250.2, 300 sec: 75567.4). Total num frames: 21270528. Throughput: 0: 17967.7. Samples: 4291380. Policy #0 lag: (min: 0.0, avg: 2.8, max: 7.0) [2024-09-30 00:37:30,987][1153456] Avg episode reward: [(0, '33.326')] [2024-09-30 00:37:31,297][1153805] Updated weights for policy 0, policy_version 5199 (0.0006) [2024-09-30 00:37:31,833][1153805] Updated weights for policy 0, policy_version 5209 (0.0006) [2024-09-30 00:37:32,347][1153805] Updated weights for policy 0, policy_version 5219 (0.0006) [2024-09-30 00:37:32,883][1153805] Updated weights for policy 0, policy_version 5229 (0.0006) [2024-09-30 00:37:33,442][1153805] Updated weights for policy 0, policy_version 5239 (0.0006) [2024-09-30 00:37:33,983][1153805] Updated weights for policy 0, policy_version 5249 (0.0006) [2024-09-30 00:37:34,535][1153805] Updated weights for policy 0, policy_version 5259 (0.0006) [2024-09-30 00:37:35,084][1153805] Updated weights for policy 0, policy_version 5269 (0.0006) [2024-09-30 00:37:35,680][1153805] Updated weights for policy 0, policy_version 5279 (0.0006) [2024-09-30 00:37:35,987][1153456] Fps is (10 sec: 75367.2, 60 sec: 72772.2, 300 sec: 75528.0). Total num frames: 21639168. Throughput: 0: 18204.5. Samples: 4404080. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 00:37:35,987][1153456] Avg episode reward: [(0, '35.834')] [2024-09-30 00:37:35,996][1153683] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000005284_21643264.pth... [2024-09-30 00:37:36,058][1153683] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth [2024-09-30 00:37:36,065][1153683] Saving new best policy, reward=35.834! [2024-09-30 00:37:36,363][1153805] Updated weights for policy 0, policy_version 5289 (0.0006) [2024-09-30 00:37:36,924][1153805] Updated weights for policy 0, policy_version 5299 (0.0006) [2024-09-30 00:37:37,469][1153805] Updated weights for policy 0, policy_version 5309 (0.0006) [2024-09-30 00:37:38,032][1153805] Updated weights for policy 0, policy_version 5319 (0.0006) [2024-09-30 00:37:38,584][1153805] Updated weights for policy 0, policy_version 5329 (0.0006) [2024-09-30 00:37:39,133][1153805] Updated weights for policy 0, policy_version 5339 (0.0006) [2024-09-30 00:37:39,666][1153805] Updated weights for policy 0, policy_version 5349 (0.0006) [2024-09-30 00:37:40,216][1153805] Updated weights for policy 0, policy_version 5359 (0.0006) [2024-09-30 00:37:40,749][1153805] Updated weights for policy 0, policy_version 5369 (0.0006) [2024-09-30 00:37:40,987][1153456] Fps is (10 sec: 73728.2, 60 sec: 72567.6, 300 sec: 75490.3). Total num frames: 22007808. Throughput: 0: 18301.4. Samples: 4456688. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 00:37:40,987][1153456] Avg episode reward: [(0, '33.306')] [2024-09-30 00:37:41,283][1153805] Updated weights for policy 0, policy_version 5379 (0.0006) [2024-09-30 00:37:41,867][1153805] Updated weights for policy 0, policy_version 5389 (0.0006) [2024-09-30 00:37:42,410][1153805] Updated weights for policy 0, policy_version 5399 (0.0006) [2024-09-30 00:37:42,993][1153805] Updated weights for policy 0, policy_version 5409 (0.0006) [2024-09-30 00:37:43,566][1153805] Updated weights for policy 0, policy_version 5419 (0.0006) [2024-09-30 00:37:44,117][1153805] Updated weights for policy 0, policy_version 5429 (0.0006) [2024-09-30 00:37:44,709][1153805] Updated weights for policy 0, policy_version 5439 (0.0006) [2024-09-30 00:37:45,294][1153805] Updated weights for policy 0, policy_version 5449 (0.0006) [2024-09-30 00:37:45,865][1153805] Updated weights for policy 0, policy_version 5459 (0.0006) [2024-09-30 00:37:45,987][1153456] Fps is (10 sec: 72499.8, 60 sec: 72226.3, 300 sec: 75403.6). Total num frames: 22364160. Throughput: 0: 18562.6. Samples: 4567244. Policy #0 lag: (min: 0.0, avg: 2.8, max: 7.0) [2024-09-30 00:37:45,987][1153456] Avg episode reward: [(0, '31.728')] [2024-09-30 00:37:46,452][1153805] Updated weights for policy 0, policy_version 5469 (0.0006) [2024-09-30 00:37:47,029][1153805] Updated weights for policy 0, policy_version 5479 (0.0006) [2024-09-30 00:37:47,603][1153805] Updated weights for policy 0, policy_version 5489 (0.0006) [2024-09-30 00:37:48,137][1153805] Updated weights for policy 0, policy_version 5499 (0.0006) [2024-09-30 00:37:48,647][1153805] Updated weights for policy 0, policy_version 5509 (0.0006) [2024-09-30 00:37:49,246][1153805] Updated weights for policy 0, policy_version 5519 (0.0006) [2024-09-30 00:37:49,837][1153805] Updated weights for policy 0, policy_version 5529 (0.0006) [2024-09-30 00:37:50,379][1153805] Updated weights for policy 0, policy_version 5539 (0.0006) [2024-09-30 00:37:50,987][1153456] Fps is (10 sec: 71679.5, 60 sec: 72703.9, 300 sec: 75336.9). Total num frames: 22724608. Throughput: 0: 18659.8. Samples: 4675108. Policy #0 lag: (min: 0.0, avg: 2.8, max: 7.0) [2024-09-30 00:37:50,987][1153456] Avg episode reward: [(0, '35.538')] [2024-09-30 00:37:50,993][1153805] Updated weights for policy 0, policy_version 5549 (0.0006) [2024-09-30 00:37:51,576][1153805] Updated weights for policy 0, policy_version 5559 (0.0006) [2024-09-30 00:37:51,586][1153683] Signal inference workers to stop experience collection... (400 times) [2024-09-30 00:37:51,587][1153683] Signal inference workers to resume experience collection... (400 times) [2024-09-30 00:37:51,592][1153805] InferenceWorker_p0-w0: stopping experience collection (400 times) [2024-09-30 00:37:51,592][1153805] InferenceWorker_p0-w0: resuming experience collection (400 times) [2024-09-30 00:37:52,137][1153805] Updated weights for policy 0, policy_version 5569 (0.0006) [2024-09-30 00:37:52,758][1153805] Updated weights for policy 0, policy_version 5579 (0.0006) [2024-09-30 00:37:53,430][1153805] Updated weights for policy 0, policy_version 5589 (0.0006) [2024-09-30 00:37:54,028][1153805] Updated weights for policy 0, policy_version 5599 (0.0006) [2024-09-30 00:37:54,654][1153805] Updated weights for policy 0, policy_version 5609 (0.0006) [2024-09-30 00:37:55,291][1153805] Updated weights for policy 0, policy_version 5619 (0.0006) [2024-09-30 00:37:55,812][1153805] Updated weights for policy 0, policy_version 5629 (0.0006) [2024-09-30 00:37:55,987][1153456] Fps is (10 sec: 70450.1, 60 sec: 72977.0, 300 sec: 75208.2). Total num frames: 23068672. Throughput: 0: 18507.6. Samples: 4726028. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 00:37:55,987][1153456] Avg episode reward: [(0, '32.545')] [2024-09-30 00:37:56,351][1153805] Updated weights for policy 0, policy_version 5639 (0.0006) [2024-09-30 00:37:56,938][1153805] Updated weights for policy 0, policy_version 5649 (0.0006) [2024-09-30 00:37:57,464][1153805] Updated weights for policy 0, policy_version 5659 (0.0006) [2024-09-30 00:37:58,014][1153805] Updated weights for policy 0, policy_version 5669 (0.0006) [2024-09-30 00:37:58,577][1153805] Updated weights for policy 0, policy_version 5679 (0.0006) [2024-09-30 00:37:59,128][1153805] Updated weights for policy 0, policy_version 5689 (0.0006) [2024-09-30 00:37:59,716][1153805] Updated weights for policy 0, policy_version 5699 (0.0006) [2024-09-30 00:38:00,277][1153805] Updated weights for policy 0, policy_version 5709 (0.0006) [2024-09-30 00:38:00,878][1153805] Updated weights for policy 0, policy_version 5719 (0.0006) [2024-09-30 00:38:00,987][1153456] Fps is (10 sec: 70860.6, 60 sec: 73591.4, 300 sec: 75163.7). Total num frames: 23433216. Throughput: 0: 18348.6. Samples: 4833624. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 00:38:00,987][1153456] Avg episode reward: [(0, '31.310')] [2024-09-30 00:38:01,425][1153805] Updated weights for policy 0, policy_version 5729 (0.0006) [2024-09-30 00:38:01,970][1153805] Updated weights for policy 0, policy_version 5739 (0.0006) [2024-09-30 00:38:02,544][1153805] Updated weights for policy 0, policy_version 5749 (0.0006) [2024-09-30 00:38:03,093][1153805] Updated weights for policy 0, policy_version 5759 (0.0006) [2024-09-30 00:38:03,645][1153805] Updated weights for policy 0, policy_version 5769 (0.0006) [2024-09-30 00:38:04,202][1153805] Updated weights for policy 0, policy_version 5779 (0.0006) [2024-09-30 00:38:04,777][1153805] Updated weights for policy 0, policy_version 5789 (0.0005) [2024-09-30 00:38:05,367][1153805] Updated weights for policy 0, policy_version 5799 (0.0006) [2024-09-30 00:38:05,950][1153805] Updated weights for policy 0, policy_version 5809 (0.0006) [2024-09-30 00:38:05,987][1153456] Fps is (10 sec: 72909.2, 60 sec: 73796.2, 300 sec: 75120.9). Total num frames: 23797760. Throughput: 0: 18245.4. Samples: 4941780. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 00:38:05,987][1153456] Avg episode reward: [(0, '35.395')] [2024-09-30 00:38:06,379][1153805] Updated weights for policy 0, policy_version 5819 (0.0006) [2024-09-30 00:38:06,834][1153805] Updated weights for policy 0, policy_version 5829 (0.0006) [2024-09-30 00:38:07,285][1153805] Updated weights for policy 0, policy_version 5839 (0.0006) [2024-09-30 00:38:07,726][1153805] Updated weights for policy 0, policy_version 5849 (0.0006) [2024-09-30 00:38:08,191][1153805] Updated weights for policy 0, policy_version 5859 (0.0006) [2024-09-30 00:38:08,664][1153805] Updated weights for policy 0, policy_version 5869 (0.0006) [2024-09-30 00:38:09,197][1153805] Updated weights for policy 0, policy_version 5879 (0.0007) [2024-09-30 00:38:09,717][1153805] Updated weights for policy 0, policy_version 5889 (0.0006) [2024-09-30 00:38:10,222][1153805] Updated weights for policy 0, policy_version 5899 (0.0006) [2024-09-30 00:38:10,734][1153805] Updated weights for policy 0, policy_version 5909 (0.0006) [2024-09-30 00:38:10,987][1153456] Fps is (10 sec: 79053.1, 60 sec: 74478.8, 300 sec: 75308.6). Total num frames: 24223744. Throughput: 0: 18459.2. Samples: 5007804. Policy #0 lag: (min: 0.0, avg: 2.6, max: 4.0) [2024-09-30 00:38:10,987][1153456] Avg episode reward: [(0, '34.288')] [2024-09-30 00:38:11,263][1153805] Updated weights for policy 0, policy_version 5919 (0.0006) [2024-09-30 00:38:11,739][1153805] Updated weights for policy 0, policy_version 5929 (0.0006) [2024-09-30 00:38:12,179][1153805] Updated weights for policy 0, policy_version 5939 (0.0006) [2024-09-30 00:38:12,651][1153805] Updated weights for policy 0, policy_version 5949 (0.0006) [2024-09-30 00:38:13,141][1153805] Updated weights for policy 0, policy_version 5959 (0.0006) [2024-09-30 00:38:13,647][1153805] Updated weights for policy 0, policy_version 5969 (0.0006) [2024-09-30 00:38:14,145][1153805] Updated weights for policy 0, policy_version 5979 (0.0006) [2024-09-30 00:38:14,653][1153805] Updated weights for policy 0, policy_version 5989 (0.0006) [2024-09-30 00:38:15,157][1153805] Updated weights for policy 0, policy_version 5999 (0.0006) [2024-09-30 00:38:15,325][1153683] Signal inference workers to stop experience collection... (450 times) [2024-09-30 00:38:15,326][1153683] Signal inference workers to resume experience collection... (450 times) [2024-09-30 00:38:15,330][1153805] InferenceWorker_p0-w0: stopping experience collection (450 times) [2024-09-30 00:38:15,330][1153805] InferenceWorker_p0-w0: resuming experience collection (450 times) [2024-09-30 00:38:15,676][1153805] Updated weights for policy 0, policy_version 6009 (0.0006) [2024-09-30 00:38:15,987][1153456] Fps is (10 sec: 83559.8, 60 sec: 74957.0, 300 sec: 75429.5). Total num frames: 24633344. Throughput: 0: 18669.2. Samples: 5131492. Policy #0 lag: (min: 0.0, avg: 2.2, max: 4.0) [2024-09-30 00:38:15,987][1153456] Avg episode reward: [(0, '34.141')] [2024-09-30 00:38:16,195][1153805] Updated weights for policy 0, policy_version 6019 (0.0006) [2024-09-30 00:38:16,744][1153805] Updated weights for policy 0, policy_version 6029 (0.0006) [2024-09-30 00:38:17,279][1153805] Updated weights for policy 0, policy_version 6039 (0.0006) [2024-09-30 00:38:17,787][1153805] Updated weights for policy 0, policy_version 6049 (0.0006) [2024-09-30 00:38:18,324][1153805] Updated weights for policy 0, policy_version 6059 (0.0006) [2024-09-30 00:38:18,780][1153805] Updated weights for policy 0, policy_version 6069 (0.0006) [2024-09-30 00:38:19,228][1153805] Updated weights for policy 0, policy_version 6079 (0.0006) [2024-09-30 00:38:19,724][1153805] Updated weights for policy 0, policy_version 6089 (0.0006) [2024-09-30 00:38:20,209][1153805] Updated weights for policy 0, policy_version 6099 (0.0006) [2024-09-30 00:38:20,656][1153805] Updated weights for policy 0, policy_version 6109 (0.0006) [2024-09-30 00:38:20,987][1153456] Fps is (10 sec: 82739.4, 60 sec: 75639.6, 300 sec: 75575.4). Total num frames: 25051136. Throughput: 0: 18873.6. Samples: 5253392. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 00:38:20,987][1153456] Avg episode reward: [(0, '36.407')] [2024-09-30 00:38:20,987][1153683] Saving new best policy, reward=36.407! [2024-09-30 00:38:21,130][1153805] Updated weights for policy 0, policy_version 6119 (0.0006) [2024-09-30 00:38:21,599][1153805] Updated weights for policy 0, policy_version 6129 (0.0006) [2024-09-30 00:38:22,096][1153805] Updated weights for policy 0, policy_version 6139 (0.0006) [2024-09-30 00:38:22,603][1153805] Updated weights for policy 0, policy_version 6149 (0.0006) [2024-09-30 00:38:23,066][1153805] Updated weights for policy 0, policy_version 6159 (0.0006) [2024-09-30 00:38:23,527][1153805] Updated weights for policy 0, policy_version 6169 (0.0006) [2024-09-30 00:38:24,044][1153805] Updated weights for policy 0, policy_version 6179 (0.0006) [2024-09-30 00:38:24,551][1153805] Updated weights for policy 0, policy_version 6189 (0.0006) [2024-09-30 00:38:25,055][1153805] Updated weights for policy 0, policy_version 6199 (0.0006) [2024-09-30 00:38:25,570][1153805] Updated weights for policy 0, policy_version 6209 (0.0006) [2024-09-30 00:38:25,987][1153456] Fps is (10 sec: 82739.0, 60 sec: 76254.2, 300 sec: 75687.4). Total num frames: 25460736. Throughput: 0: 19130.2. Samples: 5317548. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 00:38:25,987][1153456] Avg episode reward: [(0, '36.124')] [2024-09-30 00:38:26,100][1153805] Updated weights for policy 0, policy_version 6219 (0.0006) [2024-09-30 00:38:26,688][1153805] Updated weights for policy 0, policy_version 6229 (0.0006) [2024-09-30 00:38:27,155][1153805] Updated weights for policy 0, policy_version 6239 (0.0006) [2024-09-30 00:38:27,684][1153805] Updated weights for policy 0, policy_version 6249 (0.0006) [2024-09-30 00:38:28,201][1153805] Updated weights for policy 0, policy_version 6259 (0.0006) [2024-09-30 00:38:28,716][1153805] Updated weights for policy 0, policy_version 6269 (0.0006) [2024-09-30 00:38:29,221][1153805] Updated weights for policy 0, policy_version 6279 (0.0006) [2024-09-30 00:38:29,713][1153805] Updated weights for policy 0, policy_version 6289 (0.0006) [2024-09-30 00:38:30,263][1153805] Updated weights for policy 0, policy_version 6299 (0.0006) [2024-09-30 00:38:30,756][1153805] Updated weights for policy 0, policy_version 6309 (0.0006) [2024-09-30 00:38:30,987][1153456] Fps is (10 sec: 80692.2, 60 sec: 76458.8, 300 sec: 75752.8). Total num frames: 25858048. Throughput: 0: 19327.5. Samples: 5436980. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 00:38:30,987][1153456] Avg episode reward: [(0, '36.220')] [2024-09-30 00:38:31,288][1153805] Updated weights for policy 0, policy_version 6319 (0.0006) [2024-09-30 00:38:31,786][1153805] Updated weights for policy 0, policy_version 6329 (0.0006) [2024-09-30 00:38:32,300][1153805] Updated weights for policy 0, policy_version 6339 (0.0006) [2024-09-30 00:38:32,817][1153805] Updated weights for policy 0, policy_version 6349 (0.0006) [2024-09-30 00:38:33,141][1153683] Signal inference workers to stop experience collection... (500 times) [2024-09-30 00:38:33,142][1153683] Signal inference workers to resume experience collection... (500 times) [2024-09-30 00:38:33,146][1153805] InferenceWorker_p0-w0: stopping experience collection (500 times) [2024-09-30 00:38:33,146][1153805] InferenceWorker_p0-w0: resuming experience collection (500 times) [2024-09-30 00:38:33,322][1153805] Updated weights for policy 0, policy_version 6359 (0.0006) [2024-09-30 00:38:33,828][1153805] Updated weights for policy 0, policy_version 6369 (0.0006) [2024-09-30 00:38:34,375][1153805] Updated weights for policy 0, policy_version 6379 (0.0006) [2024-09-30 00:38:34,903][1153805] Updated weights for policy 0, policy_version 6389 (0.0006) [2024-09-30 00:38:35,470][1153805] Updated weights for policy 0, policy_version 6399 (0.0006) [2024-09-30 00:38:35,987][1153456] Fps is (10 sec: 78643.5, 60 sec: 76800.2, 300 sec: 75788.1). Total num frames: 26247168. Throughput: 0: 19545.0. Samples: 5554628. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 00:38:35,987][1153456] Avg episode reward: [(0, '33.533')] [2024-09-30 00:38:36,005][1153805] Updated weights for policy 0, policy_version 6409 (0.0006) [2024-09-30 00:38:36,554][1153805] Updated weights for policy 0, policy_version 6419 (0.0006) [2024-09-30 00:38:37,136][1153805] Updated weights for policy 0, policy_version 6429 (0.0006) [2024-09-30 00:38:37,662][1153805] Updated weights for policy 0, policy_version 6439 (0.0006) [2024-09-30 00:38:38,183][1153805] Updated weights for policy 0, policy_version 6449 (0.0006) [2024-09-30 00:38:38,727][1153805] Updated weights for policy 0, policy_version 6459 (0.0006) [2024-09-30 00:38:39,231][1153805] Updated weights for policy 0, policy_version 6469 (0.0006) [2024-09-30 00:38:39,770][1153805] Updated weights for policy 0, policy_version 6479 (0.0006) [2024-09-30 00:38:40,301][1153805] Updated weights for policy 0, policy_version 6489 (0.0006) [2024-09-30 00:38:40,796][1153805] Updated weights for policy 0, policy_version 6499 (0.0006) [2024-09-30 00:38:40,987][1153456] Fps is (10 sec: 77414.6, 60 sec: 77073.2, 300 sec: 76699.4). Total num frames: 26632192. Throughput: 0: 19670.1. Samples: 5611176. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 00:38:40,987][1153456] Avg episode reward: [(0, '33.683')] [2024-09-30 00:38:41,295][1153805] Updated weights for policy 0, policy_version 6509 (0.0006) [2024-09-30 00:38:41,846][1153805] Updated weights for policy 0, policy_version 6519 (0.0006) [2024-09-30 00:38:42,349][1153805] Updated weights for policy 0, policy_version 6529 (0.0006) [2024-09-30 00:38:42,868][1153805] Updated weights for policy 0, policy_version 6539 (0.0006) [2024-09-30 00:38:43,369][1153805] Updated weights for policy 0, policy_version 6549 (0.0006) [2024-09-30 00:38:43,896][1153805] Updated weights for policy 0, policy_version 6559 (0.0006) [2024-09-30 00:38:44,423][1153805] Updated weights for policy 0, policy_version 6569 (0.0006) [2024-09-30 00:38:44,983][1153805] Updated weights for policy 0, policy_version 6579 (0.0006) [2024-09-30 00:38:45,539][1153805] Updated weights for policy 0, policy_version 6589 (0.0006) [2024-09-30 00:38:45,987][1153456] Fps is (10 sec: 77823.9, 60 sec: 77687.6, 300 sec: 77310.3). Total num frames: 27025408. Throughput: 0: 19904.2. Samples: 5729312. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 00:38:45,987][1153456] Avg episode reward: [(0, '35.226')] [2024-09-30 00:38:46,040][1153805] Updated weights for policy 0, policy_version 6599 (0.0006) [2024-09-30 00:38:46,534][1153805] Updated weights for policy 0, policy_version 6609 (0.0006) [2024-09-30 00:38:47,048][1153805] Updated weights for policy 0, policy_version 6619 (0.0006) [2024-09-30 00:38:47,568][1153805] Updated weights for policy 0, policy_version 6629 (0.0006) [2024-09-30 00:38:48,136][1153805] Updated weights for policy 0, policy_version 6639 (0.0006) [2024-09-30 00:38:48,693][1153805] Updated weights for policy 0, policy_version 6649 (0.0006) [2024-09-30 00:38:49,245][1153805] Updated weights for policy 0, policy_version 6659 (0.0006) [2024-09-30 00:38:49,768][1153805] Updated weights for policy 0, policy_version 6669 (0.0006) [2024-09-30 00:38:50,257][1153805] Updated weights for policy 0, policy_version 6679 (0.0006) [2024-09-30 00:38:50,790][1153805] Updated weights for policy 0, policy_version 6689 (0.0006) [2024-09-30 00:38:50,987][1153456] Fps is (10 sec: 77823.8, 60 sec: 78097.3, 300 sec: 77310.3). Total num frames: 27410432. Throughput: 0: 20076.0. Samples: 5845196. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 00:38:50,987][1153456] Avg episode reward: [(0, '36.787')] [2024-09-30 00:38:50,987][1153683] Saving new best policy, reward=36.787! [2024-09-30 00:38:51,376][1153805] Updated weights for policy 0, policy_version 6699 (0.0006) [2024-09-30 00:38:51,882][1153805] Updated weights for policy 0, policy_version 6709 (0.0006) [2024-09-30 00:38:52,390][1153805] Updated weights for policy 0, policy_version 6719 (0.0006) [2024-09-30 00:38:52,903][1153805] Updated weights for policy 0, policy_version 6729 (0.0006) [2024-09-30 00:38:53,405][1153805] Updated weights for policy 0, policy_version 6739 (0.0006) [2024-09-30 00:38:53,881][1153805] Updated weights for policy 0, policy_version 6749 (0.0006) [2024-09-30 00:38:54,400][1153805] Updated weights for policy 0, policy_version 6759 (0.0006) [2024-09-30 00:38:54,906][1153805] Updated weights for policy 0, policy_version 6769 (0.0006) [2024-09-30 00:38:55,406][1153805] Updated weights for policy 0, policy_version 6779 (0.0006) [2024-09-30 00:38:55,933][1153805] Updated weights for policy 0, policy_version 6789 (0.0006) [2024-09-30 00:38:55,987][1153456] Fps is (10 sec: 78643.2, 60 sec: 79053.1, 300 sec: 77310.3). Total num frames: 27811840. Throughput: 0: 19923.3. Samples: 5904348. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 00:38:55,987][1153456] Avg episode reward: [(0, '36.261')] [2024-09-30 00:38:56,457][1153805] Updated weights for policy 0, policy_version 6799 (0.0006) [2024-09-30 00:38:56,994][1153805] Updated weights for policy 0, policy_version 6809 (0.0006) [2024-09-30 00:38:57,514][1153805] Updated weights for policy 0, policy_version 6819 (0.0006) [2024-09-30 00:38:57,635][1153683] Signal inference workers to stop experience collection... (550 times) [2024-09-30 00:38:57,636][1153683] Signal inference workers to resume experience collection... (550 times) [2024-09-30 00:38:57,639][1153805] InferenceWorker_p0-w0: stopping experience collection (550 times) [2024-09-30 00:38:57,642][1153805] InferenceWorker_p0-w0: resuming experience collection (550 times) [2024-09-30 00:38:58,012][1153805] Updated weights for policy 0, policy_version 6829 (0.0006) [2024-09-30 00:38:58,526][1153805] Updated weights for policy 0, policy_version 6839 (0.0006) [2024-09-30 00:38:59,060][1153805] Updated weights for policy 0, policy_version 6849 (0.0006) [2024-09-30 00:38:59,621][1153805] Updated weights for policy 0, policy_version 6859 (0.0006) [2024-09-30 00:39:00,137][1153805] Updated weights for policy 0, policy_version 6869 (0.0006) [2024-09-30 00:39:00,635][1153805] Updated weights for policy 0, policy_version 6879 (0.0006) [2024-09-30 00:39:00,987][1153456] Fps is (10 sec: 79052.6, 60 sec: 79462.6, 300 sec: 77254.8). Total num frames: 28200960. Throughput: 0: 19812.7. Samples: 6023064. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 00:39:00,987][1153456] Avg episode reward: [(0, '37.537')] [2024-09-30 00:39:00,987][1153683] Saving new best policy, reward=37.537! [2024-09-30 00:39:01,189][1153805] Updated weights for policy 0, policy_version 6889 (0.0006) [2024-09-30 00:39:01,723][1153805] Updated weights for policy 0, policy_version 6899 (0.0006) [2024-09-30 00:39:02,306][1153805] Updated weights for policy 0, policy_version 6909 (0.0006) [2024-09-30 00:39:02,857][1153805] Updated weights for policy 0, policy_version 6919 (0.0006) [2024-09-30 00:39:03,423][1153805] Updated weights for policy 0, policy_version 6929 (0.0006) [2024-09-30 00:39:03,974][1153805] Updated weights for policy 0, policy_version 6939 (0.0006) [2024-09-30 00:39:04,545][1153805] Updated weights for policy 0, policy_version 6949 (0.0006) [2024-09-30 00:39:05,108][1153805] Updated weights for policy 0, policy_version 6959 (0.0006) [2024-09-30 00:39:05,646][1153805] Updated weights for policy 0, policy_version 6969 (0.0006) [2024-09-30 00:39:05,987][1153456] Fps is (10 sec: 75776.1, 60 sec: 79530.9, 300 sec: 77129.8). Total num frames: 28569600. Throughput: 0: 19589.9. Samples: 6134936. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 00:39:05,987][1153456] Avg episode reward: [(0, '31.062')] [2024-09-30 00:39:06,198][1153805] Updated weights for policy 0, policy_version 6979 (0.0006) [2024-09-30 00:39:06,782][1153805] Updated weights for policy 0, policy_version 6989 (0.0006) [2024-09-30 00:39:07,289][1153805] Updated weights for policy 0, policy_version 6999 (0.0006) [2024-09-30 00:39:07,855][1153805] Updated weights for policy 0, policy_version 7009 (0.0006) [2024-09-30 00:39:08,422][1153805] Updated weights for policy 0, policy_version 7019 (0.0006) [2024-09-30 00:39:08,962][1153805] Updated weights for policy 0, policy_version 7029 (0.0006) [2024-09-30 00:39:09,472][1153805] Updated weights for policy 0, policy_version 7039 (0.0006) [2024-09-30 00:39:09,974][1153805] Updated weights for policy 0, policy_version 7049 (0.0006) [2024-09-30 00:39:10,493][1153805] Updated weights for policy 0, policy_version 7059 (0.0006) [2024-09-30 00:39:10,987][1153456] Fps is (10 sec: 74547.6, 60 sec: 78711.7, 300 sec: 77157.5). Total num frames: 28946432. Throughput: 0: 19397.1. Samples: 6190416. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 00:39:10,987][1153456] Avg episode reward: [(0, '37.698')] [2024-09-30 00:39:10,987][1153683] Saving new best policy, reward=37.698! [2024-09-30 00:39:11,073][1153805] Updated weights for policy 0, policy_version 7069 (0.0006) [2024-09-30 00:39:11,653][1153805] Updated weights for policy 0, policy_version 7079 (0.0006) [2024-09-30 00:39:12,211][1153805] Updated weights for policy 0, policy_version 7089 (0.0006) [2024-09-30 00:39:12,784][1153805] Updated weights for policy 0, policy_version 7099 (0.0006) [2024-09-30 00:39:13,328][1153805] Updated weights for policy 0, policy_version 7109 (0.0006) [2024-09-30 00:39:13,907][1153805] Updated weights for policy 0, policy_version 7119 (0.0006) [2024-09-30 00:39:14,423][1153805] Updated weights for policy 0, policy_version 7129 (0.0006) [2024-09-30 00:39:14,999][1153805] Updated weights for policy 0, policy_version 7139 (0.0006) [2024-09-30 00:39:15,563][1153805] Updated weights for policy 0, policy_version 7149 (0.0006) [2024-09-30 00:39:15,987][1153456] Fps is (10 sec: 73727.9, 60 sec: 77892.3, 300 sec: 76990.9). Total num frames: 29306880. Throughput: 0: 19229.0. Samples: 6302284. Policy #0 lag: (min: 0.0, avg: 1.7, max: 5.0) [2024-09-30 00:39:15,987][1153456] Avg episode reward: [(0, '37.531')] [2024-09-30 00:39:16,120][1153805] Updated weights for policy 0, policy_version 7159 (0.0006) [2024-09-30 00:39:16,717][1153805] Updated weights for policy 0, policy_version 7169 (0.0006) [2024-09-30 00:39:17,248][1153805] Updated weights for policy 0, policy_version 7179 (0.0006) [2024-09-30 00:39:17,807][1153805] Updated weights for policy 0, policy_version 7189 (0.0006) [2024-09-30 00:39:18,381][1153805] Updated weights for policy 0, policy_version 7199 (0.0006) [2024-09-30 00:39:18,895][1153805] Updated weights for policy 0, policy_version 7209 (0.0006) [2024-09-30 00:39:19,442][1153805] Updated weights for policy 0, policy_version 7219 (0.0006) [2024-09-30 00:39:19,998][1153805] Updated weights for policy 0, policy_version 7229 (0.0006) [2024-09-30 00:39:20,524][1153805] Updated weights for policy 0, policy_version 7239 (0.0006) [2024-09-30 00:39:20,987][1153456] Fps is (10 sec: 73727.2, 60 sec: 77209.7, 300 sec: 76907.6). Total num frames: 29683712. Throughput: 0: 19084.1. Samples: 6413416. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 00:39:20,987][1153456] Avg episode reward: [(0, '40.574')] [2024-09-30 00:39:20,988][1153683] Saving new best policy, reward=40.574! [2024-09-30 00:39:21,072][1153805] Updated weights for policy 0, policy_version 7249 (0.0006) [2024-09-30 00:39:21,657][1153805] Updated weights for policy 0, policy_version 7259 (0.0006) [2024-09-30 00:39:22,183][1153805] Updated weights for policy 0, policy_version 7269 (0.0006) [2024-09-30 00:39:22,755][1153805] Updated weights for policy 0, policy_version 7279 (0.0006) [2024-09-30 00:39:23,315][1153805] Updated weights for policy 0, policy_version 7289 (0.0006) [2024-09-30 00:39:23,825][1153805] Updated weights for policy 0, policy_version 7299 (0.0006) [2024-09-30 00:39:24,351][1153805] Updated weights for policy 0, policy_version 7309 (0.0006) [2024-09-30 00:39:24,854][1153805] Updated weights for policy 0, policy_version 7319 (0.0006) [2024-09-30 00:39:25,379][1153805] Updated weights for policy 0, policy_version 7329 (0.0006) [2024-09-30 00:39:25,884][1153805] Updated weights for policy 0, policy_version 7339 (0.0006) [2024-09-30 00:39:25,987][1153456] Fps is (10 sec: 76185.6, 60 sec: 76800.0, 300 sec: 76935.4). Total num frames: 30068736. Throughput: 0: 19063.5. Samples: 6469036. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 00:39:25,987][1153456] Avg episode reward: [(0, '36.766')] [2024-09-30 00:39:26,392][1153805] Updated weights for policy 0, policy_version 7349 (0.0006) [2024-09-30 00:39:26,886][1153805] Updated weights for policy 0, policy_version 7359 (0.0006) [2024-09-30 00:39:26,954][1153683] Signal inference workers to stop experience collection... (600 times) [2024-09-30 00:39:26,956][1153805] InferenceWorker_p0-w0: stopping experience collection (600 times) [2024-09-30 00:39:26,961][1153683] Signal inference workers to resume experience collection... (600 times) [2024-09-30 00:39:26,962][1153805] InferenceWorker_p0-w0: resuming experience collection (600 times) [2024-09-30 00:39:27,404][1153805] Updated weights for policy 0, policy_version 7369 (0.0007) [2024-09-30 00:39:27,906][1153805] Updated weights for policy 0, policy_version 7379 (0.0006) [2024-09-30 00:39:28,419][1153805] Updated weights for policy 0, policy_version 7389 (0.0006) [2024-09-30 00:39:28,919][1153805] Updated weights for policy 0, policy_version 7399 (0.0006) [2024-09-30 00:39:29,413][1153805] Updated weights for policy 0, policy_version 7409 (0.0006) [2024-09-30 00:39:29,959][1153805] Updated weights for policy 0, policy_version 7419 (0.0006) [2024-09-30 00:39:30,484][1153805] Updated weights for policy 0, policy_version 7429 (0.0006) [2024-09-30 00:39:30,987][1153456] Fps is (10 sec: 78233.6, 60 sec: 76799.9, 300 sec: 77157.5). Total num frames: 30466048. Throughput: 0: 19112.9. Samples: 6589396. Policy #0 lag: (min: 0.0, avg: 1.5, max: 4.0) [2024-09-30 00:39:30,987][1153456] Avg episode reward: [(0, '38.132')] [2024-09-30 00:39:31,012][1153805] Updated weights for policy 0, policy_version 7439 (0.0006) [2024-09-30 00:39:31,513][1153805] Updated weights for policy 0, policy_version 7449 (0.0006) [2024-09-30 00:39:32,024][1153805] Updated weights for policy 0, policy_version 7459 (0.0006) [2024-09-30 00:39:32,533][1153805] Updated weights for policy 0, policy_version 7469 (0.0006) [2024-09-30 00:39:33,012][1153805] Updated weights for policy 0, policy_version 7479 (0.0006) [2024-09-30 00:39:33,496][1153805] Updated weights for policy 0, policy_version 7489 (0.0006) [2024-09-30 00:39:33,987][1153805] Updated weights for policy 0, policy_version 7499 (0.0006) [2024-09-30 00:39:34,487][1153805] Updated weights for policy 0, policy_version 7509 (0.0007) [2024-09-30 00:39:34,974][1153805] Updated weights for policy 0, policy_version 7519 (0.0006) [2024-09-30 00:39:35,447][1153805] Updated weights for policy 0, policy_version 7529 (0.0007) [2024-09-30 00:39:35,957][1153805] Updated weights for policy 0, policy_version 7539 (0.0006) [2024-09-30 00:39:35,987][1153456] Fps is (10 sec: 81100.9, 60 sec: 77209.6, 300 sec: 77352.0). Total num frames: 30879744. Throughput: 0: 19258.4. Samples: 6711824. Policy #0 lag: (min: 0.0, avg: 1.5, max: 4.0) [2024-09-30 00:39:35,987][1153456] Avg episode reward: [(0, '37.346')] [2024-09-30 00:39:35,990][1153683] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000007540_30883840.pth... [2024-09-30 00:39:36,036][1153683] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000003049_12488704.pth [2024-09-30 00:39:36,450][1153805] Updated weights for policy 0, policy_version 7549 (0.0007) [2024-09-30 00:39:36,946][1153805] Updated weights for policy 0, policy_version 7559 (0.0006) [2024-09-30 00:39:37,429][1153805] Updated weights for policy 0, policy_version 7569 (0.0006) [2024-09-30 00:39:37,933][1153805] Updated weights for policy 0, policy_version 7579 (0.0006) [2024-09-30 00:39:38,436][1153805] Updated weights for policy 0, policy_version 7589 (0.0006) [2024-09-30 00:39:38,945][1153805] Updated weights for policy 0, policy_version 7599 (0.0006) [2024-09-30 00:39:39,448][1153805] Updated weights for policy 0, policy_version 7609 (0.0006) [2024-09-30 00:39:39,963][1153805] Updated weights for policy 0, policy_version 7619 (0.0006) [2024-09-30 00:39:40,477][1153805] Updated weights for policy 0, policy_version 7629 (0.0006) [2024-09-30 00:39:40,987][1153456] Fps is (10 sec: 81920.2, 60 sec: 77550.8, 300 sec: 77338.1). Total num frames: 31285248. Throughput: 0: 19327.2. Samples: 6774072. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 00:39:40,987][1153456] Avg episode reward: [(0, '32.475')] [2024-09-30 00:39:41,008][1153805] Updated weights for policy 0, policy_version 7639 (0.0006) [2024-09-30 00:39:41,562][1153805] Updated weights for policy 0, policy_version 7649 (0.0007) [2024-09-30 00:39:42,115][1153805] Updated weights for policy 0, policy_version 7659 (0.0006) [2024-09-30 00:39:42,657][1153805] Updated weights for policy 0, policy_version 7669 (0.0007) [2024-09-30 00:39:43,176][1153805] Updated weights for policy 0, policy_version 7679 (0.0006) [2024-09-30 00:39:43,704][1153805] Updated weights for policy 0, policy_version 7689 (0.0006) [2024-09-30 00:39:44,267][1153805] Updated weights for policy 0, policy_version 7699 (0.0006) [2024-09-30 00:39:44,804][1153805] Updated weights for policy 0, policy_version 7709 (0.0006) [2024-09-30 00:39:45,342][1153805] Updated weights for policy 0, policy_version 7719 (0.0006) [2024-09-30 00:39:45,916][1153805] Updated weights for policy 0, policy_version 7729 (0.0006) [2024-09-30 00:39:45,987][1153456] Fps is (10 sec: 78643.1, 60 sec: 77346.1, 300 sec: 77463.1). Total num frames: 31666176. Throughput: 0: 19257.4. Samples: 6889648. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 00:39:45,987][1153456] Avg episode reward: [(0, '38.643')] [2024-09-30 00:39:46,449][1153805] Updated weights for policy 0, policy_version 7739 (0.0006) [2024-09-30 00:39:46,982][1153805] Updated weights for policy 0, policy_version 7749 (0.0006) [2024-09-30 00:39:47,494][1153805] Updated weights for policy 0, policy_version 7759 (0.0006) [2024-09-30 00:39:48,030][1153805] Updated weights for policy 0, policy_version 7769 (0.0006) [2024-09-30 00:39:48,558][1153805] Updated weights for policy 0, policy_version 7779 (0.0006) [2024-09-30 00:39:49,115][1153805] Updated weights for policy 0, policy_version 7789 (0.0006) [2024-09-30 00:39:49,690][1153805] Updated weights for policy 0, policy_version 7799 (0.0006) [2024-09-30 00:39:50,207][1153805] Updated weights for policy 0, policy_version 7809 (0.0006) [2024-09-30 00:39:50,748][1153805] Updated weights for policy 0, policy_version 7819 (0.0006) [2024-09-30 00:39:50,987][1153456] Fps is (10 sec: 76185.5, 60 sec: 77277.8, 300 sec: 77588.0). Total num frames: 32047104. Throughput: 0: 19303.1. Samples: 7003576. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 00:39:50,987][1153456] Avg episode reward: [(0, '36.930')] [2024-09-30 00:39:51,230][1153805] Updated weights for policy 0, policy_version 7829 (0.0005) [2024-09-30 00:39:51,769][1153805] Updated weights for policy 0, policy_version 7839 (0.0006) [2024-09-30 00:39:52,271][1153805] Updated weights for policy 0, policy_version 7849 (0.0007) [2024-09-30 00:39:52,320][1153683] Signal inference workers to stop experience collection... (650 times) [2024-09-30 00:39:52,324][1153683] Signal inference workers to resume experience collection... (650 times) [2024-09-30 00:39:52,325][1153805] InferenceWorker_p0-w0: stopping experience collection (650 times) [2024-09-30 00:39:52,328][1153805] InferenceWorker_p0-w0: resuming experience collection (650 times) [2024-09-30 00:39:52,784][1153805] Updated weights for policy 0, policy_version 7859 (0.0006) [2024-09-30 00:39:53,287][1153805] Updated weights for policy 0, policy_version 7869 (0.0006) [2024-09-30 00:39:53,804][1153805] Updated weights for policy 0, policy_version 7879 (0.0006) [2024-09-30 00:39:54,327][1153805] Updated weights for policy 0, policy_version 7889 (0.0006) [2024-09-30 00:39:54,840][1153805] Updated weights for policy 0, policy_version 7899 (0.0006) [2024-09-30 00:39:55,366][1153805] Updated weights for policy 0, policy_version 7909 (0.0006) [2024-09-30 00:39:55,876][1153805] Updated weights for policy 0, policy_version 7919 (0.0006) [2024-09-30 00:39:55,994][1153456] Fps is (10 sec: 77768.1, 60 sec: 77200.4, 300 sec: 77655.5). Total num frames: 32444416. Throughput: 0: 19397.2. Samples: 7063428. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 00:39:55,995][1153456] Avg episode reward: [(0, '32.909')] [2024-09-30 00:39:56,394][1153805] Updated weights for policy 0, policy_version 7929 (0.0007) [2024-09-30 00:39:56,906][1153805] Updated weights for policy 0, policy_version 7939 (0.0006) [2024-09-30 00:39:57,435][1153805] Updated weights for policy 0, policy_version 7949 (0.0006) [2024-09-30 00:39:57,972][1153805] Updated weights for policy 0, policy_version 7959 (0.0006) [2024-09-30 00:39:58,494][1153805] Updated weights for policy 0, policy_version 7969 (0.0006) [2024-09-30 00:39:59,032][1153805] Updated weights for policy 0, policy_version 7979 (0.0006) [2024-09-30 00:39:59,555][1153805] Updated weights for policy 0, policy_version 7989 (0.0006) [2024-09-30 00:40:00,089][1153805] Updated weights for policy 0, policy_version 7999 (0.0006) [2024-09-30 00:40:00,622][1153805] Updated weights for policy 0, policy_version 8009 (0.0006) [2024-09-30 00:40:00,987][1153456] Fps is (10 sec: 78643.9, 60 sec: 77209.7, 300 sec: 77560.2). Total num frames: 32833536. Throughput: 0: 19529.9. Samples: 7181128. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 00:40:00,987][1153456] Avg episode reward: [(0, '38.021')] [2024-09-30 00:40:01,066][1153805] Updated weights for policy 0, policy_version 8019 (0.0006) [2024-09-30 00:40:01,563][1153805] Updated weights for policy 0, policy_version 8029 (0.0006) [2024-09-30 00:40:02,005][1153805] Updated weights for policy 0, policy_version 8039 (0.0006) [2024-09-30 00:40:02,491][1153805] Updated weights for policy 0, policy_version 8049 (0.0006) [2024-09-30 00:40:02,984][1153805] Updated weights for policy 0, policy_version 8059 (0.0006) [2024-09-30 00:40:03,473][1153805] Updated weights for policy 0, policy_version 8069 (0.0006) [2024-09-30 00:40:03,958][1153805] Updated weights for policy 0, policy_version 8079 (0.0006) [2024-09-30 00:40:04,448][1153805] Updated weights for policy 0, policy_version 8089 (0.0006) [2024-09-30 00:40:04,923][1153805] Updated weights for policy 0, policy_version 8099 (0.0006) [2024-09-30 00:40:05,414][1153805] Updated weights for policy 0, policy_version 8109 (0.0006) [2024-09-30 00:40:05,872][1153805] Updated weights for policy 0, policy_version 8119 (0.0006) [2024-09-30 00:40:05,987][1153456] Fps is (10 sec: 81978.8, 60 sec: 78233.6, 300 sec: 77574.1). Total num frames: 33263616. Throughput: 0: 19866.1. Samples: 7307388. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 00:40:05,987][1153456] Avg episode reward: [(0, '38.524')] [2024-09-30 00:40:06,339][1153805] Updated weights for policy 0, policy_version 8129 (0.0006) [2024-09-30 00:40:06,803][1153805] Updated weights for policy 0, policy_version 8139 (0.0006) [2024-09-30 00:40:07,296][1153805] Updated weights for policy 0, policy_version 8149 (0.0006) [2024-09-30 00:40:07,803][1153805] Updated weights for policy 0, policy_version 8159 (0.0006) [2024-09-30 00:40:08,302][1153805] Updated weights for policy 0, policy_version 8169 (0.0006) [2024-09-30 00:40:08,809][1153805] Updated weights for policy 0, policy_version 8179 (0.0006) [2024-09-30 00:40:09,281][1153805] Updated weights for policy 0, policy_version 8189 (0.0006) [2024-09-30 00:40:09,784][1153805] Updated weights for policy 0, policy_version 8199 (0.0006) [2024-09-30 00:40:10,308][1153805] Updated weights for policy 0, policy_version 8209 (0.0006) [2024-09-30 00:40:10,864][1153805] Updated weights for policy 0, policy_version 8219 (0.0006) [2024-09-30 00:40:10,987][1153456] Fps is (10 sec: 83966.9, 60 sec: 78779.6, 300 sec: 77546.3). Total num frames: 33673216. Throughput: 0: 20033.8. Samples: 7370560. Policy #0 lag: (min: 0.0, avg: 2.3, max: 4.0) [2024-09-30 00:40:10,987][1153456] Avg episode reward: [(0, '36.684')] [2024-09-30 00:40:11,401][1153805] Updated weights for policy 0, policy_version 8229 (0.0006) [2024-09-30 00:40:11,959][1153805] Updated weights for policy 0, policy_version 8239 (0.0007) [2024-09-30 00:40:12,516][1153805] Updated weights for policy 0, policy_version 8249 (0.0006) [2024-09-30 00:40:13,065][1153805] Updated weights for policy 0, policy_version 8259 (0.0006) [2024-09-30 00:40:13,619][1153805] Updated weights for policy 0, policy_version 8269 (0.0006) [2024-09-30 00:40:14,143][1153805] Updated weights for policy 0, policy_version 8279 (0.0007) [2024-09-30 00:40:14,698][1153805] Updated weights for policy 0, policy_version 8289 (0.0006) [2024-09-30 00:40:15,264][1153805] Updated weights for policy 0, policy_version 8299 (0.0006) [2024-09-30 00:40:15,843][1153805] Updated weights for policy 0, policy_version 8309 (0.0006) [2024-09-30 00:40:15,987][1153456] Fps is (10 sec: 77823.4, 60 sec: 78916.2, 300 sec: 77379.7). Total num frames: 34041856. Throughput: 0: 19911.3. Samples: 7485404. Policy #0 lag: (min: 0.0, avg: 2.3, max: 4.0) [2024-09-30 00:40:15,987][1153456] Avg episode reward: [(0, '39.926')] [2024-09-30 00:40:16,429][1153805] Updated weights for policy 0, policy_version 8319 (0.0006) [2024-09-30 00:40:17,073][1153805] Updated weights for policy 0, policy_version 8329 (0.0006) [2024-09-30 00:40:17,710][1153805] Updated weights for policy 0, policy_version 8339 (0.0006) [2024-09-30 00:40:17,710][1153683] Signal inference workers to stop experience collection... (700 times) [2024-09-30 00:40:17,711][1153683] Signal inference workers to resume experience collection... (700 times) [2024-09-30 00:40:17,714][1153805] InferenceWorker_p0-w0: stopping experience collection (700 times) [2024-09-30 00:40:17,714][1153805] InferenceWorker_p0-w0: resuming experience collection (700 times) [2024-09-30 00:40:18,299][1153805] Updated weights for policy 0, policy_version 8349 (0.0006) [2024-09-30 00:40:18,871][1153805] Updated weights for policy 0, policy_version 8359 (0.0006) [2024-09-30 00:40:19,454][1153805] Updated weights for policy 0, policy_version 8369 (0.0006) [2024-09-30 00:40:20,063][1153805] Updated weights for policy 0, policy_version 8379 (0.0006) [2024-09-30 00:40:20,608][1153805] Updated weights for policy 0, policy_version 8389 (0.0006) [2024-09-30 00:40:20,987][1153456] Fps is (10 sec: 71270.5, 60 sec: 78370.1, 300 sec: 77088.1). Total num frames: 34385920. Throughput: 0: 19502.4. Samples: 7589436. Policy #0 lag: (min: 0.0, avg: 3.3, max: 6.0) [2024-09-30 00:40:20,987][1153456] Avg episode reward: [(0, '38.640')] [2024-09-30 00:40:21,184][1153805] Updated weights for policy 0, policy_version 8399 (0.0006) [2024-09-30 00:40:21,798][1153805] Updated weights for policy 0, policy_version 8409 (0.0006) [2024-09-30 00:40:22,354][1153805] Updated weights for policy 0, policy_version 8419 (0.0006) [2024-09-30 00:40:22,894][1153805] Updated weights for policy 0, policy_version 8429 (0.0006) [2024-09-30 00:40:23,502][1153805] Updated weights for policy 0, policy_version 8439 (0.0006) [2024-09-30 00:40:24,060][1153805] Updated weights for policy 0, policy_version 8449 (0.0006) [2024-09-30 00:40:24,629][1153805] Updated weights for policy 0, policy_version 8459 (0.0006) [2024-09-30 00:40:25,202][1153805] Updated weights for policy 0, policy_version 8469 (0.0006) [2024-09-30 00:40:25,786][1153805] Updated weights for policy 0, policy_version 8479 (0.0006) [2024-09-30 00:40:25,987][1153456] Fps is (10 sec: 70451.3, 60 sec: 77960.4, 300 sec: 76893.8). Total num frames: 34746368. Throughput: 0: 19310.0. Samples: 7643024. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 00:40:25,987][1153456] Avg episode reward: [(0, '40.783')] [2024-09-30 00:40:25,990][1153683] Saving new best policy, reward=40.783! [2024-09-30 00:40:26,314][1153805] Updated weights for policy 0, policy_version 8489 (0.0006) [2024-09-30 00:40:26,845][1153805] Updated weights for policy 0, policy_version 8499 (0.0006) [2024-09-30 00:40:27,376][1153805] Updated weights for policy 0, policy_version 8509 (0.0006) [2024-09-30 00:40:27,888][1153805] Updated weights for policy 0, policy_version 8519 (0.0008) [2024-09-30 00:40:28,413][1153805] Updated weights for policy 0, policy_version 8529 (0.0008) [2024-09-30 00:40:28,860][1153805] Updated weights for policy 0, policy_version 8539 (0.0008) [2024-09-30 00:40:29,372][1153805] Updated weights for policy 0, policy_version 8549 (0.0007) [2024-09-30 00:40:29,855][1153805] Updated weights for policy 0, policy_version 8559 (0.0007) [2024-09-30 00:40:30,345][1153805] Updated weights for policy 0, policy_version 8569 (0.0006) [2024-09-30 00:40:30,847][1153805] Updated weights for policy 0, policy_version 8579 (0.0006) [2024-09-30 00:40:30,987][1153456] Fps is (10 sec: 76185.3, 60 sec: 78028.7, 300 sec: 76810.4). Total num frames: 35147776. Throughput: 0: 19307.9. Samples: 7758504. Policy #0 lag: (min: 0.0, avg: 1.7, max: 5.0) [2024-09-30 00:40:30,987][1153456] Avg episode reward: [(0, '37.805')] [2024-09-30 00:40:31,352][1153805] Updated weights for policy 0, policy_version 8589 (0.0006) [2024-09-30 00:40:31,869][1153805] Updated weights for policy 0, policy_version 8599 (0.0006) [2024-09-30 00:40:32,391][1153805] Updated weights for policy 0, policy_version 8609 (0.0006) [2024-09-30 00:40:32,916][1153805] Updated weights for policy 0, policy_version 8619 (0.0006) [2024-09-30 00:40:33,368][1153805] Updated weights for policy 0, policy_version 8629 (0.0006) [2024-09-30 00:40:33,835][1153805] Updated weights for policy 0, policy_version 8639 (0.0006) [2024-09-30 00:40:34,343][1153805] Updated weights for policy 0, policy_version 8649 (0.0006) [2024-09-30 00:40:34,787][1153805] Updated weights for policy 0, policy_version 8659 (0.0006) [2024-09-30 00:40:35,295][1153805] Updated weights for policy 0, policy_version 8669 (0.0006) [2024-09-30 00:40:35,784][1153805] Updated weights for policy 0, policy_version 8679 (0.0006) [2024-09-30 00:40:35,987][1153456] Fps is (10 sec: 81920.0, 60 sec: 78097.0, 300 sec: 76852.1). Total num frames: 35565568. Throughput: 0: 19536.3. Samples: 7882708. Policy #0 lag: (min: 0.0, avg: 1.7, max: 5.0) [2024-09-30 00:40:35,987][1153456] Avg episode reward: [(0, '37.921')] [2024-09-30 00:40:36,285][1153805] Updated weights for policy 0, policy_version 8689 (0.0006) [2024-09-30 00:40:36,803][1153805] Updated weights for policy 0, policy_version 8699 (0.0006) [2024-09-30 00:40:37,307][1153805] Updated weights for policy 0, policy_version 8709 (0.0006) [2024-09-30 00:40:37,815][1153805] Updated weights for policy 0, policy_version 8719 (0.0006) [2024-09-30 00:40:38,319][1153805] Updated weights for policy 0, policy_version 8729 (0.0006) [2024-09-30 00:40:38,805][1153805] Updated weights for policy 0, policy_version 8739 (0.0006) [2024-09-30 00:40:39,292][1153805] Updated weights for policy 0, policy_version 8749 (0.0006) [2024-09-30 00:40:39,790][1153805] Updated weights for policy 0, policy_version 8759 (0.0006) [2024-09-30 00:40:40,269][1153805] Updated weights for policy 0, policy_version 8769 (0.0006) [2024-09-30 00:40:40,776][1153805] Updated weights for policy 0, policy_version 8779 (0.0006) [2024-09-30 00:40:40,987][1153456] Fps is (10 sec: 82739.4, 60 sec: 78165.3, 300 sec: 76866.0). Total num frames: 35975168. Throughput: 0: 19563.6. Samples: 7943652. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 00:40:40,987][1153456] Avg episode reward: [(0, '38.878')] [2024-09-30 00:40:41,258][1153805] Updated weights for policy 0, policy_version 8789 (0.0006) [2024-09-30 00:40:41,711][1153805] Updated weights for policy 0, policy_version 8799 (0.0006) [2024-09-30 00:40:42,190][1153805] Updated weights for policy 0, policy_version 8809 (0.0006) [2024-09-30 00:40:42,661][1153805] Updated weights for policy 0, policy_version 8819 (0.0006) [2024-09-30 00:40:43,121][1153805] Updated weights for policy 0, policy_version 8829 (0.0006) [2024-09-30 00:40:43,576][1153805] Updated weights for policy 0, policy_version 8839 (0.0006) [2024-09-30 00:40:44,070][1153805] Updated weights for policy 0, policy_version 8849 (0.0006) [2024-09-30 00:40:44,561][1153805] Updated weights for policy 0, policy_version 8859 (0.0006) [2024-09-30 00:40:45,043][1153805] Updated weights for policy 0, policy_version 8869 (0.0006) [2024-09-30 00:40:45,518][1153805] Updated weights for policy 0, policy_version 8879 (0.0006) [2024-09-30 00:40:45,985][1153805] Updated weights for policy 0, policy_version 8889 (0.0006) [2024-09-30 00:40:45,987][1153456] Fps is (10 sec: 84377.8, 60 sec: 79052.7, 300 sec: 76949.3). Total num frames: 36409344. Throughput: 0: 19795.6. Samples: 8071932. Policy #0 lag: (min: 0.0, avg: 1.7, max: 4.0) [2024-09-30 00:40:45,987][1153456] Avg episode reward: [(0, '39.409')] [2024-09-30 00:40:46,476][1153805] Updated weights for policy 0, policy_version 8899 (0.0006) [2024-09-30 00:40:46,965][1153805] Updated weights for policy 0, policy_version 8909 (0.0006) [2024-09-30 00:40:47,435][1153805] Updated weights for policy 0, policy_version 8919 (0.0006) [2024-09-30 00:40:47,896][1153805] Updated weights for policy 0, policy_version 8929 (0.0006) [2024-09-30 00:40:48,376][1153805] Updated weights for policy 0, policy_version 8939 (0.0006) [2024-09-30 00:40:48,877][1153805] Updated weights for policy 0, policy_version 8949 (0.0006) [2024-09-30 00:40:49,370][1153805] Updated weights for policy 0, policy_version 8959 (0.0006) [2024-09-30 00:40:49,815][1153805] Updated weights for policy 0, policy_version 8969 (0.0005) [2024-09-30 00:40:50,314][1153805] Updated weights for policy 0, policy_version 8979 (0.0006) [2024-09-30 00:40:50,821][1153805] Updated weights for policy 0, policy_version 8989 (0.0006) [2024-09-30 00:40:50,987][1153456] Fps is (10 sec: 85607.4, 60 sec: 79735.6, 300 sec: 76963.2). Total num frames: 36831232. Throughput: 0: 19828.9. Samples: 8199688. Policy #0 lag: (min: 0.0, avg: 1.7, max: 4.0) [2024-09-30 00:40:50,987][1153456] Avg episode reward: [(0, '40.095')] [2024-09-30 00:40:51,317][1153805] Updated weights for policy 0, policy_version 8999 (0.0006) [2024-09-30 00:40:51,807][1153805] Updated weights for policy 0, policy_version 9009 (0.0006) [2024-09-30 00:40:52,304][1153805] Updated weights for policy 0, policy_version 9019 (0.0006) [2024-09-30 00:40:52,792][1153805] Updated weights for policy 0, policy_version 9029 (0.0006) [2024-09-30 00:40:53,249][1153805] Updated weights for policy 0, policy_version 9039 (0.0006) [2024-09-30 00:40:53,724][1153683] Signal inference workers to stop experience collection... (750 times) [2024-09-30 00:40:53,727][1153805] InferenceWorker_p0-w0: stopping experience collection (750 times) [2024-09-30 00:40:53,733][1153683] Signal inference workers to resume experience collection... (750 times) [2024-09-30 00:40:53,733][1153805] InferenceWorker_p0-w0: resuming experience collection (750 times) [2024-09-30 00:40:53,734][1153805] Updated weights for policy 0, policy_version 9049 (0.0006) [2024-09-30 00:40:54,154][1153805] Updated weights for policy 0, policy_version 9059 (0.0006) [2024-09-30 00:40:54,605][1153805] Updated weights for policy 0, policy_version 9069 (0.0006) [2024-09-30 00:40:55,068][1153805] Updated weights for policy 0, policy_version 9079 (0.0006) [2024-09-30 00:40:55,528][1153805] Updated weights for policy 0, policy_version 9089 (0.0006) [2024-09-30 00:40:55,963][1153805] Updated weights for policy 0, policy_version 9099 (0.0006) [2024-09-30 00:40:55,987][1153456] Fps is (10 sec: 86015.8, 60 sec: 80427.7, 300 sec: 77032.6). Total num frames: 37269504. Throughput: 0: 19831.6. Samples: 8262980. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 00:40:55,987][1153456] Avg episode reward: [(0, '39.394')] [2024-09-30 00:40:56,449][1153805] Updated weights for policy 0, policy_version 9109 (0.0006) [2024-09-30 00:40:56,943][1153805] Updated weights for policy 0, policy_version 9119 (0.0006) [2024-09-30 00:40:57,356][1153805] Updated weights for policy 0, policy_version 9129 (0.0006) [2024-09-30 00:40:57,845][1153805] Updated weights for policy 0, policy_version 9139 (0.0006) [2024-09-30 00:40:58,338][1153805] Updated weights for policy 0, policy_version 9149 (0.0006) [2024-09-30 00:40:58,825][1153805] Updated weights for policy 0, policy_version 9159 (0.0006) [2024-09-30 00:40:59,322][1153805] Updated weights for policy 0, policy_version 9169 (0.0007) [2024-09-30 00:40:59,747][1153805] Updated weights for policy 0, policy_version 9179 (0.0006) [2024-09-30 00:41:00,224][1153805] Updated weights for policy 0, policy_version 9189 (0.0006) [2024-09-30 00:41:00,704][1153805] Updated weights for policy 0, policy_version 9199 (0.0006) [2024-09-30 00:41:00,987][1153456] Fps is (10 sec: 86834.9, 60 sec: 81100.7, 300 sec: 77115.9). Total num frames: 37699584. Throughput: 0: 20213.8. Samples: 8395024. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 00:41:00,987][1153456] Avg episode reward: [(0, '39.348')] [2024-09-30 00:41:01,187][1153805] Updated weights for policy 0, policy_version 9209 (0.0006) [2024-09-30 00:41:01,725][1153805] Updated weights for policy 0, policy_version 9219 (0.0006) [2024-09-30 00:41:02,225][1153805] Updated weights for policy 0, policy_version 9229 (0.0006) [2024-09-30 00:41:02,750][1153805] Updated weights for policy 0, policy_version 9239 (0.0006) [2024-09-30 00:41:03,281][1153805] Updated weights for policy 0, policy_version 9249 (0.0007) [2024-09-30 00:41:03,808][1153805] Updated weights for policy 0, policy_version 9259 (0.0006) [2024-09-30 00:41:04,312][1153805] Updated weights for policy 0, policy_version 9269 (0.0006) [2024-09-30 00:41:04,850][1153805] Updated weights for policy 0, policy_version 9279 (0.0006) [2024-09-30 00:41:05,373][1153805] Updated weights for policy 0, policy_version 9289 (0.0006) [2024-09-30 00:41:05,876][1153805] Updated weights for policy 0, policy_version 9299 (0.0006) [2024-09-30 00:41:05,987][1153456] Fps is (10 sec: 82330.4, 60 sec: 80486.4, 300 sec: 77060.4). Total num frames: 38092800. Throughput: 0: 20583.2. Samples: 8515676. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 00:41:05,987][1153456] Avg episode reward: [(0, '38.735')] [2024-09-30 00:41:06,451][1153805] Updated weights for policy 0, policy_version 9309 (0.0006) [2024-09-30 00:41:06,939][1153805] Updated weights for policy 0, policy_version 9319 (0.0006) [2024-09-30 00:41:07,418][1153805] Updated weights for policy 0, policy_version 9329 (0.0006) [2024-09-30 00:41:07,901][1153805] Updated weights for policy 0, policy_version 9339 (0.0006) [2024-09-30 00:41:08,383][1153805] Updated weights for policy 0, policy_version 9349 (0.0006) [2024-09-30 00:41:08,874][1153805] Updated weights for policy 0, policy_version 9359 (0.0006) [2024-09-30 00:41:09,379][1153805] Updated weights for policy 0, policy_version 9369 (0.0006) [2024-09-30 00:41:09,857][1153805] Updated weights for policy 0, policy_version 9379 (0.0006) [2024-09-30 00:41:10,361][1153805] Updated weights for policy 0, policy_version 9389 (0.0006) [2024-09-30 00:41:10,865][1153805] Updated weights for policy 0, policy_version 9399 (0.0006) [2024-09-30 00:41:10,987][1153456] Fps is (10 sec: 80691.1, 60 sec: 80554.8, 300 sec: 77282.5). Total num frames: 38506496. Throughput: 0: 20753.0. Samples: 8576908. Policy #0 lag: (min: 0.0, avg: 2.6, max: 5.0) [2024-09-30 00:41:10,987][1153456] Avg episode reward: [(0, '40.802')] [2024-09-30 00:41:10,987][1153683] Saving new best policy, reward=40.802! [2024-09-30 00:41:11,400][1153805] Updated weights for policy 0, policy_version 9409 (0.0006) [2024-09-30 00:41:11,983][1153805] Updated weights for policy 0, policy_version 9419 (0.0006) [2024-09-30 00:41:12,567][1153805] Updated weights for policy 0, policy_version 9429 (0.0006) [2024-09-30 00:41:13,159][1153805] Updated weights for policy 0, policy_version 9439 (0.0006) [2024-09-30 00:41:13,691][1153805] Updated weights for policy 0, policy_version 9449 (0.0006) [2024-09-30 00:41:14,257][1153805] Updated weights for policy 0, policy_version 9459 (0.0006) [2024-09-30 00:41:14,794][1153805] Updated weights for policy 0, policy_version 9469 (0.0006) [2024-09-30 00:41:15,351][1153805] Updated weights for policy 0, policy_version 9479 (0.0006) [2024-09-30 00:41:15,890][1153805] Updated weights for policy 0, policy_version 9489 (0.0006) [2024-09-30 00:41:15,987][1153456] Fps is (10 sec: 77823.7, 60 sec: 80486.5, 300 sec: 77227.0). Total num frames: 38871040. Throughput: 0: 20742.6. Samples: 8691920. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 00:41:15,987][1153456] Avg episode reward: [(0, '41.660')] [2024-09-30 00:41:15,990][1153683] Saving new best policy, reward=41.660! [2024-09-30 00:41:16,508][1153805] Updated weights for policy 0, policy_version 9499 (0.0006) [2024-09-30 00:41:17,082][1153805] Updated weights for policy 0, policy_version 9509 (0.0006) [2024-09-30 00:41:17,628][1153805] Updated weights for policy 0, policy_version 9519 (0.0006) [2024-09-30 00:41:18,156][1153805] Updated weights for policy 0, policy_version 9529 (0.0006) [2024-09-30 00:41:18,679][1153805] Updated weights for policy 0, policy_version 9539 (0.0006) [2024-09-30 00:41:19,228][1153805] Updated weights for policy 0, policy_version 9549 (0.0006) [2024-09-30 00:41:19,750][1153805] Updated weights for policy 0, policy_version 9559 (0.0006) [2024-09-30 00:41:20,318][1153805] Updated weights for policy 0, policy_version 9569 (0.0006) [2024-09-30 00:41:20,879][1153805] Updated weights for policy 0, policy_version 9579 (0.0006) [2024-09-30 00:41:20,987][1153456] Fps is (10 sec: 73317.9, 60 sec: 80896.0, 300 sec: 77143.7). Total num frames: 39239680. Throughput: 0: 20449.0. Samples: 8802912. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 00:41:20,987][1153456] Avg episode reward: [(0, '38.028')] [2024-09-30 00:41:21,455][1153805] Updated weights for policy 0, policy_version 9589 (0.0006) [2024-09-30 00:41:22,014][1153805] Updated weights for policy 0, policy_version 9599 (0.0006) [2024-09-30 00:41:22,566][1153805] Updated weights for policy 0, policy_version 9609 (0.0006) [2024-09-30 00:41:23,075][1153805] Updated weights for policy 0, policy_version 9619 (0.0006) [2024-09-30 00:41:23,562][1153805] Updated weights for policy 0, policy_version 9629 (0.0006) [2024-09-30 00:41:24,088][1153805] Updated weights for policy 0, policy_version 9639 (0.0006) [2024-09-30 00:41:24,645][1153805] Updated weights for policy 0, policy_version 9649 (0.0006) [2024-09-30 00:41:25,171][1153805] Updated weights for policy 0, policy_version 9659 (0.0006) [2024-09-30 00:41:25,650][1153805] Updated weights for policy 0, policy_version 9669 (0.0006) [2024-09-30 00:41:25,987][1153456] Fps is (10 sec: 75774.7, 60 sec: 81373.7, 300 sec: 77129.8). Total num frames: 39628800. Throughput: 0: 20358.3. Samples: 8859776. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 00:41:25,987][1153456] Avg episode reward: [(0, '40.390')] [2024-09-30 00:41:26,178][1153805] Updated weights for policy 0, policy_version 9679 (0.0006) [2024-09-30 00:41:26,714][1153805] Updated weights for policy 0, policy_version 9689 (0.0006) [2024-09-30 00:41:27,245][1153805] Updated weights for policy 0, policy_version 9699 (0.0006) [2024-09-30 00:41:27,745][1153805] Updated weights for policy 0, policy_version 9709 (0.0006) [2024-09-30 00:41:28,229][1153805] Updated weights for policy 0, policy_version 9719 (0.0006) [2024-09-30 00:41:28,746][1153805] Updated weights for policy 0, policy_version 9729 (0.0006) [2024-09-30 00:41:29,299][1153805] Updated weights for policy 0, policy_version 9739 (0.0006) [2024-09-30 00:41:29,854][1153805] Updated weights for policy 0, policy_version 9749 (0.0006) [2024-09-30 00:41:30,375][1153805] Updated weights for policy 0, policy_version 9759 (0.0006) [2024-09-30 00:41:30,848][1153683] Stopping Batcher_0... [2024-09-30 00:41:30,848][1153456] Component Batcher_0 stopped! [2024-09-30 00:41:30,848][1153683] Loop batcher_evt_loop terminating... [2024-09-30 00:41:30,859][1153683] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000009768_40009728.pth... [2024-09-30 00:41:30,867][1153805] Weights refcount: 2 0 [2024-09-30 00:41:30,868][1153805] Stopping InferenceWorker_p0-w0... [2024-09-30 00:41:30,868][1153456] Component InferenceWorker_p0-w0 stopped! [2024-09-30 00:41:30,868][1153805] Loop inference_proc0-0_evt_loop terminating... [2024-09-30 00:41:30,909][1153683] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000005284_21643264.pth [2024-09-30 00:41:30,913][1153806] Stopping RolloutWorker_w0... [2024-09-30 00:41:30,913][1153456] Component RolloutWorker_w0 stopped! [2024-09-30 00:41:30,913][1153806] Loop rollout_proc0_evt_loop terminating... [2024-09-30 00:41:30,916][1153456] Component RolloutWorker_w11 stopped! [2024-09-30 00:41:30,916][1153882] Stopping RolloutWorker_w11... [2024-09-30 00:41:30,917][1153882] Loop rollout_proc11_evt_loop terminating... [2024-09-30 00:41:30,917][1153456] Component RolloutWorker_w8 stopped! [2024-09-30 00:41:30,917][1153814] Stopping RolloutWorker_w8... [2024-09-30 00:41:30,917][1153456] Component RolloutWorker_w9 stopped! [2024-09-30 00:41:30,917][1153814] Loop rollout_proc8_evt_loop terminating... [2024-09-30 00:41:30,917][1153815] Stopping RolloutWorker_w9... [2024-09-30 00:41:30,918][1153683] Saving new best policy, reward=41.848! [2024-09-30 00:41:30,918][1153456] Component RolloutWorker_w1 stopped! [2024-09-30 00:41:30,918][1153807] Stopping RolloutWorker_w1... [2024-09-30 00:41:30,918][1153815] Loop rollout_proc9_evt_loop terminating... [2024-09-30 00:41:30,918][1153807] Loop rollout_proc1_evt_loop terminating... [2024-09-30 00:41:30,920][1153456] Component RolloutWorker_w3 stopped! [2024-09-30 00:41:30,920][1153808] Stopping RolloutWorker_w3... [2024-09-30 00:41:30,921][1153456] Component RolloutWorker_w7 stopped! [2024-09-30 00:41:30,921][1153808] Loop rollout_proc3_evt_loop terminating... [2024-09-30 00:41:30,921][1153813] Stopping RolloutWorker_w7... [2024-09-30 00:41:30,921][1153456] Component RolloutWorker_w15 stopped! [2024-09-30 00:41:30,921][1154909] Stopping RolloutWorker_w15... [2024-09-30 00:41:30,921][1153813] Loop rollout_proc7_evt_loop terminating... [2024-09-30 00:41:30,921][1154909] Loop rollout_proc15_evt_loop terminating... [2024-09-30 00:41:30,922][1153456] Component RolloutWorker_w5 stopped! [2024-09-30 00:41:30,922][1153812] Stopping RolloutWorker_w5... [2024-09-30 00:41:30,922][1153812] Loop rollout_proc5_evt_loop terminating... [2024-09-30 00:41:30,923][1153456] Component RolloutWorker_w13 stopped! [2024-09-30 00:41:30,923][1153880] Stopping RolloutWorker_w13... [2024-09-30 00:41:30,923][1153880] Loop rollout_proc13_evt_loop terminating... [2024-09-30 00:41:30,925][1153456] Component RolloutWorker_w2 stopped! [2024-09-30 00:41:30,925][1153809] Stopping RolloutWorker_w2... [2024-09-30 00:41:30,926][1153809] Loop rollout_proc2_evt_loop terminating... [2024-09-30 00:41:30,926][1153456] Component RolloutWorker_w14 stopped! [2024-09-30 00:41:30,926][1153881] Stopping RolloutWorker_w14... [2024-09-30 00:41:30,927][1153881] Loop rollout_proc14_evt_loop terminating... [2024-09-30 00:41:30,927][1153456] Component RolloutWorker_w6 stopped! [2024-09-30 00:41:30,927][1153811] Stopping RolloutWorker_w6... [2024-09-30 00:41:30,927][1153456] Component RolloutWorker_w4 stopped! [2024-09-30 00:41:30,927][1153811] Loop rollout_proc6_evt_loop terminating... [2024-09-30 00:41:30,927][1153810] Stopping RolloutWorker_w4... [2024-09-30 00:41:30,927][1153810] Loop rollout_proc4_evt_loop terminating... [2024-09-30 00:41:30,928][1153456] Component RolloutWorker_w12 stopped! [2024-09-30 00:41:30,928][1153883] Stopping RolloutWorker_w12... [2024-09-30 00:41:30,928][1153883] Loop rollout_proc12_evt_loop terminating... [2024-09-30 00:41:30,929][1153456] Component RolloutWorker_w10 stopped! [2024-09-30 00:41:30,929][1153816] Stopping RolloutWorker_w10... [2024-09-30 00:41:30,930][1153816] Loop rollout_proc10_evt_loop terminating... [2024-09-30 00:41:30,973][1153683] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000009768_40009728.pth... [2024-09-30 00:41:31,094][1153683] Stopping LearnerWorker_p0... [2024-09-30 00:41:31,095][1153683] Loop learner_proc0_evt_loop terminating... [2024-09-30 00:41:31,095][1153456] Component LearnerWorker_p0 stopped! [2024-09-30 00:41:31,095][1153456] Waiting for process learner_proc0 to stop... [2024-09-30 00:41:31,697][1153456] Waiting for process inference_proc0-0 to join... [2024-09-30 00:41:31,697][1153456] Waiting for process rollout_proc0 to join... [2024-09-30 00:41:31,698][1153456] Waiting for process rollout_proc1 to join... [2024-09-30 00:41:31,698][1153456] Waiting for process rollout_proc2 to join... [2024-09-30 00:41:31,698][1153456] Waiting for process rollout_proc3 to join... [2024-09-30 00:41:31,698][1153456] Waiting for process rollout_proc4 to join... [2024-09-30 00:41:31,699][1153456] Waiting for process rollout_proc5 to join... [2024-09-30 00:41:31,699][1153456] Waiting for process rollout_proc6 to join... [2024-09-30 00:41:31,699][1153456] Waiting for process rollout_proc7 to join... [2024-09-30 00:41:31,699][1153456] Waiting for process rollout_proc8 to join... [2024-09-30 00:41:31,700][1153456] Waiting for process rollout_proc9 to join... [2024-09-30 00:41:31,700][1153456] Waiting for process rollout_proc10 to join... [2024-09-30 00:41:31,700][1153456] Waiting for process rollout_proc11 to join... [2024-09-30 00:41:31,700][1153456] Waiting for process rollout_proc12 to join... [2024-09-30 00:41:31,701][1153456] Waiting for process rollout_proc13 to join... [2024-09-30 00:41:31,701][1153456] Waiting for process rollout_proc14 to join... [2024-09-30 00:41:31,701][1153456] Waiting for process rollout_proc15 to join... [2024-09-30 00:41:31,701][1153456] Batcher 0 profile tree view: batching: 77.9109, releasing_batches: 0.2341 [2024-09-30 00:41:31,702][1153456] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0000 wait_policy_total: 14.1223 update_model: 8.9162 weight_update: 0.0006 one_step: 0.0019 handle_policy_step: 425.6982 deserialize: 29.2039, stack: 1.9383, obs_to_device_normalize: 99.8019, forward: 200.3020, send_messages: 26.0623 prepare_outputs: 51.2828 to_cpu: 27.0228 [2024-09-30 00:41:31,702][1153456] Learner 0 profile tree view: misc: 0.0281, prepare_batch: 35.7234 train: 109.3318 epoch_init: 0.0392, minibatch_init: 0.0370, losses_postprocess: 1.8251, kl_divergence: 2.1194, after_optimizer: 1.1084 calculate_losses: 42.6500 losses_init: 0.0257, forward_head: 3.9350, bptt_initial: 19.0626, tail: 3.7535, advantages_returns: 1.0221, losses: 7.0241 bptt: 6.6162 bptt_forward_core: 6.2523 update: 58.9957 clip: 4.9422 [2024-09-30 00:41:31,702][1153456] RolloutWorker_w0 profile tree view: wait_for_trajectories: 0.3443, enqueue_policy_requests: 17.9735, env_step: 350.5113, overhead: 16.8544, complete_rollouts: 0.3431 save_policy_outputs: 26.8913 split_output_tensors: 8.9608 [2024-09-30 00:41:31,702][1153456] RolloutWorker_w15 profile tree view: wait_for_trajectories: 0.3431, enqueue_policy_requests: 17.7994, env_step: 349.1141, overhead: 17.0094, complete_rollouts: 0.3473 save_policy_outputs: 27.3007 split_output_tensors: 9.0405 [2024-09-30 00:41:31,702][1153456] Loop Runner_EvtLoop terminating... [2024-09-30 00:41:31,702][1153456] Runner profile tree view: main_loop: 473.8744 [2024-09-30 00:41:31,703][1153456] Collected {0: 40009728}, FPS: 75977.6 [2024-09-30 00:41:31,893][1153456] Loading existing experiment configuration from /home/luyang/workspace/rl/train_dir/default_experiment/config.json [2024-09-30 00:41:31,894][1153456] Overriding arg 'num_workers' with value 1 passed from command line [2024-09-30 00:41:31,894][1153456] Adding new argument 'no_render'=True that is not in the saved config file! [2024-09-30 00:41:31,894][1153456] Adding new argument 'save_video'=True that is not in the saved config file! [2024-09-30 00:41:31,894][1153456] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2024-09-30 00:41:31,894][1153456] Adding new argument 'video_name'=None that is not in the saved config file! [2024-09-30 00:41:31,894][1153456] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! [2024-09-30 00:41:31,894][1153456] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2024-09-30 00:41:31,894][1153456] Adding new argument 'push_to_hub'=True that is not in the saved config file! [2024-09-30 00:41:31,894][1153456] Adding new argument 'hf_repository'='esperesa/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! [2024-09-30 00:41:31,894][1153456] Adding new argument 'policy_index'=0 that is not in the saved config file! [2024-09-30 00:41:31,894][1153456] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2024-09-30 00:41:31,894][1153456] Adding new argument 'train_script'=None that is not in the saved config file! [2024-09-30 00:41:31,894][1153456] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2024-09-30 00:41:31,894][1153456] Using frameskip 1 and render_action_repeat=4 for evaluation [2024-09-30 00:41:31,913][1153456] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:41:31,914][1153456] RunningMeanStd input shape: (3, 72, 128) [2024-09-30 00:41:31,915][1153456] RunningMeanStd input shape: (1,) [2024-09-30 00:41:31,923][1153456] ConvEncoder: input_channels=3 [2024-09-30 00:41:31,987][1153456] Conv encoder output size: 512 [2024-09-30 00:41:31,987][1153456] Policy head output size: 512 [2024-09-30 00:41:32,135][1153456] Loading state from checkpoint /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000009768_40009728.pth... [2024-09-30 00:41:32,713][1153456] Num frames 100... [2024-09-30 00:41:32,786][1153456] Num frames 200... [2024-09-30 00:41:32,860][1153456] Num frames 300... [2024-09-30 00:41:32,934][1153456] Num frames 400... [2024-09-30 00:41:33,007][1153456] Num frames 500... [2024-09-30 00:41:33,081][1153456] Num frames 600... [2024-09-30 00:41:33,155][1153456] Num frames 700... [2024-09-30 00:41:33,229][1153456] Num frames 800... [2024-09-30 00:41:33,304][1153456] Num frames 900... [2024-09-30 00:41:33,378][1153456] Num frames 1000... [2024-09-30 00:41:33,451][1153456] Num frames 1100... [2024-09-30 00:41:33,525][1153456] Num frames 1200... [2024-09-30 00:41:33,600][1153456] Num frames 1300... [2024-09-30 00:41:33,673][1153456] Num frames 1400... [2024-09-30 00:41:33,748][1153456] Num frames 1500... [2024-09-30 00:41:33,823][1153456] Num frames 1600... [2024-09-30 00:41:33,896][1153456] Num frames 1700... [2024-09-30 00:41:33,970][1153456] Num frames 1800... [2024-09-30 00:41:34,072][1153456] Avg episode rewards: #0: 56.649, true rewards: #0: 18.650 [2024-09-30 00:41:34,072][1153456] Avg episode reward: 56.649, avg true_objective: 18.650 [2024-09-30 00:41:34,099][1153456] Num frames 1900... [2024-09-30 00:41:34,172][1153456] Num frames 2000... [2024-09-30 00:41:34,247][1153456] Num frames 2100... [2024-09-30 00:41:34,322][1153456] Num frames 2200... [2024-09-30 00:41:34,396][1153456] Num frames 2300... [2024-09-30 00:41:34,470][1153456] Num frames 2400... [2024-09-30 00:41:34,545][1153456] Num frames 2500... [2024-09-30 00:41:34,618][1153456] Num frames 2600... [2024-09-30 00:41:34,693][1153456] Num frames 2700... [2024-09-30 00:41:34,768][1153456] Num frames 2800... [2024-09-30 00:41:34,842][1153456] Num frames 2900... [2024-09-30 00:41:34,916][1153456] Num frames 3000... [2024-09-30 00:41:34,991][1153456] Num frames 3100... [2024-09-30 00:41:35,065][1153456] Num frames 3200... [2024-09-30 00:41:35,140][1153456] Num frames 3300... [2024-09-30 00:41:35,214][1153456] Num frames 3400... [2024-09-30 00:41:35,289][1153456] Num frames 3500... [2024-09-30 00:41:35,368][1153456] Num frames 3600... [2024-09-30 00:41:35,457][1153456] Num frames 3700... [2024-09-30 00:41:35,548][1153456] Num frames 3800... [2024-09-30 00:41:35,638][1153456] Num frames 3900... [2024-09-30 00:41:35,752][1153456] Avg episode rewards: #0: 56.824, true rewards: #0: 19.825 [2024-09-30 00:41:35,752][1153456] Avg episode reward: 56.824, avg true_objective: 19.825 [2024-09-30 00:41:35,800][1153456] Num frames 4000... [2024-09-30 00:41:35,893][1153456] Num frames 4100... [2024-09-30 00:41:35,980][1153456] Num frames 4200... [2024-09-30 00:41:36,069][1153456] Num frames 4300... [2024-09-30 00:41:36,158][1153456] Num frames 4400... [2024-09-30 00:41:36,246][1153456] Num frames 4500... [2024-09-30 00:41:36,332][1153456] Num frames 4600... [2024-09-30 00:41:36,422][1153456] Num frames 4700... [2024-09-30 00:41:36,511][1153456] Num frames 4800... [2024-09-30 00:41:36,599][1153456] Num frames 4900... [2024-09-30 00:41:36,687][1153456] Num frames 5000... [2024-09-30 00:41:36,777][1153456] Num frames 5100... [2024-09-30 00:41:36,866][1153456] Num frames 5200... [2024-09-30 00:41:36,954][1153456] Num frames 5300... [2024-09-30 00:41:37,044][1153456] Num frames 5400... [2024-09-30 00:41:37,136][1153456] Num frames 5500... [2024-09-30 00:41:37,226][1153456] Num frames 5600... [2024-09-30 00:41:37,315][1153456] Num frames 5700... [2024-09-30 00:41:37,422][1153456] Avg episode rewards: #0: 51.856, true rewards: #0: 19.190 [2024-09-30 00:41:37,422][1153456] Avg episode reward: 51.856, avg true_objective: 19.190 [2024-09-30 00:41:37,475][1153456] Num frames 5800... [2024-09-30 00:41:37,564][1153456] Num frames 5900... [2024-09-30 00:41:37,653][1153456] Num frames 6000... [2024-09-30 00:41:37,741][1153456] Num frames 6100... [2024-09-30 00:41:37,828][1153456] Num frames 6200... [2024-09-30 00:41:37,919][1153456] Num frames 6300... [2024-09-30 00:41:38,008][1153456] Num frames 6400... [2024-09-30 00:41:38,096][1153456] Num frames 6500... [2024-09-30 00:41:38,186][1153456] Num frames 6600... [2024-09-30 00:41:38,275][1153456] Num frames 6700... [2024-09-30 00:41:38,364][1153456] Num frames 6800... [2024-09-30 00:41:38,453][1153456] Num frames 6900... [2024-09-30 00:41:38,541][1153456] Num frames 7000... [2024-09-30 00:41:38,630][1153456] Num frames 7100... [2024-09-30 00:41:38,718][1153456] Num frames 7200... [2024-09-30 00:41:38,807][1153456] Num frames 7300... [2024-09-30 00:41:38,916][1153456] Avg episode rewards: #0: 49.392, true rewards: #0: 18.393 [2024-09-30 00:41:38,916][1153456] Avg episode reward: 49.392, avg true_objective: 18.393 [2024-09-30 00:41:38,972][1153456] Num frames 7400... [2024-09-30 00:41:39,061][1153456] Num frames 7500... [2024-09-30 00:41:39,149][1153456] Num frames 7600... [2024-09-30 00:41:39,237][1153456] Num frames 7700... [2024-09-30 00:41:39,325][1153456] Num frames 7800... [2024-09-30 00:41:39,413][1153456] Num frames 7900... [2024-09-30 00:41:39,502][1153456] Num frames 8000... [2024-09-30 00:41:39,590][1153456] Num frames 8100... [2024-09-30 00:41:39,678][1153456] Num frames 8200... [2024-09-30 00:41:39,767][1153456] Num frames 8300... [2024-09-30 00:41:39,857][1153456] Num frames 8400... [2024-09-30 00:41:39,946][1153456] Num frames 8500... [2024-09-30 00:41:40,033][1153456] Num frames 8600... [2024-09-30 00:41:40,125][1153456] Num frames 8700... [2024-09-30 00:41:40,217][1153456] Num frames 8800... [2024-09-30 00:41:40,306][1153456] Num frames 8900... [2024-09-30 00:41:40,395][1153456] Num frames 9000... [2024-09-30 00:41:40,485][1153456] Num frames 9100... [2024-09-30 00:41:40,575][1153456] Num frames 9200... [2024-09-30 00:41:40,666][1153456] Num frames 9300... [2024-09-30 00:41:40,755][1153456] Num frames 9400... [2024-09-30 00:41:40,863][1153456] Avg episode rewards: #0: 53.513, true rewards: #0: 18.914 [2024-09-30 00:41:40,863][1153456] Avg episode reward: 53.513, avg true_objective: 18.914 [2024-09-30 00:41:40,926][1153456] Num frames 9500... [2024-09-30 00:41:41,014][1153456] Num frames 9600... [2024-09-30 00:41:41,102][1153456] Num frames 9700... [2024-09-30 00:41:41,191][1153456] Num frames 9800... [2024-09-30 00:41:41,279][1153456] Num frames 9900... [2024-09-30 00:41:41,367][1153456] Num frames 10000... [2024-09-30 00:41:41,456][1153456] Num frames 10100... [2024-09-30 00:41:41,543][1153456] Num frames 10200... [2024-09-30 00:41:41,633][1153456] Num frames 10300... [2024-09-30 00:41:41,723][1153456] Num frames 10400... [2024-09-30 00:41:41,815][1153456] Num frames 10500... [2024-09-30 00:41:41,904][1153456] Num frames 10600... [2024-09-30 00:41:41,993][1153456] Num frames 10700... [2024-09-30 00:41:42,082][1153456] Num frames 10800... [2024-09-30 00:41:42,171][1153456] Num frames 10900... [2024-09-30 00:41:42,262][1153456] Num frames 11000... [2024-09-30 00:41:42,353][1153456] Num frames 11100... [2024-09-30 00:41:42,443][1153456] Num frames 11200... [2024-09-30 00:41:42,533][1153456] Num frames 11300... [2024-09-30 00:41:42,622][1153456] Num frames 11400... [2024-09-30 00:41:42,711][1153456] Num frames 11500... [2024-09-30 00:41:42,818][1153456] Avg episode rewards: #0: 54.261, true rewards: #0: 19.262 [2024-09-30 00:41:42,818][1153456] Avg episode reward: 54.261, avg true_objective: 19.262 [2024-09-30 00:41:42,881][1153456] Num frames 11600... [2024-09-30 00:41:42,972][1153456] Num frames 11700... [2024-09-30 00:41:43,061][1153456] Num frames 11800... [2024-09-30 00:41:43,148][1153456] Num frames 11900... [2024-09-30 00:41:43,238][1153456] Num frames 12000... [2024-09-30 00:41:43,326][1153456] Num frames 12100... [2024-09-30 00:41:43,415][1153456] Num frames 12200... [2024-09-30 00:41:43,503][1153456] Num frames 12300... [2024-09-30 00:41:43,592][1153456] Num frames 12400... [2024-09-30 00:41:43,681][1153456] Num frames 12500... [2024-09-30 00:41:43,769][1153456] Num frames 12600... [2024-09-30 00:41:43,859][1153456] Num frames 12700... [2024-09-30 00:41:43,949][1153456] Num frames 12800... [2024-09-30 00:41:44,039][1153456] Num frames 12900... [2024-09-30 00:41:44,131][1153456] Avg episode rewards: #0: 51.486, true rewards: #0: 18.487 [2024-09-30 00:41:44,131][1153456] Avg episode reward: 51.486, avg true_objective: 18.487 [2024-09-30 00:41:44,201][1153456] Num frames 13000... [2024-09-30 00:41:44,290][1153456] Num frames 13100... [2024-09-30 00:41:44,378][1153456] Num frames 13200... [2024-09-30 00:41:44,466][1153456] Num frames 13300... [2024-09-30 00:41:44,543][1153456] Avg episode rewards: #0: 46.031, true rewards: #0: 16.656 [2024-09-30 00:41:44,543][1153456] Avg episode reward: 46.031, avg true_objective: 16.656 [2024-09-30 00:41:44,626][1153456] Num frames 13400... [2024-09-30 00:41:44,713][1153456] Num frames 13500... [2024-09-30 00:41:44,802][1153456] Num frames 13600... [2024-09-30 00:41:44,890][1153456] Num frames 13700... [2024-09-30 00:41:44,978][1153456] Num frames 13800... [2024-09-30 00:41:45,067][1153456] Num frames 13900... [2024-09-30 00:41:45,156][1153456] Num frames 14000... [2024-09-30 00:41:45,243][1153456] Num frames 14100... [2024-09-30 00:41:45,333][1153456] Num frames 14200... [2024-09-30 00:41:45,423][1153456] Num frames 14300... [2024-09-30 00:41:45,511][1153456] Num frames 14400... [2024-09-30 00:41:45,600][1153456] Num frames 14500... [2024-09-30 00:41:45,689][1153456] Num frames 14600... [2024-09-30 00:41:45,777][1153456] Num frames 14700... [2024-09-30 00:41:45,869][1153456] Num frames 14800... [2024-09-30 00:41:45,957][1153456] Num frames 14900... [2024-09-30 00:41:46,048][1153456] Num frames 15000... [2024-09-30 00:41:46,137][1153456] Num frames 15100... [2024-09-30 00:41:46,226][1153456] Num frames 15200... [2024-09-30 00:41:46,314][1153456] Num frames 15300... [2024-09-30 00:41:46,377][1153456] Avg episode rewards: #0: 47.676, true rewards: #0: 17.010 [2024-09-30 00:41:46,377][1153456] Avg episode reward: 47.676, avg true_objective: 17.010 [2024-09-30 00:41:46,476][1153456] Num frames 15400... [2024-09-30 00:41:46,563][1153456] Num frames 15500... [2024-09-30 00:41:46,651][1153456] Num frames 15600... [2024-09-30 00:41:46,738][1153456] Num frames 15700... [2024-09-30 00:41:46,826][1153456] Num frames 15800... [2024-09-30 00:41:46,915][1153456] Num frames 15900... [2024-09-30 00:41:47,004][1153456] Num frames 16000... [2024-09-30 00:41:47,092][1153456] Num frames 16100... [2024-09-30 00:41:47,182][1153456] Num frames 16200... [2024-09-30 00:41:47,272][1153456] Num frames 16300... [2024-09-30 00:41:47,362][1153456] Num frames 16400... [2024-09-30 00:41:47,451][1153456] Num frames 16500... [2024-09-30 00:41:47,539][1153456] Num frames 16600... [2024-09-30 00:41:47,629][1153456] Num frames 16700... [2024-09-30 00:41:47,720][1153456] Num frames 16800... [2024-09-30 00:41:47,811][1153456] Num frames 16900... [2024-09-30 00:41:47,901][1153456] Num frames 17000... [2024-09-30 00:41:47,989][1153456] Num frames 17100... [2024-09-30 00:41:48,079][1153456] Num frames 17200... [2024-09-30 00:41:48,169][1153456] Num frames 17300... [2024-09-30 00:41:48,259][1153456] Num frames 17400... [2024-09-30 00:41:48,322][1153456] Avg episode rewards: #0: 48.708, true rewards: #0: 17.409 [2024-09-30 00:41:48,322][1153456] Avg episode reward: 48.708, avg true_objective: 17.409 [2024-09-30 00:42:10,996][1153456] Replay video saved to /home/luyang/workspace/rl/train_dir/default_experiment/replay.mp4! [2024-09-30 00:42:51,949][1153456] The model has been pushed to https://huggingface.co/esperesa/rl_course_vizdoom_health_gathering_supreme [2024-09-30 00:44:07,086][1157520] Saving configuration to /home/luyang/workspace/rl/train_dir/default_experiment/config.json... [2024-09-30 00:44:07,091][1157520] Rollout worker 0 uses device cpu [2024-09-30 00:44:07,091][1157520] Rollout worker 1 uses device cpu [2024-09-30 00:44:07,091][1157520] Rollout worker 2 uses device cpu [2024-09-30 00:44:07,091][1157520] Rollout worker 3 uses device cpu [2024-09-30 00:44:07,091][1157520] Rollout worker 4 uses device cpu [2024-09-30 00:44:07,091][1157520] Rollout worker 5 uses device cpu [2024-09-30 00:44:07,091][1157520] Rollout worker 6 uses device cpu [2024-09-30 00:44:07,091][1157520] Rollout worker 7 uses device cpu [2024-09-30 00:44:07,091][1157520] Rollout worker 8 uses device cpu [2024-09-30 00:44:07,092][1157520] Rollout worker 9 uses device cpu [2024-09-30 00:44:07,092][1157520] Rollout worker 10 uses device cpu [2024-09-30 00:44:07,092][1157520] Rollout worker 11 uses device cpu [2024-09-30 00:44:07,092][1157520] Rollout worker 12 uses device cpu [2024-09-30 00:44:07,092][1157520] Rollout worker 13 uses device cpu [2024-09-30 00:44:07,092][1157520] Rollout worker 14 uses device cpu [2024-09-30 00:44:07,092][1157520] Rollout worker 15 uses device cpu [2024-09-30 00:44:07,206][1157520] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-09-30 00:44:07,206][1157520] InferenceWorker_p0-w0: min num requests: 5 [2024-09-30 00:44:07,281][1157520] Starting all processes... [2024-09-30 00:44:07,281][1157520] Starting process learner_proc0 [2024-09-30 00:44:08,870][1157520] Starting all processes... [2024-09-30 00:44:08,874][1157736] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-09-30 00:44:08,874][1157736] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2024-09-30 00:44:08,876][1157520] Starting process inference_proc0-0 [2024-09-30 00:44:08,877][1157520] Starting process rollout_proc0 [2024-09-30 00:44:08,877][1157520] Starting process rollout_proc1 [2024-09-30 00:44:08,877][1157520] Starting process rollout_proc2 [2024-09-30 00:44:08,877][1157520] Starting process rollout_proc3 [2024-09-30 00:44:08,878][1157520] Starting process rollout_proc4 [2024-09-30 00:44:08,881][1157520] Starting process rollout_proc5 [2024-09-30 00:44:08,881][1157520] Starting process rollout_proc6 [2024-09-30 00:44:08,881][1157520] Starting process rollout_proc7 [2024-09-30 00:44:08,884][1157520] Starting process rollout_proc8 [2024-09-30 00:44:08,885][1157520] Starting process rollout_proc9 [2024-09-30 00:44:08,906][1157736] Num visible devices: 1 [2024-09-30 00:44:08,887][1157520] Starting process rollout_proc10 [2024-09-30 00:44:08,889][1157520] Starting process rollout_proc11 [2024-09-30 00:44:08,911][1157736] Starting seed is not provided [2024-09-30 00:44:08,912][1157736] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-09-30 00:44:08,912][1157736] Initializing actor-critic model on device cuda:0 [2024-09-30 00:44:08,889][1157520] Starting process rollout_proc12 [2024-09-30 00:44:08,912][1157736] RunningMeanStd input shape: (3, 72, 128) [2024-09-30 00:44:08,913][1157736] RunningMeanStd input shape: (1,) [2024-09-30 00:44:08,892][1157520] Starting process rollout_proc13 [2024-09-30 00:44:08,892][1157520] Starting process rollout_proc14 [2024-09-30 00:44:08,921][1157736] ConvEncoder: input_channels=3 [2024-09-30 00:44:08,994][1157736] Conv encoder output size: 512 [2024-09-30 00:44:08,994][1157736] Policy head output size: 512 [2024-09-30 00:44:09,005][1157736] Created Actor Critic model with architecture: [2024-09-30 00:44:09,006][1157736] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): VizdoomEncoder( (basic_encoder): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ELU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ELU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ELU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ELU) ) ) ) ) (core): ModelCoreRNN( (core): GRU(512, 512) ) (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=5, bias=True) ) ) [2024-09-30 00:44:09,145][1157736] Using optimizer [2024-09-30 00:44:09,831][1157736] Loading state from checkpoint /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000009768_40009728.pth... [2024-09-30 00:44:09,852][1157736] Loading model from checkpoint [2024-09-30 00:44:09,853][1157736] Loaded experiment state at self.train_step=9768, self.env_steps=40009728 [2024-09-30 00:44:09,853][1157736] Initialized policy 0 weights for model version 9768 [2024-09-30 00:44:09,855][1157736] LearnerWorker_p0 finished initialization! [2024-09-30 00:44:09,856][1157736] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-09-30 00:44:10,465][1157520] Starting process rollout_proc15 [2024-09-30 00:44:10,469][1157823] Worker 3 uses CPU cores [18, 19, 20, 21, 22, 23] [2024-09-30 00:44:10,486][1157822] Worker 2 uses CPU cores [12, 13, 14, 15, 16, 17] [2024-09-30 00:44:10,498][1157894] Worker 11 uses CPU cores [66, 67, 68, 69, 70, 71] [2024-09-30 00:44:10,519][1157826] Worker 5 uses CPU cores [30, 31, 32, 33, 34, 35] [2024-09-30 00:44:10,531][1157821] Worker 0 uses CPU cores [0, 1, 2, 3, 4, 5] [2024-09-30 00:44:10,532][1157819] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-09-30 00:44:10,533][1157819] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 [2024-09-30 00:44:10,536][1157829] Worker 8 uses CPU cores [48, 49, 50, 51, 52, 53] [2024-09-30 00:44:10,540][1157824] Worker 6 uses CPU cores [36, 37, 38, 39, 40, 41] [2024-09-30 00:44:10,547][1157896] Worker 14 uses CPU cores [84, 85, 86, 87, 88, 89] [2024-09-30 00:44:10,547][1157897] Worker 12 uses CPU cores [72, 73, 74, 75, 76, 77] [2024-09-30 00:44:10,554][1157895] Worker 13 uses CPU cores [78, 79, 80, 81, 82, 83] [2024-09-30 00:44:10,555][1157825] Worker 4 uses CPU cores [24, 25, 26, 27, 28, 29] [2024-09-30 00:44:10,557][1157862] Worker 10 uses CPU cores [60, 61, 62, 63, 64, 65] [2024-09-30 00:44:10,565][1157819] Num visible devices: 1 [2024-09-30 00:44:10,566][1157828] Worker 9 uses CPU cores [54, 55, 56, 57, 58, 59] [2024-09-30 00:44:10,575][1157820] Worker 1 uses CPU cores [6, 7, 8, 9, 10, 11] [2024-09-30 00:44:10,580][1157827] Worker 7 uses CPU cores [42, 43, 44, 45, 46, 47] [2024-09-30 00:44:10,674][1157819] RunningMeanStd input shape: (3, 72, 128) [2024-09-30 00:44:10,674][1157819] RunningMeanStd input shape: (1,) [2024-09-30 00:44:10,683][1157819] ConvEncoder: input_channels=3 [2024-09-30 00:44:10,754][1157819] Conv encoder output size: 512 [2024-09-30 00:44:10,754][1157819] Policy head output size: 512 [2024-09-30 00:44:11,891][1157520] Inference worker 0-0 is ready! [2024-09-30 00:44:11,891][1157520] All inference workers are ready! Signal rollout workers to start! [2024-09-30 00:44:11,891][1157520] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 40009728. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2024-09-30 00:44:11,893][1158936] Worker 15 uses CPU cores [90, 91, 92, 93, 94, 95] [2024-09-30 00:44:11,916][1157894] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:44:11,917][1157822] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:44:11,917][1157821] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:44:11,917][1157827] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:44:11,917][1157896] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:44:11,918][1157826] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:44:11,921][1157823] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:44:11,922][1157862] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:44:11,922][1157829] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:44:11,922][1157824] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:44:11,923][1157895] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:44:11,947][1157828] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:44:11,947][1157820] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:44:11,947][1157897] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:44:11,948][1157825] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:44:12,004][1158936] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 00:44:12,195][1157894] Decorrelating experience for 0 frames... [2024-09-30 00:44:12,196][1157821] Decorrelating experience for 0 frames... [2024-09-30 00:44:12,196][1157822] Decorrelating experience for 0 frames... [2024-09-30 00:44:12,197][1157826] Decorrelating experience for 0 frames... [2024-09-30 00:44:12,198][1157829] Decorrelating experience for 0 frames... [2024-09-30 00:44:12,201][1157895] Decorrelating experience for 0 frames... [2024-09-30 00:44:12,208][1157827] Decorrelating experience for 0 frames... [2024-09-30 00:44:12,223][1157820] Decorrelating experience for 0 frames... [2024-09-30 00:44:12,225][1157825] Decorrelating experience for 0 frames... [2024-09-30 00:44:12,264][1158936] Decorrelating experience for 0 frames... [2024-09-30 00:44:12,410][1157829] Decorrelating experience for 32 frames... [2024-09-30 00:44:12,422][1157827] Decorrelating experience for 32 frames... [2024-09-30 00:44:12,422][1157828] Decorrelating experience for 0 frames... [2024-09-30 00:44:12,432][1157820] Decorrelating experience for 32 frames... [2024-09-30 00:44:12,443][1157896] Decorrelating experience for 0 frames... [2024-09-30 00:44:12,456][1157822] Decorrelating experience for 32 frames... [2024-09-30 00:44:12,482][1157825] Decorrelating experience for 32 frames... [2024-09-30 00:44:12,650][1157824] Decorrelating experience for 0 frames... [2024-09-30 00:44:12,654][1157820] Decorrelating experience for 64 frames... [2024-09-30 00:44:12,672][1157897] Decorrelating experience for 0 frames... [2024-09-30 00:44:12,679][1158936] Decorrelating experience for 32 frames... [2024-09-30 00:44:12,682][1157828] Decorrelating experience for 32 frames... [2024-09-30 00:44:12,704][1157894] Decorrelating experience for 32 frames... [2024-09-30 00:44:12,858][1157824] Decorrelating experience for 32 frames... [2024-09-30 00:44:12,885][1157897] Decorrelating experience for 32 frames... [2024-09-30 00:44:12,896][1157827] Decorrelating experience for 64 frames... [2024-09-30 00:44:12,899][1158936] Decorrelating experience for 64 frames... [2024-09-30 00:44:12,932][1157829] Decorrelating experience for 64 frames... [2024-09-30 00:44:12,936][1157820] Decorrelating experience for 96 frames... [2024-09-30 00:44:13,067][1157895] Decorrelating experience for 32 frames... [2024-09-30 00:44:13,080][1157824] Decorrelating experience for 64 frames... [2024-09-30 00:44:13,125][1157894] Decorrelating experience for 64 frames... [2024-09-30 00:44:13,151][1157862] Decorrelating experience for 0 frames... [2024-09-30 00:44:13,159][1157821] Decorrelating experience for 32 frames... [2024-09-30 00:44:13,243][1157820] Decorrelating experience for 128 frames... [2024-09-30 00:44:13,283][1157822] Decorrelating experience for 64 frames... [2024-09-30 00:44:13,289][1157895] Decorrelating experience for 64 frames... [2024-09-30 00:44:13,336][1157827] Decorrelating experience for 96 frames... [2024-09-30 00:44:13,362][1157894] Decorrelating experience for 96 frames... [2024-09-30 00:44:13,362][1157862] Decorrelating experience for 32 frames... [2024-09-30 00:44:13,381][1157821] Decorrelating experience for 64 frames... [2024-09-30 00:44:13,413][1157824] Decorrelating experience for 96 frames... [2024-09-30 00:44:13,508][1157828] Decorrelating experience for 64 frames... [2024-09-30 00:44:13,541][1158936] Decorrelating experience for 96 frames... [2024-09-30 00:44:13,568][1157822] Decorrelating experience for 96 frames... [2024-09-30 00:44:13,573][1157895] Decorrelating experience for 96 frames... [2024-09-30 00:44:13,582][1157862] Decorrelating experience for 64 frames... [2024-09-30 00:44:13,605][1157820] Decorrelating experience for 160 frames... [2024-09-30 00:44:13,668][1157894] Decorrelating experience for 128 frames... [2024-09-30 00:44:13,669][1157826] Decorrelating experience for 32 frames... [2024-09-30 00:44:13,746][1157827] Decorrelating experience for 128 frames... [2024-09-30 00:44:13,798][1157828] Decorrelating experience for 96 frames... [2024-09-30 00:44:13,814][1157824] Decorrelating experience for 128 frames... [2024-09-30 00:44:13,847][1158936] Decorrelating experience for 128 frames... [2024-09-30 00:44:13,877][1157895] Decorrelating experience for 128 frames... [2024-09-30 00:44:13,888][1157820] Decorrelating experience for 192 frames... [2024-09-30 00:44:13,961][1157897] Decorrelating experience for 64 frames... [2024-09-30 00:44:14,005][1157827] Decorrelating experience for 160 frames... [2024-09-30 00:44:14,060][1157823] Decorrelating experience for 0 frames... [2024-09-30 00:44:14,072][1157822] Decorrelating experience for 128 frames... [2024-09-30 00:44:14,081][1157824] Decorrelating experience for 160 frames... [2024-09-30 00:44:14,096][1157826] Decorrelating experience for 64 frames... [2024-09-30 00:44:14,105][1158936] Decorrelating experience for 160 frames... [2024-09-30 00:44:14,123][1157894] Decorrelating experience for 160 frames... [2024-09-30 00:44:14,190][1157897] Decorrelating experience for 96 frames... [2024-09-30 00:44:14,214][1157896] Decorrelating experience for 32 frames... [2024-09-30 00:44:14,276][1157862] Decorrelating experience for 96 frames... [2024-09-30 00:44:14,284][1157895] Decorrelating experience for 160 frames... [2024-09-30 00:44:14,332][1157827] Decorrelating experience for 192 frames... [2024-09-30 00:44:14,339][1157822] Decorrelating experience for 160 frames... [2024-09-30 00:44:14,355][1157824] Decorrelating experience for 192 frames... [2024-09-30 00:44:14,370][1157820] Decorrelating experience for 224 frames... [2024-09-30 00:44:14,396][1157894] Decorrelating experience for 192 frames... [2024-09-30 00:44:14,467][1157825] Decorrelating experience for 64 frames... [2024-09-30 00:44:14,509][1157828] Decorrelating experience for 128 frames... [2024-09-30 00:44:14,544][1157823] Decorrelating experience for 32 frames... [2024-09-30 00:44:14,578][1158936] Decorrelating experience for 192 frames... [2024-09-30 00:44:14,583][1157826] Decorrelating experience for 96 frames... [2024-09-30 00:44:14,604][1157897] Decorrelating experience for 128 frames... [2024-09-30 00:44:14,610][1157822] Decorrelating experience for 192 frames... [2024-09-30 00:44:14,635][1157824] Decorrelating experience for 224 frames... [2024-09-30 00:44:14,725][1157821] Decorrelating experience for 96 frames... [2024-09-30 00:44:14,725][1157827] Decorrelating experience for 224 frames... [2024-09-30 00:44:14,767][1157823] Decorrelating experience for 64 frames... [2024-09-30 00:44:14,789][1157896] Decorrelating experience for 64 frames... [2024-09-30 00:44:14,854][1157825] Decorrelating experience for 96 frames... [2024-09-30 00:44:14,892][1157826] Decorrelating experience for 128 frames... [2024-09-30 00:44:14,894][1157822] Decorrelating experience for 224 frames... [2024-09-30 00:44:14,952][1157895] Decorrelating experience for 192 frames... [2024-09-30 00:44:15,004][1157823] Decorrelating experience for 96 frames... [2024-09-30 00:44:15,017][1157897] Decorrelating experience for 160 frames... [2024-09-30 00:44:15,024][1158936] Decorrelating experience for 224 frames... [2024-09-30 00:44:15,030][1157896] Decorrelating experience for 96 frames... [2024-09-30 00:44:15,074][1157828] Decorrelating experience for 160 frames... [2024-09-30 00:44:15,181][1157829] Decorrelating experience for 96 frames... [2024-09-30 00:44:15,240][1157895] Decorrelating experience for 224 frames... [2024-09-30 00:44:15,252][1157826] Decorrelating experience for 160 frames... [2024-09-30 00:44:15,262][1157825] Decorrelating experience for 128 frames... [2024-09-30 00:44:15,281][1157821] Decorrelating experience for 128 frames... [2024-09-30 00:44:15,286][1157897] Decorrelating experience for 192 frames... [2024-09-30 00:44:15,310][1157894] Decorrelating experience for 224 frames... [2024-09-30 00:44:15,343][1157828] Decorrelating experience for 192 frames... [2024-09-30 00:44:15,358][1157736] Signal inference workers to stop experience collection... [2024-09-30 00:44:15,360][1157819] InferenceWorker_p0-w0: stopping experience collection [2024-09-30 00:44:15,442][1157823] Decorrelating experience for 128 frames... [2024-09-30 00:44:15,466][1157520] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 40009728. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2024-09-30 00:44:15,466][1157520] Avg episode reward: [(0, '1.513')] [2024-09-30 00:44:15,527][1157825] Decorrelating experience for 160 frames... [2024-09-30 00:44:15,535][1157826] Decorrelating experience for 192 frames... [2024-09-30 00:44:15,547][1157821] Decorrelating experience for 160 frames... [2024-09-30 00:44:15,547][1157896] Decorrelating experience for 128 frames... [2024-09-30 00:44:15,575][1157897] Decorrelating experience for 224 frames... [2024-09-30 00:44:15,623][1157828] Decorrelating experience for 224 frames... [2024-09-30 00:44:15,691][1157862] Decorrelating experience for 128 frames... [2024-09-30 00:44:15,800][1157825] Decorrelating experience for 192 frames... [2024-09-30 00:44:15,800][1157829] Decorrelating experience for 128 frames... [2024-09-30 00:44:15,802][1157823] Decorrelating experience for 160 frames... [2024-09-30 00:44:15,861][1157821] Decorrelating experience for 192 frames... [2024-09-30 00:44:15,920][1157826] Decorrelating experience for 224 frames... [2024-09-30 00:44:16,066][1157829] Decorrelating experience for 160 frames... [2024-09-30 00:44:16,066][1157896] Decorrelating experience for 160 frames... [2024-09-30 00:44:16,074][1157823] Decorrelating experience for 192 frames... [2024-09-30 00:44:16,085][1157825] Decorrelating experience for 224 frames... [2024-09-30 00:44:16,150][1157862] Decorrelating experience for 160 frames... [2024-09-30 00:44:16,287][1157821] Decorrelating experience for 224 frames... [2024-09-30 00:44:16,334][1157896] Decorrelating experience for 192 frames... [2024-09-30 00:44:16,351][1157823] Decorrelating experience for 224 frames... [2024-09-30 00:44:16,380][1157829] Decorrelating experience for 192 frames... [2024-09-30 00:44:16,411][1157862] Decorrelating experience for 192 frames... [2024-09-30 00:44:16,436][1157736] Signal inference workers to resume experience collection... [2024-09-30 00:44:16,436][1157819] InferenceWorker_p0-w0: resuming experience collection [2024-09-30 00:44:16,610][1157896] Decorrelating experience for 224 frames... [2024-09-30 00:44:16,655][1157829] Decorrelating experience for 224 frames... [2024-09-30 00:44:16,685][1157862] Decorrelating experience for 224 frames... [2024-09-30 00:44:17,335][1157819] Updated weights for policy 0, policy_version 9778 (0.0130) [2024-09-30 00:44:17,963][1157819] Updated weights for policy 0, policy_version 9788 (0.0007) [2024-09-30 00:44:18,515][1157819] Updated weights for policy 0, policy_version 9798 (0.0007) [2024-09-30 00:44:19,065][1157819] Updated weights for policy 0, policy_version 9808 (0.0006) [2024-09-30 00:44:19,615][1157819] Updated weights for policy 0, policy_version 9818 (0.0006) [2024-09-30 00:44:20,138][1157819] Updated weights for policy 0, policy_version 9828 (0.0006) [2024-09-30 00:44:20,466][1157520] Fps is (10 sec: 31528.9, 60 sec: 31528.9, 300 sec: 31528.9). Total num frames: 40280064. Throughput: 0: 6638.5. Samples: 56920. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2024-09-30 00:44:20,466][1157520] Avg episode reward: [(0, '36.243')] [2024-09-30 00:44:20,681][1157819] Updated weights for policy 0, policy_version 9838 (0.0006) [2024-09-30 00:44:21,217][1157819] Updated weights for policy 0, policy_version 9848 (0.0006) [2024-09-30 00:44:21,725][1157819] Updated weights for policy 0, policy_version 9858 (0.0006) [2024-09-30 00:44:22,285][1157819] Updated weights for policy 0, policy_version 9868 (0.0006) [2024-09-30 00:44:22,826][1157819] Updated weights for policy 0, policy_version 9878 (0.0006) [2024-09-30 00:44:23,319][1157819] Updated weights for policy 0, policy_version 9888 (0.0006) [2024-09-30 00:44:23,841][1157819] Updated weights for policy 0, policy_version 9898 (0.0006) [2024-09-30 00:44:24,466][1157819] Updated weights for policy 0, policy_version 9908 (0.0006) [2024-09-30 00:44:25,018][1157819] Updated weights for policy 0, policy_version 9918 (0.0006) [2024-09-30 00:44:25,466][1157520] Fps is (10 sec: 64306.2, 60 sec: 47373.8, 300 sec: 47373.8). Total num frames: 40652800. Throughput: 0: 8464.7. Samples: 114904. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 00:44:25,466][1157520] Avg episode reward: [(0, '40.967')] [2024-09-30 00:44:25,588][1157819] Updated weights for policy 0, policy_version 9928 (0.0006) [2024-09-30 00:44:26,220][1157819] Updated weights for policy 0, policy_version 9938 (0.0006) [2024-09-30 00:44:26,733][1157819] Updated weights for policy 0, policy_version 9948 (0.0006) [2024-09-30 00:44:27,197][1157520] Heartbeat connected on Batcher_0 [2024-09-30 00:44:27,208][1157520] Heartbeat connected on InferenceWorker_p0-w0 [2024-09-30 00:44:27,211][1157520] Heartbeat connected on LearnerWorker_p0 [2024-09-30 00:44:27,217][1157520] Heartbeat connected on RolloutWorker_w0 [2024-09-30 00:44:27,218][1157520] Heartbeat connected on RolloutWorker_w1 [2024-09-30 00:44:27,222][1157520] Heartbeat connected on RolloutWorker_w2 [2024-09-30 00:44:27,227][1157520] Heartbeat connected on RolloutWorker_w3 [2024-09-30 00:44:27,232][1157520] Heartbeat connected on RolloutWorker_w4 [2024-09-30 00:44:27,237][1157520] Heartbeat connected on RolloutWorker_w5 [2024-09-30 00:44:27,239][1157520] Heartbeat connected on RolloutWorker_w6 [2024-09-30 00:44:27,240][1157520] Heartbeat connected on RolloutWorker_w7 [2024-09-30 00:44:27,240][1157819] Updated weights for policy 0, policy_version 9958 (0.0006) [2024-09-30 00:44:27,248][1157520] Heartbeat connected on RolloutWorker_w9 [2024-09-30 00:44:27,250][1157520] Heartbeat connected on RolloutWorker_w8 [2024-09-30 00:44:27,253][1157520] Heartbeat connected on RolloutWorker_w10 [2024-09-30 00:44:27,261][1157520] Heartbeat connected on RolloutWorker_w11 [2024-09-30 00:44:27,265][1157520] Heartbeat connected on RolloutWorker_w12 [2024-09-30 00:44:27,270][1157520] Heartbeat connected on RolloutWorker_w13 [2024-09-30 00:44:27,283][1157520] Heartbeat connected on RolloutWorker_w14 [2024-09-30 00:44:27,284][1157520] Heartbeat connected on RolloutWorker_w15 [2024-09-30 00:44:27,770][1157819] Updated weights for policy 0, policy_version 9968 (0.0006) [2024-09-30 00:44:28,285][1157819] Updated weights for policy 0, policy_version 9978 (0.0006) [2024-09-30 00:44:28,787][1157819] Updated weights for policy 0, policy_version 9988 (0.0006) [2024-09-30 00:44:29,296][1157819] Updated weights for policy 0, policy_version 9998 (0.0006) [2024-09-30 00:44:29,830][1157819] Updated weights for policy 0, policy_version 10008 (0.0006) [2024-09-30 00:44:30,335][1157819] Updated weights for policy 0, policy_version 10018 (0.0006) [2024-09-30 00:44:30,466][1157520] Fps is (10 sec: 76185.9, 60 sec: 55571.3, 300 sec: 55571.3). Total num frames: 41041920. Throughput: 0: 12253.4. Samples: 227596. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 00:44:30,466][1157520] Avg episode reward: [(0, '40.298')] [2024-09-30 00:44:30,879][1157819] Updated weights for policy 0, policy_version 10028 (0.0006) [2024-09-30 00:44:31,388][1157819] Updated weights for policy 0, policy_version 10038 (0.0006) [2024-09-30 00:44:31,894][1157819] Updated weights for policy 0, policy_version 10048 (0.0006) [2024-09-30 00:44:32,391][1157819] Updated weights for policy 0, policy_version 10058 (0.0006) [2024-09-30 00:44:32,889][1157819] Updated weights for policy 0, policy_version 10068 (0.0007) [2024-09-30 00:44:33,387][1157819] Updated weights for policy 0, policy_version 10078 (0.0007) [2024-09-30 00:44:33,889][1157819] Updated weights for policy 0, policy_version 10088 (0.0006) [2024-09-30 00:44:34,393][1157819] Updated weights for policy 0, policy_version 10098 (0.0006) [2024-09-30 00:44:34,908][1157819] Updated weights for policy 0, policy_version 10108 (0.0006) [2024-09-30 00:44:35,392][1157819] Updated weights for policy 0, policy_version 10118 (0.0006) [2024-09-30 00:44:35,466][1157520] Fps is (10 sec: 79464.0, 60 sec: 60985.9, 300 sec: 60985.9). Total num frames: 41447424. Throughput: 0: 14772.7. Samples: 348256. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 00:44:35,466][1157520] Avg episode reward: [(0, '41.050')] [2024-09-30 00:44:35,892][1157819] Updated weights for policy 0, policy_version 10128 (0.0006) [2024-09-30 00:44:36,385][1157819] Updated weights for policy 0, policy_version 10138 (0.0007) [2024-09-30 00:44:36,834][1157819] Updated weights for policy 0, policy_version 10148 (0.0006) [2024-09-30 00:44:37,315][1157819] Updated weights for policy 0, policy_version 10158 (0.0006) [2024-09-30 00:44:37,809][1157819] Updated weights for policy 0, policy_version 10168 (0.0006) [2024-09-30 00:44:38,304][1157819] Updated weights for policy 0, policy_version 10178 (0.0006) [2024-09-30 00:44:38,783][1157819] Updated weights for policy 0, policy_version 10188 (0.0006) [2024-09-30 00:44:39,276][1157819] Updated weights for policy 0, policy_version 10198 (0.0006) [2024-09-30 00:44:39,814][1157819] Updated weights for policy 0, policy_version 10208 (0.0006) [2024-09-30 00:44:40,352][1157819] Updated weights for policy 0, policy_version 10218 (0.0006) [2024-09-30 00:44:40,466][1157520] Fps is (10 sec: 81918.1, 60 sec: 64792.0, 300 sec: 64792.0). Total num frames: 41861120. Throughput: 0: 14394.6. Samples: 411316. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 00:44:40,466][1157520] Avg episode reward: [(0, '43.133')] [2024-09-30 00:44:40,467][1157736] Saving new best policy, reward=43.133! [2024-09-30 00:44:40,935][1157819] Updated weights for policy 0, policy_version 10228 (0.0006) [2024-09-30 00:44:41,513][1157819] Updated weights for policy 0, policy_version 10238 (0.0006) [2024-09-30 00:44:42,048][1157819] Updated weights for policy 0, policy_version 10248 (0.0006) [2024-09-30 00:44:42,586][1157819] Updated weights for policy 0, policy_version 10258 (0.0006) [2024-09-30 00:44:43,135][1157819] Updated weights for policy 0, policy_version 10268 (0.0006) [2024-09-30 00:44:43,652][1157819] Updated weights for policy 0, policy_version 10278 (0.0006) [2024-09-30 00:44:44,186][1157819] Updated weights for policy 0, policy_version 10288 (0.0006) [2024-09-30 00:44:44,759][1157819] Updated weights for policy 0, policy_version 10298 (0.0006) [2024-09-30 00:44:45,282][1157819] Updated weights for policy 0, policy_version 10308 (0.0006) [2024-09-30 00:44:45,466][1157520] Fps is (10 sec: 78643.2, 60 sec: 66245.1, 300 sec: 66245.1). Total num frames: 42233856. Throughput: 0: 15699.7. Samples: 527104. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 00:44:45,466][1157520] Avg episode reward: [(0, '40.395')] [2024-09-30 00:44:45,808][1157819] Updated weights for policy 0, policy_version 10318 (0.0006) [2024-09-30 00:44:46,334][1157819] Updated weights for policy 0, policy_version 10328 (0.0006) [2024-09-30 00:44:46,822][1157819] Updated weights for policy 0, policy_version 10338 (0.0006) [2024-09-30 00:44:47,329][1157819] Updated weights for policy 0, policy_version 10348 (0.0006) [2024-09-30 00:44:47,841][1157819] Updated weights for policy 0, policy_version 10358 (0.0006) [2024-09-30 00:44:48,333][1157819] Updated weights for policy 0, policy_version 10368 (0.0006) [2024-09-30 00:44:48,840][1157819] Updated weights for policy 0, policy_version 10378 (0.0006) [2024-09-30 00:44:49,391][1157819] Updated weights for policy 0, policy_version 10388 (0.0006) [2024-09-30 00:44:49,903][1157819] Updated weights for policy 0, policy_version 10398 (0.0006) [2024-09-30 00:44:50,397][1157819] Updated weights for policy 0, policy_version 10408 (0.0006) [2024-09-30 00:44:50,466][1157520] Fps is (10 sec: 77416.1, 60 sec: 68064.6, 300 sec: 68064.6). Total num frames: 42635264. Throughput: 0: 16728.7. Samples: 645296. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 00:44:50,466][1157520] Avg episode reward: [(0, '44.196')] [2024-09-30 00:44:50,466][1157736] Saving new best policy, reward=44.196! [2024-09-30 00:44:50,915][1157819] Updated weights for policy 0, policy_version 10418 (0.0006) [2024-09-30 00:44:51,430][1157819] Updated weights for policy 0, policy_version 10428 (0.0006) [2024-09-30 00:44:51,922][1157819] Updated weights for policy 0, policy_version 10438 (0.0006) [2024-09-30 00:44:52,460][1157819] Updated weights for policy 0, policy_version 10448 (0.0006) [2024-09-30 00:44:52,969][1157819] Updated weights for policy 0, policy_version 10458 (0.0006) [2024-09-30 00:44:53,482][1157819] Updated weights for policy 0, policy_version 10468 (0.0006) [2024-09-30 00:44:53,983][1157819] Updated weights for policy 0, policy_version 10478 (0.0006) [2024-09-30 00:44:54,484][1157819] Updated weights for policy 0, policy_version 10488 (0.0006) [2024-09-30 00:44:54,996][1157819] Updated weights for policy 0, policy_version 10498 (0.0006) [2024-09-30 00:44:55,466][1157520] Fps is (10 sec: 80281.9, 60 sec: 69466.4, 300 sec: 69466.4). Total num frames: 43036672. Throughput: 0: 16188.4. Samples: 705396. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 00:44:55,466][1157520] Avg episode reward: [(0, '40.444')] [2024-09-30 00:44:55,527][1157819] Updated weights for policy 0, policy_version 10508 (0.0006) [2024-09-30 00:44:56,041][1157819] Updated weights for policy 0, policy_version 10518 (0.0006) [2024-09-30 00:44:56,553][1157819] Updated weights for policy 0, policy_version 10528 (0.0006) [2024-09-30 00:44:57,062][1157819] Updated weights for policy 0, policy_version 10538 (0.0006) [2024-09-30 00:44:57,555][1157819] Updated weights for policy 0, policy_version 10548 (0.0006) [2024-09-30 00:44:58,060][1157819] Updated weights for policy 0, policy_version 10558 (0.0006) [2024-09-30 00:44:58,190][1157736] Signal inference workers to stop experience collection... (50 times) [2024-09-30 00:44:58,194][1157736] Signal inference workers to resume experience collection... (50 times) [2024-09-30 00:44:58,197][1157819] InferenceWorker_p0-w0: stopping experience collection (50 times) [2024-09-30 00:44:58,199][1157819] InferenceWorker_p0-w0: resuming experience collection (50 times) [2024-09-30 00:44:58,558][1157819] Updated weights for policy 0, policy_version 10568 (0.0006) [2024-09-30 00:44:59,090][1157819] Updated weights for policy 0, policy_version 10578 (0.0006) [2024-09-30 00:44:59,603][1157819] Updated weights for policy 0, policy_version 10588 (0.0006) [2024-09-30 00:45:00,115][1157819] Updated weights for policy 0, policy_version 10598 (0.0006) [2024-09-30 00:45:00,466][1157520] Fps is (10 sec: 79871.0, 60 sec: 70495.2, 300 sec: 70495.2). Total num frames: 43433984. Throughput: 0: 18352.0. Samples: 825840. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 00:45:00,466][1157520] Avg episode reward: [(0, '40.224')] [2024-09-30 00:45:00,646][1157819] Updated weights for policy 0, policy_version 10608 (0.0006) [2024-09-30 00:45:01,201][1157819] Updated weights for policy 0, policy_version 10618 (0.0006) [2024-09-30 00:45:01,735][1157819] Updated weights for policy 0, policy_version 10628 (0.0006) [2024-09-30 00:45:02,286][1157819] Updated weights for policy 0, policy_version 10638 (0.0006) [2024-09-30 00:45:02,837][1157819] Updated weights for policy 0, policy_version 10648 (0.0006) [2024-09-30 00:45:03,371][1157819] Updated weights for policy 0, policy_version 10658 (0.0006) [2024-09-30 00:45:03,896][1157819] Updated weights for policy 0, policy_version 10668 (0.0006) [2024-09-30 00:45:04,468][1157819] Updated weights for policy 0, policy_version 10678 (0.0006) [2024-09-30 00:45:05,018][1157819] Updated weights for policy 0, policy_version 10688 (0.0006) [2024-09-30 00:45:05,466][1157520] Fps is (10 sec: 77413.8, 60 sec: 70949.9, 300 sec: 70949.9). Total num frames: 43810816. Throughput: 0: 19621.2. Samples: 939876. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 00:45:05,466][1157520] Avg episode reward: [(0, '41.786')] [2024-09-30 00:45:05,586][1157819] Updated weights for policy 0, policy_version 10698 (0.0006) [2024-09-30 00:45:06,103][1157819] Updated weights for policy 0, policy_version 10708 (0.0006) [2024-09-30 00:45:06,643][1157819] Updated weights for policy 0, policy_version 10718 (0.0006) [2024-09-30 00:45:07,151][1157819] Updated weights for policy 0, policy_version 10728 (0.0006) [2024-09-30 00:45:07,698][1157819] Updated weights for policy 0, policy_version 10738 (0.0006) [2024-09-30 00:45:08,245][1157819] Updated weights for policy 0, policy_version 10748 (0.0006) [2024-09-30 00:45:08,788][1157819] Updated weights for policy 0, policy_version 10758 (0.0006) [2024-09-30 00:45:09,355][1157819] Updated weights for policy 0, policy_version 10768 (0.0006) [2024-09-30 00:45:09,908][1157819] Updated weights for policy 0, policy_version 10778 (0.0006) [2024-09-30 00:45:10,425][1157819] Updated weights for policy 0, policy_version 10788 (0.0006) [2024-09-30 00:45:10,466][1157520] Fps is (10 sec: 75367.6, 60 sec: 71327.0, 300 sec: 71327.0). Total num frames: 44187648. Throughput: 0: 19611.9. Samples: 997436. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 00:45:10,466][1157520] Avg episode reward: [(0, '41.652')] [2024-09-30 00:45:10,955][1157819] Updated weights for policy 0, policy_version 10798 (0.0006) [2024-09-30 00:45:11,495][1157819] Updated weights for policy 0, policy_version 10808 (0.0006) [2024-09-30 00:45:12,023][1157819] Updated weights for policy 0, policy_version 10818 (0.0006) [2024-09-30 00:45:12,539][1157819] Updated weights for policy 0, policy_version 10828 (0.0006) [2024-09-30 00:45:13,037][1157819] Updated weights for policy 0, policy_version 10838 (0.0006) [2024-09-30 00:45:13,538][1157819] Updated weights for policy 0, policy_version 10848 (0.0006) [2024-09-30 00:45:14,041][1157819] Updated weights for policy 0, policy_version 10858 (0.0006) [2024-09-30 00:45:14,544][1157819] Updated weights for policy 0, policy_version 10868 (0.0006) [2024-09-30 00:45:15,052][1157819] Updated weights for policy 0, policy_version 10878 (0.0006) [2024-09-30 00:45:15,466][1157520] Fps is (10 sec: 77414.4, 60 sec: 76253.9, 300 sec: 71966.7). Total num frames: 44584960. Throughput: 0: 19689.6. Samples: 1113628. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 00:45:15,466][1157520] Avg episode reward: [(0, '44.212')] [2024-09-30 00:45:15,469][1157736] Saving new best policy, reward=44.212! [2024-09-30 00:45:15,584][1157819] Updated weights for policy 0, policy_version 10888 (0.0006) [2024-09-30 00:45:16,165][1157819] Updated weights for policy 0, policy_version 10898 (0.0006) [2024-09-30 00:45:16,704][1157819] Updated weights for policy 0, policy_version 10908 (0.0006) [2024-09-30 00:45:17,272][1157819] Updated weights for policy 0, policy_version 10918 (0.0006) [2024-09-30 00:45:17,812][1157819] Updated weights for policy 0, policy_version 10928 (0.0006) [2024-09-30 00:45:18,375][1157819] Updated weights for policy 0, policy_version 10938 (0.0006) [2024-09-30 00:45:18,914][1157819] Updated weights for policy 0, policy_version 10948 (0.0006) [2024-09-30 00:45:19,451][1157819] Updated weights for policy 0, policy_version 10958 (0.0006) [2024-09-30 00:45:19,992][1157819] Updated weights for policy 0, policy_version 10968 (0.0006) [2024-09-30 00:45:20,466][1157520] Fps is (10 sec: 77004.3, 60 sec: 77960.5, 300 sec: 72154.9). Total num frames: 44957696. Throughput: 0: 19534.0. Samples: 1227288. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 00:45:20,466][1157520] Avg episode reward: [(0, '41.266')] [2024-09-30 00:45:20,575][1157819] Updated weights for policy 0, policy_version 10978 (0.0006) [2024-09-30 00:45:21,124][1157819] Updated weights for policy 0, policy_version 10988 (0.0006) [2024-09-30 00:45:21,651][1157819] Updated weights for policy 0, policy_version 10998 (0.0006) [2024-09-30 00:45:22,220][1157819] Updated weights for policy 0, policy_version 11008 (0.0006) [2024-09-30 00:45:22,745][1157819] Updated weights for policy 0, policy_version 11018 (0.0006) [2024-09-30 00:45:23,292][1157819] Updated weights for policy 0, policy_version 11028 (0.0006) [2024-09-30 00:45:23,852][1157819] Updated weights for policy 0, policy_version 11038 (0.0006) [2024-09-30 00:45:24,397][1157819] Updated weights for policy 0, policy_version 11048 (0.0006) [2024-09-30 00:45:24,946][1157819] Updated weights for policy 0, policy_version 11058 (0.0006) [2024-09-30 00:45:25,465][1157819] Updated weights for policy 0, policy_version 11068 (0.0006) [2024-09-30 00:45:25,466][1157520] Fps is (10 sec: 74957.2, 60 sec: 78029.1, 300 sec: 72373.2). Total num frames: 45334528. Throughput: 0: 19371.3. Samples: 1283020. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 00:45:25,466][1157520] Avg episode reward: [(0, '40.844')] [2024-09-30 00:45:25,967][1157819] Updated weights for policy 0, policy_version 11078 (0.0006) [2024-09-30 00:45:26,497][1157819] Updated weights for policy 0, policy_version 11088 (0.0006) [2024-09-30 00:45:27,029][1157819] Updated weights for policy 0, policy_version 11098 (0.0006) [2024-09-30 00:45:27,564][1157819] Updated weights for policy 0, policy_version 11108 (0.0006) [2024-09-30 00:45:28,134][1157819] Updated weights for policy 0, policy_version 11118 (0.0006) [2024-09-30 00:45:28,645][1157819] Updated weights for policy 0, policy_version 11128 (0.0006) [2024-09-30 00:45:29,193][1157819] Updated weights for policy 0, policy_version 11138 (0.0006) [2024-09-30 00:45:29,727][1157819] Updated weights for policy 0, policy_version 11148 (0.0006) [2024-09-30 00:45:30,317][1157819] Updated weights for policy 0, policy_version 11158 (0.0006) [2024-09-30 00:45:30,466][1157520] Fps is (10 sec: 75775.2, 60 sec: 77892.1, 300 sec: 72615.7). Total num frames: 45715456. Throughput: 0: 19345.3. Samples: 1397644. Policy #0 lag: (min: 0.0, avg: 2.8, max: 5.0) [2024-09-30 00:45:30,466][1157520] Avg episode reward: [(0, '41.248')] [2024-09-30 00:45:30,865][1157819] Updated weights for policy 0, policy_version 11168 (0.0006) [2024-09-30 00:45:31,467][1157819] Updated weights for policy 0, policy_version 11178 (0.0006) [2024-09-30 00:45:31,980][1157819] Updated weights for policy 0, policy_version 11188 (0.0006) [2024-09-30 00:45:32,537][1157819] Updated weights for policy 0, policy_version 11198 (0.0006) [2024-09-30 00:45:33,056][1157819] Updated weights for policy 0, policy_version 11208 (0.0006) [2024-09-30 00:45:33,612][1157819] Updated weights for policy 0, policy_version 11218 (0.0006) [2024-09-30 00:45:34,136][1157819] Updated weights for policy 0, policy_version 11228 (0.0006) [2024-09-30 00:45:34,702][1157819] Updated weights for policy 0, policy_version 11238 (0.0006) [2024-09-30 00:45:35,233][1157819] Updated weights for policy 0, policy_version 11248 (0.0006) [2024-09-30 00:45:35,466][1157520] Fps is (10 sec: 75366.2, 60 sec: 77346.1, 300 sec: 72731.3). Total num frames: 46088192. Throughput: 0: 19206.6. Samples: 1509592. Policy #0 lag: (min: 0.0, avg: 2.8, max: 5.0) [2024-09-30 00:45:35,466][1157520] Avg episode reward: [(0, '38.392')] [2024-09-30 00:45:35,753][1157819] Updated weights for policy 0, policy_version 11258 (0.0006) [2024-09-30 00:45:36,253][1157819] Updated weights for policy 0, policy_version 11268 (0.0006) [2024-09-30 00:45:36,848][1157819] Updated weights for policy 0, policy_version 11278 (0.0006) [2024-09-30 00:45:37,384][1157819] Updated weights for policy 0, policy_version 11288 (0.0006) [2024-09-30 00:45:37,944][1157819] Updated weights for policy 0, policy_version 11298 (0.0006) [2024-09-30 00:45:38,508][1157819] Updated weights for policy 0, policy_version 11308 (0.0006) [2024-09-30 00:45:38,996][1157736] Signal inference workers to stop experience collection... (100 times) [2024-09-30 00:45:38,996][1157736] Signal inference workers to resume experience collection... (100 times) [2024-09-30 00:45:39,000][1157819] InferenceWorker_p0-w0: stopping experience collection (100 times) [2024-09-30 00:45:39,000][1157819] InferenceWorker_p0-w0: resuming experience collection (100 times) [2024-09-30 00:45:39,070][1157819] Updated weights for policy 0, policy_version 11318 (0.0006) [2024-09-30 00:45:39,649][1157819] Updated weights for policy 0, policy_version 11328 (0.0006) [2024-09-30 00:45:40,256][1157819] Updated weights for policy 0, policy_version 11338 (0.0006) [2024-09-30 00:45:40,466][1157520] Fps is (10 sec: 73727.2, 60 sec: 76526.9, 300 sec: 72741.2). Total num frames: 46452736. Throughput: 0: 19128.6. Samples: 1566188. Policy #0 lag: (min: 0.0, avg: 2.6, max: 7.0) [2024-09-30 00:45:40,466][1157520] Avg episode reward: [(0, '41.985')] [2024-09-30 00:45:40,844][1157819] Updated weights for policy 0, policy_version 11348 (0.0006) [2024-09-30 00:45:41,440][1157819] Updated weights for policy 0, policy_version 11358 (0.0006) [2024-09-30 00:45:42,013][1157819] Updated weights for policy 0, policy_version 11368 (0.0006) [2024-09-30 00:45:42,577][1157819] Updated weights for policy 0, policy_version 11378 (0.0006) [2024-09-30 00:45:43,002][1157819] Updated weights for policy 0, policy_version 11388 (0.0006) [2024-09-30 00:45:43,504][1157819] Updated weights for policy 0, policy_version 11398 (0.0006) [2024-09-30 00:45:43,959][1157819] Updated weights for policy 0, policy_version 11408 (0.0006) [2024-09-30 00:45:44,480][1157819] Updated weights for policy 0, policy_version 11418 (0.0006) [2024-09-30 00:45:44,983][1157819] Updated weights for policy 0, policy_version 11428 (0.0006) [2024-09-30 00:45:45,466][1157520] Fps is (10 sec: 75775.5, 60 sec: 76868.2, 300 sec: 73056.6). Total num frames: 46845952. Throughput: 0: 18942.3. Samples: 1678244. Policy #0 lag: (min: 0.0, avg: 2.6, max: 7.0) [2024-09-30 00:45:45,466][1157520] Avg episode reward: [(0, '42.496')] [2024-09-30 00:45:45,519][1157819] Updated weights for policy 0, policy_version 11438 (0.0006) [2024-09-30 00:45:46,004][1157819] Updated weights for policy 0, policy_version 11448 (0.0006) [2024-09-30 00:45:46,492][1157819] Updated weights for policy 0, policy_version 11458 (0.0006) [2024-09-30 00:45:47,015][1157819] Updated weights for policy 0, policy_version 11468 (0.0006) [2024-09-30 00:45:47,516][1157819] Updated weights for policy 0, policy_version 11478 (0.0006) [2024-09-30 00:45:48,010][1157819] Updated weights for policy 0, policy_version 11488 (0.0006) [2024-09-30 00:45:48,556][1157819] Updated weights for policy 0, policy_version 11498 (0.0006) [2024-09-30 00:45:49,093][1157819] Updated weights for policy 0, policy_version 11508 (0.0006) [2024-09-30 00:45:49,643][1157819] Updated weights for policy 0, policy_version 11518 (0.0006) [2024-09-30 00:45:50,196][1157819] Updated weights for policy 0, policy_version 11528 (0.0006) [2024-09-30 00:45:50,466][1157520] Fps is (10 sec: 78234.5, 60 sec: 76663.3, 300 sec: 73298.4). Total num frames: 47235072. Throughput: 0: 19056.4. Samples: 1797416. Policy #0 lag: (min: 1.0, avg: 2.7, max: 6.0) [2024-09-30 00:45:50,466][1157520] Avg episode reward: [(0, '42.269')] [2024-09-30 00:45:50,829][1157819] Updated weights for policy 0, policy_version 11538 (0.0006) [2024-09-30 00:45:51,390][1157819] Updated weights for policy 0, policy_version 11548 (0.0006) [2024-09-30 00:45:51,980][1157819] Updated weights for policy 0, policy_version 11558 (0.0006) [2024-09-30 00:45:52,510][1157819] Updated weights for policy 0, policy_version 11568 (0.0006) [2024-09-30 00:45:53,009][1157819] Updated weights for policy 0, policy_version 11578 (0.0006) [2024-09-30 00:45:53,513][1157819] Updated weights for policy 0, policy_version 11588 (0.0006) [2024-09-30 00:45:54,064][1157819] Updated weights for policy 0, policy_version 11598 (0.0006) [2024-09-30 00:45:54,570][1157819] Updated weights for policy 0, policy_version 11608 (0.0006) [2024-09-30 00:45:55,079][1157819] Updated weights for policy 0, policy_version 11618 (0.0006) [2024-09-30 00:45:55,466][1157520] Fps is (10 sec: 77004.6, 60 sec: 76322.0, 300 sec: 73437.8). Total num frames: 47616000. Throughput: 0: 18978.2. Samples: 1851460. Policy #0 lag: (min: 1.0, avg: 2.7, max: 6.0) [2024-09-30 00:45:55,466][1157520] Avg episode reward: [(0, '39.724')] [2024-09-30 00:45:55,586][1157819] Updated weights for policy 0, policy_version 11628 (0.0006) [2024-09-30 00:45:56,098][1157819] Updated weights for policy 0, policy_version 11638 (0.0006) [2024-09-30 00:45:56,602][1157819] Updated weights for policy 0, policy_version 11648 (0.0006) [2024-09-30 00:45:57,085][1157819] Updated weights for policy 0, policy_version 11658 (0.0006) [2024-09-30 00:45:57,576][1157819] Updated weights for policy 0, policy_version 11668 (0.0006) [2024-09-30 00:45:58,067][1157819] Updated weights for policy 0, policy_version 11678 (0.0006) [2024-09-30 00:45:58,547][1157819] Updated weights for policy 0, policy_version 11688 (0.0006) [2024-09-30 00:45:59,035][1157819] Updated weights for policy 0, policy_version 11698 (0.0006) [2024-09-30 00:45:59,507][1157819] Updated weights for policy 0, policy_version 11708 (0.0006) [2024-09-30 00:45:59,985][1157819] Updated weights for policy 0, policy_version 11718 (0.0006) [2024-09-30 00:46:00,411][1157819] Updated weights for policy 0, policy_version 11728 (0.0006) [2024-09-30 00:46:00,466][1157520] Fps is (10 sec: 80282.4, 60 sec: 76731.9, 300 sec: 73941.7). Total num frames: 48037888. Throughput: 0: 19115.3. Samples: 1973816. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 00:46:00,466][1157520] Avg episode reward: [(0, '42.239')] [2024-09-30 00:46:00,892][1157819] Updated weights for policy 0, policy_version 11738 (0.0006) [2024-09-30 00:46:01,361][1157819] Updated weights for policy 0, policy_version 11748 (0.0006) [2024-09-30 00:46:01,841][1157819] Updated weights for policy 0, policy_version 11758 (0.0006) [2024-09-30 00:46:02,262][1157819] Updated weights for policy 0, policy_version 11768 (0.0006) [2024-09-30 00:46:02,746][1157819] Updated weights for policy 0, policy_version 11778 (0.0006) [2024-09-30 00:46:03,204][1157819] Updated weights for policy 0, policy_version 11788 (0.0006) [2024-09-30 00:46:03,669][1157819] Updated weights for policy 0, policy_version 11798 (0.0006) [2024-09-30 00:46:04,136][1157819] Updated weights for policy 0, policy_version 11808 (0.0006) [2024-09-30 00:46:04,591][1157819] Updated weights for policy 0, policy_version 11818 (0.0006) [2024-09-30 00:46:05,060][1157819] Updated weights for policy 0, policy_version 11828 (0.0006) [2024-09-30 00:46:05,466][1157520] Fps is (10 sec: 86836.3, 60 sec: 77892.3, 300 sec: 74617.5). Total num frames: 48484352. Throughput: 0: 19525.9. Samples: 2105952. Policy #0 lag: (min: 0.0, avg: 2.5, max: 7.0) [2024-09-30 00:46:05,466][1157520] Avg episode reward: [(0, '39.569')] [2024-09-30 00:46:05,478][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000011838_48488448.pth... [2024-09-30 00:46:05,479][1157819] Updated weights for policy 0, policy_version 11838 (0.0006) [2024-09-30 00:46:05,524][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000007540_30883840.pth [2024-09-30 00:46:05,927][1157819] Updated weights for policy 0, policy_version 11848 (0.0006) [2024-09-30 00:46:06,398][1157819] Updated weights for policy 0, policy_version 11858 (0.0006) [2024-09-30 00:46:06,825][1157819] Updated weights for policy 0, policy_version 11868 (0.0006) [2024-09-30 00:46:07,082][1157736] Signal inference workers to stop experience collection... (150 times) [2024-09-30 00:46:07,083][1157736] Signal inference workers to resume experience collection... (150 times) [2024-09-30 00:46:07,086][1157819] InferenceWorker_p0-w0: stopping experience collection (150 times) [2024-09-30 00:46:07,086][1157819] InferenceWorker_p0-w0: resuming experience collection (150 times) [2024-09-30 00:46:07,276][1157819] Updated weights for policy 0, policy_version 11878 (0.0006) [2024-09-30 00:46:07,761][1157819] Updated weights for policy 0, policy_version 11888 (0.0006) [2024-09-30 00:46:08,151][1157819] Updated weights for policy 0, policy_version 11898 (0.0006) [2024-09-30 00:46:08,632][1157819] Updated weights for policy 0, policy_version 11908 (0.0006) [2024-09-30 00:46:09,130][1157819] Updated weights for policy 0, policy_version 11918 (0.0006) [2024-09-30 00:46:09,660][1157819] Updated weights for policy 0, policy_version 11928 (0.0006) [2024-09-30 00:46:10,183][1157819] Updated weights for policy 0, policy_version 11938 (0.0007) [2024-09-30 00:46:10,466][1157520] Fps is (10 sec: 87653.9, 60 sec: 78779.6, 300 sec: 75098.1). Total num frames: 48914432. Throughput: 0: 19816.5. Samples: 2174764. Policy #0 lag: (min: 0.0, avg: 2.5, max: 7.0) [2024-09-30 00:46:10,466][1157520] Avg episode reward: [(0, '41.833')] [2024-09-30 00:46:10,722][1157819] Updated weights for policy 0, policy_version 11948 (0.0006) [2024-09-30 00:46:11,229][1157819] Updated weights for policy 0, policy_version 11958 (0.0006) [2024-09-30 00:46:11,725][1157819] Updated weights for policy 0, policy_version 11968 (0.0006) [2024-09-30 00:46:12,260][1157819] Updated weights for policy 0, policy_version 11978 (0.0006) [2024-09-30 00:46:12,762][1157819] Updated weights for policy 0, policy_version 11988 (0.0006) [2024-09-30 00:46:13,269][1157819] Updated weights for policy 0, policy_version 11998 (0.0006) [2024-09-30 00:46:13,770][1157819] Updated weights for policy 0, policy_version 12008 (0.0006) [2024-09-30 00:46:14,269][1157819] Updated weights for policy 0, policy_version 12018 (0.0006) [2024-09-30 00:46:14,767][1157819] Updated weights for policy 0, policy_version 12028 (0.0006) [2024-09-30 00:46:15,268][1157819] Updated weights for policy 0, policy_version 12038 (0.0006) [2024-09-30 00:46:15,466][1157520] Fps is (10 sec: 83556.1, 60 sec: 78916.0, 300 sec: 75340.9). Total num frames: 49319936. Throughput: 0: 19976.4. Samples: 2296584. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 00:46:15,466][1157520] Avg episode reward: [(0, '41.069')] [2024-09-30 00:46:15,768][1157819] Updated weights for policy 0, policy_version 12048 (0.0006) [2024-09-30 00:46:16,286][1157819] Updated weights for policy 0, policy_version 12058 (0.0006) [2024-09-30 00:46:16,810][1157819] Updated weights for policy 0, policy_version 12068 (0.0006) [2024-09-30 00:46:17,308][1157819] Updated weights for policy 0, policy_version 12078 (0.0006) [2024-09-30 00:46:17,812][1157819] Updated weights for policy 0, policy_version 12088 (0.0006) [2024-09-30 00:46:18,296][1157819] Updated weights for policy 0, policy_version 12098 (0.0006) [2024-09-30 00:46:18,916][1157819] Updated weights for policy 0, policy_version 12108 (0.0006) [2024-09-30 00:46:19,573][1157819] Updated weights for policy 0, policy_version 12118 (0.0006) [2024-09-30 00:46:20,141][1157819] Updated weights for policy 0, policy_version 12128 (0.0006) [2024-09-30 00:46:20,466][1157520] Fps is (10 sec: 78233.5, 60 sec: 78984.4, 300 sec: 75342.0). Total num frames: 49696768. Throughput: 0: 20077.9. Samples: 2413100. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 00:46:20,466][1157520] Avg episode reward: [(0, '43.458')] [2024-09-30 00:46:20,745][1157819] Updated weights for policy 0, policy_version 12138 (0.0006) [2024-09-30 00:46:21,380][1157819] Updated weights for policy 0, policy_version 12148 (0.0006) [2024-09-30 00:46:21,975][1157819] Updated weights for policy 0, policy_version 12158 (0.0006) [2024-09-30 00:46:22,599][1157819] Updated weights for policy 0, policy_version 12168 (0.0006) [2024-09-30 00:46:23,168][1157819] Updated weights for policy 0, policy_version 12178 (0.0006) [2024-09-30 00:46:23,817][1157819] Updated weights for policy 0, policy_version 12188 (0.0006) [2024-09-30 00:46:24,395][1157819] Updated weights for policy 0, policy_version 12198 (0.0006) [2024-09-30 00:46:25,012][1157819] Updated weights for policy 0, policy_version 12208 (0.0006) [2024-09-30 00:46:25,466][1157520] Fps is (10 sec: 71270.3, 60 sec: 78301.5, 300 sec: 75036.1). Total num frames: 50032640. Throughput: 0: 19938.2. Samples: 2463408. Policy #0 lag: (min: 0.0, avg: 1.8, max: 6.0) [2024-09-30 00:46:25,466][1157520] Avg episode reward: [(0, '41.271')] [2024-09-30 00:46:25,609][1157819] Updated weights for policy 0, policy_version 12218 (0.0006) [2024-09-30 00:46:26,227][1157819] Updated weights for policy 0, policy_version 12228 (0.0006) [2024-09-30 00:46:26,826][1157819] Updated weights for policy 0, policy_version 12238 (0.0006) [2024-09-30 00:46:27,444][1157819] Updated weights for policy 0, policy_version 12248 (0.0006) [2024-09-30 00:46:28,016][1157819] Updated weights for policy 0, policy_version 12258 (0.0006) [2024-09-30 00:46:28,629][1157819] Updated weights for policy 0, policy_version 12268 (0.0006) [2024-09-30 00:46:29,250][1157819] Updated weights for policy 0, policy_version 12278 (0.0006) [2024-09-30 00:46:29,850][1157819] Updated weights for policy 0, policy_version 12288 (0.0006) [2024-09-30 00:46:30,466][1157520] Fps is (10 sec: 67174.3, 60 sec: 77551.0, 300 sec: 74752.6). Total num frames: 50368512. Throughput: 0: 19704.6. Samples: 2564952. Policy #0 lag: (min: 0.0, avg: 1.8, max: 6.0) [2024-09-30 00:46:30,466][1157520] Avg episode reward: [(0, '41.496')] [2024-09-30 00:46:30,468][1157819] Updated weights for policy 0, policy_version 12298 (0.0007) [2024-09-30 00:46:31,084][1157819] Updated weights for policy 0, policy_version 12308 (0.0006) [2024-09-30 00:46:31,472][1157736] Signal inference workers to stop experience collection... (200 times) [2024-09-30 00:46:31,473][1157736] Signal inference workers to resume experience collection... (200 times) [2024-09-30 00:46:31,476][1157819] InferenceWorker_p0-w0: stopping experience collection (200 times) [2024-09-30 00:46:31,478][1157819] InferenceWorker_p0-w0: resuming experience collection (200 times) [2024-09-30 00:46:31,779][1157819] Updated weights for policy 0, policy_version 12318 (0.0006) [2024-09-30 00:46:32,364][1157819] Updated weights for policy 0, policy_version 12328 (0.0006) [2024-09-30 00:46:32,829][1157819] Updated weights for policy 0, policy_version 12338 (0.0005) [2024-09-30 00:46:33,291][1157819] Updated weights for policy 0, policy_version 12348 (0.0006) [2024-09-30 00:46:33,773][1157819] Updated weights for policy 0, policy_version 12358 (0.0006) [2024-09-30 00:46:34,273][1157819] Updated weights for policy 0, policy_version 12368 (0.0006) [2024-09-30 00:46:34,745][1157819] Updated weights for policy 0, policy_version 12378 (0.0006) [2024-09-30 00:46:35,234][1157819] Updated weights for policy 0, policy_version 12388 (0.0006) [2024-09-30 00:46:35,466][1157520] Fps is (10 sec: 72910.6, 60 sec: 77892.3, 300 sec: 74888.1). Total num frames: 50761728. Throughput: 0: 19521.2. Samples: 2675868. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 00:46:35,466][1157520] Avg episode reward: [(0, '43.014')] [2024-09-30 00:46:35,659][1157819] Updated weights for policy 0, policy_version 12398 (0.0006) [2024-09-30 00:46:36,157][1157819] Updated weights for policy 0, policy_version 12408 (0.0006) [2024-09-30 00:46:36,684][1157819] Updated weights for policy 0, policy_version 12418 (0.0006) [2024-09-30 00:46:37,244][1157819] Updated weights for policy 0, policy_version 12428 (0.0005) [2024-09-30 00:46:37,735][1157819] Updated weights for policy 0, policy_version 12438 (0.0006) [2024-09-30 00:46:38,227][1157819] Updated weights for policy 0, policy_version 12448 (0.0006) [2024-09-30 00:46:38,720][1157819] Updated weights for policy 0, policy_version 12458 (0.0006) [2024-09-30 00:46:39,203][1157819] Updated weights for policy 0, policy_version 12468 (0.0006) [2024-09-30 00:46:39,703][1157819] Updated weights for policy 0, policy_version 12478 (0.0006) [2024-09-30 00:46:40,168][1157819] Updated weights for policy 0, policy_version 12488 (0.0006) [2024-09-30 00:46:40,466][1157520] Fps is (10 sec: 80282.3, 60 sec: 78643.5, 300 sec: 75124.7). Total num frames: 51171328. Throughput: 0: 19699.6. Samples: 2737940. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 00:46:40,466][1157520] Avg episode reward: [(0, '44.564')] [2024-09-30 00:46:40,473][1157736] Saving new best policy, reward=44.564! [2024-09-30 00:46:40,696][1157819] Updated weights for policy 0, policy_version 12498 (0.0006) [2024-09-30 00:46:41,179][1157819] Updated weights for policy 0, policy_version 12508 (0.0006) [2024-09-30 00:46:41,653][1157819] Updated weights for policy 0, policy_version 12518 (0.0006) [2024-09-30 00:46:42,149][1157819] Updated weights for policy 0, policy_version 12528 (0.0006) [2024-09-30 00:46:42,600][1157819] Updated weights for policy 0, policy_version 12538 (0.0006) [2024-09-30 00:46:43,089][1157819] Updated weights for policy 0, policy_version 12548 (0.0006) [2024-09-30 00:46:43,559][1157819] Updated weights for policy 0, policy_version 12558 (0.0006) [2024-09-30 00:46:43,995][1157819] Updated weights for policy 0, policy_version 12568 (0.0006) [2024-09-30 00:46:44,478][1157819] Updated weights for policy 0, policy_version 12578 (0.0006) [2024-09-30 00:46:44,971][1157819] Updated weights for policy 0, policy_version 12588 (0.0006) [2024-09-30 00:46:45,462][1157819] Updated weights for policy 0, policy_version 12598 (0.0006) [2024-09-30 00:46:45,466][1157520] Fps is (10 sec: 83968.3, 60 sec: 79257.7, 300 sec: 75479.3). Total num frames: 51601408. Throughput: 0: 19799.0. Samples: 2864772. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 00:46:45,466][1157520] Avg episode reward: [(0, '40.798')] [2024-09-30 00:46:45,975][1157819] Updated weights for policy 0, policy_version 12608 (0.0006) [2024-09-30 00:46:46,449][1157819] Updated weights for policy 0, policy_version 12618 (0.0006) [2024-09-30 00:46:46,943][1157819] Updated weights for policy 0, policy_version 12628 (0.0006) [2024-09-30 00:46:47,458][1157819] Updated weights for policy 0, policy_version 12638 (0.0006) [2024-09-30 00:46:47,938][1157819] Updated weights for policy 0, policy_version 12648 (0.0006) [2024-09-30 00:46:48,425][1157819] Updated weights for policy 0, policy_version 12658 (0.0006) [2024-09-30 00:46:48,902][1157819] Updated weights for policy 0, policy_version 12668 (0.0006) [2024-09-30 00:46:49,378][1157819] Updated weights for policy 0, policy_version 12678 (0.0006) [2024-09-30 00:46:49,859][1157819] Updated weights for policy 0, policy_version 12688 (0.0006) [2024-09-30 00:46:50,349][1157819] Updated weights for policy 0, policy_version 12698 (0.0006) [2024-09-30 00:46:50,466][1157520] Fps is (10 sec: 84377.8, 60 sec: 79667.4, 300 sec: 75708.3). Total num frames: 52015104. Throughput: 0: 19658.5. Samples: 2990584. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 00:46:50,466][1157520] Avg episode reward: [(0, '46.223')] [2024-09-30 00:46:50,467][1157736] Saving new best policy, reward=46.223! [2024-09-30 00:46:50,849][1157819] Updated weights for policy 0, policy_version 12708 (0.0006) [2024-09-30 00:46:51,299][1157819] Updated weights for policy 0, policy_version 12718 (0.0006) [2024-09-30 00:46:51,785][1157819] Updated weights for policy 0, policy_version 12728 (0.0006) [2024-09-30 00:46:52,268][1157819] Updated weights for policy 0, policy_version 12738 (0.0006) [2024-09-30 00:46:52,760][1157819] Updated weights for policy 0, policy_version 12748 (0.0006) [2024-09-30 00:46:53,242][1157819] Updated weights for policy 0, policy_version 12758 (0.0006) [2024-09-30 00:46:53,725][1157819] Updated weights for policy 0, policy_version 12768 (0.0006) [2024-09-30 00:46:54,212][1157819] Updated weights for policy 0, policy_version 12778 (0.0006) [2024-09-30 00:46:54,700][1157819] Updated weights for policy 0, policy_version 12788 (0.0006) [2024-09-30 00:46:55,187][1157819] Updated weights for policy 0, policy_version 12798 (0.0006) [2024-09-30 00:46:55,466][1157520] Fps is (10 sec: 84377.0, 60 sec: 80486.5, 300 sec: 76023.3). Total num frames: 52445184. Throughput: 0: 19556.4. Samples: 3054800. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 00:46:55,466][1157520] Avg episode reward: [(0, '44.447')] [2024-09-30 00:46:55,682][1157819] Updated weights for policy 0, policy_version 12808 (0.0006) [2024-09-30 00:46:56,123][1157819] Updated weights for policy 0, policy_version 12818 (0.0006) [2024-09-30 00:46:56,647][1157819] Updated weights for policy 0, policy_version 12828 (0.0006) [2024-09-30 00:46:57,172][1157819] Updated weights for policy 0, policy_version 12838 (0.0006) [2024-09-30 00:46:57,682][1157819] Updated weights for policy 0, policy_version 12848 (0.0006) [2024-09-30 00:46:58,150][1157819] Updated weights for policy 0, policy_version 12858 (0.0006) [2024-09-30 00:46:58,629][1157819] Updated weights for policy 0, policy_version 12868 (0.0006) [2024-09-30 00:46:59,133][1157819] Updated weights for policy 0, policy_version 12878 (0.0006) [2024-09-30 00:46:59,618][1157819] Updated weights for policy 0, policy_version 12888 (0.0006) [2024-09-30 00:47:00,096][1157819] Updated weights for policy 0, policy_version 12898 (0.0006) [2024-09-30 00:47:00,466][1157520] Fps is (10 sec: 84376.4, 60 sec: 80349.7, 300 sec: 76222.5). Total num frames: 52858880. Throughput: 0: 19627.6. Samples: 3179824. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 00:47:00,466][1157520] Avg episode reward: [(0, '44.277')] [2024-09-30 00:47:00,587][1157819] Updated weights for policy 0, policy_version 12908 (0.0006) [2024-09-30 00:47:01,061][1157819] Updated weights for policy 0, policy_version 12918 (0.0006) [2024-09-30 00:47:01,561][1157819] Updated weights for policy 0, policy_version 12928 (0.0006) [2024-09-30 00:47:01,858][1157736] Signal inference workers to stop experience collection... (250 times) [2024-09-30 00:47:01,858][1157736] Signal inference workers to resume experience collection... (250 times) [2024-09-30 00:47:01,862][1157819] InferenceWorker_p0-w0: stopping experience collection (250 times) [2024-09-30 00:47:01,862][1157819] InferenceWorker_p0-w0: resuming experience collection (250 times) [2024-09-30 00:47:02,042][1157819] Updated weights for policy 0, policy_version 12938 (0.0006) [2024-09-30 00:47:02,491][1157819] Updated weights for policy 0, policy_version 12948 (0.0006) [2024-09-30 00:47:02,979][1157819] Updated weights for policy 0, policy_version 12958 (0.0006) [2024-09-30 00:47:03,433][1157819] Updated weights for policy 0, policy_version 12968 (0.0006) [2024-09-30 00:47:03,898][1157819] Updated weights for policy 0, policy_version 12978 (0.0006) [2024-09-30 00:47:04,366][1157819] Updated weights for policy 0, policy_version 12988 (0.0006) [2024-09-30 00:47:04,836][1157819] Updated weights for policy 0, policy_version 12998 (0.0006) [2024-09-30 00:47:05,306][1157819] Updated weights for policy 0, policy_version 13008 (0.0006) [2024-09-30 00:47:05,466][1157520] Fps is (10 sec: 84787.8, 60 sec: 80145.1, 300 sec: 76528.2). Total num frames: 53293056. Throughput: 0: 19906.5. Samples: 3308892. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 00:47:05,466][1157520] Avg episode reward: [(0, '45.853')] [2024-09-30 00:47:05,768][1157819] Updated weights for policy 0, policy_version 13018 (0.0006) [2024-09-30 00:47:06,244][1157819] Updated weights for policy 0, policy_version 13028 (0.0006) [2024-09-30 00:47:06,723][1157819] Updated weights for policy 0, policy_version 13038 (0.0006) [2024-09-30 00:47:07,180][1157819] Updated weights for policy 0, policy_version 13048 (0.0006) [2024-09-30 00:47:07,643][1157819] Updated weights for policy 0, policy_version 13058 (0.0006) [2024-09-30 00:47:08,108][1157819] Updated weights for policy 0, policy_version 13068 (0.0006) [2024-09-30 00:47:08,592][1157819] Updated weights for policy 0, policy_version 13078 (0.0006) [2024-09-30 00:47:09,047][1157819] Updated weights for policy 0, policy_version 13088 (0.0006) [2024-09-30 00:47:09,515][1157819] Updated weights for policy 0, policy_version 13098 (0.0006) [2024-09-30 00:47:09,969][1157819] Updated weights for policy 0, policy_version 13108 (0.0006) [2024-09-30 00:47:10,433][1157819] Updated weights for policy 0, policy_version 13118 (0.0006) [2024-09-30 00:47:10,466][1157520] Fps is (10 sec: 87245.9, 60 sec: 80281.7, 300 sec: 76839.8). Total num frames: 53731328. Throughput: 0: 20245.1. Samples: 3374432. Policy #0 lag: (min: 0.0, avg: 2.5, max: 7.0) [2024-09-30 00:47:10,466][1157520] Avg episode reward: [(0, '43.396')] [2024-09-30 00:47:10,919][1157819] Updated weights for policy 0, policy_version 13128 (0.0006) [2024-09-30 00:47:11,395][1157819] Updated weights for policy 0, policy_version 13138 (0.0006) [2024-09-30 00:47:11,887][1157819] Updated weights for policy 0, policy_version 13148 (0.0006) [2024-09-30 00:47:12,340][1157819] Updated weights for policy 0, policy_version 13158 (0.0006) [2024-09-30 00:47:12,817][1157819] Updated weights for policy 0, policy_version 13168 (0.0006) [2024-09-30 00:47:13,323][1157819] Updated weights for policy 0, policy_version 13178 (0.0006) [2024-09-30 00:47:13,783][1157819] Updated weights for policy 0, policy_version 13188 (0.0006) [2024-09-30 00:47:14,270][1157819] Updated weights for policy 0, policy_version 13198 (0.0006) [2024-09-30 00:47:14,802][1157819] Updated weights for policy 0, policy_version 13208 (0.0006) [2024-09-30 00:47:15,347][1157819] Updated weights for policy 0, policy_version 13218 (0.0006) [2024-09-30 00:47:15,466][1157520] Fps is (10 sec: 85606.5, 60 sec: 80486.8, 300 sec: 77022.8). Total num frames: 54149120. Throughput: 0: 20879.7. Samples: 3504536. Policy #0 lag: (min: 0.0, avg: 2.5, max: 7.0) [2024-09-30 00:47:15,466][1157520] Avg episode reward: [(0, '43.479')] [2024-09-30 00:47:15,852][1157819] Updated weights for policy 0, policy_version 13228 (0.0006) [2024-09-30 00:47:16,353][1157819] Updated weights for policy 0, policy_version 13238 (0.0006) [2024-09-30 00:47:16,847][1157819] Updated weights for policy 0, policy_version 13248 (0.0006) [2024-09-30 00:47:17,370][1157819] Updated weights for policy 0, policy_version 13258 (0.0006) [2024-09-30 00:47:17,862][1157819] Updated weights for policy 0, policy_version 13268 (0.0006) [2024-09-30 00:47:18,364][1157819] Updated weights for policy 0, policy_version 13278 (0.0006) [2024-09-30 00:47:18,898][1157819] Updated weights for policy 0, policy_version 13288 (0.0006) [2024-09-30 00:47:19,386][1157819] Updated weights for policy 0, policy_version 13298 (0.0006) [2024-09-30 00:47:19,897][1157819] Updated weights for policy 0, policy_version 13308 (0.0006) [2024-09-30 00:47:20,382][1157819] Updated weights for policy 0, policy_version 13318 (0.0006) [2024-09-30 00:47:20,466][1157520] Fps is (10 sec: 82739.2, 60 sec: 81032.6, 300 sec: 77152.6). Total num frames: 54558720. Throughput: 0: 21103.1. Samples: 3625508. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 00:47:20,466][1157520] Avg episode reward: [(0, '41.022')] [2024-09-30 00:47:20,895][1157819] Updated weights for policy 0, policy_version 13328 (0.0006) [2024-09-30 00:47:21,395][1157819] Updated weights for policy 0, policy_version 13338 (0.0006) [2024-09-30 00:47:21,897][1157819] Updated weights for policy 0, policy_version 13348 (0.0006) [2024-09-30 00:47:22,390][1157819] Updated weights for policy 0, policy_version 13358 (0.0006) [2024-09-30 00:47:22,909][1157819] Updated weights for policy 0, policy_version 13368 (0.0006) [2024-09-30 00:47:23,436][1157819] Updated weights for policy 0, policy_version 13378 (0.0006) [2024-09-30 00:47:23,939][1157819] Updated weights for policy 0, policy_version 13388 (0.0006) [2024-09-30 00:47:24,435][1157819] Updated weights for policy 0, policy_version 13398 (0.0006) [2024-09-30 00:47:24,954][1157819] Updated weights for policy 0, policy_version 13408 (0.0006) [2024-09-30 00:47:25,448][1157819] Updated weights for policy 0, policy_version 13418 (0.0006) [2024-09-30 00:47:25,466][1157520] Fps is (10 sec: 81100.8, 60 sec: 82125.2, 300 sec: 77233.4). Total num frames: 54960128. Throughput: 0: 21076.3. Samples: 3686372. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 00:47:25,466][1157520] Avg episode reward: [(0, '45.504')] [2024-09-30 00:47:25,960][1157819] Updated weights for policy 0, policy_version 13428 (0.0006) [2024-09-30 00:47:26,447][1157819] Updated weights for policy 0, policy_version 13438 (0.0006) [2024-09-30 00:47:27,000][1157819] Updated weights for policy 0, policy_version 13448 (0.0006) [2024-09-30 00:47:27,516][1157819] Updated weights for policy 0, policy_version 13458 (0.0006) [2024-09-30 00:47:28,023][1157819] Updated weights for policy 0, policy_version 13468 (0.0006) [2024-09-30 00:47:28,529][1157819] Updated weights for policy 0, policy_version 13478 (0.0006) [2024-09-30 00:47:29,031][1157819] Updated weights for policy 0, policy_version 13488 (0.0006) [2024-09-30 00:47:29,542][1157819] Updated weights for policy 0, policy_version 13498 (0.0006) [2024-09-30 00:47:30,045][1157819] Updated weights for policy 0, policy_version 13508 (0.0006) [2024-09-30 00:47:30,466][1157520] Fps is (10 sec: 80281.5, 60 sec: 83217.2, 300 sec: 77310.2). Total num frames: 55361536. Throughput: 0: 20928.1. Samples: 3806536. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 00:47:30,466][1157520] Avg episode reward: [(0, '45.561')] [2024-09-30 00:47:30,537][1157819] Updated weights for policy 0, policy_version 13518 (0.0006) [2024-09-30 00:47:31,072][1157819] Updated weights for policy 0, policy_version 13528 (0.0006) [2024-09-30 00:47:31,571][1157819] Updated weights for policy 0, policy_version 13538 (0.0006) [2024-09-30 00:47:32,076][1157819] Updated weights for policy 0, policy_version 13548 (0.0006) [2024-09-30 00:47:32,631][1157819] Updated weights for policy 0, policy_version 13558 (0.0006) [2024-09-30 00:47:33,147][1157819] Updated weights for policy 0, policy_version 13568 (0.0006) [2024-09-30 00:47:33,659][1157819] Updated weights for policy 0, policy_version 13578 (0.0006) [2024-09-30 00:47:34,223][1157819] Updated weights for policy 0, policy_version 13588 (0.0006) [2024-09-30 00:47:34,775][1157819] Updated weights for policy 0, policy_version 13598 (0.0006) [2024-09-30 00:47:35,324][1157819] Updated weights for policy 0, policy_version 13608 (0.0006) [2024-09-30 00:47:35,466][1157520] Fps is (10 sec: 79052.8, 60 sec: 83148.9, 300 sec: 77322.8). Total num frames: 55750656. Throughput: 0: 20757.2. Samples: 3924660. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 00:47:35,466][1157520] Avg episode reward: [(0, '46.065')] [2024-09-30 00:47:35,858][1157819] Updated weights for policy 0, policy_version 13618 (0.0006) [2024-09-30 00:47:36,385][1157819] Updated weights for policy 0, policy_version 13628 (0.0006) [2024-09-30 00:47:36,924][1157819] Updated weights for policy 0, policy_version 13638 (0.0006) [2024-09-30 00:47:37,489][1157819] Updated weights for policy 0, policy_version 13648 (0.0006) [2024-09-30 00:47:38,019][1157819] Updated weights for policy 0, policy_version 13658 (0.0006) [2024-09-30 00:47:38,524][1157819] Updated weights for policy 0, policy_version 13668 (0.0006) [2024-09-30 00:47:38,846][1157736] Signal inference workers to stop experience collection... (300 times) [2024-09-30 00:47:38,847][1157736] Signal inference workers to resume experience collection... (300 times) [2024-09-30 00:47:38,853][1157819] InferenceWorker_p0-w0: stopping experience collection (300 times) [2024-09-30 00:47:38,853][1157819] InferenceWorker_p0-w0: resuming experience collection (300 times) [2024-09-30 00:47:39,041][1157819] Updated weights for policy 0, policy_version 13678 (0.0006) [2024-09-30 00:47:39,561][1157819] Updated weights for policy 0, policy_version 13688 (0.0006) [2024-09-30 00:47:40,136][1157819] Updated weights for policy 0, policy_version 13698 (0.0006) [2024-09-30 00:47:40,466][1157520] Fps is (10 sec: 76595.3, 60 sec: 82602.7, 300 sec: 77275.9). Total num frames: 56127488. Throughput: 0: 20588.4. Samples: 3981276. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 00:47:40,466][1157520] Avg episode reward: [(0, '41.489')] [2024-09-30 00:47:40,677][1157819] Updated weights for policy 0, policy_version 13708 (0.0006) [2024-09-30 00:47:41,217][1157819] Updated weights for policy 0, policy_version 13718 (0.0006) [2024-09-30 00:47:41,774][1157819] Updated weights for policy 0, policy_version 13728 (0.0006) [2024-09-30 00:47:42,297][1157819] Updated weights for policy 0, policy_version 13738 (0.0006) [2024-09-30 00:47:42,832][1157819] Updated weights for policy 0, policy_version 13748 (0.0006) [2024-09-30 00:47:43,336][1157819] Updated weights for policy 0, policy_version 13758 (0.0006) [2024-09-30 00:47:43,922][1157819] Updated weights for policy 0, policy_version 13768 (0.0006) [2024-09-30 00:47:44,467][1157819] Updated weights for policy 0, policy_version 13778 (0.0006) [2024-09-30 00:47:44,982][1157819] Updated weights for policy 0, policy_version 13788 (0.0006) [2024-09-30 00:47:45,466][1157520] Fps is (10 sec: 75776.1, 60 sec: 81783.5, 300 sec: 77250.4). Total num frames: 56508416. Throughput: 0: 20364.4. Samples: 4096220. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 00:47:45,466][1157520] Avg episode reward: [(0, '42.397')] [2024-09-30 00:47:45,523][1157819] Updated weights for policy 0, policy_version 13798 (0.0006) [2024-09-30 00:47:46,095][1157819] Updated weights for policy 0, policy_version 13808 (0.0006) [2024-09-30 00:47:46,625][1157819] Updated weights for policy 0, policy_version 13818 (0.0006) [2024-09-30 00:47:47,146][1157819] Updated weights for policy 0, policy_version 13828 (0.0006) [2024-09-30 00:47:47,649][1157819] Updated weights for policy 0, policy_version 13838 (0.0006) [2024-09-30 00:47:48,150][1157819] Updated weights for policy 0, policy_version 13848 (0.0006) [2024-09-30 00:47:48,699][1157819] Updated weights for policy 0, policy_version 13858 (0.0006) [2024-09-30 00:47:49,229][1157819] Updated weights for policy 0, policy_version 13868 (0.0006) [2024-09-30 00:47:49,733][1157819] Updated weights for policy 0, policy_version 13878 (0.0006) [2024-09-30 00:47:50,255][1157819] Updated weights for policy 0, policy_version 13888 (0.0006) [2024-09-30 00:47:50,466][1157520] Fps is (10 sec: 77414.2, 60 sec: 81442.1, 300 sec: 77282.2). Total num frames: 56901632. Throughput: 0: 20071.9. Samples: 4212128. Policy #0 lag: (min: 1.0, avg: 1.9, max: 5.0) [2024-09-30 00:47:50,466][1157520] Avg episode reward: [(0, '46.480')] [2024-09-30 00:47:50,466][1157736] Saving new best policy, reward=46.480! [2024-09-30 00:47:50,763][1157819] Updated weights for policy 0, policy_version 13898 (0.0005) [2024-09-30 00:47:51,290][1157819] Updated weights for policy 0, policy_version 13908 (0.0006) [2024-09-30 00:47:51,812][1157819] Updated weights for policy 0, policy_version 13918 (0.0006) [2024-09-30 00:47:52,310][1157819] Updated weights for policy 0, policy_version 13928 (0.0006) [2024-09-30 00:47:52,815][1157819] Updated weights for policy 0, policy_version 13938 (0.0006) [2024-09-30 00:47:53,344][1157819] Updated weights for policy 0, policy_version 13948 (0.0006) [2024-09-30 00:47:53,865][1157819] Updated weights for policy 0, policy_version 13958 (0.0006) [2024-09-30 00:47:54,355][1157819] Updated weights for policy 0, policy_version 13968 (0.0006) [2024-09-30 00:47:54,870][1157819] Updated weights for policy 0, policy_version 13978 (0.0006) [2024-09-30 00:47:55,355][1157819] Updated weights for policy 0, policy_version 13988 (0.0006) [2024-09-30 00:47:55,466][1157520] Fps is (10 sec: 79052.6, 60 sec: 80896.1, 300 sec: 77331.0). Total num frames: 57298944. Throughput: 0: 19932.4. Samples: 4271392. Policy #0 lag: (min: 1.0, avg: 1.9, max: 5.0) [2024-09-30 00:47:55,466][1157520] Avg episode reward: [(0, '42.907')] [2024-09-30 00:47:55,843][1157819] Updated weights for policy 0, policy_version 13998 (0.0006) [2024-09-30 00:47:56,414][1157819] Updated weights for policy 0, policy_version 14008 (0.0007) [2024-09-30 00:47:56,904][1157819] Updated weights for policy 0, policy_version 14018 (0.0006) [2024-09-30 00:47:57,397][1157819] Updated weights for policy 0, policy_version 14028 (0.0006) [2024-09-30 00:47:57,905][1157819] Updated weights for policy 0, policy_version 14038 (0.0006) [2024-09-30 00:47:58,396][1157819] Updated weights for policy 0, policy_version 14048 (0.0006) [2024-09-30 00:47:58,899][1157819] Updated weights for policy 0, policy_version 14058 (0.0006) [2024-09-30 00:47:59,400][1157819] Updated weights for policy 0, policy_version 14068 (0.0006) [2024-09-30 00:47:59,907][1157819] Updated weights for policy 0, policy_version 14078 (0.0006) [2024-09-30 00:48:00,440][1157819] Updated weights for policy 0, policy_version 14088 (0.0006) [2024-09-30 00:48:00,466][1157520] Fps is (10 sec: 80691.2, 60 sec: 80827.9, 300 sec: 77431.4). Total num frames: 57708544. Throughput: 0: 19733.0. Samples: 4392520. Policy #0 lag: (min: 0.0, avg: 2.6, max: 7.0) [2024-09-30 00:48:00,466][1157520] Avg episode reward: [(0, '43.884')] [2024-09-30 00:48:00,924][1157819] Updated weights for policy 0, policy_version 14098 (0.0006) [2024-09-30 00:48:01,408][1157819] Updated weights for policy 0, policy_version 14108 (0.0006) [2024-09-30 00:48:01,946][1157819] Updated weights for policy 0, policy_version 14118 (0.0006) [2024-09-30 00:48:02,444][1157819] Updated weights for policy 0, policy_version 14128 (0.0006) [2024-09-30 00:48:02,953][1157819] Updated weights for policy 0, policy_version 14138 (0.0006) [2024-09-30 00:48:03,488][1157819] Updated weights for policy 0, policy_version 14148 (0.0007) [2024-09-30 00:48:03,986][1157819] Updated weights for policy 0, policy_version 14158 (0.0006) [2024-09-30 00:48:04,478][1157819] Updated weights for policy 0, policy_version 14168 (0.0006) [2024-09-30 00:48:04,974][1157819] Updated weights for policy 0, policy_version 14178 (0.0006) [2024-09-30 00:48:05,466][1157520] Fps is (10 sec: 81100.7, 60 sec: 80281.6, 300 sec: 77492.4). Total num frames: 58109952. Throughput: 0: 19748.2. Samples: 4514176. Policy #0 lag: (min: 0.0, avg: 2.6, max: 7.0) [2024-09-30 00:48:05,466][1157520] Avg episode reward: [(0, '42.454')] [2024-09-30 00:48:05,480][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000014188_58114048.pth... [2024-09-30 00:48:05,481][1157819] Updated weights for policy 0, policy_version 14188 (0.0006) [2024-09-30 00:48:05,524][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000009768_40009728.pth [2024-09-30 00:48:05,992][1157819] Updated weights for policy 0, policy_version 14198 (0.0006) [2024-09-30 00:48:06,542][1157819] Updated weights for policy 0, policy_version 14208 (0.0006) [2024-09-30 00:48:07,044][1157819] Updated weights for policy 0, policy_version 14218 (0.0006) [2024-09-30 00:48:07,554][1157819] Updated weights for policy 0, policy_version 14228 (0.0006) [2024-09-30 00:48:08,038][1157819] Updated weights for policy 0, policy_version 14238 (0.0006) [2024-09-30 00:48:08,538][1157819] Updated weights for policy 0, policy_version 14248 (0.0006) [2024-09-30 00:48:09,087][1157819] Updated weights for policy 0, policy_version 14258 (0.0006) [2024-09-30 00:48:09,583][1157819] Updated weights for policy 0, policy_version 14268 (0.0006) [2024-09-30 00:48:09,803][1157736] Signal inference workers to stop experience collection... (350 times) [2024-09-30 00:48:09,803][1157736] Signal inference workers to resume experience collection... (350 times) [2024-09-30 00:48:09,809][1157819] InferenceWorker_p0-w0: stopping experience collection (350 times) [2024-09-30 00:48:09,809][1157819] InferenceWorker_p0-w0: resuming experience collection (350 times) [2024-09-30 00:48:10,082][1157819] Updated weights for policy 0, policy_version 14278 (0.0006) [2024-09-30 00:48:10,466][1157520] Fps is (10 sec: 80281.6, 60 sec: 79667.2, 300 sec: 77550.8). Total num frames: 58511360. Throughput: 0: 19736.7. Samples: 4574524. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 00:48:10,466][1157520] Avg episode reward: [(0, '44.581')] [2024-09-30 00:48:10,580][1157819] Updated weights for policy 0, policy_version 14288 (0.0006) [2024-09-30 00:48:11,084][1157819] Updated weights for policy 0, policy_version 14298 (0.0006) [2024-09-30 00:48:11,610][1157819] Updated weights for policy 0, policy_version 14308 (0.0006) [2024-09-30 00:48:12,132][1157819] Updated weights for policy 0, policy_version 14318 (0.0006) [2024-09-30 00:48:12,633][1157819] Updated weights for policy 0, policy_version 14328 (0.0006) [2024-09-30 00:48:13,170][1157819] Updated weights for policy 0, policy_version 14338 (0.0006) [2024-09-30 00:48:13,679][1157819] Updated weights for policy 0, policy_version 14348 (0.0006) [2024-09-30 00:48:14,206][1157819] Updated weights for policy 0, policy_version 14358 (0.0006) [2024-09-30 00:48:14,744][1157819] Updated weights for policy 0, policy_version 14368 (0.0006) [2024-09-30 00:48:15,267][1157819] Updated weights for policy 0, policy_version 14378 (0.0006) [2024-09-30 00:48:15,466][1157520] Fps is (10 sec: 79872.2, 60 sec: 79325.9, 300 sec: 77590.1). Total num frames: 58908672. Throughput: 0: 19725.1. Samples: 4694164. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 00:48:15,466][1157520] Avg episode reward: [(0, '45.913')] [2024-09-30 00:48:15,774][1157819] Updated weights for policy 0, policy_version 14388 (0.0006) [2024-09-30 00:48:16,295][1157819] Updated weights for policy 0, policy_version 14398 (0.0006) [2024-09-30 00:48:16,825][1157819] Updated weights for policy 0, policy_version 14408 (0.0006) [2024-09-30 00:48:17,345][1157819] Updated weights for policy 0, policy_version 14418 (0.0006) [2024-09-30 00:48:17,861][1157819] Updated weights for policy 0, policy_version 14428 (0.0006) [2024-09-30 00:48:18,420][1157819] Updated weights for policy 0, policy_version 14438 (0.0006) [2024-09-30 00:48:18,947][1157819] Updated weights for policy 0, policy_version 14448 (0.0006) [2024-09-30 00:48:19,450][1157819] Updated weights for policy 0, policy_version 14458 (0.0006) [2024-09-30 00:48:19,966][1157819] Updated weights for policy 0, policy_version 14468 (0.0006) [2024-09-30 00:48:20,466][1157520] Fps is (10 sec: 78642.5, 60 sec: 78984.4, 300 sec: 77594.8). Total num frames: 59297792. Throughput: 0: 19702.4. Samples: 4811268. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 00:48:20,466][1157520] Avg episode reward: [(0, '45.861')] [2024-09-30 00:48:20,473][1157819] Updated weights for policy 0, policy_version 14478 (0.0006) [2024-09-30 00:48:21,084][1157819] Updated weights for policy 0, policy_version 14488 (0.0006) [2024-09-30 00:48:21,718][1157819] Updated weights for policy 0, policy_version 14498 (0.0006) [2024-09-30 00:48:22,383][1157819] Updated weights for policy 0, policy_version 14508 (0.0006) [2024-09-30 00:48:23,015][1157819] Updated weights for policy 0, policy_version 14518 (0.0006) [2024-09-30 00:48:23,652][1157819] Updated weights for policy 0, policy_version 14528 (0.0006) [2024-09-30 00:48:24,233][1157819] Updated weights for policy 0, policy_version 14538 (0.0006) [2024-09-30 00:48:24,834][1157819] Updated weights for policy 0, policy_version 14548 (0.0006) [2024-09-30 00:48:25,466][1157520] Fps is (10 sec: 71678.8, 60 sec: 77755.5, 300 sec: 77357.0). Total num frames: 59625472. Throughput: 0: 19596.6. Samples: 4863128. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 00:48:25,466][1157520] Avg episode reward: [(0, '46.908')] [2024-09-30 00:48:25,472][1157736] Saving new best policy, reward=46.908! [2024-09-30 00:48:25,559][1157819] Updated weights for policy 0, policy_version 14558 (0.0006) [2024-09-30 00:48:26,135][1157819] Updated weights for policy 0, policy_version 14568 (0.0006) [2024-09-30 00:48:26,760][1157819] Updated weights for policy 0, policy_version 14578 (0.0006) [2024-09-30 00:48:27,419][1157819] Updated weights for policy 0, policy_version 14588 (0.0006) [2024-09-30 00:48:28,099][1157819] Updated weights for policy 0, policy_version 14598 (0.0006) [2024-09-30 00:48:28,724][1157819] Updated weights for policy 0, policy_version 14608 (0.0006) [2024-09-30 00:48:29,302][1157819] Updated weights for policy 0, policy_version 14618 (0.0006) [2024-09-30 00:48:29,996][1157819] Updated weights for policy 0, policy_version 14628 (0.0006) [2024-09-30 00:48:30,466][1157520] Fps is (10 sec: 64716.9, 60 sec: 76390.3, 300 sec: 77096.7). Total num frames: 59944960. Throughput: 0: 19196.8. Samples: 4960076. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 00:48:30,466][1157520] Avg episode reward: [(0, '44.208')] [2024-09-30 00:48:30,608][1157819] Updated weights for policy 0, policy_version 14638 (0.0006) [2024-09-30 00:48:31,294][1157819] Updated weights for policy 0, policy_version 14648 (0.0007) [2024-09-30 00:48:31,920][1157819] Updated weights for policy 0, policy_version 14658 (0.0006) [2024-09-30 00:48:32,533][1157819] Updated weights for policy 0, policy_version 14668 (0.0006) [2024-09-30 00:48:32,748][1157736] Signal inference workers to stop experience collection... (400 times) [2024-09-30 00:48:32,751][1157819] InferenceWorker_p0-w0: stopping experience collection (400 times) [2024-09-30 00:48:32,758][1157736] Signal inference workers to resume experience collection... (400 times) [2024-09-30 00:48:32,759][1157819] InferenceWorker_p0-w0: resuming experience collection (400 times) [2024-09-30 00:48:33,275][1157819] Updated weights for policy 0, policy_version 14678 (0.0006) [2024-09-30 00:48:33,900][1157819] Updated weights for policy 0, policy_version 14688 (0.0006) [2024-09-30 00:48:34,506][1157819] Updated weights for policy 0, policy_version 14698 (0.0006) [2024-09-30 00:48:35,094][1157819] Updated weights for policy 0, policy_version 14708 (0.0006) [2024-09-30 00:48:35,466][1157520] Fps is (10 sec: 63897.7, 60 sec: 75229.7, 300 sec: 76846.3). Total num frames: 60264448. Throughput: 0: 18742.8. Samples: 5055556. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 00:48:35,466][1157520] Avg episode reward: [(0, '48.215')] [2024-09-30 00:48:35,478][1157736] Saving new best policy, reward=48.215! [2024-09-30 00:48:35,721][1157819] Updated weights for policy 0, policy_version 14718 (0.0006) [2024-09-30 00:48:36,236][1157819] Updated weights for policy 0, policy_version 14728 (0.0006) [2024-09-30 00:48:36,888][1157819] Updated weights for policy 0, policy_version 14738 (0.0006) [2024-09-30 00:48:37,378][1157819] Updated weights for policy 0, policy_version 14748 (0.0006) [2024-09-30 00:48:37,834][1157819] Updated weights for policy 0, policy_version 14758 (0.0006) [2024-09-30 00:48:38,239][1157819] Updated weights for policy 0, policy_version 14768 (0.0006) [2024-09-30 00:48:38,719][1157819] Updated weights for policy 0, policy_version 14778 (0.0006) [2024-09-30 00:48:39,107][1157819] Updated weights for policy 0, policy_version 14788 (0.0006) [2024-09-30 00:48:39,577][1157819] Updated weights for policy 0, policy_version 14798 (0.0006) [2024-09-30 00:48:39,991][1157819] Updated weights for policy 0, policy_version 14808 (0.0006) [2024-09-30 00:48:40,463][1157819] Updated weights for policy 0, policy_version 14818 (0.0006) [2024-09-30 00:48:40,466][1157520] Fps is (10 sec: 74956.8, 60 sec: 76117.2, 300 sec: 77017.1). Total num frames: 60694528. Throughput: 0: 18667.5. Samples: 5111432. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 00:48:40,466][1157520] Avg episode reward: [(0, '42.811')] [2024-09-30 00:48:40,915][1157819] Updated weights for policy 0, policy_version 14828 (0.0006) [2024-09-30 00:48:41,353][1157819] Updated weights for policy 0, policy_version 14838 (0.0006) [2024-09-30 00:48:41,818][1157819] Updated weights for policy 0, policy_version 14848 (0.0006) [2024-09-30 00:48:42,265][1157819] Updated weights for policy 0, policy_version 14858 (0.0006) [2024-09-30 00:48:42,682][1157819] Updated weights for policy 0, policy_version 14868 (0.0006) [2024-09-30 00:48:43,129][1157819] Updated weights for policy 0, policy_version 14878 (0.0006) [2024-09-30 00:48:43,557][1157819] Updated weights for policy 0, policy_version 14888 (0.0006) [2024-09-30 00:48:44,006][1157819] Updated weights for policy 0, policy_version 14898 (0.0006) [2024-09-30 00:48:44,421][1157819] Updated weights for policy 0, policy_version 14908 (0.0006) [2024-09-30 00:48:44,865][1157819] Updated weights for policy 0, policy_version 14918 (0.0006) [2024-09-30 00:48:45,287][1157819] Updated weights for policy 0, policy_version 14928 (0.0006) [2024-09-30 00:48:45,466][1157520] Fps is (10 sec: 89703.8, 60 sec: 77550.9, 300 sec: 77316.3). Total num frames: 61161472. Throughput: 0: 19068.7. Samples: 5250612. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 00:48:45,466][1157520] Avg episode reward: [(0, '43.004')] [2024-09-30 00:48:45,731][1157819] Updated weights for policy 0, policy_version 14938 (0.0006) [2024-09-30 00:48:46,161][1157819] Updated weights for policy 0, policy_version 14948 (0.0006) [2024-09-30 00:48:46,600][1157819] Updated weights for policy 0, policy_version 14958 (0.0006) [2024-09-30 00:48:47,036][1157819] Updated weights for policy 0, policy_version 14968 (0.0006) [2024-09-30 00:48:47,477][1157819] Updated weights for policy 0, policy_version 14978 (0.0006) [2024-09-30 00:48:47,923][1157819] Updated weights for policy 0, policy_version 14988 (0.0006) [2024-09-30 00:48:48,349][1157819] Updated weights for policy 0, policy_version 14998 (0.0006) [2024-09-30 00:48:48,847][1157819] Updated weights for policy 0, policy_version 15008 (0.0006) [2024-09-30 00:48:49,347][1157819] Updated weights for policy 0, policy_version 15018 (0.0006) [2024-09-30 00:48:49,856][1157819] Updated weights for policy 0, policy_version 15028 (0.0006) [2024-09-30 00:48:50,345][1157819] Updated weights for policy 0, policy_version 15038 (0.0006) [2024-09-30 00:48:50,466][1157520] Fps is (10 sec: 90521.1, 60 sec: 78301.7, 300 sec: 77501.8). Total num frames: 61599744. Throughput: 0: 19402.1. Samples: 5387272. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 00:48:50,466][1157520] Avg episode reward: [(0, '39.283')] [2024-09-30 00:48:50,500][1157736] Signal inference workers to stop experience collection... (450 times) [2024-09-30 00:48:50,501][1157736] Signal inference workers to resume experience collection... (450 times) [2024-09-30 00:48:50,504][1157819] InferenceWorker_p0-w0: stopping experience collection (450 times) [2024-09-30 00:48:50,506][1157819] InferenceWorker_p0-w0: resuming experience collection (450 times) [2024-09-30 00:48:50,891][1157819] Updated weights for policy 0, policy_version 15048 (0.0006) [2024-09-30 00:48:51,397][1157819] Updated weights for policy 0, policy_version 15058 (0.0006) [2024-09-30 00:48:51,943][1157819] Updated weights for policy 0, policy_version 15068 (0.0006) [2024-09-30 00:48:52,511][1157819] Updated weights for policy 0, policy_version 15078 (0.0006) [2024-09-30 00:48:53,015][1157819] Updated weights for policy 0, policy_version 15088 (0.0006) [2024-09-30 00:48:53,546][1157819] Updated weights for policy 0, policy_version 15098 (0.0006) [2024-09-30 00:48:54,081][1157819] Updated weights for policy 0, policy_version 15108 (0.0006) [2024-09-30 00:48:54,628][1157819] Updated weights for policy 0, policy_version 15118 (0.0006) [2024-09-30 00:48:55,179][1157819] Updated weights for policy 0, policy_version 15128 (0.0006) [2024-09-30 00:48:55,466][1157520] Fps is (10 sec: 82327.7, 60 sec: 78096.8, 300 sec: 77493.0). Total num frames: 61984768. Throughput: 0: 19365.6. Samples: 5445980. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 00:48:55,466][1157520] Avg episode reward: [(0, '43.564')] [2024-09-30 00:48:55,756][1157819] Updated weights for policy 0, policy_version 15138 (0.0006) [2024-09-30 00:48:56,292][1157819] Updated weights for policy 0, policy_version 15148 (0.0006) [2024-09-30 00:48:56,851][1157819] Updated weights for policy 0, policy_version 15158 (0.0006) [2024-09-30 00:48:57,357][1157819] Updated weights for policy 0, policy_version 15168 (0.0006) [2024-09-30 00:48:57,923][1157819] Updated weights for policy 0, policy_version 15178 (0.0006) [2024-09-30 00:48:58,467][1157819] Updated weights for policy 0, policy_version 15188 (0.0006) [2024-09-30 00:48:59,001][1157819] Updated weights for policy 0, policy_version 15198 (0.0006) [2024-09-30 00:48:59,587][1157819] Updated weights for policy 0, policy_version 15208 (0.0006) [2024-09-30 00:49:00,096][1157819] Updated weights for policy 0, policy_version 15218 (0.0006) [2024-09-30 00:49:00,466][1157520] Fps is (10 sec: 76186.1, 60 sec: 77550.9, 300 sec: 77456.2). Total num frames: 62361600. Throughput: 0: 19211.5. Samples: 5558684. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 00:49:00,466][1157520] Avg episode reward: [(0, '41.153')] [2024-09-30 00:49:00,611][1157819] Updated weights for policy 0, policy_version 15228 (0.0006) [2024-09-30 00:49:01,130][1157819] Updated weights for policy 0, policy_version 15238 (0.0006) [2024-09-30 00:49:01,616][1157819] Updated weights for policy 0, policy_version 15248 (0.0006) [2024-09-30 00:49:02,147][1157819] Updated weights for policy 0, policy_version 15258 (0.0006) [2024-09-30 00:49:02,664][1157819] Updated weights for policy 0, policy_version 15268 (0.0006) [2024-09-30 00:49:03,171][1157819] Updated weights for policy 0, policy_version 15278 (0.0006) [2024-09-30 00:49:03,674][1157819] Updated weights for policy 0, policy_version 15288 (0.0006) [2024-09-30 00:49:04,190][1157819] Updated weights for policy 0, policy_version 15298 (0.0006) [2024-09-30 00:49:04,710][1157819] Updated weights for policy 0, policy_version 15308 (0.0006) [2024-09-30 00:49:05,295][1157819] Updated weights for policy 0, policy_version 15318 (0.0006) [2024-09-30 00:49:05,466][1157520] Fps is (10 sec: 77006.4, 60 sec: 77414.4, 300 sec: 77476.4). Total num frames: 62754816. Throughput: 0: 19241.2. Samples: 5677120. Policy #0 lag: (min: 0.0, avg: 2.5, max: 7.0) [2024-09-30 00:49:05,466][1157520] Avg episode reward: [(0, '42.841')] [2024-09-30 00:49:05,820][1157819] Updated weights for policy 0, policy_version 15328 (0.0006) [2024-09-30 00:49:06,431][1157819] Updated weights for policy 0, policy_version 15338 (0.0006) [2024-09-30 00:49:06,994][1157819] Updated weights for policy 0, policy_version 15348 (0.0006) [2024-09-30 00:49:07,574][1157819] Updated weights for policy 0, policy_version 15358 (0.0006) [2024-09-30 00:49:08,164][1157819] Updated weights for policy 0, policy_version 15368 (0.0006) [2024-09-30 00:49:08,728][1157819] Updated weights for policy 0, policy_version 15378 (0.0006) [2024-09-30 00:49:09,303][1157819] Updated weights for policy 0, policy_version 15388 (0.0006) [2024-09-30 00:49:09,869][1157819] Updated weights for policy 0, policy_version 15398 (0.0006) [2024-09-30 00:49:10,445][1157819] Updated weights for policy 0, policy_version 15408 (0.0006) [2024-09-30 00:49:10,466][1157520] Fps is (10 sec: 74957.2, 60 sec: 76663.5, 300 sec: 78310.0). Total num frames: 63111168. Throughput: 0: 19273.0. Samples: 5730408. Policy #0 lag: (min: 0.0, avg: 2.5, max: 7.0) [2024-09-30 00:49:10,466][1157520] Avg episode reward: [(0, '43.578')] [2024-09-30 00:49:11,022][1157819] Updated weights for policy 0, policy_version 15418 (0.0006) [2024-09-30 00:49:11,616][1157819] Updated weights for policy 0, policy_version 15428 (0.0006) [2024-09-30 00:49:12,185][1157819] Updated weights for policy 0, policy_version 15438 (0.0006) [2024-09-30 00:49:12,784][1157819] Updated weights for policy 0, policy_version 15448 (0.0006) [2024-09-30 00:49:13,341][1157819] Updated weights for policy 0, policy_version 15458 (0.0006) [2024-09-30 00:49:13,920][1157819] Updated weights for policy 0, policy_version 15468 (0.0006) [2024-09-30 00:49:14,451][1157819] Updated weights for policy 0, policy_version 15478 (0.0006) [2024-09-30 00:49:15,041][1157819] Updated weights for policy 0, policy_version 15488 (0.0006) [2024-09-30 00:49:15,466][1157520] Fps is (10 sec: 71679.6, 60 sec: 76049.0, 300 sec: 78615.4). Total num frames: 63471616. Throughput: 0: 19480.2. Samples: 5836684. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 00:49:15,466][1157520] Avg episode reward: [(0, '43.084')] [2024-09-30 00:49:15,559][1157819] Updated weights for policy 0, policy_version 15498 (0.0006) [2024-09-30 00:49:16,074][1157819] Updated weights for policy 0, policy_version 15508 (0.0006) [2024-09-30 00:49:16,654][1157819] Updated weights for policy 0, policy_version 15518 (0.0006) [2024-09-30 00:49:17,180][1157819] Updated weights for policy 0, policy_version 15528 (0.0006) [2024-09-30 00:49:17,760][1157819] Updated weights for policy 0, policy_version 15538 (0.0006) [2024-09-30 00:49:18,312][1157819] Updated weights for policy 0, policy_version 15548 (0.0006) [2024-09-30 00:49:18,835][1157819] Updated weights for policy 0, policy_version 15558 (0.0006) [2024-09-30 00:49:19,405][1157819] Updated weights for policy 0, policy_version 15568 (0.0006) [2024-09-30 00:49:19,908][1157819] Updated weights for policy 0, policy_version 15578 (0.0006) [2024-09-30 00:49:20,394][1157819] Updated weights for policy 0, policy_version 15588 (0.0006) [2024-09-30 00:49:20,466][1157520] Fps is (10 sec: 74137.7, 60 sec: 75912.7, 300 sec: 78643.3). Total num frames: 63852544. Throughput: 0: 19871.5. Samples: 5949772. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 00:49:20,466][1157520] Avg episode reward: [(0, '43.531')] [2024-09-30 00:49:20,905][1157819] Updated weights for policy 0, policy_version 15598 (0.0006) [2024-09-30 00:49:21,406][1157819] Updated weights for policy 0, policy_version 15608 (0.0006) [2024-09-30 00:49:21,935][1157819] Updated weights for policy 0, policy_version 15618 (0.0006) [2024-09-30 00:49:22,435][1157819] Updated weights for policy 0, policy_version 15628 (0.0006) [2024-09-30 00:49:22,941][1157819] Updated weights for policy 0, policy_version 15638 (0.0006) [2024-09-30 00:49:23,460][1157819] Updated weights for policy 0, policy_version 15648 (0.0006) [2024-09-30 00:49:23,968][1157819] Updated weights for policy 0, policy_version 15658 (0.0006) [2024-09-30 00:49:24,131][1157736] Signal inference workers to stop experience collection... (500 times) [2024-09-30 00:49:24,133][1157819] InferenceWorker_p0-w0: stopping experience collection (500 times) [2024-09-30 00:49:24,136][1157736] Signal inference workers to resume experience collection... (500 times) [2024-09-30 00:49:24,137][1157819] InferenceWorker_p0-w0: resuming experience collection (500 times) [2024-09-30 00:49:24,465][1157819] Updated weights for policy 0, policy_version 15668 (0.0006) [2024-09-30 00:49:24,963][1157819] Updated weights for policy 0, policy_version 15678 (0.0006) [2024-09-30 00:49:25,466][1157520] Fps is (10 sec: 78234.2, 60 sec: 77141.6, 300 sec: 78684.8). Total num frames: 64253952. Throughput: 0: 19970.4. Samples: 6010100. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 00:49:25,466][1157520] Avg episode reward: [(0, '43.516')] [2024-09-30 00:49:25,485][1157819] Updated weights for policy 0, policy_version 15688 (0.0006) [2024-09-30 00:49:25,980][1157819] Updated weights for policy 0, policy_version 15698 (0.0006) [2024-09-30 00:49:26,490][1157819] Updated weights for policy 0, policy_version 15708 (0.0006) [2024-09-30 00:49:27,027][1157819] Updated weights for policy 0, policy_version 15718 (0.0006) [2024-09-30 00:49:27,550][1157819] Updated weights for policy 0, policy_version 15728 (0.0006) [2024-09-30 00:49:28,041][1157819] Updated weights for policy 0, policy_version 15738 (0.0006) [2024-09-30 00:49:28,555][1157819] Updated weights for policy 0, policy_version 15748 (0.0006) [2024-09-30 00:49:29,040][1157819] Updated weights for policy 0, policy_version 15758 (0.0006) [2024-09-30 00:49:29,534][1157819] Updated weights for policy 0, policy_version 15768 (0.0006) [2024-09-30 00:49:30,030][1157819] Updated weights for policy 0, policy_version 15778 (0.0006) [2024-09-30 00:49:30,466][1157520] Fps is (10 sec: 80691.1, 60 sec: 78575.0, 300 sec: 78684.9). Total num frames: 64659456. Throughput: 0: 19562.7. Samples: 6130932. Policy #0 lag: (min: 0.0, avg: 3.1, max: 6.0) [2024-09-30 00:49:30,466][1157520] Avg episode reward: [(0, '43.151')] [2024-09-30 00:49:30,517][1157819] Updated weights for policy 0, policy_version 15788 (0.0006) [2024-09-30 00:49:31,013][1157819] Updated weights for policy 0, policy_version 15798 (0.0006) [2024-09-30 00:49:31,512][1157819] Updated weights for policy 0, policy_version 15808 (0.0006) [2024-09-30 00:49:32,051][1157819] Updated weights for policy 0, policy_version 15818 (0.0006) [2024-09-30 00:49:32,618][1157819] Updated weights for policy 0, policy_version 15828 (0.0006) [2024-09-30 00:49:33,141][1157819] Updated weights for policy 0, policy_version 15838 (0.0006) [2024-09-30 00:49:33,686][1157819] Updated weights for policy 0, policy_version 15848 (0.0006) [2024-09-30 00:49:34,246][1157819] Updated weights for policy 0, policy_version 15858 (0.0006) [2024-09-30 00:49:34,769][1157819] Updated weights for policy 0, policy_version 15868 (0.0006) [2024-09-30 00:49:35,357][1157819] Updated weights for policy 0, policy_version 15878 (0.0006) [2024-09-30 00:49:35,466][1157520] Fps is (10 sec: 79052.0, 60 sec: 79667.3, 300 sec: 78587.7). Total num frames: 65044480. Throughput: 0: 19143.7. Samples: 6248740. Policy #0 lag: (min: 0.0, avg: 3.1, max: 6.0) [2024-09-30 00:49:35,466][1157520] Avg episode reward: [(0, '44.759')] [2024-09-30 00:49:35,877][1157819] Updated weights for policy 0, policy_version 15888 (0.0006) [2024-09-30 00:49:36,449][1157819] Updated weights for policy 0, policy_version 15898 (0.0006) [2024-09-30 00:49:36,985][1157819] Updated weights for policy 0, policy_version 15908 (0.0006) [2024-09-30 00:49:37,522][1157819] Updated weights for policy 0, policy_version 15918 (0.0006) [2024-09-30 00:49:38,066][1157819] Updated weights for policy 0, policy_version 15928 (0.0006) [2024-09-30 00:49:38,627][1157819] Updated weights for policy 0, policy_version 15938 (0.0006) [2024-09-30 00:49:39,156][1157819] Updated weights for policy 0, policy_version 15948 (0.0006) [2024-09-30 00:49:39,719][1157819] Updated weights for policy 0, policy_version 15958 (0.0006) [2024-09-30 00:49:40,257][1157819] Updated weights for policy 0, policy_version 15968 (0.0006) [2024-09-30 00:49:40,466][1157520] Fps is (10 sec: 76184.8, 60 sec: 78779.7, 300 sec: 78601.5). Total num frames: 65421312. Throughput: 0: 19084.7. Samples: 6304788. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 00:49:40,466][1157520] Avg episode reward: [(0, '42.638')] [2024-09-30 00:49:40,776][1157819] Updated weights for policy 0, policy_version 15978 (0.0006) [2024-09-30 00:49:41,298][1157819] Updated weights for policy 0, policy_version 15988 (0.0006) [2024-09-30 00:49:41,871][1157819] Updated weights for policy 0, policy_version 15998 (0.0006) [2024-09-30 00:49:42,413][1157819] Updated weights for policy 0, policy_version 16008 (0.0006) [2024-09-30 00:49:42,970][1157819] Updated weights for policy 0, policy_version 16018 (0.0006) [2024-09-30 00:49:43,529][1157819] Updated weights for policy 0, policy_version 16028 (0.0006) [2024-09-30 00:49:44,062][1157819] Updated weights for policy 0, policy_version 16038 (0.0006) [2024-09-30 00:49:44,624][1157819] Updated weights for policy 0, policy_version 16048 (0.0006) [2024-09-30 00:49:45,177][1157819] Updated weights for policy 0, policy_version 16058 (0.0006) [2024-09-30 00:49:45,466][1157520] Fps is (10 sec: 74956.9, 60 sec: 77209.5, 300 sec: 78504.3). Total num frames: 65794048. Throughput: 0: 19086.4. Samples: 6417572. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 00:49:45,466][1157520] Avg episode reward: [(0, '44.103')] [2024-09-30 00:49:45,725][1157819] Updated weights for policy 0, policy_version 16068 (0.0006) [2024-09-30 00:49:45,732][1157736] Signal inference workers to stop experience collection... (550 times) [2024-09-30 00:49:45,732][1157736] Signal inference workers to resume experience collection... (550 times) [2024-09-30 00:49:45,738][1157819] InferenceWorker_p0-w0: stopping experience collection (550 times) [2024-09-30 00:49:45,738][1157819] InferenceWorker_p0-w0: resuming experience collection (550 times) [2024-09-30 00:49:46,254][1157819] Updated weights for policy 0, policy_version 16078 (0.0006) [2024-09-30 00:49:46,796][1157819] Updated weights for policy 0, policy_version 16088 (0.0006) [2024-09-30 00:49:47,322][1157819] Updated weights for policy 0, policy_version 16098 (0.0006) [2024-09-30 00:49:47,837][1157819] Updated weights for policy 0, policy_version 16108 (0.0006) [2024-09-30 00:49:48,352][1157819] Updated weights for policy 0, policy_version 16118 (0.0006) [2024-09-30 00:49:48,825][1157819] Updated weights for policy 0, policy_version 16128 (0.0006) [2024-09-30 00:49:49,315][1157819] Updated weights for policy 0, policy_version 16138 (0.0006) [2024-09-30 00:49:49,800][1157819] Updated weights for policy 0, policy_version 16148 (0.0006) [2024-09-30 00:49:50,264][1157819] Updated weights for policy 0, policy_version 16158 (0.0006) [2024-09-30 00:49:50,466][1157520] Fps is (10 sec: 77824.8, 60 sec: 76663.6, 300 sec: 78518.2). Total num frames: 66199552. Throughput: 0: 19073.8. Samples: 6535440. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 00:49:50,466][1157520] Avg episode reward: [(0, '43.500')] [2024-09-30 00:49:50,755][1157819] Updated weights for policy 0, policy_version 16168 (0.0006) [2024-09-30 00:49:51,252][1157819] Updated weights for policy 0, policy_version 16178 (0.0006) [2024-09-30 00:49:51,747][1157819] Updated weights for policy 0, policy_version 16188 (0.0006) [2024-09-30 00:49:52,250][1157819] Updated weights for policy 0, policy_version 16198 (0.0006) [2024-09-30 00:49:52,798][1157819] Updated weights for policy 0, policy_version 16208 (0.0006) [2024-09-30 00:49:53,302][1157819] Updated weights for policy 0, policy_version 16218 (0.0006) [2024-09-30 00:49:53,794][1157819] Updated weights for policy 0, policy_version 16228 (0.0006) [2024-09-30 00:49:54,288][1157819] Updated weights for policy 0, policy_version 16238 (0.0006) [2024-09-30 00:49:54,787][1157819] Updated weights for policy 0, policy_version 16248 (0.0006) [2024-09-30 00:49:55,321][1157819] Updated weights for policy 0, policy_version 16258 (0.0006) [2024-09-30 00:49:55,466][1157520] Fps is (10 sec: 80691.7, 60 sec: 76936.8, 300 sec: 78532.1). Total num frames: 66600960. Throughput: 0: 19259.6. Samples: 6597088. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 00:49:55,466][1157520] Avg episode reward: [(0, '44.696')] [2024-09-30 00:49:55,819][1157819] Updated weights for policy 0, policy_version 16268 (0.0006) [2024-09-30 00:49:56,350][1157819] Updated weights for policy 0, policy_version 16278 (0.0006) [2024-09-30 00:49:56,842][1157819] Updated weights for policy 0, policy_version 16288 (0.0006) [2024-09-30 00:49:57,373][1157819] Updated weights for policy 0, policy_version 16298 (0.0006) [2024-09-30 00:49:57,847][1157819] Updated weights for policy 0, policy_version 16308 (0.0006) [2024-09-30 00:49:58,334][1157819] Updated weights for policy 0, policy_version 16318 (0.0006) [2024-09-30 00:49:58,895][1157819] Updated weights for policy 0, policy_version 16328 (0.0006) [2024-09-30 00:49:59,393][1157819] Updated weights for policy 0, policy_version 16338 (0.0006) [2024-09-30 00:49:59,889][1157819] Updated weights for policy 0, policy_version 16348 (0.0006) [2024-09-30 00:50:00,405][1157819] Updated weights for policy 0, policy_version 16358 (0.0006) [2024-09-30 00:50:00,466][1157520] Fps is (10 sec: 80691.3, 60 sec: 77414.5, 300 sec: 78629.3). Total num frames: 67006464. Throughput: 0: 19582.9. Samples: 6717912. Policy #0 lag: (min: 0.0, avg: 1.7, max: 4.0) [2024-09-30 00:50:00,466][1157520] Avg episode reward: [(0, '47.742')] [2024-09-30 00:50:00,921][1157819] Updated weights for policy 0, policy_version 16368 (0.0006) [2024-09-30 00:50:01,421][1157819] Updated weights for policy 0, policy_version 16378 (0.0006) [2024-09-30 00:50:01,934][1157819] Updated weights for policy 0, policy_version 16388 (0.0006) [2024-09-30 00:50:02,468][1157819] Updated weights for policy 0, policy_version 16398 (0.0006) [2024-09-30 00:50:02,970][1157819] Updated weights for policy 0, policy_version 16408 (0.0006) [2024-09-30 00:50:03,458][1157819] Updated weights for policy 0, policy_version 16418 (0.0006) [2024-09-30 00:50:03,895][1157819] Updated weights for policy 0, policy_version 16428 (0.0006) [2024-09-30 00:50:04,380][1157819] Updated weights for policy 0, policy_version 16438 (0.0006) [2024-09-30 00:50:04,873][1157819] Updated weights for policy 0, policy_version 16448 (0.0006) [2024-09-30 00:50:05,355][1157819] Updated weights for policy 0, policy_version 16458 (0.0006) [2024-09-30 00:50:05,466][1157520] Fps is (10 sec: 81510.6, 60 sec: 77687.5, 300 sec: 78740.4). Total num frames: 67416064. Throughput: 0: 19796.0. Samples: 6840592. Policy #0 lag: (min: 0.0, avg: 1.7, max: 4.0) [2024-09-30 00:50:05,466][1157520] Avg episode reward: [(0, '47.343')] [2024-09-30 00:50:05,474][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000016460_67420160.pth... [2024-09-30 00:50:05,519][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000011838_48488448.pth [2024-09-30 00:50:05,992][1157819] Updated weights for policy 0, policy_version 16468 (0.0006) [2024-09-30 00:50:06,586][1157819] Updated weights for policy 0, policy_version 16478 (0.0006) [2024-09-30 00:50:07,131][1157819] Updated weights for policy 0, policy_version 16488 (0.0006) [2024-09-30 00:50:07,749][1157819] Updated weights for policy 0, policy_version 16498 (0.0006) [2024-09-30 00:50:08,283][1157819] Updated weights for policy 0, policy_version 16508 (0.0006) [2024-09-30 00:50:08,804][1157819] Updated weights for policy 0, policy_version 16518 (0.0006) [2024-09-30 00:50:09,337][1157819] Updated weights for policy 0, policy_version 16528 (0.0006) [2024-09-30 00:50:09,853][1157819] Updated weights for policy 0, policy_version 16538 (0.0006) [2024-09-30 00:50:10,398][1157819] Updated weights for policy 0, policy_version 16548 (0.0006) [2024-09-30 00:50:10,466][1157520] Fps is (10 sec: 77823.4, 60 sec: 77892.2, 300 sec: 78643.2). Total num frames: 67784704. Throughput: 0: 19667.3. Samples: 6895128. Policy #0 lag: (min: 0.0, avg: 1.7, max: 4.0) [2024-09-30 00:50:10,466][1157520] Avg episode reward: [(0, '46.049')] [2024-09-30 00:50:10,924][1157819] Updated weights for policy 0, policy_version 16558 (0.0006) [2024-09-30 00:50:11,446][1157819] Updated weights for policy 0, policy_version 16568 (0.0006) [2024-09-30 00:50:11,986][1157819] Updated weights for policy 0, policy_version 16578 (0.0006) [2024-09-30 00:50:12,481][1157819] Updated weights for policy 0, policy_version 16588 (0.0006) [2024-09-30 00:50:13,055][1157819] Updated weights for policy 0, policy_version 16598 (0.0006) [2024-09-30 00:50:13,588][1157819] Updated weights for policy 0, policy_version 16608 (0.0006) [2024-09-30 00:50:14,086][1157819] Updated weights for policy 0, policy_version 16618 (0.0006) [2024-09-30 00:50:14,593][1157819] Updated weights for policy 0, policy_version 16628 (0.0006) [2024-09-30 00:50:15,100][1157819] Updated weights for policy 0, policy_version 16638 (0.0006) [2024-09-30 00:50:15,466][1157520] Fps is (10 sec: 76184.3, 60 sec: 78438.3, 300 sec: 78712.6). Total num frames: 68177920. Throughput: 0: 19558.2. Samples: 7011056. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 00:50:15,466][1157520] Avg episode reward: [(0, '44.199')] [2024-09-30 00:50:15,620][1157819] Updated weights for policy 0, policy_version 16648 (0.0006) [2024-09-30 00:50:16,127][1157819] Updated weights for policy 0, policy_version 16658 (0.0006) [2024-09-30 00:50:16,624][1157819] Updated weights for policy 0, policy_version 16668 (0.0006) [2024-09-30 00:50:17,126][1157819] Updated weights for policy 0, policy_version 16678 (0.0006) [2024-09-30 00:50:17,619][1157819] Updated weights for policy 0, policy_version 16688 (0.0006) [2024-09-30 00:50:18,119][1157819] Updated weights for policy 0, policy_version 16698 (0.0006) [2024-09-30 00:50:18,649][1157819] Updated weights for policy 0, policy_version 16708 (0.0006) [2024-09-30 00:50:19,146][1157819] Updated weights for policy 0, policy_version 16718 (0.0006) [2024-09-30 00:50:19,670][1157819] Updated weights for policy 0, policy_version 16728 (0.0006) [2024-09-30 00:50:20,158][1157819] Updated weights for policy 0, policy_version 16738 (0.0006) [2024-09-30 00:50:20,466][1157520] Fps is (10 sec: 79872.4, 60 sec: 78848.0, 300 sec: 78809.8). Total num frames: 68583424. Throughput: 0: 19632.3. Samples: 7132192. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 00:50:20,466][1157520] Avg episode reward: [(0, '46.475')] [2024-09-30 00:50:20,677][1157819] Updated weights for policy 0, policy_version 16748 (0.0006) [2024-09-30 00:50:21,189][1157819] Updated weights for policy 0, policy_version 16758 (0.0006) [2024-09-30 00:50:21,685][1157819] Updated weights for policy 0, policy_version 16768 (0.0006) [2024-09-30 00:50:22,202][1157819] Updated weights for policy 0, policy_version 16778 (0.0006) [2024-09-30 00:50:22,721][1157819] Updated weights for policy 0, policy_version 16788 (0.0006) [2024-09-30 00:50:23,238][1157819] Updated weights for policy 0, policy_version 16798 (0.0006) [2024-09-30 00:50:23,736][1157819] Updated weights for policy 0, policy_version 16808 (0.0006) [2024-09-30 00:50:24,216][1157819] Updated weights for policy 0, policy_version 16818 (0.0006) [2024-09-30 00:50:24,705][1157819] Updated weights for policy 0, policy_version 16828 (0.0006) [2024-09-30 00:50:25,239][1157819] Updated weights for policy 0, policy_version 16838 (0.0006) [2024-09-30 00:50:25,466][1157520] Fps is (10 sec: 80282.1, 60 sec: 78779.6, 300 sec: 78865.4). Total num frames: 68980736. Throughput: 0: 19724.8. Samples: 7192404. Policy #0 lag: (min: 0.0, avg: 2.3, max: 4.0) [2024-09-30 00:50:25,466][1157520] Avg episode reward: [(0, '44.869')] [2024-09-30 00:50:25,781][1157819] Updated weights for policy 0, policy_version 16848 (0.0006) [2024-09-30 00:50:26,293][1157819] Updated weights for policy 0, policy_version 16858 (0.0006) [2024-09-30 00:50:26,843][1157819] Updated weights for policy 0, policy_version 16868 (0.0006) [2024-09-30 00:50:27,351][1157736] Signal inference workers to stop experience collection... (600 times) [2024-09-30 00:50:27,352][1157736] Signal inference workers to resume experience collection... (600 times) [2024-09-30 00:50:27,356][1157819] InferenceWorker_p0-w0: stopping experience collection (600 times) [2024-09-30 00:50:27,358][1157819] InferenceWorker_p0-w0: resuming experience collection (600 times) [2024-09-30 00:50:27,368][1157819] Updated weights for policy 0, policy_version 16878 (0.0006) [2024-09-30 00:50:27,898][1157819] Updated weights for policy 0, policy_version 16888 (0.0006) [2024-09-30 00:50:28,457][1157819] Updated weights for policy 0, policy_version 16898 (0.0006) [2024-09-30 00:50:29,014][1157819] Updated weights for policy 0, policy_version 16908 (0.0006) [2024-09-30 00:50:29,577][1157819] Updated weights for policy 0, policy_version 16918 (0.0006) [2024-09-30 00:50:30,110][1157819] Updated weights for policy 0, policy_version 16928 (0.0006) [2024-09-30 00:50:30,466][1157520] Fps is (10 sec: 77823.8, 60 sec: 78370.1, 300 sec: 78893.1). Total num frames: 69361664. Throughput: 0: 19820.6. Samples: 7309496. Policy #0 lag: (min: 0.0, avg: 2.3, max: 4.0) [2024-09-30 00:50:30,466][1157520] Avg episode reward: [(0, '49.606')] [2024-09-30 00:50:30,467][1157736] Saving new best policy, reward=49.606! [2024-09-30 00:50:30,643][1157819] Updated weights for policy 0, policy_version 16938 (0.0006) [2024-09-30 00:50:31,202][1157819] Updated weights for policy 0, policy_version 16948 (0.0006) [2024-09-30 00:50:31,779][1157819] Updated weights for policy 0, policy_version 16958 (0.0006) [2024-09-30 00:50:32,290][1157819] Updated weights for policy 0, policy_version 16968 (0.0006) [2024-09-30 00:50:32,835][1157819] Updated weights for policy 0, policy_version 16978 (0.0006) [2024-09-30 00:50:33,362][1157819] Updated weights for policy 0, policy_version 16988 (0.0006) [2024-09-30 00:50:33,889][1157819] Updated weights for policy 0, policy_version 16998 (0.0006) [2024-09-30 00:50:34,454][1157819] Updated weights for policy 0, policy_version 17008 (0.0006) [2024-09-30 00:50:34,971][1157819] Updated weights for policy 0, policy_version 17018 (0.0006) [2024-09-30 00:50:35,466][1157520] Fps is (10 sec: 75776.1, 60 sec: 78233.6, 300 sec: 78934.8). Total num frames: 69738496. Throughput: 0: 19716.1. Samples: 7422668. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 00:50:35,466][1157520] Avg episode reward: [(0, '44.025')] [2024-09-30 00:50:35,578][1157819] Updated weights for policy 0, policy_version 17028 (0.0006) [2024-09-30 00:50:36,097][1157819] Updated weights for policy 0, policy_version 17038 (0.0006) [2024-09-30 00:50:36,604][1157819] Updated weights for policy 0, policy_version 17048 (0.0006) [2024-09-30 00:50:37,178][1157819] Updated weights for policy 0, policy_version 17058 (0.0006) [2024-09-30 00:50:37,709][1157819] Updated weights for policy 0, policy_version 17068 (0.0006) [2024-09-30 00:50:38,224][1157819] Updated weights for policy 0, policy_version 17078 (0.0006) [2024-09-30 00:50:38,719][1157819] Updated weights for policy 0, policy_version 17088 (0.0006) [2024-09-30 00:50:39,262][1157819] Updated weights for policy 0, policy_version 17098 (0.0006) [2024-09-30 00:50:39,768][1157819] Updated weights for policy 0, policy_version 17108 (0.0006) [2024-09-30 00:50:40,290][1157819] Updated weights for policy 0, policy_version 17118 (0.0006) [2024-09-30 00:50:40,466][1157520] Fps is (10 sec: 76594.7, 60 sec: 78438.4, 300 sec: 78920.9). Total num frames: 70127616. Throughput: 0: 19600.1. Samples: 7479092. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 00:50:40,466][1157520] Avg episode reward: [(0, '45.646')] [2024-09-30 00:50:40,773][1157819] Updated weights for policy 0, policy_version 17128 (0.0006) [2024-09-30 00:50:41,281][1157819] Updated weights for policy 0, policy_version 17138 (0.0006) [2024-09-30 00:50:41,761][1157819] Updated weights for policy 0, policy_version 17148 (0.0006) [2024-09-30 00:50:42,274][1157819] Updated weights for policy 0, policy_version 17158 (0.0006) [2024-09-30 00:50:42,801][1157819] Updated weights for policy 0, policy_version 17168 (0.0006) [2024-09-30 00:50:43,361][1157819] Updated weights for policy 0, policy_version 17178 (0.0006) [2024-09-30 00:50:43,910][1157819] Updated weights for policy 0, policy_version 17188 (0.0006) [2024-09-30 00:50:44,430][1157819] Updated weights for policy 0, policy_version 17198 (0.0006) [2024-09-30 00:50:45,024][1157819] Updated weights for policy 0, policy_version 17208 (0.0006) [2024-09-30 00:50:45,466][1157520] Fps is (10 sec: 77414.4, 60 sec: 78643.2, 300 sec: 78907.0). Total num frames: 70512640. Throughput: 0: 19563.1. Samples: 7598252. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 00:50:45,466][1157520] Avg episode reward: [(0, '43.755')] [2024-09-30 00:50:45,536][1157819] Updated weights for policy 0, policy_version 17218 (0.0006) [2024-09-30 00:50:46,091][1157819] Updated weights for policy 0, policy_version 17228 (0.0006) [2024-09-30 00:50:46,663][1157819] Updated weights for policy 0, policy_version 17238 (0.0006) [2024-09-30 00:50:47,280][1157819] Updated weights for policy 0, policy_version 17248 (0.0006) [2024-09-30 00:50:47,765][1157819] Updated weights for policy 0, policy_version 17258 (0.0006) [2024-09-30 00:50:48,269][1157819] Updated weights for policy 0, policy_version 17268 (0.0006) [2024-09-30 00:50:48,775][1157819] Updated weights for policy 0, policy_version 17278 (0.0006) [2024-09-30 00:50:49,309][1157819] Updated weights for policy 0, policy_version 17288 (0.0006) [2024-09-30 00:50:49,825][1157819] Updated weights for policy 0, policy_version 17298 (0.0006) [2024-09-30 00:50:50,320][1157819] Updated weights for policy 0, policy_version 17308 (0.0006) [2024-09-30 00:50:50,466][1157520] Fps is (10 sec: 77414.5, 60 sec: 78370.0, 300 sec: 78934.8). Total num frames: 70901760. Throughput: 0: 19380.2. Samples: 7712704. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 00:50:50,466][1157520] Avg episode reward: [(0, '48.054')] [2024-09-30 00:50:50,846][1157819] Updated weights for policy 0, policy_version 17318 (0.0006) [2024-09-30 00:50:51,334][1157819] Updated weights for policy 0, policy_version 17328 (0.0006) [2024-09-30 00:50:51,852][1157819] Updated weights for policy 0, policy_version 17338 (0.0006) [2024-09-30 00:50:52,358][1157819] Updated weights for policy 0, policy_version 17348 (0.0006) [2024-09-30 00:50:52,871][1157819] Updated weights for policy 0, policy_version 17358 (0.0006) [2024-09-30 00:50:53,367][1157819] Updated weights for policy 0, policy_version 17368 (0.0006) [2024-09-30 00:50:53,870][1157819] Updated weights for policy 0, policy_version 17378 (0.0006) [2024-09-30 00:50:54,365][1157819] Updated weights for policy 0, policy_version 17388 (0.0006) [2024-09-30 00:50:54,369][1157736] Signal inference workers to stop experience collection... (650 times) [2024-09-30 00:50:54,370][1157736] Signal inference workers to resume experience collection... (650 times) [2024-09-30 00:50:54,373][1157819] InferenceWorker_p0-w0: stopping experience collection (650 times) [2024-09-30 00:50:54,376][1157819] InferenceWorker_p0-w0: resuming experience collection (650 times) [2024-09-30 00:50:54,843][1157819] Updated weights for policy 0, policy_version 17398 (0.0006) [2024-09-30 00:50:55,329][1157819] Updated weights for policy 0, policy_version 17408 (0.0006) [2024-09-30 00:50:55,466][1157520] Fps is (10 sec: 80281.7, 60 sec: 78574.9, 300 sec: 78907.0). Total num frames: 71315456. Throughput: 0: 19506.3. Samples: 7772912. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 00:50:55,466][1157520] Avg episode reward: [(0, '45.497')] [2024-09-30 00:50:55,780][1157819] Updated weights for policy 0, policy_version 17418 (0.0006) [2024-09-30 00:50:56,271][1157819] Updated weights for policy 0, policy_version 17428 (0.0006) [2024-09-30 00:50:56,764][1157819] Updated weights for policy 0, policy_version 17438 (0.0006) [2024-09-30 00:50:57,287][1157819] Updated weights for policy 0, policy_version 17448 (0.0006) [2024-09-30 00:50:57,809][1157819] Updated weights for policy 0, policy_version 17458 (0.0006) [2024-09-30 00:50:58,289][1157819] Updated weights for policy 0, policy_version 17468 (0.0006) [2024-09-30 00:50:58,797][1157819] Updated weights for policy 0, policy_version 17478 (0.0006) [2024-09-30 00:50:59,365][1157819] Updated weights for policy 0, policy_version 17488 (0.0006) [2024-09-30 00:50:59,907][1157819] Updated weights for policy 0, policy_version 17498 (0.0006) [2024-09-30 00:51:00,429][1157819] Updated weights for policy 0, policy_version 17508 (0.0006) [2024-09-30 00:51:00,466][1157520] Fps is (10 sec: 81101.2, 60 sec: 78438.3, 300 sec: 78740.4). Total num frames: 71712768. Throughput: 0: 19686.9. Samples: 7896964. Policy #0 lag: (min: 0.0, avg: 2.0, max: 4.0) [2024-09-30 00:51:00,466][1157520] Avg episode reward: [(0, '44.101')] [2024-09-30 00:51:00,949][1157819] Updated weights for policy 0, policy_version 17518 (0.0006) [2024-09-30 00:51:01,487][1157819] Updated weights for policy 0, policy_version 17528 (0.0006) [2024-09-30 00:51:02,042][1157819] Updated weights for policy 0, policy_version 17538 (0.0006) [2024-09-30 00:51:02,560][1157819] Updated weights for policy 0, policy_version 17548 (0.0006) [2024-09-30 00:51:03,142][1157819] Updated weights for policy 0, policy_version 17558 (0.0006) [2024-09-30 00:51:03,682][1157819] Updated weights for policy 0, policy_version 17568 (0.0006) [2024-09-30 00:51:04,188][1157819] Updated weights for policy 0, policy_version 17578 (0.0006) [2024-09-30 00:51:04,748][1157819] Updated weights for policy 0, policy_version 17588 (0.0006) [2024-09-30 00:51:05,302][1157819] Updated weights for policy 0, policy_version 17598 (0.0006) [2024-09-30 00:51:05,466][1157520] Fps is (10 sec: 77823.3, 60 sec: 77960.3, 300 sec: 78573.8). Total num frames: 72093696. Throughput: 0: 19521.7. Samples: 8010672. Policy #0 lag: (min: 0.0, avg: 2.0, max: 4.0) [2024-09-30 00:51:05,466][1157520] Avg episode reward: [(0, '44.319')] [2024-09-30 00:51:05,827][1157819] Updated weights for policy 0, policy_version 17608 (0.0006) [2024-09-30 00:51:06,338][1157819] Updated weights for policy 0, policy_version 17618 (0.0006) [2024-09-30 00:51:06,905][1157819] Updated weights for policy 0, policy_version 17628 (0.0006) [2024-09-30 00:51:07,417][1157819] Updated weights for policy 0, policy_version 17638 (0.0007) [2024-09-30 00:51:07,942][1157819] Updated weights for policy 0, policy_version 17648 (0.0007) [2024-09-30 00:51:08,502][1157819] Updated weights for policy 0, policy_version 17658 (0.0007) [2024-09-30 00:51:09,061][1157819] Updated weights for policy 0, policy_version 17668 (0.0006) [2024-09-30 00:51:09,618][1157819] Updated weights for policy 0, policy_version 17678 (0.0007) [2024-09-30 00:51:10,196][1157819] Updated weights for policy 0, policy_version 17688 (0.0007) [2024-09-30 00:51:10,466][1157520] Fps is (10 sec: 75365.8, 60 sec: 78028.7, 300 sec: 78462.7). Total num frames: 72466432. Throughput: 0: 19466.7. Samples: 8068408. Policy #0 lag: (min: 0.0, avg: 2.0, max: 4.0) [2024-09-30 00:51:10,466][1157520] Avg episode reward: [(0, '45.284')] [2024-09-30 00:51:10,754][1157819] Updated weights for policy 0, policy_version 17698 (0.0006) [2024-09-30 00:51:11,312][1157819] Updated weights for policy 0, policy_version 17708 (0.0006) [2024-09-30 00:51:11,858][1157819] Updated weights for policy 0, policy_version 17718 (0.0006) [2024-09-30 00:51:12,425][1157819] Updated weights for policy 0, policy_version 17728 (0.0006) [2024-09-30 00:51:12,954][1157819] Updated weights for policy 0, policy_version 17738 (0.0006) [2024-09-30 00:51:13,490][1157819] Updated weights for policy 0, policy_version 17748 (0.0006) [2024-09-30 00:51:14,067][1157819] Updated weights for policy 0, policy_version 17758 (0.0006) [2024-09-30 00:51:14,608][1157819] Updated weights for policy 0, policy_version 17768 (0.0006) [2024-09-30 00:51:15,138][1157819] Updated weights for policy 0, policy_version 17778 (0.0006) [2024-09-30 00:51:15,466][1157520] Fps is (10 sec: 74546.6, 60 sec: 77687.4, 300 sec: 78448.8). Total num frames: 72839168. Throughput: 0: 19331.6. Samples: 8179420. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 00:51:15,466][1157520] Avg episode reward: [(0, '45.566')] [2024-09-30 00:51:15,675][1157819] Updated weights for policy 0, policy_version 17788 (0.0006) [2024-09-30 00:51:16,233][1157819] Updated weights for policy 0, policy_version 17798 (0.0006) [2024-09-30 00:51:16,776][1157819] Updated weights for policy 0, policy_version 17808 (0.0006) [2024-09-30 00:51:17,338][1157819] Updated weights for policy 0, policy_version 17818 (0.0006) [2024-09-30 00:51:17,886][1157819] Updated weights for policy 0, policy_version 17828 (0.0007) [2024-09-30 00:51:18,422][1157819] Updated weights for policy 0, policy_version 17838 (0.0006) [2024-09-30 00:51:18,975][1157819] Updated weights for policy 0, policy_version 17848 (0.0006) [2024-09-30 00:51:19,527][1157819] Updated weights for policy 0, policy_version 17858 (0.0007) [2024-09-30 00:51:20,035][1157819] Updated weights for policy 0, policy_version 17868 (0.0006) [2024-09-30 00:51:20,466][1157520] Fps is (10 sec: 74547.8, 60 sec: 77141.3, 300 sec: 78573.8). Total num frames: 73211904. Throughput: 0: 19312.4. Samples: 8291724. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 00:51:20,466][1157520] Avg episode reward: [(0, '47.770')] [2024-09-30 00:51:20,591][1157819] Updated weights for policy 0, policy_version 17878 (0.0006) [2024-09-30 00:51:21,164][1157819] Updated weights for policy 0, policy_version 17888 (0.0006) [2024-09-30 00:51:21,669][1157819] Updated weights for policy 0, policy_version 17898 (0.0006) [2024-09-30 00:51:22,243][1157819] Updated weights for policy 0, policy_version 17908 (0.0006) [2024-09-30 00:51:22,770][1157819] Updated weights for policy 0, policy_version 17918 (0.0006) [2024-09-30 00:51:23,302][1157819] Updated weights for policy 0, policy_version 17928 (0.0006) [2024-09-30 00:51:23,851][1157819] Updated weights for policy 0, policy_version 17938 (0.0006) [2024-09-30 00:51:24,381][1157819] Updated weights for policy 0, policy_version 17948 (0.0006) [2024-09-30 00:51:24,910][1157819] Updated weights for policy 0, policy_version 17958 (0.0006) [2024-09-30 00:51:25,451][1157819] Updated weights for policy 0, policy_version 17968 (0.0006) [2024-09-30 00:51:25,466][1157520] Fps is (10 sec: 75776.8, 60 sec: 76936.5, 300 sec: 78740.4). Total num frames: 73596928. Throughput: 0: 19319.5. Samples: 8348468. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 00:51:25,466][1157520] Avg episode reward: [(0, '46.530')] [2024-09-30 00:51:26,037][1157819] Updated weights for policy 0, policy_version 17978 (0.0006) [2024-09-30 00:51:26,577][1157819] Updated weights for policy 0, policy_version 17988 (0.0006) [2024-09-30 00:51:27,113][1157819] Updated weights for policy 0, policy_version 17998 (0.0006) [2024-09-30 00:51:27,686][1157819] Updated weights for policy 0, policy_version 18008 (0.0006) [2024-09-30 00:51:28,182][1157819] Updated weights for policy 0, policy_version 18018 (0.0006) [2024-09-30 00:51:28,714][1157819] Updated weights for policy 0, policy_version 18028 (0.0006) [2024-09-30 00:51:29,207][1157819] Updated weights for policy 0, policy_version 18038 (0.0006) [2024-09-30 00:51:29,717][1157819] Updated weights for policy 0, policy_version 18048 (0.0006) [2024-09-30 00:51:30,230][1157819] Updated weights for policy 0, policy_version 18058 (0.0006) [2024-09-30 00:51:30,466][1157520] Fps is (10 sec: 77004.8, 60 sec: 77004.8, 300 sec: 78712.6). Total num frames: 73981952. Throughput: 0: 19202.2. Samples: 8462352. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 00:51:30,466][1157520] Avg episode reward: [(0, '46.224')] [2024-09-30 00:51:30,757][1157819] Updated weights for policy 0, policy_version 18068 (0.0006) [2024-09-30 00:51:31,253][1157819] Updated weights for policy 0, policy_version 18078 (0.0006) [2024-09-30 00:51:31,754][1157819] Updated weights for policy 0, policy_version 18088 (0.0006) [2024-09-30 00:51:32,255][1157819] Updated weights for policy 0, policy_version 18098 (0.0006) [2024-09-30 00:51:32,758][1157819] Updated weights for policy 0, policy_version 18108 (0.0006) [2024-09-30 00:51:33,289][1157819] Updated weights for policy 0, policy_version 18118 (0.0006) [2024-09-30 00:51:33,794][1157819] Updated weights for policy 0, policy_version 18128 (0.0006) [2024-09-30 00:51:34,313][1157819] Updated weights for policy 0, policy_version 18138 (0.0006) [2024-09-30 00:51:34,798][1157819] Updated weights for policy 0, policy_version 18148 (0.0006) [2024-09-30 00:51:35,302][1157819] Updated weights for policy 0, policy_version 18158 (0.0006) [2024-09-30 00:51:35,466][1157520] Fps is (10 sec: 78643.7, 60 sec: 77414.4, 300 sec: 78684.8). Total num frames: 74383360. Throughput: 0: 19342.2. Samples: 8583104. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 00:51:35,466][1157520] Avg episode reward: [(0, '44.787')] [2024-09-30 00:51:35,844][1157819] Updated weights for policy 0, policy_version 18168 (0.0006) [2024-09-30 00:51:36,401][1157819] Updated weights for policy 0, policy_version 18178 (0.0006) [2024-09-30 00:51:36,924][1157819] Updated weights for policy 0, policy_version 18188 (0.0006) [2024-09-30 00:51:37,462][1157819] Updated weights for policy 0, policy_version 18198 (0.0006) [2024-09-30 00:51:37,590][1157736] Signal inference workers to stop experience collection... (700 times) [2024-09-30 00:51:37,591][1157736] Signal inference workers to resume experience collection... (700 times) [2024-09-30 00:51:37,594][1157819] InferenceWorker_p0-w0: stopping experience collection (700 times) [2024-09-30 00:51:37,594][1157819] InferenceWorker_p0-w0: resuming experience collection (700 times) [2024-09-30 00:51:37,994][1157819] Updated weights for policy 0, policy_version 18208 (0.0006) [2024-09-30 00:51:38,558][1157819] Updated weights for policy 0, policy_version 18218 (0.0006) [2024-09-30 00:51:39,078][1157819] Updated weights for policy 0, policy_version 18228 (0.0006) [2024-09-30 00:51:39,643][1157819] Updated weights for policy 0, policy_version 18238 (0.0006) [2024-09-30 00:51:40,175][1157819] Updated weights for policy 0, policy_version 18248 (0.0006) [2024-09-30 00:51:40,466][1157520] Fps is (10 sec: 77823.9, 60 sec: 77209.7, 300 sec: 78504.3). Total num frames: 74760192. Throughput: 0: 19288.8. Samples: 8640908. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 00:51:40,466][1157520] Avg episode reward: [(0, '45.168')] [2024-09-30 00:51:40,701][1157819] Updated weights for policy 0, policy_version 18258 (0.0006) [2024-09-30 00:51:41,269][1157819] Updated weights for policy 0, policy_version 18268 (0.0006) [2024-09-30 00:51:41,796][1157819] Updated weights for policy 0, policy_version 18278 (0.0006) [2024-09-30 00:51:42,320][1157819] Updated weights for policy 0, policy_version 18288 (0.0006) [2024-09-30 00:51:42,894][1157819] Updated weights for policy 0, policy_version 18298 (0.0006) [2024-09-30 00:51:43,489][1157819] Updated weights for policy 0, policy_version 18308 (0.0006) [2024-09-30 00:51:44,107][1157819] Updated weights for policy 0, policy_version 18318 (0.0006) [2024-09-30 00:51:44,674][1157819] Updated weights for policy 0, policy_version 18328 (0.0006) [2024-09-30 00:51:45,332][1157819] Updated weights for policy 0, policy_version 18338 (0.0006) [2024-09-30 00:51:45,466][1157520] Fps is (10 sec: 73727.9, 60 sec: 76800.0, 300 sec: 78323.8). Total num frames: 75120640. Throughput: 0: 19008.8. Samples: 8752360. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 00:51:45,466][1157520] Avg episode reward: [(0, '45.133')] [2024-09-30 00:51:45,871][1157819] Updated weights for policy 0, policy_version 18348 (0.0006) [2024-09-30 00:51:46,465][1157819] Updated weights for policy 0, policy_version 18358 (0.0007) [2024-09-30 00:51:47,024][1157819] Updated weights for policy 0, policy_version 18368 (0.0006) [2024-09-30 00:51:47,590][1157819] Updated weights for policy 0, policy_version 18378 (0.0006) [2024-09-30 00:51:48,170][1157819] Updated weights for policy 0, policy_version 18388 (0.0006) [2024-09-30 00:51:48,722][1157819] Updated weights for policy 0, policy_version 18398 (0.0006) [2024-09-30 00:51:49,293][1157819] Updated weights for policy 0, policy_version 18408 (0.0006) [2024-09-30 00:51:49,827][1157819] Updated weights for policy 0, policy_version 18418 (0.0006) [2024-09-30 00:51:50,425][1157819] Updated weights for policy 0, policy_version 18428 (0.0006) [2024-09-30 00:51:50,466][1157520] Fps is (10 sec: 72499.3, 60 sec: 76390.5, 300 sec: 78101.7). Total num frames: 75485184. Throughput: 0: 18858.5. Samples: 8859304. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 00:51:50,466][1157520] Avg episode reward: [(0, '45.599')] [2024-09-30 00:51:50,959][1157819] Updated weights for policy 0, policy_version 18438 (0.0006) [2024-09-30 00:51:51,542][1157819] Updated weights for policy 0, policy_version 18448 (0.0006) [2024-09-30 00:51:52,137][1157819] Updated weights for policy 0, policy_version 18458 (0.0006) [2024-09-30 00:51:52,746][1157819] Updated weights for policy 0, policy_version 18468 (0.0006) [2024-09-30 00:51:53,346][1157819] Updated weights for policy 0, policy_version 18478 (0.0006) [2024-09-30 00:51:53,995][1157819] Updated weights for policy 0, policy_version 18488 (0.0006) [2024-09-30 00:51:54,605][1157819] Updated weights for policy 0, policy_version 18498 (0.0006) [2024-09-30 00:51:55,189][1157819] Updated weights for policy 0, policy_version 18508 (0.0006) [2024-09-30 00:51:55,466][1157520] Fps is (10 sec: 70450.0, 60 sec: 75161.4, 300 sec: 77851.7). Total num frames: 75825152. Throughput: 0: 18752.1. Samples: 8912256. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 00:51:55,466][1157520] Avg episode reward: [(0, '47.622')] [2024-09-30 00:51:55,797][1157819] Updated weights for policy 0, policy_version 18518 (0.0006) [2024-09-30 00:51:56,400][1157819] Updated weights for policy 0, policy_version 18528 (0.0006) [2024-09-30 00:51:57,019][1157819] Updated weights for policy 0, policy_version 18538 (0.0006) [2024-09-30 00:51:57,587][1157819] Updated weights for policy 0, policy_version 18548 (0.0006) [2024-09-30 00:51:58,184][1157819] Updated weights for policy 0, policy_version 18558 (0.0006) [2024-09-30 00:51:58,727][1157819] Updated weights for policy 0, policy_version 18568 (0.0006) [2024-09-30 00:51:59,273][1157819] Updated weights for policy 0, policy_version 18578 (0.0006) [2024-09-30 00:51:59,834][1157819] Updated weights for policy 0, policy_version 18588 (0.0006) [2024-09-30 00:52:00,364][1157819] Updated weights for policy 0, policy_version 18598 (0.0006) [2024-09-30 00:52:00,466][1157520] Fps is (10 sec: 69632.4, 60 sec: 74479.0, 300 sec: 77588.0). Total num frames: 76181504. Throughput: 0: 18564.3. Samples: 9014808. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 00:52:00,466][1157520] Avg episode reward: [(0, '46.813')] [2024-09-30 00:52:00,965][1157819] Updated weights for policy 0, policy_version 18608 (0.0006) [2024-09-30 00:52:01,501][1157819] Updated weights for policy 0, policy_version 18618 (0.0006) [2024-09-30 00:52:02,067][1157819] Updated weights for policy 0, policy_version 18628 (0.0006) [2024-09-30 00:52:02,618][1157819] Updated weights for policy 0, policy_version 18638 (0.0006) [2024-09-30 00:52:03,181][1157819] Updated weights for policy 0, policy_version 18648 (0.0006) [2024-09-30 00:52:03,737][1157819] Updated weights for policy 0, policy_version 18658 (0.0006) [2024-09-30 00:52:04,295][1157819] Updated weights for policy 0, policy_version 18668 (0.0006) [2024-09-30 00:52:04,854][1157819] Updated weights for policy 0, policy_version 18678 (0.0006) [2024-09-30 00:52:04,859][1157736] Signal inference workers to stop experience collection... (750 times) [2024-09-30 00:52:04,860][1157736] Signal inference workers to resume experience collection... (750 times) [2024-09-30 00:52:04,862][1157819] InferenceWorker_p0-w0: stopping experience collection (750 times) [2024-09-30 00:52:04,865][1157819] InferenceWorker_p0-w0: resuming experience collection (750 times) [2024-09-30 00:52:05,376][1157819] Updated weights for policy 0, policy_version 18688 (0.0006) [2024-09-30 00:52:05,466][1157520] Fps is (10 sec: 72500.8, 60 sec: 74274.3, 300 sec: 77351.9). Total num frames: 76550144. Throughput: 0: 18524.9. Samples: 9125344. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 00:52:05,466][1157520] Avg episode reward: [(0, '44.735')] [2024-09-30 00:52:05,469][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000018689_76550144.pth... [2024-09-30 00:52:05,535][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000014188_58114048.pth [2024-09-30 00:52:06,029][1157819] Updated weights for policy 0, policy_version 18698 (0.0006) [2024-09-30 00:52:06,557][1157819] Updated weights for policy 0, policy_version 18708 (0.0006) [2024-09-30 00:52:07,049][1157819] Updated weights for policy 0, policy_version 18718 (0.0006) [2024-09-30 00:52:07,571][1157819] Updated weights for policy 0, policy_version 18728 (0.0006) [2024-09-30 00:52:08,065][1157819] Updated weights for policy 0, policy_version 18738 (0.0006) [2024-09-30 00:52:08,573][1157819] Updated weights for policy 0, policy_version 18748 (0.0006) [2024-09-30 00:52:09,066][1157819] Updated weights for policy 0, policy_version 18758 (0.0006) [2024-09-30 00:52:09,572][1157819] Updated weights for policy 0, policy_version 18768 (0.0006) [2024-09-30 00:52:10,087][1157819] Updated weights for policy 0, policy_version 18778 (0.0006) [2024-09-30 00:52:10,466][1157520] Fps is (10 sec: 76184.5, 60 sec: 74615.5, 300 sec: 77268.6). Total num frames: 76943360. Throughput: 0: 18520.6. Samples: 9181896. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 00:52:10,466][1157520] Avg episode reward: [(0, '47.486')] [2024-09-30 00:52:10,613][1157819] Updated weights for policy 0, policy_version 18788 (0.0006) [2024-09-30 00:52:11,113][1157819] Updated weights for policy 0, policy_version 18798 (0.0006) [2024-09-30 00:52:11,623][1157819] Updated weights for policy 0, policy_version 18808 (0.0006) [2024-09-30 00:52:12,178][1157819] Updated weights for policy 0, policy_version 18818 (0.0006) [2024-09-30 00:52:12,682][1157819] Updated weights for policy 0, policy_version 18828 (0.0006) [2024-09-30 00:52:13,192][1157819] Updated weights for policy 0, policy_version 18838 (0.0006) [2024-09-30 00:52:13,681][1157819] Updated weights for policy 0, policy_version 18848 (0.0006) [2024-09-30 00:52:14,175][1157819] Updated weights for policy 0, policy_version 18858 (0.0006) [2024-09-30 00:52:14,670][1157819] Updated weights for policy 0, policy_version 18868 (0.0006) [2024-09-30 00:52:15,158][1157819] Updated weights for policy 0, policy_version 18878 (0.0006) [2024-09-30 00:52:15,466][1157520] Fps is (10 sec: 79872.0, 60 sec: 75161.9, 300 sec: 77254.7). Total num frames: 77348864. Throughput: 0: 18667.2. Samples: 9302376. Policy #0 lag: (min: 0.0, avg: 2.6, max: 7.0) [2024-09-30 00:52:15,466][1157520] Avg episode reward: [(0, '46.113')] [2024-09-30 00:52:15,644][1157819] Updated weights for policy 0, policy_version 18888 (0.0006) [2024-09-30 00:52:16,120][1157819] Updated weights for policy 0, policy_version 18898 (0.0006) [2024-09-30 00:52:16,607][1157819] Updated weights for policy 0, policy_version 18908 (0.0006) [2024-09-30 00:52:17,101][1157819] Updated weights for policy 0, policy_version 18918 (0.0006) [2024-09-30 00:52:17,558][1157819] Updated weights for policy 0, policy_version 18928 (0.0006) [2024-09-30 00:52:18,042][1157819] Updated weights for policy 0, policy_version 18938 (0.0006) [2024-09-30 00:52:18,532][1157819] Updated weights for policy 0, policy_version 18948 (0.0006) [2024-09-30 00:52:18,958][1157819] Updated weights for policy 0, policy_version 18958 (0.0006) [2024-09-30 00:52:19,446][1157819] Updated weights for policy 0, policy_version 18968 (0.0006) [2024-09-30 00:52:19,924][1157819] Updated weights for policy 0, policy_version 18978 (0.0006) [2024-09-30 00:52:20,370][1157819] Updated weights for policy 0, policy_version 18988 (0.0006) [2024-09-30 00:52:20,466][1157520] Fps is (10 sec: 83559.7, 60 sec: 76117.4, 300 sec: 77351.9). Total num frames: 77778944. Throughput: 0: 18835.4. Samples: 9430696. Policy #0 lag: (min: 0.0, avg: 2.6, max: 7.0) [2024-09-30 00:52:20,466][1157520] Avg episode reward: [(0, '50.122')] [2024-09-30 00:52:20,472][1157736] Saving new best policy, reward=50.122! [2024-09-30 00:52:20,831][1157819] Updated weights for policy 0, policy_version 18998 (0.0006) [2024-09-30 00:52:21,324][1157819] Updated weights for policy 0, policy_version 19008 (0.0006) [2024-09-30 00:52:21,819][1157819] Updated weights for policy 0, policy_version 19018 (0.0006) [2024-09-30 00:52:22,314][1157819] Updated weights for policy 0, policy_version 19028 (0.0006) [2024-09-30 00:52:22,811][1157819] Updated weights for policy 0, policy_version 19038 (0.0006) [2024-09-30 00:52:23,308][1157819] Updated weights for policy 0, policy_version 19048 (0.0006) [2024-09-30 00:52:23,806][1157819] Updated weights for policy 0, policy_version 19058 (0.0006) [2024-09-30 00:52:24,293][1157819] Updated weights for policy 0, policy_version 19068 (0.0006) [2024-09-30 00:52:24,782][1157819] Updated weights for policy 0, policy_version 19078 (0.0006) [2024-09-30 00:52:25,276][1157819] Updated weights for policy 0, policy_version 19088 (0.0006) [2024-09-30 00:52:25,466][1157520] Fps is (10 sec: 85197.1, 60 sec: 76731.9, 300 sec: 77421.3). Total num frames: 78200832. Throughput: 0: 18974.7. Samples: 9494768. Policy #0 lag: (min: 0.0, avg: 2.6, max: 7.0) [2024-09-30 00:52:25,466][1157520] Avg episode reward: [(0, '46.560')] [2024-09-30 00:52:25,764][1157819] Updated weights for policy 0, policy_version 19098 (0.0006) [2024-09-30 00:52:26,232][1157819] Updated weights for policy 0, policy_version 19108 (0.0006) [2024-09-30 00:52:26,734][1157819] Updated weights for policy 0, policy_version 19118 (0.0006) [2024-09-30 00:52:27,195][1157819] Updated weights for policy 0, policy_version 19128 (0.0006) [2024-09-30 00:52:27,677][1157819] Updated weights for policy 0, policy_version 19138 (0.0006) [2024-09-30 00:52:28,176][1157819] Updated weights for policy 0, policy_version 19148 (0.0006) [2024-09-30 00:52:28,664][1157819] Updated weights for policy 0, policy_version 19158 (0.0006) [2024-09-30 00:52:29,121][1157819] Updated weights for policy 0, policy_version 19168 (0.0006) [2024-09-30 00:52:29,613][1157819] Updated weights for policy 0, policy_version 19178 (0.0006) [2024-09-30 00:52:30,093][1157819] Updated weights for policy 0, policy_version 19188 (0.0006) [2024-09-30 00:52:30,466][1157520] Fps is (10 sec: 84377.2, 60 sec: 77346.2, 300 sec: 77532.4). Total num frames: 78622720. Throughput: 0: 19296.9. Samples: 9620720. Policy #0 lag: (min: 0.0, avg: 2.6, max: 7.0) [2024-09-30 00:52:30,466][1157520] Avg episode reward: [(0, '48.259')] [2024-09-30 00:52:30,583][1157819] Updated weights for policy 0, policy_version 19198 (0.0006) [2024-09-30 00:52:31,072][1157819] Updated weights for policy 0, policy_version 19208 (0.0006) [2024-09-30 00:52:31,540][1157819] Updated weights for policy 0, policy_version 19218 (0.0006) [2024-09-30 00:52:32,042][1157819] Updated weights for policy 0, policy_version 19228 (0.0006) [2024-09-30 00:52:32,540][1157819] Updated weights for policy 0, policy_version 19238 (0.0006) [2024-09-30 00:52:32,981][1157819] Updated weights for policy 0, policy_version 19248 (0.0006) [2024-09-30 00:52:33,479][1157819] Updated weights for policy 0, policy_version 19258 (0.0006) [2024-09-30 00:52:33,970][1157819] Updated weights for policy 0, policy_version 19268 (0.0006) [2024-09-30 00:52:34,450][1157819] Updated weights for policy 0, policy_version 19278 (0.0006) [2024-09-30 00:52:34,942][1157819] Updated weights for policy 0, policy_version 19288 (0.0006) [2024-09-30 00:52:35,421][1157819] Updated weights for policy 0, policy_version 19298 (0.0006) [2024-09-30 00:52:35,466][1157520] Fps is (10 sec: 84377.6, 60 sec: 77687.6, 300 sec: 77685.2). Total num frames: 79044608. Throughput: 0: 19740.7. Samples: 9747636. Policy #0 lag: (min: 0.0, avg: 2.6, max: 7.0) [2024-09-30 00:52:35,466][1157520] Avg episode reward: [(0, '48.174')] [2024-09-30 00:52:35,885][1157819] Updated weights for policy 0, policy_version 19308 (0.0006) [2024-09-30 00:52:36,384][1157819] Updated weights for policy 0, policy_version 19318 (0.0006) [2024-09-30 00:52:36,853][1157736] Signal inference workers to stop experience collection... (800 times) [2024-09-30 00:52:36,857][1157819] InferenceWorker_p0-w0: stopping experience collection (800 times) [2024-09-30 00:52:36,857][1157736] Signal inference workers to resume experience collection... (800 times) [2024-09-30 00:52:36,861][1157819] InferenceWorker_p0-w0: resuming experience collection (800 times) [2024-09-30 00:52:36,870][1157819] Updated weights for policy 0, policy_version 19328 (0.0006) [2024-09-30 00:52:37,357][1157819] Updated weights for policy 0, policy_version 19338 (0.0006) [2024-09-30 00:52:37,850][1157819] Updated weights for policy 0, policy_version 19348 (0.0006) [2024-09-30 00:52:38,340][1157819] Updated weights for policy 0, policy_version 19358 (0.0006) [2024-09-30 00:52:38,816][1157819] Updated weights for policy 0, policy_version 19368 (0.0006) [2024-09-30 00:52:39,290][1157819] Updated weights for policy 0, policy_version 19378 (0.0006) [2024-09-30 00:52:39,791][1157819] Updated weights for policy 0, policy_version 19388 (0.0006) [2024-09-30 00:52:40,284][1157819] Updated weights for policy 0, policy_version 19398 (0.0006) [2024-09-30 00:52:40,466][1157520] Fps is (10 sec: 84378.0, 60 sec: 78438.5, 300 sec: 77824.0). Total num frames: 79466496. Throughput: 0: 19972.7. Samples: 9811024. Policy #0 lag: (min: 0.0, avg: 2.6, max: 7.0) [2024-09-30 00:52:40,466][1157520] Avg episode reward: [(0, '50.632')] [2024-09-30 00:52:40,476][1157736] Saving new best policy, reward=50.632! [2024-09-30 00:52:40,761][1157819] Updated weights for policy 0, policy_version 19408 (0.0006) [2024-09-30 00:52:41,237][1157819] Updated weights for policy 0, policy_version 19418 (0.0006) [2024-09-30 00:52:41,735][1157819] Updated weights for policy 0, policy_version 19428 (0.0006) [2024-09-30 00:52:42,222][1157819] Updated weights for policy 0, policy_version 19438 (0.0006) [2024-09-30 00:52:42,696][1157819] Updated weights for policy 0, policy_version 19448 (0.0006) [2024-09-30 00:52:43,167][1157819] Updated weights for policy 0, policy_version 19458 (0.0006) [2024-09-30 00:52:43,655][1157819] Updated weights for policy 0, policy_version 19468 (0.0006) [2024-09-30 00:52:44,145][1157819] Updated weights for policy 0, policy_version 19478 (0.0006) [2024-09-30 00:52:44,627][1157819] Updated weights for policy 0, policy_version 19488 (0.0006) [2024-09-30 00:52:45,129][1157819] Updated weights for policy 0, policy_version 19498 (0.0006) [2024-09-30 00:52:45,466][1157520] Fps is (10 sec: 84787.0, 60 sec: 79530.8, 300 sec: 77935.1). Total num frames: 79892480. Throughput: 0: 20507.6. Samples: 9937652. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 00:52:45,466][1157520] Avg episode reward: [(0, '44.130')] [2024-09-30 00:52:45,611][1157819] Updated weights for policy 0, policy_version 19508 (0.0006) [2024-09-30 00:52:46,080][1157819] Updated weights for policy 0, policy_version 19518 (0.0006) [2024-09-30 00:52:46,565][1157819] Updated weights for policy 0, policy_version 19528 (0.0006) [2024-09-30 00:52:47,058][1157819] Updated weights for policy 0, policy_version 19538 (0.0006) [2024-09-30 00:52:47,526][1157819] Updated weights for policy 0, policy_version 19548 (0.0006) [2024-09-30 00:52:48,004][1157819] Updated weights for policy 0, policy_version 19558 (0.0006) [2024-09-30 00:52:48,499][1157819] Updated weights for policy 0, policy_version 19568 (0.0006) [2024-09-30 00:52:48,970][1157819] Updated weights for policy 0, policy_version 19578 (0.0006) [2024-09-30 00:52:49,462][1157819] Updated weights for policy 0, policy_version 19588 (0.0006) [2024-09-30 00:52:49,955][1157819] Updated weights for policy 0, policy_version 19598 (0.0006) [2024-09-30 00:52:50,437][1157819] Updated weights for policy 0, policy_version 19608 (0.0006) [2024-09-30 00:52:50,466][1157520] Fps is (10 sec: 84787.0, 60 sec: 80486.5, 300 sec: 78018.4). Total num frames: 80314368. Throughput: 0: 20868.7. Samples: 10064436. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 00:52:50,466][1157520] Avg episode reward: [(0, '46.753')] [2024-09-30 00:52:50,899][1157819] Updated weights for policy 0, policy_version 19618 (0.0006) [2024-09-30 00:52:51,377][1157819] Updated weights for policy 0, policy_version 19628 (0.0006) [2024-09-30 00:52:51,857][1157819] Updated weights for policy 0, policy_version 19638 (0.0006) [2024-09-30 00:52:52,339][1157819] Updated weights for policy 0, policy_version 19648 (0.0006) [2024-09-30 00:52:52,783][1157819] Updated weights for policy 0, policy_version 19658 (0.0006) [2024-09-30 00:52:53,261][1157819] Updated weights for policy 0, policy_version 19668 (0.0006) [2024-09-30 00:52:53,724][1157819] Updated weights for policy 0, policy_version 19678 (0.0006) [2024-09-30 00:52:54,194][1157819] Updated weights for policy 0, policy_version 19688 (0.0006) [2024-09-30 00:52:54,643][1157819] Updated weights for policy 0, policy_version 19698 (0.0006) [2024-09-30 00:52:55,124][1157819] Updated weights for policy 0, policy_version 19708 (0.0006) [2024-09-30 00:52:55,466][1157520] Fps is (10 sec: 86015.8, 60 sec: 82125.1, 300 sec: 78115.6). Total num frames: 80752640. Throughput: 0: 21061.2. Samples: 10129648. Policy #0 lag: (min: 0.0, avg: 3.2, max: 6.0) [2024-09-30 00:52:55,466][1157520] Avg episode reward: [(0, '46.389')] [2024-09-30 00:52:55,615][1157819] Updated weights for policy 0, policy_version 19718 (0.0006) [2024-09-30 00:52:56,109][1157819] Updated weights for policy 0, policy_version 19728 (0.0006) [2024-09-30 00:52:56,563][1157819] Updated weights for policy 0, policy_version 19738 (0.0006) [2024-09-30 00:52:57,048][1157819] Updated weights for policy 0, policy_version 19748 (0.0006) [2024-09-30 00:52:57,510][1157819] Updated weights for policy 0, policy_version 19758 (0.0006) [2024-09-30 00:52:57,992][1157819] Updated weights for policy 0, policy_version 19768 (0.0006) [2024-09-30 00:52:58,465][1157819] Updated weights for policy 0, policy_version 19778 (0.0006) [2024-09-30 00:52:58,966][1157819] Updated weights for policy 0, policy_version 19788 (0.0006) [2024-09-30 00:52:59,437][1157819] Updated weights for policy 0, policy_version 19798 (0.0006) [2024-09-30 00:52:59,937][1157819] Updated weights for policy 0, policy_version 19808 (0.0006) [2024-09-30 00:53:00,398][1157819] Updated weights for policy 0, policy_version 19818 (0.0006) [2024-09-30 00:53:00,466][1157520] Fps is (10 sec: 86425.8, 60 sec: 83285.4, 300 sec: 78198.9). Total num frames: 81178624. Throughput: 0: 21263.5. Samples: 10259232. Policy #0 lag: (min: 0.0, avg: 3.2, max: 6.0) [2024-09-30 00:53:00,466][1157520] Avg episode reward: [(0, '49.879')] [2024-09-30 00:53:00,897][1157819] Updated weights for policy 0, policy_version 19828 (0.0006) [2024-09-30 00:53:01,378][1157819] Updated weights for policy 0, policy_version 19838 (0.0006) [2024-09-30 00:53:01,846][1157819] Updated weights for policy 0, policy_version 19848 (0.0006) [2024-09-30 00:53:02,345][1157819] Updated weights for policy 0, policy_version 19858 (0.0006) [2024-09-30 00:53:02,818][1157819] Updated weights for policy 0, policy_version 19868 (0.0006) [2024-09-30 00:53:03,314][1157819] Updated weights for policy 0, policy_version 19878 (0.0006) [2024-09-30 00:53:03,794][1157819] Updated weights for policy 0, policy_version 19888 (0.0006) [2024-09-30 00:53:04,280][1157819] Updated weights for policy 0, policy_version 19898 (0.0006) [2024-09-30 00:53:04,763][1157819] Updated weights for policy 0, policy_version 19908 (0.0006) [2024-09-30 00:53:05,249][1157819] Updated weights for policy 0, policy_version 19918 (0.0006) [2024-09-30 00:53:05,466][1157520] Fps is (10 sec: 84787.5, 60 sec: 84172.8, 300 sec: 78268.3). Total num frames: 81600512. Throughput: 0: 21234.7. Samples: 10386256. Policy #0 lag: (min: 0.0, avg: 3.2, max: 6.0) [2024-09-30 00:53:05,466][1157520] Avg episode reward: [(0, '48.178')] [2024-09-30 00:53:05,728][1157819] Updated weights for policy 0, policy_version 19928 (0.0006) [2024-09-30 00:53:06,202][1157819] Updated weights for policy 0, policy_version 19938 (0.0006) [2024-09-30 00:53:06,679][1157819] Updated weights for policy 0, policy_version 19948 (0.0006) [2024-09-30 00:53:07,165][1157819] Updated weights for policy 0, policy_version 19958 (0.0006) [2024-09-30 00:53:07,661][1157819] Updated weights for policy 0, policy_version 19968 (0.0006) [2024-09-30 00:53:08,148][1157819] Updated weights for policy 0, policy_version 19978 (0.0006) [2024-09-30 00:53:08,630][1157819] Updated weights for policy 0, policy_version 19988 (0.0006) [2024-09-30 00:53:09,121][1157819] Updated weights for policy 0, policy_version 19998 (0.0006) [2024-09-30 00:53:09,612][1157819] Updated weights for policy 0, policy_version 20008 (0.0006) [2024-09-30 00:53:10,099][1157819] Updated weights for policy 0, policy_version 20018 (0.0006) [2024-09-30 00:53:10,466][1157520] Fps is (10 sec: 84786.5, 60 sec: 84719.0, 300 sec: 78365.5). Total num frames: 82026496. Throughput: 0: 21225.5. Samples: 10449916. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 00:53:10,466][1157520] Avg episode reward: [(0, '45.047')] [2024-09-30 00:53:10,546][1157819] Updated weights for policy 0, policy_version 20028 (0.0006) [2024-09-30 00:53:11,053][1157819] Updated weights for policy 0, policy_version 20038 (0.0006) [2024-09-30 00:53:11,547][1157819] Updated weights for policy 0, policy_version 20048 (0.0006) [2024-09-30 00:53:12,024][1157819] Updated weights for policy 0, policy_version 20058 (0.0006) [2024-09-30 00:53:12,531][1157819] Updated weights for policy 0, policy_version 20068 (0.0006) [2024-09-30 00:53:13,006][1157819] Updated weights for policy 0, policy_version 20078 (0.0006) [2024-09-30 00:53:13,504][1157819] Updated weights for policy 0, policy_version 20088 (0.0006) [2024-09-30 00:53:13,970][1157819] Updated weights for policy 0, policy_version 20098 (0.0006) [2024-09-30 00:53:14,441][1157819] Updated weights for policy 0, policy_version 20108 (0.0006) [2024-09-30 00:53:14,948][1157819] Updated weights for policy 0, policy_version 20118 (0.0006) [2024-09-30 00:53:15,442][1157819] Updated weights for policy 0, policy_version 20128 (0.0006) [2024-09-30 00:53:15,466][1157520] Fps is (10 sec: 84787.2, 60 sec: 84992.0, 300 sec: 78476.6). Total num frames: 82448384. Throughput: 0: 21228.3. Samples: 10575992. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 00:53:15,466][1157520] Avg episode reward: [(0, '47.635')] [2024-09-30 00:53:15,907][1157819] Updated weights for policy 0, policy_version 20138 (0.0006) [2024-09-30 00:53:16,359][1157819] Updated weights for policy 0, policy_version 20148 (0.0006) [2024-09-30 00:53:16,867][1157819] Updated weights for policy 0, policy_version 20158 (0.0006) [2024-09-30 00:53:17,353][1157819] Updated weights for policy 0, policy_version 20168 (0.0006) [2024-09-30 00:53:17,840][1157819] Updated weights for policy 0, policy_version 20178 (0.0006) [2024-09-30 00:53:18,345][1157819] Updated weights for policy 0, policy_version 20188 (0.0006) [2024-09-30 00:53:18,832][1157819] Updated weights for policy 0, policy_version 20198 (0.0006) [2024-09-30 00:53:19,333][1157819] Updated weights for policy 0, policy_version 20208 (0.0006) [2024-09-30 00:53:19,853][1157819] Updated weights for policy 0, policy_version 20218 (0.0006) [2024-09-30 00:53:20,370][1157819] Updated weights for policy 0, policy_version 20228 (0.0006) [2024-09-30 00:53:20,466][1157520] Fps is (10 sec: 83558.4, 60 sec: 84718.8, 300 sec: 78768.2). Total num frames: 82862080. Throughput: 0: 21206.6. Samples: 10701936. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 00:53:20,466][1157520] Avg episode reward: [(0, '47.378')] [2024-09-30 00:53:20,863][1157819] Updated weights for policy 0, policy_version 20238 (0.0006) [2024-09-30 00:53:21,372][1157819] Updated weights for policy 0, policy_version 20248 (0.0006) [2024-09-30 00:53:21,918][1157819] Updated weights for policy 0, policy_version 20258 (0.0006) [2024-09-30 00:53:22,441][1157819] Updated weights for policy 0, policy_version 20268 (0.0006) [2024-09-30 00:53:22,983][1157819] Updated weights for policy 0, policy_version 20278 (0.0006) [2024-09-30 00:53:23,577][1157819] Updated weights for policy 0, policy_version 20288 (0.0006) [2024-09-30 00:53:24,110][1157819] Updated weights for policy 0, policy_version 20298 (0.0006) [2024-09-30 00:53:24,644][1157819] Updated weights for policy 0, policy_version 20308 (0.0006) [2024-09-30 00:53:25,193][1157819] Updated weights for policy 0, policy_version 20318 (0.0006) [2024-09-30 00:53:25,466][1157520] Fps is (10 sec: 79461.6, 60 sec: 84036.1, 300 sec: 78976.4). Total num frames: 83243008. Throughput: 0: 21109.9. Samples: 10760972. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 00:53:25,466][1157520] Avg episode reward: [(0, '47.295')] [2024-09-30 00:53:25,719][1157819] Updated weights for policy 0, policy_version 20328 (0.0006) [2024-09-30 00:53:26,250][1157819] Updated weights for policy 0, policy_version 20338 (0.0006) [2024-09-30 00:53:26,803][1157819] Updated weights for policy 0, policy_version 20348 (0.0006) [2024-09-30 00:53:27,337][1157819] Updated weights for policy 0, policy_version 20358 (0.0006) [2024-09-30 00:53:27,860][1157819] Updated weights for policy 0, policy_version 20368 (0.0006) [2024-09-30 00:53:28,401][1157819] Updated weights for policy 0, policy_version 20378 (0.0006) [2024-09-30 00:53:28,944][1157819] Updated weights for policy 0, policy_version 20388 (0.0006) [2024-09-30 00:53:29,507][1157819] Updated weights for policy 0, policy_version 20398 (0.0006) [2024-09-30 00:53:30,042][1157819] Updated weights for policy 0, policy_version 20408 (0.0006) [2024-09-30 00:53:30,466][1157520] Fps is (10 sec: 75776.0, 60 sec: 83285.3, 300 sec: 79170.8). Total num frames: 83619840. Throughput: 0: 20812.1. Samples: 10874196. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 00:53:30,466][1157520] Avg episode reward: [(0, '45.957')] [2024-09-30 00:53:30,564][1157819] Updated weights for policy 0, policy_version 20418 (0.0006) [2024-09-30 00:53:31,136][1157819] Updated weights for policy 0, policy_version 20428 (0.0006) [2024-09-30 00:53:31,698][1157819] Updated weights for policy 0, policy_version 20438 (0.0006) [2024-09-30 00:53:32,078][1157736] Signal inference workers to stop experience collection... (850 times) [2024-09-30 00:53:32,078][1157736] Signal inference workers to resume experience collection... (850 times) [2024-09-30 00:53:32,084][1157819] InferenceWorker_p0-w0: stopping experience collection (850 times) [2024-09-30 00:53:32,084][1157819] InferenceWorker_p0-w0: resuming experience collection (850 times) [2024-09-30 00:53:32,236][1157819] Updated weights for policy 0, policy_version 20448 (0.0006) [2024-09-30 00:53:32,747][1157819] Updated weights for policy 0, policy_version 20458 (0.0006) [2024-09-30 00:53:33,280][1157819] Updated weights for policy 0, policy_version 20468 (0.0006) [2024-09-30 00:53:33,844][1157819] Updated weights for policy 0, policy_version 20478 (0.0006) [2024-09-30 00:53:34,386][1157819] Updated weights for policy 0, policy_version 20488 (0.0006) [2024-09-30 00:53:34,926][1157819] Updated weights for policy 0, policy_version 20498 (0.0006) [2024-09-30 00:53:35,466][1157520] Fps is (10 sec: 75366.2, 60 sec: 82534.2, 300 sec: 78990.3). Total num frames: 83996672. Throughput: 0: 20513.5. Samples: 10987548. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 00:53:35,466][1157520] Avg episode reward: [(0, '49.532')] [2024-09-30 00:53:35,475][1157819] Updated weights for policy 0, policy_version 20508 (0.0006) [2024-09-30 00:53:36,002][1157819] Updated weights for policy 0, policy_version 20518 (0.0006) [2024-09-30 00:53:36,540][1157819] Updated weights for policy 0, policy_version 20528 (0.0006) [2024-09-30 00:53:37,059][1157819] Updated weights for policy 0, policy_version 20538 (0.0006) [2024-09-30 00:53:37,602][1157819] Updated weights for policy 0, policy_version 20548 (0.0006) [2024-09-30 00:53:38,126][1157819] Updated weights for policy 0, policy_version 20558 (0.0006) [2024-09-30 00:53:38,662][1157819] Updated weights for policy 0, policy_version 20568 (0.0006) [2024-09-30 00:53:39,221][1157819] Updated weights for policy 0, policy_version 20578 (0.0006) [2024-09-30 00:53:39,729][1157819] Updated weights for policy 0, policy_version 20588 (0.0006) [2024-09-30 00:53:40,266][1157819] Updated weights for policy 0, policy_version 20598 (0.0006) [2024-09-30 00:53:40,466][1157520] Fps is (10 sec: 76185.4, 60 sec: 81919.9, 300 sec: 78712.6). Total num frames: 84381696. Throughput: 0: 20337.4. Samples: 11044832. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 00:53:40,466][1157520] Avg episode reward: [(0, '46.519')] [2024-09-30 00:53:40,835][1157819] Updated weights for policy 0, policy_version 20608 (0.0006) [2024-09-30 00:53:41,358][1157819] Updated weights for policy 0, policy_version 20618 (0.0006) [2024-09-30 00:53:41,866][1157819] Updated weights for policy 0, policy_version 20628 (0.0006) [2024-09-30 00:53:42,442][1157819] Updated weights for policy 0, policy_version 20638 (0.0006) [2024-09-30 00:53:42,975][1157819] Updated weights for policy 0, policy_version 20648 (0.0006) [2024-09-30 00:53:43,540][1157819] Updated weights for policy 0, policy_version 20658 (0.0006) [2024-09-30 00:53:44,101][1157819] Updated weights for policy 0, policy_version 20668 (0.0006) [2024-09-30 00:53:44,613][1157819] Updated weights for policy 0, policy_version 20678 (0.0006) [2024-09-30 00:53:45,151][1157819] Updated weights for policy 0, policy_version 20688 (0.0006) [2024-09-30 00:53:45,466][1157520] Fps is (10 sec: 76595.7, 60 sec: 81169.0, 300 sec: 78518.3). Total num frames: 84762624. Throughput: 0: 19985.2. Samples: 11158568. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 00:53:45,466][1157520] Avg episode reward: [(0, '48.575')] [2024-09-30 00:53:45,712][1157819] Updated weights for policy 0, policy_version 20698 (0.0006) [2024-09-30 00:53:46,239][1157819] Updated weights for policy 0, policy_version 20708 (0.0006) [2024-09-30 00:53:46,790][1157819] Updated weights for policy 0, policy_version 20718 (0.0006) [2024-09-30 00:53:47,325][1157819] Updated weights for policy 0, policy_version 20728 (0.0006) [2024-09-30 00:53:47,868][1157819] Updated weights for policy 0, policy_version 20738 (0.0006) [2024-09-30 00:53:48,431][1157819] Updated weights for policy 0, policy_version 20748 (0.0006) [2024-09-30 00:53:48,960][1157819] Updated weights for policy 0, policy_version 20758 (0.0006) [2024-09-30 00:53:49,494][1157819] Updated weights for policy 0, policy_version 20768 (0.0006) [2024-09-30 00:53:50,042][1157819] Updated weights for policy 0, policy_version 20778 (0.0006) [2024-09-30 00:53:50,466][1157520] Fps is (10 sec: 75366.5, 60 sec: 80349.8, 300 sec: 78476.6). Total num frames: 85135360. Throughput: 0: 19676.5. Samples: 11271700. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 00:53:50,466][1157520] Avg episode reward: [(0, '45.631')] [2024-09-30 00:53:50,582][1157819] Updated weights for policy 0, policy_version 20788 (0.0006) [2024-09-30 00:53:51,135][1157819] Updated weights for policy 0, policy_version 20798 (0.0006) [2024-09-30 00:53:51,676][1157819] Updated weights for policy 0, policy_version 20808 (0.0006) [2024-09-30 00:53:52,217][1157819] Updated weights for policy 0, policy_version 20818 (0.0006) [2024-09-30 00:53:52,753][1157819] Updated weights for policy 0, policy_version 20828 (0.0006) [2024-09-30 00:53:53,310][1157819] Updated weights for policy 0, policy_version 20838 (0.0006) [2024-09-30 00:53:53,882][1157819] Updated weights for policy 0, policy_version 20848 (0.0006) [2024-09-30 00:53:54,426][1157819] Updated weights for policy 0, policy_version 20858 (0.0006) [2024-09-30 00:53:54,959][1157819] Updated weights for policy 0, policy_version 20868 (0.0006) [2024-09-30 00:53:55,466][1157520] Fps is (10 sec: 74956.3, 60 sec: 79325.7, 300 sec: 78476.6). Total num frames: 85512192. Throughput: 0: 19517.5. Samples: 11328204. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 00:53:55,466][1157520] Avg episode reward: [(0, '48.792')] [2024-09-30 00:53:55,481][1157819] Updated weights for policy 0, policy_version 20878 (0.0006) [2024-09-30 00:53:56,036][1157819] Updated weights for policy 0, policy_version 20888 (0.0006) [2024-09-30 00:53:56,575][1157819] Updated weights for policy 0, policy_version 20898 (0.0006) [2024-09-30 00:53:57,125][1157819] Updated weights for policy 0, policy_version 20908 (0.0006) [2024-09-30 00:53:57,667][1157819] Updated weights for policy 0, policy_version 20918 (0.0006) [2024-09-30 00:53:58,244][1157819] Updated weights for policy 0, policy_version 20928 (0.0006) [2024-09-30 00:53:58,797][1157819] Updated weights for policy 0, policy_version 20938 (0.0006) [2024-09-30 00:53:59,328][1157819] Updated weights for policy 0, policy_version 20948 (0.0006) [2024-09-30 00:53:59,879][1157819] Updated weights for policy 0, policy_version 20958 (0.0006) [2024-09-30 00:54:00,449][1157819] Updated weights for policy 0, policy_version 20968 (0.0006) [2024-09-30 00:54:00,466][1157520] Fps is (10 sec: 74956.7, 60 sec: 78438.3, 300 sec: 78407.1). Total num frames: 85884928. Throughput: 0: 19213.1. Samples: 11440584. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 00:54:00,466][1157520] Avg episode reward: [(0, '50.332')] [2024-09-30 00:54:00,998][1157819] Updated weights for policy 0, policy_version 20978 (0.0006) [2024-09-30 00:54:01,564][1157819] Updated weights for policy 0, policy_version 20988 (0.0006) [2024-09-30 00:54:02,115][1157819] Updated weights for policy 0, policy_version 20998 (0.0006) [2024-09-30 00:54:02,686][1157819] Updated weights for policy 0, policy_version 21008 (0.0006) [2024-09-30 00:54:03,220][1157819] Updated weights for policy 0, policy_version 21018 (0.0006) [2024-09-30 00:54:03,745][1157819] Updated weights for policy 0, policy_version 21028 (0.0006) [2024-09-30 00:54:04,308][1157819] Updated weights for policy 0, policy_version 21038 (0.0006) [2024-09-30 00:54:04,875][1157819] Updated weights for policy 0, policy_version 21048 (0.0006) [2024-09-30 00:54:05,404][1157819] Updated weights for policy 0, policy_version 21058 (0.0006) [2024-09-30 00:54:05,466][1157520] Fps is (10 sec: 74546.7, 60 sec: 77619.0, 300 sec: 78462.7). Total num frames: 86257664. Throughput: 0: 18884.1. Samples: 11551724. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 00:54:05,466][1157520] Avg episode reward: [(0, '47.889')] [2024-09-30 00:54:05,485][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000021060_86261760.pth... [2024-09-30 00:54:05,532][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000016460_67420160.pth [2024-09-30 00:54:05,962][1157819] Updated weights for policy 0, policy_version 21068 (0.0006) [2024-09-30 00:54:06,498][1157819] Updated weights for policy 0, policy_version 21078 (0.0006) [2024-09-30 00:54:07,040][1157819] Updated weights for policy 0, policy_version 21088 (0.0006) [2024-09-30 00:54:07,603][1157819] Updated weights for policy 0, policy_version 21098 (0.0006) [2024-09-30 00:54:08,109][1157819] Updated weights for policy 0, policy_version 21108 (0.0006) [2024-09-30 00:54:08,641][1157819] Updated weights for policy 0, policy_version 21118 (0.0006) [2024-09-30 00:54:09,165][1157819] Updated weights for policy 0, policy_version 21128 (0.0006) [2024-09-30 00:54:09,720][1157819] Updated weights for policy 0, policy_version 21138 (0.0006) [2024-09-30 00:54:10,259][1157819] Updated weights for policy 0, policy_version 21148 (0.0006) [2024-09-30 00:54:10,466][1157520] Fps is (10 sec: 75366.1, 60 sec: 76868.2, 300 sec: 78532.1). Total num frames: 86638592. Throughput: 0: 18827.7. Samples: 11608220. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 00:54:10,466][1157520] Avg episode reward: [(0, '44.165')] [2024-09-30 00:54:10,790][1157819] Updated weights for policy 0, policy_version 21158 (0.0006) [2024-09-30 00:54:11,286][1157819] Updated weights for policy 0, policy_version 21168 (0.0006) [2024-09-30 00:54:11,806][1157819] Updated weights for policy 0, policy_version 21178 (0.0006) [2024-09-30 00:54:12,331][1157819] Updated weights for policy 0, policy_version 21188 (0.0006) [2024-09-30 00:54:12,875][1157819] Updated weights for policy 0, policy_version 21198 (0.0006) [2024-09-30 00:54:13,411][1157819] Updated weights for policy 0, policy_version 21208 (0.0006) [2024-09-30 00:54:13,949][1157819] Updated weights for policy 0, policy_version 21218 (0.0006) [2024-09-30 00:54:14,496][1157819] Updated weights for policy 0, policy_version 21228 (0.0006) [2024-09-30 00:54:15,005][1157819] Updated weights for policy 0, policy_version 21238 (0.0006) [2024-09-30 00:54:15,466][1157520] Fps is (10 sec: 76595.8, 60 sec: 76253.7, 300 sec: 78546.0). Total num frames: 87023616. Throughput: 0: 18884.5. Samples: 11724000. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 00:54:15,466][1157520] Avg episode reward: [(0, '47.728')] [2024-09-30 00:54:15,528][1157819] Updated weights for policy 0, policy_version 21248 (0.0006) [2024-09-30 00:54:16,116][1157819] Updated weights for policy 0, policy_version 21258 (0.0006) [2024-09-30 00:54:16,617][1157819] Updated weights for policy 0, policy_version 21268 (0.0006) [2024-09-30 00:54:17,120][1157819] Updated weights for policy 0, policy_version 21278 (0.0006) [2024-09-30 00:54:17,614][1157819] Updated weights for policy 0, policy_version 21288 (0.0006) [2024-09-30 00:54:18,109][1157819] Updated weights for policy 0, policy_version 21298 (0.0006) [2024-09-30 00:54:18,605][1157819] Updated weights for policy 0, policy_version 21308 (0.0006) [2024-09-30 00:54:19,120][1157819] Updated weights for policy 0, policy_version 21318 (0.0006) [2024-09-30 00:54:19,661][1157819] Updated weights for policy 0, policy_version 21328 (0.0006) [2024-09-30 00:54:20,209][1157819] Updated weights for policy 0, policy_version 21338 (0.0006) [2024-09-30 00:54:20,466][1157520] Fps is (10 sec: 77824.1, 60 sec: 75912.5, 300 sec: 78518.2). Total num frames: 87416832. Throughput: 0: 18991.9. Samples: 11842184. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 00:54:20,466][1157520] Avg episode reward: [(0, '47.712')] [2024-09-30 00:54:20,713][1157819] Updated weights for policy 0, policy_version 21348 (0.0006) [2024-09-30 00:54:21,227][1157819] Updated weights for policy 0, policy_version 21358 (0.0006) [2024-09-30 00:54:21,742][1157819] Updated weights for policy 0, policy_version 21368 (0.0006) [2024-09-30 00:54:22,302][1157819] Updated weights for policy 0, policy_version 21378 (0.0006) [2024-09-30 00:54:22,825][1157819] Updated weights for policy 0, policy_version 21388 (0.0006) [2024-09-30 00:54:23,408][1157819] Updated weights for policy 0, policy_version 21398 (0.0006) [2024-09-30 00:54:23,953][1157819] Updated weights for policy 0, policy_version 21408 (0.0006) [2024-09-30 00:54:24,507][1157819] Updated weights for policy 0, policy_version 21418 (0.0006) [2024-09-30 00:54:25,085][1157819] Updated weights for policy 0, policy_version 21428 (0.0007) [2024-09-30 00:54:25,466][1157520] Fps is (10 sec: 77004.8, 60 sec: 75844.3, 300 sec: 78421.0). Total num frames: 87793664. Throughput: 0: 19007.9. Samples: 11900188. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 00:54:25,466][1157520] Avg episode reward: [(0, '49.496')] [2024-09-30 00:54:25,676][1157819] Updated weights for policy 0, policy_version 21438 (0.0006) [2024-09-30 00:54:26,247][1157819] Updated weights for policy 0, policy_version 21448 (0.0006) [2024-09-30 00:54:26,839][1157819] Updated weights for policy 0, policy_version 21458 (0.0006) [2024-09-30 00:54:27,386][1157819] Updated weights for policy 0, policy_version 21468 (0.0006) [2024-09-30 00:54:27,958][1157819] Updated weights for policy 0, policy_version 21478 (0.0006) [2024-09-30 00:54:28,510][1157819] Updated weights for policy 0, policy_version 21488 (0.0006) [2024-09-30 00:54:28,794][1157736] Signal inference workers to stop experience collection... (900 times) [2024-09-30 00:54:28,797][1157819] InferenceWorker_p0-w0: stopping experience collection (900 times) [2024-09-30 00:54:28,803][1157736] Signal inference workers to resume experience collection... (900 times) [2024-09-30 00:54:28,803][1157819] InferenceWorker_p0-w0: resuming experience collection (900 times) [2024-09-30 00:54:29,074][1157819] Updated weights for policy 0, policy_version 21498 (0.0006) [2024-09-30 00:54:29,619][1157819] Updated weights for policy 0, policy_version 21508 (0.0006) [2024-09-30 00:54:30,176][1157819] Updated weights for policy 0, policy_version 21518 (0.0006) [2024-09-30 00:54:30,466][1157520] Fps is (10 sec: 74137.9, 60 sec: 75639.5, 300 sec: 78351.6). Total num frames: 88158208. Throughput: 0: 18899.7. Samples: 12009056. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 00:54:30,466][1157520] Avg episode reward: [(0, '47.688')] [2024-09-30 00:54:30,727][1157819] Updated weights for policy 0, policy_version 21528 (0.0006) [2024-09-30 00:54:31,244][1157819] Updated weights for policy 0, policy_version 21538 (0.0006) [2024-09-30 00:54:31,798][1157819] Updated weights for policy 0, policy_version 21548 (0.0006) [2024-09-30 00:54:32,364][1157819] Updated weights for policy 0, policy_version 21558 (0.0006) [2024-09-30 00:54:32,901][1157819] Updated weights for policy 0, policy_version 21568 (0.0006) [2024-09-30 00:54:33,458][1157819] Updated weights for policy 0, policy_version 21578 (0.0006) [2024-09-30 00:54:34,060][1157819] Updated weights for policy 0, policy_version 21588 (0.0006) [2024-09-30 00:54:34,663][1157819] Updated weights for policy 0, policy_version 21598 (0.0006) [2024-09-30 00:54:35,260][1157819] Updated weights for policy 0, policy_version 21608 (0.0006) [2024-09-30 00:54:35,466][1157520] Fps is (10 sec: 72499.1, 60 sec: 75366.4, 300 sec: 78296.1). Total num frames: 88518656. Throughput: 0: 18810.5. Samples: 12118172. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 00:54:35,466][1157520] Avg episode reward: [(0, '47.091')] [2024-09-30 00:54:35,855][1157819] Updated weights for policy 0, policy_version 21618 (0.0006) [2024-09-30 00:54:36,453][1157819] Updated weights for policy 0, policy_version 21628 (0.0006) [2024-09-30 00:54:37,042][1157819] Updated weights for policy 0, policy_version 21638 (0.0006) [2024-09-30 00:54:37,618][1157819] Updated weights for policy 0, policy_version 21648 (0.0006) [2024-09-30 00:54:38,206][1157819] Updated weights for policy 0, policy_version 21658 (0.0006) [2024-09-30 00:54:38,777][1157819] Updated weights for policy 0, policy_version 21668 (0.0006) [2024-09-30 00:54:39,337][1157819] Updated weights for policy 0, policy_version 21678 (0.0006) [2024-09-30 00:54:39,933][1157819] Updated weights for policy 0, policy_version 21688 (0.0006) [2024-09-30 00:54:40,466][1157520] Fps is (10 sec: 71270.4, 60 sec: 74820.3, 300 sec: 78226.7). Total num frames: 88870912. Throughput: 0: 18716.5. Samples: 12170444. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 00:54:40,466][1157520] Avg episode reward: [(0, '47.416')] [2024-09-30 00:54:40,500][1157819] Updated weights for policy 0, policy_version 21698 (0.0006) [2024-09-30 00:54:41,091][1157819] Updated weights for policy 0, policy_version 21708 (0.0006) [2024-09-30 00:54:41,640][1157819] Updated weights for policy 0, policy_version 21718 (0.0006) [2024-09-30 00:54:42,185][1157819] Updated weights for policy 0, policy_version 21728 (0.0006) [2024-09-30 00:54:42,785][1157819] Updated weights for policy 0, policy_version 21738 (0.0006) [2024-09-30 00:54:43,306][1157819] Updated weights for policy 0, policy_version 21748 (0.0006) [2024-09-30 00:54:43,882][1157819] Updated weights for policy 0, policy_version 21758 (0.0006) [2024-09-30 00:54:44,451][1157819] Updated weights for policy 0, policy_version 21768 (0.0006) [2024-09-30 00:54:44,975][1157819] Updated weights for policy 0, policy_version 21778 (0.0006) [2024-09-30 00:54:45,466][1157520] Fps is (10 sec: 72089.6, 60 sec: 74615.4, 300 sec: 78101.7). Total num frames: 89239552. Throughput: 0: 18623.1. Samples: 12278624. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 00:54:45,466][1157520] Avg episode reward: [(0, '46.928')] [2024-09-30 00:54:45,563][1157819] Updated weights for policy 0, policy_version 21788 (0.0006) [2024-09-30 00:54:46,158][1157819] Updated weights for policy 0, policy_version 21798 (0.0006) [2024-09-30 00:54:46,745][1157819] Updated weights for policy 0, policy_version 21808 (0.0006) [2024-09-30 00:54:47,336][1157819] Updated weights for policy 0, policy_version 21818 (0.0006) [2024-09-30 00:54:47,904][1157819] Updated weights for policy 0, policy_version 21828 (0.0006) [2024-09-30 00:54:48,543][1157819] Updated weights for policy 0, policy_version 21838 (0.0006) [2024-09-30 00:54:49,103][1157819] Updated weights for policy 0, policy_version 21848 (0.0006) [2024-09-30 00:54:49,729][1157819] Updated weights for policy 0, policy_version 21858 (0.0006) [2024-09-30 00:54:50,271][1157819] Updated weights for policy 0, policy_version 21868 (0.0006) [2024-09-30 00:54:50,466][1157520] Fps is (10 sec: 71270.8, 60 sec: 74137.7, 300 sec: 77907.3). Total num frames: 89583616. Throughput: 0: 18505.4. Samples: 12384464. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 00:54:50,466][1157520] Avg episode reward: [(0, '47.970')] [2024-09-30 00:54:50,697][1157736] Signal inference workers to stop experience collection... (950 times) [2024-09-30 00:54:50,701][1157819] InferenceWorker_p0-w0: stopping experience collection (950 times) [2024-09-30 00:54:50,703][1157736] Signal inference workers to resume experience collection... (950 times) [2024-09-30 00:54:50,703][1157819] InferenceWorker_p0-w0: resuming experience collection (950 times) [2024-09-30 00:54:50,859][1157819] Updated weights for policy 0, policy_version 21878 (0.0006) [2024-09-30 00:54:51,447][1157819] Updated weights for policy 0, policy_version 21888 (0.0006) [2024-09-30 00:54:52,020][1157819] Updated weights for policy 0, policy_version 21898 (0.0006) [2024-09-30 00:54:52,611][1157819] Updated weights for policy 0, policy_version 21908 (0.0006) [2024-09-30 00:54:53,238][1157819] Updated weights for policy 0, policy_version 21918 (0.0006) [2024-09-30 00:54:53,822][1157819] Updated weights for policy 0, policy_version 21928 (0.0006) [2024-09-30 00:54:54,421][1157819] Updated weights for policy 0, policy_version 21938 (0.0006) [2024-09-30 00:54:55,056][1157819] Updated weights for policy 0, policy_version 21948 (0.0006) [2024-09-30 00:54:55,466][1157520] Fps is (10 sec: 68813.8, 60 sec: 73591.6, 300 sec: 77699.0). Total num frames: 89927680. Throughput: 0: 18405.5. Samples: 12436464. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 00:54:55,466][1157520] Avg episode reward: [(0, '46.753')] [2024-09-30 00:54:55,605][1157819] Updated weights for policy 0, policy_version 21958 (0.0006) [2024-09-30 00:54:56,234][1157819] Updated weights for policy 0, policy_version 21968 (0.0006) [2024-09-30 00:54:56,842][1157819] Updated weights for policy 0, policy_version 21978 (0.0006) [2024-09-30 00:54:57,443][1157819] Updated weights for policy 0, policy_version 21988 (0.0006) [2024-09-30 00:54:58,036][1157819] Updated weights for policy 0, policy_version 21998 (0.0006) [2024-09-30 00:54:58,619][1157819] Updated weights for policy 0, policy_version 22008 (0.0006) [2024-09-30 00:54:59,218][1157819] Updated weights for policy 0, policy_version 22018 (0.0006) [2024-09-30 00:54:59,795][1157819] Updated weights for policy 0, policy_version 22028 (0.0006) [2024-09-30 00:55:00,407][1157819] Updated weights for policy 0, policy_version 22038 (0.0006) [2024-09-30 00:55:00,466][1157520] Fps is (10 sec: 68812.9, 60 sec: 73113.7, 300 sec: 77476.9). Total num frames: 90271744. Throughput: 0: 18118.5. Samples: 12539332. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 00:55:00,466][1157520] Avg episode reward: [(0, '45.403')] [2024-09-30 00:55:01,003][1157819] Updated weights for policy 0, policy_version 22048 (0.0006) [2024-09-30 00:55:01,505][1157819] Updated weights for policy 0, policy_version 22058 (0.0006) [2024-09-30 00:55:01,965][1157819] Updated weights for policy 0, policy_version 22068 (0.0006) [2024-09-30 00:55:02,491][1157819] Updated weights for policy 0, policy_version 22078 (0.0006) [2024-09-30 00:55:03,114][1157819] Updated weights for policy 0, policy_version 22088 (0.0007) [2024-09-30 00:55:03,741][1157819] Updated weights for policy 0, policy_version 22098 (0.0006) [2024-09-30 00:55:04,375][1157819] Updated weights for policy 0, policy_version 22108 (0.0006) [2024-09-30 00:55:04,955][1157819] Updated weights for policy 0, policy_version 22118 (0.0006) [2024-09-30 00:55:05,466][1157520] Fps is (10 sec: 70040.8, 60 sec: 72840.6, 300 sec: 77435.2). Total num frames: 90628096. Throughput: 0: 17866.0. Samples: 12646156. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 00:55:05,466][1157520] Avg episode reward: [(0, '47.904')] [2024-09-30 00:55:05,554][1157819] Updated weights for policy 0, policy_version 22128 (0.0006) [2024-09-30 00:55:06,120][1157819] Updated weights for policy 0, policy_version 22138 (0.0006) [2024-09-30 00:55:06,819][1157819] Updated weights for policy 0, policy_version 22148 (0.0006) [2024-09-30 00:55:07,453][1157819] Updated weights for policy 0, policy_version 22158 (0.0006) [2024-09-30 00:55:08,078][1157819] Updated weights for policy 0, policy_version 22168 (0.0006) [2024-09-30 00:55:08,714][1157819] Updated weights for policy 0, policy_version 22178 (0.0006) [2024-09-30 00:55:09,368][1157819] Updated weights for policy 0, policy_version 22188 (0.0006) [2024-09-30 00:55:09,997][1157819] Updated weights for policy 0, policy_version 22198 (0.0007) [2024-09-30 00:55:10,466][1157520] Fps is (10 sec: 67992.4, 60 sec: 71884.7, 300 sec: 77199.2). Total num frames: 90951680. Throughput: 0: 17665.1. Samples: 12695116. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 00:55:10,466][1157520] Avg episode reward: [(0, '48.649')] [2024-09-30 00:55:10,656][1157819] Updated weights for policy 0, policy_version 22208 (0.0006) [2024-09-30 00:55:11,271][1157819] Updated weights for policy 0, policy_version 22218 (0.0007) [2024-09-30 00:55:11,888][1157819] Updated weights for policy 0, policy_version 22228 (0.0006) [2024-09-30 00:55:12,481][1157819] Updated weights for policy 0, policy_version 22238 (0.0006) [2024-09-30 00:55:13,089][1157819] Updated weights for policy 0, policy_version 22248 (0.0006) [2024-09-30 00:55:13,727][1157819] Updated weights for policy 0, policy_version 22258 (0.0006) [2024-09-30 00:55:14,371][1157819] Updated weights for policy 0, policy_version 22268 (0.0006) [2024-09-30 00:55:15,051][1157819] Updated weights for policy 0, policy_version 22278 (0.0007) [2024-09-30 00:55:15,324][1157736] Signal inference workers to stop experience collection... (1000 times) [2024-09-30 00:55:15,327][1157819] InferenceWorker_p0-w0: stopping experience collection (1000 times) [2024-09-30 00:55:15,332][1157736] Signal inference workers to resume experience collection... (1000 times) [2024-09-30 00:55:15,332][1157819] InferenceWorker_p0-w0: resuming experience collection (1000 times) [2024-09-30 00:55:15,466][1157520] Fps is (10 sec: 64716.4, 60 sec: 70860.7, 300 sec: 76921.5). Total num frames: 91275264. Throughput: 0: 17422.3. Samples: 12793060. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 00:55:15,466][1157520] Avg episode reward: [(0, '49.623')] [2024-09-30 00:55:15,647][1157819] Updated weights for policy 0, policy_version 22288 (0.0006) [2024-09-30 00:55:16,334][1157819] Updated weights for policy 0, policy_version 22298 (0.0006) [2024-09-30 00:55:16,968][1157819] Updated weights for policy 0, policy_version 22308 (0.0006) [2024-09-30 00:55:17,588][1157819] Updated weights for policy 0, policy_version 22318 (0.0006) [2024-09-30 00:55:18,186][1157819] Updated weights for policy 0, policy_version 22328 (0.0006) [2024-09-30 00:55:18,762][1157819] Updated weights for policy 0, policy_version 22338 (0.0006) [2024-09-30 00:55:19,333][1157819] Updated weights for policy 0, policy_version 22348 (0.0006) [2024-09-30 00:55:19,920][1157819] Updated weights for policy 0, policy_version 22358 (0.0006) [2024-09-30 00:55:20,466][1157520] Fps is (10 sec: 66355.6, 60 sec: 69973.3, 300 sec: 76727.1). Total num frames: 91615232. Throughput: 0: 17199.0. Samples: 12892128. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 00:55:20,466][1157520] Avg episode reward: [(0, '51.082')] [2024-09-30 00:55:20,467][1157736] Saving new best policy, reward=51.082! [2024-09-30 00:55:20,548][1157819] Updated weights for policy 0, policy_version 22368 (0.0006) [2024-09-30 00:55:21,056][1157819] Updated weights for policy 0, policy_version 22378 (0.0006) [2024-09-30 00:55:21,598][1157819] Updated weights for policy 0, policy_version 22388 (0.0006) [2024-09-30 00:55:22,137][1157819] Updated weights for policy 0, policy_version 22398 (0.0006) [2024-09-30 00:55:22,697][1157819] Updated weights for policy 0, policy_version 22408 (0.0006) [2024-09-30 00:55:23,239][1157819] Updated weights for policy 0, policy_version 22418 (0.0006) [2024-09-30 00:55:23,806][1157819] Updated weights for policy 0, policy_version 22428 (0.0006) [2024-09-30 00:55:24,370][1157819] Updated weights for policy 0, policy_version 22438 (0.0006) [2024-09-30 00:55:24,874][1157819] Updated weights for policy 0, policy_version 22448 (0.0006) [2024-09-30 00:55:25,401][1157819] Updated weights for policy 0, policy_version 22458 (0.0006) [2024-09-30 00:55:25,466][1157520] Fps is (10 sec: 71270.7, 60 sec: 69905.1, 300 sec: 76699.3). Total num frames: 91987968. Throughput: 0: 17270.9. Samples: 12947636. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 00:55:25,466][1157520] Avg episode reward: [(0, '49.108')] [2024-09-30 00:55:25,962][1157819] Updated weights for policy 0, policy_version 22468 (0.0006) [2024-09-30 00:55:26,465][1157819] Updated weights for policy 0, policy_version 22478 (0.0006) [2024-09-30 00:55:26,989][1157819] Updated weights for policy 0, policy_version 22488 (0.0006) [2024-09-30 00:55:27,527][1157819] Updated weights for policy 0, policy_version 22498 (0.0006) [2024-09-30 00:55:28,044][1157819] Updated weights for policy 0, policy_version 22508 (0.0006) [2024-09-30 00:55:28,568][1157819] Updated weights for policy 0, policy_version 22518 (0.0006) [2024-09-30 00:55:29,077][1157819] Updated weights for policy 0, policy_version 22528 (0.0006) [2024-09-30 00:55:29,596][1157819] Updated weights for policy 0, policy_version 22538 (0.0006) [2024-09-30 00:55:30,127][1157819] Updated weights for policy 0, policy_version 22548 (0.0006) [2024-09-30 00:55:30,466][1157520] Fps is (10 sec: 76595.0, 60 sec: 70382.9, 300 sec: 76754.9). Total num frames: 92381184. Throughput: 0: 17428.1. Samples: 13062888. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 00:55:30,466][1157520] Avg episode reward: [(0, '48.844')] [2024-09-30 00:55:30,649][1157819] Updated weights for policy 0, policy_version 22558 (0.0006) [2024-09-30 00:55:31,163][1157819] Updated weights for policy 0, policy_version 22568 (0.0006) [2024-09-30 00:55:31,668][1157819] Updated weights for policy 0, policy_version 22578 (0.0006) [2024-09-30 00:55:32,207][1157819] Updated weights for policy 0, policy_version 22588 (0.0006) [2024-09-30 00:55:32,704][1157819] Updated weights for policy 0, policy_version 22598 (0.0006) [2024-09-30 00:55:33,219][1157819] Updated weights for policy 0, policy_version 22608 (0.0006) [2024-09-30 00:55:33,776][1157819] Updated weights for policy 0, policy_version 22618 (0.0006) [2024-09-30 00:55:34,292][1157819] Updated weights for policy 0, policy_version 22628 (0.0006) [2024-09-30 00:55:34,810][1157819] Updated weights for policy 0, policy_version 22638 (0.0006) [2024-09-30 00:55:35,311][1157819] Updated weights for policy 0, policy_version 22648 (0.0006) [2024-09-30 00:55:35,466][1157520] Fps is (10 sec: 78643.1, 60 sec: 70929.1, 300 sec: 76768.8). Total num frames: 92774400. Throughput: 0: 17700.6. Samples: 13180992. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 00:55:35,466][1157520] Avg episode reward: [(0, '49.076')] [2024-09-30 00:55:35,813][1157819] Updated weights for policy 0, policy_version 22658 (0.0006) [2024-09-30 00:55:36,343][1157819] Updated weights for policy 0, policy_version 22668 (0.0006) [2024-09-30 00:55:36,858][1157819] Updated weights for policy 0, policy_version 22678 (0.0006) [2024-09-30 00:55:37,363][1157819] Updated weights for policy 0, policy_version 22688 (0.0006) [2024-09-30 00:55:37,886][1157819] Updated weights for policy 0, policy_version 22698 (0.0006) [2024-09-30 00:55:38,383][1157819] Updated weights for policy 0, policy_version 22708 (0.0006) [2024-09-30 00:55:38,882][1157819] Updated weights for policy 0, policy_version 22718 (0.0006) [2024-09-30 00:55:39,449][1157819] Updated weights for policy 0, policy_version 22728 (0.0006) [2024-09-30 00:55:39,968][1157819] Updated weights for policy 0, policy_version 22738 (0.0006) [2024-09-30 00:55:40,457][1157819] Updated weights for policy 0, policy_version 22748 (0.0006) [2024-09-30 00:55:40,466][1157520] Fps is (10 sec: 79462.9, 60 sec: 71748.3, 300 sec: 76824.3). Total num frames: 93175808. Throughput: 0: 17874.0. Samples: 13240796. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 00:55:40,466][1157520] Avg episode reward: [(0, '48.284')] [2024-09-30 00:55:40,994][1157819] Updated weights for policy 0, policy_version 22758 (0.0006) [2024-09-30 00:55:41,508][1157819] Updated weights for policy 0, policy_version 22768 (0.0006) [2024-09-30 00:55:42,043][1157819] Updated weights for policy 0, policy_version 22778 (0.0006) [2024-09-30 00:55:42,538][1157819] Updated weights for policy 0, policy_version 22788 (0.0006) [2024-09-30 00:55:43,076][1157819] Updated weights for policy 0, policy_version 22798 (0.0006) [2024-09-30 00:55:43,590][1157819] Updated weights for policy 0, policy_version 22808 (0.0006) [2024-09-30 00:55:44,121][1157819] Updated weights for policy 0, policy_version 22818 (0.0006) [2024-09-30 00:55:44,621][1157819] Updated weights for policy 0, policy_version 22828 (0.0006) [2024-09-30 00:55:45,160][1157819] Updated weights for policy 0, policy_version 22838 (0.0006) [2024-09-30 00:55:45,466][1157520] Fps is (10 sec: 79053.1, 60 sec: 72089.7, 300 sec: 76824.3). Total num frames: 93564928. Throughput: 0: 18223.3. Samples: 13359384. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 00:55:45,466][1157520] Avg episode reward: [(0, '51.046')] [2024-09-30 00:55:45,694][1157819] Updated weights for policy 0, policy_version 22848 (0.0006) [2024-09-30 00:55:46,192][1157819] Updated weights for policy 0, policy_version 22858 (0.0006) [2024-09-30 00:55:46,693][1157819] Updated weights for policy 0, policy_version 22868 (0.0006) [2024-09-30 00:55:47,244][1157819] Updated weights for policy 0, policy_version 22878 (0.0006) [2024-09-30 00:55:47,766][1157819] Updated weights for policy 0, policy_version 22888 (0.0006) [2024-09-30 00:55:48,274][1157819] Updated weights for policy 0, policy_version 22898 (0.0006) [2024-09-30 00:55:48,840][1157819] Updated weights for policy 0, policy_version 22908 (0.0006) [2024-09-30 00:55:49,377][1157819] Updated weights for policy 0, policy_version 22918 (0.0006) [2024-09-30 00:55:49,860][1157819] Updated weights for policy 0, policy_version 22928 (0.0006) [2024-09-30 00:55:50,403][1157819] Updated weights for policy 0, policy_version 22938 (0.0006) [2024-09-30 00:55:50,466][1157520] Fps is (10 sec: 78233.0, 60 sec: 72908.6, 300 sec: 76754.9). Total num frames: 93958144. Throughput: 0: 18453.6. Samples: 13476568. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 00:55:50,466][1157520] Avg episode reward: [(0, '44.742')] [2024-09-30 00:55:50,912][1157819] Updated weights for policy 0, policy_version 22948 (0.0006) [2024-09-30 00:55:51,459][1157819] Updated weights for policy 0, policy_version 22958 (0.0006) [2024-09-30 00:55:52,001][1157819] Updated weights for policy 0, policy_version 22968 (0.0006) [2024-09-30 00:55:52,499][1157819] Updated weights for policy 0, policy_version 22978 (0.0006) [2024-09-30 00:55:53,041][1157819] Updated weights for policy 0, policy_version 22988 (0.0006) [2024-09-30 00:55:53,531][1157819] Updated weights for policy 0, policy_version 22998 (0.0006) [2024-09-30 00:55:54,023][1157819] Updated weights for policy 0, policy_version 23008 (0.0006) [2024-09-30 00:55:54,522][1157819] Updated weights for policy 0, policy_version 23018 (0.0006) [2024-09-30 00:55:55,006][1157819] Updated weights for policy 0, policy_version 23028 (0.0006) [2024-09-30 00:55:55,466][1157520] Fps is (10 sec: 79462.2, 60 sec: 73864.4, 300 sec: 76768.7). Total num frames: 94359552. Throughput: 0: 18663.0. Samples: 13534948. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 00:55:55,466][1157520] Avg episode reward: [(0, '48.223')] [2024-09-30 00:55:55,497][1157819] Updated weights for policy 0, policy_version 23038 (0.0006) [2024-09-30 00:55:55,997][1157819] Updated weights for policy 0, policy_version 23048 (0.0006) [2024-09-30 00:55:56,491][1157819] Updated weights for policy 0, policy_version 23058 (0.0006) [2024-09-30 00:55:56,954][1157819] Updated weights for policy 0, policy_version 23068 (0.0006) [2024-09-30 00:55:57,453][1157819] Updated weights for policy 0, policy_version 23078 (0.0006) [2024-09-30 00:55:57,951][1157819] Updated weights for policy 0, policy_version 23088 (0.0006) [2024-09-30 00:55:58,430][1157819] Updated weights for policy 0, policy_version 23098 (0.0006) [2024-09-30 00:55:58,921][1157819] Updated weights for policy 0, policy_version 23108 (0.0006) [2024-09-30 00:55:59,447][1157819] Updated weights for policy 0, policy_version 23118 (0.0006) [2024-09-30 00:55:59,800][1157736] Signal inference workers to stop experience collection... (1050 times) [2024-09-30 00:55:59,800][1157736] Signal inference workers to resume experience collection... (1050 times) [2024-09-30 00:55:59,804][1157819] InferenceWorker_p0-w0: stopping experience collection (1050 times) [2024-09-30 00:55:59,806][1157819] InferenceWorker_p0-w0: resuming experience collection (1050 times) [2024-09-30 00:55:59,962][1157819] Updated weights for policy 0, policy_version 23128 (0.0006) [2024-09-30 00:56:00,459][1157819] Updated weights for policy 0, policy_version 23138 (0.0006) [2024-09-30 00:56:00,466][1157520] Fps is (10 sec: 81510.3, 60 sec: 75024.9, 300 sec: 76879.8). Total num frames: 94773248. Throughput: 0: 19260.6. Samples: 13659788. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 00:56:00,466][1157520] Avg episode reward: [(0, '45.701')] [2024-09-30 00:56:00,945][1157819] Updated weights for policy 0, policy_version 23148 (0.0006) [2024-09-30 00:56:01,433][1157819] Updated weights for policy 0, policy_version 23158 (0.0006) [2024-09-30 00:56:01,928][1157819] Updated weights for policy 0, policy_version 23168 (0.0006) [2024-09-30 00:56:02,423][1157819] Updated weights for policy 0, policy_version 23178 (0.0006) [2024-09-30 00:56:02,942][1157819] Updated weights for policy 0, policy_version 23188 (0.0007) [2024-09-30 00:56:03,437][1157819] Updated weights for policy 0, policy_version 23198 (0.0006) [2024-09-30 00:56:03,920][1157819] Updated weights for policy 0, policy_version 23208 (0.0006) [2024-09-30 00:56:04,419][1157819] Updated weights for policy 0, policy_version 23218 (0.0006) [2024-09-30 00:56:04,951][1157819] Updated weights for policy 0, policy_version 23228 (0.0006) [2024-09-30 00:56:05,466][1157520] Fps is (10 sec: 81919.7, 60 sec: 75844.2, 300 sec: 76990.9). Total num frames: 95178752. Throughput: 0: 19773.7. Samples: 13781944. Policy #0 lag: (min: 0.0, avg: 1.5, max: 5.0) [2024-09-30 00:56:05,466][1157520] Avg episode reward: [(0, '49.090')] [2024-09-30 00:56:05,474][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000023237_95178752.pth... [2024-09-30 00:56:05,524][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000018689_76550144.pth [2024-09-30 00:56:05,544][1157819] Updated weights for policy 0, policy_version 23238 (0.0006) [2024-09-30 00:56:06,104][1157819] Updated weights for policy 0, policy_version 23248 (0.0006) [2024-09-30 00:56:06,682][1157819] Updated weights for policy 0, policy_version 23258 (0.0006) [2024-09-30 00:56:07,275][1157819] Updated weights for policy 0, policy_version 23268 (0.0006) [2024-09-30 00:56:07,856][1157819] Updated weights for policy 0, policy_version 23278 (0.0006) [2024-09-30 00:56:08,428][1157819] Updated weights for policy 0, policy_version 23288 (0.0006) [2024-09-30 00:56:09,018][1157819] Updated weights for policy 0, policy_version 23298 (0.0006) [2024-09-30 00:56:09,618][1157819] Updated weights for policy 0, policy_version 23308 (0.0006) [2024-09-30 00:56:10,150][1157819] Updated weights for policy 0, policy_version 23318 (0.0006) [2024-09-30 00:56:10,466][1157520] Fps is (10 sec: 75776.4, 60 sec: 76322.2, 300 sec: 76921.5). Total num frames: 95531008. Throughput: 0: 19728.5. Samples: 13835420. Policy #0 lag: (min: 0.0, avg: 1.5, max: 5.0) [2024-09-30 00:56:10,466][1157520] Avg episode reward: [(0, '46.740')] [2024-09-30 00:56:10,694][1157819] Updated weights for policy 0, policy_version 23328 (0.0006) [2024-09-30 00:56:11,231][1157819] Updated weights for policy 0, policy_version 23338 (0.0006) [2024-09-30 00:56:11,776][1157819] Updated weights for policy 0, policy_version 23348 (0.0006) [2024-09-30 00:56:12,295][1157819] Updated weights for policy 0, policy_version 23358 (0.0006) [2024-09-30 00:56:12,848][1157819] Updated weights for policy 0, policy_version 23368 (0.0006) [2024-09-30 00:56:13,416][1157819] Updated weights for policy 0, policy_version 23378 (0.0006) [2024-09-30 00:56:13,925][1157819] Updated weights for policy 0, policy_version 23388 (0.0006) [2024-09-30 00:56:14,502][1157819] Updated weights for policy 0, policy_version 23398 (0.0006) [2024-09-30 00:56:14,993][1157819] Updated weights for policy 0, policy_version 23408 (0.0006) [2024-09-30 00:56:15,466][1157520] Fps is (10 sec: 73728.2, 60 sec: 77346.2, 300 sec: 76963.1). Total num frames: 95916032. Throughput: 0: 19634.0. Samples: 13946416. Policy #0 lag: (min: 0.0, avg: 1.5, max: 5.0) [2024-09-30 00:56:15,466][1157520] Avg episode reward: [(0, '50.924')] [2024-09-30 00:56:15,495][1157819] Updated weights for policy 0, policy_version 23418 (0.0006) [2024-09-30 00:56:16,007][1157819] Updated weights for policy 0, policy_version 23428 (0.0006) [2024-09-30 00:56:16,547][1157819] Updated weights for policy 0, policy_version 23438 (0.0006) [2024-09-30 00:56:17,100][1157819] Updated weights for policy 0, policy_version 23448 (0.0006) [2024-09-30 00:56:17,639][1157819] Updated weights for policy 0, policy_version 23458 (0.0006) [2024-09-30 00:56:18,179][1157819] Updated weights for policy 0, policy_version 23468 (0.0006) [2024-09-30 00:56:18,693][1157819] Updated weights for policy 0, policy_version 23478 (0.0006) [2024-09-30 00:56:19,247][1157819] Updated weights for policy 0, policy_version 23488 (0.0006) [2024-09-30 00:56:19,787][1157819] Updated weights for policy 0, policy_version 23498 (0.0006) [2024-09-30 00:56:20,326][1157819] Updated weights for policy 0, policy_version 23508 (0.0006) [2024-09-30 00:56:20,466][1157520] Fps is (10 sec: 76595.2, 60 sec: 78028.8, 300 sec: 76949.3). Total num frames: 96296960. Throughput: 0: 19579.4. Samples: 14062064. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 00:56:20,466][1157520] Avg episode reward: [(0, '49.439')] [2024-09-30 00:56:20,856][1157819] Updated weights for policy 0, policy_version 23518 (0.0006) [2024-09-30 00:56:21,398][1157819] Updated weights for policy 0, policy_version 23528 (0.0006) [2024-09-30 00:56:21,934][1157819] Updated weights for policy 0, policy_version 23538 (0.0006) [2024-09-30 00:56:22,459][1157819] Updated weights for policy 0, policy_version 23548 (0.0006) [2024-09-30 00:56:23,046][1157819] Updated weights for policy 0, policy_version 23558 (0.0006) [2024-09-30 00:56:23,552][1157819] Updated weights for policy 0, policy_version 23568 (0.0006) [2024-09-30 00:56:24,069][1157819] Updated weights for policy 0, policy_version 23578 (0.0006) [2024-09-30 00:56:24,641][1157819] Updated weights for policy 0, policy_version 23588 (0.0006) [2024-09-30 00:56:25,214][1157736] Signal inference workers to stop experience collection... (1100 times) [2024-09-30 00:56:25,218][1157736] Signal inference workers to resume experience collection... (1100 times) [2024-09-30 00:56:25,218][1157819] Updated weights for policy 0, policy_version 23598 (0.0006) [2024-09-30 00:56:25,221][1157819] InferenceWorker_p0-w0: stopping experience collection (1100 times) [2024-09-30 00:56:25,224][1157819] InferenceWorker_p0-w0: resuming experience collection (1100 times) [2024-09-30 00:56:25,466][1157520] Fps is (10 sec: 75776.1, 60 sec: 78097.1, 300 sec: 76921.5). Total num frames: 96673792. Throughput: 0: 19514.1. Samples: 14118932. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 00:56:25,466][1157520] Avg episode reward: [(0, '48.146')] [2024-09-30 00:56:25,776][1157819] Updated weights for policy 0, policy_version 23608 (0.0006) [2024-09-30 00:56:26,298][1157819] Updated weights for policy 0, policy_version 23618 (0.0006) [2024-09-30 00:56:26,885][1157819] Updated weights for policy 0, policy_version 23628 (0.0006) [2024-09-30 00:56:27,401][1157819] Updated weights for policy 0, policy_version 23638 (0.0006) [2024-09-30 00:56:27,948][1157819] Updated weights for policy 0, policy_version 23648 (0.0006) [2024-09-30 00:56:28,474][1157819] Updated weights for policy 0, policy_version 23658 (0.0006) [2024-09-30 00:56:29,036][1157819] Updated weights for policy 0, policy_version 23668 (0.0006) [2024-09-30 00:56:29,553][1157819] Updated weights for policy 0, policy_version 23678 (0.0006) [2024-09-30 00:56:30,075][1157819] Updated weights for policy 0, policy_version 23688 (0.0006) [2024-09-30 00:56:30,466][1157520] Fps is (10 sec: 75776.0, 60 sec: 77892.3, 300 sec: 76852.1). Total num frames: 97054720. Throughput: 0: 19379.1. Samples: 14231444. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 00:56:30,466][1157520] Avg episode reward: [(0, '46.481')] [2024-09-30 00:56:30,588][1157819] Updated weights for policy 0, policy_version 23698 (0.0006) [2024-09-30 00:56:31,144][1157819] Updated weights for policy 0, policy_version 23708 (0.0006) [2024-09-30 00:56:31,636][1157819] Updated weights for policy 0, policy_version 23718 (0.0006) [2024-09-30 00:56:32,127][1157819] Updated weights for policy 0, policy_version 23728 (0.0006) [2024-09-30 00:56:32,659][1157819] Updated weights for policy 0, policy_version 23738 (0.0006) [2024-09-30 00:56:33,172][1157819] Updated weights for policy 0, policy_version 23748 (0.0006) [2024-09-30 00:56:33,720][1157819] Updated weights for policy 0, policy_version 23758 (0.0006) [2024-09-30 00:56:34,231][1157819] Updated weights for policy 0, policy_version 23768 (0.0006) [2024-09-30 00:56:34,735][1157819] Updated weights for policy 0, policy_version 23778 (0.0006) [2024-09-30 00:56:35,264][1157819] Updated weights for policy 0, policy_version 23788 (0.0006) [2024-09-30 00:56:35,466][1157520] Fps is (10 sec: 77824.2, 60 sec: 77960.6, 300 sec: 76921.5). Total num frames: 97452032. Throughput: 0: 19395.9. Samples: 14349384. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 00:56:35,466][1157520] Avg episode reward: [(0, '48.478')] [2024-09-30 00:56:35,788][1157819] Updated weights for policy 0, policy_version 23798 (0.0006) [2024-09-30 00:56:36,316][1157819] Updated weights for policy 0, policy_version 23808 (0.0006) [2024-09-30 00:56:36,856][1157819] Updated weights for policy 0, policy_version 23818 (0.0006) [2024-09-30 00:56:37,383][1157819] Updated weights for policy 0, policy_version 23828 (0.0006) [2024-09-30 00:56:37,894][1157819] Updated weights for policy 0, policy_version 23838 (0.0006) [2024-09-30 00:56:38,415][1157819] Updated weights for policy 0, policy_version 23848 (0.0006) [2024-09-30 00:56:38,912][1157819] Updated weights for policy 0, policy_version 23858 (0.0006) [2024-09-30 00:56:39,481][1157819] Updated weights for policy 0, policy_version 23868 (0.0006) [2024-09-30 00:56:39,960][1157819] Updated weights for policy 0, policy_version 23878 (0.0006) [2024-09-30 00:56:40,466][1157819] Updated weights for policy 0, policy_version 23888 (0.0006) [2024-09-30 00:56:40,466][1157520] Fps is (10 sec: 79052.3, 60 sec: 77823.9, 300 sec: 77032.6). Total num frames: 97845248. Throughput: 0: 19400.5. Samples: 14407972. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 00:56:40,466][1157520] Avg episode reward: [(0, '47.119')] [2024-09-30 00:56:40,971][1157819] Updated weights for policy 0, policy_version 23898 (0.0006) [2024-09-30 00:56:41,495][1157819] Updated weights for policy 0, policy_version 23908 (0.0006) [2024-09-30 00:56:42,005][1157819] Updated weights for policy 0, policy_version 23918 (0.0006) [2024-09-30 00:56:42,524][1157819] Updated weights for policy 0, policy_version 23928 (0.0006) [2024-09-30 00:56:43,037][1157819] Updated weights for policy 0, policy_version 23938 (0.0006) [2024-09-30 00:56:43,552][1157819] Updated weights for policy 0, policy_version 23948 (0.0006) [2024-09-30 00:56:44,049][1157819] Updated weights for policy 0, policy_version 23958 (0.0006) [2024-09-30 00:56:44,551][1157819] Updated weights for policy 0, policy_version 23968 (0.0006) [2024-09-30 00:56:45,119][1157819] Updated weights for policy 0, policy_version 23978 (0.0006) [2024-09-30 00:56:45,466][1157520] Fps is (10 sec: 78643.3, 60 sec: 77892.3, 300 sec: 77129.8). Total num frames: 98238464. Throughput: 0: 19283.7. Samples: 14527552. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 00:56:45,466][1157520] Avg episode reward: [(0, '51.327')] [2024-09-30 00:56:45,474][1157736] Saving new best policy, reward=51.327! [2024-09-30 00:56:45,633][1157819] Updated weights for policy 0, policy_version 23988 (0.0006) [2024-09-30 00:56:46,139][1157819] Updated weights for policy 0, policy_version 23998 (0.0006) [2024-09-30 00:56:46,656][1157819] Updated weights for policy 0, policy_version 24008 (0.0006) [2024-09-30 00:56:47,220][1157819] Updated weights for policy 0, policy_version 24018 (0.0006) [2024-09-30 00:56:47,763][1157819] Updated weights for policy 0, policy_version 24028 (0.0006) [2024-09-30 00:56:48,310][1157819] Updated weights for policy 0, policy_version 24038 (0.0006) [2024-09-30 00:56:48,876][1157819] Updated weights for policy 0, policy_version 24048 (0.0006) [2024-09-30 00:56:49,417][1157819] Updated weights for policy 0, policy_version 24058 (0.0006) [2024-09-30 00:56:50,033][1157819] Updated weights for policy 0, policy_version 24068 (0.0006) [2024-09-30 00:56:50,466][1157520] Fps is (10 sec: 76595.5, 60 sec: 77551.0, 300 sec: 77240.9). Total num frames: 98611200. Throughput: 0: 19096.8. Samples: 14641300. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 00:56:50,466][1157520] Avg episode reward: [(0, '48.788')] [2024-09-30 00:56:50,639][1157819] Updated weights for policy 0, policy_version 24078 (0.0006) [2024-09-30 00:56:51,228][1157819] Updated weights for policy 0, policy_version 24088 (0.0006) [2024-09-30 00:56:51,761][1157819] Updated weights for policy 0, policy_version 24098 (0.0006) [2024-09-30 00:56:52,318][1157819] Updated weights for policy 0, policy_version 24108 (0.0006) [2024-09-30 00:56:52,884][1157819] Updated weights for policy 0, policy_version 24118 (0.0006) [2024-09-30 00:56:53,419][1157819] Updated weights for policy 0, policy_version 24128 (0.0006) [2024-09-30 00:56:53,979][1157819] Updated weights for policy 0, policy_version 24138 (0.0006) [2024-09-30 00:56:54,536][1157819] Updated weights for policy 0, policy_version 24148 (0.0006) [2024-09-30 00:56:55,087][1157819] Updated weights for policy 0, policy_version 24158 (0.0006) [2024-09-30 00:56:55,466][1157520] Fps is (10 sec: 74136.6, 60 sec: 77004.7, 300 sec: 77282.4). Total num frames: 98979840. Throughput: 0: 19107.3. Samples: 14695252. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 00:56:55,466][1157520] Avg episode reward: [(0, '50.313')] [2024-09-30 00:56:55,612][1157819] Updated weights for policy 0, policy_version 24168 (0.0006) [2024-09-30 00:56:56,164][1157819] Updated weights for policy 0, policy_version 24178 (0.0006) [2024-09-30 00:56:56,728][1157819] Updated weights for policy 0, policy_version 24188 (0.0006) [2024-09-30 00:56:57,244][1157819] Updated weights for policy 0, policy_version 24198 (0.0006) [2024-09-30 00:56:57,775][1157819] Updated weights for policy 0, policy_version 24208 (0.0006) [2024-09-30 00:56:58,365][1157819] Updated weights for policy 0, policy_version 24218 (0.0006) [2024-09-30 00:56:58,927][1157819] Updated weights for policy 0, policy_version 24228 (0.0006) [2024-09-30 00:56:59,494][1157819] Updated weights for policy 0, policy_version 24238 (0.0006) [2024-09-30 00:57:00,053][1157819] Updated weights for policy 0, policy_version 24248 (0.0006) [2024-09-30 00:57:00,433][1157736] Signal inference workers to stop experience collection... (1150 times) [2024-09-30 00:57:00,434][1157736] Signal inference workers to resume experience collection... (1150 times) [2024-09-30 00:57:00,437][1157819] InferenceWorker_p0-w0: stopping experience collection (1150 times) [2024-09-30 00:57:00,437][1157819] InferenceWorker_p0-w0: resuming experience collection (1150 times) [2024-09-30 00:57:00,466][1157520] Fps is (10 sec: 74137.9, 60 sec: 76322.2, 300 sec: 77296.4). Total num frames: 99352576. Throughput: 0: 19118.6. Samples: 14806752. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 00:57:00,466][1157520] Avg episode reward: [(0, '51.364')] [2024-09-30 00:57:00,467][1157736] Saving new best policy, reward=51.364! [2024-09-30 00:57:00,620][1157819] Updated weights for policy 0, policy_version 24258 (0.0006) [2024-09-30 00:57:01,205][1157819] Updated weights for policy 0, policy_version 24268 (0.0006) [2024-09-30 00:57:01,696][1157819] Updated weights for policy 0, policy_version 24278 (0.0006) [2024-09-30 00:57:02,223][1157819] Updated weights for policy 0, policy_version 24288 (0.0006) [2024-09-30 00:57:02,816][1157819] Updated weights for policy 0, policy_version 24298 (0.0006) [2024-09-30 00:57:03,371][1157819] Updated weights for policy 0, policy_version 24308 (0.0006) [2024-09-30 00:57:03,946][1157819] Updated weights for policy 0, policy_version 24318 (0.0006) [2024-09-30 00:57:04,449][1157819] Updated weights for policy 0, policy_version 24328 (0.0006) [2024-09-30 00:57:05,027][1157819] Updated weights for policy 0, policy_version 24338 (0.0006) [2024-09-30 00:57:05,466][1157520] Fps is (10 sec: 74138.5, 60 sec: 75707.8, 300 sec: 77213.1). Total num frames: 99721216. Throughput: 0: 19022.8. Samples: 14918092. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 00:57:05,466][1157520] Avg episode reward: [(0, '47.718')] [2024-09-30 00:57:05,554][1157819] Updated weights for policy 0, policy_version 24348 (0.0006) [2024-09-30 00:57:06,107][1157819] Updated weights for policy 0, policy_version 24358 (0.0006) [2024-09-30 00:57:06,666][1157819] Updated weights for policy 0, policy_version 24368 (0.0006) [2024-09-30 00:57:07,194][1157819] Updated weights for policy 0, policy_version 24378 (0.0006) [2024-09-30 00:57:07,769][1157819] Updated weights for policy 0, policy_version 24388 (0.0006) [2024-09-30 00:57:08,320][1157819] Updated weights for policy 0, policy_version 24398 (0.0006) [2024-09-30 00:57:08,873][1157819] Updated weights for policy 0, policy_version 24408 (0.0006) [2024-09-30 00:57:09,446][1157819] Updated weights for policy 0, policy_version 24418 (0.0006) [2024-09-30 00:57:09,969][1157819] Updated weights for policy 0, policy_version 24428 (0.0006) [2024-09-30 00:57:10,466][1157520] Fps is (10 sec: 73727.4, 60 sec: 75980.7, 300 sec: 77088.1). Total num frames: 100089856. Throughput: 0: 18987.2. Samples: 14973356. Policy #0 lag: (min: 0.0, avg: 2.6, max: 5.0) [2024-09-30 00:57:10,466][1157520] Avg episode reward: [(0, '49.183')] [2024-09-30 00:57:10,550][1157819] Updated weights for policy 0, policy_version 24438 (0.0006) [2024-09-30 00:57:11,081][1157819] Updated weights for policy 0, policy_version 24448 (0.0006) [2024-09-30 00:57:11,619][1157819] Updated weights for policy 0, policy_version 24458 (0.0006) [2024-09-30 00:57:12,193][1157819] Updated weights for policy 0, policy_version 24468 (0.0006) [2024-09-30 00:57:12,695][1157819] Updated weights for policy 0, policy_version 24478 (0.0006) [2024-09-30 00:57:13,256][1157819] Updated weights for policy 0, policy_version 24488 (0.0006) [2024-09-30 00:57:13,828][1157819] Updated weights for policy 0, policy_version 24498 (0.0006) [2024-09-30 00:57:14,378][1157819] Updated weights for policy 0, policy_version 24508 (0.0006) [2024-09-30 00:57:14,930][1157819] Updated weights for policy 0, policy_version 24518 (0.0006) [2024-09-30 00:57:15,466][1157520] Fps is (10 sec: 74137.5, 60 sec: 75776.0, 300 sec: 76893.7). Total num frames: 100462592. Throughput: 0: 18976.1. Samples: 15085368. Policy #0 lag: (min: 0.0, avg: 2.6, max: 5.0) [2024-09-30 00:57:15,466][1157520] Avg episode reward: [(0, '47.203')] [2024-09-30 00:57:15,483][1157819] Updated weights for policy 0, policy_version 24528 (0.0006) [2024-09-30 00:57:16,022][1157819] Updated weights for policy 0, policy_version 24538 (0.0006) [2024-09-30 00:57:16,577][1157819] Updated weights for policy 0, policy_version 24548 (0.0006) [2024-09-30 00:57:17,147][1157819] Updated weights for policy 0, policy_version 24558 (0.0006) [2024-09-30 00:57:17,697][1157819] Updated weights for policy 0, policy_version 24568 (0.0006) [2024-09-30 00:57:18,266][1157819] Updated weights for policy 0, policy_version 24578 (0.0006) [2024-09-30 00:57:18,830][1157819] Updated weights for policy 0, policy_version 24588 (0.0006) [2024-09-30 00:57:19,410][1157819] Updated weights for policy 0, policy_version 24598 (0.0006) [2024-09-30 00:57:19,961][1157819] Updated weights for policy 0, policy_version 24608 (0.0006) [2024-09-30 00:57:20,463][1157819] Updated weights for policy 0, policy_version 24618 (0.0006) [2024-09-30 00:57:20,466][1157520] Fps is (10 sec: 74548.0, 60 sec: 75639.5, 300 sec: 76727.1). Total num frames: 100835328. Throughput: 0: 18813.3. Samples: 15195980. Policy #0 lag: (min: 0.0, avg: 2.6, max: 5.0) [2024-09-30 00:57:20,466][1157520] Avg episode reward: [(0, '48.345')] [2024-09-30 00:57:21,041][1157819] Updated weights for policy 0, policy_version 24628 (0.0006) [2024-09-30 00:57:21,567][1157819] Updated weights for policy 0, policy_version 24638 (0.0006) [2024-09-30 00:57:22,115][1157819] Updated weights for policy 0, policy_version 24648 (0.0006) [2024-09-30 00:57:22,693][1157819] Updated weights for policy 0, policy_version 24658 (0.0006) [2024-09-30 00:57:23,196][1157819] Updated weights for policy 0, policy_version 24668 (0.0006) [2024-09-30 00:57:23,721][1157819] Updated weights for policy 0, policy_version 24678 (0.0006) [2024-09-30 00:57:24,301][1157819] Updated weights for policy 0, policy_version 24688 (0.0006) [2024-09-30 00:57:24,755][1157819] Updated weights for policy 0, policy_version 24698 (0.0006) [2024-09-30 00:57:25,250][1157819] Updated weights for policy 0, policy_version 24708 (0.0006) [2024-09-30 00:57:25,466][1157520] Fps is (10 sec: 75776.8, 60 sec: 75776.1, 300 sec: 76602.2). Total num frames: 101220352. Throughput: 0: 18752.4. Samples: 15251828. Policy #0 lag: (min: 0.0, avg: 3.1, max: 7.0) [2024-09-30 00:57:25,466][1157520] Avg episode reward: [(0, '48.243')] [2024-09-30 00:57:25,749][1157819] Updated weights for policy 0, policy_version 24718 (0.0006) [2024-09-30 00:57:26,172][1157736] Signal inference workers to stop experience collection... (1200 times) [2024-09-30 00:57:26,173][1157736] Signal inference workers to resume experience collection... (1200 times) [2024-09-30 00:57:26,177][1157819] InferenceWorker_p0-w0: stopping experience collection (1200 times) [2024-09-30 00:57:26,177][1157819] InferenceWorker_p0-w0: resuming experience collection (1200 times) [2024-09-30 00:57:26,251][1157819] Updated weights for policy 0, policy_version 24728 (0.0006) [2024-09-30 00:57:26,751][1157819] Updated weights for policy 0, policy_version 24738 (0.0006) [2024-09-30 00:57:27,241][1157819] Updated weights for policy 0, policy_version 24748 (0.0006) [2024-09-30 00:57:27,766][1157819] Updated weights for policy 0, policy_version 24758 (0.0006) [2024-09-30 00:57:28,273][1157819] Updated weights for policy 0, policy_version 24768 (0.0006) [2024-09-30 00:57:28,815][1157819] Updated weights for policy 0, policy_version 24778 (0.0006) [2024-09-30 00:57:29,330][1157819] Updated weights for policy 0, policy_version 24788 (0.0006) [2024-09-30 00:57:29,882][1157819] Updated weights for policy 0, policy_version 24798 (0.0006) [2024-09-30 00:57:30,388][1157819] Updated weights for policy 0, policy_version 24808 (0.0006) [2024-09-30 00:57:30,466][1157520] Fps is (10 sec: 78233.8, 60 sec: 76049.2, 300 sec: 76518.8). Total num frames: 101617664. Throughput: 0: 18777.2. Samples: 15372524. Policy #0 lag: (min: 0.0, avg: 3.1, max: 7.0) [2024-09-30 00:57:30,466][1157520] Avg episode reward: [(0, '47.007')] [2024-09-30 00:57:30,913][1157819] Updated weights for policy 0, policy_version 24818 (0.0006) [2024-09-30 00:57:31,441][1157819] Updated weights for policy 0, policy_version 24828 (0.0006) [2024-09-30 00:57:31,935][1157819] Updated weights for policy 0, policy_version 24838 (0.0006) [2024-09-30 00:57:32,449][1157819] Updated weights for policy 0, policy_version 24848 (0.0006) [2024-09-30 00:57:32,968][1157819] Updated weights for policy 0, policy_version 24858 (0.0006) [2024-09-30 00:57:33,513][1157819] Updated weights for policy 0, policy_version 24868 (0.0006) [2024-09-30 00:57:34,025][1157819] Updated weights for policy 0, policy_version 24878 (0.0006) [2024-09-30 00:57:34,536][1157819] Updated weights for policy 0, policy_version 24888 (0.0006) [2024-09-30 00:57:35,034][1157819] Updated weights for policy 0, policy_version 24898 (0.0006) [2024-09-30 00:57:35,466][1157520] Fps is (10 sec: 79052.6, 60 sec: 75980.9, 300 sec: 76421.6). Total num frames: 102010880. Throughput: 0: 18866.4. Samples: 15490288. Policy #0 lag: (min: 0.0, avg: 3.1, max: 7.0) [2024-09-30 00:57:35,466][1157520] Avg episode reward: [(0, '50.131')] [2024-09-30 00:57:35,544][1157819] Updated weights for policy 0, policy_version 24908 (0.0006) [2024-09-30 00:57:36,064][1157819] Updated weights for policy 0, policy_version 24918 (0.0006) [2024-09-30 00:57:36,571][1157819] Updated weights for policy 0, policy_version 24928 (0.0006) [2024-09-30 00:57:37,136][1157819] Updated weights for policy 0, policy_version 24938 (0.0006) [2024-09-30 00:57:37,663][1157819] Updated weights for policy 0, policy_version 24948 (0.0006) [2024-09-30 00:57:38,162][1157819] Updated weights for policy 0, policy_version 24958 (0.0006) [2024-09-30 00:57:38,664][1157819] Updated weights for policy 0, policy_version 24968 (0.0006) [2024-09-30 00:57:39,239][1157819] Updated weights for policy 0, policy_version 24978 (0.0006) [2024-09-30 00:57:39,784][1157819] Updated weights for policy 0, policy_version 24988 (0.0006) [2024-09-30 00:57:40,341][1157819] Updated weights for policy 0, policy_version 24998 (0.0006) [2024-09-30 00:57:40,466][1157520] Fps is (10 sec: 77824.2, 60 sec: 75844.5, 300 sec: 76282.8). Total num frames: 102395904. Throughput: 0: 18984.4. Samples: 15549548. Policy #0 lag: (min: 0.0, avg: 3.1, max: 7.0) [2024-09-30 00:57:40,466][1157520] Avg episode reward: [(0, '47.866')] [2024-09-30 00:57:40,923][1157819] Updated weights for policy 0, policy_version 25008 (0.0006) [2024-09-30 00:57:41,488][1157819] Updated weights for policy 0, policy_version 25018 (0.0006) [2024-09-30 00:57:42,041][1157819] Updated weights for policy 0, policy_version 25028 (0.0006) [2024-09-30 00:57:42,622][1157819] Updated weights for policy 0, policy_version 25038 (0.0006) [2024-09-30 00:57:43,170][1157819] Updated weights for policy 0, policy_version 25048 (0.0006) [2024-09-30 00:57:43,712][1157819] Updated weights for policy 0, policy_version 25058 (0.0006) [2024-09-30 00:57:44,269][1157819] Updated weights for policy 0, policy_version 25068 (0.0006) [2024-09-30 00:57:44,808][1157819] Updated weights for policy 0, policy_version 25078 (0.0006) [2024-09-30 00:57:45,327][1157819] Updated weights for policy 0, policy_version 25088 (0.0006) [2024-09-30 00:57:45,466][1157520] Fps is (10 sec: 75776.1, 60 sec: 75503.0, 300 sec: 76116.2). Total num frames: 102768640. Throughput: 0: 18971.3. Samples: 15660460. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 00:57:45,466][1157520] Avg episode reward: [(0, '50.203')] [2024-09-30 00:57:45,860][1157819] Updated weights for policy 0, policy_version 25098 (0.0006) [2024-09-30 00:57:46,351][1157819] Updated weights for policy 0, policy_version 25108 (0.0006) [2024-09-30 00:57:46,852][1157819] Updated weights for policy 0, policy_version 25118 (0.0006) [2024-09-30 00:57:47,403][1157819] Updated weights for policy 0, policy_version 25128 (0.0006) [2024-09-30 00:57:47,939][1157819] Updated weights for policy 0, policy_version 25138 (0.0006) [2024-09-30 00:57:48,450][1157819] Updated weights for policy 0, policy_version 25148 (0.0006) [2024-09-30 00:57:48,947][1157819] Updated weights for policy 0, policy_version 25158 (0.0006) [2024-09-30 00:57:49,453][1157819] Updated weights for policy 0, policy_version 25168 (0.0006) [2024-09-30 00:57:49,954][1157819] Updated weights for policy 0, policy_version 25178 (0.0006) [2024-09-30 00:57:50,431][1157819] Updated weights for policy 0, policy_version 25188 (0.0006) [2024-09-30 00:57:50,466][1157520] Fps is (10 sec: 77824.1, 60 sec: 76049.3, 300 sec: 76005.1). Total num frames: 103174144. Throughput: 0: 19134.9. Samples: 15779160. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 00:57:50,466][1157520] Avg episode reward: [(0, '48.547')] [2024-09-30 00:57:50,913][1157819] Updated weights for policy 0, policy_version 25198 (0.0006) [2024-09-30 00:57:51,435][1157819] Updated weights for policy 0, policy_version 25208 (0.0006) [2024-09-30 00:57:51,943][1157819] Updated weights for policy 0, policy_version 25218 (0.0006) [2024-09-30 00:57:52,385][1157819] Updated weights for policy 0, policy_version 25228 (0.0006) [2024-09-30 00:57:52,915][1157819] Updated weights for policy 0, policy_version 25238 (0.0006) [2024-09-30 00:57:53,391][1157819] Updated weights for policy 0, policy_version 25248 (0.0006) [2024-09-30 00:57:53,865][1157819] Updated weights for policy 0, policy_version 25258 (0.0006) [2024-09-30 00:57:54,385][1157819] Updated weights for policy 0, policy_version 25268 (0.0006) [2024-09-30 00:57:54,951][1157819] Updated weights for policy 0, policy_version 25278 (0.0006) [2024-09-30 00:57:55,452][1157819] Updated weights for policy 0, policy_version 25288 (0.0006) [2024-09-30 00:57:55,466][1157520] Fps is (10 sec: 81101.0, 60 sec: 76663.7, 300 sec: 75935.7). Total num frames: 103579648. Throughput: 0: 19291.3. Samples: 15841460. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 00:57:55,466][1157520] Avg episode reward: [(0, '46.440')] [2024-09-30 00:57:55,946][1157819] Updated weights for policy 0, policy_version 25298 (0.0006) [2024-09-30 00:57:56,447][1157819] Updated weights for policy 0, policy_version 25308 (0.0006) [2024-09-30 00:57:56,956][1157819] Updated weights for policy 0, policy_version 25318 (0.0006) [2024-09-30 00:57:57,487][1157819] Updated weights for policy 0, policy_version 25328 (0.0006) [2024-09-30 00:57:57,972][1157819] Updated weights for policy 0, policy_version 25338 (0.0006) [2024-09-30 00:57:58,538][1157819] Updated weights for policy 0, policy_version 25348 (0.0006) [2024-09-30 00:57:59,046][1157819] Updated weights for policy 0, policy_version 25358 (0.0006) [2024-09-30 00:57:59,556][1157819] Updated weights for policy 0, policy_version 25368 (0.0006) [2024-09-30 00:58:00,063][1157819] Updated weights for policy 0, policy_version 25378 (0.0006) [2024-09-30 00:58:00,466][1157520] Fps is (10 sec: 80281.2, 60 sec: 77073.1, 300 sec: 75852.4). Total num frames: 103976960. Throughput: 0: 19471.9. Samples: 15961604. Policy #0 lag: (min: 0.0, avg: 3.0, max: 6.0) [2024-09-30 00:58:00,466][1157520] Avg episode reward: [(0, '47.855')] [2024-09-30 00:58:00,566][1157819] Updated weights for policy 0, policy_version 25388 (0.0006) [2024-09-30 00:58:01,091][1157819] Updated weights for policy 0, policy_version 25398 (0.0006) [2024-09-30 00:58:01,594][1157819] Updated weights for policy 0, policy_version 25408 (0.0006) [2024-09-30 00:58:02,118][1157819] Updated weights for policy 0, policy_version 25418 (0.0006) [2024-09-30 00:58:02,656][1157819] Updated weights for policy 0, policy_version 25428 (0.0006) [2024-09-30 00:58:03,167][1157819] Updated weights for policy 0, policy_version 25438 (0.0006) [2024-09-30 00:58:03,667][1157819] Updated weights for policy 0, policy_version 25448 (0.0006) [2024-09-30 00:58:04,182][1157819] Updated weights for policy 0, policy_version 25458 (0.0006) [2024-09-30 00:58:04,715][1157819] Updated weights for policy 0, policy_version 25468 (0.0006) [2024-09-30 00:58:05,225][1157819] Updated weights for policy 0, policy_version 25478 (0.0006) [2024-09-30 00:58:05,466][1157520] Fps is (10 sec: 79871.9, 60 sec: 77619.3, 300 sec: 75769.1). Total num frames: 104378368. Throughput: 0: 19664.9. Samples: 16080900. Policy #0 lag: (min: 0.0, avg: 3.0, max: 6.0) [2024-09-30 00:58:05,466][1157520] Avg episode reward: [(0, '50.282')] [2024-09-30 00:58:05,469][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000025483_104378368.pth... [2024-09-30 00:58:05,517][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000021060_86261760.pth [2024-09-30 00:58:05,723][1157819] Updated weights for policy 0, policy_version 25488 (0.0006) [2024-09-30 00:58:06,229][1157819] Updated weights for policy 0, policy_version 25498 (0.0006) [2024-09-30 00:58:06,734][1157819] Updated weights for policy 0, policy_version 25508 (0.0006) [2024-09-30 00:58:07,235][1157819] Updated weights for policy 0, policy_version 25518 (0.0006) [2024-09-30 00:58:07,794][1157819] Updated weights for policy 0, policy_version 25528 (0.0006) [2024-09-30 00:58:08,347][1157819] Updated weights for policy 0, policy_version 25538 (0.0006) [2024-09-30 00:58:08,915][1157819] Updated weights for policy 0, policy_version 25548 (0.0006) [2024-09-30 00:58:09,448][1157819] Updated weights for policy 0, policy_version 25558 (0.0006) [2024-09-30 00:58:10,026][1157819] Updated weights for policy 0, policy_version 25568 (0.0006) [2024-09-30 00:58:10,466][1157520] Fps is (10 sec: 78232.9, 60 sec: 77824.1, 300 sec: 75630.2). Total num frames: 104759296. Throughput: 0: 19747.2. Samples: 16140456. Policy #0 lag: (min: 0.0, avg: 3.0, max: 6.0) [2024-09-30 00:58:10,466][1157520] Avg episode reward: [(0, '47.689')] [2024-09-30 00:58:10,534][1157819] Updated weights for policy 0, policy_version 25578 (0.0006) [2024-09-30 00:58:10,847][1157736] Signal inference workers to stop experience collection... (1250 times) [2024-09-30 00:58:10,847][1157736] Signal inference workers to resume experience collection... (1250 times) [2024-09-30 00:58:10,851][1157819] InferenceWorker_p0-w0: stopping experience collection (1250 times) [2024-09-30 00:58:10,851][1157819] InferenceWorker_p0-w0: resuming experience collection (1250 times) [2024-09-30 00:58:11,084][1157819] Updated weights for policy 0, policy_version 25588 (0.0006) [2024-09-30 00:58:11,657][1157819] Updated weights for policy 0, policy_version 25598 (0.0006) [2024-09-30 00:58:12,177][1157819] Updated weights for policy 0, policy_version 25608 (0.0006) [2024-09-30 00:58:12,738][1157819] Updated weights for policy 0, policy_version 25618 (0.0006) [2024-09-30 00:58:13,289][1157819] Updated weights for policy 0, policy_version 25628 (0.0006) [2024-09-30 00:58:13,847][1157819] Updated weights for policy 0, policy_version 25638 (0.0006) [2024-09-30 00:58:14,399][1157819] Updated weights for policy 0, policy_version 25648 (0.0006) [2024-09-30 00:58:14,949][1157819] Updated weights for policy 0, policy_version 25658 (0.0006) [2024-09-30 00:58:15,466][1157520] Fps is (10 sec: 75365.7, 60 sec: 77824.0, 300 sec: 75491.4). Total num frames: 105132032. Throughput: 0: 19544.9. Samples: 16252044. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 00:58:15,466][1157520] Avg episode reward: [(0, '47.785')] [2024-09-30 00:58:15,498][1157819] Updated weights for policy 0, policy_version 25668 (0.0006) [2024-09-30 00:58:16,055][1157819] Updated weights for policy 0, policy_version 25678 (0.0006) [2024-09-30 00:58:16,596][1157819] Updated weights for policy 0, policy_version 25688 (0.0006) [2024-09-30 00:58:17,136][1157819] Updated weights for policy 0, policy_version 25698 (0.0006) [2024-09-30 00:58:17,647][1157819] Updated weights for policy 0, policy_version 25708 (0.0006) [2024-09-30 00:58:18,167][1157819] Updated weights for policy 0, policy_version 25718 (0.0006) [2024-09-30 00:58:18,702][1157819] Updated weights for policy 0, policy_version 25728 (0.0006) [2024-09-30 00:58:19,201][1157819] Updated weights for policy 0, policy_version 25738 (0.0006) [2024-09-30 00:58:19,751][1157819] Updated weights for policy 0, policy_version 25748 (0.0006) [2024-09-30 00:58:20,242][1157819] Updated weights for policy 0, policy_version 25758 (0.0006) [2024-09-30 00:58:20,466][1157520] Fps is (10 sec: 75776.3, 60 sec: 78028.8, 300 sec: 75505.3). Total num frames: 105517056. Throughput: 0: 19483.5. Samples: 16367048. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 00:58:20,466][1157520] Avg episode reward: [(0, '45.486')] [2024-09-30 00:58:20,797][1157819] Updated weights for policy 0, policy_version 25768 (0.0006) [2024-09-30 00:58:21,266][1157819] Updated weights for policy 0, policy_version 25778 (0.0006) [2024-09-30 00:58:21,771][1157819] Updated weights for policy 0, policy_version 25788 (0.0006) [2024-09-30 00:58:22,267][1157819] Updated weights for policy 0, policy_version 25798 (0.0006) [2024-09-30 00:58:22,766][1157819] Updated weights for policy 0, policy_version 25808 (0.0006) [2024-09-30 00:58:23,234][1157819] Updated weights for policy 0, policy_version 25818 (0.0006) [2024-09-30 00:58:23,785][1157819] Updated weights for policy 0, policy_version 25828 (0.0006) [2024-09-30 00:58:24,301][1157819] Updated weights for policy 0, policy_version 25838 (0.0006) [2024-09-30 00:58:24,797][1157819] Updated weights for policy 0, policy_version 25848 (0.0006) [2024-09-30 00:58:25,315][1157819] Updated weights for policy 0, policy_version 25858 (0.0006) [2024-09-30 00:58:25,466][1157520] Fps is (10 sec: 79053.0, 60 sec: 78370.0, 300 sec: 75602.4). Total num frames: 105922560. Throughput: 0: 19527.3. Samples: 16428276. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 00:58:25,466][1157520] Avg episode reward: [(0, '48.056')] [2024-09-30 00:58:25,827][1157819] Updated weights for policy 0, policy_version 25868 (0.0006) [2024-09-30 00:58:26,364][1157819] Updated weights for policy 0, policy_version 25878 (0.0006) [2024-09-30 00:58:26,902][1157819] Updated weights for policy 0, policy_version 25888 (0.0006) [2024-09-30 00:58:27,451][1157819] Updated weights for policy 0, policy_version 25898 (0.0006) [2024-09-30 00:58:27,967][1157819] Updated weights for policy 0, policy_version 25908 (0.0006) [2024-09-30 00:58:28,578][1157819] Updated weights for policy 0, policy_version 25918 (0.0006) [2024-09-30 00:58:29,088][1157819] Updated weights for policy 0, policy_version 25928 (0.0006) [2024-09-30 00:58:29,627][1157819] Updated weights for policy 0, policy_version 25938 (0.0006) [2024-09-30 00:58:30,169][1157819] Updated weights for policy 0, policy_version 25948 (0.0006) [2024-09-30 00:58:30,466][1157520] Fps is (10 sec: 78643.2, 60 sec: 78097.0, 300 sec: 75616.3). Total num frames: 106303488. Throughput: 0: 19644.2. Samples: 16544448. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 00:58:30,466][1157520] Avg episode reward: [(0, '48.569')] [2024-09-30 00:58:30,723][1157819] Updated weights for policy 0, policy_version 25958 (0.0006) [2024-09-30 00:58:31,267][1157819] Updated weights for policy 0, policy_version 25968 (0.0006) [2024-09-30 00:58:31,802][1157819] Updated weights for policy 0, policy_version 25978 (0.0006) [2024-09-30 00:58:32,374][1157819] Updated weights for policy 0, policy_version 25988 (0.0006) [2024-09-30 00:58:32,931][1157819] Updated weights for policy 0, policy_version 25998 (0.0006) [2024-09-30 00:58:33,472][1157819] Updated weights for policy 0, policy_version 26008 (0.0006) [2024-09-30 00:58:34,022][1157819] Updated weights for policy 0, policy_version 26018 (0.0006) [2024-09-30 00:58:34,550][1157819] Updated weights for policy 0, policy_version 26028 (0.0006) [2024-09-30 00:58:35,092][1157819] Updated weights for policy 0, policy_version 26038 (0.0006) [2024-09-30 00:58:35,466][1157520] Fps is (10 sec: 75365.3, 60 sec: 77755.5, 300 sec: 75574.6). Total num frames: 106676224. Throughput: 0: 19513.6. Samples: 16657276. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 00:58:35,466][1157520] Avg episode reward: [(0, '43.645')] [2024-09-30 00:58:35,635][1157819] Updated weights for policy 0, policy_version 26048 (0.0006) [2024-09-30 00:58:36,182][1157819] Updated weights for policy 0, policy_version 26058 (0.0006) [2024-09-30 00:58:36,747][1157819] Updated weights for policy 0, policy_version 26068 (0.0006) [2024-09-30 00:58:37,287][1157819] Updated weights for policy 0, policy_version 26078 (0.0006) [2024-09-30 00:58:37,812][1157819] Updated weights for policy 0, policy_version 26088 (0.0006) [2024-09-30 00:58:38,374][1157819] Updated weights for policy 0, policy_version 26098 (0.0006) [2024-09-30 00:58:38,929][1157819] Updated weights for policy 0, policy_version 26108 (0.0006) [2024-09-30 00:58:39,471][1157819] Updated weights for policy 0, policy_version 26118 (0.0006) [2024-09-30 00:58:39,995][1157819] Updated weights for policy 0, policy_version 26128 (0.0006) [2024-09-30 00:58:40,466][1157520] Fps is (10 sec: 74957.0, 60 sec: 77619.1, 300 sec: 75560.8). Total num frames: 107053056. Throughput: 0: 19377.8. Samples: 16713464. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 00:58:40,466][1157520] Avg episode reward: [(0, '48.024')] [2024-09-30 00:58:40,549][1157819] Updated weights for policy 0, policy_version 26138 (0.0006) [2024-09-30 00:58:41,104][1157819] Updated weights for policy 0, policy_version 26148 (0.0006) [2024-09-30 00:58:41,620][1157819] Updated weights for policy 0, policy_version 26158 (0.0006) [2024-09-30 00:58:42,174][1157819] Updated weights for policy 0, policy_version 26168 (0.0006) [2024-09-30 00:58:42,743][1157819] Updated weights for policy 0, policy_version 26178 (0.0006) [2024-09-30 00:58:43,316][1157819] Updated weights for policy 0, policy_version 26188 (0.0006) [2024-09-30 00:58:43,882][1157819] Updated weights for policy 0, policy_version 26198 (0.0006) [2024-09-30 00:58:44,480][1157819] Updated weights for policy 0, policy_version 26208 (0.0006) [2024-09-30 00:58:45,060][1157819] Updated weights for policy 0, policy_version 26218 (0.0006) [2024-09-30 00:58:45,466][1157520] Fps is (10 sec: 73729.0, 60 sec: 77414.3, 300 sec: 75519.1). Total num frames: 107413504. Throughput: 0: 19186.5. Samples: 16824996. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 00:58:45,466][1157520] Avg episode reward: [(0, '47.287')] [2024-09-30 00:58:45,654][1157819] Updated weights for policy 0, policy_version 26228 (0.0006) [2024-09-30 00:58:46,230][1157819] Updated weights for policy 0, policy_version 26238 (0.0006) [2024-09-30 00:58:46,777][1157819] Updated weights for policy 0, policy_version 26248 (0.0006) [2024-09-30 00:58:47,415][1157819] Updated weights for policy 0, policy_version 26258 (0.0006) [2024-09-30 00:58:48,001][1157819] Updated weights for policy 0, policy_version 26268 (0.0006) [2024-09-30 00:58:48,554][1157819] Updated weights for policy 0, policy_version 26278 (0.0006) [2024-09-30 00:58:49,160][1157819] Updated weights for policy 0, policy_version 26288 (0.0006) [2024-09-30 00:58:49,737][1157819] Updated weights for policy 0, policy_version 26298 (0.0006) [2024-09-30 00:58:50,310][1157819] Updated weights for policy 0, policy_version 26308 (0.0006) [2024-09-30 00:58:50,466][1157520] Fps is (10 sec: 71270.2, 60 sec: 76526.8, 300 sec: 75435.8). Total num frames: 107765760. Throughput: 0: 18865.5. Samples: 16929848. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 00:58:50,466][1157520] Avg episode reward: [(0, '47.707')] [2024-09-30 00:58:50,884][1157819] Updated weights for policy 0, policy_version 26318 (0.0006) [2024-09-30 00:58:51,443][1157819] Updated weights for policy 0, policy_version 26328 (0.0006) [2024-09-30 00:58:52,030][1157819] Updated weights for policy 0, policy_version 26338 (0.0006) [2024-09-30 00:58:52,566][1157819] Updated weights for policy 0, policy_version 26348 (0.0006) [2024-09-30 00:58:53,112][1157819] Updated weights for policy 0, policy_version 26358 (0.0006) [2024-09-30 00:58:53,659][1157819] Updated weights for policy 0, policy_version 26368 (0.0006) [2024-09-30 00:58:54,213][1157819] Updated weights for policy 0, policy_version 26378 (0.0006) [2024-09-30 00:58:54,787][1157819] Updated weights for policy 0, policy_version 26388 (0.0006) [2024-09-30 00:58:55,324][1157819] Updated weights for policy 0, policy_version 26398 (0.0006) [2024-09-30 00:58:55,466][1157520] Fps is (10 sec: 72089.7, 60 sec: 75912.4, 300 sec: 75421.9). Total num frames: 108134400. Throughput: 0: 18758.0. Samples: 16984564. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 00:58:55,466][1157520] Avg episode reward: [(0, '47.631')] [2024-09-30 00:58:55,894][1157819] Updated weights for policy 0, policy_version 26408 (0.0006) [2024-09-30 00:58:56,451][1157819] Updated weights for policy 0, policy_version 26418 (0.0006) [2024-09-30 00:58:57,028][1157819] Updated weights for policy 0, policy_version 26428 (0.0006) [2024-09-30 00:58:57,579][1157819] Updated weights for policy 0, policy_version 26438 (0.0006) [2024-09-30 00:58:58,131][1157819] Updated weights for policy 0, policy_version 26448 (0.0006) [2024-09-30 00:58:58,710][1157819] Updated weights for policy 0, policy_version 26458 (0.0006) [2024-09-30 00:58:59,264][1157819] Updated weights for policy 0, policy_version 26468 (0.0006) [2024-09-30 00:58:59,827][1157819] Updated weights for policy 0, policy_version 26478 (0.0006) [2024-09-30 00:59:00,403][1157819] Updated weights for policy 0, policy_version 26488 (0.0006) [2024-09-30 00:59:00,466][1157520] Fps is (10 sec: 73317.8, 60 sec: 75366.2, 300 sec: 75394.2). Total num frames: 108498944. Throughput: 0: 18724.1. Samples: 17094628. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 00:59:00,466][1157520] Avg episode reward: [(0, '50.029')] [2024-09-30 00:59:00,907][1157819] Updated weights for policy 0, policy_version 26498 (0.0006) [2024-09-30 00:59:01,445][1157819] Updated weights for policy 0, policy_version 26508 (0.0006) [2024-09-30 00:59:02,022][1157819] Updated weights for policy 0, policy_version 26518 (0.0006) [2024-09-30 00:59:02,565][1157819] Updated weights for policy 0, policy_version 26528 (0.0006) [2024-09-30 00:59:03,137][1157819] Updated weights for policy 0, policy_version 26538 (0.0006) [2024-09-30 00:59:03,730][1157819] Updated weights for policy 0, policy_version 26548 (0.0006) [2024-09-30 00:59:04,297][1157819] Updated weights for policy 0, policy_version 26558 (0.0006) [2024-09-30 00:59:04,866][1157819] Updated weights for policy 0, policy_version 26568 (0.0006) [2024-09-30 00:59:05,451][1157819] Updated weights for policy 0, policy_version 26578 (0.0006) [2024-09-30 00:59:05,466][1157520] Fps is (10 sec: 72908.1, 60 sec: 74751.8, 300 sec: 75338.6). Total num frames: 108863488. Throughput: 0: 18600.7. Samples: 17204080. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 00:59:05,466][1157520] Avg episode reward: [(0, '45.472')] [2024-09-30 00:59:05,984][1157819] Updated weights for policy 0, policy_version 26588 (0.0006) [2024-09-30 00:59:06,534][1157819] Updated weights for policy 0, policy_version 26598 (0.0006) [2024-09-30 00:59:07,069][1157819] Updated weights for policy 0, policy_version 26608 (0.0006) [2024-09-30 00:59:07,602][1157819] Updated weights for policy 0, policy_version 26618 (0.0006) [2024-09-30 00:59:08,162][1157819] Updated weights for policy 0, policy_version 26628 (0.0006) [2024-09-30 00:59:08,676][1157819] Updated weights for policy 0, policy_version 26638 (0.0006) [2024-09-30 00:59:09,165][1157736] Signal inference workers to stop experience collection... (1300 times) [2024-09-30 00:59:09,170][1157819] InferenceWorker_p0-w0: stopping experience collection (1300 times) [2024-09-30 00:59:09,171][1157736] Signal inference workers to resume experience collection... (1300 times) [2024-09-30 00:59:09,175][1157819] InferenceWorker_p0-w0: resuming experience collection (1300 times) [2024-09-30 00:59:09,185][1157819] Updated weights for policy 0, policy_version 26648 (0.0006) [2024-09-30 00:59:09,720][1157819] Updated weights for policy 0, policy_version 26658 (0.0006) [2024-09-30 00:59:10,285][1157819] Updated weights for policy 0, policy_version 26668 (0.0006) [2024-09-30 00:59:10,466][1157520] Fps is (10 sec: 74547.4, 60 sec: 74752.0, 300 sec: 75324.7). Total num frames: 109244416. Throughput: 0: 18486.2. Samples: 17260156. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 00:59:10,466][1157520] Avg episode reward: [(0, '48.158')] [2024-09-30 00:59:10,796][1157819] Updated weights for policy 0, policy_version 26678 (0.0006) [2024-09-30 00:59:11,334][1157819] Updated weights for policy 0, policy_version 26688 (0.0006) [2024-09-30 00:59:11,906][1157819] Updated weights for policy 0, policy_version 26698 (0.0006) [2024-09-30 00:59:12,424][1157819] Updated weights for policy 0, policy_version 26708 (0.0006) [2024-09-30 00:59:12,978][1157819] Updated weights for policy 0, policy_version 26718 (0.0006) [2024-09-30 00:59:13,537][1157819] Updated weights for policy 0, policy_version 26728 (0.0006) [2024-09-30 00:59:14,059][1157819] Updated weights for policy 0, policy_version 26738 (0.0006) [2024-09-30 00:59:14,602][1157819] Updated weights for policy 0, policy_version 26748 (0.0006) [2024-09-30 00:59:15,135][1157819] Updated weights for policy 0, policy_version 26758 (0.0006) [2024-09-30 00:59:15,466][1157520] Fps is (10 sec: 75776.5, 60 sec: 74820.3, 300 sec: 75269.2). Total num frames: 109621248. Throughput: 0: 18441.9. Samples: 17374332. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 00:59:15,466][1157520] Avg episode reward: [(0, '47.033')] [2024-09-30 00:59:15,679][1157819] Updated weights for policy 0, policy_version 26768 (0.0006) [2024-09-30 00:59:16,235][1157819] Updated weights for policy 0, policy_version 26778 (0.0006) [2024-09-30 00:59:16,732][1157819] Updated weights for policy 0, policy_version 26788 (0.0006) [2024-09-30 00:59:17,230][1157819] Updated weights for policy 0, policy_version 26798 (0.0006) [2024-09-30 00:59:17,716][1157819] Updated weights for policy 0, policy_version 26808 (0.0006) [2024-09-30 00:59:18,214][1157819] Updated weights for policy 0, policy_version 26818 (0.0006) [2024-09-30 00:59:18,707][1157819] Updated weights for policy 0, policy_version 26828 (0.0006) [2024-09-30 00:59:19,199][1157819] Updated weights for policy 0, policy_version 26838 (0.0006) [2024-09-30 00:59:19,733][1157819] Updated weights for policy 0, policy_version 26848 (0.0006) [2024-09-30 00:59:20,264][1157819] Updated weights for policy 0, policy_version 26858 (0.0006) [2024-09-30 00:59:20,466][1157520] Fps is (10 sec: 77825.3, 60 sec: 75093.5, 300 sec: 75352.5). Total num frames: 110022656. Throughput: 0: 18578.6. Samples: 17493308. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 00:59:20,466][1157520] Avg episode reward: [(0, '46.711')] [2024-09-30 00:59:20,792][1157819] Updated weights for policy 0, policy_version 26868 (0.0006) [2024-09-30 00:59:21,290][1157819] Updated weights for policy 0, policy_version 26878 (0.0006) [2024-09-30 00:59:21,795][1157819] Updated weights for policy 0, policy_version 26888 (0.0006) [2024-09-30 00:59:22,322][1157819] Updated weights for policy 0, policy_version 26898 (0.0006) [2024-09-30 00:59:22,811][1157819] Updated weights for policy 0, policy_version 26908 (0.0006) [2024-09-30 00:59:23,334][1157819] Updated weights for policy 0, policy_version 26918 (0.0006) [2024-09-30 00:59:23,880][1157819] Updated weights for policy 0, policy_version 26928 (0.0006) [2024-09-30 00:59:24,415][1157819] Updated weights for policy 0, policy_version 26938 (0.0006) [2024-09-30 00:59:24,875][1157819] Updated weights for policy 0, policy_version 26948 (0.0006) [2024-09-30 00:59:25,383][1157819] Updated weights for policy 0, policy_version 26958 (0.0006) [2024-09-30 00:59:25,466][1157520] Fps is (10 sec: 80282.5, 60 sec: 75025.2, 300 sec: 75477.5). Total num frames: 110424064. Throughput: 0: 18651.2. Samples: 17552768. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 00:59:25,466][1157520] Avg episode reward: [(0, '46.825')] [2024-09-30 00:59:25,934][1157819] Updated weights for policy 0, policy_version 26968 (0.0006) [2024-09-30 00:59:26,434][1157819] Updated weights for policy 0, policy_version 26978 (0.0006) [2024-09-30 00:59:26,937][1157819] Updated weights for policy 0, policy_version 26988 (0.0006) [2024-09-30 00:59:27,473][1157819] Updated weights for policy 0, policy_version 26998 (0.0006) [2024-09-30 00:59:27,988][1157819] Updated weights for policy 0, policy_version 27008 (0.0006) [2024-09-30 00:59:28,489][1157819] Updated weights for policy 0, policy_version 27018 (0.0006) [2024-09-30 00:59:29,006][1157819] Updated weights for policy 0, policy_version 27028 (0.0006) [2024-09-30 00:59:29,524][1157819] Updated weights for policy 0, policy_version 27038 (0.0006) [2024-09-30 00:59:30,092][1157819] Updated weights for policy 0, policy_version 27048 (0.0006) [2024-09-30 00:59:30,466][1157520] Fps is (10 sec: 79462.4, 60 sec: 75230.0, 300 sec: 75588.6). Total num frames: 110817280. Throughput: 0: 18823.5. Samples: 17672052. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 00:59:30,466][1157520] Avg episode reward: [(0, '47.519')] [2024-09-30 00:59:30,596][1157819] Updated weights for policy 0, policy_version 27058 (0.0006) [2024-09-30 00:59:31,143][1157819] Updated weights for policy 0, policy_version 27068 (0.0006) [2024-09-30 00:59:31,643][1157819] Updated weights for policy 0, policy_version 27078 (0.0006) [2024-09-30 00:59:32,145][1157819] Updated weights for policy 0, policy_version 27088 (0.0006) [2024-09-30 00:59:32,671][1157819] Updated weights for policy 0, policy_version 27098 (0.0006) [2024-09-30 00:59:33,254][1157819] Updated weights for policy 0, policy_version 27108 (0.0006) [2024-09-30 00:59:33,779][1157819] Updated weights for policy 0, policy_version 27118 (0.0006) [2024-09-30 00:59:34,325][1157819] Updated weights for policy 0, policy_version 27128 (0.0006) [2024-09-30 00:59:34,871][1157819] Updated weights for policy 0, policy_version 27138 (0.0006) [2024-09-30 00:59:35,405][1157819] Updated weights for policy 0, policy_version 27148 (0.0006) [2024-09-30 00:59:35,466][1157520] Fps is (10 sec: 77823.8, 60 sec: 75434.9, 300 sec: 75699.6). Total num frames: 111202304. Throughput: 0: 19065.9. Samples: 17787812. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 00:59:35,466][1157520] Avg episode reward: [(0, '49.212')] [2024-09-30 00:59:35,893][1157819] Updated weights for policy 0, policy_version 27158 (0.0006) [2024-09-30 00:59:36,390][1157736] Signal inference workers to stop experience collection... (1350 times) [2024-09-30 00:59:36,391][1157736] Signal inference workers to resume experience collection... (1350 times) [2024-09-30 00:59:36,394][1157819] InferenceWorker_p0-w0: stopping experience collection (1350 times) [2024-09-30 00:59:36,395][1157819] InferenceWorker_p0-w0: resuming experience collection (1350 times) [2024-09-30 00:59:36,404][1157819] Updated weights for policy 0, policy_version 27168 (0.0006) [2024-09-30 00:59:36,950][1157819] Updated weights for policy 0, policy_version 27178 (0.0006) [2024-09-30 00:59:37,475][1157819] Updated weights for policy 0, policy_version 27188 (0.0006) [2024-09-30 00:59:37,971][1157819] Updated weights for policy 0, policy_version 27198 (0.0006) [2024-09-30 00:59:38,506][1157819] Updated weights for policy 0, policy_version 27208 (0.0006) [2024-09-30 00:59:39,030][1157819] Updated weights for policy 0, policy_version 27218 (0.0006) [2024-09-30 00:59:39,554][1157819] Updated weights for policy 0, policy_version 27228 (0.0005) [2024-09-30 00:59:40,086][1157819] Updated weights for policy 0, policy_version 27238 (0.0006) [2024-09-30 00:59:40,466][1157520] Fps is (10 sec: 77824.0, 60 sec: 75707.8, 300 sec: 75783.0). Total num frames: 111595520. Throughput: 0: 19164.1. Samples: 17846948. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 00:59:40,466][1157520] Avg episode reward: [(0, '47.359')] [2024-09-30 00:59:40,600][1157819] Updated weights for policy 0, policy_version 27248 (0.0006) [2024-09-30 00:59:41,087][1157819] Updated weights for policy 0, policy_version 27258 (0.0006) [2024-09-30 00:59:41,597][1157819] Updated weights for policy 0, policy_version 27268 (0.0006) [2024-09-30 00:59:42,087][1157819] Updated weights for policy 0, policy_version 27278 (0.0006) [2024-09-30 00:59:42,585][1157819] Updated weights for policy 0, policy_version 27288 (0.0006) [2024-09-30 00:59:43,066][1157819] Updated weights for policy 0, policy_version 27298 (0.0006) [2024-09-30 00:59:43,567][1157819] Updated weights for policy 0, policy_version 27308 (0.0006) [2024-09-30 00:59:44,101][1157819] Updated weights for policy 0, policy_version 27318 (0.0006) [2024-09-30 00:59:44,597][1157819] Updated weights for policy 0, policy_version 27328 (0.0006) [2024-09-30 00:59:45,153][1157819] Updated weights for policy 0, policy_version 27338 (0.0006) [2024-09-30 00:59:45,466][1157520] Fps is (10 sec: 79872.2, 60 sec: 76458.8, 300 sec: 75991.2). Total num frames: 112001024. Throughput: 0: 19393.9. Samples: 17967352. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 00:59:45,466][1157520] Avg episode reward: [(0, '48.573')] [2024-09-30 00:59:45,666][1157819] Updated weights for policy 0, policy_version 27348 (0.0006) [2024-09-30 00:59:46,157][1157819] Updated weights for policy 0, policy_version 27358 (0.0006) [2024-09-30 00:59:46,644][1157819] Updated weights for policy 0, policy_version 27368 (0.0006) [2024-09-30 00:59:47,144][1157819] Updated weights for policy 0, policy_version 27378 (0.0006) [2024-09-30 00:59:47,611][1157819] Updated weights for policy 0, policy_version 27388 (0.0006) [2024-09-30 00:59:48,121][1157819] Updated weights for policy 0, policy_version 27398 (0.0006) [2024-09-30 00:59:48,586][1157819] Updated weights for policy 0, policy_version 27408 (0.0006) [2024-09-30 00:59:49,069][1157819] Updated weights for policy 0, policy_version 27418 (0.0006) [2024-09-30 00:59:49,589][1157819] Updated weights for policy 0, policy_version 27428 (0.0006) [2024-09-30 00:59:50,143][1157819] Updated weights for policy 0, policy_version 27438 (0.0006) [2024-09-30 00:59:50,466][1157520] Fps is (10 sec: 81509.8, 60 sec: 77414.4, 300 sec: 76213.4). Total num frames: 112410624. Throughput: 0: 19683.4. Samples: 18089832. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 00:59:50,466][1157520] Avg episode reward: [(0, '48.385')] [2024-09-30 00:59:50,637][1157819] Updated weights for policy 0, policy_version 27448 (0.0006) [2024-09-30 00:59:51,134][1157819] Updated weights for policy 0, policy_version 27458 (0.0006) [2024-09-30 00:59:51,646][1157819] Updated weights for policy 0, policy_version 27468 (0.0006) [2024-09-30 00:59:52,149][1157819] Updated weights for policy 0, policy_version 27478 (0.0006) [2024-09-30 00:59:52,644][1157819] Updated weights for policy 0, policy_version 27488 (0.0006) [2024-09-30 00:59:53,159][1157819] Updated weights for policy 0, policy_version 27498 (0.0006) [2024-09-30 00:59:53,649][1157819] Updated weights for policy 0, policy_version 27508 (0.0006) [2024-09-30 00:59:54,175][1157819] Updated weights for policy 0, policy_version 27518 (0.0006) [2024-09-30 00:59:54,684][1157819] Updated weights for policy 0, policy_version 27528 (0.0006) [2024-09-30 00:59:55,168][1157819] Updated weights for policy 0, policy_version 27538 (0.0006) [2024-09-30 00:59:55,466][1157520] Fps is (10 sec: 81509.6, 60 sec: 78028.8, 300 sec: 76421.6). Total num frames: 112816128. Throughput: 0: 19785.9. Samples: 18150520. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 00:59:55,466][1157520] Avg episode reward: [(0, '48.528')] [2024-09-30 00:59:55,677][1157819] Updated weights for policy 0, policy_version 27548 (0.0006) [2024-09-30 00:59:56,192][1157819] Updated weights for policy 0, policy_version 27558 (0.0006) [2024-09-30 00:59:56,696][1157819] Updated weights for policy 0, policy_version 27568 (0.0006) [2024-09-30 00:59:57,206][1157819] Updated weights for policy 0, policy_version 27578 (0.0006) [2024-09-30 00:59:57,724][1157819] Updated weights for policy 0, policy_version 27588 (0.0006) [2024-09-30 00:59:58,247][1157819] Updated weights for policy 0, policy_version 27598 (0.0006) [2024-09-30 00:59:58,763][1157819] Updated weights for policy 0, policy_version 27608 (0.0006) [2024-09-30 00:59:59,292][1157819] Updated weights for policy 0, policy_version 27618 (0.0006) [2024-09-30 00:59:59,793][1157819] Updated weights for policy 0, policy_version 27628 (0.0006) [2024-09-30 01:00:00,296][1157819] Updated weights for policy 0, policy_version 27638 (0.0006) [2024-09-30 01:00:00,466][1157520] Fps is (10 sec: 80690.5, 60 sec: 78643.2, 300 sec: 76574.4). Total num frames: 113217536. Throughput: 0: 19925.1. Samples: 18270964. Policy #0 lag: (min: 0.0, avg: 1.4, max: 5.0) [2024-09-30 01:00:00,466][1157520] Avg episode reward: [(0, '48.684')] [2024-09-30 01:00:00,847][1157819] Updated weights for policy 0, policy_version 27648 (0.0006) [2024-09-30 01:00:01,343][1157819] Updated weights for policy 0, policy_version 27658 (0.0006) [2024-09-30 01:00:01,844][1157819] Updated weights for policy 0, policy_version 27668 (0.0006) [2024-09-30 01:00:02,351][1157819] Updated weights for policy 0, policy_version 27678 (0.0006) [2024-09-30 01:00:02,851][1157819] Updated weights for policy 0, policy_version 27688 (0.0006) [2024-09-30 01:00:03,349][1157819] Updated weights for policy 0, policy_version 27698 (0.0006) [2024-09-30 01:00:03,857][1157819] Updated weights for policy 0, policy_version 27708 (0.0006) [2024-09-30 01:00:04,348][1157819] Updated weights for policy 0, policy_version 27718 (0.0006) [2024-09-30 01:00:04,870][1157819] Updated weights for policy 0, policy_version 27728 (0.0006) [2024-09-30 01:00:05,364][1157819] Updated weights for policy 0, policy_version 27738 (0.0006) [2024-09-30 01:00:05,466][1157520] Fps is (10 sec: 80281.0, 60 sec: 79257.6, 300 sec: 76838.2). Total num frames: 113618944. Throughput: 0: 19964.5. Samples: 18391716. Policy #0 lag: (min: 0.0, avg: 1.4, max: 5.0) [2024-09-30 01:00:05,466][1157520] Avg episode reward: [(0, '46.619')] [2024-09-30 01:00:05,472][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000027740_113623040.pth... [2024-09-30 01:00:05,520][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000023237_95178752.pth [2024-09-30 01:00:05,875][1157819] Updated weights for policy 0, policy_version 27748 (0.0006) [2024-09-30 01:00:06,369][1157819] Updated weights for policy 0, policy_version 27758 (0.0006) [2024-09-30 01:00:06,901][1157819] Updated weights for policy 0, policy_version 27768 (0.0006) [2024-09-30 01:00:07,418][1157819] Updated weights for policy 0, policy_version 27778 (0.0006) [2024-09-30 01:00:07,925][1157819] Updated weights for policy 0, policy_version 27788 (0.0006) [2024-09-30 01:00:08,403][1157819] Updated weights for policy 0, policy_version 27798 (0.0006) [2024-09-30 01:00:08,873][1157819] Updated weights for policy 0, policy_version 27808 (0.0006) [2024-09-30 01:00:09,338][1157819] Updated weights for policy 0, policy_version 27818 (0.0006) [2024-09-30 01:00:09,784][1157819] Updated weights for policy 0, policy_version 27828 (0.0006) [2024-09-30 01:00:10,276][1157819] Updated weights for policy 0, policy_version 27838 (0.0006) [2024-09-30 01:00:10,466][1157520] Fps is (10 sec: 82331.1, 60 sec: 79940.5, 300 sec: 77171.5). Total num frames: 114040832. Throughput: 0: 19991.2. Samples: 18452372. Policy #0 lag: (min: 0.0, avg: 1.4, max: 5.0) [2024-09-30 01:00:10,466][1157520] Avg episode reward: [(0, '47.882')] [2024-09-30 01:00:10,754][1157819] Updated weights for policy 0, policy_version 27848 (0.0006) [2024-09-30 01:00:11,207][1157819] Updated weights for policy 0, policy_version 27858 (0.0006) [2024-09-30 01:00:11,672][1157819] Updated weights for policy 0, policy_version 27868 (0.0006) [2024-09-30 01:00:12,130][1157819] Updated weights for policy 0, policy_version 27878 (0.0006) [2024-09-30 01:00:12,606][1157819] Updated weights for policy 0, policy_version 27888 (0.0006) [2024-09-30 01:00:13,087][1157819] Updated weights for policy 0, policy_version 27898 (0.0006) [2024-09-30 01:00:13,548][1157819] Updated weights for policy 0, policy_version 27908 (0.0006) [2024-09-30 01:00:13,985][1157819] Updated weights for policy 0, policy_version 27918 (0.0006) [2024-09-30 01:00:14,444][1157819] Updated weights for policy 0, policy_version 27928 (0.0006) [2024-09-30 01:00:14,935][1157819] Updated weights for policy 0, policy_version 27938 (0.0006) [2024-09-30 01:00:15,403][1157819] Updated weights for policy 0, policy_version 27948 (0.0006) [2024-09-30 01:00:15,466][1157520] Fps is (10 sec: 86017.2, 60 sec: 80964.4, 300 sec: 77504.7). Total num frames: 114479104. Throughput: 0: 20251.5. Samples: 18583368. Policy #0 lag: (min: 0.0, avg: 1.4, max: 5.0) [2024-09-30 01:00:15,466][1157520] Avg episode reward: [(0, '47.662')] [2024-09-30 01:00:15,843][1157819] Updated weights for policy 0, policy_version 27958 (0.0006) [2024-09-30 01:00:16,317][1157819] Updated weights for policy 0, policy_version 27968 (0.0006) [2024-09-30 01:00:16,784][1157819] Updated weights for policy 0, policy_version 27978 (0.0006) [2024-09-30 01:00:17,255][1157819] Updated weights for policy 0, policy_version 27988 (0.0006) [2024-09-30 01:00:17,735][1157819] Updated weights for policy 0, policy_version 27998 (0.0006) [2024-09-30 01:00:18,158][1157819] Updated weights for policy 0, policy_version 28008 (0.0006) [2024-09-30 01:00:18,627][1157819] Updated weights for policy 0, policy_version 28018 (0.0006) [2024-09-30 01:00:19,116][1157819] Updated weights for policy 0, policy_version 28028 (0.0006) [2024-09-30 01:00:19,593][1157819] Updated weights for policy 0, policy_version 28038 (0.0006) [2024-09-30 01:00:20,084][1157819] Updated weights for policy 0, policy_version 28048 (0.0006) [2024-09-30 01:00:20,466][1157520] Fps is (10 sec: 87654.2, 60 sec: 81578.7, 300 sec: 77726.8). Total num frames: 114917376. Throughput: 0: 20611.2. Samples: 18715316. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:00:20,466][1157520] Avg episode reward: [(0, '45.598')] [2024-09-30 01:00:20,496][1157819] Updated weights for policy 0, policy_version 28058 (0.0006) [2024-09-30 01:00:20,990][1157819] Updated weights for policy 0, policy_version 28068 (0.0006) [2024-09-30 01:00:21,444][1157819] Updated weights for policy 0, policy_version 28078 (0.0006) [2024-09-30 01:00:21,924][1157819] Updated weights for policy 0, policy_version 28088 (0.0006) [2024-09-30 01:00:22,410][1157819] Updated weights for policy 0, policy_version 28098 (0.0006) [2024-09-30 01:00:22,872][1157819] Updated weights for policy 0, policy_version 28108 (0.0006) [2024-09-30 01:00:23,313][1157819] Updated weights for policy 0, policy_version 28118 (0.0006) [2024-09-30 01:00:23,790][1157819] Updated weights for policy 0, policy_version 28128 (0.0006) [2024-09-30 01:00:24,259][1157819] Updated weights for policy 0, policy_version 28138 (0.0006) [2024-09-30 01:00:24,727][1157819] Updated weights for policy 0, policy_version 28148 (0.0006) [2024-09-30 01:00:25,091][1157736] Signal inference workers to stop experience collection... (1400 times) [2024-09-30 01:00:25,092][1157736] Signal inference workers to resume experience collection... (1400 times) [2024-09-30 01:00:25,097][1157819] InferenceWorker_p0-w0: stopping experience collection (1400 times) [2024-09-30 01:00:25,097][1157819] InferenceWorker_p0-w0: resuming experience collection (1400 times) [2024-09-30 01:00:25,166][1157819] Updated weights for policy 0, policy_version 28158 (0.0006) [2024-09-30 01:00:25,466][1157520] Fps is (10 sec: 88064.2, 60 sec: 82261.3, 300 sec: 77893.5). Total num frames: 115359744. Throughput: 0: 20754.0. Samples: 18780876. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:00:25,466][1157520] Avg episode reward: [(0, '48.513')] [2024-09-30 01:00:25,616][1157819] Updated weights for policy 0, policy_version 28168 (0.0006) [2024-09-30 01:00:26,071][1157819] Updated weights for policy 0, policy_version 28178 (0.0006) [2024-09-30 01:00:26,526][1157819] Updated weights for policy 0, policy_version 28188 (0.0006) [2024-09-30 01:00:26,956][1157819] Updated weights for policy 0, policy_version 28198 (0.0006) [2024-09-30 01:00:27,427][1157819] Updated weights for policy 0, policy_version 28208 (0.0006) [2024-09-30 01:00:27,854][1157819] Updated weights for policy 0, policy_version 28218 (0.0006) [2024-09-30 01:00:28,323][1157819] Updated weights for policy 0, policy_version 28228 (0.0006) [2024-09-30 01:00:28,749][1157819] Updated weights for policy 0, policy_version 28238 (0.0006) [2024-09-30 01:00:29,212][1157819] Updated weights for policy 0, policy_version 28248 (0.0006) [2024-09-30 01:00:29,652][1157819] Updated weights for policy 0, policy_version 28258 (0.0006) [2024-09-30 01:00:30,087][1157819] Updated weights for policy 0, policy_version 28268 (0.0006) [2024-09-30 01:00:30,466][1157520] Fps is (10 sec: 90112.1, 60 sec: 83353.6, 300 sec: 78115.6). Total num frames: 115818496. Throughput: 0: 21089.3. Samples: 18916372. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:00:30,466][1157520] Avg episode reward: [(0, '47.047')] [2024-09-30 01:00:30,548][1157819] Updated weights for policy 0, policy_version 28278 (0.0006) [2024-09-30 01:00:30,993][1157819] Updated weights for policy 0, policy_version 28288 (0.0006) [2024-09-30 01:00:31,424][1157819] Updated weights for policy 0, policy_version 28298 (0.0006) [2024-09-30 01:00:31,914][1157819] Updated weights for policy 0, policy_version 28308 (0.0006) [2024-09-30 01:00:32,328][1157819] Updated weights for policy 0, policy_version 28318 (0.0006) [2024-09-30 01:00:32,794][1157819] Updated weights for policy 0, policy_version 28328 (0.0006) [2024-09-30 01:00:33,245][1157819] Updated weights for policy 0, policy_version 28338 (0.0006) [2024-09-30 01:00:33,733][1157819] Updated weights for policy 0, policy_version 28348 (0.0006) [2024-09-30 01:00:34,218][1157819] Updated weights for policy 0, policy_version 28358 (0.0006) [2024-09-30 01:00:34,688][1157819] Updated weights for policy 0, policy_version 28368 (0.0006) [2024-09-30 01:00:35,130][1157819] Updated weights for policy 0, policy_version 28378 (0.0006) [2024-09-30 01:00:35,466][1157520] Fps is (10 sec: 90521.7, 60 sec: 84377.6, 300 sec: 78268.3). Total num frames: 116264960. Throughput: 0: 21367.7. Samples: 19051376. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:00:35,466][1157520] Avg episode reward: [(0, '46.437')] [2024-09-30 01:00:35,585][1157819] Updated weights for policy 0, policy_version 28388 (0.0006) [2024-09-30 01:00:36,062][1157819] Updated weights for policy 0, policy_version 28398 (0.0006) [2024-09-30 01:00:36,539][1157819] Updated weights for policy 0, policy_version 28408 (0.0006) [2024-09-30 01:00:37,014][1157819] Updated weights for policy 0, policy_version 28418 (0.0006) [2024-09-30 01:00:37,476][1157819] Updated weights for policy 0, policy_version 28428 (0.0006) [2024-09-30 01:00:37,919][1157819] Updated weights for policy 0, policy_version 28438 (0.0006) [2024-09-30 01:00:38,391][1157819] Updated weights for policy 0, policy_version 28448 (0.0006) [2024-09-30 01:00:38,865][1157819] Updated weights for policy 0, policy_version 28458 (0.0006) [2024-09-30 01:00:39,339][1157819] Updated weights for policy 0, policy_version 28468 (0.0006) [2024-09-30 01:00:39,822][1157819] Updated weights for policy 0, policy_version 28478 (0.0006) [2024-09-30 01:00:40,286][1157819] Updated weights for policy 0, policy_version 28488 (0.0006) [2024-09-30 01:00:40,466][1157520] Fps is (10 sec: 88063.8, 60 sec: 85060.3, 300 sec: 78421.1). Total num frames: 116699136. Throughput: 0: 21484.8. Samples: 19117332. Policy #0 lag: (min: 1.0, avg: 2.2, max: 5.0) [2024-09-30 01:00:40,466][1157520] Avg episode reward: [(0, '49.236')] [2024-09-30 01:00:40,752][1157819] Updated weights for policy 0, policy_version 28498 (0.0006) [2024-09-30 01:00:41,223][1157819] Updated weights for policy 0, policy_version 28508 (0.0006) [2024-09-30 01:00:41,712][1157819] Updated weights for policy 0, policy_version 28518 (0.0007) [2024-09-30 01:00:42,205][1157819] Updated weights for policy 0, policy_version 28528 (0.0007) [2024-09-30 01:00:42,660][1157819] Updated weights for policy 0, policy_version 28538 (0.0006) [2024-09-30 01:00:43,134][1157819] Updated weights for policy 0, policy_version 28548 (0.0006) [2024-09-30 01:00:43,585][1157819] Updated weights for policy 0, policy_version 28558 (0.0006) [2024-09-30 01:00:44,073][1157819] Updated weights for policy 0, policy_version 28568 (0.0006) [2024-09-30 01:00:44,543][1157819] Updated weights for policy 0, policy_version 28578 (0.0006) [2024-09-30 01:00:45,020][1157819] Updated weights for policy 0, policy_version 28588 (0.0006) [2024-09-30 01:00:45,466][1157520] Fps is (10 sec: 86834.5, 60 sec: 85538.0, 300 sec: 78559.9). Total num frames: 117133312. Throughput: 0: 21698.7. Samples: 19247404. Policy #0 lag: (min: 1.0, avg: 2.2, max: 5.0) [2024-09-30 01:00:45,466][1157520] Avg episode reward: [(0, '48.119')] [2024-09-30 01:00:45,479][1157819] Updated weights for policy 0, policy_version 28598 (0.0006) [2024-09-30 01:00:45,942][1157819] Updated weights for policy 0, policy_version 28608 (0.0006) [2024-09-30 01:00:46,426][1157819] Updated weights for policy 0, policy_version 28618 (0.0006) [2024-09-30 01:00:46,893][1157819] Updated weights for policy 0, policy_version 28628 (0.0006) [2024-09-30 01:00:47,366][1157819] Updated weights for policy 0, policy_version 28638 (0.0006) [2024-09-30 01:00:47,831][1157819] Updated weights for policy 0, policy_version 28648 (0.0006) [2024-09-30 01:00:48,292][1157819] Updated weights for policy 0, policy_version 28658 (0.0006) [2024-09-30 01:00:48,773][1157819] Updated weights for policy 0, policy_version 28668 (0.0006) [2024-09-30 01:00:49,233][1157819] Updated weights for policy 0, policy_version 28678 (0.0006) [2024-09-30 01:00:49,682][1157819] Updated weights for policy 0, policy_version 28688 (0.0006) [2024-09-30 01:00:50,161][1157819] Updated weights for policy 0, policy_version 28698 (0.0006) [2024-09-30 01:00:50,466][1157520] Fps is (10 sec: 87244.7, 60 sec: 86016.1, 300 sec: 78684.9). Total num frames: 117571584. Throughput: 0: 21931.2. Samples: 19378616. Policy #0 lag: (min: 1.0, avg: 2.2, max: 5.0) [2024-09-30 01:00:50,466][1157520] Avg episode reward: [(0, '49.148')] [2024-09-30 01:00:50,647][1157819] Updated weights for policy 0, policy_version 28708 (0.0006) [2024-09-30 01:00:51,100][1157819] Updated weights for policy 0, policy_version 28718 (0.0006) [2024-09-30 01:00:51,576][1157819] Updated weights for policy 0, policy_version 28728 (0.0006) [2024-09-30 01:00:52,028][1157819] Updated weights for policy 0, policy_version 28738 (0.0006) [2024-09-30 01:00:52,510][1157819] Updated weights for policy 0, policy_version 28748 (0.0006) [2024-09-30 01:00:52,988][1157819] Updated weights for policy 0, policy_version 28758 (0.0006) [2024-09-30 01:00:53,466][1157819] Updated weights for policy 0, policy_version 28768 (0.0006) [2024-09-30 01:00:53,959][1157819] Updated weights for policy 0, policy_version 28778 (0.0006) [2024-09-30 01:00:54,371][1157819] Updated weights for policy 0, policy_version 28788 (0.0006) [2024-09-30 01:00:54,867][1157819] Updated weights for policy 0, policy_version 28798 (0.0006) [2024-09-30 01:00:55,328][1157819] Updated weights for policy 0, policy_version 28808 (0.0006) [2024-09-30 01:00:55,466][1157520] Fps is (10 sec: 87245.5, 60 sec: 86494.0, 300 sec: 78754.3). Total num frames: 118005760. Throughput: 0: 22032.6. Samples: 19443840. Policy #0 lag: (min: 1.0, avg: 2.2, max: 5.0) [2024-09-30 01:00:55,466][1157520] Avg episode reward: [(0, '47.830')] [2024-09-30 01:00:55,818][1157819] Updated weights for policy 0, policy_version 28818 (0.0006) [2024-09-30 01:00:56,275][1157819] Updated weights for policy 0, policy_version 28828 (0.0006) [2024-09-30 01:00:56,720][1157819] Updated weights for policy 0, policy_version 28838 (0.0006) [2024-09-30 01:00:57,205][1157819] Updated weights for policy 0, policy_version 28848 (0.0006) [2024-09-30 01:00:57,674][1157819] Updated weights for policy 0, policy_version 28858 (0.0006) [2024-09-30 01:00:58,145][1157819] Updated weights for policy 0, policy_version 28868 (0.0006) [2024-09-30 01:00:58,591][1157819] Updated weights for policy 0, policy_version 28878 (0.0006) [2024-09-30 01:00:58,879][1157736] Signal inference workers to stop experience collection... (1450 times) [2024-09-30 01:00:58,880][1157736] Signal inference workers to resume experience collection... (1450 times) [2024-09-30 01:00:58,885][1157819] InferenceWorker_p0-w0: stopping experience collection (1450 times) [2024-09-30 01:00:58,885][1157819] InferenceWorker_p0-w0: resuming experience collection (1450 times) [2024-09-30 01:00:59,048][1157819] Updated weights for policy 0, policy_version 28888 (0.0006) [2024-09-30 01:00:59,530][1157819] Updated weights for policy 0, policy_version 28898 (0.0006) [2024-09-30 01:01:00,014][1157819] Updated weights for policy 0, policy_version 28908 (0.0006) [2024-09-30 01:01:00,466][1157520] Fps is (10 sec: 87244.4, 60 sec: 87108.4, 300 sec: 78865.4). Total num frames: 118444032. Throughput: 0: 22041.0. Samples: 19575216. Policy #0 lag: (min: 0.0, avg: 1.9, max: 4.0) [2024-09-30 01:01:00,466][1157520] Avg episode reward: [(0, '48.611')] [2024-09-30 01:01:00,468][1157819] Updated weights for policy 0, policy_version 28918 (0.0006) [2024-09-30 01:01:00,880][1157819] Updated weights for policy 0, policy_version 28928 (0.0006) [2024-09-30 01:01:01,369][1157819] Updated weights for policy 0, policy_version 28938 (0.0006) [2024-09-30 01:01:01,858][1157819] Updated weights for policy 0, policy_version 28948 (0.0006) [2024-09-30 01:01:02,321][1157819] Updated weights for policy 0, policy_version 28958 (0.0006) [2024-09-30 01:01:02,774][1157819] Updated weights for policy 0, policy_version 28968 (0.0006) [2024-09-30 01:01:03,224][1157819] Updated weights for policy 0, policy_version 28978 (0.0006) [2024-09-30 01:01:03,712][1157819] Updated weights for policy 0, policy_version 28988 (0.0006) [2024-09-30 01:01:04,195][1157819] Updated weights for policy 0, policy_version 28998 (0.0006) [2024-09-30 01:01:04,643][1157819] Updated weights for policy 0, policy_version 29008 (0.0006) [2024-09-30 01:01:05,080][1157819] Updated weights for policy 0, policy_version 29018 (0.0006) [2024-09-30 01:01:05,466][1157520] Fps is (10 sec: 88063.9, 60 sec: 87791.2, 300 sec: 79170.8). Total num frames: 118886400. Throughput: 0: 22049.4. Samples: 19707540. Policy #0 lag: (min: 0.0, avg: 1.9, max: 4.0) [2024-09-30 01:01:05,466][1157520] Avg episode reward: [(0, '48.624')] [2024-09-30 01:01:05,563][1157819] Updated weights for policy 0, policy_version 29028 (0.0006) [2024-09-30 01:01:06,046][1157819] Updated weights for policy 0, policy_version 29038 (0.0006) [2024-09-30 01:01:06,489][1157819] Updated weights for policy 0, policy_version 29048 (0.0006) [2024-09-30 01:01:06,953][1157819] Updated weights for policy 0, policy_version 29058 (0.0006) [2024-09-30 01:01:07,432][1157819] Updated weights for policy 0, policy_version 29068 (0.0006) [2024-09-30 01:01:07,888][1157819] Updated weights for policy 0, policy_version 29078 (0.0006) [2024-09-30 01:01:08,359][1157819] Updated weights for policy 0, policy_version 29088 (0.0006) [2024-09-30 01:01:08,807][1157819] Updated weights for policy 0, policy_version 29098 (0.0006) [2024-09-30 01:01:09,271][1157819] Updated weights for policy 0, policy_version 29108 (0.0006) [2024-09-30 01:01:09,735][1157819] Updated weights for policy 0, policy_version 29118 (0.0006) [2024-09-30 01:01:10,181][1157819] Updated weights for policy 0, policy_version 29128 (0.0006) [2024-09-30 01:01:10,466][1157520] Fps is (10 sec: 88474.3, 60 sec: 88132.3, 300 sec: 79365.2). Total num frames: 119328768. Throughput: 0: 22061.9. Samples: 19773660. Policy #0 lag: (min: 0.0, avg: 1.9, max: 4.0) [2024-09-30 01:01:10,466][1157520] Avg episode reward: [(0, '49.916')] [2024-09-30 01:01:10,676][1157819] Updated weights for policy 0, policy_version 29138 (0.0006) [2024-09-30 01:01:11,127][1157819] Updated weights for policy 0, policy_version 29148 (0.0006) [2024-09-30 01:01:11,610][1157819] Updated weights for policy 0, policy_version 29158 (0.0006) [2024-09-30 01:01:12,072][1157819] Updated weights for policy 0, policy_version 29168 (0.0006) [2024-09-30 01:01:12,552][1157819] Updated weights for policy 0, policy_version 29178 (0.0006) [2024-09-30 01:01:13,056][1157819] Updated weights for policy 0, policy_version 29188 (0.0006) [2024-09-30 01:01:13,547][1157819] Updated weights for policy 0, policy_version 29198 (0.0006) [2024-09-30 01:01:14,047][1157819] Updated weights for policy 0, policy_version 29208 (0.0006) [2024-09-30 01:01:14,543][1157819] Updated weights for policy 0, policy_version 29218 (0.0006) [2024-09-30 01:01:15,042][1157819] Updated weights for policy 0, policy_version 29228 (0.0006) [2024-09-30 01:01:15,466][1157520] Fps is (10 sec: 86015.7, 60 sec: 87790.9, 300 sec: 79490.2). Total num frames: 119746560. Throughput: 0: 21946.1. Samples: 19903948. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:01:15,466][1157520] Avg episode reward: [(0, '49.560')] [2024-09-30 01:01:15,538][1157819] Updated weights for policy 0, policy_version 29238 (0.0006) [2024-09-30 01:01:16,067][1157819] Updated weights for policy 0, policy_version 29248 (0.0006) [2024-09-30 01:01:16,599][1157819] Updated weights for policy 0, policy_version 29258 (0.0006) [2024-09-30 01:01:17,114][1157819] Updated weights for policy 0, policy_version 29268 (0.0006) [2024-09-30 01:01:17,627][1157819] Updated weights for policy 0, policy_version 29278 (0.0006) [2024-09-30 01:01:18,194][1157819] Updated weights for policy 0, policy_version 29288 (0.0006) [2024-09-30 01:01:18,729][1157819] Updated weights for policy 0, policy_version 29298 (0.0006) [2024-09-30 01:01:19,287][1157819] Updated weights for policy 0, policy_version 29308 (0.0006) [2024-09-30 01:01:19,824][1157819] Updated weights for policy 0, policy_version 29318 (0.0006) [2024-09-30 01:01:20,325][1157819] Updated weights for policy 0, policy_version 29328 (0.0006) [2024-09-30 01:01:20,466][1157520] Fps is (10 sec: 80280.9, 60 sec: 86903.3, 300 sec: 79517.9). Total num frames: 120131584. Throughput: 0: 21549.3. Samples: 20021096. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:01:20,466][1157520] Avg episode reward: [(0, '47.751')] [2024-09-30 01:01:20,863][1157819] Updated weights for policy 0, policy_version 29338 (0.0006) [2024-09-30 01:01:21,434][1157819] Updated weights for policy 0, policy_version 29348 (0.0006) [2024-09-30 01:01:21,966][1157819] Updated weights for policy 0, policy_version 29358 (0.0006) [2024-09-30 01:01:22,510][1157819] Updated weights for policy 0, policy_version 29368 (0.0006) [2024-09-30 01:01:23,046][1157819] Updated weights for policy 0, policy_version 29378 (0.0006) [2024-09-30 01:01:23,563][1157819] Updated weights for policy 0, policy_version 29388 (0.0006) [2024-09-30 01:01:24,049][1157819] Updated weights for policy 0, policy_version 29398 (0.0006) [2024-09-30 01:01:24,559][1157819] Updated weights for policy 0, policy_version 29408 (0.0006) [2024-09-30 01:01:25,078][1157819] Updated weights for policy 0, policy_version 29418 (0.0006) [2024-09-30 01:01:25,466][1157520] Fps is (10 sec: 77823.5, 60 sec: 86084.1, 300 sec: 79559.6). Total num frames: 120524800. Throughput: 0: 21356.9. Samples: 20078396. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:01:25,466][1157520] Avg episode reward: [(0, '49.572')] [2024-09-30 01:01:25,586][1157819] Updated weights for policy 0, policy_version 29428 (0.0006) [2024-09-30 01:01:26,082][1157819] Updated weights for policy 0, policy_version 29438 (0.0006) [2024-09-30 01:01:26,593][1157819] Updated weights for policy 0, policy_version 29448 (0.0006) [2024-09-30 01:01:27,078][1157819] Updated weights for policy 0, policy_version 29458 (0.0006) [2024-09-30 01:01:27,584][1157819] Updated weights for policy 0, policy_version 29468 (0.0006) [2024-09-30 01:01:28,108][1157819] Updated weights for policy 0, policy_version 29478 (0.0006) [2024-09-30 01:01:28,600][1157819] Updated weights for policy 0, policy_version 29488 (0.0006) [2024-09-30 01:01:29,106][1157819] Updated weights for policy 0, policy_version 29498 (0.0006) [2024-09-30 01:01:29,622][1157819] Updated weights for policy 0, policy_version 29508 (0.0006) [2024-09-30 01:01:30,116][1157819] Updated weights for policy 0, policy_version 29518 (0.0006) [2024-09-30 01:01:30,466][1157520] Fps is (10 sec: 80282.3, 60 sec: 85265.1, 300 sec: 79601.3). Total num frames: 120934400. Throughput: 0: 21155.6. Samples: 20199404. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:01:30,466][1157520] Avg episode reward: [(0, '48.075')] [2024-09-30 01:01:30,596][1157819] Updated weights for policy 0, policy_version 29528 (0.0006) [2024-09-30 01:01:31,081][1157819] Updated weights for policy 0, policy_version 29538 (0.0006) [2024-09-30 01:01:31,559][1157819] Updated weights for policy 0, policy_version 29548 (0.0006) [2024-09-30 01:01:32,054][1157736] Signal inference workers to stop experience collection... (1500 times) [2024-09-30 01:01:32,057][1157736] Signal inference workers to resume experience collection... (1500 times) [2024-09-30 01:01:32,057][1157819] Updated weights for policy 0, policy_version 29558 (0.0006) [2024-09-30 01:01:32,061][1157819] InferenceWorker_p0-w0: stopping experience collection (1500 times) [2024-09-30 01:01:32,063][1157819] InferenceWorker_p0-w0: resuming experience collection (1500 times) [2024-09-30 01:01:32,554][1157819] Updated weights for policy 0, policy_version 29568 (0.0006) [2024-09-30 01:01:33,053][1157819] Updated weights for policy 0, policy_version 29578 (0.0006) [2024-09-30 01:01:33,540][1157819] Updated weights for policy 0, policy_version 29588 (0.0006) [2024-09-30 01:01:34,023][1157819] Updated weights for policy 0, policy_version 29598 (0.0006) [2024-09-30 01:01:34,520][1157819] Updated weights for policy 0, policy_version 29608 (0.0006) [2024-09-30 01:01:35,005][1157819] Updated weights for policy 0, policy_version 29618 (0.0006) [2024-09-30 01:01:35,466][1157520] Fps is (10 sec: 82739.8, 60 sec: 84787.2, 300 sec: 79684.6). Total num frames: 121352192. Throughput: 0: 21018.9. Samples: 20324468. Policy #0 lag: (min: 0.0, avg: 2.6, max: 7.0) [2024-09-30 01:01:35,466][1157520] Avg episode reward: [(0, '48.407')] [2024-09-30 01:01:35,500][1157819] Updated weights for policy 0, policy_version 29628 (0.0006) [2024-09-30 01:01:35,997][1157819] Updated weights for policy 0, policy_version 29638 (0.0006) [2024-09-30 01:01:36,490][1157819] Updated weights for policy 0, policy_version 29648 (0.0006) [2024-09-30 01:01:36,981][1157819] Updated weights for policy 0, policy_version 29658 (0.0006) [2024-09-30 01:01:37,513][1157819] Updated weights for policy 0, policy_version 29668 (0.0006) [2024-09-30 01:01:38,016][1157819] Updated weights for policy 0, policy_version 29678 (0.0006) [2024-09-30 01:01:38,523][1157819] Updated weights for policy 0, policy_version 29688 (0.0006) [2024-09-30 01:01:39,037][1157819] Updated weights for policy 0, policy_version 29698 (0.0006) [2024-09-30 01:01:39,606][1157819] Updated weights for policy 0, policy_version 29708 (0.0006) [2024-09-30 01:01:40,096][1157819] Updated weights for policy 0, policy_version 29718 (0.0006) [2024-09-30 01:01:40,466][1157520] Fps is (10 sec: 81920.0, 60 sec: 84241.1, 300 sec: 79712.3). Total num frames: 121753600. Throughput: 0: 20928.8. Samples: 20385636. Policy #0 lag: (min: 0.0, avg: 2.6, max: 7.0) [2024-09-30 01:01:40,466][1157520] Avg episode reward: [(0, '50.812')] [2024-09-30 01:01:40,606][1157819] Updated weights for policy 0, policy_version 29728 (0.0006) [2024-09-30 01:01:41,123][1157819] Updated weights for policy 0, policy_version 29738 (0.0006) [2024-09-30 01:01:41,651][1157819] Updated weights for policy 0, policy_version 29748 (0.0006) [2024-09-30 01:01:42,155][1157819] Updated weights for policy 0, policy_version 29758 (0.0006) [2024-09-30 01:01:42,653][1157819] Updated weights for policy 0, policy_version 29768 (0.0006) [2024-09-30 01:01:43,155][1157819] Updated weights for policy 0, policy_version 29778 (0.0006) [2024-09-30 01:01:43,692][1157819] Updated weights for policy 0, policy_version 29788 (0.0006) [2024-09-30 01:01:44,203][1157819] Updated weights for policy 0, policy_version 29798 (0.0006) [2024-09-30 01:01:44,707][1157819] Updated weights for policy 0, policy_version 29808 (0.0006) [2024-09-30 01:01:45,222][1157819] Updated weights for policy 0, policy_version 29818 (0.0006) [2024-09-30 01:01:45,466][1157520] Fps is (10 sec: 79872.2, 60 sec: 83626.8, 300 sec: 79795.7). Total num frames: 122150912. Throughput: 0: 20658.5. Samples: 20504848. Policy #0 lag: (min: 0.0, avg: 2.6, max: 7.0) [2024-09-30 01:01:45,466][1157520] Avg episode reward: [(0, '50.107')] [2024-09-30 01:01:45,718][1157819] Updated weights for policy 0, policy_version 29828 (0.0006) [2024-09-30 01:01:46,225][1157819] Updated weights for policy 0, policy_version 29838 (0.0006) [2024-09-30 01:01:46,742][1157819] Updated weights for policy 0, policy_version 29848 (0.0006) [2024-09-30 01:01:47,235][1157819] Updated weights for policy 0, policy_version 29858 (0.0006) [2024-09-30 01:01:47,740][1157819] Updated weights for policy 0, policy_version 29868 (0.0006) [2024-09-30 01:01:48,269][1157819] Updated weights for policy 0, policy_version 29878 (0.0006) [2024-09-30 01:01:48,791][1157819] Updated weights for policy 0, policy_version 29888 (0.0006) [2024-09-30 01:01:49,308][1157819] Updated weights for policy 0, policy_version 29898 (0.0006) [2024-09-30 01:01:49,795][1157819] Updated weights for policy 0, policy_version 29908 (0.0006) [2024-09-30 01:01:50,304][1157819] Updated weights for policy 0, policy_version 29918 (0.0006) [2024-09-30 01:01:50,466][1157520] Fps is (10 sec: 80281.5, 60 sec: 83080.6, 300 sec: 79920.7). Total num frames: 122556416. Throughput: 0: 20411.3. Samples: 20626048. Policy #0 lag: (min: 0.0, avg: 2.6, max: 7.0) [2024-09-30 01:01:50,466][1157520] Avg episode reward: [(0, '51.060')] [2024-09-30 01:01:50,800][1157819] Updated weights for policy 0, policy_version 29928 (0.0006) [2024-09-30 01:01:51,296][1157819] Updated weights for policy 0, policy_version 29938 (0.0006) [2024-09-30 01:01:51,828][1157819] Updated weights for policy 0, policy_version 29948 (0.0006) [2024-09-30 01:01:52,345][1157819] Updated weights for policy 0, policy_version 29958 (0.0006) [2024-09-30 01:01:52,835][1157819] Updated weights for policy 0, policy_version 29968 (0.0006) [2024-09-30 01:01:53,336][1157819] Updated weights for policy 0, policy_version 29978 (0.0006) [2024-09-30 01:01:53,855][1157819] Updated weights for policy 0, policy_version 29988 (0.0006) [2024-09-30 01:01:54,358][1157819] Updated weights for policy 0, policy_version 29998 (0.0006) [2024-09-30 01:01:54,857][1157819] Updated weights for policy 0, policy_version 30008 (0.0006) [2024-09-30 01:01:55,353][1157819] Updated weights for policy 0, policy_version 30018 (0.0006) [2024-09-30 01:01:55,466][1157520] Fps is (10 sec: 81100.6, 60 sec: 82602.6, 300 sec: 80031.7). Total num frames: 122961920. Throughput: 0: 20278.2. Samples: 20686180. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:01:55,466][1157520] Avg episode reward: [(0, '49.369')] [2024-09-30 01:01:55,903][1157819] Updated weights for policy 0, policy_version 30028 (0.0006) [2024-09-30 01:01:56,412][1157819] Updated weights for policy 0, policy_version 30038 (0.0006) [2024-09-30 01:01:56,905][1157819] Updated weights for policy 0, policy_version 30048 (0.0006) [2024-09-30 01:01:57,399][1157819] Updated weights for policy 0, policy_version 30058 (0.0006) [2024-09-30 01:01:57,938][1157819] Updated weights for policy 0, policy_version 30068 (0.0006) [2024-09-30 01:01:58,424][1157819] Updated weights for policy 0, policy_version 30078 (0.0006) [2024-09-30 01:01:58,926][1157819] Updated weights for policy 0, policy_version 30088 (0.0006) [2024-09-30 01:01:59,461][1157819] Updated weights for policy 0, policy_version 30098 (0.0006) [2024-09-30 01:02:00,000][1157819] Updated weights for policy 0, policy_version 30108 (0.0006) [2024-09-30 01:02:00,466][1157520] Fps is (10 sec: 80281.7, 60 sec: 81920.1, 300 sec: 80128.9). Total num frames: 123359232. Throughput: 0: 20061.8. Samples: 20806728. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:02:00,466][1157520] Avg episode reward: [(0, '49.057')] [2024-09-30 01:02:00,513][1157819] Updated weights for policy 0, policy_version 30118 (0.0006) [2024-09-30 01:02:01,072][1157819] Updated weights for policy 0, policy_version 30128 (0.0006) [2024-09-30 01:02:01,581][1157819] Updated weights for policy 0, policy_version 30138 (0.0006) [2024-09-30 01:02:02,111][1157819] Updated weights for policy 0, policy_version 30148 (0.0006) [2024-09-30 01:02:02,684][1157819] Updated weights for policy 0, policy_version 30158 (0.0006) [2024-09-30 01:02:03,231][1157819] Updated weights for policy 0, policy_version 30168 (0.0006) [2024-09-30 01:02:03,771][1157819] Updated weights for policy 0, policy_version 30178 (0.0006) [2024-09-30 01:02:04,344][1157819] Updated weights for policy 0, policy_version 30188 (0.0006) [2024-09-30 01:02:04,920][1157819] Updated weights for policy 0, policy_version 30198 (0.0007) [2024-09-30 01:02:05,455][1157819] Updated weights for policy 0, policy_version 30208 (0.0006) [2024-09-30 01:02:05,466][1157520] Fps is (10 sec: 77004.1, 60 sec: 80759.3, 300 sec: 80142.8). Total num frames: 123731968. Throughput: 0: 19981.8. Samples: 20920276. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:02:05,466][1157520] Avg episode reward: [(0, '48.864')] [2024-09-30 01:02:05,471][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000030208_123731968.pth... [2024-09-30 01:02:05,515][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000025483_104378368.pth [2024-09-30 01:02:06,048][1157819] Updated weights for policy 0, policy_version 30218 (0.0006) [2024-09-30 01:02:06,632][1157819] Updated weights for policy 0, policy_version 30228 (0.0006) [2024-09-30 01:02:07,192][1157819] Updated weights for policy 0, policy_version 30238 (0.0006) [2024-09-30 01:02:07,800][1157819] Updated weights for policy 0, policy_version 30248 (0.0006) [2024-09-30 01:02:08,408][1157819] Updated weights for policy 0, policy_version 30258 (0.0006) [2024-09-30 01:02:08,941][1157819] Updated weights for policy 0, policy_version 30268 (0.0006) [2024-09-30 01:02:09,582][1157819] Updated weights for policy 0, policy_version 30278 (0.0006) [2024-09-30 01:02:10,121][1157819] Updated weights for policy 0, policy_version 30288 (0.0006) [2024-09-30 01:02:10,466][1157520] Fps is (10 sec: 72089.0, 60 sec: 79189.2, 300 sec: 80059.5). Total num frames: 124080128. Throughput: 0: 19896.9. Samples: 20973756. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:02:10,466][1157520] Avg episode reward: [(0, '51.573')] [2024-09-30 01:02:10,467][1157736] Saving new best policy, reward=51.573! [2024-09-30 01:02:10,728][1157819] Updated weights for policy 0, policy_version 30298 (0.0006) [2024-09-30 01:02:11,239][1157819] Updated weights for policy 0, policy_version 30308 (0.0006) [2024-09-30 01:02:11,680][1157736] Signal inference workers to stop experience collection... (1550 times) [2024-09-30 01:02:11,682][1157736] Signal inference workers to resume experience collection... (1550 times) [2024-09-30 01:02:11,685][1157819] InferenceWorker_p0-w0: stopping experience collection (1550 times) [2024-09-30 01:02:11,685][1157819] InferenceWorker_p0-w0: resuming experience collection (1550 times) [2024-09-30 01:02:11,838][1157819] Updated weights for policy 0, policy_version 30318 (0.0006) [2024-09-30 01:02:12,370][1157819] Updated weights for policy 0, policy_version 30328 (0.0006) [2024-09-30 01:02:12,978][1157819] Updated weights for policy 0, policy_version 30338 (0.0006) [2024-09-30 01:02:13,477][1157819] Updated weights for policy 0, policy_version 30348 (0.0006) [2024-09-30 01:02:14,024][1157819] Updated weights for policy 0, policy_version 30358 (0.0006) [2024-09-30 01:02:14,604][1157819] Updated weights for policy 0, policy_version 30368 (0.0006) [2024-09-30 01:02:15,117][1157819] Updated weights for policy 0, policy_version 30378 (0.0006) [2024-09-30 01:02:15,466][1157520] Fps is (10 sec: 72089.7, 60 sec: 78438.3, 300 sec: 80059.4). Total num frames: 124452864. Throughput: 0: 19614.3. Samples: 21082048. Policy #0 lag: (min: 0.0, avg: 1.4, max: 5.0) [2024-09-30 01:02:15,466][1157520] Avg episode reward: [(0, '50.025')] [2024-09-30 01:02:15,661][1157819] Updated weights for policy 0, policy_version 30388 (0.0006) [2024-09-30 01:02:16,253][1157819] Updated weights for policy 0, policy_version 30398 (0.0006) [2024-09-30 01:02:16,776][1157819] Updated weights for policy 0, policy_version 30408 (0.0006) [2024-09-30 01:02:17,304][1157819] Updated weights for policy 0, policy_version 30418 (0.0006) [2024-09-30 01:02:17,833][1157819] Updated weights for policy 0, policy_version 30428 (0.0006) [2024-09-30 01:02:18,401][1157819] Updated weights for policy 0, policy_version 30438 (0.0006) [2024-09-30 01:02:18,909][1157819] Updated weights for policy 0, policy_version 30448 (0.0006) [2024-09-30 01:02:19,407][1157819] Updated weights for policy 0, policy_version 30458 (0.0006) [2024-09-30 01:02:19,958][1157819] Updated weights for policy 0, policy_version 30468 (0.0006) [2024-09-30 01:02:20,466][1157520] Fps is (10 sec: 75367.0, 60 sec: 78370.3, 300 sec: 80045.6). Total num frames: 124833792. Throughput: 0: 19358.8. Samples: 21195612. Policy #0 lag: (min: 0.0, avg: 1.4, max: 5.0) [2024-09-30 01:02:20,466][1157520] Avg episode reward: [(0, '50.645')] [2024-09-30 01:02:20,484][1157819] Updated weights for policy 0, policy_version 30478 (0.0006) [2024-09-30 01:02:21,054][1157819] Updated weights for policy 0, policy_version 30488 (0.0006) [2024-09-30 01:02:21,582][1157819] Updated weights for policy 0, policy_version 30498 (0.0006) [2024-09-30 01:02:22,111][1157819] Updated weights for policy 0, policy_version 30508 (0.0006) [2024-09-30 01:02:22,669][1157819] Updated weights for policy 0, policy_version 30518 (0.0006) [2024-09-30 01:02:23,233][1157819] Updated weights for policy 0, policy_version 30528 (0.0006) [2024-09-30 01:02:23,819][1157819] Updated weights for policy 0, policy_version 30538 (0.0006) [2024-09-30 01:02:24,391][1157819] Updated weights for policy 0, policy_version 30548 (0.0006) [2024-09-30 01:02:24,983][1157819] Updated weights for policy 0, policy_version 30558 (0.0006) [2024-09-30 01:02:25,466][1157520] Fps is (10 sec: 74138.3, 60 sec: 77824.1, 300 sec: 79920.6). Total num frames: 125194240. Throughput: 0: 19264.3. Samples: 21252532. Policy #0 lag: (min: 0.0, avg: 1.4, max: 5.0) [2024-09-30 01:02:25,466][1157520] Avg episode reward: [(0, '47.417')] [2024-09-30 01:02:25,543][1157819] Updated weights for policy 0, policy_version 30568 (0.0006) [2024-09-30 01:02:26,139][1157819] Updated weights for policy 0, policy_version 30578 (0.0006) [2024-09-30 01:02:26,712][1157819] Updated weights for policy 0, policy_version 30588 (0.0006) [2024-09-30 01:02:27,287][1157819] Updated weights for policy 0, policy_version 30598 (0.0006) [2024-09-30 01:02:27,849][1157819] Updated weights for policy 0, policy_version 30608 (0.0006) [2024-09-30 01:02:28,393][1157819] Updated weights for policy 0, policy_version 30618 (0.0006) [2024-09-30 01:02:28,922][1157819] Updated weights for policy 0, policy_version 30628 (0.0006) [2024-09-30 01:02:29,491][1157819] Updated weights for policy 0, policy_version 30638 (0.0006) [2024-09-30 01:02:30,029][1157819] Updated weights for policy 0, policy_version 30648 (0.0006) [2024-09-30 01:02:30,466][1157520] Fps is (10 sec: 72908.7, 60 sec: 77141.3, 300 sec: 79837.3). Total num frames: 125562880. Throughput: 0: 19014.8. Samples: 21360516. Policy #0 lag: (min: 0.0, avg: 1.4, max: 5.0) [2024-09-30 01:02:30,466][1157520] Avg episode reward: [(0, '49.722')] [2024-09-30 01:02:30,566][1157819] Updated weights for policy 0, policy_version 30658 (0.0006) [2024-09-30 01:02:31,155][1157819] Updated weights for policy 0, policy_version 30668 (0.0006) [2024-09-30 01:02:31,693][1157819] Updated weights for policy 0, policy_version 30678 (0.0006) [2024-09-30 01:02:32,284][1157819] Updated weights for policy 0, policy_version 30688 (0.0006) [2024-09-30 01:02:32,842][1157819] Updated weights for policy 0, policy_version 30698 (0.0006) [2024-09-30 01:02:33,359][1157819] Updated weights for policy 0, policy_version 30708 (0.0006) [2024-09-30 01:02:33,934][1157819] Updated weights for policy 0, policy_version 30718 (0.0006) [2024-09-30 01:02:34,476][1157819] Updated weights for policy 0, policy_version 30728 (0.0006) [2024-09-30 01:02:35,037][1157819] Updated weights for policy 0, policy_version 30738 (0.0006) [2024-09-30 01:02:35,466][1157520] Fps is (10 sec: 73727.8, 60 sec: 76322.1, 300 sec: 79781.7). Total num frames: 125931520. Throughput: 0: 18780.5. Samples: 21471172. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:02:35,466][1157520] Avg episode reward: [(0, '50.741')] [2024-09-30 01:02:35,601][1157819] Updated weights for policy 0, policy_version 30748 (0.0006) [2024-09-30 01:02:36,163][1157819] Updated weights for policy 0, policy_version 30758 (0.0006) [2024-09-30 01:02:36,731][1157819] Updated weights for policy 0, policy_version 30768 (0.0006) [2024-09-30 01:02:37,270][1157819] Updated weights for policy 0, policy_version 30778 (0.0006) [2024-09-30 01:02:37,803][1157819] Updated weights for policy 0, policy_version 30788 (0.0006) [2024-09-30 01:02:38,368][1157819] Updated weights for policy 0, policy_version 30798 (0.0006) [2024-09-30 01:02:38,920][1157819] Updated weights for policy 0, policy_version 30808 (0.0006) [2024-09-30 01:02:39,499][1157819] Updated weights for policy 0, policy_version 30818 (0.0006) [2024-09-30 01:02:40,035][1157819] Updated weights for policy 0, policy_version 30828 (0.0006) [2024-09-30 01:02:40,466][1157520] Fps is (10 sec: 73728.1, 60 sec: 75776.0, 300 sec: 79767.9). Total num frames: 126300160. Throughput: 0: 18672.9. Samples: 21526460. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:02:40,466][1157520] Avg episode reward: [(0, '50.021')] [2024-09-30 01:02:40,620][1157819] Updated weights for policy 0, policy_version 30838 (0.0006) [2024-09-30 01:02:41,171][1157819] Updated weights for policy 0, policy_version 30848 (0.0006) [2024-09-30 01:02:41,723][1157819] Updated weights for policy 0, policy_version 30858 (0.0006) [2024-09-30 01:02:42,263][1157819] Updated weights for policy 0, policy_version 30868 (0.0006) [2024-09-30 01:02:42,415][1157736] Signal inference workers to stop experience collection... (1600 times) [2024-09-30 01:02:42,419][1157819] InferenceWorker_p0-w0: stopping experience collection (1600 times) [2024-09-30 01:02:42,425][1157736] Signal inference workers to resume experience collection... (1600 times) [2024-09-30 01:02:42,425][1157819] InferenceWorker_p0-w0: resuming experience collection (1600 times) [2024-09-30 01:02:42,805][1157819] Updated weights for policy 0, policy_version 30878 (0.0006) [2024-09-30 01:02:43,383][1157819] Updated weights for policy 0, policy_version 30888 (0.0006) [2024-09-30 01:02:43,895][1157819] Updated weights for policy 0, policy_version 30898 (0.0006) [2024-09-30 01:02:44,498][1157819] Updated weights for policy 0, policy_version 30908 (0.0007) [2024-09-30 01:02:45,092][1157819] Updated weights for policy 0, policy_version 30918 (0.0007) [2024-09-30 01:02:45,466][1157520] Fps is (10 sec: 73728.3, 60 sec: 75298.1, 300 sec: 79642.9). Total num frames: 126668800. Throughput: 0: 18462.8. Samples: 21637556. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:02:45,466][1157520] Avg episode reward: [(0, '47.817')] [2024-09-30 01:02:45,688][1157819] Updated weights for policy 0, policy_version 30928 (0.0007) [2024-09-30 01:02:46,223][1157819] Updated weights for policy 0, policy_version 30938 (0.0007) [2024-09-30 01:02:46,858][1157819] Updated weights for policy 0, policy_version 30948 (0.0006) [2024-09-30 01:02:47,478][1157819] Updated weights for policy 0, policy_version 30958 (0.0007) [2024-09-30 01:02:48,078][1157819] Updated weights for policy 0, policy_version 30968 (0.0008) [2024-09-30 01:02:48,729][1157819] Updated weights for policy 0, policy_version 30978 (0.0006) [2024-09-30 01:02:49,320][1157819] Updated weights for policy 0, policy_version 30988 (0.0006) [2024-09-30 01:02:49,874][1157819] Updated weights for policy 0, policy_version 30998 (0.0006) [2024-09-30 01:02:50,466][1157520] Fps is (10 sec: 70451.2, 60 sec: 74137.6, 300 sec: 79406.9). Total num frames: 127004672. Throughput: 0: 18216.8. Samples: 21740028. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:02:50,466][1157520] Avg episode reward: [(0, '49.036')] [2024-09-30 01:02:50,519][1157819] Updated weights for policy 0, policy_version 31008 (0.0006) [2024-09-30 01:02:51,079][1157819] Updated weights for policy 0, policy_version 31018 (0.0006) [2024-09-30 01:02:51,656][1157819] Updated weights for policy 0, policy_version 31028 (0.0006) [2024-09-30 01:02:52,214][1157819] Updated weights for policy 0, policy_version 31038 (0.0006) [2024-09-30 01:02:52,809][1157819] Updated weights for policy 0, policy_version 31048 (0.0006) [2024-09-30 01:02:53,324][1157819] Updated weights for policy 0, policy_version 31058 (0.0006) [2024-09-30 01:02:53,901][1157819] Updated weights for policy 0, policy_version 31068 (0.0006) [2024-09-30 01:02:54,437][1157819] Updated weights for policy 0, policy_version 31078 (0.0006) [2024-09-30 01:02:54,972][1157819] Updated weights for policy 0, policy_version 31088 (0.0006) [2024-09-30 01:02:55,466][1157520] Fps is (10 sec: 70041.4, 60 sec: 73454.9, 300 sec: 79295.8). Total num frames: 127369216. Throughput: 0: 18208.2. Samples: 21793124. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:02:55,466][1157520] Avg episode reward: [(0, '51.280')] [2024-09-30 01:02:55,525][1157819] Updated weights for policy 0, policy_version 31098 (0.0006) [2024-09-30 01:02:56,065][1157819] Updated weights for policy 0, policy_version 31108 (0.0006) [2024-09-30 01:02:56,645][1157819] Updated weights for policy 0, policy_version 31118 (0.0006) [2024-09-30 01:02:57,220][1157819] Updated weights for policy 0, policy_version 31128 (0.0006) [2024-09-30 01:02:57,782][1157819] Updated weights for policy 0, policy_version 31138 (0.0006) [2024-09-30 01:02:58,328][1157819] Updated weights for policy 0, policy_version 31148 (0.0006) [2024-09-30 01:02:58,863][1157819] Updated weights for policy 0, policy_version 31158 (0.0006) [2024-09-30 01:02:59,405][1157819] Updated weights for policy 0, policy_version 31168 (0.0006) [2024-09-30 01:02:59,949][1157819] Updated weights for policy 0, policy_version 31178 (0.0006) [2024-09-30 01:03:00,466][1157520] Fps is (10 sec: 73728.0, 60 sec: 73045.3, 300 sec: 79198.6). Total num frames: 127741952. Throughput: 0: 18270.4. Samples: 21904212. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:03:00,466][1157520] Avg episode reward: [(0, '51.108')] [2024-09-30 01:03:00,498][1157819] Updated weights for policy 0, policy_version 31188 (0.0006) [2024-09-30 01:03:01,053][1157819] Updated weights for policy 0, policy_version 31198 (0.0006) [2024-09-30 01:03:01,582][1157819] Updated weights for policy 0, policy_version 31208 (0.0006) [2024-09-30 01:03:02,142][1157819] Updated weights for policy 0, policy_version 31218 (0.0006) [2024-09-30 01:03:02,679][1157819] Updated weights for policy 0, policy_version 31228 (0.0006) [2024-09-30 01:03:03,289][1157819] Updated weights for policy 0, policy_version 31238 (0.0006) [2024-09-30 01:03:03,815][1157819] Updated weights for policy 0, policy_version 31248 (0.0006) [2024-09-30 01:03:04,406][1157819] Updated weights for policy 0, policy_version 31258 (0.0006) [2024-09-30 01:03:05,006][1157819] Updated weights for policy 0, policy_version 31268 (0.0006) [2024-09-30 01:03:05,466][1157520] Fps is (10 sec: 73318.6, 60 sec: 72840.7, 300 sec: 79129.2). Total num frames: 128102400. Throughput: 0: 18205.7. Samples: 22014868. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:03:05,466][1157520] Avg episode reward: [(0, '50.861')] [2024-09-30 01:03:05,579][1157819] Updated weights for policy 0, policy_version 31278 (0.0006) [2024-09-30 01:03:06,146][1157819] Updated weights for policy 0, policy_version 31288 (0.0006) [2024-09-30 01:03:06,678][1157819] Updated weights for policy 0, policy_version 31298 (0.0006) [2024-09-30 01:03:07,219][1157819] Updated weights for policy 0, policy_version 31308 (0.0006) [2024-09-30 01:03:07,768][1157819] Updated weights for policy 0, policy_version 31318 (0.0006) [2024-09-30 01:03:08,315][1157819] Updated weights for policy 0, policy_version 31328 (0.0006) [2024-09-30 01:03:08,896][1157819] Updated weights for policy 0, policy_version 31338 (0.0006) [2024-09-30 01:03:09,442][1157819] Updated weights for policy 0, policy_version 31348 (0.0007) [2024-09-30 01:03:09,952][1157819] Updated weights for policy 0, policy_version 31358 (0.0006) [2024-09-30 01:03:10,466][1157520] Fps is (10 sec: 73727.9, 60 sec: 73318.5, 300 sec: 79143.1). Total num frames: 128479232. Throughput: 0: 18162.8. Samples: 22069860. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:03:10,466][1157520] Avg episode reward: [(0, '47.604')] [2024-09-30 01:03:10,497][1157819] Updated weights for policy 0, policy_version 31368 (0.0006) [2024-09-30 01:03:11,028][1157819] Updated weights for policy 0, policy_version 31378 (0.0006) [2024-09-30 01:03:11,329][1157736] Signal inference workers to stop experience collection... (1650 times) [2024-09-30 01:03:11,330][1157736] Signal inference workers to resume experience collection... (1650 times) [2024-09-30 01:03:11,334][1157819] InferenceWorker_p0-w0: stopping experience collection (1650 times) [2024-09-30 01:03:11,336][1157819] InferenceWorker_p0-w0: resuming experience collection (1650 times) [2024-09-30 01:03:11,558][1157819] Updated weights for policy 0, policy_version 31388 (0.0006) [2024-09-30 01:03:12,086][1157819] Updated weights for policy 0, policy_version 31398 (0.0006) [2024-09-30 01:03:12,638][1157819] Updated weights for policy 0, policy_version 31408 (0.0006) [2024-09-30 01:03:13,164][1157819] Updated weights for policy 0, policy_version 31418 (0.0006) [2024-09-30 01:03:13,707][1157819] Updated weights for policy 0, policy_version 31428 (0.0006) [2024-09-30 01:03:14,268][1157819] Updated weights for policy 0, policy_version 31438 (0.0006) [2024-09-30 01:03:14,814][1157819] Updated weights for policy 0, policy_version 31448 (0.0006) [2024-09-30 01:03:15,385][1157819] Updated weights for policy 0, policy_version 31458 (0.0006) [2024-09-30 01:03:15,466][1157520] Fps is (10 sec: 75365.3, 60 sec: 73386.6, 300 sec: 79115.3). Total num frames: 128856064. Throughput: 0: 18290.7. Samples: 22183600. Policy #0 lag: (min: 0.0, avg: 2.7, max: 5.0) [2024-09-30 01:03:15,466][1157520] Avg episode reward: [(0, '47.418')] [2024-09-30 01:03:15,950][1157819] Updated weights for policy 0, policy_version 31468 (0.0006) [2024-09-30 01:03:16,539][1157819] Updated weights for policy 0, policy_version 31478 (0.0006) [2024-09-30 01:03:17,131][1157819] Updated weights for policy 0, policy_version 31488 (0.0006) [2024-09-30 01:03:17,659][1157819] Updated weights for policy 0, policy_version 31498 (0.0006) [2024-09-30 01:03:18,206][1157819] Updated weights for policy 0, policy_version 31508 (0.0006) [2024-09-30 01:03:18,775][1157819] Updated weights for policy 0, policy_version 31518 (0.0006) [2024-09-30 01:03:19,351][1157819] Updated weights for policy 0, policy_version 31528 (0.0006) [2024-09-30 01:03:19,920][1157819] Updated weights for policy 0, policy_version 31538 (0.0006) [2024-09-30 01:03:20,466][1157520] Fps is (10 sec: 73726.8, 60 sec: 73045.1, 300 sec: 78962.5). Total num frames: 129216512. Throughput: 0: 18260.5. Samples: 22292896. Policy #0 lag: (min: 0.0, avg: 2.7, max: 5.0) [2024-09-30 01:03:20,466][1157520] Avg episode reward: [(0, '51.289')] [2024-09-30 01:03:20,503][1157819] Updated weights for policy 0, policy_version 31548 (0.0006) [2024-09-30 01:03:21,064][1157819] Updated weights for policy 0, policy_version 31558 (0.0006) [2024-09-30 01:03:21,633][1157819] Updated weights for policy 0, policy_version 31568 (0.0006) [2024-09-30 01:03:22,256][1157819] Updated weights for policy 0, policy_version 31578 (0.0006) [2024-09-30 01:03:22,832][1157819] Updated weights for policy 0, policy_version 31588 (0.0006) [2024-09-30 01:03:23,409][1157819] Updated weights for policy 0, policy_version 31598 (0.0006) [2024-09-30 01:03:24,032][1157819] Updated weights for policy 0, policy_version 31608 (0.0006) [2024-09-30 01:03:24,567][1157819] Updated weights for policy 0, policy_version 31618 (0.0006) [2024-09-30 01:03:25,193][1157819] Updated weights for policy 0, policy_version 31628 (0.0006) [2024-09-30 01:03:25,466][1157520] Fps is (10 sec: 70860.6, 60 sec: 72840.3, 300 sec: 78851.5). Total num frames: 129564672. Throughput: 0: 18204.9. Samples: 22345684. Policy #0 lag: (min: 0.0, avg: 2.7, max: 5.0) [2024-09-30 01:03:25,466][1157520] Avg episode reward: [(0, '49.128')] [2024-09-30 01:03:25,775][1157819] Updated weights for policy 0, policy_version 31638 (0.0006) [2024-09-30 01:03:26,333][1157819] Updated weights for policy 0, policy_version 31648 (0.0006) [2024-09-30 01:03:26,931][1157819] Updated weights for policy 0, policy_version 31658 (0.0006) [2024-09-30 01:03:27,523][1157819] Updated weights for policy 0, policy_version 31668 (0.0006) [2024-09-30 01:03:28,111][1157819] Updated weights for policy 0, policy_version 31678 (0.0006) [2024-09-30 01:03:28,687][1157819] Updated weights for policy 0, policy_version 31688 (0.0006) [2024-09-30 01:03:29,250][1157819] Updated weights for policy 0, policy_version 31698 (0.0006) [2024-09-30 01:03:29,843][1157819] Updated weights for policy 0, policy_version 31708 (0.0006) [2024-09-30 01:03:30,389][1157819] Updated weights for policy 0, policy_version 31718 (0.0006) [2024-09-30 01:03:30,466][1157520] Fps is (10 sec: 70451.2, 60 sec: 72635.5, 300 sec: 78796.0). Total num frames: 129921024. Throughput: 0: 18066.0. Samples: 22450528. Policy #0 lag: (min: 0.0, avg: 2.7, max: 5.0) [2024-09-30 01:03:30,466][1157520] Avg episode reward: [(0, '52.163')] [2024-09-30 01:03:30,467][1157736] Saving new best policy, reward=52.163! [2024-09-30 01:03:30,903][1157819] Updated weights for policy 0, policy_version 31728 (0.0006) [2024-09-30 01:03:31,478][1157819] Updated weights for policy 0, policy_version 31738 (0.0006) [2024-09-30 01:03:32,000][1157819] Updated weights for policy 0, policy_version 31748 (0.0006) [2024-09-30 01:03:32,551][1157819] Updated weights for policy 0, policy_version 31758 (0.0006) [2024-09-30 01:03:33,117][1157819] Updated weights for policy 0, policy_version 31768 (0.0006) [2024-09-30 01:03:33,659][1157819] Updated weights for policy 0, policy_version 31778 (0.0006) [2024-09-30 01:03:34,232][1157819] Updated weights for policy 0, policy_version 31788 (0.0006) [2024-09-30 01:03:34,723][1157819] Updated weights for policy 0, policy_version 31798 (0.0006) [2024-09-30 01:03:35,288][1157819] Updated weights for policy 0, policy_version 31808 (0.0006) [2024-09-30 01:03:35,466][1157520] Fps is (10 sec: 73317.7, 60 sec: 72772.0, 300 sec: 78795.9). Total num frames: 130297856. Throughput: 0: 18267.1. Samples: 22562052. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 01:03:35,466][1157520] Avg episode reward: [(0, '50.797')] [2024-09-30 01:03:35,848][1157819] Updated weights for policy 0, policy_version 31818 (0.0006) [2024-09-30 01:03:36,368][1157819] Updated weights for policy 0, policy_version 31828 (0.0007) [2024-09-30 01:03:36,925][1157819] Updated weights for policy 0, policy_version 31838 (0.0006) [2024-09-30 01:03:37,476][1157819] Updated weights for policy 0, policy_version 31848 (0.0007) [2024-09-30 01:03:38,015][1157819] Updated weights for policy 0, policy_version 31858 (0.0006) [2024-09-30 01:03:38,585][1157819] Updated weights for policy 0, policy_version 31868 (0.0006) [2024-09-30 01:03:39,143][1157819] Updated weights for policy 0, policy_version 31878 (0.0006) [2024-09-30 01:03:39,689][1157819] Updated weights for policy 0, policy_version 31888 (0.0007) [2024-09-30 01:03:40,272][1157819] Updated weights for policy 0, policy_version 31898 (0.0006) [2024-09-30 01:03:40,466][1157520] Fps is (10 sec: 74137.8, 60 sec: 72703.8, 300 sec: 78809.8). Total num frames: 130662400. Throughput: 0: 18324.0. Samples: 22617704. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 01:03:40,466][1157520] Avg episode reward: [(0, '51.119')] [2024-09-30 01:03:40,894][1157819] Updated weights for policy 0, policy_version 31908 (0.0006) [2024-09-30 01:03:41,431][1157819] Updated weights for policy 0, policy_version 31918 (0.0007) [2024-09-30 01:03:42,038][1157819] Updated weights for policy 0, policy_version 31928 (0.0007) [2024-09-30 01:03:42,613][1157819] Updated weights for policy 0, policy_version 31938 (0.0006) [2024-09-30 01:03:43,199][1157819] Updated weights for policy 0, policy_version 31948 (0.0006) [2024-09-30 01:03:43,799][1157819] Updated weights for policy 0, policy_version 31958 (0.0006) [2024-09-30 01:03:44,355][1157819] Updated weights for policy 0, policy_version 31968 (0.0006) [2024-09-30 01:03:44,803][1157736] Signal inference workers to stop experience collection... (1700 times) [2024-09-30 01:03:44,804][1157736] Signal inference workers to resume experience collection... (1700 times) [2024-09-30 01:03:44,807][1157819] InferenceWorker_p0-w0: stopping experience collection (1700 times) [2024-09-30 01:03:44,807][1157819] InferenceWorker_p0-w0: resuming experience collection (1700 times) [2024-09-30 01:03:44,945][1157819] Updated weights for policy 0, policy_version 31978 (0.0006) [2024-09-30 01:03:45,466][1157520] Fps is (10 sec: 71680.7, 60 sec: 72430.7, 300 sec: 78809.8). Total num frames: 131014656. Throughput: 0: 18236.8. Samples: 22724872. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 01:03:45,466][1157520] Avg episode reward: [(0, '50.333')] [2024-09-30 01:03:45,525][1157819] Updated weights for policy 0, policy_version 31988 (0.0006) [2024-09-30 01:03:46,101][1157819] Updated weights for policy 0, policy_version 31998 (0.0006) [2024-09-30 01:03:46,679][1157819] Updated weights for policy 0, policy_version 32008 (0.0006) [2024-09-30 01:03:47,285][1157819] Updated weights for policy 0, policy_version 32018 (0.0006) [2024-09-30 01:03:47,852][1157819] Updated weights for policy 0, policy_version 32028 (0.0006) [2024-09-30 01:03:48,428][1157819] Updated weights for policy 0, policy_version 32038 (0.0006) [2024-09-30 01:03:49,016][1157819] Updated weights for policy 0, policy_version 32048 (0.0006) [2024-09-30 01:03:49,569][1157819] Updated weights for policy 0, policy_version 32058 (0.0006) [2024-09-30 01:03:50,150][1157819] Updated weights for policy 0, policy_version 32068 (0.0006) [2024-09-30 01:03:50,466][1157520] Fps is (10 sec: 70451.2, 60 sec: 72703.8, 300 sec: 78754.3). Total num frames: 131366912. Throughput: 0: 18139.7. Samples: 22831156. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 01:03:50,466][1157520] Avg episode reward: [(0, '50.844')] [2024-09-30 01:03:50,761][1157819] Updated weights for policy 0, policy_version 32078 (0.0006) [2024-09-30 01:03:51,289][1157819] Updated weights for policy 0, policy_version 32088 (0.0006) [2024-09-30 01:03:51,799][1157819] Updated weights for policy 0, policy_version 32098 (0.0006) [2024-09-30 01:03:52,367][1157819] Updated weights for policy 0, policy_version 32108 (0.0006) [2024-09-30 01:03:52,917][1157819] Updated weights for policy 0, policy_version 32118 (0.0006) [2024-09-30 01:03:53,462][1157819] Updated weights for policy 0, policy_version 32128 (0.0006) [2024-09-30 01:03:53,975][1157819] Updated weights for policy 0, policy_version 32138 (0.0006) [2024-09-30 01:03:54,512][1157819] Updated weights for policy 0, policy_version 32148 (0.0006) [2024-09-30 01:03:55,046][1157819] Updated weights for policy 0, policy_version 32158 (0.0006) [2024-09-30 01:03:55,466][1157520] Fps is (10 sec: 73727.4, 60 sec: 73045.1, 300 sec: 78823.7). Total num frames: 131751936. Throughput: 0: 18146.4. Samples: 22886452. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 01:03:55,466][1157520] Avg episode reward: [(0, '49.029')] [2024-09-30 01:03:55,556][1157819] Updated weights for policy 0, policy_version 32168 (0.0006) [2024-09-30 01:03:56,073][1157819] Updated weights for policy 0, policy_version 32178 (0.0006) [2024-09-30 01:03:56,561][1157819] Updated weights for policy 0, policy_version 32188 (0.0006) [2024-09-30 01:03:57,076][1157819] Updated weights for policy 0, policy_version 32198 (0.0006) [2024-09-30 01:03:57,603][1157819] Updated weights for policy 0, policy_version 32208 (0.0006) [2024-09-30 01:03:58,156][1157819] Updated weights for policy 0, policy_version 32218 (0.0006) [2024-09-30 01:03:58,678][1157819] Updated weights for policy 0, policy_version 32228 (0.0006) [2024-09-30 01:03:59,166][1157819] Updated weights for policy 0, policy_version 32238 (0.0006) [2024-09-30 01:03:59,664][1157819] Updated weights for policy 0, policy_version 32248 (0.0006) [2024-09-30 01:04:00,169][1157819] Updated weights for policy 0, policy_version 32258 (0.0006) [2024-09-30 01:04:00,466][1157520] Fps is (10 sec: 78233.4, 60 sec: 73454.7, 300 sec: 78934.8). Total num frames: 132149248. Throughput: 0: 18233.9. Samples: 23004128. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 01:04:00,466][1157520] Avg episode reward: [(0, '49.348')] [2024-09-30 01:04:00,707][1157819] Updated weights for policy 0, policy_version 32268 (0.0006) [2024-09-30 01:04:01,211][1157819] Updated weights for policy 0, policy_version 32278 (0.0006) [2024-09-30 01:04:01,694][1157819] Updated weights for policy 0, policy_version 32288 (0.0006) [2024-09-30 01:04:02,211][1157819] Updated weights for policy 0, policy_version 32298 (0.0006) [2024-09-30 01:04:02,751][1157819] Updated weights for policy 0, policy_version 32308 (0.0006) [2024-09-30 01:04:03,261][1157819] Updated weights for policy 0, policy_version 32318 (0.0006) [2024-09-30 01:04:03,754][1157819] Updated weights for policy 0, policy_version 32328 (0.0006) [2024-09-30 01:04:04,276][1157819] Updated weights for policy 0, policy_version 32338 (0.0006) [2024-09-30 01:04:04,768][1157819] Updated weights for policy 0, policy_version 32348 (0.0006) [2024-09-30 01:04:05,284][1157819] Updated weights for policy 0, policy_version 32358 (0.0006) [2024-09-30 01:04:05,466][1157520] Fps is (10 sec: 79872.6, 60 sec: 74137.4, 300 sec: 79004.2). Total num frames: 132550656. Throughput: 0: 18482.9. Samples: 23124628. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 01:04:05,466][1157520] Avg episode reward: [(0, '50.103')] [2024-09-30 01:04:05,472][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000032361_132550656.pth... [2024-09-30 01:04:05,518][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000027740_113623040.pth [2024-09-30 01:04:05,805][1157819] Updated weights for policy 0, policy_version 32368 (0.0006) [2024-09-30 01:04:06,316][1157819] Updated weights for policy 0, policy_version 32378 (0.0006) [2024-09-30 01:04:06,834][1157819] Updated weights for policy 0, policy_version 32388 (0.0006) [2024-09-30 01:04:07,374][1157819] Updated weights for policy 0, policy_version 32398 (0.0006) [2024-09-30 01:04:07,954][1157819] Updated weights for policy 0, policy_version 32408 (0.0006) [2024-09-30 01:04:08,485][1157819] Updated weights for policy 0, policy_version 32418 (0.0006) [2024-09-30 01:04:08,997][1157819] Updated weights for policy 0, policy_version 32428 (0.0006) [2024-09-30 01:04:09,512][1157819] Updated weights for policy 0, policy_version 32438 (0.0006) [2024-09-30 01:04:10,003][1157819] Updated weights for policy 0, policy_version 32448 (0.0006) [2024-09-30 01:04:10,466][1157520] Fps is (10 sec: 79462.6, 60 sec: 74410.5, 300 sec: 79059.7). Total num frames: 132943872. Throughput: 0: 18607.5. Samples: 23183020. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 01:04:10,466][1157520] Avg episode reward: [(0, '50.844')] [2024-09-30 01:04:10,502][1157819] Updated weights for policy 0, policy_version 32458 (0.0006) [2024-09-30 01:04:11,013][1157819] Updated weights for policy 0, policy_version 32468 (0.0006) [2024-09-30 01:04:11,499][1157819] Updated weights for policy 0, policy_version 32478 (0.0006) [2024-09-30 01:04:12,002][1157819] Updated weights for policy 0, policy_version 32488 (0.0006) [2024-09-30 01:04:12,463][1157819] Updated weights for policy 0, policy_version 32498 (0.0006) [2024-09-30 01:04:12,967][1157819] Updated weights for policy 0, policy_version 32508 (0.0006) [2024-09-30 01:04:13,462][1157819] Updated weights for policy 0, policy_version 32518 (0.0006) [2024-09-30 01:04:13,929][1157819] Updated weights for policy 0, policy_version 32528 (0.0006) [2024-09-30 01:04:14,455][1157819] Updated weights for policy 0, policy_version 32538 (0.0006) [2024-09-30 01:04:14,958][1157819] Updated weights for policy 0, policy_version 32548 (0.0006) [2024-09-30 01:04:15,405][1157819] Updated weights for policy 0, policy_version 32558 (0.0006) [2024-09-30 01:04:15,466][1157520] Fps is (10 sec: 80691.4, 60 sec: 75025.1, 300 sec: 79101.4). Total num frames: 133357568. Throughput: 0: 18997.0. Samples: 23305392. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:04:15,466][1157520] Avg episode reward: [(0, '49.471')] [2024-09-30 01:04:15,920][1157819] Updated weights for policy 0, policy_version 32568 (0.0006) [2024-09-30 01:04:16,413][1157819] Updated weights for policy 0, policy_version 32578 (0.0006) [2024-09-30 01:04:16,881][1157819] Updated weights for policy 0, policy_version 32588 (0.0006) [2024-09-30 01:04:17,409][1157819] Updated weights for policy 0, policy_version 32598 (0.0006) [2024-09-30 01:04:17,865][1157819] Updated weights for policy 0, policy_version 32608 (0.0006) [2024-09-30 01:04:18,396][1157819] Updated weights for policy 0, policy_version 32618 (0.0006) [2024-09-30 01:04:18,927][1157819] Updated weights for policy 0, policy_version 32628 (0.0006) [2024-09-30 01:04:19,433][1157819] Updated weights for policy 0, policy_version 32638 (0.0006) [2024-09-30 01:04:19,951][1157819] Updated weights for policy 0, policy_version 32648 (0.0006) [2024-09-30 01:04:20,466][1157520] Fps is (10 sec: 81920.1, 60 sec: 75776.0, 300 sec: 79115.2). Total num frames: 133763072. Throughput: 0: 19248.7. Samples: 23428240. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:04:20,466][1157520] Avg episode reward: [(0, '50.028')] [2024-09-30 01:04:20,515][1157819] Updated weights for policy 0, policy_version 32658 (0.0006) [2024-09-30 01:04:21,006][1157819] Updated weights for policy 0, policy_version 32668 (0.0006) [2024-09-30 01:04:21,539][1157819] Updated weights for policy 0, policy_version 32678 (0.0006) [2024-09-30 01:04:22,068][1157819] Updated weights for policy 0, policy_version 32688 (0.0006) [2024-09-30 01:04:22,666][1157819] Updated weights for policy 0, policy_version 32698 (0.0006) [2024-09-30 01:04:23,217][1157819] Updated weights for policy 0, policy_version 32708 (0.0006) [2024-09-30 01:04:23,814][1157819] Updated weights for policy 0, policy_version 32718 (0.0006) [2024-09-30 01:04:24,308][1157736] Signal inference workers to stop experience collection... (1750 times) [2024-09-30 01:04:24,308][1157736] Signal inference workers to resume experience collection... (1750 times) [2024-09-30 01:04:24,312][1157819] InferenceWorker_p0-w0: stopping experience collection (1750 times) [2024-09-30 01:04:24,312][1157819] InferenceWorker_p0-w0: resuming experience collection (1750 times) [2024-09-30 01:04:24,363][1157819] Updated weights for policy 0, policy_version 32728 (0.0006) [2024-09-30 01:04:24,963][1157819] Updated weights for policy 0, policy_version 32738 (0.0006) [2024-09-30 01:04:25,466][1157520] Fps is (10 sec: 77004.4, 60 sec: 76049.0, 300 sec: 79018.0). Total num frames: 134127616. Throughput: 0: 19277.0. Samples: 23485168. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:04:25,466][1157520] Avg episode reward: [(0, '52.663')] [2024-09-30 01:04:25,472][1157736] Saving new best policy, reward=52.663! [2024-09-30 01:04:25,554][1157819] Updated weights for policy 0, policy_version 32748 (0.0006) [2024-09-30 01:04:26,117][1157819] Updated weights for policy 0, policy_version 32758 (0.0006) [2024-09-30 01:04:26,694][1157819] Updated weights for policy 0, policy_version 32768 (0.0006) [2024-09-30 01:04:27,251][1157819] Updated weights for policy 0, policy_version 32778 (0.0006) [2024-09-30 01:04:27,768][1157819] Updated weights for policy 0, policy_version 32788 (0.0006) [2024-09-30 01:04:28,296][1157819] Updated weights for policy 0, policy_version 32798 (0.0006) [2024-09-30 01:04:28,850][1157819] Updated weights for policy 0, policy_version 32808 (0.0006) [2024-09-30 01:04:29,403][1157819] Updated weights for policy 0, policy_version 32818 (0.0006) [2024-09-30 01:04:29,979][1157819] Updated weights for policy 0, policy_version 32828 (0.0006) [2024-09-30 01:04:30,466][1157520] Fps is (10 sec: 73318.6, 60 sec: 76253.9, 300 sec: 78962.5). Total num frames: 134496256. Throughput: 0: 19308.4. Samples: 23593748. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:04:30,466][1157520] Avg episode reward: [(0, '48.851')] [2024-09-30 01:04:30,528][1157819] Updated weights for policy 0, policy_version 32838 (0.0006) [2024-09-30 01:04:31,044][1157819] Updated weights for policy 0, policy_version 32848 (0.0006) [2024-09-30 01:04:31,622][1157819] Updated weights for policy 0, policy_version 32858 (0.0006) [2024-09-30 01:04:32,145][1157819] Updated weights for policy 0, policy_version 32868 (0.0006) [2024-09-30 01:04:32,690][1157819] Updated weights for policy 0, policy_version 32878 (0.0006) [2024-09-30 01:04:33,267][1157819] Updated weights for policy 0, policy_version 32888 (0.0006) [2024-09-30 01:04:33,797][1157819] Updated weights for policy 0, policy_version 32898 (0.0007) [2024-09-30 01:04:34,349][1157819] Updated weights for policy 0, policy_version 32908 (0.0006) [2024-09-30 01:04:34,910][1157819] Updated weights for policy 0, policy_version 32918 (0.0006) [2024-09-30 01:04:35,466][1157520] Fps is (10 sec: 74138.7, 60 sec: 76185.9, 300 sec: 78893.1). Total num frames: 134868992. Throughput: 0: 19433.0. Samples: 23705640. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:04:35,466][1157520] Avg episode reward: [(0, '50.095')] [2024-09-30 01:04:35,470][1157819] Updated weights for policy 0, policy_version 32928 (0.0006) [2024-09-30 01:04:36,012][1157819] Updated weights for policy 0, policy_version 32938 (0.0006) [2024-09-30 01:04:36,544][1157819] Updated weights for policy 0, policy_version 32948 (0.0006) [2024-09-30 01:04:37,055][1157819] Updated weights for policy 0, policy_version 32958 (0.0006) [2024-09-30 01:04:37,567][1157819] Updated weights for policy 0, policy_version 32968 (0.0006) [2024-09-30 01:04:38,059][1157819] Updated weights for policy 0, policy_version 32978 (0.0006) [2024-09-30 01:04:38,570][1157819] Updated weights for policy 0, policy_version 32988 (0.0006) [2024-09-30 01:04:39,130][1157819] Updated weights for policy 0, policy_version 32998 (0.0006) [2024-09-30 01:04:39,657][1157819] Updated weights for policy 0, policy_version 33008 (0.0006) [2024-09-30 01:04:40,144][1157819] Updated weights for policy 0, policy_version 33018 (0.0006) [2024-09-30 01:04:40,466][1157520] Fps is (10 sec: 76595.0, 60 sec: 76663.5, 300 sec: 78851.4). Total num frames: 135262208. Throughput: 0: 19498.8. Samples: 23763896. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:04:40,466][1157520] Avg episode reward: [(0, '49.931')] [2024-09-30 01:04:40,660][1157819] Updated weights for policy 0, policy_version 33028 (0.0006) [2024-09-30 01:04:41,178][1157819] Updated weights for policy 0, policy_version 33038 (0.0006) [2024-09-30 01:04:41,681][1157819] Updated weights for policy 0, policy_version 33048 (0.0006) [2024-09-30 01:04:42,219][1157819] Updated weights for policy 0, policy_version 33058 (0.0006) [2024-09-30 01:04:42,732][1157819] Updated weights for policy 0, policy_version 33068 (0.0006) [2024-09-30 01:04:43,230][1157819] Updated weights for policy 0, policy_version 33078 (0.0006) [2024-09-30 01:04:43,718][1157819] Updated weights for policy 0, policy_version 33088 (0.0006) [2024-09-30 01:04:44,202][1157819] Updated weights for policy 0, policy_version 33098 (0.0006) [2024-09-30 01:04:44,704][1157819] Updated weights for policy 0, policy_version 33108 (0.0006) [2024-09-30 01:04:45,206][1157819] Updated weights for policy 0, policy_version 33118 (0.0006) [2024-09-30 01:04:45,466][1157520] Fps is (10 sec: 79871.1, 60 sec: 77551.0, 300 sec: 78837.6). Total num frames: 135667712. Throughput: 0: 19541.1. Samples: 23883476. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:04:45,466][1157520] Avg episode reward: [(0, '52.543')] [2024-09-30 01:04:45,706][1157819] Updated weights for policy 0, policy_version 33128 (0.0006) [2024-09-30 01:04:46,210][1157819] Updated weights for policy 0, policy_version 33138 (0.0006) [2024-09-30 01:04:46,707][1157819] Updated weights for policy 0, policy_version 33148 (0.0006) [2024-09-30 01:04:47,219][1157819] Updated weights for policy 0, policy_version 33158 (0.0006) [2024-09-30 01:04:47,736][1157819] Updated weights for policy 0, policy_version 33168 (0.0006) [2024-09-30 01:04:48,238][1157819] Updated weights for policy 0, policy_version 33178 (0.0006) [2024-09-30 01:04:48,741][1157819] Updated weights for policy 0, policy_version 33188 (0.0006) [2024-09-30 01:04:49,265][1157819] Updated weights for policy 0, policy_version 33198 (0.0006) [2024-09-30 01:04:49,752][1157819] Updated weights for policy 0, policy_version 33208 (0.0006) [2024-09-30 01:04:50,254][1157819] Updated weights for policy 0, policy_version 33218 (0.0006) [2024-09-30 01:04:50,466][1157520] Fps is (10 sec: 81100.9, 60 sec: 78438.4, 300 sec: 78837.6). Total num frames: 136073216. Throughput: 0: 19571.7. Samples: 24005352. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:04:50,466][1157520] Avg episode reward: [(0, '49.238')] [2024-09-30 01:04:50,779][1157819] Updated weights for policy 0, policy_version 33228 (0.0006) [2024-09-30 01:04:51,280][1157819] Updated weights for policy 0, policy_version 33238 (0.0006) [2024-09-30 01:04:51,817][1157819] Updated weights for policy 0, policy_version 33248 (0.0006) [2024-09-30 01:04:52,333][1157819] Updated weights for policy 0, policy_version 33258 (0.0006) [2024-09-30 01:04:52,846][1157819] Updated weights for policy 0, policy_version 33268 (0.0006) [2024-09-30 01:04:53,335][1157819] Updated weights for policy 0, policy_version 33278 (0.0006) [2024-09-30 01:04:53,843][1157819] Updated weights for policy 0, policy_version 33288 (0.0006) [2024-09-30 01:04:54,384][1157819] Updated weights for policy 0, policy_version 33298 (0.0006) [2024-09-30 01:04:54,888][1157819] Updated weights for policy 0, policy_version 33308 (0.0006) [2024-09-30 01:04:55,403][1157819] Updated weights for policy 0, policy_version 33318 (0.0006) [2024-09-30 01:04:55,466][1157520] Fps is (10 sec: 80691.0, 60 sec: 78711.6, 300 sec: 78837.6). Total num frames: 136474624. Throughput: 0: 19607.6. Samples: 24065364. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:04:55,466][1157520] Avg episode reward: [(0, '52.269')] [2024-09-30 01:04:55,944][1157819] Updated weights for policy 0, policy_version 33328 (0.0006) [2024-09-30 01:04:56,445][1157819] Updated weights for policy 0, policy_version 33338 (0.0006) [2024-09-30 01:04:57,007][1157819] Updated weights for policy 0, policy_version 33348 (0.0006) [2024-09-30 01:04:57,512][1157819] Updated weights for policy 0, policy_version 33358 (0.0006) [2024-09-30 01:04:58,013][1157819] Updated weights for policy 0, policy_version 33368 (0.0006) [2024-09-30 01:04:58,460][1157819] Updated weights for policy 0, policy_version 33378 (0.0006) [2024-09-30 01:04:58,947][1157819] Updated weights for policy 0, policy_version 33388 (0.0006) [2024-09-30 01:04:59,404][1157819] Updated weights for policy 0, policy_version 33398 (0.0006) [2024-09-30 01:04:59,876][1157819] Updated weights for policy 0, policy_version 33408 (0.0006) [2024-09-30 01:05:00,323][1157819] Updated weights for policy 0, policy_version 33418 (0.0006) [2024-09-30 01:05:00,466][1157520] Fps is (10 sec: 81510.1, 60 sec: 78984.5, 300 sec: 78879.2). Total num frames: 136888320. Throughput: 0: 19570.1. Samples: 24186048. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:05:00,466][1157520] Avg episode reward: [(0, '50.940')] [2024-09-30 01:05:00,811][1157819] Updated weights for policy 0, policy_version 33428 (0.0006) [2024-09-30 01:05:01,292][1157819] Updated weights for policy 0, policy_version 33438 (0.0006) [2024-09-30 01:05:01,735][1157819] Updated weights for policy 0, policy_version 33448 (0.0006) [2024-09-30 01:05:02,205][1157819] Updated weights for policy 0, policy_version 33458 (0.0006) [2024-09-30 01:05:02,685][1157819] Updated weights for policy 0, policy_version 33468 (0.0006) [2024-09-30 01:05:03,184][1157819] Updated weights for policy 0, policy_version 33478 (0.0006) [2024-09-30 01:05:03,683][1157819] Updated weights for policy 0, policy_version 33488 (0.0006) [2024-09-30 01:05:04,148][1157819] Updated weights for policy 0, policy_version 33498 (0.0006) [2024-09-30 01:05:04,648][1157819] Updated weights for policy 0, policy_version 33508 (0.0006) [2024-09-30 01:05:05,147][1157819] Updated weights for policy 0, policy_version 33518 (0.0006) [2024-09-30 01:05:05,466][1157520] Fps is (10 sec: 83558.4, 60 sec: 79325.9, 300 sec: 78879.2). Total num frames: 137310208. Throughput: 0: 19706.7. Samples: 24315040. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:05:05,466][1157520] Avg episode reward: [(0, '52.410')] [2024-09-30 01:05:05,637][1157819] Updated weights for policy 0, policy_version 33528 (0.0006) [2024-09-30 01:05:06,135][1157819] Updated weights for policy 0, policy_version 33538 (0.0006) [2024-09-30 01:05:06,631][1157819] Updated weights for policy 0, policy_version 33548 (0.0006) [2024-09-30 01:05:07,114][1157819] Updated weights for policy 0, policy_version 33558 (0.0006) [2024-09-30 01:05:07,609][1157819] Updated weights for policy 0, policy_version 33568 (0.0006) [2024-09-30 01:05:08,103][1157819] Updated weights for policy 0, policy_version 33578 (0.0006) [2024-09-30 01:05:08,596][1157819] Updated weights for policy 0, policy_version 33588 (0.0007) [2024-09-30 01:05:09,099][1157819] Updated weights for policy 0, policy_version 33598 (0.0006) [2024-09-30 01:05:09,583][1157819] Updated weights for policy 0, policy_version 33608 (0.0006) [2024-09-30 01:05:10,090][1157819] Updated weights for policy 0, policy_version 33618 (0.0006) [2024-09-30 01:05:10,466][1157520] Fps is (10 sec: 83968.0, 60 sec: 79735.4, 300 sec: 78809.8). Total num frames: 137728000. Throughput: 0: 19831.3. Samples: 24377576. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:05:10,466][1157520] Avg episode reward: [(0, '51.902')] [2024-09-30 01:05:10,598][1157819] Updated weights for policy 0, policy_version 33628 (0.0006) [2024-09-30 01:05:11,070][1157819] Updated weights for policy 0, policy_version 33638 (0.0006) [2024-09-30 01:05:11,575][1157819] Updated weights for policy 0, policy_version 33648 (0.0006) [2024-09-30 01:05:12,087][1157819] Updated weights for policy 0, policy_version 33658 (0.0006) [2024-09-30 01:05:12,597][1157819] Updated weights for policy 0, policy_version 33668 (0.0006) [2024-09-30 01:05:13,117][1157819] Updated weights for policy 0, policy_version 33678 (0.0006) [2024-09-30 01:05:13,666][1157819] Updated weights for policy 0, policy_version 33688 (0.0006) [2024-09-30 01:05:14,205][1157819] Updated weights for policy 0, policy_version 33698 (0.0006) [2024-09-30 01:05:14,667][1157736] Signal inference workers to stop experience collection... (1800 times) [2024-09-30 01:05:14,668][1157736] Signal inference workers to resume experience collection... (1800 times) [2024-09-30 01:05:14,672][1157819] InferenceWorker_p0-w0: stopping experience collection (1800 times) [2024-09-30 01:05:14,672][1157819] InferenceWorker_p0-w0: resuming experience collection (1800 times) [2024-09-30 01:05:14,741][1157819] Updated weights for policy 0, policy_version 33708 (0.0006) [2024-09-30 01:05:15,307][1157819] Updated weights for policy 0, policy_version 33718 (0.0006) [2024-09-30 01:05:15,466][1157520] Fps is (10 sec: 80691.8, 60 sec: 79325.9, 300 sec: 78643.2). Total num frames: 138117120. Throughput: 0: 20113.6. Samples: 24498860. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:05:15,466][1157520] Avg episode reward: [(0, '52.052')] [2024-09-30 01:05:15,854][1157819] Updated weights for policy 0, policy_version 33728 (0.0006) [2024-09-30 01:05:16,423][1157819] Updated weights for policy 0, policy_version 33738 (0.0006) [2024-09-30 01:05:16,953][1157819] Updated weights for policy 0, policy_version 33748 (0.0006) [2024-09-30 01:05:17,549][1157819] Updated weights for policy 0, policy_version 33758 (0.0006) [2024-09-30 01:05:18,047][1157819] Updated weights for policy 0, policy_version 33768 (0.0006) [2024-09-30 01:05:18,549][1157819] Updated weights for policy 0, policy_version 33778 (0.0006) [2024-09-30 01:05:19,045][1157819] Updated weights for policy 0, policy_version 33788 (0.0006) [2024-09-30 01:05:19,529][1157819] Updated weights for policy 0, policy_version 33798 (0.0006) [2024-09-30 01:05:20,015][1157819] Updated weights for policy 0, policy_version 33808 (0.0006) [2024-09-30 01:05:20,466][1157520] Fps is (10 sec: 78644.2, 60 sec: 79189.5, 300 sec: 78490.5). Total num frames: 138514432. Throughput: 0: 20198.3. Samples: 24614564. Policy #0 lag: (min: 0.0, avg: 1.8, max: 7.0) [2024-09-30 01:05:20,466][1157520] Avg episode reward: [(0, '50.705')] [2024-09-30 01:05:20,511][1157819] Updated weights for policy 0, policy_version 33818 (0.0006) [2024-09-30 01:05:21,003][1157819] Updated weights for policy 0, policy_version 33828 (0.0006) [2024-09-30 01:05:21,508][1157819] Updated weights for policy 0, policy_version 33838 (0.0006) [2024-09-30 01:05:22,016][1157819] Updated weights for policy 0, policy_version 33848 (0.0006) [2024-09-30 01:05:22,527][1157819] Updated weights for policy 0, policy_version 33858 (0.0006) [2024-09-30 01:05:23,055][1157819] Updated weights for policy 0, policy_version 33868 (0.0006) [2024-09-30 01:05:23,586][1157819] Updated weights for policy 0, policy_version 33878 (0.0006) [2024-09-30 01:05:24,075][1157819] Updated weights for policy 0, policy_version 33888 (0.0006) [2024-09-30 01:05:24,590][1157819] Updated weights for policy 0, policy_version 33898 (0.0006) [2024-09-30 01:05:25,105][1157819] Updated weights for policy 0, policy_version 33908 (0.0006) [2024-09-30 01:05:25,466][1157520] Fps is (10 sec: 79872.4, 60 sec: 79803.9, 300 sec: 78296.1). Total num frames: 138915840. Throughput: 0: 20258.6. Samples: 24675532. Policy #0 lag: (min: 0.0, avg: 1.8, max: 7.0) [2024-09-30 01:05:25,466][1157520] Avg episode reward: [(0, '47.509')] [2024-09-30 01:05:25,607][1157819] Updated weights for policy 0, policy_version 33918 (0.0006) [2024-09-30 01:05:26,100][1157819] Updated weights for policy 0, policy_version 33928 (0.0006) [2024-09-30 01:05:26,647][1157819] Updated weights for policy 0, policy_version 33938 (0.0006) [2024-09-30 01:05:27,183][1157819] Updated weights for policy 0, policy_version 33948 (0.0006) [2024-09-30 01:05:27,744][1157819] Updated weights for policy 0, policy_version 33958 (0.0006) [2024-09-30 01:05:28,295][1157819] Updated weights for policy 0, policy_version 33968 (0.0006) [2024-09-30 01:05:28,820][1157819] Updated weights for policy 0, policy_version 33978 (0.0006) [2024-09-30 01:05:29,412][1157819] Updated weights for policy 0, policy_version 33988 (0.0006) [2024-09-30 01:05:29,933][1157819] Updated weights for policy 0, policy_version 33998 (0.0006) [2024-09-30 01:05:30,465][1157819] Updated weights for policy 0, policy_version 34008 (0.0006) [2024-09-30 01:05:30,466][1157520] Fps is (10 sec: 78233.7, 60 sec: 80008.7, 300 sec: 78073.9). Total num frames: 139296768. Throughput: 0: 20199.9. Samples: 24792468. Policy #0 lag: (min: 0.0, avg: 1.8, max: 7.0) [2024-09-30 01:05:30,466][1157520] Avg episode reward: [(0, '49.332')] [2024-09-30 01:05:31,042][1157819] Updated weights for policy 0, policy_version 34018 (0.0006) [2024-09-30 01:05:31,617][1157819] Updated weights for policy 0, policy_version 34028 (0.0006) [2024-09-30 01:05:32,120][1157819] Updated weights for policy 0, policy_version 34038 (0.0006) [2024-09-30 01:05:32,615][1157819] Updated weights for policy 0, policy_version 34048 (0.0006) [2024-09-30 01:05:33,142][1157819] Updated weights for policy 0, policy_version 34058 (0.0006) [2024-09-30 01:05:33,690][1157819] Updated weights for policy 0, policy_version 34068 (0.0005) [2024-09-30 01:05:34,219][1157819] Updated weights for policy 0, policy_version 34078 (0.0006) [2024-09-30 01:05:34,720][1157819] Updated weights for policy 0, policy_version 34088 (0.0006) [2024-09-30 01:05:35,215][1157819] Updated weights for policy 0, policy_version 34098 (0.0006) [2024-09-30 01:05:35,466][1157520] Fps is (10 sec: 76595.2, 60 sec: 80213.3, 300 sec: 77907.3). Total num frames: 139681792. Throughput: 0: 20042.5. Samples: 24907264. Policy #0 lag: (min: 0.0, avg: 1.8, max: 7.0) [2024-09-30 01:05:35,466][1157520] Avg episode reward: [(0, '49.479')] [2024-09-30 01:05:35,772][1157819] Updated weights for policy 0, policy_version 34108 (0.0006) [2024-09-30 01:05:36,308][1157819] Updated weights for policy 0, policy_version 34118 (0.0006) [2024-09-30 01:05:36,835][1157819] Updated weights for policy 0, policy_version 34128 (0.0006) [2024-09-30 01:05:37,364][1157819] Updated weights for policy 0, policy_version 34138 (0.0006) [2024-09-30 01:05:37,858][1157819] Updated weights for policy 0, policy_version 34148 (0.0006) [2024-09-30 01:05:38,387][1157819] Updated weights for policy 0, policy_version 34158 (0.0006) [2024-09-30 01:05:38,891][1157819] Updated weights for policy 0, policy_version 34168 (0.0006) [2024-09-30 01:05:39,414][1157819] Updated weights for policy 0, policy_version 34178 (0.0005) [2024-09-30 01:05:39,931][1157819] Updated weights for policy 0, policy_version 34188 (0.0006) [2024-09-30 01:05:40,462][1157819] Updated weights for policy 0, policy_version 34198 (0.0006) [2024-09-30 01:05:40,466][1157520] Fps is (10 sec: 77824.0, 60 sec: 80213.5, 300 sec: 77768.5). Total num frames: 140075008. Throughput: 0: 20011.4. Samples: 24965872. Policy #0 lag: (min: 0.0, avg: 1.8, max: 7.0) [2024-09-30 01:05:40,466][1157520] Avg episode reward: [(0, '49.994')] [2024-09-30 01:05:40,955][1157819] Updated weights for policy 0, policy_version 34208 (0.0006) [2024-09-30 01:05:41,458][1157819] Updated weights for policy 0, policy_version 34218 (0.0006) [2024-09-30 01:05:42,005][1157819] Updated weights for policy 0, policy_version 34228 (0.0006) [2024-09-30 01:05:42,519][1157819] Updated weights for policy 0, policy_version 34238 (0.0006) [2024-09-30 01:05:43,042][1157819] Updated weights for policy 0, policy_version 34248 (0.0006) [2024-09-30 01:05:43,528][1157819] Updated weights for policy 0, policy_version 34258 (0.0006) [2024-09-30 01:05:44,064][1157819] Updated weights for policy 0, policy_version 34268 (0.0006) [2024-09-30 01:05:44,579][1157819] Updated weights for policy 0, policy_version 34278 (0.0006) [2024-09-30 01:05:45,103][1157819] Updated weights for policy 0, policy_version 34288 (0.0006) [2024-09-30 01:05:45,466][1157520] Fps is (10 sec: 79052.8, 60 sec: 80076.9, 300 sec: 77629.6). Total num frames: 140472320. Throughput: 0: 19976.7. Samples: 25084996. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 01:05:45,466][1157520] Avg episode reward: [(0, '50.772')] [2024-09-30 01:05:45,602][1157819] Updated weights for policy 0, policy_version 34298 (0.0006) [2024-09-30 01:05:46,112][1157819] Updated weights for policy 0, policy_version 34308 (0.0006) [2024-09-30 01:05:46,657][1157819] Updated weights for policy 0, policy_version 34318 (0.0006) [2024-09-30 01:05:47,153][1157819] Updated weights for policy 0, policy_version 34328 (0.0006) [2024-09-30 01:05:47,670][1157819] Updated weights for policy 0, policy_version 34338 (0.0006) [2024-09-30 01:05:48,186][1157819] Updated weights for policy 0, policy_version 34348 (0.0006) [2024-09-30 01:05:48,680][1157819] Updated weights for policy 0, policy_version 34358 (0.0006) [2024-09-30 01:05:49,181][1157819] Updated weights for policy 0, policy_version 34368 (0.0006) [2024-09-30 01:05:49,722][1157819] Updated weights for policy 0, policy_version 34378 (0.0006) [2024-09-30 01:05:50,218][1157819] Updated weights for policy 0, policy_version 34388 (0.0006) [2024-09-30 01:05:50,466][1157520] Fps is (10 sec: 79462.5, 60 sec: 79940.4, 300 sec: 77504.7). Total num frames: 140869632. Throughput: 0: 19764.4. Samples: 25204436. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 01:05:50,466][1157520] Avg episode reward: [(0, '48.776')] [2024-09-30 01:05:50,740][1157819] Updated weights for policy 0, policy_version 34398 (0.0006) [2024-09-30 01:05:51,240][1157819] Updated weights for policy 0, policy_version 34408 (0.0006) [2024-09-30 01:05:51,747][1157819] Updated weights for policy 0, policy_version 34418 (0.0006) [2024-09-30 01:05:52,295][1157819] Updated weights for policy 0, policy_version 34428 (0.0006) [2024-09-30 01:05:52,795][1157819] Updated weights for policy 0, policy_version 34438 (0.0006) [2024-09-30 01:05:53,307][1157819] Updated weights for policy 0, policy_version 34448 (0.0006) [2024-09-30 01:05:53,430][1157736] Signal inference workers to stop experience collection... (1850 times) [2024-09-30 01:05:53,431][1157736] Signal inference workers to resume experience collection... (1850 times) [2024-09-30 01:05:53,434][1157819] InferenceWorker_p0-w0: stopping experience collection (1850 times) [2024-09-30 01:05:53,434][1157819] InferenceWorker_p0-w0: resuming experience collection (1850 times) [2024-09-30 01:05:53,824][1157819] Updated weights for policy 0, policy_version 34458 (0.0006) [2024-09-30 01:05:54,336][1157819] Updated weights for policy 0, policy_version 34468 (0.0006) [2024-09-30 01:05:54,881][1157819] Updated weights for policy 0, policy_version 34478 (0.0006) [2024-09-30 01:05:55,403][1157819] Updated weights for policy 0, policy_version 34488 (0.0006) [2024-09-30 01:05:55,466][1157520] Fps is (10 sec: 79052.1, 60 sec: 79803.8, 300 sec: 77351.9). Total num frames: 141262848. Throughput: 0: 19710.4. Samples: 25264544. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 01:05:55,466][1157520] Avg episode reward: [(0, '47.919')] [2024-09-30 01:05:55,947][1157819] Updated weights for policy 0, policy_version 34498 (0.0006) [2024-09-30 01:05:56,477][1157819] Updated weights for policy 0, policy_version 34508 (0.0006) [2024-09-30 01:05:57,015][1157819] Updated weights for policy 0, policy_version 34518 (0.0006) [2024-09-30 01:05:57,557][1157819] Updated weights for policy 0, policy_version 34528 (0.0006) [2024-09-30 01:05:58,061][1157819] Updated weights for policy 0, policy_version 34538 (0.0006) [2024-09-30 01:05:58,573][1157819] Updated weights for policy 0, policy_version 34548 (0.0006) [2024-09-30 01:05:59,071][1157819] Updated weights for policy 0, policy_version 34558 (0.0006) [2024-09-30 01:05:59,570][1157819] Updated weights for policy 0, policy_version 34568 (0.0006) [2024-09-30 01:06:00,066][1157819] Updated weights for policy 0, policy_version 34578 (0.0006) [2024-09-30 01:06:00,466][1157520] Fps is (10 sec: 79052.7, 60 sec: 79530.9, 300 sec: 77199.2). Total num frames: 141660160. Throughput: 0: 19609.0. Samples: 25381264. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 01:06:00,466][1157520] Avg episode reward: [(0, '49.391')] [2024-09-30 01:06:00,567][1157819] Updated weights for policy 0, policy_version 34588 (0.0006) [2024-09-30 01:06:01,109][1157819] Updated weights for policy 0, policy_version 34598 (0.0006) [2024-09-30 01:06:01,642][1157819] Updated weights for policy 0, policy_version 34608 (0.0006) [2024-09-30 01:06:02,171][1157819] Updated weights for policy 0, policy_version 34618 (0.0006) [2024-09-30 01:06:02,723][1157819] Updated weights for policy 0, policy_version 34628 (0.0006) [2024-09-30 01:06:03,262][1157819] Updated weights for policy 0, policy_version 34638 (0.0006) [2024-09-30 01:06:03,802][1157819] Updated weights for policy 0, policy_version 34648 (0.0006) [2024-09-30 01:06:04,326][1157819] Updated weights for policy 0, policy_version 34658 (0.0006) [2024-09-30 01:06:04,841][1157819] Updated weights for policy 0, policy_version 34668 (0.0006) [2024-09-30 01:06:05,345][1157819] Updated weights for policy 0, policy_version 34678 (0.0006) [2024-09-30 01:06:05,466][1157520] Fps is (10 sec: 78234.0, 60 sec: 78916.4, 300 sec: 77004.8). Total num frames: 142045184. Throughput: 0: 19641.1. Samples: 25498416. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 01:06:05,466][1157520] Avg episode reward: [(0, '47.367')] [2024-09-30 01:06:05,475][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000034680_142049280.pth... [2024-09-30 01:06:05,518][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000030208_123731968.pth [2024-09-30 01:06:05,843][1157819] Updated weights for policy 0, policy_version 34688 (0.0006) [2024-09-30 01:06:06,353][1157819] Updated weights for policy 0, policy_version 34698 (0.0006) [2024-09-30 01:06:06,877][1157819] Updated weights for policy 0, policy_version 34708 (0.0006) [2024-09-30 01:06:07,407][1157819] Updated weights for policy 0, policy_version 34718 (0.0006) [2024-09-30 01:06:07,948][1157819] Updated weights for policy 0, policy_version 34728 (0.0006) [2024-09-30 01:06:08,488][1157819] Updated weights for policy 0, policy_version 34738 (0.0006) [2024-09-30 01:06:09,048][1157819] Updated weights for policy 0, policy_version 34748 (0.0007) [2024-09-30 01:06:09,592][1157819] Updated weights for policy 0, policy_version 34758 (0.0006) [2024-09-30 01:06:10,116][1157819] Updated weights for policy 0, policy_version 34768 (0.0006) [2024-09-30 01:06:10,466][1157520] Fps is (10 sec: 77414.6, 60 sec: 78438.6, 300 sec: 76907.6). Total num frames: 142434304. Throughput: 0: 19615.8. Samples: 25558244. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 01:06:10,466][1157520] Avg episode reward: [(0, '51.168')] [2024-09-30 01:06:10,626][1157819] Updated weights for policy 0, policy_version 34778 (0.0006) [2024-09-30 01:06:11,169][1157819] Updated weights for policy 0, policy_version 34788 (0.0006) [2024-09-30 01:06:11,698][1157819] Updated weights for policy 0, policy_version 34798 (0.0006) [2024-09-30 01:06:12,207][1157819] Updated weights for policy 0, policy_version 34808 (0.0006) [2024-09-30 01:06:12,720][1157819] Updated weights for policy 0, policy_version 34818 (0.0006) [2024-09-30 01:06:13,224][1157819] Updated weights for policy 0, policy_version 34828 (0.0006) [2024-09-30 01:06:13,692][1157819] Updated weights for policy 0, policy_version 34838 (0.0006) [2024-09-30 01:06:14,199][1157819] Updated weights for policy 0, policy_version 34848 (0.0006) [2024-09-30 01:06:14,705][1157819] Updated weights for policy 0, policy_version 34858 (0.0006) [2024-09-30 01:06:15,204][1157819] Updated weights for policy 0, policy_version 34868 (0.0006) [2024-09-30 01:06:15,466][1157520] Fps is (10 sec: 79462.9, 60 sec: 78711.6, 300 sec: 76977.1). Total num frames: 142839808. Throughput: 0: 19627.2. Samples: 25675692. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 01:06:15,466][1157520] Avg episode reward: [(0, '50.134')] [2024-09-30 01:06:15,710][1157819] Updated weights for policy 0, policy_version 34878 (0.0006) [2024-09-30 01:06:16,198][1157819] Updated weights for policy 0, policy_version 34888 (0.0006) [2024-09-30 01:06:16,700][1157819] Updated weights for policy 0, policy_version 34898 (0.0006) [2024-09-30 01:06:17,185][1157819] Updated weights for policy 0, policy_version 34908 (0.0006) [2024-09-30 01:06:17,684][1157819] Updated weights for policy 0, policy_version 34918 (0.0006) [2024-09-30 01:06:18,191][1157819] Updated weights for policy 0, policy_version 34928 (0.0006) [2024-09-30 01:06:18,665][1157819] Updated weights for policy 0, policy_version 34938 (0.0006) [2024-09-30 01:06:19,176][1157819] Updated weights for policy 0, policy_version 34948 (0.0006) [2024-09-30 01:06:19,711][1157819] Updated weights for policy 0, policy_version 34958 (0.0006) [2024-09-30 01:06:20,240][1157819] Updated weights for policy 0, policy_version 34968 (0.0006) [2024-09-30 01:06:20,466][1157520] Fps is (10 sec: 81100.8, 60 sec: 78848.0, 300 sec: 77018.7). Total num frames: 143245312. Throughput: 0: 19813.5. Samples: 25798872. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 01:06:20,466][1157520] Avg episode reward: [(0, '48.153')] [2024-09-30 01:06:20,739][1157819] Updated weights for policy 0, policy_version 34978 (0.0006) [2024-09-30 01:06:21,268][1157819] Updated weights for policy 0, policy_version 34988 (0.0006) [2024-09-30 01:06:21,794][1157819] Updated weights for policy 0, policy_version 34998 (0.0006) [2024-09-30 01:06:22,329][1157819] Updated weights for policy 0, policy_version 35008 (0.0006) [2024-09-30 01:06:22,846][1157819] Updated weights for policy 0, policy_version 35018 (0.0006) [2024-09-30 01:06:23,345][1157819] Updated weights for policy 0, policy_version 35028 (0.0006) [2024-09-30 01:06:23,866][1157819] Updated weights for policy 0, policy_version 35038 (0.0006) [2024-09-30 01:06:24,389][1157819] Updated weights for policy 0, policy_version 35048 (0.0006) [2024-09-30 01:06:24,948][1157819] Updated weights for policy 0, policy_version 35058 (0.0006) [2024-09-30 01:06:25,466][1157520] Fps is (10 sec: 79462.4, 60 sec: 78643.2, 300 sec: 76949.3). Total num frames: 143634432. Throughput: 0: 19821.5. Samples: 25857840. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 01:06:25,466][1157520] Avg episode reward: [(0, '51.079')] [2024-09-30 01:06:25,472][1157819] Updated weights for policy 0, policy_version 35068 (0.0006) [2024-09-30 01:06:25,960][1157819] Updated weights for policy 0, policy_version 35078 (0.0006) [2024-09-30 01:06:26,499][1157819] Updated weights for policy 0, policy_version 35088 (0.0006) [2024-09-30 01:06:27,030][1157819] Updated weights for policy 0, policy_version 35098 (0.0006) [2024-09-30 01:06:27,609][1157819] Updated weights for policy 0, policy_version 35108 (0.0006) [2024-09-30 01:06:27,899][1157736] Signal inference workers to stop experience collection... (1900 times) [2024-09-30 01:06:27,900][1157736] Signal inference workers to resume experience collection... (1900 times) [2024-09-30 01:06:27,903][1157819] InferenceWorker_p0-w0: stopping experience collection (1900 times) [2024-09-30 01:06:27,905][1157819] InferenceWorker_p0-w0: resuming experience collection (1900 times) [2024-09-30 01:06:28,220][1157819] Updated weights for policy 0, policy_version 35118 (0.0006) [2024-09-30 01:06:28,813][1157819] Updated weights for policy 0, policy_version 35128 (0.0006) [2024-09-30 01:06:29,420][1157819] Updated weights for policy 0, policy_version 35138 (0.0006) [2024-09-30 01:06:29,989][1157819] Updated weights for policy 0, policy_version 35148 (0.0006) [2024-09-30 01:06:30,466][1157520] Fps is (10 sec: 74956.5, 60 sec: 78301.9, 300 sec: 76754.9). Total num frames: 143994880. Throughput: 0: 19678.0. Samples: 25970504. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:06:30,466][1157520] Avg episode reward: [(0, '48.603')] [2024-09-30 01:06:30,563][1157819] Updated weights for policy 0, policy_version 35158 (0.0006) [2024-09-30 01:06:31,179][1157819] Updated weights for policy 0, policy_version 35168 (0.0006) [2024-09-30 01:06:31,729][1157819] Updated weights for policy 0, policy_version 35178 (0.0006) [2024-09-30 01:06:32,350][1157819] Updated weights for policy 0, policy_version 35188 (0.0006) [2024-09-30 01:06:32,932][1157819] Updated weights for policy 0, policy_version 35198 (0.0006) [2024-09-30 01:06:33,518][1157819] Updated weights for policy 0, policy_version 35208 (0.0006) [2024-09-30 01:06:34,102][1157819] Updated weights for policy 0, policy_version 35218 (0.0006) [2024-09-30 01:06:34,668][1157819] Updated weights for policy 0, policy_version 35228 (0.0006) [2024-09-30 01:06:35,284][1157819] Updated weights for policy 0, policy_version 35238 (0.0006) [2024-09-30 01:06:35,466][1157520] Fps is (10 sec: 71270.4, 60 sec: 77755.8, 300 sec: 76588.3). Total num frames: 144347136. Throughput: 0: 19353.7. Samples: 26075352. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:06:35,466][1157520] Avg episode reward: [(0, '47.982')] [2024-09-30 01:06:35,868][1157819] Updated weights for policy 0, policy_version 35248 (0.0006) [2024-09-30 01:06:36,434][1157819] Updated weights for policy 0, policy_version 35258 (0.0006) [2024-09-30 01:06:37,004][1157819] Updated weights for policy 0, policy_version 35268 (0.0006) [2024-09-30 01:06:37,577][1157819] Updated weights for policy 0, policy_version 35278 (0.0006) [2024-09-30 01:06:38,145][1157819] Updated weights for policy 0, policy_version 35288 (0.0006) [2024-09-30 01:06:38,759][1157819] Updated weights for policy 0, policy_version 35298 (0.0006) [2024-09-30 01:06:39,316][1157819] Updated weights for policy 0, policy_version 35308 (0.0006) [2024-09-30 01:06:39,869][1157819] Updated weights for policy 0, policy_version 35318 (0.0006) [2024-09-30 01:06:40,395][1157819] Updated weights for policy 0, policy_version 35328 (0.0006) [2024-09-30 01:06:40,466][1157520] Fps is (10 sec: 71270.7, 60 sec: 77209.6, 300 sec: 76463.3). Total num frames: 144707584. Throughput: 0: 19197.1. Samples: 26128412. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:06:40,466][1157520] Avg episode reward: [(0, '47.535')] [2024-09-30 01:06:40,961][1157819] Updated weights for policy 0, policy_version 35338 (0.0006) [2024-09-30 01:06:41,491][1157819] Updated weights for policy 0, policy_version 35348 (0.0006) [2024-09-30 01:06:42,063][1157819] Updated weights for policy 0, policy_version 35358 (0.0006) [2024-09-30 01:06:42,596][1157819] Updated weights for policy 0, policy_version 35368 (0.0006) [2024-09-30 01:06:43,107][1157819] Updated weights for policy 0, policy_version 35378 (0.0006) [2024-09-30 01:06:43,679][1157819] Updated weights for policy 0, policy_version 35388 (0.0006) [2024-09-30 01:06:44,204][1157819] Updated weights for policy 0, policy_version 35398 (0.0006) [2024-09-30 01:06:44,756][1157819] Updated weights for policy 0, policy_version 35408 (0.0006) [2024-09-30 01:06:45,317][1157819] Updated weights for policy 0, policy_version 35418 (0.0006) [2024-09-30 01:06:45,466][1157520] Fps is (10 sec: 73318.5, 60 sec: 76800.0, 300 sec: 76352.2). Total num frames: 145080320. Throughput: 0: 19069.0. Samples: 26239368. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:06:45,466][1157520] Avg episode reward: [(0, '46.979')] [2024-09-30 01:06:45,893][1157819] Updated weights for policy 0, policy_version 35428 (0.0006) [2024-09-30 01:06:46,408][1157819] Updated weights for policy 0, policy_version 35438 (0.0005) [2024-09-30 01:06:46,913][1157819] Updated weights for policy 0, policy_version 35448 (0.0006) [2024-09-30 01:06:47,511][1157819] Updated weights for policy 0, policy_version 35458 (0.0007) [2024-09-30 01:06:48,010][1157819] Updated weights for policy 0, policy_version 35468 (0.0006) [2024-09-30 01:06:48,559][1157819] Updated weights for policy 0, policy_version 35478 (0.0006) [2024-09-30 01:06:49,127][1157819] Updated weights for policy 0, policy_version 35488 (0.0006) [2024-09-30 01:06:49,687][1157819] Updated weights for policy 0, policy_version 35498 (0.0006) [2024-09-30 01:06:50,231][1157819] Updated weights for policy 0, policy_version 35508 (0.0006) [2024-09-30 01:06:50,466][1157520] Fps is (10 sec: 74956.5, 60 sec: 76458.7, 300 sec: 76255.0). Total num frames: 145457152. Throughput: 0: 18969.2. Samples: 26352028. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:06:50,466][1157520] Avg episode reward: [(0, '49.003')] [2024-09-30 01:06:50,767][1157819] Updated weights for policy 0, policy_version 35518 (0.0006) [2024-09-30 01:06:51,346][1157819] Updated weights for policy 0, policy_version 35528 (0.0006) [2024-09-30 01:06:51,893][1157819] Updated weights for policy 0, policy_version 35538 (0.0006) [2024-09-30 01:06:52,433][1157819] Updated weights for policy 0, policy_version 35548 (0.0006) [2024-09-30 01:06:52,987][1157819] Updated weights for policy 0, policy_version 35558 (0.0006) [2024-09-30 01:06:53,531][1157819] Updated weights for policy 0, policy_version 35568 (0.0006) [2024-09-30 01:06:54,073][1157819] Updated weights for policy 0, policy_version 35578 (0.0006) [2024-09-30 01:06:54,611][1157819] Updated weights for policy 0, policy_version 35588 (0.0006) [2024-09-30 01:06:55,176][1157819] Updated weights for policy 0, policy_version 35598 (0.0006) [2024-09-30 01:06:55,466][1157520] Fps is (10 sec: 74547.0, 60 sec: 76049.2, 300 sec: 76157.8). Total num frames: 145825792. Throughput: 0: 18875.2. Samples: 26407628. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:06:55,466][1157520] Avg episode reward: [(0, '47.160')] [2024-09-30 01:06:55,742][1157819] Updated weights for policy 0, policy_version 35608 (0.0006) [2024-09-30 01:06:56,270][1157819] Updated weights for policy 0, policy_version 35618 (0.0006) [2024-09-30 01:06:56,810][1157819] Updated weights for policy 0, policy_version 35628 (0.0006) [2024-09-30 01:06:57,352][1157819] Updated weights for policy 0, policy_version 35638 (0.0006) [2024-09-30 01:06:57,855][1157819] Updated weights for policy 0, policy_version 35648 (0.0006) [2024-09-30 01:06:58,448][1157819] Updated weights for policy 0, policy_version 35658 (0.0006) [2024-09-30 01:06:58,976][1157819] Updated weights for policy 0, policy_version 35668 (0.0006) [2024-09-30 01:06:59,532][1157819] Updated weights for policy 0, policy_version 35678 (0.0006) [2024-09-30 01:07:00,079][1157819] Updated weights for policy 0, policy_version 35688 (0.0006) [2024-09-30 01:07:00,466][1157520] Fps is (10 sec: 74957.0, 60 sec: 75776.0, 300 sec: 76185.6). Total num frames: 146206720. Throughput: 0: 18766.3. Samples: 26520176. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:07:00,466][1157520] Avg episode reward: [(0, '50.225')] [2024-09-30 01:07:00,554][1157736] Signal inference workers to stop experience collection... (1950 times) [2024-09-30 01:07:00,554][1157736] Signal inference workers to resume experience collection... (1950 times) [2024-09-30 01:07:00,558][1157819] InferenceWorker_p0-w0: stopping experience collection (1950 times) [2024-09-30 01:07:00,560][1157819] InferenceWorker_p0-w0: resuming experience collection (1950 times) [2024-09-30 01:07:00,620][1157819] Updated weights for policy 0, policy_version 35698 (0.0006) [2024-09-30 01:07:01,193][1157819] Updated weights for policy 0, policy_version 35708 (0.0006) [2024-09-30 01:07:01,707][1157819] Updated weights for policy 0, policy_version 35718 (0.0006) [2024-09-30 01:07:02,227][1157819] Updated weights for policy 0, policy_version 35728 (0.0006) [2024-09-30 01:07:02,781][1157819] Updated weights for policy 0, policy_version 35738 (0.0005) [2024-09-30 01:07:03,324][1157819] Updated weights for policy 0, policy_version 35748 (0.0006) [2024-09-30 01:07:03,903][1157819] Updated weights for policy 0, policy_version 35758 (0.0006) [2024-09-30 01:07:04,418][1157819] Updated weights for policy 0, policy_version 35768 (0.0006) [2024-09-30 01:07:04,978][1157819] Updated weights for policy 0, policy_version 35778 (0.0006) [2024-09-30 01:07:05,466][1157520] Fps is (10 sec: 75366.3, 60 sec: 75571.3, 300 sec: 76268.9). Total num frames: 146579456. Throughput: 0: 18538.9. Samples: 26633124. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:07:05,466][1157520] Avg episode reward: [(0, '48.351')] [2024-09-30 01:07:05,523][1157819] Updated weights for policy 0, policy_version 35788 (0.0006) [2024-09-30 01:07:06,058][1157819] Updated weights for policy 0, policy_version 35798 (0.0006) [2024-09-30 01:07:06,570][1157819] Updated weights for policy 0, policy_version 35808 (0.0006) [2024-09-30 01:07:07,075][1157819] Updated weights for policy 0, policy_version 35818 (0.0006) [2024-09-30 01:07:07,603][1157819] Updated weights for policy 0, policy_version 35828 (0.0006) [2024-09-30 01:07:08,111][1157819] Updated weights for policy 0, policy_version 35838 (0.0006) [2024-09-30 01:07:08,608][1157819] Updated weights for policy 0, policy_version 35848 (0.0006) [2024-09-30 01:07:09,100][1157819] Updated weights for policy 0, policy_version 35858 (0.0006) [2024-09-30 01:07:09,618][1157819] Updated weights for policy 0, policy_version 35868 (0.0006) [2024-09-30 01:07:10,181][1157819] Updated weights for policy 0, policy_version 35878 (0.0006) [2024-09-30 01:07:10,466][1157520] Fps is (10 sec: 77004.8, 60 sec: 75707.7, 300 sec: 76352.2). Total num frames: 146976768. Throughput: 0: 18535.1. Samples: 26691920. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:07:10,466][1157520] Avg episode reward: [(0, '53.651')] [2024-09-30 01:07:10,466][1157736] Saving new best policy, reward=53.651! [2024-09-30 01:07:10,686][1157819] Updated weights for policy 0, policy_version 35888 (0.0006) [2024-09-30 01:07:11,199][1157819] Updated weights for policy 0, policy_version 35898 (0.0006) [2024-09-30 01:07:11,718][1157819] Updated weights for policy 0, policy_version 35908 (0.0006) [2024-09-30 01:07:12,271][1157819] Updated weights for policy 0, policy_version 35918 (0.0006) [2024-09-30 01:07:12,804][1157819] Updated weights for policy 0, policy_version 35928 (0.0006) [2024-09-30 01:07:13,331][1157819] Updated weights for policy 0, policy_version 35938 (0.0006) [2024-09-30 01:07:13,834][1157819] Updated weights for policy 0, policy_version 35948 (0.0006) [2024-09-30 01:07:14,346][1157819] Updated weights for policy 0, policy_version 35958 (0.0006) [2024-09-30 01:07:14,863][1157819] Updated weights for policy 0, policy_version 35968 (0.0006) [2024-09-30 01:07:15,374][1157819] Updated weights for policy 0, policy_version 35978 (0.0006) [2024-09-30 01:07:15,466][1157520] Fps is (10 sec: 79052.9, 60 sec: 75502.9, 300 sec: 76393.9). Total num frames: 147369984. Throughput: 0: 18646.0. Samples: 26809572. Policy #0 lag: (min: 0.0, avg: 2.5, max: 7.0) [2024-09-30 01:07:15,466][1157520] Avg episode reward: [(0, '51.435')] [2024-09-30 01:07:15,884][1157819] Updated weights for policy 0, policy_version 35988 (0.0006) [2024-09-30 01:07:16,416][1157819] Updated weights for policy 0, policy_version 35998 (0.0006) [2024-09-30 01:07:16,962][1157819] Updated weights for policy 0, policy_version 36008 (0.0006) [2024-09-30 01:07:17,461][1157819] Updated weights for policy 0, policy_version 36018 (0.0006) [2024-09-30 01:07:17,968][1157819] Updated weights for policy 0, policy_version 36028 (0.0006) [2024-09-30 01:07:18,477][1157819] Updated weights for policy 0, policy_version 36038 (0.0006) [2024-09-30 01:07:19,062][1157819] Updated weights for policy 0, policy_version 36048 (0.0006) [2024-09-30 01:07:19,625][1157819] Updated weights for policy 0, policy_version 36058 (0.0006) [2024-09-30 01:07:20,184][1157819] Updated weights for policy 0, policy_version 36068 (0.0006) [2024-09-30 01:07:20,466][1157520] Fps is (10 sec: 77823.7, 60 sec: 75161.6, 300 sec: 76477.2). Total num frames: 147755008. Throughput: 0: 18916.7. Samples: 26926604. Policy #0 lag: (min: 0.0, avg: 2.5, max: 7.0) [2024-09-30 01:07:20,466][1157520] Avg episode reward: [(0, '51.591')] [2024-09-30 01:07:20,691][1157819] Updated weights for policy 0, policy_version 36078 (0.0006) [2024-09-30 01:07:21,246][1157819] Updated weights for policy 0, policy_version 36088 (0.0006) [2024-09-30 01:07:21,798][1157819] Updated weights for policy 0, policy_version 36098 (0.0006) [2024-09-30 01:07:22,340][1157819] Updated weights for policy 0, policy_version 36108 (0.0006) [2024-09-30 01:07:22,852][1157819] Updated weights for policy 0, policy_version 36118 (0.0006) [2024-09-30 01:07:23,399][1157819] Updated weights for policy 0, policy_version 36128 (0.0006) [2024-09-30 01:07:23,928][1157819] Updated weights for policy 0, policy_version 36138 (0.0006) [2024-09-30 01:07:24,440][1157819] Updated weights for policy 0, policy_version 36148 (0.0006) [2024-09-30 01:07:24,942][1157819] Updated weights for policy 0, policy_version 36158 (0.0006) [2024-09-30 01:07:25,464][1157819] Updated weights for policy 0, policy_version 36168 (0.0006) [2024-09-30 01:07:25,466][1157520] Fps is (10 sec: 77414.5, 60 sec: 75161.6, 300 sec: 76546.6). Total num frames: 148144128. Throughput: 0: 18998.8. Samples: 26983360. Policy #0 lag: (min: 0.0, avg: 2.5, max: 7.0) [2024-09-30 01:07:25,466][1157520] Avg episode reward: [(0, '51.562')] [2024-09-30 01:07:25,975][1157819] Updated weights for policy 0, policy_version 36178 (0.0006) [2024-09-30 01:07:26,493][1157819] Updated weights for policy 0, policy_version 36188 (0.0006) [2024-09-30 01:07:27,018][1157819] Updated weights for policy 0, policy_version 36198 (0.0006) [2024-09-30 01:07:27,531][1157819] Updated weights for policy 0, policy_version 36208 (0.0006) [2024-09-30 01:07:28,036][1157819] Updated weights for policy 0, policy_version 36218 (0.0006) [2024-09-30 01:07:28,574][1157819] Updated weights for policy 0, policy_version 36228 (0.0006) [2024-09-30 01:07:29,120][1157819] Updated weights for policy 0, policy_version 36238 (0.0006) [2024-09-30 01:07:29,661][1157819] Updated weights for policy 0, policy_version 36248 (0.0006) [2024-09-30 01:07:30,154][1157819] Updated weights for policy 0, policy_version 36258 (0.0006) [2024-09-30 01:07:30,466][1157520] Fps is (10 sec: 77824.2, 60 sec: 75639.5, 300 sec: 76616.0). Total num frames: 148533248. Throughput: 0: 19157.5. Samples: 27101456. Policy #0 lag: (min: 0.0, avg: 2.5, max: 7.0) [2024-09-30 01:07:30,466][1157520] Avg episode reward: [(0, '50.904')] [2024-09-30 01:07:30,667][1157819] Updated weights for policy 0, policy_version 36268 (0.0005) [2024-09-30 01:07:31,202][1157819] Updated weights for policy 0, policy_version 36278 (0.0006) [2024-09-30 01:07:31,703][1157819] Updated weights for policy 0, policy_version 36288 (0.0006) [2024-09-30 01:07:32,214][1157819] Updated weights for policy 0, policy_version 36298 (0.0006) [2024-09-30 01:07:32,743][1157819] Updated weights for policy 0, policy_version 36308 (0.0006) [2024-09-30 01:07:33,287][1157819] Updated weights for policy 0, policy_version 36318 (0.0006) [2024-09-30 01:07:33,797][1157819] Updated weights for policy 0, policy_version 36328 (0.0006) [2024-09-30 01:07:34,302][1157819] Updated weights for policy 0, policy_version 36338 (0.0006) [2024-09-30 01:07:34,806][1157819] Updated weights for policy 0, policy_version 36348 (0.0006) [2024-09-30 01:07:35,355][1157819] Updated weights for policy 0, policy_version 36358 (0.0006) [2024-09-30 01:07:35,466][1157520] Fps is (10 sec: 78643.0, 60 sec: 76390.4, 300 sec: 76713.2). Total num frames: 148930560. Throughput: 0: 19273.3. Samples: 27219328. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:07:35,466][1157520] Avg episode reward: [(0, '50.691')] [2024-09-30 01:07:35,885][1157819] Updated weights for policy 0, policy_version 36368 (0.0006) [2024-09-30 01:07:36,380][1157819] Updated weights for policy 0, policy_version 36378 (0.0006) [2024-09-30 01:07:36,901][1157819] Updated weights for policy 0, policy_version 36388 (0.0006) [2024-09-30 01:07:37,408][1157819] Updated weights for policy 0, policy_version 36398 (0.0006) [2024-09-30 01:07:37,928][1157819] Updated weights for policy 0, policy_version 36408 (0.0006) [2024-09-30 01:07:38,416][1157819] Updated weights for policy 0, policy_version 36418 (0.0006) [2024-09-30 01:07:38,961][1157819] Updated weights for policy 0, policy_version 36428 (0.0006) [2024-09-30 01:07:39,474][1157819] Updated weights for policy 0, policy_version 36438 (0.0006) [2024-09-30 01:07:39,975][1157819] Updated weights for policy 0, policy_version 36448 (0.0006) [2024-09-30 01:07:40,466][1157520] Fps is (10 sec: 79462.1, 60 sec: 77004.7, 300 sec: 76810.4). Total num frames: 149327872. Throughput: 0: 19365.9. Samples: 27279092. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:07:40,466][1157520] Avg episode reward: [(0, '50.921')] [2024-09-30 01:07:40,498][1157819] Updated weights for policy 0, policy_version 36458 (0.0006) [2024-09-30 01:07:40,988][1157819] Updated weights for policy 0, policy_version 36468 (0.0006) [2024-09-30 01:07:41,506][1157819] Updated weights for policy 0, policy_version 36478 (0.0006) [2024-09-30 01:07:42,072][1157819] Updated weights for policy 0, policy_version 36488 (0.0006) [2024-09-30 01:07:42,608][1157819] Updated weights for policy 0, policy_version 36498 (0.0006) [2024-09-30 01:07:43,190][1157819] Updated weights for policy 0, policy_version 36508 (0.0006) [2024-09-30 01:07:43,747][1157819] Updated weights for policy 0, policy_version 36518 (0.0006) [2024-09-30 01:07:44,333][1157819] Updated weights for policy 0, policy_version 36528 (0.0006) [2024-09-30 01:07:44,849][1157819] Updated weights for policy 0, policy_version 36538 (0.0006) [2024-09-30 01:07:45,366][1157819] Updated weights for policy 0, policy_version 36548 (0.0006) [2024-09-30 01:07:45,466][1157520] Fps is (10 sec: 77414.5, 60 sec: 77073.0, 300 sec: 76949.3). Total num frames: 149704704. Throughput: 0: 19436.3. Samples: 27394812. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:07:45,466][1157520] Avg episode reward: [(0, '49.524')] [2024-09-30 01:07:45,866][1157819] Updated weights for policy 0, policy_version 36558 (0.0006) [2024-09-30 01:07:46,369][1157819] Updated weights for policy 0, policy_version 36568 (0.0006) [2024-09-30 01:07:46,903][1157819] Updated weights for policy 0, policy_version 36578 (0.0006) [2024-09-30 01:07:47,416][1157819] Updated weights for policy 0, policy_version 36588 (0.0006) [2024-09-30 01:07:47,930][1157819] Updated weights for policy 0, policy_version 36598 (0.0006) [2024-09-30 01:07:48,448][1157819] Updated weights for policy 0, policy_version 36608 (0.0006) [2024-09-30 01:07:48,970][1157819] Updated weights for policy 0, policy_version 36618 (0.0006) [2024-09-30 01:07:49,472][1157819] Updated weights for policy 0, policy_version 36628 (0.0006) [2024-09-30 01:07:49,992][1157819] Updated weights for policy 0, policy_version 36638 (0.0006) [2024-09-30 01:07:50,466][1157520] Fps is (10 sec: 77414.6, 60 sec: 77414.4, 300 sec: 77060.3). Total num frames: 150102016. Throughput: 0: 19550.9. Samples: 27512916. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:07:50,466][1157520] Avg episode reward: [(0, '50.851')] [2024-09-30 01:07:50,539][1157819] Updated weights for policy 0, policy_version 36648 (0.0006) [2024-09-30 01:07:51,041][1157819] Updated weights for policy 0, policy_version 36658 (0.0006) [2024-09-30 01:07:51,561][1157819] Updated weights for policy 0, policy_version 36668 (0.0006) [2024-09-30 01:07:52,072][1157819] Updated weights for policy 0, policy_version 36678 (0.0006) [2024-09-30 01:07:52,580][1157819] Updated weights for policy 0, policy_version 36688 (0.0006) [2024-09-30 01:07:53,107][1157819] Updated weights for policy 0, policy_version 36698 (0.0006) [2024-09-30 01:07:53,616][1157819] Updated weights for policy 0, policy_version 36708 (0.0006) [2024-09-30 01:07:54,118][1157819] Updated weights for policy 0, policy_version 36718 (0.0006) [2024-09-30 01:07:54,645][1157819] Updated weights for policy 0, policy_version 36728 (0.0006) [2024-09-30 01:07:55,157][1157819] Updated weights for policy 0, policy_version 36738 (0.0006) [2024-09-30 01:07:55,466][1157520] Fps is (10 sec: 79872.2, 60 sec: 77960.6, 300 sec: 77157.5). Total num frames: 150503424. Throughput: 0: 19565.4. Samples: 27572364. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:07:55,466][1157520] Avg episode reward: [(0, '47.266')] [2024-09-30 01:07:55,692][1157819] Updated weights for policy 0, policy_version 36748 (0.0006) [2024-09-30 01:07:56,220][1157819] Updated weights for policy 0, policy_version 36758 (0.0006) [2024-09-30 01:07:56,719][1157819] Updated weights for policy 0, policy_version 36768 (0.0006) [2024-09-30 01:07:57,245][1157819] Updated weights for policy 0, policy_version 36778 (0.0006) [2024-09-30 01:07:57,739][1157819] Updated weights for policy 0, policy_version 36788 (0.0006) [2024-09-30 01:07:58,243][1157819] Updated weights for policy 0, policy_version 36798 (0.0006) [2024-09-30 01:07:58,748][1157819] Updated weights for policy 0, policy_version 36808 (0.0006) [2024-09-30 01:07:59,246][1157819] Updated weights for policy 0, policy_version 36818 (0.0006) [2024-09-30 01:07:59,747][1157819] Updated weights for policy 0, policy_version 36828 (0.0006) [2024-09-30 01:08:00,261][1157819] Updated weights for policy 0, policy_version 36838 (0.0006) [2024-09-30 01:08:00,466][1157520] Fps is (10 sec: 80281.7, 60 sec: 78301.9, 300 sec: 77296.4). Total num frames: 150904832. Throughput: 0: 19616.2. Samples: 27692300. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 01:08:00,466][1157520] Avg episode reward: [(0, '49.453')] [2024-09-30 01:08:00,756][1157819] Updated weights for policy 0, policy_version 36848 (0.0006) [2024-09-30 01:08:01,259][1157819] Updated weights for policy 0, policy_version 36858 (0.0006) [2024-09-30 01:08:01,776][1157819] Updated weights for policy 0, policy_version 36868 (0.0006) [2024-09-30 01:08:02,301][1157819] Updated weights for policy 0, policy_version 36878 (0.0006) [2024-09-30 01:08:02,806][1157819] Updated weights for policy 0, policy_version 36888 (0.0006) [2024-09-30 01:08:03,308][1157819] Updated weights for policy 0, policy_version 36898 (0.0006) [2024-09-30 01:08:03,803][1157819] Updated weights for policy 0, policy_version 36908 (0.0006) [2024-09-30 01:08:04,312][1157819] Updated weights for policy 0, policy_version 36918 (0.0006) [2024-09-30 01:08:04,843][1157819] Updated weights for policy 0, policy_version 36928 (0.0006) [2024-09-30 01:08:05,358][1157819] Updated weights for policy 0, policy_version 36938 (0.0006) [2024-09-30 01:08:05,466][1157520] Fps is (10 sec: 80281.0, 60 sec: 78779.7, 300 sec: 77379.7). Total num frames: 151306240. Throughput: 0: 19703.5. Samples: 27813264. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 01:08:05,466][1157520] Avg episode reward: [(0, '53.193')] [2024-09-30 01:08:05,469][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000036940_151306240.pth... [2024-09-30 01:08:05,514][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000032361_132550656.pth [2024-09-30 01:08:05,868][1157819] Updated weights for policy 0, policy_version 36948 (0.0006) [2024-09-30 01:08:06,106][1157736] Signal inference workers to stop experience collection... (2000 times) [2024-09-30 01:08:06,107][1157736] Signal inference workers to resume experience collection... (2000 times) [2024-09-30 01:08:06,112][1157819] InferenceWorker_p0-w0: stopping experience collection (2000 times) [2024-09-30 01:08:06,112][1157819] InferenceWorker_p0-w0: resuming experience collection (2000 times) [2024-09-30 01:08:06,349][1157819] Updated weights for policy 0, policy_version 36958 (0.0006) [2024-09-30 01:08:06,836][1157819] Updated weights for policy 0, policy_version 36968 (0.0006) [2024-09-30 01:08:07,348][1157819] Updated weights for policy 0, policy_version 36978 (0.0006) [2024-09-30 01:08:07,881][1157819] Updated weights for policy 0, policy_version 36988 (0.0006) [2024-09-30 01:08:08,378][1157819] Updated weights for policy 0, policy_version 36998 (0.0006) [2024-09-30 01:08:08,875][1157819] Updated weights for policy 0, policy_version 37008 (0.0006) [2024-09-30 01:08:09,381][1157819] Updated weights for policy 0, policy_version 37018 (0.0006) [2024-09-30 01:08:09,876][1157819] Updated weights for policy 0, policy_version 37028 (0.0006) [2024-09-30 01:08:10,379][1157819] Updated weights for policy 0, policy_version 37038 (0.0006) [2024-09-30 01:08:10,466][1157520] Fps is (10 sec: 80690.8, 60 sec: 78916.2, 300 sec: 77476.9). Total num frames: 151711744. Throughput: 0: 19798.8. Samples: 27874308. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 01:08:10,466][1157520] Avg episode reward: [(0, '50.772')] [2024-09-30 01:08:10,889][1157819] Updated weights for policy 0, policy_version 37048 (0.0006) [2024-09-30 01:08:11,412][1157819] Updated weights for policy 0, policy_version 37058 (0.0006) [2024-09-30 01:08:11,907][1157819] Updated weights for policy 0, policy_version 37068 (0.0006) [2024-09-30 01:08:12,455][1157819] Updated weights for policy 0, policy_version 37078 (0.0006) [2024-09-30 01:08:12,941][1157819] Updated weights for policy 0, policy_version 37088 (0.0006) [2024-09-30 01:08:13,453][1157819] Updated weights for policy 0, policy_version 37098 (0.0006) [2024-09-30 01:08:13,964][1157819] Updated weights for policy 0, policy_version 37108 (0.0006) [2024-09-30 01:08:14,472][1157819] Updated weights for policy 0, policy_version 37118 (0.0006) [2024-09-30 01:08:14,985][1157819] Updated weights for policy 0, policy_version 37128 (0.0006) [2024-09-30 01:08:15,466][1157520] Fps is (10 sec: 80689.6, 60 sec: 79052.5, 300 sec: 77615.7). Total num frames: 152113152. Throughput: 0: 19857.0. Samples: 27995024. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 01:08:15,466][1157520] Avg episode reward: [(0, '49.269')] [2024-09-30 01:08:15,513][1157819] Updated weights for policy 0, policy_version 37138 (0.0006) [2024-09-30 01:08:16,028][1157819] Updated weights for policy 0, policy_version 37148 (0.0006) [2024-09-30 01:08:16,519][1157819] Updated weights for policy 0, policy_version 37158 (0.0006) [2024-09-30 01:08:17,042][1157819] Updated weights for policy 0, policy_version 37168 (0.0006) [2024-09-30 01:08:17,548][1157819] Updated weights for policy 0, policy_version 37178 (0.0006) [2024-09-30 01:08:18,039][1157819] Updated weights for policy 0, policy_version 37188 (0.0006) [2024-09-30 01:08:18,564][1157819] Updated weights for policy 0, policy_version 37198 (0.0006) [2024-09-30 01:08:19,067][1157819] Updated weights for policy 0, policy_version 37208 (0.0006) [2024-09-30 01:08:19,581][1157819] Updated weights for policy 0, policy_version 37218 (0.0006) [2024-09-30 01:08:20,077][1157819] Updated weights for policy 0, policy_version 37228 (0.0006) [2024-09-30 01:08:20,466][1157520] Fps is (10 sec: 80280.9, 60 sec: 79325.7, 300 sec: 77796.2). Total num frames: 152514560. Throughput: 0: 19914.5. Samples: 28115484. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 01:08:20,466][1157520] Avg episode reward: [(0, '50.554')] [2024-09-30 01:08:20,568][1157819] Updated weights for policy 0, policy_version 37238 (0.0006) [2024-09-30 01:08:21,081][1157819] Updated weights for policy 0, policy_version 37248 (0.0006) [2024-09-30 01:08:21,580][1157819] Updated weights for policy 0, policy_version 37258 (0.0006) [2024-09-30 01:08:22,088][1157819] Updated weights for policy 0, policy_version 37268 (0.0006) [2024-09-30 01:08:22,591][1157819] Updated weights for policy 0, policy_version 37278 (0.0006) [2024-09-30 01:08:23,076][1157819] Updated weights for policy 0, policy_version 37288 (0.0006) [2024-09-30 01:08:23,567][1157819] Updated weights for policy 0, policy_version 37298 (0.0006) [2024-09-30 01:08:24,065][1157819] Updated weights for policy 0, policy_version 37308 (0.0006) [2024-09-30 01:08:24,528][1157819] Updated weights for policy 0, policy_version 37318 (0.0006) [2024-09-30 01:08:25,017][1157819] Updated weights for policy 0, policy_version 37328 (0.0006) [2024-09-30 01:08:25,466][1157520] Fps is (10 sec: 81921.6, 60 sec: 79803.7, 300 sec: 78004.5). Total num frames: 152932352. Throughput: 0: 19948.9. Samples: 28176792. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:08:25,466][1157520] Avg episode reward: [(0, '51.096')] [2024-09-30 01:08:25,505][1157819] Updated weights for policy 0, policy_version 37338 (0.0006) [2024-09-30 01:08:25,969][1157819] Updated weights for policy 0, policy_version 37348 (0.0006) [2024-09-30 01:08:26,423][1157819] Updated weights for policy 0, policy_version 37358 (0.0006) [2024-09-30 01:08:26,909][1157819] Updated weights for policy 0, policy_version 37368 (0.0006) [2024-09-30 01:08:27,462][1157819] Updated weights for policy 0, policy_version 37378 (0.0006) [2024-09-30 01:08:27,951][1157819] Updated weights for policy 0, policy_version 37388 (0.0006) [2024-09-30 01:08:28,451][1157819] Updated weights for policy 0, policy_version 37398 (0.0006) [2024-09-30 01:08:28,971][1157819] Updated weights for policy 0, policy_version 37408 (0.0006) [2024-09-30 01:08:29,492][1157819] Updated weights for policy 0, policy_version 37418 (0.0006) [2024-09-30 01:08:30,017][1157819] Updated weights for policy 0, policy_version 37428 (0.0006) [2024-09-30 01:08:30,466][1157520] Fps is (10 sec: 82739.2, 60 sec: 80144.9, 300 sec: 78115.6). Total num frames: 153341952. Throughput: 0: 20163.5. Samples: 28302172. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:08:30,466][1157520] Avg episode reward: [(0, '53.846')] [2024-09-30 01:08:30,467][1157736] Saving new best policy, reward=53.846! [2024-09-30 01:08:30,543][1157819] Updated weights for policy 0, policy_version 37438 (0.0006) [2024-09-30 01:08:31,002][1157819] Updated weights for policy 0, policy_version 37448 (0.0006) [2024-09-30 01:08:31,500][1157819] Updated weights for policy 0, policy_version 37458 (0.0006) [2024-09-30 01:08:32,027][1157819] Updated weights for policy 0, policy_version 37468 (0.0006) [2024-09-30 01:08:32,585][1157819] Updated weights for policy 0, policy_version 37478 (0.0006) [2024-09-30 01:08:33,149][1157819] Updated weights for policy 0, policy_version 37488 (0.0006) [2024-09-30 01:08:33,694][1157819] Updated weights for policy 0, policy_version 37498 (0.0006) [2024-09-30 01:08:34,225][1157819] Updated weights for policy 0, policy_version 37508 (0.0006) [2024-09-30 01:08:34,756][1157819] Updated weights for policy 0, policy_version 37518 (0.0006) [2024-09-30 01:08:35,303][1157819] Updated weights for policy 0, policy_version 37528 (0.0006) [2024-09-30 01:08:35,466][1157520] Fps is (10 sec: 79052.4, 60 sec: 79871.9, 300 sec: 78171.1). Total num frames: 153722880. Throughput: 0: 20129.8. Samples: 28418760. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:08:35,466][1157520] Avg episode reward: [(0, '52.111')] [2024-09-30 01:08:35,869][1157819] Updated weights for policy 0, policy_version 37538 (0.0006) [2024-09-30 01:08:36,416][1157819] Updated weights for policy 0, policy_version 37548 (0.0006) [2024-09-30 01:08:36,927][1157819] Updated weights for policy 0, policy_version 37558 (0.0006) [2024-09-30 01:08:37,449][1157819] Updated weights for policy 0, policy_version 37568 (0.0006) [2024-09-30 01:08:37,980][1157819] Updated weights for policy 0, policy_version 37578 (0.0006) [2024-09-30 01:08:38,516][1157819] Updated weights for policy 0, policy_version 37588 (0.0006) [2024-09-30 01:08:39,057][1157819] Updated weights for policy 0, policy_version 37598 (0.0006) [2024-09-30 01:08:39,600][1157819] Updated weights for policy 0, policy_version 37608 (0.0006) [2024-09-30 01:08:40,125][1157819] Updated weights for policy 0, policy_version 37618 (0.0006) [2024-09-30 01:08:40,466][1157520] Fps is (10 sec: 76596.3, 60 sec: 79667.3, 300 sec: 78282.2). Total num frames: 154107904. Throughput: 0: 20081.4. Samples: 28476028. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:08:40,466][1157520] Avg episode reward: [(0, '46.898')] [2024-09-30 01:08:40,672][1157819] Updated weights for policy 0, policy_version 37628 (0.0006) [2024-09-30 01:08:41,229][1157819] Updated weights for policy 0, policy_version 37638 (0.0006) [2024-09-30 01:08:41,823][1157819] Updated weights for policy 0, policy_version 37648 (0.0006) [2024-09-30 01:08:42,342][1157819] Updated weights for policy 0, policy_version 37658 (0.0006) [2024-09-30 01:08:42,866][1157819] Updated weights for policy 0, policy_version 37668 (0.0006) [2024-09-30 01:08:43,393][1157819] Updated weights for policy 0, policy_version 37678 (0.0006) [2024-09-30 01:08:43,946][1157819] Updated weights for policy 0, policy_version 37688 (0.0006) [2024-09-30 01:08:44,453][1157819] Updated weights for policy 0, policy_version 37698 (0.0006) [2024-09-30 01:08:44,978][1157819] Updated weights for policy 0, policy_version 37708 (0.0006) [2024-09-30 01:08:45,198][1157736] Signal inference workers to stop experience collection... (2050 times) [2024-09-30 01:08:45,199][1157736] Signal inference workers to resume experience collection... (2050 times) [2024-09-30 01:08:45,202][1157819] InferenceWorker_p0-w0: stopping experience collection (2050 times) [2024-09-30 01:08:45,203][1157819] InferenceWorker_p0-w0: resuming experience collection (2050 times) [2024-09-30 01:08:45,466][1157520] Fps is (10 sec: 76186.3, 60 sec: 79667.2, 300 sec: 78365.5). Total num frames: 154484736. Throughput: 0: 19946.7. Samples: 28589904. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 01:08:45,466][1157520] Avg episode reward: [(0, '51.459')] [2024-09-30 01:08:45,540][1157819] Updated weights for policy 0, policy_version 37718 (0.0006) [2024-09-30 01:08:46,082][1157819] Updated weights for policy 0, policy_version 37728 (0.0006) [2024-09-30 01:08:46,629][1157819] Updated weights for policy 0, policy_version 37738 (0.0006) [2024-09-30 01:08:47,164][1157819] Updated weights for policy 0, policy_version 37748 (0.0006) [2024-09-30 01:08:47,660][1157819] Updated weights for policy 0, policy_version 37758 (0.0006) [2024-09-30 01:08:48,154][1157819] Updated weights for policy 0, policy_version 37768 (0.0006) [2024-09-30 01:08:48,662][1157819] Updated weights for policy 0, policy_version 37778 (0.0006) [2024-09-30 01:08:49,171][1157819] Updated weights for policy 0, policy_version 37788 (0.0006) [2024-09-30 01:08:49,654][1157819] Updated weights for policy 0, policy_version 37798 (0.0006) [2024-09-30 01:08:50,148][1157819] Updated weights for policy 0, policy_version 37808 (0.0006) [2024-09-30 01:08:50,466][1157520] Fps is (10 sec: 77414.2, 60 sec: 79667.2, 300 sec: 78407.2). Total num frames: 154882048. Throughput: 0: 19875.4. Samples: 28707656. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 01:08:50,466][1157520] Avg episode reward: [(0, '50.946')] [2024-09-30 01:08:50,667][1157819] Updated weights for policy 0, policy_version 37818 (0.0006) [2024-09-30 01:08:51,197][1157819] Updated weights for policy 0, policy_version 37828 (0.0006) [2024-09-30 01:08:51,726][1157819] Updated weights for policy 0, policy_version 37838 (0.0006) [2024-09-30 01:08:52,250][1157819] Updated weights for policy 0, policy_version 37848 (0.0006) [2024-09-30 01:08:52,747][1157819] Updated weights for policy 0, policy_version 37858 (0.0006) [2024-09-30 01:08:53,273][1157819] Updated weights for policy 0, policy_version 37868 (0.0006) [2024-09-30 01:08:53,783][1157819] Updated weights for policy 0, policy_version 37878 (0.0006) [2024-09-30 01:08:54,316][1157819] Updated weights for policy 0, policy_version 37888 (0.0006) [2024-09-30 01:08:54,851][1157819] Updated weights for policy 0, policy_version 37898 (0.0006) [2024-09-30 01:08:55,349][1157819] Updated weights for policy 0, policy_version 37908 (0.0006) [2024-09-30 01:08:55,466][1157520] Fps is (10 sec: 79462.5, 60 sec: 79598.9, 300 sec: 78407.2). Total num frames: 155279360. Throughput: 0: 19847.6. Samples: 28767448. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 01:08:55,466][1157520] Avg episode reward: [(0, '51.826')] [2024-09-30 01:08:55,881][1157819] Updated weights for policy 0, policy_version 37918 (0.0006) [2024-09-30 01:08:56,388][1157819] Updated weights for policy 0, policy_version 37928 (0.0006) [2024-09-30 01:08:56,911][1157819] Updated weights for policy 0, policy_version 37938 (0.0006) [2024-09-30 01:08:57,427][1157819] Updated weights for policy 0, policy_version 37948 (0.0006) [2024-09-30 01:08:57,989][1157819] Updated weights for policy 0, policy_version 37958 (0.0006) [2024-09-30 01:08:58,469][1157819] Updated weights for policy 0, policy_version 37968 (0.0006) [2024-09-30 01:08:58,978][1157819] Updated weights for policy 0, policy_version 37978 (0.0006) [2024-09-30 01:08:59,485][1157819] Updated weights for policy 0, policy_version 37988 (0.0006) [2024-09-30 01:09:00,012][1157819] Updated weights for policy 0, policy_version 37998 (0.0006) [2024-09-30 01:09:00,466][1157520] Fps is (10 sec: 79052.8, 60 sec: 79462.4, 300 sec: 78379.4). Total num frames: 155672576. Throughput: 0: 19791.2. Samples: 28885624. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 01:09:00,466][1157520] Avg episode reward: [(0, '50.653')] [2024-09-30 01:09:00,555][1157819] Updated weights for policy 0, policy_version 38008 (0.0006) [2024-09-30 01:09:01,049][1157819] Updated weights for policy 0, policy_version 38018 (0.0006) [2024-09-30 01:09:01,548][1157819] Updated weights for policy 0, policy_version 38028 (0.0006) [2024-09-30 01:09:02,047][1157819] Updated weights for policy 0, policy_version 38038 (0.0006) [2024-09-30 01:09:02,560][1157819] Updated weights for policy 0, policy_version 38048 (0.0006) [2024-09-30 01:09:03,074][1157819] Updated weights for policy 0, policy_version 38058 (0.0006) [2024-09-30 01:09:03,552][1157819] Updated weights for policy 0, policy_version 38068 (0.0006) [2024-09-30 01:09:04,045][1157819] Updated weights for policy 0, policy_version 38078 (0.0006) [2024-09-30 01:09:04,548][1157819] Updated weights for policy 0, policy_version 38088 (0.0006) [2024-09-30 01:09:05,031][1157819] Updated weights for policy 0, policy_version 38098 (0.0006) [2024-09-30 01:09:05,466][1157520] Fps is (10 sec: 80691.2, 60 sec: 79667.3, 300 sec: 78448.8). Total num frames: 156086272. Throughput: 0: 19816.4. Samples: 29007220. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 01:09:05,466][1157520] Avg episode reward: [(0, '50.579')] [2024-09-30 01:09:05,520][1157819] Updated weights for policy 0, policy_version 38108 (0.0006) [2024-09-30 01:09:06,008][1157819] Updated weights for policy 0, policy_version 38118 (0.0006) [2024-09-30 01:09:06,489][1157819] Updated weights for policy 0, policy_version 38128 (0.0006) [2024-09-30 01:09:06,963][1157819] Updated weights for policy 0, policy_version 38138 (0.0006) [2024-09-30 01:09:07,463][1157819] Updated weights for policy 0, policy_version 38148 (0.0006) [2024-09-30 01:09:07,986][1157819] Updated weights for policy 0, policy_version 38158 (0.0006) [2024-09-30 01:09:08,522][1157819] Updated weights for policy 0, policy_version 38168 (0.0006) [2024-09-30 01:09:09,029][1157819] Updated weights for policy 0, policy_version 38178 (0.0006) [2024-09-30 01:09:09,528][1157819] Updated weights for policy 0, policy_version 38188 (0.0006) [2024-09-30 01:09:10,067][1157819] Updated weights for policy 0, policy_version 38198 (0.0006) [2024-09-30 01:09:10,466][1157520] Fps is (10 sec: 81920.2, 60 sec: 79667.3, 300 sec: 78421.1). Total num frames: 156491776. Throughput: 0: 19839.6. Samples: 29069572. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 01:09:10,466][1157520] Avg episode reward: [(0, '51.865')] [2024-09-30 01:09:10,587][1157819] Updated weights for policy 0, policy_version 38208 (0.0006) [2024-09-30 01:09:11,106][1157819] Updated weights for policy 0, policy_version 38218 (0.0006) [2024-09-30 01:09:11,621][1157819] Updated weights for policy 0, policy_version 38228 (0.0006) [2024-09-30 01:09:12,122][1157819] Updated weights for policy 0, policy_version 38238 (0.0006) [2024-09-30 01:09:12,640][1157819] Updated weights for policy 0, policy_version 38248 (0.0006) [2024-09-30 01:09:13,172][1157819] Updated weights for policy 0, policy_version 38258 (0.0006) [2024-09-30 01:09:13,689][1157819] Updated weights for policy 0, policy_version 38268 (0.0006) [2024-09-30 01:09:14,217][1157819] Updated weights for policy 0, policy_version 38278 (0.0006) [2024-09-30 01:09:14,709][1157819] Updated weights for policy 0, policy_version 38288 (0.0006) [2024-09-30 01:09:15,224][1157819] Updated weights for policy 0, policy_version 38298 (0.0006) [2024-09-30 01:09:15,466][1157520] Fps is (10 sec: 79462.1, 60 sec: 79462.7, 300 sec: 78365.5). Total num frames: 156880896. Throughput: 0: 19693.0. Samples: 29188356. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 01:09:15,466][1157520] Avg episode reward: [(0, '50.337')] [2024-09-30 01:09:15,794][1157819] Updated weights for policy 0, policy_version 38308 (0.0006) [2024-09-30 01:09:16,324][1157819] Updated weights for policy 0, policy_version 38318 (0.0006) [2024-09-30 01:09:16,651][1157736] Signal inference workers to stop experience collection... (2100 times) [2024-09-30 01:09:16,654][1157819] InferenceWorker_p0-w0: stopping experience collection (2100 times) [2024-09-30 01:09:16,658][1157736] Signal inference workers to resume experience collection... (2100 times) [2024-09-30 01:09:16,658][1157819] InferenceWorker_p0-w0: resuming experience collection (2100 times) [2024-09-30 01:09:16,812][1157819] Updated weights for policy 0, policy_version 38328 (0.0006) [2024-09-30 01:09:17,292][1157819] Updated weights for policy 0, policy_version 38338 (0.0006) [2024-09-30 01:09:17,809][1157819] Updated weights for policy 0, policy_version 38348 (0.0006) [2024-09-30 01:09:18,355][1157819] Updated weights for policy 0, policy_version 38358 (0.0006) [2024-09-30 01:09:18,868][1157819] Updated weights for policy 0, policy_version 38368 (0.0006) [2024-09-30 01:09:19,389][1157819] Updated weights for policy 0, policy_version 38378 (0.0006) [2024-09-30 01:09:19,893][1157819] Updated weights for policy 0, policy_version 38388 (0.0006) [2024-09-30 01:09:20,399][1157819] Updated weights for policy 0, policy_version 38398 (0.0006) [2024-09-30 01:09:20,466][1157520] Fps is (10 sec: 79052.9, 60 sec: 79462.6, 300 sec: 78490.5). Total num frames: 157282304. Throughput: 0: 19741.3. Samples: 29307116. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 01:09:20,466][1157520] Avg episode reward: [(0, '53.267')] [2024-09-30 01:09:20,918][1157819] Updated weights for policy 0, policy_version 38408 (0.0006) [2024-09-30 01:09:21,434][1157819] Updated weights for policy 0, policy_version 38418 (0.0006) [2024-09-30 01:09:21,977][1157819] Updated weights for policy 0, policy_version 38428 (0.0006) [2024-09-30 01:09:22,511][1157819] Updated weights for policy 0, policy_version 38438 (0.0006) [2024-09-30 01:09:23,021][1157819] Updated weights for policy 0, policy_version 38448 (0.0006) [2024-09-30 01:09:23,523][1157819] Updated weights for policy 0, policy_version 38458 (0.0006) [2024-09-30 01:09:24,057][1157819] Updated weights for policy 0, policy_version 38468 (0.0006) [2024-09-30 01:09:24,583][1157819] Updated weights for policy 0, policy_version 38478 (0.0006) [2024-09-30 01:09:25,080][1157819] Updated weights for policy 0, policy_version 38488 (0.0006) [2024-09-30 01:09:25,466][1157520] Fps is (10 sec: 79052.9, 60 sec: 78984.6, 300 sec: 78559.9). Total num frames: 157671424. Throughput: 0: 19779.0. Samples: 29366084. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 01:09:25,466][1157520] Avg episode reward: [(0, '53.805')] [2024-09-30 01:09:25,616][1157819] Updated weights for policy 0, policy_version 38498 (0.0006) [2024-09-30 01:09:26,121][1157819] Updated weights for policy 0, policy_version 38508 (0.0006) [2024-09-30 01:09:26,662][1157819] Updated weights for policy 0, policy_version 38518 (0.0006) [2024-09-30 01:09:27,174][1157819] Updated weights for policy 0, policy_version 38528 (0.0006) [2024-09-30 01:09:27,671][1157819] Updated weights for policy 0, policy_version 38538 (0.0006) [2024-09-30 01:09:28,194][1157819] Updated weights for policy 0, policy_version 38548 (0.0006) [2024-09-30 01:09:28,730][1157819] Updated weights for policy 0, policy_version 38558 (0.0006) [2024-09-30 01:09:29,264][1157819] Updated weights for policy 0, policy_version 38568 (0.0006) [2024-09-30 01:09:29,787][1157819] Updated weights for policy 0, policy_version 38578 (0.0006) [2024-09-30 01:09:30,278][1157819] Updated weights for policy 0, policy_version 38588 (0.0006) [2024-09-30 01:09:30,466][1157520] Fps is (10 sec: 78643.2, 60 sec: 78779.9, 300 sec: 78643.2). Total num frames: 158068736. Throughput: 0: 19876.2. Samples: 29484332. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 01:09:30,466][1157520] Avg episode reward: [(0, '51.621')] [2024-09-30 01:09:30,800][1157819] Updated weights for policy 0, policy_version 38598 (0.0006) [2024-09-30 01:09:31,319][1157819] Updated weights for policy 0, policy_version 38608 (0.0006) [2024-09-30 01:09:31,807][1157819] Updated weights for policy 0, policy_version 38618 (0.0006) [2024-09-30 01:09:32,314][1157819] Updated weights for policy 0, policy_version 38628 (0.0006) [2024-09-30 01:09:32,836][1157819] Updated weights for policy 0, policy_version 38638 (0.0006) [2024-09-30 01:09:33,385][1157819] Updated weights for policy 0, policy_version 38648 (0.0006) [2024-09-30 01:09:33,881][1157819] Updated weights for policy 0, policy_version 38658 (0.0006) [2024-09-30 01:09:34,394][1157819] Updated weights for policy 0, policy_version 38668 (0.0006) [2024-09-30 01:09:34,907][1157819] Updated weights for policy 0, policy_version 38678 (0.0006) [2024-09-30 01:09:35,419][1157819] Updated weights for policy 0, policy_version 38688 (0.0006) [2024-09-30 01:09:35,466][1157520] Fps is (10 sec: 79462.1, 60 sec: 79052.8, 300 sec: 78657.1). Total num frames: 158466048. Throughput: 0: 19911.5. Samples: 29603676. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 01:09:35,466][1157520] Avg episode reward: [(0, '48.526')] [2024-09-30 01:09:35,942][1157819] Updated weights for policy 0, policy_version 38698 (0.0006) [2024-09-30 01:09:36,453][1157819] Updated weights for policy 0, policy_version 38708 (0.0006) [2024-09-30 01:09:37,005][1157819] Updated weights for policy 0, policy_version 38718 (0.0006) [2024-09-30 01:09:37,518][1157819] Updated weights for policy 0, policy_version 38728 (0.0006) [2024-09-30 01:09:38,044][1157819] Updated weights for policy 0, policy_version 38738 (0.0006) [2024-09-30 01:09:38,526][1157819] Updated weights for policy 0, policy_version 38748 (0.0006) [2024-09-30 01:09:39,049][1157819] Updated weights for policy 0, policy_version 38758 (0.0006) [2024-09-30 01:09:39,563][1157819] Updated weights for policy 0, policy_version 38768 (0.0006) [2024-09-30 01:09:40,072][1157819] Updated weights for policy 0, policy_version 38778 (0.0006) [2024-09-30 01:09:40,466][1157520] Fps is (10 sec: 79461.9, 60 sec: 79257.5, 300 sec: 78629.3). Total num frames: 158863360. Throughput: 0: 19896.9. Samples: 29662808. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 01:09:40,466][1157520] Avg episode reward: [(0, '49.242')] [2024-09-30 01:09:40,608][1157819] Updated weights for policy 0, policy_version 38788 (0.0006) [2024-09-30 01:09:41,084][1157819] Updated weights for policy 0, policy_version 38798 (0.0006) [2024-09-30 01:09:41,574][1157819] Updated weights for policy 0, policy_version 38808 (0.0006) [2024-09-30 01:09:42,080][1157819] Updated weights for policy 0, policy_version 38818 (0.0006) [2024-09-30 01:09:42,563][1157819] Updated weights for policy 0, policy_version 38828 (0.0006) [2024-09-30 01:09:43,053][1157819] Updated weights for policy 0, policy_version 38838 (0.0006) [2024-09-30 01:09:43,549][1157819] Updated weights for policy 0, policy_version 38848 (0.0006) [2024-09-30 01:09:44,040][1157819] Updated weights for policy 0, policy_version 38858 (0.0006) [2024-09-30 01:09:44,546][1157819] Updated weights for policy 0, policy_version 38868 (0.0006) [2024-09-30 01:09:45,016][1157819] Updated weights for policy 0, policy_version 38878 (0.0006) [2024-09-30 01:09:45,466][1157520] Fps is (10 sec: 81510.7, 60 sec: 79940.3, 300 sec: 78671.0). Total num frames: 159281152. Throughput: 0: 19989.2. Samples: 29785140. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 01:09:45,466][1157520] Avg episode reward: [(0, '48.310')] [2024-09-30 01:09:45,512][1157819] Updated weights for policy 0, policy_version 38888 (0.0006) [2024-09-30 01:09:46,008][1157819] Updated weights for policy 0, policy_version 38898 (0.0005) [2024-09-30 01:09:46,483][1157819] Updated weights for policy 0, policy_version 38908 (0.0006) [2024-09-30 01:09:46,979][1157819] Updated weights for policy 0, policy_version 38918 (0.0006) [2024-09-30 01:09:47,481][1157819] Updated weights for policy 0, policy_version 38928 (0.0006) [2024-09-30 01:09:47,969][1157819] Updated weights for policy 0, policy_version 38938 (0.0006) [2024-09-30 01:09:48,461][1157819] Updated weights for policy 0, policy_version 38948 (0.0006) [2024-09-30 01:09:48,950][1157819] Updated weights for policy 0, policy_version 38958 (0.0006) [2024-09-30 01:09:49,441][1157819] Updated weights for policy 0, policy_version 38968 (0.0006) [2024-09-30 01:09:49,950][1157819] Updated weights for policy 0, policy_version 38978 (0.0006) [2024-09-30 01:09:50,433][1157819] Updated weights for policy 0, policy_version 38988 (0.0006) [2024-09-30 01:09:50,466][1157520] Fps is (10 sec: 83149.1, 60 sec: 80213.3, 300 sec: 78712.7). Total num frames: 159694848. Throughput: 0: 20060.1. Samples: 29909924. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 01:09:50,466][1157520] Avg episode reward: [(0, '53.472')] [2024-09-30 01:09:50,931][1157819] Updated weights for policy 0, policy_version 38998 (0.0006) [2024-09-30 01:09:51,436][1157819] Updated weights for policy 0, policy_version 39008 (0.0006) [2024-09-30 01:09:51,911][1157819] Updated weights for policy 0, policy_version 39018 (0.0006) [2024-09-30 01:09:52,419][1157819] Updated weights for policy 0, policy_version 39028 (0.0006) [2024-09-30 01:09:52,907][1157819] Updated weights for policy 0, policy_version 39038 (0.0006) [2024-09-30 01:09:53,390][1157819] Updated weights for policy 0, policy_version 39048 (0.0006) [2024-09-30 01:09:53,909][1157819] Updated weights for policy 0, policy_version 39058 (0.0006) [2024-09-30 01:09:54,394][1157819] Updated weights for policy 0, policy_version 39068 (0.0006) [2024-09-30 01:09:54,875][1157819] Updated weights for policy 0, policy_version 39078 (0.0006) [2024-09-30 01:09:55,370][1157819] Updated weights for policy 0, policy_version 39088 (0.0006) [2024-09-30 01:09:55,466][1157520] Fps is (10 sec: 83148.8, 60 sec: 80554.6, 300 sec: 78726.5). Total num frames: 160112640. Throughput: 0: 20062.7. Samples: 29972396. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 01:09:55,466][1157520] Avg episode reward: [(0, '49.747')] [2024-09-30 01:09:55,829][1157819] Updated weights for policy 0, policy_version 39098 (0.0006) [2024-09-30 01:09:56,324][1157819] Updated weights for policy 0, policy_version 39108 (0.0006) [2024-09-30 01:09:56,788][1157819] Updated weights for policy 0, policy_version 39118 (0.0006) [2024-09-30 01:09:57,270][1157819] Updated weights for policy 0, policy_version 39128 (0.0006) [2024-09-30 01:09:57,762][1157819] Updated weights for policy 0, policy_version 39138 (0.0006) [2024-09-30 01:09:58,194][1157819] Updated weights for policy 0, policy_version 39148 (0.0006) [2024-09-30 01:09:58,676][1157819] Updated weights for policy 0, policy_version 39158 (0.0006) [2024-09-30 01:09:59,129][1157819] Updated weights for policy 0, policy_version 39168 (0.0006) [2024-09-30 01:09:59,610][1157819] Updated weights for policy 0, policy_version 39178 (0.0006) [2024-09-30 01:10:00,098][1157819] Updated weights for policy 0, policy_version 39188 (0.0006) [2024-09-30 01:10:00,466][1157520] Fps is (10 sec: 84786.8, 60 sec: 81169.0, 300 sec: 78754.3). Total num frames: 160542720. Throughput: 0: 20260.3. Samples: 30100068. Policy #0 lag: (min: 0.0, avg: 1.7, max: 4.0) [2024-09-30 01:10:00,466][1157520] Avg episode reward: [(0, '50.380')] [2024-09-30 01:10:00,557][1157819] Updated weights for policy 0, policy_version 39198 (0.0006) [2024-09-30 01:10:01,017][1157819] Updated weights for policy 0, policy_version 39208 (0.0006) [2024-09-30 01:10:01,489][1157819] Updated weights for policy 0, policy_version 39218 (0.0006) [2024-09-30 01:10:01,972][1157819] Updated weights for policy 0, policy_version 39228 (0.0006) [2024-09-30 01:10:02,445][1157819] Updated weights for policy 0, policy_version 39238 (0.0006) [2024-09-30 01:10:02,918][1157819] Updated weights for policy 0, policy_version 39248 (0.0005) [2024-09-30 01:10:03,379][1157819] Updated weights for policy 0, policy_version 39258 (0.0006) [2024-09-30 01:10:03,854][1157819] Updated weights for policy 0, policy_version 39268 (0.0006) [2024-09-30 01:10:04,317][1157819] Updated weights for policy 0, policy_version 39278 (0.0006) [2024-09-30 01:10:04,804][1157819] Updated weights for policy 0, policy_version 39288 (0.0006) [2024-09-30 01:10:05,304][1157819] Updated weights for policy 0, policy_version 39298 (0.0006) [2024-09-30 01:10:05,466][1157520] Fps is (10 sec: 86425.5, 60 sec: 81510.4, 300 sec: 78809.8). Total num frames: 160976896. Throughput: 0: 20516.3. Samples: 30230352. Policy #0 lag: (min: 0.0, avg: 1.7, max: 4.0) [2024-09-30 01:10:05,466][1157520] Avg episode reward: [(0, '53.140')] [2024-09-30 01:10:05,470][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000039301_160976896.pth... [2024-09-30 01:10:05,515][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000034680_142049280.pth [2024-09-30 01:10:05,801][1157819] Updated weights for policy 0, policy_version 39308 (0.0006) [2024-09-30 01:10:06,284][1157819] Updated weights for policy 0, policy_version 39318 (0.0006) [2024-09-30 01:10:06,787][1157819] Updated weights for policy 0, policy_version 39328 (0.0006) [2024-09-30 01:10:07,290][1157819] Updated weights for policy 0, policy_version 39338 (0.0006) [2024-09-30 01:10:07,771][1157819] Updated weights for policy 0, policy_version 39348 (0.0006) [2024-09-30 01:10:08,258][1157819] Updated weights for policy 0, policy_version 39358 (0.0006) [2024-09-30 01:10:08,782][1157819] Updated weights for policy 0, policy_version 39368 (0.0006) [2024-09-30 01:10:09,261][1157819] Updated weights for policy 0, policy_version 39378 (0.0006) [2024-09-30 01:10:09,719][1157819] Updated weights for policy 0, policy_version 39388 (0.0006) [2024-09-30 01:10:10,203][1157819] Updated weights for policy 0, policy_version 39398 (0.0006) [2024-09-30 01:10:10,466][1157520] Fps is (10 sec: 85197.5, 60 sec: 81715.2, 300 sec: 78907.0). Total num frames: 161394688. Throughput: 0: 20590.2. Samples: 30292644. Policy #0 lag: (min: 0.0, avg: 1.7, max: 4.0) [2024-09-30 01:10:10,466][1157520] Avg episode reward: [(0, '50.766')] [2024-09-30 01:10:10,665][1157819] Updated weights for policy 0, policy_version 39408 (0.0006) [2024-09-30 01:10:11,168][1157819] Updated weights for policy 0, policy_version 39418 (0.0006) [2024-09-30 01:10:11,635][1157819] Updated weights for policy 0, policy_version 39428 (0.0006) [2024-09-30 01:10:12,083][1157819] Updated weights for policy 0, policy_version 39438 (0.0006) [2024-09-30 01:10:12,574][1157819] Updated weights for policy 0, policy_version 39448 (0.0006) [2024-09-30 01:10:13,024][1157819] Updated weights for policy 0, policy_version 39458 (0.0006) [2024-09-30 01:10:13,524][1157819] Updated weights for policy 0, policy_version 39468 (0.0006) [2024-09-30 01:10:14,014][1157819] Updated weights for policy 0, policy_version 39478 (0.0006) [2024-09-30 01:10:14,503][1157819] Updated weights for policy 0, policy_version 39488 (0.0006) [2024-09-30 01:10:14,998][1157819] Updated weights for policy 0, policy_version 39498 (0.0006) [2024-09-30 01:10:15,437][1157819] Updated weights for policy 0, policy_version 39508 (0.0006) [2024-09-30 01:10:15,466][1157520] Fps is (10 sec: 84786.3, 60 sec: 82397.7, 300 sec: 79018.1). Total num frames: 161824768. Throughput: 0: 20810.1. Samples: 30420788. Policy #0 lag: (min: 0.0, avg: 1.7, max: 4.0) [2024-09-30 01:10:15,466][1157520] Avg episode reward: [(0, '53.519')] [2024-09-30 01:10:15,921][1157819] Updated weights for policy 0, policy_version 39518 (0.0006) [2024-09-30 01:10:16,421][1157819] Updated weights for policy 0, policy_version 39528 (0.0006) [2024-09-30 01:10:16,850][1157819] Updated weights for policy 0, policy_version 39538 (0.0006) [2024-09-30 01:10:17,309][1157819] Updated weights for policy 0, policy_version 39548 (0.0006) [2024-09-30 01:10:17,762][1157819] Updated weights for policy 0, policy_version 39558 (0.0006) [2024-09-30 01:10:18,219][1157819] Updated weights for policy 0, policy_version 39568 (0.0006) [2024-09-30 01:10:18,657][1157819] Updated weights for policy 0, policy_version 39578 (0.0006) [2024-09-30 01:10:19,076][1157736] Signal inference workers to stop experience collection... (2150 times) [2024-09-30 01:10:19,077][1157736] Signal inference workers to resume experience collection... (2150 times) [2024-09-30 01:10:19,082][1157819] InferenceWorker_p0-w0: stopping experience collection (2150 times) [2024-09-30 01:10:19,082][1157819] InferenceWorker_p0-w0: resuming experience collection (2150 times) [2024-09-30 01:10:19,090][1157819] Updated weights for policy 0, policy_version 39588 (0.0006) [2024-09-30 01:10:19,537][1157819] Updated weights for policy 0, policy_version 39598 (0.0006) [2024-09-30 01:10:19,964][1157819] Updated weights for policy 0, policy_version 39608 (0.0006) [2024-09-30 01:10:20,415][1157819] Updated weights for policy 0, policy_version 39618 (0.0005) [2024-09-30 01:10:20,466][1157520] Fps is (10 sec: 88064.1, 60 sec: 83217.1, 300 sec: 79184.7). Total num frames: 162275328. Throughput: 0: 21123.8. Samples: 30554244. Policy #0 lag: (min: 0.0, avg: 1.7, max: 4.0) [2024-09-30 01:10:20,466][1157520] Avg episode reward: [(0, '47.662')] [2024-09-30 01:10:20,839][1157819] Updated weights for policy 0, policy_version 39628 (0.0005) [2024-09-30 01:10:21,275][1157819] Updated weights for policy 0, policy_version 39638 (0.0006) [2024-09-30 01:10:21,756][1157819] Updated weights for policy 0, policy_version 39648 (0.0006) [2024-09-30 01:10:22,147][1157819] Updated weights for policy 0, policy_version 39658 (0.0005) [2024-09-30 01:10:22,629][1157819] Updated weights for policy 0, policy_version 39668 (0.0006) [2024-09-30 01:10:23,119][1157819] Updated weights for policy 0, policy_version 39678 (0.0006) [2024-09-30 01:10:23,601][1157819] Updated weights for policy 0, policy_version 39688 (0.0006) [2024-09-30 01:10:24,092][1157819] Updated weights for policy 0, policy_version 39698 (0.0006) [2024-09-30 01:10:24,523][1157819] Updated weights for policy 0, policy_version 39708 (0.0006) [2024-09-30 01:10:24,982][1157819] Updated weights for policy 0, policy_version 39718 (0.0006) [2024-09-30 01:10:25,410][1157819] Updated weights for policy 0, policy_version 39728 (0.0006) [2024-09-30 01:10:25,466][1157520] Fps is (10 sec: 90112.8, 60 sec: 84241.0, 300 sec: 79420.7). Total num frames: 162725888. Throughput: 0: 21344.8. Samples: 30623324. Policy #0 lag: (min: 0.0, avg: 1.8, max: 4.0) [2024-09-30 01:10:25,466][1157520] Avg episode reward: [(0, '51.816')] [2024-09-30 01:10:25,843][1157819] Updated weights for policy 0, policy_version 39738 (0.0006) [2024-09-30 01:10:26,309][1157819] Updated weights for policy 0, policy_version 39748 (0.0006) [2024-09-30 01:10:26,730][1157819] Updated weights for policy 0, policy_version 39758 (0.0006) [2024-09-30 01:10:27,190][1157819] Updated weights for policy 0, policy_version 39768 (0.0006) [2024-09-30 01:10:27,627][1157819] Updated weights for policy 0, policy_version 39778 (0.0006) [2024-09-30 01:10:28,066][1157819] Updated weights for policy 0, policy_version 39788 (0.0006) [2024-09-30 01:10:28,538][1157819] Updated weights for policy 0, policy_version 39798 (0.0006) [2024-09-30 01:10:28,981][1157819] Updated weights for policy 0, policy_version 39808 (0.0006) [2024-09-30 01:10:29,437][1157819] Updated weights for policy 0, policy_version 39818 (0.0006) [2024-09-30 01:10:29,890][1157819] Updated weights for policy 0, policy_version 39828 (0.0006) [2024-09-30 01:10:30,382][1157819] Updated weights for policy 0, policy_version 39838 (0.0006) [2024-09-30 01:10:30,466][1157520] Fps is (10 sec: 90930.0, 60 sec: 85264.9, 300 sec: 79670.6). Total num frames: 163184640. Throughput: 0: 21641.1. Samples: 30758992. Policy #0 lag: (min: 0.0, avg: 1.8, max: 4.0) [2024-09-30 01:10:30,466][1157520] Avg episode reward: [(0, '48.382')] [2024-09-30 01:10:30,880][1157819] Updated weights for policy 0, policy_version 39848 (0.0006) [2024-09-30 01:10:31,318][1157819] Updated weights for policy 0, policy_version 39858 (0.0006) [2024-09-30 01:10:31,875][1157819] Updated weights for policy 0, policy_version 39868 (0.0006) [2024-09-30 01:10:32,415][1157819] Updated weights for policy 0, policy_version 39878 (0.0006) [2024-09-30 01:10:32,920][1157819] Updated weights for policy 0, policy_version 39888 (0.0006) [2024-09-30 01:10:33,439][1157819] Updated weights for policy 0, policy_version 39898 (0.0006) [2024-09-30 01:10:33,942][1157819] Updated weights for policy 0, policy_version 39908 (0.0006) [2024-09-30 01:10:34,444][1157819] Updated weights for policy 0, policy_version 39918 (0.0006) [2024-09-30 01:10:34,945][1157819] Updated weights for policy 0, policy_version 39928 (0.0007) [2024-09-30 01:10:35,441][1157819] Updated weights for policy 0, policy_version 39938 (0.0006) [2024-09-30 01:10:35,466][1157520] Fps is (10 sec: 86016.6, 60 sec: 85333.5, 300 sec: 79698.4). Total num frames: 163586048. Throughput: 0: 21626.7. Samples: 30883124. Policy #0 lag: (min: 0.0, avg: 1.8, max: 4.0) [2024-09-30 01:10:35,466][1157520] Avg episode reward: [(0, '51.494')] [2024-09-30 01:10:35,925][1157819] Updated weights for policy 0, policy_version 39948 (0.0006) [2024-09-30 01:10:36,420][1157819] Updated weights for policy 0, policy_version 39958 (0.0006) [2024-09-30 01:10:36,912][1157819] Updated weights for policy 0, policy_version 39968 (0.0006) [2024-09-30 01:10:37,401][1157819] Updated weights for policy 0, policy_version 39978 (0.0006) [2024-09-30 01:10:37,900][1157819] Updated weights for policy 0, policy_version 39988 (0.0006) [2024-09-30 01:10:38,388][1157819] Updated weights for policy 0, policy_version 39998 (0.0006) [2024-09-30 01:10:38,863][1157819] Updated weights for policy 0, policy_version 40008 (0.0006) [2024-09-30 01:10:39,295][1157819] Updated weights for policy 0, policy_version 40018 (0.0006) [2024-09-30 01:10:39,753][1157819] Updated weights for policy 0, policy_version 40028 (0.0006) [2024-09-30 01:10:40,194][1157819] Updated weights for policy 0, policy_version 40038 (0.0005) [2024-09-30 01:10:40,466][1157520] Fps is (10 sec: 83559.3, 60 sec: 85947.8, 300 sec: 79823.4). Total num frames: 164020224. Throughput: 0: 21619.2. Samples: 30945260. Policy #0 lag: (min: 0.0, avg: 1.8, max: 4.0) [2024-09-30 01:10:40,466][1157520] Avg episode reward: [(0, '52.409')] [2024-09-30 01:10:40,619][1157819] Updated weights for policy 0, policy_version 40048 (0.0005) [2024-09-30 01:10:41,097][1157819] Updated weights for policy 0, policy_version 40058 (0.0006) [2024-09-30 01:10:41,585][1157819] Updated weights for policy 0, policy_version 40068 (0.0006) [2024-09-30 01:10:42,058][1157819] Updated weights for policy 0, policy_version 40078 (0.0006) [2024-09-30 01:10:42,510][1157819] Updated weights for policy 0, policy_version 40088 (0.0006) [2024-09-30 01:10:42,976][1157819] Updated weights for policy 0, policy_version 40098 (0.0005) [2024-09-30 01:10:43,402][1157819] Updated weights for policy 0, policy_version 40108 (0.0005) [2024-09-30 01:10:43,841][1157819] Updated weights for policy 0, policy_version 40118 (0.0006) [2024-09-30 01:10:44,278][1157819] Updated weights for policy 0, policy_version 40128 (0.0006) [2024-09-30 01:10:44,724][1157819] Updated weights for policy 0, policy_version 40138 (0.0006) [2024-09-30 01:10:45,156][1157819] Updated weights for policy 0, policy_version 40148 (0.0006) [2024-09-30 01:10:45,466][1157520] Fps is (10 sec: 88883.1, 60 sec: 86562.2, 300 sec: 80017.8). Total num frames: 164474880. Throughput: 0: 21763.6. Samples: 31079428. Policy #0 lag: (min: 0.0, avg: 1.8, max: 4.0) [2024-09-30 01:10:45,466][1157520] Avg episode reward: [(0, '49.533')] [2024-09-30 01:10:45,596][1157819] Updated weights for policy 0, policy_version 40158 (0.0005) [2024-09-30 01:10:46,035][1157819] Updated weights for policy 0, policy_version 40168 (0.0006) [2024-09-30 01:10:46,486][1157819] Updated weights for policy 0, policy_version 40178 (0.0006) [2024-09-30 01:10:46,926][1157819] Updated weights for policy 0, policy_version 40188 (0.0006) [2024-09-30 01:10:47,366][1157819] Updated weights for policy 0, policy_version 40198 (0.0006) [2024-09-30 01:10:47,836][1157819] Updated weights for policy 0, policy_version 40208 (0.0006) [2024-09-30 01:10:48,062][1157736] Signal inference workers to stop experience collection... (2200 times) [2024-09-30 01:10:48,063][1157736] Signal inference workers to resume experience collection... (2200 times) [2024-09-30 01:10:48,066][1157819] InferenceWorker_p0-w0: stopping experience collection (2200 times) [2024-09-30 01:10:48,069][1157819] InferenceWorker_p0-w0: resuming experience collection (2200 times) [2024-09-30 01:10:48,249][1157819] Updated weights for policy 0, policy_version 40218 (0.0006) [2024-09-30 01:10:48,712][1157819] Updated weights for policy 0, policy_version 40228 (0.0006) [2024-09-30 01:10:49,186][1157819] Updated weights for policy 0, policy_version 40238 (0.0006) [2024-09-30 01:10:49,610][1157819] Updated weights for policy 0, policy_version 40248 (0.0006) [2024-09-30 01:10:50,069][1157819] Updated weights for policy 0, policy_version 40258 (0.0006) [2024-09-30 01:10:50,466][1157520] Fps is (10 sec: 91340.5, 60 sec: 87313.0, 300 sec: 80240.0). Total num frames: 164933632. Throughput: 0: 21943.6. Samples: 31217816. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 01:10:50,466][1157520] Avg episode reward: [(0, '49.463')] [2024-09-30 01:10:50,534][1157819] Updated weights for policy 0, policy_version 40268 (0.0006) [2024-09-30 01:10:50,956][1157819] Updated weights for policy 0, policy_version 40278 (0.0006) [2024-09-30 01:10:51,440][1157819] Updated weights for policy 0, policy_version 40288 (0.0006) [2024-09-30 01:10:51,930][1157819] Updated weights for policy 0, policy_version 40298 (0.0006) [2024-09-30 01:10:52,381][1157819] Updated weights for policy 0, policy_version 40308 (0.0006) [2024-09-30 01:10:52,801][1157819] Updated weights for policy 0, policy_version 40318 (0.0006) [2024-09-30 01:10:53,279][1157819] Updated weights for policy 0, policy_version 40328 (0.0006) [2024-09-30 01:10:53,739][1157819] Updated weights for policy 0, policy_version 40338 (0.0006) [2024-09-30 01:10:54,203][1157819] Updated weights for policy 0, policy_version 40348 (0.0006) [2024-09-30 01:10:54,659][1157819] Updated weights for policy 0, policy_version 40358 (0.0006) [2024-09-30 01:10:55,127][1157819] Updated weights for policy 0, policy_version 40368 (0.0006) [2024-09-30 01:10:55,466][1157520] Fps is (10 sec: 90111.8, 60 sec: 87722.7, 300 sec: 80392.7). Total num frames: 165376000. Throughput: 0: 22052.3. Samples: 31285000. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 01:10:55,466][1157520] Avg episode reward: [(0, '47.531')] [2024-09-30 01:10:55,518][1157819] Updated weights for policy 0, policy_version 40378 (0.0006) [2024-09-30 01:10:56,000][1157819] Updated weights for policy 0, policy_version 40388 (0.0006) [2024-09-30 01:10:56,404][1157819] Updated weights for policy 0, policy_version 40398 (0.0005) [2024-09-30 01:10:56,885][1157819] Updated weights for policy 0, policy_version 40408 (0.0006) [2024-09-30 01:10:57,346][1157819] Updated weights for policy 0, policy_version 40418 (0.0006) [2024-09-30 01:10:57,765][1157819] Updated weights for policy 0, policy_version 40428 (0.0006) [2024-09-30 01:10:58,234][1157819] Updated weights for policy 0, policy_version 40438 (0.0006) [2024-09-30 01:10:58,673][1157819] Updated weights for policy 0, policy_version 40448 (0.0006) [2024-09-30 01:10:59,114][1157819] Updated weights for policy 0, policy_version 40458 (0.0006) [2024-09-30 01:10:59,629][1157819] Updated weights for policy 0, policy_version 40468 (0.0007) [2024-09-30 01:11:00,131][1157819] Updated weights for policy 0, policy_version 40478 (0.0006) [2024-09-30 01:11:00,466][1157520] Fps is (10 sec: 88882.0, 60 sec: 87995.6, 300 sec: 80600.9). Total num frames: 165822464. Throughput: 0: 22229.6. Samples: 31421120. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 01:11:00,466][1157520] Avg episode reward: [(0, '51.170')] [2024-09-30 01:11:00,654][1157819] Updated weights for policy 0, policy_version 40488 (0.0006) [2024-09-30 01:11:01,177][1157819] Updated weights for policy 0, policy_version 40498 (0.0006) [2024-09-30 01:11:01,694][1157819] Updated weights for policy 0, policy_version 40508 (0.0006) [2024-09-30 01:11:02,210][1157819] Updated weights for policy 0, policy_version 40518 (0.0006) [2024-09-30 01:11:02,723][1157819] Updated weights for policy 0, policy_version 40528 (0.0006) [2024-09-30 01:11:03,239][1157819] Updated weights for policy 0, policy_version 40538 (0.0006) [2024-09-30 01:11:03,739][1157819] Updated weights for policy 0, policy_version 40548 (0.0006) [2024-09-30 01:11:04,241][1157819] Updated weights for policy 0, policy_version 40558 (0.0006) [2024-09-30 01:11:04,750][1157819] Updated weights for policy 0, policy_version 40568 (0.0006) [2024-09-30 01:11:05,270][1157819] Updated weights for policy 0, policy_version 40578 (0.0006) [2024-09-30 01:11:05,466][1157520] Fps is (10 sec: 84376.3, 60 sec: 87381.1, 300 sec: 80628.7). Total num frames: 166219776. Throughput: 0: 21961.8. Samples: 31542528. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 01:11:05,466][1157520] Avg episode reward: [(0, '50.314')] [2024-09-30 01:11:05,835][1157819] Updated weights for policy 0, policy_version 40588 (0.0006) [2024-09-30 01:11:06,368][1157819] Updated weights for policy 0, policy_version 40598 (0.0006) [2024-09-30 01:11:06,871][1157819] Updated weights for policy 0, policy_version 40608 (0.0006) [2024-09-30 01:11:07,401][1157819] Updated weights for policy 0, policy_version 40618 (0.0006) [2024-09-30 01:11:07,924][1157819] Updated weights for policy 0, policy_version 40628 (0.0006) [2024-09-30 01:11:08,420][1157819] Updated weights for policy 0, policy_version 40638 (0.0006) [2024-09-30 01:11:08,928][1157819] Updated weights for policy 0, policy_version 40648 (0.0006) [2024-09-30 01:11:09,423][1157819] Updated weights for policy 0, policy_version 40658 (0.0006) [2024-09-30 01:11:09,943][1157819] Updated weights for policy 0, policy_version 40668 (0.0006) [2024-09-30 01:11:10,412][1157819] Updated weights for policy 0, policy_version 40678 (0.0006) [2024-09-30 01:11:10,466][1157520] Fps is (10 sec: 79873.4, 60 sec: 87108.3, 300 sec: 80614.8). Total num frames: 166621184. Throughput: 0: 21724.2. Samples: 31600912. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 01:11:10,466][1157520] Avg episode reward: [(0, '51.503')] [2024-09-30 01:11:10,888][1157819] Updated weights for policy 0, policy_version 40688 (0.0006) [2024-09-30 01:11:11,401][1157819] Updated weights for policy 0, policy_version 40698 (0.0006) [2024-09-30 01:11:11,460][1157736] Signal inference workers to stop experience collection... (2250 times) [2024-09-30 01:11:11,461][1157736] Signal inference workers to resume experience collection... (2250 times) [2024-09-30 01:11:11,465][1157819] InferenceWorker_p0-w0: stopping experience collection (2250 times) [2024-09-30 01:11:11,466][1157819] InferenceWorker_p0-w0: resuming experience collection (2250 times) [2024-09-30 01:11:11,868][1157819] Updated weights for policy 0, policy_version 40708 (0.0006) [2024-09-30 01:11:12,352][1157819] Updated weights for policy 0, policy_version 40718 (0.0006) [2024-09-30 01:11:12,841][1157819] Updated weights for policy 0, policy_version 40728 (0.0006) [2024-09-30 01:11:13,324][1157819] Updated weights for policy 0, policy_version 40738 (0.0006) [2024-09-30 01:11:13,845][1157819] Updated weights for policy 0, policy_version 40748 (0.0006) [2024-09-30 01:11:14,332][1157819] Updated weights for policy 0, policy_version 40758 (0.0006) [2024-09-30 01:11:14,838][1157819] Updated weights for policy 0, policy_version 40768 (0.0006) [2024-09-30 01:11:15,319][1157819] Updated weights for policy 0, policy_version 40778 (0.0006) [2024-09-30 01:11:15,466][1157520] Fps is (10 sec: 81511.7, 60 sec: 86835.4, 300 sec: 80642.6). Total num frames: 167034880. Throughput: 0: 21466.1. Samples: 31724964. Policy #0 lag: (min: 0.0, avg: 2.5, max: 4.0) [2024-09-30 01:11:15,466][1157520] Avg episode reward: [(0, '49.509')] [2024-09-30 01:11:15,811][1157819] Updated weights for policy 0, policy_version 40788 (0.0006) [2024-09-30 01:11:16,326][1157819] Updated weights for policy 0, policy_version 40798 (0.0006) [2024-09-30 01:11:16,960][1157819] Updated weights for policy 0, policy_version 40808 (0.0006) [2024-09-30 01:11:17,491][1157819] Updated weights for policy 0, policy_version 40818 (0.0006) [2024-09-30 01:11:18,065][1157819] Updated weights for policy 0, policy_version 40828 (0.0006) [2024-09-30 01:11:18,681][1157819] Updated weights for policy 0, policy_version 40838 (0.0006) [2024-09-30 01:11:19,238][1157819] Updated weights for policy 0, policy_version 40848 (0.0006) [2024-09-30 01:11:19,823][1157819] Updated weights for policy 0, policy_version 40858 (0.0006) [2024-09-30 01:11:20,420][1157819] Updated weights for policy 0, policy_version 40868 (0.0006) [2024-09-30 01:11:20,466][1157520] Fps is (10 sec: 77413.8, 60 sec: 85333.2, 300 sec: 80545.4). Total num frames: 167395328. Throughput: 0: 21226.3. Samples: 31838308. Policy #0 lag: (min: 0.0, avg: 2.5, max: 4.0) [2024-09-30 01:11:20,466][1157520] Avg episode reward: [(0, '48.416')] [2024-09-30 01:11:20,997][1157819] Updated weights for policy 0, policy_version 40878 (0.0006) [2024-09-30 01:11:21,599][1157819] Updated weights for policy 0, policy_version 40888 (0.0006) [2024-09-30 01:11:22,176][1157819] Updated weights for policy 0, policy_version 40898 (0.0006) [2024-09-30 01:11:22,826][1157819] Updated weights for policy 0, policy_version 40908 (0.0006) [2024-09-30 01:11:23,407][1157819] Updated weights for policy 0, policy_version 40918 (0.0006) [2024-09-30 01:11:24,040][1157819] Updated weights for policy 0, policy_version 40928 (0.0006) [2024-09-30 01:11:24,647][1157819] Updated weights for policy 0, policy_version 40938 (0.0006) [2024-09-30 01:11:25,216][1157819] Updated weights for policy 0, policy_version 40948 (0.0006) [2024-09-30 01:11:25,466][1157520] Fps is (10 sec: 70040.8, 60 sec: 83490.0, 300 sec: 80476.0). Total num frames: 167735296. Throughput: 0: 20983.3. Samples: 31889512. Policy #0 lag: (min: 0.0, avg: 2.5, max: 4.0) [2024-09-30 01:11:25,466][1157520] Avg episode reward: [(0, '52.864')] [2024-09-30 01:11:25,871][1157819] Updated weights for policy 0, policy_version 40958 (0.0006) [2024-09-30 01:11:26,408][1157819] Updated weights for policy 0, policy_version 40968 (0.0006) [2024-09-30 01:11:26,993][1157819] Updated weights for policy 0, policy_version 40978 (0.0006) [2024-09-30 01:11:27,500][1157819] Updated weights for policy 0, policy_version 40988 (0.0006) [2024-09-30 01:11:28,029][1157819] Updated weights for policy 0, policy_version 40998 (0.0006) [2024-09-30 01:11:28,568][1157819] Updated weights for policy 0, policy_version 41008 (0.0006) [2024-09-30 01:11:29,103][1157819] Updated weights for policy 0, policy_version 41018 (0.0006) [2024-09-30 01:11:29,647][1157819] Updated weights for policy 0, policy_version 41028 (0.0006) [2024-09-30 01:11:30,159][1157819] Updated weights for policy 0, policy_version 41038 (0.0006) [2024-09-30 01:11:30,466][1157520] Fps is (10 sec: 71680.8, 60 sec: 82125.0, 300 sec: 80559.3). Total num frames: 168112128. Throughput: 0: 20377.6. Samples: 31996420. Policy #0 lag: (min: 0.0, avg: 2.5, max: 4.0) [2024-09-30 01:11:30,466][1157520] Avg episode reward: [(0, '50.159')] [2024-09-30 01:11:30,676][1157819] Updated weights for policy 0, policy_version 41048 (0.0006) [2024-09-30 01:11:31,223][1157819] Updated weights for policy 0, policy_version 41058 (0.0006) [2024-09-30 01:11:31,776][1157819] Updated weights for policy 0, policy_version 41068 (0.0006) [2024-09-30 01:11:32,298][1157819] Updated weights for policy 0, policy_version 41078 (0.0006) [2024-09-30 01:11:32,813][1157819] Updated weights for policy 0, policy_version 41088 (0.0006) [2024-09-30 01:11:33,109][1157736] Signal inference workers to stop experience collection... (2300 times) [2024-09-30 01:11:33,109][1157736] Signal inference workers to resume experience collection... (2300 times) [2024-09-30 01:11:33,114][1157819] InferenceWorker_p0-w0: stopping experience collection (2300 times) [2024-09-30 01:11:33,114][1157819] InferenceWorker_p0-w0: resuming experience collection (2300 times) [2024-09-30 01:11:33,362][1157819] Updated weights for policy 0, policy_version 41098 (0.0006) [2024-09-30 01:11:33,871][1157819] Updated weights for policy 0, policy_version 41108 (0.0006) [2024-09-30 01:11:34,403][1157819] Updated weights for policy 0, policy_version 41118 (0.0006) [2024-09-30 01:11:34,930][1157819] Updated weights for policy 0, policy_version 41128 (0.0006) [2024-09-30 01:11:35,432][1157819] Updated weights for policy 0, policy_version 41138 (0.0006) [2024-09-30 01:11:35,466][1157520] Fps is (10 sec: 76596.3, 60 sec: 81920.0, 300 sec: 80656.5). Total num frames: 168501248. Throughput: 0: 19870.8. Samples: 32112000. Policy #0 lag: (min: 0.0, avg: 2.5, max: 4.0) [2024-09-30 01:11:35,466][1157520] Avg episode reward: [(0, '51.050')] [2024-09-30 01:11:35,945][1157819] Updated weights for policy 0, policy_version 41148 (0.0006) [2024-09-30 01:11:36,451][1157819] Updated weights for policy 0, policy_version 41158 (0.0006) [2024-09-30 01:11:36,963][1157819] Updated weights for policy 0, policy_version 41168 (0.0006) [2024-09-30 01:11:37,466][1157819] Updated weights for policy 0, policy_version 41178 (0.0006) [2024-09-30 01:11:37,967][1157819] Updated weights for policy 0, policy_version 41188 (0.0006) [2024-09-30 01:11:38,472][1157819] Updated weights for policy 0, policy_version 41198 (0.0006) [2024-09-30 01:11:38,968][1157819] Updated weights for policy 0, policy_version 41208 (0.0006) [2024-09-30 01:11:39,470][1157819] Updated weights for policy 0, policy_version 41218 (0.0006) [2024-09-30 01:11:40,004][1157819] Updated weights for policy 0, policy_version 41228 (0.0006) [2024-09-30 01:11:40,466][1157520] Fps is (10 sec: 79462.3, 60 sec: 81442.2, 300 sec: 80767.6). Total num frames: 168906752. Throughput: 0: 19731.9. Samples: 32172936. Policy #0 lag: (min: 0.0, avg: 2.5, max: 4.0) [2024-09-30 01:11:40,466][1157520] Avg episode reward: [(0, '52.265')] [2024-09-30 01:11:40,489][1157819] Updated weights for policy 0, policy_version 41238 (0.0006) [2024-09-30 01:11:40,983][1157819] Updated weights for policy 0, policy_version 41248 (0.0006) [2024-09-30 01:11:41,488][1157819] Updated weights for policy 0, policy_version 41258 (0.0006) [2024-09-30 01:11:41,990][1157819] Updated weights for policy 0, policy_version 41268 (0.0006) [2024-09-30 01:11:42,482][1157819] Updated weights for policy 0, policy_version 41278 (0.0006) [2024-09-30 01:11:43,006][1157819] Updated weights for policy 0, policy_version 41288 (0.0006) [2024-09-30 01:11:43,520][1157819] Updated weights for policy 0, policy_version 41298 (0.0006) [2024-09-30 01:11:44,070][1157819] Updated weights for policy 0, policy_version 41308 (0.0006) [2024-09-30 01:11:44,559][1157819] Updated weights for policy 0, policy_version 41318 (0.0006) [2024-09-30 01:11:45,059][1157819] Updated weights for policy 0, policy_version 41328 (0.0006) [2024-09-30 01:11:45,466][1157520] Fps is (10 sec: 81100.4, 60 sec: 80622.9, 300 sec: 80864.7). Total num frames: 169312256. Throughput: 0: 19411.6. Samples: 32294640. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 01:11:45,466][1157520] Avg episode reward: [(0, '49.925')] [2024-09-30 01:11:45,569][1157819] Updated weights for policy 0, policy_version 41338 (0.0006) [2024-09-30 01:11:46,081][1157819] Updated weights for policy 0, policy_version 41348 (0.0006) [2024-09-30 01:11:46,590][1157819] Updated weights for policy 0, policy_version 41358 (0.0006) [2024-09-30 01:11:47,096][1157819] Updated weights for policy 0, policy_version 41368 (0.0006) [2024-09-30 01:11:47,597][1157819] Updated weights for policy 0, policy_version 41378 (0.0006) [2024-09-30 01:11:48,104][1157819] Updated weights for policy 0, policy_version 41388 (0.0006) [2024-09-30 01:11:48,595][1157819] Updated weights for policy 0, policy_version 41398 (0.0006) [2024-09-30 01:11:49,112][1157819] Updated weights for policy 0, policy_version 41408 (0.0006) [2024-09-30 01:11:49,618][1157819] Updated weights for policy 0, policy_version 41418 (0.0006) [2024-09-30 01:11:50,102][1157819] Updated weights for policy 0, policy_version 41428 (0.0006) [2024-09-30 01:11:50,466][1157520] Fps is (10 sec: 81100.8, 60 sec: 79735.5, 300 sec: 80989.7). Total num frames: 169717760. Throughput: 0: 19406.0. Samples: 32415796. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 01:11:50,466][1157520] Avg episode reward: [(0, '51.066')] [2024-09-30 01:11:50,601][1157819] Updated weights for policy 0, policy_version 41438 (0.0006) [2024-09-30 01:11:51,120][1157819] Updated weights for policy 0, policy_version 41448 (0.0006) [2024-09-30 01:11:51,620][1157819] Updated weights for policy 0, policy_version 41458 (0.0006) [2024-09-30 01:11:52,122][1157819] Updated weights for policy 0, policy_version 41468 (0.0006) [2024-09-30 01:11:52,626][1157819] Updated weights for policy 0, policy_version 41478 (0.0006) [2024-09-30 01:11:53,130][1157819] Updated weights for policy 0, policy_version 41488 (0.0006) [2024-09-30 01:11:53,637][1157819] Updated weights for policy 0, policy_version 41498 (0.0006) [2024-09-30 01:11:54,128][1157819] Updated weights for policy 0, policy_version 41508 (0.0006) [2024-09-30 01:11:54,635][1157819] Updated weights for policy 0, policy_version 41518 (0.0006) [2024-09-30 01:11:55,163][1157819] Updated weights for policy 0, policy_version 41528 (0.0006) [2024-09-30 01:11:55,466][1157520] Fps is (10 sec: 81101.3, 60 sec: 79121.1, 300 sec: 81073.0). Total num frames: 170123264. Throughput: 0: 19467.9. Samples: 32476968. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 01:11:55,466][1157520] Avg episode reward: [(0, '49.629')] [2024-09-30 01:11:55,661][1157819] Updated weights for policy 0, policy_version 41538 (0.0006) [2024-09-30 01:11:56,164][1157819] Updated weights for policy 0, policy_version 41548 (0.0006) [2024-09-30 01:11:56,672][1157819] Updated weights for policy 0, policy_version 41558 (0.0006) [2024-09-30 01:11:57,208][1157819] Updated weights for policy 0, policy_version 41568 (0.0006) [2024-09-30 01:11:57,744][1157819] Updated weights for policy 0, policy_version 41578 (0.0006) [2024-09-30 01:11:58,275][1157819] Updated weights for policy 0, policy_version 41588 (0.0006) [2024-09-30 01:11:58,794][1157819] Updated weights for policy 0, policy_version 41598 (0.0006) [2024-09-30 01:11:59,305][1157819] Updated weights for policy 0, policy_version 41608 (0.0006) [2024-09-30 01:11:59,813][1157819] Updated weights for policy 0, policy_version 41618 (0.0006) [2024-09-30 01:12:00,349][1157819] Updated weights for policy 0, policy_version 41628 (0.0006) [2024-09-30 01:12:00,466][1157520] Fps is (10 sec: 79871.8, 60 sec: 78233.8, 300 sec: 81142.5). Total num frames: 170516480. Throughput: 0: 19365.8. Samples: 32596424. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 01:12:00,466][1157520] Avg episode reward: [(0, '52.515')] [2024-09-30 01:12:00,871][1157819] Updated weights for policy 0, policy_version 41638 (0.0006) [2024-09-30 01:12:01,379][1157819] Updated weights for policy 0, policy_version 41648 (0.0006) [2024-09-30 01:12:01,918][1157819] Updated weights for policy 0, policy_version 41658 (0.0006) [2024-09-30 01:12:02,429][1157819] Updated weights for policy 0, policy_version 41668 (0.0006) [2024-09-30 01:12:02,923][1157819] Updated weights for policy 0, policy_version 41678 (0.0006) [2024-09-30 01:12:03,440][1157819] Updated weights for policy 0, policy_version 41688 (0.0006) [2024-09-30 01:12:03,953][1157819] Updated weights for policy 0, policy_version 41698 (0.0006) [2024-09-30 01:12:04,469][1157819] Updated weights for policy 0, policy_version 41708 (0.0006) [2024-09-30 01:12:04,994][1157819] Updated weights for policy 0, policy_version 41718 (0.0006) [2024-09-30 01:12:05,466][1157520] Fps is (10 sec: 79052.4, 60 sec: 78233.8, 300 sec: 81142.4). Total num frames: 170913792. Throughput: 0: 19492.0. Samples: 32715448. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 01:12:05,466][1157520] Avg episode reward: [(0, '51.043')] [2024-09-30 01:12:05,480][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000041728_170917888.pth... [2024-09-30 01:12:05,480][1157819] Updated weights for policy 0, policy_version 41728 (0.0006) [2024-09-30 01:12:05,521][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000036940_151306240.pth [2024-09-30 01:12:06,015][1157819] Updated weights for policy 0, policy_version 41738 (0.0006) [2024-09-30 01:12:06,520][1157819] Updated weights for policy 0, policy_version 41748 (0.0006) [2024-09-30 01:12:07,012][1157819] Updated weights for policy 0, policy_version 41758 (0.0006) [2024-09-30 01:12:07,508][1157819] Updated weights for policy 0, policy_version 41768 (0.0006) [2024-09-30 01:12:08,045][1157819] Updated weights for policy 0, policy_version 41778 (0.0006) [2024-09-30 01:12:08,558][1157819] Updated weights for policy 0, policy_version 41788 (0.0006) [2024-09-30 01:12:09,068][1157819] Updated weights for policy 0, policy_version 41798 (0.0006) [2024-09-30 01:12:09,570][1157819] Updated weights for policy 0, policy_version 41808 (0.0006) [2024-09-30 01:12:10,085][1157819] Updated weights for policy 0, policy_version 41818 (0.0006) [2024-09-30 01:12:10,466][1157520] Fps is (10 sec: 79871.8, 60 sec: 78233.6, 300 sec: 81170.2). Total num frames: 171315200. Throughput: 0: 19691.0. Samples: 32775604. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 01:12:10,466][1157520] Avg episode reward: [(0, '50.087')] [2024-09-30 01:12:10,626][1157819] Updated weights for policy 0, policy_version 41828 (0.0006) [2024-09-30 01:12:11,167][1157819] Updated weights for policy 0, policy_version 41838 (0.0006) [2024-09-30 01:12:11,682][1157819] Updated weights for policy 0, policy_version 41848 (0.0006) [2024-09-30 01:12:12,179][1157819] Updated weights for policy 0, policy_version 41858 (0.0006) [2024-09-30 01:12:12,698][1157819] Updated weights for policy 0, policy_version 41868 (0.0006) [2024-09-30 01:12:13,216][1157819] Updated weights for policy 0, policy_version 41878 (0.0006) [2024-09-30 01:12:13,717][1157819] Updated weights for policy 0, policy_version 41888 (0.0006) [2024-09-30 01:12:14,246][1157819] Updated weights for policy 0, policy_version 41898 (0.0006) [2024-09-30 01:12:14,776][1157819] Updated weights for policy 0, policy_version 41908 (0.0006) [2024-09-30 01:12:15,275][1157819] Updated weights for policy 0, policy_version 41918 (0.0006) [2024-09-30 01:12:15,466][1157520] Fps is (10 sec: 79462.6, 60 sec: 77892.3, 300 sec: 81198.0). Total num frames: 171708416. Throughput: 0: 19957.0. Samples: 32894484. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 01:12:15,466][1157520] Avg episode reward: [(0, '51.565')] [2024-09-30 01:12:15,773][1157819] Updated weights for policy 0, policy_version 41928 (0.0006) [2024-09-30 01:12:16,301][1157819] Updated weights for policy 0, policy_version 41938 (0.0006) [2024-09-30 01:12:16,837][1157819] Updated weights for policy 0, policy_version 41948 (0.0007) [2024-09-30 01:12:17,355][1157819] Updated weights for policy 0, policy_version 41958 (0.0006) [2024-09-30 01:12:17,867][1157819] Updated weights for policy 0, policy_version 41968 (0.0007) [2024-09-30 01:12:18,389][1157819] Updated weights for policy 0, policy_version 41978 (0.0007) [2024-09-30 01:12:18,940][1157819] Updated weights for policy 0, policy_version 41988 (0.0007) [2024-09-30 01:12:19,460][1157819] Updated weights for policy 0, policy_version 41998 (0.0006) [2024-09-30 01:12:19,993][1157819] Updated weights for policy 0, policy_version 42008 (0.0006) [2024-09-30 01:12:20,466][1157520] Fps is (10 sec: 78643.6, 60 sec: 78438.5, 300 sec: 81211.9). Total num frames: 172101632. Throughput: 0: 20005.2. Samples: 33012232. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 01:12:20,466][1157520] Avg episode reward: [(0, '51.479')] [2024-09-30 01:12:20,528][1157819] Updated weights for policy 0, policy_version 42018 (0.0006) [2024-09-30 01:12:21,039][1157819] Updated weights for policy 0, policy_version 42028 (0.0006) [2024-09-30 01:12:21,558][1157819] Updated weights for policy 0, policy_version 42038 (0.0006) [2024-09-30 01:12:22,064][1157819] Updated weights for policy 0, policy_version 42048 (0.0006) [2024-09-30 01:12:22,619][1157819] Updated weights for policy 0, policy_version 42058 (0.0006) [2024-09-30 01:12:23,125][1157819] Updated weights for policy 0, policy_version 42068 (0.0006) [2024-09-30 01:12:23,675][1157819] Updated weights for policy 0, policy_version 42078 (0.0006) [2024-09-30 01:12:24,197][1157819] Updated weights for policy 0, policy_version 42088 (0.0006) [2024-09-30 01:12:24,712][1157819] Updated weights for policy 0, policy_version 42098 (0.0006) [2024-09-30 01:12:25,241][1157819] Updated weights for policy 0, policy_version 42108 (0.0006) [2024-09-30 01:12:25,466][1157520] Fps is (10 sec: 77824.1, 60 sec: 79189.5, 300 sec: 81198.0). Total num frames: 172486656. Throughput: 0: 19951.1. Samples: 33070736. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 01:12:25,466][1157520] Avg episode reward: [(0, '50.983')] [2024-09-30 01:12:25,771][1157819] Updated weights for policy 0, policy_version 42118 (0.0006) [2024-09-30 01:12:26,337][1157819] Updated weights for policy 0, policy_version 42128 (0.0006) [2024-09-30 01:12:26,836][1157819] Updated weights for policy 0, policy_version 42138 (0.0006) [2024-09-30 01:12:27,341][1157819] Updated weights for policy 0, policy_version 42148 (0.0006) [2024-09-30 01:12:27,862][1157819] Updated weights for policy 0, policy_version 42158 (0.0006) [2024-09-30 01:12:28,379][1157819] Updated weights for policy 0, policy_version 42168 (0.0006) [2024-09-30 01:12:28,921][1157819] Updated weights for policy 0, policy_version 42178 (0.0006) [2024-09-30 01:12:29,439][1157819] Updated weights for policy 0, policy_version 42188 (0.0006) [2024-09-30 01:12:29,964][1157819] Updated weights for policy 0, policy_version 42198 (0.0006) [2024-09-30 01:12:30,466][1157520] Fps is (10 sec: 77823.9, 60 sec: 79462.4, 300 sec: 81184.1). Total num frames: 172879872. Throughput: 0: 19849.9. Samples: 33187884. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 01:12:30,466][1157520] Avg episode reward: [(0, '49.432')] [2024-09-30 01:12:30,495][1157819] Updated weights for policy 0, policy_version 42208 (0.0006) [2024-09-30 01:12:30,922][1157736] Signal inference workers to stop experience collection... (2350 times) [2024-09-30 01:12:30,925][1157736] Signal inference workers to resume experience collection... (2350 times) [2024-09-30 01:12:30,929][1157819] InferenceWorker_p0-w0: stopping experience collection (2350 times) [2024-09-30 01:12:30,931][1157819] InferenceWorker_p0-w0: resuming experience collection (2350 times) [2024-09-30 01:12:31,012][1157819] Updated weights for policy 0, policy_version 42218 (0.0006) [2024-09-30 01:12:31,597][1157819] Updated weights for policy 0, policy_version 42228 (0.0006) [2024-09-30 01:12:32,128][1157819] Updated weights for policy 0, policy_version 42238 (0.0006) [2024-09-30 01:12:32,664][1157819] Updated weights for policy 0, policy_version 42248 (0.0006) [2024-09-30 01:12:33,227][1157819] Updated weights for policy 0, policy_version 42258 (0.0006) [2024-09-30 01:12:33,796][1157819] Updated weights for policy 0, policy_version 42268 (0.0006) [2024-09-30 01:12:34,312][1157819] Updated weights for policy 0, policy_version 42278 (0.0006) [2024-09-30 01:12:34,840][1157819] Updated weights for policy 0, policy_version 42288 (0.0006) [2024-09-30 01:12:35,409][1157819] Updated weights for policy 0, policy_version 42298 (0.0006) [2024-09-30 01:12:35,466][1157520] Fps is (10 sec: 77004.4, 60 sec: 79257.5, 300 sec: 81114.7). Total num frames: 173256704. Throughput: 0: 19684.0. Samples: 33301576. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:12:35,466][1157520] Avg episode reward: [(0, '50.806')] [2024-09-30 01:12:35,958][1157819] Updated weights for policy 0, policy_version 42308 (0.0006) [2024-09-30 01:12:36,501][1157819] Updated weights for policy 0, policy_version 42318 (0.0006) [2024-09-30 01:12:37,009][1157819] Updated weights for policy 0, policy_version 42328 (0.0006) [2024-09-30 01:12:37,528][1157819] Updated weights for policy 0, policy_version 42338 (0.0006) [2024-09-30 01:12:38,074][1157819] Updated weights for policy 0, policy_version 42348 (0.0006) [2024-09-30 01:12:38,580][1157819] Updated weights for policy 0, policy_version 42358 (0.0006) [2024-09-30 01:12:39,078][1157819] Updated weights for policy 0, policy_version 42368 (0.0006) [2024-09-30 01:12:39,583][1157819] Updated weights for policy 0, policy_version 42378 (0.0006) [2024-09-30 01:12:40,106][1157819] Updated weights for policy 0, policy_version 42388 (0.0006) [2024-09-30 01:12:40,466][1157520] Fps is (10 sec: 76595.1, 60 sec: 78984.5, 300 sec: 81156.3). Total num frames: 173645824. Throughput: 0: 19603.0. Samples: 33359104. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:12:40,466][1157520] Avg episode reward: [(0, '51.636')] [2024-09-30 01:12:40,632][1157819] Updated weights for policy 0, policy_version 42398 (0.0006) [2024-09-30 01:12:41,165][1157819] Updated weights for policy 0, policy_version 42408 (0.0007) [2024-09-30 01:12:41,683][1157819] Updated weights for policy 0, policy_version 42418 (0.0006) [2024-09-30 01:12:42,199][1157819] Updated weights for policy 0, policy_version 42428 (0.0006) [2024-09-30 01:12:42,740][1157819] Updated weights for policy 0, policy_version 42438 (0.0006) [2024-09-30 01:12:43,254][1157819] Updated weights for policy 0, policy_version 42448 (0.0006) [2024-09-30 01:12:43,760][1157819] Updated weights for policy 0, policy_version 42458 (0.0006) [2024-09-30 01:12:44,254][1157819] Updated weights for policy 0, policy_version 42468 (0.0006) [2024-09-30 01:12:44,764][1157819] Updated weights for policy 0, policy_version 42478 (0.0006) [2024-09-30 01:12:45,274][1157819] Updated weights for policy 0, policy_version 42488 (0.0006) [2024-09-30 01:12:45,466][1157520] Fps is (10 sec: 78643.6, 60 sec: 78848.1, 300 sec: 81156.3). Total num frames: 174043136. Throughput: 0: 19589.6. Samples: 33477956. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:12:45,466][1157520] Avg episode reward: [(0, '51.869')] [2024-09-30 01:12:45,778][1157819] Updated weights for policy 0, policy_version 42498 (0.0006) [2024-09-30 01:12:46,279][1157819] Updated weights for policy 0, policy_version 42508 (0.0006) [2024-09-30 01:12:46,823][1157819] Updated weights for policy 0, policy_version 42518 (0.0006) [2024-09-30 01:12:47,407][1157819] Updated weights for policy 0, policy_version 42528 (0.0006) [2024-09-30 01:12:47,918][1157819] Updated weights for policy 0, policy_version 42538 (0.0006) [2024-09-30 01:12:48,467][1157819] Updated weights for policy 0, policy_version 42548 (0.0006) [2024-09-30 01:12:49,029][1157819] Updated weights for policy 0, policy_version 42558 (0.0006) [2024-09-30 01:12:49,552][1157819] Updated weights for policy 0, policy_version 42568 (0.0006) [2024-09-30 01:12:50,121][1157819] Updated weights for policy 0, policy_version 42578 (0.0006) [2024-09-30 01:12:50,466][1157520] Fps is (10 sec: 77824.3, 60 sec: 78438.4, 300 sec: 81086.9). Total num frames: 174424064. Throughput: 0: 19515.9. Samples: 33593664. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:12:50,466][1157520] Avg episode reward: [(0, '50.494')] [2024-09-30 01:12:50,690][1157819] Updated weights for policy 0, policy_version 42588 (0.0006) [2024-09-30 01:12:51,223][1157819] Updated weights for policy 0, policy_version 42598 (0.0006) [2024-09-30 01:12:51,784][1157819] Updated weights for policy 0, policy_version 42608 (0.0007) [2024-09-30 01:12:52,309][1157819] Updated weights for policy 0, policy_version 42618 (0.0006) [2024-09-30 01:12:52,795][1157819] Updated weights for policy 0, policy_version 42628 (0.0006) [2024-09-30 01:12:53,306][1157819] Updated weights for policy 0, policy_version 42638 (0.0006) [2024-09-30 01:12:53,852][1157819] Updated weights for policy 0, policy_version 42648 (0.0006) [2024-09-30 01:12:54,354][1157819] Updated weights for policy 0, policy_version 42658 (0.0006) [2024-09-30 01:12:54,881][1157819] Updated weights for policy 0, policy_version 42668 (0.0006) [2024-09-30 01:12:55,369][1157819] Updated weights for policy 0, policy_version 42678 (0.0006) [2024-09-30 01:12:55,466][1157520] Fps is (10 sec: 77414.4, 60 sec: 78233.6, 300 sec: 81059.1). Total num frames: 174817280. Throughput: 0: 19441.8. Samples: 33650484. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:12:55,466][1157520] Avg episode reward: [(0, '50.712')] [2024-09-30 01:12:55,883][1157819] Updated weights for policy 0, policy_version 42688 (0.0006) [2024-09-30 01:12:56,384][1157819] Updated weights for policy 0, policy_version 42698 (0.0006) [2024-09-30 01:12:56,881][1157819] Updated weights for policy 0, policy_version 42708 (0.0006) [2024-09-30 01:12:57,381][1157819] Updated weights for policy 0, policy_version 42718 (0.0006) [2024-09-30 01:12:57,623][1157736] Signal inference workers to stop experience collection... (2400 times) [2024-09-30 01:12:57,624][1157736] Signal inference workers to resume experience collection... (2400 times) [2024-09-30 01:12:57,629][1157819] InferenceWorker_p0-w0: stopping experience collection (2400 times) [2024-09-30 01:12:57,630][1157819] InferenceWorker_p0-w0: resuming experience collection (2400 times) [2024-09-30 01:12:57,899][1157819] Updated weights for policy 0, policy_version 42728 (0.0006) [2024-09-30 01:12:58,404][1157819] Updated weights for policy 0, policy_version 42738 (0.0006) [2024-09-30 01:12:58,836][1157819] Updated weights for policy 0, policy_version 42748 (0.0006) [2024-09-30 01:12:59,305][1157819] Updated weights for policy 0, policy_version 42758 (0.0006) [2024-09-30 01:12:59,769][1157819] Updated weights for policy 0, policy_version 42768 (0.0006) [2024-09-30 01:13:00,244][1157819] Updated weights for policy 0, policy_version 42778 (0.0006) [2024-09-30 01:13:00,466][1157520] Fps is (10 sec: 81510.2, 60 sec: 78711.5, 300 sec: 81128.6). Total num frames: 175239168. Throughput: 0: 19509.9. Samples: 33772428. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 01:13:00,466][1157520] Avg episode reward: [(0, '49.184')] [2024-09-30 01:13:00,670][1157819] Updated weights for policy 0, policy_version 42788 (0.0006) [2024-09-30 01:13:01,135][1157819] Updated weights for policy 0, policy_version 42798 (0.0006) [2024-09-30 01:13:01,593][1157819] Updated weights for policy 0, policy_version 42808 (0.0006) [2024-09-30 01:13:02,036][1157819] Updated weights for policy 0, policy_version 42818 (0.0006) [2024-09-30 01:13:02,488][1157819] Updated weights for policy 0, policy_version 42828 (0.0006) [2024-09-30 01:13:02,966][1157819] Updated weights for policy 0, policy_version 42838 (0.0006) [2024-09-30 01:13:03,400][1157819] Updated weights for policy 0, policy_version 42848 (0.0006) [2024-09-30 01:13:03,852][1157819] Updated weights for policy 0, policy_version 42858 (0.0006) [2024-09-30 01:13:04,340][1157819] Updated weights for policy 0, policy_version 42868 (0.0006) [2024-09-30 01:13:04,769][1157819] Updated weights for policy 0, policy_version 42878 (0.0006) [2024-09-30 01:13:05,223][1157819] Updated weights for policy 0, policy_version 42888 (0.0006) [2024-09-30 01:13:05,466][1157520] Fps is (10 sec: 87244.5, 60 sec: 79598.9, 300 sec: 81281.3). Total num frames: 175689728. Throughput: 0: 19898.7. Samples: 33907676. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 01:13:05,466][1157520] Avg episode reward: [(0, '51.128')] [2024-09-30 01:13:05,666][1157819] Updated weights for policy 0, policy_version 42898 (0.0006) [2024-09-30 01:13:06,096][1157819] Updated weights for policy 0, policy_version 42908 (0.0006) [2024-09-30 01:13:06,579][1157819] Updated weights for policy 0, policy_version 42918 (0.0006) [2024-09-30 01:13:07,045][1157819] Updated weights for policy 0, policy_version 42928 (0.0006) [2024-09-30 01:13:07,537][1157819] Updated weights for policy 0, policy_version 42938 (0.0006) [2024-09-30 01:13:08,034][1157819] Updated weights for policy 0, policy_version 42948 (0.0006) [2024-09-30 01:13:08,528][1157819] Updated weights for policy 0, policy_version 42958 (0.0006) [2024-09-30 01:13:08,983][1157819] Updated weights for policy 0, policy_version 42968 (0.0006) [2024-09-30 01:13:09,440][1157819] Updated weights for policy 0, policy_version 42978 (0.0006) [2024-09-30 01:13:09,929][1157819] Updated weights for policy 0, policy_version 42988 (0.0006) [2024-09-30 01:13:10,376][1157819] Updated weights for policy 0, policy_version 42998 (0.0006) [2024-09-30 01:13:10,466][1157520] Fps is (10 sec: 88473.4, 60 sec: 80145.1, 300 sec: 81392.4). Total num frames: 176123904. Throughput: 0: 20064.4. Samples: 33973636. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 01:13:10,466][1157520] Avg episode reward: [(0, '51.251')] [2024-09-30 01:13:10,872][1157819] Updated weights for policy 0, policy_version 43008 (0.0006) [2024-09-30 01:13:11,334][1157819] Updated weights for policy 0, policy_version 43018 (0.0006) [2024-09-30 01:13:11,843][1157819] Updated weights for policy 0, policy_version 43028 (0.0006) [2024-09-30 01:13:12,299][1157819] Updated weights for policy 0, policy_version 43038 (0.0006) [2024-09-30 01:13:12,772][1157819] Updated weights for policy 0, policy_version 43048 (0.0006) [2024-09-30 01:13:13,261][1157819] Updated weights for policy 0, policy_version 43058 (0.0006) [2024-09-30 01:13:13,758][1157819] Updated weights for policy 0, policy_version 43068 (0.0006) [2024-09-30 01:13:13,760][1157736] Signal inference workers to stop experience collection... (2450 times) [2024-09-30 01:13:13,761][1157736] Signal inference workers to resume experience collection... (2450 times) [2024-09-30 01:13:13,765][1157819] InferenceWorker_p0-w0: stopping experience collection (2450 times) [2024-09-30 01:13:13,765][1157819] InferenceWorker_p0-w0: resuming experience collection (2450 times) [2024-09-30 01:13:14,269][1157819] Updated weights for policy 0, policy_version 43078 (0.0006) [2024-09-30 01:13:14,682][1157819] Updated weights for policy 0, policy_version 43088 (0.0006) [2024-09-30 01:13:15,171][1157819] Updated weights for policy 0, policy_version 43098 (0.0006) [2024-09-30 01:13:15,466][1157520] Fps is (10 sec: 86016.3, 60 sec: 80691.2, 300 sec: 81475.7). Total num frames: 176549888. Throughput: 0: 20324.7. Samples: 34102496. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 01:13:15,466][1157520] Avg episode reward: [(0, '51.962')] [2024-09-30 01:13:15,652][1157819] Updated weights for policy 0, policy_version 43108 (0.0006) [2024-09-30 01:13:16,140][1157819] Updated weights for policy 0, policy_version 43118 (0.0006) [2024-09-30 01:13:16,629][1157819] Updated weights for policy 0, policy_version 43128 (0.0006) [2024-09-30 01:13:17,113][1157819] Updated weights for policy 0, policy_version 43138 (0.0006) [2024-09-30 01:13:17,576][1157819] Updated weights for policy 0, policy_version 43148 (0.0006) [2024-09-30 01:13:18,057][1157819] Updated weights for policy 0, policy_version 43158 (0.0006) [2024-09-30 01:13:18,536][1157819] Updated weights for policy 0, policy_version 43168 (0.0006) [2024-09-30 01:13:19,026][1157819] Updated weights for policy 0, policy_version 43178 (0.0006) [2024-09-30 01:13:19,527][1157819] Updated weights for policy 0, policy_version 43188 (0.0006) [2024-09-30 01:13:20,041][1157819] Updated weights for policy 0, policy_version 43198 (0.0006) [2024-09-30 01:13:20,466][1157520] Fps is (10 sec: 85196.9, 60 sec: 81237.3, 300 sec: 81503.5). Total num frames: 176975872. Throughput: 0: 20631.7. Samples: 34230000. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 01:13:20,466][1157520] Avg episode reward: [(0, '49.941')] [2024-09-30 01:13:20,505][1157819] Updated weights for policy 0, policy_version 43208 (0.0006) [2024-09-30 01:13:21,008][1157819] Updated weights for policy 0, policy_version 43218 (0.0007) [2024-09-30 01:13:21,487][1157819] Updated weights for policy 0, policy_version 43228 (0.0007) [2024-09-30 01:13:21,986][1157819] Updated weights for policy 0, policy_version 43238 (0.0007) [2024-09-30 01:13:22,501][1157819] Updated weights for policy 0, policy_version 43248 (0.0006) [2024-09-30 01:13:23,020][1157819] Updated weights for policy 0, policy_version 43258 (0.0006) [2024-09-30 01:13:23,512][1157819] Updated weights for policy 0, policy_version 43268 (0.0007) [2024-09-30 01:13:24,026][1157819] Updated weights for policy 0, policy_version 43278 (0.0006) [2024-09-30 01:13:24,545][1157819] Updated weights for policy 0, policy_version 43288 (0.0006) [2024-09-30 01:13:25,016][1157819] Updated weights for policy 0, policy_version 43298 (0.0007) [2024-09-30 01:13:25,466][1157520] Fps is (10 sec: 82739.2, 60 sec: 81510.4, 300 sec: 81475.7). Total num frames: 177377280. Throughput: 0: 20723.3. Samples: 34291652. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 01:13:25,466][1157520] Avg episode reward: [(0, '50.781')] [2024-09-30 01:13:25,535][1157819] Updated weights for policy 0, policy_version 43308 (0.0007) [2024-09-30 01:13:26,089][1157819] Updated weights for policy 0, policy_version 43318 (0.0006) [2024-09-30 01:13:26,591][1157819] Updated weights for policy 0, policy_version 43328 (0.0006) [2024-09-30 01:13:27,093][1157819] Updated weights for policy 0, policy_version 43338 (0.0006) [2024-09-30 01:13:27,652][1157819] Updated weights for policy 0, policy_version 43348 (0.0006) [2024-09-30 01:13:28,200][1157819] Updated weights for policy 0, policy_version 43358 (0.0006) [2024-09-30 01:13:28,773][1157819] Updated weights for policy 0, policy_version 43368 (0.0006) [2024-09-30 01:13:29,312][1157819] Updated weights for policy 0, policy_version 43378 (0.0006) [2024-09-30 01:13:29,858][1157819] Updated weights for policy 0, policy_version 43388 (0.0006) [2024-09-30 01:13:30,364][1157819] Updated weights for policy 0, policy_version 43398 (0.0006) [2024-09-30 01:13:30,466][1157520] Fps is (10 sec: 78643.2, 60 sec: 81373.9, 300 sec: 81489.6). Total num frames: 177762304. Throughput: 0: 20698.4. Samples: 34409384. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:13:30,466][1157520] Avg episode reward: [(0, '52.577')] [2024-09-30 01:13:30,901][1157819] Updated weights for policy 0, policy_version 43408 (0.0006) [2024-09-30 01:13:31,443][1157819] Updated weights for policy 0, policy_version 43418 (0.0006) [2024-09-30 01:13:31,993][1157819] Updated weights for policy 0, policy_version 43428 (0.0006) [2024-09-30 01:13:32,561][1157819] Updated weights for policy 0, policy_version 43438 (0.0006) [2024-09-30 01:13:33,118][1157819] Updated weights for policy 0, policy_version 43448 (0.0006) [2024-09-30 01:13:33,647][1157819] Updated weights for policy 0, policy_version 43458 (0.0006) [2024-09-30 01:13:34,202][1157819] Updated weights for policy 0, policy_version 43468 (0.0006) [2024-09-30 01:13:34,754][1157819] Updated weights for policy 0, policy_version 43478 (0.0006) [2024-09-30 01:13:35,271][1157819] Updated weights for policy 0, policy_version 43488 (0.0006) [2024-09-30 01:13:35,466][1157520] Fps is (10 sec: 76185.6, 60 sec: 81373.9, 300 sec: 81461.8). Total num frames: 178139136. Throughput: 0: 20631.6. Samples: 34522088. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:13:35,466][1157520] Avg episode reward: [(0, '51.227')] [2024-09-30 01:13:35,767][1157819] Updated weights for policy 0, policy_version 43498 (0.0006) [2024-09-30 01:13:36,323][1157819] Updated weights for policy 0, policy_version 43508 (0.0006) [2024-09-30 01:13:36,819][1157819] Updated weights for policy 0, policy_version 43518 (0.0006) [2024-09-30 01:13:37,366][1157819] Updated weights for policy 0, policy_version 43528 (0.0006) [2024-09-30 01:13:37,905][1157819] Updated weights for policy 0, policy_version 43538 (0.0006) [2024-09-30 01:13:38,433][1157819] Updated weights for policy 0, policy_version 43548 (0.0006) [2024-09-30 01:13:38,963][1157819] Updated weights for policy 0, policy_version 43558 (0.0006) [2024-09-30 01:13:39,503][1157819] Updated weights for policy 0, policy_version 43568 (0.0006) [2024-09-30 01:13:40,060][1157819] Updated weights for policy 0, policy_version 43578 (0.0006) [2024-09-30 01:13:40,466][1157520] Fps is (10 sec: 76185.8, 60 sec: 81305.6, 300 sec: 81489.6). Total num frames: 178524160. Throughput: 0: 20678.8. Samples: 34581028. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:13:40,466][1157520] Avg episode reward: [(0, '52.146')] [2024-09-30 01:13:40,615][1157819] Updated weights for policy 0, policy_version 43588 (0.0006) [2024-09-30 01:13:41,192][1157819] Updated weights for policy 0, policy_version 43598 (0.0006) [2024-09-30 01:13:41,758][1157819] Updated weights for policy 0, policy_version 43608 (0.0006) [2024-09-30 01:13:42,325][1157819] Updated weights for policy 0, policy_version 43618 (0.0006) [2024-09-30 01:13:42,898][1157819] Updated weights for policy 0, policy_version 43628 (0.0006) [2024-09-30 01:13:43,434][1157819] Updated weights for policy 0, policy_version 43638 (0.0006) [2024-09-30 01:13:43,983][1157819] Updated weights for policy 0, policy_version 43648 (0.0006) [2024-09-30 01:13:44,563][1157819] Updated weights for policy 0, policy_version 43658 (0.0006) [2024-09-30 01:13:45,103][1157819] Updated weights for policy 0, policy_version 43668 (0.0006) [2024-09-30 01:13:45,466][1157520] Fps is (10 sec: 74956.6, 60 sec: 80759.4, 300 sec: 81378.5). Total num frames: 178888704. Throughput: 0: 20429.2. Samples: 34691744. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:13:45,466][1157520] Avg episode reward: [(0, '50.126')] [2024-09-30 01:13:45,670][1157819] Updated weights for policy 0, policy_version 43678 (0.0006) [2024-09-30 01:13:46,231][1157819] Updated weights for policy 0, policy_version 43688 (0.0006) [2024-09-30 01:13:46,802][1157819] Updated weights for policy 0, policy_version 43698 (0.0006) [2024-09-30 01:13:47,368][1157819] Updated weights for policy 0, policy_version 43708 (0.0006) [2024-09-30 01:13:47,451][1157736] Signal inference workers to stop experience collection... (2500 times) [2024-09-30 01:13:47,452][1157736] Signal inference workers to resume experience collection... (2500 times) [2024-09-30 01:13:47,456][1157819] InferenceWorker_p0-w0: stopping experience collection (2500 times) [2024-09-30 01:13:47,456][1157819] InferenceWorker_p0-w0: resuming experience collection (2500 times) [2024-09-30 01:13:47,927][1157819] Updated weights for policy 0, policy_version 43718 (0.0006) [2024-09-30 01:13:48,460][1157819] Updated weights for policy 0, policy_version 43728 (0.0006) [2024-09-30 01:13:49,038][1157819] Updated weights for policy 0, policy_version 43738 (0.0006) [2024-09-30 01:13:49,577][1157819] Updated weights for policy 0, policy_version 43748 (0.0006) [2024-09-30 01:13:50,174][1157819] Updated weights for policy 0, policy_version 43758 (0.0006) [2024-09-30 01:13:50,466][1157520] Fps is (10 sec: 72908.8, 60 sec: 80486.4, 300 sec: 81267.4). Total num frames: 179253248. Throughput: 0: 19863.9. Samples: 34801552. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:13:50,466][1157520] Avg episode reward: [(0, '51.217')] [2024-09-30 01:13:50,706][1157819] Updated weights for policy 0, policy_version 43768 (0.0006) [2024-09-30 01:13:51,248][1157819] Updated weights for policy 0, policy_version 43778 (0.0006) [2024-09-30 01:13:51,811][1157819] Updated weights for policy 0, policy_version 43788 (0.0006) [2024-09-30 01:13:52,348][1157819] Updated weights for policy 0, policy_version 43798 (0.0006) [2024-09-30 01:13:52,891][1157819] Updated weights for policy 0, policy_version 43808 (0.0006) [2024-09-30 01:13:53,426][1157819] Updated weights for policy 0, policy_version 43818 (0.0006) [2024-09-30 01:13:53,942][1157819] Updated weights for policy 0, policy_version 43828 (0.0006) [2024-09-30 01:13:54,450][1157819] Updated weights for policy 0, policy_version 43838 (0.0006) [2024-09-30 01:13:54,982][1157819] Updated weights for policy 0, policy_version 43848 (0.0006) [2024-09-30 01:13:55,466][1157520] Fps is (10 sec: 74547.0, 60 sec: 80281.5, 300 sec: 81225.8). Total num frames: 179634176. Throughput: 0: 19634.6. Samples: 34857192. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 01:13:55,466][1157520] Avg episode reward: [(0, '48.973')] [2024-09-30 01:13:55,519][1157819] Updated weights for policy 0, policy_version 43858 (0.0006) [2024-09-30 01:13:56,046][1157819] Updated weights for policy 0, policy_version 43868 (0.0006) [2024-09-30 01:13:56,540][1157819] Updated weights for policy 0, policy_version 43878 (0.0006) [2024-09-30 01:13:57,096][1157819] Updated weights for policy 0, policy_version 43888 (0.0006) [2024-09-30 01:13:57,634][1157819] Updated weights for policy 0, policy_version 43898 (0.0006) [2024-09-30 01:13:58,147][1157819] Updated weights for policy 0, policy_version 43908 (0.0006) [2024-09-30 01:13:58,718][1157819] Updated weights for policy 0, policy_version 43918 (0.0006) [2024-09-30 01:13:59,280][1157819] Updated weights for policy 0, policy_version 43928 (0.0006) [2024-09-30 01:13:59,802][1157819] Updated weights for policy 0, policy_version 43938 (0.0006) [2024-09-30 01:14:00,403][1157819] Updated weights for policy 0, policy_version 43948 (0.0006) [2024-09-30 01:14:00,466][1157520] Fps is (10 sec: 76185.2, 60 sec: 79598.9, 300 sec: 81114.7). Total num frames: 180015104. Throughput: 0: 19353.4. Samples: 34973400. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 01:14:00,466][1157520] Avg episode reward: [(0, '52.384')] [2024-09-30 01:14:00,934][1157819] Updated weights for policy 0, policy_version 43958 (0.0006) [2024-09-30 01:14:01,516][1157819] Updated weights for policy 0, policy_version 43968 (0.0006) [2024-09-30 01:14:02,032][1157819] Updated weights for policy 0, policy_version 43978 (0.0006) [2024-09-30 01:14:02,588][1157819] Updated weights for policy 0, policy_version 43988 (0.0006) [2024-09-30 01:14:03,149][1157819] Updated weights for policy 0, policy_version 43998 (0.0006) [2024-09-30 01:14:03,703][1157819] Updated weights for policy 0, policy_version 44008 (0.0006) [2024-09-30 01:14:04,278][1157819] Updated weights for policy 0, policy_version 44018 (0.0006) [2024-09-30 01:14:04,789][1157819] Updated weights for policy 0, policy_version 44028 (0.0006) [2024-09-30 01:14:05,326][1157819] Updated weights for policy 0, policy_version 44038 (0.0006) [2024-09-30 01:14:05,466][1157520] Fps is (10 sec: 75366.5, 60 sec: 78301.9, 300 sec: 81003.6). Total num frames: 180387840. Throughput: 0: 18994.2. Samples: 35084740. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 01:14:05,466][1157520] Avg episode reward: [(0, '52.870')] [2024-09-30 01:14:05,469][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000044040_180387840.pth... [2024-09-30 01:14:05,529][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000039301_160976896.pth [2024-09-30 01:14:05,901][1157819] Updated weights for policy 0, policy_version 44048 (0.0006) [2024-09-30 01:14:06,485][1157819] Updated weights for policy 0, policy_version 44058 (0.0006) [2024-09-30 01:14:07,023][1157819] Updated weights for policy 0, policy_version 44068 (0.0006) [2024-09-30 01:14:07,600][1157819] Updated weights for policy 0, policy_version 44078 (0.0006) [2024-09-30 01:14:08,146][1157819] Updated weights for policy 0, policy_version 44088 (0.0006) [2024-09-30 01:14:08,717][1157819] Updated weights for policy 0, policy_version 44098 (0.0006) [2024-09-30 01:14:09,251][1157819] Updated weights for policy 0, policy_version 44108 (0.0006) [2024-09-30 01:14:09,834][1157819] Updated weights for policy 0, policy_version 44118 (0.0006) [2024-09-30 01:14:10,382][1157819] Updated weights for policy 0, policy_version 44128 (0.0006) [2024-09-30 01:14:10,466][1157520] Fps is (10 sec: 73728.3, 60 sec: 77141.4, 300 sec: 80920.3). Total num frames: 180752384. Throughput: 0: 18838.4. Samples: 35139380. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 01:14:10,466][1157520] Avg episode reward: [(0, '51.017')] [2024-09-30 01:14:10,957][1157819] Updated weights for policy 0, policy_version 44138 (0.0006) [2024-09-30 01:14:11,511][1157819] Updated weights for policy 0, policy_version 44148 (0.0006) [2024-09-30 01:14:12,104][1157819] Updated weights for policy 0, policy_version 44158 (0.0006) [2024-09-30 01:14:12,638][1157819] Updated weights for policy 0, policy_version 44168 (0.0006) [2024-09-30 01:14:13,218][1157819] Updated weights for policy 0, policy_version 44178 (0.0006) [2024-09-30 01:14:13,735][1157819] Updated weights for policy 0, policy_version 44188 (0.0006) [2024-09-30 01:14:14,321][1157819] Updated weights for policy 0, policy_version 44198 (0.0006) [2024-09-30 01:14:14,871][1157819] Updated weights for policy 0, policy_version 44208 (0.0006) [2024-09-30 01:14:15,466][1157520] Fps is (10 sec: 72499.4, 60 sec: 76049.1, 300 sec: 80781.4). Total num frames: 181112832. Throughput: 0: 18658.4. Samples: 35249012. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 01:14:15,466][1157520] Avg episode reward: [(0, '51.589')] [2024-09-30 01:14:15,474][1157819] Updated weights for policy 0, policy_version 44218 (0.0006) [2024-09-30 01:14:15,990][1157819] Updated weights for policy 0, policy_version 44228 (0.0006) [2024-09-30 01:14:16,550][1157819] Updated weights for policy 0, policy_version 44238 (0.0006) [2024-09-30 01:14:17,122][1157819] Updated weights for policy 0, policy_version 44248 (0.0006) [2024-09-30 01:14:17,669][1157819] Updated weights for policy 0, policy_version 44258 (0.0006) [2024-09-30 01:14:18,244][1157819] Updated weights for policy 0, policy_version 44268 (0.0006) [2024-09-30 01:14:18,791][1157819] Updated weights for policy 0, policy_version 44278 (0.0006) [2024-09-30 01:14:19,385][1157819] Updated weights for policy 0, policy_version 44288 (0.0006) [2024-09-30 01:14:19,988][1157819] Updated weights for policy 0, policy_version 44298 (0.0006) [2024-09-30 01:14:20,466][1157520] Fps is (10 sec: 72499.4, 60 sec: 75025.1, 300 sec: 80698.2). Total num frames: 181477376. Throughput: 0: 18572.8. Samples: 35357864. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 01:14:20,466][1157520] Avg episode reward: [(0, '53.319')] [2024-09-30 01:14:20,575][1157819] Updated weights for policy 0, policy_version 44308 (0.0006) [2024-09-30 01:14:21,166][1157819] Updated weights for policy 0, policy_version 44318 (0.0006) [2024-09-30 01:14:21,769][1157819] Updated weights for policy 0, policy_version 44328 (0.0006) [2024-09-30 01:14:22,370][1157819] Updated weights for policy 0, policy_version 44338 (0.0006) [2024-09-30 01:14:22,935][1157819] Updated weights for policy 0, policy_version 44348 (0.0006) [2024-09-30 01:14:23,528][1157819] Updated weights for policy 0, policy_version 44358 (0.0006) [2024-09-30 01:14:24,144][1157819] Updated weights for policy 0, policy_version 44368 (0.0006) [2024-09-30 01:14:24,202][1157736] Signal inference workers to stop experience collection... (2550 times) [2024-09-30 01:14:24,203][1157736] Signal inference workers to resume experience collection... (2550 times) [2024-09-30 01:14:24,206][1157819] InferenceWorker_p0-w0: stopping experience collection (2550 times) [2024-09-30 01:14:24,209][1157819] InferenceWorker_p0-w0: resuming experience collection (2550 times) [2024-09-30 01:14:24,706][1157819] Updated weights for policy 0, policy_version 44378 (0.0006) [2024-09-30 01:14:25,327][1157819] Updated weights for policy 0, policy_version 44388 (0.0006) [2024-09-30 01:14:25,466][1157520] Fps is (10 sec: 70860.6, 60 sec: 74069.3, 300 sec: 80517.6). Total num frames: 181821440. Throughput: 0: 18416.5. Samples: 35409772. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 01:14:25,466][1157520] Avg episode reward: [(0, '52.950')] [2024-09-30 01:14:25,917][1157819] Updated weights for policy 0, policy_version 44398 (0.0006) [2024-09-30 01:14:26,499][1157819] Updated weights for policy 0, policy_version 44408 (0.0006) [2024-09-30 01:14:27,112][1157819] Updated weights for policy 0, policy_version 44418 (0.0006) [2024-09-30 01:14:27,712][1157819] Updated weights for policy 0, policy_version 44428 (0.0006) [2024-09-30 01:14:28,309][1157819] Updated weights for policy 0, policy_version 44438 (0.0006) [2024-09-30 01:14:28,893][1157819] Updated weights for policy 0, policy_version 44448 (0.0006) [2024-09-30 01:14:29,481][1157819] Updated weights for policy 0, policy_version 44458 (0.0006) [2024-09-30 01:14:30,083][1157819] Updated weights for policy 0, policy_version 44468 (0.0006) [2024-09-30 01:14:30,466][1157520] Fps is (10 sec: 68812.6, 60 sec: 73386.7, 300 sec: 80337.2). Total num frames: 182165504. Throughput: 0: 18253.7. Samples: 35513160. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 01:14:30,466][1157520] Avg episode reward: [(0, '52.418')] [2024-09-30 01:14:30,693][1157819] Updated weights for policy 0, policy_version 44478 (0.0006) [2024-09-30 01:14:31,272][1157819] Updated weights for policy 0, policy_version 44488 (0.0006) [2024-09-30 01:14:31,882][1157819] Updated weights for policy 0, policy_version 44498 (0.0006) [2024-09-30 01:14:32,484][1157819] Updated weights for policy 0, policy_version 44508 (0.0006) [2024-09-30 01:14:33,099][1157819] Updated weights for policy 0, policy_version 44518 (0.0006) [2024-09-30 01:14:33,657][1157819] Updated weights for policy 0, policy_version 44528 (0.0006) [2024-09-30 01:14:34,220][1157819] Updated weights for policy 0, policy_version 44538 (0.0006) [2024-09-30 01:14:34,769][1157819] Updated weights for policy 0, policy_version 44548 (0.0006) [2024-09-30 01:14:35,348][1157819] Updated weights for policy 0, policy_version 44558 (0.0006) [2024-09-30 01:14:35,466][1157520] Fps is (10 sec: 69632.2, 60 sec: 72977.1, 300 sec: 80184.4). Total num frames: 182517760. Throughput: 0: 18129.9. Samples: 35617400. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 01:14:35,466][1157520] Avg episode reward: [(0, '52.070')] [2024-09-30 01:14:35,900][1157819] Updated weights for policy 0, policy_version 44568 (0.0006) [2024-09-30 01:14:36,463][1157819] Updated weights for policy 0, policy_version 44578 (0.0006) [2024-09-30 01:14:37,011][1157819] Updated weights for policy 0, policy_version 44588 (0.0006) [2024-09-30 01:14:37,540][1157819] Updated weights for policy 0, policy_version 44598 (0.0006) [2024-09-30 01:14:38,115][1157819] Updated weights for policy 0, policy_version 44608 (0.0006) [2024-09-30 01:14:38,653][1157819] Updated weights for policy 0, policy_version 44618 (0.0006) [2024-09-30 01:14:39,221][1157819] Updated weights for policy 0, policy_version 44628 (0.0006) [2024-09-30 01:14:39,781][1157819] Updated weights for policy 0, policy_version 44638 (0.0006) [2024-09-30 01:14:40,348][1157819] Updated weights for policy 0, policy_version 44648 (0.0006) [2024-09-30 01:14:40,466][1157520] Fps is (10 sec: 72499.3, 60 sec: 72772.3, 300 sec: 80031.7). Total num frames: 182890496. Throughput: 0: 18119.8. Samples: 35672584. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 01:14:40,466][1157520] Avg episode reward: [(0, '50.762')] [2024-09-30 01:14:40,917][1157819] Updated weights for policy 0, policy_version 44658 (0.0006) [2024-09-30 01:14:41,443][1157819] Updated weights for policy 0, policy_version 44668 (0.0007) [2024-09-30 01:14:42,026][1157819] Updated weights for policy 0, policy_version 44678 (0.0006) [2024-09-30 01:14:42,566][1157819] Updated weights for policy 0, policy_version 44688 (0.0006) [2024-09-30 01:14:43,138][1157819] Updated weights for policy 0, policy_version 44698 (0.0007) [2024-09-30 01:14:43,696][1157819] Updated weights for policy 0, policy_version 44708 (0.0006) [2024-09-30 01:14:44,255][1157819] Updated weights for policy 0, policy_version 44718 (0.0006) [2024-09-30 01:14:44,859][1157819] Updated weights for policy 0, policy_version 44728 (0.0006) [2024-09-30 01:14:45,391][1157819] Updated weights for policy 0, policy_version 44738 (0.0006) [2024-09-30 01:14:45,466][1157520] Fps is (10 sec: 73318.6, 60 sec: 72704.0, 300 sec: 79851.2). Total num frames: 183250944. Throughput: 0: 17982.1. Samples: 35782592. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 01:14:45,466][1157520] Avg episode reward: [(0, '52.346')] [2024-09-30 01:14:45,925][1157819] Updated weights for policy 0, policy_version 44748 (0.0007) [2024-09-30 01:14:46,511][1157819] Updated weights for policy 0, policy_version 44758 (0.0006) [2024-09-30 01:14:47,043][1157819] Updated weights for policy 0, policy_version 44768 (0.0006) [2024-09-30 01:14:47,630][1157819] Updated weights for policy 0, policy_version 44778 (0.0006) [2024-09-30 01:14:48,155][1157819] Updated weights for policy 0, policy_version 44788 (0.0006) [2024-09-30 01:14:48,760][1157819] Updated weights for policy 0, policy_version 44798 (0.0006) [2024-09-30 01:14:49,271][1157819] Updated weights for policy 0, policy_version 44808 (0.0006) [2024-09-30 01:14:49,863][1157819] Updated weights for policy 0, policy_version 44818 (0.0006) [2024-09-30 01:14:50,380][1157819] Updated weights for policy 0, policy_version 44828 (0.0006) [2024-09-30 01:14:50,466][1157520] Fps is (10 sec: 72908.8, 60 sec: 72772.3, 300 sec: 79684.6). Total num frames: 183619584. Throughput: 0: 17959.6. Samples: 35892920. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 01:14:50,466][1157520] Avg episode reward: [(0, '53.227')] [2024-09-30 01:14:50,910][1157819] Updated weights for policy 0, policy_version 44838 (0.0006) [2024-09-30 01:14:51,495][1157819] Updated weights for policy 0, policy_version 44848 (0.0006) [2024-09-30 01:14:52,021][1157819] Updated weights for policy 0, policy_version 44858 (0.0006) [2024-09-30 01:14:52,572][1157819] Updated weights for policy 0, policy_version 44868 (0.0006) [2024-09-30 01:14:53,152][1157819] Updated weights for policy 0, policy_version 44878 (0.0006) [2024-09-30 01:14:53,665][1157819] Updated weights for policy 0, policy_version 44888 (0.0006) [2024-09-30 01:14:54,120][1157736] Signal inference workers to stop experience collection... (2600 times) [2024-09-30 01:14:54,122][1157736] Signal inference workers to resume experience collection... (2600 times) [2024-09-30 01:14:54,126][1157819] InferenceWorker_p0-w0: stopping experience collection (2600 times) [2024-09-30 01:14:54,129][1157819] InferenceWorker_p0-w0: resuming experience collection (2600 times) [2024-09-30 01:14:54,273][1157819] Updated weights for policy 0, policy_version 44898 (0.0006) [2024-09-30 01:14:54,882][1157819] Updated weights for policy 0, policy_version 44908 (0.0006) [2024-09-30 01:14:55,458][1157819] Updated weights for policy 0, policy_version 44918 (0.0006) [2024-09-30 01:14:55,466][1157520] Fps is (10 sec: 73318.0, 60 sec: 72499.2, 300 sec: 79462.4). Total num frames: 183984128. Throughput: 0: 17981.4. Samples: 35948544. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:14:55,466][1157520] Avg episode reward: [(0, '49.482')] [2024-09-30 01:14:56,036][1157819] Updated weights for policy 0, policy_version 44928 (0.0007) [2024-09-30 01:14:56,628][1157819] Updated weights for policy 0, policy_version 44938 (0.0006) [2024-09-30 01:14:57,211][1157819] Updated weights for policy 0, policy_version 44948 (0.0006) [2024-09-30 01:14:57,829][1157819] Updated weights for policy 0, policy_version 44958 (0.0006) [2024-09-30 01:14:58,413][1157819] Updated weights for policy 0, policy_version 44968 (0.0006) [2024-09-30 01:14:59,001][1157819] Updated weights for policy 0, policy_version 44978 (0.0006) [2024-09-30 01:14:59,610][1157819] Updated weights for policy 0, policy_version 44988 (0.0006) [2024-09-30 01:15:00,204][1157819] Updated weights for policy 0, policy_version 44998 (0.0006) [2024-09-30 01:15:00,466][1157520] Fps is (10 sec: 70860.7, 60 sec: 71884.9, 300 sec: 79156.9). Total num frames: 184328192. Throughput: 0: 17878.9. Samples: 36053564. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:15:00,466][1157520] Avg episode reward: [(0, '54.316')] [2024-09-30 01:15:00,466][1157736] Saving new best policy, reward=54.316! [2024-09-30 01:15:00,769][1157819] Updated weights for policy 0, policy_version 45008 (0.0006) [2024-09-30 01:15:01,341][1157819] Updated weights for policy 0, policy_version 45018 (0.0006) [2024-09-30 01:15:01,802][1157819] Updated weights for policy 0, policy_version 45028 (0.0006) [2024-09-30 01:15:02,246][1157819] Updated weights for policy 0, policy_version 45038 (0.0006) [2024-09-30 01:15:02,733][1157819] Updated weights for policy 0, policy_version 45048 (0.0006) [2024-09-30 01:15:03,211][1157819] Updated weights for policy 0, policy_version 45058 (0.0006) [2024-09-30 01:15:03,692][1157819] Updated weights for policy 0, policy_version 45068 (0.0006) [2024-09-30 01:15:04,190][1157819] Updated weights for policy 0, policy_version 45078 (0.0006) [2024-09-30 01:15:04,645][1157819] Updated weights for policy 0, policy_version 45088 (0.0006) [2024-09-30 01:15:05,143][1157819] Updated weights for policy 0, policy_version 45098 (0.0006) [2024-09-30 01:15:05,466][1157520] Fps is (10 sec: 76183.8, 60 sec: 72635.4, 300 sec: 79156.9). Total num frames: 184745984. Throughput: 0: 18114.8. Samples: 36173036. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:15:05,466][1157520] Avg episode reward: [(0, '51.901')] [2024-09-30 01:15:05,683][1157819] Updated weights for policy 0, policy_version 45108 (0.0006) [2024-09-30 01:15:06,208][1157819] Updated weights for policy 0, policy_version 45118 (0.0006) [2024-09-30 01:15:06,692][1157819] Updated weights for policy 0, policy_version 45128 (0.0006) [2024-09-30 01:15:07,189][1157819] Updated weights for policy 0, policy_version 45138 (0.0006) [2024-09-30 01:15:07,713][1157819] Updated weights for policy 0, policy_version 45148 (0.0006) [2024-09-30 01:15:08,247][1157819] Updated weights for policy 0, policy_version 45158 (0.0006) [2024-09-30 01:15:08,768][1157819] Updated weights for policy 0, policy_version 45168 (0.0006) [2024-09-30 01:15:09,313][1157819] Updated weights for policy 0, policy_version 45178 (0.0006) [2024-09-30 01:15:09,844][1157819] Updated weights for policy 0, policy_version 45188 (0.0006) [2024-09-30 01:15:10,383][1157819] Updated weights for policy 0, policy_version 45198 (0.0006) [2024-09-30 01:15:10,466][1157520] Fps is (10 sec: 80690.8, 60 sec: 73045.3, 300 sec: 79018.1). Total num frames: 185135104. Throughput: 0: 18297.6. Samples: 36233164. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:15:10,466][1157520] Avg episode reward: [(0, '52.952')] [2024-09-30 01:15:10,919][1157819] Updated weights for policy 0, policy_version 45208 (0.0006) [2024-09-30 01:15:11,458][1157819] Updated weights for policy 0, policy_version 45218 (0.0006) [2024-09-30 01:15:12,022][1157819] Updated weights for policy 0, policy_version 45228 (0.0006) [2024-09-30 01:15:12,575][1157819] Updated weights for policy 0, policy_version 45238 (0.0006) [2024-09-30 01:15:13,204][1157819] Updated weights for policy 0, policy_version 45248 (0.0006) [2024-09-30 01:15:13,749][1157819] Updated weights for policy 0, policy_version 45258 (0.0006) [2024-09-30 01:15:14,323][1157819] Updated weights for policy 0, policy_version 45268 (0.0006) [2024-09-30 01:15:14,922][1157819] Updated weights for policy 0, policy_version 45278 (0.0006) [2024-09-30 01:15:15,169][1157736] Signal inference workers to stop experience collection... (2650 times) [2024-09-30 01:15:15,171][1157736] Signal inference workers to resume experience collection... (2650 times) [2024-09-30 01:15:15,172][1157819] InferenceWorker_p0-w0: stopping experience collection (2650 times) [2024-09-30 01:15:15,176][1157819] InferenceWorker_p0-w0: resuming experience collection (2650 times) [2024-09-30 01:15:15,466][1157520] Fps is (10 sec: 74957.8, 60 sec: 73045.2, 300 sec: 78712.6). Total num frames: 185495552. Throughput: 0: 18481.5. Samples: 36344828. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:15:15,466][1157520] Avg episode reward: [(0, '49.262')] [2024-09-30 01:15:15,480][1157819] Updated weights for policy 0, policy_version 45288 (0.0006) [2024-09-30 01:15:16,017][1157819] Updated weights for policy 0, policy_version 45298 (0.0006) [2024-09-30 01:15:16,607][1157819] Updated weights for policy 0, policy_version 45308 (0.0006) [2024-09-30 01:15:17,126][1157819] Updated weights for policy 0, policy_version 45318 (0.0006) [2024-09-30 01:15:17,653][1157819] Updated weights for policy 0, policy_version 45328 (0.0006) [2024-09-30 01:15:18,206][1157819] Updated weights for policy 0, policy_version 45338 (0.0006) [2024-09-30 01:15:18,752][1157819] Updated weights for policy 0, policy_version 45348 (0.0006) [2024-09-30 01:15:19,278][1157819] Updated weights for policy 0, policy_version 45358 (0.0006) [2024-09-30 01:15:19,838][1157819] Updated weights for policy 0, policy_version 45368 (0.0006) [2024-09-30 01:15:20,375][1157819] Updated weights for policy 0, policy_version 45378 (0.0006) [2024-09-30 01:15:20,466][1157520] Fps is (10 sec: 73727.1, 60 sec: 73249.9, 300 sec: 78462.7). Total num frames: 185872384. Throughput: 0: 18637.9. Samples: 36456108. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:15:20,466][1157520] Avg episode reward: [(0, '49.980')] [2024-09-30 01:15:20,908][1157819] Updated weights for policy 0, policy_version 45388 (0.0006) [2024-09-30 01:15:21,451][1157819] Updated weights for policy 0, policy_version 45398 (0.0006) [2024-09-30 01:15:22,020][1157819] Updated weights for policy 0, policy_version 45408 (0.0006) [2024-09-30 01:15:22,574][1157819] Updated weights for policy 0, policy_version 45418 (0.0006) [2024-09-30 01:15:23,135][1157819] Updated weights for policy 0, policy_version 45428 (0.0006) [2024-09-30 01:15:23,714][1157819] Updated weights for policy 0, policy_version 45438 (0.0006) [2024-09-30 01:15:24,230][1157819] Updated weights for policy 0, policy_version 45448 (0.0006) [2024-09-30 01:15:24,778][1157819] Updated weights for policy 0, policy_version 45458 (0.0006) [2024-09-30 01:15:25,311][1157819] Updated weights for policy 0, policy_version 45468 (0.0006) [2024-09-30 01:15:25,466][1157520] Fps is (10 sec: 74957.1, 60 sec: 73727.9, 300 sec: 78171.1). Total num frames: 186245120. Throughput: 0: 18653.1. Samples: 36511976. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:15:25,466][1157520] Avg episode reward: [(0, '50.296')] [2024-09-30 01:15:25,870][1157819] Updated weights for policy 0, policy_version 45478 (0.0006) [2024-09-30 01:15:26,389][1157819] Updated weights for policy 0, policy_version 45488 (0.0006) [2024-09-30 01:15:26,929][1157819] Updated weights for policy 0, policy_version 45498 (0.0006) [2024-09-30 01:15:27,524][1157819] Updated weights for policy 0, policy_version 45508 (0.0006) [2024-09-30 01:15:28,133][1157819] Updated weights for policy 0, policy_version 45518 (0.0006) [2024-09-30 01:15:28,814][1157819] Updated weights for policy 0, policy_version 45528 (0.0007) [2024-09-30 01:15:29,520][1157819] Updated weights for policy 0, policy_version 45538 (0.0007) [2024-09-30 01:15:30,131][1157819] Updated weights for policy 0, policy_version 45548 (0.0006) [2024-09-30 01:15:30,466][1157520] Fps is (10 sec: 71271.0, 60 sec: 73659.6, 300 sec: 77962.8). Total num frames: 186585088. Throughput: 0: 18612.1. Samples: 36620140. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:15:30,466][1157520] Avg episode reward: [(0, '51.427')] [2024-09-30 01:15:30,777][1157819] Updated weights for policy 0, policy_version 45558 (0.0006) [2024-09-30 01:15:31,405][1157819] Updated weights for policy 0, policy_version 45568 (0.0006) [2024-09-30 01:15:32,050][1157819] Updated weights for policy 0, policy_version 45578 (0.0006) [2024-09-30 01:15:32,665][1157819] Updated weights for policy 0, policy_version 45588 (0.0006) [2024-09-30 01:15:33,202][1157819] Updated weights for policy 0, policy_version 45598 (0.0006) [2024-09-30 01:15:33,767][1157819] Updated weights for policy 0, policy_version 45608 (0.0006) [2024-09-30 01:15:34,292][1157819] Updated weights for policy 0, policy_version 45618 (0.0006) [2024-09-30 01:15:34,827][1157819] Updated weights for policy 0, policy_version 45628 (0.0006) [2024-09-30 01:15:35,328][1157819] Updated weights for policy 0, policy_version 45638 (0.0006) [2024-09-30 01:15:35,466][1157520] Fps is (10 sec: 70041.4, 60 sec: 73796.1, 300 sec: 77712.9). Total num frames: 186945536. Throughput: 0: 18426.0. Samples: 36722092. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:15:35,466][1157520] Avg episode reward: [(0, '52.469')] [2024-09-30 01:15:35,862][1157819] Updated weights for policy 0, policy_version 45648 (0.0006) [2024-09-30 01:15:36,363][1157819] Updated weights for policy 0, policy_version 45658 (0.0006) [2024-09-30 01:15:36,860][1157819] Updated weights for policy 0, policy_version 45668 (0.0006) [2024-09-30 01:15:37,369][1157819] Updated weights for policy 0, policy_version 45678 (0.0006) [2024-09-30 01:15:37,873][1157819] Updated weights for policy 0, policy_version 45688 (0.0006) [2024-09-30 01:15:38,418][1157819] Updated weights for policy 0, policy_version 45698 (0.0006) [2024-09-30 01:15:38,918][1157819] Updated weights for policy 0, policy_version 45708 (0.0006) [2024-09-30 01:15:39,422][1157819] Updated weights for policy 0, policy_version 45718 (0.0006) [2024-09-30 01:15:39,950][1157819] Updated weights for policy 0, policy_version 45728 (0.0006) [2024-09-30 01:15:40,440][1157819] Updated weights for policy 0, policy_version 45738 (0.0006) [2024-09-30 01:15:40,466][1157520] Fps is (10 sec: 75775.0, 60 sec: 74205.6, 300 sec: 77518.5). Total num frames: 187342848. Throughput: 0: 18537.2. Samples: 36782720. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:15:40,466][1157520] Avg episode reward: [(0, '54.816')] [2024-09-30 01:15:40,467][1157736] Saving new best policy, reward=54.816! [2024-09-30 01:15:40,960][1157819] Updated weights for policy 0, policy_version 45748 (0.0006) [2024-09-30 01:15:41,452][1157819] Updated weights for policy 0, policy_version 45758 (0.0006) [2024-09-30 01:15:41,975][1157819] Updated weights for policy 0, policy_version 45768 (0.0006) [2024-09-30 01:15:42,512][1157819] Updated weights for policy 0, policy_version 45778 (0.0006) [2024-09-30 01:15:43,037][1157819] Updated weights for policy 0, policy_version 45788 (0.0006) [2024-09-30 01:15:43,544][1157819] Updated weights for policy 0, policy_version 45798 (0.0006) [2024-09-30 01:15:44,071][1157819] Updated weights for policy 0, policy_version 45808 (0.0006) [2024-09-30 01:15:44,576][1157819] Updated weights for policy 0, policy_version 45818 (0.0006) [2024-09-30 01:15:45,135][1157819] Updated weights for policy 0, policy_version 45828 (0.0006) [2024-09-30 01:15:45,466][1157520] Fps is (10 sec: 79051.8, 60 sec: 74751.7, 300 sec: 77296.3). Total num frames: 187736064. Throughput: 0: 18849.6. Samples: 36901800. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:15:45,466][1157520] Avg episode reward: [(0, '52.602')] [2024-09-30 01:15:45,642][1157819] Updated weights for policy 0, policy_version 45838 (0.0006) [2024-09-30 01:15:46,178][1157819] Updated weights for policy 0, policy_version 45848 (0.0006) [2024-09-30 01:15:46,711][1157819] Updated weights for policy 0, policy_version 45858 (0.0006) [2024-09-30 01:15:47,218][1157819] Updated weights for policy 0, policy_version 45868 (0.0006) [2024-09-30 01:15:47,722][1157819] Updated weights for policy 0, policy_version 45878 (0.0006) [2024-09-30 01:15:48,254][1157819] Updated weights for policy 0, policy_version 45888 (0.0006) [2024-09-30 01:15:48,787][1157819] Updated weights for policy 0, policy_version 45898 (0.0006) [2024-09-30 01:15:49,301][1157819] Updated weights for policy 0, policy_version 45908 (0.0006) [2024-09-30 01:15:49,809][1157819] Updated weights for policy 0, policy_version 45918 (0.0006) [2024-09-30 01:15:50,337][1157819] Updated weights for policy 0, policy_version 45928 (0.0006) [2024-09-30 01:15:50,466][1157520] Fps is (10 sec: 78643.2, 60 sec: 75161.3, 300 sec: 77129.7). Total num frames: 188129280. Throughput: 0: 18806.4. Samples: 37019324. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:15:50,466][1157520] Avg episode reward: [(0, '50.580')] [2024-09-30 01:15:50,857][1157819] Updated weights for policy 0, policy_version 45938 (0.0006) [2024-09-30 01:15:51,390][1157819] Updated weights for policy 0, policy_version 45948 (0.0006) [2024-09-30 01:15:51,912][1157819] Updated weights for policy 0, policy_version 45958 (0.0006) [2024-09-30 01:15:52,418][1157819] Updated weights for policy 0, policy_version 45968 (0.0006) [2024-09-30 01:15:52,943][1157819] Updated weights for policy 0, policy_version 45978 (0.0006) [2024-09-30 01:15:53,440][1157819] Updated weights for policy 0, policy_version 45988 (0.0006) [2024-09-30 01:15:53,611][1157736] Signal inference workers to stop experience collection... (2700 times) [2024-09-30 01:15:53,611][1157736] Signal inference workers to resume experience collection... (2700 times) [2024-09-30 01:15:53,617][1157819] InferenceWorker_p0-w0: stopping experience collection (2700 times) [2024-09-30 01:15:53,617][1157819] InferenceWorker_p0-w0: resuming experience collection (2700 times) [2024-09-30 01:15:53,943][1157819] Updated weights for policy 0, policy_version 45998 (0.0006) [2024-09-30 01:15:54,447][1157819] Updated weights for policy 0, policy_version 46008 (0.0006) [2024-09-30 01:15:54,948][1157819] Updated weights for policy 0, policy_version 46018 (0.0006) [2024-09-30 01:15:55,466][1157520] Fps is (10 sec: 79053.6, 60 sec: 75707.6, 300 sec: 76963.1). Total num frames: 188526592. Throughput: 0: 18786.0. Samples: 37078536. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:15:55,466][1157520] Avg episode reward: [(0, '50.268')] [2024-09-30 01:15:55,488][1157819] Updated weights for policy 0, policy_version 46028 (0.0006) [2024-09-30 01:15:56,010][1157819] Updated weights for policy 0, policy_version 46038 (0.0006) [2024-09-30 01:15:56,586][1157819] Updated weights for policy 0, policy_version 46048 (0.0006) [2024-09-30 01:15:57,154][1157819] Updated weights for policy 0, policy_version 46058 (0.0006) [2024-09-30 01:15:57,706][1157819] Updated weights for policy 0, policy_version 46068 (0.0006) [2024-09-30 01:15:58,272][1157819] Updated weights for policy 0, policy_version 46078 (0.0006) [2024-09-30 01:15:58,843][1157819] Updated weights for policy 0, policy_version 46088 (0.0006) [2024-09-30 01:15:59,400][1157819] Updated weights for policy 0, policy_version 46098 (0.0006) [2024-09-30 01:15:59,955][1157819] Updated weights for policy 0, policy_version 46108 (0.0006) [2024-09-30 01:16:00,466][1157520] Fps is (10 sec: 76595.5, 60 sec: 76117.1, 300 sec: 76866.0). Total num frames: 188895232. Throughput: 0: 18848.1. Samples: 37192992. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:16:00,466][1157520] Avg episode reward: [(0, '52.520')] [2024-09-30 01:16:00,539][1157819] Updated weights for policy 0, policy_version 46118 (0.0006) [2024-09-30 01:16:01,090][1157819] Updated weights for policy 0, policy_version 46128 (0.0006) [2024-09-30 01:16:01,645][1157819] Updated weights for policy 0, policy_version 46138 (0.0006) [2024-09-30 01:16:02,194][1157819] Updated weights for policy 0, policy_version 46148 (0.0006) [2024-09-30 01:16:02,717][1157819] Updated weights for policy 0, policy_version 46158 (0.0006) [2024-09-30 01:16:03,269][1157819] Updated weights for policy 0, policy_version 46168 (0.0006) [2024-09-30 01:16:03,824][1157819] Updated weights for policy 0, policy_version 46178 (0.0006) [2024-09-30 01:16:04,356][1157819] Updated weights for policy 0, policy_version 46188 (0.0006) [2024-09-30 01:16:04,912][1157819] Updated weights for policy 0, policy_version 46198 (0.0006) [2024-09-30 01:16:05,466][1157520] Fps is (10 sec: 73727.4, 60 sec: 75298.2, 300 sec: 76754.8). Total num frames: 189263872. Throughput: 0: 18851.1. Samples: 37304408. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:16:05,466][1157520] Avg episode reward: [(0, '52.369')] [2024-09-30 01:16:05,471][1157819] Updated weights for policy 0, policy_version 46208 (0.0006) [2024-09-30 01:16:05,473][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000046208_189267968.pth... [2024-09-30 01:16:05,525][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000041728_170917888.pth [2024-09-30 01:16:06,028][1157819] Updated weights for policy 0, policy_version 46218 (0.0006) [2024-09-30 01:16:06,558][1157819] Updated weights for policy 0, policy_version 46228 (0.0006) [2024-09-30 01:16:07,118][1157819] Updated weights for policy 0, policy_version 46238 (0.0006) [2024-09-30 01:16:07,661][1157819] Updated weights for policy 0, policy_version 46248 (0.0006) [2024-09-30 01:16:08,189][1157819] Updated weights for policy 0, policy_version 46258 (0.0006) [2024-09-30 01:16:08,765][1157819] Updated weights for policy 0, policy_version 46268 (0.0006) [2024-09-30 01:16:09,326][1157819] Updated weights for policy 0, policy_version 46278 (0.0006) [2024-09-30 01:16:09,907][1157819] Updated weights for policy 0, policy_version 46288 (0.0006) [2024-09-30 01:16:10,466][1157520] Fps is (10 sec: 73728.4, 60 sec: 74956.7, 300 sec: 76602.1). Total num frames: 189632512. Throughput: 0: 18854.9. Samples: 37360448. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:16:10,466][1157520] Avg episode reward: [(0, '52.008')] [2024-09-30 01:16:10,487][1157819] Updated weights for policy 0, policy_version 46298 (0.0006) [2024-09-30 01:16:11,065][1157819] Updated weights for policy 0, policy_version 46308 (0.0006) [2024-09-30 01:16:11,658][1157819] Updated weights for policy 0, policy_version 46318 (0.0006) [2024-09-30 01:16:12,248][1157819] Updated weights for policy 0, policy_version 46328 (0.0006) [2024-09-30 01:16:12,842][1157819] Updated weights for policy 0, policy_version 46338 (0.0006) [2024-09-30 01:16:13,379][1157819] Updated weights for policy 0, policy_version 46348 (0.0006) [2024-09-30 01:16:13,957][1157819] Updated weights for policy 0, policy_version 46358 (0.0006) [2024-09-30 01:16:14,495][1157819] Updated weights for policy 0, policy_version 46368 (0.0006) [2024-09-30 01:16:15,088][1157819] Updated weights for policy 0, policy_version 46378 (0.0006) [2024-09-30 01:16:15,466][1157520] Fps is (10 sec: 73318.9, 60 sec: 75025.0, 300 sec: 76616.0). Total num frames: 189997056. Throughput: 0: 18839.7. Samples: 37467928. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:16:15,466][1157520] Avg episode reward: [(0, '51.477')] [2024-09-30 01:16:15,611][1157819] Updated weights for policy 0, policy_version 46388 (0.0006) [2024-09-30 01:16:16,167][1157819] Updated weights for policy 0, policy_version 46398 (0.0006) [2024-09-30 01:16:16,703][1157819] Updated weights for policy 0, policy_version 46408 (0.0006) [2024-09-30 01:16:17,249][1157819] Updated weights for policy 0, policy_version 46418 (0.0006) [2024-09-30 01:16:17,836][1157819] Updated weights for policy 0, policy_version 46428 (0.0006) [2024-09-30 01:16:18,346][1157819] Updated weights for policy 0, policy_version 46438 (0.0006) [2024-09-30 01:16:18,868][1157819] Updated weights for policy 0, policy_version 46448 (0.0006) [2024-09-30 01:16:19,356][1157819] Updated weights for policy 0, policy_version 46458 (0.0006) [2024-09-30 01:16:19,892][1157819] Updated weights for policy 0, policy_version 46468 (0.0006) [2024-09-30 01:16:20,396][1157819] Updated weights for policy 0, policy_version 46478 (0.0006) [2024-09-30 01:16:20,466][1157520] Fps is (10 sec: 74546.1, 60 sec: 75093.2, 300 sec: 76754.8). Total num frames: 190377984. Throughput: 0: 19091.7. Samples: 37581220. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:16:20,466][1157520] Avg episode reward: [(0, '52.742')] [2024-09-30 01:16:20,866][1157819] Updated weights for policy 0, policy_version 46488 (0.0006) [2024-09-30 01:16:21,377][1157819] Updated weights for policy 0, policy_version 46498 (0.0006) [2024-09-30 01:16:21,885][1157819] Updated weights for policy 0, policy_version 46508 (0.0006) [2024-09-30 01:16:22,390][1157819] Updated weights for policy 0, policy_version 46518 (0.0006) [2024-09-30 01:16:22,892][1157819] Updated weights for policy 0, policy_version 46528 (0.0006) [2024-09-30 01:16:23,430][1157819] Updated weights for policy 0, policy_version 46538 (0.0006) [2024-09-30 01:16:23,973][1157819] Updated weights for policy 0, policy_version 46548 (0.0007) [2024-09-30 01:16:24,463][1157819] Updated weights for policy 0, policy_version 46558 (0.0006) [2024-09-30 01:16:24,970][1157819] Updated weights for policy 0, policy_version 46568 (0.0006) [2024-09-30 01:16:25,466][1157520] Fps is (10 sec: 78233.8, 60 sec: 75571.1, 300 sec: 76838.1). Total num frames: 190779392. Throughput: 0: 19108.1. Samples: 37642584. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:16:25,466][1157520] Avg episode reward: [(0, '51.314')] [2024-09-30 01:16:25,491][1157819] Updated weights for policy 0, policy_version 46578 (0.0006) [2024-09-30 01:16:26,033][1157819] Updated weights for policy 0, policy_version 46588 (0.0006) [2024-09-30 01:16:26,530][1157819] Updated weights for policy 0, policy_version 46598 (0.0006) [2024-09-30 01:16:27,034][1157819] Updated weights for policy 0, policy_version 46608 (0.0006) [2024-09-30 01:16:27,560][1157819] Updated weights for policy 0, policy_version 46618 (0.0006) [2024-09-30 01:16:28,079][1157819] Updated weights for policy 0, policy_version 46628 (0.0006) [2024-09-30 01:16:28,576][1157819] Updated weights for policy 0, policy_version 46638 (0.0006) [2024-09-30 01:16:29,070][1157819] Updated weights for policy 0, policy_version 46648 (0.0006) [2024-09-30 01:16:29,567][1157819] Updated weights for policy 0, policy_version 46658 (0.0006) [2024-09-30 01:16:30,035][1157736] Signal inference workers to stop experience collection... (2750 times) [2024-09-30 01:16:30,040][1157819] InferenceWorker_p0-w0: stopping experience collection (2750 times) [2024-09-30 01:16:30,041][1157736] Signal inference workers to resume experience collection... (2750 times) [2024-09-30 01:16:30,044][1157819] InferenceWorker_p0-w0: resuming experience collection (2750 times) [2024-09-30 01:16:30,072][1157819] Updated weights for policy 0, policy_version 46668 (0.0006) [2024-09-30 01:16:30,466][1157520] Fps is (10 sec: 79873.1, 60 sec: 76526.9, 300 sec: 76865.9). Total num frames: 191176704. Throughput: 0: 19105.6. Samples: 37761548. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:16:30,466][1157520] Avg episode reward: [(0, '52.969')] [2024-09-30 01:16:30,598][1157819] Updated weights for policy 0, policy_version 46678 (0.0006) [2024-09-30 01:16:31,161][1157819] Updated weights for policy 0, policy_version 46688 (0.0006) [2024-09-30 01:16:31,659][1157819] Updated weights for policy 0, policy_version 46698 (0.0006) [2024-09-30 01:16:32,177][1157819] Updated weights for policy 0, policy_version 46708 (0.0006) [2024-09-30 01:16:32,668][1157819] Updated weights for policy 0, policy_version 46718 (0.0006) [2024-09-30 01:16:33,162][1157819] Updated weights for policy 0, policy_version 46728 (0.0006) [2024-09-30 01:16:33,635][1157819] Updated weights for policy 0, policy_version 46738 (0.0006) [2024-09-30 01:16:34,106][1157819] Updated weights for policy 0, policy_version 46748 (0.0006) [2024-09-30 01:16:34,553][1157819] Updated weights for policy 0, policy_version 46758 (0.0006) [2024-09-30 01:16:35,022][1157819] Updated weights for policy 0, policy_version 46768 (0.0006) [2024-09-30 01:16:35,466][1157520] Fps is (10 sec: 81919.7, 60 sec: 77550.9, 300 sec: 76921.4). Total num frames: 191598592. Throughput: 0: 19239.7. Samples: 37885108. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:16:35,466][1157520] Avg episode reward: [(0, '51.356')] [2024-09-30 01:16:35,482][1157819] Updated weights for policy 0, policy_version 46778 (0.0006) [2024-09-30 01:16:35,964][1157819] Updated weights for policy 0, policy_version 46788 (0.0006) [2024-09-30 01:16:36,441][1157819] Updated weights for policy 0, policy_version 46798 (0.0006) [2024-09-30 01:16:36,898][1157819] Updated weights for policy 0, policy_version 46808 (0.0006) [2024-09-30 01:16:37,347][1157819] Updated weights for policy 0, policy_version 46818 (0.0006) [2024-09-30 01:16:37,820][1157819] Updated weights for policy 0, policy_version 46828 (0.0006) [2024-09-30 01:16:38,273][1157819] Updated weights for policy 0, policy_version 46838 (0.0006) [2024-09-30 01:16:38,746][1157819] Updated weights for policy 0, policy_version 46848 (0.0006) [2024-09-30 01:16:39,202][1157819] Updated weights for policy 0, policy_version 46858 (0.0006) [2024-09-30 01:16:39,676][1157819] Updated weights for policy 0, policy_version 46868 (0.0006) [2024-09-30 01:16:40,168][1157819] Updated weights for policy 0, policy_version 46878 (0.0006) [2024-09-30 01:16:40,466][1157520] Fps is (10 sec: 86015.8, 60 sec: 78233.7, 300 sec: 77032.5). Total num frames: 192036864. Throughput: 0: 19389.8. Samples: 37951076. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:16:40,466][1157520] Avg episode reward: [(0, '50.820')] [2024-09-30 01:16:40,663][1157819] Updated weights for policy 0, policy_version 46888 (0.0006) [2024-09-30 01:16:41,159][1157819] Updated weights for policy 0, policy_version 46898 (0.0006) [2024-09-30 01:16:41,664][1157819] Updated weights for policy 0, policy_version 46908 (0.0006) [2024-09-30 01:16:42,145][1157819] Updated weights for policy 0, policy_version 46918 (0.0006) [2024-09-30 01:16:42,639][1157819] Updated weights for policy 0, policy_version 46928 (0.0006) [2024-09-30 01:16:43,112][1157819] Updated weights for policy 0, policy_version 46938 (0.0006) [2024-09-30 01:16:43,616][1157819] Updated weights for policy 0, policy_version 46948 (0.0006) [2024-09-30 01:16:44,114][1157819] Updated weights for policy 0, policy_version 46958 (0.0006) [2024-09-30 01:16:44,580][1157819] Updated weights for policy 0, policy_version 46968 (0.0006) [2024-09-30 01:16:45,087][1157819] Updated weights for policy 0, policy_version 46978 (0.0006) [2024-09-30 01:16:45,466][1157520] Fps is (10 sec: 85605.7, 60 sec: 78643.2, 300 sec: 77074.2). Total num frames: 192454656. Throughput: 0: 19672.5. Samples: 38078256. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:16:45,466][1157520] Avg episode reward: [(0, '51.464')] [2024-09-30 01:16:45,587][1157819] Updated weights for policy 0, policy_version 46988 (0.0006) [2024-09-30 01:16:46,052][1157819] Updated weights for policy 0, policy_version 46998 (0.0006) [2024-09-30 01:16:46,556][1157819] Updated weights for policy 0, policy_version 47008 (0.0006) [2024-09-30 01:16:47,062][1157819] Updated weights for policy 0, policy_version 47018 (0.0006) [2024-09-30 01:16:47,552][1157819] Updated weights for policy 0, policy_version 47028 (0.0006) [2024-09-30 01:16:48,046][1157819] Updated weights for policy 0, policy_version 47038 (0.0006) [2024-09-30 01:16:48,519][1157819] Updated weights for policy 0, policy_version 47048 (0.0006) [2024-09-30 01:16:49,017][1157819] Updated weights for policy 0, policy_version 47058 (0.0006) [2024-09-30 01:16:49,510][1157819] Updated weights for policy 0, policy_version 47068 (0.0006) [2024-09-30 01:16:49,992][1157819] Updated weights for policy 0, policy_version 47078 (0.0006) [2024-09-30 01:16:50,466][1157520] Fps is (10 sec: 83148.6, 60 sec: 78984.6, 300 sec: 77101.9). Total num frames: 192868352. Throughput: 0: 19981.7. Samples: 38203584. Policy #0 lag: (min: 0.0, avg: 1.7, max: 4.0) [2024-09-30 01:16:50,466][1157520] Avg episode reward: [(0, '51.596')] [2024-09-30 01:16:50,496][1157819] Updated weights for policy 0, policy_version 47088 (0.0006) [2024-09-30 01:16:50,994][1157819] Updated weights for policy 0, policy_version 47098 (0.0006) [2024-09-30 01:16:51,492][1157819] Updated weights for policy 0, policy_version 47108 (0.0006) [2024-09-30 01:16:51,972][1157819] Updated weights for policy 0, policy_version 47118 (0.0006) [2024-09-30 01:16:52,471][1157819] Updated weights for policy 0, policy_version 47128 (0.0006) [2024-09-30 01:16:52,971][1157819] Updated weights for policy 0, policy_version 47138 (0.0006) [2024-09-30 01:16:53,490][1157819] Updated weights for policy 0, policy_version 47148 (0.0006) [2024-09-30 01:16:54,015][1157819] Updated weights for policy 0, policy_version 47158 (0.0006) [2024-09-30 01:16:54,509][1157819] Updated weights for policy 0, policy_version 47168 (0.0006) [2024-09-30 01:16:55,007][1157819] Updated weights for policy 0, policy_version 47178 (0.0006) [2024-09-30 01:16:55,466][1157520] Fps is (10 sec: 81921.2, 60 sec: 79121.1, 300 sec: 77143.6). Total num frames: 193273856. Throughput: 0: 20111.7. Samples: 38265472. Policy #0 lag: (min: 0.0, avg: 1.7, max: 4.0) [2024-09-30 01:16:55,466][1157520] Avg episode reward: [(0, '53.647')] [2024-09-30 01:16:55,525][1157819] Updated weights for policy 0, policy_version 47188 (0.0006) [2024-09-30 01:16:56,037][1157819] Updated weights for policy 0, policy_version 47198 (0.0006) [2024-09-30 01:16:56,544][1157819] Updated weights for policy 0, policy_version 47208 (0.0006) [2024-09-30 01:16:57,046][1157819] Updated weights for policy 0, policy_version 47218 (0.0006) [2024-09-30 01:16:57,558][1157819] Updated weights for policy 0, policy_version 47228 (0.0006) [2024-09-30 01:16:58,121][1157819] Updated weights for policy 0, policy_version 47238 (0.0006) [2024-09-30 01:16:58,653][1157819] Updated weights for policy 0, policy_version 47248 (0.0006) [2024-09-30 01:16:59,192][1157819] Updated weights for policy 0, policy_version 47258 (0.0006) [2024-09-30 01:16:59,753][1157819] Updated weights for policy 0, policy_version 47268 (0.0006) [2024-09-30 01:17:00,292][1157819] Updated weights for policy 0, policy_version 47278 (0.0006) [2024-09-30 01:17:00,466][1157520] Fps is (10 sec: 79872.9, 60 sec: 79530.8, 300 sec: 77129.8). Total num frames: 193667072. Throughput: 0: 20371.2. Samples: 38384628. Policy #0 lag: (min: 0.0, avg: 1.7, max: 4.0) [2024-09-30 01:17:00,466][1157520] Avg episode reward: [(0, '51.644')] [2024-09-30 01:17:00,787][1157819] Updated weights for policy 0, policy_version 47288 (0.0006) [2024-09-30 01:17:01,306][1157819] Updated weights for policy 0, policy_version 47298 (0.0006) [2024-09-30 01:17:01,835][1157819] Updated weights for policy 0, policy_version 47308 (0.0006) [2024-09-30 01:17:02,335][1157819] Updated weights for policy 0, policy_version 47318 (0.0006) [2024-09-30 01:17:02,868][1157819] Updated weights for policy 0, policy_version 47328 (0.0006) [2024-09-30 01:17:03,365][1157819] Updated weights for policy 0, policy_version 47338 (0.0006) [2024-09-30 01:17:03,889][1157819] Updated weights for policy 0, policy_version 47348 (0.0006) [2024-09-30 01:17:04,367][1157819] Updated weights for policy 0, policy_version 47358 (0.0006) [2024-09-30 01:17:04,907][1157819] Updated weights for policy 0, policy_version 47368 (0.0007) [2024-09-30 01:17:05,446][1157819] Updated weights for policy 0, policy_version 47378 (0.0006) [2024-09-30 01:17:05,466][1157520] Fps is (10 sec: 78643.1, 60 sec: 79940.4, 300 sec: 77102.0). Total num frames: 194060288. Throughput: 0: 20473.8. Samples: 38502536. Policy #0 lag: (min: 0.0, avg: 1.7, max: 4.0) [2024-09-30 01:17:05,466][1157520] Avg episode reward: [(0, '48.699')] [2024-09-30 01:17:05,980][1157819] Updated weights for policy 0, policy_version 47388 (0.0006) [2024-09-30 01:17:06,518][1157819] Updated weights for policy 0, policy_version 47398 (0.0006) [2024-09-30 01:17:07,037][1157819] Updated weights for policy 0, policy_version 47408 (0.0006) [2024-09-30 01:17:07,572][1157819] Updated weights for policy 0, policy_version 47418 (0.0006) [2024-09-30 01:17:08,066][1157819] Updated weights for policy 0, policy_version 47428 (0.0006) [2024-09-30 01:17:08,570][1157819] Updated weights for policy 0, policy_version 47438 (0.0006) [2024-09-30 01:17:09,077][1157819] Updated weights for policy 0, policy_version 47448 (0.0006) [2024-09-30 01:17:09,610][1157819] Updated weights for policy 0, policy_version 47458 (0.0006) [2024-09-30 01:17:10,135][1157819] Updated weights for policy 0, policy_version 47468 (0.0006) [2024-09-30 01:17:10,466][1157520] Fps is (10 sec: 78643.0, 60 sec: 80349.9, 300 sec: 77102.0). Total num frames: 194453504. Throughput: 0: 20405.7. Samples: 38560840. Policy #0 lag: (min: 0.0, avg: 1.7, max: 4.0) [2024-09-30 01:17:10,466][1157520] Avg episode reward: [(0, '51.988')] [2024-09-30 01:17:10,644][1157819] Updated weights for policy 0, policy_version 47478 (0.0006) [2024-09-30 01:17:11,138][1157819] Updated weights for policy 0, policy_version 47488 (0.0006) [2024-09-30 01:17:11,629][1157819] Updated weights for policy 0, policy_version 47498 (0.0006) [2024-09-30 01:17:12,115][1157819] Updated weights for policy 0, policy_version 47508 (0.0006) [2024-09-30 01:17:12,585][1157819] Updated weights for policy 0, policy_version 47518 (0.0006) [2024-09-30 01:17:13,034][1157819] Updated weights for policy 0, policy_version 47528 (0.0006) [2024-09-30 01:17:13,525][1157819] Updated weights for policy 0, policy_version 47538 (0.0006) [2024-09-30 01:17:14,021][1157819] Updated weights for policy 0, policy_version 47548 (0.0007) [2024-09-30 01:17:14,517][1157819] Updated weights for policy 0, policy_version 47558 (0.0006) [2024-09-30 01:17:15,013][1157819] Updated weights for policy 0, policy_version 47568 (0.0006) [2024-09-30 01:17:15,466][1157520] Fps is (10 sec: 81510.6, 60 sec: 81305.7, 300 sec: 77199.2). Total num frames: 194875392. Throughput: 0: 20504.3. Samples: 38684240. Policy #0 lag: (min: 0.0, avg: 1.7, max: 4.0) [2024-09-30 01:17:15,466][1157520] Avg episode reward: [(0, '49.637')] [2024-09-30 01:17:15,522][1157819] Updated weights for policy 0, policy_version 47578 (0.0006) [2024-09-30 01:17:15,990][1157819] Updated weights for policy 0, policy_version 47588 (0.0006) [2024-09-30 01:17:16,483][1157819] Updated weights for policy 0, policy_version 47598 (0.0006) [2024-09-30 01:17:16,971][1157819] Updated weights for policy 0, policy_version 47608 (0.0006) [2024-09-30 01:17:17,471][1157819] Updated weights for policy 0, policy_version 47618 (0.0006) [2024-09-30 01:17:17,958][1157819] Updated weights for policy 0, policy_version 47628 (0.0006) [2024-09-30 01:17:18,443][1157819] Updated weights for policy 0, policy_version 47638 (0.0006) [2024-09-30 01:17:18,955][1157819] Updated weights for policy 0, policy_version 47648 (0.0006) [2024-09-30 01:17:19,425][1157819] Updated weights for policy 0, policy_version 47658 (0.0006) [2024-09-30 01:17:19,914][1157819] Updated weights for policy 0, policy_version 47668 (0.0006) [2024-09-30 01:17:20,428][1157819] Updated weights for policy 0, policy_version 47678 (0.0006) [2024-09-30 01:17:20,466][1157520] Fps is (10 sec: 83966.6, 60 sec: 81920.0, 300 sec: 77310.2). Total num frames: 195293184. Throughput: 0: 20536.2. Samples: 38809240. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:17:20,466][1157520] Avg episode reward: [(0, '53.869')] [2024-09-30 01:17:20,922][1157819] Updated weights for policy 0, policy_version 47688 (0.0006) [2024-09-30 01:17:21,408][1157819] Updated weights for policy 0, policy_version 47698 (0.0006) [2024-09-30 01:17:21,908][1157819] Updated weights for policy 0, policy_version 47708 (0.0006) [2024-09-30 01:17:22,409][1157819] Updated weights for policy 0, policy_version 47718 (0.0006) [2024-09-30 01:17:22,929][1157819] Updated weights for policy 0, policy_version 47728 (0.0006) [2024-09-30 01:17:23,460][1157819] Updated weights for policy 0, policy_version 47738 (0.0006) [2024-09-30 01:17:23,960][1157819] Updated weights for policy 0, policy_version 47748 (0.0006) [2024-09-30 01:17:24,449][1157819] Updated weights for policy 0, policy_version 47758 (0.0006) [2024-09-30 01:17:24,968][1157819] Updated weights for policy 0, policy_version 47768 (0.0006) [2024-09-30 01:17:25,083][1157736] Signal inference workers to stop experience collection... (2800 times) [2024-09-30 01:17:25,087][1157819] InferenceWorker_p0-w0: stopping experience collection (2800 times) [2024-09-30 01:17:25,092][1157736] Signal inference workers to resume experience collection... (2800 times) [2024-09-30 01:17:25,092][1157819] InferenceWorker_p0-w0: resuming experience collection (2800 times) [2024-09-30 01:17:25,464][1157819] Updated weights for policy 0, policy_version 47778 (0.0006) [2024-09-30 01:17:25,466][1157520] Fps is (10 sec: 82329.5, 60 sec: 81988.3, 300 sec: 77351.9). Total num frames: 195698688. Throughput: 0: 20441.0. Samples: 38870920. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:17:25,466][1157520] Avg episode reward: [(0, '51.307')] [2024-09-30 01:17:25,951][1157819] Updated weights for policy 0, policy_version 47788 (0.0006) [2024-09-30 01:17:26,466][1157819] Updated weights for policy 0, policy_version 47798 (0.0006) [2024-09-30 01:17:26,985][1157819] Updated weights for policy 0, policy_version 47808 (0.0006) [2024-09-30 01:17:27,508][1157819] Updated weights for policy 0, policy_version 47818 (0.0006) [2024-09-30 01:17:28,037][1157819] Updated weights for policy 0, policy_version 47828 (0.0006) [2024-09-30 01:17:28,550][1157819] Updated weights for policy 0, policy_version 47838 (0.0006) [2024-09-30 01:17:29,074][1157819] Updated weights for policy 0, policy_version 47848 (0.0006) [2024-09-30 01:17:29,612][1157819] Updated weights for policy 0, policy_version 47858 (0.0006) [2024-09-30 01:17:30,186][1157819] Updated weights for policy 0, policy_version 47868 (0.0006) [2024-09-30 01:17:30,466][1157520] Fps is (10 sec: 79463.7, 60 sec: 81851.8, 300 sec: 77393.6). Total num frames: 196087808. Throughput: 0: 20277.9. Samples: 38990756. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:17:30,466][1157520] Avg episode reward: [(0, '54.023')] [2024-09-30 01:17:30,709][1157819] Updated weights for policy 0, policy_version 47878 (0.0006) [2024-09-30 01:17:31,227][1157819] Updated weights for policy 0, policy_version 47888 (0.0006) [2024-09-30 01:17:31,731][1157819] Updated weights for policy 0, policy_version 47898 (0.0006) [2024-09-30 01:17:32,252][1157819] Updated weights for policy 0, policy_version 47908 (0.0006) [2024-09-30 01:17:32,793][1157819] Updated weights for policy 0, policy_version 47918 (0.0006) [2024-09-30 01:17:33,320][1157819] Updated weights for policy 0, policy_version 47928 (0.0006) [2024-09-30 01:17:33,823][1157819] Updated weights for policy 0, policy_version 47938 (0.0006) [2024-09-30 01:17:34,322][1157819] Updated weights for policy 0, policy_version 47948 (0.0006) [2024-09-30 01:17:34,862][1157819] Updated weights for policy 0, policy_version 47958 (0.0006) [2024-09-30 01:17:35,355][1157819] Updated weights for policy 0, policy_version 47968 (0.0006) [2024-09-30 01:17:35,466][1157520] Fps is (10 sec: 78643.1, 60 sec: 81442.2, 300 sec: 77421.3). Total num frames: 196485120. Throughput: 0: 20096.1. Samples: 39107908. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:17:35,466][1157520] Avg episode reward: [(0, '51.862')] [2024-09-30 01:17:35,884][1157819] Updated weights for policy 0, policy_version 47978 (0.0006) [2024-09-30 01:17:36,389][1157819] Updated weights for policy 0, policy_version 47988 (0.0006) [2024-09-30 01:17:36,899][1157819] Updated weights for policy 0, policy_version 47998 (0.0006) [2024-09-30 01:17:37,436][1157819] Updated weights for policy 0, policy_version 48008 (0.0006) [2024-09-30 01:17:37,971][1157819] Updated weights for policy 0, policy_version 48018 (0.0006) [2024-09-30 01:17:38,505][1157819] Updated weights for policy 0, policy_version 48028 (0.0006) [2024-09-30 01:17:39,057][1157819] Updated weights for policy 0, policy_version 48038 (0.0006) [2024-09-30 01:17:39,584][1157819] Updated weights for policy 0, policy_version 48048 (0.0006) [2024-09-30 01:17:40,121][1157819] Updated weights for policy 0, policy_version 48058 (0.0006) [2024-09-30 01:17:40,466][1157520] Fps is (10 sec: 78233.5, 60 sec: 80554.7, 300 sec: 77379.7). Total num frames: 196870144. Throughput: 0: 20033.6. Samples: 39166984. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:17:40,466][1157520] Avg episode reward: [(0, '51.752')] [2024-09-30 01:17:40,656][1157819] Updated weights for policy 0, policy_version 48068 (0.0006) [2024-09-30 01:17:41,198][1157819] Updated weights for policy 0, policy_version 48078 (0.0006) [2024-09-30 01:17:41,743][1157819] Updated weights for policy 0, policy_version 48088 (0.0006) [2024-09-30 01:17:42,279][1157819] Updated weights for policy 0, policy_version 48098 (0.0006) [2024-09-30 01:17:42,805][1157819] Updated weights for policy 0, policy_version 48108 (0.0006) [2024-09-30 01:17:43,328][1157819] Updated weights for policy 0, policy_version 48118 (0.0006) [2024-09-30 01:17:43,871][1157819] Updated weights for policy 0, policy_version 48128 (0.0006) [2024-09-30 01:17:44,404][1157819] Updated weights for policy 0, policy_version 48138 (0.0006) [2024-09-30 01:17:44,945][1157819] Updated weights for policy 0, policy_version 48148 (0.0006) [2024-09-30 01:17:45,466][1157520] Fps is (10 sec: 76595.2, 60 sec: 79940.4, 300 sec: 77379.7). Total num frames: 197251072. Throughput: 0: 19924.5. Samples: 39281232. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:17:45,466][1157520] Avg episode reward: [(0, '54.004')] [2024-09-30 01:17:45,483][1157819] Updated weights for policy 0, policy_version 48158 (0.0006) [2024-09-30 01:17:45,965][1157819] Updated weights for policy 0, policy_version 48168 (0.0006) [2024-09-30 01:17:46,497][1157819] Updated weights for policy 0, policy_version 48178 (0.0006) [2024-09-30 01:17:47,001][1157819] Updated weights for policy 0, policy_version 48188 (0.0006) [2024-09-30 01:17:47,503][1157819] Updated weights for policy 0, policy_version 48198 (0.0006) [2024-09-30 01:17:48,010][1157819] Updated weights for policy 0, policy_version 48208 (0.0006) [2024-09-30 01:17:48,515][1157819] Updated weights for policy 0, policy_version 48218 (0.0006) [2024-09-30 01:17:49,094][1157819] Updated weights for policy 0, policy_version 48228 (0.0006) [2024-09-30 01:17:49,617][1157819] Updated weights for policy 0, policy_version 48238 (0.0006) [2024-09-30 01:17:50,138][1157819] Updated weights for policy 0, policy_version 48248 (0.0006) [2024-09-30 01:17:50,466][1157520] Fps is (10 sec: 77823.5, 60 sec: 79667.2, 300 sec: 77393.5). Total num frames: 197648384. Throughput: 0: 19922.3. Samples: 39399040. Policy #0 lag: (min: 0.0, avg: 2.0, max: 4.0) [2024-09-30 01:17:50,466][1157520] Avg episode reward: [(0, '53.099')] [2024-09-30 01:17:50,679][1157819] Updated weights for policy 0, policy_version 48258 (0.0006) [2024-09-30 01:17:51,217][1157819] Updated weights for policy 0, policy_version 48268 (0.0006) [2024-09-30 01:17:51,746][1157819] Updated weights for policy 0, policy_version 48278 (0.0006) [2024-09-30 01:17:52,301][1157819] Updated weights for policy 0, policy_version 48288 (0.0006) [2024-09-30 01:17:52,862][1157819] Updated weights for policy 0, policy_version 48298 (0.0006) [2024-09-30 01:17:53,357][1157819] Updated weights for policy 0, policy_version 48308 (0.0006) [2024-09-30 01:17:53,870][1157819] Updated weights for policy 0, policy_version 48318 (0.0006) [2024-09-30 01:17:54,413][1157819] Updated weights for policy 0, policy_version 48328 (0.0006) [2024-09-30 01:17:54,921][1157819] Updated weights for policy 0, policy_version 48338 (0.0006) [2024-09-30 01:17:55,448][1157819] Updated weights for policy 0, policy_version 48348 (0.0006) [2024-09-30 01:17:55,466][1157520] Fps is (10 sec: 78233.3, 60 sec: 79325.8, 300 sec: 77268.6). Total num frames: 198033408. Throughput: 0: 19903.9. Samples: 39456516. Policy #0 lag: (min: 0.0, avg: 2.0, max: 4.0) [2024-09-30 01:17:55,466][1157520] Avg episode reward: [(0, '52.635')] [2024-09-30 01:17:55,965][1157819] Updated weights for policy 0, policy_version 48358 (0.0006) [2024-09-30 01:17:56,493][1157819] Updated weights for policy 0, policy_version 48368 (0.0006) [2024-09-30 01:17:57,010][1157819] Updated weights for policy 0, policy_version 48378 (0.0006) [2024-09-30 01:17:57,522][1157819] Updated weights for policy 0, policy_version 48388 (0.0006) [2024-09-30 01:17:58,049][1157819] Updated weights for policy 0, policy_version 48398 (0.0006) [2024-09-30 01:17:58,572][1157819] Updated weights for policy 0, policy_version 48408 (0.0006) [2024-09-30 01:17:59,128][1157819] Updated weights for policy 0, policy_version 48418 (0.0006) [2024-09-30 01:17:59,611][1157819] Updated weights for policy 0, policy_version 48428 (0.0006) [2024-09-30 01:18:00,115][1157819] Updated weights for policy 0, policy_version 48438 (0.0006) [2024-09-30 01:18:00,466][1157520] Fps is (10 sec: 77824.4, 60 sec: 79325.8, 300 sec: 77074.2). Total num frames: 198426624. Throughput: 0: 19772.3. Samples: 39573992. Policy #0 lag: (min: 0.0, avg: 2.0, max: 4.0) [2024-09-30 01:18:00,466][1157520] Avg episode reward: [(0, '54.157')] [2024-09-30 01:18:00,642][1157819] Updated weights for policy 0, policy_version 48448 (0.0006) [2024-09-30 01:18:00,900][1157736] Signal inference workers to stop experience collection... (2850 times) [2024-09-30 01:18:00,901][1157736] Signal inference workers to resume experience collection... (2850 times) [2024-09-30 01:18:00,905][1157819] InferenceWorker_p0-w0: stopping experience collection (2850 times) [2024-09-30 01:18:00,907][1157819] InferenceWorker_p0-w0: resuming experience collection (2850 times) [2024-09-30 01:18:01,215][1157819] Updated weights for policy 0, policy_version 48458 (0.0006) [2024-09-30 01:18:01,724][1157819] Updated weights for policy 0, policy_version 48468 (0.0006) [2024-09-30 01:18:02,234][1157819] Updated weights for policy 0, policy_version 48478 (0.0006) [2024-09-30 01:18:02,744][1157819] Updated weights for policy 0, policy_version 48488 (0.0006) [2024-09-30 01:18:03,265][1157819] Updated weights for policy 0, policy_version 48498 (0.0006) [2024-09-30 01:18:03,802][1157819] Updated weights for policy 0, policy_version 48508 (0.0006) [2024-09-30 01:18:04,336][1157819] Updated weights for policy 0, policy_version 48518 (0.0006) [2024-09-30 01:18:04,897][1157819] Updated weights for policy 0, policy_version 48528 (0.0006) [2024-09-30 01:18:05,466][1157520] Fps is (10 sec: 77414.4, 60 sec: 79121.0, 300 sec: 76893.7). Total num frames: 198807552. Throughput: 0: 19591.5. Samples: 39690856. Policy #0 lag: (min: 0.0, avg: 2.0, max: 4.0) [2024-09-30 01:18:05,466][1157520] Avg episode reward: [(0, '51.514')] [2024-09-30 01:18:05,487][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000048538_198811648.pth... [2024-09-30 01:18:05,488][1157819] Updated weights for policy 0, policy_version 48538 (0.0006) [2024-09-30 01:18:05,551][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000044040_180387840.pth [2024-09-30 01:18:06,056][1157819] Updated weights for policy 0, policy_version 48548 (0.0006) [2024-09-30 01:18:06,607][1157819] Updated weights for policy 0, policy_version 48558 (0.0006) [2024-09-30 01:18:07,125][1157819] Updated weights for policy 0, policy_version 48568 (0.0006) [2024-09-30 01:18:07,639][1157819] Updated weights for policy 0, policy_version 48578 (0.0006) [2024-09-30 01:18:08,168][1157819] Updated weights for policy 0, policy_version 48588 (0.0006) [2024-09-30 01:18:08,721][1157819] Updated weights for policy 0, policy_version 48598 (0.0006) [2024-09-30 01:18:09,236][1157819] Updated weights for policy 0, policy_version 48608 (0.0006) [2024-09-30 01:18:09,741][1157819] Updated weights for policy 0, policy_version 48618 (0.0006) [2024-09-30 01:18:10,279][1157819] Updated weights for policy 0, policy_version 48628 (0.0006) [2024-09-30 01:18:10,466][1157520] Fps is (10 sec: 76596.2, 60 sec: 78984.6, 300 sec: 76754.9). Total num frames: 199192576. Throughput: 0: 19463.2. Samples: 39746760. Policy #0 lag: (min: 0.0, avg: 2.0, max: 4.0) [2024-09-30 01:18:10,466][1157520] Avg episode reward: [(0, '53.530')] [2024-09-30 01:18:10,798][1157819] Updated weights for policy 0, policy_version 48638 (0.0006) [2024-09-30 01:18:11,352][1157819] Updated weights for policy 0, policy_version 48648 (0.0006) [2024-09-30 01:18:11,887][1157819] Updated weights for policy 0, policy_version 48658 (0.0006) [2024-09-30 01:18:12,389][1157819] Updated weights for policy 0, policy_version 48668 (0.0006) [2024-09-30 01:18:12,888][1157819] Updated weights for policy 0, policy_version 48678 (0.0006) [2024-09-30 01:18:13,434][1157819] Updated weights for policy 0, policy_version 48688 (0.0006) [2024-09-30 01:18:13,951][1157819] Updated weights for policy 0, policy_version 48698 (0.0007) [2024-09-30 01:18:14,463][1157819] Updated weights for policy 0, policy_version 48708 (0.0006) [2024-09-30 01:18:14,979][1157819] Updated weights for policy 0, policy_version 48718 (0.0006) [2024-09-30 01:18:15,466][1157520] Fps is (10 sec: 77825.1, 60 sec: 78506.8, 300 sec: 76643.8). Total num frames: 199585792. Throughput: 0: 19395.8. Samples: 39863564. Policy #0 lag: (min: 0.0, avg: 2.0, max: 4.0) [2024-09-30 01:18:15,466][1157520] Avg episode reward: [(0, '52.360')] [2024-09-30 01:18:15,468][1157819] Updated weights for policy 0, policy_version 48728 (0.0006) [2024-09-30 01:18:15,957][1157819] Updated weights for policy 0, policy_version 48738 (0.0006) [2024-09-30 01:18:16,452][1157819] Updated weights for policy 0, policy_version 48748 (0.0006) [2024-09-30 01:18:16,958][1157819] Updated weights for policy 0, policy_version 48758 (0.0006) [2024-09-30 01:18:17,462][1157819] Updated weights for policy 0, policy_version 48768 (0.0006) [2024-09-30 01:18:17,962][1157819] Updated weights for policy 0, policy_version 48778 (0.0006) [2024-09-30 01:18:18,468][1157819] Updated weights for policy 0, policy_version 48788 (0.0006) [2024-09-30 01:18:18,987][1157819] Updated weights for policy 0, policy_version 48798 (0.0006) [2024-09-30 01:18:19,477][1157819] Updated weights for policy 0, policy_version 48808 (0.0006) [2024-09-30 01:18:20,004][1157819] Updated weights for policy 0, policy_version 48818 (0.0006) [2024-09-30 01:18:20,466][1157520] Fps is (10 sec: 80280.7, 60 sec: 78370.3, 300 sec: 76671.5). Total num frames: 199995392. Throughput: 0: 19500.1. Samples: 39985412. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 01:18:20,466][1157520] Avg episode reward: [(0, '51.204')] [2024-09-30 01:18:20,508][1157819] Updated weights for policy 0, policy_version 48828 (0.0006) [2024-09-30 01:18:20,973][1157819] Updated weights for policy 0, policy_version 48838 (0.0006) [2024-09-30 01:18:21,451][1157819] Updated weights for policy 0, policy_version 48848 (0.0006) [2024-09-30 01:18:21,949][1157819] Updated weights for policy 0, policy_version 48858 (0.0006) [2024-09-30 01:18:22,411][1157819] Updated weights for policy 0, policy_version 48868 (0.0006) [2024-09-30 01:18:22,918][1157819] Updated weights for policy 0, policy_version 48878 (0.0006) [2024-09-30 01:18:23,452][1157819] Updated weights for policy 0, policy_version 48888 (0.0006) [2024-09-30 01:18:23,985][1157819] Updated weights for policy 0, policy_version 48898 (0.0006) [2024-09-30 01:18:24,524][1157819] Updated weights for policy 0, policy_version 48908 (0.0006) [2024-09-30 01:18:25,046][1157819] Updated weights for policy 0, policy_version 48918 (0.0006) [2024-09-30 01:18:25,466][1157520] Fps is (10 sec: 81099.9, 60 sec: 78301.8, 300 sec: 76727.1). Total num frames: 200396800. Throughput: 0: 19583.6. Samples: 40048248. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 01:18:25,466][1157520] Avg episode reward: [(0, '51.927')] [2024-09-30 01:18:25,578][1157819] Updated weights for policy 0, policy_version 48928 (0.0006) [2024-09-30 01:18:26,142][1157819] Updated weights for policy 0, policy_version 48938 (0.0006) [2024-09-30 01:18:26,675][1157819] Updated weights for policy 0, policy_version 48948 (0.0006) [2024-09-30 01:18:27,260][1157819] Updated weights for policy 0, policy_version 48958 (0.0006) [2024-09-30 01:18:27,808][1157819] Updated weights for policy 0, policy_version 48968 (0.0006) [2024-09-30 01:18:28,392][1157819] Updated weights for policy 0, policy_version 48978 (0.0006) [2024-09-30 01:18:28,946][1157819] Updated weights for policy 0, policy_version 48988 (0.0006) [2024-09-30 01:18:29,491][1157819] Updated weights for policy 0, policy_version 48998 (0.0006) [2024-09-30 01:18:30,041][1157819] Updated weights for policy 0, policy_version 49008 (0.0006) [2024-09-30 01:18:30,466][1157520] Fps is (10 sec: 77414.7, 60 sec: 78028.8, 300 sec: 76713.2). Total num frames: 200769536. Throughput: 0: 19559.1. Samples: 40161392. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 01:18:30,466][1157520] Avg episode reward: [(0, '51.926')] [2024-09-30 01:18:30,557][1157819] Updated weights for policy 0, policy_version 49018 (0.0006) [2024-09-30 01:18:31,062][1157819] Updated weights for policy 0, policy_version 49028 (0.0006) [2024-09-30 01:18:31,594][1157819] Updated weights for policy 0, policy_version 49038 (0.0006) [2024-09-30 01:18:32,119][1157819] Updated weights for policy 0, policy_version 49048 (0.0006) [2024-09-30 01:18:32,619][1157819] Updated weights for policy 0, policy_version 49058 (0.0006) [2024-09-30 01:18:33,121][1157736] Signal inference workers to stop experience collection... (2900 times) [2024-09-30 01:18:33,126][1157819] InferenceWorker_p0-w0: stopping experience collection (2900 times) [2024-09-30 01:18:33,126][1157736] Signal inference workers to resume experience collection... (2900 times) [2024-09-30 01:18:33,127][1157819] Updated weights for policy 0, policy_version 49068 (0.0006) [2024-09-30 01:18:33,130][1157819] InferenceWorker_p0-w0: resuming experience collection (2900 times) [2024-09-30 01:18:33,625][1157819] Updated weights for policy 0, policy_version 49078 (0.0006) [2024-09-30 01:18:34,166][1157819] Updated weights for policy 0, policy_version 49088 (0.0006) [2024-09-30 01:18:34,734][1157819] Updated weights for policy 0, policy_version 49098 (0.0006) [2024-09-30 01:18:35,304][1157819] Updated weights for policy 0, policy_version 49108 (0.0006) [2024-09-30 01:18:35,466][1157520] Fps is (10 sec: 76185.8, 60 sec: 77892.3, 300 sec: 76727.1). Total num frames: 201158656. Throughput: 0: 19527.6. Samples: 40277780. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 01:18:35,466][1157520] Avg episode reward: [(0, '53.571')] [2024-09-30 01:18:35,866][1157819] Updated weights for policy 0, policy_version 49118 (0.0006) [2024-09-30 01:18:36,390][1157819] Updated weights for policy 0, policy_version 49128 (0.0006) [2024-09-30 01:18:37,005][1157819] Updated weights for policy 0, policy_version 49138 (0.0006) [2024-09-30 01:18:37,576][1157819] Updated weights for policy 0, policy_version 49148 (0.0006) [2024-09-30 01:18:38,132][1157819] Updated weights for policy 0, policy_version 49158 (0.0006) [2024-09-30 01:18:38,679][1157819] Updated weights for policy 0, policy_version 49168 (0.0006) [2024-09-30 01:18:39,201][1157819] Updated weights for policy 0, policy_version 49178 (0.0006) [2024-09-30 01:18:39,755][1157819] Updated weights for policy 0, policy_version 49188 (0.0006) [2024-09-30 01:18:40,293][1157819] Updated weights for policy 0, policy_version 49198 (0.0006) [2024-09-30 01:18:40,466][1157520] Fps is (10 sec: 75366.2, 60 sec: 77551.0, 300 sec: 76727.1). Total num frames: 201523200. Throughput: 0: 19443.9. Samples: 40331492. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 01:18:40,466][1157520] Avg episode reward: [(0, '52.321')] [2024-09-30 01:18:40,826][1157819] Updated weights for policy 0, policy_version 49208 (0.0006) [2024-09-30 01:18:41,395][1157819] Updated weights for policy 0, policy_version 49218 (0.0006) [2024-09-30 01:18:41,923][1157819] Updated weights for policy 0, policy_version 49228 (0.0007) [2024-09-30 01:18:42,526][1157819] Updated weights for policy 0, policy_version 49238 (0.0006) [2024-09-30 01:18:43,080][1157819] Updated weights for policy 0, policy_version 49248 (0.0006) [2024-09-30 01:18:43,622][1157819] Updated weights for policy 0, policy_version 49258 (0.0006) [2024-09-30 01:18:44,182][1157819] Updated weights for policy 0, policy_version 49268 (0.0006) [2024-09-30 01:18:44,699][1157819] Updated weights for policy 0, policy_version 49278 (0.0006) [2024-09-30 01:18:45,233][1157819] Updated weights for policy 0, policy_version 49288 (0.0006) [2024-09-30 01:18:45,466][1157520] Fps is (10 sec: 74137.6, 60 sec: 77482.7, 300 sec: 76768.7). Total num frames: 201900032. Throughput: 0: 19322.0. Samples: 40443480. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 01:18:45,466][1157520] Avg episode reward: [(0, '49.871')] [2024-09-30 01:18:45,782][1157819] Updated weights for policy 0, policy_version 49298 (0.0006) [2024-09-30 01:18:46,324][1157819] Updated weights for policy 0, policy_version 49308 (0.0006) [2024-09-30 01:18:46,828][1157819] Updated weights for policy 0, policy_version 49318 (0.0006) [2024-09-30 01:18:47,352][1157819] Updated weights for policy 0, policy_version 49328 (0.0006) [2024-09-30 01:18:47,834][1157819] Updated weights for policy 0, policy_version 49338 (0.0006) [2024-09-30 01:18:48,344][1157819] Updated weights for policy 0, policy_version 49348 (0.0006) [2024-09-30 01:18:48,833][1157819] Updated weights for policy 0, policy_version 49358 (0.0006) [2024-09-30 01:18:49,366][1157819] Updated weights for policy 0, policy_version 49368 (0.0006) [2024-09-30 01:18:49,859][1157819] Updated weights for policy 0, policy_version 49378 (0.0006) [2024-09-30 01:18:50,380][1157819] Updated weights for policy 0, policy_version 49388 (0.0006) [2024-09-30 01:18:50,466][1157520] Fps is (10 sec: 77414.2, 60 sec: 77482.8, 300 sec: 76824.3). Total num frames: 202297344. Throughput: 0: 19349.4. Samples: 40561580. Policy #0 lag: (min: 0.0, avg: 2.5, max: 7.0) [2024-09-30 01:18:50,466][1157520] Avg episode reward: [(0, '54.263')] [2024-09-30 01:18:50,875][1157819] Updated weights for policy 0, policy_version 49398 (0.0006) [2024-09-30 01:18:51,369][1157819] Updated weights for policy 0, policy_version 49408 (0.0006) [2024-09-30 01:18:51,853][1157819] Updated weights for policy 0, policy_version 49418 (0.0006) [2024-09-30 01:18:52,349][1157819] Updated weights for policy 0, policy_version 49428 (0.0006) [2024-09-30 01:18:52,825][1157819] Updated weights for policy 0, policy_version 49438 (0.0006) [2024-09-30 01:18:53,311][1157819] Updated weights for policy 0, policy_version 49448 (0.0006) [2024-09-30 01:18:53,799][1157819] Updated weights for policy 0, policy_version 49458 (0.0006) [2024-09-30 01:18:54,278][1157819] Updated weights for policy 0, policy_version 49468 (0.0006) [2024-09-30 01:18:54,764][1157819] Updated weights for policy 0, policy_version 49478 (0.0006) [2024-09-30 01:18:55,244][1157819] Updated weights for policy 0, policy_version 49488 (0.0006) [2024-09-30 01:18:55,466][1157520] Fps is (10 sec: 81920.1, 60 sec: 78097.2, 300 sec: 76963.1). Total num frames: 202719232. Throughput: 0: 19482.6. Samples: 40623480. Policy #0 lag: (min: 0.0, avg: 2.5, max: 7.0) [2024-09-30 01:18:55,466][1157520] Avg episode reward: [(0, '52.239')] [2024-09-30 01:18:55,728][1157819] Updated weights for policy 0, policy_version 49498 (0.0006) [2024-09-30 01:18:56,201][1157819] Updated weights for policy 0, policy_version 49508 (0.0006) [2024-09-30 01:18:56,711][1157819] Updated weights for policy 0, policy_version 49518 (0.0006) [2024-09-30 01:18:57,232][1157819] Updated weights for policy 0, policy_version 49528 (0.0006) [2024-09-30 01:18:57,741][1157819] Updated weights for policy 0, policy_version 49538 (0.0006) [2024-09-30 01:18:58,238][1157819] Updated weights for policy 0, policy_version 49548 (0.0006) [2024-09-30 01:18:58,754][1157819] Updated weights for policy 0, policy_version 49558 (0.0006) [2024-09-30 01:18:59,292][1157819] Updated weights for policy 0, policy_version 49568 (0.0006) [2024-09-30 01:18:59,833][1157819] Updated weights for policy 0, policy_version 49578 (0.0006) [2024-09-30 01:19:00,344][1157819] Updated weights for policy 0, policy_version 49588 (0.0006) [2024-09-30 01:19:00,466][1157520] Fps is (10 sec: 82330.0, 60 sec: 78233.7, 300 sec: 77060.3). Total num frames: 203120640. Throughput: 0: 19647.4. Samples: 40747700. Policy #0 lag: (min: 0.0, avg: 2.5, max: 7.0) [2024-09-30 01:19:00,466][1157520] Avg episode reward: [(0, '52.428')] [2024-09-30 01:19:00,846][1157819] Updated weights for policy 0, policy_version 49598 (0.0006) [2024-09-30 01:19:01,352][1157819] Updated weights for policy 0, policy_version 49608 (0.0006) [2024-09-30 01:19:01,894][1157819] Updated weights for policy 0, policy_version 49618 (0.0006) [2024-09-30 01:19:02,451][1157819] Updated weights for policy 0, policy_version 49628 (0.0006) [2024-09-30 01:19:02,962][1157819] Updated weights for policy 0, policy_version 49638 (0.0006) [2024-09-30 01:19:03,493][1157819] Updated weights for policy 0, policy_version 49648 (0.0006) [2024-09-30 01:19:03,993][1157819] Updated weights for policy 0, policy_version 49658 (0.0006) [2024-09-30 01:19:04,558][1157819] Updated weights for policy 0, policy_version 49668 (0.0006) [2024-09-30 01:19:05,166][1157819] Updated weights for policy 0, policy_version 49678 (0.0006) [2024-09-30 01:19:05,466][1157520] Fps is (10 sec: 78233.5, 60 sec: 78233.7, 300 sec: 77115.9). Total num frames: 203501568. Throughput: 0: 19517.6. Samples: 40863704. Policy #0 lag: (min: 0.0, avg: 2.5, max: 7.0) [2024-09-30 01:19:05,466][1157520] Avg episode reward: [(0, '51.568')] [2024-09-30 01:19:05,738][1157819] Updated weights for policy 0, policy_version 49688 (0.0006) [2024-09-30 01:19:06,339][1157736] Signal inference workers to stop experience collection... (2950 times) [2024-09-30 01:19:06,341][1157736] Signal inference workers to resume experience collection... (2950 times) [2024-09-30 01:19:06,341][1157819] Updated weights for policy 0, policy_version 49698 (0.0006) [2024-09-30 01:19:06,344][1157819] InferenceWorker_p0-w0: stopping experience collection (2950 times) [2024-09-30 01:19:06,346][1157819] InferenceWorker_p0-w0: resuming experience collection (2950 times) [2024-09-30 01:19:06,904][1157819] Updated weights for policy 0, policy_version 49708 (0.0006) [2024-09-30 01:19:07,509][1157819] Updated weights for policy 0, policy_version 49718 (0.0006) [2024-09-30 01:19:08,037][1157819] Updated weights for policy 0, policy_version 49728 (0.0006) [2024-09-30 01:19:08,618][1157819] Updated weights for policy 0, policy_version 49738 (0.0006) [2024-09-30 01:19:09,140][1157819] Updated weights for policy 0, policy_version 49748 (0.0006) [2024-09-30 01:19:09,692][1157819] Updated weights for policy 0, policy_version 49758 (0.0006) [2024-09-30 01:19:10,251][1157819] Updated weights for policy 0, policy_version 49768 (0.0006) [2024-09-30 01:19:10,466][1157520] Fps is (10 sec: 74546.8, 60 sec: 77892.1, 300 sec: 77129.7). Total num frames: 203866112. Throughput: 0: 19296.1. Samples: 40916572. Policy #0 lag: (min: 0.0, avg: 2.5, max: 7.0) [2024-09-30 01:19:10,466][1157520] Avg episode reward: [(0, '54.025')] [2024-09-30 01:19:10,787][1157819] Updated weights for policy 0, policy_version 49778 (0.0006) [2024-09-30 01:19:11,342][1157819] Updated weights for policy 0, policy_version 49788 (0.0006) [2024-09-30 01:19:11,836][1157819] Updated weights for policy 0, policy_version 49798 (0.0006) [2024-09-30 01:19:12,362][1157819] Updated weights for policy 0, policy_version 49808 (0.0006) [2024-09-30 01:19:12,934][1157819] Updated weights for policy 0, policy_version 49818 (0.0006) [2024-09-30 01:19:13,456][1157819] Updated weights for policy 0, policy_version 49828 (0.0006) [2024-09-30 01:19:13,976][1157819] Updated weights for policy 0, policy_version 49838 (0.0006) [2024-09-30 01:19:14,523][1157819] Updated weights for policy 0, policy_version 49848 (0.0006) [2024-09-30 01:19:15,058][1157819] Updated weights for policy 0, policy_version 49858 (0.0006) [2024-09-30 01:19:15,466][1157520] Fps is (10 sec: 74547.2, 60 sec: 77687.3, 300 sec: 77185.3). Total num frames: 204247040. Throughput: 0: 19309.4. Samples: 41030316. Policy #0 lag: (min: 0.0, avg: 2.5, max: 7.0) [2024-09-30 01:19:15,466][1157520] Avg episode reward: [(0, '52.231')] [2024-09-30 01:19:15,630][1157819] Updated weights for policy 0, policy_version 49868 (0.0006) [2024-09-30 01:19:16,189][1157819] Updated weights for policy 0, policy_version 49878 (0.0006) [2024-09-30 01:19:16,720][1157819] Updated weights for policy 0, policy_version 49888 (0.0006) [2024-09-30 01:19:17,242][1157819] Updated weights for policy 0, policy_version 49898 (0.0006) [2024-09-30 01:19:17,760][1157819] Updated weights for policy 0, policy_version 49908 (0.0006) [2024-09-30 01:19:18,307][1157819] Updated weights for policy 0, policy_version 49918 (0.0006) [2024-09-30 01:19:18,820][1157819] Updated weights for policy 0, policy_version 49928 (0.0006) [2024-09-30 01:19:19,323][1157819] Updated weights for policy 0, policy_version 49938 (0.0006) [2024-09-30 01:19:19,840][1157819] Updated weights for policy 0, policy_version 49948 (0.0006) [2024-09-30 01:19:20,367][1157819] Updated weights for policy 0, policy_version 49958 (0.0006) [2024-09-30 01:19:20,466][1157520] Fps is (10 sec: 77004.7, 60 sec: 77346.1, 300 sec: 77338.0). Total num frames: 204636160. Throughput: 0: 19282.5. Samples: 41145492. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 01:19:20,466][1157520] Avg episode reward: [(0, '50.262')] [2024-09-30 01:19:20,913][1157819] Updated weights for policy 0, policy_version 49968 (0.0006) [2024-09-30 01:19:21,435][1157819] Updated weights for policy 0, policy_version 49978 (0.0006) [2024-09-30 01:19:21,926][1157819] Updated weights for policy 0, policy_version 49988 (0.0006) [2024-09-30 01:19:22,474][1157819] Updated weights for policy 0, policy_version 49998 (0.0006) [2024-09-30 01:19:22,977][1157819] Updated weights for policy 0, policy_version 50008 (0.0006) [2024-09-30 01:19:23,509][1157819] Updated weights for policy 0, policy_version 50018 (0.0006) [2024-09-30 01:19:24,030][1157819] Updated weights for policy 0, policy_version 50028 (0.0006) [2024-09-30 01:19:24,532][1157819] Updated weights for policy 0, policy_version 50038 (0.0006) [2024-09-30 01:19:25,069][1157819] Updated weights for policy 0, policy_version 50048 (0.0006) [2024-09-30 01:19:25,466][1157520] Fps is (10 sec: 77824.1, 60 sec: 77141.4, 300 sec: 77490.7). Total num frames: 205025280. Throughput: 0: 19398.3. Samples: 41204416. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 01:19:25,466][1157520] Avg episode reward: [(0, '50.267')] [2024-09-30 01:19:25,609][1157819] Updated weights for policy 0, policy_version 50058 (0.0006) [2024-09-30 01:19:26,140][1157819] Updated weights for policy 0, policy_version 50068 (0.0006) [2024-09-30 01:19:26,704][1157819] Updated weights for policy 0, policy_version 50078 (0.0006) [2024-09-30 01:19:27,235][1157819] Updated weights for policy 0, policy_version 50088 (0.0006) [2024-09-30 01:19:27,769][1157819] Updated weights for policy 0, policy_version 50098 (0.0006) [2024-09-30 01:19:28,329][1157819] Updated weights for policy 0, policy_version 50108 (0.0006) [2024-09-30 01:19:28,854][1157819] Updated weights for policy 0, policy_version 50118 (0.0006) [2024-09-30 01:19:29,416][1157819] Updated weights for policy 0, policy_version 50128 (0.0006) [2024-09-30 01:19:29,978][1157819] Updated weights for policy 0, policy_version 50138 (0.0006) [2024-09-30 01:19:30,466][1157520] Fps is (10 sec: 76185.2, 60 sec: 77141.2, 300 sec: 77560.1). Total num frames: 205398016. Throughput: 0: 19458.7. Samples: 41319124. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 01:19:30,466][1157520] Avg episode reward: [(0, '49.841')] [2024-09-30 01:19:30,525][1157819] Updated weights for policy 0, policy_version 50148 (0.0006) [2024-09-30 01:19:31,063][1157819] Updated weights for policy 0, policy_version 50158 (0.0006) [2024-09-30 01:19:31,603][1157819] Updated weights for policy 0, policy_version 50168 (0.0006) [2024-09-30 01:19:32,153][1157819] Updated weights for policy 0, policy_version 50178 (0.0006) [2024-09-30 01:19:32,702][1157819] Updated weights for policy 0, policy_version 50188 (0.0006) [2024-09-30 01:19:33,250][1157819] Updated weights for policy 0, policy_version 50198 (0.0006) [2024-09-30 01:19:33,792][1157819] Updated weights for policy 0, policy_version 50208 (0.0006) [2024-09-30 01:19:34,328][1157819] Updated weights for policy 0, policy_version 50218 (0.0006) [2024-09-30 01:19:34,870][1157819] Updated weights for policy 0, policy_version 50228 (0.0006) [2024-09-30 01:19:35,441][1157819] Updated weights for policy 0, policy_version 50238 (0.0006) [2024-09-30 01:19:35,466][1157520] Fps is (10 sec: 74956.2, 60 sec: 76936.4, 300 sec: 77574.0). Total num frames: 205774848. Throughput: 0: 19338.1. Samples: 41431796. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 01:19:35,466][1157520] Avg episode reward: [(0, '50.035')] [2024-09-30 01:19:35,995][1157819] Updated weights for policy 0, policy_version 50248 (0.0006) [2024-09-30 01:19:36,522][1157819] Updated weights for policy 0, policy_version 50258 (0.0006) [2024-09-30 01:19:37,075][1157819] Updated weights for policy 0, policy_version 50268 (0.0006) [2024-09-30 01:19:37,632][1157819] Updated weights for policy 0, policy_version 50278 (0.0006) [2024-09-30 01:19:38,183][1157819] Updated weights for policy 0, policy_version 50288 (0.0006) [2024-09-30 01:19:38,754][1157819] Updated weights for policy 0, policy_version 50298 (0.0006) [2024-09-30 01:19:39,288][1157819] Updated weights for policy 0, policy_version 50308 (0.0006) [2024-09-30 01:19:39,801][1157819] Updated weights for policy 0, policy_version 50318 (0.0006) [2024-09-30 01:19:40,371][1157819] Updated weights for policy 0, policy_version 50328 (0.0006) [2024-09-30 01:19:40,466][1157520] Fps is (10 sec: 74957.3, 60 sec: 77073.0, 300 sec: 77615.7). Total num frames: 206147584. Throughput: 0: 19200.6. Samples: 41487508. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 01:19:40,466][1157520] Avg episode reward: [(0, '49.908')] [2024-09-30 01:19:40,911][1157819] Updated weights for policy 0, policy_version 50338 (0.0006) [2024-09-30 01:19:41,484][1157819] Updated weights for policy 0, policy_version 50348 (0.0006) [2024-09-30 01:19:41,999][1157819] Updated weights for policy 0, policy_version 50358 (0.0006) [2024-09-30 01:19:42,527][1157819] Updated weights for policy 0, policy_version 50368 (0.0006) [2024-09-30 01:19:43,077][1157819] Updated weights for policy 0, policy_version 50378 (0.0006) [2024-09-30 01:19:43,630][1157819] Updated weights for policy 0, policy_version 50388 (0.0006) [2024-09-30 01:19:44,203][1157819] Updated weights for policy 0, policy_version 50398 (0.0007) [2024-09-30 01:19:44,786][1157819] Updated weights for policy 0, policy_version 50408 (0.0006) [2024-09-30 01:19:44,791][1157736] Signal inference workers to stop experience collection... (3000 times) [2024-09-30 01:19:44,792][1157736] Signal inference workers to resume experience collection... (3000 times) [2024-09-30 01:19:44,796][1157819] InferenceWorker_p0-w0: stopping experience collection (3000 times) [2024-09-30 01:19:44,796][1157819] InferenceWorker_p0-w0: resuming experience collection (3000 times) [2024-09-30 01:19:45,324][1157819] Updated weights for policy 0, policy_version 50418 (0.0006) [2024-09-30 01:19:45,466][1157520] Fps is (10 sec: 74546.8, 60 sec: 77004.6, 300 sec: 77629.6). Total num frames: 206520320. Throughput: 0: 18935.6. Samples: 41599804. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 01:19:45,466][1157520] Avg episode reward: [(0, '51.720')] [2024-09-30 01:19:45,898][1157819] Updated weights for policy 0, policy_version 50428 (0.0006) [2024-09-30 01:19:46,415][1157819] Updated weights for policy 0, policy_version 50438 (0.0006) [2024-09-30 01:19:46,963][1157819] Updated weights for policy 0, policy_version 50448 (0.0006) [2024-09-30 01:19:47,529][1157819] Updated weights for policy 0, policy_version 50458 (0.0006) [2024-09-30 01:19:48,104][1157819] Updated weights for policy 0, policy_version 50468 (0.0006) [2024-09-30 01:19:48,686][1157819] Updated weights for policy 0, policy_version 50478 (0.0006) [2024-09-30 01:19:49,222][1157819] Updated weights for policy 0, policy_version 50488 (0.0006) [2024-09-30 01:19:49,750][1157819] Updated weights for policy 0, policy_version 50498 (0.0006) [2024-09-30 01:19:50,294][1157819] Updated weights for policy 0, policy_version 50508 (0.0006) [2024-09-30 01:19:50,466][1157520] Fps is (10 sec: 74136.7, 60 sec: 76526.8, 300 sec: 77643.4). Total num frames: 206888960. Throughput: 0: 18817.1. Samples: 41710476. Policy #0 lag: (min: 0.0, avg: 1.8, max: 6.0) [2024-09-30 01:19:50,466][1157520] Avg episode reward: [(0, '52.481')] [2024-09-30 01:19:50,869][1157819] Updated weights for policy 0, policy_version 50518 (0.0006) [2024-09-30 01:19:51,431][1157819] Updated weights for policy 0, policy_version 50528 (0.0006) [2024-09-30 01:19:52,007][1157819] Updated weights for policy 0, policy_version 50538 (0.0006) [2024-09-30 01:19:52,544][1157819] Updated weights for policy 0, policy_version 50548 (0.0006) [2024-09-30 01:19:53,110][1157819] Updated weights for policy 0, policy_version 50558 (0.0006) [2024-09-30 01:19:53,646][1157819] Updated weights for policy 0, policy_version 50568 (0.0006) [2024-09-30 01:19:54,144][1157819] Updated weights for policy 0, policy_version 50578 (0.0006) [2024-09-30 01:19:54,647][1157819] Updated weights for policy 0, policy_version 50588 (0.0006) [2024-09-30 01:19:55,145][1157819] Updated weights for policy 0, policy_version 50598 (0.0006) [2024-09-30 01:19:55,466][1157520] Fps is (10 sec: 75366.5, 60 sec: 75912.4, 300 sec: 77782.3). Total num frames: 207273984. Throughput: 0: 18869.9. Samples: 41765720. Policy #0 lag: (min: 0.0, avg: 1.8, max: 6.0) [2024-09-30 01:19:55,466][1157520] Avg episode reward: [(0, '52.204')] [2024-09-30 01:19:55,646][1157819] Updated weights for policy 0, policy_version 50608 (0.0006) [2024-09-30 01:19:56,171][1157819] Updated weights for policy 0, policy_version 50618 (0.0006) [2024-09-30 01:19:56,692][1157819] Updated weights for policy 0, policy_version 50628 (0.0006) [2024-09-30 01:19:57,186][1157819] Updated weights for policy 0, policy_version 50638 (0.0006) [2024-09-30 01:19:57,714][1157819] Updated weights for policy 0, policy_version 50648 (0.0006) [2024-09-30 01:19:58,218][1157819] Updated weights for policy 0, policy_version 50658 (0.0006) [2024-09-30 01:19:58,744][1157819] Updated weights for policy 0, policy_version 50668 (0.0006) [2024-09-30 01:19:59,317][1157819] Updated weights for policy 0, policy_version 50678 (0.0006) [2024-09-30 01:19:59,870][1157819] Updated weights for policy 0, policy_version 50688 (0.0006) [2024-09-30 01:20:00,398][1157819] Updated weights for policy 0, policy_version 50698 (0.0006) [2024-09-30 01:20:00,466][1157520] Fps is (10 sec: 77005.9, 60 sec: 75639.4, 300 sec: 77671.3). Total num frames: 207659008. Throughput: 0: 18986.9. Samples: 41884728. Policy #0 lag: (min: 0.0, avg: 1.8, max: 6.0) [2024-09-30 01:20:00,466][1157520] Avg episode reward: [(0, '53.325')] [2024-09-30 01:20:00,933][1157819] Updated weights for policy 0, policy_version 50708 (0.0006) [2024-09-30 01:20:01,454][1157819] Updated weights for policy 0, policy_version 50718 (0.0006) [2024-09-30 01:20:02,021][1157819] Updated weights for policy 0, policy_version 50728 (0.0006) [2024-09-30 01:20:02,532][1157819] Updated weights for policy 0, policy_version 50738 (0.0006) [2024-09-30 01:20:03,094][1157819] Updated weights for policy 0, policy_version 50748 (0.0006) [2024-09-30 01:20:03,617][1157819] Updated weights for policy 0, policy_version 50758 (0.0006) [2024-09-30 01:20:04,147][1157819] Updated weights for policy 0, policy_version 50768 (0.0006) [2024-09-30 01:20:04,685][1157819] Updated weights for policy 0, policy_version 50778 (0.0006) [2024-09-30 01:20:05,226][1157819] Updated weights for policy 0, policy_version 50788 (0.0006) [2024-09-30 01:20:05,466][1157520] Fps is (10 sec: 77005.1, 60 sec: 75707.7, 300 sec: 77657.4). Total num frames: 208044032. Throughput: 0: 18959.1. Samples: 41998652. Policy #0 lag: (min: 0.0, avg: 1.8, max: 6.0) [2024-09-30 01:20:05,466][1157520] Avg episode reward: [(0, '52.811')] [2024-09-30 01:20:05,472][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000050792_208044032.pth... [2024-09-30 01:20:05,512][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000046208_189267968.pth [2024-09-30 01:20:05,829][1157819] Updated weights for policy 0, policy_version 50798 (0.0006) [2024-09-30 01:20:06,429][1157819] Updated weights for policy 0, policy_version 50808 (0.0006) [2024-09-30 01:20:07,001][1157819] Updated weights for policy 0, policy_version 50818 (0.0006) [2024-09-30 01:20:07,574][1157819] Updated weights for policy 0, policy_version 50828 (0.0006) [2024-09-30 01:20:08,087][1157819] Updated weights for policy 0, policy_version 50838 (0.0006) [2024-09-30 01:20:08,606][1157819] Updated weights for policy 0, policy_version 50848 (0.0006) [2024-09-30 01:20:09,159][1157819] Updated weights for policy 0, policy_version 50858 (0.0006) [2024-09-30 01:20:09,718][1157819] Updated weights for policy 0, policy_version 50868 (0.0006) [2024-09-30 01:20:10,275][1157819] Updated weights for policy 0, policy_version 50878 (0.0006) [2024-09-30 01:20:10,466][1157520] Fps is (10 sec: 74957.0, 60 sec: 75707.8, 300 sec: 77671.3). Total num frames: 208408576. Throughput: 0: 18842.7. Samples: 42052336. Policy #0 lag: (min: 0.0, avg: 1.8, max: 6.0) [2024-09-30 01:20:10,466][1157520] Avg episode reward: [(0, '52.200')] [2024-09-30 01:20:10,839][1157819] Updated weights for policy 0, policy_version 50888 (0.0006) [2024-09-30 01:20:11,363][1157819] Updated weights for policy 0, policy_version 50898 (0.0006) [2024-09-30 01:20:11,922][1157819] Updated weights for policy 0, policy_version 50908 (0.0006) [2024-09-30 01:20:12,466][1157819] Updated weights for policy 0, policy_version 50918 (0.0006) [2024-09-30 01:20:13,010][1157819] Updated weights for policy 0, policy_version 50928 (0.0006) [2024-09-30 01:20:13,593][1157819] Updated weights for policy 0, policy_version 50938 (0.0006) [2024-09-30 01:20:14,125][1157819] Updated weights for policy 0, policy_version 50948 (0.0006) [2024-09-30 01:20:14,694][1157819] Updated weights for policy 0, policy_version 50958 (0.0006) [2024-09-30 01:20:15,200][1157819] Updated weights for policy 0, policy_version 50968 (0.0006) [2024-09-30 01:20:15,466][1157520] Fps is (10 sec: 74138.9, 60 sec: 75639.6, 300 sec: 77671.3). Total num frames: 208785408. Throughput: 0: 18779.1. Samples: 42164180. Policy #0 lag: (min: 0.0, avg: 1.8, max: 6.0) [2024-09-30 01:20:15,466][1157520] Avg episode reward: [(0, '50.992')] [2024-09-30 01:20:15,725][1157819] Updated weights for policy 0, policy_version 50978 (0.0006) [2024-09-30 01:20:16,224][1157819] Updated weights for policy 0, policy_version 50988 (0.0006) [2024-09-30 01:20:16,748][1157819] Updated weights for policy 0, policy_version 50998 (0.0006) [2024-09-30 01:20:17,230][1157819] Updated weights for policy 0, policy_version 51008 (0.0006) [2024-09-30 01:20:17,745][1157819] Updated weights for policy 0, policy_version 51018 (0.0006) [2024-09-30 01:20:18,267][1157819] Updated weights for policy 0, policy_version 51028 (0.0006) [2024-09-30 01:20:18,798][1157819] Updated weights for policy 0, policy_version 51038 (0.0006) [2024-09-30 01:20:19,312][1157819] Updated weights for policy 0, policy_version 51048 (0.0006) [2024-09-30 01:20:19,436][1157736] Signal inference workers to stop experience collection... (3050 times) [2024-09-30 01:20:19,437][1157736] Signal inference workers to resume experience collection... (3050 times) [2024-09-30 01:20:19,441][1157819] InferenceWorker_p0-w0: stopping experience collection (3050 times) [2024-09-30 01:20:19,444][1157819] InferenceWorker_p0-w0: resuming experience collection (3050 times) [2024-09-30 01:20:19,824][1157819] Updated weights for policy 0, policy_version 51058 (0.0006) [2024-09-30 01:20:20,349][1157819] Updated weights for policy 0, policy_version 51068 (0.0006) [2024-09-30 01:20:20,466][1157520] Fps is (10 sec: 77413.8, 60 sec: 75776.0, 300 sec: 77754.6). Total num frames: 209182720. Throughput: 0: 18903.5. Samples: 42282452. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 01:20:20,466][1157520] Avg episode reward: [(0, '52.195')] [2024-09-30 01:20:20,872][1157819] Updated weights for policy 0, policy_version 51078 (0.0006) [2024-09-30 01:20:21,402][1157819] Updated weights for policy 0, policy_version 51088 (0.0006) [2024-09-30 01:20:21,914][1157819] Updated weights for policy 0, policy_version 51098 (0.0006) [2024-09-30 01:20:22,434][1157819] Updated weights for policy 0, policy_version 51108 (0.0006) [2024-09-30 01:20:22,982][1157819] Updated weights for policy 0, policy_version 51118 (0.0006) [2024-09-30 01:20:23,509][1157819] Updated weights for policy 0, policy_version 51128 (0.0006) [2024-09-30 01:20:24,026][1157819] Updated weights for policy 0, policy_version 51138 (0.0006) [2024-09-30 01:20:24,557][1157819] Updated weights for policy 0, policy_version 51148 (0.0006) [2024-09-30 01:20:25,063][1157819] Updated weights for policy 0, policy_version 51158 (0.0006) [2024-09-30 01:20:25,466][1157520] Fps is (10 sec: 78641.4, 60 sec: 75775.8, 300 sec: 77921.2). Total num frames: 209571840. Throughput: 0: 18968.5. Samples: 42341092. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 01:20:25,466][1157520] Avg episode reward: [(0, '50.616')] [2024-09-30 01:20:25,565][1157819] Updated weights for policy 0, policy_version 51168 (0.0006) [2024-09-30 01:20:26,105][1157819] Updated weights for policy 0, policy_version 51178 (0.0006) [2024-09-30 01:20:26,638][1157819] Updated weights for policy 0, policy_version 51188 (0.0006) [2024-09-30 01:20:27,143][1157819] Updated weights for policy 0, policy_version 51198 (0.0006) [2024-09-30 01:20:27,669][1157819] Updated weights for policy 0, policy_version 51208 (0.0006) [2024-09-30 01:20:28,191][1157819] Updated weights for policy 0, policy_version 51218 (0.0006) [2024-09-30 01:20:28,741][1157819] Updated weights for policy 0, policy_version 51228 (0.0006) [2024-09-30 01:20:29,297][1157819] Updated weights for policy 0, policy_version 51238 (0.0006) [2024-09-30 01:20:29,842][1157819] Updated weights for policy 0, policy_version 51248 (0.0006) [2024-09-30 01:20:30,385][1157819] Updated weights for policy 0, policy_version 51258 (0.0006) [2024-09-30 01:20:30,466][1157520] Fps is (10 sec: 77414.3, 60 sec: 75980.8, 300 sec: 78004.5). Total num frames: 209956864. Throughput: 0: 19076.6. Samples: 42458248. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 01:20:30,466][1157520] Avg episode reward: [(0, '49.038')] [2024-09-30 01:20:30,949][1157819] Updated weights for policy 0, policy_version 51268 (0.0006) [2024-09-30 01:20:31,500][1157819] Updated weights for policy 0, policy_version 51278 (0.0006) [2024-09-30 01:20:32,032][1157819] Updated weights for policy 0, policy_version 51288 (0.0006) [2024-09-30 01:20:32,585][1157819] Updated weights for policy 0, policy_version 51298 (0.0006) [2024-09-30 01:20:33,099][1157819] Updated weights for policy 0, policy_version 51308 (0.0006) [2024-09-30 01:20:33,654][1157819] Updated weights for policy 0, policy_version 51318 (0.0006) [2024-09-30 01:20:34,199][1157819] Updated weights for policy 0, policy_version 51328 (0.0006) [2024-09-30 01:20:34,736][1157819] Updated weights for policy 0, policy_version 51338 (0.0006) [2024-09-30 01:20:35,285][1157819] Updated weights for policy 0, policy_version 51348 (0.0006) [2024-09-30 01:20:35,466][1157520] Fps is (10 sec: 76185.5, 60 sec: 75980.7, 300 sec: 77935.1). Total num frames: 210333696. Throughput: 0: 19124.6. Samples: 42571084. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 01:20:35,466][1157520] Avg episode reward: [(0, '52.822')] [2024-09-30 01:20:35,809][1157819] Updated weights for policy 0, policy_version 51358 (0.0006) [2024-09-30 01:20:36,345][1157819] Updated weights for policy 0, policy_version 51368 (0.0006) [2024-09-30 01:20:36,914][1157819] Updated weights for policy 0, policy_version 51378 (0.0006) [2024-09-30 01:20:37,468][1157819] Updated weights for policy 0, policy_version 51388 (0.0006) [2024-09-30 01:20:38,026][1157819] Updated weights for policy 0, policy_version 51398 (0.0006) [2024-09-30 01:20:38,558][1157819] Updated weights for policy 0, policy_version 51408 (0.0006) [2024-09-30 01:20:39,069][1157819] Updated weights for policy 0, policy_version 51418 (0.0006) [2024-09-30 01:20:39,627][1157819] Updated weights for policy 0, policy_version 51428 (0.0006) [2024-09-30 01:20:40,163][1157819] Updated weights for policy 0, policy_version 51438 (0.0006) [2024-09-30 01:20:40,466][1157520] Fps is (10 sec: 75366.2, 60 sec: 76049.0, 300 sec: 77879.6). Total num frames: 210710528. Throughput: 0: 19153.4. Samples: 42627624. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 01:20:40,466][1157520] Avg episode reward: [(0, '51.319')] [2024-09-30 01:20:40,743][1157819] Updated weights for policy 0, policy_version 51448 (0.0006) [2024-09-30 01:20:41,291][1157819] Updated weights for policy 0, policy_version 51458 (0.0006) [2024-09-30 01:20:41,831][1157819] Updated weights for policy 0, policy_version 51468 (0.0006) [2024-09-30 01:20:42,393][1157819] Updated weights for policy 0, policy_version 51478 (0.0006) [2024-09-30 01:20:42,957][1157819] Updated weights for policy 0, policy_version 51488 (0.0006) [2024-09-30 01:20:43,477][1157819] Updated weights for policy 0, policy_version 51498 (0.0006) [2024-09-30 01:20:44,038][1157819] Updated weights for policy 0, policy_version 51508 (0.0006) [2024-09-30 01:20:44,595][1157819] Updated weights for policy 0, policy_version 51518 (0.0006) [2024-09-30 01:20:45,154][1157819] Updated weights for policy 0, policy_version 51528 (0.0006) [2024-09-30 01:20:45,466][1157520] Fps is (10 sec: 74548.1, 60 sec: 75980.9, 300 sec: 77796.3). Total num frames: 211079168. Throughput: 0: 18990.1. Samples: 42739284. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 01:20:45,466][1157520] Avg episode reward: [(0, '48.385')] [2024-09-30 01:20:45,690][1157819] Updated weights for policy 0, policy_version 51538 (0.0006) [2024-09-30 01:20:46,235][1157819] Updated weights for policy 0, policy_version 51548 (0.0006) [2024-09-30 01:20:46,770][1157819] Updated weights for policy 0, policy_version 51558 (0.0006) [2024-09-30 01:20:47,271][1157819] Updated weights for policy 0, policy_version 51568 (0.0006) [2024-09-30 01:20:47,787][1157819] Updated weights for policy 0, policy_version 51578 (0.0006) [2024-09-30 01:20:48,305][1157819] Updated weights for policy 0, policy_version 51588 (0.0006) [2024-09-30 01:20:48,830][1157819] Updated weights for policy 0, policy_version 51598 (0.0006) [2024-09-30 01:20:49,390][1157819] Updated weights for policy 0, policy_version 51608 (0.0006) [2024-09-30 01:20:49,888][1157819] Updated weights for policy 0, policy_version 51618 (0.0006) [2024-09-30 01:20:50,404][1157819] Updated weights for policy 0, policy_version 51628 (0.0006) [2024-09-30 01:20:50,466][1157520] Fps is (10 sec: 76186.0, 60 sec: 76390.5, 300 sec: 77782.4). Total num frames: 211472384. Throughput: 0: 19032.5. Samples: 42855112. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 01:20:50,466][1157520] Avg episode reward: [(0, '51.199')] [2024-09-30 01:20:50,906][1157819] Updated weights for policy 0, policy_version 51638 (0.0006) [2024-09-30 01:20:51,411][1157819] Updated weights for policy 0, policy_version 51648 (0.0006) [2024-09-30 01:20:51,953][1157819] Updated weights for policy 0, policy_version 51658 (0.0006) [2024-09-30 01:20:52,475][1157819] Updated weights for policy 0, policy_version 51668 (0.0006) [2024-09-30 01:20:52,980][1157819] Updated weights for policy 0, policy_version 51678 (0.0006) [2024-09-30 01:20:53,522][1157819] Updated weights for policy 0, policy_version 51688 (0.0006) [2024-09-30 01:20:54,035][1157819] Updated weights for policy 0, policy_version 51698 (0.0006) [2024-09-30 01:20:54,563][1157819] Updated weights for policy 0, policy_version 51708 (0.0006) [2024-09-30 01:20:55,073][1157819] Updated weights for policy 0, policy_version 51718 (0.0006) [2024-09-30 01:20:55,466][1157520] Fps is (10 sec: 78642.0, 60 sec: 76526.9, 300 sec: 77865.6). Total num frames: 211865600. Throughput: 0: 19151.5. Samples: 42914156. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 01:20:55,466][1157520] Avg episode reward: [(0, '51.945')] [2024-09-30 01:20:55,597][1157819] Updated weights for policy 0, policy_version 51728 (0.0006) [2024-09-30 01:20:56,126][1157819] Updated weights for policy 0, policy_version 51738 (0.0006) [2024-09-30 01:20:56,648][1157819] Updated weights for policy 0, policy_version 51748 (0.0006) [2024-09-30 01:20:57,038][1157736] Signal inference workers to stop experience collection... (3100 times) [2024-09-30 01:20:57,042][1157819] InferenceWorker_p0-w0: stopping experience collection (3100 times) [2024-09-30 01:20:57,046][1157736] Signal inference workers to resume experience collection... (3100 times) [2024-09-30 01:20:57,047][1157819] InferenceWorker_p0-w0: resuming experience collection (3100 times) [2024-09-30 01:20:57,160][1157819] Updated weights for policy 0, policy_version 51758 (0.0006) [2024-09-30 01:20:57,704][1157819] Updated weights for policy 0, policy_version 51768 (0.0006) [2024-09-30 01:20:58,198][1157819] Updated weights for policy 0, policy_version 51778 (0.0006) [2024-09-30 01:20:58,672][1157819] Updated weights for policy 0, policy_version 51788 (0.0006) [2024-09-30 01:20:59,163][1157819] Updated weights for policy 0, policy_version 51798 (0.0006) [2024-09-30 01:20:59,652][1157819] Updated weights for policy 0, policy_version 51808 (0.0006) [2024-09-30 01:21:00,142][1157819] Updated weights for policy 0, policy_version 51818 (0.0006) [2024-09-30 01:21:00,466][1157520] Fps is (10 sec: 79872.2, 60 sec: 76868.2, 300 sec: 77990.7). Total num frames: 212271104. Throughput: 0: 19316.5. Samples: 43033424. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 01:21:00,466][1157520] Avg episode reward: [(0, '50.561')] [2024-09-30 01:21:00,649][1157819] Updated weights for policy 0, policy_version 51828 (0.0006) [2024-09-30 01:21:01,138][1157819] Updated weights for policy 0, policy_version 51838 (0.0006) [2024-09-30 01:21:01,626][1157819] Updated weights for policy 0, policy_version 51848 (0.0006) [2024-09-30 01:21:02,109][1157819] Updated weights for policy 0, policy_version 51858 (0.0006) [2024-09-30 01:21:02,601][1157819] Updated weights for policy 0, policy_version 51868 (0.0006) [2024-09-30 01:21:03,095][1157819] Updated weights for policy 0, policy_version 51878 (0.0006) [2024-09-30 01:21:03,585][1157819] Updated weights for policy 0, policy_version 51888 (0.0006) [2024-09-30 01:21:04,075][1157819] Updated weights for policy 0, policy_version 51898 (0.0006) [2024-09-30 01:21:04,583][1157819] Updated weights for policy 0, policy_version 51908 (0.0006) [2024-09-30 01:21:05,072][1157819] Updated weights for policy 0, policy_version 51918 (0.0006) [2024-09-30 01:21:05,466][1157520] Fps is (10 sec: 82740.9, 60 sec: 77482.8, 300 sec: 78171.1). Total num frames: 212692992. Throughput: 0: 19472.5. Samples: 43158712. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 01:21:05,466][1157520] Avg episode reward: [(0, '52.749')] [2024-09-30 01:21:05,542][1157819] Updated weights for policy 0, policy_version 51928 (0.0006) [2024-09-30 01:21:06,036][1157819] Updated weights for policy 0, policy_version 51938 (0.0006) [2024-09-30 01:21:06,525][1157819] Updated weights for policy 0, policy_version 51948 (0.0006) [2024-09-30 01:21:06,953][1157819] Updated weights for policy 0, policy_version 51958 (0.0007) [2024-09-30 01:21:07,478][1157819] Updated weights for policy 0, policy_version 51968 (0.0006) [2024-09-30 01:21:07,950][1157819] Updated weights for policy 0, policy_version 51978 (0.0006) [2024-09-30 01:21:08,433][1157819] Updated weights for policy 0, policy_version 51988 (0.0006) [2024-09-30 01:21:08,920][1157819] Updated weights for policy 0, policy_version 51998 (0.0006) [2024-09-30 01:21:09,421][1157819] Updated weights for policy 0, policy_version 52008 (0.0006) [2024-09-30 01:21:09,930][1157819] Updated weights for policy 0, policy_version 52018 (0.0006) [2024-09-30 01:21:10,393][1157819] Updated weights for policy 0, policy_version 52028 (0.0006) [2024-09-30 01:21:10,466][1157520] Fps is (10 sec: 83558.4, 60 sec: 78301.8, 300 sec: 78337.8). Total num frames: 213106688. Throughput: 0: 19566.6. Samples: 43221588. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 01:21:10,466][1157520] Avg episode reward: [(0, '53.136')] [2024-09-30 01:21:10,880][1157819] Updated weights for policy 0, policy_version 52038 (0.0006) [2024-09-30 01:21:11,378][1157819] Updated weights for policy 0, policy_version 52048 (0.0006) [2024-09-30 01:21:11,862][1157819] Updated weights for policy 0, policy_version 52058 (0.0006) [2024-09-30 01:21:12,350][1157819] Updated weights for policy 0, policy_version 52068 (0.0006) [2024-09-30 01:21:12,842][1157819] Updated weights for policy 0, policy_version 52078 (0.0006) [2024-09-30 01:21:13,333][1157819] Updated weights for policy 0, policy_version 52088 (0.0006) [2024-09-30 01:21:13,811][1157819] Updated weights for policy 0, policy_version 52098 (0.0006) [2024-09-30 01:21:14,315][1157819] Updated weights for policy 0, policy_version 52108 (0.0006) [2024-09-30 01:21:14,807][1157819] Updated weights for policy 0, policy_version 52118 (0.0006) [2024-09-30 01:21:15,272][1157819] Updated weights for policy 0, policy_version 52128 (0.0006) [2024-09-30 01:21:15,466][1157520] Fps is (10 sec: 83557.9, 60 sec: 79052.6, 300 sec: 78476.6). Total num frames: 213528576. Throughput: 0: 19764.2. Samples: 43347636. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 01:21:15,466][1157520] Avg episode reward: [(0, '52.803')] [2024-09-30 01:21:15,765][1157819] Updated weights for policy 0, policy_version 52138 (0.0006) [2024-09-30 01:21:16,269][1157819] Updated weights for policy 0, policy_version 52148 (0.0006) [2024-09-30 01:21:16,751][1157819] Updated weights for policy 0, policy_version 52158 (0.0006) [2024-09-30 01:21:17,233][1157819] Updated weights for policy 0, policy_version 52168 (0.0006) [2024-09-30 01:21:17,731][1157819] Updated weights for policy 0, policy_version 52178 (0.0006) [2024-09-30 01:21:18,228][1157819] Updated weights for policy 0, policy_version 52188 (0.0006) [2024-09-30 01:21:18,716][1157819] Updated weights for policy 0, policy_version 52198 (0.0006) [2024-09-30 01:21:19,198][1157819] Updated weights for policy 0, policy_version 52208 (0.0006) [2024-09-30 01:21:19,691][1157819] Updated weights for policy 0, policy_version 52218 (0.0006) [2024-09-30 01:21:20,179][1157819] Updated weights for policy 0, policy_version 52228 (0.0006) [2024-09-30 01:21:20,466][1157520] Fps is (10 sec: 83968.1, 60 sec: 79394.2, 300 sec: 78532.1). Total num frames: 213946368. Throughput: 0: 20051.2. Samples: 43473384. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 01:21:20,466][1157520] Avg episode reward: [(0, '51.386')] [2024-09-30 01:21:20,661][1157819] Updated weights for policy 0, policy_version 52238 (0.0006) [2024-09-30 01:21:21,155][1157819] Updated weights for policy 0, policy_version 52248 (0.0006) [2024-09-30 01:21:21,648][1157819] Updated weights for policy 0, policy_version 52258 (0.0006) [2024-09-30 01:21:22,146][1157819] Updated weights for policy 0, policy_version 52268 (0.0006) [2024-09-30 01:21:22,631][1157819] Updated weights for policy 0, policy_version 52278 (0.0006) [2024-09-30 01:21:22,632][1157736] Signal inference workers to stop experience collection... (3150 times) [2024-09-30 01:21:22,633][1157736] Signal inference workers to resume experience collection... (3150 times) [2024-09-30 01:21:22,636][1157819] InferenceWorker_p0-w0: stopping experience collection (3150 times) [2024-09-30 01:21:22,636][1157819] InferenceWorker_p0-w0: resuming experience collection (3150 times) [2024-09-30 01:21:23,120][1157819] Updated weights for policy 0, policy_version 52288 (0.0006) [2024-09-30 01:21:23,608][1157819] Updated weights for policy 0, policy_version 52298 (0.0006) [2024-09-30 01:21:24,102][1157819] Updated weights for policy 0, policy_version 52308 (0.0006) [2024-09-30 01:21:24,582][1157819] Updated weights for policy 0, policy_version 52318 (0.0006) [2024-09-30 01:21:25,073][1157819] Updated weights for policy 0, policy_version 52328 (0.0006) [2024-09-30 01:21:25,466][1157520] Fps is (10 sec: 83967.1, 60 sec: 79940.3, 300 sec: 78615.4). Total num frames: 214368256. Throughput: 0: 20191.2. Samples: 43536228. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:21:25,466][1157520] Avg episode reward: [(0, '51.836')] [2024-09-30 01:21:25,557][1157819] Updated weights for policy 0, policy_version 52338 (0.0006) [2024-09-30 01:21:26,062][1157819] Updated weights for policy 0, policy_version 52348 (0.0006) [2024-09-30 01:21:26,537][1157819] Updated weights for policy 0, policy_version 52358 (0.0006) [2024-09-30 01:21:27,030][1157819] Updated weights for policy 0, policy_version 52368 (0.0006) [2024-09-30 01:21:27,512][1157819] Updated weights for policy 0, policy_version 52378 (0.0006) [2024-09-30 01:21:27,988][1157819] Updated weights for policy 0, policy_version 52388 (0.0006) [2024-09-30 01:21:28,486][1157819] Updated weights for policy 0, policy_version 52398 (0.0006) [2024-09-30 01:21:28,943][1157819] Updated weights for policy 0, policy_version 52408 (0.0006) [2024-09-30 01:21:29,440][1157819] Updated weights for policy 0, policy_version 52418 (0.0006) [2024-09-30 01:21:29,940][1157819] Updated weights for policy 0, policy_version 52428 (0.0007) [2024-09-30 01:21:30,418][1157819] Updated weights for policy 0, policy_version 52438 (0.0006) [2024-09-30 01:21:30,466][1157520] Fps is (10 sec: 83967.9, 60 sec: 80486.5, 300 sec: 78601.6). Total num frames: 214786048. Throughput: 0: 20509.8. Samples: 43662224. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:21:30,466][1157520] Avg episode reward: [(0, '52.038')] [2024-09-30 01:21:30,886][1157819] Updated weights for policy 0, policy_version 52448 (0.0006) [2024-09-30 01:21:31,440][1157819] Updated weights for policy 0, policy_version 52458 (0.0006) [2024-09-30 01:21:31,945][1157819] Updated weights for policy 0, policy_version 52468 (0.0007) [2024-09-30 01:21:32,472][1157819] Updated weights for policy 0, policy_version 52478 (0.0006) [2024-09-30 01:21:33,005][1157819] Updated weights for policy 0, policy_version 52488 (0.0007) [2024-09-30 01:21:33,510][1157819] Updated weights for policy 0, policy_version 52498 (0.0007) [2024-09-30 01:21:34,015][1157819] Updated weights for policy 0, policy_version 52508 (0.0007) [2024-09-30 01:21:34,544][1157819] Updated weights for policy 0, policy_version 52518 (0.0006) [2024-09-30 01:21:35,060][1157819] Updated weights for policy 0, policy_version 52528 (0.0006) [2024-09-30 01:21:35,466][1157520] Fps is (10 sec: 81511.5, 60 sec: 80827.9, 300 sec: 78462.7). Total num frames: 215183360. Throughput: 0: 20625.5. Samples: 43783260. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:21:35,466][1157520] Avg episode reward: [(0, '50.587')] [2024-09-30 01:21:35,610][1157819] Updated weights for policy 0, policy_version 52538 (0.0007) [2024-09-30 01:21:36,160][1157819] Updated weights for policy 0, policy_version 52548 (0.0006) [2024-09-30 01:21:36,686][1157819] Updated weights for policy 0, policy_version 52558 (0.0006) [2024-09-30 01:21:37,249][1157819] Updated weights for policy 0, policy_version 52568 (0.0006) [2024-09-30 01:21:37,811][1157819] Updated weights for policy 0, policy_version 52578 (0.0006) [2024-09-30 01:21:38,335][1157819] Updated weights for policy 0, policy_version 52588 (0.0006) [2024-09-30 01:21:38,869][1157819] Updated weights for policy 0, policy_version 52598 (0.0007) [2024-09-30 01:21:39,432][1157819] Updated weights for policy 0, policy_version 52608 (0.0007) [2024-09-30 01:21:39,948][1157819] Updated weights for policy 0, policy_version 52618 (0.0006) [2024-09-30 01:21:40,456][1157819] Updated weights for policy 0, policy_version 52628 (0.0007) [2024-09-30 01:21:40,466][1157520] Fps is (10 sec: 77824.1, 60 sec: 80896.1, 300 sec: 78337.8). Total num frames: 215564288. Throughput: 0: 20564.7. Samples: 43839564. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:21:40,466][1157520] Avg episode reward: [(0, '51.422')] [2024-09-30 01:21:40,983][1157819] Updated weights for policy 0, policy_version 52638 (0.0006) [2024-09-30 01:21:41,485][1157819] Updated weights for policy 0, policy_version 52648 (0.0006) [2024-09-30 01:21:42,014][1157819] Updated weights for policy 0, policy_version 52658 (0.0006) [2024-09-30 01:21:42,603][1157819] Updated weights for policy 0, policy_version 52668 (0.0007) [2024-09-30 01:21:43,139][1157819] Updated weights for policy 0, policy_version 52678 (0.0006) [2024-09-30 01:21:43,682][1157819] Updated weights for policy 0, policy_version 52688 (0.0006) [2024-09-30 01:21:44,255][1157819] Updated weights for policy 0, policy_version 52698 (0.0006) [2024-09-30 01:21:44,768][1157819] Updated weights for policy 0, policy_version 52708 (0.0006) [2024-09-30 01:21:45,327][1157819] Updated weights for policy 0, policy_version 52718 (0.0006) [2024-09-30 01:21:45,466][1157520] Fps is (10 sec: 75775.2, 60 sec: 81032.4, 300 sec: 78212.8). Total num frames: 215941120. Throughput: 0: 20467.6. Samples: 43954468. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:21:45,466][1157520] Avg episode reward: [(0, '52.183')] [2024-09-30 01:21:45,889][1157819] Updated weights for policy 0, policy_version 52728 (0.0006) [2024-09-30 01:21:46,417][1157819] Updated weights for policy 0, policy_version 52738 (0.0006) [2024-09-30 01:21:46,995][1157819] Updated weights for policy 0, policy_version 52748 (0.0006) [2024-09-30 01:21:47,534][1157819] Updated weights for policy 0, policy_version 52758 (0.0006) [2024-09-30 01:21:48,104][1157819] Updated weights for policy 0, policy_version 52768 (0.0006) [2024-09-30 01:21:48,665][1157819] Updated weights for policy 0, policy_version 52778 (0.0006) [2024-09-30 01:21:49,212][1157819] Updated weights for policy 0, policy_version 52788 (0.0006) [2024-09-30 01:21:49,736][1157819] Updated weights for policy 0, policy_version 52798 (0.0006) [2024-09-30 01:21:50,219][1157819] Updated weights for policy 0, policy_version 52808 (0.0006) [2024-09-30 01:21:50,466][1157520] Fps is (10 sec: 75366.0, 60 sec: 80759.5, 300 sec: 78115.6). Total num frames: 216317952. Throughput: 0: 20174.0. Samples: 44066544. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:21:50,466][1157520] Avg episode reward: [(0, '52.427')] [2024-09-30 01:21:50,753][1157819] Updated weights for policy 0, policy_version 52818 (0.0006) [2024-09-30 01:21:51,274][1157819] Updated weights for policy 0, policy_version 52828 (0.0006) [2024-09-30 01:21:51,792][1157819] Updated weights for policy 0, policy_version 52838 (0.0006) [2024-09-30 01:21:52,286][1157819] Updated weights for policy 0, policy_version 52848 (0.0006) [2024-09-30 01:21:52,799][1157819] Updated weights for policy 0, policy_version 52858 (0.0006) [2024-09-30 01:21:53,330][1157819] Updated weights for policy 0, policy_version 52868 (0.0006) [2024-09-30 01:21:53,876][1157819] Updated weights for policy 0, policy_version 52878 (0.0006) [2024-09-30 01:21:54,367][1157819] Updated weights for policy 0, policy_version 52888 (0.0006) [2024-09-30 01:21:54,878][1157819] Updated weights for policy 0, policy_version 52898 (0.0006) [2024-09-30 01:21:55,380][1157819] Updated weights for policy 0, policy_version 52908 (0.0006) [2024-09-30 01:21:55,466][1157520] Fps is (10 sec: 77414.3, 60 sec: 80827.8, 300 sec: 78129.4). Total num frames: 216715264. Throughput: 0: 20100.7. Samples: 44126120. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:21:55,466][1157520] Avg episode reward: [(0, '49.494')] [2024-09-30 01:21:55,928][1157819] Updated weights for policy 0, policy_version 52918 (0.0006) [2024-09-30 01:21:56,436][1157819] Updated weights for policy 0, policy_version 52928 (0.0006) [2024-09-30 01:21:56,975][1157819] Updated weights for policy 0, policy_version 52938 (0.0006) [2024-09-30 01:21:57,136][1157736] Signal inference workers to stop experience collection... (3200 times) [2024-09-30 01:21:57,140][1157819] InferenceWorker_p0-w0: stopping experience collection (3200 times) [2024-09-30 01:21:57,140][1157736] Signal inference workers to resume experience collection... (3200 times) [2024-09-30 01:21:57,145][1157819] InferenceWorker_p0-w0: resuming experience collection (3200 times) [2024-09-30 01:21:57,470][1157819] Updated weights for policy 0, policy_version 52948 (0.0006) [2024-09-30 01:21:57,994][1157819] Updated weights for policy 0, policy_version 52958 (0.0006) [2024-09-30 01:21:58,525][1157819] Updated weights for policy 0, policy_version 52968 (0.0006) [2024-09-30 01:21:59,056][1157819] Updated weights for policy 0, policy_version 52978 (0.0006) [2024-09-30 01:21:59,568][1157819] Updated weights for policy 0, policy_version 52988 (0.0006) [2024-09-30 01:22:00,056][1157819] Updated weights for policy 0, policy_version 52998 (0.0006) [2024-09-30 01:22:00,466][1157520] Fps is (10 sec: 79053.0, 60 sec: 80622.9, 300 sec: 78129.5). Total num frames: 217108480. Throughput: 0: 19930.9. Samples: 44244528. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:22:00,466][1157520] Avg episode reward: [(0, '54.035')] [2024-09-30 01:22:00,595][1157819] Updated weights for policy 0, policy_version 53008 (0.0006) [2024-09-30 01:22:01,116][1157819] Updated weights for policy 0, policy_version 53018 (0.0006) [2024-09-30 01:22:01,621][1157819] Updated weights for policy 0, policy_version 53028 (0.0006) [2024-09-30 01:22:02,158][1157819] Updated weights for policy 0, policy_version 53038 (0.0006) [2024-09-30 01:22:02,648][1157819] Updated weights for policy 0, policy_version 53048 (0.0006) [2024-09-30 01:22:03,175][1157819] Updated weights for policy 0, policy_version 53058 (0.0006) [2024-09-30 01:22:03,691][1157819] Updated weights for policy 0, policy_version 53068 (0.0006) [2024-09-30 01:22:04,218][1157819] Updated weights for policy 0, policy_version 53078 (0.0006) [2024-09-30 01:22:04,747][1157819] Updated weights for policy 0, policy_version 53088 (0.0006) [2024-09-30 01:22:05,263][1157819] Updated weights for policy 0, policy_version 53098 (0.0006) [2024-09-30 01:22:05,466][1157520] Fps is (10 sec: 78643.6, 60 sec: 80144.9, 300 sec: 78129.4). Total num frames: 217501696. Throughput: 0: 19768.9. Samples: 44362984. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:22:05,466][1157520] Avg episode reward: [(0, '52.151')] [2024-09-30 01:22:05,481][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000053102_217505792.pth... [2024-09-30 01:22:05,534][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000048538_198811648.pth [2024-09-30 01:22:05,773][1157819] Updated weights for policy 0, policy_version 53108 (0.0006) [2024-09-30 01:22:06,287][1157819] Updated weights for policy 0, policy_version 53118 (0.0006) [2024-09-30 01:22:06,828][1157819] Updated weights for policy 0, policy_version 53128 (0.0006) [2024-09-30 01:22:07,343][1157819] Updated weights for policy 0, policy_version 53138 (0.0006) [2024-09-30 01:22:07,827][1157819] Updated weights for policy 0, policy_version 53148 (0.0006) [2024-09-30 01:22:08,356][1157819] Updated weights for policy 0, policy_version 53158 (0.0006) [2024-09-30 01:22:08,909][1157819] Updated weights for policy 0, policy_version 53168 (0.0006) [2024-09-30 01:22:09,405][1157819] Updated weights for policy 0, policy_version 53178 (0.0006) [2024-09-30 01:22:09,894][1157819] Updated weights for policy 0, policy_version 53188 (0.0006) [2024-09-30 01:22:10,396][1157819] Updated weights for policy 0, policy_version 53198 (0.0006) [2024-09-30 01:22:10,466][1157520] Fps is (10 sec: 79052.6, 60 sec: 79872.0, 300 sec: 78046.1). Total num frames: 217899008. Throughput: 0: 19685.0. Samples: 44422052. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:22:10,466][1157520] Avg episode reward: [(0, '54.741')] [2024-09-30 01:22:10,925][1157819] Updated weights for policy 0, policy_version 53208 (0.0006) [2024-09-30 01:22:11,482][1157819] Updated weights for policy 0, policy_version 53218 (0.0006) [2024-09-30 01:22:12,002][1157819] Updated weights for policy 0, policy_version 53228 (0.0006) [2024-09-30 01:22:12,496][1157819] Updated weights for policy 0, policy_version 53238 (0.0006) [2024-09-30 01:22:13,020][1157819] Updated weights for policy 0, policy_version 53248 (0.0006) [2024-09-30 01:22:13,563][1157819] Updated weights for policy 0, policy_version 53258 (0.0006) [2024-09-30 01:22:14,119][1157819] Updated weights for policy 0, policy_version 53268 (0.0006) [2024-09-30 01:22:14,649][1157819] Updated weights for policy 0, policy_version 53278 (0.0006) [2024-09-30 01:22:15,160][1157819] Updated weights for policy 0, policy_version 53288 (0.0006) [2024-09-30 01:22:15,466][1157520] Fps is (10 sec: 78643.5, 60 sec: 79325.9, 300 sec: 77949.0). Total num frames: 218288128. Throughput: 0: 19526.9. Samples: 44540936. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:22:15,466][1157520] Avg episode reward: [(0, '52.270')] [2024-09-30 01:22:15,679][1157819] Updated weights for policy 0, policy_version 53298 (0.0006) [2024-09-30 01:22:16,261][1157819] Updated weights for policy 0, policy_version 53308 (0.0006) [2024-09-30 01:22:16,763][1157819] Updated weights for policy 0, policy_version 53318 (0.0006) [2024-09-30 01:22:17,277][1157819] Updated weights for policy 0, policy_version 53328 (0.0006) [2024-09-30 01:22:17,780][1157819] Updated weights for policy 0, policy_version 53338 (0.0006) [2024-09-30 01:22:18,282][1157819] Updated weights for policy 0, policy_version 53348 (0.0006) [2024-09-30 01:22:18,801][1157819] Updated weights for policy 0, policy_version 53358 (0.0006) [2024-09-30 01:22:19,375][1157819] Updated weights for policy 0, policy_version 53368 (0.0006) [2024-09-30 01:22:19,910][1157819] Updated weights for policy 0, policy_version 53378 (0.0006) [2024-09-30 01:22:20,466][1157520] Fps is (10 sec: 77414.9, 60 sec: 78779.8, 300 sec: 77879.5). Total num frames: 218673152. Throughput: 0: 19416.3. Samples: 44656992. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:22:20,466][1157520] Avg episode reward: [(0, '54.547')] [2024-09-30 01:22:20,472][1157819] Updated weights for policy 0, policy_version 53388 (0.0006) [2024-09-30 01:22:21,033][1157819] Updated weights for policy 0, policy_version 53398 (0.0006) [2024-09-30 01:22:21,605][1157819] Updated weights for policy 0, policy_version 53408 (0.0006) [2024-09-30 01:22:22,186][1157819] Updated weights for policy 0, policy_version 53418 (0.0006) [2024-09-30 01:22:22,737][1157819] Updated weights for policy 0, policy_version 53428 (0.0006) [2024-09-30 01:22:23,286][1157819] Updated weights for policy 0, policy_version 53438 (0.0006) [2024-09-30 01:22:23,813][1157736] Signal inference workers to stop experience collection... (3250 times) [2024-09-30 01:22:23,814][1157736] Signal inference workers to resume experience collection... (3250 times) [2024-09-30 01:22:23,817][1157819] InferenceWorker_p0-w0: stopping experience collection (3250 times) [2024-09-30 01:22:23,820][1157819] InferenceWorker_p0-w0: resuming experience collection (3250 times) [2024-09-30 01:22:23,835][1157819] Updated weights for policy 0, policy_version 53448 (0.0006) [2024-09-30 01:22:24,447][1157819] Updated weights for policy 0, policy_version 53458 (0.0006) [2024-09-30 01:22:24,976][1157819] Updated weights for policy 0, policy_version 53468 (0.0006) [2024-09-30 01:22:25,466][1157520] Fps is (10 sec: 74957.1, 60 sec: 77824.2, 300 sec: 77796.2). Total num frames: 219037696. Throughput: 0: 19369.2. Samples: 44711180. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:22:25,466][1157520] Avg episode reward: [(0, '53.441')] [2024-09-30 01:22:25,539][1157819] Updated weights for policy 0, policy_version 53478 (0.0006) [2024-09-30 01:22:26,123][1157819] Updated weights for policy 0, policy_version 53488 (0.0006) [2024-09-30 01:22:26,745][1157819] Updated weights for policy 0, policy_version 53498 (0.0006) [2024-09-30 01:22:27,327][1157819] Updated weights for policy 0, policy_version 53508 (0.0006) [2024-09-30 01:22:27,955][1157819] Updated weights for policy 0, policy_version 53518 (0.0006) [2024-09-30 01:22:28,616][1157819] Updated weights for policy 0, policy_version 53528 (0.0006) [2024-09-30 01:22:29,212][1157819] Updated weights for policy 0, policy_version 53538 (0.0006) [2024-09-30 01:22:29,844][1157819] Updated weights for policy 0, policy_version 53548 (0.0006) [2024-09-30 01:22:30,466][1157520] Fps is (10 sec: 69631.9, 60 sec: 76390.4, 300 sec: 77574.1). Total num frames: 219369472. Throughput: 0: 19138.9. Samples: 44815716. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:22:30,466][1157520] Avg episode reward: [(0, '51.538')] [2024-09-30 01:22:30,508][1157819] Updated weights for policy 0, policy_version 53558 (0.0006) [2024-09-30 01:22:31,122][1157819] Updated weights for policy 0, policy_version 53568 (0.0006) [2024-09-30 01:22:31,767][1157819] Updated weights for policy 0, policy_version 53578 (0.0006) [2024-09-30 01:22:32,414][1157819] Updated weights for policy 0, policy_version 53588 (0.0006) [2024-09-30 01:22:33,041][1157819] Updated weights for policy 0, policy_version 53598 (0.0006) [2024-09-30 01:22:33,641][1157819] Updated weights for policy 0, policy_version 53608 (0.0007) [2024-09-30 01:22:34,274][1157819] Updated weights for policy 0, policy_version 53618 (0.0007) [2024-09-30 01:22:34,890][1157819] Updated weights for policy 0, policy_version 53628 (0.0006) [2024-09-30 01:22:35,466][1157520] Fps is (10 sec: 65945.6, 60 sec: 75229.9, 300 sec: 77379.7). Total num frames: 219697152. Throughput: 0: 18815.0. Samples: 44913216. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:22:35,466][1157520] Avg episode reward: [(0, '51.906')] [2024-09-30 01:22:35,516][1157819] Updated weights for policy 0, policy_version 53638 (0.0006) [2024-09-30 01:22:36,232][1157819] Updated weights for policy 0, policy_version 53648 (0.0006) [2024-09-30 01:22:36,875][1157819] Updated weights for policy 0, policy_version 53658 (0.0006) [2024-09-30 01:22:37,524][1157819] Updated weights for policy 0, policy_version 53668 (0.0006) [2024-09-30 01:22:38,121][1157819] Updated weights for policy 0, policy_version 53678 (0.0006) [2024-09-30 01:22:38,739][1157819] Updated weights for policy 0, policy_version 53688 (0.0006) [2024-09-30 01:22:39,353][1157819] Updated weights for policy 0, policy_version 53698 (0.0006) [2024-09-30 01:22:40,078][1157819] Updated weights for policy 0, policy_version 53708 (0.0006) [2024-09-30 01:22:40,466][1157520] Fps is (10 sec: 64716.2, 60 sec: 74205.8, 300 sec: 77171.4). Total num frames: 220016640. Throughput: 0: 18548.5. Samples: 44960800. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:22:40,466][1157520] Avg episode reward: [(0, '53.532')] [2024-09-30 01:22:40,669][1157819] Updated weights for policy 0, policy_version 53718 (0.0006) [2024-09-30 01:22:41,281][1157819] Updated weights for policy 0, policy_version 53728 (0.0006) [2024-09-30 01:22:41,943][1157819] Updated weights for policy 0, policy_version 53738 (0.0006) [2024-09-30 01:22:42,583][1157819] Updated weights for policy 0, policy_version 53748 (0.0006) [2024-09-30 01:22:43,110][1157819] Updated weights for policy 0, policy_version 53758 (0.0006) [2024-09-30 01:22:43,658][1157819] Updated weights for policy 0, policy_version 53768 (0.0006) [2024-09-30 01:22:44,218][1157819] Updated weights for policy 0, policy_version 53778 (0.0006) [2024-09-30 01:22:44,757][1157819] Updated weights for policy 0, policy_version 53788 (0.0006) [2024-09-30 01:22:45,346][1157819] Updated weights for policy 0, policy_version 53798 (0.0006) [2024-09-30 01:22:45,466][1157520] Fps is (10 sec: 66355.1, 60 sec: 73659.9, 300 sec: 76990.9). Total num frames: 220360704. Throughput: 0: 18133.7. Samples: 45060544. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:22:45,466][1157520] Avg episode reward: [(0, '51.328')] [2024-09-30 01:22:45,943][1157819] Updated weights for policy 0, policy_version 53808 (0.0006) [2024-09-30 01:22:46,547][1157819] Updated weights for policy 0, policy_version 53818 (0.0006) [2024-09-30 01:22:47,164][1157819] Updated weights for policy 0, policy_version 53828 (0.0006) [2024-09-30 01:22:47,751][1157819] Updated weights for policy 0, policy_version 53838 (0.0006) [2024-09-30 01:22:48,369][1157819] Updated weights for policy 0, policy_version 53848 (0.0006) [2024-09-30 01:22:48,942][1157819] Updated weights for policy 0, policy_version 53858 (0.0006) [2024-09-30 01:22:49,545][1157819] Updated weights for policy 0, policy_version 53868 (0.0006) [2024-09-30 01:22:50,152][1157819] Updated weights for policy 0, policy_version 53878 (0.0006) [2024-09-30 01:22:50,466][1157520] Fps is (10 sec: 68813.4, 60 sec: 73113.7, 300 sec: 76852.1). Total num frames: 220704768. Throughput: 0: 17822.3. Samples: 45164984. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:22:50,466][1157520] Avg episode reward: [(0, '50.061')] [2024-09-30 01:22:50,720][1157819] Updated weights for policy 0, policy_version 53888 (0.0006) [2024-09-30 01:22:51,335][1157819] Updated weights for policy 0, policy_version 53898 (0.0006) [2024-09-30 01:22:51,902][1157819] Updated weights for policy 0, policy_version 53908 (0.0006) [2024-09-30 01:22:52,493][1157819] Updated weights for policy 0, policy_version 53918 (0.0006) [2024-09-30 01:22:53,094][1157819] Updated weights for policy 0, policy_version 53928 (0.0006) [2024-09-30 01:22:53,669][1157819] Updated weights for policy 0, policy_version 53938 (0.0006) [2024-09-30 01:22:54,252][1157819] Updated weights for policy 0, policy_version 53948 (0.0006) [2024-09-30 01:22:54,870][1157819] Updated weights for policy 0, policy_version 53958 (0.0006) [2024-09-30 01:22:55,428][1157819] Updated weights for policy 0, policy_version 53968 (0.0006) [2024-09-30 01:22:55,466][1157520] Fps is (10 sec: 69222.6, 60 sec: 72294.6, 300 sec: 76699.3). Total num frames: 221052928. Throughput: 0: 17665.8. Samples: 45217012. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:22:55,466][1157520] Avg episode reward: [(0, '54.123')] [2024-09-30 01:22:56,010][1157819] Updated weights for policy 0, policy_version 53978 (0.0006) [2024-09-30 01:22:56,598][1157819] Updated weights for policy 0, policy_version 53988 (0.0007) [2024-09-30 01:22:57,174][1157819] Updated weights for policy 0, policy_version 53998 (0.0006) [2024-09-30 01:22:57,765][1157819] Updated weights for policy 0, policy_version 54008 (0.0006) [2024-09-30 01:22:58,384][1157819] Updated weights for policy 0, policy_version 54018 (0.0007) [2024-09-30 01:22:58,923][1157736] Signal inference workers to stop experience collection... (3300 times) [2024-09-30 01:22:58,927][1157736] Signal inference workers to resume experience collection... (3300 times) [2024-09-30 01:22:58,928][1157819] InferenceWorker_p0-w0: stopping experience collection (3300 times) [2024-09-30 01:22:58,931][1157819] InferenceWorker_p0-w0: resuming experience collection (3300 times) [2024-09-30 01:22:58,973][1157819] Updated weights for policy 0, policy_version 54028 (0.0006) [2024-09-30 01:22:59,697][1157819] Updated weights for policy 0, policy_version 54038 (0.0006) [2024-09-30 01:23:00,320][1157819] Updated weights for policy 0, policy_version 54048 (0.0006) [2024-09-30 01:23:00,466][1157520] Fps is (10 sec: 68403.3, 60 sec: 71338.7, 300 sec: 76546.6). Total num frames: 221388800. Throughput: 0: 17336.3. Samples: 45321068. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:23:00,466][1157520] Avg episode reward: [(0, '50.009')] [2024-09-30 01:23:00,902][1157819] Updated weights for policy 0, policy_version 54058 (0.0006) [2024-09-30 01:23:01,480][1157819] Updated weights for policy 0, policy_version 54068 (0.0006) [2024-09-30 01:23:02,046][1157819] Updated weights for policy 0, policy_version 54078 (0.0006) [2024-09-30 01:23:02,670][1157819] Updated weights for policy 0, policy_version 54088 (0.0006) [2024-09-30 01:23:03,256][1157819] Updated weights for policy 0, policy_version 54098 (0.0006) [2024-09-30 01:23:03,851][1157819] Updated weights for policy 0, policy_version 54108 (0.0006) [2024-09-30 01:23:04,468][1157819] Updated weights for policy 0, policy_version 54118 (0.0006) [2024-09-30 01:23:05,016][1157819] Updated weights for policy 0, policy_version 54128 (0.0006) [2024-09-30 01:23:05,466][1157520] Fps is (10 sec: 68403.1, 60 sec: 70587.8, 300 sec: 76421.6). Total num frames: 221736960. Throughput: 0: 17020.0. Samples: 45422892. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 01:23:05,466][1157520] Avg episode reward: [(0, '51.128')] [2024-09-30 01:23:05,610][1157819] Updated weights for policy 0, policy_version 54138 (0.0006) [2024-09-30 01:23:06,203][1157819] Updated weights for policy 0, policy_version 54148 (0.0006) [2024-09-30 01:23:06,789][1157819] Updated weights for policy 0, policy_version 54158 (0.0006) [2024-09-30 01:23:07,352][1157819] Updated weights for policy 0, policy_version 54168 (0.0006) [2024-09-30 01:23:07,916][1157819] Updated weights for policy 0, policy_version 54178 (0.0006) [2024-09-30 01:23:08,554][1157819] Updated weights for policy 0, policy_version 54188 (0.0006) [2024-09-30 01:23:09,082][1157819] Updated weights for policy 0, policy_version 54198 (0.0006) [2024-09-30 01:23:09,677][1157819] Updated weights for policy 0, policy_version 54208 (0.0006) [2024-09-30 01:23:10,287][1157819] Updated weights for policy 0, policy_version 54218 (0.0006) [2024-09-30 01:23:10,466][1157520] Fps is (10 sec: 70041.4, 60 sec: 69836.8, 300 sec: 76282.8). Total num frames: 222089216. Throughput: 0: 16985.2. Samples: 45475516. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 01:23:10,466][1157520] Avg episode reward: [(0, '51.465')] [2024-09-30 01:23:10,881][1157819] Updated weights for policy 0, policy_version 54228 (0.0006) [2024-09-30 01:23:11,466][1157819] Updated weights for policy 0, policy_version 54238 (0.0006) [2024-09-30 01:23:12,082][1157819] Updated weights for policy 0, policy_version 54248 (0.0006) [2024-09-30 01:23:12,660][1157819] Updated weights for policy 0, policy_version 54258 (0.0006) [2024-09-30 01:23:13,235][1157819] Updated weights for policy 0, policy_version 54268 (0.0006) [2024-09-30 01:23:13,855][1157819] Updated weights for policy 0, policy_version 54278 (0.0006) [2024-09-30 01:23:14,422][1157819] Updated weights for policy 0, policy_version 54288 (0.0006) [2024-09-30 01:23:15,028][1157819] Updated weights for policy 0, policy_version 54298 (0.0006) [2024-09-30 01:23:15,466][1157520] Fps is (10 sec: 69631.5, 60 sec: 69085.8, 300 sec: 76060.6). Total num frames: 222433280. Throughput: 0: 16979.5. Samples: 45579796. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 01:23:15,466][1157520] Avg episode reward: [(0, '53.786')] [2024-09-30 01:23:15,643][1157819] Updated weights for policy 0, policy_version 54308 (0.0006) [2024-09-30 01:23:16,317][1157819] Updated weights for policy 0, policy_version 54318 (0.0006) [2024-09-30 01:23:16,937][1157819] Updated weights for policy 0, policy_version 54328 (0.0006) [2024-09-30 01:23:17,551][1157819] Updated weights for policy 0, policy_version 54338 (0.0006) [2024-09-30 01:23:18,191][1157819] Updated weights for policy 0, policy_version 54348 (0.0006) [2024-09-30 01:23:18,794][1157819] Updated weights for policy 0, policy_version 54358 (0.0006) [2024-09-30 01:23:19,401][1157819] Updated weights for policy 0, policy_version 54368 (0.0006) [2024-09-30 01:23:19,984][1157819] Updated weights for policy 0, policy_version 54378 (0.0006) [2024-09-30 01:23:20,466][1157520] Fps is (10 sec: 67174.8, 60 sec: 68130.2, 300 sec: 75810.7). Total num frames: 222760960. Throughput: 0: 17033.0. Samples: 45679700. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 01:23:20,466][1157520] Avg episode reward: [(0, '51.449')] [2024-09-30 01:23:20,563][1157819] Updated weights for policy 0, policy_version 54388 (0.0006) [2024-09-30 01:23:21,168][1157819] Updated weights for policy 0, policy_version 54398 (0.0006) [2024-09-30 01:23:21,724][1157819] Updated weights for policy 0, policy_version 54408 (0.0006) [2024-09-30 01:23:22,266][1157819] Updated weights for policy 0, policy_version 54418 (0.0006) [2024-09-30 01:23:22,886][1157819] Updated weights for policy 0, policy_version 54428 (0.0006) [2024-09-30 01:23:23,444][1157819] Updated weights for policy 0, policy_version 54438 (0.0006) [2024-09-30 01:23:24,008][1157819] Updated weights for policy 0, policy_version 54448 (0.0006) [2024-09-30 01:23:24,591][1157819] Updated weights for policy 0, policy_version 54458 (0.0006) [2024-09-30 01:23:25,178][1157819] Updated weights for policy 0, policy_version 54468 (0.0006) [2024-09-30 01:23:25,466][1157520] Fps is (10 sec: 68403.0, 60 sec: 67993.5, 300 sec: 75755.1). Total num frames: 223117312. Throughput: 0: 17156.3. Samples: 45732836. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 01:23:25,466][1157520] Avg episode reward: [(0, '51.456')] [2024-09-30 01:23:25,770][1157819] Updated weights for policy 0, policy_version 54478 (0.0006) [2024-09-30 01:23:26,340][1157819] Updated weights for policy 0, policy_version 54488 (0.0006) [2024-09-30 01:23:26,941][1157819] Updated weights for policy 0, policy_version 54498 (0.0006) [2024-09-30 01:23:27,511][1157819] Updated weights for policy 0, policy_version 54508 (0.0007) [2024-09-30 01:23:28,105][1157819] Updated weights for policy 0, policy_version 54518 (0.0006) [2024-09-30 01:23:28,696][1157819] Updated weights for policy 0, policy_version 54528 (0.0006) [2024-09-30 01:23:29,266][1157819] Updated weights for policy 0, policy_version 54538 (0.0006) [2024-09-30 01:23:29,852][1157819] Updated weights for policy 0, policy_version 54548 (0.0006) [2024-09-30 01:23:30,403][1157819] Updated weights for policy 0, policy_version 54558 (0.0006) [2024-09-30 01:23:30,466][1157520] Fps is (10 sec: 70860.7, 60 sec: 68334.9, 300 sec: 75630.2). Total num frames: 223469568. Throughput: 0: 17284.3. Samples: 45838336. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 01:23:30,466][1157520] Avg episode reward: [(0, '52.649')] [2024-09-30 01:23:30,987][1157819] Updated weights for policy 0, policy_version 54568 (0.0006) [2024-09-30 01:23:31,569][1157819] Updated weights for policy 0, policy_version 54578 (0.0006) [2024-09-30 01:23:32,119][1157819] Updated weights for policy 0, policy_version 54588 (0.0006) [2024-09-30 01:23:32,700][1157819] Updated weights for policy 0, policy_version 54598 (0.0006) [2024-09-30 01:23:33,291][1157819] Updated weights for policy 0, policy_version 54608 (0.0006) [2024-09-30 01:23:33,864][1157819] Updated weights for policy 0, policy_version 54618 (0.0006) [2024-09-30 01:23:34,448][1157819] Updated weights for policy 0, policy_version 54628 (0.0006) [2024-09-30 01:23:35,058][1157819] Updated weights for policy 0, policy_version 54638 (0.0006) [2024-09-30 01:23:35,466][1157520] Fps is (10 sec: 70861.4, 60 sec: 68812.8, 300 sec: 75602.4). Total num frames: 223825920. Throughput: 0: 17323.1. Samples: 45944524. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 01:23:35,466][1157520] Avg episode reward: [(0, '53.258')] [2024-09-30 01:23:35,639][1157819] Updated weights for policy 0, policy_version 54648 (0.0006) [2024-09-30 01:23:36,162][1157819] Updated weights for policy 0, policy_version 54658 (0.0006) [2024-09-30 01:23:36,704][1157819] Updated weights for policy 0, policy_version 54668 (0.0006) [2024-09-30 01:23:37,268][1157819] Updated weights for policy 0, policy_version 54678 (0.0006) [2024-09-30 01:23:37,800][1157819] Updated weights for policy 0, policy_version 54688 (0.0006) [2024-09-30 01:23:38,357][1157819] Updated weights for policy 0, policy_version 54698 (0.0006) [2024-09-30 01:23:38,895][1157819] Updated weights for policy 0, policy_version 54708 (0.0006) [2024-09-30 01:23:39,434][1157819] Updated weights for policy 0, policy_version 54718 (0.0006) [2024-09-30 01:23:39,980][1157819] Updated weights for policy 0, policy_version 54728 (0.0006) [2024-09-30 01:23:40,466][1157520] Fps is (10 sec: 72907.6, 60 sec: 69700.2, 300 sec: 75588.5). Total num frames: 224198656. Throughput: 0: 17401.2. Samples: 46000068. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:23:40,466][1157520] Avg episode reward: [(0, '51.718')] [2024-09-30 01:23:40,527][1157819] Updated weights for policy 0, policy_version 54738 (0.0006) [2024-09-30 01:23:41,065][1157819] Updated weights for policy 0, policy_version 54748 (0.0006) [2024-09-30 01:23:41,617][1157819] Updated weights for policy 0, policy_version 54758 (0.0007) [2024-09-30 01:23:42,165][1157819] Updated weights for policy 0, policy_version 54768 (0.0007) [2024-09-30 01:23:42,689][1157819] Updated weights for policy 0, policy_version 54778 (0.0006) [2024-09-30 01:23:43,238][1157819] Updated weights for policy 0, policy_version 54788 (0.0006) [2024-09-30 01:23:43,779][1157819] Updated weights for policy 0, policy_version 54798 (0.0006) [2024-09-30 01:23:44,310][1157819] Updated weights for policy 0, policy_version 54808 (0.0006) [2024-09-30 01:23:44,862][1157819] Updated weights for policy 0, policy_version 54818 (0.0006) [2024-09-30 01:23:45,396][1157819] Updated weights for policy 0, policy_version 54828 (0.0006) [2024-09-30 01:23:45,466][1157520] Fps is (10 sec: 75366.6, 60 sec: 70314.7, 300 sec: 75533.0). Total num frames: 224579584. Throughput: 0: 17606.9. Samples: 46113380. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:23:45,466][1157520] Avg episode reward: [(0, '50.993')] [2024-09-30 01:23:45,728][1157736] Signal inference workers to stop experience collection... (3350 times) [2024-09-30 01:23:45,728][1157736] Signal inference workers to resume experience collection... (3350 times) [2024-09-30 01:23:45,732][1157819] InferenceWorker_p0-w0: stopping experience collection (3350 times) [2024-09-30 01:23:45,734][1157819] InferenceWorker_p0-w0: resuming experience collection (3350 times) [2024-09-30 01:23:45,908][1157819] Updated weights for policy 0, policy_version 54838 (0.0006) [2024-09-30 01:23:46,475][1157819] Updated weights for policy 0, policy_version 54848 (0.0006) [2024-09-30 01:23:46,997][1157819] Updated weights for policy 0, policy_version 54858 (0.0006) [2024-09-30 01:23:47,549][1157819] Updated weights for policy 0, policy_version 54868 (0.0006) [2024-09-30 01:23:48,103][1157819] Updated weights for policy 0, policy_version 54878 (0.0006) [2024-09-30 01:23:48,614][1157819] Updated weights for policy 0, policy_version 54888 (0.0006) [2024-09-30 01:23:49,179][1157819] Updated weights for policy 0, policy_version 54898 (0.0006) [2024-09-30 01:23:49,741][1157819] Updated weights for policy 0, policy_version 54908 (0.0006) [2024-09-30 01:23:50,274][1157819] Updated weights for policy 0, policy_version 54918 (0.0006) [2024-09-30 01:23:50,466][1157520] Fps is (10 sec: 75777.2, 60 sec: 70860.8, 300 sec: 75380.3). Total num frames: 224956416. Throughput: 0: 17864.5. Samples: 46226796. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:23:50,466][1157520] Avg episode reward: [(0, '51.743')] [2024-09-30 01:23:50,864][1157819] Updated weights for policy 0, policy_version 54928 (0.0006) [2024-09-30 01:23:51,356][1157819] Updated weights for policy 0, policy_version 54938 (0.0006) [2024-09-30 01:23:51,923][1157819] Updated weights for policy 0, policy_version 54948 (0.0006) [2024-09-30 01:23:52,488][1157819] Updated weights for policy 0, policy_version 54958 (0.0006) [2024-09-30 01:23:53,043][1157819] Updated weights for policy 0, policy_version 54968 (0.0006) [2024-09-30 01:23:53,615][1157819] Updated weights for policy 0, policy_version 54978 (0.0006) [2024-09-30 01:23:54,164][1157819] Updated weights for policy 0, policy_version 54988 (0.0006) [2024-09-30 01:23:54,710][1157819] Updated weights for policy 0, policy_version 54998 (0.0006) [2024-09-30 01:23:55,268][1157819] Updated weights for policy 0, policy_version 55008 (0.0006) [2024-09-30 01:23:55,466][1157520] Fps is (10 sec: 74546.1, 60 sec: 71202.0, 300 sec: 75269.2). Total num frames: 225325056. Throughput: 0: 17929.3. Samples: 46282336. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:23:55,466][1157520] Avg episode reward: [(0, '50.071')] [2024-09-30 01:23:55,810][1157819] Updated weights for policy 0, policy_version 55018 (0.0006) [2024-09-30 01:23:56,384][1157819] Updated weights for policy 0, policy_version 55028 (0.0006) [2024-09-30 01:23:56,910][1157819] Updated weights for policy 0, policy_version 55038 (0.0006) [2024-09-30 01:23:57,464][1157819] Updated weights for policy 0, policy_version 55048 (0.0006) [2024-09-30 01:23:58,021][1157819] Updated weights for policy 0, policy_version 55058 (0.0006) [2024-09-30 01:23:58,577][1157819] Updated weights for policy 0, policy_version 55068 (0.0006) [2024-09-30 01:23:59,123][1157819] Updated weights for policy 0, policy_version 55078 (0.0006) [2024-09-30 01:23:59,690][1157819] Updated weights for policy 0, policy_version 55088 (0.0006) [2024-09-30 01:24:00,226][1157819] Updated weights for policy 0, policy_version 55098 (0.0006) [2024-09-30 01:24:00,466][1157520] Fps is (10 sec: 73728.0, 60 sec: 71748.3, 300 sec: 75227.6). Total num frames: 225693696. Throughput: 0: 18081.8. Samples: 46393476. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:24:00,466][1157520] Avg episode reward: [(0, '52.743')] [2024-09-30 01:24:00,780][1157819] Updated weights for policy 0, policy_version 55108 (0.0006) [2024-09-30 01:24:01,351][1157819] Updated weights for policy 0, policy_version 55118 (0.0006) [2024-09-30 01:24:01,914][1157819] Updated weights for policy 0, policy_version 55128 (0.0006) [2024-09-30 01:24:02,452][1157819] Updated weights for policy 0, policy_version 55138 (0.0006) [2024-09-30 01:24:03,009][1157819] Updated weights for policy 0, policy_version 55148 (0.0006) [2024-09-30 01:24:03,592][1157819] Updated weights for policy 0, policy_version 55158 (0.0006) [2024-09-30 01:24:04,109][1157819] Updated weights for policy 0, policy_version 55168 (0.0006) [2024-09-30 01:24:04,631][1157819] Updated weights for policy 0, policy_version 55178 (0.0006) [2024-09-30 01:24:05,215][1157819] Updated weights for policy 0, policy_version 55188 (0.0006) [2024-09-30 01:24:05,466][1157520] Fps is (10 sec: 74138.1, 60 sec: 72157.8, 300 sec: 75255.3). Total num frames: 226066432. Throughput: 0: 18336.5. Samples: 46504844. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:24:05,466][1157520] Avg episode reward: [(0, '50.677')] [2024-09-30 01:24:05,477][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000055193_226070528.pth... [2024-09-30 01:24:05,527][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000050792_208044032.pth [2024-09-30 01:24:05,742][1157819] Updated weights for policy 0, policy_version 55198 (0.0006) [2024-09-30 01:24:06,260][1157819] Updated weights for policy 0, policy_version 55208 (0.0006) [2024-09-30 01:24:06,845][1157819] Updated weights for policy 0, policy_version 55218 (0.0006) [2024-09-30 01:24:07,363][1157819] Updated weights for policy 0, policy_version 55228 (0.0006) [2024-09-30 01:24:07,894][1157819] Updated weights for policy 0, policy_version 55238 (0.0006) [2024-09-30 01:24:08,431][1157819] Updated weights for policy 0, policy_version 55248 (0.0006) [2024-09-30 01:24:08,946][1157819] Updated weights for policy 0, policy_version 55258 (0.0006) [2024-09-30 01:24:09,496][1157819] Updated weights for policy 0, policy_version 55268 (0.0006) [2024-09-30 01:24:10,030][1157819] Updated weights for policy 0, policy_version 55278 (0.0006) [2024-09-30 01:24:10,466][1157520] Fps is (10 sec: 75776.2, 60 sec: 72704.1, 300 sec: 75269.2). Total num frames: 226451456. Throughput: 0: 18419.5. Samples: 46561712. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:24:10,466][1157520] Avg episode reward: [(0, '51.673')] [2024-09-30 01:24:10,536][1157819] Updated weights for policy 0, policy_version 55288 (0.0006) [2024-09-30 01:24:11,043][1157819] Updated weights for policy 0, policy_version 55298 (0.0006) [2024-09-30 01:24:11,537][1157819] Updated weights for policy 0, policy_version 55308 (0.0006) [2024-09-30 01:24:12,058][1157819] Updated weights for policy 0, policy_version 55318 (0.0006) [2024-09-30 01:24:12,592][1157819] Updated weights for policy 0, policy_version 55328 (0.0006) [2024-09-30 01:24:13,111][1157819] Updated weights for policy 0, policy_version 55338 (0.0006) [2024-09-30 01:24:13,622][1157819] Updated weights for policy 0, policy_version 55348 (0.0006) [2024-09-30 01:24:14,121][1157819] Updated weights for policy 0, policy_version 55358 (0.0006) [2024-09-30 01:24:14,707][1157819] Updated weights for policy 0, policy_version 55368 (0.0006) [2024-09-30 01:24:15,232][1157819] Updated weights for policy 0, policy_version 55378 (0.0006) [2024-09-30 01:24:15,466][1157520] Fps is (10 sec: 77824.1, 60 sec: 73523.2, 300 sec: 75283.1). Total num frames: 226844672. Throughput: 0: 18690.3. Samples: 46679400. Policy #0 lag: (min: 0.0, avg: 1.8, max: 4.0) [2024-09-30 01:24:15,466][1157520] Avg episode reward: [(0, '51.627')] [2024-09-30 01:24:15,718][1157819] Updated weights for policy 0, policy_version 55388 (0.0006) [2024-09-30 01:24:16,220][1157819] Updated weights for policy 0, policy_version 55398 (0.0006) [2024-09-30 01:24:16,678][1157736] Signal inference workers to stop experience collection... (3400 times) [2024-09-30 01:24:16,681][1157819] InferenceWorker_p0-w0: stopping experience collection (3400 times) [2024-09-30 01:24:16,684][1157736] Signal inference workers to resume experience collection... (3400 times) [2024-09-30 01:24:16,685][1157819] InferenceWorker_p0-w0: resuming experience collection (3400 times) [2024-09-30 01:24:16,764][1157819] Updated weights for policy 0, policy_version 55408 (0.0006) [2024-09-30 01:24:17,275][1157819] Updated weights for policy 0, policy_version 55418 (0.0006) [2024-09-30 01:24:17,817][1157819] Updated weights for policy 0, policy_version 55428 (0.0006) [2024-09-30 01:24:18,319][1157819] Updated weights for policy 0, policy_version 55438 (0.0006) [2024-09-30 01:24:18,818][1157819] Updated weights for policy 0, policy_version 55448 (0.0006) [2024-09-30 01:24:19,317][1157819] Updated weights for policy 0, policy_version 55458 (0.0006) [2024-09-30 01:24:19,820][1157819] Updated weights for policy 0, policy_version 55468 (0.0006) [2024-09-30 01:24:20,357][1157819] Updated weights for policy 0, policy_version 55478 (0.0006) [2024-09-30 01:24:20,466][1157520] Fps is (10 sec: 79462.1, 60 sec: 74752.0, 300 sec: 75324.7). Total num frames: 227246080. Throughput: 0: 18969.3. Samples: 46798144. Policy #0 lag: (min: 0.0, avg: 1.8, max: 4.0) [2024-09-30 01:24:20,466][1157520] Avg episode reward: [(0, '50.254')] [2024-09-30 01:24:20,876][1157819] Updated weights for policy 0, policy_version 55488 (0.0006) [2024-09-30 01:24:21,377][1157819] Updated weights for policy 0, policy_version 55498 (0.0006) [2024-09-30 01:24:21,892][1157819] Updated weights for policy 0, policy_version 55508 (0.0006) [2024-09-30 01:24:22,384][1157819] Updated weights for policy 0, policy_version 55518 (0.0006) [2024-09-30 01:24:22,894][1157819] Updated weights for policy 0, policy_version 55528 (0.0006) [2024-09-30 01:24:23,391][1157819] Updated weights for policy 0, policy_version 55538 (0.0006) [2024-09-30 01:24:23,916][1157819] Updated weights for policy 0, policy_version 55548 (0.0006) [2024-09-30 01:24:24,472][1157819] Updated weights for policy 0, policy_version 55558 (0.0006) [2024-09-30 01:24:24,979][1157819] Updated weights for policy 0, policy_version 55568 (0.0006) [2024-09-30 01:24:25,466][1157520] Fps is (10 sec: 79872.4, 60 sec: 75434.8, 300 sec: 75408.1). Total num frames: 227643392. Throughput: 0: 19076.9. Samples: 46858528. Policy #0 lag: (min: 0.0, avg: 1.8, max: 4.0) [2024-09-30 01:24:25,466][1157520] Avg episode reward: [(0, '51.326')] [2024-09-30 01:24:25,492][1157819] Updated weights for policy 0, policy_version 55578 (0.0006) [2024-09-30 01:24:25,998][1157819] Updated weights for policy 0, policy_version 55588 (0.0006) [2024-09-30 01:24:26,509][1157819] Updated weights for policy 0, policy_version 55598 (0.0006) [2024-09-30 01:24:27,045][1157819] Updated weights for policy 0, policy_version 55608 (0.0006) [2024-09-30 01:24:27,573][1157819] Updated weights for policy 0, policy_version 55618 (0.0006) [2024-09-30 01:24:28,071][1157819] Updated weights for policy 0, policy_version 55628 (0.0006) [2024-09-30 01:24:28,623][1157819] Updated weights for policy 0, policy_version 55638 (0.0006) [2024-09-30 01:24:29,123][1157819] Updated weights for policy 0, policy_version 55648 (0.0006) [2024-09-30 01:24:29,653][1157819] Updated weights for policy 0, policy_version 55658 (0.0006) [2024-09-30 01:24:30,158][1157819] Updated weights for policy 0, policy_version 55668 (0.0006) [2024-09-30 01:24:30,466][1157520] Fps is (10 sec: 79052.5, 60 sec: 76117.3, 300 sec: 75463.6). Total num frames: 228036608. Throughput: 0: 19194.5. Samples: 46977132. Policy #0 lag: (min: 0.0, avg: 1.8, max: 4.0) [2024-09-30 01:24:30,466][1157520] Avg episode reward: [(0, '50.947')] [2024-09-30 01:24:30,679][1157819] Updated weights for policy 0, policy_version 55678 (0.0006) [2024-09-30 01:24:31,210][1157819] Updated weights for policy 0, policy_version 55688 (0.0006) [2024-09-30 01:24:31,750][1157819] Updated weights for policy 0, policy_version 55698 (0.0006) [2024-09-30 01:24:32,241][1157819] Updated weights for policy 0, policy_version 55708 (0.0006) [2024-09-30 01:24:32,783][1157819] Updated weights for policy 0, policy_version 55718 (0.0006) [2024-09-30 01:24:33,293][1157819] Updated weights for policy 0, policy_version 55728 (0.0006) [2024-09-30 01:24:33,831][1157819] Updated weights for policy 0, policy_version 55738 (0.0006) [2024-09-30 01:24:34,322][1157819] Updated weights for policy 0, policy_version 55748 (0.0006) [2024-09-30 01:24:34,828][1157819] Updated weights for policy 0, policy_version 55758 (0.0006) [2024-09-30 01:24:35,341][1157819] Updated weights for policy 0, policy_version 55768 (0.0006) [2024-09-30 01:24:35,466][1157520] Fps is (10 sec: 79052.8, 60 sec: 76800.0, 300 sec: 75546.9). Total num frames: 228433920. Throughput: 0: 19298.4. Samples: 47095224. Policy #0 lag: (min: 0.0, avg: 1.8, max: 4.0) [2024-09-30 01:24:35,466][1157520] Avg episode reward: [(0, '50.976')] [2024-09-30 01:24:35,861][1157819] Updated weights for policy 0, policy_version 55778 (0.0006) [2024-09-30 01:24:36,383][1157819] Updated weights for policy 0, policy_version 55788 (0.0006) [2024-09-30 01:24:36,901][1157819] Updated weights for policy 0, policy_version 55798 (0.0006) [2024-09-30 01:24:37,448][1157819] Updated weights for policy 0, policy_version 55808 (0.0006) [2024-09-30 01:24:37,981][1157819] Updated weights for policy 0, policy_version 55818 (0.0006) [2024-09-30 01:24:38,532][1157819] Updated weights for policy 0, policy_version 55828 (0.0006) [2024-09-30 01:24:39,075][1157819] Updated weights for policy 0, policy_version 55838 (0.0006) [2024-09-30 01:24:39,612][1157819] Updated weights for policy 0, policy_version 55848 (0.0006) [2024-09-30 01:24:40,185][1157819] Updated weights for policy 0, policy_version 55858 (0.0006) [2024-09-30 01:24:40,466][1157520] Fps is (10 sec: 77824.2, 60 sec: 76936.7, 300 sec: 75574.7). Total num frames: 228814848. Throughput: 0: 19368.6. Samples: 47153920. Policy #0 lag: (min: 0.0, avg: 1.8, max: 4.0) [2024-09-30 01:24:40,466][1157520] Avg episode reward: [(0, '51.421')] [2024-09-30 01:24:40,695][1157819] Updated weights for policy 0, policy_version 55868 (0.0006) [2024-09-30 01:24:41,221][1157819] Updated weights for policy 0, policy_version 55878 (0.0006) [2024-09-30 01:24:41,791][1157819] Updated weights for policy 0, policy_version 55888 (0.0006) [2024-09-30 01:24:42,320][1157819] Updated weights for policy 0, policy_version 55898 (0.0006) [2024-09-30 01:24:42,870][1157819] Updated weights for policy 0, policy_version 55908 (0.0006) [2024-09-30 01:24:43,412][1157819] Updated weights for policy 0, policy_version 55918 (0.0006) [2024-09-30 01:24:43,945][1157819] Updated weights for policy 0, policy_version 55928 (0.0006) [2024-09-30 01:24:44,502][1157819] Updated weights for policy 0, policy_version 55938 (0.0006) [2024-09-30 01:24:45,053][1157819] Updated weights for policy 0, policy_version 55948 (0.0005) [2024-09-30 01:24:45,466][1157520] Fps is (10 sec: 75776.0, 60 sec: 76868.2, 300 sec: 75602.5). Total num frames: 229191680. Throughput: 0: 19411.6. Samples: 47267000. Policy #0 lag: (min: 0.0, avg: 1.8, max: 4.0) [2024-09-30 01:24:45,466][1157520] Avg episode reward: [(0, '52.161')] [2024-09-30 01:24:45,575][1157819] Updated weights for policy 0, policy_version 55958 (0.0006) [2024-09-30 01:24:46,089][1157819] Updated weights for policy 0, policy_version 55968 (0.0005) [2024-09-30 01:24:46,597][1157819] Updated weights for policy 0, policy_version 55978 (0.0006) [2024-09-30 01:24:47,089][1157819] Updated weights for policy 0, policy_version 55988 (0.0006) [2024-09-30 01:24:47,622][1157819] Updated weights for policy 0, policy_version 55998 (0.0006) [2024-09-30 01:24:48,129][1157819] Updated weights for policy 0, policy_version 56008 (0.0006) [2024-09-30 01:24:48,650][1157819] Updated weights for policy 0, policy_version 56018 (0.0006) [2024-09-30 01:24:49,174][1157819] Updated weights for policy 0, policy_version 56028 (0.0006) [2024-09-30 01:24:49,667][1157819] Updated weights for policy 0, policy_version 56038 (0.0006) [2024-09-30 01:24:50,187][1157819] Updated weights for policy 0, policy_version 56048 (0.0006) [2024-09-30 01:24:50,466][1157520] Fps is (10 sec: 77824.2, 60 sec: 77277.9, 300 sec: 75658.0). Total num frames: 229593088. Throughput: 0: 19571.8. Samples: 47385572. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:24:50,466][1157520] Avg episode reward: [(0, '52.039')] [2024-09-30 01:24:50,681][1157819] Updated weights for policy 0, policy_version 56058 (0.0006) [2024-09-30 01:24:51,257][1157819] Updated weights for policy 0, policy_version 56068 (0.0006) [2024-09-30 01:24:51,794][1157819] Updated weights for policy 0, policy_version 56078 (0.0006) [2024-09-30 01:24:52,328][1157819] Updated weights for policy 0, policy_version 56088 (0.0006) [2024-09-30 01:24:52,899][1157819] Updated weights for policy 0, policy_version 56098 (0.0006) [2024-09-30 01:24:53,395][1157819] Updated weights for policy 0, policy_version 56108 (0.0006) [2024-09-30 01:24:53,945][1157819] Updated weights for policy 0, policy_version 56118 (0.0006) [2024-09-30 01:24:54,512][1157819] Updated weights for policy 0, policy_version 56128 (0.0006) [2024-09-30 01:24:55,018][1157819] Updated weights for policy 0, policy_version 56138 (0.0006) [2024-09-30 01:24:55,466][1157520] Fps is (10 sec: 77822.8, 60 sec: 77414.3, 300 sec: 75630.2). Total num frames: 229969920. Throughput: 0: 19574.8. Samples: 47442584. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:24:55,466][1157520] Avg episode reward: [(0, '54.287')] [2024-09-30 01:24:55,564][1157819] Updated weights for policy 0, policy_version 56148 (0.0006) [2024-09-30 01:24:56,105][1157819] Updated weights for policy 0, policy_version 56158 (0.0006) [2024-09-30 01:24:56,666][1157819] Updated weights for policy 0, policy_version 56168 (0.0006) [2024-09-30 01:24:57,147][1157736] Signal inference workers to stop experience collection... (3450 times) [2024-09-30 01:24:57,148][1157736] Signal inference workers to resume experience collection... (3450 times) [2024-09-30 01:24:57,153][1157819] InferenceWorker_p0-w0: stopping experience collection (3450 times) [2024-09-30 01:24:57,154][1157819] InferenceWorker_p0-w0: resuming experience collection (3450 times) [2024-09-30 01:24:57,164][1157819] Updated weights for policy 0, policy_version 56178 (0.0006) [2024-09-30 01:24:57,701][1157819] Updated weights for policy 0, policy_version 56188 (0.0006) [2024-09-30 01:24:58,198][1157819] Updated weights for policy 0, policy_version 56198 (0.0006) [2024-09-30 01:24:58,685][1157819] Updated weights for policy 0, policy_version 56208 (0.0006) [2024-09-30 01:24:59,189][1157819] Updated weights for policy 0, policy_version 56218 (0.0006) [2024-09-30 01:24:59,701][1157819] Updated weights for policy 0, policy_version 56228 (0.0006) [2024-09-30 01:25:00,203][1157819] Updated weights for policy 0, policy_version 56238 (0.0006) [2024-09-30 01:25:00,466][1157520] Fps is (10 sec: 77824.9, 60 sec: 77960.7, 300 sec: 75685.8). Total num frames: 230371328. Throughput: 0: 19543.0. Samples: 47558832. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:25:00,466][1157520] Avg episode reward: [(0, '52.218')] [2024-09-30 01:25:00,674][1157819] Updated weights for policy 0, policy_version 56248 (0.0006) [2024-09-30 01:25:01,163][1157819] Updated weights for policy 0, policy_version 56258 (0.0006) [2024-09-30 01:25:01,575][1157819] Updated weights for policy 0, policy_version 56268 (0.0006) [2024-09-30 01:25:02,026][1157819] Updated weights for policy 0, policy_version 56278 (0.0006) [2024-09-30 01:25:02,468][1157819] Updated weights for policy 0, policy_version 56288 (0.0006) [2024-09-30 01:25:02,896][1157819] Updated weights for policy 0, policy_version 56298 (0.0006) [2024-09-30 01:25:03,347][1157819] Updated weights for policy 0, policy_version 56308 (0.0006) [2024-09-30 01:25:03,767][1157819] Updated weights for policy 0, policy_version 56318 (0.0006) [2024-09-30 01:25:04,214][1157819] Updated weights for policy 0, policy_version 56328 (0.0006) [2024-09-30 01:25:04,634][1157819] Updated weights for policy 0, policy_version 56338 (0.0005) [2024-09-30 01:25:05,095][1157819] Updated weights for policy 0, policy_version 56348 (0.0006) [2024-09-30 01:25:05,466][1157520] Fps is (10 sec: 86428.0, 60 sec: 79462.6, 300 sec: 76019.0). Total num frames: 230834176. Throughput: 0: 19896.4. Samples: 47693480. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:25:05,466][1157520] Avg episode reward: [(0, '51.997')] [2024-09-30 01:25:05,569][1157819] Updated weights for policy 0, policy_version 56358 (0.0006) [2024-09-30 01:25:06,009][1157819] Updated weights for policy 0, policy_version 56368 (0.0006) [2024-09-30 01:25:06,492][1157819] Updated weights for policy 0, policy_version 56378 (0.0006) [2024-09-30 01:25:07,001][1157819] Updated weights for policy 0, policy_version 56388 (0.0006) [2024-09-30 01:25:07,497][1157819] Updated weights for policy 0, policy_version 56398 (0.0006) [2024-09-30 01:25:07,985][1157819] Updated weights for policy 0, policy_version 56408 (0.0006) [2024-09-30 01:25:08,496][1157819] Updated weights for policy 0, policy_version 56418 (0.0006) [2024-09-30 01:25:09,021][1157819] Updated weights for policy 0, policy_version 56428 (0.0006) [2024-09-30 01:25:09,547][1157819] Updated weights for policy 0, policy_version 56438 (0.0006) [2024-09-30 01:25:10,048][1157819] Updated weights for policy 0, policy_version 56448 (0.0006) [2024-09-30 01:25:10,466][1157520] Fps is (10 sec: 87244.6, 60 sec: 79872.1, 300 sec: 76130.1). Total num frames: 231243776. Throughput: 0: 19980.1. Samples: 47757632. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:25:10,466][1157520] Avg episode reward: [(0, '52.946')] [2024-09-30 01:25:10,543][1157819] Updated weights for policy 0, policy_version 56458 (0.0006) [2024-09-30 01:25:11,039][1157819] Updated weights for policy 0, policy_version 56468 (0.0006) [2024-09-30 01:25:11,518][1157819] Updated weights for policy 0, policy_version 56478 (0.0006) [2024-09-30 01:25:12,009][1157819] Updated weights for policy 0, policy_version 56488 (0.0006) [2024-09-30 01:25:12,504][1157819] Updated weights for policy 0, policy_version 56498 (0.0006) [2024-09-30 01:25:13,000][1157819] Updated weights for policy 0, policy_version 56508 (0.0006) [2024-09-30 01:25:13,502][1157819] Updated weights for policy 0, policy_version 56518 (0.0006) [2024-09-30 01:25:13,991][1157819] Updated weights for policy 0, policy_version 56528 (0.0006) [2024-09-30 01:25:14,483][1157819] Updated weights for policy 0, policy_version 56538 (0.0006) [2024-09-30 01:25:14,992][1157819] Updated weights for policy 0, policy_version 56548 (0.0006) [2024-09-30 01:25:15,466][1157520] Fps is (10 sec: 82329.6, 60 sec: 80213.6, 300 sec: 76185.6). Total num frames: 231657472. Throughput: 0: 20076.8. Samples: 47880584. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:25:15,466][1157520] Avg episode reward: [(0, '51.753')] [2024-09-30 01:25:15,485][1157819] Updated weights for policy 0, policy_version 56558 (0.0006) [2024-09-30 01:25:15,607][1157736] Signal inference workers to stop experience collection... (3500 times) [2024-09-30 01:25:15,611][1157736] Signal inference workers to resume experience collection... (3500 times) [2024-09-30 01:25:15,612][1157819] InferenceWorker_p0-w0: stopping experience collection (3500 times) [2024-09-30 01:25:15,615][1157819] InferenceWorker_p0-w0: resuming experience collection (3500 times) [2024-09-30 01:25:15,986][1157819] Updated weights for policy 0, policy_version 56568 (0.0006) [2024-09-30 01:25:16,513][1157819] Updated weights for policy 0, policy_version 56578 (0.0006) [2024-09-30 01:25:17,034][1157819] Updated weights for policy 0, policy_version 56588 (0.0006) [2024-09-30 01:25:17,534][1157819] Updated weights for policy 0, policy_version 56598 (0.0006) [2024-09-30 01:25:18,038][1157819] Updated weights for policy 0, policy_version 56608 (0.0006) [2024-09-30 01:25:18,496][1157819] Updated weights for policy 0, policy_version 56618 (0.0006) [2024-09-30 01:25:18,968][1157819] Updated weights for policy 0, policy_version 56628 (0.0006) [2024-09-30 01:25:19,456][1157819] Updated weights for policy 0, policy_version 56638 (0.0006) [2024-09-30 01:25:19,952][1157819] Updated weights for policy 0, policy_version 56648 (0.0006) [2024-09-30 01:25:20,457][1157819] Updated weights for policy 0, policy_version 56658 (0.0006) [2024-09-30 01:25:20,466][1157520] Fps is (10 sec: 82739.2, 60 sec: 80418.3, 300 sec: 76269.0). Total num frames: 232071168. Throughput: 0: 20190.6. Samples: 48003800. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:25:20,466][1157520] Avg episode reward: [(0, '49.723')] [2024-09-30 01:25:20,939][1157819] Updated weights for policy 0, policy_version 56668 (0.0006) [2024-09-30 01:25:21,473][1157819] Updated weights for policy 0, policy_version 56678 (0.0006) [2024-09-30 01:25:21,930][1157819] Updated weights for policy 0, policy_version 56688 (0.0006) [2024-09-30 01:25:22,425][1157819] Updated weights for policy 0, policy_version 56698 (0.0006) [2024-09-30 01:25:22,967][1157819] Updated weights for policy 0, policy_version 56708 (0.0006) [2024-09-30 01:25:23,418][1157819] Updated weights for policy 0, policy_version 56718 (0.0005) [2024-09-30 01:25:23,909][1157819] Updated weights for policy 0, policy_version 56728 (0.0006) [2024-09-30 01:25:24,411][1157819] Updated weights for policy 0, policy_version 56738 (0.0006) [2024-09-30 01:25:24,902][1157819] Updated weights for policy 0, policy_version 56748 (0.0006) [2024-09-30 01:25:25,394][1157819] Updated weights for policy 0, policy_version 56758 (0.0006) [2024-09-30 01:25:25,466][1157520] Fps is (10 sec: 82738.8, 60 sec: 80691.3, 300 sec: 76366.1). Total num frames: 232484864. Throughput: 0: 20267.5. Samples: 48065956. Policy #0 lag: (min: 0.0, avg: 2.5, max: 7.0) [2024-09-30 01:25:25,466][1157520] Avg episode reward: [(0, '46.540')] [2024-09-30 01:25:25,903][1157819] Updated weights for policy 0, policy_version 56768 (0.0006) [2024-09-30 01:25:26,385][1157819] Updated weights for policy 0, policy_version 56778 (0.0006) [2024-09-30 01:25:26,884][1157819] Updated weights for policy 0, policy_version 56788 (0.0006) [2024-09-30 01:25:27,376][1157819] Updated weights for policy 0, policy_version 56798 (0.0006) [2024-09-30 01:25:27,869][1157819] Updated weights for policy 0, policy_version 56808 (0.0006) [2024-09-30 01:25:28,353][1157819] Updated weights for policy 0, policy_version 56818 (0.0006) [2024-09-30 01:25:28,844][1157819] Updated weights for policy 0, policy_version 56828 (0.0006) [2024-09-30 01:25:29,348][1157819] Updated weights for policy 0, policy_version 56838 (0.0006) [2024-09-30 01:25:29,822][1157819] Updated weights for policy 0, policy_version 56848 (0.0006) [2024-09-30 01:25:30,308][1157819] Updated weights for policy 0, policy_version 56858 (0.0006) [2024-09-30 01:25:30,466][1157520] Fps is (10 sec: 82739.2, 60 sec: 81032.7, 300 sec: 76491.1). Total num frames: 232898560. Throughput: 0: 20520.1. Samples: 48190400. Policy #0 lag: (min: 0.0, avg: 2.5, max: 7.0) [2024-09-30 01:25:30,466][1157520] Avg episode reward: [(0, '53.509')] [2024-09-30 01:25:30,802][1157819] Updated weights for policy 0, policy_version 56868 (0.0006) [2024-09-30 01:25:31,287][1157819] Updated weights for policy 0, policy_version 56878 (0.0006) [2024-09-30 01:25:31,787][1157819] Updated weights for policy 0, policy_version 56888 (0.0006) [2024-09-30 01:25:32,278][1157819] Updated weights for policy 0, policy_version 56898 (0.0006) [2024-09-30 01:25:32,773][1157819] Updated weights for policy 0, policy_version 56908 (0.0006) [2024-09-30 01:25:33,271][1157819] Updated weights for policy 0, policy_version 56918 (0.0006) [2024-09-30 01:25:33,809][1157819] Updated weights for policy 0, policy_version 56928 (0.0006) [2024-09-30 01:25:34,364][1157819] Updated weights for policy 0, policy_version 56938 (0.0006) [2024-09-30 01:25:34,865][1157819] Updated weights for policy 0, policy_version 56948 (0.0006) [2024-09-30 01:25:35,384][1157819] Updated weights for policy 0, policy_version 56958 (0.0006) [2024-09-30 01:25:35,466][1157520] Fps is (10 sec: 82330.0, 60 sec: 81237.5, 300 sec: 76602.2). Total num frames: 233308160. Throughput: 0: 20619.5. Samples: 48313448. Policy #0 lag: (min: 0.0, avg: 2.5, max: 7.0) [2024-09-30 01:25:35,466][1157520] Avg episode reward: [(0, '49.372')] [2024-09-30 01:25:35,896][1157819] Updated weights for policy 0, policy_version 56968 (0.0006) [2024-09-30 01:25:36,424][1157819] Updated weights for policy 0, policy_version 56978 (0.0006) [2024-09-30 01:25:36,983][1157819] Updated weights for policy 0, policy_version 56988 (0.0006) [2024-09-30 01:25:37,530][1157819] Updated weights for policy 0, policy_version 56998 (0.0006) [2024-09-30 01:25:38,071][1157819] Updated weights for policy 0, policy_version 57008 (0.0006) [2024-09-30 01:25:38,623][1157819] Updated weights for policy 0, policy_version 57018 (0.0006) [2024-09-30 01:25:39,177][1157819] Updated weights for policy 0, policy_version 57028 (0.0006) [2024-09-30 01:25:39,727][1157819] Updated weights for policy 0, policy_version 57038 (0.0006) [2024-09-30 01:25:40,224][1157819] Updated weights for policy 0, policy_version 57048 (0.0006) [2024-09-30 01:25:40,466][1157520] Fps is (10 sec: 79052.8, 60 sec: 81237.5, 300 sec: 76643.8). Total num frames: 233689088. Throughput: 0: 20629.3. Samples: 48370896. Policy #0 lag: (min: 0.0, avg: 2.5, max: 7.0) [2024-09-30 01:25:40,466][1157520] Avg episode reward: [(0, '52.040')] [2024-09-30 01:25:40,732][1157819] Updated weights for policy 0, policy_version 57058 (0.0006) [2024-09-30 01:25:41,217][1157819] Updated weights for policy 0, policy_version 57068 (0.0006) [2024-09-30 01:25:41,704][1157819] Updated weights for policy 0, policy_version 57078 (0.0006) [2024-09-30 01:25:42,189][1157819] Updated weights for policy 0, policy_version 57088 (0.0006) [2024-09-30 01:25:42,338][1157736] Signal inference workers to stop experience collection... (3550 times) [2024-09-30 01:25:42,340][1157736] Signal inference workers to resume experience collection... (3550 times) [2024-09-30 01:25:42,345][1157819] InferenceWorker_p0-w0: stopping experience collection (3550 times) [2024-09-30 01:25:42,348][1157819] InferenceWorker_p0-w0: resuming experience collection (3550 times) [2024-09-30 01:25:42,682][1157819] Updated weights for policy 0, policy_version 57098 (0.0006) [2024-09-30 01:25:43,206][1157819] Updated weights for policy 0, policy_version 57108 (0.0006) [2024-09-30 01:25:43,755][1157819] Updated weights for policy 0, policy_version 57118 (0.0006) [2024-09-30 01:25:44,272][1157819] Updated weights for policy 0, policy_version 57128 (0.0006) [2024-09-30 01:25:44,768][1157819] Updated weights for policy 0, policy_version 57138 (0.0006) [2024-09-30 01:25:45,260][1157819] Updated weights for policy 0, policy_version 57148 (0.0006) [2024-09-30 01:25:45,466][1157520] Fps is (10 sec: 78232.7, 60 sec: 81646.9, 300 sec: 76671.6). Total num frames: 234090496. Throughput: 0: 20693.3. Samples: 48490032. Policy #0 lag: (min: 0.0, avg: 2.5, max: 7.0) [2024-09-30 01:25:45,466][1157520] Avg episode reward: [(0, '51.973')] [2024-09-30 01:25:45,792][1157819] Updated weights for policy 0, policy_version 57158 (0.0006) [2024-09-30 01:25:46,301][1157819] Updated weights for policy 0, policy_version 57168 (0.0006) [2024-09-30 01:25:46,809][1157819] Updated weights for policy 0, policy_version 57178 (0.0006) [2024-09-30 01:25:47,324][1157819] Updated weights for policy 0, policy_version 57188 (0.0006) [2024-09-30 01:25:47,820][1157819] Updated weights for policy 0, policy_version 57198 (0.0006) [2024-09-30 01:25:48,323][1157819] Updated weights for policy 0, policy_version 57208 (0.0006) [2024-09-30 01:25:48,885][1157819] Updated weights for policy 0, policy_version 57218 (0.0006) [2024-09-30 01:25:49,438][1157819] Updated weights for policy 0, policy_version 57228 (0.0006) [2024-09-30 01:25:49,996][1157819] Updated weights for policy 0, policy_version 57238 (0.0006) [2024-09-30 01:25:50,466][1157520] Fps is (10 sec: 79051.7, 60 sec: 81442.1, 300 sec: 76657.7). Total num frames: 234479616. Throughput: 0: 20312.1. Samples: 48607528. Policy #0 lag: (min: 0.0, avg: 2.5, max: 7.0) [2024-09-30 01:25:50,466][1157520] Avg episode reward: [(0, '50.432')] [2024-09-30 01:25:50,560][1157819] Updated weights for policy 0, policy_version 57248 (0.0006) [2024-09-30 01:25:51,100][1157819] Updated weights for policy 0, policy_version 57258 (0.0005) [2024-09-30 01:25:51,652][1157819] Updated weights for policy 0, policy_version 57268 (0.0006) [2024-09-30 01:25:52,174][1157819] Updated weights for policy 0, policy_version 57278 (0.0006) [2024-09-30 01:25:52,708][1157819] Updated weights for policy 0, policy_version 57288 (0.0006) [2024-09-30 01:25:53,272][1157819] Updated weights for policy 0, policy_version 57298 (0.0006) [2024-09-30 01:25:53,814][1157819] Updated weights for policy 0, policy_version 57308 (0.0006) [2024-09-30 01:25:54,380][1157819] Updated weights for policy 0, policy_version 57318 (0.0006) [2024-09-30 01:25:54,924][1157819] Updated weights for policy 0, policy_version 57328 (0.0006) [2024-09-30 01:25:55,443][1157819] Updated weights for policy 0, policy_version 57338 (0.0006) [2024-09-30 01:25:55,466][1157520] Fps is (10 sec: 76595.0, 60 sec: 81442.3, 300 sec: 76560.5). Total num frames: 234856448. Throughput: 0: 20144.6. Samples: 48664144. Policy #0 lag: (min: 0.0, avg: 2.5, max: 7.0) [2024-09-30 01:25:55,466][1157520] Avg episode reward: [(0, '53.278')] [2024-09-30 01:25:56,003][1157819] Updated weights for policy 0, policy_version 57348 (0.0006) [2024-09-30 01:25:56,540][1157819] Updated weights for policy 0, policy_version 57358 (0.0006) [2024-09-30 01:25:57,052][1157819] Updated weights for policy 0, policy_version 57368 (0.0006) [2024-09-30 01:25:57,560][1157819] Updated weights for policy 0, policy_version 57378 (0.0006) [2024-09-30 01:25:58,057][1157819] Updated weights for policy 0, policy_version 57388 (0.0006) [2024-09-30 01:25:58,678][1157819] Updated weights for policy 0, policy_version 57398 (0.0006) [2024-09-30 01:25:59,223][1157819] Updated weights for policy 0, policy_version 57408 (0.0007) [2024-09-30 01:25:59,801][1157819] Updated weights for policy 0, policy_version 57418 (0.0006) [2024-09-30 01:26:00,372][1157819] Updated weights for policy 0, policy_version 57428 (0.0006) [2024-09-30 01:26:00,465][1157520] Fps is (10 sec: 74958.4, 60 sec: 80964.3, 300 sec: 76393.9). Total num frames: 235229184. Throughput: 0: 19943.9. Samples: 48778056. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 01:26:00,466][1157520] Avg episode reward: [(0, '52.679')] [2024-09-30 01:26:00,931][1157819] Updated weights for policy 0, policy_version 57438 (0.0007) [2024-09-30 01:26:01,495][1157819] Updated weights for policy 0, policy_version 57448 (0.0006) [2024-09-30 01:26:02,010][1157819] Updated weights for policy 0, policy_version 57458 (0.0006) [2024-09-30 01:26:02,613][1157819] Updated weights for policy 0, policy_version 57468 (0.0006) [2024-09-30 01:26:03,159][1157819] Updated weights for policy 0, policy_version 57478 (0.0006) [2024-09-30 01:26:03,793][1157819] Updated weights for policy 0, policy_version 57488 (0.0006) [2024-09-30 01:26:04,340][1157819] Updated weights for policy 0, policy_version 57498 (0.0006) [2024-09-30 01:26:04,936][1157819] Updated weights for policy 0, policy_version 57508 (0.0006) [2024-09-30 01:26:05,466][1157520] Fps is (10 sec: 73319.3, 60 sec: 79257.6, 300 sec: 76213.4). Total num frames: 235589632. Throughput: 0: 19589.8. Samples: 48885340. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 01:26:05,466][1157520] Avg episode reward: [(0, '52.424')] [2024-09-30 01:26:05,469][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000057517_235589632.pth... [2024-09-30 01:26:05,537][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000053102_217505792.pth [2024-09-30 01:26:05,567][1157819] Updated weights for policy 0, policy_version 57518 (0.0006) [2024-09-30 01:26:06,077][1157819] Updated weights for policy 0, policy_version 57528 (0.0006) [2024-09-30 01:26:06,643][1157819] Updated weights for policy 0, policy_version 57538 (0.0006) [2024-09-30 01:26:07,178][1157736] Signal inference workers to stop experience collection... (3600 times) [2024-09-30 01:26:07,178][1157736] Signal inference workers to resume experience collection... (3600 times) [2024-09-30 01:26:07,183][1157819] InferenceWorker_p0-w0: stopping experience collection (3600 times) [2024-09-30 01:26:07,183][1157819] InferenceWorker_p0-w0: resuming experience collection (3600 times) [2024-09-30 01:26:07,201][1157819] Updated weights for policy 0, policy_version 57548 (0.0006) [2024-09-30 01:26:07,776][1157819] Updated weights for policy 0, policy_version 57558 (0.0006) [2024-09-30 01:26:08,410][1157819] Updated weights for policy 0, policy_version 57568 (0.0006) [2024-09-30 01:26:09,011][1157819] Updated weights for policy 0, policy_version 57578 (0.0006) [2024-09-30 01:26:09,598][1157819] Updated weights for policy 0, policy_version 57588 (0.0006) [2024-09-30 01:26:10,185][1157819] Updated weights for policy 0, policy_version 57598 (0.0006) [2024-09-30 01:26:10,466][1157520] Fps is (10 sec: 71269.3, 60 sec: 78301.7, 300 sec: 75977.3). Total num frames: 235941888. Throughput: 0: 19415.9. Samples: 48939672. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 01:26:10,466][1157520] Avg episode reward: [(0, '50.127')] [2024-09-30 01:26:10,734][1157819] Updated weights for policy 0, policy_version 57608 (0.0006) [2024-09-30 01:26:11,270][1157819] Updated weights for policy 0, policy_version 57618 (0.0006) [2024-09-30 01:26:11,830][1157819] Updated weights for policy 0, policy_version 57628 (0.0006) [2024-09-30 01:26:12,374][1157819] Updated weights for policy 0, policy_version 57638 (0.0006) [2024-09-30 01:26:12,907][1157819] Updated weights for policy 0, policy_version 57648 (0.0006) [2024-09-30 01:26:13,495][1157819] Updated weights for policy 0, policy_version 57658 (0.0006) [2024-09-30 01:26:14,078][1157819] Updated weights for policy 0, policy_version 57668 (0.0006) [2024-09-30 01:26:14,684][1157819] Updated weights for policy 0, policy_version 57678 (0.0006) [2024-09-30 01:26:15,279][1157819] Updated weights for policy 0, policy_version 57688 (0.0006) [2024-09-30 01:26:15,466][1157520] Fps is (10 sec: 71268.8, 60 sec: 77414.1, 300 sec: 75782.9). Total num frames: 236302336. Throughput: 0: 19022.6. Samples: 49046420. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 01:26:15,466][1157520] Avg episode reward: [(0, '49.727')] [2024-09-30 01:26:15,909][1157819] Updated weights for policy 0, policy_version 57698 (0.0006) [2024-09-30 01:26:16,494][1157819] Updated weights for policy 0, policy_version 57708 (0.0006) [2024-09-30 01:26:17,085][1157819] Updated weights for policy 0, policy_version 57718 (0.0006) [2024-09-30 01:26:17,680][1157819] Updated weights for policy 0, policy_version 57728 (0.0006) [2024-09-30 01:26:18,247][1157819] Updated weights for policy 0, policy_version 57738 (0.0006) [2024-09-30 01:26:18,831][1157819] Updated weights for policy 0, policy_version 57748 (0.0006) [2024-09-30 01:26:19,410][1157819] Updated weights for policy 0, policy_version 57758 (0.0006) [2024-09-30 01:26:20,016][1157819] Updated weights for policy 0, policy_version 57768 (0.0006) [2024-09-30 01:26:20,466][1157520] Fps is (10 sec: 70450.9, 60 sec: 76253.7, 300 sec: 75519.2). Total num frames: 236646400. Throughput: 0: 18602.6. Samples: 49150568. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 01:26:20,466][1157520] Avg episode reward: [(0, '48.974')] [2024-09-30 01:26:20,598][1157819] Updated weights for policy 0, policy_version 57778 (0.0006) [2024-09-30 01:26:21,167][1157819] Updated weights for policy 0, policy_version 57788 (0.0006) [2024-09-30 01:26:21,752][1157819] Updated weights for policy 0, policy_version 57798 (0.0006) [2024-09-30 01:26:22,353][1157819] Updated weights for policy 0, policy_version 57808 (0.0006) [2024-09-30 01:26:22,939][1157819] Updated weights for policy 0, policy_version 57818 (0.0006) [2024-09-30 01:26:23,499][1157819] Updated weights for policy 0, policy_version 57828 (0.0006) [2024-09-30 01:26:24,105][1157819] Updated weights for policy 0, policy_version 57838 (0.0006) [2024-09-30 01:26:24,662][1157819] Updated weights for policy 0, policy_version 57848 (0.0006) [2024-09-30 01:26:25,236][1157819] Updated weights for policy 0, policy_version 57858 (0.0006) [2024-09-30 01:26:25,466][1157520] Fps is (10 sec: 70042.5, 60 sec: 75298.1, 300 sec: 75310.9). Total num frames: 237002752. Throughput: 0: 18492.8. Samples: 49203072. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 01:26:25,466][1157520] Avg episode reward: [(0, '50.194')] [2024-09-30 01:26:25,757][1157819] Updated weights for policy 0, policy_version 57868 (0.0006) [2024-09-30 01:26:26,289][1157819] Updated weights for policy 0, policy_version 57878 (0.0006) [2024-09-30 01:26:26,853][1157819] Updated weights for policy 0, policy_version 57888 (0.0006) [2024-09-30 01:26:27,373][1157819] Updated weights for policy 0, policy_version 57898 (0.0006) [2024-09-30 01:26:27,900][1157819] Updated weights for policy 0, policy_version 57908 (0.0006) [2024-09-30 01:26:28,470][1157819] Updated weights for policy 0, policy_version 57918 (0.0006) [2024-09-30 01:26:29,001][1157819] Updated weights for policy 0, policy_version 57928 (0.0006) [2024-09-30 01:26:29,570][1157819] Updated weights for policy 0, policy_version 57938 (0.0006) [2024-09-30 01:26:30,110][1157819] Updated weights for policy 0, policy_version 57948 (0.0006) [2024-09-30 01:26:30,466][1157520] Fps is (10 sec: 73728.3, 60 sec: 74751.9, 300 sec: 75255.3). Total num frames: 237383680. Throughput: 0: 18308.8. Samples: 49313928. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 01:26:30,466][1157520] Avg episode reward: [(0, '52.058')] [2024-09-30 01:26:30,625][1157819] Updated weights for policy 0, policy_version 57958 (0.0006) [2024-09-30 01:26:31,125][1157819] Updated weights for policy 0, policy_version 57968 (0.0006) [2024-09-30 01:26:31,652][1157819] Updated weights for policy 0, policy_version 57978 (0.0006) [2024-09-30 01:26:32,195][1157819] Updated weights for policy 0, policy_version 57988 (0.0006) [2024-09-30 01:26:32,697][1157819] Updated weights for policy 0, policy_version 57998 (0.0006) [2024-09-30 01:26:33,186][1157819] Updated weights for policy 0, policy_version 58008 (0.0006) [2024-09-30 01:26:33,692][1157819] Updated weights for policy 0, policy_version 58018 (0.0006) [2024-09-30 01:26:34,213][1157819] Updated weights for policy 0, policy_version 58028 (0.0006) [2024-09-30 01:26:34,687][1157819] Updated weights for policy 0, policy_version 58038 (0.0006) [2024-09-30 01:26:35,178][1157819] Updated weights for policy 0, policy_version 58048 (0.0006) [2024-09-30 01:26:35,466][1157520] Fps is (10 sec: 78233.1, 60 sec: 74615.3, 300 sec: 75324.7). Total num frames: 237785088. Throughput: 0: 18335.3. Samples: 49432616. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:26:35,466][1157520] Avg episode reward: [(0, '49.499')] [2024-09-30 01:26:35,668][1157819] Updated weights for policy 0, policy_version 58058 (0.0006) [2024-09-30 01:26:36,160][1157819] Updated weights for policy 0, policy_version 58068 (0.0006) [2024-09-30 01:26:36,648][1157819] Updated weights for policy 0, policy_version 58078 (0.0006) [2024-09-30 01:26:37,142][1157819] Updated weights for policy 0, policy_version 58088 (0.0006) [2024-09-30 01:26:37,640][1157819] Updated weights for policy 0, policy_version 58098 (0.0006) [2024-09-30 01:26:38,124][1157819] Updated weights for policy 0, policy_version 58108 (0.0006) [2024-09-30 01:26:38,623][1157819] Updated weights for policy 0, policy_version 58118 (0.0006) [2024-09-30 01:26:39,112][1157819] Updated weights for policy 0, policy_version 58128 (0.0006) [2024-09-30 01:26:39,601][1157819] Updated weights for policy 0, policy_version 58138 (0.0006) [2024-09-30 01:26:40,093][1157819] Updated weights for policy 0, policy_version 58148 (0.0006) [2024-09-30 01:26:40,466][1157520] Fps is (10 sec: 81919.3, 60 sec: 75229.6, 300 sec: 75463.6). Total num frames: 238202880. Throughput: 0: 18472.1. Samples: 49495388. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:26:40,466][1157520] Avg episode reward: [(0, '52.598')] [2024-09-30 01:26:40,585][1157819] Updated weights for policy 0, policy_version 58158 (0.0006) [2024-09-30 01:26:40,712][1157736] Signal inference workers to stop experience collection... (3650 times) [2024-09-30 01:26:40,712][1157736] Signal inference workers to resume experience collection... (3650 times) [2024-09-30 01:26:40,716][1157819] InferenceWorker_p0-w0: stopping experience collection (3650 times) [2024-09-30 01:26:40,716][1157819] InferenceWorker_p0-w0: resuming experience collection (3650 times) [2024-09-30 01:26:41,081][1157819] Updated weights for policy 0, policy_version 58168 (0.0006) [2024-09-30 01:26:41,589][1157819] Updated weights for policy 0, policy_version 58178 (0.0006) [2024-09-30 01:26:42,085][1157819] Updated weights for policy 0, policy_version 58188 (0.0006) [2024-09-30 01:26:42,578][1157819] Updated weights for policy 0, policy_version 58198 (0.0006) [2024-09-30 01:26:43,074][1157819] Updated weights for policy 0, policy_version 58208 (0.0006) [2024-09-30 01:26:43,578][1157819] Updated weights for policy 0, policy_version 58218 (0.0006) [2024-09-30 01:26:44,084][1157819] Updated weights for policy 0, policy_version 58228 (0.0006) [2024-09-30 01:26:44,569][1157819] Updated weights for policy 0, policy_version 58238 (0.0006) [2024-09-30 01:26:45,088][1157819] Updated weights for policy 0, policy_version 58248 (0.0006) [2024-09-30 01:26:45,466][1157520] Fps is (10 sec: 82739.1, 60 sec: 75366.3, 300 sec: 75574.7). Total num frames: 238612480. Throughput: 0: 18702.3. Samples: 49619664. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:26:45,466][1157520] Avg episode reward: [(0, '52.482')] [2024-09-30 01:26:45,585][1157819] Updated weights for policy 0, policy_version 58258 (0.0006) [2024-09-30 01:26:46,101][1157819] Updated weights for policy 0, policy_version 58268 (0.0006) [2024-09-30 01:26:46,626][1157819] Updated weights for policy 0, policy_version 58278 (0.0006) [2024-09-30 01:26:47,179][1157819] Updated weights for policy 0, policy_version 58288 (0.0006) [2024-09-30 01:26:47,696][1157819] Updated weights for policy 0, policy_version 58298 (0.0006) [2024-09-30 01:26:48,210][1157819] Updated weights for policy 0, policy_version 58308 (0.0006) [2024-09-30 01:26:48,699][1157819] Updated weights for policy 0, policy_version 58318 (0.0006) [2024-09-30 01:26:49,225][1157819] Updated weights for policy 0, policy_version 58328 (0.0006) [2024-09-30 01:26:49,739][1157819] Updated weights for policy 0, policy_version 58338 (0.0006) [2024-09-30 01:26:50,230][1157819] Updated weights for policy 0, policy_version 58348 (0.0006) [2024-09-30 01:26:50,466][1157520] Fps is (10 sec: 80692.3, 60 sec: 75503.1, 300 sec: 75574.7). Total num frames: 239009792. Throughput: 0: 18977.9. Samples: 49739344. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:26:50,466][1157520] Avg episode reward: [(0, '52.322')] [2024-09-30 01:26:50,750][1157819] Updated weights for policy 0, policy_version 58358 (0.0006) [2024-09-30 01:26:51,294][1157819] Updated weights for policy 0, policy_version 58368 (0.0006) [2024-09-30 01:26:51,816][1157819] Updated weights for policy 0, policy_version 58378 (0.0006) [2024-09-30 01:26:52,363][1157819] Updated weights for policy 0, policy_version 58388 (0.0006) [2024-09-30 01:26:52,900][1157819] Updated weights for policy 0, policy_version 58398 (0.0006) [2024-09-30 01:26:53,447][1157819] Updated weights for policy 0, policy_version 58408 (0.0006) [2024-09-30 01:26:54,017][1157819] Updated weights for policy 0, policy_version 58418 (0.0006) [2024-09-30 01:26:54,556][1157819] Updated weights for policy 0, policy_version 58428 (0.0006) [2024-09-30 01:26:55,079][1157819] Updated weights for policy 0, policy_version 58438 (0.0006) [2024-09-30 01:26:55,466][1157520] Fps is (10 sec: 77825.1, 60 sec: 75571.3, 300 sec: 75533.0). Total num frames: 239390720. Throughput: 0: 19058.0. Samples: 49797280. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:26:55,466][1157520] Avg episode reward: [(0, '51.687')] [2024-09-30 01:26:55,568][1157819] Updated weights for policy 0, policy_version 58448 (0.0006) [2024-09-30 01:26:56,102][1157819] Updated weights for policy 0, policy_version 58458 (0.0006) [2024-09-30 01:26:56,621][1157819] Updated weights for policy 0, policy_version 58468 (0.0006) [2024-09-30 01:26:57,135][1157819] Updated weights for policy 0, policy_version 58478 (0.0006) [2024-09-30 01:26:57,651][1157819] Updated weights for policy 0, policy_version 58488 (0.0006) [2024-09-30 01:26:58,224][1157819] Updated weights for policy 0, policy_version 58498 (0.0006) [2024-09-30 01:26:58,790][1157819] Updated weights for policy 0, policy_version 58508 (0.0006) [2024-09-30 01:26:59,423][1157819] Updated weights for policy 0, policy_version 58518 (0.0006) [2024-09-30 01:27:00,026][1157819] Updated weights for policy 0, policy_version 58528 (0.0006) [2024-09-30 01:27:00,466][1157520] Fps is (10 sec: 74546.5, 60 sec: 75434.4, 300 sec: 75435.8). Total num frames: 239755264. Throughput: 0: 19235.8. Samples: 49912028. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:27:00,466][1157520] Avg episode reward: [(0, '51.567')] [2024-09-30 01:27:00,717][1157819] Updated weights for policy 0, policy_version 58538 (0.0006) [2024-09-30 01:27:01,356][1157819] Updated weights for policy 0, policy_version 58548 (0.0006) [2024-09-30 01:27:01,978][1157819] Updated weights for policy 0, policy_version 58558 (0.0006) [2024-09-30 01:27:02,600][1157819] Updated weights for policy 0, policy_version 58568 (0.0006) [2024-09-30 01:27:03,185][1157819] Updated weights for policy 0, policy_version 58578 (0.0006) [2024-09-30 01:27:03,762][1157819] Updated weights for policy 0, policy_version 58588 (0.0006) [2024-09-30 01:27:04,364][1157819] Updated weights for policy 0, policy_version 58598 (0.0006) [2024-09-30 01:27:04,950][1157819] Updated weights for policy 0, policy_version 58608 (0.0006) [2024-09-30 01:27:05,466][1157520] Fps is (10 sec: 70040.3, 60 sec: 75024.8, 300 sec: 75227.5). Total num frames: 240091136. Throughput: 0: 19136.9. Samples: 50011728. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:27:05,466][1157520] Avg episode reward: [(0, '50.848')] [2024-09-30 01:27:05,557][1157819] Updated weights for policy 0, policy_version 58618 (0.0006) [2024-09-30 01:27:06,138][1157819] Updated weights for policy 0, policy_version 58628 (0.0006) [2024-09-30 01:27:06,693][1157819] Updated weights for policy 0, policy_version 58638 (0.0006) [2024-09-30 01:27:07,272][1157819] Updated weights for policy 0, policy_version 58648 (0.0006) [2024-09-30 01:27:07,802][1157819] Updated weights for policy 0, policy_version 58658 (0.0006) [2024-09-30 01:27:08,318][1157819] Updated weights for policy 0, policy_version 58668 (0.0006) [2024-09-30 01:27:08,827][1157819] Updated weights for policy 0, policy_version 58678 (0.0006) [2024-09-30 01:27:09,342][1157819] Updated weights for policy 0, policy_version 58688 (0.0006) [2024-09-30 01:27:09,846][1157819] Updated weights for policy 0, policy_version 58698 (0.0006) [2024-09-30 01:27:10,339][1157819] Updated weights for policy 0, policy_version 58708 (0.0006) [2024-09-30 01:27:10,466][1157520] Fps is (10 sec: 72090.4, 60 sec: 75571.3, 300 sec: 75213.7). Total num frames: 240476160. Throughput: 0: 19173.5. Samples: 50065876. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 01:27:10,466][1157520] Avg episode reward: [(0, '50.637')] [2024-09-30 01:27:10,856][1157819] Updated weights for policy 0, policy_version 58718 (0.0006) [2024-09-30 01:27:11,384][1157819] Updated weights for policy 0, policy_version 58728 (0.0006) [2024-09-30 01:27:11,872][1157819] Updated weights for policy 0, policy_version 58738 (0.0006) [2024-09-30 01:27:12,365][1157819] Updated weights for policy 0, policy_version 58748 (0.0006) [2024-09-30 01:27:12,865][1157819] Updated weights for policy 0, policy_version 58758 (0.0006) [2024-09-30 01:27:13,356][1157819] Updated weights for policy 0, policy_version 58768 (0.0006) [2024-09-30 01:27:13,868][1157819] Updated weights for policy 0, policy_version 58778 (0.0006) [2024-09-30 01:27:14,368][1157819] Updated weights for policy 0, policy_version 58788 (0.0006) [2024-09-30 01:27:14,851][1157819] Updated weights for policy 0, policy_version 58798 (0.0006) [2024-09-30 01:27:15,356][1157819] Updated weights for policy 0, policy_version 58808 (0.0006) [2024-09-30 01:27:15,466][1157520] Fps is (10 sec: 79464.1, 60 sec: 76390.7, 300 sec: 75297.0). Total num frames: 240885760. Throughput: 0: 19403.3. Samples: 50187076. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 01:27:15,466][1157520] Avg episode reward: [(0, '51.894')] [2024-09-30 01:27:15,868][1157819] Updated weights for policy 0, policy_version 58818 (0.0006) [2024-09-30 01:27:16,361][1157819] Updated weights for policy 0, policy_version 58828 (0.0006) [2024-09-30 01:27:16,846][1157736] Signal inference workers to stop experience collection... (3700 times) [2024-09-30 01:27:16,847][1157736] Signal inference workers to resume experience collection... (3700 times) [2024-09-30 01:27:16,851][1157819] InferenceWorker_p0-w0: stopping experience collection (3700 times) [2024-09-30 01:27:16,851][1157819] InferenceWorker_p0-w0: resuming experience collection (3700 times) [2024-09-30 01:27:16,862][1157819] Updated weights for policy 0, policy_version 58838 (0.0006) [2024-09-30 01:27:17,397][1157819] Updated weights for policy 0, policy_version 58848 (0.0006) [2024-09-30 01:27:17,887][1157819] Updated weights for policy 0, policy_version 58858 (0.0006) [2024-09-30 01:27:18,400][1157819] Updated weights for policy 0, policy_version 58868 (0.0006) [2024-09-30 01:27:18,936][1157819] Updated weights for policy 0, policy_version 58878 (0.0006) [2024-09-30 01:27:19,427][1157819] Updated weights for policy 0, policy_version 58888 (0.0006) [2024-09-30 01:27:19,935][1157819] Updated weights for policy 0, policy_version 58898 (0.0006) [2024-09-30 01:27:20,436][1157819] Updated weights for policy 0, policy_version 58908 (0.0006) [2024-09-30 01:27:20,466][1157520] Fps is (10 sec: 81101.1, 60 sec: 77346.3, 300 sec: 75422.0). Total num frames: 241287168. Throughput: 0: 19467.1. Samples: 50308632. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 01:27:20,466][1157520] Avg episode reward: [(0, '51.137')] [2024-09-30 01:27:20,953][1157819] Updated weights for policy 0, policy_version 58918 (0.0006) [2024-09-30 01:27:21,455][1157819] Updated weights for policy 0, policy_version 58928 (0.0006) [2024-09-30 01:27:21,955][1157819] Updated weights for policy 0, policy_version 58938 (0.0006) [2024-09-30 01:27:22,474][1157819] Updated weights for policy 0, policy_version 58948 (0.0006) [2024-09-30 01:27:22,996][1157819] Updated weights for policy 0, policy_version 58958 (0.0006) [2024-09-30 01:27:23,553][1157819] Updated weights for policy 0, policy_version 58968 (0.0006) [2024-09-30 01:27:24,042][1157819] Updated weights for policy 0, policy_version 58978 (0.0006) [2024-09-30 01:27:24,539][1157819] Updated weights for policy 0, policy_version 58988 (0.0006) [2024-09-30 01:27:25,056][1157819] Updated weights for policy 0, policy_version 58998 (0.0006) [2024-09-30 01:27:25,466][1157520] Fps is (10 sec: 79871.7, 60 sec: 78028.9, 300 sec: 75644.1). Total num frames: 241684480. Throughput: 0: 19406.0. Samples: 50368656. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 01:27:25,466][1157520] Avg episode reward: [(0, '47.767')] [2024-09-30 01:27:25,557][1157819] Updated weights for policy 0, policy_version 59008 (0.0006) [2024-09-30 01:27:26,083][1157819] Updated weights for policy 0, policy_version 59018 (0.0006) [2024-09-30 01:27:26,600][1157819] Updated weights for policy 0, policy_version 59028 (0.0006) [2024-09-30 01:27:27,143][1157819] Updated weights for policy 0, policy_version 59038 (0.0006) [2024-09-30 01:27:27,633][1157819] Updated weights for policy 0, policy_version 59048 (0.0006) [2024-09-30 01:27:28,140][1157819] Updated weights for policy 0, policy_version 59058 (0.0005) [2024-09-30 01:27:28,676][1157819] Updated weights for policy 0, policy_version 59068 (0.0006) [2024-09-30 01:27:29,232][1157819] Updated weights for policy 0, policy_version 59078 (0.0006) [2024-09-30 01:27:29,755][1157819] Updated weights for policy 0, policy_version 59088 (0.0006) [2024-09-30 01:27:30,281][1157819] Updated weights for policy 0, policy_version 59098 (0.0006) [2024-09-30 01:27:30,466][1157520] Fps is (10 sec: 79052.7, 60 sec: 78233.7, 300 sec: 75866.3). Total num frames: 242077696. Throughput: 0: 19296.3. Samples: 50487992. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 01:27:30,466][1157520] Avg episode reward: [(0, '52.419')] [2024-09-30 01:27:30,839][1157819] Updated weights for policy 0, policy_version 59108 (0.0006) [2024-09-30 01:27:31,383][1157819] Updated weights for policy 0, policy_version 59118 (0.0006) [2024-09-30 01:27:31,942][1157819] Updated weights for policy 0, policy_version 59128 (0.0006) [2024-09-30 01:27:32,507][1157819] Updated weights for policy 0, policy_version 59138 (0.0006) [2024-09-30 01:27:33,056][1157819] Updated weights for policy 0, policy_version 59148 (0.0006) [2024-09-30 01:27:33,655][1157819] Updated weights for policy 0, policy_version 59158 (0.0006) [2024-09-30 01:27:34,192][1157819] Updated weights for policy 0, policy_version 59168 (0.0006) [2024-09-30 01:27:34,767][1157819] Updated weights for policy 0, policy_version 59178 (0.0006) [2024-09-30 01:27:35,315][1157819] Updated weights for policy 0, policy_version 59188 (0.0006) [2024-09-30 01:27:35,466][1157520] Fps is (10 sec: 75776.3, 60 sec: 77619.4, 300 sec: 76019.0). Total num frames: 242442240. Throughput: 0: 19099.4. Samples: 50598816. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 01:27:35,466][1157520] Avg episode reward: [(0, '53.023')] [2024-09-30 01:27:35,893][1157819] Updated weights for policy 0, policy_version 59198 (0.0006) [2024-09-30 01:27:36,427][1157819] Updated weights for policy 0, policy_version 59208 (0.0006) [2024-09-30 01:27:36,987][1157819] Updated weights for policy 0, policy_version 59218 (0.0006) [2024-09-30 01:27:37,540][1157819] Updated weights for policy 0, policy_version 59228 (0.0006) [2024-09-30 01:27:38,115][1157819] Updated weights for policy 0, policy_version 59238 (0.0006) [2024-09-30 01:27:38,676][1157819] Updated weights for policy 0, policy_version 59248 (0.0006) [2024-09-30 01:27:39,255][1157819] Updated weights for policy 0, policy_version 59258 (0.0006) [2024-09-30 01:27:39,782][1157819] Updated weights for policy 0, policy_version 59268 (0.0006) [2024-09-30 01:27:40,331][1157819] Updated weights for policy 0, policy_version 59278 (0.0006) [2024-09-30 01:27:40,466][1157520] Fps is (10 sec: 72908.6, 60 sec: 76731.9, 300 sec: 76088.4). Total num frames: 242806784. Throughput: 0: 19032.9. Samples: 50653760. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 01:27:40,466][1157520] Avg episode reward: [(0, '53.170')] [2024-09-30 01:27:40,871][1157819] Updated weights for policy 0, policy_version 59288 (0.0006) [2024-09-30 01:27:41,400][1157819] Updated weights for policy 0, policy_version 59298 (0.0006) [2024-09-30 01:27:41,909][1157819] Updated weights for policy 0, policy_version 59308 (0.0006) [2024-09-30 01:27:42,439][1157819] Updated weights for policy 0, policy_version 59318 (0.0006) [2024-09-30 01:27:42,946][1157819] Updated weights for policy 0, policy_version 59328 (0.0006) [2024-09-30 01:27:43,491][1157819] Updated weights for policy 0, policy_version 59338 (0.0006) [2024-09-30 01:27:43,988][1157819] Updated weights for policy 0, policy_version 59348 (0.0006) [2024-09-30 01:27:44,498][1157819] Updated weights for policy 0, policy_version 59358 (0.0006) [2024-09-30 01:27:45,025][1157819] Updated weights for policy 0, policy_version 59368 (0.0006) [2024-09-30 01:27:45,466][1157520] Fps is (10 sec: 76594.8, 60 sec: 76595.4, 300 sec: 76282.8). Total num frames: 243208192. Throughput: 0: 19033.0. Samples: 50768512. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:27:45,466][1157520] Avg episode reward: [(0, '52.437')] [2024-09-30 01:27:45,529][1157819] Updated weights for policy 0, policy_version 59378 (0.0006) [2024-09-30 01:27:46,041][1157819] Updated weights for policy 0, policy_version 59388 (0.0006) [2024-09-30 01:27:46,530][1157819] Updated weights for policy 0, policy_version 59398 (0.0006) [2024-09-30 01:27:47,035][1157819] Updated weights for policy 0, policy_version 59408 (0.0006) [2024-09-30 01:27:47,539][1157819] Updated weights for policy 0, policy_version 59418 (0.0006) [2024-09-30 01:27:48,063][1157819] Updated weights for policy 0, policy_version 59428 (0.0006) [2024-09-30 01:27:48,559][1157819] Updated weights for policy 0, policy_version 59438 (0.0006) [2024-09-30 01:27:49,060][1157819] Updated weights for policy 0, policy_version 59448 (0.0006) [2024-09-30 01:27:49,578][1157819] Updated weights for policy 0, policy_version 59458 (0.0007) [2024-09-30 01:27:50,052][1157819] Updated weights for policy 0, policy_version 59468 (0.0006) [2024-09-30 01:27:50,466][1157520] Fps is (10 sec: 80691.1, 60 sec: 76731.8, 300 sec: 76477.2). Total num frames: 243613696. Throughput: 0: 19513.2. Samples: 50889820. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:27:50,466][1157520] Avg episode reward: [(0, '52.901')] [2024-09-30 01:27:50,547][1157819] Updated weights for policy 0, policy_version 59478 (0.0006) [2024-09-30 01:27:51,049][1157819] Updated weights for policy 0, policy_version 59488 (0.0006) [2024-09-30 01:27:51,549][1157819] Updated weights for policy 0, policy_version 59498 (0.0006) [2024-09-30 01:27:52,039][1157819] Updated weights for policy 0, policy_version 59508 (0.0006) [2024-09-30 01:27:52,560][1157819] Updated weights for policy 0, policy_version 59518 (0.0006) [2024-09-30 01:27:53,068][1157819] Updated weights for policy 0, policy_version 59528 (0.0006) [2024-09-30 01:27:53,226][1157736] Signal inference workers to stop experience collection... (3750 times) [2024-09-30 01:27:53,230][1157736] Signal inference workers to resume experience collection... (3750 times) [2024-09-30 01:27:53,230][1157819] InferenceWorker_p0-w0: stopping experience collection (3750 times) [2024-09-30 01:27:53,234][1157819] InferenceWorker_p0-w0: resuming experience collection (3750 times) [2024-09-30 01:27:53,579][1157819] Updated weights for policy 0, policy_version 59538 (0.0006) [2024-09-30 01:27:54,077][1157819] Updated weights for policy 0, policy_version 59548 (0.0006) [2024-09-30 01:27:54,572][1157819] Updated weights for policy 0, policy_version 59558 (0.0006) [2024-09-30 01:27:55,051][1157819] Updated weights for policy 0, policy_version 59568 (0.0006) [2024-09-30 01:27:55,466][1157520] Fps is (10 sec: 81510.2, 60 sec: 77209.6, 300 sec: 76727.1). Total num frames: 244023296. Throughput: 0: 19674.4. Samples: 50951224. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:27:55,466][1157520] Avg episode reward: [(0, '51.939')] [2024-09-30 01:27:55,565][1157819] Updated weights for policy 0, policy_version 59578 (0.0006) [2024-09-30 01:27:56,052][1157819] Updated weights for policy 0, policy_version 59588 (0.0006) [2024-09-30 01:27:56,548][1157819] Updated weights for policy 0, policy_version 59598 (0.0006) [2024-09-30 01:27:57,056][1157819] Updated weights for policy 0, policy_version 59608 (0.0006) [2024-09-30 01:27:57,551][1157819] Updated weights for policy 0, policy_version 59618 (0.0006) [2024-09-30 01:27:58,013][1157819] Updated weights for policy 0, policy_version 59628 (0.0006) [2024-09-30 01:27:58,547][1157819] Updated weights for policy 0, policy_version 59638 (0.0006) [2024-09-30 01:27:59,040][1157819] Updated weights for policy 0, policy_version 59648 (0.0006) [2024-09-30 01:27:59,543][1157819] Updated weights for policy 0, policy_version 59658 (0.0005) [2024-09-30 01:28:00,143][1157819] Updated weights for policy 0, policy_version 59668 (0.0006) [2024-09-30 01:28:00,466][1157520] Fps is (10 sec: 80691.0, 60 sec: 77755.8, 300 sec: 76893.7). Total num frames: 244420608. Throughput: 0: 19729.0. Samples: 51074880. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:28:00,466][1157520] Avg episode reward: [(0, '53.030')] [2024-09-30 01:28:00,708][1157819] Updated weights for policy 0, policy_version 59678 (0.0006) [2024-09-30 01:28:01,260][1157819] Updated weights for policy 0, policy_version 59688 (0.0006) [2024-09-30 01:28:01,849][1157819] Updated weights for policy 0, policy_version 59698 (0.0006) [2024-09-30 01:28:02,347][1157819] Updated weights for policy 0, policy_version 59708 (0.0006) [2024-09-30 01:28:02,875][1157819] Updated weights for policy 0, policy_version 59718 (0.0006) [2024-09-30 01:28:03,406][1157819] Updated weights for policy 0, policy_version 59728 (0.0006) [2024-09-30 01:28:03,936][1157819] Updated weights for policy 0, policy_version 59738 (0.0006) [2024-09-30 01:28:04,473][1157819] Updated weights for policy 0, policy_version 59748 (0.0006) [2024-09-30 01:28:05,065][1157819] Updated weights for policy 0, policy_version 59758 (0.0006) [2024-09-30 01:28:05,466][1157520] Fps is (10 sec: 77824.0, 60 sec: 78506.9, 300 sec: 76990.9). Total num frames: 244801536. Throughput: 0: 19532.1. Samples: 51187576. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:28:05,466][1157520] Avg episode reward: [(0, '50.367')] [2024-09-30 01:28:05,470][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000059766_244801536.pth... [2024-09-30 01:28:05,513][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000055193_226070528.pth [2024-09-30 01:28:05,619][1157819] Updated weights for policy 0, policy_version 59768 (0.0006) [2024-09-30 01:28:06,164][1157819] Updated weights for policy 0, policy_version 59778 (0.0006) [2024-09-30 01:28:06,702][1157819] Updated weights for policy 0, policy_version 59788 (0.0006) [2024-09-30 01:28:07,252][1157819] Updated weights for policy 0, policy_version 59798 (0.0006) [2024-09-30 01:28:07,790][1157819] Updated weights for policy 0, policy_version 59808 (0.0006) [2024-09-30 01:28:08,335][1157819] Updated weights for policy 0, policy_version 59818 (0.0006) [2024-09-30 01:28:08,898][1157819] Updated weights for policy 0, policy_version 59828 (0.0006) [2024-09-30 01:28:09,430][1157819] Updated weights for policy 0, policy_version 59838 (0.0006) [2024-09-30 01:28:09,994][1157819] Updated weights for policy 0, policy_version 59848 (0.0006) [2024-09-30 01:28:10,466][1157520] Fps is (10 sec: 74957.2, 60 sec: 78233.6, 300 sec: 77074.3). Total num frames: 245170176. Throughput: 0: 19431.0. Samples: 51243052. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:28:10,466][1157520] Avg episode reward: [(0, '49.992')] [2024-09-30 01:28:10,535][1157819] Updated weights for policy 0, policy_version 59858 (0.0006) [2024-09-30 01:28:11,017][1157819] Updated weights for policy 0, policy_version 59868 (0.0006) [2024-09-30 01:28:11,560][1157819] Updated weights for policy 0, policy_version 59878 (0.0006) [2024-09-30 01:28:12,076][1157819] Updated weights for policy 0, policy_version 59888 (0.0006) [2024-09-30 01:28:12,558][1157819] Updated weights for policy 0, policy_version 59898 (0.0006) [2024-09-30 01:28:13,061][1157819] Updated weights for policy 0, policy_version 59908 (0.0005) [2024-09-30 01:28:13,567][1157819] Updated weights for policy 0, policy_version 59918 (0.0006) [2024-09-30 01:28:14,070][1157819] Updated weights for policy 0, policy_version 59928 (0.0006) [2024-09-30 01:28:14,580][1157819] Updated weights for policy 0, policy_version 59938 (0.0006) [2024-09-30 01:28:15,111][1157819] Updated weights for policy 0, policy_version 59948 (0.0006) [2024-09-30 01:28:15,466][1157520] Fps is (10 sec: 77414.9, 60 sec: 78165.3, 300 sec: 77338.0). Total num frames: 245575680. Throughput: 0: 19382.2. Samples: 51360192. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:28:15,466][1157520] Avg episode reward: [(0, '51.770')] [2024-09-30 01:28:15,605][1157819] Updated weights for policy 0, policy_version 59958 (0.0006) [2024-09-30 01:28:16,093][1157819] Updated weights for policy 0, policy_version 59968 (0.0006) [2024-09-30 01:28:16,579][1157819] Updated weights for policy 0, policy_version 59978 (0.0006) [2024-09-30 01:28:17,119][1157819] Updated weights for policy 0, policy_version 59988 (0.0006) [2024-09-30 01:28:17,604][1157819] Updated weights for policy 0, policy_version 59998 (0.0006) [2024-09-30 01:28:18,098][1157819] Updated weights for policy 0, policy_version 60008 (0.0006) [2024-09-30 01:28:18,588][1157819] Updated weights for policy 0, policy_version 60018 (0.0006) [2024-09-30 01:28:19,090][1157819] Updated weights for policy 0, policy_version 60028 (0.0006) [2024-09-30 01:28:19,569][1157819] Updated weights for policy 0, policy_version 60038 (0.0006) [2024-09-30 01:28:20,056][1157819] Updated weights for policy 0, policy_version 60048 (0.0006) [2024-09-30 01:28:20,466][1157520] Fps is (10 sec: 81920.0, 60 sec: 78370.1, 300 sec: 77532.5). Total num frames: 245989376. Throughput: 0: 19657.6. Samples: 51483408. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:28:20,466][1157520] Avg episode reward: [(0, '53.605')] [2024-09-30 01:28:20,548][1157819] Updated weights for policy 0, policy_version 60058 (0.0006) [2024-09-30 01:28:21,054][1157819] Updated weights for policy 0, policy_version 60068 (0.0006) [2024-09-30 01:28:21,513][1157819] Updated weights for policy 0, policy_version 60078 (0.0006) [2024-09-30 01:28:22,004][1157819] Updated weights for policy 0, policy_version 60088 (0.0006) [2024-09-30 01:28:22,500][1157819] Updated weights for policy 0, policy_version 60098 (0.0006) [2024-09-30 01:28:22,982][1157819] Updated weights for policy 0, policy_version 60108 (0.0006) [2024-09-30 01:28:23,479][1157819] Updated weights for policy 0, policy_version 60118 (0.0006) [2024-09-30 01:28:23,964][1157819] Updated weights for policy 0, policy_version 60128 (0.0006) [2024-09-30 01:28:24,464][1157819] Updated weights for policy 0, policy_version 60138 (0.0006) [2024-09-30 01:28:24,951][1157819] Updated weights for policy 0, policy_version 60148 (0.0006) [2024-09-30 01:28:25,438][1157819] Updated weights for policy 0, policy_version 60158 (0.0006) [2024-09-30 01:28:25,466][1157520] Fps is (10 sec: 83148.4, 60 sec: 78711.4, 300 sec: 77754.6). Total num frames: 246407168. Throughput: 0: 19837.8. Samples: 51546460. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 01:28:25,466][1157520] Avg episode reward: [(0, '50.657')] [2024-09-30 01:28:25,933][1157819] Updated weights for policy 0, policy_version 60168 (0.0006) [2024-09-30 01:28:26,423][1157819] Updated weights for policy 0, policy_version 60178 (0.0006) [2024-09-30 01:28:26,916][1157819] Updated weights for policy 0, policy_version 60188 (0.0006) [2024-09-30 01:28:27,398][1157819] Updated weights for policy 0, policy_version 60198 (0.0006) [2024-09-30 01:28:27,889][1157819] Updated weights for policy 0, policy_version 60208 (0.0006) [2024-09-30 01:28:28,386][1157819] Updated weights for policy 0, policy_version 60218 (0.0006) [2024-09-30 01:28:28,864][1157819] Updated weights for policy 0, policy_version 60228 (0.0006) [2024-09-30 01:28:29,354][1157819] Updated weights for policy 0, policy_version 60238 (0.0006) [2024-09-30 01:28:29,851][1157819] Updated weights for policy 0, policy_version 60248 (0.0006) [2024-09-30 01:28:30,328][1157819] Updated weights for policy 0, policy_version 60258 (0.0006) [2024-09-30 01:28:30,466][1157520] Fps is (10 sec: 83967.9, 60 sec: 79189.3, 300 sec: 77976.8). Total num frames: 246829056. Throughput: 0: 20074.3. Samples: 51671856. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 01:28:30,466][1157520] Avg episode reward: [(0, '48.628')] [2024-09-30 01:28:30,799][1157819] Updated weights for policy 0, policy_version 60268 (0.0006) [2024-09-30 01:28:31,284][1157819] Updated weights for policy 0, policy_version 60278 (0.0006) [2024-09-30 01:28:31,781][1157819] Updated weights for policy 0, policy_version 60288 (0.0006) [2024-09-30 01:28:32,264][1157819] Updated weights for policy 0, policy_version 60298 (0.0006) [2024-09-30 01:28:32,750][1157819] Updated weights for policy 0, policy_version 60308 (0.0006) [2024-09-30 01:28:33,242][1157819] Updated weights for policy 0, policy_version 60318 (0.0006) [2024-09-30 01:28:33,735][1157819] Updated weights for policy 0, policy_version 60328 (0.0006) [2024-09-30 01:28:33,897][1157736] Signal inference workers to stop experience collection... (3800 times) [2024-09-30 01:28:33,897][1157736] Signal inference workers to resume experience collection... (3800 times) [2024-09-30 01:28:33,903][1157819] InferenceWorker_p0-w0: stopping experience collection (3800 times) [2024-09-30 01:28:33,903][1157819] InferenceWorker_p0-w0: resuming experience collection (3800 times) [2024-09-30 01:28:34,226][1157819] Updated weights for policy 0, policy_version 60338 (0.0006) [2024-09-30 01:28:34,720][1157819] Updated weights for policy 0, policy_version 60348 (0.0006) [2024-09-30 01:28:35,214][1157819] Updated weights for policy 0, policy_version 60358 (0.0006) [2024-09-30 01:28:35,466][1157520] Fps is (10 sec: 84378.1, 60 sec: 80145.1, 300 sec: 78143.4). Total num frames: 247250944. Throughput: 0: 20186.8. Samples: 51798224. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 01:28:35,466][1157520] Avg episode reward: [(0, '52.170')] [2024-09-30 01:28:35,698][1157819] Updated weights for policy 0, policy_version 60368 (0.0006) [2024-09-30 01:28:36,195][1157819] Updated weights for policy 0, policy_version 60378 (0.0006) [2024-09-30 01:28:36,646][1157819] Updated weights for policy 0, policy_version 60388 (0.0006) [2024-09-30 01:28:37,114][1157819] Updated weights for policy 0, policy_version 60398 (0.0006) [2024-09-30 01:28:37,609][1157819] Updated weights for policy 0, policy_version 60408 (0.0006) [2024-09-30 01:28:38,102][1157819] Updated weights for policy 0, policy_version 60418 (0.0006) [2024-09-30 01:28:38,601][1157819] Updated weights for policy 0, policy_version 60428 (0.0006) [2024-09-30 01:28:39,100][1157819] Updated weights for policy 0, policy_version 60438 (0.0006) [2024-09-30 01:28:39,534][1157819] Updated weights for policy 0, policy_version 60448 (0.0006) [2024-09-30 01:28:40,051][1157819] Updated weights for policy 0, policy_version 60458 (0.0006) [2024-09-30 01:28:40,466][1157520] Fps is (10 sec: 83558.5, 60 sec: 80964.3, 300 sec: 78254.5). Total num frames: 247664640. Throughput: 0: 20223.8. Samples: 51861292. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 01:28:40,466][1157520] Avg episode reward: [(0, '51.455')] [2024-09-30 01:28:40,546][1157819] Updated weights for policy 0, policy_version 60468 (0.0006) [2024-09-30 01:28:41,019][1157819] Updated weights for policy 0, policy_version 60478 (0.0006) [2024-09-30 01:28:41,519][1157819] Updated weights for policy 0, policy_version 60488 (0.0006) [2024-09-30 01:28:42,000][1157819] Updated weights for policy 0, policy_version 60498 (0.0006) [2024-09-30 01:28:42,468][1157819] Updated weights for policy 0, policy_version 60508 (0.0006) [2024-09-30 01:28:42,958][1157819] Updated weights for policy 0, policy_version 60518 (0.0006) [2024-09-30 01:28:43,423][1157819] Updated weights for policy 0, policy_version 60528 (0.0006) [2024-09-30 01:28:43,907][1157819] Updated weights for policy 0, policy_version 60538 (0.0006) [2024-09-30 01:28:44,336][1157819] Updated weights for policy 0, policy_version 60548 (0.0006) [2024-09-30 01:28:44,820][1157819] Updated weights for policy 0, policy_version 60558 (0.0006) [2024-09-30 01:28:45,288][1157819] Updated weights for policy 0, policy_version 60568 (0.0006) [2024-09-30 01:28:45,466][1157520] Fps is (10 sec: 85196.7, 60 sec: 81578.7, 300 sec: 78462.7). Total num frames: 248102912. Throughput: 0: 20305.9. Samples: 51988644. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 01:28:45,466][1157520] Avg episode reward: [(0, '51.837')] [2024-09-30 01:28:45,770][1157819] Updated weights for policy 0, policy_version 60578 (0.0006) [2024-09-30 01:28:46,247][1157819] Updated weights for policy 0, policy_version 60588 (0.0006) [2024-09-30 01:28:46,719][1157819] Updated weights for policy 0, policy_version 60598 (0.0006) [2024-09-30 01:28:47,160][1157819] Updated weights for policy 0, policy_version 60608 (0.0006) [2024-09-30 01:28:47,625][1157819] Updated weights for policy 0, policy_version 60618 (0.0006) [2024-09-30 01:28:48,117][1157819] Updated weights for policy 0, policy_version 60628 (0.0006) [2024-09-30 01:28:48,580][1157819] Updated weights for policy 0, policy_version 60638 (0.0006) [2024-09-30 01:28:49,068][1157819] Updated weights for policy 0, policy_version 60648 (0.0006) [2024-09-30 01:28:49,562][1157819] Updated weights for policy 0, policy_version 60658 (0.0006) [2024-09-30 01:28:50,012][1157819] Updated weights for policy 0, policy_version 60668 (0.0006) [2024-09-30 01:28:50,466][1157520] Fps is (10 sec: 86425.6, 60 sec: 81920.0, 300 sec: 78657.2). Total num frames: 248528896. Throughput: 0: 20698.2. Samples: 52118992. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 01:28:50,466][1157520] Avg episode reward: [(0, '52.271')] [2024-09-30 01:28:50,538][1157819] Updated weights for policy 0, policy_version 60678 (0.0006) [2024-09-30 01:28:51,010][1157819] Updated weights for policy 0, policy_version 60688 (0.0006) [2024-09-30 01:28:51,494][1157819] Updated weights for policy 0, policy_version 60698 (0.0006) [2024-09-30 01:28:51,996][1157819] Updated weights for policy 0, policy_version 60708 (0.0006) [2024-09-30 01:28:52,460][1157819] Updated weights for policy 0, policy_version 60718 (0.0006) [2024-09-30 01:28:52,955][1157819] Updated weights for policy 0, policy_version 60728 (0.0006) [2024-09-30 01:28:53,464][1157819] Updated weights for policy 0, policy_version 60738 (0.0006) [2024-09-30 01:28:53,951][1157819] Updated weights for policy 0, policy_version 60748 (0.0006) [2024-09-30 01:28:54,432][1157819] Updated weights for policy 0, policy_version 60758 (0.0006) [2024-09-30 01:28:54,925][1157819] Updated weights for policy 0, policy_version 60768 (0.0006) [2024-09-30 01:28:55,422][1157819] Updated weights for policy 0, policy_version 60778 (0.0006) [2024-09-30 01:28:55,466][1157520] Fps is (10 sec: 84377.5, 60 sec: 82056.6, 300 sec: 78823.7). Total num frames: 248946688. Throughput: 0: 20860.0. Samples: 52181752. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 01:28:55,466][1157520] Avg episode reward: [(0, '53.347')] [2024-09-30 01:28:55,933][1157819] Updated weights for policy 0, policy_version 60788 (0.0006) [2024-09-30 01:28:56,375][1157819] Updated weights for policy 0, policy_version 60798 (0.0006) [2024-09-30 01:28:56,866][1157819] Updated weights for policy 0, policy_version 60808 (0.0006) [2024-09-30 01:28:57,432][1157819] Updated weights for policy 0, policy_version 60818 (0.0006) [2024-09-30 01:28:57,928][1157819] Updated weights for policy 0, policy_version 60828 (0.0006) [2024-09-30 01:28:58,383][1157819] Updated weights for policy 0, policy_version 60838 (0.0006) [2024-09-30 01:28:58,931][1157819] Updated weights for policy 0, policy_version 60848 (0.0006) [2024-09-30 01:28:59,420][1157819] Updated weights for policy 0, policy_version 60858 (0.0006) [2024-09-30 01:28:59,929][1157819] Updated weights for policy 0, policy_version 60868 (0.0006) [2024-09-30 01:29:00,455][1157819] Updated weights for policy 0, policy_version 60878 (0.0006) [2024-09-30 01:29:00,466][1157520] Fps is (10 sec: 82739.0, 60 sec: 82261.4, 300 sec: 78948.7). Total num frames: 249356288. Throughput: 0: 21023.0. Samples: 52306228. Policy #0 lag: (min: 1.0, avg: 2.3, max: 7.0) [2024-09-30 01:29:00,466][1157520] Avg episode reward: [(0, '51.274')] [2024-09-30 01:29:00,958][1157819] Updated weights for policy 0, policy_version 60888 (0.0006) [2024-09-30 01:29:01,516][1157819] Updated weights for policy 0, policy_version 60898 (0.0006) [2024-09-30 01:29:02,059][1157819] Updated weights for policy 0, policy_version 60908 (0.0006) [2024-09-30 01:29:02,590][1157819] Updated weights for policy 0, policy_version 60918 (0.0006) [2024-09-30 01:29:03,124][1157819] Updated weights for policy 0, policy_version 60928 (0.0006) [2024-09-30 01:29:03,678][1157819] Updated weights for policy 0, policy_version 60938 (0.0006) [2024-09-30 01:29:04,228][1157819] Updated weights for policy 0, policy_version 60948 (0.0006) [2024-09-30 01:29:04,814][1157819] Updated weights for policy 0, policy_version 60958 (0.0006) [2024-09-30 01:29:05,363][1157819] Updated weights for policy 0, policy_version 60968 (0.0006) [2024-09-30 01:29:05,466][1157520] Fps is (10 sec: 78643.4, 60 sec: 82193.2, 300 sec: 78920.9). Total num frames: 249733120. Throughput: 0: 20843.5. Samples: 52421364. Policy #0 lag: (min: 1.0, avg: 2.3, max: 7.0) [2024-09-30 01:29:05,466][1157520] Avg episode reward: [(0, '51.349')] [2024-09-30 01:29:05,975][1157736] Signal inference workers to stop experience collection... (3850 times) [2024-09-30 01:29:05,977][1157736] Signal inference workers to resume experience collection... (3850 times) [2024-09-30 01:29:05,977][1157819] InferenceWorker_p0-w0: stopping experience collection (3850 times) [2024-09-30 01:29:05,979][1157819] Updated weights for policy 0, policy_version 60978 (0.0007) [2024-09-30 01:29:05,981][1157819] InferenceWorker_p0-w0: resuming experience collection (3850 times) [2024-09-30 01:29:06,546][1157819] Updated weights for policy 0, policy_version 60988 (0.0006) [2024-09-30 01:29:07,130][1157819] Updated weights for policy 0, policy_version 60998 (0.0007) [2024-09-30 01:29:07,751][1157819] Updated weights for policy 0, policy_version 61008 (0.0006) [2024-09-30 01:29:08,370][1157819] Updated weights for policy 0, policy_version 61018 (0.0006) [2024-09-30 01:29:08,959][1157819] Updated weights for policy 0, policy_version 61028 (0.0006) [2024-09-30 01:29:09,604][1157819] Updated weights for policy 0, policy_version 61038 (0.0006) [2024-09-30 01:29:10,200][1157819] Updated weights for policy 0, policy_version 61048 (0.0006) [2024-09-30 01:29:10,466][1157520] Fps is (10 sec: 71270.0, 60 sec: 81646.8, 300 sec: 78726.5). Total num frames: 250068992. Throughput: 0: 20589.4. Samples: 52472984. Policy #0 lag: (min: 1.0, avg: 2.3, max: 7.0) [2024-09-30 01:29:10,466][1157520] Avg episode reward: [(0, '52.632')] [2024-09-30 01:29:10,835][1157819] Updated weights for policy 0, policy_version 61058 (0.0006) [2024-09-30 01:29:11,395][1157819] Updated weights for policy 0, policy_version 61068 (0.0006) [2024-09-30 01:29:12,042][1157819] Updated weights for policy 0, policy_version 61078 (0.0006) [2024-09-30 01:29:12,645][1157819] Updated weights for policy 0, policy_version 61088 (0.0006) [2024-09-30 01:29:13,287][1157819] Updated weights for policy 0, policy_version 61098 (0.0006) [2024-09-30 01:29:13,840][1157819] Updated weights for policy 0, policy_version 61108 (0.0006) [2024-09-30 01:29:14,494][1157819] Updated weights for policy 0, policy_version 61118 (0.0006) [2024-09-30 01:29:15,032][1157819] Updated weights for policy 0, policy_version 61128 (0.0006) [2024-09-30 01:29:15,466][1157520] Fps is (10 sec: 67583.3, 60 sec: 80554.5, 300 sec: 78518.2). Total num frames: 250408960. Throughput: 0: 20027.2. Samples: 52573080. Policy #0 lag: (min: 1.0, avg: 2.3, max: 7.0) [2024-09-30 01:29:15,466][1157520] Avg episode reward: [(0, '52.571')] [2024-09-30 01:29:15,609][1157819] Updated weights for policy 0, policy_version 61138 (0.0006) [2024-09-30 01:29:16,200][1157819] Updated weights for policy 0, policy_version 61148 (0.0006) [2024-09-30 01:29:16,717][1157819] Updated weights for policy 0, policy_version 61158 (0.0006) [2024-09-30 01:29:17,239][1157819] Updated weights for policy 0, policy_version 61168 (0.0006) [2024-09-30 01:29:17,826][1157819] Updated weights for policy 0, policy_version 61178 (0.0006) [2024-09-30 01:29:18,358][1157819] Updated weights for policy 0, policy_version 61188 (0.0006) [2024-09-30 01:29:18,918][1157819] Updated weights for policy 0, policy_version 61198 (0.0006) [2024-09-30 01:29:19,443][1157819] Updated weights for policy 0, policy_version 61208 (0.0006) [2024-09-30 01:29:19,963][1157819] Updated weights for policy 0, policy_version 61218 (0.0006) [2024-09-30 01:29:20,466][1157520] Fps is (10 sec: 71680.4, 60 sec: 79940.2, 300 sec: 78448.8). Total num frames: 250785792. Throughput: 0: 19676.9. Samples: 52683684. Policy #0 lag: (min: 1.0, avg: 2.3, max: 7.0) [2024-09-30 01:29:20,466][1157520] Avg episode reward: [(0, '51.944')] [2024-09-30 01:29:20,523][1157819] Updated weights for policy 0, policy_version 61228 (0.0006) [2024-09-30 01:29:21,053][1157819] Updated weights for policy 0, policy_version 61238 (0.0006) [2024-09-30 01:29:21,574][1157819] Updated weights for policy 0, policy_version 61248 (0.0006) [2024-09-30 01:29:22,139][1157819] Updated weights for policy 0, policy_version 61258 (0.0006) [2024-09-30 01:29:22,665][1157819] Updated weights for policy 0, policy_version 61268 (0.0006) [2024-09-30 01:29:23,183][1157819] Updated weights for policy 0, policy_version 61278 (0.0006) [2024-09-30 01:29:23,723][1157819] Updated weights for policy 0, policy_version 61288 (0.0006) [2024-09-30 01:29:24,275][1157819] Updated weights for policy 0, policy_version 61298 (0.0006) [2024-09-30 01:29:24,792][1157819] Updated weights for policy 0, policy_version 61308 (0.0006) [2024-09-30 01:29:25,348][1157819] Updated weights for policy 0, policy_version 61318 (0.0006) [2024-09-30 01:29:25,466][1157520] Fps is (10 sec: 75776.2, 60 sec: 79325.8, 300 sec: 78407.2). Total num frames: 251166720. Throughput: 0: 19547.6. Samples: 52740936. Policy #0 lag: (min: 1.0, avg: 2.3, max: 7.0) [2024-09-30 01:29:25,466][1157520] Avg episode reward: [(0, '51.596')] [2024-09-30 01:29:25,902][1157819] Updated weights for policy 0, policy_version 61328 (0.0006) [2024-09-30 01:29:26,432][1157819] Updated weights for policy 0, policy_version 61338 (0.0006) [2024-09-30 01:29:27,033][1157819] Updated weights for policy 0, policy_version 61348 (0.0006) [2024-09-30 01:29:27,599][1157819] Updated weights for policy 0, policy_version 61358 (0.0006) [2024-09-30 01:29:28,176][1157819] Updated weights for policy 0, policy_version 61368 (0.0006) [2024-09-30 01:29:28,784][1157819] Updated weights for policy 0, policy_version 61378 (0.0006) [2024-09-30 01:29:29,385][1157819] Updated weights for policy 0, policy_version 61388 (0.0006) [2024-09-30 01:29:29,924][1157819] Updated weights for policy 0, policy_version 61398 (0.0006) [2024-09-30 01:29:30,455][1157819] Updated weights for policy 0, policy_version 61408 (0.0006) [2024-09-30 01:29:30,466][1157520] Fps is (10 sec: 74137.4, 60 sec: 78301.8, 300 sec: 78282.2). Total num frames: 251527168. Throughput: 0: 19170.2. Samples: 52851304. Policy #0 lag: (min: 1.0, avg: 2.3, max: 7.0) [2024-09-30 01:29:30,466][1157520] Avg episode reward: [(0, '52.432')] [2024-09-30 01:29:31,011][1157819] Updated weights for policy 0, policy_version 61418 (0.0006) [2024-09-30 01:29:31,537][1157819] Updated weights for policy 0, policy_version 61428 (0.0006) [2024-09-30 01:29:32,076][1157819] Updated weights for policy 0, policy_version 61438 (0.0006) [2024-09-30 01:29:32,590][1157819] Updated weights for policy 0, policy_version 61448 (0.0006) [2024-09-30 01:29:33,085][1157819] Updated weights for policy 0, policy_version 61458 (0.0006) [2024-09-30 01:29:33,620][1157819] Updated weights for policy 0, policy_version 61468 (0.0006) [2024-09-30 01:29:34,130][1157819] Updated weights for policy 0, policy_version 61478 (0.0006) [2024-09-30 01:29:34,633][1157819] Updated weights for policy 0, policy_version 61488 (0.0006) [2024-09-30 01:29:35,175][1157819] Updated weights for policy 0, policy_version 61498 (0.0006) [2024-09-30 01:29:35,466][1157520] Fps is (10 sec: 74957.2, 60 sec: 77755.7, 300 sec: 78310.0). Total num frames: 251916288. Throughput: 0: 18831.5. Samples: 52966412. Policy #0 lag: (min: 1.0, avg: 2.3, max: 7.0) [2024-09-30 01:29:35,466][1157520] Avg episode reward: [(0, '53.302')] [2024-09-30 01:29:35,678][1157819] Updated weights for policy 0, policy_version 61508 (0.0006) [2024-09-30 01:29:36,060][1157736] Signal inference workers to stop experience collection... (3900 times) [2024-09-30 01:29:36,061][1157736] Signal inference workers to resume experience collection... (3900 times) [2024-09-30 01:29:36,065][1157819] InferenceWorker_p0-w0: stopping experience collection (3900 times) [2024-09-30 01:29:36,067][1157819] InferenceWorker_p0-w0: resuming experience collection (3900 times) [2024-09-30 01:29:36,200][1157819] Updated weights for policy 0, policy_version 61518 (0.0006) [2024-09-30 01:29:36,725][1157819] Updated weights for policy 0, policy_version 61528 (0.0006) [2024-09-30 01:29:37,231][1157819] Updated weights for policy 0, policy_version 61538 (0.0006) [2024-09-30 01:29:37,739][1157819] Updated weights for policy 0, policy_version 61548 (0.0006) [2024-09-30 01:29:38,252][1157819] Updated weights for policy 0, policy_version 61558 (0.0006) [2024-09-30 01:29:38,786][1157819] Updated weights for policy 0, policy_version 61568 (0.0006) [2024-09-30 01:29:39,295][1157819] Updated weights for policy 0, policy_version 61578 (0.0006) [2024-09-30 01:29:39,819][1157819] Updated weights for policy 0, policy_version 61588 (0.0006) [2024-09-30 01:29:40,331][1157819] Updated weights for policy 0, policy_version 61598 (0.0006) [2024-09-30 01:29:40,466][1157520] Fps is (10 sec: 79052.9, 60 sec: 77550.9, 300 sec: 78393.3). Total num frames: 252317696. Throughput: 0: 18761.2. Samples: 53026008. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 01:29:40,466][1157520] Avg episode reward: [(0, '54.277')] [2024-09-30 01:29:40,830][1157819] Updated weights for policy 0, policy_version 61608 (0.0006) [2024-09-30 01:29:41,317][1157819] Updated weights for policy 0, policy_version 61618 (0.0006) [2024-09-30 01:29:41,860][1157819] Updated weights for policy 0, policy_version 61628 (0.0006) [2024-09-30 01:29:42,398][1157819] Updated weights for policy 0, policy_version 61638 (0.0006) [2024-09-30 01:29:42,902][1157819] Updated weights for policy 0, policy_version 61648 (0.0006) [2024-09-30 01:29:43,392][1157819] Updated weights for policy 0, policy_version 61658 (0.0006) [2024-09-30 01:29:43,902][1157819] Updated weights for policy 0, policy_version 61668 (0.0006) [2024-09-30 01:29:44,449][1157819] Updated weights for policy 0, policy_version 61678 (0.0006) [2024-09-30 01:29:44,961][1157819] Updated weights for policy 0, policy_version 61688 (0.0006) [2024-09-30 01:29:45,466][1157520] Fps is (10 sec: 79461.5, 60 sec: 76799.8, 300 sec: 78365.5). Total num frames: 252710912. Throughput: 0: 18645.3. Samples: 53145268. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 01:29:45,466][1157520] Avg episode reward: [(0, '53.564')] [2024-09-30 01:29:45,519][1157819] Updated weights for policy 0, policy_version 61698 (0.0006) [2024-09-30 01:29:46,020][1157819] Updated weights for policy 0, policy_version 61708 (0.0006) [2024-09-30 01:29:46,568][1157819] Updated weights for policy 0, policy_version 61718 (0.0006) [2024-09-30 01:29:47,142][1157819] Updated weights for policy 0, policy_version 61728 (0.0006) [2024-09-30 01:29:47,683][1157819] Updated weights for policy 0, policy_version 61738 (0.0006) [2024-09-30 01:29:48,210][1157819] Updated weights for policy 0, policy_version 61748 (0.0006) [2024-09-30 01:29:48,758][1157819] Updated weights for policy 0, policy_version 61758 (0.0006) [2024-09-30 01:29:49,313][1157819] Updated weights for policy 0, policy_version 61768 (0.0006) [2024-09-30 01:29:49,888][1157819] Updated weights for policy 0, policy_version 61778 (0.0006) [2024-09-30 01:29:50,419][1157819] Updated weights for policy 0, policy_version 61788 (0.0006) [2024-09-30 01:29:50,466][1157520] Fps is (10 sec: 76594.2, 60 sec: 75912.3, 300 sec: 78351.7). Total num frames: 253083648. Throughput: 0: 18616.5. Samples: 53259108. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 01:29:50,466][1157520] Avg episode reward: [(0, '54.476')] [2024-09-30 01:29:51,008][1157819] Updated weights for policy 0, policy_version 61798 (0.0006) [2024-09-30 01:29:51,547][1157819] Updated weights for policy 0, policy_version 61808 (0.0006) [2024-09-30 01:29:52,068][1157819] Updated weights for policy 0, policy_version 61818 (0.0006) [2024-09-30 01:29:52,651][1157819] Updated weights for policy 0, policy_version 61828 (0.0006) [2024-09-30 01:29:53,175][1157819] Updated weights for policy 0, policy_version 61838 (0.0006) [2024-09-30 01:29:53,725][1157819] Updated weights for policy 0, policy_version 61848 (0.0006) [2024-09-30 01:29:54,269][1157819] Updated weights for policy 0, policy_version 61858 (0.0006) [2024-09-30 01:29:54,821][1157819] Updated weights for policy 0, policy_version 61868 (0.0006) [2024-09-30 01:29:55,393][1157819] Updated weights for policy 0, policy_version 61878 (0.0006) [2024-09-30 01:29:55,466][1157520] Fps is (10 sec: 74547.2, 60 sec: 75161.5, 300 sec: 78254.4). Total num frames: 253456384. Throughput: 0: 18703.5. Samples: 53314644. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 01:29:55,466][1157520] Avg episode reward: [(0, '53.405')] [2024-09-30 01:29:55,906][1157819] Updated weights for policy 0, policy_version 61888 (0.0006) [2024-09-30 01:29:56,444][1157819] Updated weights for policy 0, policy_version 61898 (0.0006) [2024-09-30 01:29:56,958][1157819] Updated weights for policy 0, policy_version 61908 (0.0006) [2024-09-30 01:29:57,478][1157819] Updated weights for policy 0, policy_version 61918 (0.0006) [2024-09-30 01:29:57,983][1157819] Updated weights for policy 0, policy_version 61928 (0.0006) [2024-09-30 01:29:58,523][1157819] Updated weights for policy 0, policy_version 61938 (0.0006) [2024-09-30 01:29:59,031][1157819] Updated weights for policy 0, policy_version 61948 (0.0006) [2024-09-30 01:29:59,516][1157819] Updated weights for policy 0, policy_version 61958 (0.0006) [2024-09-30 01:30:00,035][1157819] Updated weights for policy 0, policy_version 61968 (0.0006) [2024-09-30 01:30:00,466][1157520] Fps is (10 sec: 76596.2, 60 sec: 74888.5, 300 sec: 78018.4). Total num frames: 253849600. Throughput: 0: 19047.9. Samples: 53430236. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 01:30:00,466][1157520] Avg episode reward: [(0, '54.460')] [2024-09-30 01:30:00,554][1157819] Updated weights for policy 0, policy_version 61978 (0.0006) [2024-09-30 01:30:01,056][1157819] Updated weights for policy 0, policy_version 61988 (0.0006) [2024-09-30 01:30:01,607][1157819] Updated weights for policy 0, policy_version 61998 (0.0006) [2024-09-30 01:30:02,104][1157819] Updated weights for policy 0, policy_version 62008 (0.0005) [2024-09-30 01:30:02,580][1157819] Updated weights for policy 0, policy_version 62018 (0.0006) [2024-09-30 01:30:03,022][1157819] Updated weights for policy 0, policy_version 62028 (0.0006) [2024-09-30 01:30:03,505][1157819] Updated weights for policy 0, policy_version 62038 (0.0006) [2024-09-30 01:30:03,963][1157819] Updated weights for policy 0, policy_version 62048 (0.0006) [2024-09-30 01:30:04,417][1157819] Updated weights for policy 0, policy_version 62058 (0.0006) [2024-09-30 01:30:04,862][1157819] Updated weights for policy 0, policy_version 62068 (0.0006) [2024-09-30 01:30:05,319][1157819] Updated weights for policy 0, policy_version 62078 (0.0006) [2024-09-30 01:30:05,466][1157520] Fps is (10 sec: 82329.4, 60 sec: 75775.8, 300 sec: 78087.8). Total num frames: 254279680. Throughput: 0: 19383.7. Samples: 53555952. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 01:30:05,466][1157520] Avg episode reward: [(0, '52.532')] [2024-09-30 01:30:05,471][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000062081_254283776.pth... [2024-09-30 01:30:05,514][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000057517_235589632.pth [2024-09-30 01:30:05,786][1157819] Updated weights for policy 0, policy_version 62088 (0.0006) [2024-09-30 01:30:06,230][1157819] Updated weights for policy 0, policy_version 62098 (0.0006) [2024-09-30 01:30:06,677][1157819] Updated weights for policy 0, policy_version 62108 (0.0006) [2024-09-30 01:30:07,133][1157819] Updated weights for policy 0, policy_version 62118 (0.0006) [2024-09-30 01:30:07,585][1157819] Updated weights for policy 0, policy_version 62128 (0.0005) [2024-09-30 01:30:08,032][1157819] Updated weights for policy 0, policy_version 62138 (0.0006) [2024-09-30 01:30:08,485][1157819] Updated weights for policy 0, policy_version 62148 (0.0006) [2024-09-30 01:30:08,933][1157819] Updated weights for policy 0, policy_version 62158 (0.0006) [2024-09-30 01:30:09,391][1157819] Updated weights for policy 0, policy_version 62168 (0.0006) [2024-09-30 01:30:09,841][1157819] Updated weights for policy 0, policy_version 62178 (0.0006) [2024-09-30 01:30:10,306][1157819] Updated weights for policy 0, policy_version 62188 (0.0006) [2024-09-30 01:30:10,466][1157520] Fps is (10 sec: 88473.9, 60 sec: 77755.8, 300 sec: 78226.7). Total num frames: 254734336. Throughput: 0: 19616.1. Samples: 53623660. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 01:30:10,466][1157520] Avg episode reward: [(0, '52.195')] [2024-09-30 01:30:10,716][1157819] Updated weights for policy 0, policy_version 62198 (0.0005) [2024-09-30 01:30:11,206][1157819] Updated weights for policy 0, policy_version 62208 (0.0006) [2024-09-30 01:30:11,665][1157819] Updated weights for policy 0, policy_version 62218 (0.0006) [2024-09-30 01:30:12,091][1157819] Updated weights for policy 0, policy_version 62228 (0.0006) [2024-09-30 01:30:12,556][1157819] Updated weights for policy 0, policy_version 62238 (0.0006) [2024-09-30 01:30:13,035][1157819] Updated weights for policy 0, policy_version 62248 (0.0006) [2024-09-30 01:30:13,480][1157819] Updated weights for policy 0, policy_version 62258 (0.0006) [2024-09-30 01:30:13,932][1157819] Updated weights for policy 0, policy_version 62268 (0.0006) [2024-09-30 01:30:14,425][1157819] Updated weights for policy 0, policy_version 62278 (0.0006) [2024-09-30 01:30:14,926][1157819] Updated weights for policy 0, policy_version 62288 (0.0006) [2024-09-30 01:30:15,373][1157819] Updated weights for policy 0, policy_version 62298 (0.0006) [2024-09-30 01:30:15,466][1157520] Fps is (10 sec: 89702.6, 60 sec: 79462.4, 300 sec: 78323.8). Total num frames: 255176704. Throughput: 0: 20168.5. Samples: 53758888. Policy #0 lag: (min: 0.0, avg: 2.1, max: 7.0) [2024-09-30 01:30:15,466][1157520] Avg episode reward: [(0, '53.747')] [2024-09-30 01:30:15,832][1157819] Updated weights for policy 0, policy_version 62308 (0.0006) [2024-09-30 01:30:16,301][1157819] Updated weights for policy 0, policy_version 62318 (0.0006) [2024-09-30 01:30:16,795][1157819] Updated weights for policy 0, policy_version 62328 (0.0006) [2024-09-30 01:30:17,265][1157819] Updated weights for policy 0, policy_version 62338 (0.0006) [2024-09-30 01:30:17,753][1157819] Updated weights for policy 0, policy_version 62348 (0.0006) [2024-09-30 01:30:18,221][1157819] Updated weights for policy 0, policy_version 62358 (0.0006) [2024-09-30 01:30:18,653][1157819] Updated weights for policy 0, policy_version 62368 (0.0006) [2024-09-30 01:30:19,144][1157819] Updated weights for policy 0, policy_version 62378 (0.0006) [2024-09-30 01:30:19,645][1157819] Updated weights for policy 0, policy_version 62388 (0.0006) [2024-09-30 01:30:20,116][1157819] Updated weights for policy 0, policy_version 62398 (0.0006) [2024-09-30 01:30:20,466][1157520] Fps is (10 sec: 87653.5, 60 sec: 80418.0, 300 sec: 78393.3). Total num frames: 255610880. Throughput: 0: 20496.8. Samples: 53888768. Policy #0 lag: (min: 0.0, avg: 2.1, max: 7.0) [2024-09-30 01:30:20,466][1157520] Avg episode reward: [(0, '51.226')] [2024-09-30 01:30:20,599][1157819] Updated weights for policy 0, policy_version 62408 (0.0006) [2024-09-30 01:30:21,097][1157819] Updated weights for policy 0, policy_version 62418 (0.0006) [2024-09-30 01:30:21,603][1157819] Updated weights for policy 0, policy_version 62428 (0.0006) [2024-09-30 01:30:22,094][1157819] Updated weights for policy 0, policy_version 62438 (0.0006) [2024-09-30 01:30:22,574][1157819] Updated weights for policy 0, policy_version 62448 (0.0006) [2024-09-30 01:30:23,063][1157819] Updated weights for policy 0, policy_version 62458 (0.0006) [2024-09-30 01:30:23,564][1157819] Updated weights for policy 0, policy_version 62468 (0.0006) [2024-09-30 01:30:24,051][1157819] Updated weights for policy 0, policy_version 62478 (0.0005) [2024-09-30 01:30:24,545][1157819] Updated weights for policy 0, policy_version 62488 (0.0006) [2024-09-30 01:30:25,035][1157819] Updated weights for policy 0, policy_version 62498 (0.0005) [2024-09-30 01:30:25,466][1157520] Fps is (10 sec: 84787.2, 60 sec: 80964.2, 300 sec: 78393.2). Total num frames: 256024576. Throughput: 0: 20564.2. Samples: 53951400. Policy #0 lag: (min: 0.0, avg: 2.1, max: 7.0) [2024-09-30 01:30:25,466][1157520] Avg episode reward: [(0, '53.381')] [2024-09-30 01:30:25,521][1157819] Updated weights for policy 0, policy_version 62508 (0.0006) [2024-09-30 01:30:26,005][1157819] Updated weights for policy 0, policy_version 62518 (0.0005) [2024-09-30 01:30:26,488][1157819] Updated weights for policy 0, policy_version 62528 (0.0006) [2024-09-30 01:30:26,910][1157819] Updated weights for policy 0, policy_version 62538 (0.0005) [2024-09-30 01:30:27,385][1157819] Updated weights for policy 0, policy_version 62548 (0.0006) [2024-09-30 01:30:27,851][1157819] Updated weights for policy 0, policy_version 62558 (0.0006) [2024-09-30 01:30:28,346][1157819] Updated weights for policy 0, policy_version 62568 (0.0006) [2024-09-30 01:30:28,350][1157736] Signal inference workers to stop experience collection... (3950 times) [2024-09-30 01:30:28,350][1157736] Signal inference workers to resume experience collection... (3950 times) [2024-09-30 01:30:28,356][1157819] InferenceWorker_p0-w0: stopping experience collection (3950 times) [2024-09-30 01:30:28,356][1157819] InferenceWorker_p0-w0: resuming experience collection (3950 times) [2024-09-30 01:30:28,776][1157819] Updated weights for policy 0, policy_version 62578 (0.0006) [2024-09-30 01:30:29,247][1157819] Updated weights for policy 0, policy_version 62588 (0.0006) [2024-09-30 01:30:29,726][1157819] Updated weights for policy 0, policy_version 62598 (0.0006) [2024-09-30 01:30:30,216][1157819] Updated weights for policy 0, policy_version 62608 (0.0006) [2024-09-30 01:30:30,466][1157520] Fps is (10 sec: 85196.9, 60 sec: 82261.3, 300 sec: 78490.4). Total num frames: 256462848. Throughput: 0: 20764.0. Samples: 54079648. Policy #0 lag: (min: 0.0, avg: 2.1, max: 7.0) [2024-09-30 01:30:30,466][1157520] Avg episode reward: [(0, '52.340')] [2024-09-30 01:30:30,670][1157819] Updated weights for policy 0, policy_version 62618 (0.0006) [2024-09-30 01:30:31,131][1157819] Updated weights for policy 0, policy_version 62628 (0.0006) [2024-09-30 01:30:31,656][1157819] Updated weights for policy 0, policy_version 62638 (0.0006) [2024-09-30 01:30:32,202][1157819] Updated weights for policy 0, policy_version 62648 (0.0006) [2024-09-30 01:30:32,698][1157819] Updated weights for policy 0, policy_version 62658 (0.0006) [2024-09-30 01:30:33,223][1157819] Updated weights for policy 0, policy_version 62668 (0.0006) [2024-09-30 01:30:33,696][1157819] Updated weights for policy 0, policy_version 62678 (0.0006) [2024-09-30 01:30:34,187][1157819] Updated weights for policy 0, policy_version 62688 (0.0006) [2024-09-30 01:30:34,681][1157819] Updated weights for policy 0, policy_version 62698 (0.0006) [2024-09-30 01:30:35,134][1157819] Updated weights for policy 0, policy_version 62708 (0.0006) [2024-09-30 01:30:35,466][1157520] Fps is (10 sec: 85197.8, 60 sec: 82670.9, 300 sec: 78601.5). Total num frames: 256876544. Throughput: 0: 21030.3. Samples: 54205468. Policy #0 lag: (min: 0.0, avg: 2.1, max: 7.0) [2024-09-30 01:30:35,466][1157520] Avg episode reward: [(0, '51.552')] [2024-09-30 01:30:35,662][1157819] Updated weights for policy 0, policy_version 62718 (0.0006) [2024-09-30 01:30:36,121][1157819] Updated weights for policy 0, policy_version 62728 (0.0006) [2024-09-30 01:30:36,610][1157819] Updated weights for policy 0, policy_version 62738 (0.0006) [2024-09-30 01:30:37,064][1157819] Updated weights for policy 0, policy_version 62748 (0.0006) [2024-09-30 01:30:37,560][1157819] Updated weights for policy 0, policy_version 62758 (0.0006) [2024-09-30 01:30:38,057][1157819] Updated weights for policy 0, policy_version 62768 (0.0006) [2024-09-30 01:30:38,549][1157819] Updated weights for policy 0, policy_version 62778 (0.0006) [2024-09-30 01:30:39,045][1157819] Updated weights for policy 0, policy_version 62788 (0.0006) [2024-09-30 01:30:39,535][1157819] Updated weights for policy 0, policy_version 62798 (0.0006) [2024-09-30 01:30:39,746][1157736] Signal inference workers to stop experience collection... (4000 times) [2024-09-30 01:30:39,749][1157819] InferenceWorker_p0-w0: stopping experience collection (4000 times) [2024-09-30 01:30:39,752][1157736] Signal inference workers to resume experience collection... (4000 times) [2024-09-30 01:30:39,752][1157819] InferenceWorker_p0-w0: resuming experience collection (4000 times) [2024-09-30 01:30:40,033][1157819] Updated weights for policy 0, policy_version 62808 (0.0006) [2024-09-30 01:30:40,466][1157520] Fps is (10 sec: 83147.7, 60 sec: 82943.7, 300 sec: 78657.1). Total num frames: 257294336. Throughput: 0: 21194.4. Samples: 54268396. Policy #0 lag: (min: 0.0, avg: 2.1, max: 7.0) [2024-09-30 01:30:40,466][1157520] Avg episode reward: [(0, '50.345')] [2024-09-30 01:30:40,554][1157819] Updated weights for policy 0, policy_version 62818 (0.0006) [2024-09-30 01:30:41,045][1157819] Updated weights for policy 0, policy_version 62828 (0.0006) [2024-09-30 01:30:41,537][1157819] Updated weights for policy 0, policy_version 62838 (0.0006) [2024-09-30 01:30:42,041][1157819] Updated weights for policy 0, policy_version 62848 (0.0006) [2024-09-30 01:30:42,529][1157819] Updated weights for policy 0, policy_version 62858 (0.0006) [2024-09-30 01:30:43,079][1157819] Updated weights for policy 0, policy_version 62868 (0.0006) [2024-09-30 01:30:43,576][1157819] Updated weights for policy 0, policy_version 62878 (0.0006) [2024-09-30 01:30:44,076][1157819] Updated weights for policy 0, policy_version 62888 (0.0006) [2024-09-30 01:30:44,615][1157819] Updated weights for policy 0, policy_version 62898 (0.0006) [2024-09-30 01:30:45,128][1157819] Updated weights for policy 0, policy_version 62908 (0.0006) [2024-09-30 01:30:45,466][1157520] Fps is (10 sec: 82328.9, 60 sec: 83148.8, 300 sec: 78712.6). Total num frames: 257699840. Throughput: 0: 21356.9. Samples: 54391296. Policy #0 lag: (min: 0.0, avg: 2.1, max: 7.0) [2024-09-30 01:30:45,466][1157520] Avg episode reward: [(0, '50.825')] [2024-09-30 01:30:45,616][1157819] Updated weights for policy 0, policy_version 62918 (0.0006) [2024-09-30 01:30:46,127][1157819] Updated weights for policy 0, policy_version 62928 (0.0006) [2024-09-30 01:30:46,655][1157819] Updated weights for policy 0, policy_version 62938 (0.0006) [2024-09-30 01:30:47,186][1157819] Updated weights for policy 0, policy_version 62948 (0.0006) [2024-09-30 01:30:47,685][1157819] Updated weights for policy 0, policy_version 62958 (0.0006) [2024-09-30 01:30:48,230][1157819] Updated weights for policy 0, policy_version 62968 (0.0006) [2024-09-30 01:30:48,728][1157819] Updated weights for policy 0, policy_version 62978 (0.0006) [2024-09-30 01:30:49,247][1157819] Updated weights for policy 0, policy_version 62988 (0.0006) [2024-09-30 01:30:49,734][1157819] Updated weights for policy 0, policy_version 62998 (0.0006) [2024-09-30 01:30:50,239][1157819] Updated weights for policy 0, policy_version 63008 (0.0006) [2024-09-30 01:30:50,466][1157520] Fps is (10 sec: 80692.2, 60 sec: 83626.7, 300 sec: 78795.9). Total num frames: 258101248. Throughput: 0: 21220.8. Samples: 54510888. Policy #0 lag: (min: 0.0, avg: 2.1, max: 7.0) [2024-09-30 01:30:50,466][1157520] Avg episode reward: [(0, '51.583')] [2024-09-30 01:30:50,711][1157819] Updated weights for policy 0, policy_version 63018 (0.0006) [2024-09-30 01:30:51,191][1157819] Updated weights for policy 0, policy_version 63028 (0.0006) [2024-09-30 01:30:51,627][1157819] Updated weights for policy 0, policy_version 63038 (0.0006) [2024-09-30 01:30:52,091][1157819] Updated weights for policy 0, policy_version 63048 (0.0006) [2024-09-30 01:30:52,562][1157819] Updated weights for policy 0, policy_version 63058 (0.0006) [2024-09-30 01:30:53,059][1157819] Updated weights for policy 0, policy_version 63068 (0.0006) [2024-09-30 01:30:53,579][1157819] Updated weights for policy 0, policy_version 63078 (0.0006) [2024-09-30 01:30:54,172][1157819] Updated weights for policy 0, policy_version 63088 (0.0006) [2024-09-30 01:30:54,784][1157819] Updated weights for policy 0, policy_version 63098 (0.0006) [2024-09-30 01:30:55,387][1157819] Updated weights for policy 0, policy_version 63108 (0.0006) [2024-09-30 01:30:55,466][1157520] Fps is (10 sec: 79461.3, 60 sec: 83967.9, 300 sec: 78865.3). Total num frames: 258494464. Throughput: 0: 21166.5. Samples: 54576156. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 01:30:55,466][1157520] Avg episode reward: [(0, '50.966')] [2024-09-30 01:30:55,978][1157819] Updated weights for policy 0, policy_version 63118 (0.0006) [2024-09-30 01:30:56,643][1157819] Updated weights for policy 0, policy_version 63128 (0.0006) [2024-09-30 01:30:57,287][1157819] Updated weights for policy 0, policy_version 63138 (0.0006) [2024-09-30 01:30:57,889][1157819] Updated weights for policy 0, policy_version 63148 (0.0006) [2024-09-30 01:30:58,493][1157819] Updated weights for policy 0, policy_version 63158 (0.0006) [2024-09-30 01:30:59,097][1157819] Updated weights for policy 0, policy_version 63168 (0.0006) [2024-09-30 01:30:59,692][1157819] Updated weights for policy 0, policy_version 63178 (0.0006) [2024-09-30 01:31:00,243][1157819] Updated weights for policy 0, policy_version 63188 (0.0006) [2024-09-30 01:31:00,466][1157520] Fps is (10 sec: 73319.1, 60 sec: 83080.6, 300 sec: 78795.9). Total num frames: 258834432. Throughput: 0: 20435.2. Samples: 54678468. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 01:31:00,466][1157520] Avg episode reward: [(0, '52.075')] [2024-09-30 01:31:00,760][1157819] Updated weights for policy 0, policy_version 63198 (0.0006) [2024-09-30 01:31:01,263][1157819] Updated weights for policy 0, policy_version 63208 (0.0006) [2024-09-30 01:31:01,808][1157819] Updated weights for policy 0, policy_version 63218 (0.0006) [2024-09-30 01:31:02,299][1157819] Updated weights for policy 0, policy_version 63228 (0.0006) [2024-09-30 01:31:02,796][1157819] Updated weights for policy 0, policy_version 63238 (0.0006) [2024-09-30 01:31:03,305][1157819] Updated weights for policy 0, policy_version 63248 (0.0006) [2024-09-30 01:31:03,832][1157819] Updated weights for policy 0, policy_version 63258 (0.0006) [2024-09-30 01:31:04,336][1157819] Updated weights for policy 0, policy_version 63268 (0.0006) [2024-09-30 01:31:04,839][1157819] Updated weights for policy 0, policy_version 63278 (0.0006) [2024-09-30 01:31:05,336][1157819] Updated weights for policy 0, policy_version 63288 (0.0006) [2024-09-30 01:31:05,466][1157520] Fps is (10 sec: 74139.3, 60 sec: 82602.9, 300 sec: 78962.6). Total num frames: 259235840. Throughput: 0: 20155.0. Samples: 54795740. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 01:31:05,466][1157520] Avg episode reward: [(0, '54.243')] [2024-09-30 01:31:05,861][1157819] Updated weights for policy 0, policy_version 63298 (0.0006) [2024-09-30 01:31:06,419][1157819] Updated weights for policy 0, policy_version 63308 (0.0006) [2024-09-30 01:31:06,968][1157819] Updated weights for policy 0, policy_version 63318 (0.0006) [2024-09-30 01:31:07,132][1157736] Signal inference workers to stop experience collection... (4050 times) [2024-09-30 01:31:07,133][1157736] Signal inference workers to resume experience collection... (4050 times) [2024-09-30 01:31:07,136][1157819] InferenceWorker_p0-w0: stopping experience collection (4050 times) [2024-09-30 01:31:07,138][1157819] InferenceWorker_p0-w0: resuming experience collection (4050 times) [2024-09-30 01:31:07,488][1157819] Updated weights for policy 0, policy_version 63328 (0.0006) [2024-09-30 01:31:08,024][1157819] Updated weights for policy 0, policy_version 63338 (0.0006) [2024-09-30 01:31:08,575][1157819] Updated weights for policy 0, policy_version 63348 (0.0006) [2024-09-30 01:31:09,144][1157819] Updated weights for policy 0, policy_version 63358 (0.0006) [2024-09-30 01:31:09,708][1157819] Updated weights for policy 0, policy_version 63368 (0.0006) [2024-09-30 01:31:10,272][1157819] Updated weights for policy 0, policy_version 63378 (0.0006) [2024-09-30 01:31:10,466][1157520] Fps is (10 sec: 77412.7, 60 sec: 81237.0, 300 sec: 79004.2). Total num frames: 259608576. Throughput: 0: 20051.7. Samples: 54853728. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 01:31:10,466][1157520] Avg episode reward: [(0, '52.579')] [2024-09-30 01:31:10,833][1157819] Updated weights for policy 0, policy_version 63388 (0.0006) [2024-09-30 01:31:11,404][1157819] Updated weights for policy 0, policy_version 63398 (0.0006) [2024-09-30 01:31:11,907][1157819] Updated weights for policy 0, policy_version 63408 (0.0006) [2024-09-30 01:31:12,436][1157819] Updated weights for policy 0, policy_version 63418 (0.0006) [2024-09-30 01:31:12,924][1157819] Updated weights for policy 0, policy_version 63428 (0.0006) [2024-09-30 01:31:13,456][1157819] Updated weights for policy 0, policy_version 63438 (0.0006) [2024-09-30 01:31:13,967][1157819] Updated weights for policy 0, policy_version 63448 (0.0006) [2024-09-30 01:31:14,467][1157819] Updated weights for policy 0, policy_version 63458 (0.0006) [2024-09-30 01:31:14,961][1157819] Updated weights for policy 0, policy_version 63468 (0.0006) [2024-09-30 01:31:15,457][1157819] Updated weights for policy 0, policy_version 63478 (0.0006) [2024-09-30 01:31:15,466][1157520] Fps is (10 sec: 77004.8, 60 sec: 80486.6, 300 sec: 79184.7). Total num frames: 260005888. Throughput: 0: 19732.7. Samples: 54967616. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 01:31:15,466][1157520] Avg episode reward: [(0, '55.168')] [2024-09-30 01:31:15,469][1157736] Saving new best policy, reward=55.168! [2024-09-30 01:31:15,981][1157819] Updated weights for policy 0, policy_version 63488 (0.0006) [2024-09-30 01:31:16,509][1157819] Updated weights for policy 0, policy_version 63498 (0.0006) [2024-09-30 01:31:17,029][1157819] Updated weights for policy 0, policy_version 63508 (0.0006) [2024-09-30 01:31:17,525][1157819] Updated weights for policy 0, policy_version 63518 (0.0006) [2024-09-30 01:31:18,026][1157819] Updated weights for policy 0, policy_version 63528 (0.0006) [2024-09-30 01:31:18,533][1157819] Updated weights for policy 0, policy_version 63538 (0.0006) [2024-09-30 01:31:19,021][1157819] Updated weights for policy 0, policy_version 63548 (0.0006) [2024-09-30 01:31:19,503][1157819] Updated weights for policy 0, policy_version 63558 (0.0006) [2024-09-30 01:31:20,010][1157819] Updated weights for policy 0, policy_version 63568 (0.0006) [2024-09-30 01:31:20,466][1157520] Fps is (10 sec: 80283.1, 60 sec: 80008.6, 300 sec: 79351.3). Total num frames: 260411392. Throughput: 0: 19638.6. Samples: 55089204. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 01:31:20,466][1157520] Avg episode reward: [(0, '54.890')] [2024-09-30 01:31:20,494][1157819] Updated weights for policy 0, policy_version 63578 (0.0006) [2024-09-30 01:31:20,990][1157819] Updated weights for policy 0, policy_version 63588 (0.0006) [2024-09-30 01:31:21,483][1157819] Updated weights for policy 0, policy_version 63598 (0.0006) [2024-09-30 01:31:21,968][1157819] Updated weights for policy 0, policy_version 63608 (0.0006) [2024-09-30 01:31:22,479][1157819] Updated weights for policy 0, policy_version 63618 (0.0006) [2024-09-30 01:31:23,017][1157819] Updated weights for policy 0, policy_version 63628 (0.0006) [2024-09-30 01:31:23,606][1157819] Updated weights for policy 0, policy_version 63638 (0.0006) [2024-09-30 01:31:24,161][1157819] Updated weights for policy 0, policy_version 63648 (0.0006) [2024-09-30 01:31:24,751][1157819] Updated weights for policy 0, policy_version 63658 (0.0006) [2024-09-30 01:31:25,285][1157819] Updated weights for policy 0, policy_version 63668 (0.0006) [2024-09-30 01:31:25,466][1157520] Fps is (10 sec: 79051.5, 60 sec: 79530.6, 300 sec: 79365.2). Total num frames: 260796416. Throughput: 0: 19603.0. Samples: 55150528. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 01:31:25,466][1157520] Avg episode reward: [(0, '56.165')] [2024-09-30 01:31:25,470][1157736] Saving new best policy, reward=56.165! [2024-09-30 01:31:25,874][1157819] Updated weights for policy 0, policy_version 63678 (0.0006) [2024-09-30 01:31:26,388][1157819] Updated weights for policy 0, policy_version 63688 (0.0006) [2024-09-30 01:31:26,901][1157819] Updated weights for policy 0, policy_version 63698 (0.0006) [2024-09-30 01:31:27,498][1157819] Updated weights for policy 0, policy_version 63708 (0.0006) [2024-09-30 01:31:28,035][1157819] Updated weights for policy 0, policy_version 63718 (0.0006) [2024-09-30 01:31:28,616][1157819] Updated weights for policy 0, policy_version 63728 (0.0006) [2024-09-30 01:31:29,128][1157819] Updated weights for policy 0, policy_version 63738 (0.0006) [2024-09-30 01:31:29,658][1157819] Updated weights for policy 0, policy_version 63748 (0.0006) [2024-09-30 01:31:30,196][1157819] Updated weights for policy 0, policy_version 63758 (0.0006) [2024-09-30 01:31:30,466][1157520] Fps is (10 sec: 76186.0, 60 sec: 78506.8, 300 sec: 79281.9). Total num frames: 261173248. Throughput: 0: 19336.1. Samples: 55261420. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 01:31:30,466][1157520] Avg episode reward: [(0, '53.937')] [2024-09-30 01:31:30,702][1157819] Updated weights for policy 0, policy_version 63768 (0.0006) [2024-09-30 01:31:31,192][1157819] Updated weights for policy 0, policy_version 63778 (0.0006) [2024-09-30 01:31:31,705][1157819] Updated weights for policy 0, policy_version 63788 (0.0006) [2024-09-30 01:31:32,189][1157819] Updated weights for policy 0, policy_version 63798 (0.0006) [2024-09-30 01:31:32,686][1157819] Updated weights for policy 0, policy_version 63808 (0.0006) [2024-09-30 01:31:33,184][1157819] Updated weights for policy 0, policy_version 63818 (0.0006) [2024-09-30 01:31:33,665][1157819] Updated weights for policy 0, policy_version 63828 (0.0006) [2024-09-30 01:31:34,132][1157819] Updated weights for policy 0, policy_version 63838 (0.0006) [2024-09-30 01:31:34,645][1157819] Updated weights for policy 0, policy_version 63848 (0.0006) [2024-09-30 01:31:35,118][1157736] Signal inference workers to stop experience collection... (4100 times) [2024-09-30 01:31:35,119][1157736] Signal inference workers to resume experience collection... (4100 times) [2024-09-30 01:31:35,122][1157819] InferenceWorker_p0-w0: stopping experience collection (4100 times) [2024-09-30 01:31:35,124][1157819] InferenceWorker_p0-w0: resuming experience collection (4100 times) [2024-09-30 01:31:35,175][1157819] Updated weights for policy 0, policy_version 63858 (0.0006) [2024-09-30 01:31:35,466][1157520] Fps is (10 sec: 78644.2, 60 sec: 78438.3, 300 sec: 79254.2). Total num frames: 261582848. Throughput: 0: 19374.4. Samples: 55382736. Policy #0 lag: (min: 0.0, avg: 2.1, max: 7.0) [2024-09-30 01:31:35,466][1157520] Avg episode reward: [(0, '53.802')] [2024-09-30 01:31:35,682][1157819] Updated weights for policy 0, policy_version 63868 (0.0006) [2024-09-30 01:31:36,170][1157819] Updated weights for policy 0, policy_version 63878 (0.0006) [2024-09-30 01:31:36,661][1157819] Updated weights for policy 0, policy_version 63888 (0.0006) [2024-09-30 01:31:37,146][1157819] Updated weights for policy 0, policy_version 63898 (0.0006) [2024-09-30 01:31:37,626][1157819] Updated weights for policy 0, policy_version 63908 (0.0006) [2024-09-30 01:31:38,108][1157819] Updated weights for policy 0, policy_version 63918 (0.0006) [2024-09-30 01:31:38,577][1157819] Updated weights for policy 0, policy_version 63928 (0.0006) [2024-09-30 01:31:39,072][1157819] Updated weights for policy 0, policy_version 63938 (0.0006) [2024-09-30 01:31:39,556][1157819] Updated weights for policy 0, policy_version 63948 (0.0006) [2024-09-30 01:31:40,008][1157819] Updated weights for policy 0, policy_version 63958 (0.0006) [2024-09-30 01:31:40,466][1157520] Fps is (10 sec: 83558.5, 60 sec: 78575.3, 300 sec: 79309.7). Total num frames: 262008832. Throughput: 0: 19307.8. Samples: 55445004. Policy #0 lag: (min: 0.0, avg: 2.1, max: 7.0) [2024-09-30 01:31:40,466][1157520] Avg episode reward: [(0, '54.367')] [2024-09-30 01:31:40,475][1157819] Updated weights for policy 0, policy_version 63968 (0.0006) [2024-09-30 01:31:40,941][1157819] Updated weights for policy 0, policy_version 63978 (0.0006) [2024-09-30 01:31:41,420][1157819] Updated weights for policy 0, policy_version 63988 (0.0006) [2024-09-30 01:31:41,885][1157819] Updated weights for policy 0, policy_version 63998 (0.0006) [2024-09-30 01:31:42,339][1157819] Updated weights for policy 0, policy_version 64008 (0.0006) [2024-09-30 01:31:42,841][1157819] Updated weights for policy 0, policy_version 64018 (0.0006) [2024-09-30 01:31:43,381][1157819] Updated weights for policy 0, policy_version 64028 (0.0006) [2024-09-30 01:31:43,908][1157819] Updated weights for policy 0, policy_version 64038 (0.0006) [2024-09-30 01:31:44,438][1157819] Updated weights for policy 0, policy_version 64048 (0.0006) [2024-09-30 01:31:44,969][1157819] Updated weights for policy 0, policy_version 64058 (0.0006) [2024-09-30 01:31:45,466][1157520] Fps is (10 sec: 83558.6, 60 sec: 78643.3, 300 sec: 79351.3). Total num frames: 262418432. Throughput: 0: 19878.9. Samples: 55573020. Policy #0 lag: (min: 0.0, avg: 2.1, max: 7.0) [2024-09-30 01:31:45,466][1157520] Avg episode reward: [(0, '53.291')] [2024-09-30 01:31:45,503][1157819] Updated weights for policy 0, policy_version 64068 (0.0006) [2024-09-30 01:31:46,065][1157819] Updated weights for policy 0, policy_version 64078 (0.0006) [2024-09-30 01:31:46,593][1157819] Updated weights for policy 0, policy_version 64088 (0.0006) [2024-09-30 01:31:47,110][1157819] Updated weights for policy 0, policy_version 64098 (0.0006) [2024-09-30 01:31:47,649][1157819] Updated weights for policy 0, policy_version 64108 (0.0006) [2024-09-30 01:31:48,201][1157819] Updated weights for policy 0, policy_version 64118 (0.0006) [2024-09-30 01:31:48,733][1157819] Updated weights for policy 0, policy_version 64128 (0.0006) [2024-09-30 01:31:49,246][1157819] Updated weights for policy 0, policy_version 64138 (0.0006) [2024-09-30 01:31:49,741][1157819] Updated weights for policy 0, policy_version 64148 (0.0006) [2024-09-30 01:31:50,243][1157819] Updated weights for policy 0, policy_version 64158 (0.0006) [2024-09-30 01:31:50,466][1157520] Fps is (10 sec: 79871.5, 60 sec: 78438.5, 300 sec: 79379.1). Total num frames: 262807552. Throughput: 0: 19842.2. Samples: 55688640. Policy #0 lag: (min: 0.0, avg: 2.1, max: 7.0) [2024-09-30 01:31:50,466][1157520] Avg episode reward: [(0, '49.933')] [2024-09-30 01:31:50,758][1157819] Updated weights for policy 0, policy_version 64168 (0.0006) [2024-09-30 01:31:51,277][1157819] Updated weights for policy 0, policy_version 64178 (0.0006) [2024-09-30 01:31:51,803][1157819] Updated weights for policy 0, policy_version 64188 (0.0006) [2024-09-30 01:31:52,328][1157819] Updated weights for policy 0, policy_version 64198 (0.0006) [2024-09-30 01:31:52,840][1157819] Updated weights for policy 0, policy_version 64208 (0.0006) [2024-09-30 01:31:53,341][1157819] Updated weights for policy 0, policy_version 64218 (0.0006) [2024-09-30 01:31:53,887][1157819] Updated weights for policy 0, policy_version 64228 (0.0006) [2024-09-30 01:31:54,413][1157819] Updated weights for policy 0, policy_version 64238 (0.0006) [2024-09-30 01:31:54,934][1157819] Updated weights for policy 0, policy_version 64248 (0.0006) [2024-09-30 01:31:55,448][1157819] Updated weights for policy 0, policy_version 64258 (0.0006) [2024-09-30 01:31:55,466][1157520] Fps is (10 sec: 78233.7, 60 sec: 78438.7, 300 sec: 79476.3). Total num frames: 263200768. Throughput: 0: 19870.0. Samples: 55747876. Policy #0 lag: (min: 0.0, avg: 2.1, max: 7.0) [2024-09-30 01:31:55,466][1157520] Avg episode reward: [(0, '51.313')] [2024-09-30 01:31:55,965][1157819] Updated weights for policy 0, policy_version 64268 (0.0006) [2024-09-30 01:31:56,487][1157819] Updated weights for policy 0, policy_version 64278 (0.0006) [2024-09-30 01:31:57,011][1157819] Updated weights for policy 0, policy_version 64288 (0.0006) [2024-09-30 01:31:57,545][1157819] Updated weights for policy 0, policy_version 64298 (0.0006) [2024-09-30 01:31:58,060][1157819] Updated weights for policy 0, policy_version 64308 (0.0006) [2024-09-30 01:31:58,555][1157819] Updated weights for policy 0, policy_version 64318 (0.0006) [2024-09-30 01:31:59,041][1157819] Updated weights for policy 0, policy_version 64328 (0.0006) [2024-09-30 01:31:59,566][1157819] Updated weights for policy 0, policy_version 64338 (0.0006) [2024-09-30 01:32:00,084][1157819] Updated weights for policy 0, policy_version 64348 (0.0006) [2024-09-30 01:32:00,466][1157520] Fps is (10 sec: 79462.5, 60 sec: 79462.4, 300 sec: 79698.5). Total num frames: 263602176. Throughput: 0: 19972.3. Samples: 55866368. Policy #0 lag: (min: 0.0, avg: 2.1, max: 7.0) [2024-09-30 01:32:00,466][1157520] Avg episode reward: [(0, '50.468')] [2024-09-30 01:32:00,574][1157819] Updated weights for policy 0, policy_version 64358 (0.0006) [2024-09-30 01:32:01,112][1157819] Updated weights for policy 0, policy_version 64368 (0.0006) [2024-09-30 01:32:01,607][1157819] Updated weights for policy 0, policy_version 64378 (0.0006) [2024-09-30 01:32:02,108][1157819] Updated weights for policy 0, policy_version 64388 (0.0006) [2024-09-30 01:32:02,603][1157819] Updated weights for policy 0, policy_version 64398 (0.0006) [2024-09-30 01:32:03,131][1157819] Updated weights for policy 0, policy_version 64408 (0.0006) [2024-09-30 01:32:03,648][1157819] Updated weights for policy 0, policy_version 64418 (0.0006) [2024-09-30 01:32:04,183][1157819] Updated weights for policy 0, policy_version 64428 (0.0006) [2024-09-30 01:32:04,698][1157819] Updated weights for policy 0, policy_version 64438 (0.0006) [2024-09-30 01:32:05,197][1157819] Updated weights for policy 0, policy_version 64448 (0.0006) [2024-09-30 01:32:05,466][1157520] Fps is (10 sec: 79871.9, 60 sec: 79394.1, 300 sec: 79740.1). Total num frames: 263999488. Throughput: 0: 19939.1. Samples: 55986464. Policy #0 lag: (min: 0.0, avg: 2.1, max: 7.0) [2024-09-30 01:32:05,466][1157520] Avg episode reward: [(0, '50.632')] [2024-09-30 01:32:05,470][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000064453_263999488.pth... [2024-09-30 01:32:05,517][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000059766_244801536.pth [2024-09-30 01:32:05,714][1157819] Updated weights for policy 0, policy_version 64458 (0.0006) [2024-09-30 01:32:06,275][1157819] Updated weights for policy 0, policy_version 64468 (0.0006) [2024-09-30 01:32:06,790][1157819] Updated weights for policy 0, policy_version 64478 (0.0006) [2024-09-30 01:32:07,344][1157819] Updated weights for policy 0, policy_version 64488 (0.0006) [2024-09-30 01:32:07,882][1157819] Updated weights for policy 0, policy_version 64498 (0.0006) [2024-09-30 01:32:08,424][1157819] Updated weights for policy 0, policy_version 64508 (0.0006) [2024-09-30 01:32:09,000][1157819] Updated weights for policy 0, policy_version 64518 (0.0006) [2024-09-30 01:32:09,505][1157819] Updated weights for policy 0, policy_version 64528 (0.0006) [2024-09-30 01:32:10,053][1157819] Updated weights for policy 0, policy_version 64538 (0.0006) [2024-09-30 01:32:10,466][1157520] Fps is (10 sec: 77414.4, 60 sec: 79462.7, 300 sec: 79629.0). Total num frames: 264376320. Throughput: 0: 19856.4. Samples: 56044064. Policy #0 lag: (min: 0.0, avg: 2.1, max: 7.0) [2024-09-30 01:32:10,466][1157520] Avg episode reward: [(0, '52.914')] [2024-09-30 01:32:10,604][1157819] Updated weights for policy 0, policy_version 64548 (0.0006) [2024-09-30 01:32:11,147][1157819] Updated weights for policy 0, policy_version 64558 (0.0006) [2024-09-30 01:32:11,712][1157819] Updated weights for policy 0, policy_version 64568 (0.0006) [2024-09-30 01:32:12,220][1157819] Updated weights for policy 0, policy_version 64578 (0.0006) [2024-09-30 01:32:12,767][1157819] Updated weights for policy 0, policy_version 64588 (0.0006) [2024-09-30 01:32:13,277][1157819] Updated weights for policy 0, policy_version 64598 (0.0006) [2024-09-30 01:32:13,797][1157819] Updated weights for policy 0, policy_version 64608 (0.0006) [2024-09-30 01:32:14,369][1157819] Updated weights for policy 0, policy_version 64618 (0.0006) [2024-09-30 01:32:14,912][1157819] Updated weights for policy 0, policy_version 64628 (0.0006) [2024-09-30 01:32:15,456][1157819] Updated weights for policy 0, policy_version 64638 (0.0006) [2024-09-30 01:32:15,466][1157520] Fps is (10 sec: 75775.6, 60 sec: 79189.2, 300 sec: 79559.6). Total num frames: 264757248. Throughput: 0: 19916.5. Samples: 56157664. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:32:15,466][1157520] Avg episode reward: [(0, '50.162')] [2024-09-30 01:32:16,004][1157819] Updated weights for policy 0, policy_version 64648 (0.0006) [2024-09-30 01:32:16,496][1157819] Updated weights for policy 0, policy_version 64658 (0.0006) [2024-09-30 01:32:17,009][1157819] Updated weights for policy 0, policy_version 64668 (0.0006) [2024-09-30 01:32:17,510][1157819] Updated weights for policy 0, policy_version 64678 (0.0006) [2024-09-30 01:32:18,033][1157819] Updated weights for policy 0, policy_version 64688 (0.0006) [2024-09-30 01:32:18,528][1157819] Updated weights for policy 0, policy_version 64698 (0.0006) [2024-09-30 01:32:19,077][1157736] Signal inference workers to stop experience collection... (4150 times) [2024-09-30 01:32:19,077][1157736] Signal inference workers to resume experience collection... (4150 times) [2024-09-30 01:32:19,077][1157819] Updated weights for policy 0, policy_version 64708 (0.0006) [2024-09-30 01:32:19,083][1157819] InferenceWorker_p0-w0: stopping experience collection (4150 times) [2024-09-30 01:32:19,083][1157819] InferenceWorker_p0-w0: resuming experience collection (4150 times) [2024-09-30 01:32:19,565][1157819] Updated weights for policy 0, policy_version 64718 (0.0006) [2024-09-30 01:32:20,086][1157819] Updated weights for policy 0, policy_version 64728 (0.0006) [2024-09-30 01:32:20,466][1157520] Fps is (10 sec: 77823.9, 60 sec: 79052.8, 300 sec: 79559.6). Total num frames: 265154560. Throughput: 0: 19840.4. Samples: 56275552. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:32:20,466][1157520] Avg episode reward: [(0, '53.055')] [2024-09-30 01:32:20,590][1157819] Updated weights for policy 0, policy_version 64738 (0.0006) [2024-09-30 01:32:21,097][1157819] Updated weights for policy 0, policy_version 64748 (0.0006) [2024-09-30 01:32:21,593][1157819] Updated weights for policy 0, policy_version 64758 (0.0006) [2024-09-30 01:32:22,098][1157819] Updated weights for policy 0, policy_version 64768 (0.0006) [2024-09-30 01:32:22,601][1157819] Updated weights for policy 0, policy_version 64778 (0.0006) [2024-09-30 01:32:23,082][1157819] Updated weights for policy 0, policy_version 64788 (0.0006) [2024-09-30 01:32:23,559][1157819] Updated weights for policy 0, policy_version 64798 (0.0006) [2024-09-30 01:32:24,054][1157819] Updated weights for policy 0, policy_version 64808 (0.0006) [2024-09-30 01:32:24,544][1157819] Updated weights for policy 0, policy_version 64818 (0.0006) [2024-09-30 01:32:25,039][1157819] Updated weights for policy 0, policy_version 64828 (0.0006) [2024-09-30 01:32:25,466][1157520] Fps is (10 sec: 81100.9, 60 sec: 79530.8, 300 sec: 79629.0). Total num frames: 265568256. Throughput: 0: 19814.2. Samples: 56336644. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:32:25,466][1157520] Avg episode reward: [(0, '53.591')] [2024-09-30 01:32:25,537][1157819] Updated weights for policy 0, policy_version 64838 (0.0006) [2024-09-30 01:32:26,026][1157819] Updated weights for policy 0, policy_version 64848 (0.0006) [2024-09-30 01:32:26,522][1157819] Updated weights for policy 0, policy_version 64858 (0.0006) [2024-09-30 01:32:27,018][1157819] Updated weights for policy 0, policy_version 64868 (0.0006) [2024-09-30 01:32:27,522][1157819] Updated weights for policy 0, policy_version 64878 (0.0006) [2024-09-30 01:32:27,999][1157819] Updated weights for policy 0, policy_version 64888 (0.0006) [2024-09-30 01:32:28,508][1157819] Updated weights for policy 0, policy_version 64898 (0.0006) [2024-09-30 01:32:29,000][1157819] Updated weights for policy 0, policy_version 64908 (0.0006) [2024-09-30 01:32:29,480][1157819] Updated weights for policy 0, policy_version 64918 (0.0006) [2024-09-30 01:32:29,956][1157819] Updated weights for policy 0, policy_version 64928 (0.0006) [2024-09-30 01:32:30,450][1157819] Updated weights for policy 0, policy_version 64938 (0.0006) [2024-09-30 01:32:30,466][1157520] Fps is (10 sec: 83149.2, 60 sec: 80213.3, 300 sec: 79809.5). Total num frames: 265986048. Throughput: 0: 19747.4. Samples: 56461652. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:32:30,466][1157520] Avg episode reward: [(0, '54.025')] [2024-09-30 01:32:30,936][1157819] Updated weights for policy 0, policy_version 64948 (0.0006) [2024-09-30 01:32:31,433][1157819] Updated weights for policy 0, policy_version 64958 (0.0006) [2024-09-30 01:32:31,882][1157819] Updated weights for policy 0, policy_version 64968 (0.0006) [2024-09-30 01:32:32,366][1157819] Updated weights for policy 0, policy_version 64978 (0.0006) [2024-09-30 01:32:32,842][1157819] Updated weights for policy 0, policy_version 64988 (0.0006) [2024-09-30 01:32:33,308][1157819] Updated weights for policy 0, policy_version 64998 (0.0006) [2024-09-30 01:32:33,760][1157819] Updated weights for policy 0, policy_version 65008 (0.0006) [2024-09-30 01:32:34,245][1157819] Updated weights for policy 0, policy_version 65018 (0.0006) [2024-09-30 01:32:34,724][1157819] Updated weights for policy 0, policy_version 65028 (0.0006) [2024-09-30 01:32:35,182][1157819] Updated weights for policy 0, policy_version 65038 (0.0006) [2024-09-30 01:32:35,466][1157520] Fps is (10 sec: 84787.5, 60 sec: 80554.7, 300 sec: 80031.7). Total num frames: 266416128. Throughput: 0: 20033.6. Samples: 56590152. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:32:35,466][1157520] Avg episode reward: [(0, '52.942')] [2024-09-30 01:32:35,684][1157819] Updated weights for policy 0, policy_version 65048 (0.0006) [2024-09-30 01:32:36,186][1157819] Updated weights for policy 0, policy_version 65058 (0.0006) [2024-09-30 01:32:36,641][1157819] Updated weights for policy 0, policy_version 65068 (0.0006) [2024-09-30 01:32:37,138][1157819] Updated weights for policy 0, policy_version 65078 (0.0006) [2024-09-30 01:32:37,639][1157819] Updated weights for policy 0, policy_version 65088 (0.0006) [2024-09-30 01:32:38,104][1157819] Updated weights for policy 0, policy_version 65098 (0.0006) [2024-09-30 01:32:38,600][1157819] Updated weights for policy 0, policy_version 65108 (0.0006) [2024-09-30 01:32:39,066][1157819] Updated weights for policy 0, policy_version 65118 (0.0006) [2024-09-30 01:32:39,538][1157819] Updated weights for policy 0, policy_version 65128 (0.0006) [2024-09-30 01:32:40,030][1157819] Updated weights for policy 0, policy_version 65138 (0.0006) [2024-09-30 01:32:40,466][1157520] Fps is (10 sec: 85196.5, 60 sec: 80486.3, 300 sec: 80101.1). Total num frames: 266838016. Throughput: 0: 20127.5. Samples: 56653612. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:32:40,466][1157520] Avg episode reward: [(0, '50.923')] [2024-09-30 01:32:40,519][1157819] Updated weights for policy 0, policy_version 65148 (0.0006) [2024-09-30 01:32:41,023][1157819] Updated weights for policy 0, policy_version 65158 (0.0006) [2024-09-30 01:32:41,549][1157819] Updated weights for policy 0, policy_version 65168 (0.0006) [2024-09-30 01:32:42,052][1157819] Updated weights for policy 0, policy_version 65178 (0.0006) [2024-09-30 01:32:42,548][1157819] Updated weights for policy 0, policy_version 65188 (0.0006) [2024-09-30 01:32:43,048][1157819] Updated weights for policy 0, policy_version 65198 (0.0006) [2024-09-30 01:32:43,569][1157819] Updated weights for policy 0, policy_version 65208 (0.0006) [2024-09-30 01:32:44,061][1157819] Updated weights for policy 0, policy_version 65218 (0.0006) [2024-09-30 01:32:44,564][1157819] Updated weights for policy 0, policy_version 65228 (0.0006) [2024-09-30 01:32:45,058][1157819] Updated weights for policy 0, policy_version 65238 (0.0006) [2024-09-30 01:32:45,466][1157520] Fps is (10 sec: 83149.0, 60 sec: 80486.4, 300 sec: 80115.0). Total num frames: 267247616. Throughput: 0: 20244.9. Samples: 56777388. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:32:45,466][1157520] Avg episode reward: [(0, '53.490')] [2024-09-30 01:32:45,540][1157819] Updated weights for policy 0, policy_version 65248 (0.0006) [2024-09-30 01:32:46,024][1157819] Updated weights for policy 0, policy_version 65258 (0.0006) [2024-09-30 01:32:46,510][1157819] Updated weights for policy 0, policy_version 65268 (0.0006) [2024-09-30 01:32:46,993][1157819] Updated weights for policy 0, policy_version 65278 (0.0006) [2024-09-30 01:32:47,488][1157819] Updated weights for policy 0, policy_version 65288 (0.0006) [2024-09-30 01:32:47,978][1157819] Updated weights for policy 0, policy_version 65298 (0.0006) [2024-09-30 01:32:48,474][1157819] Updated weights for policy 0, policy_version 65308 (0.0006) [2024-09-30 01:32:48,968][1157819] Updated weights for policy 0, policy_version 65318 (0.0006) [2024-09-30 01:32:49,465][1157819] Updated weights for policy 0, policy_version 65328 (0.0006) [2024-09-30 01:32:49,968][1157819] Updated weights for policy 0, policy_version 65338 (0.0006) [2024-09-30 01:32:50,466][1157520] Fps is (10 sec: 82329.7, 60 sec: 80896.0, 300 sec: 80128.9). Total num frames: 267661312. Throughput: 0: 20354.0. Samples: 56902392. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:32:50,466][1157520] Avg episode reward: [(0, '51.500')] [2024-09-30 01:32:50,474][1157819] Updated weights for policy 0, policy_version 65348 (0.0006) [2024-09-30 01:32:50,970][1157819] Updated weights for policy 0, policy_version 65358 (0.0006) [2024-09-30 01:32:51,483][1157819] Updated weights for policy 0, policy_version 65368 (0.0006) [2024-09-30 01:32:52,019][1157819] Updated weights for policy 0, policy_version 65378 (0.0006) [2024-09-30 01:32:52,544][1157819] Updated weights for policy 0, policy_version 65388 (0.0006) [2024-09-30 01:32:53,058][1157819] Updated weights for policy 0, policy_version 65398 (0.0006) [2024-09-30 01:32:53,581][1157819] Updated weights for policy 0, policy_version 65408 (0.0006) [2024-09-30 01:32:54,113][1157819] Updated weights for policy 0, policy_version 65418 (0.0006) [2024-09-30 01:32:54,654][1157819] Updated weights for policy 0, policy_version 65428 (0.0006) [2024-09-30 01:32:55,185][1157819] Updated weights for policy 0, policy_version 65438 (0.0006) [2024-09-30 01:32:55,466][1157520] Fps is (10 sec: 80690.9, 60 sec: 80896.0, 300 sec: 80115.0). Total num frames: 268054528. Throughput: 0: 20407.0. Samples: 56962380. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 01:32:55,466][1157520] Avg episode reward: [(0, '55.054')] [2024-09-30 01:32:55,731][1157819] Updated weights for policy 0, policy_version 65448 (0.0006) [2024-09-30 01:32:56,315][1157819] Updated weights for policy 0, policy_version 65458 (0.0006) [2024-09-30 01:32:56,816][1157819] Updated weights for policy 0, policy_version 65468 (0.0006) [2024-09-30 01:32:57,339][1157819] Updated weights for policy 0, policy_version 65478 (0.0006) [2024-09-30 01:32:57,926][1157819] Updated weights for policy 0, policy_version 65488 (0.0006) [2024-09-30 01:32:58,416][1157819] Updated weights for policy 0, policy_version 65498 (0.0006) [2024-09-30 01:32:58,934][1157819] Updated weights for policy 0, policy_version 65508 (0.0006) [2024-09-30 01:32:59,513][1157819] Updated weights for policy 0, policy_version 65518 (0.0006) [2024-09-30 01:33:00,035][1157819] Updated weights for policy 0, policy_version 65528 (0.0006) [2024-09-30 01:33:00,466][1157520] Fps is (10 sec: 77414.4, 60 sec: 80554.7, 300 sec: 80115.0). Total num frames: 268435456. Throughput: 0: 20431.2. Samples: 57077068. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 01:33:00,466][1157520] Avg episode reward: [(0, '55.222')] [2024-09-30 01:33:00,543][1157819] Updated weights for policy 0, policy_version 65538 (0.0006) [2024-09-30 01:33:01,071][1157819] Updated weights for policy 0, policy_version 65548 (0.0006) [2024-09-30 01:33:01,563][1157819] Updated weights for policy 0, policy_version 65558 (0.0006) [2024-09-30 01:33:02,061][1157819] Updated weights for policy 0, policy_version 65568 (0.0006) [2024-09-30 01:33:02,575][1157819] Updated weights for policy 0, policy_version 65578 (0.0006) [2024-09-30 01:33:03,110][1157819] Updated weights for policy 0, policy_version 65588 (0.0006) [2024-09-30 01:33:03,670][1157819] Updated weights for policy 0, policy_version 65598 (0.0006) [2024-09-30 01:33:04,234][1157819] Updated weights for policy 0, policy_version 65608 (0.0006) [2024-09-30 01:33:04,743][1157819] Updated weights for policy 0, policy_version 65618 (0.0006) [2024-09-30 01:33:05,296][1157819] Updated weights for policy 0, policy_version 65628 (0.0006) [2024-09-30 01:33:05,466][1157520] Fps is (10 sec: 77004.9, 60 sec: 80418.1, 300 sec: 80184.4). Total num frames: 268824576. Throughput: 0: 20409.3. Samples: 57193972. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 01:33:05,466][1157520] Avg episode reward: [(0, '52.203')] [2024-09-30 01:33:05,801][1157819] Updated weights for policy 0, policy_version 65638 (0.0006) [2024-09-30 01:33:06,358][1157819] Updated weights for policy 0, policy_version 65648 (0.0006) [2024-09-30 01:33:06,908][1157819] Updated weights for policy 0, policy_version 65658 (0.0006) [2024-09-30 01:33:07,441][1157819] Updated weights for policy 0, policy_version 65668 (0.0006) [2024-09-30 01:33:07,942][1157819] Updated weights for policy 0, policy_version 65678 (0.0006) [2024-09-30 01:33:08,489][1157819] Updated weights for policy 0, policy_version 65688 (0.0006) [2024-09-30 01:33:09,030][1157819] Updated weights for policy 0, policy_version 65698 (0.0006) [2024-09-30 01:33:09,541][1157819] Updated weights for policy 0, policy_version 65708 (0.0006) [2024-09-30 01:33:10,043][1157819] Updated weights for policy 0, policy_version 65718 (0.0006) [2024-09-30 01:33:10,466][1157520] Fps is (10 sec: 77823.4, 60 sec: 80622.9, 300 sec: 80128.8). Total num frames: 269213696. Throughput: 0: 20312.1. Samples: 57250688. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 01:33:10,466][1157520] Avg episode reward: [(0, '49.083')] [2024-09-30 01:33:10,533][1157819] Updated weights for policy 0, policy_version 65728 (0.0006) [2024-09-30 01:33:11,030][1157819] Updated weights for policy 0, policy_version 65738 (0.0006) [2024-09-30 01:33:11,552][1157819] Updated weights for policy 0, policy_version 65748 (0.0006) [2024-09-30 01:33:11,822][1157736] Signal inference workers to stop experience collection... (4200 times) [2024-09-30 01:33:11,823][1157736] Signal inference workers to resume experience collection... (4200 times) [2024-09-30 01:33:11,826][1157819] InferenceWorker_p0-w0: stopping experience collection (4200 times) [2024-09-30 01:33:11,826][1157819] InferenceWorker_p0-w0: resuming experience collection (4200 times) [2024-09-30 01:33:12,076][1157819] Updated weights for policy 0, policy_version 65758 (0.0006) [2024-09-30 01:33:12,557][1157819] Updated weights for policy 0, policy_version 65768 (0.0006) [2024-09-30 01:33:13,090][1157819] Updated weights for policy 0, policy_version 65778 (0.0006) [2024-09-30 01:33:13,582][1157819] Updated weights for policy 0, policy_version 65788 (0.0006) [2024-09-30 01:33:14,100][1157819] Updated weights for policy 0, policy_version 65798 (0.0006) [2024-09-30 01:33:14,615][1157819] Updated weights for policy 0, policy_version 65808 (0.0006) [2024-09-30 01:33:15,112][1157819] Updated weights for policy 0, policy_version 65818 (0.0006) [2024-09-30 01:33:15,466][1157520] Fps is (10 sec: 79462.3, 60 sec: 81032.6, 300 sec: 80101.1). Total num frames: 269619200. Throughput: 0: 20209.9. Samples: 57371100. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 01:33:15,466][1157520] Avg episode reward: [(0, '53.495')] [2024-09-30 01:33:15,604][1157819] Updated weights for policy 0, policy_version 65828 (0.0006) [2024-09-30 01:33:16,142][1157819] Updated weights for policy 0, policy_version 65838 (0.0006) [2024-09-30 01:33:16,641][1157819] Updated weights for policy 0, policy_version 65848 (0.0006) [2024-09-30 01:33:17,171][1157819] Updated weights for policy 0, policy_version 65858 (0.0006) [2024-09-30 01:33:17,707][1157819] Updated weights for policy 0, policy_version 65868 (0.0006) [2024-09-30 01:33:18,199][1157819] Updated weights for policy 0, policy_version 65878 (0.0006) [2024-09-30 01:33:18,693][1157819] Updated weights for policy 0, policy_version 65888 (0.0006) [2024-09-30 01:33:19,188][1157819] Updated weights for policy 0, policy_version 65898 (0.0006) [2024-09-30 01:33:19,755][1157819] Updated weights for policy 0, policy_version 65908 (0.0006) [2024-09-30 01:33:20,288][1157819] Updated weights for policy 0, policy_version 65918 (0.0006) [2024-09-30 01:33:20,466][1157520] Fps is (10 sec: 79872.6, 60 sec: 80964.3, 300 sec: 80017.8). Total num frames: 270012416. Throughput: 0: 20018.7. Samples: 57490992. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 01:33:20,466][1157520] Avg episode reward: [(0, '52.012')] [2024-09-30 01:33:20,814][1157819] Updated weights for policy 0, policy_version 65928 (0.0006) [2024-09-30 01:33:21,369][1157819] Updated weights for policy 0, policy_version 65938 (0.0006) [2024-09-30 01:33:21,908][1157819] Updated weights for policy 0, policy_version 65948 (0.0006) [2024-09-30 01:33:22,487][1157819] Updated weights for policy 0, policy_version 65958 (0.0006) [2024-09-30 01:33:23,033][1157819] Updated weights for policy 0, policy_version 65968 (0.0006) [2024-09-30 01:33:23,611][1157819] Updated weights for policy 0, policy_version 65978 (0.0006) [2024-09-30 01:33:24,160][1157819] Updated weights for policy 0, policy_version 65988 (0.0006) [2024-09-30 01:33:24,680][1157819] Updated weights for policy 0, policy_version 65998 (0.0006) [2024-09-30 01:33:25,260][1157819] Updated weights for policy 0, policy_version 66008 (0.0006) [2024-09-30 01:33:25,466][1157520] Fps is (10 sec: 76595.5, 60 sec: 80281.7, 300 sec: 79851.2). Total num frames: 270385152. Throughput: 0: 19852.2. Samples: 57546960. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 01:33:25,466][1157520] Avg episode reward: [(0, '53.485')] [2024-09-30 01:33:25,784][1157819] Updated weights for policy 0, policy_version 66018 (0.0006) [2024-09-30 01:33:26,388][1157819] Updated weights for policy 0, policy_version 66028 (0.0006) [2024-09-30 01:33:26,889][1157819] Updated weights for policy 0, policy_version 66038 (0.0006) [2024-09-30 01:33:27,372][1157819] Updated weights for policy 0, policy_version 66048 (0.0006) [2024-09-30 01:33:27,859][1157819] Updated weights for policy 0, policy_version 66058 (0.0006) [2024-09-30 01:33:28,363][1157819] Updated weights for policy 0, policy_version 66068 (0.0006) [2024-09-30 01:33:28,876][1157819] Updated weights for policy 0, policy_version 66078 (0.0006) [2024-09-30 01:33:29,368][1157819] Updated weights for policy 0, policy_version 66088 (0.0006) [2024-09-30 01:33:29,861][1157819] Updated weights for policy 0, policy_version 66098 (0.0006) [2024-09-30 01:33:30,356][1157819] Updated weights for policy 0, policy_version 66108 (0.0006) [2024-09-30 01:33:30,466][1157520] Fps is (10 sec: 77004.7, 60 sec: 79940.2, 300 sec: 79767.9). Total num frames: 270782464. Throughput: 0: 19662.3. Samples: 57662192. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 01:33:30,466][1157520] Avg episode reward: [(0, '52.879')] [2024-09-30 01:33:30,873][1157819] Updated weights for policy 0, policy_version 66118 (0.0006) [2024-09-30 01:33:31,382][1157819] Updated weights for policy 0, policy_version 66128 (0.0006) [2024-09-30 01:33:31,948][1157819] Updated weights for policy 0, policy_version 66138 (0.0006) [2024-09-30 01:33:32,448][1157819] Updated weights for policy 0, policy_version 66148 (0.0006) [2024-09-30 01:33:32,946][1157819] Updated weights for policy 0, policy_version 66158 (0.0006) [2024-09-30 01:33:33,476][1157819] Updated weights for policy 0, policy_version 66168 (0.0006) [2024-09-30 01:33:33,994][1157819] Updated weights for policy 0, policy_version 66178 (0.0006) [2024-09-30 01:33:34,533][1157819] Updated weights for policy 0, policy_version 66188 (0.0006) [2024-09-30 01:33:35,040][1157819] Updated weights for policy 0, policy_version 66198 (0.0006) [2024-09-30 01:33:35,466][1157520] Fps is (10 sec: 79871.7, 60 sec: 79462.4, 300 sec: 79726.2). Total num frames: 271183872. Throughput: 0: 19546.3. Samples: 57781976. Policy #0 lag: (min: 0.0, avg: 1.5, max: 4.0) [2024-09-30 01:33:35,466][1157520] Avg episode reward: [(0, '53.910')] [2024-09-30 01:33:35,511][1157819] Updated weights for policy 0, policy_version 66208 (0.0006) [2024-09-30 01:33:36,007][1157819] Updated weights for policy 0, policy_version 66218 (0.0006) [2024-09-30 01:33:36,506][1157819] Updated weights for policy 0, policy_version 66228 (0.0006) [2024-09-30 01:33:36,987][1157819] Updated weights for policy 0, policy_version 66238 (0.0006) [2024-09-30 01:33:37,471][1157819] Updated weights for policy 0, policy_version 66248 (0.0006) [2024-09-30 01:33:37,970][1157819] Updated weights for policy 0, policy_version 66258 (0.0006) [2024-09-30 01:33:38,446][1157819] Updated weights for policy 0, policy_version 66268 (0.0006) [2024-09-30 01:33:38,942][1157819] Updated weights for policy 0, policy_version 66278 (0.0006) [2024-09-30 01:33:39,429][1157819] Updated weights for policy 0, policy_version 66288 (0.0006) [2024-09-30 01:33:39,882][1157819] Updated weights for policy 0, policy_version 66298 (0.0006) [2024-09-30 01:33:40,376][1157819] Updated weights for policy 0, policy_version 66308 (0.0006) [2024-09-30 01:33:40,466][1157520] Fps is (10 sec: 81920.0, 60 sec: 79394.1, 300 sec: 79656.8). Total num frames: 271601664. Throughput: 0: 19604.0. Samples: 57844560. Policy #0 lag: (min: 0.0, avg: 1.5, max: 4.0) [2024-09-30 01:33:40,466][1157520] Avg episode reward: [(0, '52.520')] [2024-09-30 01:33:40,913][1157819] Updated weights for policy 0, policy_version 66318 (0.0006) [2024-09-30 01:33:41,413][1157819] Updated weights for policy 0, policy_version 66328 (0.0006) [2024-09-30 01:33:41,902][1157819] Updated weights for policy 0, policy_version 66338 (0.0006) [2024-09-30 01:33:42,380][1157819] Updated weights for policy 0, policy_version 66348 (0.0006) [2024-09-30 01:33:42,905][1157819] Updated weights for policy 0, policy_version 66358 (0.0006) [2024-09-30 01:33:43,391][1157819] Updated weights for policy 0, policy_version 66368 (0.0006) [2024-09-30 01:33:43,887][1157819] Updated weights for policy 0, policy_version 66378 (0.0006) [2024-09-30 01:33:44,342][1157819] Updated weights for policy 0, policy_version 66388 (0.0006) [2024-09-30 01:33:44,835][1157819] Updated weights for policy 0, policy_version 66398 (0.0006) [2024-09-30 01:33:45,339][1157819] Updated weights for policy 0, policy_version 66408 (0.0006) [2024-09-30 01:33:45,466][1157520] Fps is (10 sec: 83149.0, 60 sec: 79462.4, 300 sec: 79615.1). Total num frames: 272015360. Throughput: 0: 19828.8. Samples: 57969364. Policy #0 lag: (min: 0.0, avg: 1.5, max: 4.0) [2024-09-30 01:33:45,466][1157520] Avg episode reward: [(0, '55.643')] [2024-09-30 01:33:45,832][1157819] Updated weights for policy 0, policy_version 66418 (0.0006) [2024-09-30 01:33:46,329][1157819] Updated weights for policy 0, policy_version 66428 (0.0006) [2024-09-30 01:33:46,827][1157819] Updated weights for policy 0, policy_version 66438 (0.0006) [2024-09-30 01:33:47,291][1157819] Updated weights for policy 0, policy_version 66448 (0.0006) [2024-09-30 01:33:47,778][1157819] Updated weights for policy 0, policy_version 66458 (0.0006) [2024-09-30 01:33:48,234][1157819] Updated weights for policy 0, policy_version 66468 (0.0006) [2024-09-30 01:33:48,719][1157819] Updated weights for policy 0, policy_version 66478 (0.0006) [2024-09-30 01:33:48,962][1157736] Signal inference workers to stop experience collection... (4250 times) [2024-09-30 01:33:48,963][1157736] Signal inference workers to resume experience collection... (4250 times) [2024-09-30 01:33:48,966][1157819] InferenceWorker_p0-w0: stopping experience collection (4250 times) [2024-09-30 01:33:48,967][1157819] InferenceWorker_p0-w0: resuming experience collection (4250 times) [2024-09-30 01:33:49,218][1157819] Updated weights for policy 0, policy_version 66488 (0.0006) [2024-09-30 01:33:49,695][1157819] Updated weights for policy 0, policy_version 66498 (0.0006) [2024-09-30 01:33:50,189][1157819] Updated weights for policy 0, policy_version 66508 (0.0006) [2024-09-30 01:33:50,466][1157520] Fps is (10 sec: 83558.7, 60 sec: 79599.0, 300 sec: 79629.0). Total num frames: 272437248. Throughput: 0: 20035.3. Samples: 58095560. Policy #0 lag: (min: 0.0, avg: 1.5, max: 4.0) [2024-09-30 01:33:50,466][1157520] Avg episode reward: [(0, '52.958')] [2024-09-30 01:33:50,670][1157819] Updated weights for policy 0, policy_version 66518 (0.0006) [2024-09-30 01:33:51,198][1157819] Updated weights for policy 0, policy_version 66528 (0.0006) [2024-09-30 01:33:51,659][1157819] Updated weights for policy 0, policy_version 66538 (0.0006) [2024-09-30 01:33:52,199][1157819] Updated weights for policy 0, policy_version 66548 (0.0006) [2024-09-30 01:33:52,697][1157819] Updated weights for policy 0, policy_version 66558 (0.0006) [2024-09-30 01:33:53,186][1157819] Updated weights for policy 0, policy_version 66568 (0.0006) [2024-09-30 01:33:53,709][1157819] Updated weights for policy 0, policy_version 66578 (0.0006) [2024-09-30 01:33:54,204][1157819] Updated weights for policy 0, policy_version 66588 (0.0006) [2024-09-30 01:33:54,697][1157819] Updated weights for policy 0, policy_version 66598 (0.0006) [2024-09-30 01:33:55,204][1157819] Updated weights for policy 0, policy_version 66608 (0.0006) [2024-09-30 01:33:55,466][1157520] Fps is (10 sec: 82739.5, 60 sec: 79803.8, 300 sec: 79615.1). Total num frames: 272842752. Throughput: 0: 20152.8. Samples: 58157564. Policy #0 lag: (min: 0.0, avg: 1.5, max: 4.0) [2024-09-30 01:33:55,466][1157520] Avg episode reward: [(0, '54.206')] [2024-09-30 01:33:55,702][1157819] Updated weights for policy 0, policy_version 66618 (0.0006) [2024-09-30 01:33:56,201][1157819] Updated weights for policy 0, policy_version 66628 (0.0006) [2024-09-30 01:33:56,655][1157819] Updated weights for policy 0, policy_version 66638 (0.0006) [2024-09-30 01:33:57,151][1157819] Updated weights for policy 0, policy_version 66648 (0.0006) [2024-09-30 01:33:57,648][1157819] Updated weights for policy 0, policy_version 66658 (0.0006) [2024-09-30 01:33:58,118][1157819] Updated weights for policy 0, policy_version 66668 (0.0006) [2024-09-30 01:33:58,618][1157819] Updated weights for policy 0, policy_version 66678 (0.0006) [2024-09-30 01:33:59,102][1157819] Updated weights for policy 0, policy_version 66688 (0.0006) [2024-09-30 01:33:59,603][1157819] Updated weights for policy 0, policy_version 66698 (0.0006) [2024-09-30 01:34:00,109][1157819] Updated weights for policy 0, policy_version 66708 (0.0006) [2024-09-30 01:34:00,466][1157520] Fps is (10 sec: 82738.9, 60 sec: 80486.4, 300 sec: 79767.9). Total num frames: 273264640. Throughput: 0: 20235.5. Samples: 58281696. Policy #0 lag: (min: 0.0, avg: 1.5, max: 4.0) [2024-09-30 01:34:00,466][1157520] Avg episode reward: [(0, '53.939')] [2024-09-30 01:34:00,630][1157819] Updated weights for policy 0, policy_version 66718 (0.0006) [2024-09-30 01:34:01,134][1157819] Updated weights for policy 0, policy_version 66728 (0.0006) [2024-09-30 01:34:01,657][1157819] Updated weights for policy 0, policy_version 66738 (0.0006) [2024-09-30 01:34:02,184][1157819] Updated weights for policy 0, policy_version 66748 (0.0006) [2024-09-30 01:34:02,694][1157819] Updated weights for policy 0, policy_version 66758 (0.0006) [2024-09-30 01:34:03,200][1157819] Updated weights for policy 0, policy_version 66768 (0.0006) [2024-09-30 01:34:03,708][1157819] Updated weights for policy 0, policy_version 66778 (0.0006) [2024-09-30 01:34:04,288][1157819] Updated weights for policy 0, policy_version 66788 (0.0006) [2024-09-30 01:34:04,839][1157819] Updated weights for policy 0, policy_version 66798 (0.0006) [2024-09-30 01:34:05,382][1157819] Updated weights for policy 0, policy_version 66808 (0.0006) [2024-09-30 01:34:05,466][1157520] Fps is (10 sec: 80690.5, 60 sec: 80418.1, 300 sec: 79934.5). Total num frames: 273649664. Throughput: 0: 20207.2. Samples: 58400316. Policy #0 lag: (min: 0.0, avg: 1.5, max: 4.0) [2024-09-30 01:34:05,466][1157520] Avg episode reward: [(0, '52.286')] [2024-09-30 01:34:05,479][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000066810_273653760.pth... [2024-09-30 01:34:05,523][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000062081_254283776.pth [2024-09-30 01:34:05,918][1157819] Updated weights for policy 0, policy_version 66818 (0.0006) [2024-09-30 01:34:06,465][1157819] Updated weights for policy 0, policy_version 66828 (0.0006) [2024-09-30 01:34:07,010][1157819] Updated weights for policy 0, policy_version 66838 (0.0006) [2024-09-30 01:34:07,543][1157819] Updated weights for policy 0, policy_version 66848 (0.0006) [2024-09-30 01:34:08,103][1157819] Updated weights for policy 0, policy_version 66858 (0.0006) [2024-09-30 01:34:08,655][1157819] Updated weights for policy 0, policy_version 66868 (0.0006) [2024-09-30 01:34:09,206][1157819] Updated weights for policy 0, policy_version 66878 (0.0006) [2024-09-30 01:34:09,760][1157819] Updated weights for policy 0, policy_version 66888 (0.0006) [2024-09-30 01:34:10,287][1157819] Updated weights for policy 0, policy_version 66898 (0.0006) [2024-09-30 01:34:10,466][1157520] Fps is (10 sec: 76595.5, 60 sec: 80281.7, 300 sec: 80073.4). Total num frames: 274030592. Throughput: 0: 20215.0. Samples: 58456636. Policy #0 lag: (min: 0.0, avg: 1.5, max: 4.0) [2024-09-30 01:34:10,466][1157520] Avg episode reward: [(0, '54.926')] [2024-09-30 01:34:10,789][1157819] Updated weights for policy 0, policy_version 66908 (0.0006) [2024-09-30 01:34:11,303][1157819] Updated weights for policy 0, policy_version 66918 (0.0006) [2024-09-30 01:34:11,849][1157819] Updated weights for policy 0, policy_version 66928 (0.0006) [2024-09-30 01:34:12,390][1157819] Updated weights for policy 0, policy_version 66938 (0.0006) [2024-09-30 01:34:12,878][1157819] Updated weights for policy 0, policy_version 66948 (0.0006) [2024-09-30 01:34:13,389][1157819] Updated weights for policy 0, policy_version 66958 (0.0006) [2024-09-30 01:34:13,897][1157819] Updated weights for policy 0, policy_version 66968 (0.0006) [2024-09-30 01:34:14,424][1157819] Updated weights for policy 0, policy_version 66978 (0.0006) [2024-09-30 01:34:14,944][1157819] Updated weights for policy 0, policy_version 66988 (0.0006) [2024-09-30 01:34:15,463][1157819] Updated weights for policy 0, policy_version 66998 (0.0006) [2024-09-30 01:34:15,466][1157520] Fps is (10 sec: 77414.8, 60 sec: 80076.8, 300 sec: 80128.9). Total num frames: 274423808. Throughput: 0: 20237.8. Samples: 58572892. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:34:15,466][1157520] Avg episode reward: [(0, '54.823')] [2024-09-30 01:34:15,993][1157819] Updated weights for policy 0, policy_version 67008 (0.0006) [2024-09-30 01:34:16,496][1157819] Updated weights for policy 0, policy_version 67018 (0.0006) [2024-09-30 01:34:17,013][1157819] Updated weights for policy 0, policy_version 67028 (0.0006) [2024-09-30 01:34:17,518][1157819] Updated weights for policy 0, policy_version 67038 (0.0006) [2024-09-30 01:34:18,036][1157819] Updated weights for policy 0, policy_version 67048 (0.0006) [2024-09-30 01:34:18,561][1157819] Updated weights for policy 0, policy_version 67058 (0.0006) [2024-09-30 01:34:19,095][1157819] Updated weights for policy 0, policy_version 67068 (0.0006) [2024-09-30 01:34:19,590][1157819] Updated weights for policy 0, policy_version 67078 (0.0006) [2024-09-30 01:34:20,079][1157819] Updated weights for policy 0, policy_version 67088 (0.0006) [2024-09-30 01:34:20,466][1157520] Fps is (10 sec: 79052.6, 60 sec: 80145.1, 300 sec: 80184.4). Total num frames: 274821120. Throughput: 0: 20219.8. Samples: 58691868. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:34:20,466][1157520] Avg episode reward: [(0, '52.012')] [2024-09-30 01:34:20,582][1157819] Updated weights for policy 0, policy_version 67098 (0.0006) [2024-09-30 01:34:21,118][1157819] Updated weights for policy 0, policy_version 67108 (0.0006) [2024-09-30 01:34:21,639][1157819] Updated weights for policy 0, policy_version 67118 (0.0006) [2024-09-30 01:34:22,139][1157819] Updated weights for policy 0, policy_version 67128 (0.0006) [2024-09-30 01:34:22,647][1157819] Updated weights for policy 0, policy_version 67138 (0.0006) [2024-09-30 01:34:23,161][1157819] Updated weights for policy 0, policy_version 67148 (0.0006) [2024-09-30 01:34:23,677][1157819] Updated weights for policy 0, policy_version 67158 (0.0006) [2024-09-30 01:34:24,200][1157819] Updated weights for policy 0, policy_version 67168 (0.0006) [2024-09-30 01:34:24,694][1157819] Updated weights for policy 0, policy_version 67178 (0.0006) [2024-09-30 01:34:25,210][1157819] Updated weights for policy 0, policy_version 67188 (0.0006) [2024-09-30 01:34:25,466][1157520] Fps is (10 sec: 79052.6, 60 sec: 80486.4, 300 sec: 80295.5). Total num frames: 275214336. Throughput: 0: 20164.4. Samples: 58751960. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:34:25,466][1157520] Avg episode reward: [(0, '54.061')] [2024-09-30 01:34:25,780][1157819] Updated weights for policy 0, policy_version 67198 (0.0006) [2024-09-30 01:34:26,290][1157819] Updated weights for policy 0, policy_version 67208 (0.0006) [2024-09-30 01:34:26,790][1157819] Updated weights for policy 0, policy_version 67218 (0.0006) [2024-09-30 01:34:27,292][1157819] Updated weights for policy 0, policy_version 67228 (0.0006) [2024-09-30 01:34:27,815][1157819] Updated weights for policy 0, policy_version 67238 (0.0006) [2024-09-30 01:34:28,338][1157819] Updated weights for policy 0, policy_version 67248 (0.0006) [2024-09-30 01:34:28,858][1157819] Updated weights for policy 0, policy_version 67258 (0.0006) [2024-09-30 01:34:29,390][1157819] Updated weights for policy 0, policy_version 67268 (0.0006) [2024-09-30 01:34:29,906][1157819] Updated weights for policy 0, policy_version 67278 (0.0006) [2024-09-30 01:34:30,435][1157819] Updated weights for policy 0, policy_version 67288 (0.0006) [2024-09-30 01:34:30,466][1157520] Fps is (10 sec: 79053.0, 60 sec: 80486.5, 300 sec: 80323.3). Total num frames: 275611648. Throughput: 0: 20028.8. Samples: 58870660. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:34:30,466][1157520] Avg episode reward: [(0, '52.612')] [2024-09-30 01:34:30,942][1157819] Updated weights for policy 0, policy_version 67298 (0.0006) [2024-09-30 01:34:31,444][1157819] Updated weights for policy 0, policy_version 67308 (0.0006) [2024-09-30 01:34:31,990][1157819] Updated weights for policy 0, policy_version 67318 (0.0006) [2024-09-30 01:34:32,494][1157819] Updated weights for policy 0, policy_version 67328 (0.0006) [2024-09-30 01:34:32,994][1157819] Updated weights for policy 0, policy_version 67338 (0.0006) [2024-09-30 01:34:33,523][1157819] Updated weights for policy 0, policy_version 67348 (0.0006) [2024-09-30 01:34:34,046][1157819] Updated weights for policy 0, policy_version 67358 (0.0006) [2024-09-30 01:34:34,537][1157819] Updated weights for policy 0, policy_version 67368 (0.0006) [2024-09-30 01:34:34,941][1157819] Updated weights for policy 0, policy_version 67378 (0.0006) [2024-09-30 01:34:35,418][1157819] Updated weights for policy 0, policy_version 67388 (0.0006) [2024-09-30 01:34:35,466][1157520] Fps is (10 sec: 81510.8, 60 sec: 80759.6, 300 sec: 80378.8). Total num frames: 276029440. Throughput: 0: 19899.3. Samples: 58991028. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:34:35,466][1157520] Avg episode reward: [(0, '52.116')] [2024-09-30 01:34:35,866][1157819] Updated weights for policy 0, policy_version 67398 (0.0006) [2024-09-30 01:34:36,308][1157819] Updated weights for policy 0, policy_version 67408 (0.0006) [2024-09-30 01:34:36,761][1157819] Updated weights for policy 0, policy_version 67418 (0.0006) [2024-09-30 01:34:37,194][1157819] Updated weights for policy 0, policy_version 67428 (0.0006) [2024-09-30 01:34:37,684][1157819] Updated weights for policy 0, policy_version 67438 (0.0006) [2024-09-30 01:34:38,167][1157819] Updated weights for policy 0, policy_version 67448 (0.0006) [2024-09-30 01:34:38,645][1157819] Updated weights for policy 0, policy_version 67458 (0.0006) [2024-09-30 01:34:39,145][1157819] Updated weights for policy 0, policy_version 67468 (0.0006) [2024-09-30 01:34:39,171][1157736] Signal inference workers to stop experience collection... (4300 times) [2024-09-30 01:34:39,176][1157736] Signal inference workers to resume experience collection... (4300 times) [2024-09-30 01:34:39,177][1157819] InferenceWorker_p0-w0: stopping experience collection (4300 times) [2024-09-30 01:34:39,181][1157819] InferenceWorker_p0-w0: resuming experience collection (4300 times) [2024-09-30 01:34:39,590][1157819] Updated weights for policy 0, policy_version 67478 (0.0006) [2024-09-30 01:34:40,061][1157819] Updated weights for policy 0, policy_version 67488 (0.0006) [2024-09-30 01:34:40,466][1157520] Fps is (10 sec: 84786.9, 60 sec: 80964.3, 300 sec: 80503.8). Total num frames: 276459520. Throughput: 0: 20025.0. Samples: 59058688. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:34:40,466][1157520] Avg episode reward: [(0, '53.319')] [2024-09-30 01:34:40,580][1157819] Updated weights for policy 0, policy_version 67498 (0.0006) [2024-09-30 01:34:41,068][1157819] Updated weights for policy 0, policy_version 67508 (0.0006) [2024-09-30 01:34:41,562][1157819] Updated weights for policy 0, policy_version 67518 (0.0006) [2024-09-30 01:34:42,046][1157819] Updated weights for policy 0, policy_version 67528 (0.0006) [2024-09-30 01:34:42,539][1157819] Updated weights for policy 0, policy_version 67538 (0.0006) [2024-09-30 01:34:43,045][1157819] Updated weights for policy 0, policy_version 67548 (0.0006) [2024-09-30 01:34:43,538][1157819] Updated weights for policy 0, policy_version 67558 (0.0006) [2024-09-30 01:34:44,027][1157819] Updated weights for policy 0, policy_version 67568 (0.0006) [2024-09-30 01:34:44,524][1157819] Updated weights for policy 0, policy_version 67578 (0.0006) [2024-09-30 01:34:45,007][1157819] Updated weights for policy 0, policy_version 67588 (0.0006) [2024-09-30 01:34:45,466][1157520] Fps is (10 sec: 84786.4, 60 sec: 81032.5, 300 sec: 80656.5). Total num frames: 276877312. Throughput: 0: 20066.5. Samples: 59184692. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:34:45,466][1157520] Avg episode reward: [(0, '55.413')] [2024-09-30 01:34:45,506][1157819] Updated weights for policy 0, policy_version 67598 (0.0006) [2024-09-30 01:34:46,003][1157819] Updated weights for policy 0, policy_version 67608 (0.0006) [2024-09-30 01:34:46,505][1157819] Updated weights for policy 0, policy_version 67618 (0.0006) [2024-09-30 01:34:47,008][1157819] Updated weights for policy 0, policy_version 67628 (0.0006) [2024-09-30 01:34:47,511][1157819] Updated weights for policy 0, policy_version 67638 (0.0006) [2024-09-30 01:34:48,010][1157819] Updated weights for policy 0, policy_version 67648 (0.0006) [2024-09-30 01:34:48,563][1157819] Updated weights for policy 0, policy_version 67658 (0.0006) [2024-09-30 01:34:49,089][1157819] Updated weights for policy 0, policy_version 67668 (0.0006) [2024-09-30 01:34:49,599][1157819] Updated weights for policy 0, policy_version 67678 (0.0006) [2024-09-30 01:34:50,116][1157819] Updated weights for policy 0, policy_version 67688 (0.0006) [2024-09-30 01:34:50,466][1157520] Fps is (10 sec: 81510.3, 60 sec: 80622.9, 300 sec: 80739.8). Total num frames: 277274624. Throughput: 0: 20125.3. Samples: 59305952. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:34:50,466][1157520] Avg episode reward: [(0, '51.173')] [2024-09-30 01:34:50,650][1157819] Updated weights for policy 0, policy_version 67698 (0.0006) [2024-09-30 01:34:51,155][1157819] Updated weights for policy 0, policy_version 67708 (0.0006) [2024-09-30 01:34:51,676][1157819] Updated weights for policy 0, policy_version 67718 (0.0006) [2024-09-30 01:34:52,177][1157819] Updated weights for policy 0, policy_version 67728 (0.0006) [2024-09-30 01:34:52,686][1157819] Updated weights for policy 0, policy_version 67738 (0.0006) [2024-09-30 01:34:53,177][1157819] Updated weights for policy 0, policy_version 67748 (0.0006) [2024-09-30 01:34:53,674][1157819] Updated weights for policy 0, policy_version 67758 (0.0006) [2024-09-30 01:34:54,180][1157819] Updated weights for policy 0, policy_version 67768 (0.0006) [2024-09-30 01:34:54,655][1157819] Updated weights for policy 0, policy_version 67778 (0.0006) [2024-09-30 01:34:55,144][1157819] Updated weights for policy 0, policy_version 67788 (0.0006) [2024-09-30 01:34:55,466][1157520] Fps is (10 sec: 80691.4, 60 sec: 80691.1, 300 sec: 80795.3). Total num frames: 277684224. Throughput: 0: 20197.0. Samples: 59365504. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 01:34:55,466][1157520] Avg episode reward: [(0, '53.278')] [2024-09-30 01:34:55,619][1157819] Updated weights for policy 0, policy_version 67798 (0.0006) [2024-09-30 01:34:56,051][1157819] Updated weights for policy 0, policy_version 67808 (0.0006) [2024-09-30 01:34:56,551][1157819] Updated weights for policy 0, policy_version 67818 (0.0006) [2024-09-30 01:34:57,029][1157819] Updated weights for policy 0, policy_version 67828 (0.0006) [2024-09-30 01:34:57,500][1157819] Updated weights for policy 0, policy_version 67838 (0.0006) [2024-09-30 01:34:57,996][1157819] Updated weights for policy 0, policy_version 67848 (0.0006) [2024-09-30 01:34:58,476][1157819] Updated weights for policy 0, policy_version 67858 (0.0006) [2024-09-30 01:34:58,936][1157819] Updated weights for policy 0, policy_version 67868 (0.0006) [2024-09-30 01:34:59,437][1157819] Updated weights for policy 0, policy_version 67878 (0.0006) [2024-09-30 01:34:59,920][1157819] Updated weights for policy 0, policy_version 67888 (0.0006) [2024-09-30 01:35:00,379][1157819] Updated weights for policy 0, policy_version 67898 (0.0006) [2024-09-30 01:35:00,466][1157520] Fps is (10 sec: 83968.0, 60 sec: 80827.7, 300 sec: 80795.4). Total num frames: 278114304. Throughput: 0: 20460.4. Samples: 59493608. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 01:35:00,466][1157520] Avg episode reward: [(0, '50.259')] [2024-09-30 01:35:00,843][1157819] Updated weights for policy 0, policy_version 67908 (0.0006) [2024-09-30 01:35:01,312][1157819] Updated weights for policy 0, policy_version 67918 (0.0006) [2024-09-30 01:35:01,802][1157819] Updated weights for policy 0, policy_version 67928 (0.0006) [2024-09-30 01:35:02,251][1157819] Updated weights for policy 0, policy_version 67938 (0.0006) [2024-09-30 01:35:02,740][1157819] Updated weights for policy 0, policy_version 67948 (0.0006) [2024-09-30 01:35:03,211][1157819] Updated weights for policy 0, policy_version 67958 (0.0006) [2024-09-30 01:35:03,649][1157819] Updated weights for policy 0, policy_version 67968 (0.0006) [2024-09-30 01:35:04,137][1157819] Updated weights for policy 0, policy_version 67978 (0.0006) [2024-09-30 01:35:04,609][1157819] Updated weights for policy 0, policy_version 67988 (0.0006) [2024-09-30 01:35:05,084][1157819] Updated weights for policy 0, policy_version 67998 (0.0006) [2024-09-30 01:35:05,466][1157520] Fps is (10 sec: 86835.6, 60 sec: 81715.3, 300 sec: 80739.8). Total num frames: 278552576. Throughput: 0: 20703.0. Samples: 59623504. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 01:35:05,466][1157520] Avg episode reward: [(0, '51.373')] [2024-09-30 01:35:05,573][1157819] Updated weights for policy 0, policy_version 68008 (0.0006) [2024-09-30 01:35:06,012][1157819] Updated weights for policy 0, policy_version 68018 (0.0006) [2024-09-30 01:35:06,485][1157819] Updated weights for policy 0, policy_version 68028 (0.0006) [2024-09-30 01:35:06,970][1157819] Updated weights for policy 0, policy_version 68038 (0.0006) [2024-09-30 01:35:07,469][1157819] Updated weights for policy 0, policy_version 68048 (0.0006) [2024-09-30 01:35:07,947][1157819] Updated weights for policy 0, policy_version 68058 (0.0006) [2024-09-30 01:35:08,398][1157819] Updated weights for policy 0, policy_version 68068 (0.0006) [2024-09-30 01:35:08,856][1157819] Updated weights for policy 0, policy_version 68078 (0.0006) [2024-09-30 01:35:09,344][1157819] Updated weights for policy 0, policy_version 68088 (0.0006) [2024-09-30 01:35:09,832][1157819] Updated weights for policy 0, policy_version 68098 (0.0006) [2024-09-30 01:35:10,314][1157819] Updated weights for policy 0, policy_version 68108 (0.0006) [2024-09-30 01:35:10,466][1157520] Fps is (10 sec: 86835.6, 60 sec: 82534.4, 300 sec: 80698.2). Total num frames: 278982656. Throughput: 0: 20809.4. Samples: 59688384. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 01:35:10,466][1157520] Avg episode reward: [(0, '53.353')] [2024-09-30 01:35:10,758][1157819] Updated weights for policy 0, policy_version 68118 (0.0006) [2024-09-30 01:35:11,212][1157819] Updated weights for policy 0, policy_version 68128 (0.0006) [2024-09-30 01:35:11,703][1157819] Updated weights for policy 0, policy_version 68138 (0.0006) [2024-09-30 01:35:12,198][1157819] Updated weights for policy 0, policy_version 68148 (0.0006) [2024-09-30 01:35:12,670][1157819] Updated weights for policy 0, policy_version 68158 (0.0006) [2024-09-30 01:35:13,153][1157819] Updated weights for policy 0, policy_version 68168 (0.0006) [2024-09-30 01:35:13,586][1157819] Updated weights for policy 0, policy_version 68178 (0.0006) [2024-09-30 01:35:14,076][1157819] Updated weights for policy 0, policy_version 68188 (0.0006) [2024-09-30 01:35:14,564][1157819] Updated weights for policy 0, policy_version 68198 (0.0006) [2024-09-30 01:35:15,035][1157819] Updated weights for policy 0, policy_version 68208 (0.0006) [2024-09-30 01:35:15,466][1157520] Fps is (10 sec: 86835.3, 60 sec: 83285.4, 300 sec: 80712.1). Total num frames: 279420928. Throughput: 0: 21051.9. Samples: 59817996. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 01:35:15,466][1157819] Updated weights for policy 0, policy_version 68218 (0.0006) [2024-09-30 01:35:15,466][1157520] Avg episode reward: [(0, '55.446')] [2024-09-30 01:35:15,947][1157819] Updated weights for policy 0, policy_version 68228 (0.0006) [2024-09-30 01:35:16,429][1157819] Updated weights for policy 0, policy_version 68238 (0.0006) [2024-09-30 01:35:16,938][1157819] Updated weights for policy 0, policy_version 68248 (0.0006) [2024-09-30 01:35:17,442][1157819] Updated weights for policy 0, policy_version 68258 (0.0006) [2024-09-30 01:35:17,446][1157736] Signal inference workers to stop experience collection... (4350 times) [2024-09-30 01:35:17,446][1157736] Signal inference workers to resume experience collection... (4350 times) [2024-09-30 01:35:17,450][1157819] InferenceWorker_p0-w0: stopping experience collection (4350 times) [2024-09-30 01:35:17,450][1157819] InferenceWorker_p0-w0: resuming experience collection (4350 times) [2024-09-30 01:35:17,934][1157819] Updated weights for policy 0, policy_version 68268 (0.0006) [2024-09-30 01:35:18,423][1157819] Updated weights for policy 0, policy_version 68278 (0.0006) [2024-09-30 01:35:18,921][1157819] Updated weights for policy 0, policy_version 68288 (0.0006) [2024-09-30 01:35:19,406][1157819] Updated weights for policy 0, policy_version 68298 (0.0006) [2024-09-30 01:35:19,881][1157819] Updated weights for policy 0, policy_version 68308 (0.0006) [2024-09-30 01:35:20,300][1157819] Updated weights for policy 0, policy_version 68318 (0.0006) [2024-09-30 01:35:20,466][1157520] Fps is (10 sec: 85605.9, 60 sec: 83626.6, 300 sec: 80725.9). Total num frames: 279838720. Throughput: 0: 21217.0. Samples: 59945792. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 01:35:20,466][1157520] Avg episode reward: [(0, '53.219')] [2024-09-30 01:35:20,790][1157819] Updated weights for policy 0, policy_version 68328 (0.0006) [2024-09-30 01:35:21,285][1157819] Updated weights for policy 0, policy_version 68338 (0.0006) [2024-09-30 01:35:21,781][1157819] Updated weights for policy 0, policy_version 68348 (0.0006) [2024-09-30 01:35:22,264][1157819] Updated weights for policy 0, policy_version 68358 (0.0006) [2024-09-30 01:35:22,759][1157819] Updated weights for policy 0, policy_version 68368 (0.0006) [2024-09-30 01:35:23,251][1157819] Updated weights for policy 0, policy_version 68378 (0.0006) [2024-09-30 01:35:23,739][1157819] Updated weights for policy 0, policy_version 68388 (0.0006) [2024-09-30 01:35:24,235][1157819] Updated weights for policy 0, policy_version 68398 (0.0006) [2024-09-30 01:35:24,725][1157819] Updated weights for policy 0, policy_version 68408 (0.0006) [2024-09-30 01:35:25,223][1157819] Updated weights for policy 0, policy_version 68418 (0.0006) [2024-09-30 01:35:25,466][1157520] Fps is (10 sec: 83967.8, 60 sec: 84104.6, 300 sec: 80670.4). Total num frames: 280260608. Throughput: 0: 21128.4. Samples: 60009464. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 01:35:25,466][1157520] Avg episode reward: [(0, '51.963')] [2024-09-30 01:35:25,720][1157819] Updated weights for policy 0, policy_version 68428 (0.0006) [2024-09-30 01:35:26,214][1157819] Updated weights for policy 0, policy_version 68438 (0.0006) [2024-09-30 01:35:26,685][1157819] Updated weights for policy 0, policy_version 68448 (0.0006) [2024-09-30 01:35:27,142][1157819] Updated weights for policy 0, policy_version 68458 (0.0006) [2024-09-30 01:35:27,612][1157819] Updated weights for policy 0, policy_version 68468 (0.0006) [2024-09-30 01:35:28,087][1157819] Updated weights for policy 0, policy_version 68478 (0.0006) [2024-09-30 01:35:28,585][1157819] Updated weights for policy 0, policy_version 68488 (0.0006) [2024-09-30 01:35:29,078][1157819] Updated weights for policy 0, policy_version 68498 (0.0007) [2024-09-30 01:35:29,592][1157819] Updated weights for policy 0, policy_version 68508 (0.0007) [2024-09-30 01:35:30,093][1157819] Updated weights for policy 0, policy_version 68518 (0.0006) [2024-09-30 01:35:30,466][1157520] Fps is (10 sec: 83968.1, 60 sec: 84445.8, 300 sec: 80684.3). Total num frames: 280678400. Throughput: 0: 21144.1. Samples: 60136176. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 01:35:30,466][1157520] Avg episode reward: [(0, '53.837')] [2024-09-30 01:35:30,583][1157819] Updated weights for policy 0, policy_version 68528 (0.0006) [2024-09-30 01:35:31,035][1157819] Updated weights for policy 0, policy_version 68538 (0.0006) [2024-09-30 01:35:31,520][1157819] Updated weights for policy 0, policy_version 68548 (0.0006) [2024-09-30 01:35:32,002][1157819] Updated weights for policy 0, policy_version 68558 (0.0006) [2024-09-30 01:35:32,476][1157819] Updated weights for policy 0, policy_version 68568 (0.0006) [2024-09-30 01:35:32,950][1157819] Updated weights for policy 0, policy_version 68578 (0.0006) [2024-09-30 01:35:33,453][1157819] Updated weights for policy 0, policy_version 68588 (0.0006) [2024-09-30 01:35:33,928][1157819] Updated weights for policy 0, policy_version 68598 (0.0006) [2024-09-30 01:35:34,412][1157819] Updated weights for policy 0, policy_version 68608 (0.0006) [2024-09-30 01:35:34,865][1157819] Updated weights for policy 0, policy_version 68618 (0.0006) [2024-09-30 01:35:35,333][1157819] Updated weights for policy 0, policy_version 68628 (0.0006) [2024-09-30 01:35:35,466][1157520] Fps is (10 sec: 84787.0, 60 sec: 84650.6, 300 sec: 80726.0). Total num frames: 281108480. Throughput: 0: 21269.1. Samples: 60263064. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 01:35:35,466][1157520] Avg episode reward: [(0, '55.118')] [2024-09-30 01:35:35,812][1157819] Updated weights for policy 0, policy_version 68638 (0.0006) [2024-09-30 01:35:36,261][1157819] Updated weights for policy 0, policy_version 68648 (0.0006) [2024-09-30 01:35:36,688][1157819] Updated weights for policy 0, policy_version 68658 (0.0006) [2024-09-30 01:35:37,184][1157819] Updated weights for policy 0, policy_version 68668 (0.0006) [2024-09-30 01:35:37,683][1157819] Updated weights for policy 0, policy_version 68678 (0.0006) [2024-09-30 01:35:38,142][1157819] Updated weights for policy 0, policy_version 68688 (0.0006) [2024-09-30 01:35:38,574][1157819] Updated weights for policy 0, policy_version 68698 (0.0006) [2024-09-30 01:35:39,051][1157819] Updated weights for policy 0, policy_version 68708 (0.0006) [2024-09-30 01:35:39,536][1157819] Updated weights for policy 0, policy_version 68718 (0.0006) [2024-09-30 01:35:40,003][1157819] Updated weights for policy 0, policy_version 68728 (0.0006) [2024-09-30 01:35:40,445][1157819] Updated weights for policy 0, policy_version 68738 (0.0006) [2024-09-30 01:35:40,466][1157520] Fps is (10 sec: 87244.3, 60 sec: 84855.4, 300 sec: 80850.9). Total num frames: 281550848. Throughput: 0: 21415.6. Samples: 60329208. Policy #0 lag: (min: 0.0, avg: 2.8, max: 7.0) [2024-09-30 01:35:40,466][1157520] Avg episode reward: [(0, '52.696')] [2024-09-30 01:35:40,936][1157819] Updated weights for policy 0, policy_version 68748 (0.0006) [2024-09-30 01:35:41,396][1157819] Updated weights for policy 0, policy_version 68758 (0.0006) [2024-09-30 01:35:41,857][1157819] Updated weights for policy 0, policy_version 68768 (0.0006) [2024-09-30 01:35:42,353][1157819] Updated weights for policy 0, policy_version 68778 (0.0006) [2024-09-30 01:35:42,802][1157819] Updated weights for policy 0, policy_version 68788 (0.0006) [2024-09-30 01:35:43,293][1157819] Updated weights for policy 0, policy_version 68798 (0.0006) [2024-09-30 01:35:43,790][1157819] Updated weights for policy 0, policy_version 68808 (0.0006) [2024-09-30 01:35:44,332][1157819] Updated weights for policy 0, policy_version 68818 (0.0006) [2024-09-30 01:35:44,905][1157819] Updated weights for policy 0, policy_version 68828 (0.0006) [2024-09-30 01:35:45,466][1157520] Fps is (10 sec: 84787.0, 60 sec: 84650.7, 300 sec: 80864.8). Total num frames: 281956352. Throughput: 0: 21475.4. Samples: 60460004. Policy #0 lag: (min: 0.0, avg: 2.8, max: 7.0) [2024-09-30 01:35:45,466][1157520] Avg episode reward: [(0, '52.168')] [2024-09-30 01:35:45,478][1157819] Updated weights for policy 0, policy_version 68838 (0.0006) [2024-09-30 01:35:46,030][1157819] Updated weights for policy 0, policy_version 68848 (0.0006) [2024-09-30 01:35:46,128][1157736] Signal inference workers to stop experience collection... (4400 times) [2024-09-30 01:35:46,130][1157736] Signal inference workers to resume experience collection... (4400 times) [2024-09-30 01:35:46,133][1157819] InferenceWorker_p0-w0: stopping experience collection (4400 times) [2024-09-30 01:35:46,137][1157819] InferenceWorker_p0-w0: resuming experience collection (4400 times) [2024-09-30 01:35:46,583][1157819] Updated weights for policy 0, policy_version 68858 (0.0006) [2024-09-30 01:35:47,151][1157819] Updated weights for policy 0, policy_version 68868 (0.0006) [2024-09-30 01:35:47,694][1157819] Updated weights for policy 0, policy_version 68878 (0.0006) [2024-09-30 01:35:48,220][1157819] Updated weights for policy 0, policy_version 68888 (0.0006) [2024-09-30 01:35:48,823][1157819] Updated weights for policy 0, policy_version 68898 (0.0006) [2024-09-30 01:35:49,391][1157819] Updated weights for policy 0, policy_version 68908 (0.0006) [2024-09-30 01:35:49,994][1157819] Updated weights for policy 0, policy_version 68918 (0.0006) [2024-09-30 01:35:50,466][1157520] Fps is (10 sec: 77005.0, 60 sec: 84104.5, 300 sec: 80767.6). Total num frames: 282320896. Throughput: 0: 21004.4. Samples: 60568704. Policy #0 lag: (min: 0.0, avg: 2.8, max: 7.0) [2024-09-30 01:35:50,466][1157520] Avg episode reward: [(0, '51.242')] [2024-09-30 01:35:50,559][1157819] Updated weights for policy 0, policy_version 68928 (0.0006) [2024-09-30 01:35:51,209][1157819] Updated weights for policy 0, policy_version 68938 (0.0007) [2024-09-30 01:35:51,839][1157819] Updated weights for policy 0, policy_version 68948 (0.0006) [2024-09-30 01:35:52,488][1157819] Updated weights for policy 0, policy_version 68958 (0.0006) [2024-09-30 01:35:53,096][1157819] Updated weights for policy 0, policy_version 68968 (0.0006) [2024-09-30 01:35:53,739][1157819] Updated weights for policy 0, policy_version 68978 (0.0006) [2024-09-30 01:35:54,341][1157819] Updated weights for policy 0, policy_version 68988 (0.0006) [2024-09-30 01:35:54,965][1157819] Updated weights for policy 0, policy_version 68998 (0.0006) [2024-09-30 01:35:55,466][1157520] Fps is (10 sec: 69222.7, 60 sec: 82739.2, 300 sec: 80725.9). Total num frames: 282648576. Throughput: 0: 20662.5. Samples: 60618196. Policy #0 lag: (min: 0.0, avg: 2.8, max: 7.0) [2024-09-30 01:35:55,466][1157520] Avg episode reward: [(0, '53.100')] [2024-09-30 01:35:55,590][1157819] Updated weights for policy 0, policy_version 69008 (0.0006) [2024-09-30 01:35:56,196][1157819] Updated weights for policy 0, policy_version 69018 (0.0006) [2024-09-30 01:35:56,835][1157819] Updated weights for policy 0, policy_version 69028 (0.0006) [2024-09-30 01:35:57,442][1157819] Updated weights for policy 0, policy_version 69038 (0.0006) [2024-09-30 01:35:58,036][1157819] Updated weights for policy 0, policy_version 69048 (0.0006) [2024-09-30 01:35:58,607][1157819] Updated weights for policy 0, policy_version 69058 (0.0006) [2024-09-30 01:35:59,154][1157819] Updated weights for policy 0, policy_version 69068 (0.0006) [2024-09-30 01:35:59,668][1157819] Updated weights for policy 0, policy_version 69078 (0.0006) [2024-09-30 01:36:00,257][1157819] Updated weights for policy 0, policy_version 69088 (0.0006) [2024-09-30 01:36:00,466][1157520] Fps is (10 sec: 67993.6, 60 sec: 81442.1, 300 sec: 80559.3). Total num frames: 283000832. Throughput: 0: 20019.2. Samples: 60718860. Policy #0 lag: (min: 0.0, avg: 2.8, max: 7.0) [2024-09-30 01:36:00,466][1157520] Avg episode reward: [(0, '52.389')] [2024-09-30 01:36:00,782][1157819] Updated weights for policy 0, policy_version 69098 (0.0006) [2024-09-30 01:36:01,297][1157819] Updated weights for policy 0, policy_version 69108 (0.0006) [2024-09-30 01:36:01,865][1157819] Updated weights for policy 0, policy_version 69118 (0.0006) [2024-09-30 01:36:02,402][1157819] Updated weights for policy 0, policy_version 69128 (0.0006) [2024-09-30 01:36:02,922][1157819] Updated weights for policy 0, policy_version 69138 (0.0006) [2024-09-30 01:36:03,472][1157819] Updated weights for policy 0, policy_version 69148 (0.0006) [2024-09-30 01:36:04,024][1157819] Updated weights for policy 0, policy_version 69158 (0.0006) [2024-09-30 01:36:04,567][1157819] Updated weights for policy 0, policy_version 69168 (0.0006) [2024-09-30 01:36:05,128][1157819] Updated weights for policy 0, policy_version 69178 (0.0006) [2024-09-30 01:36:05,466][1157520] Fps is (10 sec: 72908.8, 60 sec: 80418.1, 300 sec: 80573.2). Total num frames: 283377664. Throughput: 0: 19697.4. Samples: 60832176. Policy #0 lag: (min: 0.0, avg: 2.8, max: 7.0) [2024-09-30 01:36:05,466][1157520] Avg episode reward: [(0, '53.954')] [2024-09-30 01:36:05,488][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000069185_283381760.pth... [2024-09-30 01:36:05,545][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000064453_263999488.pth [2024-09-30 01:36:05,634][1157819] Updated weights for policy 0, policy_version 69188 (0.0006) [2024-09-30 01:36:06,167][1157819] Updated weights for policy 0, policy_version 69198 (0.0006) [2024-09-30 01:36:06,773][1157819] Updated weights for policy 0, policy_version 69208 (0.0007) [2024-09-30 01:36:07,302][1157819] Updated weights for policy 0, policy_version 69218 (0.0006) [2024-09-30 01:36:07,891][1157819] Updated weights for policy 0, policy_version 69228 (0.0006) [2024-09-30 01:36:08,492][1157819] Updated weights for policy 0, policy_version 69238 (0.0006) [2024-09-30 01:36:09,059][1157819] Updated weights for policy 0, policy_version 69248 (0.0006) [2024-09-30 01:36:09,661][1157819] Updated weights for policy 0, policy_version 69258 (0.0006) [2024-09-30 01:36:10,225][1157819] Updated weights for policy 0, policy_version 69268 (0.0006) [2024-09-30 01:36:10,466][1157520] Fps is (10 sec: 73728.5, 60 sec: 79257.6, 300 sec: 80448.2). Total num frames: 283738112. Throughput: 0: 19513.9. Samples: 60887588. Policy #0 lag: (min: 0.0, avg: 2.8, max: 7.0) [2024-09-30 01:36:10,466][1157520] Avg episode reward: [(0, '52.312')] [2024-09-30 01:36:10,778][1157819] Updated weights for policy 0, policy_version 69278 (0.0006) [2024-09-30 01:36:11,317][1157819] Updated weights for policy 0, policy_version 69288 (0.0006) [2024-09-30 01:36:11,934][1157819] Updated weights for policy 0, policy_version 69298 (0.0007) [2024-09-30 01:36:12,489][1157819] Updated weights for policy 0, policy_version 69308 (0.0006) [2024-09-30 01:36:13,086][1157819] Updated weights for policy 0, policy_version 69318 (0.0006) [2024-09-30 01:36:13,648][1157819] Updated weights for policy 0, policy_version 69328 (0.0006) [2024-09-30 01:36:13,907][1157736] Signal inference workers to stop experience collection... (4450 times) [2024-09-30 01:36:13,908][1157736] Signal inference workers to resume experience collection... (4450 times) [2024-09-30 01:36:13,911][1157819] InferenceWorker_p0-w0: stopping experience collection (4450 times) [2024-09-30 01:36:13,911][1157819] InferenceWorker_p0-w0: resuming experience collection (4450 times) [2024-09-30 01:36:14,209][1157819] Updated weights for policy 0, policy_version 69338 (0.0006) [2024-09-30 01:36:14,771][1157819] Updated weights for policy 0, policy_version 69348 (0.0006) [2024-09-30 01:36:15,337][1157819] Updated weights for policy 0, policy_version 69358 (0.0006) [2024-09-30 01:36:15,466][1157520] Fps is (10 sec: 72089.5, 60 sec: 77960.5, 300 sec: 80295.5). Total num frames: 284098560. Throughput: 0: 19068.4. Samples: 60994256. Policy #0 lag: (min: 0.0, avg: 2.8, max: 7.0) [2024-09-30 01:36:15,466][1157520] Avg episode reward: [(0, '54.237')] [2024-09-30 01:36:15,925][1157819] Updated weights for policy 0, policy_version 69368 (0.0006) [2024-09-30 01:36:16,449][1157819] Updated weights for policy 0, policy_version 69378 (0.0006) [2024-09-30 01:36:17,002][1157819] Updated weights for policy 0, policy_version 69388 (0.0006) [2024-09-30 01:36:17,570][1157819] Updated weights for policy 0, policy_version 69398 (0.0006) [2024-09-30 01:36:18,151][1157819] Updated weights for policy 0, policy_version 69408 (0.0006) [2024-09-30 01:36:18,725][1157819] Updated weights for policy 0, policy_version 69418 (0.0006) [2024-09-30 01:36:19,348][1157819] Updated weights for policy 0, policy_version 69428 (0.0006) [2024-09-30 01:36:19,914][1157819] Updated weights for policy 0, policy_version 69438 (0.0006) [2024-09-30 01:36:20,466][1157520] Fps is (10 sec: 71270.0, 60 sec: 76868.3, 300 sec: 80184.4). Total num frames: 284450816. Throughput: 0: 18651.6. Samples: 61102384. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 01:36:20,466][1157520] Avg episode reward: [(0, '53.359')] [2024-09-30 01:36:20,563][1157819] Updated weights for policy 0, policy_version 69448 (0.0006) [2024-09-30 01:36:21,181][1157819] Updated weights for policy 0, policy_version 69458 (0.0007) [2024-09-30 01:36:21,776][1157819] Updated weights for policy 0, policy_version 69468 (0.0005) [2024-09-30 01:36:22,428][1157819] Updated weights for policy 0, policy_version 69478 (0.0006) [2024-09-30 01:36:23,033][1157819] Updated weights for policy 0, policy_version 69488 (0.0006) [2024-09-30 01:36:23,698][1157819] Updated weights for policy 0, policy_version 69498 (0.0006) [2024-09-30 01:36:24,323][1157819] Updated weights for policy 0, policy_version 69508 (0.0006) [2024-09-30 01:36:24,941][1157819] Updated weights for policy 0, policy_version 69518 (0.0006) [2024-09-30 01:36:25,466][1157520] Fps is (10 sec: 67584.1, 60 sec: 75229.9, 300 sec: 80003.9). Total num frames: 284774400. Throughput: 0: 18270.7. Samples: 61151388. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 01:36:25,466][1157520] Avg episode reward: [(0, '51.654')] [2024-09-30 01:36:25,584][1157819] Updated weights for policy 0, policy_version 69528 (0.0006) [2024-09-30 01:36:26,251][1157819] Updated weights for policy 0, policy_version 69538 (0.0006) [2024-09-30 01:36:26,858][1157819] Updated weights for policy 0, policy_version 69548 (0.0006) [2024-09-30 01:36:27,510][1157819] Updated weights for policy 0, policy_version 69558 (0.0006) [2024-09-30 01:36:28,085][1157819] Updated weights for policy 0, policy_version 69568 (0.0006) [2024-09-30 01:36:28,676][1157819] Updated weights for policy 0, policy_version 69578 (0.0006) [2024-09-30 01:36:29,278][1157819] Updated weights for policy 0, policy_version 69588 (0.0006) [2024-09-30 01:36:29,871][1157819] Updated weights for policy 0, policy_version 69598 (0.0006) [2024-09-30 01:36:30,452][1157819] Updated weights for policy 0, policy_version 69608 (0.0006) [2024-09-30 01:36:30,466][1157520] Fps is (10 sec: 66355.5, 60 sec: 73932.8, 300 sec: 79767.9). Total num frames: 285114368. Throughput: 0: 17556.3. Samples: 61250036. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 01:36:30,466][1157520] Avg episode reward: [(0, '50.592')] [2024-09-30 01:36:31,041][1157819] Updated weights for policy 0, policy_version 69618 (0.0006) [2024-09-30 01:36:31,614][1157819] Updated weights for policy 0, policy_version 69628 (0.0006) [2024-09-30 01:36:32,215][1157819] Updated weights for policy 0, policy_version 69638 (0.0006) [2024-09-30 01:36:32,814][1157819] Updated weights for policy 0, policy_version 69648 (0.0006) [2024-09-30 01:36:33,399][1157819] Updated weights for policy 0, policy_version 69658 (0.0006) [2024-09-30 01:36:33,980][1157819] Updated weights for policy 0, policy_version 69668 (0.0006) [2024-09-30 01:36:34,587][1157819] Updated weights for policy 0, policy_version 69678 (0.0006) [2024-09-30 01:36:35,171][1157819] Updated weights for policy 0, policy_version 69688 (0.0006) [2024-09-30 01:36:35,466][1157520] Fps is (10 sec: 68812.9, 60 sec: 72567.5, 300 sec: 79504.0). Total num frames: 285462528. Throughput: 0: 17447.7. Samples: 61353848. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 01:36:35,466][1157520] Avg episode reward: [(0, '52.362')] [2024-09-30 01:36:35,766][1157819] Updated weights for policy 0, policy_version 69698 (0.0006) [2024-09-30 01:36:36,338][1157819] Updated weights for policy 0, policy_version 69708 (0.0006) [2024-09-30 01:36:36,973][1157819] Updated weights for policy 0, policy_version 69718 (0.0006) [2024-09-30 01:36:37,568][1157819] Updated weights for policy 0, policy_version 69728 (0.0006) [2024-09-30 01:36:38,160][1157819] Updated weights for policy 0, policy_version 69738 (0.0006) [2024-09-30 01:36:38,721][1157819] Updated weights for policy 0, policy_version 69748 (0.0006) [2024-09-30 01:36:39,265][1157819] Updated weights for policy 0, policy_version 69758 (0.0006) [2024-09-30 01:36:39,809][1157819] Updated weights for policy 0, policy_version 69768 (0.0006) [2024-09-30 01:36:40,336][1157819] Updated weights for policy 0, policy_version 69778 (0.0006) [2024-09-30 01:36:40,466][1157520] Fps is (10 sec: 70041.7, 60 sec: 71065.7, 300 sec: 79309.7). Total num frames: 285814784. Throughput: 0: 17487.5. Samples: 61405132. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 01:36:40,466][1157520] Avg episode reward: [(0, '52.961')] [2024-09-30 01:36:40,881][1157819] Updated weights for policy 0, policy_version 69788 (0.0006) [2024-09-30 01:36:41,413][1157819] Updated weights for policy 0, policy_version 69798 (0.0006) [2024-09-30 01:36:41,961][1157819] Updated weights for policy 0, policy_version 69808 (0.0006) [2024-09-30 01:36:42,515][1157819] Updated weights for policy 0, policy_version 69818 (0.0006) [2024-09-30 01:36:43,067][1157819] Updated weights for policy 0, policy_version 69828 (0.0007) [2024-09-30 01:36:43,655][1157819] Updated weights for policy 0, policy_version 69838 (0.0006) [2024-09-30 01:36:44,165][1157819] Updated weights for policy 0, policy_version 69848 (0.0006) [2024-09-30 01:36:44,712][1157819] Updated weights for policy 0, policy_version 69858 (0.0006) [2024-09-30 01:36:45,290][1157819] Updated weights for policy 0, policy_version 69868 (0.0006) [2024-09-30 01:36:45,466][1157520] Fps is (10 sec: 72908.6, 60 sec: 70587.8, 300 sec: 79268.0). Total num frames: 286191616. Throughput: 0: 17745.4. Samples: 61517404. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 01:36:45,466][1157520] Avg episode reward: [(0, '51.449')] [2024-09-30 01:36:45,825][1157819] Updated weights for policy 0, policy_version 69878 (0.0006) [2024-09-30 01:36:46,367][1157819] Updated weights for policy 0, policy_version 69888 (0.0006) [2024-09-30 01:36:46,913][1157819] Updated weights for policy 0, policy_version 69898 (0.0006) [2024-09-30 01:36:47,462][1157819] Updated weights for policy 0, policy_version 69908 (0.0006) [2024-09-30 01:36:48,007][1157819] Updated weights for policy 0, policy_version 69918 (0.0006) [2024-09-30 01:36:48,578][1157819] Updated weights for policy 0, policy_version 69928 (0.0006) [2024-09-30 01:36:49,091][1157819] Updated weights for policy 0, policy_version 69938 (0.0006) [2024-09-30 01:36:49,617][1157819] Updated weights for policy 0, policy_version 69948 (0.0006) [2024-09-30 01:36:50,196][1157819] Updated weights for policy 0, policy_version 69958 (0.0006) [2024-09-30 01:36:50,466][1157520] Fps is (10 sec: 75366.4, 60 sec: 70792.6, 300 sec: 79212.5). Total num frames: 286568448. Throughput: 0: 17721.7. Samples: 61629652. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 01:36:50,466][1157520] Avg episode reward: [(0, '54.119')] [2024-09-30 01:36:50,713][1157819] Updated weights for policy 0, policy_version 69968 (0.0006) [2024-09-30 01:36:51,240][1157819] Updated weights for policy 0, policy_version 69978 (0.0006) [2024-09-30 01:36:51,807][1157819] Updated weights for policy 0, policy_version 69988 (0.0006) [2024-09-30 01:36:52,353][1157819] Updated weights for policy 0, policy_version 69998 (0.0006) [2024-09-30 01:36:52,895][1157819] Updated weights for policy 0, policy_version 70008 (0.0006) [2024-09-30 01:36:53,420][1157819] Updated weights for policy 0, policy_version 70018 (0.0006) [2024-09-30 01:36:53,922][1157819] Updated weights for policy 0, policy_version 70028 (0.0006) [2024-09-30 01:36:54,441][1157819] Updated weights for policy 0, policy_version 70038 (0.0006) [2024-09-30 01:36:54,985][1157819] Updated weights for policy 0, policy_version 70048 (0.0006) [2024-09-30 01:36:55,466][1157520] Fps is (10 sec: 76185.1, 60 sec: 71748.2, 300 sec: 79156.9). Total num frames: 286953472. Throughput: 0: 17759.7. Samples: 61686776. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 01:36:55,466][1157520] Avg episode reward: [(0, '53.156')] [2024-09-30 01:36:55,521][1157819] Updated weights for policy 0, policy_version 70058 (0.0006) [2024-09-30 01:36:56,043][1157819] Updated weights for policy 0, policy_version 70068 (0.0006) [2024-09-30 01:36:56,584][1157819] Updated weights for policy 0, policy_version 70078 (0.0006) [2024-09-30 01:36:57,113][1157736] Signal inference workers to stop experience collection... (4500 times) [2024-09-30 01:36:57,117][1157736] Signal inference workers to resume experience collection... (4500 times) [2024-09-30 01:36:57,118][1157819] InferenceWorker_p0-w0: stopping experience collection (4500 times) [2024-09-30 01:36:57,119][1157819] Updated weights for policy 0, policy_version 70088 (0.0006) [2024-09-30 01:36:57,121][1157819] InferenceWorker_p0-w0: resuming experience collection (4500 times) [2024-09-30 01:36:57,649][1157819] Updated weights for policy 0, policy_version 70098 (0.0006) [2024-09-30 01:36:58,204][1157819] Updated weights for policy 0, policy_version 70108 (0.0006) [2024-09-30 01:36:58,757][1157819] Updated weights for policy 0, policy_version 70118 (0.0006) [2024-09-30 01:36:59,267][1157819] Updated weights for policy 0, policy_version 70128 (0.0006) [2024-09-30 01:36:59,840][1157819] Updated weights for policy 0, policy_version 70138 (0.0006) [2024-09-30 01:37:00,342][1157819] Updated weights for policy 0, policy_version 70148 (0.0006) [2024-09-30 01:37:00,466][1157520] Fps is (10 sec: 76594.3, 60 sec: 72226.1, 300 sec: 79101.4). Total num frames: 287334400. Throughput: 0: 17943.5. Samples: 61801716. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 01:37:00,466][1157520] Avg episode reward: [(0, '53.952')] [2024-09-30 01:37:00,838][1157819] Updated weights for policy 0, policy_version 70158 (0.0006) [2024-09-30 01:37:01,336][1157819] Updated weights for policy 0, policy_version 70168 (0.0006) [2024-09-30 01:37:01,848][1157819] Updated weights for policy 0, policy_version 70178 (0.0006) [2024-09-30 01:37:02,394][1157819] Updated weights for policy 0, policy_version 70188 (0.0006) [2024-09-30 01:37:02,900][1157819] Updated weights for policy 0, policy_version 70198 (0.0006) [2024-09-30 01:37:03,412][1157819] Updated weights for policy 0, policy_version 70208 (0.0006) [2024-09-30 01:37:03,968][1157819] Updated weights for policy 0, policy_version 70218 (0.0006) [2024-09-30 01:37:04,510][1157819] Updated weights for policy 0, policy_version 70228 (0.0006) [2024-09-30 01:37:05,077][1157819] Updated weights for policy 0, policy_version 70238 (0.0006) [2024-09-30 01:37:05,466][1157520] Fps is (10 sec: 76594.2, 60 sec: 72362.4, 300 sec: 79129.1). Total num frames: 287719424. Throughput: 0: 18137.5. Samples: 61918576. Policy #0 lag: (min: 0.0, avg: 2.5, max: 5.0) [2024-09-30 01:37:05,466][1157520] Avg episode reward: [(0, '54.636')] [2024-09-30 01:37:05,652][1157819] Updated weights for policy 0, policy_version 70248 (0.0006) [2024-09-30 01:37:06,225][1157819] Updated weights for policy 0, policy_version 70258 (0.0006) [2024-09-30 01:37:06,782][1157819] Updated weights for policy 0, policy_version 70268 (0.0006) [2024-09-30 01:37:07,361][1157819] Updated weights for policy 0, policy_version 70278 (0.0006) [2024-09-30 01:37:07,905][1157819] Updated weights for policy 0, policy_version 70288 (0.0006) [2024-09-30 01:37:08,460][1157819] Updated weights for policy 0, policy_version 70298 (0.0006) [2024-09-30 01:37:09,032][1157819] Updated weights for policy 0, policy_version 70308 (0.0006) [2024-09-30 01:37:09,605][1157819] Updated weights for policy 0, policy_version 70318 (0.0006) [2024-09-30 01:37:10,160][1157819] Updated weights for policy 0, policy_version 70328 (0.0006) [2024-09-30 01:37:10,466][1157520] Fps is (10 sec: 74956.8, 60 sec: 72430.8, 300 sec: 79073.6). Total num frames: 288083968. Throughput: 0: 18253.8. Samples: 61972812. Policy #0 lag: (min: 0.0, avg: 2.5, max: 5.0) [2024-09-30 01:37:10,466][1157520] Avg episode reward: [(0, '52.342')] [2024-09-30 01:37:10,656][1157819] Updated weights for policy 0, policy_version 70338 (0.0006) [2024-09-30 01:37:11,215][1157819] Updated weights for policy 0, policy_version 70348 (0.0006) [2024-09-30 01:37:11,744][1157819] Updated weights for policy 0, policy_version 70358 (0.0006) [2024-09-30 01:37:12,287][1157819] Updated weights for policy 0, policy_version 70368 (0.0006) [2024-09-30 01:37:12,828][1157819] Updated weights for policy 0, policy_version 70378 (0.0006) [2024-09-30 01:37:13,328][1157819] Updated weights for policy 0, policy_version 70388 (0.0006) [2024-09-30 01:37:13,839][1157819] Updated weights for policy 0, policy_version 70398 (0.0006) [2024-09-30 01:37:14,413][1157819] Updated weights for policy 0, policy_version 70408 (0.0006) [2024-09-30 01:37:14,933][1157819] Updated weights for policy 0, policy_version 70418 (0.0006) [2024-09-30 01:37:15,466][1157520] Fps is (10 sec: 74957.7, 60 sec: 72840.4, 300 sec: 79031.9). Total num frames: 288468992. Throughput: 0: 18577.1. Samples: 62086008. Policy #0 lag: (min: 0.0, avg: 2.5, max: 5.0) [2024-09-30 01:37:15,466][1157520] Avg episode reward: [(0, '53.448')] [2024-09-30 01:37:15,484][1157819] Updated weights for policy 0, policy_version 70428 (0.0006) [2024-09-30 01:37:16,011][1157819] Updated weights for policy 0, policy_version 70438 (0.0006) [2024-09-30 01:37:16,555][1157819] Updated weights for policy 0, policy_version 70448 (0.0006) [2024-09-30 01:37:17,075][1157819] Updated weights for policy 0, policy_version 70458 (0.0006) [2024-09-30 01:37:17,637][1157819] Updated weights for policy 0, policy_version 70468 (0.0006) [2024-09-30 01:37:18,167][1157819] Updated weights for policy 0, policy_version 70478 (0.0006) [2024-09-30 01:37:18,705][1157819] Updated weights for policy 0, policy_version 70488 (0.0006) [2024-09-30 01:37:19,229][1157819] Updated weights for policy 0, policy_version 70498 (0.0006) [2024-09-30 01:37:19,766][1157819] Updated weights for policy 0, policy_version 70508 (0.0006) [2024-09-30 01:37:20,329][1157819] Updated weights for policy 0, policy_version 70518 (0.0006) [2024-09-30 01:37:20,466][1157520] Fps is (10 sec: 76594.6, 60 sec: 73318.2, 300 sec: 78920.9). Total num frames: 288849920. Throughput: 0: 18817.4. Samples: 62200636. Policy #0 lag: (min: 0.0, avg: 2.5, max: 5.0) [2024-09-30 01:37:20,466][1157520] Avg episode reward: [(0, '52.995')] [2024-09-30 01:37:20,843][1157819] Updated weights for policy 0, policy_version 70528 (0.0006) [2024-09-30 01:37:21,369][1157819] Updated weights for policy 0, policy_version 70538 (0.0006) [2024-09-30 01:37:21,919][1157819] Updated weights for policy 0, policy_version 70548 (0.0006) [2024-09-30 01:37:22,444][1157819] Updated weights for policy 0, policy_version 70558 (0.0006) [2024-09-30 01:37:22,989][1157819] Updated weights for policy 0, policy_version 70568 (0.0006) [2024-09-30 01:37:23,537][1157819] Updated weights for policy 0, policy_version 70578 (0.0006) [2024-09-30 01:37:24,058][1157819] Updated weights for policy 0, policy_version 70588 (0.0006) [2024-09-30 01:37:24,613][1157819] Updated weights for policy 0, policy_version 70598 (0.0006) [2024-09-30 01:37:25,164][1157819] Updated weights for policy 0, policy_version 70608 (0.0006) [2024-09-30 01:37:25,466][1157520] Fps is (10 sec: 76185.1, 60 sec: 74273.9, 300 sec: 78795.9). Total num frames: 289230848. Throughput: 0: 18944.8. Samples: 62257652. Policy #0 lag: (min: 0.0, avg: 2.5, max: 5.0) [2024-09-30 01:37:25,466][1157520] Avg episode reward: [(0, '53.952')] [2024-09-30 01:37:25,687][1157819] Updated weights for policy 0, policy_version 70618 (0.0006) [2024-09-30 01:37:26,195][1157819] Updated weights for policy 0, policy_version 70628 (0.0006) [2024-09-30 01:37:26,695][1157819] Updated weights for policy 0, policy_version 70638 (0.0006) [2024-09-30 01:37:27,188][1157819] Updated weights for policy 0, policy_version 70648 (0.0006) [2024-09-30 01:37:27,678][1157819] Updated weights for policy 0, policy_version 70658 (0.0006) [2024-09-30 01:37:28,182][1157819] Updated weights for policy 0, policy_version 70668 (0.0006) [2024-09-30 01:37:28,692][1157819] Updated weights for policy 0, policy_version 70678 (0.0006) [2024-09-30 01:37:29,230][1157819] Updated weights for policy 0, policy_version 70688 (0.0006) [2024-09-30 01:37:29,728][1157819] Updated weights for policy 0, policy_version 70698 (0.0006) [2024-09-30 01:37:30,219][1157819] Updated weights for policy 0, policy_version 70708 (0.0006) [2024-09-30 01:37:30,466][1157520] Fps is (10 sec: 78644.7, 60 sec: 75366.4, 300 sec: 78712.6). Total num frames: 289636352. Throughput: 0: 19079.2. Samples: 62375968. Policy #0 lag: (min: 0.0, avg: 2.5, max: 5.0) [2024-09-30 01:37:30,466][1157520] Avg episode reward: [(0, '54.157')] [2024-09-30 01:37:30,718][1157819] Updated weights for policy 0, policy_version 70718 (0.0006) [2024-09-30 01:37:30,865][1157736] Signal inference workers to stop experience collection... (4550 times) [2024-09-30 01:37:30,865][1157736] Signal inference workers to resume experience collection... (4550 times) [2024-09-30 01:37:30,869][1157819] InferenceWorker_p0-w0: stopping experience collection (4550 times) [2024-09-30 01:37:30,869][1157819] InferenceWorker_p0-w0: resuming experience collection (4550 times) [2024-09-30 01:37:31,234][1157819] Updated weights for policy 0, policy_version 70728 (0.0006) [2024-09-30 01:37:31,735][1157819] Updated weights for policy 0, policy_version 70738 (0.0006) [2024-09-30 01:37:32,279][1157819] Updated weights for policy 0, policy_version 70748 (0.0006) [2024-09-30 01:37:32,794][1157819] Updated weights for policy 0, policy_version 70758 (0.0006) [2024-09-30 01:37:33,301][1157819] Updated weights for policy 0, policy_version 70768 (0.0006) [2024-09-30 01:37:33,811][1157819] Updated weights for policy 0, policy_version 70778 (0.0006) [2024-09-30 01:37:34,312][1157819] Updated weights for policy 0, policy_version 70788 (0.0006) [2024-09-30 01:37:34,827][1157819] Updated weights for policy 0, policy_version 70798 (0.0006) [2024-09-30 01:37:35,342][1157819] Updated weights for policy 0, policy_version 70808 (0.0006) [2024-09-30 01:37:35,466][1157520] Fps is (10 sec: 80692.5, 60 sec: 76253.9, 300 sec: 78643.2). Total num frames: 290037760. Throughput: 0: 19260.6. Samples: 62496380. Policy #0 lag: (min: 0.0, avg: 2.5, max: 5.0) [2024-09-30 01:37:35,466][1157520] Avg episode reward: [(0, '50.621')] [2024-09-30 01:37:35,847][1157819] Updated weights for policy 0, policy_version 70818 (0.0006) [2024-09-30 01:37:36,364][1157819] Updated weights for policy 0, policy_version 70828 (0.0006) [2024-09-30 01:37:36,868][1157819] Updated weights for policy 0, policy_version 70838 (0.0006) [2024-09-30 01:37:37,373][1157819] Updated weights for policy 0, policy_version 70848 (0.0006) [2024-09-30 01:37:37,888][1157819] Updated weights for policy 0, policy_version 70858 (0.0006) [2024-09-30 01:37:38,385][1157819] Updated weights for policy 0, policy_version 70868 (0.0006) [2024-09-30 01:37:38,940][1157819] Updated weights for policy 0, policy_version 70878 (0.0006) [2024-09-30 01:37:39,432][1157819] Updated weights for policy 0, policy_version 70888 (0.0006) [2024-09-30 01:37:39,957][1157819] Updated weights for policy 0, policy_version 70898 (0.0006) [2024-09-30 01:37:40,434][1157819] Updated weights for policy 0, policy_version 70908 (0.0006) [2024-09-30 01:37:40,466][1157520] Fps is (10 sec: 80281.2, 60 sec: 77073.0, 300 sec: 78615.4). Total num frames: 290439168. Throughput: 0: 19329.3. Samples: 62556592. Policy #0 lag: (min: 0.0, avg: 2.5, max: 5.0) [2024-09-30 01:37:40,466][1157520] Avg episode reward: [(0, '51.944')] [2024-09-30 01:37:40,945][1157819] Updated weights for policy 0, policy_version 70918 (0.0006) [2024-09-30 01:37:41,439][1157819] Updated weights for policy 0, policy_version 70928 (0.0006) [2024-09-30 01:37:41,944][1157819] Updated weights for policy 0, policy_version 70938 (0.0006) [2024-09-30 01:37:42,437][1157819] Updated weights for policy 0, policy_version 70948 (0.0006) [2024-09-30 01:37:42,931][1157819] Updated weights for policy 0, policy_version 70958 (0.0006) [2024-09-30 01:37:43,425][1157819] Updated weights for policy 0, policy_version 70968 (0.0006) [2024-09-30 01:37:43,964][1157819] Updated weights for policy 0, policy_version 70978 (0.0006) [2024-09-30 01:37:44,453][1157819] Updated weights for policy 0, policy_version 70988 (0.0006) [2024-09-30 01:37:44,942][1157819] Updated weights for policy 0, policy_version 70998 (0.0006) [2024-09-30 01:37:45,448][1157819] Updated weights for policy 0, policy_version 71008 (0.0006) [2024-09-30 01:37:45,466][1157520] Fps is (10 sec: 81100.5, 60 sec: 77619.2, 300 sec: 78601.5). Total num frames: 290848768. Throughput: 0: 19476.6. Samples: 62678160. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 01:37:45,466][1157520] Avg episode reward: [(0, '55.117')] [2024-09-30 01:37:45,986][1157819] Updated weights for policy 0, policy_version 71018 (0.0006) [2024-09-30 01:37:46,476][1157819] Updated weights for policy 0, policy_version 71028 (0.0006) [2024-09-30 01:37:46,979][1157819] Updated weights for policy 0, policy_version 71038 (0.0006) [2024-09-30 01:37:47,495][1157819] Updated weights for policy 0, policy_version 71048 (0.0006) [2024-09-30 01:37:47,988][1157819] Updated weights for policy 0, policy_version 71058 (0.0006) [2024-09-30 01:37:48,493][1157819] Updated weights for policy 0, policy_version 71068 (0.0006) [2024-09-30 01:37:48,982][1157819] Updated weights for policy 0, policy_version 71078 (0.0006) [2024-09-30 01:37:49,491][1157819] Updated weights for policy 0, policy_version 71088 (0.0006) [2024-09-30 01:37:50,032][1157819] Updated weights for policy 0, policy_version 71098 (0.0006) [2024-09-30 01:37:50,466][1157520] Fps is (10 sec: 81100.7, 60 sec: 78028.7, 300 sec: 78629.3). Total num frames: 291250176. Throughput: 0: 19574.7. Samples: 62799436. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 01:37:50,466][1157520] Avg episode reward: [(0, '52.556')] [2024-09-30 01:37:50,555][1157819] Updated weights for policy 0, policy_version 71108 (0.0006) [2024-09-30 01:37:51,052][1157819] Updated weights for policy 0, policy_version 71118 (0.0006) [2024-09-30 01:37:51,556][1157819] Updated weights for policy 0, policy_version 71128 (0.0006) [2024-09-30 01:37:52,087][1157819] Updated weights for policy 0, policy_version 71138 (0.0006) [2024-09-30 01:37:52,591][1157819] Updated weights for policy 0, policy_version 71148 (0.0006) [2024-09-30 01:37:53,111][1157819] Updated weights for policy 0, policy_version 71158 (0.0006) [2024-09-30 01:37:53,608][1157819] Updated weights for policy 0, policy_version 71168 (0.0006) [2024-09-30 01:37:54,134][1157819] Updated weights for policy 0, policy_version 71178 (0.0006) [2024-09-30 01:37:54,662][1157819] Updated weights for policy 0, policy_version 71188 (0.0006) [2024-09-30 01:37:55,156][1157819] Updated weights for policy 0, policy_version 71198 (0.0006) [2024-09-30 01:37:55,466][1157520] Fps is (10 sec: 79872.3, 60 sec: 78233.7, 300 sec: 78684.9). Total num frames: 291647488. Throughput: 0: 19699.1. Samples: 62859272. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 01:37:55,466][1157520] Avg episode reward: [(0, '53.873')] [2024-09-30 01:37:55,714][1157819] Updated weights for policy 0, policy_version 71208 (0.0006) [2024-09-30 01:37:56,200][1157819] Updated weights for policy 0, policy_version 71218 (0.0006) [2024-09-30 01:37:56,711][1157819] Updated weights for policy 0, policy_version 71228 (0.0006) [2024-09-30 01:37:57,198][1157819] Updated weights for policy 0, policy_version 71238 (0.0006) [2024-09-30 01:37:57,745][1157819] Updated weights for policy 0, policy_version 71248 (0.0006) [2024-09-30 01:37:58,264][1157819] Updated weights for policy 0, policy_version 71258 (0.0006) [2024-09-30 01:37:58,769][1157819] Updated weights for policy 0, policy_version 71268 (0.0006) [2024-09-30 01:37:59,293][1157819] Updated weights for policy 0, policy_version 71278 (0.0006) [2024-09-30 01:37:59,782][1157819] Updated weights for policy 0, policy_version 71288 (0.0006) [2024-09-30 01:38:00,300][1157819] Updated weights for policy 0, policy_version 71298 (0.0006) [2024-09-30 01:38:00,466][1157520] Fps is (10 sec: 79872.5, 60 sec: 78575.1, 300 sec: 78726.5). Total num frames: 292048896. Throughput: 0: 19834.8. Samples: 62978572. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 01:38:00,466][1157520] Avg episode reward: [(0, '51.864')] [2024-09-30 01:38:00,798][1157819] Updated weights for policy 0, policy_version 71308 (0.0006) [2024-09-30 01:38:01,262][1157819] Updated weights for policy 0, policy_version 71318 (0.0006) [2024-09-30 01:38:01,792][1157819] Updated weights for policy 0, policy_version 71328 (0.0006) [2024-09-30 01:38:02,300][1157819] Updated weights for policy 0, policy_version 71338 (0.0006) [2024-09-30 01:38:02,801][1157819] Updated weights for policy 0, policy_version 71348 (0.0006) [2024-09-30 01:38:03,325][1157819] Updated weights for policy 0, policy_version 71358 (0.0006) [2024-09-30 01:38:03,828][1157819] Updated weights for policy 0, policy_version 71368 (0.0006) [2024-09-30 01:38:04,331][1157819] Updated weights for policy 0, policy_version 71378 (0.0006) [2024-09-30 01:38:04,846][1157819] Updated weights for policy 0, policy_version 71388 (0.0006) [2024-09-30 01:38:05,362][1157819] Updated weights for policy 0, policy_version 71398 (0.0006) [2024-09-30 01:38:05,466][1157520] Fps is (10 sec: 80691.1, 60 sec: 78916.6, 300 sec: 78782.1). Total num frames: 292454400. Throughput: 0: 19987.8. Samples: 63100084. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 01:38:05,466][1157520] Avg episode reward: [(0, '51.379')] [2024-09-30 01:38:05,469][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000071400_292454400.pth... [2024-09-30 01:38:05,515][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000066810_273653760.pth [2024-09-30 01:38:05,886][1157819] Updated weights for policy 0, policy_version 71408 (0.0006) [2024-09-30 01:38:06,397][1157819] Updated weights for policy 0, policy_version 71418 (0.0006) [2024-09-30 01:38:06,917][1157819] Updated weights for policy 0, policy_version 71428 (0.0006) [2024-09-30 01:38:07,466][1157819] Updated weights for policy 0, policy_version 71438 (0.0006) [2024-09-30 01:38:07,994][1157819] Updated weights for policy 0, policy_version 71448 (0.0006) [2024-09-30 01:38:08,506][1157819] Updated weights for policy 0, policy_version 71458 (0.0006) [2024-09-30 01:38:09,012][1157819] Updated weights for policy 0, policy_version 71468 (0.0006) [2024-09-30 01:38:09,546][1157819] Updated weights for policy 0, policy_version 71478 (0.0006) [2024-09-30 01:38:10,057][1157819] Updated weights for policy 0, policy_version 71488 (0.0006) [2024-09-30 01:38:10,466][1157520] Fps is (10 sec: 79872.1, 60 sec: 79394.3, 300 sec: 78740.4). Total num frames: 292847616. Throughput: 0: 20030.6. Samples: 63159024. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 01:38:10,466][1157520] Avg episode reward: [(0, '53.337')] [2024-09-30 01:38:10,600][1157819] Updated weights for policy 0, policy_version 71498 (0.0006) [2024-09-30 01:38:11,107][1157819] Updated weights for policy 0, policy_version 71508 (0.0006) [2024-09-30 01:38:11,618][1157819] Updated weights for policy 0, policy_version 71518 (0.0006) [2024-09-30 01:38:12,138][1157819] Updated weights for policy 0, policy_version 71528 (0.0006) [2024-09-30 01:38:12,657][1157819] Updated weights for policy 0, policy_version 71538 (0.0006) [2024-09-30 01:38:13,176][1157819] Updated weights for policy 0, policy_version 71548 (0.0006) [2024-09-30 01:38:13,685][1157819] Updated weights for policy 0, policy_version 71558 (0.0005) [2024-09-30 01:38:14,177][1157819] Updated weights for policy 0, policy_version 71568 (0.0006) [2024-09-30 01:38:14,709][1157819] Updated weights for policy 0, policy_version 71578 (0.0006) [2024-09-30 01:38:15,174][1157819] Updated weights for policy 0, policy_version 71588 (0.0006) [2024-09-30 01:38:15,466][1157520] Fps is (10 sec: 79052.7, 60 sec: 79599.1, 300 sec: 78754.3). Total num frames: 293244928. Throughput: 0: 20034.8. Samples: 63277536. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 01:38:15,466][1157520] Avg episode reward: [(0, '54.140')] [2024-09-30 01:38:15,669][1157819] Updated weights for policy 0, policy_version 71598 (0.0006) [2024-09-30 01:38:16,198][1157819] Updated weights for policy 0, policy_version 71608 (0.0006) [2024-09-30 01:38:16,673][1157819] Updated weights for policy 0, policy_version 71618 (0.0006) [2024-09-30 01:38:17,143][1157819] Updated weights for policy 0, policy_version 71628 (0.0006) [2024-09-30 01:38:17,362][1157736] Signal inference workers to stop experience collection... (4600 times) [2024-09-30 01:38:17,363][1157736] Signal inference workers to resume experience collection... (4600 times) [2024-09-30 01:38:17,367][1157819] InferenceWorker_p0-w0: stopping experience collection (4600 times) [2024-09-30 01:38:17,370][1157819] InferenceWorker_p0-w0: resuming experience collection (4600 times) [2024-09-30 01:38:17,642][1157819] Updated weights for policy 0, policy_version 71638 (0.0006) [2024-09-30 01:38:18,159][1157819] Updated weights for policy 0, policy_version 71648 (0.0006) [2024-09-30 01:38:18,641][1157819] Updated weights for policy 0, policy_version 71658 (0.0006) [2024-09-30 01:38:19,122][1157819] Updated weights for policy 0, policy_version 71668 (0.0006) [2024-09-30 01:38:19,623][1157819] Updated weights for policy 0, policy_version 71678 (0.0006) [2024-09-30 01:38:20,097][1157819] Updated weights for policy 0, policy_version 71688 (0.0006) [2024-09-30 01:38:20,466][1157520] Fps is (10 sec: 81510.1, 60 sec: 80213.6, 300 sec: 78907.0). Total num frames: 293662720. Throughput: 0: 20127.8. Samples: 63402132. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 01:38:20,466][1157520] Avg episode reward: [(0, '54.270')] [2024-09-30 01:38:20,588][1157819] Updated weights for policy 0, policy_version 71698 (0.0006) [2024-09-30 01:38:21,091][1157819] Updated weights for policy 0, policy_version 71708 (0.0006) [2024-09-30 01:38:21,574][1157819] Updated weights for policy 0, policy_version 71718 (0.0006) [2024-09-30 01:38:22,033][1157819] Updated weights for policy 0, policy_version 71728 (0.0006) [2024-09-30 01:38:22,534][1157819] Updated weights for policy 0, policy_version 71738 (0.0006) [2024-09-30 01:38:23,038][1157819] Updated weights for policy 0, policy_version 71748 (0.0006) [2024-09-30 01:38:23,514][1157819] Updated weights for policy 0, policy_version 71758 (0.0006) [2024-09-30 01:38:23,997][1157819] Updated weights for policy 0, policy_version 71768 (0.0006) [2024-09-30 01:38:24,505][1157819] Updated weights for policy 0, policy_version 71778 (0.0006) [2024-09-30 01:38:24,947][1157819] Updated weights for policy 0, policy_version 71788 (0.0006) [2024-09-30 01:38:25,411][1157819] Updated weights for policy 0, policy_version 71798 (0.0006) [2024-09-30 01:38:25,466][1157520] Fps is (10 sec: 84377.6, 60 sec: 80964.5, 300 sec: 79004.2). Total num frames: 294088704. Throughput: 0: 20189.3. Samples: 63465112. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 01:38:25,466][1157520] Avg episode reward: [(0, '55.177')] [2024-09-30 01:38:25,912][1157819] Updated weights for policy 0, policy_version 71808 (0.0006) [2024-09-30 01:38:26,340][1157819] Updated weights for policy 0, policy_version 71818 (0.0006) [2024-09-30 01:38:26,797][1157819] Updated weights for policy 0, policy_version 71828 (0.0006) [2024-09-30 01:38:27,282][1157819] Updated weights for policy 0, policy_version 71838 (0.0006) [2024-09-30 01:38:27,743][1157819] Updated weights for policy 0, policy_version 71848 (0.0006) [2024-09-30 01:38:28,201][1157819] Updated weights for policy 0, policy_version 71858 (0.0006) [2024-09-30 01:38:28,666][1157819] Updated weights for policy 0, policy_version 71868 (0.0006) [2024-09-30 01:38:29,112][1157819] Updated weights for policy 0, policy_version 71878 (0.0006) [2024-09-30 01:38:29,596][1157819] Updated weights for policy 0, policy_version 71888 (0.0006) [2024-09-30 01:38:30,076][1157819] Updated weights for policy 0, policy_version 71898 (0.0006) [2024-09-30 01:38:30,466][1157520] Fps is (10 sec: 86425.4, 60 sec: 81510.3, 300 sec: 79129.2). Total num frames: 294526976. Throughput: 0: 20385.5. Samples: 63595508. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:38:30,466][1157520] Avg episode reward: [(0, '51.911')] [2024-09-30 01:38:30,526][1157819] Updated weights for policy 0, policy_version 71908 (0.0006) [2024-09-30 01:38:31,002][1157819] Updated weights for policy 0, policy_version 71918 (0.0006) [2024-09-30 01:38:31,462][1157819] Updated weights for policy 0, policy_version 71928 (0.0006) [2024-09-30 01:38:31,930][1157819] Updated weights for policy 0, policy_version 71938 (0.0006) [2024-09-30 01:38:32,383][1157819] Updated weights for policy 0, policy_version 71948 (0.0006) [2024-09-30 01:38:32,832][1157819] Updated weights for policy 0, policy_version 71958 (0.0006) [2024-09-30 01:38:33,306][1157819] Updated weights for policy 0, policy_version 71968 (0.0006) [2024-09-30 01:38:33,778][1157819] Updated weights for policy 0, policy_version 71978 (0.0006) [2024-09-30 01:38:34,271][1157819] Updated weights for policy 0, policy_version 71988 (0.0006) [2024-09-30 01:38:34,692][1157819] Updated weights for policy 0, policy_version 71998 (0.0006) [2024-09-30 01:38:35,182][1157819] Updated weights for policy 0, policy_version 72008 (0.0006) [2024-09-30 01:38:35,466][1157520] Fps is (10 sec: 88064.0, 60 sec: 82193.1, 300 sec: 79212.5). Total num frames: 294969344. Throughput: 0: 20627.4. Samples: 63727668. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:38:35,466][1157520] Avg episode reward: [(0, '55.128')] [2024-09-30 01:38:35,641][1157819] Updated weights for policy 0, policy_version 72018 (0.0006) [2024-09-30 01:38:36,104][1157819] Updated weights for policy 0, policy_version 72028 (0.0006) [2024-09-30 01:38:36,581][1157819] Updated weights for policy 0, policy_version 72038 (0.0006) [2024-09-30 01:38:37,036][1157819] Updated weights for policy 0, policy_version 72048 (0.0006) [2024-09-30 01:38:37,491][1157819] Updated weights for policy 0, policy_version 72058 (0.0006) [2024-09-30 01:38:37,993][1157819] Updated weights for policy 0, policy_version 72068 (0.0006) [2024-09-30 01:38:38,481][1157819] Updated weights for policy 0, policy_version 72078 (0.0006) [2024-09-30 01:38:38,962][1157819] Updated weights for policy 0, policy_version 72088 (0.0006) [2024-09-30 01:38:39,441][1157819] Updated weights for policy 0, policy_version 72098 (0.0006) [2024-09-30 01:38:39,909][1157819] Updated weights for policy 0, policy_version 72108 (0.0006) [2024-09-30 01:38:40,367][1157819] Updated weights for policy 0, policy_version 72118 (0.0006) [2024-09-30 01:38:40,466][1157520] Fps is (10 sec: 87654.9, 60 sec: 82739.3, 300 sec: 79281.9). Total num frames: 295403520. Throughput: 0: 20756.0. Samples: 63793292. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:38:40,466][1157520] Avg episode reward: [(0, '52.776')] [2024-09-30 01:38:40,828][1157819] Updated weights for policy 0, policy_version 72128 (0.0006) [2024-09-30 01:38:41,311][1157819] Updated weights for policy 0, policy_version 72138 (0.0006) [2024-09-30 01:38:41,771][1157819] Updated weights for policy 0, policy_version 72148 (0.0006) [2024-09-30 01:38:42,225][1157819] Updated weights for policy 0, policy_version 72158 (0.0006) [2024-09-30 01:38:42,705][1157819] Updated weights for policy 0, policy_version 72168 (0.0006) [2024-09-30 01:38:43,163][1157819] Updated weights for policy 0, policy_version 72178 (0.0006) [2024-09-30 01:38:43,637][1157819] Updated weights for policy 0, policy_version 72188 (0.0006) [2024-09-30 01:38:44,110][1157819] Updated weights for policy 0, policy_version 72198 (0.0006) [2024-09-30 01:38:44,578][1157819] Updated weights for policy 0, policy_version 72208 (0.0006) [2024-09-30 01:38:45,020][1157819] Updated weights for policy 0, policy_version 72218 (0.0006) [2024-09-30 01:38:45,466][1157520] Fps is (10 sec: 87244.9, 60 sec: 83217.1, 300 sec: 79337.4). Total num frames: 295841792. Throughput: 0: 21005.2. Samples: 63923808. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:38:45,466][1157520] Avg episode reward: [(0, '53.552')] [2024-09-30 01:38:45,484][1157819] Updated weights for policy 0, policy_version 72228 (0.0006) [2024-09-30 01:38:45,959][1157819] Updated weights for policy 0, policy_version 72238 (0.0006) [2024-09-30 01:38:46,447][1157819] Updated weights for policy 0, policy_version 72248 (0.0006) [2024-09-30 01:38:46,919][1157819] Updated weights for policy 0, policy_version 72258 (0.0006) [2024-09-30 01:38:47,374][1157819] Updated weights for policy 0, policy_version 72268 (0.0006) [2024-09-30 01:38:47,818][1157819] Updated weights for policy 0, policy_version 72278 (0.0006) [2024-09-30 01:38:48,290][1157819] Updated weights for policy 0, policy_version 72288 (0.0006) [2024-09-30 01:38:48,776][1157819] Updated weights for policy 0, policy_version 72298 (0.0006) [2024-09-30 01:38:49,265][1157819] Updated weights for policy 0, policy_version 72308 (0.0006) [2024-09-30 01:38:49,736][1157819] Updated weights for policy 0, policy_version 72318 (0.0006) [2024-09-30 01:38:50,199][1157819] Updated weights for policy 0, policy_version 72328 (0.0006) [2024-09-30 01:38:50,466][1157520] Fps is (10 sec: 87654.3, 60 sec: 83831.5, 300 sec: 79448.5). Total num frames: 296280064. Throughput: 0: 21224.2. Samples: 64055172. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:38:50,466][1157520] Avg episode reward: [(0, '52.230')] [2024-09-30 01:38:50,640][1157819] Updated weights for policy 0, policy_version 72338 (0.0006) [2024-09-30 01:38:51,104][1157819] Updated weights for policy 0, policy_version 72348 (0.0006) [2024-09-30 01:38:51,583][1157819] Updated weights for policy 0, policy_version 72358 (0.0006) [2024-09-30 01:38:52,061][1157819] Updated weights for policy 0, policy_version 72368 (0.0006) [2024-09-30 01:38:52,554][1157819] Updated weights for policy 0, policy_version 72378 (0.0006) [2024-09-30 01:38:52,977][1157819] Updated weights for policy 0, policy_version 72388 (0.0006) [2024-09-30 01:38:53,419][1157819] Updated weights for policy 0, policy_version 72398 (0.0006) [2024-09-30 01:38:53,903][1157819] Updated weights for policy 0, policy_version 72408 (0.0006) [2024-09-30 01:38:54,370][1157736] Signal inference workers to stop experience collection... (4650 times) [2024-09-30 01:38:54,373][1157819] InferenceWorker_p0-w0: stopping experience collection (4650 times) [2024-09-30 01:38:54,379][1157736] Signal inference workers to resume experience collection... (4650 times) [2024-09-30 01:38:54,379][1157819] InferenceWorker_p0-w0: resuming experience collection (4650 times) [2024-09-30 01:38:54,400][1157819] Updated weights for policy 0, policy_version 72418 (0.0006) [2024-09-30 01:38:54,883][1157819] Updated weights for policy 0, policy_version 72428 (0.0007) [2024-09-30 01:38:55,323][1157819] Updated weights for policy 0, policy_version 72438 (0.0006) [2024-09-30 01:38:55,466][1157520] Fps is (10 sec: 87654.5, 60 sec: 84514.1, 300 sec: 79504.1). Total num frames: 296718336. Throughput: 0: 21379.6. Samples: 64121108. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:38:55,466][1157520] Avg episode reward: [(0, '53.696')] [2024-09-30 01:38:55,780][1157819] Updated weights for policy 0, policy_version 72448 (0.0006) [2024-09-30 01:38:56,232][1157819] Updated weights for policy 0, policy_version 72458 (0.0007) [2024-09-30 01:38:56,717][1157819] Updated weights for policy 0, policy_version 72468 (0.0006) [2024-09-30 01:38:57,193][1157819] Updated weights for policy 0, policy_version 72478 (0.0006) [2024-09-30 01:38:57,671][1157819] Updated weights for policy 0, policy_version 72488 (0.0006) [2024-09-30 01:38:58,086][1157819] Updated weights for policy 0, policy_version 72498 (0.0006) [2024-09-30 01:38:58,572][1157819] Updated weights for policy 0, policy_version 72508 (0.0006) [2024-09-30 01:38:59,065][1157819] Updated weights for policy 0, policy_version 72518 (0.0006) [2024-09-30 01:38:59,515][1157819] Updated weights for policy 0, policy_version 72528 (0.0006) [2024-09-30 01:38:59,980][1157819] Updated weights for policy 0, policy_version 72538 (0.0006) [2024-09-30 01:39:00,461][1157819] Updated weights for policy 0, policy_version 72548 (0.0006) [2024-09-30 01:39:00,466][1157520] Fps is (10 sec: 87653.9, 60 sec: 85128.5, 300 sec: 79684.6). Total num frames: 297156608. Throughput: 0: 21670.5. Samples: 64252708. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:39:00,466][1157520] Avg episode reward: [(0, '52.904')] [2024-09-30 01:39:00,916][1157819] Updated weights for policy 0, policy_version 72558 (0.0006) [2024-09-30 01:39:01,393][1157819] Updated weights for policy 0, policy_version 72568 (0.0006) [2024-09-30 01:39:01,834][1157819] Updated weights for policy 0, policy_version 72578 (0.0006) [2024-09-30 01:39:02,294][1157819] Updated weights for policy 0, policy_version 72588 (0.0006) [2024-09-30 01:39:02,772][1157819] Updated weights for policy 0, policy_version 72598 (0.0006) [2024-09-30 01:39:03,266][1157819] Updated weights for policy 0, policy_version 72608 (0.0006) [2024-09-30 01:39:03,714][1157819] Updated weights for policy 0, policy_version 72618 (0.0006) [2024-09-30 01:39:04,161][1157819] Updated weights for policy 0, policy_version 72628 (0.0006) [2024-09-30 01:39:04,627][1157819] Updated weights for policy 0, policy_version 72638 (0.0006) [2024-09-30 01:39:05,126][1157819] Updated weights for policy 0, policy_version 72648 (0.0006) [2024-09-30 01:39:05,466][1157520] Fps is (10 sec: 87654.4, 60 sec: 85674.7, 300 sec: 79878.9). Total num frames: 297594880. Throughput: 0: 21828.6. Samples: 64384420. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:39:05,466][1157520] Avg episode reward: [(0, '51.159')] [2024-09-30 01:39:05,580][1157819] Updated weights for policy 0, policy_version 72658 (0.0006) [2024-09-30 01:39:06,061][1157819] Updated weights for policy 0, policy_version 72668 (0.0006) [2024-09-30 01:39:06,549][1157819] Updated weights for policy 0, policy_version 72678 (0.0006) [2024-09-30 01:39:06,997][1157819] Updated weights for policy 0, policy_version 72688 (0.0006) [2024-09-30 01:39:07,498][1157819] Updated weights for policy 0, policy_version 72698 (0.0006) [2024-09-30 01:39:07,936][1157819] Updated weights for policy 0, policy_version 72708 (0.0006) [2024-09-30 01:39:08,436][1157819] Updated weights for policy 0, policy_version 72718 (0.0006) [2024-09-30 01:39:08,925][1157819] Updated weights for policy 0, policy_version 72728 (0.0006) [2024-09-30 01:39:09,410][1157819] Updated weights for policy 0, policy_version 72738 (0.0007) [2024-09-30 01:39:09,913][1157819] Updated weights for policy 0, policy_version 72748 (0.0006) [2024-09-30 01:39:10,374][1157819] Updated weights for policy 0, policy_version 72758 (0.0006) [2024-09-30 01:39:10,466][1157520] Fps is (10 sec: 86425.8, 60 sec: 86220.7, 300 sec: 79990.0). Total num frames: 298020864. Throughput: 0: 21872.1. Samples: 64449356. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:39:10,466][1157520] Avg episode reward: [(0, '51.584')] [2024-09-30 01:39:10,863][1157819] Updated weights for policy 0, policy_version 72768 (0.0006) [2024-09-30 01:39:11,366][1157819] Updated weights for policy 0, policy_version 72778 (0.0006) [2024-09-30 01:39:11,857][1157819] Updated weights for policy 0, policy_version 72788 (0.0006) [2024-09-30 01:39:12,356][1157819] Updated weights for policy 0, policy_version 72798 (0.0006) [2024-09-30 01:39:12,843][1157819] Updated weights for policy 0, policy_version 72808 (0.0006) [2024-09-30 01:39:13,330][1157819] Updated weights for policy 0, policy_version 72818 (0.0006) [2024-09-30 01:39:13,812][1157819] Updated weights for policy 0, policy_version 72828 (0.0006) [2024-09-30 01:39:14,315][1157819] Updated weights for policy 0, policy_version 72838 (0.0006) [2024-09-30 01:39:14,771][1157819] Updated weights for policy 0, policy_version 72848 (0.0006) [2024-09-30 01:39:15,294][1157819] Updated weights for policy 0, policy_version 72858 (0.0006) [2024-09-30 01:39:15,466][1157520] Fps is (10 sec: 84786.9, 60 sec: 86630.4, 300 sec: 80073.3). Total num frames: 298442752. Throughput: 0: 21770.9. Samples: 64575200. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 01:39:15,466][1157520] Avg episode reward: [(0, '51.949')] [2024-09-30 01:39:15,775][1157819] Updated weights for policy 0, policy_version 72868 (0.0006) [2024-09-30 01:39:16,267][1157819] Updated weights for policy 0, policy_version 72878 (0.0006) [2024-09-30 01:39:16,574][1157736] Signal inference workers to stop experience collection... (4700 times) [2024-09-30 01:39:16,575][1157736] Signal inference workers to resume experience collection... (4700 times) [2024-09-30 01:39:16,580][1157819] InferenceWorker_p0-w0: stopping experience collection (4700 times) [2024-09-30 01:39:16,580][1157819] InferenceWorker_p0-w0: resuming experience collection (4700 times) [2024-09-30 01:39:16,753][1157819] Updated weights for policy 0, policy_version 72888 (0.0006) [2024-09-30 01:39:17,244][1157819] Updated weights for policy 0, policy_version 72898 (0.0006) [2024-09-30 01:39:17,725][1157819] Updated weights for policy 0, policy_version 72908 (0.0006) [2024-09-30 01:39:18,233][1157819] Updated weights for policy 0, policy_version 72918 (0.0006) [2024-09-30 01:39:18,714][1157819] Updated weights for policy 0, policy_version 72928 (0.0006) [2024-09-30 01:39:19,210][1157819] Updated weights for policy 0, policy_version 72938 (0.0006) [2024-09-30 01:39:19,719][1157819] Updated weights for policy 0, policy_version 72948 (0.0006) [2024-09-30 01:39:20,211][1157819] Updated weights for policy 0, policy_version 72958 (0.0006) [2024-09-30 01:39:20,466][1157520] Fps is (10 sec: 83558.4, 60 sec: 86562.1, 300 sec: 80142.8). Total num frames: 298856448. Throughput: 0: 21610.0. Samples: 64700116. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 01:39:20,466][1157520] Avg episode reward: [(0, '53.917')] [2024-09-30 01:39:20,728][1157819] Updated weights for policy 0, policy_version 72968 (0.0006) [2024-09-30 01:39:21,269][1157819] Updated weights for policy 0, policy_version 72978 (0.0006) [2024-09-30 01:39:21,775][1157819] Updated weights for policy 0, policy_version 72988 (0.0006) [2024-09-30 01:39:22,290][1157819] Updated weights for policy 0, policy_version 72998 (0.0006) [2024-09-30 01:39:22,794][1157819] Updated weights for policy 0, policy_version 73008 (0.0006) [2024-09-30 01:39:23,346][1157819] Updated weights for policy 0, policy_version 73018 (0.0006) [2024-09-30 01:39:23,877][1157819] Updated weights for policy 0, policy_version 73028 (0.0006) [2024-09-30 01:39:24,373][1157819] Updated weights for policy 0, policy_version 73038 (0.0006) [2024-09-30 01:39:24,870][1157819] Updated weights for policy 0, policy_version 73048 (0.0006) [2024-09-30 01:39:25,421][1157819] Updated weights for policy 0, policy_version 73058 (0.0006) [2024-09-30 01:39:25,466][1157520] Fps is (10 sec: 80691.4, 60 sec: 86016.0, 300 sec: 80128.9). Total num frames: 299249664. Throughput: 0: 21470.7. Samples: 64759476. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 01:39:25,466][1157520] Avg episode reward: [(0, '53.839')] [2024-09-30 01:39:25,973][1157819] Updated weights for policy 0, policy_version 73068 (0.0006) [2024-09-30 01:39:26,488][1157819] Updated weights for policy 0, policy_version 73078 (0.0006) [2024-09-30 01:39:26,986][1157819] Updated weights for policy 0, policy_version 73088 (0.0006) [2024-09-30 01:39:27,508][1157819] Updated weights for policy 0, policy_version 73098 (0.0006) [2024-09-30 01:39:28,003][1157819] Updated weights for policy 0, policy_version 73108 (0.0006) [2024-09-30 01:39:28,505][1157819] Updated weights for policy 0, policy_version 73118 (0.0006) [2024-09-30 01:39:29,042][1157819] Updated weights for policy 0, policy_version 73128 (0.0006) [2024-09-30 01:39:29,526][1157819] Updated weights for policy 0, policy_version 73138 (0.0006) [2024-09-30 01:39:30,025][1157819] Updated weights for policy 0, policy_version 73148 (0.0006) [2024-09-30 01:39:30,466][1157520] Fps is (10 sec: 79053.0, 60 sec: 85333.4, 300 sec: 80059.4). Total num frames: 299646976. Throughput: 0: 21208.7. Samples: 64878200. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 01:39:30,466][1157520] Avg episode reward: [(0, '54.509')] [2024-09-30 01:39:30,510][1157819] Updated weights for policy 0, policy_version 73158 (0.0006) [2024-09-30 01:39:30,997][1157819] Updated weights for policy 0, policy_version 73168 (0.0006) [2024-09-30 01:39:31,499][1157819] Updated weights for policy 0, policy_version 73178 (0.0007) [2024-09-30 01:39:32,050][1157819] Updated weights for policy 0, policy_version 73188 (0.0006) [2024-09-30 01:39:32,624][1157819] Updated weights for policy 0, policy_version 73198 (0.0006) [2024-09-30 01:39:33,160][1157819] Updated weights for policy 0, policy_version 73208 (0.0006) [2024-09-30 01:39:33,741][1157819] Updated weights for policy 0, policy_version 73218 (0.0006) [2024-09-30 01:39:34,278][1157819] Updated weights for policy 0, policy_version 73228 (0.0006) [2024-09-30 01:39:34,849][1157819] Updated weights for policy 0, policy_version 73238 (0.0006) [2024-09-30 01:39:35,437][1157819] Updated weights for policy 0, policy_version 73248 (0.0006) [2024-09-30 01:39:35,466][1157520] Fps is (10 sec: 77412.8, 60 sec: 84240.8, 300 sec: 79878.9). Total num frames: 300023808. Throughput: 0: 20881.3. Samples: 64994836. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 01:39:35,466][1157520] Avg episode reward: [(0, '51.588')] [2024-09-30 01:39:35,995][1157819] Updated weights for policy 0, policy_version 73258 (0.0006) [2024-09-30 01:39:36,552][1157819] Updated weights for policy 0, policy_version 73268 (0.0006) [2024-09-30 01:39:37,117][1157819] Updated weights for policy 0, policy_version 73278 (0.0006) [2024-09-30 01:39:37,689][1157819] Updated weights for policy 0, policy_version 73288 (0.0006) [2024-09-30 01:39:38,230][1157819] Updated weights for policy 0, policy_version 73298 (0.0006) [2024-09-30 01:39:38,776][1157819] Updated weights for policy 0, policy_version 73308 (0.0006) [2024-09-30 01:39:39,282][1157819] Updated weights for policy 0, policy_version 73318 (0.0006) [2024-09-30 01:39:39,813][1157819] Updated weights for policy 0, policy_version 73328 (0.0006) [2024-09-30 01:39:40,307][1157819] Updated weights for policy 0, policy_version 73338 (0.0006) [2024-09-30 01:39:40,466][1157520] Fps is (10 sec: 75365.9, 60 sec: 83285.2, 300 sec: 79740.1). Total num frames: 300400640. Throughput: 0: 20614.5. Samples: 65048760. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 01:39:40,466][1157520] Avg episode reward: [(0, '52.527')] [2024-09-30 01:39:40,840][1157819] Updated weights for policy 0, policy_version 73348 (0.0006) [2024-09-30 01:39:41,417][1157819] Updated weights for policy 0, policy_version 73358 (0.0006) [2024-09-30 01:39:41,961][1157819] Updated weights for policy 0, policy_version 73368 (0.0006) [2024-09-30 01:39:42,486][1157819] Updated weights for policy 0, policy_version 73378 (0.0006) [2024-09-30 01:39:43,013][1157819] Updated weights for policy 0, policy_version 73388 (0.0006) [2024-09-30 01:39:43,571][1157819] Updated weights for policy 0, policy_version 73398 (0.0006) [2024-09-30 01:39:44,130][1157819] Updated weights for policy 0, policy_version 73408 (0.0006) [2024-09-30 01:39:44,664][1157819] Updated weights for policy 0, policy_version 73418 (0.0006) [2024-09-30 01:39:45,184][1157819] Updated weights for policy 0, policy_version 73428 (0.0007) [2024-09-30 01:39:45,466][1157520] Fps is (10 sec: 75367.3, 60 sec: 82261.2, 300 sec: 79670.6). Total num frames: 300777472. Throughput: 0: 20243.1. Samples: 65163648. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 01:39:45,466][1157520] Avg episode reward: [(0, '51.594')] [2024-09-30 01:39:45,824][1157819] Updated weights for policy 0, policy_version 73438 (0.0006) [2024-09-30 01:39:46,423][1157819] Updated weights for policy 0, policy_version 73448 (0.0006) [2024-09-30 01:39:46,999][1157819] Updated weights for policy 0, policy_version 73458 (0.0006) [2024-09-30 01:39:47,285][1157736] Signal inference workers to stop experience collection... (4750 times) [2024-09-30 01:39:47,289][1157736] Signal inference workers to resume experience collection... (4750 times) [2024-09-30 01:39:47,289][1157819] InferenceWorker_p0-w0: stopping experience collection (4750 times) [2024-09-30 01:39:47,293][1157819] InferenceWorker_p0-w0: resuming experience collection (4750 times) [2024-09-30 01:39:47,620][1157819] Updated weights for policy 0, policy_version 73468 (0.0006) [2024-09-30 01:39:48,206][1157819] Updated weights for policy 0, policy_version 73478 (0.0006) [2024-09-30 01:39:48,786][1157819] Updated weights for policy 0, policy_version 73488 (0.0006) [2024-09-30 01:39:49,397][1157819] Updated weights for policy 0, policy_version 73498 (0.0006) [2024-09-30 01:39:49,993][1157819] Updated weights for policy 0, policy_version 73508 (0.0006) [2024-09-30 01:39:50,466][1157520] Fps is (10 sec: 71270.3, 60 sec: 80554.6, 300 sec: 79420.7). Total num frames: 301113344. Throughput: 0: 19643.4. Samples: 65268372. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 01:39:50,466][1157520] Avg episode reward: [(0, '50.950')] [2024-09-30 01:39:50,674][1157819] Updated weights for policy 0, policy_version 73518 (0.0006) [2024-09-30 01:39:51,299][1157819] Updated weights for policy 0, policy_version 73528 (0.0006) [2024-09-30 01:39:51,891][1157819] Updated weights for policy 0, policy_version 73538 (0.0006) [2024-09-30 01:39:52,487][1157819] Updated weights for policy 0, policy_version 73548 (0.0006) [2024-09-30 01:39:53,079][1157819] Updated weights for policy 0, policy_version 73558 (0.0006) [2024-09-30 01:39:53,656][1157819] Updated weights for policy 0, policy_version 73568 (0.0006) [2024-09-30 01:39:54,227][1157819] Updated weights for policy 0, policy_version 73578 (0.0006) [2024-09-30 01:39:54,787][1157819] Updated weights for policy 0, policy_version 73588 (0.0006) [2024-09-30 01:39:55,398][1157819] Updated weights for policy 0, policy_version 73598 (0.0006) [2024-09-30 01:39:55,466][1157520] Fps is (10 sec: 68402.3, 60 sec: 79052.5, 300 sec: 79143.0). Total num frames: 301461504. Throughput: 0: 19311.4. Samples: 65318372. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 01:39:55,466][1157520] Avg episode reward: [(0, '53.560')] [2024-09-30 01:39:55,935][1157819] Updated weights for policy 0, policy_version 73608 (0.0006) [2024-09-30 01:39:56,548][1157819] Updated weights for policy 0, policy_version 73618 (0.0006) [2024-09-30 01:39:57,133][1157819] Updated weights for policy 0, policy_version 73628 (0.0006) [2024-09-30 01:39:57,685][1157819] Updated weights for policy 0, policy_version 73638 (0.0006) [2024-09-30 01:39:58,232][1157819] Updated weights for policy 0, policy_version 73648 (0.0006) [2024-09-30 01:39:58,801][1157819] Updated weights for policy 0, policy_version 73658 (0.0006) [2024-09-30 01:39:59,385][1157819] Updated weights for policy 0, policy_version 73668 (0.0006) [2024-09-30 01:39:59,958][1157819] Updated weights for policy 0, policy_version 73678 (0.0006) [2024-09-30 01:40:00,466][1157520] Fps is (10 sec: 70451.0, 60 sec: 77687.4, 300 sec: 78865.3). Total num frames: 301817856. Throughput: 0: 18889.5. Samples: 65425228. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:40:00,466][1157520] Avg episode reward: [(0, '52.914')] [2024-09-30 01:40:00,531][1157819] Updated weights for policy 0, policy_version 73688 (0.0006) [2024-09-30 01:40:01,131][1157819] Updated weights for policy 0, policy_version 73698 (0.0006) [2024-09-30 01:40:01,689][1157819] Updated weights for policy 0, policy_version 73708 (0.0006) [2024-09-30 01:40:02,267][1157819] Updated weights for policy 0, policy_version 73718 (0.0006) [2024-09-30 01:40:02,770][1157819] Updated weights for policy 0, policy_version 73728 (0.0006) [2024-09-30 01:40:03,334][1157819] Updated weights for policy 0, policy_version 73738 (0.0006) [2024-09-30 01:40:03,882][1157819] Updated weights for policy 0, policy_version 73748 (0.0006) [2024-09-30 01:40:04,406][1157819] Updated weights for policy 0, policy_version 73758 (0.0006) [2024-09-30 01:40:04,971][1157819] Updated weights for policy 0, policy_version 73768 (0.0006) [2024-09-30 01:40:05,466][1157520] Fps is (10 sec: 72908.6, 60 sec: 76594.9, 300 sec: 78670.9). Total num frames: 302190592. Throughput: 0: 18560.8. Samples: 65535356. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:40:05,466][1157520] Avg episode reward: [(0, '53.028')] [2024-09-30 01:40:05,477][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000073778_302194688.pth... [2024-09-30 01:40:05,478][1157819] Updated weights for policy 0, policy_version 73778 (0.0006) [2024-09-30 01:40:05,529][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000069185_283381760.pth [2024-09-30 01:40:06,044][1157819] Updated weights for policy 0, policy_version 73788 (0.0006) [2024-09-30 01:40:06,574][1157819] Updated weights for policy 0, policy_version 73798 (0.0006) [2024-09-30 01:40:07,121][1157819] Updated weights for policy 0, policy_version 73808 (0.0006) [2024-09-30 01:40:07,669][1157819] Updated weights for policy 0, policy_version 73818 (0.0006) [2024-09-30 01:40:08,182][1157819] Updated weights for policy 0, policy_version 73828 (0.0006) [2024-09-30 01:40:08,733][1157819] Updated weights for policy 0, policy_version 73838 (0.0006) [2024-09-30 01:40:09,296][1157819] Updated weights for policy 0, policy_version 73848 (0.0006) [2024-09-30 01:40:09,825][1157819] Updated weights for policy 0, policy_version 73858 (0.0006) [2024-09-30 01:40:10,390][1157819] Updated weights for policy 0, policy_version 73868 (0.0006) [2024-09-30 01:40:10,466][1157520] Fps is (10 sec: 74956.4, 60 sec: 75775.8, 300 sec: 78462.7). Total num frames: 302567424. Throughput: 0: 18495.4. Samples: 65591772. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:40:10,466][1157520] Avg episode reward: [(0, '52.494')] [2024-09-30 01:40:10,932][1157819] Updated weights for policy 0, policy_version 73878 (0.0006) [2024-09-30 01:40:11,501][1157819] Updated weights for policy 0, policy_version 73888 (0.0006) [2024-09-30 01:40:12,011][1157819] Updated weights for policy 0, policy_version 73898 (0.0006) [2024-09-30 01:40:12,568][1157819] Updated weights for policy 0, policy_version 73908 (0.0006) [2024-09-30 01:40:13,103][1157819] Updated weights for policy 0, policy_version 73918 (0.0006) [2024-09-30 01:40:13,604][1157819] Updated weights for policy 0, policy_version 73928 (0.0006) [2024-09-30 01:40:14,134][1157819] Updated weights for policy 0, policy_version 73938 (0.0006) [2024-09-30 01:40:14,653][1157819] Updated weights for policy 0, policy_version 73948 (0.0006) [2024-09-30 01:40:15,201][1157819] Updated weights for policy 0, policy_version 73958 (0.0006) [2024-09-30 01:40:15,466][1157520] Fps is (10 sec: 76187.1, 60 sec: 75161.6, 300 sec: 78351.6). Total num frames: 302952448. Throughput: 0: 18384.4. Samples: 65705500. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:40:15,466][1157520] Avg episode reward: [(0, '51.804')] [2024-09-30 01:40:15,702][1157819] Updated weights for policy 0, policy_version 73968 (0.0006) [2024-09-30 01:40:16,236][1157819] Updated weights for policy 0, policy_version 73978 (0.0006) [2024-09-30 01:40:16,762][1157819] Updated weights for policy 0, policy_version 73988 (0.0006) [2024-09-30 01:40:17,309][1157819] Updated weights for policy 0, policy_version 73998 (0.0006) [2024-09-30 01:40:17,817][1157819] Updated weights for policy 0, policy_version 74008 (0.0006) [2024-09-30 01:40:18,305][1157819] Updated weights for policy 0, policy_version 74018 (0.0006) [2024-09-30 01:40:18,814][1157819] Updated weights for policy 0, policy_version 74028 (0.0006) [2024-09-30 01:40:19,331][1157819] Updated weights for policy 0, policy_version 74038 (0.0006) [2024-09-30 01:40:19,840][1157819] Updated weights for policy 0, policy_version 74048 (0.0006) [2024-09-30 01:40:20,343][1157819] Updated weights for policy 0, policy_version 74058 (0.0006) [2024-09-30 01:40:20,466][1157520] Fps is (10 sec: 78234.4, 60 sec: 74888.5, 300 sec: 78268.3). Total num frames: 303349760. Throughput: 0: 18440.2. Samples: 65824644. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:40:20,466][1157520] Avg episode reward: [(0, '53.200')] [2024-09-30 01:40:20,842][1157819] Updated weights for policy 0, policy_version 74068 (0.0006) [2024-09-30 01:40:21,339][1157819] Updated weights for policy 0, policy_version 74078 (0.0006) [2024-09-30 01:40:21,841][1157819] Updated weights for policy 0, policy_version 74088 (0.0006) [2024-09-30 01:40:22,361][1157819] Updated weights for policy 0, policy_version 74098 (0.0006) [2024-09-30 01:40:22,902][1157819] Updated weights for policy 0, policy_version 74108 (0.0006) [2024-09-30 01:40:23,440][1157819] Updated weights for policy 0, policy_version 74118 (0.0006) [2024-09-30 01:40:23,967][1157819] Updated weights for policy 0, policy_version 74128 (0.0006) [2024-09-30 01:40:24,516][1157819] Updated weights for policy 0, policy_version 74138 (0.0006) [2024-09-30 01:40:25,070][1157819] Updated weights for policy 0, policy_version 74148 (0.0006) [2024-09-30 01:40:25,466][1157520] Fps is (10 sec: 78642.6, 60 sec: 74820.1, 300 sec: 78171.1). Total num frames: 303738880. Throughput: 0: 18571.5. Samples: 65884480. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:40:25,466][1157520] Avg episode reward: [(0, '52.577')] [2024-09-30 01:40:25,601][1157819] Updated weights for policy 0, policy_version 74158 (0.0006) [2024-09-30 01:40:26,139][1157819] Updated weights for policy 0, policy_version 74168 (0.0006) [2024-09-30 01:40:26,649][1157819] Updated weights for policy 0, policy_version 74178 (0.0006) [2024-09-30 01:40:27,169][1157819] Updated weights for policy 0, policy_version 74188 (0.0006) [2024-09-30 01:40:27,678][1157819] Updated weights for policy 0, policy_version 74198 (0.0006) [2024-09-30 01:40:28,164][1157819] Updated weights for policy 0, policy_version 74208 (0.0006) [2024-09-30 01:40:28,672][1157819] Updated weights for policy 0, policy_version 74218 (0.0006) [2024-09-30 01:40:29,202][1157819] Updated weights for policy 0, policy_version 74228 (0.0006) [2024-09-30 01:40:29,668][1157819] Updated weights for policy 0, policy_version 74238 (0.0006) [2024-09-30 01:40:30,150][1157819] Updated weights for policy 0, policy_version 74248 (0.0006) [2024-09-30 01:40:30,466][1157520] Fps is (10 sec: 79462.9, 60 sec: 74956.8, 300 sec: 78087.8). Total num frames: 304144384. Throughput: 0: 18615.2. Samples: 66001328. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:40:30,466][1157520] Avg episode reward: [(0, '55.556')] [2024-09-30 01:40:30,647][1157819] Updated weights for policy 0, policy_version 74258 (0.0006) [2024-09-30 01:40:31,102][1157819] Updated weights for policy 0, policy_version 74268 (0.0006) [2024-09-30 01:40:31,596][1157819] Updated weights for policy 0, policy_version 74278 (0.0006) [2024-09-30 01:40:32,079][1157819] Updated weights for policy 0, policy_version 74288 (0.0006) [2024-09-30 01:40:32,571][1157819] Updated weights for policy 0, policy_version 74298 (0.0006) [2024-09-30 01:40:33,059][1157819] Updated weights for policy 0, policy_version 74308 (0.0006) [2024-09-30 01:40:33,543][1157819] Updated weights for policy 0, policy_version 74318 (0.0006) [2024-09-30 01:40:34,027][1157819] Updated weights for policy 0, policy_version 74328 (0.0006) [2024-09-30 01:40:34,515][1157819] Updated weights for policy 0, policy_version 74338 (0.0006) [2024-09-30 01:40:34,964][1157819] Updated weights for policy 0, policy_version 74348 (0.0006) [2024-09-30 01:40:35,466][1157520] Fps is (10 sec: 82740.4, 60 sec: 75708.0, 300 sec: 78018.4). Total num frames: 304566272. Throughput: 0: 19104.3. Samples: 66128064. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:40:35,466][1157520] Avg episode reward: [(0, '52.504')] [2024-09-30 01:40:35,479][1157819] Updated weights for policy 0, policy_version 74358 (0.0006) [2024-09-30 01:40:35,965][1157819] Updated weights for policy 0, policy_version 74368 (0.0006) [2024-09-30 01:40:36,445][1157819] Updated weights for policy 0, policy_version 74378 (0.0006) [2024-09-30 01:40:36,941][1157819] Updated weights for policy 0, policy_version 74388 (0.0006) [2024-09-30 01:40:37,462][1157819] Updated weights for policy 0, policy_version 74398 (0.0006) [2024-09-30 01:40:37,970][1157819] Updated weights for policy 0, policy_version 74408 (0.0007) [2024-09-30 01:40:38,455][1157819] Updated weights for policy 0, policy_version 74418 (0.0006) [2024-09-30 01:40:38,976][1157819] Updated weights for policy 0, policy_version 74428 (0.0006) [2024-09-30 01:40:39,520][1157819] Updated weights for policy 0, policy_version 74438 (0.0006) [2024-09-30 01:40:40,043][1157819] Updated weights for policy 0, policy_version 74448 (0.0006) [2024-09-30 01:40:40,466][1157520] Fps is (10 sec: 83148.0, 60 sec: 76253.8, 300 sec: 78032.3). Total num frames: 304975872. Throughput: 0: 19380.2. Samples: 66190476. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:40:40,466][1157520] Avg episode reward: [(0, '52.772')] [2024-09-30 01:40:40,571][1157819] Updated weights for policy 0, policy_version 74458 (0.0006) [2024-09-30 01:40:41,068][1157819] Updated weights for policy 0, policy_version 74468 (0.0006) [2024-09-30 01:40:41,575][1157819] Updated weights for policy 0, policy_version 74478 (0.0006) [2024-09-30 01:40:42,100][1157819] Updated weights for policy 0, policy_version 74488 (0.0006) [2024-09-30 01:40:42,603][1157819] Updated weights for policy 0, policy_version 74498 (0.0006) [2024-09-30 01:40:43,126][1157819] Updated weights for policy 0, policy_version 74508 (0.0006) [2024-09-30 01:40:43,645][1157819] Updated weights for policy 0, policy_version 74518 (0.0006) [2024-09-30 01:40:44,165][1157819] Updated weights for policy 0, policy_version 74528 (0.0006) [2024-09-30 01:40:44,689][1157819] Updated weights for policy 0, policy_version 74538 (0.0006) [2024-09-30 01:40:45,188][1157819] Updated weights for policy 0, policy_version 74548 (0.0006) [2024-09-30 01:40:45,466][1157520] Fps is (10 sec: 80280.2, 60 sec: 76526.8, 300 sec: 78129.4). Total num frames: 305369088. Throughput: 0: 19642.6. Samples: 66309148. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 01:40:45,466][1157520] Avg episode reward: [(0, '54.833')] [2024-09-30 01:40:45,705][1157819] Updated weights for policy 0, policy_version 74558 (0.0006) [2024-09-30 01:40:46,224][1157819] Updated weights for policy 0, policy_version 74568 (0.0006) [2024-09-30 01:40:46,749][1157819] Updated weights for policy 0, policy_version 74578 (0.0006) [2024-09-30 01:40:47,259][1157819] Updated weights for policy 0, policy_version 74588 (0.0006) [2024-09-30 01:40:47,586][1157736] Signal inference workers to stop experience collection... (4800 times) [2024-09-30 01:40:47,586][1157736] Signal inference workers to resume experience collection... (4800 times) [2024-09-30 01:40:47,590][1157819] InferenceWorker_p0-w0: stopping experience collection (4800 times) [2024-09-30 01:40:47,590][1157819] InferenceWorker_p0-w0: resuming experience collection (4800 times) [2024-09-30 01:40:47,755][1157819] Updated weights for policy 0, policy_version 74598 (0.0006) [2024-09-30 01:40:48,268][1157819] Updated weights for policy 0, policy_version 74608 (0.0006) [2024-09-30 01:40:48,842][1157819] Updated weights for policy 0, policy_version 74618 (0.0006) [2024-09-30 01:40:49,349][1157819] Updated weights for policy 0, policy_version 74628 (0.0006) [2024-09-30 01:40:49,858][1157819] Updated weights for policy 0, policy_version 74638 (0.0006) [2024-09-30 01:40:50,374][1157819] Updated weights for policy 0, policy_version 74648 (0.0006) [2024-09-30 01:40:50,466][1157520] Fps is (10 sec: 78643.1, 60 sec: 77482.6, 300 sec: 78351.6). Total num frames: 305762304. Throughput: 0: 19841.6. Samples: 66428224. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 01:40:50,466][1157520] Avg episode reward: [(0, '53.508')] [2024-09-30 01:40:50,883][1157819] Updated weights for policy 0, policy_version 74658 (0.0006) [2024-09-30 01:40:51,385][1157819] Updated weights for policy 0, policy_version 74668 (0.0006) [2024-09-30 01:40:51,899][1157819] Updated weights for policy 0, policy_version 74678 (0.0006) [2024-09-30 01:40:52,419][1157819] Updated weights for policy 0, policy_version 74688 (0.0006) [2024-09-30 01:40:52,976][1157819] Updated weights for policy 0, policy_version 74698 (0.0006) [2024-09-30 01:40:53,484][1157819] Updated weights for policy 0, policy_version 74708 (0.0006) [2024-09-30 01:40:53,982][1157819] Updated weights for policy 0, policy_version 74718 (0.0006) [2024-09-30 01:40:54,510][1157819] Updated weights for policy 0, policy_version 74728 (0.0006) [2024-09-30 01:40:55,006][1157819] Updated weights for policy 0, policy_version 74738 (0.0006) [2024-09-30 01:40:55,466][1157520] Fps is (10 sec: 79463.8, 60 sec: 78370.4, 300 sec: 78518.2). Total num frames: 306163712. Throughput: 0: 19903.1. Samples: 66487408. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 01:40:55,466][1157520] Avg episode reward: [(0, '54.289')] [2024-09-30 01:40:55,473][1157819] Updated weights for policy 0, policy_version 74748 (0.0006) [2024-09-30 01:40:55,980][1157819] Updated weights for policy 0, policy_version 74758 (0.0006) [2024-09-30 01:40:56,468][1157819] Updated weights for policy 0, policy_version 74768 (0.0006) [2024-09-30 01:40:56,951][1157819] Updated weights for policy 0, policy_version 74778 (0.0006) [2024-09-30 01:40:57,409][1157819] Updated weights for policy 0, policy_version 74788 (0.0006) [2024-09-30 01:40:57,884][1157819] Updated weights for policy 0, policy_version 74798 (0.0006) [2024-09-30 01:40:58,372][1157819] Updated weights for policy 0, policy_version 74808 (0.0006) [2024-09-30 01:40:58,837][1157819] Updated weights for policy 0, policy_version 74818 (0.0006) [2024-09-30 01:40:59,316][1157819] Updated weights for policy 0, policy_version 74828 (0.0006) [2024-09-30 01:40:59,800][1157819] Updated weights for policy 0, policy_version 74838 (0.0006) [2024-09-30 01:41:00,284][1157819] Updated weights for policy 0, policy_version 74848 (0.0006) [2024-09-30 01:41:00,466][1157520] Fps is (10 sec: 82740.0, 60 sec: 79530.8, 300 sec: 78684.9). Total num frames: 306589696. Throughput: 0: 20145.5. Samples: 66612048. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 01:41:00,466][1157520] Avg episode reward: [(0, '54.478')] [2024-09-30 01:41:00,774][1157819] Updated weights for policy 0, policy_version 74858 (0.0006) [2024-09-30 01:41:01,247][1157819] Updated weights for policy 0, policy_version 74868 (0.0006) [2024-09-30 01:41:01,711][1157819] Updated weights for policy 0, policy_version 74878 (0.0006) [2024-09-30 01:41:02,122][1157819] Updated weights for policy 0, policy_version 74888 (0.0006) [2024-09-30 01:41:02,615][1157819] Updated weights for policy 0, policy_version 74898 (0.0006) [2024-09-30 01:41:03,085][1157819] Updated weights for policy 0, policy_version 74908 (0.0006) [2024-09-30 01:41:03,568][1157819] Updated weights for policy 0, policy_version 74918 (0.0006) [2024-09-30 01:41:04,031][1157819] Updated weights for policy 0, policy_version 74928 (0.0006) [2024-09-30 01:41:04,483][1157819] Updated weights for policy 0, policy_version 74938 (0.0006) [2024-09-30 01:41:04,924][1157819] Updated weights for policy 0, policy_version 74948 (0.0006) [2024-09-30 01:41:05,402][1157819] Updated weights for policy 0, policy_version 74958 (0.0006) [2024-09-30 01:41:05,466][1157520] Fps is (10 sec: 86835.2, 60 sec: 80691.6, 300 sec: 78962.5). Total num frames: 307032064. Throughput: 0: 20412.4. Samples: 66743200. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 01:41:05,466][1157520] Avg episode reward: [(0, '52.381')] [2024-09-30 01:41:05,875][1157819] Updated weights for policy 0, policy_version 74968 (0.0006) [2024-09-30 01:41:06,350][1157819] Updated weights for policy 0, policy_version 74978 (0.0006) [2024-09-30 01:41:06,814][1157819] Updated weights for policy 0, policy_version 74988 (0.0007) [2024-09-30 01:41:07,251][1157819] Updated weights for policy 0, policy_version 74998 (0.0006) [2024-09-30 01:41:07,733][1157819] Updated weights for policy 0, policy_version 75008 (0.0006) [2024-09-30 01:41:08,229][1157819] Updated weights for policy 0, policy_version 75018 (0.0006) [2024-09-30 01:41:08,741][1157819] Updated weights for policy 0, policy_version 75028 (0.0006) [2024-09-30 01:41:09,222][1157819] Updated weights for policy 0, policy_version 75038 (0.0006) [2024-09-30 01:41:09,717][1157819] Updated weights for policy 0, policy_version 75048 (0.0006) [2024-09-30 01:41:10,200][1157819] Updated weights for policy 0, policy_version 75058 (0.0006) [2024-09-30 01:41:10,466][1157520] Fps is (10 sec: 86835.5, 60 sec: 81510.7, 300 sec: 79184.7). Total num frames: 307458048. Throughput: 0: 20542.2. Samples: 66808876. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 01:41:10,466][1157520] Avg episode reward: [(0, '52.914')] [2024-09-30 01:41:10,691][1157819] Updated weights for policy 0, policy_version 75068 (0.0006) [2024-09-30 01:41:11,183][1157819] Updated weights for policy 0, policy_version 75078 (0.0006) [2024-09-30 01:41:11,652][1157819] Updated weights for policy 0, policy_version 75088 (0.0006) [2024-09-30 01:41:12,174][1157819] Updated weights for policy 0, policy_version 75098 (0.0006) [2024-09-30 01:41:12,666][1157819] Updated weights for policy 0, policy_version 75108 (0.0006) [2024-09-30 01:41:13,183][1157819] Updated weights for policy 0, policy_version 75118 (0.0006) [2024-09-30 01:41:13,673][1157819] Updated weights for policy 0, policy_version 75128 (0.0006) [2024-09-30 01:41:14,182][1157819] Updated weights for policy 0, policy_version 75138 (0.0006) [2024-09-30 01:41:14,698][1157819] Updated weights for policy 0, policy_version 75148 (0.0006) [2024-09-30 01:41:15,228][1157819] Updated weights for policy 0, policy_version 75158 (0.0006) [2024-09-30 01:41:15,466][1157520] Fps is (10 sec: 83148.0, 60 sec: 81851.7, 300 sec: 79365.2). Total num frames: 307863552. Throughput: 0: 20711.3. Samples: 66933340. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 01:41:15,466][1157520] Avg episode reward: [(0, '52.743')] [2024-09-30 01:41:15,758][1157819] Updated weights for policy 0, policy_version 75168 (0.0006) [2024-09-30 01:41:16,262][1157819] Updated weights for policy 0, policy_version 75178 (0.0006) [2024-09-30 01:41:16,753][1157819] Updated weights for policy 0, policy_version 75188 (0.0006) [2024-09-30 01:41:17,260][1157819] Updated weights for policy 0, policy_version 75198 (0.0006) [2024-09-30 01:41:17,755][1157819] Updated weights for policy 0, policy_version 75208 (0.0006) [2024-09-30 01:41:18,243][1157819] Updated weights for policy 0, policy_version 75218 (0.0006) [2024-09-30 01:41:18,660][1157736] Signal inference workers to stop experience collection... (4850 times) [2024-09-30 01:41:18,663][1157819] InferenceWorker_p0-w0: stopping experience collection (4850 times) [2024-09-30 01:41:18,667][1157736] Signal inference workers to resume experience collection... (4850 times) [2024-09-30 01:41:18,667][1157819] InferenceWorker_p0-w0: resuming experience collection (4850 times) [2024-09-30 01:41:18,774][1157819] Updated weights for policy 0, policy_version 75228 (0.0006) [2024-09-30 01:41:19,408][1157819] Updated weights for policy 0, policy_version 75238 (0.0006) [2024-09-30 01:41:20,035][1157819] Updated weights for policy 0, policy_version 75248 (0.0006) [2024-09-30 01:41:20,466][1157520] Fps is (10 sec: 78642.3, 60 sec: 81578.6, 300 sec: 79559.6). Total num frames: 308244480. Throughput: 0: 20490.0. Samples: 67050116. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 01:41:20,466][1157520] Avg episode reward: [(0, '54.292')] [2024-09-30 01:41:20,640][1157819] Updated weights for policy 0, policy_version 75258 (0.0006) [2024-09-30 01:41:21,202][1157819] Updated weights for policy 0, policy_version 75268 (0.0006) [2024-09-30 01:41:21,866][1157819] Updated weights for policy 0, policy_version 75278 (0.0006) [2024-09-30 01:41:22,440][1157819] Updated weights for policy 0, policy_version 75288 (0.0006) [2024-09-30 01:41:22,961][1157819] Updated weights for policy 0, policy_version 75298 (0.0006) [2024-09-30 01:41:23,475][1157819] Updated weights for policy 0, policy_version 75308 (0.0006) [2024-09-30 01:41:24,047][1157819] Updated weights for policy 0, policy_version 75318 (0.0006) [2024-09-30 01:41:24,600][1157819] Updated weights for policy 0, policy_version 75328 (0.0006) [2024-09-30 01:41:25,117][1157819] Updated weights for policy 0, policy_version 75338 (0.0006) [2024-09-30 01:41:25,466][1157520] Fps is (10 sec: 74547.2, 60 sec: 81169.1, 300 sec: 79642.9). Total num frames: 308609024. Throughput: 0: 20250.6. Samples: 67101752. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 01:41:25,466][1157520] Avg episode reward: [(0, '51.488')] [2024-09-30 01:41:25,657][1157819] Updated weights for policy 0, policy_version 75348 (0.0006) [2024-09-30 01:41:26,158][1157819] Updated weights for policy 0, policy_version 75358 (0.0006) [2024-09-30 01:41:26,670][1157819] Updated weights for policy 0, policy_version 75368 (0.0006) [2024-09-30 01:41:27,152][1157819] Updated weights for policy 0, policy_version 75378 (0.0006) [2024-09-30 01:41:27,681][1157819] Updated weights for policy 0, policy_version 75388 (0.0006) [2024-09-30 01:41:28,193][1157819] Updated weights for policy 0, policy_version 75398 (0.0006) [2024-09-30 01:41:28,702][1157819] Updated weights for policy 0, policy_version 75408 (0.0006) [2024-09-30 01:41:29,211][1157819] Updated weights for policy 0, policy_version 75418 (0.0006) [2024-09-30 01:41:29,707][1157819] Updated weights for policy 0, policy_version 75428 (0.0006) [2024-09-30 01:41:30,256][1157819] Updated weights for policy 0, policy_version 75438 (0.0006) [2024-09-30 01:41:30,466][1157520] Fps is (10 sec: 76595.0, 60 sec: 81100.6, 300 sec: 79823.4). Total num frames: 309010432. Throughput: 0: 20230.8. Samples: 67219532. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:41:30,466][1157520] Avg episode reward: [(0, '55.528')] [2024-09-30 01:41:30,758][1157819] Updated weights for policy 0, policy_version 75448 (0.0006) [2024-09-30 01:41:31,286][1157819] Updated weights for policy 0, policy_version 75458 (0.0006) [2024-09-30 01:41:31,809][1157819] Updated weights for policy 0, policy_version 75468 (0.0006) [2024-09-30 01:41:32,316][1157819] Updated weights for policy 0, policy_version 75478 (0.0006) [2024-09-30 01:41:32,841][1157819] Updated weights for policy 0, policy_version 75488 (0.0006) [2024-09-30 01:41:33,382][1157819] Updated weights for policy 0, policy_version 75498 (0.0006) [2024-09-30 01:41:33,878][1157819] Updated weights for policy 0, policy_version 75508 (0.0006) [2024-09-30 01:41:34,412][1157819] Updated weights for policy 0, policy_version 75518 (0.0006) [2024-09-30 01:41:34,923][1157819] Updated weights for policy 0, policy_version 75528 (0.0006) [2024-09-30 01:41:35,459][1157819] Updated weights for policy 0, policy_version 75538 (0.0006) [2024-09-30 01:41:35,466][1157520] Fps is (10 sec: 79462.4, 60 sec: 80622.8, 300 sec: 79962.2). Total num frames: 309403648. Throughput: 0: 20218.2. Samples: 67338044. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:41:35,466][1157520] Avg episode reward: [(0, '53.968')] [2024-09-30 01:41:35,947][1157819] Updated weights for policy 0, policy_version 75548 (0.0006) [2024-09-30 01:41:36,474][1157819] Updated weights for policy 0, policy_version 75558 (0.0006) [2024-09-30 01:41:37,003][1157819] Updated weights for policy 0, policy_version 75568 (0.0006) [2024-09-30 01:41:37,526][1157819] Updated weights for policy 0, policy_version 75578 (0.0006) [2024-09-30 01:41:38,040][1157819] Updated weights for policy 0, policy_version 75588 (0.0006) [2024-09-30 01:41:38,539][1157819] Updated weights for policy 0, policy_version 75598 (0.0006) [2024-09-30 01:41:39,058][1157819] Updated weights for policy 0, policy_version 75608 (0.0006) [2024-09-30 01:41:39,597][1157819] Updated weights for policy 0, policy_version 75618 (0.0006) [2024-09-30 01:41:40,115][1157819] Updated weights for policy 0, policy_version 75628 (0.0006) [2024-09-30 01:41:40,466][1157520] Fps is (10 sec: 78643.4, 60 sec: 80349.9, 300 sec: 80017.8). Total num frames: 309796864. Throughput: 0: 20220.3. Samples: 67397324. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:41:40,466][1157520] Avg episode reward: [(0, '52.638')] [2024-09-30 01:41:40,607][1157819] Updated weights for policy 0, policy_version 75638 (0.0006) [2024-09-30 01:41:41,179][1157819] Updated weights for policy 0, policy_version 75648 (0.0006) [2024-09-30 01:41:41,768][1157819] Updated weights for policy 0, policy_version 75658 (0.0006) [2024-09-30 01:41:42,343][1157819] Updated weights for policy 0, policy_version 75668 (0.0006) [2024-09-30 01:41:42,914][1157819] Updated weights for policy 0, policy_version 75678 (0.0006) [2024-09-30 01:41:43,470][1157819] Updated weights for policy 0, policy_version 75688 (0.0006) [2024-09-30 01:41:44,040][1157819] Updated weights for policy 0, policy_version 75698 (0.0006) [2024-09-30 01:41:44,640][1157819] Updated weights for policy 0, policy_version 75708 (0.0006) [2024-09-30 01:41:45,200][1157819] Updated weights for policy 0, policy_version 75718 (0.0006) [2024-09-30 01:41:45,466][1157520] Fps is (10 sec: 74956.7, 60 sec: 79735.6, 300 sec: 79948.3). Total num frames: 310153216. Throughput: 0: 19951.6. Samples: 67509872. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:41:45,466][1157520] Avg episode reward: [(0, '51.107')] [2024-09-30 01:41:45,791][1157819] Updated weights for policy 0, policy_version 75728 (0.0006) [2024-09-30 01:41:46,419][1157819] Updated weights for policy 0, policy_version 75738 (0.0006) [2024-09-30 01:41:46,957][1157819] Updated weights for policy 0, policy_version 75748 (0.0006) [2024-09-30 01:41:47,566][1157819] Updated weights for policy 0, policy_version 75758 (0.0006) [2024-09-30 01:41:48,149][1157819] Updated weights for policy 0, policy_version 75768 (0.0006) [2024-09-30 01:41:48,735][1157819] Updated weights for policy 0, policy_version 75778 (0.0006) [2024-09-30 01:41:49,333][1157819] Updated weights for policy 0, policy_version 75788 (0.0006) [2024-09-30 01:41:49,499][1157736] Signal inference workers to stop experience collection... (4900 times) [2024-09-30 01:41:49,499][1157736] Signal inference workers to resume experience collection... (4900 times) [2024-09-30 01:41:49,503][1157819] InferenceWorker_p0-w0: stopping experience collection (4900 times) [2024-09-30 01:41:49,503][1157819] InferenceWorker_p0-w0: resuming experience collection (4900 times) [2024-09-30 01:41:49,916][1157819] Updated weights for policy 0, policy_version 75798 (0.0006) [2024-09-30 01:41:50,466][1157520] Fps is (10 sec: 70860.7, 60 sec: 79052.8, 300 sec: 79837.3). Total num frames: 310505472. Throughput: 0: 19379.1. Samples: 67615260. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:41:50,466][1157520] Avg episode reward: [(0, '54.052')] [2024-09-30 01:41:50,472][1157819] Updated weights for policy 0, policy_version 75808 (0.0006) [2024-09-30 01:41:51,016][1157819] Updated weights for policy 0, policy_version 75818 (0.0006) [2024-09-30 01:41:51,583][1157819] Updated weights for policy 0, policy_version 75828 (0.0006) [2024-09-30 01:41:52,202][1157819] Updated weights for policy 0, policy_version 75838 (0.0006) [2024-09-30 01:41:52,719][1157819] Updated weights for policy 0, policy_version 75848 (0.0006) [2024-09-30 01:41:53,312][1157819] Updated weights for policy 0, policy_version 75858 (0.0006) [2024-09-30 01:41:53,815][1157819] Updated weights for policy 0, policy_version 75868 (0.0006) [2024-09-30 01:41:54,402][1157819] Updated weights for policy 0, policy_version 75878 (0.0006) [2024-09-30 01:41:54,912][1157819] Updated weights for policy 0, policy_version 75888 (0.0006) [2024-09-30 01:41:55,466][1157520] Fps is (10 sec: 72089.1, 60 sec: 78506.4, 300 sec: 79795.6). Total num frames: 310874112. Throughput: 0: 19123.4. Samples: 67669432. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:41:55,466][1157520] Avg episode reward: [(0, '49.999')] [2024-09-30 01:41:55,485][1157819] Updated weights for policy 0, policy_version 75898 (0.0006) [2024-09-30 01:41:56,022][1157819] Updated weights for policy 0, policy_version 75908 (0.0006) [2024-09-30 01:41:56,544][1157819] Updated weights for policy 0, policy_version 75918 (0.0006) [2024-09-30 01:41:57,127][1157819] Updated weights for policy 0, policy_version 75928 (0.0006) [2024-09-30 01:41:57,643][1157819] Updated weights for policy 0, policy_version 75938 (0.0006) [2024-09-30 01:41:58,156][1157819] Updated weights for policy 0, policy_version 75948 (0.0006) [2024-09-30 01:41:58,759][1157819] Updated weights for policy 0, policy_version 75958 (0.0006) [2024-09-30 01:41:59,277][1157819] Updated weights for policy 0, policy_version 75968 (0.0006) [2024-09-30 01:41:59,880][1157819] Updated weights for policy 0, policy_version 75978 (0.0006) [2024-09-30 01:42:00,415][1157819] Updated weights for policy 0, policy_version 75988 (0.0006) [2024-09-30 01:42:00,466][1157520] Fps is (10 sec: 74137.0, 60 sec: 77619.0, 300 sec: 79754.0). Total num frames: 311246848. Throughput: 0: 18853.3. Samples: 67781740. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:42:00,466][1157520] Avg episode reward: [(0, '52.130')] [2024-09-30 01:42:00,990][1157819] Updated weights for policy 0, policy_version 75998 (0.0006) [2024-09-30 01:42:01,529][1157819] Updated weights for policy 0, policy_version 76008 (0.0006) [2024-09-30 01:42:02,094][1157819] Updated weights for policy 0, policy_version 76018 (0.0006) [2024-09-30 01:42:02,648][1157819] Updated weights for policy 0, policy_version 76028 (0.0006) [2024-09-30 01:42:03,209][1157819] Updated weights for policy 0, policy_version 76038 (0.0006) [2024-09-30 01:42:03,739][1157819] Updated weights for policy 0, policy_version 76048 (0.0006) [2024-09-30 01:42:04,295][1157819] Updated weights for policy 0, policy_version 76058 (0.0006) [2024-09-30 01:42:04,862][1157819] Updated weights for policy 0, policy_version 76068 (0.0006) [2024-09-30 01:42:05,382][1157819] Updated weights for policy 0, policy_version 76078 (0.0006) [2024-09-30 01:42:05,466][1157520] Fps is (10 sec: 74547.3, 60 sec: 76458.5, 300 sec: 79781.7). Total num frames: 311619584. Throughput: 0: 18723.2. Samples: 67892660. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:42:05,466][1157520] Avg episode reward: [(0, '52.862')] [2024-09-30 01:42:05,472][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000076079_311619584.pth... [2024-09-30 01:42:05,536][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000071400_292454400.pth [2024-09-30 01:42:05,998][1157819] Updated weights for policy 0, policy_version 76088 (0.0006) [2024-09-30 01:42:06,556][1157819] Updated weights for policy 0, policy_version 76098 (0.0006) [2024-09-30 01:42:07,124][1157819] Updated weights for policy 0, policy_version 76108 (0.0006) [2024-09-30 01:42:07,723][1157819] Updated weights for policy 0, policy_version 76118 (0.0006) [2024-09-30 01:42:08,287][1157819] Updated weights for policy 0, policy_version 76128 (0.0006) [2024-09-30 01:42:08,861][1157819] Updated weights for policy 0, policy_version 76138 (0.0006) [2024-09-30 01:42:09,445][1157819] Updated weights for policy 0, policy_version 76148 (0.0006) [2024-09-30 01:42:10,035][1157819] Updated weights for policy 0, policy_version 76158 (0.0006) [2024-09-30 01:42:10,466][1157520] Fps is (10 sec: 72498.6, 60 sec: 75229.5, 300 sec: 79670.6). Total num frames: 311971840. Throughput: 0: 18770.0. Samples: 67946404. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 01:42:10,466][1157520] Avg episode reward: [(0, '54.524')] [2024-09-30 01:42:10,609][1157819] Updated weights for policy 0, policy_version 76168 (0.0006) [2024-09-30 01:42:11,176][1157819] Updated weights for policy 0, policy_version 76178 (0.0006) [2024-09-30 01:42:11,764][1157819] Updated weights for policy 0, policy_version 76188 (0.0006) [2024-09-30 01:42:12,340][1157819] Updated weights for policy 0, policy_version 76198 (0.0006) [2024-09-30 01:42:12,928][1157819] Updated weights for policy 0, policy_version 76208 (0.0006) [2024-09-30 01:42:13,501][1157819] Updated weights for policy 0, policy_version 76218 (0.0006) [2024-09-30 01:42:14,096][1157819] Updated weights for policy 0, policy_version 76228 (0.0006) [2024-09-30 01:42:14,663][1157819] Updated weights for policy 0, policy_version 76238 (0.0006) [2024-09-30 01:42:15,245][1157819] Updated weights for policy 0, policy_version 76248 (0.0006) [2024-09-30 01:42:15,466][1157520] Fps is (10 sec: 70451.3, 60 sec: 74342.3, 300 sec: 79573.5). Total num frames: 312324096. Throughput: 0: 18506.2. Samples: 68052312. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:42:15,466][1157520] Avg episode reward: [(0, '53.925')] [2024-09-30 01:42:15,814][1157819] Updated weights for policy 0, policy_version 76258 (0.0006) [2024-09-30 01:42:16,392][1157819] Updated weights for policy 0, policy_version 76268 (0.0006) [2024-09-30 01:42:16,953][1157819] Updated weights for policy 0, policy_version 76278 (0.0006) [2024-09-30 01:42:17,534][1157819] Updated weights for policy 0, policy_version 76288 (0.0006) [2024-09-30 01:42:18,114][1157819] Updated weights for policy 0, policy_version 76298 (0.0006) [2024-09-30 01:42:18,677][1157819] Updated weights for policy 0, policy_version 76308 (0.0006) [2024-09-30 01:42:19,221][1157819] Updated weights for policy 0, policy_version 76318 (0.0006) [2024-09-30 01:42:19,766][1157819] Updated weights for policy 0, policy_version 76328 (0.0006) [2024-09-30 01:42:20,277][1157819] Updated weights for policy 0, policy_version 76338 (0.0006) [2024-09-30 01:42:20,466][1157520] Fps is (10 sec: 71681.2, 60 sec: 74069.3, 300 sec: 79518.0). Total num frames: 312688640. Throughput: 0: 18278.4. Samples: 68160572. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:42:20,466][1157520] Avg episode reward: [(0, '54.049')] [2024-09-30 01:42:20,902][1157819] Updated weights for policy 0, policy_version 76348 (0.0006) [2024-09-30 01:42:21,434][1157819] Updated weights for policy 0, policy_version 76358 (0.0006) [2024-09-30 01:42:22,057][1157819] Updated weights for policy 0, policy_version 76368 (0.0006) [2024-09-30 01:42:22,183][1157736] Signal inference workers to stop experience collection... (4950 times) [2024-09-30 01:42:22,184][1157736] Signal inference workers to resume experience collection... (4950 times) [2024-09-30 01:42:22,187][1157819] InferenceWorker_p0-w0: stopping experience collection (4950 times) [2024-09-30 01:42:22,187][1157819] InferenceWorker_p0-w0: resuming experience collection (4950 times) [2024-09-30 01:42:22,600][1157819] Updated weights for policy 0, policy_version 76378 (0.0006) [2024-09-30 01:42:23,197][1157819] Updated weights for policy 0, policy_version 76388 (0.0006) [2024-09-30 01:42:23,726][1157819] Updated weights for policy 0, policy_version 76398 (0.0006) [2024-09-30 01:42:24,250][1157819] Updated weights for policy 0, policy_version 76408 (0.0006) [2024-09-30 01:42:24,809][1157819] Updated weights for policy 0, policy_version 76418 (0.0006) [2024-09-30 01:42:25,382][1157819] Updated weights for policy 0, policy_version 76428 (0.0006) [2024-09-30 01:42:25,466][1157520] Fps is (10 sec: 72908.9, 60 sec: 74069.3, 300 sec: 79379.1). Total num frames: 313053184. Throughput: 0: 18152.5. Samples: 68214188. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:42:25,466][1157520] Avg episode reward: [(0, '54.355')] [2024-09-30 01:42:25,959][1157819] Updated weights for policy 0, policy_version 76438 (0.0006) [2024-09-30 01:42:26,526][1157819] Updated weights for policy 0, policy_version 76448 (0.0006) [2024-09-30 01:42:27,126][1157819] Updated weights for policy 0, policy_version 76458 (0.0006) [2024-09-30 01:42:27,761][1157819] Updated weights for policy 0, policy_version 76468 (0.0006) [2024-09-30 01:42:28,355][1157819] Updated weights for policy 0, policy_version 76478 (0.0006) [2024-09-30 01:42:28,936][1157819] Updated weights for policy 0, policy_version 76488 (0.0006) [2024-09-30 01:42:29,544][1157819] Updated weights for policy 0, policy_version 76498 (0.0006) [2024-09-30 01:42:30,104][1157819] Updated weights for policy 0, policy_version 76508 (0.0006) [2024-09-30 01:42:30,466][1157520] Fps is (10 sec: 71269.8, 60 sec: 73181.8, 300 sec: 79198.5). Total num frames: 313401344. Throughput: 0: 18032.2. Samples: 68321320. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:42:30,466][1157520] Avg episode reward: [(0, '51.909')] [2024-09-30 01:42:30,690][1157819] Updated weights for policy 0, policy_version 76518 (0.0006) [2024-09-30 01:42:31,206][1157819] Updated weights for policy 0, policy_version 76528 (0.0006) [2024-09-30 01:42:31,804][1157819] Updated weights for policy 0, policy_version 76538 (0.0006) [2024-09-30 01:42:32,368][1157819] Updated weights for policy 0, policy_version 76548 (0.0006) [2024-09-30 01:42:32,938][1157819] Updated weights for policy 0, policy_version 76558 (0.0006) [2024-09-30 01:42:33,538][1157819] Updated weights for policy 0, policy_version 76568 (0.0006) [2024-09-30 01:42:34,150][1157819] Updated weights for policy 0, policy_version 76578 (0.0006) [2024-09-30 01:42:34,747][1157819] Updated weights for policy 0, policy_version 76588 (0.0006) [2024-09-30 01:42:35,312][1157819] Updated weights for policy 0, policy_version 76598 (0.0006) [2024-09-30 01:42:35,466][1157520] Fps is (10 sec: 70041.8, 60 sec: 72499.2, 300 sec: 79032.0). Total num frames: 313753600. Throughput: 0: 18050.2. Samples: 68427520. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:42:35,466][1157520] Avg episode reward: [(0, '53.895')] [2024-09-30 01:42:35,846][1157819] Updated weights for policy 0, policy_version 76608 (0.0006) [2024-09-30 01:42:36,466][1157819] Updated weights for policy 0, policy_version 76618 (0.0006) [2024-09-30 01:42:37,034][1157819] Updated weights for policy 0, policy_version 76628 (0.0006) [2024-09-30 01:42:37,627][1157819] Updated weights for policy 0, policy_version 76638 (0.0006) [2024-09-30 01:42:38,216][1157819] Updated weights for policy 0, policy_version 76648 (0.0006) [2024-09-30 01:42:38,768][1157819] Updated weights for policy 0, policy_version 76658 (0.0006) [2024-09-30 01:42:39,325][1157819] Updated weights for policy 0, policy_version 76668 (0.0006) [2024-09-30 01:42:39,868][1157819] Updated weights for policy 0, policy_version 76678 (0.0006) [2024-09-30 01:42:40,330][1157736] Signal inference workers to stop experience collection... (5000 times) [2024-09-30 01:42:40,331][1157736] Signal inference workers to resume experience collection... (5000 times) [2024-09-30 01:42:40,335][1157819] InferenceWorker_p0-w0: stopping experience collection (5000 times) [2024-09-30 01:42:40,335][1157819] InferenceWorker_p0-w0: resuming experience collection (5000 times) [2024-09-30 01:42:40,466][1157520] Fps is (10 sec: 70860.9, 60 sec: 71884.7, 300 sec: 78851.4). Total num frames: 314109952. Throughput: 0: 18022.2. Samples: 68480432. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:42:40,466][1157520] Avg episode reward: [(0, '53.154')] [2024-09-30 01:42:40,483][1157819] Updated weights for policy 0, policy_version 76688 (0.0006) [2024-09-30 01:42:41,066][1157819] Updated weights for policy 0, policy_version 76698 (0.0006) [2024-09-30 01:42:41,625][1157819] Updated weights for policy 0, policy_version 76708 (0.0006) [2024-09-30 01:42:42,212][1157819] Updated weights for policy 0, policy_version 76718 (0.0006) [2024-09-30 01:42:42,722][1157819] Updated weights for policy 0, policy_version 76728 (0.0006) [2024-09-30 01:42:43,255][1157819] Updated weights for policy 0, policy_version 76738 (0.0006) [2024-09-30 01:42:43,805][1157819] Updated weights for policy 0, policy_version 76748 (0.0006) [2024-09-30 01:42:44,356][1157819] Updated weights for policy 0, policy_version 76758 (0.0006) [2024-09-30 01:42:44,850][1157819] Updated weights for policy 0, policy_version 76768 (0.0006) [2024-09-30 01:42:45,352][1157819] Updated weights for policy 0, policy_version 76778 (0.0006) [2024-09-30 01:42:45,466][1157520] Fps is (10 sec: 73318.1, 60 sec: 72226.1, 300 sec: 78768.1). Total num frames: 314486784. Throughput: 0: 17952.4. Samples: 68589596. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:42:45,466][1157520] Avg episode reward: [(0, '53.139')] [2024-09-30 01:42:45,864][1157819] Updated weights for policy 0, policy_version 76788 (0.0007) [2024-09-30 01:42:46,383][1157819] Updated weights for policy 0, policy_version 76798 (0.0006) [2024-09-30 01:42:46,907][1157819] Updated weights for policy 0, policy_version 76808 (0.0006) [2024-09-30 01:42:47,418][1157819] Updated weights for policy 0, policy_version 76818 (0.0006) [2024-09-30 01:42:47,918][1157819] Updated weights for policy 0, policy_version 76828 (0.0006) [2024-09-30 01:42:48,406][1157819] Updated weights for policy 0, policy_version 76838 (0.0006) [2024-09-30 01:42:48,946][1157819] Updated weights for policy 0, policy_version 76848 (0.0006) [2024-09-30 01:42:49,489][1157819] Updated weights for policy 0, policy_version 76858 (0.0006) [2024-09-30 01:42:49,985][1157819] Updated weights for policy 0, policy_version 76868 (0.0006) [2024-09-30 01:42:50,466][1157520] Fps is (10 sec: 77414.3, 60 sec: 72977.0, 300 sec: 78768.1). Total num frames: 314884096. Throughput: 0: 18133.2. Samples: 68708656. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:42:50,466][1157520] Avg episode reward: [(0, '53.614')] [2024-09-30 01:42:50,533][1157819] Updated weights for policy 0, policy_version 76878 (0.0006) [2024-09-30 01:42:51,046][1157819] Updated weights for policy 0, policy_version 76888 (0.0006) [2024-09-30 01:42:51,554][1157819] Updated weights for policy 0, policy_version 76898 (0.0006) [2024-09-30 01:42:52,095][1157819] Updated weights for policy 0, policy_version 76908 (0.0006) [2024-09-30 01:42:52,645][1157819] Updated weights for policy 0, policy_version 76918 (0.0007) [2024-09-30 01:42:53,186][1157819] Updated weights for policy 0, policy_version 76928 (0.0006) [2024-09-30 01:42:53,743][1157819] Updated weights for policy 0, policy_version 76938 (0.0006) [2024-09-30 01:42:54,265][1157819] Updated weights for policy 0, policy_version 76948 (0.0006) [2024-09-30 01:42:54,812][1157819] Updated weights for policy 0, policy_version 76958 (0.0006) [2024-09-30 01:42:55,366][1157819] Updated weights for policy 0, policy_version 76968 (0.0006) [2024-09-30 01:42:55,466][1157520] Fps is (10 sec: 77823.8, 60 sec: 73181.9, 300 sec: 78698.7). Total num frames: 315265024. Throughput: 0: 18228.5. Samples: 68766684. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:42:55,466][1157520] Avg episode reward: [(0, '54.339')] [2024-09-30 01:42:55,911][1157819] Updated weights for policy 0, policy_version 76978 (0.0006) [2024-09-30 01:42:56,444][1157819] Updated weights for policy 0, policy_version 76988 (0.0006) [2024-09-30 01:42:57,010][1157819] Updated weights for policy 0, policy_version 76998 (0.0006) [2024-09-30 01:42:57,564][1157819] Updated weights for policy 0, policy_version 77008 (0.0007) [2024-09-30 01:42:58,081][1157819] Updated weights for policy 0, policy_version 77018 (0.0006) [2024-09-30 01:42:58,640][1157819] Updated weights for policy 0, policy_version 77028 (0.0007) [2024-09-30 01:42:59,179][1157819] Updated weights for policy 0, policy_version 77038 (0.0006) [2024-09-30 01:42:59,711][1157819] Updated weights for policy 0, policy_version 77048 (0.0006) [2024-09-30 01:43:00,276][1157819] Updated weights for policy 0, policy_version 77058 (0.0006) [2024-09-30 01:43:00,466][1157520] Fps is (10 sec: 75776.0, 60 sec: 73250.1, 300 sec: 78601.5). Total num frames: 315641856. Throughput: 0: 18377.0. Samples: 68879276. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:43:00,466][1157520] Avg episode reward: [(0, '50.786')] [2024-09-30 01:43:00,796][1157819] Updated weights for policy 0, policy_version 77068 (0.0006) [2024-09-30 01:43:01,344][1157819] Updated weights for policy 0, policy_version 77078 (0.0006) [2024-09-30 01:43:01,850][1157819] Updated weights for policy 0, policy_version 77088 (0.0006) [2024-09-30 01:43:02,369][1157819] Updated weights for policy 0, policy_version 77098 (0.0006) [2024-09-30 01:43:02,870][1157819] Updated weights for policy 0, policy_version 77108 (0.0007) [2024-09-30 01:43:03,431][1157819] Updated weights for policy 0, policy_version 77118 (0.0006) [2024-09-30 01:43:03,921][1157819] Updated weights for policy 0, policy_version 77128 (0.0006) [2024-09-30 01:43:04,420][1157819] Updated weights for policy 0, policy_version 77138 (0.0006) [2024-09-30 01:43:04,921][1157819] Updated weights for policy 0, policy_version 77148 (0.0006) [2024-09-30 01:43:05,466][1157520] Fps is (10 sec: 77005.2, 60 sec: 73591.5, 300 sec: 78601.5). Total num frames: 316035072. Throughput: 0: 18572.2. Samples: 68996320. Policy #0 lag: (min: 0.0, avg: 1.7, max: 4.0) [2024-09-30 01:43:05,466][1157520] Avg episode reward: [(0, '51.806')] [2024-09-30 01:43:05,468][1157819] Updated weights for policy 0, policy_version 77158 (0.0006) [2024-09-30 01:43:06,027][1157819] Updated weights for policy 0, policy_version 77168 (0.0006) [2024-09-30 01:43:06,523][1157819] Updated weights for policy 0, policy_version 77178 (0.0006) [2024-09-30 01:43:07,029][1157819] Updated weights for policy 0, policy_version 77188 (0.0006) [2024-09-30 01:43:07,533][1157819] Updated weights for policy 0, policy_version 77198 (0.0006) [2024-09-30 01:43:08,032][1157819] Updated weights for policy 0, policy_version 77208 (0.0006) [2024-09-30 01:43:08,558][1157819] Updated weights for policy 0, policy_version 77218 (0.0006) [2024-09-30 01:43:09,079][1157819] Updated weights for policy 0, policy_version 77228 (0.0006) [2024-09-30 01:43:09,597][1157819] Updated weights for policy 0, policy_version 77238 (0.0006) [2024-09-30 01:43:10,112][1157819] Updated weights for policy 0, policy_version 77248 (0.0006) [2024-09-30 01:43:10,466][1157520] Fps is (10 sec: 79462.9, 60 sec: 74410.8, 300 sec: 78615.4). Total num frames: 316436480. Throughput: 0: 18704.3. Samples: 69055880. Policy #0 lag: (min: 0.0, avg: 1.7, max: 4.0) [2024-09-30 01:43:10,466][1157520] Avg episode reward: [(0, '52.617')] [2024-09-30 01:43:10,630][1157819] Updated weights for policy 0, policy_version 77258 (0.0006) [2024-09-30 01:43:11,128][1157819] Updated weights for policy 0, policy_version 77268 (0.0006) [2024-09-30 01:43:11,619][1157819] Updated weights for policy 0, policy_version 77278 (0.0006) [2024-09-30 01:43:12,130][1157819] Updated weights for policy 0, policy_version 77288 (0.0006) [2024-09-30 01:43:12,640][1157819] Updated weights for policy 0, policy_version 77298 (0.0006) [2024-09-30 01:43:13,152][1157819] Updated weights for policy 0, policy_version 77308 (0.0006) [2024-09-30 01:43:13,669][1157819] Updated weights for policy 0, policy_version 77318 (0.0006) [2024-09-30 01:43:14,176][1157819] Updated weights for policy 0, policy_version 77328 (0.0006) [2024-09-30 01:43:14,669][1157819] Updated weights for policy 0, policy_version 77338 (0.0006) [2024-09-30 01:43:15,185][1157819] Updated weights for policy 0, policy_version 77348 (0.0006) [2024-09-30 01:43:15,466][1157520] Fps is (10 sec: 80280.8, 60 sec: 75229.8, 300 sec: 78559.8). Total num frames: 316837888. Throughput: 0: 18997.2. Samples: 69176196. Policy #0 lag: (min: 0.0, avg: 1.7, max: 4.0) [2024-09-30 01:43:15,466][1157520] Avg episode reward: [(0, '55.269')] [2024-09-30 01:43:15,689][1157819] Updated weights for policy 0, policy_version 77358 (0.0006) [2024-09-30 01:43:16,181][1157819] Updated weights for policy 0, policy_version 77368 (0.0006) [2024-09-30 01:43:16,692][1157819] Updated weights for policy 0, policy_version 77378 (0.0006) [2024-09-30 01:43:17,224][1157819] Updated weights for policy 0, policy_version 77388 (0.0006) [2024-09-30 01:43:17,728][1157819] Updated weights for policy 0, policy_version 77398 (0.0006) [2024-09-30 01:43:18,225][1157819] Updated weights for policy 0, policy_version 77408 (0.0006) [2024-09-30 01:43:18,723][1157819] Updated weights for policy 0, policy_version 77418 (0.0006) [2024-09-30 01:43:19,230][1157819] Updated weights for policy 0, policy_version 77428 (0.0006) [2024-09-30 01:43:19,727][1157819] Updated weights for policy 0, policy_version 77438 (0.0006) [2024-09-30 01:43:20,228][1157819] Updated weights for policy 0, policy_version 77448 (0.0006) [2024-09-30 01:43:20,466][1157520] Fps is (10 sec: 80691.3, 60 sec: 75912.5, 300 sec: 78490.5). Total num frames: 317243392. Throughput: 0: 19332.9. Samples: 69297500. Policy #0 lag: (min: 0.0, avg: 1.7, max: 4.0) [2024-09-30 01:43:20,466][1157520] Avg episode reward: [(0, '55.290')] [2024-09-30 01:43:20,738][1157819] Updated weights for policy 0, policy_version 77458 (0.0006) [2024-09-30 01:43:21,257][1157819] Updated weights for policy 0, policy_version 77468 (0.0006) [2024-09-30 01:43:21,774][1157819] Updated weights for policy 0, policy_version 77478 (0.0006) [2024-09-30 01:43:22,267][1157819] Updated weights for policy 0, policy_version 77488 (0.0006) [2024-09-30 01:43:22,745][1157819] Updated weights for policy 0, policy_version 77498 (0.0006) [2024-09-30 01:43:23,260][1157819] Updated weights for policy 0, policy_version 77508 (0.0006) [2024-09-30 01:43:23,760][1157819] Updated weights for policy 0, policy_version 77518 (0.0006) [2024-09-30 01:43:24,262][1157819] Updated weights for policy 0, policy_version 77528 (0.0006) [2024-09-30 01:43:24,781][1157819] Updated weights for policy 0, policy_version 77538 (0.0006) [2024-09-30 01:43:25,276][1157819] Updated weights for policy 0, policy_version 77548 (0.0006) [2024-09-30 01:43:25,466][1157520] Fps is (10 sec: 81101.5, 60 sec: 76595.2, 300 sec: 78379.4). Total num frames: 317648896. Throughput: 0: 19517.9. Samples: 69358736. Policy #0 lag: (min: 0.0, avg: 1.7, max: 4.0) [2024-09-30 01:43:25,466][1157520] Avg episode reward: [(0, '54.369')] [2024-09-30 01:43:25,792][1157819] Updated weights for policy 0, policy_version 77558 (0.0006) [2024-09-30 01:43:26,320][1157819] Updated weights for policy 0, policy_version 77568 (0.0006) [2024-09-30 01:43:26,810][1157819] Updated weights for policy 0, policy_version 77578 (0.0006) [2024-09-30 01:43:27,327][1157819] Updated weights for policy 0, policy_version 77588 (0.0006) [2024-09-30 01:43:27,843][1157819] Updated weights for policy 0, policy_version 77598 (0.0006) [2024-09-30 01:43:28,374][1157819] Updated weights for policy 0, policy_version 77608 (0.0006) [2024-09-30 01:43:28,896][1157819] Updated weights for policy 0, policy_version 77618 (0.0006) [2024-09-30 01:43:29,153][1157736] Signal inference workers to stop experience collection... (5050 times) [2024-09-30 01:43:29,159][1157819] InferenceWorker_p0-w0: stopping experience collection (5050 times) [2024-09-30 01:43:29,159][1157736] Signal inference workers to resume experience collection... (5050 times) [2024-09-30 01:43:29,163][1157819] InferenceWorker_p0-w0: resuming experience collection (5050 times) [2024-09-30 01:43:29,403][1157819] Updated weights for policy 0, policy_version 77628 (0.0006) [2024-09-30 01:43:29,937][1157819] Updated weights for policy 0, policy_version 77638 (0.0006) [2024-09-30 01:43:30,449][1157819] Updated weights for policy 0, policy_version 77648 (0.0006) [2024-09-30 01:43:30,466][1157520] Fps is (10 sec: 80281.8, 60 sec: 77414.5, 300 sec: 78226.6). Total num frames: 318046208. Throughput: 0: 19757.3. Samples: 69478672. Policy #0 lag: (min: 0.0, avg: 1.7, max: 4.0) [2024-09-30 01:43:30,466][1157520] Avg episode reward: [(0, '52.686')] [2024-09-30 01:43:30,938][1157819] Updated weights for policy 0, policy_version 77658 (0.0006) [2024-09-30 01:43:31,443][1157819] Updated weights for policy 0, policy_version 77668 (0.0006) [2024-09-30 01:43:31,950][1157819] Updated weights for policy 0, policy_version 77678 (0.0006) [2024-09-30 01:43:32,499][1157819] Updated weights for policy 0, policy_version 77688 (0.0006) [2024-09-30 01:43:33,006][1157819] Updated weights for policy 0, policy_version 77698 (0.0006) [2024-09-30 01:43:33,535][1157819] Updated weights for policy 0, policy_version 77708 (0.0006) [2024-09-30 01:43:34,032][1157819] Updated weights for policy 0, policy_version 77718 (0.0006) [2024-09-30 01:43:34,537][1157819] Updated weights for policy 0, policy_version 77728 (0.0006) [2024-09-30 01:43:35,051][1157819] Updated weights for policy 0, policy_version 77738 (0.0006) [2024-09-30 01:43:35,466][1157520] Fps is (10 sec: 79872.3, 60 sec: 78233.6, 300 sec: 78115.6). Total num frames: 318447616. Throughput: 0: 19770.3. Samples: 69598316. Policy #0 lag: (min: 0.0, avg: 1.7, max: 4.0) [2024-09-30 01:43:35,466][1157520] Avg episode reward: [(0, '52.857')] [2024-09-30 01:43:35,555][1157819] Updated weights for policy 0, policy_version 77748 (0.0006) [2024-09-30 01:43:36,094][1157819] Updated weights for policy 0, policy_version 77758 (0.0006) [2024-09-30 01:43:36,587][1157819] Updated weights for policy 0, policy_version 77768 (0.0006) [2024-09-30 01:43:37,076][1157819] Updated weights for policy 0, policy_version 77778 (0.0006) [2024-09-30 01:43:37,612][1157819] Updated weights for policy 0, policy_version 77788 (0.0006) [2024-09-30 01:43:38,117][1157819] Updated weights for policy 0, policy_version 77798 (0.0006) [2024-09-30 01:43:38,626][1157819] Updated weights for policy 0, policy_version 77808 (0.0006) [2024-09-30 01:43:39,144][1157819] Updated weights for policy 0, policy_version 77818 (0.0006) [2024-09-30 01:43:39,673][1157819] Updated weights for policy 0, policy_version 77828 (0.0006) [2024-09-30 01:43:40,174][1157819] Updated weights for policy 0, policy_version 77838 (0.0006) [2024-09-30 01:43:40,466][1157520] Fps is (10 sec: 79871.7, 60 sec: 78916.3, 300 sec: 77976.7). Total num frames: 318844928. Throughput: 0: 19814.8. Samples: 69658348. Policy #0 lag: (min: 0.0, avg: 1.7, max: 4.0) [2024-09-30 01:43:40,466][1157520] Avg episode reward: [(0, '53.672')] [2024-09-30 01:43:40,681][1157819] Updated weights for policy 0, policy_version 77848 (0.0006) [2024-09-30 01:43:41,173][1157819] Updated weights for policy 0, policy_version 77858 (0.0006) [2024-09-30 01:43:41,690][1157819] Updated weights for policy 0, policy_version 77868 (0.0006) [2024-09-30 01:43:42,237][1157819] Updated weights for policy 0, policy_version 77878 (0.0006) [2024-09-30 01:43:42,760][1157819] Updated weights for policy 0, policy_version 77888 (0.0006) [2024-09-30 01:43:43,275][1157819] Updated weights for policy 0, policy_version 77898 (0.0006) [2024-09-30 01:43:43,790][1157819] Updated weights for policy 0, policy_version 77908 (0.0006) [2024-09-30 01:43:44,292][1157819] Updated weights for policy 0, policy_version 77918 (0.0006) [2024-09-30 01:43:44,809][1157819] Updated weights for policy 0, policy_version 77928 (0.0006) [2024-09-30 01:43:45,366][1157819] Updated weights for policy 0, policy_version 77938 (0.0006) [2024-09-30 01:43:45,466][1157520] Fps is (10 sec: 79462.0, 60 sec: 79257.6, 300 sec: 77837.8). Total num frames: 319242240. Throughput: 0: 19967.7. Samples: 69777820. Policy #0 lag: (min: 0.0, avg: 1.7, max: 4.0) [2024-09-30 01:43:45,466][1157520] Avg episode reward: [(0, '54.538')] [2024-09-30 01:43:45,868][1157819] Updated weights for policy 0, policy_version 77948 (0.0006) [2024-09-30 01:43:46,374][1157819] Updated weights for policy 0, policy_version 77958 (0.0006) [2024-09-30 01:43:46,884][1157819] Updated weights for policy 0, policy_version 77968 (0.0006) [2024-09-30 01:43:47,376][1157819] Updated weights for policy 0, policy_version 77978 (0.0006) [2024-09-30 01:43:47,871][1157819] Updated weights for policy 0, policy_version 77988 (0.0006) [2024-09-30 01:43:48,402][1157819] Updated weights for policy 0, policy_version 77998 (0.0006) [2024-09-30 01:43:48,936][1157819] Updated weights for policy 0, policy_version 78008 (0.0006) [2024-09-30 01:43:49,462][1157819] Updated weights for policy 0, policy_version 78018 (0.0006) [2024-09-30 01:43:49,960][1157819] Updated weights for policy 0, policy_version 78028 (0.0006) [2024-09-30 01:43:50,466][1157520] Fps is (10 sec: 79463.1, 60 sec: 79257.8, 300 sec: 77699.0). Total num frames: 319639552. Throughput: 0: 20022.6. Samples: 69897336. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:43:50,466][1157520] Avg episode reward: [(0, '55.323')] [2024-09-30 01:43:50,466][1157819] Updated weights for policy 0, policy_version 78038 (0.0006) [2024-09-30 01:43:50,987][1157819] Updated weights for policy 0, policy_version 78048 (0.0006) [2024-09-30 01:43:51,499][1157819] Updated weights for policy 0, policy_version 78058 (0.0006) [2024-09-30 01:43:52,050][1157819] Updated weights for policy 0, policy_version 78068 (0.0006) [2024-09-30 01:43:52,543][1157819] Updated weights for policy 0, policy_version 78078 (0.0006) [2024-09-30 01:43:53,042][1157819] Updated weights for policy 0, policy_version 78088 (0.0006) [2024-09-30 01:43:53,563][1157819] Updated weights for policy 0, policy_version 78098 (0.0006) [2024-09-30 01:43:54,090][1157819] Updated weights for policy 0, policy_version 78108 (0.0006) [2024-09-30 01:43:54,601][1157819] Updated weights for policy 0, policy_version 78118 (0.0006) [2024-09-30 01:43:55,116][1157819] Updated weights for policy 0, policy_version 78128 (0.0006) [2024-09-30 01:43:55,466][1157520] Fps is (10 sec: 79463.4, 60 sec: 79530.9, 300 sec: 77560.2). Total num frames: 320036864. Throughput: 0: 20021.0. Samples: 69956824. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:43:55,466][1157520] Avg episode reward: [(0, '54.078')] [2024-09-30 01:43:55,606][1157819] Updated weights for policy 0, policy_version 78138 (0.0006) [2024-09-30 01:43:56,130][1157819] Updated weights for policy 0, policy_version 78148 (0.0006) [2024-09-30 01:43:56,636][1157819] Updated weights for policy 0, policy_version 78158 (0.0006) [2024-09-30 01:43:57,149][1157819] Updated weights for policy 0, policy_version 78168 (0.0006) [2024-09-30 01:43:57,669][1157819] Updated weights for policy 0, policy_version 78178 (0.0006) [2024-09-30 01:43:58,184][1157819] Updated weights for policy 0, policy_version 78188 (0.0006) [2024-09-30 01:43:58,707][1157819] Updated weights for policy 0, policy_version 78198 (0.0006) [2024-09-30 01:43:59,226][1157819] Updated weights for policy 0, policy_version 78208 (0.0006) [2024-09-30 01:43:59,751][1157819] Updated weights for policy 0, policy_version 78218 (0.0006) [2024-09-30 01:44:00,249][1157819] Updated weights for policy 0, policy_version 78228 (0.0006) [2024-09-30 01:44:00,466][1157520] Fps is (10 sec: 79871.9, 60 sec: 79940.4, 300 sec: 77435.2). Total num frames: 320438272. Throughput: 0: 20002.9. Samples: 70076324. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:44:00,466][1157520] Avg episode reward: [(0, '54.358')] [2024-09-30 01:44:00,755][1157819] Updated weights for policy 0, policy_version 78238 (0.0006) [2024-09-30 01:44:01,266][1157819] Updated weights for policy 0, policy_version 78248 (0.0006) [2024-09-30 01:44:01,761][1157819] Updated weights for policy 0, policy_version 78258 (0.0006) [2024-09-30 01:44:02,275][1157819] Updated weights for policy 0, policy_version 78268 (0.0006) [2024-09-30 01:44:02,795][1157819] Updated weights for policy 0, policy_version 78278 (0.0006) [2024-09-30 01:44:03,300][1157819] Updated weights for policy 0, policy_version 78288 (0.0006) [2024-09-30 01:44:03,788][1157819] Updated weights for policy 0, policy_version 78298 (0.0006) [2024-09-30 01:44:04,284][1157819] Updated weights for policy 0, policy_version 78308 (0.0006) [2024-09-30 01:44:04,770][1157819] Updated weights for policy 0, policy_version 78318 (0.0006) [2024-09-30 01:44:05,263][1157819] Updated weights for policy 0, policy_version 78328 (0.0006) [2024-09-30 01:44:05,466][1157520] Fps is (10 sec: 81100.7, 60 sec: 80213.5, 300 sec: 77379.7). Total num frames: 320847872. Throughput: 0: 20008.0. Samples: 70197860. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:44:05,466][1157520] Avg episode reward: [(0, '53.892')] [2024-09-30 01:44:05,471][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000078332_320847872.pth... [2024-09-30 01:44:05,517][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000073778_302194688.pth [2024-09-30 01:44:05,762][1157819] Updated weights for policy 0, policy_version 78338 (0.0006) [2024-09-30 01:44:06,238][1157819] Updated weights for policy 0, policy_version 78348 (0.0006) [2024-09-30 01:44:06,719][1157819] Updated weights for policy 0, policy_version 78358 (0.0006) [2024-09-30 01:44:07,213][1157819] Updated weights for policy 0, policy_version 78368 (0.0006) [2024-09-30 01:44:07,703][1157819] Updated weights for policy 0, policy_version 78378 (0.0006) [2024-09-30 01:44:08,199][1157819] Updated weights for policy 0, policy_version 78388 (0.0006) [2024-09-30 01:44:08,243][1157736] Signal inference workers to stop experience collection... (5100 times) [2024-09-30 01:44:08,246][1157736] Signal inference workers to resume experience collection... (5100 times) [2024-09-30 01:44:08,250][1157819] InferenceWorker_p0-w0: stopping experience collection (5100 times) [2024-09-30 01:44:08,250][1157819] InferenceWorker_p0-w0: resuming experience collection (5100 times) [2024-09-30 01:44:08,683][1157819] Updated weights for policy 0, policy_version 78398 (0.0006) [2024-09-30 01:44:09,216][1157819] Updated weights for policy 0, policy_version 78408 (0.0006) [2024-09-30 01:44:09,726][1157819] Updated weights for policy 0, policy_version 78418 (0.0006) [2024-09-30 01:44:10,254][1157819] Updated weights for policy 0, policy_version 78428 (0.0006) [2024-09-30 01:44:10,466][1157520] Fps is (10 sec: 81920.2, 60 sec: 80350.0, 300 sec: 77338.0). Total num frames: 321257472. Throughput: 0: 20050.3. Samples: 70260996. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:44:10,466][1157520] Avg episode reward: [(0, '54.110')] [2024-09-30 01:44:10,751][1157819] Updated weights for policy 0, policy_version 78438 (0.0006) [2024-09-30 01:44:11,303][1157819] Updated weights for policy 0, policy_version 78448 (0.0006) [2024-09-30 01:44:11,816][1157819] Updated weights for policy 0, policy_version 78458 (0.0006) [2024-09-30 01:44:12,308][1157819] Updated weights for policy 0, policy_version 78468 (0.0006) [2024-09-30 01:44:12,816][1157819] Updated weights for policy 0, policy_version 78478 (0.0006) [2024-09-30 01:44:13,326][1157819] Updated weights for policy 0, policy_version 78488 (0.0006) [2024-09-30 01:44:13,844][1157819] Updated weights for policy 0, policy_version 78498 (0.0006) [2024-09-30 01:44:14,387][1157819] Updated weights for policy 0, policy_version 78508 (0.0006) [2024-09-30 01:44:14,879][1157819] Updated weights for policy 0, policy_version 78518 (0.0006) [2024-09-30 01:44:15,378][1157819] Updated weights for policy 0, policy_version 78528 (0.0006) [2024-09-30 01:44:15,466][1157520] Fps is (10 sec: 80691.5, 60 sec: 80281.9, 300 sec: 77282.5). Total num frames: 321654784. Throughput: 0: 20044.9. Samples: 70380692. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:44:15,466][1157520] Avg episode reward: [(0, '56.635')] [2024-09-30 01:44:15,469][1157736] Saving new best policy, reward=56.635! [2024-09-30 01:44:15,868][1157819] Updated weights for policy 0, policy_version 78538 (0.0006) [2024-09-30 01:44:16,362][1157819] Updated weights for policy 0, policy_version 78548 (0.0006) [2024-09-30 01:44:16,857][1157819] Updated weights for policy 0, policy_version 78558 (0.0006) [2024-09-30 01:44:17,358][1157819] Updated weights for policy 0, policy_version 78568 (0.0006) [2024-09-30 01:44:17,863][1157819] Updated weights for policy 0, policy_version 78578 (0.0006) [2024-09-30 01:44:18,340][1157819] Updated weights for policy 0, policy_version 78588 (0.0006) [2024-09-30 01:44:18,858][1157819] Updated weights for policy 0, policy_version 78598 (0.0006) [2024-09-30 01:44:19,362][1157819] Updated weights for policy 0, policy_version 78608 (0.0006) [2024-09-30 01:44:19,889][1157819] Updated weights for policy 0, policy_version 78618 (0.0006) [2024-09-30 01:44:20,382][1157819] Updated weights for policy 0, policy_version 78628 (0.0006) [2024-09-30 01:44:20,466][1157520] Fps is (10 sec: 80281.7, 60 sec: 80281.7, 300 sec: 77324.2). Total num frames: 322060288. Throughput: 0: 20100.3. Samples: 70502828. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:44:20,466][1157520] Avg episode reward: [(0, '52.670')] [2024-09-30 01:44:20,921][1157819] Updated weights for policy 0, policy_version 78638 (0.0006) [2024-09-30 01:44:21,447][1157819] Updated weights for policy 0, policy_version 78648 (0.0006) [2024-09-30 01:44:21,964][1157819] Updated weights for policy 0, policy_version 78658 (0.0006) [2024-09-30 01:44:22,466][1157819] Updated weights for policy 0, policy_version 78668 (0.0006) [2024-09-30 01:44:22,991][1157819] Updated weights for policy 0, policy_version 78678 (0.0006) [2024-09-30 01:44:23,495][1157819] Updated weights for policy 0, policy_version 78688 (0.0006) [2024-09-30 01:44:23,995][1157819] Updated weights for policy 0, policy_version 78698 (0.0006) [2024-09-30 01:44:24,532][1157819] Updated weights for policy 0, policy_version 78708 (0.0006) [2024-09-30 01:44:25,043][1157819] Updated weights for policy 0, policy_version 78718 (0.0006) [2024-09-30 01:44:25,466][1157520] Fps is (10 sec: 80691.1, 60 sec: 80213.5, 300 sec: 77338.0). Total num frames: 322461696. Throughput: 0: 20089.2. Samples: 70562360. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:44:25,466][1157520] Avg episode reward: [(0, '54.564')] [2024-09-30 01:44:25,581][1157819] Updated weights for policy 0, policy_version 78728 (0.0006) [2024-09-30 01:44:26,095][1157819] Updated weights for policy 0, policy_version 78738 (0.0006) [2024-09-30 01:44:26,650][1157819] Updated weights for policy 0, policy_version 78748 (0.0006) [2024-09-30 01:44:27,214][1157819] Updated weights for policy 0, policy_version 78758 (0.0006) [2024-09-30 01:44:27,743][1157819] Updated weights for policy 0, policy_version 78768 (0.0006) [2024-09-30 01:44:28,284][1157819] Updated weights for policy 0, policy_version 78778 (0.0006) [2024-09-30 01:44:28,840][1157819] Updated weights for policy 0, policy_version 78788 (0.0006) [2024-09-30 01:44:29,398][1157819] Updated weights for policy 0, policy_version 78798 (0.0006) [2024-09-30 01:44:29,961][1157819] Updated weights for policy 0, policy_version 78808 (0.0006) [2024-09-30 01:44:30,445][1157819] Updated weights for policy 0, policy_version 78818 (0.0006) [2024-09-30 01:44:30,466][1157520] Fps is (10 sec: 77823.7, 60 sec: 79872.0, 300 sec: 77338.1). Total num frames: 322838528. Throughput: 0: 19996.0. Samples: 70677636. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:44:30,466][1157520] Avg episode reward: [(0, '51.781')] [2024-09-30 01:44:30,934][1157819] Updated weights for policy 0, policy_version 78828 (0.0006) [2024-09-30 01:44:31,413][1157819] Updated weights for policy 0, policy_version 78838 (0.0006) [2024-09-30 01:44:31,902][1157819] Updated weights for policy 0, policy_version 78848 (0.0006) [2024-09-30 01:44:32,396][1157819] Updated weights for policy 0, policy_version 78858 (0.0006) [2024-09-30 01:44:32,956][1157819] Updated weights for policy 0, policy_version 78868 (0.0006) [2024-09-30 01:44:33,486][1157819] Updated weights for policy 0, policy_version 78878 (0.0006) [2024-09-30 01:44:34,030][1157819] Updated weights for policy 0, policy_version 78888 (0.0006) [2024-09-30 01:44:34,550][1157819] Updated weights for policy 0, policy_version 78898 (0.0006) [2024-09-30 01:44:35,104][1157819] Updated weights for policy 0, policy_version 78908 (0.0006) [2024-09-30 01:44:35,466][1157520] Fps is (10 sec: 77005.0, 60 sec: 79735.6, 300 sec: 77393.6). Total num frames: 323231744. Throughput: 0: 19975.9. Samples: 70796252. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:44:35,466][1157520] Avg episode reward: [(0, '53.344')] [2024-09-30 01:44:35,665][1157819] Updated weights for policy 0, policy_version 78918 (0.0006) [2024-09-30 01:44:36,193][1157819] Updated weights for policy 0, policy_version 78928 (0.0006) [2024-09-30 01:44:36,752][1157819] Updated weights for policy 0, policy_version 78938 (0.0006) [2024-09-30 01:44:37,312][1157819] Updated weights for policy 0, policy_version 78948 (0.0006) [2024-09-30 01:44:37,852][1157819] Updated weights for policy 0, policy_version 78958 (0.0006) [2024-09-30 01:44:38,423][1157819] Updated weights for policy 0, policy_version 78968 (0.0006) [2024-09-30 01:44:38,952][1157819] Updated weights for policy 0, policy_version 78978 (0.0006) [2024-09-30 01:44:39,491][1157819] Updated weights for policy 0, policy_version 78988 (0.0006) [2024-09-30 01:44:40,049][1157819] Updated weights for policy 0, policy_version 78998 (0.0006) [2024-09-30 01:44:40,466][1157520] Fps is (10 sec: 77004.9, 60 sec: 79394.3, 300 sec: 77393.6). Total num frames: 323608576. Throughput: 0: 19888.8. Samples: 70851820. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:44:40,466][1157520] Avg episode reward: [(0, '54.369')] [2024-09-30 01:44:40,602][1157819] Updated weights for policy 0, policy_version 79008 (0.0006) [2024-09-30 01:44:41,118][1157819] Updated weights for policy 0, policy_version 79018 (0.0006) [2024-09-30 01:44:41,663][1157819] Updated weights for policy 0, policy_version 79028 (0.0006) [2024-09-30 01:44:42,211][1157819] Updated weights for policy 0, policy_version 79038 (0.0006) [2024-09-30 01:44:42,741][1157819] Updated weights for policy 0, policy_version 79048 (0.0006) [2024-09-30 01:44:43,260][1157819] Updated weights for policy 0, policy_version 79058 (0.0006) [2024-09-30 01:44:43,772][1157819] Updated weights for policy 0, policy_version 79068 (0.0006) [2024-09-30 01:44:44,332][1157819] Updated weights for policy 0, policy_version 79078 (0.0006) [2024-09-30 01:44:44,589][1157736] Signal inference workers to stop experience collection... (5150 times) [2024-09-30 01:44:44,592][1157819] InferenceWorker_p0-w0: stopping experience collection (5150 times) [2024-09-30 01:44:44,599][1157736] Signal inference workers to resume experience collection... (5150 times) [2024-09-30 01:44:44,600][1157819] InferenceWorker_p0-w0: resuming experience collection (5150 times) [2024-09-30 01:44:44,862][1157819] Updated weights for policy 0, policy_version 79088 (0.0006) [2024-09-30 01:44:45,420][1157819] Updated weights for policy 0, policy_version 79098 (0.0006) [2024-09-30 01:44:45,466][1157520] Fps is (10 sec: 75366.3, 60 sec: 79053.0, 300 sec: 77532.4). Total num frames: 323985408. Throughput: 0: 19769.4. Samples: 70965948. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:44:45,466][1157520] Avg episode reward: [(0, '53.275')] [2024-09-30 01:44:45,964][1157819] Updated weights for policy 0, policy_version 79108 (0.0006) [2024-09-30 01:44:46,529][1157819] Updated weights for policy 0, policy_version 79118 (0.0006) [2024-09-30 01:44:47,107][1157819] Updated weights for policy 0, policy_version 79128 (0.0006) [2024-09-30 01:44:47,613][1157819] Updated weights for policy 0, policy_version 79138 (0.0006) [2024-09-30 01:44:48,146][1157819] Updated weights for policy 0, policy_version 79148 (0.0006) [2024-09-30 01:44:48,725][1157819] Updated weights for policy 0, policy_version 79158 (0.0006) [2024-09-30 01:44:49,261][1157819] Updated weights for policy 0, policy_version 79168 (0.0006) [2024-09-30 01:44:49,832][1157819] Updated weights for policy 0, policy_version 79178 (0.0006) [2024-09-30 01:44:50,371][1157819] Updated weights for policy 0, policy_version 79188 (0.0006) [2024-09-30 01:44:50,466][1157520] Fps is (10 sec: 74956.3, 60 sec: 78643.1, 300 sec: 77615.8). Total num frames: 324358144. Throughput: 0: 19550.5. Samples: 71077632. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:44:50,466][1157520] Avg episode reward: [(0, '54.558')] [2024-09-30 01:44:50,897][1157819] Updated weights for policy 0, policy_version 79198 (0.0006) [2024-09-30 01:44:51,408][1157819] Updated weights for policy 0, policy_version 79208 (0.0006) [2024-09-30 01:44:51,934][1157819] Updated weights for policy 0, policy_version 79218 (0.0006) [2024-09-30 01:44:52,468][1157819] Updated weights for policy 0, policy_version 79228 (0.0006) [2024-09-30 01:44:52,983][1157819] Updated weights for policy 0, policy_version 79238 (0.0006) [2024-09-30 01:44:53,501][1157819] Updated weights for policy 0, policy_version 79248 (0.0006) [2024-09-30 01:44:54,008][1157819] Updated weights for policy 0, policy_version 79258 (0.0006) [2024-09-30 01:44:54,526][1157819] Updated weights for policy 0, policy_version 79268 (0.0006) [2024-09-30 01:44:55,085][1157819] Updated weights for policy 0, policy_version 79278 (0.0006) [2024-09-30 01:44:55,466][1157520] Fps is (10 sec: 76593.9, 60 sec: 78574.7, 300 sec: 77740.7). Total num frames: 324751360. Throughput: 0: 19446.9. Samples: 71136108. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:44:55,466][1157520] Avg episode reward: [(0, '54.795')] [2024-09-30 01:44:55,622][1157819] Updated weights for policy 0, policy_version 79288 (0.0006) [2024-09-30 01:44:56,139][1157819] Updated weights for policy 0, policy_version 79298 (0.0006) [2024-09-30 01:44:56,651][1157819] Updated weights for policy 0, policy_version 79308 (0.0006) [2024-09-30 01:44:57,188][1157819] Updated weights for policy 0, policy_version 79318 (0.0006) [2024-09-30 01:44:57,688][1157819] Updated weights for policy 0, policy_version 79328 (0.0006) [2024-09-30 01:44:58,212][1157819] Updated weights for policy 0, policy_version 79338 (0.0006) [2024-09-30 01:44:58,734][1157819] Updated weights for policy 0, policy_version 79348 (0.0006) [2024-09-30 01:44:59,246][1157819] Updated weights for policy 0, policy_version 79358 (0.0006) [2024-09-30 01:44:59,752][1157819] Updated weights for policy 0, policy_version 79368 (0.0006) [2024-09-30 01:45:00,268][1157819] Updated weights for policy 0, policy_version 79378 (0.0006) [2024-09-30 01:45:00,466][1157520] Fps is (10 sec: 78643.2, 60 sec: 78438.3, 300 sec: 77810.2). Total num frames: 325144576. Throughput: 0: 19398.1. Samples: 71253608. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:45:00,466][1157520] Avg episode reward: [(0, '53.237')] [2024-09-30 01:45:00,778][1157819] Updated weights for policy 0, policy_version 79388 (0.0006) [2024-09-30 01:45:01,287][1157819] Updated weights for policy 0, policy_version 79398 (0.0006) [2024-09-30 01:45:01,781][1157819] Updated weights for policy 0, policy_version 79408 (0.0006) [2024-09-30 01:45:02,286][1157819] Updated weights for policy 0, policy_version 79418 (0.0006) [2024-09-30 01:45:02,791][1157819] Updated weights for policy 0, policy_version 79428 (0.0006) [2024-09-30 01:45:03,285][1157819] Updated weights for policy 0, policy_version 79438 (0.0006) [2024-09-30 01:45:03,809][1157819] Updated weights for policy 0, policy_version 79448 (0.0006) [2024-09-30 01:45:04,301][1157819] Updated weights for policy 0, policy_version 79458 (0.0006) [2024-09-30 01:45:04,790][1157819] Updated weights for policy 0, policy_version 79468 (0.0006) [2024-09-30 01:45:05,293][1157819] Updated weights for policy 0, policy_version 79478 (0.0006) [2024-09-30 01:45:05,466][1157520] Fps is (10 sec: 80282.3, 60 sec: 78438.3, 300 sec: 77921.2). Total num frames: 325554176. Throughput: 0: 19382.9. Samples: 71375060. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:45:05,466][1157520] Avg episode reward: [(0, '52.071')] [2024-09-30 01:45:05,807][1157819] Updated weights for policy 0, policy_version 79488 (0.0006) [2024-09-30 01:45:06,318][1157819] Updated weights for policy 0, policy_version 79498 (0.0006) [2024-09-30 01:45:06,826][1157819] Updated weights for policy 0, policy_version 79508 (0.0006) [2024-09-30 01:45:07,336][1157819] Updated weights for policy 0, policy_version 79518 (0.0006) [2024-09-30 01:45:07,877][1157819] Updated weights for policy 0, policy_version 79528 (0.0006) [2024-09-30 01:45:08,409][1157819] Updated weights for policy 0, policy_version 79538 (0.0006) [2024-09-30 01:45:08,971][1157819] Updated weights for policy 0, policy_version 79548 (0.0006) [2024-09-30 01:45:09,607][1157819] Updated weights for policy 0, policy_version 79558 (0.0006) [2024-09-30 01:45:10,137][1157819] Updated weights for policy 0, policy_version 79568 (0.0006) [2024-09-30 01:45:10,466][1157520] Fps is (10 sec: 78642.6, 60 sec: 77892.1, 300 sec: 77893.4). Total num frames: 325931008. Throughput: 0: 19395.8. Samples: 71435172. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:45:10,466][1157520] Avg episode reward: [(0, '54.777')] [2024-09-30 01:45:10,700][1157819] Updated weights for policy 0, policy_version 79578 (0.0006) [2024-09-30 01:45:11,299][1157819] Updated weights for policy 0, policy_version 79588 (0.0006) [2024-09-30 01:45:11,919][1157819] Updated weights for policy 0, policy_version 79598 (0.0006) [2024-09-30 01:45:12,513][1157819] Updated weights for policy 0, policy_version 79608 (0.0006) [2024-09-30 01:45:13,105][1157819] Updated weights for policy 0, policy_version 79618 (0.0006) [2024-09-30 01:45:13,699][1157819] Updated weights for policy 0, policy_version 79628 (0.0006) [2024-09-30 01:45:14,287][1157819] Updated weights for policy 0, policy_version 79638 (0.0006) [2024-09-30 01:45:14,847][1157819] Updated weights for policy 0, policy_version 79648 (0.0006) [2024-09-30 01:45:15,430][1157819] Updated weights for policy 0, policy_version 79658 (0.0006) [2024-09-30 01:45:15,466][1157520] Fps is (10 sec: 72498.9, 60 sec: 77072.9, 300 sec: 77726.8). Total num frames: 326279168. Throughput: 0: 19180.4. Samples: 71540756. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:45:15,466][1157520] Avg episode reward: [(0, '54.983')] [2024-09-30 01:45:15,971][1157819] Updated weights for policy 0, policy_version 79668 (0.0006) [2024-09-30 01:45:16,545][1157819] Updated weights for policy 0, policy_version 79678 (0.0006) [2024-09-30 01:45:17,156][1157819] Updated weights for policy 0, policy_version 79688 (0.0006) [2024-09-30 01:45:17,272][1157736] Signal inference workers to stop experience collection... (5200 times) [2024-09-30 01:45:17,272][1157736] Signal inference workers to resume experience collection... (5200 times) [2024-09-30 01:45:17,275][1157819] InferenceWorker_p0-w0: stopping experience collection (5200 times) [2024-09-30 01:45:17,275][1157819] InferenceWorker_p0-w0: resuming experience collection (5200 times) [2024-09-30 01:45:17,730][1157819] Updated weights for policy 0, policy_version 79698 (0.0006) [2024-09-30 01:45:18,278][1157819] Updated weights for policy 0, policy_version 79708 (0.0006) [2024-09-30 01:45:18,857][1157819] Updated weights for policy 0, policy_version 79718 (0.0006) [2024-09-30 01:45:19,431][1157819] Updated weights for policy 0, policy_version 79728 (0.0006) [2024-09-30 01:45:19,968][1157819] Updated weights for policy 0, policy_version 79738 (0.0006) [2024-09-30 01:45:20,466][1157520] Fps is (10 sec: 70860.9, 60 sec: 76321.9, 300 sec: 77629.6). Total num frames: 326639616. Throughput: 0: 18931.2. Samples: 71648160. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 01:45:20,466][1157520] Avg episode reward: [(0, '54.293')] [2024-09-30 01:45:20,551][1157819] Updated weights for policy 0, policy_version 79748 (0.0006) [2024-09-30 01:45:21,081][1157819] Updated weights for policy 0, policy_version 79758 (0.0006) [2024-09-30 01:45:21,709][1157819] Updated weights for policy 0, policy_version 79768 (0.0006) [2024-09-30 01:45:22,244][1157819] Updated weights for policy 0, policy_version 79778 (0.0006) [2024-09-30 01:45:22,779][1157819] Updated weights for policy 0, policy_version 79788 (0.0006) [2024-09-30 01:45:23,319][1157819] Updated weights for policy 0, policy_version 79798 (0.0006) [2024-09-30 01:45:23,830][1157819] Updated weights for policy 0, policy_version 79808 (0.0006) [2024-09-30 01:45:24,380][1157819] Updated weights for policy 0, policy_version 79818 (0.0006) [2024-09-30 01:45:24,931][1157819] Updated weights for policy 0, policy_version 79828 (0.0006) [2024-09-30 01:45:25,466][1157520] Fps is (10 sec: 73318.2, 60 sec: 75844.1, 300 sec: 77518.5). Total num frames: 327012352. Throughput: 0: 18913.3. Samples: 71702920. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 01:45:25,466][1157520] Avg episode reward: [(0, '52.646')] [2024-09-30 01:45:25,476][1157819] Updated weights for policy 0, policy_version 79838 (0.0006) [2024-09-30 01:45:26,025][1157819] Updated weights for policy 0, policy_version 79848 (0.0007) [2024-09-30 01:45:26,573][1157819] Updated weights for policy 0, policy_version 79858 (0.0006) [2024-09-30 01:45:27,124][1157819] Updated weights for policy 0, policy_version 79868 (0.0006) [2024-09-30 01:45:27,619][1157819] Updated weights for policy 0, policy_version 79878 (0.0006) [2024-09-30 01:45:28,148][1157819] Updated weights for policy 0, policy_version 79888 (0.0006) [2024-09-30 01:45:28,643][1157819] Updated weights for policy 0, policy_version 79898 (0.0006) [2024-09-30 01:45:29,150][1157819] Updated weights for policy 0, policy_version 79908 (0.0006) [2024-09-30 01:45:29,655][1157819] Updated weights for policy 0, policy_version 79918 (0.0006) [2024-09-30 01:45:30,161][1157819] Updated weights for policy 0, policy_version 79928 (0.0006) [2024-09-30 01:45:30,466][1157520] Fps is (10 sec: 76186.0, 60 sec: 76049.0, 300 sec: 77407.4). Total num frames: 327401472. Throughput: 0: 18945.6. Samples: 71818500. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 01:45:30,466][1157520] Avg episode reward: [(0, '52.678')] [2024-09-30 01:45:30,685][1157819] Updated weights for policy 0, policy_version 79938 (0.0006) [2024-09-30 01:45:31,272][1157819] Updated weights for policy 0, policy_version 79948 (0.0006) [2024-09-30 01:45:31,797][1157819] Updated weights for policy 0, policy_version 79958 (0.0006) [2024-09-30 01:45:32,320][1157819] Updated weights for policy 0, policy_version 79968 (0.0006) [2024-09-30 01:45:32,854][1157819] Updated weights for policy 0, policy_version 79978 (0.0006) [2024-09-30 01:45:33,431][1157819] Updated weights for policy 0, policy_version 79988 (0.0006) [2024-09-30 01:45:33,990][1157819] Updated weights for policy 0, policy_version 79998 (0.0006) [2024-09-30 01:45:34,510][1157819] Updated weights for policy 0, policy_version 80008 (0.0006) [2024-09-30 01:45:35,031][1157819] Updated weights for policy 0, policy_version 80018 (0.0006) [2024-09-30 01:45:35,466][1157520] Fps is (10 sec: 77414.7, 60 sec: 75912.4, 300 sec: 77324.1). Total num frames: 327786496. Throughput: 0: 19028.1. Samples: 71933896. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 01:45:35,466][1157520] Avg episode reward: [(0, '52.933')] [2024-09-30 01:45:35,522][1157819] Updated weights for policy 0, policy_version 80028 (0.0006) [2024-09-30 01:45:36,030][1157819] Updated weights for policy 0, policy_version 80038 (0.0006) [2024-09-30 01:45:36,531][1157819] Updated weights for policy 0, policy_version 80048 (0.0006) [2024-09-30 01:45:37,053][1157819] Updated weights for policy 0, policy_version 80058 (0.0006) [2024-09-30 01:45:37,564][1157819] Updated weights for policy 0, policy_version 80068 (0.0006) [2024-09-30 01:45:38,080][1157819] Updated weights for policy 0, policy_version 80078 (0.0006) [2024-09-30 01:45:38,606][1157819] Updated weights for policy 0, policy_version 80088 (0.0006) [2024-09-30 01:45:39,118][1157819] Updated weights for policy 0, policy_version 80098 (0.0006) [2024-09-30 01:45:39,615][1157819] Updated weights for policy 0, policy_version 80108 (0.0006) [2024-09-30 01:45:40,126][1157819] Updated weights for policy 0, policy_version 80118 (0.0006) [2024-09-30 01:45:40,466][1157520] Fps is (10 sec: 79052.1, 60 sec: 76390.2, 300 sec: 77365.8). Total num frames: 328192000. Throughput: 0: 19068.5. Samples: 71994192. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 01:45:40,466][1157520] Avg episode reward: [(0, '52.776')] [2024-09-30 01:45:40,624][1157819] Updated weights for policy 0, policy_version 80128 (0.0006) [2024-09-30 01:45:41,132][1157819] Updated weights for policy 0, policy_version 80138 (0.0006) [2024-09-30 01:45:41,629][1157819] Updated weights for policy 0, policy_version 80148 (0.0006) [2024-09-30 01:45:42,185][1157819] Updated weights for policy 0, policy_version 80158 (0.0006) [2024-09-30 01:45:42,693][1157819] Updated weights for policy 0, policy_version 80168 (0.0006) [2024-09-30 01:45:43,253][1157819] Updated weights for policy 0, policy_version 80178 (0.0006) [2024-09-30 01:45:43,777][1157819] Updated weights for policy 0, policy_version 80188 (0.0006) [2024-09-30 01:45:44,325][1157819] Updated weights for policy 0, policy_version 80198 (0.0007) [2024-09-30 01:45:44,863][1157819] Updated weights for policy 0, policy_version 80208 (0.0006) [2024-09-30 01:45:45,392][1157819] Updated weights for policy 0, policy_version 80218 (0.0006) [2024-09-30 01:45:45,466][1157520] Fps is (10 sec: 78642.2, 60 sec: 76458.4, 300 sec: 77324.1). Total num frames: 328572928. Throughput: 0: 19090.8. Samples: 72112696. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 01:45:45,466][1157520] Avg episode reward: [(0, '54.174')] [2024-09-30 01:45:45,941][1157819] Updated weights for policy 0, policy_version 80228 (0.0006) [2024-09-30 01:45:46,516][1157819] Updated weights for policy 0, policy_version 80238 (0.0006) [2024-09-30 01:45:47,069][1157819] Updated weights for policy 0, policy_version 80248 (0.0006) [2024-09-30 01:45:47,605][1157819] Updated weights for policy 0, policy_version 80258 (0.0006) [2024-09-30 01:45:48,048][1157736] Signal inference workers to stop experience collection... (5250 times) [2024-09-30 01:45:48,048][1157736] Signal inference workers to resume experience collection... (5250 times) [2024-09-30 01:45:48,054][1157819] InferenceWorker_p0-w0: stopping experience collection (5250 times) [2024-09-30 01:45:48,054][1157819] InferenceWorker_p0-w0: resuming experience collection (5250 times) [2024-09-30 01:45:48,152][1157819] Updated weights for policy 0, policy_version 80268 (0.0006) [2024-09-30 01:45:48,706][1157819] Updated weights for policy 0, policy_version 80278 (0.0006) [2024-09-30 01:45:49,266][1157819] Updated weights for policy 0, policy_version 80288 (0.0007) [2024-09-30 01:45:49,801][1157819] Updated weights for policy 0, policy_version 80298 (0.0007) [2024-09-30 01:45:50,366][1157819] Updated weights for policy 0, policy_version 80308 (0.0007) [2024-09-30 01:45:50,466][1157520] Fps is (10 sec: 75366.6, 60 sec: 76458.6, 300 sec: 77226.9). Total num frames: 328945664. Throughput: 0: 18880.6. Samples: 72224688. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 01:45:50,466][1157520] Avg episode reward: [(0, '52.189')] [2024-09-30 01:45:50,933][1157819] Updated weights for policy 0, policy_version 80318 (0.0007) [2024-09-30 01:45:51,486][1157819] Updated weights for policy 0, policy_version 80328 (0.0006) [2024-09-30 01:45:52,050][1157819] Updated weights for policy 0, policy_version 80338 (0.0006) [2024-09-30 01:45:52,603][1157819] Updated weights for policy 0, policy_version 80348 (0.0006) [2024-09-30 01:45:53,136][1157819] Updated weights for policy 0, policy_version 80358 (0.0007) [2024-09-30 01:45:53,696][1157819] Updated weights for policy 0, policy_version 80368 (0.0007) [2024-09-30 01:45:54,248][1157819] Updated weights for policy 0, policy_version 80378 (0.0006) [2024-09-30 01:45:54,778][1157819] Updated weights for policy 0, policy_version 80388 (0.0006) [2024-09-30 01:45:55,337][1157819] Updated weights for policy 0, policy_version 80398 (0.0006) [2024-09-30 01:45:55,466][1157520] Fps is (10 sec: 74547.3, 60 sec: 76117.3, 300 sec: 77046.4). Total num frames: 329318400. Throughput: 0: 18775.3. Samples: 72280060. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 01:45:55,466][1157520] Avg episode reward: [(0, '53.833')] [2024-09-30 01:45:55,896][1157819] Updated weights for policy 0, policy_version 80408 (0.0006) [2024-09-30 01:45:56,443][1157819] Updated weights for policy 0, policy_version 80418 (0.0006) [2024-09-30 01:45:56,953][1157819] Updated weights for policy 0, policy_version 80428 (0.0006) [2024-09-30 01:45:57,542][1157819] Updated weights for policy 0, policy_version 80438 (0.0006) [2024-09-30 01:45:58,066][1157819] Updated weights for policy 0, policy_version 80448 (0.0006) [2024-09-30 01:45:58,622][1157819] Updated weights for policy 0, policy_version 80458 (0.0006) [2024-09-30 01:45:59,167][1157819] Updated weights for policy 0, policy_version 80468 (0.0006) [2024-09-30 01:45:59,720][1157819] Updated weights for policy 0, policy_version 80478 (0.0006) [2024-09-30 01:46:00,293][1157819] Updated weights for policy 0, policy_version 80488 (0.0006) [2024-09-30 01:46:00,466][1157520] Fps is (10 sec: 74547.0, 60 sec: 75775.9, 300 sec: 76810.4). Total num frames: 329691136. Throughput: 0: 18912.9. Samples: 72391840. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 01:46:00,466][1157520] Avg episode reward: [(0, '54.316')] [2024-09-30 01:46:00,815][1157819] Updated weights for policy 0, policy_version 80498 (0.0006) [2024-09-30 01:46:01,403][1157819] Updated weights for policy 0, policy_version 80508 (0.0006) [2024-09-30 01:46:01,919][1157819] Updated weights for policy 0, policy_version 80518 (0.0006) [2024-09-30 01:46:02,478][1157819] Updated weights for policy 0, policy_version 80528 (0.0006) [2024-09-30 01:46:03,018][1157819] Updated weights for policy 0, policy_version 80538 (0.0006) [2024-09-30 01:46:03,585][1157819] Updated weights for policy 0, policy_version 80548 (0.0006) [2024-09-30 01:46:04,133][1157819] Updated weights for policy 0, policy_version 80558 (0.0006) [2024-09-30 01:46:04,670][1157819] Updated weights for policy 0, policy_version 80568 (0.0006) [2024-09-30 01:46:05,198][1157819] Updated weights for policy 0, policy_version 80578 (0.0006) [2024-09-30 01:46:05,466][1157520] Fps is (10 sec: 74547.8, 60 sec: 75161.5, 300 sec: 76629.9). Total num frames: 330063872. Throughput: 0: 19018.7. Samples: 72504000. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 01:46:05,466][1157520] Avg episode reward: [(0, '51.306')] [2024-09-30 01:46:05,472][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000080583_330067968.pth... [2024-09-30 01:46:05,518][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000076079_311619584.pth [2024-09-30 01:46:05,747][1157819] Updated weights for policy 0, policy_version 80588 (0.0006) [2024-09-30 01:46:06,265][1157819] Updated weights for policy 0, policy_version 80598 (0.0006) [2024-09-30 01:46:06,803][1157819] Updated weights for policy 0, policy_version 80608 (0.0006) [2024-09-30 01:46:07,357][1157819] Updated weights for policy 0, policy_version 80618 (0.0006) [2024-09-30 01:46:07,881][1157819] Updated weights for policy 0, policy_version 80628 (0.0006) [2024-09-30 01:46:08,417][1157819] Updated weights for policy 0, policy_version 80638 (0.0006) [2024-09-30 01:46:08,939][1157819] Updated weights for policy 0, policy_version 80648 (0.0006) [2024-09-30 01:46:09,481][1157819] Updated weights for policy 0, policy_version 80658 (0.0006) [2024-09-30 01:46:09,989][1157819] Updated weights for policy 0, policy_version 80668 (0.0006) [2024-09-30 01:46:10,466][1157520] Fps is (10 sec: 75776.7, 60 sec: 75298.2, 300 sec: 76560.5). Total num frames: 330448896. Throughput: 0: 19074.8. Samples: 72561284. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 01:46:10,466][1157520] Avg episode reward: [(0, '51.167')] [2024-09-30 01:46:10,527][1157819] Updated weights for policy 0, policy_version 80678 (0.0006) [2024-09-30 01:46:11,075][1157819] Updated weights for policy 0, policy_version 80688 (0.0006) [2024-09-30 01:46:11,593][1157819] Updated weights for policy 0, policy_version 80698 (0.0006) [2024-09-30 01:46:12,134][1157819] Updated weights for policy 0, policy_version 80708 (0.0006) [2024-09-30 01:46:12,707][1157819] Updated weights for policy 0, policy_version 80718 (0.0006) [2024-09-30 01:46:13,222][1157819] Updated weights for policy 0, policy_version 80728 (0.0006) [2024-09-30 01:46:13,794][1157819] Updated weights for policy 0, policy_version 80738 (0.0006) [2024-09-30 01:46:14,342][1157819] Updated weights for policy 0, policy_version 80748 (0.0006) [2024-09-30 01:46:14,865][1157819] Updated weights for policy 0, policy_version 80758 (0.0006) [2024-09-30 01:46:15,439][1157819] Updated weights for policy 0, policy_version 80768 (0.0006) [2024-09-30 01:46:15,466][1157520] Fps is (10 sec: 76185.3, 60 sec: 75775.9, 300 sec: 76546.6). Total num frames: 330825728. Throughput: 0: 19050.7. Samples: 72675784. Policy #0 lag: (min: 0.0, avg: 3.0, max: 6.0) [2024-09-30 01:46:15,466][1157520] Avg episode reward: [(0, '53.623')] [2024-09-30 01:46:15,969][1157819] Updated weights for policy 0, policy_version 80778 (0.0006) [2024-09-30 01:46:16,490][1157819] Updated weights for policy 0, policy_version 80788 (0.0006) [2024-09-30 01:46:17,071][1157819] Updated weights for policy 0, policy_version 80798 (0.0006) [2024-09-30 01:46:17,594][1157819] Updated weights for policy 0, policy_version 80808 (0.0006) [2024-09-30 01:46:18,139][1157819] Updated weights for policy 0, policy_version 80818 (0.0006) [2024-09-30 01:46:18,707][1157819] Updated weights for policy 0, policy_version 80828 (0.0006) [2024-09-30 01:46:19,257][1157819] Updated weights for policy 0, policy_version 80838 (0.0006) [2024-09-30 01:46:19,799][1157819] Updated weights for policy 0, policy_version 80848 (0.0006) [2024-09-30 01:46:20,330][1157819] Updated weights for policy 0, policy_version 80858 (0.0006) [2024-09-30 01:46:20,466][1157520] Fps is (10 sec: 75366.0, 60 sec: 76049.1, 300 sec: 76588.2). Total num frames: 331202560. Throughput: 0: 18987.8. Samples: 72788348. Policy #0 lag: (min: 0.0, avg: 3.0, max: 6.0) [2024-09-30 01:46:20,466][1157520] Avg episode reward: [(0, '50.578')] [2024-09-30 01:46:20,867][1157819] Updated weights for policy 0, policy_version 80868 (0.0006) [2024-09-30 01:46:21,370][1157819] Updated weights for policy 0, policy_version 80878 (0.0006) [2024-09-30 01:46:21,935][1157819] Updated weights for policy 0, policy_version 80888 (0.0006) [2024-09-30 01:46:22,480][1157819] Updated weights for policy 0, policy_version 80898 (0.0006) [2024-09-30 01:46:23,045][1157819] Updated weights for policy 0, policy_version 80908 (0.0006) [2024-09-30 01:46:23,566][1157819] Updated weights for policy 0, policy_version 80918 (0.0006) [2024-09-30 01:46:24,144][1157819] Updated weights for policy 0, policy_version 80928 (0.0006) [2024-09-30 01:46:24,657][1157819] Updated weights for policy 0, policy_version 80938 (0.0006) [2024-09-30 01:46:25,231][1157819] Updated weights for policy 0, policy_version 80948 (0.0006) [2024-09-30 01:46:25,466][1157520] Fps is (10 sec: 75366.8, 60 sec: 76117.3, 300 sec: 76504.9). Total num frames: 331579392. Throughput: 0: 18911.8. Samples: 72845220. Policy #0 lag: (min: 0.0, avg: 3.0, max: 6.0) [2024-09-30 01:46:25,466][1157520] Avg episode reward: [(0, '52.167')] [2024-09-30 01:46:25,759][1157819] Updated weights for policy 0, policy_version 80958 (0.0006) [2024-09-30 01:46:26,295][1157819] Updated weights for policy 0, policy_version 80968 (0.0006) [2024-09-30 01:46:26,825][1157819] Updated weights for policy 0, policy_version 80978 (0.0006) [2024-09-30 01:46:27,390][1157819] Updated weights for policy 0, policy_version 80988 (0.0006) [2024-09-30 01:46:27,939][1157819] Updated weights for policy 0, policy_version 80998 (0.0006) [2024-09-30 01:46:28,486][1157819] Updated weights for policy 0, policy_version 81008 (0.0006) [2024-09-30 01:46:29,036][1157819] Updated weights for policy 0, policy_version 81018 (0.0006) [2024-09-30 01:46:29,584][1157819] Updated weights for policy 0, policy_version 81028 (0.0006) [2024-09-30 01:46:30,139][1157819] Updated weights for policy 0, policy_version 81038 (0.0006) [2024-09-30 01:46:30,466][1157520] Fps is (10 sec: 74956.6, 60 sec: 75844.2, 300 sec: 76435.5). Total num frames: 331952128. Throughput: 0: 18773.8. Samples: 72957516. Policy #0 lag: (min: 0.0, avg: 3.0, max: 6.0) [2024-09-30 01:46:30,466][1157520] Avg episode reward: [(0, '52.761')] [2024-09-30 01:46:30,705][1157819] Updated weights for policy 0, policy_version 81048 (0.0006) [2024-09-30 01:46:31,239][1157819] Updated weights for policy 0, policy_version 81058 (0.0006) [2024-09-30 01:46:31,827][1157819] Updated weights for policy 0, policy_version 81068 (0.0006) [2024-09-30 01:46:32,352][1157819] Updated weights for policy 0, policy_version 81078 (0.0006) [2024-09-30 01:46:32,953][1157819] Updated weights for policy 0, policy_version 81088 (0.0006) [2024-09-30 01:46:33,479][1157819] Updated weights for policy 0, policy_version 81098 (0.0006) [2024-09-30 01:46:34,070][1157819] Updated weights for policy 0, policy_version 81108 (0.0006) [2024-09-30 01:46:34,572][1157819] Updated weights for policy 0, policy_version 81118 (0.0006) [2024-09-30 01:46:35,089][1157819] Updated weights for policy 0, policy_version 81128 (0.0006) [2024-09-30 01:46:35,466][1157520] Fps is (10 sec: 74547.0, 60 sec: 75639.4, 300 sec: 76366.1). Total num frames: 332324864. Throughput: 0: 18761.2. Samples: 73068944. Policy #0 lag: (min: 0.0, avg: 3.0, max: 6.0) [2024-09-30 01:46:35,466][1157520] Avg episode reward: [(0, '52.913')] [2024-09-30 01:46:35,665][1157819] Updated weights for policy 0, policy_version 81138 (0.0006) [2024-09-30 01:46:36,189][1157819] Updated weights for policy 0, policy_version 81148 (0.0006) [2024-09-30 01:46:36,792][1157819] Updated weights for policy 0, policy_version 81158 (0.0006) [2024-09-30 01:46:37,304][1157819] Updated weights for policy 0, policy_version 81168 (0.0006) [2024-09-30 01:46:37,900][1157819] Updated weights for policy 0, policy_version 81178 (0.0006) [2024-09-30 01:46:38,401][1157819] Updated weights for policy 0, policy_version 81188 (0.0006) [2024-09-30 01:46:38,942][1157819] Updated weights for policy 0, policy_version 81198 (0.0006) [2024-09-30 01:46:39,514][1157819] Updated weights for policy 0, policy_version 81208 (0.0006) [2024-09-30 01:46:40,075][1157819] Updated weights for policy 0, policy_version 81218 (0.0006) [2024-09-30 01:46:40,466][1157520] Fps is (10 sec: 74137.9, 60 sec: 75025.1, 300 sec: 76407.8). Total num frames: 332693504. Throughput: 0: 18767.6. Samples: 73124600. Policy #0 lag: (min: 0.0, avg: 3.0, max: 6.0) [2024-09-30 01:46:40,466][1157520] Avg episode reward: [(0, '54.916')] [2024-09-30 01:46:40,631][1157819] Updated weights for policy 0, policy_version 81228 (0.0006) [2024-09-30 01:46:41,200][1157819] Updated weights for policy 0, policy_version 81238 (0.0006) [2024-09-30 01:46:41,754][1157819] Updated weights for policy 0, policy_version 81248 (0.0006) [2024-09-30 01:46:42,292][1157736] Signal inference workers to stop experience collection... (5300 times) [2024-09-30 01:46:42,294][1157819] InferenceWorker_p0-w0: stopping experience collection (5300 times) [2024-09-30 01:46:42,297][1157736] Signal inference workers to resume experience collection... (5300 times) [2024-09-30 01:46:42,297][1157819] InferenceWorker_p0-w0: resuming experience collection (5300 times) [2024-09-30 01:46:42,313][1157819] Updated weights for policy 0, policy_version 81258 (0.0006) [2024-09-30 01:46:42,877][1157819] Updated weights for policy 0, policy_version 81268 (0.0006) [2024-09-30 01:46:43,424][1157819] Updated weights for policy 0, policy_version 81278 (0.0006) [2024-09-30 01:46:43,980][1157819] Updated weights for policy 0, policy_version 81288 (0.0006) [2024-09-30 01:46:44,491][1157819] Updated weights for policy 0, policy_version 81298 (0.0006) [2024-09-30 01:46:45,085][1157819] Updated weights for policy 0, policy_version 81308 (0.0006) [2024-09-30 01:46:45,466][1157520] Fps is (10 sec: 74136.8, 60 sec: 74888.5, 300 sec: 76477.1). Total num frames: 333066240. Throughput: 0: 18752.8. Samples: 73235716. Policy #0 lag: (min: 0.0, avg: 3.0, max: 6.0) [2024-09-30 01:46:45,466][1157520] Avg episode reward: [(0, '54.735')] [2024-09-30 01:46:45,614][1157819] Updated weights for policy 0, policy_version 81318 (0.0006) [2024-09-30 01:46:46,197][1157819] Updated weights for policy 0, policy_version 81328 (0.0006) [2024-09-30 01:46:46,714][1157819] Updated weights for policy 0, policy_version 81338 (0.0006) [2024-09-30 01:46:47,286][1157819] Updated weights for policy 0, policy_version 81348 (0.0006) [2024-09-30 01:46:47,825][1157819] Updated weights for policy 0, policy_version 81358 (0.0006) [2024-09-30 01:46:48,378][1157819] Updated weights for policy 0, policy_version 81368 (0.0006) [2024-09-30 01:46:48,960][1157819] Updated weights for policy 0, policy_version 81378 (0.0006) [2024-09-30 01:46:49,495][1157819] Updated weights for policy 0, policy_version 81388 (0.0006) [2024-09-30 01:46:50,069][1157819] Updated weights for policy 0, policy_version 81398 (0.0006) [2024-09-30 01:46:50,466][1157520] Fps is (10 sec: 74137.2, 60 sec: 74820.2, 300 sec: 76477.2). Total num frames: 333434880. Throughput: 0: 18717.1. Samples: 73346272. Policy #0 lag: (min: 0.0, avg: 3.0, max: 6.0) [2024-09-30 01:46:50,466][1157520] Avg episode reward: [(0, '54.023')] [2024-09-30 01:46:50,644][1157819] Updated weights for policy 0, policy_version 81408 (0.0006) [2024-09-30 01:46:51,201][1157819] Updated weights for policy 0, policy_version 81418 (0.0006) [2024-09-30 01:46:51,755][1157819] Updated weights for policy 0, policy_version 81428 (0.0006) [2024-09-30 01:46:52,297][1157819] Updated weights for policy 0, policy_version 81438 (0.0006) [2024-09-30 01:46:52,790][1157819] Updated weights for policy 0, policy_version 81448 (0.0006) [2024-09-30 01:46:53,394][1157819] Updated weights for policy 0, policy_version 81458 (0.0006) [2024-09-30 01:46:53,903][1157819] Updated weights for policy 0, policy_version 81468 (0.0006) [2024-09-30 01:46:54,480][1157819] Updated weights for policy 0, policy_version 81478 (0.0006) [2024-09-30 01:46:55,030][1157819] Updated weights for policy 0, policy_version 81488 (0.0006) [2024-09-30 01:46:55,466][1157520] Fps is (10 sec: 73728.7, 60 sec: 74752.1, 300 sec: 76463.3). Total num frames: 333803520. Throughput: 0: 18679.0. Samples: 73401840. Policy #0 lag: (min: 0.0, avg: 3.0, max: 6.0) [2024-09-30 01:46:55,466][1157520] Avg episode reward: [(0, '54.786')] [2024-09-30 01:46:55,557][1157819] Updated weights for policy 0, policy_version 81498 (0.0006) [2024-09-30 01:46:56,103][1157819] Updated weights for policy 0, policy_version 81508 (0.0006) [2024-09-30 01:46:56,628][1157819] Updated weights for policy 0, policy_version 81518 (0.0006) [2024-09-30 01:46:57,198][1157819] Updated weights for policy 0, policy_version 81528 (0.0006) [2024-09-30 01:46:57,716][1157819] Updated weights for policy 0, policy_version 81538 (0.0006) [2024-09-30 01:46:58,228][1157819] Updated weights for policy 0, policy_version 81548 (0.0006) [2024-09-30 01:46:58,782][1157819] Updated weights for policy 0, policy_version 81558 (0.0006) [2024-09-30 01:46:59,283][1157819] Updated weights for policy 0, policy_version 81568 (0.0006) [2024-09-30 01:46:59,783][1157819] Updated weights for policy 0, policy_version 81578 (0.0006) [2024-09-30 01:47:00,281][1157819] Updated weights for policy 0, policy_version 81588 (0.0006) [2024-09-30 01:47:00,466][1157520] Fps is (10 sec: 76185.8, 60 sec: 75093.4, 300 sec: 76532.7). Total num frames: 334196736. Throughput: 0: 18669.5. Samples: 73515912. Policy #0 lag: (min: 0.0, avg: 3.0, max: 6.0) [2024-09-30 01:47:00,466][1157520] Avg episode reward: [(0, '53.142')] [2024-09-30 01:47:00,813][1157819] Updated weights for policy 0, policy_version 81598 (0.0006) [2024-09-30 01:47:01,308][1157819] Updated weights for policy 0, policy_version 81608 (0.0006) [2024-09-30 01:47:01,853][1157819] Updated weights for policy 0, policy_version 81618 (0.0006) [2024-09-30 01:47:02,357][1157819] Updated weights for policy 0, policy_version 81628 (0.0006) [2024-09-30 01:47:02,859][1157819] Updated weights for policy 0, policy_version 81638 (0.0006) [2024-09-30 01:47:03,366][1157819] Updated weights for policy 0, policy_version 81648 (0.0006) [2024-09-30 01:47:03,884][1157819] Updated weights for policy 0, policy_version 81658 (0.0006) [2024-09-30 01:47:04,377][1157819] Updated weights for policy 0, policy_version 81668 (0.0006) [2024-09-30 01:47:04,869][1157819] Updated weights for policy 0, policy_version 81678 (0.0006) [2024-09-30 01:47:05,370][1157819] Updated weights for policy 0, policy_version 81688 (0.0006) [2024-09-30 01:47:05,466][1157520] Fps is (10 sec: 79053.5, 60 sec: 75503.0, 300 sec: 76685.5). Total num frames: 334594048. Throughput: 0: 18839.0. Samples: 73636100. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:47:05,466][1157520] Avg episode reward: [(0, '54.167')] [2024-09-30 01:47:05,931][1157819] Updated weights for policy 0, policy_version 81698 (0.0006) [2024-09-30 01:47:06,436][1157819] Updated weights for policy 0, policy_version 81708 (0.0006) [2024-09-30 01:47:06,957][1157819] Updated weights for policy 0, policy_version 81718 (0.0006) [2024-09-30 01:47:07,449][1157819] Updated weights for policy 0, policy_version 81728 (0.0006) [2024-09-30 01:47:07,957][1157819] Updated weights for policy 0, policy_version 81738 (0.0006) [2024-09-30 01:47:08,489][1157819] Updated weights for policy 0, policy_version 81748 (0.0006) [2024-09-30 01:47:08,991][1157819] Updated weights for policy 0, policy_version 81758 (0.0006) [2024-09-30 01:47:09,496][1157819] Updated weights for policy 0, policy_version 81768 (0.0006) [2024-09-30 01:47:10,014][1157819] Updated weights for policy 0, policy_version 81778 (0.0006) [2024-09-30 01:47:10,466][1157520] Fps is (10 sec: 80282.1, 60 sec: 75844.3, 300 sec: 76866.0). Total num frames: 334999552. Throughput: 0: 18906.7. Samples: 73696020. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:47:10,466][1157520] Avg episode reward: [(0, '51.638')] [2024-09-30 01:47:10,524][1157819] Updated weights for policy 0, policy_version 81788 (0.0006) [2024-09-30 01:47:11,028][1157819] Updated weights for policy 0, policy_version 81798 (0.0006) [2024-09-30 01:47:11,537][1157819] Updated weights for policy 0, policy_version 81808 (0.0006) [2024-09-30 01:47:12,056][1157819] Updated weights for policy 0, policy_version 81818 (0.0006) [2024-09-30 01:47:12,541][1157819] Updated weights for policy 0, policy_version 81828 (0.0006) [2024-09-30 01:47:13,079][1157819] Updated weights for policy 0, policy_version 81838 (0.0006) [2024-09-30 01:47:13,605][1157819] Updated weights for policy 0, policy_version 81848 (0.0006) [2024-09-30 01:47:14,110][1157819] Updated weights for policy 0, policy_version 81858 (0.0006) [2024-09-30 01:47:14,597][1157819] Updated weights for policy 0, policy_version 81868 (0.0006) [2024-09-30 01:47:15,129][1157819] Updated weights for policy 0, policy_version 81878 (0.0006) [2024-09-30 01:47:15,466][1157520] Fps is (10 sec: 80280.4, 60 sec: 76185.5, 300 sec: 76977.0). Total num frames: 335396864. Throughput: 0: 19083.0. Samples: 73816252. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:47:15,466][1157520] Avg episode reward: [(0, '55.515')] [2024-09-30 01:47:15,632][1157819] Updated weights for policy 0, policy_version 81888 (0.0006) [2024-09-30 01:47:16,159][1157819] Updated weights for policy 0, policy_version 81898 (0.0006) [2024-09-30 01:47:16,656][1157819] Updated weights for policy 0, policy_version 81908 (0.0006) [2024-09-30 01:47:17,155][1157819] Updated weights for policy 0, policy_version 81918 (0.0006) [2024-09-30 01:47:17,661][1157819] Updated weights for policy 0, policy_version 81928 (0.0006) [2024-09-30 01:47:18,169][1157819] Updated weights for policy 0, policy_version 81938 (0.0006) [2024-09-30 01:47:18,686][1157819] Updated weights for policy 0, policy_version 81948 (0.0006) [2024-09-30 01:47:19,204][1157819] Updated weights for policy 0, policy_version 81958 (0.0006) [2024-09-30 01:47:19,699][1157819] Updated weights for policy 0, policy_version 81968 (0.0006) [2024-09-30 01:47:20,211][1157819] Updated weights for policy 0, policy_version 81978 (0.0006) [2024-09-30 01:47:20,466][1157520] Fps is (10 sec: 80281.3, 60 sec: 76663.5, 300 sec: 77115.9). Total num frames: 335802368. Throughput: 0: 19292.7. Samples: 73937116. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:47:20,466][1157520] Avg episode reward: [(0, '52.609')] [2024-09-30 01:47:20,706][1157819] Updated weights for policy 0, policy_version 81988 (0.0006) [2024-09-30 01:47:21,225][1157819] Updated weights for policy 0, policy_version 81998 (0.0006) [2024-09-30 01:47:21,755][1157819] Updated weights for policy 0, policy_version 82008 (0.0006) [2024-09-30 01:47:22,245][1157819] Updated weights for policy 0, policy_version 82018 (0.0006) [2024-09-30 01:47:22,755][1157819] Updated weights for policy 0, policy_version 82028 (0.0006) [2024-09-30 01:47:23,294][1157819] Updated weights for policy 0, policy_version 82038 (0.0006) [2024-09-30 01:47:23,800][1157819] Updated weights for policy 0, policy_version 82048 (0.0006) [2024-09-30 01:47:24,312][1157819] Updated weights for policy 0, policy_version 82058 (0.0006) [2024-09-30 01:47:24,828][1157819] Updated weights for policy 0, policy_version 82068 (0.0006) [2024-09-30 01:47:25,367][1157819] Updated weights for policy 0, policy_version 82078 (0.0006) [2024-09-30 01:47:25,466][1157520] Fps is (10 sec: 80282.3, 60 sec: 77004.8, 300 sec: 77282.5). Total num frames: 336199680. Throughput: 0: 19388.9. Samples: 73997100. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:47:25,466][1157520] Avg episode reward: [(0, '53.196')] [2024-09-30 01:47:25,890][1157819] Updated weights for policy 0, policy_version 82088 (0.0006) [2024-09-30 01:47:26,417][1157819] Updated weights for policy 0, policy_version 82098 (0.0006) [2024-09-30 01:47:26,909][1157819] Updated weights for policy 0, policy_version 82108 (0.0006) [2024-09-30 01:47:27,437][1157819] Updated weights for policy 0, policy_version 82118 (0.0006) [2024-09-30 01:47:27,940][1157819] Updated weights for policy 0, policy_version 82128 (0.0006) [2024-09-30 01:47:28,476][1157819] Updated weights for policy 0, policy_version 82138 (0.0006) [2024-09-30 01:47:28,995][1157819] Updated weights for policy 0, policy_version 82148 (0.0006) [2024-09-30 01:47:29,515][1157819] Updated weights for policy 0, policy_version 82158 (0.0006) [2024-09-30 01:47:29,993][1157819] Updated weights for policy 0, policy_version 82168 (0.0006) [2024-09-30 01:47:30,466][1157520] Fps is (10 sec: 79462.3, 60 sec: 77414.4, 300 sec: 77435.2). Total num frames: 336596992. Throughput: 0: 19547.6. Samples: 74115356. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:47:30,466][1157520] Avg episode reward: [(0, '52.370')] [2024-09-30 01:47:30,475][1157819] Updated weights for policy 0, policy_version 82178 (0.0006) [2024-09-30 01:47:30,973][1157819] Updated weights for policy 0, policy_version 82188 (0.0006) [2024-09-30 01:47:31,469][1157819] Updated weights for policy 0, policy_version 82198 (0.0006) [2024-09-30 01:47:31,956][1157819] Updated weights for policy 0, policy_version 82208 (0.0006) [2024-09-30 01:47:32,450][1157819] Updated weights for policy 0, policy_version 82218 (0.0006) [2024-09-30 01:47:32,943][1157819] Updated weights for policy 0, policy_version 82228 (0.0006) [2024-09-30 01:47:33,444][1157819] Updated weights for policy 0, policy_version 82238 (0.0006) [2024-09-30 01:47:33,924][1157819] Updated weights for policy 0, policy_version 82248 (0.0006) [2024-09-30 01:47:34,406][1157819] Updated weights for policy 0, policy_version 82258 (0.0006) [2024-09-30 01:47:34,878][1157819] Updated weights for policy 0, policy_version 82268 (0.0006) [2024-09-30 01:47:35,351][1157819] Updated weights for policy 0, policy_version 82278 (0.0006) [2024-09-30 01:47:35,466][1157520] Fps is (10 sec: 81920.1, 60 sec: 78233.6, 300 sec: 77657.4). Total num frames: 337018880. Throughput: 0: 19873.1. Samples: 74240560. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:47:35,466][1157520] Avg episode reward: [(0, '54.378')] [2024-09-30 01:47:35,831][1157819] Updated weights for policy 0, policy_version 82288 (0.0006) [2024-09-30 01:47:36,323][1157819] Updated weights for policy 0, policy_version 82298 (0.0006) [2024-09-30 01:47:36,811][1157819] Updated weights for policy 0, policy_version 82308 (0.0006) [2024-09-30 01:47:37,292][1157819] Updated weights for policy 0, policy_version 82318 (0.0006) [2024-09-30 01:47:37,781][1157819] Updated weights for policy 0, policy_version 82328 (0.0006) [2024-09-30 01:47:38,269][1157819] Updated weights for policy 0, policy_version 82338 (0.0006) [2024-09-30 01:47:38,757][1157819] Updated weights for policy 0, policy_version 82348 (0.0006) [2024-09-30 01:47:39,244][1157819] Updated weights for policy 0, policy_version 82358 (0.0006) [2024-09-30 01:47:39,736][1157819] Updated weights for policy 0, policy_version 82368 (0.0006) [2024-09-30 01:47:40,224][1157819] Updated weights for policy 0, policy_version 82378 (0.0006) [2024-09-30 01:47:40,466][1157520] Fps is (10 sec: 83968.1, 60 sec: 79052.8, 300 sec: 77796.2). Total num frames: 337436672. Throughput: 0: 20046.7. Samples: 74303940. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:47:40,466][1157520] Avg episode reward: [(0, '55.122')] [2024-09-30 01:47:40,717][1157819] Updated weights for policy 0, policy_version 82388 (0.0006) [2024-09-30 01:47:41,200][1157819] Updated weights for policy 0, policy_version 82398 (0.0006) [2024-09-30 01:47:41,692][1157819] Updated weights for policy 0, policy_version 82408 (0.0006) [2024-09-30 01:47:42,180][1157819] Updated weights for policy 0, policy_version 82418 (0.0006) [2024-09-30 01:47:42,668][1157819] Updated weights for policy 0, policy_version 82428 (0.0006) [2024-09-30 01:47:43,162][1157819] Updated weights for policy 0, policy_version 82438 (0.0006) [2024-09-30 01:47:43,648][1157819] Updated weights for policy 0, policy_version 82448 (0.0006) [2024-09-30 01:47:44,146][1157819] Updated weights for policy 0, policy_version 82458 (0.0006) [2024-09-30 01:47:44,627][1157736] Signal inference workers to stop experience collection... (5350 times) [2024-09-30 01:47:44,628][1157736] Signal inference workers to resume experience collection... (5350 times) [2024-09-30 01:47:44,631][1157819] InferenceWorker_p0-w0: stopping experience collection (5350 times) [2024-09-30 01:47:44,632][1157819] InferenceWorker_p0-w0: resuming experience collection (5350 times) [2024-09-30 01:47:44,643][1157819] Updated weights for policy 0, policy_version 82468 (0.0006) [2024-09-30 01:47:45,122][1157819] Updated weights for policy 0, policy_version 82478 (0.0006) [2024-09-30 01:47:45,466][1157520] Fps is (10 sec: 83558.7, 60 sec: 79803.9, 300 sec: 77865.7). Total num frames: 337854464. Throughput: 0: 20298.2. Samples: 74429328. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:47:45,466][1157520] Avg episode reward: [(0, '52.580')] [2024-09-30 01:47:45,620][1157819] Updated weights for policy 0, policy_version 82488 (0.0006) [2024-09-30 01:47:46,130][1157819] Updated weights for policy 0, policy_version 82498 (0.0006) [2024-09-30 01:47:46,615][1157819] Updated weights for policy 0, policy_version 82508 (0.0006) [2024-09-30 01:47:47,105][1157819] Updated weights for policy 0, policy_version 82518 (0.0006) [2024-09-30 01:47:47,604][1157819] Updated weights for policy 0, policy_version 82528 (0.0006) [2024-09-30 01:47:48,102][1157819] Updated weights for policy 0, policy_version 82538 (0.0006) [2024-09-30 01:47:48,599][1157819] Updated weights for policy 0, policy_version 82548 (0.0006) [2024-09-30 01:47:49,106][1157819] Updated weights for policy 0, policy_version 82558 (0.0006) [2024-09-30 01:47:49,609][1157819] Updated weights for policy 0, policy_version 82568 (0.0006) [2024-09-30 01:47:50,091][1157819] Updated weights for policy 0, policy_version 82578 (0.0006) [2024-09-30 01:47:50,466][1157520] Fps is (10 sec: 83148.3, 60 sec: 80554.7, 300 sec: 77976.7). Total num frames: 338268160. Throughput: 0: 20387.7. Samples: 74553548. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:47:50,466][1157520] Avg episode reward: [(0, '53.477')] [2024-09-30 01:47:50,593][1157819] Updated weights for policy 0, policy_version 82588 (0.0006) [2024-09-30 01:47:51,096][1157819] Updated weights for policy 0, policy_version 82598 (0.0006) [2024-09-30 01:47:51,598][1157819] Updated weights for policy 0, policy_version 82608 (0.0006) [2024-09-30 01:47:52,069][1157819] Updated weights for policy 0, policy_version 82618 (0.0006) [2024-09-30 01:47:52,584][1157819] Updated weights for policy 0, policy_version 82628 (0.0006) [2024-09-30 01:47:53,080][1157819] Updated weights for policy 0, policy_version 82638 (0.0006) [2024-09-30 01:47:53,563][1157819] Updated weights for policy 0, policy_version 82648 (0.0006) [2024-09-30 01:47:54,081][1157819] Updated weights for policy 0, policy_version 82658 (0.0006) [2024-09-30 01:47:54,527][1157819] Updated weights for policy 0, policy_version 82668 (0.0006) [2024-09-30 01:47:55,038][1157819] Updated weights for policy 0, policy_version 82678 (0.0006) [2024-09-30 01:47:55,466][1157520] Fps is (10 sec: 82739.0, 60 sec: 81305.7, 300 sec: 78101.7). Total num frames: 338681856. Throughput: 0: 20434.2. Samples: 74615560. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 01:47:55,466][1157520] Avg episode reward: [(0, '53.819')] [2024-09-30 01:47:55,552][1157819] Updated weights for policy 0, policy_version 82688 (0.0006) [2024-09-30 01:47:56,023][1157819] Updated weights for policy 0, policy_version 82698 (0.0006) [2024-09-30 01:47:56,513][1157819] Updated weights for policy 0, policy_version 82708 (0.0006) [2024-09-30 01:47:57,014][1157819] Updated weights for policy 0, policy_version 82718 (0.0006) [2024-09-30 01:47:57,518][1157819] Updated weights for policy 0, policy_version 82728 (0.0006) [2024-09-30 01:47:58,026][1157819] Updated weights for policy 0, policy_version 82738 (0.0006) [2024-09-30 01:47:58,518][1157819] Updated weights for policy 0, policy_version 82748 (0.0006) [2024-09-30 01:47:59,028][1157819] Updated weights for policy 0, policy_version 82758 (0.0006) [2024-09-30 01:47:59,532][1157819] Updated weights for policy 0, policy_version 82768 (0.0006) [2024-09-30 01:48:00,013][1157819] Updated weights for policy 0, policy_version 82778 (0.0006) [2024-09-30 01:48:00,466][1157520] Fps is (10 sec: 82330.1, 60 sec: 81578.7, 300 sec: 78157.2). Total num frames: 339091456. Throughput: 0: 20508.9. Samples: 74739152. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 01:48:00,466][1157520] Avg episode reward: [(0, '54.807')] [2024-09-30 01:48:00,511][1157819] Updated weights for policy 0, policy_version 82788 (0.0006) [2024-09-30 01:48:01,032][1157819] Updated weights for policy 0, policy_version 82798 (0.0006) [2024-09-30 01:48:01,498][1157819] Updated weights for policy 0, policy_version 82808 (0.0006) [2024-09-30 01:48:01,994][1157819] Updated weights for policy 0, policy_version 82818 (0.0006) [2024-09-30 01:48:02,490][1157819] Updated weights for policy 0, policy_version 82828 (0.0006) [2024-09-30 01:48:02,973][1157819] Updated weights for policy 0, policy_version 82838 (0.0006) [2024-09-30 01:48:03,458][1157819] Updated weights for policy 0, policy_version 82848 (0.0006) [2024-09-30 01:48:03,945][1157819] Updated weights for policy 0, policy_version 82858 (0.0006) [2024-09-30 01:48:04,445][1157819] Updated weights for policy 0, policy_version 82868 (0.0006) [2024-09-30 01:48:04,935][1157819] Updated weights for policy 0, policy_version 82878 (0.0006) [2024-09-30 01:48:05,396][1157819] Updated weights for policy 0, policy_version 82888 (0.0006) [2024-09-30 01:48:05,466][1157520] Fps is (10 sec: 83149.0, 60 sec: 81988.2, 300 sec: 78226.7). Total num frames: 339513344. Throughput: 0: 20602.5. Samples: 74864228. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 01:48:05,466][1157520] Avg episode reward: [(0, '51.658')] [2024-09-30 01:48:05,471][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000082889_339513344.pth... [2024-09-30 01:48:05,516][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000078332_320847872.pth [2024-09-30 01:48:05,891][1157819] Updated weights for policy 0, policy_version 82898 (0.0006) [2024-09-30 01:48:06,393][1157819] Updated weights for policy 0, policy_version 82908 (0.0006) [2024-09-30 01:48:06,894][1157819] Updated weights for policy 0, policy_version 82918 (0.0006) [2024-09-30 01:48:07,377][1157819] Updated weights for policy 0, policy_version 82928 (0.0006) [2024-09-30 01:48:07,904][1157819] Updated weights for policy 0, policy_version 82938 (0.0006) [2024-09-30 01:48:08,393][1157819] Updated weights for policy 0, policy_version 82948 (0.0006) [2024-09-30 01:48:08,851][1157819] Updated weights for policy 0, policy_version 82958 (0.0006) [2024-09-30 01:48:09,370][1157819] Updated weights for policy 0, policy_version 82968 (0.0006) [2024-09-30 01:48:09,856][1157819] Updated weights for policy 0, policy_version 82978 (0.0006) [2024-09-30 01:48:10,346][1157819] Updated weights for policy 0, policy_version 82988 (0.0006) [2024-09-30 01:48:10,466][1157520] Fps is (10 sec: 83148.8, 60 sec: 82056.5, 300 sec: 78254.5). Total num frames: 339922944. Throughput: 0: 20655.3. Samples: 74926588. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 01:48:10,466][1157520] Avg episode reward: [(0, '52.009')] [2024-09-30 01:48:10,844][1157819] Updated weights for policy 0, policy_version 82998 (0.0006) [2024-09-30 01:48:11,355][1157819] Updated weights for policy 0, policy_version 83008 (0.0006) [2024-09-30 01:48:11,855][1157819] Updated weights for policy 0, policy_version 83018 (0.0007) [2024-09-30 01:48:12,350][1157819] Updated weights for policy 0, policy_version 83028 (0.0006) [2024-09-30 01:48:12,845][1157819] Updated weights for policy 0, policy_version 83038 (0.0006) [2024-09-30 01:48:13,344][1157819] Updated weights for policy 0, policy_version 83048 (0.0006) [2024-09-30 01:48:13,872][1157819] Updated weights for policy 0, policy_version 83058 (0.0006) [2024-09-30 01:48:14,421][1157819] Updated weights for policy 0, policy_version 83068 (0.0006) [2024-09-30 01:48:14,914][1157819] Updated weights for policy 0, policy_version 83078 (0.0006) [2024-09-30 01:48:15,435][1157819] Updated weights for policy 0, policy_version 83088 (0.0006) [2024-09-30 01:48:15,466][1157520] Fps is (10 sec: 81510.3, 60 sec: 82193.2, 300 sec: 78254.4). Total num frames: 340328448. Throughput: 0: 20762.0. Samples: 75049644. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 01:48:15,466][1157520] Avg episode reward: [(0, '52.248')] [2024-09-30 01:48:15,951][1157819] Updated weights for policy 0, policy_version 83098 (0.0006) [2024-09-30 01:48:16,449][1157819] Updated weights for policy 0, policy_version 83108 (0.0006) [2024-09-30 01:48:16,963][1157819] Updated weights for policy 0, policy_version 83118 (0.0006) [2024-09-30 01:48:17,496][1157819] Updated weights for policy 0, policy_version 83128 (0.0006) [2024-09-30 01:48:17,998][1157819] Updated weights for policy 0, policy_version 83138 (0.0006) [2024-09-30 01:48:18,464][1157819] Updated weights for policy 0, policy_version 83148 (0.0006) [2024-09-30 01:48:18,974][1157819] Updated weights for policy 0, policy_version 83158 (0.0006) [2024-09-30 01:48:19,474][1157819] Updated weights for policy 0, policy_version 83168 (0.0006) [2024-09-30 01:48:19,955][1157819] Updated weights for policy 0, policy_version 83178 (0.0006) [2024-09-30 01:48:20,431][1157819] Updated weights for policy 0, policy_version 83188 (0.0006) [2024-09-30 01:48:20,466][1157520] Fps is (10 sec: 81510.3, 60 sec: 82261.3, 300 sec: 78268.3). Total num frames: 340738048. Throughput: 0: 20669.8. Samples: 75170700. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 01:48:20,466][1157520] Avg episode reward: [(0, '52.398')] [2024-09-30 01:48:20,926][1157819] Updated weights for policy 0, policy_version 83198 (0.0006) [2024-09-30 01:48:21,366][1157819] Updated weights for policy 0, policy_version 83208 (0.0006) [2024-09-30 01:48:21,854][1157819] Updated weights for policy 0, policy_version 83218 (0.0006) [2024-09-30 01:48:22,342][1157819] Updated weights for policy 0, policy_version 83228 (0.0006) [2024-09-30 01:48:22,822][1157819] Updated weights for policy 0, policy_version 83238 (0.0006) [2024-09-30 01:48:23,324][1157819] Updated weights for policy 0, policy_version 83248 (0.0006) [2024-09-30 01:48:23,819][1157819] Updated weights for policy 0, policy_version 83258 (0.0006) [2024-09-30 01:48:24,278][1157819] Updated weights for policy 0, policy_version 83268 (0.0006) [2024-09-30 01:48:24,777][1157819] Updated weights for policy 0, policy_version 83278 (0.0006) [2024-09-30 01:48:25,291][1157819] Updated weights for policy 0, policy_version 83288 (0.0006) [2024-09-30 01:48:25,466][1157520] Fps is (10 sec: 83558.0, 60 sec: 82739.2, 300 sec: 78365.5). Total num frames: 341164032. Throughput: 0: 20688.5. Samples: 75234924. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 01:48:25,466][1157520] Avg episode reward: [(0, '51.132')] [2024-09-30 01:48:25,605][1157736] Signal inference workers to stop experience collection... (5400 times) [2024-09-30 01:48:25,606][1157736] Signal inference workers to resume experience collection... (5400 times) [2024-09-30 01:48:25,609][1157819] InferenceWorker_p0-w0: stopping experience collection (5400 times) [2024-09-30 01:48:25,611][1157819] InferenceWorker_p0-w0: resuming experience collection (5400 times) [2024-09-30 01:48:25,778][1157819] Updated weights for policy 0, policy_version 83298 (0.0006) [2024-09-30 01:48:26,280][1157819] Updated weights for policy 0, policy_version 83308 (0.0006) [2024-09-30 01:48:26,789][1157819] Updated weights for policy 0, policy_version 83318 (0.0006) [2024-09-30 01:48:27,303][1157819] Updated weights for policy 0, policy_version 83328 (0.0006) [2024-09-30 01:48:27,781][1157819] Updated weights for policy 0, policy_version 83338 (0.0006) [2024-09-30 01:48:28,278][1157819] Updated weights for policy 0, policy_version 83348 (0.0006) [2024-09-30 01:48:28,773][1157819] Updated weights for policy 0, policy_version 83358 (0.0006) [2024-09-30 01:48:29,252][1157819] Updated weights for policy 0, policy_version 83368 (0.0006) [2024-09-30 01:48:29,750][1157819] Updated weights for policy 0, policy_version 83378 (0.0006) [2024-09-30 01:48:30,261][1157819] Updated weights for policy 0, policy_version 83388 (0.0006) [2024-09-30 01:48:30,466][1157520] Fps is (10 sec: 83558.4, 60 sec: 82944.0, 300 sec: 78393.3). Total num frames: 341573632. Throughput: 0: 20651.2. Samples: 75358632. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 01:48:30,466][1157520] Avg episode reward: [(0, '52.513')] [2024-09-30 01:48:30,721][1157819] Updated weights for policy 0, policy_version 83398 (0.0006) [2024-09-30 01:48:31,193][1157819] Updated weights for policy 0, policy_version 83408 (0.0006) [2024-09-30 01:48:31,673][1157819] Updated weights for policy 0, policy_version 83418 (0.0006) [2024-09-30 01:48:32,142][1157819] Updated weights for policy 0, policy_version 83428 (0.0006) [2024-09-30 01:48:32,623][1157819] Updated weights for policy 0, policy_version 83438 (0.0006) [2024-09-30 01:48:33,121][1157819] Updated weights for policy 0, policy_version 83448 (0.0006) [2024-09-30 01:48:33,600][1157819] Updated weights for policy 0, policy_version 83458 (0.0006) [2024-09-30 01:48:34,116][1157819] Updated weights for policy 0, policy_version 83468 (0.0006) [2024-09-30 01:48:34,560][1157819] Updated weights for policy 0, policy_version 83478 (0.0006) [2024-09-30 01:48:35,052][1157819] Updated weights for policy 0, policy_version 83488 (0.0006) [2024-09-30 01:48:35,466][1157520] Fps is (10 sec: 83557.7, 60 sec: 83012.1, 300 sec: 78490.4). Total num frames: 341999616. Throughput: 0: 20719.2. Samples: 75485912. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 01:48:35,466][1157520] Avg episode reward: [(0, '54.917')] [2024-09-30 01:48:35,552][1157819] Updated weights for policy 0, policy_version 83498 (0.0006) [2024-09-30 01:48:36,041][1157819] Updated weights for policy 0, policy_version 83508 (0.0006) [2024-09-30 01:48:36,508][1157819] Updated weights for policy 0, policy_version 83518 (0.0006) [2024-09-30 01:48:37,010][1157819] Updated weights for policy 0, policy_version 83528 (0.0006) [2024-09-30 01:48:37,466][1157819] Updated weights for policy 0, policy_version 83538 (0.0006) [2024-09-30 01:48:37,958][1157819] Updated weights for policy 0, policy_version 83548 (0.0006) [2024-09-30 01:48:38,452][1157819] Updated weights for policy 0, policy_version 83558 (0.0006) [2024-09-30 01:48:38,942][1157819] Updated weights for policy 0, policy_version 83568 (0.0006) [2024-09-30 01:48:39,433][1157819] Updated weights for policy 0, policy_version 83578 (0.0006) [2024-09-30 01:48:39,930][1157819] Updated weights for policy 0, policy_version 83588 (0.0006) [2024-09-30 01:48:40,407][1157819] Updated weights for policy 0, policy_version 83598 (0.0006) [2024-09-30 01:48:40,466][1157520] Fps is (10 sec: 84377.9, 60 sec: 83012.3, 300 sec: 78559.9). Total num frames: 342417408. Throughput: 0: 20748.2. Samples: 75549228. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 01:48:40,466][1157520] Avg episode reward: [(0, '54.395')] [2024-09-30 01:48:40,899][1157819] Updated weights for policy 0, policy_version 83608 (0.0006) [2024-09-30 01:48:41,387][1157819] Updated weights for policy 0, policy_version 83618 (0.0006) [2024-09-30 01:48:41,876][1157819] Updated weights for policy 0, policy_version 83628 (0.0006) [2024-09-30 01:48:42,354][1157819] Updated weights for policy 0, policy_version 83638 (0.0006) [2024-09-30 01:48:42,846][1157819] Updated weights for policy 0, policy_version 83648 (0.0006) [2024-09-30 01:48:43,341][1157819] Updated weights for policy 0, policy_version 83658 (0.0006) [2024-09-30 01:48:43,809][1157819] Updated weights for policy 0, policy_version 83668 (0.0006) [2024-09-30 01:48:44,337][1157819] Updated weights for policy 0, policy_version 83678 (0.0006) [2024-09-30 01:48:44,825][1157819] Updated weights for policy 0, policy_version 83688 (0.0006) [2024-09-30 01:48:45,325][1157819] Updated weights for policy 0, policy_version 83698 (0.0006) [2024-09-30 01:48:45,466][1157520] Fps is (10 sec: 83558.9, 60 sec: 83012.2, 300 sec: 78629.3). Total num frames: 342835200. Throughput: 0: 20805.9. Samples: 75675420. Policy #0 lag: (min: 0.0, avg: 1.4, max: 4.0) [2024-09-30 01:48:45,466][1157520] Avg episode reward: [(0, '56.170')] [2024-09-30 01:48:45,836][1157819] Updated weights for policy 0, policy_version 83708 (0.0006) [2024-09-30 01:48:46,336][1157819] Updated weights for policy 0, policy_version 83718 (0.0006) [2024-09-30 01:48:46,870][1157819] Updated weights for policy 0, policy_version 83728 (0.0006) [2024-09-30 01:48:47,376][1157819] Updated weights for policy 0, policy_version 83738 (0.0006) [2024-09-30 01:48:47,894][1157819] Updated weights for policy 0, policy_version 83748 (0.0006) [2024-09-30 01:48:48,435][1157819] Updated weights for policy 0, policy_version 83758 (0.0006) [2024-09-30 01:48:48,941][1157819] Updated weights for policy 0, policy_version 83768 (0.0006) [2024-09-30 01:48:49,453][1157819] Updated weights for policy 0, policy_version 83778 (0.0006) [2024-09-30 01:48:49,971][1157819] Updated weights for policy 0, policy_version 83788 (0.0006) [2024-09-30 01:48:50,466][1157520] Fps is (10 sec: 81510.5, 60 sec: 82739.3, 300 sec: 78629.3). Total num frames: 343232512. Throughput: 0: 20690.4. Samples: 75795296. Policy #0 lag: (min: 0.0, avg: 1.4, max: 4.0) [2024-09-30 01:48:50,466][1157520] Avg episode reward: [(0, '56.263')] [2024-09-30 01:48:50,476][1157819] Updated weights for policy 0, policy_version 83798 (0.0006) [2024-09-30 01:48:51,010][1157819] Updated weights for policy 0, policy_version 83808 (0.0007) [2024-09-30 01:48:51,559][1157819] Updated weights for policy 0, policy_version 83818 (0.0007) [2024-09-30 01:48:52,077][1157819] Updated weights for policy 0, policy_version 83828 (0.0006) [2024-09-30 01:48:52,594][1157819] Updated weights for policy 0, policy_version 83838 (0.0006) [2024-09-30 01:48:53,101][1157819] Updated weights for policy 0, policy_version 83848 (0.0006) [2024-09-30 01:48:53,632][1157819] Updated weights for policy 0, policy_version 83858 (0.0006) [2024-09-30 01:48:54,144][1157819] Updated weights for policy 0, policy_version 83868 (0.0005) [2024-09-30 01:48:54,658][1157819] Updated weights for policy 0, policy_version 83878 (0.0006) [2024-09-30 01:48:55,142][1157819] Updated weights for policy 0, policy_version 83888 (0.0006) [2024-09-30 01:48:55,466][1157520] Fps is (10 sec: 79462.9, 60 sec: 82466.1, 300 sec: 78615.4). Total num frames: 343629824. Throughput: 0: 20604.5. Samples: 75853792. Policy #0 lag: (min: 0.0, avg: 1.4, max: 4.0) [2024-09-30 01:48:55,466][1157520] Avg episode reward: [(0, '53.116')] [2024-09-30 01:48:55,626][1157819] Updated weights for policy 0, policy_version 83898 (0.0006) [2024-09-30 01:48:56,135][1157819] Updated weights for policy 0, policy_version 83908 (0.0006) [2024-09-30 01:48:56,676][1157819] Updated weights for policy 0, policy_version 83918 (0.0006) [2024-09-30 01:48:57,194][1157819] Updated weights for policy 0, policy_version 83928 (0.0006) [2024-09-30 01:48:57,735][1157819] Updated weights for policy 0, policy_version 83938 (0.0006) [2024-09-30 01:48:58,247][1157819] Updated weights for policy 0, policy_version 83948 (0.0006) [2024-09-30 01:48:58,758][1157819] Updated weights for policy 0, policy_version 83958 (0.0006) [2024-09-30 01:48:59,250][1157819] Updated weights for policy 0, policy_version 83968 (0.0006) [2024-09-30 01:48:59,773][1157819] Updated weights for policy 0, policy_version 83978 (0.0006) [2024-09-30 01:49:00,299][1157819] Updated weights for policy 0, policy_version 83988 (0.0006) [2024-09-30 01:49:00,466][1157520] Fps is (10 sec: 79462.3, 60 sec: 82261.4, 300 sec: 78573.8). Total num frames: 344027136. Throughput: 0: 20540.1. Samples: 75973948. Policy #0 lag: (min: 0.0, avg: 1.4, max: 4.0) [2024-09-30 01:49:00,466][1157520] Avg episode reward: [(0, '55.604')] [2024-09-30 01:49:00,846][1157819] Updated weights for policy 0, policy_version 83998 (0.0006) [2024-09-30 01:49:01,354][1157819] Updated weights for policy 0, policy_version 84008 (0.0006) [2024-09-30 01:49:01,888][1157819] Updated weights for policy 0, policy_version 84018 (0.0006) [2024-09-30 01:49:02,415][1157819] Updated weights for policy 0, policy_version 84028 (0.0006) [2024-09-30 01:49:02,947][1157819] Updated weights for policy 0, policy_version 84038 (0.0006) [2024-09-30 01:49:03,357][1157736] Signal inference workers to stop experience collection... (5450 times) [2024-09-30 01:49:03,357][1157736] Signal inference workers to resume experience collection... (5450 times) [2024-09-30 01:49:03,361][1157819] InferenceWorker_p0-w0: stopping experience collection (5450 times) [2024-09-30 01:49:03,363][1157819] InferenceWorker_p0-w0: resuming experience collection (5450 times) [2024-09-30 01:49:03,477][1157819] Updated weights for policy 0, policy_version 84048 (0.0006) [2024-09-30 01:49:03,985][1157819] Updated weights for policy 0, policy_version 84058 (0.0006) [2024-09-30 01:49:04,492][1157819] Updated weights for policy 0, policy_version 84068 (0.0006) [2024-09-30 01:49:04,977][1157819] Updated weights for policy 0, policy_version 84078 (0.0006) [2024-09-30 01:49:05,466][1157520] Fps is (10 sec: 79052.8, 60 sec: 81783.4, 300 sec: 78518.2). Total num frames: 344420352. Throughput: 0: 20472.5. Samples: 76091960. Policy #0 lag: (min: 0.0, avg: 1.4, max: 4.0) [2024-09-30 01:49:05,466][1157520] Avg episode reward: [(0, '54.607')] [2024-09-30 01:49:05,489][1157819] Updated weights for policy 0, policy_version 84088 (0.0006) [2024-09-30 01:49:06,027][1157819] Updated weights for policy 0, policy_version 84098 (0.0006) [2024-09-30 01:49:06,573][1157819] Updated weights for policy 0, policy_version 84108 (0.0006) [2024-09-30 01:49:07,069][1157819] Updated weights for policy 0, policy_version 84118 (0.0006) [2024-09-30 01:49:07,592][1157819] Updated weights for policy 0, policy_version 84128 (0.0006) [2024-09-30 01:49:08,162][1157819] Updated weights for policy 0, policy_version 84138 (0.0006) [2024-09-30 01:49:08,696][1157819] Updated weights for policy 0, policy_version 84148 (0.0006) [2024-09-30 01:49:09,191][1157819] Updated weights for policy 0, policy_version 84158 (0.0006) [2024-09-30 01:49:09,732][1157819] Updated weights for policy 0, policy_version 84168 (0.0006) [2024-09-30 01:49:10,293][1157819] Updated weights for policy 0, policy_version 84178 (0.0006) [2024-09-30 01:49:10,466][1157520] Fps is (10 sec: 77824.1, 60 sec: 81373.9, 300 sec: 78476.6). Total num frames: 344805376. Throughput: 0: 20341.8. Samples: 76150304. Policy #0 lag: (min: 0.0, avg: 1.4, max: 4.0) [2024-09-30 01:49:10,466][1157520] Avg episode reward: [(0, '54.642')] [2024-09-30 01:49:10,823][1157819] Updated weights for policy 0, policy_version 84188 (0.0006) [2024-09-30 01:49:11,356][1157819] Updated weights for policy 0, policy_version 84198 (0.0006) [2024-09-30 01:49:11,874][1157819] Updated weights for policy 0, policy_version 84208 (0.0006) [2024-09-30 01:49:12,359][1157819] Updated weights for policy 0, policy_version 84218 (0.0005) [2024-09-30 01:49:12,851][1157819] Updated weights for policy 0, policy_version 84228 (0.0006) [2024-09-30 01:49:13,370][1157819] Updated weights for policy 0, policy_version 84238 (0.0006) [2024-09-30 01:49:13,867][1157819] Updated weights for policy 0, policy_version 84248 (0.0006) [2024-09-30 01:49:14,355][1157819] Updated weights for policy 0, policy_version 84258 (0.0006) [2024-09-30 01:49:14,898][1157819] Updated weights for policy 0, policy_version 84268 (0.0006) [2024-09-30 01:49:15,394][1157819] Updated weights for policy 0, policy_version 84278 (0.0006) [2024-09-30 01:49:15,466][1157520] Fps is (10 sec: 78643.2, 60 sec: 81305.6, 300 sec: 78462.7). Total num frames: 345206784. Throughput: 0: 20215.8. Samples: 76268344. Policy #0 lag: (min: 0.0, avg: 1.4, max: 4.0) [2024-09-30 01:49:15,466][1157520] Avg episode reward: [(0, '54.755')] [2024-09-30 01:49:15,911][1157819] Updated weights for policy 0, policy_version 84288 (0.0006) [2024-09-30 01:49:16,404][1157819] Updated weights for policy 0, policy_version 84298 (0.0006) [2024-09-30 01:49:16,916][1157819] Updated weights for policy 0, policy_version 84308 (0.0006) [2024-09-30 01:49:17,403][1157819] Updated weights for policy 0, policy_version 84318 (0.0006) [2024-09-30 01:49:17,918][1157819] Updated weights for policy 0, policy_version 84328 (0.0006) [2024-09-30 01:49:18,411][1157819] Updated weights for policy 0, policy_version 84338 (0.0006) [2024-09-30 01:49:18,930][1157819] Updated weights for policy 0, policy_version 84348 (0.0006) [2024-09-30 01:49:19,480][1157819] Updated weights for policy 0, policy_version 84358 (0.0006) [2024-09-30 01:49:19,983][1157819] Updated weights for policy 0, policy_version 84368 (0.0006) [2024-09-30 01:49:20,466][1157520] Fps is (10 sec: 79871.9, 60 sec: 81100.9, 300 sec: 78448.8). Total num frames: 345604096. Throughput: 0: 20070.3. Samples: 76389072. Policy #0 lag: (min: 0.0, avg: 1.4, max: 4.0) [2024-09-30 01:49:20,466][1157520] Avg episode reward: [(0, '54.229')] [2024-09-30 01:49:20,502][1157819] Updated weights for policy 0, policy_version 84378 (0.0006) [2024-09-30 01:49:21,010][1157819] Updated weights for policy 0, policy_version 84388 (0.0006) [2024-09-30 01:49:21,542][1157819] Updated weights for policy 0, policy_version 84398 (0.0006) [2024-09-30 01:49:22,047][1157819] Updated weights for policy 0, policy_version 84408 (0.0006) [2024-09-30 01:49:22,560][1157819] Updated weights for policy 0, policy_version 84418 (0.0006) [2024-09-30 01:49:23,106][1157819] Updated weights for policy 0, policy_version 84428 (0.0006) [2024-09-30 01:49:23,616][1157819] Updated weights for policy 0, policy_version 84438 (0.0006) [2024-09-30 01:49:24,140][1157819] Updated weights for policy 0, policy_version 84448 (0.0006) [2024-09-30 01:49:24,644][1157819] Updated weights for policy 0, policy_version 84458 (0.0006) [2024-09-30 01:49:25,220][1157819] Updated weights for policy 0, policy_version 84468 (0.0006) [2024-09-30 01:49:25,466][1157520] Fps is (10 sec: 79051.9, 60 sec: 80554.6, 300 sec: 78504.3). Total num frames: 345997312. Throughput: 0: 19977.1. Samples: 76448200. Policy #0 lag: (min: 0.0, avg: 1.4, max: 4.0) [2024-09-30 01:49:25,466][1157520] Avg episode reward: [(0, '53.421')] [2024-09-30 01:49:25,771][1157819] Updated weights for policy 0, policy_version 84478 (0.0006) [2024-09-30 01:49:26,260][1157819] Updated weights for policy 0, policy_version 84488 (0.0006) [2024-09-30 01:49:26,820][1157819] Updated weights for policy 0, policy_version 84498 (0.0006) [2024-09-30 01:49:27,382][1157819] Updated weights for policy 0, policy_version 84508 (0.0006) [2024-09-30 01:49:27,943][1157819] Updated weights for policy 0, policy_version 84518 (0.0006) [2024-09-30 01:49:28,482][1157819] Updated weights for policy 0, policy_version 84528 (0.0006) [2024-09-30 01:49:28,999][1157819] Updated weights for policy 0, policy_version 84538 (0.0006) [2024-09-30 01:49:29,508][1157819] Updated weights for policy 0, policy_version 84548 (0.0006) [2024-09-30 01:49:30,037][1157819] Updated weights for policy 0, policy_version 84558 (0.0006) [2024-09-30 01:49:30,466][1157520] Fps is (10 sec: 77414.6, 60 sec: 80076.9, 300 sec: 78462.7). Total num frames: 346378240. Throughput: 0: 19715.3. Samples: 76562608. Policy #0 lag: (min: 0.0, avg: 1.4, max: 4.0) [2024-09-30 01:49:30,466][1157520] Avg episode reward: [(0, '55.317')] [2024-09-30 01:49:30,580][1157819] Updated weights for policy 0, policy_version 84568 (0.0006) [2024-09-30 01:49:31,143][1157819] Updated weights for policy 0, policy_version 84578 (0.0006) [2024-09-30 01:49:31,680][1157819] Updated weights for policy 0, policy_version 84588 (0.0006) [2024-09-30 01:49:32,250][1157819] Updated weights for policy 0, policy_version 84598 (0.0006) [2024-09-30 01:49:32,774][1157819] Updated weights for policy 0, policy_version 84608 (0.0006) [2024-09-30 01:49:33,361][1157819] Updated weights for policy 0, policy_version 84618 (0.0006) [2024-09-30 01:49:33,541][1157736] Signal inference workers to stop experience collection... (5500 times) [2024-09-30 01:49:33,541][1157736] Signal inference workers to resume experience collection... (5500 times) [2024-09-30 01:49:33,547][1157819] InferenceWorker_p0-w0: stopping experience collection (5500 times) [2024-09-30 01:49:33,547][1157819] InferenceWorker_p0-w0: resuming experience collection (5500 times) [2024-09-30 01:49:33,875][1157819] Updated weights for policy 0, policy_version 84628 (0.0006) [2024-09-30 01:49:34,451][1157819] Updated weights for policy 0, policy_version 84638 (0.0006) [2024-09-30 01:49:35,019][1157819] Updated weights for policy 0, policy_version 84648 (0.0006) [2024-09-30 01:49:35,466][1157520] Fps is (10 sec: 74957.3, 60 sec: 79121.2, 300 sec: 78434.9). Total num frames: 346746880. Throughput: 0: 19560.0. Samples: 76675496. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 01:49:35,466][1157520] Avg episode reward: [(0, '55.547')] [2024-09-30 01:49:35,553][1157819] Updated weights for policy 0, policy_version 84658 (0.0006) [2024-09-30 01:49:36,109][1157819] Updated weights for policy 0, policy_version 84668 (0.0006) [2024-09-30 01:49:36,656][1157819] Updated weights for policy 0, policy_version 84678 (0.0006) [2024-09-30 01:49:37,172][1157819] Updated weights for policy 0, policy_version 84688 (0.0006) [2024-09-30 01:49:37,732][1157819] Updated weights for policy 0, policy_version 84698 (0.0006) [2024-09-30 01:49:38,250][1157819] Updated weights for policy 0, policy_version 84708 (0.0006) [2024-09-30 01:49:38,811][1157819] Updated weights for policy 0, policy_version 84718 (0.0006) [2024-09-30 01:49:39,354][1157819] Updated weights for policy 0, policy_version 84728 (0.0006) [2024-09-30 01:49:39,932][1157819] Updated weights for policy 0, policy_version 84738 (0.0006) [2024-09-30 01:49:40,463][1157819] Updated weights for policy 0, policy_version 84748 (0.0006) [2024-09-30 01:49:40,466][1157520] Fps is (10 sec: 74956.6, 60 sec: 78506.7, 300 sec: 78448.8). Total num frames: 347127808. Throughput: 0: 19521.7. Samples: 76732268. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 01:49:40,466][1157520] Avg episode reward: [(0, '54.208')] [2024-09-30 01:49:41,028][1157819] Updated weights for policy 0, policy_version 84758 (0.0006) [2024-09-30 01:49:41,592][1157819] Updated weights for policy 0, policy_version 84768 (0.0006) [2024-09-30 01:49:42,125][1157819] Updated weights for policy 0, policy_version 84778 (0.0006) [2024-09-30 01:49:42,711][1157819] Updated weights for policy 0, policy_version 84788 (0.0006) [2024-09-30 01:49:43,243][1157819] Updated weights for policy 0, policy_version 84798 (0.0006) [2024-09-30 01:49:43,798][1157819] Updated weights for policy 0, policy_version 84808 (0.0006) [2024-09-30 01:49:44,338][1157819] Updated weights for policy 0, policy_version 84818 (0.0006) [2024-09-30 01:49:44,895][1157819] Updated weights for policy 0, policy_version 84828 (0.0006) [2024-09-30 01:49:45,418][1157819] Updated weights for policy 0, policy_version 84838 (0.0006) [2024-09-30 01:49:45,466][1157520] Fps is (10 sec: 74956.5, 60 sec: 77687.4, 300 sec: 78434.9). Total num frames: 347496448. Throughput: 0: 19320.9. Samples: 76843392. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 01:49:45,466][1157520] Avg episode reward: [(0, '55.494')] [2024-09-30 01:49:45,962][1157819] Updated weights for policy 0, policy_version 84848 (0.0006) [2024-09-30 01:49:46,505][1157819] Updated weights for policy 0, policy_version 84858 (0.0006) [2024-09-30 01:49:47,013][1157819] Updated weights for policy 0, policy_version 84868 (0.0006) [2024-09-30 01:49:47,533][1157819] Updated weights for policy 0, policy_version 84878 (0.0006) [2024-09-30 01:49:48,036][1157819] Updated weights for policy 0, policy_version 84888 (0.0006) [2024-09-30 01:49:48,530][1157819] Updated weights for policy 0, policy_version 84898 (0.0006) [2024-09-30 01:49:49,034][1157819] Updated weights for policy 0, policy_version 84908 (0.0006) [2024-09-30 01:49:49,558][1157819] Updated weights for policy 0, policy_version 84918 (0.0006) [2024-09-30 01:49:50,095][1157819] Updated weights for policy 0, policy_version 84928 (0.0006) [2024-09-30 01:49:50,466][1157520] Fps is (10 sec: 76594.7, 60 sec: 77687.4, 300 sec: 78448.8). Total num frames: 347893760. Throughput: 0: 19298.5. Samples: 76960392. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 01:49:50,466][1157520] Avg episode reward: [(0, '55.109')] [2024-09-30 01:49:50,604][1157819] Updated weights for policy 0, policy_version 84938 (0.0006) [2024-09-30 01:49:51,094][1157819] Updated weights for policy 0, policy_version 84948 (0.0006) [2024-09-30 01:49:51,642][1157819] Updated weights for policy 0, policy_version 84958 (0.0006) [2024-09-30 01:49:52,158][1157819] Updated weights for policy 0, policy_version 84968 (0.0006) [2024-09-30 01:49:52,682][1157819] Updated weights for policy 0, policy_version 84978 (0.0006) [2024-09-30 01:49:53,181][1157819] Updated weights for policy 0, policy_version 84988 (0.0006) [2024-09-30 01:49:53,689][1157819] Updated weights for policy 0, policy_version 84998 (0.0006) [2024-09-30 01:49:54,222][1157819] Updated weights for policy 0, policy_version 85008 (0.0006) [2024-09-30 01:49:54,744][1157819] Updated weights for policy 0, policy_version 85018 (0.0006) [2024-09-30 01:49:55,273][1157819] Updated weights for policy 0, policy_version 85028 (0.0006) [2024-09-30 01:49:55,466][1157520] Fps is (10 sec: 79462.7, 60 sec: 77687.4, 300 sec: 78462.7). Total num frames: 348291072. Throughput: 0: 19321.8. Samples: 77019788. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 01:49:55,466][1157520] Avg episode reward: [(0, '53.355')] [2024-09-30 01:49:55,782][1157819] Updated weights for policy 0, policy_version 85038 (0.0006) [2024-09-30 01:49:56,302][1157819] Updated weights for policy 0, policy_version 85048 (0.0006) [2024-09-30 01:49:56,828][1157819] Updated weights for policy 0, policy_version 85058 (0.0006) [2024-09-30 01:49:57,334][1157819] Updated weights for policy 0, policy_version 85068 (0.0006) [2024-09-30 01:49:57,857][1157819] Updated weights for policy 0, policy_version 85078 (0.0006) [2024-09-30 01:49:58,358][1157819] Updated weights for policy 0, policy_version 85088 (0.0006) [2024-09-30 01:49:58,865][1157819] Updated weights for policy 0, policy_version 85098 (0.0006) [2024-09-30 01:49:59,364][1157819] Updated weights for policy 0, policy_version 85108 (0.0006) [2024-09-30 01:49:59,877][1157819] Updated weights for policy 0, policy_version 85118 (0.0006) [2024-09-30 01:50:00,398][1157819] Updated weights for policy 0, policy_version 85128 (0.0006) [2024-09-30 01:50:00,466][1157520] Fps is (10 sec: 79463.0, 60 sec: 77687.5, 300 sec: 78421.0). Total num frames: 348688384. Throughput: 0: 19342.7. Samples: 77138764. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 01:50:00,466][1157520] Avg episode reward: [(0, '54.392')] [2024-09-30 01:50:00,950][1157819] Updated weights for policy 0, policy_version 85138 (0.0006) [2024-09-30 01:50:01,451][1157819] Updated weights for policy 0, policy_version 85148 (0.0006) [2024-09-30 01:50:01,962][1157819] Updated weights for policy 0, policy_version 85158 (0.0006) [2024-09-30 01:50:02,471][1157819] Updated weights for policy 0, policy_version 85168 (0.0006) [2024-09-30 01:50:02,982][1157819] Updated weights for policy 0, policy_version 85178 (0.0006) [2024-09-30 01:50:03,515][1157819] Updated weights for policy 0, policy_version 85188 (0.0006) [2024-09-30 01:50:04,062][1157819] Updated weights for policy 0, policy_version 85198 (0.0006) [2024-09-30 01:50:04,527][1157819] Updated weights for policy 0, policy_version 85208 (0.0006) [2024-09-30 01:50:05,014][1157819] Updated weights for policy 0, policy_version 85218 (0.0006) [2024-09-30 01:50:05,466][1157520] Fps is (10 sec: 79873.5, 60 sec: 77824.2, 300 sec: 78504.4). Total num frames: 349089792. Throughput: 0: 19319.5. Samples: 77258448. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 01:50:05,466][1157520] Avg episode reward: [(0, '54.106')] [2024-09-30 01:50:05,469][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000085227_349089792.pth... [2024-09-30 01:50:05,521][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000080583_330067968.pth [2024-09-30 01:50:05,543][1157819] Updated weights for policy 0, policy_version 85228 (0.0006) [2024-09-30 01:50:06,009][1157819] Updated weights for policy 0, policy_version 85238 (0.0006) [2024-09-30 01:50:06,507][1157819] Updated weights for policy 0, policy_version 85248 (0.0006) [2024-09-30 01:50:06,993][1157819] Updated weights for policy 0, policy_version 85258 (0.0006) [2024-09-30 01:50:07,471][1157819] Updated weights for policy 0, policy_version 85268 (0.0006) [2024-09-30 01:50:07,985][1157819] Updated weights for policy 0, policy_version 85278 (0.0006) [2024-09-30 01:50:08,495][1157819] Updated weights for policy 0, policy_version 85288 (0.0006) [2024-09-30 01:50:09,001][1157819] Updated weights for policy 0, policy_version 85298 (0.0006) [2024-09-30 01:50:09,324][1157736] Signal inference workers to stop experience collection... (5550 times) [2024-09-30 01:50:09,325][1157736] Signal inference workers to resume experience collection... (5550 times) [2024-09-30 01:50:09,330][1157819] InferenceWorker_p0-w0: stopping experience collection (5550 times) [2024-09-30 01:50:09,331][1157819] InferenceWorker_p0-w0: resuming experience collection (5550 times) [2024-09-30 01:50:09,490][1157819] Updated weights for policy 0, policy_version 85308 (0.0006) [2024-09-30 01:50:10,026][1157819] Updated weights for policy 0, policy_version 85318 (0.0006) [2024-09-30 01:50:10,466][1157520] Fps is (10 sec: 80692.0, 60 sec: 78165.5, 300 sec: 78698.8). Total num frames: 349495296. Throughput: 0: 19387.2. Samples: 77320620. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 01:50:10,466][1157520] Avg episode reward: [(0, '55.180')] [2024-09-30 01:50:10,552][1157819] Updated weights for policy 0, policy_version 85328 (0.0006) [2024-09-30 01:50:11,060][1157819] Updated weights for policy 0, policy_version 85338 (0.0006) [2024-09-30 01:50:11,567][1157819] Updated weights for policy 0, policy_version 85348 (0.0006) [2024-09-30 01:50:12,088][1157819] Updated weights for policy 0, policy_version 85358 (0.0006) [2024-09-30 01:50:12,600][1157819] Updated weights for policy 0, policy_version 85368 (0.0006) [2024-09-30 01:50:13,152][1157819] Updated weights for policy 0, policy_version 85378 (0.0006) [2024-09-30 01:50:13,688][1157819] Updated weights for policy 0, policy_version 85388 (0.0006) [2024-09-30 01:50:14,169][1157819] Updated weights for policy 0, policy_version 85398 (0.0006) [2024-09-30 01:50:14,675][1157819] Updated weights for policy 0, policy_version 85408 (0.0006) [2024-09-30 01:50:15,211][1157819] Updated weights for policy 0, policy_version 85418 (0.0006) [2024-09-30 01:50:15,466][1157520] Fps is (10 sec: 79871.8, 60 sec: 78028.9, 300 sec: 78809.9). Total num frames: 349888512. Throughput: 0: 19495.8. Samples: 77439916. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 01:50:15,466][1157520] Avg episode reward: [(0, '52.843')] [2024-09-30 01:50:15,761][1157819] Updated weights for policy 0, policy_version 85428 (0.0006) [2024-09-30 01:50:16,253][1157819] Updated weights for policy 0, policy_version 85438 (0.0006) [2024-09-30 01:50:16,741][1157819] Updated weights for policy 0, policy_version 85448 (0.0006) [2024-09-30 01:50:17,268][1157819] Updated weights for policy 0, policy_version 85458 (0.0006) [2024-09-30 01:50:17,788][1157819] Updated weights for policy 0, policy_version 85468 (0.0006) [2024-09-30 01:50:18,309][1157819] Updated weights for policy 0, policy_version 85478 (0.0006) [2024-09-30 01:50:18,780][1157819] Updated weights for policy 0, policy_version 85488 (0.0006) [2024-09-30 01:50:19,278][1157819] Updated weights for policy 0, policy_version 85498 (0.0006) [2024-09-30 01:50:19,768][1157819] Updated weights for policy 0, policy_version 85508 (0.0006) [2024-09-30 01:50:20,253][1157819] Updated weights for policy 0, policy_version 85518 (0.0006) [2024-09-30 01:50:20,466][1157520] Fps is (10 sec: 80281.6, 60 sec: 78233.7, 300 sec: 78934.8). Total num frames: 350298112. Throughput: 0: 19667.5. Samples: 77560528. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 01:50:20,466][1157520] Avg episode reward: [(0, '54.668')] [2024-09-30 01:50:20,752][1157819] Updated weights for policy 0, policy_version 85528 (0.0006) [2024-09-30 01:50:21,247][1157819] Updated weights for policy 0, policy_version 85538 (0.0006) [2024-09-30 01:50:21,706][1157819] Updated weights for policy 0, policy_version 85548 (0.0006) [2024-09-30 01:50:22,170][1157819] Updated weights for policy 0, policy_version 85558 (0.0006) [2024-09-30 01:50:22,683][1157819] Updated weights for policy 0, policy_version 85568 (0.0006) [2024-09-30 01:50:23,221][1157819] Updated weights for policy 0, policy_version 85578 (0.0006) [2024-09-30 01:50:23,745][1157819] Updated weights for policy 0, policy_version 85588 (0.0006) [2024-09-30 01:50:24,245][1157819] Updated weights for policy 0, policy_version 85598 (0.0006) [2024-09-30 01:50:24,763][1157819] Updated weights for policy 0, policy_version 85608 (0.0006) [2024-09-30 01:50:25,247][1157819] Updated weights for policy 0, policy_version 85618 (0.0006) [2024-09-30 01:50:25,466][1157520] Fps is (10 sec: 81920.1, 60 sec: 78507.0, 300 sec: 79004.2). Total num frames: 350707712. Throughput: 0: 19802.6. Samples: 77623384. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 01:50:25,466][1157520] Avg episode reward: [(0, '53.293')] [2024-09-30 01:50:25,755][1157819] Updated weights for policy 0, policy_version 85628 (0.0006) [2024-09-30 01:50:26,264][1157819] Updated weights for policy 0, policy_version 85638 (0.0006) [2024-09-30 01:50:26,774][1157819] Updated weights for policy 0, policy_version 85648 (0.0006) [2024-09-30 01:50:27,287][1157819] Updated weights for policy 0, policy_version 85658 (0.0006) [2024-09-30 01:50:27,782][1157819] Updated weights for policy 0, policy_version 85668 (0.0006) [2024-09-30 01:50:28,312][1157819] Updated weights for policy 0, policy_version 85678 (0.0006) [2024-09-30 01:50:28,802][1157819] Updated weights for policy 0, policy_version 85688 (0.0006) [2024-09-30 01:50:29,308][1157819] Updated weights for policy 0, policy_version 85698 (0.0006) [2024-09-30 01:50:29,819][1157819] Updated weights for policy 0, policy_version 85708 (0.0006) [2024-09-30 01:50:30,332][1157819] Updated weights for policy 0, policy_version 85718 (0.0006) [2024-09-30 01:50:30,466][1157520] Fps is (10 sec: 81100.6, 60 sec: 78848.1, 300 sec: 79059.8). Total num frames: 351109120. Throughput: 0: 20014.4. Samples: 77744036. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:50:30,466][1157520] Avg episode reward: [(0, '52.172')] [2024-09-30 01:50:30,856][1157819] Updated weights for policy 0, policy_version 85728 (0.0006) [2024-09-30 01:50:31,375][1157819] Updated weights for policy 0, policy_version 85738 (0.0006) [2024-09-30 01:50:31,876][1157819] Updated weights for policy 0, policy_version 85748 (0.0006) [2024-09-30 01:50:32,394][1157819] Updated weights for policy 0, policy_version 85758 (0.0006) [2024-09-30 01:50:32,895][1157819] Updated weights for policy 0, policy_version 85768 (0.0006) [2024-09-30 01:50:33,408][1157819] Updated weights for policy 0, policy_version 85778 (0.0006) [2024-09-30 01:50:33,937][1157819] Updated weights for policy 0, policy_version 85788 (0.0006) [2024-09-30 01:50:34,428][1157819] Updated weights for policy 0, policy_version 85798 (0.0006) [2024-09-30 01:50:34,942][1157819] Updated weights for policy 0, policy_version 85808 (0.0006) [2024-09-30 01:50:35,444][1157819] Updated weights for policy 0, policy_version 85818 (0.0006) [2024-09-30 01:50:35,466][1157520] Fps is (10 sec: 80281.6, 60 sec: 79394.3, 300 sec: 79045.9). Total num frames: 351510528. Throughput: 0: 20087.8. Samples: 77864340. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:50:35,466][1157520] Avg episode reward: [(0, '53.495')] [2024-09-30 01:50:35,946][1157819] Updated weights for policy 0, policy_version 85828 (0.0006) [2024-09-30 01:50:36,447][1157819] Updated weights for policy 0, policy_version 85838 (0.0006) [2024-09-30 01:50:36,979][1157819] Updated weights for policy 0, policy_version 85848 (0.0006) [2024-09-30 01:50:37,563][1157819] Updated weights for policy 0, policy_version 85858 (0.0006) [2024-09-30 01:50:38,136][1157819] Updated weights for policy 0, policy_version 85868 (0.0006) [2024-09-30 01:50:38,696][1157819] Updated weights for policy 0, policy_version 85878 (0.0006) [2024-09-30 01:50:39,279][1157819] Updated weights for policy 0, policy_version 85888 (0.0006) [2024-09-30 01:50:39,822][1157819] Updated weights for policy 0, policy_version 85898 (0.0006) [2024-09-30 01:50:40,358][1157819] Updated weights for policy 0, policy_version 85908 (0.0006) [2024-09-30 01:50:40,466][1157520] Fps is (10 sec: 77413.4, 60 sec: 79257.5, 300 sec: 79018.1). Total num frames: 351883264. Throughput: 0: 20067.3. Samples: 77922816. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:50:40,466][1157520] Avg episode reward: [(0, '54.680')] [2024-09-30 01:50:40,933][1157819] Updated weights for policy 0, policy_version 85918 (0.0006) [2024-09-30 01:50:41,452][1157819] Updated weights for policy 0, policy_version 85928 (0.0006) [2024-09-30 01:50:41,964][1157819] Updated weights for policy 0, policy_version 85938 (0.0006) [2024-09-30 01:50:42,564][1157819] Updated weights for policy 0, policy_version 85948 (0.0006) [2024-09-30 01:50:43,069][1157819] Updated weights for policy 0, policy_version 85958 (0.0006) [2024-09-30 01:50:43,613][1157819] Updated weights for policy 0, policy_version 85968 (0.0006) [2024-09-30 01:50:44,175][1157819] Updated weights for policy 0, policy_version 85978 (0.0006) [2024-09-30 01:50:44,730][1157819] Updated weights for policy 0, policy_version 85988 (0.0006) [2024-09-30 01:50:45,277][1157819] Updated weights for policy 0, policy_version 85998 (0.0006) [2024-09-30 01:50:45,466][1157520] Fps is (10 sec: 74956.1, 60 sec: 79394.3, 300 sec: 79032.0). Total num frames: 352260096. Throughput: 0: 19899.5. Samples: 78034244. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:50:45,466][1157520] Avg episode reward: [(0, '55.740')] [2024-09-30 01:50:45,809][1157819] Updated weights for policy 0, policy_version 86008 (0.0006) [2024-09-30 01:50:46,353][1157819] Updated weights for policy 0, policy_version 86018 (0.0006) [2024-09-30 01:50:46,905][1157819] Updated weights for policy 0, policy_version 86028 (0.0006) [2024-09-30 01:50:47,428][1157819] Updated weights for policy 0, policy_version 86038 (0.0006) [2024-09-30 01:50:47,990][1157819] Updated weights for policy 0, policy_version 86048 (0.0006) [2024-09-30 01:50:48,542][1157819] Updated weights for policy 0, policy_version 86058 (0.0006) [2024-09-30 01:50:49,065][1157819] Updated weights for policy 0, policy_version 86068 (0.0006) [2024-09-30 01:50:49,622][1157819] Updated weights for policy 0, policy_version 86078 (0.0006) [2024-09-30 01:50:50,168][1157819] Updated weights for policy 0, policy_version 86088 (0.0006) [2024-09-30 01:50:50,466][1157520] Fps is (10 sec: 75366.9, 60 sec: 79052.9, 300 sec: 79045.9). Total num frames: 352636928. Throughput: 0: 19742.9. Samples: 78146880. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:50:50,466][1157520] Avg episode reward: [(0, '53.327')] [2024-09-30 01:50:50,718][1157819] Updated weights for policy 0, policy_version 86098 (0.0006) [2024-09-30 01:50:51,264][1157819] Updated weights for policy 0, policy_version 86108 (0.0006) [2024-09-30 01:50:51,779][1157819] Updated weights for policy 0, policy_version 86118 (0.0006) [2024-09-30 01:50:52,320][1157819] Updated weights for policy 0, policy_version 86128 (0.0006) [2024-09-30 01:50:52,903][1157819] Updated weights for policy 0, policy_version 86138 (0.0006) [2024-09-30 01:50:53,423][1157819] Updated weights for policy 0, policy_version 86148 (0.0006) [2024-09-30 01:50:53,999][1157819] Updated weights for policy 0, policy_version 86158 (0.0006) [2024-09-30 01:50:54,194][1157736] Signal inference workers to stop experience collection... (5600 times) [2024-09-30 01:50:54,194][1157736] Signal inference workers to resume experience collection... (5600 times) [2024-09-30 01:50:54,197][1157819] InferenceWorker_p0-w0: stopping experience collection (5600 times) [2024-09-30 01:50:54,198][1157819] InferenceWorker_p0-w0: resuming experience collection (5600 times) [2024-09-30 01:50:54,534][1157819] Updated weights for policy 0, policy_version 86168 (0.0006) [2024-09-30 01:50:55,103][1157819] Updated weights for policy 0, policy_version 86178 (0.0006) [2024-09-30 01:50:55,466][1157520] Fps is (10 sec: 75366.6, 60 sec: 78711.6, 300 sec: 79059.8). Total num frames: 353013760. Throughput: 0: 19612.3. Samples: 78203176. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:50:55,466][1157520] Avg episode reward: [(0, '55.016')] [2024-09-30 01:50:55,643][1157819] Updated weights for policy 0, policy_version 86188 (0.0006) [2024-09-30 01:50:56,178][1157819] Updated weights for policy 0, policy_version 86198 (0.0006) [2024-09-30 01:50:56,718][1157819] Updated weights for policy 0, policy_version 86208 (0.0006) [2024-09-30 01:50:57,266][1157819] Updated weights for policy 0, policy_version 86218 (0.0006) [2024-09-30 01:50:57,837][1157819] Updated weights for policy 0, policy_version 86228 (0.0006) [2024-09-30 01:50:58,386][1157819] Updated weights for policy 0, policy_version 86238 (0.0006) [2024-09-30 01:50:58,953][1157819] Updated weights for policy 0, policy_version 86248 (0.0006) [2024-09-30 01:50:59,493][1157819] Updated weights for policy 0, policy_version 86258 (0.0006) [2024-09-30 01:51:00,028][1157819] Updated weights for policy 0, policy_version 86268 (0.0006) [2024-09-30 01:51:00,466][1157520] Fps is (10 sec: 74956.8, 60 sec: 78301.9, 300 sec: 79059.8). Total num frames: 353386496. Throughput: 0: 19453.1. Samples: 78315308. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:51:00,466][1157520] Avg episode reward: [(0, '52.702')] [2024-09-30 01:51:00,571][1157819] Updated weights for policy 0, policy_version 86278 (0.0006) [2024-09-30 01:51:01,100][1157819] Updated weights for policy 0, policy_version 86288 (0.0006) [2024-09-30 01:51:01,646][1157819] Updated weights for policy 0, policy_version 86298 (0.0006) [2024-09-30 01:51:02,203][1157819] Updated weights for policy 0, policy_version 86308 (0.0006) [2024-09-30 01:51:02,699][1157819] Updated weights for policy 0, policy_version 86318 (0.0006) [2024-09-30 01:51:03,258][1157819] Updated weights for policy 0, policy_version 86328 (0.0006) [2024-09-30 01:51:03,794][1157819] Updated weights for policy 0, policy_version 86338 (0.0006) [2024-09-30 01:51:04,341][1157819] Updated weights for policy 0, policy_version 86348 (0.0006) [2024-09-30 01:51:04,864][1157819] Updated weights for policy 0, policy_version 86358 (0.0006) [2024-09-30 01:51:05,427][1157819] Updated weights for policy 0, policy_version 86368 (0.0006) [2024-09-30 01:51:05,466][1157520] Fps is (10 sec: 74956.7, 60 sec: 77892.1, 300 sec: 79032.0). Total num frames: 353763328. Throughput: 0: 19297.6. Samples: 78428920. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:51:05,466][1157520] Avg episode reward: [(0, '55.198')] [2024-09-30 01:51:05,949][1157819] Updated weights for policy 0, policy_version 86378 (0.0006) [2024-09-30 01:51:06,465][1157819] Updated weights for policy 0, policy_version 86388 (0.0006) [2024-09-30 01:51:07,031][1157819] Updated weights for policy 0, policy_version 86398 (0.0006) [2024-09-30 01:51:07,566][1157819] Updated weights for policy 0, policy_version 86408 (0.0006) [2024-09-30 01:51:08,058][1157819] Updated weights for policy 0, policy_version 86418 (0.0006) [2024-09-30 01:51:08,562][1157819] Updated weights for policy 0, policy_version 86428 (0.0006) [2024-09-30 01:51:09,072][1157819] Updated weights for policy 0, policy_version 86438 (0.0006) [2024-09-30 01:51:09,608][1157819] Updated weights for policy 0, policy_version 86448 (0.0006) [2024-09-30 01:51:10,166][1157819] Updated weights for policy 0, policy_version 86458 (0.0006) [2024-09-30 01:51:10,466][1157520] Fps is (10 sec: 76595.2, 60 sec: 77619.1, 300 sec: 79073.7). Total num frames: 354152448. Throughput: 0: 19176.1. Samples: 78486312. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:51:10,466][1157520] Avg episode reward: [(0, '53.776')] [2024-09-30 01:51:10,687][1157819] Updated weights for policy 0, policy_version 86468 (0.0006) [2024-09-30 01:51:11,200][1157819] Updated weights for policy 0, policy_version 86478 (0.0006) [2024-09-30 01:51:11,701][1157819] Updated weights for policy 0, policy_version 86488 (0.0006) [2024-09-30 01:51:12,264][1157819] Updated weights for policy 0, policy_version 86498 (0.0006) [2024-09-30 01:51:12,825][1157819] Updated weights for policy 0, policy_version 86508 (0.0006) [2024-09-30 01:51:13,319][1157819] Updated weights for policy 0, policy_version 86518 (0.0006) [2024-09-30 01:51:13,831][1157819] Updated weights for policy 0, policy_version 86528 (0.0006) [2024-09-30 01:51:14,415][1157819] Updated weights for policy 0, policy_version 86538 (0.0006) [2024-09-30 01:51:14,930][1157819] Updated weights for policy 0, policy_version 86548 (0.0006) [2024-09-30 01:51:15,433][1157819] Updated weights for policy 0, policy_version 86558 (0.0006) [2024-09-30 01:51:15,466][1157520] Fps is (10 sec: 77823.4, 60 sec: 77550.7, 300 sec: 79115.3). Total num frames: 354541568. Throughput: 0: 19096.0. Samples: 78603360. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:51:15,466][1157520] Avg episode reward: [(0, '53.397')] [2024-09-30 01:51:15,975][1157819] Updated weights for policy 0, policy_version 86568 (0.0006) [2024-09-30 01:51:16,510][1157819] Updated weights for policy 0, policy_version 86578 (0.0006) [2024-09-30 01:51:17,036][1157819] Updated weights for policy 0, policy_version 86588 (0.0006) [2024-09-30 01:51:17,560][1157819] Updated weights for policy 0, policy_version 86598 (0.0006) [2024-09-30 01:51:18,149][1157819] Updated weights for policy 0, policy_version 86608 (0.0006) [2024-09-30 01:51:18,703][1157819] Updated weights for policy 0, policy_version 86618 (0.0006) [2024-09-30 01:51:19,232][1157819] Updated weights for policy 0, policy_version 86628 (0.0006) [2024-09-30 01:51:19,815][1157819] Updated weights for policy 0, policy_version 86638 (0.0006) [2024-09-30 01:51:20,341][1157819] Updated weights for policy 0, policy_version 86648 (0.0006) [2024-09-30 01:51:20,466][1157520] Fps is (10 sec: 76185.4, 60 sec: 76936.4, 300 sec: 79101.4). Total num frames: 354914304. Throughput: 0: 18953.1. Samples: 78717232. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:51:20,466][1157520] Avg episode reward: [(0, '52.628')] [2024-09-30 01:51:20,892][1157819] Updated weights for policy 0, policy_version 86658 (0.0006) [2024-09-30 01:51:21,430][1157819] Updated weights for policy 0, policy_version 86668 (0.0006) [2024-09-30 01:51:21,959][1157819] Updated weights for policy 0, policy_version 86678 (0.0006) [2024-09-30 01:51:22,559][1157819] Updated weights for policy 0, policy_version 86688 (0.0006) [2024-09-30 01:51:23,076][1157819] Updated weights for policy 0, policy_version 86698 (0.0006) [2024-09-30 01:51:23,623][1157819] Updated weights for policy 0, policy_version 86708 (0.0006) [2024-09-30 01:51:24,189][1157819] Updated weights for policy 0, policy_version 86718 (0.0006) [2024-09-30 01:51:24,755][1157819] Updated weights for policy 0, policy_version 86728 (0.0006) [2024-09-30 01:51:25,287][1157819] Updated weights for policy 0, policy_version 86738 (0.0006) [2024-09-30 01:51:25,466][1157520] Fps is (10 sec: 74957.5, 60 sec: 76390.3, 300 sec: 79115.3). Total num frames: 355291136. Throughput: 0: 18894.7. Samples: 78773076. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:51:25,466][1157520] Avg episode reward: [(0, '53.055')] [2024-09-30 01:51:25,811][1157819] Updated weights for policy 0, policy_version 86748 (0.0006) [2024-09-30 01:51:26,345][1157819] Updated weights for policy 0, policy_version 86758 (0.0006) [2024-09-30 01:51:26,908][1157819] Updated weights for policy 0, policy_version 86768 (0.0006) [2024-09-30 01:51:27,447][1157819] Updated weights for policy 0, policy_version 86778 (0.0006) [2024-09-30 01:51:27,957][1157819] Updated weights for policy 0, policy_version 86788 (0.0006) [2024-09-30 01:51:28,512][1157819] Updated weights for policy 0, policy_version 86798 (0.0006) [2024-09-30 01:51:29,051][1157819] Updated weights for policy 0, policy_version 86808 (0.0006) [2024-09-30 01:51:29,581][1157819] Updated weights for policy 0, policy_version 86818 (0.0006) [2024-09-30 01:51:30,126][1157819] Updated weights for policy 0, policy_version 86828 (0.0006) [2024-09-30 01:51:30,466][1157520] Fps is (10 sec: 75776.2, 60 sec: 76049.0, 300 sec: 79143.1). Total num frames: 355672064. Throughput: 0: 18935.3. Samples: 78886332. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:51:30,466][1157520] Avg episode reward: [(0, '54.390')] [2024-09-30 01:51:30,680][1157819] Updated weights for policy 0, policy_version 86838 (0.0006) [2024-09-30 01:51:31,211][1157819] Updated weights for policy 0, policy_version 86848 (0.0006) [2024-09-30 01:51:31,789][1157819] Updated weights for policy 0, policy_version 86858 (0.0006) [2024-09-30 01:51:32,308][1157819] Updated weights for policy 0, policy_version 86868 (0.0006) [2024-09-30 01:51:32,845][1157819] Updated weights for policy 0, policy_version 86878 (0.0006) [2024-09-30 01:51:33,417][1157819] Updated weights for policy 0, policy_version 86888 (0.0006) [2024-09-30 01:51:33,955][1157819] Updated weights for policy 0, policy_version 86898 (0.0006) [2024-09-30 01:51:34,476][1157819] Updated weights for policy 0, policy_version 86908 (0.0006) [2024-09-30 01:51:35,055][1157819] Updated weights for policy 0, policy_version 86918 (0.0006) [2024-09-30 01:51:35,466][1157520] Fps is (10 sec: 75775.8, 60 sec: 75639.3, 300 sec: 79170.8). Total num frames: 356048896. Throughput: 0: 18941.8. Samples: 78999260. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:51:35,466][1157520] Avg episode reward: [(0, '54.635')] [2024-09-30 01:51:35,596][1157819] Updated weights for policy 0, policy_version 86928 (0.0006) [2024-09-30 01:51:36,132][1157819] Updated weights for policy 0, policy_version 86938 (0.0006) [2024-09-30 01:51:36,675][1157819] Updated weights for policy 0, policy_version 86948 (0.0006) [2024-09-30 01:51:37,228][1157819] Updated weights for policy 0, policy_version 86958 (0.0006) [2024-09-30 01:51:37,734][1157819] Updated weights for policy 0, policy_version 86968 (0.0006) [2024-09-30 01:51:38,251][1157819] Updated weights for policy 0, policy_version 86978 (0.0006) [2024-09-30 01:51:38,743][1157819] Updated weights for policy 0, policy_version 86988 (0.0006) [2024-09-30 01:51:39,250][1157819] Updated weights for policy 0, policy_version 86998 (0.0006) [2024-09-30 01:51:39,762][1157819] Updated weights for policy 0, policy_version 87008 (0.0006) [2024-09-30 01:51:40,253][1157819] Updated weights for policy 0, policy_version 87018 (0.0006) [2024-09-30 01:51:40,466][1157520] Fps is (10 sec: 77004.9, 60 sec: 75980.9, 300 sec: 79240.3). Total num frames: 356442112. Throughput: 0: 18965.6. Samples: 79056628. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:51:40,466][1157520] Avg episode reward: [(0, '54.897')] [2024-09-30 01:51:40,775][1157819] Updated weights for policy 0, policy_version 87028 (0.0006) [2024-09-30 01:51:41,311][1157819] Updated weights for policy 0, policy_version 87038 (0.0006) [2024-09-30 01:51:41,816][1157819] Updated weights for policy 0, policy_version 87048 (0.0006) [2024-09-30 01:51:42,309][1157819] Updated weights for policy 0, policy_version 87058 (0.0006) [2024-09-30 01:51:42,837][1157819] Updated weights for policy 0, policy_version 87068 (0.0006) [2024-09-30 01:51:43,340][1157819] Updated weights for policy 0, policy_version 87078 (0.0006) [2024-09-30 01:51:43,654][1157736] Signal inference workers to stop experience collection... (5650 times) [2024-09-30 01:51:43,659][1157736] Signal inference workers to resume experience collection... (5650 times) [2024-09-30 01:51:43,659][1157819] InferenceWorker_p0-w0: stopping experience collection (5650 times) [2024-09-30 01:51:43,662][1157819] InferenceWorker_p0-w0: resuming experience collection (5650 times) [2024-09-30 01:51:43,827][1157819] Updated weights for policy 0, policy_version 87088 (0.0006) [2024-09-30 01:51:44,333][1157819] Updated weights for policy 0, policy_version 87098 (0.0006) [2024-09-30 01:51:44,829][1157819] Updated weights for policy 0, policy_version 87108 (0.0006) [2024-09-30 01:51:45,335][1157819] Updated weights for policy 0, policy_version 87118 (0.0006) [2024-09-30 01:51:45,466][1157520] Fps is (10 sec: 79053.0, 60 sec: 76322.1, 300 sec: 79337.5). Total num frames: 356839424. Throughput: 0: 19151.2. Samples: 79177112. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:51:45,466][1157520] Avg episode reward: [(0, '53.863')] [2024-09-30 01:51:45,834][1157819] Updated weights for policy 0, policy_version 87128 (0.0006) [2024-09-30 01:51:46,372][1157819] Updated weights for policy 0, policy_version 87138 (0.0006) [2024-09-30 01:51:46,898][1157819] Updated weights for policy 0, policy_version 87148 (0.0006) [2024-09-30 01:51:47,406][1157819] Updated weights for policy 0, policy_version 87158 (0.0006) [2024-09-30 01:51:47,902][1157819] Updated weights for policy 0, policy_version 87168 (0.0006) [2024-09-30 01:51:48,367][1157819] Updated weights for policy 0, policy_version 87178 (0.0006) [2024-09-30 01:51:48,840][1157819] Updated weights for policy 0, policy_version 87188 (0.0006) [2024-09-30 01:51:49,330][1157819] Updated weights for policy 0, policy_version 87198 (0.0006) [2024-09-30 01:51:49,787][1157819] Updated weights for policy 0, policy_version 87208 (0.0006) [2024-09-30 01:51:50,275][1157819] Updated weights for policy 0, policy_version 87218 (0.0006) [2024-09-30 01:51:50,466][1157520] Fps is (10 sec: 81510.2, 60 sec: 77004.8, 300 sec: 79504.1). Total num frames: 357257216. Throughput: 0: 19368.4. Samples: 79300496. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:51:50,466][1157520] Avg episode reward: [(0, '53.291')] [2024-09-30 01:51:50,774][1157819] Updated weights for policy 0, policy_version 87228 (0.0006) [2024-09-30 01:51:51,269][1157819] Updated weights for policy 0, policy_version 87238 (0.0006) [2024-09-30 01:51:51,742][1157819] Updated weights for policy 0, policy_version 87248 (0.0006) [2024-09-30 01:51:52,227][1157819] Updated weights for policy 0, policy_version 87258 (0.0006) [2024-09-30 01:51:52,720][1157819] Updated weights for policy 0, policy_version 87268 (0.0006) [2024-09-30 01:51:53,199][1157819] Updated weights for policy 0, policy_version 87278 (0.0006) [2024-09-30 01:51:53,690][1157819] Updated weights for policy 0, policy_version 87288 (0.0006) [2024-09-30 01:51:54,168][1157819] Updated weights for policy 0, policy_version 87298 (0.0006) [2024-09-30 01:51:54,634][1157819] Updated weights for policy 0, policy_version 87308 (0.0006) [2024-09-30 01:51:55,127][1157819] Updated weights for policy 0, policy_version 87318 (0.0006) [2024-09-30 01:51:55,466][1157520] Fps is (10 sec: 83967.3, 60 sec: 77755.6, 300 sec: 79601.2). Total num frames: 357679104. Throughput: 0: 19502.0. Samples: 79363904. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:51:55,466][1157520] Avg episode reward: [(0, '55.599')] [2024-09-30 01:51:55,624][1157819] Updated weights for policy 0, policy_version 87328 (0.0006) [2024-09-30 01:51:56,107][1157819] Updated weights for policy 0, policy_version 87338 (0.0006) [2024-09-30 01:51:56,611][1157819] Updated weights for policy 0, policy_version 87348 (0.0006) [2024-09-30 01:51:57,103][1157819] Updated weights for policy 0, policy_version 87358 (0.0006) [2024-09-30 01:51:57,570][1157819] Updated weights for policy 0, policy_version 87368 (0.0006) [2024-09-30 01:51:58,090][1157819] Updated weights for policy 0, policy_version 87378 (0.0006) [2024-09-30 01:51:58,591][1157819] Updated weights for policy 0, policy_version 87388 (0.0006) [2024-09-30 01:51:59,082][1157819] Updated weights for policy 0, policy_version 87398 (0.0006) [2024-09-30 01:51:59,579][1157819] Updated weights for policy 0, policy_version 87408 (0.0006) [2024-09-30 01:52:00,060][1157819] Updated weights for policy 0, policy_version 87418 (0.0006) [2024-09-30 01:52:00,466][1157520] Fps is (10 sec: 83967.8, 60 sec: 78506.6, 300 sec: 79670.7). Total num frames: 358096896. Throughput: 0: 19690.8. Samples: 79489444. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:52:00,466][1157520] Avg episode reward: [(0, '51.812')] [2024-09-30 01:52:00,562][1157819] Updated weights for policy 0, policy_version 87428 (0.0006) [2024-09-30 01:52:01,053][1157819] Updated weights for policy 0, policy_version 87438 (0.0006) [2024-09-30 01:52:01,517][1157819] Updated weights for policy 0, policy_version 87448 (0.0005) [2024-09-30 01:52:02,016][1157819] Updated weights for policy 0, policy_version 87458 (0.0006) [2024-09-30 01:52:02,517][1157819] Updated weights for policy 0, policy_version 87468 (0.0006) [2024-09-30 01:52:03,016][1157819] Updated weights for policy 0, policy_version 87478 (0.0006) [2024-09-30 01:52:03,501][1157819] Updated weights for policy 0, policy_version 87488 (0.0006) [2024-09-30 01:52:03,996][1157819] Updated weights for policy 0, policy_version 87498 (0.0006) [2024-09-30 01:52:04,510][1157819] Updated weights for policy 0, policy_version 87508 (0.0006) [2024-09-30 01:52:04,997][1157819] Updated weights for policy 0, policy_version 87518 (0.0006) [2024-09-30 01:52:05,466][1157520] Fps is (10 sec: 83148.6, 60 sec: 79120.9, 300 sec: 79698.4). Total num frames: 358510592. Throughput: 0: 19930.5. Samples: 79614108. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:52:05,466][1157520] Avg episode reward: [(0, '50.990')] [2024-09-30 01:52:05,475][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000087528_358514688.pth... [2024-09-30 01:52:05,476][1157819] Updated weights for policy 0, policy_version 87528 (0.0006) [2024-09-30 01:52:05,520][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000082889_339513344.pth [2024-09-30 01:52:06,004][1157819] Updated weights for policy 0, policy_version 87538 (0.0006) [2024-09-30 01:52:06,502][1157819] Updated weights for policy 0, policy_version 87548 (0.0006) [2024-09-30 01:52:06,981][1157819] Updated weights for policy 0, policy_version 87558 (0.0006) [2024-09-30 01:52:07,473][1157819] Updated weights for policy 0, policy_version 87568 (0.0006) [2024-09-30 01:52:07,972][1157819] Updated weights for policy 0, policy_version 87578 (0.0006) [2024-09-30 01:52:08,460][1157819] Updated weights for policy 0, policy_version 87588 (0.0006) [2024-09-30 01:52:08,965][1157819] Updated weights for policy 0, policy_version 87598 (0.0006) [2024-09-30 01:52:09,459][1157819] Updated weights for policy 0, policy_version 87608 (0.0006) [2024-09-30 01:52:09,959][1157819] Updated weights for policy 0, policy_version 87618 (0.0006) [2024-09-30 01:52:10,464][1157819] Updated weights for policy 0, policy_version 87628 (0.0006) [2024-09-30 01:52:10,466][1157520] Fps is (10 sec: 82739.5, 60 sec: 79530.7, 300 sec: 79754.0). Total num frames: 358924288. Throughput: 0: 20061.5. Samples: 79675844. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 01:52:10,466][1157520] Avg episode reward: [(0, '53.967')] [2024-09-30 01:52:10,972][1157819] Updated weights for policy 0, policy_version 87638 (0.0006) [2024-09-30 01:52:11,453][1157819] Updated weights for policy 0, policy_version 87648 (0.0006) [2024-09-30 01:52:11,981][1157819] Updated weights for policy 0, policy_version 87658 (0.0006) [2024-09-30 01:52:12,454][1157819] Updated weights for policy 0, policy_version 87668 (0.0006) [2024-09-30 01:52:12,937][1157819] Updated weights for policy 0, policy_version 87678 (0.0006) [2024-09-30 01:52:13,448][1157819] Updated weights for policy 0, policy_version 87688 (0.0006) [2024-09-30 01:52:13,895][1157819] Updated weights for policy 0, policy_version 87698 (0.0006) [2024-09-30 01:52:14,430][1157819] Updated weights for policy 0, policy_version 87708 (0.0006) [2024-09-30 01:52:14,940][1157819] Updated weights for policy 0, policy_version 87718 (0.0006) [2024-09-30 01:52:15,461][1157819] Updated weights for policy 0, policy_version 87728 (0.0006) [2024-09-30 01:52:15,466][1157520] Fps is (10 sec: 82329.5, 60 sec: 79871.9, 300 sec: 79767.8). Total num frames: 359333888. Throughput: 0: 20304.6. Samples: 79800040. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:52:15,466][1157520] Avg episode reward: [(0, '52.731')] [2024-09-30 01:52:15,970][1157819] Updated weights for policy 0, policy_version 87738 (0.0006) [2024-09-30 01:52:16,458][1157819] Updated weights for policy 0, policy_version 87748 (0.0006) [2024-09-30 01:52:16,941][1157819] Updated weights for policy 0, policy_version 87758 (0.0006) [2024-09-30 01:52:17,445][1157819] Updated weights for policy 0, policy_version 87768 (0.0006) [2024-09-30 01:52:17,945][1157819] Updated weights for policy 0, policy_version 87778 (0.0006) [2024-09-30 01:52:18,439][1157819] Updated weights for policy 0, policy_version 87788 (0.0006) [2024-09-30 01:52:18,931][1157819] Updated weights for policy 0, policy_version 87798 (0.0006) [2024-09-30 01:52:19,434][1157819] Updated weights for policy 0, policy_version 87808 (0.0006) [2024-09-30 01:52:19,922][1157819] Updated weights for policy 0, policy_version 87818 (0.0006) [2024-09-30 01:52:20,428][1157819] Updated weights for policy 0, policy_version 87828 (0.0006) [2024-09-30 01:52:20,466][1157520] Fps is (10 sec: 82328.9, 60 sec: 80554.6, 300 sec: 79823.4). Total num frames: 359747584. Throughput: 0: 20525.8. Samples: 79922924. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:52:20,466][1157520] Avg episode reward: [(0, '52.626')] [2024-09-30 01:52:20,928][1157819] Updated weights for policy 0, policy_version 87838 (0.0006) [2024-09-30 01:52:21,388][1157819] Updated weights for policy 0, policy_version 87848 (0.0006) [2024-09-30 01:52:21,869][1157819] Updated weights for policy 0, policy_version 87858 (0.0006) [2024-09-30 01:52:22,373][1157819] Updated weights for policy 0, policy_version 87868 (0.0006) [2024-09-30 01:52:22,872][1157819] Updated weights for policy 0, policy_version 87878 (0.0006) [2024-09-30 01:52:23,325][1157819] Updated weights for policy 0, policy_version 87888 (0.0006) [2024-09-30 01:52:23,823][1157819] Updated weights for policy 0, policy_version 87898 (0.0006) [2024-09-30 01:52:24,329][1157819] Updated weights for policy 0, policy_version 87908 (0.0006) [2024-09-30 01:52:24,831][1157819] Updated weights for policy 0, policy_version 87918 (0.0006) [2024-09-30 01:52:25,324][1157819] Updated weights for policy 0, policy_version 87928 (0.0006) [2024-09-30 01:52:25,466][1157520] Fps is (10 sec: 82740.2, 60 sec: 81169.0, 300 sec: 79879.0). Total num frames: 360161280. Throughput: 0: 20644.6. Samples: 79985636. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:52:25,466][1157520] Avg episode reward: [(0, '54.785')] [2024-09-30 01:52:25,827][1157819] Updated weights for policy 0, policy_version 87938 (0.0006) [2024-09-30 01:52:26,332][1157819] Updated weights for policy 0, policy_version 87948 (0.0006) [2024-09-30 01:52:26,819][1157819] Updated weights for policy 0, policy_version 87958 (0.0006) [2024-09-30 01:52:27,322][1157819] Updated weights for policy 0, policy_version 87968 (0.0006) [2024-09-30 01:52:27,823][1157819] Updated weights for policy 0, policy_version 87978 (0.0006) [2024-09-30 01:52:28,311][1157819] Updated weights for policy 0, policy_version 87988 (0.0006) [2024-09-30 01:52:28,810][1157819] Updated weights for policy 0, policy_version 87998 (0.0006) [2024-09-30 01:52:29,306][1157819] Updated weights for policy 0, policy_version 88008 (0.0006) [2024-09-30 01:52:29,804][1157819] Updated weights for policy 0, policy_version 88018 (0.0006) [2024-09-30 01:52:30,292][1157819] Updated weights for policy 0, policy_version 88028 (0.0006) [2024-09-30 01:52:30,466][1157520] Fps is (10 sec: 82740.0, 60 sec: 81715.2, 300 sec: 79851.2). Total num frames: 360574976. Throughput: 0: 20722.6. Samples: 80109628. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:52:30,466][1157520] Avg episode reward: [(0, '56.296')] [2024-09-30 01:52:30,786][1157819] Updated weights for policy 0, policy_version 88038 (0.0006) [2024-09-30 01:52:31,327][1157819] Updated weights for policy 0, policy_version 88048 (0.0006) [2024-09-30 01:52:31,847][1157819] Updated weights for policy 0, policy_version 88058 (0.0006) [2024-09-30 01:52:32,358][1157819] Updated weights for policy 0, policy_version 88068 (0.0006) [2024-09-30 01:52:32,856][1157819] Updated weights for policy 0, policy_version 88078 (0.0006) [2024-09-30 01:52:33,359][1157819] Updated weights for policy 0, policy_version 88088 (0.0006) [2024-09-30 01:52:33,892][1157819] Updated weights for policy 0, policy_version 88098 (0.0006) [2024-09-30 01:52:34,436][1157819] Updated weights for policy 0, policy_version 88108 (0.0006) [2024-09-30 01:52:34,964][1157819] Updated weights for policy 0, policy_version 88118 (0.0006) [2024-09-30 01:52:35,466][1157520] Fps is (10 sec: 80691.1, 60 sec: 81988.3, 300 sec: 79767.9). Total num frames: 360968192. Throughput: 0: 20644.7. Samples: 80229508. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:52:35,466][1157520] Avg episode reward: [(0, '55.462')] [2024-09-30 01:52:35,513][1157819] Updated weights for policy 0, policy_version 88128 (0.0006) [2024-09-30 01:52:36,051][1157819] Updated weights for policy 0, policy_version 88138 (0.0006) [2024-09-30 01:52:36,616][1157819] Updated weights for policy 0, policy_version 88148 (0.0006) [2024-09-30 01:52:37,159][1157819] Updated weights for policy 0, policy_version 88158 (0.0006) [2024-09-30 01:52:37,718][1157819] Updated weights for policy 0, policy_version 88168 (0.0006) [2024-09-30 01:52:38,234][1157819] Updated weights for policy 0, policy_version 88178 (0.0006) [2024-09-30 01:52:38,773][1157819] Updated weights for policy 0, policy_version 88188 (0.0006) [2024-09-30 01:52:39,274][1157819] Updated weights for policy 0, policy_version 88198 (0.0006) [2024-09-30 01:52:39,770][1157819] Updated weights for policy 0, policy_version 88208 (0.0006) [2024-09-30 01:52:40,271][1157819] Updated weights for policy 0, policy_version 88218 (0.0006) [2024-09-30 01:52:40,466][1157520] Fps is (10 sec: 77823.9, 60 sec: 81851.7, 300 sec: 79656.8). Total num frames: 361353216. Throughput: 0: 20478.6. Samples: 80285440. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:52:40,466][1157520] Avg episode reward: [(0, '54.136')] [2024-09-30 01:52:40,784][1157819] Updated weights for policy 0, policy_version 88228 (0.0006) [2024-09-30 01:52:41,300][1157819] Updated weights for policy 0, policy_version 88238 (0.0006) [2024-09-30 01:52:41,818][1157819] Updated weights for policy 0, policy_version 88248 (0.0006) [2024-09-30 01:52:42,323][1157819] Updated weights for policy 0, policy_version 88258 (0.0006) [2024-09-30 01:52:42,830][1157819] Updated weights for policy 0, policy_version 88268 (0.0006) [2024-09-30 01:52:43,349][1157819] Updated weights for policy 0, policy_version 88278 (0.0006) [2024-09-30 01:52:43,895][1157819] Updated weights for policy 0, policy_version 88288 (0.0006) [2024-09-30 01:52:44,410][1157819] Updated weights for policy 0, policy_version 88298 (0.0006) [2024-09-30 01:52:44,898][1157819] Updated weights for policy 0, policy_version 88308 (0.0006) [2024-09-30 01:52:45,435][1157819] Updated weights for policy 0, policy_version 88318 (0.0006) [2024-09-30 01:52:45,466][1157520] Fps is (10 sec: 78233.4, 60 sec: 81851.7, 300 sec: 79601.3). Total num frames: 361750528. Throughput: 0: 20347.3. Samples: 80405072. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:52:45,466][1157520] Avg episode reward: [(0, '53.592')] [2024-09-30 01:52:45,957][1157819] Updated weights for policy 0, policy_version 88328 (0.0006) [2024-09-30 01:52:46,489][1157819] Updated weights for policy 0, policy_version 88338 (0.0006) [2024-09-30 01:52:46,646][1157736] Signal inference workers to stop experience collection... (5700 times) [2024-09-30 01:52:46,646][1157736] Signal inference workers to resume experience collection... (5700 times) [2024-09-30 01:52:46,650][1157819] InferenceWorker_p0-w0: stopping experience collection (5700 times) [2024-09-30 01:52:46,650][1157819] InferenceWorker_p0-w0: resuming experience collection (5700 times) [2024-09-30 01:52:46,979][1157819] Updated weights for policy 0, policy_version 88348 (0.0006) [2024-09-30 01:52:47,478][1157819] Updated weights for policy 0, policy_version 88358 (0.0006) [2024-09-30 01:52:48,052][1157819] Updated weights for policy 0, policy_version 88368 (0.0006) [2024-09-30 01:52:48,563][1157819] Updated weights for policy 0, policy_version 88378 (0.0006) [2024-09-30 01:52:49,065][1157819] Updated weights for policy 0, policy_version 88388 (0.0006) [2024-09-30 01:52:49,551][1157819] Updated weights for policy 0, policy_version 88398 (0.0006) [2024-09-30 01:52:50,061][1157819] Updated weights for policy 0, policy_version 88408 (0.0006) [2024-09-30 01:52:50,466][1157520] Fps is (10 sec: 79462.0, 60 sec: 81510.4, 300 sec: 79545.7). Total num frames: 362147840. Throughput: 0: 20218.7. Samples: 80523948. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:52:50,466][1157520] Avg episode reward: [(0, '53.494')] [2024-09-30 01:52:50,603][1157819] Updated weights for policy 0, policy_version 88418 (0.0006) [2024-09-30 01:52:51,124][1157819] Updated weights for policy 0, policy_version 88428 (0.0006) [2024-09-30 01:52:51,636][1157819] Updated weights for policy 0, policy_version 88438 (0.0006) [2024-09-30 01:52:52,134][1157819] Updated weights for policy 0, policy_version 88448 (0.0006) [2024-09-30 01:52:52,657][1157819] Updated weights for policy 0, policy_version 88458 (0.0007) [2024-09-30 01:52:53,217][1157819] Updated weights for policy 0, policy_version 88468 (0.0006) [2024-09-30 01:52:53,768][1157819] Updated weights for policy 0, policy_version 88478 (0.0007) [2024-09-30 01:52:54,284][1157819] Updated weights for policy 0, policy_version 88488 (0.0006) [2024-09-30 01:52:54,793][1157819] Updated weights for policy 0, policy_version 88498 (0.0007) [2024-09-30 01:52:55,333][1157819] Updated weights for policy 0, policy_version 88508 (0.0006) [2024-09-30 01:52:55,466][1157520] Fps is (10 sec: 78644.4, 60 sec: 80964.5, 300 sec: 79476.3). Total num frames: 362536960. Throughput: 0: 20157.3. Samples: 80582920. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:52:55,466][1157520] Avg episode reward: [(0, '52.384')] [2024-09-30 01:52:55,853][1157819] Updated weights for policy 0, policy_version 88518 (0.0007) [2024-09-30 01:52:56,406][1157819] Updated weights for policy 0, policy_version 88528 (0.0006) [2024-09-30 01:52:56,967][1157819] Updated weights for policy 0, policy_version 88538 (0.0006) [2024-09-30 01:52:57,506][1157819] Updated weights for policy 0, policy_version 88548 (0.0006) [2024-09-30 01:52:58,027][1157819] Updated weights for policy 0, policy_version 88558 (0.0006) [2024-09-30 01:52:58,593][1157819] Updated weights for policy 0, policy_version 88568 (0.0006) [2024-09-30 01:52:59,158][1157819] Updated weights for policy 0, policy_version 88578 (0.0006) [2024-09-30 01:52:59,720][1157819] Updated weights for policy 0, policy_version 88588 (0.0006) [2024-09-30 01:53:00,294][1157819] Updated weights for policy 0, policy_version 88598 (0.0006) [2024-09-30 01:53:00,466][1157520] Fps is (10 sec: 76185.6, 60 sec: 80213.3, 300 sec: 79309.7). Total num frames: 362909696. Throughput: 0: 19931.5. Samples: 80696956. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:53:00,466][1157520] Avg episode reward: [(0, '56.157')] [2024-09-30 01:53:00,862][1157819] Updated weights for policy 0, policy_version 88608 (0.0006) [2024-09-30 01:53:01,385][1157819] Updated weights for policy 0, policy_version 88618 (0.0006) [2024-09-30 01:53:01,917][1157819] Updated weights for policy 0, policy_version 88628 (0.0006) [2024-09-30 01:53:02,461][1157819] Updated weights for policy 0, policy_version 88638 (0.0006) [2024-09-30 01:53:02,986][1157819] Updated weights for policy 0, policy_version 88648 (0.0006) [2024-09-30 01:53:03,504][1157819] Updated weights for policy 0, policy_version 88658 (0.0006) [2024-09-30 01:53:04,026][1157819] Updated weights for policy 0, policy_version 88668 (0.0006) [2024-09-30 01:53:04,624][1157819] Updated weights for policy 0, policy_version 88678 (0.0006) [2024-09-30 01:53:05,165][1157819] Updated weights for policy 0, policy_version 88688 (0.0007) [2024-09-30 01:53:05,466][1157520] Fps is (10 sec: 74956.0, 60 sec: 79599.1, 300 sec: 79198.6). Total num frames: 363286528. Throughput: 0: 19709.9. Samples: 80809868. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 01:53:05,466][1157520] Avg episode reward: [(0, '54.980')] [2024-09-30 01:53:05,670][1157819] Updated weights for policy 0, policy_version 88698 (0.0006) [2024-09-30 01:53:06,205][1157819] Updated weights for policy 0, policy_version 88708 (0.0006) [2024-09-30 01:53:06,756][1157819] Updated weights for policy 0, policy_version 88718 (0.0006) [2024-09-30 01:53:07,256][1157819] Updated weights for policy 0, policy_version 88728 (0.0006) [2024-09-30 01:53:07,745][1157819] Updated weights for policy 0, policy_version 88738 (0.0006) [2024-09-30 01:53:08,263][1157819] Updated weights for policy 0, policy_version 88748 (0.0006) [2024-09-30 01:53:08,840][1157819] Updated weights for policy 0, policy_version 88758 (0.0006) [2024-09-30 01:53:09,390][1157819] Updated weights for policy 0, policy_version 88768 (0.0006) [2024-09-30 01:53:09,950][1157819] Updated weights for policy 0, policy_version 88778 (0.0006) [2024-09-30 01:53:10,466][1157520] Fps is (10 sec: 76185.6, 60 sec: 79121.0, 300 sec: 79129.2). Total num frames: 363671552. Throughput: 0: 19611.9. Samples: 80868172. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 01:53:10,466][1157520] Avg episode reward: [(0, '55.928')] [2024-09-30 01:53:10,505][1157819] Updated weights for policy 0, policy_version 88788 (0.0006) [2024-09-30 01:53:11,061][1157819] Updated weights for policy 0, policy_version 88798 (0.0006) [2024-09-30 01:53:11,638][1157819] Updated weights for policy 0, policy_version 88808 (0.0007) [2024-09-30 01:53:11,640][1157736] Signal inference workers to stop experience collection... (5750 times) [2024-09-30 01:53:11,641][1157736] Signal inference workers to resume experience collection... (5750 times) [2024-09-30 01:53:11,644][1157819] InferenceWorker_p0-w0: stopping experience collection (5750 times) [2024-09-30 01:53:11,644][1157819] InferenceWorker_p0-w0: resuming experience collection (5750 times) [2024-09-30 01:53:12,164][1157819] Updated weights for policy 0, policy_version 88818 (0.0006) [2024-09-30 01:53:12,729][1157819] Updated weights for policy 0, policy_version 88828 (0.0006) [2024-09-30 01:53:13,269][1157819] Updated weights for policy 0, policy_version 88838 (0.0006) [2024-09-30 01:53:13,815][1157819] Updated weights for policy 0, policy_version 88848 (0.0006) [2024-09-30 01:53:14,362][1157819] Updated weights for policy 0, policy_version 88858 (0.0006) [2024-09-30 01:53:14,916][1157819] Updated weights for policy 0, policy_version 88868 (0.0006) [2024-09-30 01:53:15,466][1157520] Fps is (10 sec: 75366.5, 60 sec: 78438.6, 300 sec: 78990.3). Total num frames: 364040192. Throughput: 0: 19335.0. Samples: 80979704. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 01:53:15,466][1157520] Avg episode reward: [(0, '54.793')] [2024-09-30 01:53:15,495][1157819] Updated weights for policy 0, policy_version 88878 (0.0006) [2024-09-30 01:53:16,038][1157819] Updated weights for policy 0, policy_version 88888 (0.0006) [2024-09-30 01:53:16,600][1157819] Updated weights for policy 0, policy_version 88898 (0.0006) [2024-09-30 01:53:17,147][1157819] Updated weights for policy 0, policy_version 88908 (0.0006) [2024-09-30 01:53:17,693][1157819] Updated weights for policy 0, policy_version 88918 (0.0006) [2024-09-30 01:53:18,282][1157819] Updated weights for policy 0, policy_version 88928 (0.0006) [2024-09-30 01:53:18,835][1157819] Updated weights for policy 0, policy_version 88938 (0.0006) [2024-09-30 01:53:19,418][1157819] Updated weights for policy 0, policy_version 88948 (0.0006) [2024-09-30 01:53:19,981][1157819] Updated weights for policy 0, policy_version 88958 (0.0006) [2024-09-30 01:53:20,466][1157520] Fps is (10 sec: 73318.9, 60 sec: 77619.3, 300 sec: 78782.1). Total num frames: 364404736. Throughput: 0: 19113.2. Samples: 81089600. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 01:53:20,466][1157520] Avg episode reward: [(0, '53.400')] [2024-09-30 01:53:20,561][1157819] Updated weights for policy 0, policy_version 88968 (0.0006) [2024-09-30 01:53:21,146][1157819] Updated weights for policy 0, policy_version 88978 (0.0007) [2024-09-30 01:53:21,739][1157819] Updated weights for policy 0, policy_version 88988 (0.0006) [2024-09-30 01:53:22,299][1157819] Updated weights for policy 0, policy_version 88998 (0.0007) [2024-09-30 01:53:22,867][1157819] Updated weights for policy 0, policy_version 89008 (0.0006) [2024-09-30 01:53:23,431][1157819] Updated weights for policy 0, policy_version 89018 (0.0006) [2024-09-30 01:53:23,991][1157819] Updated weights for policy 0, policy_version 89028 (0.0006) [2024-09-30 01:53:24,526][1157819] Updated weights for policy 0, policy_version 89038 (0.0006) [2024-09-30 01:53:25,096][1157819] Updated weights for policy 0, policy_version 89048 (0.0006) [2024-09-30 01:53:25,466][1157520] Fps is (10 sec: 72908.8, 60 sec: 76800.0, 300 sec: 78629.3). Total num frames: 364769280. Throughput: 0: 19050.6. Samples: 81142716. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 01:53:25,466][1157520] Avg episode reward: [(0, '57.567')] [2024-09-30 01:53:25,471][1157736] Saving new best policy, reward=57.567! [2024-09-30 01:53:25,633][1157819] Updated weights for policy 0, policy_version 89058 (0.0006) [2024-09-30 01:53:26,251][1157819] Updated weights for policy 0, policy_version 89068 (0.0006) [2024-09-30 01:53:26,813][1157819] Updated weights for policy 0, policy_version 89078 (0.0007) [2024-09-30 01:53:27,422][1157819] Updated weights for policy 0, policy_version 89088 (0.0006) [2024-09-30 01:53:27,986][1157819] Updated weights for policy 0, policy_version 89098 (0.0006) [2024-09-30 01:53:28,540][1157819] Updated weights for policy 0, policy_version 89108 (0.0006) [2024-09-30 01:53:29,117][1157819] Updated weights for policy 0, policy_version 89118 (0.0006) [2024-09-30 01:53:29,656][1157819] Updated weights for policy 0, policy_version 89128 (0.0006) [2024-09-30 01:53:30,216][1157819] Updated weights for policy 0, policy_version 89138 (0.0006) [2024-09-30 01:53:30,466][1157520] Fps is (10 sec: 71679.4, 60 sec: 75775.9, 300 sec: 78379.4). Total num frames: 365121536. Throughput: 0: 18796.9. Samples: 81250932. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 01:53:30,466][1157520] Avg episode reward: [(0, '54.535')] [2024-09-30 01:53:30,775][1157819] Updated weights for policy 0, policy_version 89148 (0.0006) [2024-09-30 01:53:31,350][1157819] Updated weights for policy 0, policy_version 89158 (0.0006) [2024-09-30 01:53:31,932][1157819] Updated weights for policy 0, policy_version 89168 (0.0006) [2024-09-30 01:53:32,480][1157819] Updated weights for policy 0, policy_version 89178 (0.0006) [2024-09-30 01:53:32,986][1157819] Updated weights for policy 0, policy_version 89188 (0.0006) [2024-09-30 01:53:33,494][1157819] Updated weights for policy 0, policy_version 89198 (0.0006) [2024-09-30 01:53:34,052][1157819] Updated weights for policy 0, policy_version 89208 (0.0006) [2024-09-30 01:53:34,609][1157819] Updated weights for policy 0, policy_version 89218 (0.0006) [2024-09-30 01:53:35,186][1157819] Updated weights for policy 0, policy_version 89228 (0.0006) [2024-09-30 01:53:35,466][1157520] Fps is (10 sec: 72499.0, 60 sec: 75434.7, 300 sec: 78226.7). Total num frames: 365494272. Throughput: 0: 18632.9. Samples: 81362428. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 01:53:35,466][1157520] Avg episode reward: [(0, '54.895')] [2024-09-30 01:53:35,727][1157819] Updated weights for policy 0, policy_version 89238 (0.0006) [2024-09-30 01:53:36,277][1157819] Updated weights for policy 0, policy_version 89248 (0.0006) [2024-09-30 01:53:36,833][1157819] Updated weights for policy 0, policy_version 89258 (0.0006) [2024-09-30 01:53:37,358][1157819] Updated weights for policy 0, policy_version 89268 (0.0006) [2024-09-30 01:53:37,920][1157819] Updated weights for policy 0, policy_version 89278 (0.0006) [2024-09-30 01:53:38,473][1157819] Updated weights for policy 0, policy_version 89288 (0.0006) [2024-09-30 01:53:39,036][1157819] Updated weights for policy 0, policy_version 89298 (0.0006) [2024-09-30 01:53:39,579][1157819] Updated weights for policy 0, policy_version 89308 (0.0006) [2024-09-30 01:53:40,120][1157819] Updated weights for policy 0, policy_version 89318 (0.0006) [2024-09-30 01:53:40,466][1157520] Fps is (10 sec: 74957.1, 60 sec: 75298.1, 300 sec: 78087.8). Total num frames: 365871104. Throughput: 0: 18556.9. Samples: 81417984. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 01:53:40,466][1157520] Avg episode reward: [(0, '52.748')] [2024-09-30 01:53:40,658][1157819] Updated weights for policy 0, policy_version 89328 (0.0006) [2024-09-30 01:53:41,188][1157819] Updated weights for policy 0, policy_version 89338 (0.0006) [2024-09-30 01:53:41,754][1157819] Updated weights for policy 0, policy_version 89348 (0.0006) [2024-09-30 01:53:42,307][1157819] Updated weights for policy 0, policy_version 89358 (0.0006) [2024-09-30 01:53:42,898][1157819] Updated weights for policy 0, policy_version 89368 (0.0006) [2024-09-30 01:53:43,474][1157819] Updated weights for policy 0, policy_version 89378 (0.0006) [2024-09-30 01:53:44,041][1157819] Updated weights for policy 0, policy_version 89388 (0.0006) [2024-09-30 01:53:44,668][1157819] Updated weights for policy 0, policy_version 89398 (0.0006) [2024-09-30 01:53:45,214][1157819] Updated weights for policy 0, policy_version 89408 (0.0006) [2024-09-30 01:53:45,466][1157520] Fps is (10 sec: 73728.1, 60 sec: 74683.8, 300 sec: 77962.8). Total num frames: 366231552. Throughput: 0: 18487.9. Samples: 81528912. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 01:53:45,466][1157520] Avg episode reward: [(0, '55.630')] [2024-09-30 01:53:45,776][1157819] Updated weights for policy 0, policy_version 89418 (0.0006) [2024-09-30 01:53:46,305][1157819] Updated weights for policy 0, policy_version 89428 (0.0006) [2024-09-30 01:53:46,841][1157819] Updated weights for policy 0, policy_version 89438 (0.0006) [2024-09-30 01:53:47,360][1157819] Updated weights for policy 0, policy_version 89448 (0.0006) [2024-09-30 01:53:47,867][1157819] Updated weights for policy 0, policy_version 89458 (0.0006) [2024-09-30 01:53:48,370][1157819] Updated weights for policy 0, policy_version 89468 (0.0006) [2024-09-30 01:53:48,899][1157819] Updated weights for policy 0, policy_version 89478 (0.0006) [2024-09-30 01:53:49,404][1157819] Updated weights for policy 0, policy_version 89488 (0.0006) [2024-09-30 01:53:49,922][1157819] Updated weights for policy 0, policy_version 89498 (0.0006) [2024-09-30 01:53:50,445][1157819] Updated weights for policy 0, policy_version 89508 (0.0006) [2024-09-30 01:53:50,466][1157520] Fps is (10 sec: 75366.4, 60 sec: 74615.5, 300 sec: 77949.0). Total num frames: 366624768. Throughput: 0: 18520.3. Samples: 81643280. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 01:53:50,466][1157520] Avg episode reward: [(0, '55.352')] [2024-09-30 01:53:50,940][1157819] Updated weights for policy 0, policy_version 89518 (0.0006) [2024-09-30 01:53:51,453][1157819] Updated weights for policy 0, policy_version 89528 (0.0006) [2024-09-30 01:53:51,975][1157819] Updated weights for policy 0, policy_version 89538 (0.0006) [2024-09-30 01:53:52,543][1157819] Updated weights for policy 0, policy_version 89548 (0.0006) [2024-09-30 01:53:53,112][1157819] Updated weights for policy 0, policy_version 89558 (0.0006) [2024-09-30 01:53:53,636][1157819] Updated weights for policy 0, policy_version 89568 (0.0006) [2024-09-30 01:53:54,180][1157819] Updated weights for policy 0, policy_version 89578 (0.0006) [2024-09-30 01:53:54,711][1157819] Updated weights for policy 0, policy_version 89588 (0.0006) [2024-09-30 01:53:55,276][1157819] Updated weights for policy 0, policy_version 89598 (0.0006) [2024-09-30 01:53:55,466][1157520] Fps is (10 sec: 77413.9, 60 sec: 74478.7, 300 sec: 77893.4). Total num frames: 367005696. Throughput: 0: 18524.8. Samples: 81701788. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 01:53:55,466][1157520] Avg episode reward: [(0, '54.167')] [2024-09-30 01:53:55,828][1157819] Updated weights for policy 0, policy_version 89608 (0.0006) [2024-09-30 01:53:56,345][1157819] Updated weights for policy 0, policy_version 89618 (0.0006) [2024-09-30 01:53:56,927][1157819] Updated weights for policy 0, policy_version 89628 (0.0006) [2024-09-30 01:53:57,446][1157819] Updated weights for policy 0, policy_version 89638 (0.0006) [2024-09-30 01:53:57,987][1157819] Updated weights for policy 0, policy_version 89648 (0.0006) [2024-09-30 01:53:58,521][1157819] Updated weights for policy 0, policy_version 89658 (0.0006) [2024-09-30 01:53:59,070][1157819] Updated weights for policy 0, policy_version 89668 (0.0006) [2024-09-30 01:53:59,136][1157736] Signal inference workers to stop experience collection... (5800 times) [2024-09-30 01:53:59,137][1157736] Signal inference workers to resume experience collection... (5800 times) [2024-09-30 01:53:59,140][1157819] InferenceWorker_p0-w0: stopping experience collection (5800 times) [2024-09-30 01:53:59,140][1157819] InferenceWorker_p0-w0: resuming experience collection (5800 times) [2024-09-30 01:53:59,622][1157819] Updated weights for policy 0, policy_version 89678 (0.0006) [2024-09-30 01:54:00,132][1157819] Updated weights for policy 0, policy_version 89688 (0.0006) [2024-09-30 01:54:00,466][1157520] Fps is (10 sec: 76184.8, 60 sec: 74615.4, 300 sec: 77851.7). Total num frames: 367386624. Throughput: 0: 18553.8. Samples: 81814628. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:54:00,466][1157520] Avg episode reward: [(0, '53.981')] [2024-09-30 01:54:00,649][1157819] Updated weights for policy 0, policy_version 89698 (0.0006) [2024-09-30 01:54:01,188][1157819] Updated weights for policy 0, policy_version 89708 (0.0006) [2024-09-30 01:54:01,711][1157819] Updated weights for policy 0, policy_version 89718 (0.0006) [2024-09-30 01:54:02,220][1157819] Updated weights for policy 0, policy_version 89728 (0.0006) [2024-09-30 01:54:02,744][1157819] Updated weights for policy 0, policy_version 89738 (0.0006) [2024-09-30 01:54:03,263][1157819] Updated weights for policy 0, policy_version 89748 (0.0006) [2024-09-30 01:54:03,786][1157819] Updated weights for policy 0, policy_version 89758 (0.0006) [2024-09-30 01:54:04,294][1157819] Updated weights for policy 0, policy_version 89768 (0.0006) [2024-09-30 01:54:04,850][1157819] Updated weights for policy 0, policy_version 89778 (0.0006) [2024-09-30 01:54:05,438][1157819] Updated weights for policy 0, policy_version 89788 (0.0006) [2024-09-30 01:54:05,466][1157520] Fps is (10 sec: 77004.2, 60 sec: 74820.1, 300 sec: 77865.6). Total num frames: 367775744. Throughput: 0: 18705.5. Samples: 81931352. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:54:05,466][1157520] Avg episode reward: [(0, '54.155')] [2024-09-30 01:54:05,473][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000089789_367775744.pth... [2024-09-30 01:54:05,529][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000085227_349089792.pth [2024-09-30 01:54:05,922][1157819] Updated weights for policy 0, policy_version 89798 (0.0006) [2024-09-30 01:54:06,437][1157819] Updated weights for policy 0, policy_version 89808 (0.0006) [2024-09-30 01:54:06,964][1157819] Updated weights for policy 0, policy_version 89818 (0.0006) [2024-09-30 01:54:07,483][1157819] Updated weights for policy 0, policy_version 89828 (0.0006) [2024-09-30 01:54:07,992][1157819] Updated weights for policy 0, policy_version 89838 (0.0006) [2024-09-30 01:54:08,566][1157819] Updated weights for policy 0, policy_version 89848 (0.0006) [2024-09-30 01:54:09,102][1157819] Updated weights for policy 0, policy_version 89858 (0.0006) [2024-09-30 01:54:09,648][1157819] Updated weights for policy 0, policy_version 89868 (0.0006) [2024-09-30 01:54:10,231][1157819] Updated weights for policy 0, policy_version 89878 (0.0006) [2024-09-30 01:54:10,466][1157520] Fps is (10 sec: 77005.3, 60 sec: 74752.0, 300 sec: 77796.2). Total num frames: 368156672. Throughput: 0: 18823.3. Samples: 81989764. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:54:10,466][1157520] Avg episode reward: [(0, '55.218')] [2024-09-30 01:54:10,772][1157819] Updated weights for policy 0, policy_version 89888 (0.0006) [2024-09-30 01:54:11,361][1157819] Updated weights for policy 0, policy_version 89898 (0.0006) [2024-09-30 01:54:11,898][1157819] Updated weights for policy 0, policy_version 89908 (0.0006) [2024-09-30 01:54:12,446][1157819] Updated weights for policy 0, policy_version 89918 (0.0006) [2024-09-30 01:54:13,021][1157819] Updated weights for policy 0, policy_version 89928 (0.0006) [2024-09-30 01:54:13,595][1157819] Updated weights for policy 0, policy_version 89938 (0.0006) [2024-09-30 01:54:14,170][1157819] Updated weights for policy 0, policy_version 89948 (0.0006) [2024-09-30 01:54:14,769][1157819] Updated weights for policy 0, policy_version 89958 (0.0006) [2024-09-30 01:54:15,373][1157819] Updated weights for policy 0, policy_version 89968 (0.0006) [2024-09-30 01:54:15,466][1157520] Fps is (10 sec: 74137.6, 60 sec: 74615.3, 300 sec: 77671.2). Total num frames: 368517120. Throughput: 0: 18872.6. Samples: 82100200. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:54:15,466][1157520] Avg episode reward: [(0, '55.454')] [2024-09-30 01:54:15,947][1157819] Updated weights for policy 0, policy_version 89978 (0.0006) [2024-09-30 01:54:16,494][1157819] Updated weights for policy 0, policy_version 89988 (0.0006) [2024-09-30 01:54:17,031][1157819] Updated weights for policy 0, policy_version 89998 (0.0006) [2024-09-30 01:54:17,540][1157819] Updated weights for policy 0, policy_version 90008 (0.0006) [2024-09-30 01:54:18,066][1157819] Updated weights for policy 0, policy_version 90018 (0.0006) [2024-09-30 01:54:18,601][1157819] Updated weights for policy 0, policy_version 90028 (0.0006) [2024-09-30 01:54:19,094][1157819] Updated weights for policy 0, policy_version 90038 (0.0006) [2024-09-30 01:54:19,582][1157819] Updated weights for policy 0, policy_version 90048 (0.0006) [2024-09-30 01:54:20,115][1157819] Updated weights for policy 0, policy_version 90058 (0.0006) [2024-09-30 01:54:20,466][1157520] Fps is (10 sec: 74547.3, 60 sec: 74956.7, 300 sec: 77643.5). Total num frames: 368902144. Throughput: 0: 18892.6. Samples: 82212596. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:54:20,466][1157520] Avg episode reward: [(0, '55.002')] [2024-09-30 01:54:20,678][1157819] Updated weights for policy 0, policy_version 90068 (0.0006) [2024-09-30 01:54:21,201][1157819] Updated weights for policy 0, policy_version 90078 (0.0006) [2024-09-30 01:54:21,717][1157819] Updated weights for policy 0, policy_version 90088 (0.0006) [2024-09-30 01:54:22,220][1157819] Updated weights for policy 0, policy_version 90098 (0.0006) [2024-09-30 01:54:22,743][1157819] Updated weights for policy 0, policy_version 90108 (0.0006) [2024-09-30 01:54:23,290][1157819] Updated weights for policy 0, policy_version 90118 (0.0006) [2024-09-30 01:54:23,809][1157819] Updated weights for policy 0, policy_version 90128 (0.0006) [2024-09-30 01:54:24,361][1157819] Updated weights for policy 0, policy_version 90138 (0.0006) [2024-09-30 01:54:24,882][1157819] Updated weights for policy 0, policy_version 90148 (0.0006) [2024-09-30 01:54:25,394][1157819] Updated weights for policy 0, policy_version 90158 (0.0006) [2024-09-30 01:54:25,466][1157520] Fps is (10 sec: 77415.0, 60 sec: 75366.3, 300 sec: 77671.2). Total num frames: 369291264. Throughput: 0: 18957.9. Samples: 82271092. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:54:25,466][1157520] Avg episode reward: [(0, '55.150')] [2024-09-30 01:54:25,920][1157819] Updated weights for policy 0, policy_version 90168 (0.0006) [2024-09-30 01:54:26,451][1157819] Updated weights for policy 0, policy_version 90178 (0.0006) [2024-09-30 01:54:26,963][1157819] Updated weights for policy 0, policy_version 90188 (0.0006) [2024-09-30 01:54:27,456][1157819] Updated weights for policy 0, policy_version 90198 (0.0006) [2024-09-30 01:54:27,992][1157819] Updated weights for policy 0, policy_version 90208 (0.0006) [2024-09-30 01:54:28,545][1157819] Updated weights for policy 0, policy_version 90218 (0.0006) [2024-09-30 01:54:29,074][1157819] Updated weights for policy 0, policy_version 90228 (0.0006) [2024-09-30 01:54:29,587][1157819] Updated weights for policy 0, policy_version 90238 (0.0006) [2024-09-30 01:54:30,104][1157819] Updated weights for policy 0, policy_version 90248 (0.0006) [2024-09-30 01:54:30,466][1157520] Fps is (10 sec: 77823.8, 60 sec: 75980.8, 300 sec: 77740.7). Total num frames: 369680384. Throughput: 0: 19093.1. Samples: 82388104. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:54:30,466][1157520] Avg episode reward: [(0, '54.060')] [2024-09-30 01:54:30,623][1157819] Updated weights for policy 0, policy_version 90258 (0.0006) [2024-09-30 01:54:31,138][1157819] Updated weights for policy 0, policy_version 90268 (0.0006) [2024-09-30 01:54:31,698][1157819] Updated weights for policy 0, policy_version 90278 (0.0006) [2024-09-30 01:54:32,224][1157819] Updated weights for policy 0, policy_version 90288 (0.0006) [2024-09-30 01:54:32,732][1157819] Updated weights for policy 0, policy_version 90298 (0.0006) [2024-09-30 01:54:33,257][1157819] Updated weights for policy 0, policy_version 90308 (0.0006) [2024-09-30 01:54:33,796][1157819] Updated weights for policy 0, policy_version 90318 (0.0007) [2024-09-30 01:54:34,331][1157819] Updated weights for policy 0, policy_version 90328 (0.0007) [2024-09-30 01:54:34,868][1157819] Updated weights for policy 0, policy_version 90338 (0.0006) [2024-09-30 01:54:35,391][1157819] Updated weights for policy 0, policy_version 90348 (0.0006) [2024-09-30 01:54:35,466][1157520] Fps is (10 sec: 77824.0, 60 sec: 76253.8, 300 sec: 77768.4). Total num frames: 370069504. Throughput: 0: 19143.5. Samples: 82504740. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:54:35,466][1157520] Avg episode reward: [(0, '55.782')] [2024-09-30 01:54:35,899][1157819] Updated weights for policy 0, policy_version 90358 (0.0006) [2024-09-30 01:54:36,416][1157819] Updated weights for policy 0, policy_version 90368 (0.0006) [2024-09-30 01:54:36,927][1157819] Updated weights for policy 0, policy_version 90378 (0.0006) [2024-09-30 01:54:37,475][1157819] Updated weights for policy 0, policy_version 90388 (0.0006) [2024-09-30 01:54:37,998][1157819] Updated weights for policy 0, policy_version 90398 (0.0006) [2024-09-30 01:54:38,490][1157819] Updated weights for policy 0, policy_version 90408 (0.0006) [2024-09-30 01:54:38,995][1157819] Updated weights for policy 0, policy_version 90418 (0.0006) [2024-09-30 01:54:39,543][1157819] Updated weights for policy 0, policy_version 90428 (0.0006) [2024-09-30 01:54:40,042][1157819] Updated weights for policy 0, policy_version 90438 (0.0006) [2024-09-30 01:54:40,466][1157520] Fps is (10 sec: 78233.5, 60 sec: 76526.8, 300 sec: 77851.8). Total num frames: 370462720. Throughput: 0: 19151.3. Samples: 82563596. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:54:40,466][1157520] Avg episode reward: [(0, '53.771')] [2024-09-30 01:54:40,599][1157819] Updated weights for policy 0, policy_version 90448 (0.0006) [2024-09-30 01:54:41,135][1157819] Updated weights for policy 0, policy_version 90458 (0.0006) [2024-09-30 01:54:41,682][1157819] Updated weights for policy 0, policy_version 90468 (0.0006) [2024-09-30 01:54:42,179][1157819] Updated weights for policy 0, policy_version 90478 (0.0006) [2024-09-30 01:54:42,677][1157819] Updated weights for policy 0, policy_version 90488 (0.0006) [2024-09-30 01:54:43,239][1157819] Updated weights for policy 0, policy_version 90498 (0.0006) [2024-09-30 01:54:43,757][1157819] Updated weights for policy 0, policy_version 90508 (0.0006) [2024-09-30 01:54:44,267][1157819] Updated weights for policy 0, policy_version 90518 (0.0006) [2024-09-30 01:54:44,824][1157819] Updated weights for policy 0, policy_version 90528 (0.0006) [2024-09-30 01:54:45,366][1157819] Updated weights for policy 0, policy_version 90538 (0.0006) [2024-09-30 01:54:45,466][1157520] Fps is (10 sec: 78643.1, 60 sec: 77073.0, 300 sec: 77837.9). Total num frames: 370855936. Throughput: 0: 19243.2. Samples: 82680572. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:54:45,466][1157520] Avg episode reward: [(0, '52.898')] [2024-09-30 01:54:45,826][1157819] Updated weights for policy 0, policy_version 90548 (0.0006) [2024-09-30 01:54:46,329][1157819] Updated weights for policy 0, policy_version 90558 (0.0006) [2024-09-30 01:54:46,826][1157819] Updated weights for policy 0, policy_version 90568 (0.0006) [2024-09-30 01:54:47,336][1157819] Updated weights for policy 0, policy_version 90578 (0.0006) [2024-09-30 01:54:47,810][1157819] Updated weights for policy 0, policy_version 90588 (0.0006) [2024-09-30 01:54:48,300][1157819] Updated weights for policy 0, policy_version 90598 (0.0006) [2024-09-30 01:54:48,834][1157819] Updated weights for policy 0, policy_version 90608 (0.0006) [2024-09-30 01:54:49,301][1157819] Updated weights for policy 0, policy_version 90618 (0.0006) [2024-09-30 01:54:49,785][1157819] Updated weights for policy 0, policy_version 90628 (0.0006) [2024-09-30 01:54:50,296][1157819] Updated weights for policy 0, policy_version 90638 (0.0006) [2024-09-30 01:54:50,466][1157520] Fps is (10 sec: 80281.8, 60 sec: 77346.1, 300 sec: 77879.5). Total num frames: 371265536. Throughput: 0: 19355.7. Samples: 82802356. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:54:50,466][1157520] Avg episode reward: [(0, '55.256')] [2024-09-30 01:54:50,610][1157736] Signal inference workers to stop experience collection... (5850 times) [2024-09-30 01:54:50,613][1157736] Signal inference workers to resume experience collection... (5850 times) [2024-09-30 01:54:50,617][1157819] InferenceWorker_p0-w0: stopping experience collection (5850 times) [2024-09-30 01:54:50,619][1157819] InferenceWorker_p0-w0: resuming experience collection (5850 times) [2024-09-30 01:54:50,797][1157819] Updated weights for policy 0, policy_version 90648 (0.0006) [2024-09-30 01:54:51,299][1157819] Updated weights for policy 0, policy_version 90658 (0.0006) [2024-09-30 01:54:51,775][1157819] Updated weights for policy 0, policy_version 90668 (0.0006) [2024-09-30 01:54:52,286][1157819] Updated weights for policy 0, policy_version 90678 (0.0006) [2024-09-30 01:54:52,776][1157819] Updated weights for policy 0, policy_version 90688 (0.0006) [2024-09-30 01:54:53,280][1157819] Updated weights for policy 0, policy_version 90698 (0.0006) [2024-09-30 01:54:53,755][1157819] Updated weights for policy 0, policy_version 90708 (0.0006) [2024-09-30 01:54:54,249][1157819] Updated weights for policy 0, policy_version 90718 (0.0006) [2024-09-30 01:54:54,704][1157819] Updated weights for policy 0, policy_version 90728 (0.0006) [2024-09-30 01:54:55,192][1157819] Updated weights for policy 0, policy_version 90738 (0.0006) [2024-09-30 01:54:55,466][1157520] Fps is (10 sec: 82739.7, 60 sec: 77960.6, 300 sec: 77949.0). Total num frames: 371683328. Throughput: 0: 19446.1. Samples: 82864840. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 01:54:55,466][1157520] Avg episode reward: [(0, '54.735')] [2024-09-30 01:54:55,696][1157819] Updated weights for policy 0, policy_version 90748 (0.0006) [2024-09-30 01:54:56,170][1157819] Updated weights for policy 0, policy_version 90758 (0.0006) [2024-09-30 01:54:56,624][1157819] Updated weights for policy 0, policy_version 90768 (0.0006) [2024-09-30 01:54:57,126][1157819] Updated weights for policy 0, policy_version 90778 (0.0006) [2024-09-30 01:54:57,606][1157819] Updated weights for policy 0, policy_version 90788 (0.0006) [2024-09-30 01:54:58,096][1157819] Updated weights for policy 0, policy_version 90798 (0.0006) [2024-09-30 01:54:58,589][1157819] Updated weights for policy 0, policy_version 90808 (0.0006) [2024-09-30 01:54:59,086][1157819] Updated weights for policy 0, policy_version 90818 (0.0006) [2024-09-30 01:54:59,566][1157819] Updated weights for policy 0, policy_version 90828 (0.0006) [2024-09-30 01:55:00,054][1157819] Updated weights for policy 0, policy_version 90838 (0.0006) [2024-09-30 01:55:00,466][1157520] Fps is (10 sec: 83968.2, 60 sec: 78643.3, 300 sec: 78018.3). Total num frames: 372105216. Throughput: 0: 19801.5. Samples: 82991264. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 01:55:00,466][1157520] Avg episode reward: [(0, '54.982')] [2024-09-30 01:55:00,549][1157819] Updated weights for policy 0, policy_version 90848 (0.0006) [2024-09-30 01:55:01,043][1157819] Updated weights for policy 0, policy_version 90858 (0.0006) [2024-09-30 01:55:01,539][1157819] Updated weights for policy 0, policy_version 90868 (0.0006) [2024-09-30 01:55:02,029][1157819] Updated weights for policy 0, policy_version 90878 (0.0006) [2024-09-30 01:55:02,529][1157819] Updated weights for policy 0, policy_version 90888 (0.0006) [2024-09-30 01:55:03,014][1157819] Updated weights for policy 0, policy_version 90898 (0.0006) [2024-09-30 01:55:03,463][1157819] Updated weights for policy 0, policy_version 90908 (0.0006) [2024-09-30 01:55:03,945][1157819] Updated weights for policy 0, policy_version 90918 (0.0006) [2024-09-30 01:55:04,465][1157819] Updated weights for policy 0, policy_version 90928 (0.0006) [2024-09-30 01:55:04,925][1157819] Updated weights for policy 0, policy_version 90938 (0.0006) [2024-09-30 01:55:05,414][1157819] Updated weights for policy 0, policy_version 90948 (0.0006) [2024-09-30 01:55:05,466][1157520] Fps is (10 sec: 83967.1, 60 sec: 79121.1, 300 sec: 78060.0). Total num frames: 372523008. Throughput: 0: 20107.2. Samples: 83117420. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 01:55:05,466][1157520] Avg episode reward: [(0, '50.365')] [2024-09-30 01:55:05,891][1157819] Updated weights for policy 0, policy_version 90958 (0.0006) [2024-09-30 01:55:06,393][1157819] Updated weights for policy 0, policy_version 90968 (0.0006) [2024-09-30 01:55:06,905][1157819] Updated weights for policy 0, policy_version 90978 (0.0006) [2024-09-30 01:55:07,390][1157819] Updated weights for policy 0, policy_version 90988 (0.0006) [2024-09-30 01:55:07,888][1157819] Updated weights for policy 0, policy_version 90998 (0.0006) [2024-09-30 01:55:08,386][1157819] Updated weights for policy 0, policy_version 91008 (0.0006) [2024-09-30 01:55:08,890][1157819] Updated weights for policy 0, policy_version 91018 (0.0006) [2024-09-30 01:55:09,394][1157819] Updated weights for policy 0, policy_version 91028 (0.0006) [2024-09-30 01:55:09,861][1157819] Updated weights for policy 0, policy_version 91038 (0.0006) [2024-09-30 01:55:10,364][1157819] Updated weights for policy 0, policy_version 91048 (0.0006) [2024-09-30 01:55:10,466][1157520] Fps is (10 sec: 83147.1, 60 sec: 79666.9, 300 sec: 78129.4). Total num frames: 372936704. Throughput: 0: 20191.7. Samples: 83179720. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 01:55:10,466][1157520] Avg episode reward: [(0, '50.456')] [2024-09-30 01:55:10,861][1157819] Updated weights for policy 0, policy_version 91058 (0.0006) [2024-09-30 01:55:11,356][1157819] Updated weights for policy 0, policy_version 91068 (0.0006) [2024-09-30 01:55:11,845][1157819] Updated weights for policy 0, policy_version 91078 (0.0006) [2024-09-30 01:55:12,334][1157819] Updated weights for policy 0, policy_version 91088 (0.0006) [2024-09-30 01:55:12,832][1157819] Updated weights for policy 0, policy_version 91098 (0.0006) [2024-09-30 01:55:13,337][1157819] Updated weights for policy 0, policy_version 91108 (0.0006) [2024-09-30 01:55:13,841][1157819] Updated weights for policy 0, policy_version 91118 (0.0006) [2024-09-30 01:55:14,343][1157819] Updated weights for policy 0, policy_version 91128 (0.0006) [2024-09-30 01:55:14,856][1157819] Updated weights for policy 0, policy_version 91138 (0.0006) [2024-09-30 01:55:15,363][1157819] Updated weights for policy 0, policy_version 91148 (0.0006) [2024-09-30 01:55:15,466][1157520] Fps is (10 sec: 82330.7, 60 sec: 80486.6, 300 sec: 78129.4). Total num frames: 373346304. Throughput: 0: 20347.2. Samples: 83303728. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 01:55:15,466][1157520] Avg episode reward: [(0, '54.774')] [2024-09-30 01:55:15,893][1157819] Updated weights for policy 0, policy_version 91158 (0.0006) [2024-09-30 01:55:16,442][1157819] Updated weights for policy 0, policy_version 91168 (0.0006) [2024-09-30 01:55:16,966][1157819] Updated weights for policy 0, policy_version 91178 (0.0006) [2024-09-30 01:55:17,518][1157819] Updated weights for policy 0, policy_version 91188 (0.0006) [2024-09-30 01:55:18,066][1157819] Updated weights for policy 0, policy_version 91198 (0.0006) [2024-09-30 01:55:18,595][1157819] Updated weights for policy 0, policy_version 91208 (0.0006) [2024-09-30 01:55:19,150][1157819] Updated weights for policy 0, policy_version 91218 (0.0006) [2024-09-30 01:55:19,704][1157819] Updated weights for policy 0, policy_version 91228 (0.0006) [2024-09-30 01:55:20,197][1157819] Updated weights for policy 0, policy_version 91238 (0.0006) [2024-09-30 01:55:20,466][1157520] Fps is (10 sec: 79054.0, 60 sec: 80418.1, 300 sec: 78032.2). Total num frames: 373727232. Throughput: 0: 20333.3. Samples: 83419740. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 01:55:20,466][1157520] Avg episode reward: [(0, '53.754')] [2024-09-30 01:55:20,719][1157819] Updated weights for policy 0, policy_version 91248 (0.0006) [2024-09-30 01:55:21,263][1157819] Updated weights for policy 0, policy_version 91258 (0.0006) [2024-09-30 01:55:21,760][1157819] Updated weights for policy 0, policy_version 91268 (0.0006) [2024-09-30 01:55:22,300][1157819] Updated weights for policy 0, policy_version 91278 (0.0006) [2024-09-30 01:55:22,833][1157819] Updated weights for policy 0, policy_version 91288 (0.0006) [2024-09-30 01:55:23,352][1157819] Updated weights for policy 0, policy_version 91298 (0.0006) [2024-09-30 01:55:23,889][1157819] Updated weights for policy 0, policy_version 91308 (0.0006) [2024-09-30 01:55:24,403][1157819] Updated weights for policy 0, policy_version 91318 (0.0006) [2024-09-30 01:55:24,929][1157819] Updated weights for policy 0, policy_version 91328 (0.0006) [2024-09-30 01:55:25,466][1157520] Fps is (10 sec: 77004.2, 60 sec: 80418.1, 300 sec: 77990.6). Total num frames: 374116352. Throughput: 0: 20322.1. Samples: 83478092. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 01:55:25,466][1157520] Avg episode reward: [(0, '54.653')] [2024-09-30 01:55:25,480][1157819] Updated weights for policy 0, policy_version 91338 (0.0006) [2024-09-30 01:55:25,990][1157819] Updated weights for policy 0, policy_version 91348 (0.0006) [2024-09-30 01:55:26,536][1157819] Updated weights for policy 0, policy_version 91358 (0.0006) [2024-09-30 01:55:27,053][1157819] Updated weights for policy 0, policy_version 91368 (0.0006) [2024-09-30 01:55:27,599][1157819] Updated weights for policy 0, policy_version 91378 (0.0006) [2024-09-30 01:55:28,110][1157819] Updated weights for policy 0, policy_version 91388 (0.0006) [2024-09-30 01:55:28,625][1157819] Updated weights for policy 0, policy_version 91398 (0.0006) [2024-09-30 01:55:29,151][1157819] Updated weights for policy 0, policy_version 91408 (0.0006) [2024-09-30 01:55:29,675][1157819] Updated weights for policy 0, policy_version 91418 (0.0006) [2024-09-30 01:55:30,225][1157819] Updated weights for policy 0, policy_version 91428 (0.0006) [2024-09-30 01:55:30,466][1157520] Fps is (10 sec: 77824.2, 60 sec: 80418.1, 300 sec: 77948.9). Total num frames: 374505472. Throughput: 0: 20306.2. Samples: 83594352. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 01:55:30,466][1157520] Avg episode reward: [(0, '52.775')] [2024-09-30 01:55:30,750][1157819] Updated weights for policy 0, policy_version 91438 (0.0006) [2024-09-30 01:55:31,279][1157819] Updated weights for policy 0, policy_version 91448 (0.0006) [2024-09-30 01:55:31,796][1157819] Updated weights for policy 0, policy_version 91458 (0.0006) [2024-09-30 01:55:32,315][1157819] Updated weights for policy 0, policy_version 91468 (0.0006) [2024-09-30 01:55:32,888][1157819] Updated weights for policy 0, policy_version 91478 (0.0006) [2024-09-30 01:55:33,420][1157819] Updated weights for policy 0, policy_version 91488 (0.0006) [2024-09-30 01:55:33,919][1157819] Updated weights for policy 0, policy_version 91498 (0.0006) [2024-09-30 01:55:34,449][1157819] Updated weights for policy 0, policy_version 91508 (0.0006) [2024-09-30 01:55:34,984][1157819] Updated weights for policy 0, policy_version 91518 (0.0006) [2024-09-30 01:55:35,466][1157520] Fps is (10 sec: 77414.9, 60 sec: 80350.0, 300 sec: 77990.6). Total num frames: 374890496. Throughput: 0: 20178.7. Samples: 83710396. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 01:55:35,466][1157520] Avg episode reward: [(0, '54.025')] [2024-09-30 01:55:35,529][1157819] Updated weights for policy 0, policy_version 91528 (0.0007) [2024-09-30 01:55:36,055][1157819] Updated weights for policy 0, policy_version 91538 (0.0007) [2024-09-30 01:55:36,587][1157819] Updated weights for policy 0, policy_version 91548 (0.0007) [2024-09-30 01:55:37,101][1157819] Updated weights for policy 0, policy_version 91558 (0.0007) [2024-09-30 01:55:37,620][1157819] Updated weights for policy 0, policy_version 91568 (0.0007) [2024-09-30 01:55:38,177][1157819] Updated weights for policy 0, policy_version 91578 (0.0007) [2024-09-30 01:55:38,715][1157819] Updated weights for policy 0, policy_version 91588 (0.0006) [2024-09-30 01:55:39,294][1157819] Updated weights for policy 0, policy_version 91598 (0.0007) [2024-09-30 01:55:39,844][1157819] Updated weights for policy 0, policy_version 91608 (0.0007) [2024-09-30 01:55:40,154][1157736] Signal inference workers to stop experience collection... (5900 times) [2024-09-30 01:55:40,154][1157736] Signal inference workers to resume experience collection... (5900 times) [2024-09-30 01:55:40,160][1157819] InferenceWorker_p0-w0: stopping experience collection (5900 times) [2024-09-30 01:55:40,160][1157819] InferenceWorker_p0-w0: resuming experience collection (5900 times) [2024-09-30 01:55:40,415][1157819] Updated weights for policy 0, policy_version 91618 (0.0006) [2024-09-30 01:55:40,466][1157520] Fps is (10 sec: 76595.4, 60 sec: 80145.1, 300 sec: 78004.5). Total num frames: 375271424. Throughput: 0: 20077.3. Samples: 83768320. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 01:55:40,466][1157520] Avg episode reward: [(0, '50.720')] [2024-09-30 01:55:40,941][1157819] Updated weights for policy 0, policy_version 91628 (0.0006) [2024-09-30 01:55:41,445][1157819] Updated weights for policy 0, policy_version 91638 (0.0006) [2024-09-30 01:55:41,959][1157819] Updated weights for policy 0, policy_version 91648 (0.0006) [2024-09-30 01:55:42,488][1157819] Updated weights for policy 0, policy_version 91658 (0.0006) [2024-09-30 01:55:43,009][1157819] Updated weights for policy 0, policy_version 91668 (0.0006) [2024-09-30 01:55:43,541][1157819] Updated weights for policy 0, policy_version 91678 (0.0006) [2024-09-30 01:55:44,049][1157819] Updated weights for policy 0, policy_version 91688 (0.0006) [2024-09-30 01:55:44,562][1157819] Updated weights for policy 0, policy_version 91698 (0.0006) [2024-09-30 01:55:45,080][1157819] Updated weights for policy 0, policy_version 91708 (0.0006) [2024-09-30 01:55:45,466][1157520] Fps is (10 sec: 77414.3, 60 sec: 80145.2, 300 sec: 78060.0). Total num frames: 375664640. Throughput: 0: 19822.9. Samples: 83883292. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 01:55:45,466][1157520] Avg episode reward: [(0, '51.306')] [2024-09-30 01:55:45,572][1157819] Updated weights for policy 0, policy_version 91718 (0.0006) [2024-09-30 01:55:46,091][1157819] Updated weights for policy 0, policy_version 91728 (0.0006) [2024-09-30 01:55:46,594][1157819] Updated weights for policy 0, policy_version 91738 (0.0006) [2024-09-30 01:55:47,120][1157819] Updated weights for policy 0, policy_version 91748 (0.0006) [2024-09-30 01:55:47,664][1157819] Updated weights for policy 0, policy_version 91758 (0.0006) [2024-09-30 01:55:48,175][1157819] Updated weights for policy 0, policy_version 91768 (0.0006) [2024-09-30 01:55:48,689][1157819] Updated weights for policy 0, policy_version 91778 (0.0006) [2024-09-30 01:55:49,190][1157819] Updated weights for policy 0, policy_version 91788 (0.0006) [2024-09-30 01:55:49,729][1157819] Updated weights for policy 0, policy_version 91798 (0.0006) [2024-09-30 01:55:50,264][1157819] Updated weights for policy 0, policy_version 91808 (0.0006) [2024-09-30 01:55:50,466][1157520] Fps is (10 sec: 78643.5, 60 sec: 79872.1, 300 sec: 78115.6). Total num frames: 376057856. Throughput: 0: 19657.8. Samples: 84002016. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:55:50,466][1157520] Avg episode reward: [(0, '52.993')] [2024-09-30 01:55:50,801][1157819] Updated weights for policy 0, policy_version 91818 (0.0006) [2024-09-30 01:55:51,293][1157819] Updated weights for policy 0, policy_version 91828 (0.0006) [2024-09-30 01:55:51,815][1157819] Updated weights for policy 0, policy_version 91838 (0.0006) [2024-09-30 01:55:52,321][1157819] Updated weights for policy 0, policy_version 91848 (0.0006) [2024-09-30 01:55:52,825][1157819] Updated weights for policy 0, policy_version 91858 (0.0006) [2024-09-30 01:55:53,326][1157819] Updated weights for policy 0, policy_version 91868 (0.0006) [2024-09-30 01:55:53,871][1157819] Updated weights for policy 0, policy_version 91878 (0.0006) [2024-09-30 01:55:54,412][1157819] Updated weights for policy 0, policy_version 91888 (0.0006) [2024-09-30 01:55:54,937][1157819] Updated weights for policy 0, policy_version 91898 (0.0006) [2024-09-30 01:55:55,452][1157819] Updated weights for policy 0, policy_version 91908 (0.0006) [2024-09-30 01:55:55,466][1157520] Fps is (10 sec: 79052.3, 60 sec: 79530.6, 300 sec: 78198.9). Total num frames: 376455168. Throughput: 0: 19593.6. Samples: 84061428. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:55:55,466][1157520] Avg episode reward: [(0, '54.961')] [2024-09-30 01:55:55,948][1157819] Updated weights for policy 0, policy_version 91918 (0.0006) [2024-09-30 01:55:56,456][1157819] Updated weights for policy 0, policy_version 91928 (0.0006) [2024-09-30 01:55:56,997][1157819] Updated weights for policy 0, policy_version 91938 (0.0006) [2024-09-30 01:55:57,512][1157819] Updated weights for policy 0, policy_version 91948 (0.0006) [2024-09-30 01:55:58,011][1157819] Updated weights for policy 0, policy_version 91958 (0.0006) [2024-09-30 01:55:58,508][1157819] Updated weights for policy 0, policy_version 91968 (0.0006) [2024-09-30 01:55:59,051][1157819] Updated weights for policy 0, policy_version 91978 (0.0006) [2024-09-30 01:55:59,573][1157819] Updated weights for policy 0, policy_version 91988 (0.0006) [2024-09-30 01:56:00,084][1157819] Updated weights for policy 0, policy_version 91998 (0.0006) [2024-09-30 01:56:00,466][1157520] Fps is (10 sec: 79462.1, 60 sec: 79121.1, 300 sec: 78268.3). Total num frames: 376852480. Throughput: 0: 19479.9. Samples: 84180324. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:56:00,466][1157520] Avg episode reward: [(0, '51.990')] [2024-09-30 01:56:00,605][1157819] Updated weights for policy 0, policy_version 92008 (0.0006) [2024-09-30 01:56:01,114][1157819] Updated weights for policy 0, policy_version 92018 (0.0006) [2024-09-30 01:56:01,620][1157819] Updated weights for policy 0, policy_version 92028 (0.0006) [2024-09-30 01:56:02,147][1157819] Updated weights for policy 0, policy_version 92038 (0.0006) [2024-09-30 01:56:02,665][1157819] Updated weights for policy 0, policy_version 92048 (0.0006) [2024-09-30 01:56:03,159][1157819] Updated weights for policy 0, policy_version 92058 (0.0006) [2024-09-30 01:56:03,679][1157819] Updated weights for policy 0, policy_version 92068 (0.0006) [2024-09-30 01:56:04,223][1157819] Updated weights for policy 0, policy_version 92078 (0.0006) [2024-09-30 01:56:04,718][1157819] Updated weights for policy 0, policy_version 92088 (0.0006) [2024-09-30 01:56:05,242][1157819] Updated weights for policy 0, policy_version 92098 (0.0006) [2024-09-30 01:56:05,466][1157520] Fps is (10 sec: 79462.2, 60 sec: 78779.8, 300 sec: 78296.0). Total num frames: 377249792. Throughput: 0: 19547.2. Samples: 84299364. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:56:05,466][1157520] Avg episode reward: [(0, '55.860')] [2024-09-30 01:56:05,475][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000092102_377249792.pth... [2024-09-30 01:56:05,523][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000087528_358514688.pth [2024-09-30 01:56:05,757][1157819] Updated weights for policy 0, policy_version 92108 (0.0006) [2024-09-30 01:56:06,284][1157819] Updated weights for policy 0, policy_version 92118 (0.0006) [2024-09-30 01:56:06,792][1157819] Updated weights for policy 0, policy_version 92128 (0.0006) [2024-09-30 01:56:07,291][1157819] Updated weights for policy 0, policy_version 92138 (0.0006) [2024-09-30 01:56:07,806][1157819] Updated weights for policy 0, policy_version 92148 (0.0006) [2024-09-30 01:56:08,363][1157819] Updated weights for policy 0, policy_version 92158 (0.0006) [2024-09-30 01:56:08,918][1157819] Updated weights for policy 0, policy_version 92168 (0.0006) [2024-09-30 01:56:09,417][1157819] Updated weights for policy 0, policy_version 92178 (0.0006) [2024-09-30 01:56:09,911][1157819] Updated weights for policy 0, policy_version 92188 (0.0006) [2024-09-30 01:56:10,418][1157819] Updated weights for policy 0, policy_version 92198 (0.0006) [2024-09-30 01:56:10,466][1157520] Fps is (10 sec: 79462.1, 60 sec: 78506.9, 300 sec: 78323.9). Total num frames: 377647104. Throughput: 0: 19562.9. Samples: 84358424. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:56:10,466][1157520] Avg episode reward: [(0, '53.444')] [2024-09-30 01:56:10,942][1157819] Updated weights for policy 0, policy_version 92208 (0.0006) [2024-09-30 01:56:11,459][1157819] Updated weights for policy 0, policy_version 92218 (0.0006) [2024-09-30 01:56:11,968][1157819] Updated weights for policy 0, policy_version 92228 (0.0006) [2024-09-30 01:56:12,484][1157819] Updated weights for policy 0, policy_version 92238 (0.0006) [2024-09-30 01:56:13,014][1157819] Updated weights for policy 0, policy_version 92248 (0.0006) [2024-09-30 01:56:13,551][1157819] Updated weights for policy 0, policy_version 92258 (0.0006) [2024-09-30 01:56:14,071][1157819] Updated weights for policy 0, policy_version 92268 (0.0006) [2024-09-30 01:56:14,584][1157819] Updated weights for policy 0, policy_version 92278 (0.0006) [2024-09-30 01:56:15,084][1157819] Updated weights for policy 0, policy_version 92288 (0.0006) [2024-09-30 01:56:15,466][1157520] Fps is (10 sec: 79053.6, 60 sec: 78233.6, 300 sec: 78393.3). Total num frames: 378040320. Throughput: 0: 19612.5. Samples: 84476912. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:56:15,466][1157520] Avg episode reward: [(0, '54.451')] [2024-09-30 01:56:15,589][1157819] Updated weights for policy 0, policy_version 92298 (0.0006) [2024-09-30 01:56:16,124][1157819] Updated weights for policy 0, policy_version 92308 (0.0007) [2024-09-30 01:56:16,660][1157819] Updated weights for policy 0, policy_version 92318 (0.0006) [2024-09-30 01:56:17,195][1157819] Updated weights for policy 0, policy_version 92328 (0.0006) [2024-09-30 01:56:17,709][1157819] Updated weights for policy 0, policy_version 92338 (0.0006) [2024-09-30 01:56:18,209][1157819] Updated weights for policy 0, policy_version 92348 (0.0006) [2024-09-30 01:56:18,701][1157819] Updated weights for policy 0, policy_version 92358 (0.0006) [2024-09-30 01:56:19,265][1157819] Updated weights for policy 0, policy_version 92368 (0.0006) [2024-09-30 01:56:19,778][1157819] Updated weights for policy 0, policy_version 92378 (0.0006) [2024-09-30 01:56:20,278][1157819] Updated weights for policy 0, policy_version 92388 (0.0006) [2024-09-30 01:56:20,466][1157520] Fps is (10 sec: 78643.6, 60 sec: 78438.5, 300 sec: 78448.8). Total num frames: 378433536. Throughput: 0: 19666.7. Samples: 84595396. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:56:20,466][1157520] Avg episode reward: [(0, '55.025')] [2024-09-30 01:56:20,809][1157819] Updated weights for policy 0, policy_version 92398 (0.0006) [2024-09-30 01:56:21,319][1157819] Updated weights for policy 0, policy_version 92408 (0.0006) [2024-09-30 01:56:21,817][1157819] Updated weights for policy 0, policy_version 92418 (0.0006) [2024-09-30 01:56:22,326][1157819] Updated weights for policy 0, policy_version 92428 (0.0006) [2024-09-30 01:56:22,839][1157819] Updated weights for policy 0, policy_version 92438 (0.0006) [2024-09-30 01:56:23,350][1157819] Updated weights for policy 0, policy_version 92448 (0.0006) [2024-09-30 01:56:23,889][1157819] Updated weights for policy 0, policy_version 92458 (0.0006) [2024-09-30 01:56:24,377][1157819] Updated weights for policy 0, policy_version 92468 (0.0006) [2024-09-30 01:56:24,899][1157819] Updated weights for policy 0, policy_version 92478 (0.0006) [2024-09-30 01:56:25,404][1157819] Updated weights for policy 0, policy_version 92488 (0.0006) [2024-09-30 01:56:25,466][1157520] Fps is (10 sec: 79462.5, 60 sec: 78643.3, 300 sec: 78518.2). Total num frames: 378834944. Throughput: 0: 19709.4. Samples: 84655244. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:56:25,466][1157520] Avg episode reward: [(0, '54.658')] [2024-09-30 01:56:25,921][1157819] Updated weights for policy 0, policy_version 92498 (0.0006) [2024-09-30 01:56:26,437][1157819] Updated weights for policy 0, policy_version 92508 (0.0006) [2024-09-30 01:56:26,974][1157819] Updated weights for policy 0, policy_version 92518 (0.0006) [2024-09-30 01:56:27,454][1157819] Updated weights for policy 0, policy_version 92528 (0.0006) [2024-09-30 01:56:27,954][1157819] Updated weights for policy 0, policy_version 92538 (0.0006) [2024-09-30 01:56:28,470][1157819] Updated weights for policy 0, policy_version 92548 (0.0006) [2024-09-30 01:56:28,976][1157819] Updated weights for policy 0, policy_version 92558 (0.0006) [2024-09-30 01:56:29,475][1157819] Updated weights for policy 0, policy_version 92568 (0.0006) [2024-09-30 01:56:29,955][1157819] Updated weights for policy 0, policy_version 92578 (0.0006) [2024-09-30 01:56:30,441][1157819] Updated weights for policy 0, policy_version 92588 (0.0006) [2024-09-30 01:56:30,466][1157520] Fps is (10 sec: 80692.4, 60 sec: 78916.5, 300 sec: 78615.5). Total num frames: 379240448. Throughput: 0: 19821.3. Samples: 84775248. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:56:30,466][1157520] Avg episode reward: [(0, '54.348')] [2024-09-30 01:56:30,938][1157819] Updated weights for policy 0, policy_version 92598 (0.0006) [2024-09-30 01:56:31,418][1157819] Updated weights for policy 0, policy_version 92608 (0.0006) [2024-09-30 01:56:31,919][1157819] Updated weights for policy 0, policy_version 92618 (0.0006) [2024-09-30 01:56:32,090][1157736] Signal inference workers to stop experience collection... (5950 times) [2024-09-30 01:56:32,090][1157736] Signal inference workers to resume experience collection... (5950 times) [2024-09-30 01:56:32,093][1157819] InferenceWorker_p0-w0: stopping experience collection (5950 times) [2024-09-30 01:56:32,094][1157819] InferenceWorker_p0-w0: resuming experience collection (5950 times) [2024-09-30 01:56:32,411][1157819] Updated weights for policy 0, policy_version 92628 (0.0006) [2024-09-30 01:56:32,910][1157819] Updated weights for policy 0, policy_version 92638 (0.0006) [2024-09-30 01:56:33,406][1157819] Updated weights for policy 0, policy_version 92648 (0.0006) [2024-09-30 01:56:33,931][1157819] Updated weights for policy 0, policy_version 92658 (0.0007) [2024-09-30 01:56:34,427][1157819] Updated weights for policy 0, policy_version 92668 (0.0006) [2024-09-30 01:56:34,925][1157819] Updated weights for policy 0, policy_version 92678 (0.0006) [2024-09-30 01:56:35,436][1157819] Updated weights for policy 0, policy_version 92688 (0.0006) [2024-09-30 01:56:35,466][1157520] Fps is (10 sec: 81511.2, 60 sec: 79326.0, 300 sec: 78671.0). Total num frames: 379650048. Throughput: 0: 19942.4. Samples: 84899424. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:56:35,466][1157520] Avg episode reward: [(0, '54.751')] [2024-09-30 01:56:35,942][1157819] Updated weights for policy 0, policy_version 92698 (0.0006) [2024-09-30 01:56:36,443][1157819] Updated weights for policy 0, policy_version 92708 (0.0006) [2024-09-30 01:56:36,954][1157819] Updated weights for policy 0, policy_version 92718 (0.0006) [2024-09-30 01:56:37,457][1157819] Updated weights for policy 0, policy_version 92728 (0.0006) [2024-09-30 01:56:37,960][1157819] Updated weights for policy 0, policy_version 92738 (0.0006) [2024-09-30 01:56:38,436][1157819] Updated weights for policy 0, policy_version 92748 (0.0006) [2024-09-30 01:56:38,920][1157819] Updated weights for policy 0, policy_version 92758 (0.0006) [2024-09-30 01:56:39,384][1157819] Updated weights for policy 0, policy_version 92768 (0.0006) [2024-09-30 01:56:39,896][1157819] Updated weights for policy 0, policy_version 92778 (0.0006) [2024-09-30 01:56:40,389][1157819] Updated weights for policy 0, policy_version 92788 (0.0006) [2024-09-30 01:56:40,466][1157520] Fps is (10 sec: 82329.4, 60 sec: 79872.2, 300 sec: 78726.5). Total num frames: 380063744. Throughput: 0: 19979.9. Samples: 84960520. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 01:56:40,466][1157520] Avg episode reward: [(0, '52.165')] [2024-09-30 01:56:40,849][1157819] Updated weights for policy 0, policy_version 92798 (0.0006) [2024-09-30 01:56:41,344][1157819] Updated weights for policy 0, policy_version 92808 (0.0006) [2024-09-30 01:56:41,851][1157819] Updated weights for policy 0, policy_version 92818 (0.0006) [2024-09-30 01:56:42,347][1157819] Updated weights for policy 0, policy_version 92828 (0.0006) [2024-09-30 01:56:42,843][1157819] Updated weights for policy 0, policy_version 92838 (0.0006) [2024-09-30 01:56:43,376][1157819] Updated weights for policy 0, policy_version 92848 (0.0006) [2024-09-30 01:56:43,877][1157819] Updated weights for policy 0, policy_version 92858 (0.0006) [2024-09-30 01:56:44,370][1157819] Updated weights for policy 0, policy_version 92868 (0.0006) [2024-09-30 01:56:44,877][1157819] Updated weights for policy 0, policy_version 92878 (0.0006) [2024-09-30 01:56:45,385][1157819] Updated weights for policy 0, policy_version 92888 (0.0006) [2024-09-30 01:56:45,466][1157520] Fps is (10 sec: 82328.8, 60 sec: 80145.1, 300 sec: 78698.7). Total num frames: 380473344. Throughput: 0: 20103.8. Samples: 85084996. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 01:56:45,466][1157520] Avg episode reward: [(0, '53.418')] [2024-09-30 01:56:45,918][1157819] Updated weights for policy 0, policy_version 92898 (0.0006) [2024-09-30 01:56:46,410][1157819] Updated weights for policy 0, policy_version 92908 (0.0006) [2024-09-30 01:56:46,930][1157819] Updated weights for policy 0, policy_version 92918 (0.0006) [2024-09-30 01:56:47,428][1157819] Updated weights for policy 0, policy_version 92928 (0.0006) [2024-09-30 01:56:47,925][1157819] Updated weights for policy 0, policy_version 92938 (0.0006) [2024-09-30 01:56:48,443][1157819] Updated weights for policy 0, policy_version 92948 (0.0006) [2024-09-30 01:56:48,960][1157819] Updated weights for policy 0, policy_version 92958 (0.0006) [2024-09-30 01:56:49,468][1157819] Updated weights for policy 0, policy_version 92968 (0.0006) [2024-09-30 01:56:49,982][1157819] Updated weights for policy 0, policy_version 92978 (0.0006) [2024-09-30 01:56:50,466][1157520] Fps is (10 sec: 81100.2, 60 sec: 80281.6, 300 sec: 78629.3). Total num frames: 380874752. Throughput: 0: 20140.4. Samples: 85205680. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 01:56:50,466][1157520] Avg episode reward: [(0, '54.610')] [2024-09-30 01:56:50,495][1157819] Updated weights for policy 0, policy_version 92988 (0.0006) [2024-09-30 01:56:51,003][1157819] Updated weights for policy 0, policy_version 92998 (0.0006) [2024-09-30 01:56:51,531][1157819] Updated weights for policy 0, policy_version 93008 (0.0006) [2024-09-30 01:56:52,032][1157819] Updated weights for policy 0, policy_version 93018 (0.0006) [2024-09-30 01:56:52,593][1157819] Updated weights for policy 0, policy_version 93028 (0.0006) [2024-09-30 01:56:53,165][1157819] Updated weights for policy 0, policy_version 93038 (0.0006) [2024-09-30 01:56:53,670][1157819] Updated weights for policy 0, policy_version 93048 (0.0006) [2024-09-30 01:56:54,193][1157819] Updated weights for policy 0, policy_version 93058 (0.0006) [2024-09-30 01:56:54,702][1157819] Updated weights for policy 0, policy_version 93068 (0.0006) [2024-09-30 01:56:55,230][1157819] Updated weights for policy 0, policy_version 93078 (0.0006) [2024-09-30 01:56:55,466][1157520] Fps is (10 sec: 78643.2, 60 sec: 80076.9, 300 sec: 78518.2). Total num frames: 381259776. Throughput: 0: 20136.0. Samples: 85264544. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 01:56:55,466][1157520] Avg episode reward: [(0, '52.603')] [2024-09-30 01:56:55,787][1157819] Updated weights for policy 0, policy_version 93088 (0.0006) [2024-09-30 01:56:56,339][1157819] Updated weights for policy 0, policy_version 93098 (0.0006) [2024-09-30 01:56:56,851][1157819] Updated weights for policy 0, policy_version 93108 (0.0006) [2024-09-30 01:56:57,347][1157819] Updated weights for policy 0, policy_version 93118 (0.0006) [2024-09-30 01:56:57,852][1157819] Updated weights for policy 0, policy_version 93128 (0.0006) [2024-09-30 01:56:58,407][1157819] Updated weights for policy 0, policy_version 93138 (0.0006) [2024-09-30 01:56:58,953][1157819] Updated weights for policy 0, policy_version 93148 (0.0006) [2024-09-30 01:56:59,448][1157819] Updated weights for policy 0, policy_version 93158 (0.0006) [2024-09-30 01:56:59,941][1157819] Updated weights for policy 0, policy_version 93168 (0.0006) [2024-09-30 01:57:00,445][1157819] Updated weights for policy 0, policy_version 93178 (0.0006) [2024-09-30 01:57:00,466][1157520] Fps is (10 sec: 78233.4, 60 sec: 80076.8, 300 sec: 78462.7). Total num frames: 381657088. Throughput: 0: 20088.8. Samples: 85380908. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 01:57:00,466][1157520] Avg episode reward: [(0, '54.755')] [2024-09-30 01:57:00,975][1157819] Updated weights for policy 0, policy_version 93188 (0.0006) [2024-09-30 01:57:01,503][1157819] Updated weights for policy 0, policy_version 93198 (0.0006) [2024-09-30 01:57:02,010][1157819] Updated weights for policy 0, policy_version 93208 (0.0006) [2024-09-30 01:57:02,535][1157819] Updated weights for policy 0, policy_version 93218 (0.0006) [2024-09-30 01:57:03,043][1157819] Updated weights for policy 0, policy_version 93228 (0.0006) [2024-09-30 01:57:03,572][1157819] Updated weights for policy 0, policy_version 93238 (0.0006) [2024-09-30 01:57:04,102][1157819] Updated weights for policy 0, policy_version 93248 (0.0006) [2024-09-30 01:57:04,628][1157819] Updated weights for policy 0, policy_version 93258 (0.0006) [2024-09-30 01:57:05,132][1157819] Updated weights for policy 0, policy_version 93268 (0.0006) [2024-09-30 01:57:05,466][1157520] Fps is (10 sec: 79052.6, 60 sec: 80008.6, 300 sec: 78393.3). Total num frames: 382050304. Throughput: 0: 20102.7. Samples: 85500016. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 01:57:05,466][1157520] Avg episode reward: [(0, '51.343')] [2024-09-30 01:57:05,623][1157819] Updated weights for policy 0, policy_version 93278 (0.0006) [2024-09-30 01:57:06,134][1157819] Updated weights for policy 0, policy_version 93288 (0.0006) [2024-09-30 01:57:06,641][1157819] Updated weights for policy 0, policy_version 93298 (0.0006) [2024-09-30 01:57:07,145][1157819] Updated weights for policy 0, policy_version 93308 (0.0006) [2024-09-30 01:57:07,422][1157736] Signal inference workers to stop experience collection... (6000 times) [2024-09-30 01:57:07,422][1157736] Signal inference workers to resume experience collection... (6000 times) [2024-09-30 01:57:07,428][1157819] InferenceWorker_p0-w0: stopping experience collection (6000 times) [2024-09-30 01:57:07,428][1157819] InferenceWorker_p0-w0: resuming experience collection (6000 times) [2024-09-30 01:57:07,644][1157819] Updated weights for policy 0, policy_version 93318 (0.0006) [2024-09-30 01:57:08,196][1157819] Updated weights for policy 0, policy_version 93328 (0.0006) [2024-09-30 01:57:08,699][1157819] Updated weights for policy 0, policy_version 93338 (0.0006) [2024-09-30 01:57:09,228][1157819] Updated weights for policy 0, policy_version 93348 (0.0006) [2024-09-30 01:57:09,736][1157819] Updated weights for policy 0, policy_version 93358 (0.0006) [2024-09-30 01:57:10,263][1157819] Updated weights for policy 0, policy_version 93368 (0.0006) [2024-09-30 01:57:10,466][1157520] Fps is (10 sec: 79052.8, 60 sec: 80008.6, 300 sec: 78351.7). Total num frames: 382447616. Throughput: 0: 20105.0. Samples: 85559968. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 01:57:10,466][1157520] Avg episode reward: [(0, '53.392')] [2024-09-30 01:57:10,796][1157819] Updated weights for policy 0, policy_version 93378 (0.0006) [2024-09-30 01:57:11,301][1157819] Updated weights for policy 0, policy_version 93388 (0.0006) [2024-09-30 01:57:11,809][1157819] Updated weights for policy 0, policy_version 93398 (0.0006) [2024-09-30 01:57:12,302][1157819] Updated weights for policy 0, policy_version 93408 (0.0006) [2024-09-30 01:57:12,785][1157819] Updated weights for policy 0, policy_version 93418 (0.0006) [2024-09-30 01:57:13,296][1157819] Updated weights for policy 0, policy_version 93428 (0.0006) [2024-09-30 01:57:13,782][1157819] Updated weights for policy 0, policy_version 93438 (0.0006) [2024-09-30 01:57:14,274][1157819] Updated weights for policy 0, policy_version 93448 (0.0006) [2024-09-30 01:57:14,749][1157819] Updated weights for policy 0, policy_version 93458 (0.0006) [2024-09-30 01:57:15,269][1157819] Updated weights for policy 0, policy_version 93468 (0.0006) [2024-09-30 01:57:15,466][1157520] Fps is (10 sec: 80690.4, 60 sec: 80281.4, 300 sec: 78337.7). Total num frames: 382857216. Throughput: 0: 20115.6. Samples: 85680456. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 01:57:15,466][1157520] Avg episode reward: [(0, '54.618')] [2024-09-30 01:57:15,780][1157819] Updated weights for policy 0, policy_version 93478 (0.0006) [2024-09-30 01:57:16,279][1157819] Updated weights for policy 0, policy_version 93488 (0.0006) [2024-09-30 01:57:16,782][1157819] Updated weights for policy 0, policy_version 93498 (0.0006) [2024-09-30 01:57:17,332][1157819] Updated weights for policy 0, policy_version 93508 (0.0006) [2024-09-30 01:57:17,852][1157819] Updated weights for policy 0, policy_version 93518 (0.0006) [2024-09-30 01:57:18,407][1157819] Updated weights for policy 0, policy_version 93528 (0.0006) [2024-09-30 01:57:18,927][1157819] Updated weights for policy 0, policy_version 93538 (0.0006) [2024-09-30 01:57:19,424][1157819] Updated weights for policy 0, policy_version 93548 (0.0006) [2024-09-30 01:57:19,921][1157819] Updated weights for policy 0, policy_version 93558 (0.0006) [2024-09-30 01:57:20,466][1157520] Fps is (10 sec: 80281.0, 60 sec: 80281.5, 300 sec: 78268.3). Total num frames: 383250432. Throughput: 0: 20020.7. Samples: 85800360. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 01:57:20,466][1157520] Avg episode reward: [(0, '56.078')] [2024-09-30 01:57:20,473][1157819] Updated weights for policy 0, policy_version 93568 (0.0006) [2024-09-30 01:57:20,970][1157819] Updated weights for policy 0, policy_version 93578 (0.0006) [2024-09-30 01:57:21,476][1157819] Updated weights for policy 0, policy_version 93588 (0.0006) [2024-09-30 01:57:22,059][1157819] Updated weights for policy 0, policy_version 93598 (0.0006) [2024-09-30 01:57:22,554][1157819] Updated weights for policy 0, policy_version 93608 (0.0006) [2024-09-30 01:57:23,057][1157819] Updated weights for policy 0, policy_version 93618 (0.0006) [2024-09-30 01:57:23,583][1157819] Updated weights for policy 0, policy_version 93628 (0.0006) [2024-09-30 01:57:24,101][1157819] Updated weights for policy 0, policy_version 93638 (0.0006) [2024-09-30 01:57:24,598][1157819] Updated weights for policy 0, policy_version 93648 (0.0006) [2024-09-30 01:57:25,138][1157819] Updated weights for policy 0, policy_version 93658 (0.0006) [2024-09-30 01:57:25,466][1157520] Fps is (10 sec: 79053.8, 60 sec: 80213.3, 300 sec: 78212.8). Total num frames: 383647744. Throughput: 0: 19974.4. Samples: 85859368. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 01:57:25,466][1157520] Avg episode reward: [(0, '53.062')] [2024-09-30 01:57:25,684][1157819] Updated weights for policy 0, policy_version 93668 (0.0006) [2024-09-30 01:57:26,190][1157819] Updated weights for policy 0, policy_version 93678 (0.0006) [2024-09-30 01:57:26,709][1157819] Updated weights for policy 0, policy_version 93688 (0.0006) [2024-09-30 01:57:27,223][1157819] Updated weights for policy 0, policy_version 93698 (0.0006) [2024-09-30 01:57:27,739][1157819] Updated weights for policy 0, policy_version 93708 (0.0006) [2024-09-30 01:57:28,292][1157819] Updated weights for policy 0, policy_version 93718 (0.0006) [2024-09-30 01:57:28,805][1157819] Updated weights for policy 0, policy_version 93728 (0.0006) [2024-09-30 01:57:29,321][1157819] Updated weights for policy 0, policy_version 93738 (0.0006) [2024-09-30 01:57:29,836][1157819] Updated weights for policy 0, policy_version 93748 (0.0006) [2024-09-30 01:57:30,400][1157819] Updated weights for policy 0, policy_version 93758 (0.0006) [2024-09-30 01:57:30,466][1157520] Fps is (10 sec: 78643.8, 60 sec: 79940.1, 300 sec: 78198.9). Total num frames: 384036864. Throughput: 0: 19827.5. Samples: 85977232. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 01:57:30,466][1157520] Avg episode reward: [(0, '53.427')] [2024-09-30 01:57:30,912][1157819] Updated weights for policy 0, policy_version 93768 (0.0006) [2024-09-30 01:57:31,425][1157819] Updated weights for policy 0, policy_version 93778 (0.0006) [2024-09-30 01:57:31,951][1157819] Updated weights for policy 0, policy_version 93788 (0.0006) [2024-09-30 01:57:32,478][1157819] Updated weights for policy 0, policy_version 93798 (0.0006) [2024-09-30 01:57:32,995][1157819] Updated weights for policy 0, policy_version 93808 (0.0006) [2024-09-30 01:57:33,507][1157819] Updated weights for policy 0, policy_version 93818 (0.0006) [2024-09-30 01:57:34,041][1157819] Updated weights for policy 0, policy_version 93828 (0.0006) [2024-09-30 01:57:34,588][1157819] Updated weights for policy 0, policy_version 93838 (0.0006) [2024-09-30 01:57:35,128][1157819] Updated weights for policy 0, policy_version 93848 (0.0006) [2024-09-30 01:57:35,466][1157520] Fps is (10 sec: 77823.4, 60 sec: 79598.7, 300 sec: 78212.7). Total num frames: 384425984. Throughput: 0: 19749.6. Samples: 86094416. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 01:57:35,466][1157520] Avg episode reward: [(0, '55.648')] [2024-09-30 01:57:35,647][1157819] Updated weights for policy 0, policy_version 93858 (0.0006) [2024-09-30 01:57:36,191][1157819] Updated weights for policy 0, policy_version 93868 (0.0006) [2024-09-30 01:57:36,714][1157819] Updated weights for policy 0, policy_version 93878 (0.0006) [2024-09-30 01:57:37,247][1157819] Updated weights for policy 0, policy_version 93888 (0.0006) [2024-09-30 01:57:37,803][1157819] Updated weights for policy 0, policy_version 93898 (0.0006) [2024-09-30 01:57:38,348][1157819] Updated weights for policy 0, policy_version 93908 (0.0006) [2024-09-30 01:57:38,912][1157819] Updated weights for policy 0, policy_version 93918 (0.0006) [2024-09-30 01:57:39,455][1157819] Updated weights for policy 0, policy_version 93928 (0.0006) [2024-09-30 01:57:40,026][1157819] Updated weights for policy 0, policy_version 93938 (0.0006) [2024-09-30 01:57:40,466][1157520] Fps is (10 sec: 76595.2, 60 sec: 78984.4, 300 sec: 78143.4). Total num frames: 384802816. Throughput: 0: 19698.9. Samples: 86150996. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 01:57:40,466][1157520] Avg episode reward: [(0, '56.713')] [2024-09-30 01:57:40,535][1157819] Updated weights for policy 0, policy_version 93948 (0.0006) [2024-09-30 01:57:41,067][1157819] Updated weights for policy 0, policy_version 93958 (0.0006) [2024-09-30 01:57:41,624][1157819] Updated weights for policy 0, policy_version 93968 (0.0006) [2024-09-30 01:57:42,188][1157819] Updated weights for policy 0, policy_version 93978 (0.0006) [2024-09-30 01:57:42,724][1157819] Updated weights for policy 0, policy_version 93988 (0.0006) [2024-09-30 01:57:43,259][1157819] Updated weights for policy 0, policy_version 93998 (0.0006) [2024-09-30 01:57:43,829][1157819] Updated weights for policy 0, policy_version 94008 (0.0006) [2024-09-30 01:57:44,321][1157819] Updated weights for policy 0, policy_version 94018 (0.0006) [2024-09-30 01:57:44,866][1157819] Updated weights for policy 0, policy_version 94028 (0.0006) [2024-09-30 01:57:45,429][1157819] Updated weights for policy 0, policy_version 94038 (0.0006) [2024-09-30 01:57:45,466][1157520] Fps is (10 sec: 75365.9, 60 sec: 78438.2, 300 sec: 78073.9). Total num frames: 385179648. Throughput: 0: 19623.0. Samples: 86263948. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:57:45,466][1157520] Avg episode reward: [(0, '51.069')] [2024-09-30 01:57:45,968][1157819] Updated weights for policy 0, policy_version 94048 (0.0006) [2024-09-30 01:57:46,508][1157819] Updated weights for policy 0, policy_version 94058 (0.0006) [2024-09-30 01:57:47,050][1157819] Updated weights for policy 0, policy_version 94068 (0.0006) [2024-09-30 01:57:47,596][1157819] Updated weights for policy 0, policy_version 94078 (0.0006) [2024-09-30 01:57:48,163][1157819] Updated weights for policy 0, policy_version 94088 (0.0006) [2024-09-30 01:57:48,685][1157819] Updated weights for policy 0, policy_version 94098 (0.0006) [2024-09-30 01:57:49,199][1157819] Updated weights for policy 0, policy_version 94108 (0.0006) [2024-09-30 01:57:49,701][1157819] Updated weights for policy 0, policy_version 94118 (0.0005) [2024-09-30 01:57:50,199][1157819] Updated weights for policy 0, policy_version 94128 (0.0006) [2024-09-30 01:57:50,466][1157520] Fps is (10 sec: 76594.8, 60 sec: 78233.5, 300 sec: 78073.9). Total num frames: 385568768. Throughput: 0: 19517.6. Samples: 86378308. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:57:50,466][1157520] Avg episode reward: [(0, '53.961')] [2024-09-30 01:57:50,728][1157819] Updated weights for policy 0, policy_version 94138 (0.0006) [2024-09-30 01:57:51,257][1157819] Updated weights for policy 0, policy_version 94148 (0.0006) [2024-09-30 01:57:51,766][1157819] Updated weights for policy 0, policy_version 94158 (0.0006) [2024-09-30 01:57:52,296][1157819] Updated weights for policy 0, policy_version 94168 (0.0006) [2024-09-30 01:57:52,791][1157819] Updated weights for policy 0, policy_version 94178 (0.0006) [2024-09-30 01:57:53,301][1157819] Updated weights for policy 0, policy_version 94188 (0.0006) [2024-09-30 01:57:53,851][1157819] Updated weights for policy 0, policy_version 94198 (0.0006) [2024-09-30 01:57:54,348][1157819] Updated weights for policy 0, policy_version 94208 (0.0006) [2024-09-30 01:57:54,877][1157819] Updated weights for policy 0, policy_version 94218 (0.0006) [2024-09-30 01:57:55,365][1157819] Updated weights for policy 0, policy_version 94228 (0.0006) [2024-09-30 01:57:55,466][1157520] Fps is (10 sec: 78234.2, 60 sec: 78370.0, 300 sec: 78143.3). Total num frames: 385961984. Throughput: 0: 19511.4. Samples: 86437984. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:57:55,466][1157520] Avg episode reward: [(0, '53.534')] [2024-09-30 01:57:55,925][1157819] Updated weights for policy 0, policy_version 94238 (0.0006) [2024-09-30 01:57:56,465][1157819] Updated weights for policy 0, policy_version 94248 (0.0006) [2024-09-30 01:57:56,981][1157819] Updated weights for policy 0, policy_version 94258 (0.0006) [2024-09-30 01:57:57,566][1157819] Updated weights for policy 0, policy_version 94268 (0.0006) [2024-09-30 01:57:58,074][1157819] Updated weights for policy 0, policy_version 94278 (0.0006) [2024-09-30 01:57:58,653][1157819] Updated weights for policy 0, policy_version 94288 (0.0006) [2024-09-30 01:57:59,192][1157819] Updated weights for policy 0, policy_version 94298 (0.0006) [2024-09-30 01:57:59,735][1157819] Updated weights for policy 0, policy_version 94308 (0.0006) [2024-09-30 01:58:00,277][1157819] Updated weights for policy 0, policy_version 94318 (0.0006) [2024-09-30 01:58:00,466][1157520] Fps is (10 sec: 77004.5, 60 sec: 78028.7, 300 sec: 78143.3). Total num frames: 386338816. Throughput: 0: 19403.0. Samples: 86553592. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:58:00,466][1157520] Avg episode reward: [(0, '53.910')] [2024-09-30 01:58:00,837][1157819] Updated weights for policy 0, policy_version 94328 (0.0006) [2024-09-30 01:58:01,382][1157819] Updated weights for policy 0, policy_version 94338 (0.0006) [2024-09-30 01:58:01,885][1157819] Updated weights for policy 0, policy_version 94348 (0.0006) [2024-09-30 01:58:02,401][1157819] Updated weights for policy 0, policy_version 94358 (0.0006) [2024-09-30 01:58:02,923][1157819] Updated weights for policy 0, policy_version 94368 (0.0006) [2024-09-30 01:58:03,143][1157736] Signal inference workers to stop experience collection... (6050 times) [2024-09-30 01:58:03,147][1157819] InferenceWorker_p0-w0: stopping experience collection (6050 times) [2024-09-30 01:58:03,148][1157736] Signal inference workers to resume experience collection... (6050 times) [2024-09-30 01:58:03,152][1157819] InferenceWorker_p0-w0: resuming experience collection (6050 times) [2024-09-30 01:58:03,435][1157819] Updated weights for policy 0, policy_version 94378 (0.0006) [2024-09-30 01:58:03,934][1157819] Updated weights for policy 0, policy_version 94388 (0.0006) [2024-09-30 01:58:04,434][1157819] Updated weights for policy 0, policy_version 94398 (0.0006) [2024-09-30 01:58:04,936][1157819] Updated weights for policy 0, policy_version 94408 (0.0006) [2024-09-30 01:58:05,426][1157819] Updated weights for policy 0, policy_version 94418 (0.0006) [2024-09-30 01:58:05,466][1157520] Fps is (10 sec: 77824.2, 60 sec: 78165.3, 300 sec: 78198.9). Total num frames: 386740224. Throughput: 0: 19340.9. Samples: 86670700. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:58:05,466][1157520] Avg episode reward: [(0, '56.479')] [2024-09-30 01:58:05,472][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000094419_386740224.pth... [2024-09-30 01:58:05,513][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000089789_367775744.pth [2024-09-30 01:58:05,906][1157819] Updated weights for policy 0, policy_version 94428 (0.0006) [2024-09-30 01:58:06,398][1157819] Updated weights for policy 0, policy_version 94438 (0.0006) [2024-09-30 01:58:06,888][1157819] Updated weights for policy 0, policy_version 94448 (0.0006) [2024-09-30 01:58:07,338][1157819] Updated weights for policy 0, policy_version 94458 (0.0006) [2024-09-30 01:58:07,828][1157819] Updated weights for policy 0, policy_version 94468 (0.0006) [2024-09-30 01:58:08,331][1157819] Updated weights for policy 0, policy_version 94478 (0.0006) [2024-09-30 01:58:08,824][1157819] Updated weights for policy 0, policy_version 94488 (0.0006) [2024-09-30 01:58:09,291][1157819] Updated weights for policy 0, policy_version 94498 (0.0006) [2024-09-30 01:58:09,799][1157819] Updated weights for policy 0, policy_version 94508 (0.0006) [2024-09-30 01:58:10,293][1157819] Updated weights for policy 0, policy_version 94518 (0.0006) [2024-09-30 01:58:10,466][1157520] Fps is (10 sec: 82330.4, 60 sec: 78575.0, 300 sec: 78379.4). Total num frames: 387162112. Throughput: 0: 19437.3. Samples: 86734048. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:58:10,466][1157520] Avg episode reward: [(0, '53.151')] [2024-09-30 01:58:10,784][1157819] Updated weights for policy 0, policy_version 94528 (0.0006) [2024-09-30 01:58:11,230][1157819] Updated weights for policy 0, policy_version 94538 (0.0006) [2024-09-30 01:58:11,770][1157819] Updated weights for policy 0, policy_version 94548 (0.0006) [2024-09-30 01:58:12,266][1157819] Updated weights for policy 0, policy_version 94558 (0.0006) [2024-09-30 01:58:12,768][1157819] Updated weights for policy 0, policy_version 94568 (0.0006) [2024-09-30 01:58:13,295][1157819] Updated weights for policy 0, policy_version 94578 (0.0006) [2024-09-30 01:58:13,793][1157819] Updated weights for policy 0, policy_version 94588 (0.0006) [2024-09-30 01:58:14,294][1157819] Updated weights for policy 0, policy_version 94598 (0.0006) [2024-09-30 01:58:14,809][1157819] Updated weights for policy 0, policy_version 94608 (0.0006) [2024-09-30 01:58:15,313][1157819] Updated weights for policy 0, policy_version 94618 (0.0006) [2024-09-30 01:58:15,466][1157520] Fps is (10 sec: 82329.9, 60 sec: 78438.6, 300 sec: 78504.3). Total num frames: 387563520. Throughput: 0: 19571.4. Samples: 86857944. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:58:15,466][1157520] Avg episode reward: [(0, '53.628')] [2024-09-30 01:58:15,827][1157819] Updated weights for policy 0, policy_version 94628 (0.0006) [2024-09-30 01:58:16,336][1157819] Updated weights for policy 0, policy_version 94638 (0.0006) [2024-09-30 01:58:16,841][1157819] Updated weights for policy 0, policy_version 94648 (0.0006) [2024-09-30 01:58:17,349][1157819] Updated weights for policy 0, policy_version 94658 (0.0006) [2024-09-30 01:58:17,848][1157819] Updated weights for policy 0, policy_version 94668 (0.0006) [2024-09-30 01:58:18,332][1157819] Updated weights for policy 0, policy_version 94678 (0.0006) [2024-09-30 01:58:18,827][1157819] Updated weights for policy 0, policy_version 94688 (0.0006) [2024-09-30 01:58:19,346][1157819] Updated weights for policy 0, policy_version 94698 (0.0006) [2024-09-30 01:58:19,854][1157819] Updated weights for policy 0, policy_version 94708 (0.0006) [2024-09-30 01:58:20,367][1157819] Updated weights for policy 0, policy_version 94718 (0.0006) [2024-09-30 01:58:20,466][1157520] Fps is (10 sec: 81100.7, 60 sec: 78711.6, 300 sec: 78657.1). Total num frames: 387973120. Throughput: 0: 19665.5. Samples: 86979360. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:58:20,466][1157520] Avg episode reward: [(0, '56.502')] [2024-09-30 01:58:20,866][1157819] Updated weights for policy 0, policy_version 94728 (0.0006) [2024-09-30 01:58:21,386][1157819] Updated weights for policy 0, policy_version 94738 (0.0006) [2024-09-30 01:58:21,926][1157819] Updated weights for policy 0, policy_version 94748 (0.0006) [2024-09-30 01:58:22,492][1157819] Updated weights for policy 0, policy_version 94758 (0.0006) [2024-09-30 01:58:23,118][1157819] Updated weights for policy 0, policy_version 94768 (0.0007) [2024-09-30 01:58:23,685][1157819] Updated weights for policy 0, policy_version 94778 (0.0006) [2024-09-30 01:58:24,279][1157819] Updated weights for policy 0, policy_version 94788 (0.0007) [2024-09-30 01:58:24,834][1157819] Updated weights for policy 0, policy_version 94798 (0.0006) [2024-09-30 01:58:25,434][1157819] Updated weights for policy 0, policy_version 94808 (0.0006) [2024-09-30 01:58:25,466][1157520] Fps is (10 sec: 77003.6, 60 sec: 78096.9, 300 sec: 78684.8). Total num frames: 388333568. Throughput: 0: 19682.1. Samples: 87036692. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:58:25,466][1157520] Avg episode reward: [(0, '52.768')] [2024-09-30 01:58:25,972][1157819] Updated weights for policy 0, policy_version 94818 (0.0007) [2024-09-30 01:58:26,514][1157819] Updated weights for policy 0, policy_version 94828 (0.0006) [2024-09-30 01:58:27,036][1157819] Updated weights for policy 0, policy_version 94838 (0.0006) [2024-09-30 01:58:27,601][1157819] Updated weights for policy 0, policy_version 94848 (0.0006) [2024-09-30 01:58:28,137][1157819] Updated weights for policy 0, policy_version 94858 (0.0006) [2024-09-30 01:58:28,739][1157819] Updated weights for policy 0, policy_version 94868 (0.0006) [2024-09-30 01:58:29,250][1157819] Updated weights for policy 0, policy_version 94878 (0.0006) [2024-09-30 01:58:29,803][1157819] Updated weights for policy 0, policy_version 94888 (0.0006) [2024-09-30 01:58:30,343][1157819] Updated weights for policy 0, policy_version 94898 (0.0006) [2024-09-30 01:58:30,466][1157520] Fps is (10 sec: 73728.0, 60 sec: 77892.3, 300 sec: 78698.7). Total num frames: 388710400. Throughput: 0: 19612.6. Samples: 87146512. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:58:30,466][1157520] Avg episode reward: [(0, '54.146')] [2024-09-30 01:58:30,901][1157819] Updated weights for policy 0, policy_version 94908 (0.0006) [2024-09-30 01:58:31,456][1157819] Updated weights for policy 0, policy_version 94918 (0.0006) [2024-09-30 01:58:31,974][1157819] Updated weights for policy 0, policy_version 94928 (0.0006) [2024-09-30 01:58:32,551][1157819] Updated weights for policy 0, policy_version 94938 (0.0006) [2024-09-30 01:58:33,087][1157819] Updated weights for policy 0, policy_version 94948 (0.0006) [2024-09-30 01:58:33,664][1157819] Updated weights for policy 0, policy_version 94958 (0.0006) [2024-09-30 01:58:34,194][1157819] Updated weights for policy 0, policy_version 94968 (0.0006) [2024-09-30 01:58:34,777][1157819] Updated weights for policy 0, policy_version 94978 (0.0006) [2024-09-30 01:58:35,329][1157819] Updated weights for policy 0, policy_version 94988 (0.0006) [2024-09-30 01:58:35,466][1157520] Fps is (10 sec: 74548.3, 60 sec: 77551.0, 300 sec: 78671.0). Total num frames: 389079040. Throughput: 0: 19545.6. Samples: 87257860. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 01:58:35,466][1157520] Avg episode reward: [(0, '55.302')] [2024-09-30 01:58:35,868][1157819] Updated weights for policy 0, policy_version 94998 (0.0006) [2024-09-30 01:58:36,409][1157819] Updated weights for policy 0, policy_version 95008 (0.0006) [2024-09-30 01:58:36,955][1157819] Updated weights for policy 0, policy_version 95018 (0.0006) [2024-09-30 01:58:37,549][1157819] Updated weights for policy 0, policy_version 95028 (0.0006) [2024-09-30 01:58:38,075][1157819] Updated weights for policy 0, policy_version 95038 (0.0006) [2024-09-30 01:58:38,630][1157819] Updated weights for policy 0, policy_version 95048 (0.0006) [2024-09-30 01:58:39,210][1157819] Updated weights for policy 0, policy_version 95058 (0.0006) [2024-09-30 01:58:39,749][1157819] Updated weights for policy 0, policy_version 95068 (0.0006) [2024-09-30 01:58:40,332][1157819] Updated weights for policy 0, policy_version 95078 (0.0006) [2024-09-30 01:58:40,466][1157520] Fps is (10 sec: 73727.8, 60 sec: 77414.4, 300 sec: 78698.7). Total num frames: 389447680. Throughput: 0: 19453.8. Samples: 87313404. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 01:58:40,466][1157520] Avg episode reward: [(0, '52.795')] [2024-09-30 01:58:40,858][1157819] Updated weights for policy 0, policy_version 95088 (0.0006) [2024-09-30 01:58:41,367][1157819] Updated weights for policy 0, policy_version 95098 (0.0006) [2024-09-30 01:58:41,979][1157819] Updated weights for policy 0, policy_version 95108 (0.0006) [2024-09-30 01:58:42,498][1157819] Updated weights for policy 0, policy_version 95118 (0.0006) [2024-09-30 01:58:43,087][1157819] Updated weights for policy 0, policy_version 95128 (0.0006) [2024-09-30 01:58:43,625][1157819] Updated weights for policy 0, policy_version 95138 (0.0006) [2024-09-30 01:58:44,209][1157819] Updated weights for policy 0, policy_version 95148 (0.0006) [2024-09-30 01:58:44,772][1157819] Updated weights for policy 0, policy_version 95158 (0.0006) [2024-09-30 01:58:45,360][1157819] Updated weights for policy 0, policy_version 95168 (0.0006) [2024-09-30 01:58:45,466][1157520] Fps is (10 sec: 73318.5, 60 sec: 77209.8, 300 sec: 78601.5). Total num frames: 389812224. Throughput: 0: 19350.7. Samples: 87424372. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 01:58:45,466][1157520] Avg episode reward: [(0, '54.139')] [2024-09-30 01:58:45,969][1157819] Updated weights for policy 0, policy_version 95178 (0.0006) [2024-09-30 01:58:46,541][1157819] Updated weights for policy 0, policy_version 95188 (0.0006) [2024-09-30 01:58:46,919][1157736] Signal inference workers to stop experience collection... (6100 times) [2024-09-30 01:58:46,920][1157736] Signal inference workers to resume experience collection... (6100 times) [2024-09-30 01:58:46,925][1157819] InferenceWorker_p0-w0: stopping experience collection (6100 times) [2024-09-30 01:58:46,925][1157819] InferenceWorker_p0-w0: resuming experience collection (6100 times) [2024-09-30 01:58:47,134][1157819] Updated weights for policy 0, policy_version 95198 (0.0006) [2024-09-30 01:58:47,739][1157819] Updated weights for policy 0, policy_version 95208 (0.0006) [2024-09-30 01:58:48,338][1157819] Updated weights for policy 0, policy_version 95218 (0.0006) [2024-09-30 01:58:48,919][1157819] Updated weights for policy 0, policy_version 95228 (0.0006) [2024-09-30 01:58:49,501][1157819] Updated weights for policy 0, policy_version 95238 (0.0006) [2024-09-30 01:58:50,090][1157819] Updated weights for policy 0, policy_version 95248 (0.0006) [2024-09-30 01:58:50,466][1157520] Fps is (10 sec: 71270.5, 60 sec: 76527.0, 300 sec: 78490.5). Total num frames: 390160384. Throughput: 0: 19068.8. Samples: 87528796. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 01:58:50,466][1157520] Avg episode reward: [(0, '56.830')] [2024-09-30 01:58:50,702][1157819] Updated weights for policy 0, policy_version 95258 (0.0006) [2024-09-30 01:58:51,267][1157819] Updated weights for policy 0, policy_version 95268 (0.0006) [2024-09-30 01:58:51,882][1157819] Updated weights for policy 0, policy_version 95278 (0.0006) [2024-09-30 01:58:52,475][1157819] Updated weights for policy 0, policy_version 95288 (0.0006) [2024-09-30 01:58:52,995][1157819] Updated weights for policy 0, policy_version 95298 (0.0006) [2024-09-30 01:58:53,556][1157819] Updated weights for policy 0, policy_version 95308 (0.0006) [2024-09-30 01:58:54,121][1157819] Updated weights for policy 0, policy_version 95318 (0.0006) [2024-09-30 01:58:54,625][1157819] Updated weights for policy 0, policy_version 95328 (0.0006) [2024-09-30 01:58:55,132][1157819] Updated weights for policy 0, policy_version 95338 (0.0006) [2024-09-30 01:58:55,466][1157520] Fps is (10 sec: 71679.8, 60 sec: 76117.4, 300 sec: 78448.8). Total num frames: 390529024. Throughput: 0: 18826.4. Samples: 87581236. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 01:58:55,466][1157520] Avg episode reward: [(0, '54.322')] [2024-09-30 01:58:55,647][1157819] Updated weights for policy 0, policy_version 95348 (0.0006) [2024-09-30 01:58:56,171][1157819] Updated weights for policy 0, policy_version 95358 (0.0006) [2024-09-30 01:58:56,699][1157819] Updated weights for policy 0, policy_version 95368 (0.0006) [2024-09-30 01:58:57,202][1157819] Updated weights for policy 0, policy_version 95378 (0.0006) [2024-09-30 01:58:57,714][1157819] Updated weights for policy 0, policy_version 95388 (0.0006) [2024-09-30 01:58:58,213][1157819] Updated weights for policy 0, policy_version 95398 (0.0006) [2024-09-30 01:58:58,706][1157819] Updated weights for policy 0, policy_version 95408 (0.0006) [2024-09-30 01:58:59,252][1157819] Updated weights for policy 0, policy_version 95418 (0.0006) [2024-09-30 01:58:59,742][1157819] Updated weights for policy 0, policy_version 95428 (0.0006) [2024-09-30 01:59:00,224][1157819] Updated weights for policy 0, policy_version 95438 (0.0006) [2024-09-30 01:59:00,466][1157520] Fps is (10 sec: 77005.1, 60 sec: 76527.1, 300 sec: 78490.5). Total num frames: 390930432. Throughput: 0: 18691.0. Samples: 87699040. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 01:59:00,466][1157520] Avg episode reward: [(0, '55.835')] [2024-09-30 01:59:00,770][1157819] Updated weights for policy 0, policy_version 95448 (0.0006) [2024-09-30 01:59:01,218][1157819] Updated weights for policy 0, policy_version 95458 (0.0006) [2024-09-30 01:59:01,708][1157819] Updated weights for policy 0, policy_version 95468 (0.0006) [2024-09-30 01:59:02,221][1157819] Updated weights for policy 0, policy_version 95478 (0.0006) [2024-09-30 01:59:02,684][1157819] Updated weights for policy 0, policy_version 95488 (0.0006) [2024-09-30 01:59:03,189][1157819] Updated weights for policy 0, policy_version 95498 (0.0006) [2024-09-30 01:59:03,686][1157819] Updated weights for policy 0, policy_version 95508 (0.0006) [2024-09-30 01:59:04,146][1157819] Updated weights for policy 0, policy_version 95518 (0.0006) [2024-09-30 01:59:04,679][1157819] Updated weights for policy 0, policy_version 95528 (0.0006) [2024-09-30 01:59:05,154][1157819] Updated weights for policy 0, policy_version 95538 (0.0006) [2024-09-30 01:59:05,466][1157520] Fps is (10 sec: 81921.0, 60 sec: 76800.2, 300 sec: 78615.5). Total num frames: 391348224. Throughput: 0: 18754.2. Samples: 87823296. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 01:59:05,466][1157520] Avg episode reward: [(0, '55.492')] [2024-09-30 01:59:05,645][1157819] Updated weights for policy 0, policy_version 95548 (0.0006) [2024-09-30 01:59:06,137][1157819] Updated weights for policy 0, policy_version 95558 (0.0006) [2024-09-30 01:59:06,614][1157819] Updated weights for policy 0, policy_version 95568 (0.0006) [2024-09-30 01:59:07,081][1157819] Updated weights for policy 0, policy_version 95578 (0.0006) [2024-09-30 01:59:07,577][1157819] Updated weights for policy 0, policy_version 95588 (0.0006) [2024-09-30 01:59:08,058][1157819] Updated weights for policy 0, policy_version 95598 (0.0006) [2024-09-30 01:59:08,577][1157819] Updated weights for policy 0, policy_version 95608 (0.0006) [2024-09-30 01:59:09,043][1157819] Updated weights for policy 0, policy_version 95618 (0.0006) [2024-09-30 01:59:09,522][1157819] Updated weights for policy 0, policy_version 95628 (0.0006) [2024-09-30 01:59:10,016][1157819] Updated weights for policy 0, policy_version 95638 (0.0006) [2024-09-30 01:59:10,466][1157520] Fps is (10 sec: 83968.8, 60 sec: 76800.1, 300 sec: 78823.8). Total num frames: 391770112. Throughput: 0: 18886.1. Samples: 87886560. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 01:59:10,466][1157520] Avg episode reward: [(0, '54.685')] [2024-09-30 01:59:10,486][1157819] Updated weights for policy 0, policy_version 95648 (0.0006) [2024-09-30 01:59:10,991][1157819] Updated weights for policy 0, policy_version 95658 (0.0006) [2024-09-30 01:59:11,455][1157819] Updated weights for policy 0, policy_version 95668 (0.0006) [2024-09-30 01:59:11,950][1157819] Updated weights for policy 0, policy_version 95678 (0.0006) [2024-09-30 01:59:12,463][1157819] Updated weights for policy 0, policy_version 95688 (0.0006) [2024-09-30 01:59:12,903][1157819] Updated weights for policy 0, policy_version 95698 (0.0006) [2024-09-30 01:59:13,389][1157819] Updated weights for policy 0, policy_version 95708 (0.0006) [2024-09-30 01:59:13,875][1157819] Updated weights for policy 0, policy_version 95718 (0.0006) [2024-09-30 01:59:14,367][1157819] Updated weights for policy 0, policy_version 95728 (0.0006) [2024-09-30 01:59:14,848][1157819] Updated weights for policy 0, policy_version 95738 (0.0006) [2024-09-30 01:59:15,336][1157819] Updated weights for policy 0, policy_version 95748 (0.0006) [2024-09-30 01:59:15,466][1157520] Fps is (10 sec: 84377.6, 60 sec: 77141.5, 300 sec: 78948.7). Total num frames: 392192000. Throughput: 0: 19258.3. Samples: 88013132. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 01:59:15,466][1157520] Avg episode reward: [(0, '54.947')] [2024-09-30 01:59:15,826][1157819] Updated weights for policy 0, policy_version 95758 (0.0006) [2024-09-30 01:59:16,311][1157819] Updated weights for policy 0, policy_version 95768 (0.0006) [2024-09-30 01:59:16,796][1157819] Updated weights for policy 0, policy_version 95778 (0.0006) [2024-09-30 01:59:17,254][1157819] Updated weights for policy 0, policy_version 95788 (0.0006) [2024-09-30 01:59:17,719][1157819] Updated weights for policy 0, policy_version 95798 (0.0006) [2024-09-30 01:59:18,138][1157819] Updated weights for policy 0, policy_version 95808 (0.0006) [2024-09-30 01:59:18,604][1157819] Updated weights for policy 0, policy_version 95818 (0.0006) [2024-09-30 01:59:19,057][1157819] Updated weights for policy 0, policy_version 95828 (0.0006) [2024-09-30 01:59:19,482][1157819] Updated weights for policy 0, policy_version 95838 (0.0006) [2024-09-30 01:59:19,929][1157819] Updated weights for policy 0, policy_version 95848 (0.0006) [2024-09-30 01:59:20,348][1157819] Updated weights for policy 0, policy_version 95858 (0.0006) [2024-09-30 01:59:20,466][1157520] Fps is (10 sec: 87244.5, 60 sec: 77824.1, 300 sec: 79157.0). Total num frames: 392642560. Throughput: 0: 19721.7. Samples: 88145336. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 01:59:20,466][1157520] Avg episode reward: [(0, '53.720')] [2024-09-30 01:59:20,803][1157819] Updated weights for policy 0, policy_version 95868 (0.0006) [2024-09-30 01:59:21,245][1157819] Updated weights for policy 0, policy_version 95878 (0.0006) [2024-09-30 01:59:21,680][1157819] Updated weights for policy 0, policy_version 95888 (0.0006) [2024-09-30 01:59:22,125][1157819] Updated weights for policy 0, policy_version 95898 (0.0006) [2024-09-30 01:59:22,589][1157819] Updated weights for policy 0, policy_version 95908 (0.0006) [2024-09-30 01:59:23,029][1157819] Updated weights for policy 0, policy_version 95918 (0.0006) [2024-09-30 01:59:23,466][1157819] Updated weights for policy 0, policy_version 95928 (0.0006) [2024-09-30 01:59:23,936][1157819] Updated weights for policy 0, policy_version 95938 (0.0006) [2024-09-30 01:59:24,389][1157819] Updated weights for policy 0, policy_version 95948 (0.0006) [2024-09-30 01:59:24,814][1157819] Updated weights for policy 0, policy_version 95958 (0.0006) [2024-09-30 01:59:25,303][1157819] Updated weights for policy 0, policy_version 95968 (0.0006) [2024-09-30 01:59:25,466][1157520] Fps is (10 sec: 90520.6, 60 sec: 79394.3, 300 sec: 79379.1). Total num frames: 393097216. Throughput: 0: 20023.9. Samples: 88214480. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 01:59:25,466][1157520] Avg episode reward: [(0, '53.831')] [2024-09-30 01:59:25,814][1157819] Updated weights for policy 0, policy_version 95978 (0.0006) [2024-09-30 01:59:26,310][1157819] Updated weights for policy 0, policy_version 95988 (0.0006) [2024-09-30 01:59:26,801][1157819] Updated weights for policy 0, policy_version 95998 (0.0006) [2024-09-30 01:59:27,326][1157819] Updated weights for policy 0, policy_version 96008 (0.0006) [2024-09-30 01:59:27,832][1157819] Updated weights for policy 0, policy_version 96018 (0.0006) [2024-09-30 01:59:28,341][1157819] Updated weights for policy 0, policy_version 96028 (0.0006) [2024-09-30 01:59:28,845][1157819] Updated weights for policy 0, policy_version 96038 (0.0006) [2024-09-30 01:59:29,319][1157736] Signal inference workers to stop experience collection... (6150 times) [2024-09-30 01:59:29,321][1157736] Signal inference workers to resume experience collection... (6150 times) [2024-09-30 01:59:29,324][1157819] InferenceWorker_p0-w0: stopping experience collection (6150 times) [2024-09-30 01:59:29,327][1157819] InferenceWorker_p0-w0: resuming experience collection (6150 times) [2024-09-30 01:59:29,336][1157819] Updated weights for policy 0, policy_version 96048 (0.0006) [2024-09-30 01:59:29,850][1157819] Updated weights for policy 0, policy_version 96058 (0.0006) [2024-09-30 01:59:30,366][1157819] Updated weights for policy 0, policy_version 96068 (0.0006) [2024-09-30 01:59:30,466][1157520] Fps is (10 sec: 86015.6, 60 sec: 79872.0, 300 sec: 79434.7). Total num frames: 393502720. Throughput: 0: 20388.0. Samples: 88341832. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 01:59:30,466][1157520] Avg episode reward: [(0, '54.660')] [2024-09-30 01:59:30,860][1157819] Updated weights for policy 0, policy_version 96078 (0.0006) [2024-09-30 01:59:31,365][1157819] Updated weights for policy 0, policy_version 96088 (0.0006) [2024-09-30 01:59:31,874][1157819] Updated weights for policy 0, policy_version 96098 (0.0006) [2024-09-30 01:59:32,395][1157819] Updated weights for policy 0, policy_version 96108 (0.0006) [2024-09-30 01:59:32,899][1157819] Updated weights for policy 0, policy_version 96118 (0.0006) [2024-09-30 01:59:33,390][1157819] Updated weights for policy 0, policy_version 96128 (0.0006) [2024-09-30 01:59:33,886][1157819] Updated weights for policy 0, policy_version 96138 (0.0006) [2024-09-30 01:59:34,397][1157819] Updated weights for policy 0, policy_version 96148 (0.0006) [2024-09-30 01:59:34,892][1157819] Updated weights for policy 0, policy_version 96158 (0.0006) [2024-09-30 01:59:35,403][1157819] Updated weights for policy 0, policy_version 96168 (0.0006) [2024-09-30 01:59:35,466][1157520] Fps is (10 sec: 81100.7, 60 sec: 80486.4, 300 sec: 79476.3). Total num frames: 393908224. Throughput: 0: 20772.7. Samples: 88463568. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 01:59:35,466][1157520] Avg episode reward: [(0, '53.050')] [2024-09-30 01:59:35,903][1157819] Updated weights for policy 0, policy_version 96178 (0.0006) [2024-09-30 01:59:36,393][1157819] Updated weights for policy 0, policy_version 96188 (0.0006) [2024-09-30 01:59:36,875][1157819] Updated weights for policy 0, policy_version 96198 (0.0006) [2024-09-30 01:59:37,370][1157819] Updated weights for policy 0, policy_version 96208 (0.0006) [2024-09-30 01:59:37,867][1157819] Updated weights for policy 0, policy_version 96218 (0.0006) [2024-09-30 01:59:38,297][1157819] Updated weights for policy 0, policy_version 96228 (0.0006) [2024-09-30 01:59:38,765][1157819] Updated weights for policy 0, policy_version 96238 (0.0006) [2024-09-30 01:59:39,237][1157819] Updated weights for policy 0, policy_version 96248 (0.0006) [2024-09-30 01:59:39,720][1157819] Updated weights for policy 0, policy_version 96258 (0.0006) [2024-09-30 01:59:40,192][1157819] Updated weights for policy 0, policy_version 96268 (0.0006) [2024-09-30 01:59:40,466][1157520] Fps is (10 sec: 83558.5, 60 sec: 81510.5, 300 sec: 79601.3). Total num frames: 394338304. Throughput: 0: 20998.4. Samples: 88526164. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 01:59:40,466][1157520] Avg episode reward: [(0, '53.775')] [2024-09-30 01:59:40,628][1157819] Updated weights for policy 0, policy_version 96278 (0.0006) [2024-09-30 01:59:41,122][1157819] Updated weights for policy 0, policy_version 96288 (0.0006) [2024-09-30 01:59:41,616][1157819] Updated weights for policy 0, policy_version 96298 (0.0006) [2024-09-30 01:59:42,097][1157819] Updated weights for policy 0, policy_version 96308 (0.0006) [2024-09-30 01:59:42,597][1157819] Updated weights for policy 0, policy_version 96318 (0.0006) [2024-09-30 01:59:43,067][1157819] Updated weights for policy 0, policy_version 96328 (0.0006) [2024-09-30 01:59:43,566][1157819] Updated weights for policy 0, policy_version 96338 (0.0006) [2024-09-30 01:59:44,060][1157819] Updated weights for policy 0, policy_version 96348 (0.0006) [2024-09-30 01:59:44,579][1157819] Updated weights for policy 0, policy_version 96358 (0.0006) [2024-09-30 01:59:45,031][1157819] Updated weights for policy 0, policy_version 96368 (0.0006) [2024-09-30 01:59:45,466][1157520] Fps is (10 sec: 84788.8, 60 sec: 82398.1, 300 sec: 79629.1). Total num frames: 394756096. Throughput: 0: 21236.7. Samples: 88654688. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 01:59:45,466][1157520] Avg episode reward: [(0, '53.497')] [2024-09-30 01:59:45,514][1157819] Updated weights for policy 0, policy_version 96378 (0.0006) [2024-09-30 01:59:46,035][1157819] Updated weights for policy 0, policy_version 96388 (0.0006) [2024-09-30 01:59:46,482][1157819] Updated weights for policy 0, policy_version 96398 (0.0006) [2024-09-30 01:59:46,972][1157819] Updated weights for policy 0, policy_version 96408 (0.0006) [2024-09-30 01:59:47,480][1157819] Updated weights for policy 0, policy_version 96418 (0.0006) [2024-09-30 01:59:47,991][1157819] Updated weights for policy 0, policy_version 96428 (0.0006) [2024-09-30 01:59:48,479][1157819] Updated weights for policy 0, policy_version 96438 (0.0006) [2024-09-30 01:59:48,934][1157819] Updated weights for policy 0, policy_version 96448 (0.0006) [2024-09-30 01:59:49,409][1157819] Updated weights for policy 0, policy_version 96458 (0.0006) [2024-09-30 01:59:49,884][1157819] Updated weights for policy 0, policy_version 96468 (0.0006) [2024-09-30 01:59:50,389][1157819] Updated weights for policy 0, policy_version 96478 (0.0006) [2024-09-30 01:59:50,466][1157520] Fps is (10 sec: 84378.0, 60 sec: 83695.1, 300 sec: 79656.8). Total num frames: 395182080. Throughput: 0: 21280.5. Samples: 88780920. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 01:59:50,466][1157520] Avg episode reward: [(0, '56.328')] [2024-09-30 01:59:50,837][1157819] Updated weights for policy 0, policy_version 96488 (0.0006) [2024-09-30 01:59:51,267][1157819] Updated weights for policy 0, policy_version 96498 (0.0006) [2024-09-30 01:59:51,764][1157819] Updated weights for policy 0, policy_version 96508 (0.0006) [2024-09-30 01:59:52,258][1157819] Updated weights for policy 0, policy_version 96518 (0.0006) [2024-09-30 01:59:52,747][1157819] Updated weights for policy 0, policy_version 96528 (0.0006) [2024-09-30 01:59:53,249][1157819] Updated weights for policy 0, policy_version 96538 (0.0006) [2024-09-30 01:59:53,748][1157819] Updated weights for policy 0, policy_version 96548 (0.0006) [2024-09-30 01:59:54,230][1157819] Updated weights for policy 0, policy_version 96558 (0.0006) [2024-09-30 01:59:54,732][1157819] Updated weights for policy 0, policy_version 96568 (0.0006) [2024-09-30 01:59:55,222][1157819] Updated weights for policy 0, policy_version 96578 (0.0006) [2024-09-30 01:59:55,466][1157520] Fps is (10 sec: 84786.8, 60 sec: 84582.6, 300 sec: 79656.8). Total num frames: 395603968. Throughput: 0: 21301.8. Samples: 88845140. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 01:59:55,466][1157520] Avg episode reward: [(0, '55.044')] [2024-09-30 01:59:55,648][1157819] Updated weights for policy 0, policy_version 96588 (0.0005) [2024-09-30 01:59:56,145][1157819] Updated weights for policy 0, policy_version 96598 (0.0006) [2024-09-30 01:59:56,640][1157819] Updated weights for policy 0, policy_version 96608 (0.0006) [2024-09-30 01:59:57,122][1157819] Updated weights for policy 0, policy_version 96618 (0.0006) [2024-09-30 01:59:57,610][1157819] Updated weights for policy 0, policy_version 96628 (0.0006) [2024-09-30 01:59:58,095][1157819] Updated weights for policy 0, policy_version 96638 (0.0006) [2024-09-30 01:59:58,583][1157819] Updated weights for policy 0, policy_version 96648 (0.0006) [2024-09-30 01:59:59,068][1157819] Updated weights for policy 0, policy_version 96658 (0.0006) [2024-09-30 01:59:59,560][1157819] Updated weights for policy 0, policy_version 96668 (0.0006) [2024-09-30 02:00:00,056][1157819] Updated weights for policy 0, policy_version 96678 (0.0006) [2024-09-30 02:00:00,466][1157520] Fps is (10 sec: 84377.7, 60 sec: 84923.9, 300 sec: 79670.7). Total num frames: 396025856. Throughput: 0: 21297.2. Samples: 88971504. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 02:00:00,466][1157520] Avg episode reward: [(0, '52.265')] [2024-09-30 02:00:00,540][1157819] Updated weights for policy 0, policy_version 96688 (0.0006) [2024-09-30 02:00:01,012][1157819] Updated weights for policy 0, policy_version 96698 (0.0006) [2024-09-30 02:00:01,518][1157819] Updated weights for policy 0, policy_version 96708 (0.0006) [2024-09-30 02:00:02,015][1157819] Updated weights for policy 0, policy_version 96718 (0.0006) [2024-09-30 02:00:02,069][1157736] Signal inference workers to stop experience collection... (6200 times) [2024-09-30 02:00:02,074][1157819] InferenceWorker_p0-w0: stopping experience collection (6200 times) [2024-09-30 02:00:02,075][1157736] Signal inference workers to resume experience collection... (6200 times) [2024-09-30 02:00:02,079][1157819] InferenceWorker_p0-w0: resuming experience collection (6200 times) [2024-09-30 02:00:02,461][1157819] Updated weights for policy 0, policy_version 96728 (0.0006) [2024-09-30 02:00:02,959][1157819] Updated weights for policy 0, policy_version 96738 (0.0006) [2024-09-30 02:00:03,467][1157819] Updated weights for policy 0, policy_version 96748 (0.0006) [2024-09-30 02:00:03,943][1157819] Updated weights for policy 0, policy_version 96758 (0.0006) [2024-09-30 02:00:04,423][1157819] Updated weights for policy 0, policy_version 96768 (0.0006) [2024-09-30 02:00:04,938][1157819] Updated weights for policy 0, policy_version 96778 (0.0006) [2024-09-30 02:00:05,426][1157819] Updated weights for policy 0, policy_version 96788 (0.0006) [2024-09-30 02:00:05,466][1157520] Fps is (10 sec: 83967.9, 60 sec: 84923.7, 300 sec: 79684.6). Total num frames: 396443648. Throughput: 0: 21163.0. Samples: 89097672. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 02:00:05,466][1157520] Avg episode reward: [(0, '53.596')] [2024-09-30 02:00:05,469][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000096788_396443648.pth... [2024-09-30 02:00:05,516][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000092102_377249792.pth [2024-09-30 02:00:06,025][1157819] Updated weights for policy 0, policy_version 96798 (0.0006) [2024-09-30 02:00:06,607][1157819] Updated weights for policy 0, policy_version 96808 (0.0006) [2024-09-30 02:00:07,174][1157819] Updated weights for policy 0, policy_version 96818 (0.0006) [2024-09-30 02:00:07,782][1157819] Updated weights for policy 0, policy_version 96828 (0.0006) [2024-09-30 02:00:08,343][1157819] Updated weights for policy 0, policy_version 96838 (0.0006) [2024-09-30 02:00:08,857][1157819] Updated weights for policy 0, policy_version 96848 (0.0006) [2024-09-30 02:00:09,370][1157819] Updated weights for policy 0, policy_version 96858 (0.0006) [2024-09-30 02:00:09,894][1157819] Updated weights for policy 0, policy_version 96868 (0.0006) [2024-09-30 02:00:10,403][1157819] Updated weights for policy 0, policy_version 96878 (0.0006) [2024-09-30 02:00:10,466][1157520] Fps is (10 sec: 78643.1, 60 sec: 84036.2, 300 sec: 79545.7). Total num frames: 396812288. Throughput: 0: 20842.4. Samples: 89152384. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 02:00:10,466][1157520] Avg episode reward: [(0, '52.798')] [2024-09-30 02:00:10,957][1157819] Updated weights for policy 0, policy_version 96888 (0.0006) [2024-09-30 02:00:11,458][1157819] Updated weights for policy 0, policy_version 96898 (0.0006) [2024-09-30 02:00:11,959][1157819] Updated weights for policy 0, policy_version 96908 (0.0006) [2024-09-30 02:00:12,453][1157819] Updated weights for policy 0, policy_version 96918 (0.0006) [2024-09-30 02:00:12,963][1157819] Updated weights for policy 0, policy_version 96928 (0.0006) [2024-09-30 02:00:13,520][1157819] Updated weights for policy 0, policy_version 96938 (0.0006) [2024-09-30 02:00:14,032][1157819] Updated weights for policy 0, policy_version 96948 (0.0006) [2024-09-30 02:00:14,517][1157819] Updated weights for policy 0, policy_version 96958 (0.0006) [2024-09-30 02:00:15,037][1157819] Updated weights for policy 0, policy_version 96968 (0.0006) [2024-09-30 02:00:15,466][1157520] Fps is (10 sec: 77004.7, 60 sec: 83694.9, 300 sec: 79615.2). Total num frames: 397213696. Throughput: 0: 20639.4. Samples: 89270604. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 02:00:15,466][1157520] Avg episode reward: [(0, '52.337')] [2024-09-30 02:00:15,542][1157819] Updated weights for policy 0, policy_version 96978 (0.0006) [2024-09-30 02:00:16,053][1157819] Updated weights for policy 0, policy_version 96988 (0.0006) [2024-09-30 02:00:16,575][1157819] Updated weights for policy 0, policy_version 96998 (0.0006) [2024-09-30 02:00:17,096][1157819] Updated weights for policy 0, policy_version 97008 (0.0006) [2024-09-30 02:00:17,621][1157819] Updated weights for policy 0, policy_version 97018 (0.0006) [2024-09-30 02:00:18,148][1157819] Updated weights for policy 0, policy_version 97028 (0.0006) [2024-09-30 02:00:18,653][1157819] Updated weights for policy 0, policy_version 97038 (0.0006) [2024-09-30 02:00:19,168][1157819] Updated weights for policy 0, policy_version 97048 (0.0006) [2024-09-30 02:00:19,697][1157819] Updated weights for policy 0, policy_version 97058 (0.0006) [2024-09-30 02:00:20,228][1157819] Updated weights for policy 0, policy_version 97068 (0.0006) [2024-09-30 02:00:20,466][1157520] Fps is (10 sec: 79462.3, 60 sec: 82739.2, 300 sec: 79629.1). Total num frames: 397606912. Throughput: 0: 20566.5. Samples: 89389056. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 02:00:20,466][1157520] Avg episode reward: [(0, '53.941')] [2024-09-30 02:00:20,776][1157819] Updated weights for policy 0, policy_version 97078 (0.0006) [2024-09-30 02:00:21,290][1157819] Updated weights for policy 0, policy_version 97088 (0.0006) [2024-09-30 02:00:21,819][1157819] Updated weights for policy 0, policy_version 97098 (0.0006) [2024-09-30 02:00:22,365][1157819] Updated weights for policy 0, policy_version 97108 (0.0006) [2024-09-30 02:00:22,902][1157819] Updated weights for policy 0, policy_version 97118 (0.0006) [2024-09-30 02:00:23,466][1157819] Updated weights for policy 0, policy_version 97128 (0.0006) [2024-09-30 02:00:24,027][1157819] Updated weights for policy 0, policy_version 97138 (0.0006) [2024-09-30 02:00:24,604][1157819] Updated weights for policy 0, policy_version 97148 (0.0006) [2024-09-30 02:00:25,154][1157819] Updated weights for policy 0, policy_version 97158 (0.0006) [2024-09-30 02:00:25,466][1157520] Fps is (10 sec: 76595.4, 60 sec: 81374.0, 300 sec: 79573.5). Total num frames: 397979648. Throughput: 0: 20455.6. Samples: 89446664. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 02:00:25,466][1157520] Avg episode reward: [(0, '56.110')] [2024-09-30 02:00:25,740][1157819] Updated weights for policy 0, policy_version 97168 (0.0006) [2024-09-30 02:00:26,265][1157819] Updated weights for policy 0, policy_version 97178 (0.0006) [2024-09-30 02:00:26,767][1157819] Updated weights for policy 0, policy_version 97188 (0.0006) [2024-09-30 02:00:27,283][1157819] Updated weights for policy 0, policy_version 97198 (0.0006) [2024-09-30 02:00:27,792][1157819] Updated weights for policy 0, policy_version 97208 (0.0006) [2024-09-30 02:00:28,330][1157819] Updated weights for policy 0, policy_version 97218 (0.0006) [2024-09-30 02:00:28,848][1157819] Updated weights for policy 0, policy_version 97228 (0.0006) [2024-09-30 02:00:29,342][1157819] Updated weights for policy 0, policy_version 97238 (0.0006) [2024-09-30 02:00:29,862][1157819] Updated weights for policy 0, policy_version 97248 (0.0006) [2024-09-30 02:00:30,441][1157819] Updated weights for policy 0, policy_version 97258 (0.0006) [2024-09-30 02:00:30,466][1157520] Fps is (10 sec: 76185.5, 60 sec: 81100.9, 300 sec: 79587.4). Total num frames: 398368768. Throughput: 0: 20130.6. Samples: 89560568. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 02:00:30,466][1157520] Avg episode reward: [(0, '52.576')] [2024-09-30 02:00:30,957][1157819] Updated weights for policy 0, policy_version 97268 (0.0006) [2024-09-30 02:00:31,524][1157819] Updated weights for policy 0, policy_version 97278 (0.0006) [2024-09-30 02:00:32,079][1157819] Updated weights for policy 0, policy_version 97288 (0.0006) [2024-09-30 02:00:32,707][1157819] Updated weights for policy 0, policy_version 97298 (0.0006) [2024-09-30 02:00:33,024][1157736] Signal inference workers to stop experience collection... (6250 times) [2024-09-30 02:00:33,026][1157819] InferenceWorker_p0-w0: stopping experience collection (6250 times) [2024-09-30 02:00:33,034][1157736] Signal inference workers to resume experience collection... (6250 times) [2024-09-30 02:00:33,035][1157819] InferenceWorker_p0-w0: resuming experience collection (6250 times) [2024-09-30 02:00:33,322][1157819] Updated weights for policy 0, policy_version 97308 (0.0006) [2024-09-30 02:00:33,887][1157819] Updated weights for policy 0, policy_version 97318 (0.0006) [2024-09-30 02:00:34,466][1157819] Updated weights for policy 0, policy_version 97328 (0.0006) [2024-09-30 02:00:35,057][1157819] Updated weights for policy 0, policy_version 97338 (0.0006) [2024-09-30 02:00:35,466][1157520] Fps is (10 sec: 74137.2, 60 sec: 80213.4, 300 sec: 79490.2). Total num frames: 398721024. Throughput: 0: 19744.2. Samples: 89669412. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 02:00:35,466][1157520] Avg episode reward: [(0, '52.796')] [2024-09-30 02:00:35,638][1157819] Updated weights for policy 0, policy_version 97348 (0.0006) [2024-09-30 02:00:36,253][1157819] Updated weights for policy 0, policy_version 97358 (0.0006) [2024-09-30 02:00:36,824][1157819] Updated weights for policy 0, policy_version 97368 (0.0006) [2024-09-30 02:00:37,377][1157819] Updated weights for policy 0, policy_version 97378 (0.0006) [2024-09-30 02:00:37,970][1157819] Updated weights for policy 0, policy_version 97388 (0.0006) [2024-09-30 02:00:38,539][1157819] Updated weights for policy 0, policy_version 97398 (0.0006) [2024-09-30 02:00:39,130][1157819] Updated weights for policy 0, policy_version 97408 (0.0006) [2024-09-30 02:00:39,662][1157819] Updated weights for policy 0, policy_version 97418 (0.0006) [2024-09-30 02:00:40,223][1157819] Updated weights for policy 0, policy_version 97428 (0.0006) [2024-09-30 02:00:40,466][1157520] Fps is (10 sec: 71270.3, 60 sec: 79052.8, 300 sec: 79379.1). Total num frames: 399081472. Throughput: 0: 19488.6. Samples: 89722128. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 02:00:40,466][1157520] Avg episode reward: [(0, '55.144')] [2024-09-30 02:00:40,779][1157819] Updated weights for policy 0, policy_version 97438 (0.0006) [2024-09-30 02:00:41,334][1157819] Updated weights for policy 0, policy_version 97448 (0.0006) [2024-09-30 02:00:41,885][1157819] Updated weights for policy 0, policy_version 97458 (0.0006) [2024-09-30 02:00:42,443][1157819] Updated weights for policy 0, policy_version 97468 (0.0006) [2024-09-30 02:00:42,958][1157819] Updated weights for policy 0, policy_version 97478 (0.0006) [2024-09-30 02:00:43,518][1157819] Updated weights for policy 0, policy_version 97488 (0.0007) [2024-09-30 02:00:44,048][1157819] Updated weights for policy 0, policy_version 97498 (0.0006) [2024-09-30 02:00:44,592][1157819] Updated weights for policy 0, policy_version 97508 (0.0006) [2024-09-30 02:00:45,144][1157819] Updated weights for policy 0, policy_version 97518 (0.0006) [2024-09-30 02:00:45,466][1157520] Fps is (10 sec: 73728.0, 60 sec: 78370.0, 300 sec: 79323.6). Total num frames: 399458304. Throughput: 0: 19143.7. Samples: 89832972. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 02:00:45,466][1157520] Avg episode reward: [(0, '53.402')] [2024-09-30 02:00:45,660][1157819] Updated weights for policy 0, policy_version 97528 (0.0006) [2024-09-30 02:00:46,188][1157819] Updated weights for policy 0, policy_version 97538 (0.0006) [2024-09-30 02:00:46,769][1157819] Updated weights for policy 0, policy_version 97548 (0.0006) [2024-09-30 02:00:47,297][1157819] Updated weights for policy 0, policy_version 97558 (0.0006) [2024-09-30 02:00:47,841][1157819] Updated weights for policy 0, policy_version 97568 (0.0006) [2024-09-30 02:00:48,413][1157819] Updated weights for policy 0, policy_version 97578 (0.0006) [2024-09-30 02:00:48,961][1157819] Updated weights for policy 0, policy_version 97588 (0.0006) [2024-09-30 02:00:49,531][1157819] Updated weights for policy 0, policy_version 97598 (0.0006) [2024-09-30 02:00:50,063][1157819] Updated weights for policy 0, policy_version 97608 (0.0006) [2024-09-30 02:00:50,466][1157520] Fps is (10 sec: 74956.9, 60 sec: 77482.7, 300 sec: 79240.3). Total num frames: 399831040. Throughput: 0: 18837.5. Samples: 89945360. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 02:00:50,466][1157520] Avg episode reward: [(0, '52.903')] [2024-09-30 02:00:50,626][1157819] Updated weights for policy 0, policy_version 97618 (0.0006) [2024-09-30 02:00:51,195][1157819] Updated weights for policy 0, policy_version 97628 (0.0006) [2024-09-30 02:00:51,768][1157819] Updated weights for policy 0, policy_version 97638 (0.0006) [2024-09-30 02:00:52,334][1157819] Updated weights for policy 0, policy_version 97648 (0.0006) [2024-09-30 02:00:52,904][1157819] Updated weights for policy 0, policy_version 97658 (0.0006) [2024-09-30 02:00:53,448][1157819] Updated weights for policy 0, policy_version 97668 (0.0006) [2024-09-30 02:00:53,978][1157819] Updated weights for policy 0, policy_version 97678 (0.0006) [2024-09-30 02:00:54,573][1157819] Updated weights for policy 0, policy_version 97688 (0.0006) [2024-09-30 02:00:55,098][1157819] Updated weights for policy 0, policy_version 97698 (0.0006) [2024-09-30 02:00:55,466][1157520] Fps is (10 sec: 73728.3, 60 sec: 76526.9, 300 sec: 79129.2). Total num frames: 400195584. Throughput: 0: 18838.4. Samples: 90000112. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 02:00:55,466][1157520] Avg episode reward: [(0, '53.071')] [2024-09-30 02:00:55,710][1157819] Updated weights for policy 0, policy_version 97708 (0.0006) [2024-09-30 02:00:56,245][1157819] Updated weights for policy 0, policy_version 97718 (0.0006) [2024-09-30 02:00:56,805][1157819] Updated weights for policy 0, policy_version 97728 (0.0006) [2024-09-30 02:00:57,359][1157819] Updated weights for policy 0, policy_version 97738 (0.0006) [2024-09-30 02:00:57,943][1157819] Updated weights for policy 0, policy_version 97748 (0.0006) [2024-09-30 02:00:58,486][1157819] Updated weights for policy 0, policy_version 97758 (0.0006) [2024-09-30 02:00:59,048][1157819] Updated weights for policy 0, policy_version 97768 (0.0006) [2024-09-30 02:00:59,590][1157819] Updated weights for policy 0, policy_version 97778 (0.0006) [2024-09-30 02:01:00,147][1157819] Updated weights for policy 0, policy_version 97788 (0.0006) [2024-09-30 02:01:00,466][1157520] Fps is (10 sec: 72908.6, 60 sec: 75571.1, 300 sec: 79018.1). Total num frames: 400560128. Throughput: 0: 18644.2. Samples: 90109592. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 02:01:00,466][1157520] Avg episode reward: [(0, '53.113')] [2024-09-30 02:01:00,699][1157819] Updated weights for policy 0, policy_version 97798 (0.0006) [2024-09-30 02:01:01,225][1157819] Updated weights for policy 0, policy_version 97808 (0.0006) [2024-09-30 02:01:01,757][1157819] Updated weights for policy 0, policy_version 97818 (0.0006) [2024-09-30 02:01:02,257][1157819] Updated weights for policy 0, policy_version 97828 (0.0006) [2024-09-30 02:01:02,810][1157819] Updated weights for policy 0, policy_version 97838 (0.0006) [2024-09-30 02:01:03,340][1157819] Updated weights for policy 0, policy_version 97848 (0.0006) [2024-09-30 02:01:03,875][1157819] Updated weights for policy 0, policy_version 97858 (0.0006) [2024-09-30 02:01:04,422][1157819] Updated weights for policy 0, policy_version 97868 (0.0006) [2024-09-30 02:01:04,846][1157736] Signal inference workers to stop experience collection... (6300 times) [2024-09-30 02:01:04,847][1157736] Signal inference workers to resume experience collection... (6300 times) [2024-09-30 02:01:04,851][1157819] InferenceWorker_p0-w0: stopping experience collection (6300 times) [2024-09-30 02:01:04,852][1157819] InferenceWorker_p0-w0: resuming experience collection (6300 times) [2024-09-30 02:01:04,928][1157819] Updated weights for policy 0, policy_version 97878 (0.0006) [2024-09-30 02:01:05,425][1157819] Updated weights for policy 0, policy_version 97888 (0.0006) [2024-09-30 02:01:05,466][1157520] Fps is (10 sec: 75366.0, 60 sec: 75093.3, 300 sec: 78990.3). Total num frames: 400949248. Throughput: 0: 18568.8. Samples: 90224652. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 02:01:05,466][1157520] Avg episode reward: [(0, '52.163')] [2024-09-30 02:01:06,017][1157819] Updated weights for policy 0, policy_version 97898 (0.0006) [2024-09-30 02:01:06,575][1157819] Updated weights for policy 0, policy_version 97908 (0.0006) [2024-09-30 02:01:07,178][1157819] Updated weights for policy 0, policy_version 97918 (0.0006) [2024-09-30 02:01:07,682][1157819] Updated weights for policy 0, policy_version 97928 (0.0006) [2024-09-30 02:01:08,294][1157819] Updated weights for policy 0, policy_version 97938 (0.0006) [2024-09-30 02:01:08,787][1157819] Updated weights for policy 0, policy_version 97948 (0.0006) [2024-09-30 02:01:09,288][1157819] Updated weights for policy 0, policy_version 97958 (0.0006) [2024-09-30 02:01:09,792][1157819] Updated weights for policy 0, policy_version 97968 (0.0006) [2024-09-30 02:01:10,330][1157819] Updated weights for policy 0, policy_version 97978 (0.0006) [2024-09-30 02:01:10,466][1157520] Fps is (10 sec: 76595.2, 60 sec: 75229.8, 300 sec: 78934.8). Total num frames: 401326080. Throughput: 0: 18513.4. Samples: 90279768. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 02:01:10,466][1157520] Avg episode reward: [(0, '54.177')] [2024-09-30 02:01:10,830][1157819] Updated weights for policy 0, policy_version 97988 (0.0006) [2024-09-30 02:01:11,332][1157819] Updated weights for policy 0, policy_version 97998 (0.0006) [2024-09-30 02:01:11,841][1157819] Updated weights for policy 0, policy_version 98008 (0.0006) [2024-09-30 02:01:12,342][1157819] Updated weights for policy 0, policy_version 98018 (0.0006) [2024-09-30 02:01:12,839][1157819] Updated weights for policy 0, policy_version 98028 (0.0006) [2024-09-30 02:01:13,325][1157819] Updated weights for policy 0, policy_version 98038 (0.0006) [2024-09-30 02:01:13,802][1157819] Updated weights for policy 0, policy_version 98048 (0.0006) [2024-09-30 02:01:14,279][1157819] Updated weights for policy 0, policy_version 98058 (0.0006) [2024-09-30 02:01:14,774][1157819] Updated weights for policy 0, policy_version 98068 (0.0006) [2024-09-30 02:01:15,254][1157819] Updated weights for policy 0, policy_version 98078 (0.0006) [2024-09-30 02:01:15,466][1157520] Fps is (10 sec: 79462.3, 60 sec: 75502.9, 300 sec: 79018.1). Total num frames: 401743872. Throughput: 0: 18672.4. Samples: 90400828. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 02:01:15,466][1157520] Avg episode reward: [(0, '55.106')] [2024-09-30 02:01:15,749][1157819] Updated weights for policy 0, policy_version 98088 (0.0006) [2024-09-30 02:01:16,227][1157819] Updated weights for policy 0, policy_version 98098 (0.0006) [2024-09-30 02:01:16,716][1157819] Updated weights for policy 0, policy_version 98108 (0.0006) [2024-09-30 02:01:17,215][1157819] Updated weights for policy 0, policy_version 98118 (0.0006) [2024-09-30 02:01:17,706][1157819] Updated weights for policy 0, policy_version 98128 (0.0006) [2024-09-30 02:01:18,204][1157819] Updated weights for policy 0, policy_version 98138 (0.0006) [2024-09-30 02:01:18,705][1157819] Updated weights for policy 0, policy_version 98148 (0.0006) [2024-09-30 02:01:19,202][1157819] Updated weights for policy 0, policy_version 98158 (0.0006) [2024-09-30 02:01:19,715][1157819] Updated weights for policy 0, policy_version 98168 (0.0006) [2024-09-30 02:01:20,223][1157819] Updated weights for policy 0, policy_version 98178 (0.0006) [2024-09-30 02:01:20,466][1157520] Fps is (10 sec: 82739.1, 60 sec: 75776.0, 300 sec: 79045.9). Total num frames: 402153472. Throughput: 0: 19027.0. Samples: 90525628. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 02:01:20,466][1157520] Avg episode reward: [(0, '54.300')] [2024-09-30 02:01:20,760][1157819] Updated weights for policy 0, policy_version 98188 (0.0006) [2024-09-30 02:01:21,337][1157819] Updated weights for policy 0, policy_version 98198 (0.0006) [2024-09-30 02:01:21,845][1157819] Updated weights for policy 0, policy_version 98208 (0.0006) [2024-09-30 02:01:22,394][1157819] Updated weights for policy 0, policy_version 98218 (0.0006) [2024-09-30 02:01:22,925][1157819] Updated weights for policy 0, policy_version 98228 (0.0006) [2024-09-30 02:01:23,421][1157819] Updated weights for policy 0, policy_version 98238 (0.0006) [2024-09-30 02:01:23,941][1157819] Updated weights for policy 0, policy_version 98248 (0.0006) [2024-09-30 02:01:24,452][1157819] Updated weights for policy 0, policy_version 98258 (0.0006) [2024-09-30 02:01:24,983][1157819] Updated weights for policy 0, policy_version 98268 (0.0006) [2024-09-30 02:01:25,466][1157520] Fps is (10 sec: 79872.4, 60 sec: 76049.0, 300 sec: 78990.3). Total num frames: 402542592. Throughput: 0: 19130.9. Samples: 90583020. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 02:01:25,466][1157520] Avg episode reward: [(0, '54.062')] [2024-09-30 02:01:25,473][1157819] Updated weights for policy 0, policy_version 98278 (0.0006) [2024-09-30 02:01:25,997][1157819] Updated weights for policy 0, policy_version 98288 (0.0006) [2024-09-30 02:01:26,525][1157819] Updated weights for policy 0, policy_version 98298 (0.0006) [2024-09-30 02:01:27,071][1157819] Updated weights for policy 0, policy_version 98308 (0.0006) [2024-09-30 02:01:27,613][1157819] Updated weights for policy 0, policy_version 98318 (0.0006) [2024-09-30 02:01:28,112][1157819] Updated weights for policy 0, policy_version 98328 (0.0006) [2024-09-30 02:01:28,637][1157819] Updated weights for policy 0, policy_version 98338 (0.0006) [2024-09-30 02:01:29,165][1157819] Updated weights for policy 0, policy_version 98348 (0.0006) [2024-09-30 02:01:29,687][1157819] Updated weights for policy 0, policy_version 98358 (0.0006) [2024-09-30 02:01:30,226][1157819] Updated weights for policy 0, policy_version 98368 (0.0006) [2024-09-30 02:01:30,466][1157520] Fps is (10 sec: 77824.0, 60 sec: 76049.0, 300 sec: 78920.9). Total num frames: 402931712. Throughput: 0: 19292.1. Samples: 90701116. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 02:01:30,466][1157520] Avg episode reward: [(0, '55.674')] [2024-09-30 02:01:30,737][1157819] Updated weights for policy 0, policy_version 98378 (0.0007) [2024-09-30 02:01:31,283][1157819] Updated weights for policy 0, policy_version 98388 (0.0006) [2024-09-30 02:01:31,790][1157819] Updated weights for policy 0, policy_version 98398 (0.0006) [2024-09-30 02:01:32,340][1157819] Updated weights for policy 0, policy_version 98408 (0.0006) [2024-09-30 02:01:32,845][1157819] Updated weights for policy 0, policy_version 98418 (0.0006) [2024-09-30 02:01:33,338][1157819] Updated weights for policy 0, policy_version 98428 (0.0006) [2024-09-30 02:01:33,904][1157819] Updated weights for policy 0, policy_version 98438 (0.0006) [2024-09-30 02:01:34,387][1157819] Updated weights for policy 0, policy_version 98448 (0.0006) [2024-09-30 02:01:34,869][1157819] Updated weights for policy 0, policy_version 98458 (0.0006) [2024-09-30 02:01:35,347][1157819] Updated weights for policy 0, policy_version 98468 (0.0006) [2024-09-30 02:01:35,466][1157520] Fps is (10 sec: 78642.9, 60 sec: 76800.0, 300 sec: 78865.3). Total num frames: 403329024. Throughput: 0: 19416.5. Samples: 90819104. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 02:01:35,466][1157520] Avg episode reward: [(0, '53.804')] [2024-09-30 02:01:35,859][1157819] Updated weights for policy 0, policy_version 98478 (0.0006) [2024-09-30 02:01:36,340][1157819] Updated weights for policy 0, policy_version 98488 (0.0006) [2024-09-30 02:01:36,843][1157819] Updated weights for policy 0, policy_version 98498 (0.0006) [2024-09-30 02:01:36,941][1157736] Signal inference workers to stop experience collection... (6350 times) [2024-09-30 02:01:36,941][1157736] Signal inference workers to resume experience collection... (6350 times) [2024-09-30 02:01:36,947][1157819] InferenceWorker_p0-w0: stopping experience collection (6350 times) [2024-09-30 02:01:36,947][1157819] InferenceWorker_p0-w0: resuming experience collection (6350 times) [2024-09-30 02:01:37,335][1157819] Updated weights for policy 0, policy_version 98508 (0.0006) [2024-09-30 02:01:37,821][1157819] Updated weights for policy 0, policy_version 98518 (0.0006) [2024-09-30 02:01:38,339][1157819] Updated weights for policy 0, policy_version 98528 (0.0006) [2024-09-30 02:01:38,799][1157819] Updated weights for policy 0, policy_version 98538 (0.0006) [2024-09-30 02:01:39,284][1157819] Updated weights for policy 0, policy_version 98548 (0.0006) [2024-09-30 02:01:39,788][1157819] Updated weights for policy 0, policy_version 98558 (0.0006) [2024-09-30 02:01:40,269][1157819] Updated weights for policy 0, policy_version 98568 (0.0006) [2024-09-30 02:01:40,466][1157520] Fps is (10 sec: 81510.6, 60 sec: 77755.7, 300 sec: 78893.2). Total num frames: 403746816. Throughput: 0: 19590.4. Samples: 90881680. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 02:01:40,466][1157520] Avg episode reward: [(0, '52.030')] [2024-09-30 02:01:40,766][1157819] Updated weights for policy 0, policy_version 98578 (0.0006) [2024-09-30 02:01:41,254][1157819] Updated weights for policy 0, policy_version 98588 (0.0006) [2024-09-30 02:01:41,743][1157819] Updated weights for policy 0, policy_version 98598 (0.0006) [2024-09-30 02:01:42,241][1157819] Updated weights for policy 0, policy_version 98608 (0.0006) [2024-09-30 02:01:42,730][1157819] Updated weights for policy 0, policy_version 98618 (0.0006) [2024-09-30 02:01:43,212][1157819] Updated weights for policy 0, policy_version 98628 (0.0006) [2024-09-30 02:01:43,705][1157819] Updated weights for policy 0, policy_version 98638 (0.0006) [2024-09-30 02:01:44,194][1157819] Updated weights for policy 0, policy_version 98648 (0.0006) [2024-09-30 02:01:44,678][1157819] Updated weights for policy 0, policy_version 98658 (0.0006) [2024-09-30 02:01:45,179][1157819] Updated weights for policy 0, policy_version 98668 (0.0006) [2024-09-30 02:01:45,466][1157520] Fps is (10 sec: 83558.4, 60 sec: 78438.4, 300 sec: 78948.7). Total num frames: 404164608. Throughput: 0: 19944.9. Samples: 91007112. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 02:01:45,466][1157520] Avg episode reward: [(0, '52.834')] [2024-09-30 02:01:45,674][1157819] Updated weights for policy 0, policy_version 98678 (0.0006) [2024-09-30 02:01:46,158][1157819] Updated weights for policy 0, policy_version 98688 (0.0005) [2024-09-30 02:01:46,660][1157819] Updated weights for policy 0, policy_version 98698 (0.0006) [2024-09-30 02:01:47,160][1157819] Updated weights for policy 0, policy_version 98708 (0.0006) [2024-09-30 02:01:47,625][1157819] Updated weights for policy 0, policy_version 98718 (0.0006) [2024-09-30 02:01:48,134][1157819] Updated weights for policy 0, policy_version 98728 (0.0006) [2024-09-30 02:01:48,636][1157819] Updated weights for policy 0, policy_version 98738 (0.0006) [2024-09-30 02:01:49,123][1157819] Updated weights for policy 0, policy_version 98748 (0.0006) [2024-09-30 02:01:49,644][1157819] Updated weights for policy 0, policy_version 98758 (0.0006) [2024-09-30 02:01:50,171][1157819] Updated weights for policy 0, policy_version 98768 (0.0006) [2024-09-30 02:01:50,466][1157520] Fps is (10 sec: 82739.2, 60 sec: 79052.8, 300 sec: 79032.0). Total num frames: 404574208. Throughput: 0: 20141.8. Samples: 91131032. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 02:01:50,466][1157520] Avg episode reward: [(0, '53.447')] [2024-09-30 02:01:50,689][1157819] Updated weights for policy 0, policy_version 98778 (0.0006) [2024-09-30 02:01:51,189][1157819] Updated weights for policy 0, policy_version 98788 (0.0006) [2024-09-30 02:01:51,732][1157819] Updated weights for policy 0, policy_version 98798 (0.0006) [2024-09-30 02:01:52,225][1157819] Updated weights for policy 0, policy_version 98808 (0.0006) [2024-09-30 02:01:52,755][1157819] Updated weights for policy 0, policy_version 98818 (0.0006) [2024-09-30 02:01:53,293][1157819] Updated weights for policy 0, policy_version 98828 (0.0006) [2024-09-30 02:01:53,806][1157819] Updated weights for policy 0, policy_version 98838 (0.0006) [2024-09-30 02:01:54,296][1157819] Updated weights for policy 0, policy_version 98848 (0.0006) [2024-09-30 02:01:54,824][1157819] Updated weights for policy 0, policy_version 98858 (0.0006) [2024-09-30 02:01:55,333][1157819] Updated weights for policy 0, policy_version 98868 (0.0006) [2024-09-30 02:01:55,466][1157520] Fps is (10 sec: 80691.9, 60 sec: 79599.0, 300 sec: 79032.0). Total num frames: 404971520. Throughput: 0: 20229.4. Samples: 91190088. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 02:01:55,466][1157520] Avg episode reward: [(0, '54.021')] [2024-09-30 02:01:55,883][1157819] Updated weights for policy 0, policy_version 98878 (0.0006) [2024-09-30 02:01:56,375][1157819] Updated weights for policy 0, policy_version 98888 (0.0006) [2024-09-30 02:01:56,874][1157819] Updated weights for policy 0, policy_version 98898 (0.0006) [2024-09-30 02:01:57,391][1157819] Updated weights for policy 0, policy_version 98908 (0.0006) [2024-09-30 02:01:57,910][1157819] Updated weights for policy 0, policy_version 98918 (0.0006) [2024-09-30 02:01:58,413][1157819] Updated weights for policy 0, policy_version 98928 (0.0006) [2024-09-30 02:01:58,917][1157819] Updated weights for policy 0, policy_version 98938 (0.0006) [2024-09-30 02:01:59,437][1157819] Updated weights for policy 0, policy_version 98948 (0.0006) [2024-09-30 02:01:59,991][1157819] Updated weights for policy 0, policy_version 98958 (0.0006) [2024-09-30 02:02:00,466][1157520] Fps is (10 sec: 79462.7, 60 sec: 80145.1, 300 sec: 79045.9). Total num frames: 405368832. Throughput: 0: 20205.7. Samples: 91310084. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 02:02:00,466][1157520] Avg episode reward: [(0, '54.418')] [2024-09-30 02:02:00,506][1157819] Updated weights for policy 0, policy_version 98968 (0.0006) [2024-09-30 02:02:01,015][1157819] Updated weights for policy 0, policy_version 98978 (0.0006) [2024-09-30 02:02:01,530][1157819] Updated weights for policy 0, policy_version 98988 (0.0006) [2024-09-30 02:02:02,044][1157819] Updated weights for policy 0, policy_version 98998 (0.0006) [2024-09-30 02:02:02,574][1157819] Updated weights for policy 0, policy_version 99008 (0.0006) [2024-09-30 02:02:03,077][1157819] Updated weights for policy 0, policy_version 99018 (0.0006) [2024-09-30 02:02:03,614][1157819] Updated weights for policy 0, policy_version 99028 (0.0006) [2024-09-30 02:02:04,153][1157819] Updated weights for policy 0, policy_version 99038 (0.0006) [2024-09-30 02:02:04,705][1157819] Updated weights for policy 0, policy_version 99048 (0.0006) [2024-09-30 02:02:05,260][1157819] Updated weights for policy 0, policy_version 99058 (0.0006) [2024-09-30 02:02:05,466][1157520] Fps is (10 sec: 78233.2, 60 sec: 80076.8, 300 sec: 79004.2). Total num frames: 405753856. Throughput: 0: 20034.9. Samples: 91427200. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 02:02:05,466][1157520] Avg episode reward: [(0, '56.140')] [2024-09-30 02:02:05,470][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000099061_405753856.pth... [2024-09-30 02:02:05,533][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000094419_386740224.pth [2024-09-30 02:02:05,857][1157819] Updated weights for policy 0, policy_version 99068 (0.0006) [2024-09-30 02:02:06,384][1157819] Updated weights for policy 0, policy_version 99078 (0.0006) [2024-09-30 02:02:06,908][1157819] Updated weights for policy 0, policy_version 99088 (0.0006) [2024-09-30 02:02:07,435][1157819] Updated weights for policy 0, policy_version 99098 (0.0006) [2024-09-30 02:02:07,957][1157819] Updated weights for policy 0, policy_version 99108 (0.0006) [2024-09-30 02:02:08,448][1157819] Updated weights for policy 0, policy_version 99118 (0.0006) [2024-09-30 02:02:08,958][1157819] Updated weights for policy 0, policy_version 99128 (0.0006) [2024-09-30 02:02:09,503][1157819] Updated weights for policy 0, policy_version 99138 (0.0006) [2024-09-30 02:02:09,942][1157819] Updated weights for policy 0, policy_version 99148 (0.0006) [2024-09-30 02:02:10,466][1157520] Fps is (10 sec: 77823.7, 60 sec: 80349.9, 300 sec: 78948.7). Total num frames: 406147072. Throughput: 0: 20010.1. Samples: 91483472. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 02:02:10,466][1157520] Avg episode reward: [(0, '55.582')] [2024-09-30 02:02:10,481][1157819] Updated weights for policy 0, policy_version 99158 (0.0006) [2024-09-30 02:02:10,947][1157819] Updated weights for policy 0, policy_version 99168 (0.0006) [2024-09-30 02:02:11,437][1157819] Updated weights for policy 0, policy_version 99178 (0.0006) [2024-09-30 02:02:11,926][1157819] Updated weights for policy 0, policy_version 99188 (0.0006) [2024-09-30 02:02:12,415][1157819] Updated weights for policy 0, policy_version 99198 (0.0006) [2024-09-30 02:02:12,920][1157819] Updated weights for policy 0, policy_version 99208 (0.0006) [2024-09-30 02:02:13,408][1157819] Updated weights for policy 0, policy_version 99218 (0.0006) [2024-09-30 02:02:13,891][1157819] Updated weights for policy 0, policy_version 99228 (0.0006) [2024-09-30 02:02:14,405][1157819] Updated weights for policy 0, policy_version 99238 (0.0006) [2024-09-30 02:02:14,919][1157819] Updated weights for policy 0, policy_version 99248 (0.0006) [2024-09-30 02:02:15,420][1157819] Updated weights for policy 0, policy_version 99258 (0.0006) [2024-09-30 02:02:15,466][1157520] Fps is (10 sec: 80691.3, 60 sec: 80281.7, 300 sec: 79018.1). Total num frames: 406560768. Throughput: 0: 20130.0. Samples: 91606964. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 02:02:15,466][1157520] Avg episode reward: [(0, '55.599')] [2024-09-30 02:02:15,934][1157819] Updated weights for policy 0, policy_version 99268 (0.0006) [2024-09-30 02:02:16,462][1157819] Updated weights for policy 0, policy_version 99278 (0.0006) [2024-09-30 02:02:17,023][1157819] Updated weights for policy 0, policy_version 99288 (0.0006) [2024-09-30 02:02:17,544][1157819] Updated weights for policy 0, policy_version 99298 (0.0006) [2024-09-30 02:02:18,088][1157819] Updated weights for policy 0, policy_version 99308 (0.0007) [2024-09-30 02:02:18,633][1157819] Updated weights for policy 0, policy_version 99318 (0.0006) [2024-09-30 02:02:19,160][1157819] Updated weights for policy 0, policy_version 99328 (0.0006) [2024-09-30 02:02:19,689][1157819] Updated weights for policy 0, policy_version 99338 (0.0006) [2024-09-30 02:02:20,198][1157819] Updated weights for policy 0, policy_version 99348 (0.0006) [2024-09-30 02:02:20,466][1157520] Fps is (10 sec: 80281.7, 60 sec: 79940.3, 300 sec: 78990.3). Total num frames: 406949888. Throughput: 0: 20114.7. Samples: 91724264. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 02:02:20,466][1157520] Avg episode reward: [(0, '52.436')] [2024-09-30 02:02:20,711][1157819] Updated weights for policy 0, policy_version 99358 (0.0006) [2024-09-30 02:02:21,221][1157819] Updated weights for policy 0, policy_version 99368 (0.0006) [2024-09-30 02:02:21,736][1157819] Updated weights for policy 0, policy_version 99378 (0.0006) [2024-09-30 02:02:22,281][1157819] Updated weights for policy 0, policy_version 99388 (0.0006) [2024-09-30 02:02:22,831][1157736] Signal inference workers to stop experience collection... (6400 times) [2024-09-30 02:02:22,833][1157736] Signal inference workers to resume experience collection... (6400 times) [2024-09-30 02:02:22,833][1157819] InferenceWorker_p0-w0: stopping experience collection (6400 times) [2024-09-30 02:02:22,835][1157819] Updated weights for policy 0, policy_version 99398 (0.0006) [2024-09-30 02:02:22,837][1157819] InferenceWorker_p0-w0: resuming experience collection (6400 times) [2024-09-30 02:02:23,326][1157819] Updated weights for policy 0, policy_version 99408 (0.0006) [2024-09-30 02:02:23,833][1157819] Updated weights for policy 0, policy_version 99418 (0.0006) [2024-09-30 02:02:24,373][1157819] Updated weights for policy 0, policy_version 99428 (0.0006) [2024-09-30 02:02:24,904][1157819] Updated weights for policy 0, policy_version 99438 (0.0006) [2024-09-30 02:02:25,441][1157819] Updated weights for policy 0, policy_version 99448 (0.0006) [2024-09-30 02:02:25,466][1157520] Fps is (10 sec: 77824.0, 60 sec: 79940.3, 300 sec: 78990.3). Total num frames: 407339008. Throughput: 0: 20038.5. Samples: 91783412. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 02:02:25,466][1157520] Avg episode reward: [(0, '55.097')] [2024-09-30 02:02:25,969][1157819] Updated weights for policy 0, policy_version 99458 (0.0006) [2024-09-30 02:02:26,520][1157819] Updated weights for policy 0, policy_version 99468 (0.0006) [2024-09-30 02:02:27,031][1157819] Updated weights for policy 0, policy_version 99478 (0.0006) [2024-09-30 02:02:27,611][1157819] Updated weights for policy 0, policy_version 99488 (0.0006) [2024-09-30 02:02:28,140][1157819] Updated weights for policy 0, policy_version 99498 (0.0006) [2024-09-30 02:02:28,664][1157819] Updated weights for policy 0, policy_version 99508 (0.0006) [2024-09-30 02:02:29,238][1157819] Updated weights for policy 0, policy_version 99518 (0.0006) [2024-09-30 02:02:29,757][1157819] Updated weights for policy 0, policy_version 99528 (0.0006) [2024-09-30 02:02:30,301][1157819] Updated weights for policy 0, policy_version 99538 (0.0006) [2024-09-30 02:02:30,466][1157520] Fps is (10 sec: 76595.1, 60 sec: 79735.5, 300 sec: 78948.7). Total num frames: 407715840. Throughput: 0: 19801.2. Samples: 91898164. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 02:02:30,466][1157520] Avg episode reward: [(0, '54.565')] [2024-09-30 02:02:30,875][1157819] Updated weights for policy 0, policy_version 99548 (0.0006) [2024-09-30 02:02:31,438][1157819] Updated weights for policy 0, policy_version 99558 (0.0006) [2024-09-30 02:02:32,018][1157819] Updated weights for policy 0, policy_version 99568 (0.0006) [2024-09-30 02:02:32,566][1157819] Updated weights for policy 0, policy_version 99578 (0.0006) [2024-09-30 02:02:33,158][1157819] Updated weights for policy 0, policy_version 99588 (0.0006) [2024-09-30 02:02:33,723][1157819] Updated weights for policy 0, policy_version 99598 (0.0006) [2024-09-30 02:02:34,302][1157819] Updated weights for policy 0, policy_version 99608 (0.0006) [2024-09-30 02:02:34,852][1157819] Updated weights for policy 0, policy_version 99618 (0.0006) [2024-09-30 02:02:35,353][1157819] Updated weights for policy 0, policy_version 99628 (0.0006) [2024-09-30 02:02:35,466][1157520] Fps is (10 sec: 74137.6, 60 sec: 79189.4, 300 sec: 78907.0). Total num frames: 408080384. Throughput: 0: 19491.4. Samples: 92008144. Policy #0 lag: (min: 0.0, avg: 2.7, max: 5.0) [2024-09-30 02:02:35,466][1157520] Avg episode reward: [(0, '53.562')] [2024-09-30 02:02:35,896][1157819] Updated weights for policy 0, policy_version 99638 (0.0006) [2024-09-30 02:02:36,460][1157819] Updated weights for policy 0, policy_version 99648 (0.0006) [2024-09-30 02:02:36,985][1157819] Updated weights for policy 0, policy_version 99658 (0.0006) [2024-09-30 02:02:37,539][1157819] Updated weights for policy 0, policy_version 99668 (0.0006) [2024-09-30 02:02:38,115][1157819] Updated weights for policy 0, policy_version 99678 (0.0006) [2024-09-30 02:02:38,625][1157819] Updated weights for policy 0, policy_version 99688 (0.0006) [2024-09-30 02:02:39,208][1157819] Updated weights for policy 0, policy_version 99698 (0.0006) [2024-09-30 02:02:39,742][1157819] Updated weights for policy 0, policy_version 99708 (0.0006) [2024-09-30 02:02:40,277][1157819] Updated weights for policy 0, policy_version 99718 (0.0006) [2024-09-30 02:02:40,466][1157520] Fps is (10 sec: 74137.7, 60 sec: 78506.7, 300 sec: 78907.1). Total num frames: 408457216. Throughput: 0: 19427.3. Samples: 92064316. Policy #0 lag: (min: 0.0, avg: 2.7, max: 5.0) [2024-09-30 02:02:40,466][1157520] Avg episode reward: [(0, '54.530')] [2024-09-30 02:02:40,816][1157819] Updated weights for policy 0, policy_version 99728 (0.0006) [2024-09-30 02:02:41,346][1157819] Updated weights for policy 0, policy_version 99738 (0.0006) [2024-09-30 02:02:41,915][1157819] Updated weights for policy 0, policy_version 99748 (0.0006) [2024-09-30 02:02:42,472][1157819] Updated weights for policy 0, policy_version 99758 (0.0006) [2024-09-30 02:02:42,982][1157819] Updated weights for policy 0, policy_version 99768 (0.0006) [2024-09-30 02:02:43,528][1157819] Updated weights for policy 0, policy_version 99778 (0.0006) [2024-09-30 02:02:44,087][1157819] Updated weights for policy 0, policy_version 99788 (0.0006) [2024-09-30 02:02:44,598][1157819] Updated weights for policy 0, policy_version 99798 (0.0006) [2024-09-30 02:02:45,146][1157819] Updated weights for policy 0, policy_version 99808 (0.0006) [2024-09-30 02:02:45,466][1157520] Fps is (10 sec: 75776.1, 60 sec: 77892.3, 300 sec: 78879.3). Total num frames: 408838144. Throughput: 0: 19278.4. Samples: 92177612. Policy #0 lag: (min: 0.0, avg: 2.7, max: 5.0) [2024-09-30 02:02:45,466][1157520] Avg episode reward: [(0, '51.562')] [2024-09-30 02:02:45,708][1157819] Updated weights for policy 0, policy_version 99818 (0.0006) [2024-09-30 02:02:46,240][1157819] Updated weights for policy 0, policy_version 99828 (0.0006) [2024-09-30 02:02:46,794][1157819] Updated weights for policy 0, policy_version 99838 (0.0006) [2024-09-30 02:02:47,355][1157819] Updated weights for policy 0, policy_version 99848 (0.0006) [2024-09-30 02:02:47,849][1157819] Updated weights for policy 0, policy_version 99858 (0.0006) [2024-09-30 02:02:48,349][1157819] Updated weights for policy 0, policy_version 99868 (0.0006) [2024-09-30 02:02:48,874][1157819] Updated weights for policy 0, policy_version 99878 (0.0006) [2024-09-30 02:02:49,387][1157819] Updated weights for policy 0, policy_version 99888 (0.0006) [2024-09-30 02:02:49,902][1157819] Updated weights for policy 0, policy_version 99898 (0.0006) [2024-09-30 02:02:50,414][1157819] Updated weights for policy 0, policy_version 99908 (0.0006) [2024-09-30 02:02:50,466][1157520] Fps is (10 sec: 76595.3, 60 sec: 77482.7, 300 sec: 78851.5). Total num frames: 409223168. Throughput: 0: 19248.3. Samples: 92293372. Policy #0 lag: (min: 0.0, avg: 2.7, max: 5.0) [2024-09-30 02:02:50,466][1157520] Avg episode reward: [(0, '55.296')] [2024-09-30 02:02:50,964][1157819] Updated weights for policy 0, policy_version 99918 (0.0006) [2024-09-30 02:02:50,969][1157736] Signal inference workers to stop experience collection... (6450 times) [2024-09-30 02:02:50,970][1157736] Signal inference workers to resume experience collection... (6450 times) [2024-09-30 02:02:50,973][1157819] InferenceWorker_p0-w0: stopping experience collection (6450 times) [2024-09-30 02:02:50,974][1157819] InferenceWorker_p0-w0: resuming experience collection (6450 times) [2024-09-30 02:02:51,479][1157819] Updated weights for policy 0, policy_version 99928 (0.0006) [2024-09-30 02:02:51,982][1157819] Updated weights for policy 0, policy_version 99938 (0.0006) [2024-09-30 02:02:52,506][1157819] Updated weights for policy 0, policy_version 99948 (0.0006) [2024-09-30 02:02:53,008][1157819] Updated weights for policy 0, policy_version 99958 (0.0006) [2024-09-30 02:02:53,526][1157819] Updated weights for policy 0, policy_version 99968 (0.0006) [2024-09-30 02:02:54,019][1157819] Updated weights for policy 0, policy_version 99978 (0.0006) [2024-09-30 02:02:54,533][1157819] Updated weights for policy 0, policy_version 99988 (0.0006) [2024-09-30 02:02:55,037][1157819] Updated weights for policy 0, policy_version 99998 (0.0006) [2024-09-30 02:02:55,466][1157520] Fps is (10 sec: 78642.8, 60 sec: 77550.8, 300 sec: 78934.8). Total num frames: 409624576. Throughput: 0: 19315.2. Samples: 92352656. Policy #0 lag: (min: 0.0, avg: 2.7, max: 5.0) [2024-09-30 02:02:55,466][1157520] Avg episode reward: [(0, '56.090')] [2024-09-30 02:02:55,528][1157819] Updated weights for policy 0, policy_version 100008 (0.0006) [2024-09-30 02:02:56,004][1157819] Updated weights for policy 0, policy_version 100018 (0.0006) [2024-09-30 02:02:56,506][1157819] Updated weights for policy 0, policy_version 100028 (0.0006) [2024-09-30 02:02:57,020][1157819] Updated weights for policy 0, policy_version 100038 (0.0006) [2024-09-30 02:02:57,488][1157819] Updated weights for policy 0, policy_version 100048 (0.0006) [2024-09-30 02:02:57,980][1157819] Updated weights for policy 0, policy_version 100058 (0.0006) [2024-09-30 02:02:58,482][1157819] Updated weights for policy 0, policy_version 100068 (0.0006) [2024-09-30 02:02:58,961][1157819] Updated weights for policy 0, policy_version 100078 (0.0006) [2024-09-30 02:02:59,466][1157819] Updated weights for policy 0, policy_version 100088 (0.0006) [2024-09-30 02:02:59,962][1157819] Updated weights for policy 0, policy_version 100098 (0.0006) [2024-09-30 02:03:00,466][1157520] Fps is (10 sec: 81510.4, 60 sec: 77824.0, 300 sec: 78976.5). Total num frames: 410038272. Throughput: 0: 19315.4. Samples: 92476156. Policy #0 lag: (min: 0.0, avg: 2.7, max: 5.0) [2024-09-30 02:03:00,466][1157520] Avg episode reward: [(0, '55.103')] [2024-09-30 02:03:00,469][1157819] Updated weights for policy 0, policy_version 100108 (0.0006) [2024-09-30 02:03:00,958][1157819] Updated weights for policy 0, policy_version 100118 (0.0006) [2024-09-30 02:03:01,462][1157819] Updated weights for policy 0, policy_version 100128 (0.0006) [2024-09-30 02:03:01,970][1157819] Updated weights for policy 0, policy_version 100138 (0.0006) [2024-09-30 02:03:02,485][1157819] Updated weights for policy 0, policy_version 100148 (0.0006) [2024-09-30 02:03:02,987][1157819] Updated weights for policy 0, policy_version 100158 (0.0006) [2024-09-30 02:03:03,513][1157819] Updated weights for policy 0, policy_version 100168 (0.0006) [2024-09-30 02:03:04,044][1157819] Updated weights for policy 0, policy_version 100178 (0.0006) [2024-09-30 02:03:04,573][1157819] Updated weights for policy 0, policy_version 100188 (0.0006) [2024-09-30 02:03:05,062][1157819] Updated weights for policy 0, policy_version 100198 (0.0006) [2024-09-30 02:03:05,466][1157520] Fps is (10 sec: 81510.9, 60 sec: 78097.1, 300 sec: 78907.0). Total num frames: 410439680. Throughput: 0: 19399.2. Samples: 92597228. Policy #0 lag: (min: 0.0, avg: 2.7, max: 5.0) [2024-09-30 02:03:05,466][1157520] Avg episode reward: [(0, '55.165')] [2024-09-30 02:03:05,582][1157819] Updated weights for policy 0, policy_version 100208 (0.0006) [2024-09-30 02:03:06,092][1157819] Updated weights for policy 0, policy_version 100218 (0.0006) [2024-09-30 02:03:06,580][1157819] Updated weights for policy 0, policy_version 100228 (0.0006) [2024-09-30 02:03:07,078][1157819] Updated weights for policy 0, policy_version 100238 (0.0006) [2024-09-30 02:03:07,590][1157819] Updated weights for policy 0, policy_version 100248 (0.0006) [2024-09-30 02:03:08,156][1157819] Updated weights for policy 0, policy_version 100258 (0.0006) [2024-09-30 02:03:08,689][1157819] Updated weights for policy 0, policy_version 100268 (0.0006) [2024-09-30 02:03:09,220][1157819] Updated weights for policy 0, policy_version 100278 (0.0006) [2024-09-30 02:03:09,781][1157819] Updated weights for policy 0, policy_version 100288 (0.0006) [2024-09-30 02:03:10,041][1157736] Signal inference workers to stop experience collection... (6500 times) [2024-09-30 02:03:10,042][1157736] Signal inference workers to resume experience collection... (6500 times) [2024-09-30 02:03:10,045][1157819] InferenceWorker_p0-w0: stopping experience collection (6500 times) [2024-09-30 02:03:10,045][1157819] InferenceWorker_p0-w0: resuming experience collection (6500 times) [2024-09-30 02:03:10,336][1157819] Updated weights for policy 0, policy_version 100298 (0.0006) [2024-09-30 02:03:10,466][1157520] Fps is (10 sec: 79052.8, 60 sec: 78028.8, 300 sec: 78865.4). Total num frames: 410828800. Throughput: 0: 19425.8. Samples: 92657572. Policy #0 lag: (min: 0.0, avg: 2.7, max: 5.0) [2024-09-30 02:03:10,466][1157520] Avg episode reward: [(0, '54.626')] [2024-09-30 02:03:10,926][1157819] Updated weights for policy 0, policy_version 100308 (0.0006) [2024-09-30 02:03:11,456][1157819] Updated weights for policy 0, policy_version 100318 (0.0006) [2024-09-30 02:03:12,022][1157819] Updated weights for policy 0, policy_version 100328 (0.0006) [2024-09-30 02:03:12,575][1157819] Updated weights for policy 0, policy_version 100338 (0.0006) [2024-09-30 02:03:13,095][1157819] Updated weights for policy 0, policy_version 100348 (0.0006) [2024-09-30 02:03:13,659][1157819] Updated weights for policy 0, policy_version 100358 (0.0006) [2024-09-30 02:03:14,185][1157819] Updated weights for policy 0, policy_version 100368 (0.0006) [2024-09-30 02:03:14,716][1157819] Updated weights for policy 0, policy_version 100378 (0.0006) [2024-09-30 02:03:15,274][1157819] Updated weights for policy 0, policy_version 100388 (0.0006) [2024-09-30 02:03:15,466][1157520] Fps is (10 sec: 76185.4, 60 sec: 77346.1, 300 sec: 78740.4). Total num frames: 411201536. Throughput: 0: 19355.0. Samples: 92769140. Policy #0 lag: (min: 0.0, avg: 2.7, max: 5.0) [2024-09-30 02:03:15,466][1157520] Avg episode reward: [(0, '53.157')] [2024-09-30 02:03:15,810][1157819] Updated weights for policy 0, policy_version 100398 (0.0006) [2024-09-30 02:03:16,359][1157819] Updated weights for policy 0, policy_version 100408 (0.0006) [2024-09-30 02:03:16,897][1157819] Updated weights for policy 0, policy_version 100418 (0.0006) [2024-09-30 02:03:17,416][1157819] Updated weights for policy 0, policy_version 100428 (0.0006) [2024-09-30 02:03:17,961][1157819] Updated weights for policy 0, policy_version 100438 (0.0006) [2024-09-30 02:03:18,493][1157819] Updated weights for policy 0, policy_version 100448 (0.0006) [2024-09-30 02:03:19,016][1157819] Updated weights for policy 0, policy_version 100458 (0.0006) [2024-09-30 02:03:19,515][1157819] Updated weights for policy 0, policy_version 100468 (0.0006) [2024-09-30 02:03:20,049][1157819] Updated weights for policy 0, policy_version 100478 (0.0006) [2024-09-30 02:03:20,466][1157520] Fps is (10 sec: 75776.0, 60 sec: 77277.9, 300 sec: 78823.8). Total num frames: 411586560. Throughput: 0: 19465.5. Samples: 92884092. Policy #0 lag: (min: 0.0, avg: 2.7, max: 5.0) [2024-09-30 02:03:20,466][1157520] Avg episode reward: [(0, '53.137')] [2024-09-30 02:03:20,574][1157819] Updated weights for policy 0, policy_version 100488 (0.0006) [2024-09-30 02:03:21,104][1157819] Updated weights for policy 0, policy_version 100498 (0.0006) [2024-09-30 02:03:21,602][1157819] Updated weights for policy 0, policy_version 100508 (0.0006) [2024-09-30 02:03:22,095][1157819] Updated weights for policy 0, policy_version 100518 (0.0006) [2024-09-30 02:03:22,617][1157819] Updated weights for policy 0, policy_version 100528 (0.0006) [2024-09-30 02:03:23,139][1157819] Updated weights for policy 0, policy_version 100538 (0.0006) [2024-09-30 02:03:23,673][1157819] Updated weights for policy 0, policy_version 100548 (0.0006) [2024-09-30 02:03:24,169][1157819] Updated weights for policy 0, policy_version 100558 (0.0006) [2024-09-30 02:03:24,660][1157819] Updated weights for policy 0, policy_version 100568 (0.0006) [2024-09-30 02:03:25,220][1157819] Updated weights for policy 0, policy_version 100578 (0.0006) [2024-09-30 02:03:25,466][1157520] Fps is (10 sec: 78233.6, 60 sec: 77414.4, 300 sec: 78893.1). Total num frames: 411983872. Throughput: 0: 19546.5. Samples: 92943908. Policy #0 lag: (min: 0.0, avg: 2.7, max: 5.0) [2024-09-30 02:03:25,466][1157520] Avg episode reward: [(0, '56.458')] [2024-09-30 02:03:25,727][1157819] Updated weights for policy 0, policy_version 100588 (0.0006) [2024-09-30 02:03:26,205][1157819] Updated weights for policy 0, policy_version 100598 (0.0006) [2024-09-30 02:03:26,701][1157819] Updated weights for policy 0, policy_version 100608 (0.0006) [2024-09-30 02:03:27,186][1157819] Updated weights for policy 0, policy_version 100618 (0.0006) [2024-09-30 02:03:27,656][1157819] Updated weights for policy 0, policy_version 100628 (0.0006) [2024-09-30 02:03:28,143][1157819] Updated weights for policy 0, policy_version 100638 (0.0006) [2024-09-30 02:03:28,629][1157819] Updated weights for policy 0, policy_version 100648 (0.0006) [2024-09-30 02:03:29,115][1157819] Updated weights for policy 0, policy_version 100658 (0.0006) [2024-09-30 02:03:29,611][1157819] Updated weights for policy 0, policy_version 100668 (0.0006) [2024-09-30 02:03:30,098][1157819] Updated weights for policy 0, policy_version 100678 (0.0006) [2024-09-30 02:03:30,466][1157520] Fps is (10 sec: 81510.1, 60 sec: 78097.0, 300 sec: 79059.8). Total num frames: 412401664. Throughput: 0: 19751.6. Samples: 93066436. Policy #0 lag: (min: 0.0, avg: 2.7, max: 5.0) [2024-09-30 02:03:30,466][1157520] Avg episode reward: [(0, '56.292')] [2024-09-30 02:03:30,587][1157819] Updated weights for policy 0, policy_version 100688 (0.0006) [2024-09-30 02:03:31,084][1157819] Updated weights for policy 0, policy_version 100698 (0.0006) [2024-09-30 02:03:31,570][1157819] Updated weights for policy 0, policy_version 100708 (0.0006) [2024-09-30 02:03:32,092][1157819] Updated weights for policy 0, policy_version 100718 (0.0006) [2024-09-30 02:03:32,600][1157819] Updated weights for policy 0, policy_version 100728 (0.0006) [2024-09-30 02:03:33,119][1157819] Updated weights for policy 0, policy_version 100738 (0.0006) [2024-09-30 02:03:33,617][1157819] Updated weights for policy 0, policy_version 100748 (0.0006) [2024-09-30 02:03:34,125][1157819] Updated weights for policy 0, policy_version 100758 (0.0006) [2024-09-30 02:03:34,634][1157819] Updated weights for policy 0, policy_version 100768 (0.0006) [2024-09-30 02:03:35,158][1157819] Updated weights for policy 0, policy_version 100778 (0.0006) [2024-09-30 02:03:35,466][1157520] Fps is (10 sec: 82739.0, 60 sec: 78848.0, 300 sec: 79198.6). Total num frames: 412811264. Throughput: 0: 19908.7. Samples: 93189264. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 02:03:35,466][1157520] Avg episode reward: [(0, '54.103')] [2024-09-30 02:03:35,633][1157819] Updated weights for policy 0, policy_version 100788 (0.0006) [2024-09-30 02:03:36,072][1157736] Signal inference workers to stop experience collection... (6550 times) [2024-09-30 02:03:36,072][1157736] Signal inference workers to resume experience collection... (6550 times) [2024-09-30 02:03:36,078][1157819] InferenceWorker_p0-w0: stopping experience collection (6550 times) [2024-09-30 02:03:36,078][1157819] InferenceWorker_p0-w0: resuming experience collection (6550 times) [2024-09-30 02:03:36,087][1157819] Updated weights for policy 0, policy_version 100798 (0.0006) [2024-09-30 02:03:36,579][1157819] Updated weights for policy 0, policy_version 100808 (0.0006) [2024-09-30 02:03:37,073][1157819] Updated weights for policy 0, policy_version 100818 (0.0006) [2024-09-30 02:03:37,550][1157819] Updated weights for policy 0, policy_version 100828 (0.0006) [2024-09-30 02:03:38,043][1157819] Updated weights for policy 0, policy_version 100838 (0.0006) [2024-09-30 02:03:38,527][1157819] Updated weights for policy 0, policy_version 100848 (0.0006) [2024-09-30 02:03:39,013][1157819] Updated weights for policy 0, policy_version 100858 (0.0006) [2024-09-30 02:03:39,498][1157819] Updated weights for policy 0, policy_version 100868 (0.0006) [2024-09-30 02:03:39,983][1157819] Updated weights for policy 0, policy_version 100878 (0.0006) [2024-09-30 02:03:40,466][1157520] Fps is (10 sec: 83148.4, 60 sec: 79598.8, 300 sec: 79393.0). Total num frames: 413233152. Throughput: 0: 19996.5. Samples: 93252500. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 02:03:40,466][1157520] Avg episode reward: [(0, '56.350')] [2024-09-30 02:03:40,467][1157819] Updated weights for policy 0, policy_version 100888 (0.0006) [2024-09-30 02:03:40,952][1157819] Updated weights for policy 0, policy_version 100898 (0.0006) [2024-09-30 02:03:41,456][1157819] Updated weights for policy 0, policy_version 100908 (0.0006) [2024-09-30 02:03:41,947][1157819] Updated weights for policy 0, policy_version 100918 (0.0006) [2024-09-30 02:03:42,437][1157819] Updated weights for policy 0, policy_version 100928 (0.0006) [2024-09-30 02:03:42,869][1157819] Updated weights for policy 0, policy_version 100938 (0.0006) [2024-09-30 02:03:43,339][1157819] Updated weights for policy 0, policy_version 100948 (0.0006) [2024-09-30 02:03:43,805][1157819] Updated weights for policy 0, policy_version 100958 (0.0006) [2024-09-30 02:03:44,292][1157819] Updated weights for policy 0, policy_version 100968 (0.0006) [2024-09-30 02:03:44,776][1157819] Updated weights for policy 0, policy_version 100978 (0.0006) [2024-09-30 02:03:45,274][1157819] Updated weights for policy 0, policy_version 100988 (0.0006) [2024-09-30 02:03:45,466][1157520] Fps is (10 sec: 84787.6, 60 sec: 80349.9, 300 sec: 79656.8). Total num frames: 413659136. Throughput: 0: 20092.2. Samples: 93380304. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 02:03:45,466][1157520] Avg episode reward: [(0, '52.484')] [2024-09-30 02:03:45,797][1157819] Updated weights for policy 0, policy_version 100998 (0.0006) [2024-09-30 02:03:46,297][1157819] Updated weights for policy 0, policy_version 101008 (0.0006) [2024-09-30 02:03:46,770][1157819] Updated weights for policy 0, policy_version 101018 (0.0006) [2024-09-30 02:03:47,249][1157819] Updated weights for policy 0, policy_version 101028 (0.0006) [2024-09-30 02:03:47,748][1157819] Updated weights for policy 0, policy_version 101038 (0.0006) [2024-09-30 02:03:48,212][1157819] Updated weights for policy 0, policy_version 101048 (0.0006) [2024-09-30 02:03:48,706][1157819] Updated weights for policy 0, policy_version 101058 (0.0006) [2024-09-30 02:03:49,213][1157819] Updated weights for policy 0, policy_version 101068 (0.0006) [2024-09-30 02:03:49,687][1157819] Updated weights for policy 0, policy_version 101078 (0.0006) [2024-09-30 02:03:50,179][1157819] Updated weights for policy 0, policy_version 101088 (0.0006) [2024-09-30 02:03:50,466][1157520] Fps is (10 sec: 84378.1, 60 sec: 80895.9, 300 sec: 79823.4). Total num frames: 414076928. Throughput: 0: 20189.2. Samples: 93505740. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 02:03:50,466][1157520] Avg episode reward: [(0, '54.803')] [2024-09-30 02:03:50,675][1157819] Updated weights for policy 0, policy_version 101098 (0.0006) [2024-09-30 02:03:51,168][1157819] Updated weights for policy 0, policy_version 101108 (0.0006) [2024-09-30 02:03:51,653][1157819] Updated weights for policy 0, policy_version 101118 (0.0006) [2024-09-30 02:03:52,147][1157819] Updated weights for policy 0, policy_version 101128 (0.0006) [2024-09-30 02:03:52,620][1157819] Updated weights for policy 0, policy_version 101138 (0.0006) [2024-09-30 02:03:53,079][1157819] Updated weights for policy 0, policy_version 101148 (0.0006) [2024-09-30 02:03:53,597][1157819] Updated weights for policy 0, policy_version 101158 (0.0006) [2024-09-30 02:03:54,131][1157819] Updated weights for policy 0, policy_version 101168 (0.0006) [2024-09-30 02:03:54,638][1157819] Updated weights for policy 0, policy_version 101178 (0.0006) [2024-09-30 02:03:55,155][1157819] Updated weights for policy 0, policy_version 101188 (0.0006) [2024-09-30 02:03:55,466][1157520] Fps is (10 sec: 82739.2, 60 sec: 81032.6, 300 sec: 79851.2). Total num frames: 414486528. Throughput: 0: 20250.2. Samples: 93568832. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 02:03:55,466][1157520] Avg episode reward: [(0, '52.106')] [2024-09-30 02:03:55,656][1157819] Updated weights for policy 0, policy_version 101198 (0.0006) [2024-09-30 02:03:56,192][1157819] Updated weights for policy 0, policy_version 101208 (0.0006) [2024-09-30 02:03:56,718][1157819] Updated weights for policy 0, policy_version 101218 (0.0006) [2024-09-30 02:03:57,229][1157819] Updated weights for policy 0, policy_version 101228 (0.0006) [2024-09-30 02:03:57,763][1157819] Updated weights for policy 0, policy_version 101238 (0.0006) [2024-09-30 02:03:58,269][1157819] Updated weights for policy 0, policy_version 101248 (0.0006) [2024-09-30 02:03:58,762][1157819] Updated weights for policy 0, policy_version 101258 (0.0006) [2024-09-30 02:03:59,257][1157819] Updated weights for policy 0, policy_version 101268 (0.0006) [2024-09-30 02:03:59,753][1157819] Updated weights for policy 0, policy_version 101278 (0.0006) [2024-09-30 02:04:00,229][1157819] Updated weights for policy 0, policy_version 101288 (0.0006) [2024-09-30 02:04:00,466][1157520] Fps is (10 sec: 81510.6, 60 sec: 80896.0, 300 sec: 79809.5). Total num frames: 414892032. Throughput: 0: 20431.0. Samples: 93688532. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 02:04:00,466][1157520] Avg episode reward: [(0, '55.133')] [2024-09-30 02:04:00,738][1157819] Updated weights for policy 0, policy_version 101298 (0.0006) [2024-09-30 02:04:01,226][1157819] Updated weights for policy 0, policy_version 101308 (0.0006) [2024-09-30 02:04:01,733][1157819] Updated weights for policy 0, policy_version 101318 (0.0006) [2024-09-30 02:04:02,239][1157819] Updated weights for policy 0, policy_version 101328 (0.0006) [2024-09-30 02:04:02,732][1157819] Updated weights for policy 0, policy_version 101338 (0.0006) [2024-09-30 02:04:03,226][1157819] Updated weights for policy 0, policy_version 101348 (0.0006) [2024-09-30 02:04:03,711][1157819] Updated weights for policy 0, policy_version 101358 (0.0006) [2024-09-30 02:04:04,235][1157819] Updated weights for policy 0, policy_version 101368 (0.0006) [2024-09-30 02:04:04,694][1157819] Updated weights for policy 0, policy_version 101378 (0.0006) [2024-09-30 02:04:05,196][1157819] Updated weights for policy 0, policy_version 101388 (0.0006) [2024-09-30 02:04:05,466][1157520] Fps is (10 sec: 81920.1, 60 sec: 81100.8, 300 sec: 79781.7). Total num frames: 415305728. Throughput: 0: 20631.7. Samples: 93812520. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 02:04:05,466][1157520] Avg episode reward: [(0, '53.770')] [2024-09-30 02:04:05,469][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000101393_415305728.pth... [2024-09-30 02:04:05,515][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000096788_396443648.pth [2024-09-30 02:04:05,728][1157819] Updated weights for policy 0, policy_version 101398 (0.0006) [2024-09-30 02:04:06,301][1157819] Updated weights for policy 0, policy_version 101408 (0.0006) [2024-09-30 02:04:06,836][1157819] Updated weights for policy 0, policy_version 101418 (0.0006) [2024-09-30 02:04:07,411][1157819] Updated weights for policy 0, policy_version 101428 (0.0006) [2024-09-30 02:04:07,938][1157819] Updated weights for policy 0, policy_version 101438 (0.0006) [2024-09-30 02:04:08,490][1157819] Updated weights for policy 0, policy_version 101448 (0.0006) [2024-09-30 02:04:09,054][1157819] Updated weights for policy 0, policy_version 101458 (0.0006) [2024-09-30 02:04:09,597][1157819] Updated weights for policy 0, policy_version 101468 (0.0006) [2024-09-30 02:04:10,168][1157819] Updated weights for policy 0, policy_version 101478 (0.0006) [2024-09-30 02:04:10,466][1157520] Fps is (10 sec: 78233.2, 60 sec: 80759.4, 300 sec: 79601.2). Total num frames: 415674368. Throughput: 0: 20568.5. Samples: 93869492. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 02:04:10,466][1157520] Avg episode reward: [(0, '55.979')] [2024-09-30 02:04:10,738][1157819] Updated weights for policy 0, policy_version 101488 (0.0006) [2024-09-30 02:04:11,285][1157819] Updated weights for policy 0, policy_version 101498 (0.0006) [2024-09-30 02:04:11,833][1157819] Updated weights for policy 0, policy_version 101508 (0.0006) [2024-09-30 02:04:12,377][1157819] Updated weights for policy 0, policy_version 101518 (0.0006) [2024-09-30 02:04:12,933][1157819] Updated weights for policy 0, policy_version 101528 (0.0006) [2024-09-30 02:04:13,485][1157819] Updated weights for policy 0, policy_version 101538 (0.0006) [2024-09-30 02:04:14,041][1157819] Updated weights for policy 0, policy_version 101548 (0.0006) [2024-09-30 02:04:14,606][1157819] Updated weights for policy 0, policy_version 101558 (0.0006) [2024-09-30 02:04:15,151][1157819] Updated weights for policy 0, policy_version 101568 (0.0006) [2024-09-30 02:04:15,466][1157520] Fps is (10 sec: 73727.4, 60 sec: 80691.1, 300 sec: 79323.5). Total num frames: 416043008. Throughput: 0: 20309.2. Samples: 93980352. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 02:04:15,466][1157520] Avg episode reward: [(0, '53.635')] [2024-09-30 02:04:15,692][1157819] Updated weights for policy 0, policy_version 101578 (0.0006) [2024-09-30 02:04:16,281][1157819] Updated weights for policy 0, policy_version 101588 (0.0006) [2024-09-30 02:04:16,804][1157819] Updated weights for policy 0, policy_version 101598 (0.0006) [2024-09-30 02:04:17,328][1157819] Updated weights for policy 0, policy_version 101608 (0.0006) [2024-09-30 02:04:17,902][1157819] Updated weights for policy 0, policy_version 101618 (0.0006) [2024-09-30 02:04:18,419][1157819] Updated weights for policy 0, policy_version 101628 (0.0006) [2024-09-30 02:04:18,942][1157819] Updated weights for policy 0, policy_version 101638 (0.0006) [2024-09-30 02:04:19,445][1157819] Updated weights for policy 0, policy_version 101648 (0.0006) [2024-09-30 02:04:19,937][1157819] Updated weights for policy 0, policy_version 101658 (0.0006) [2024-09-30 02:04:20,466][1157520] Fps is (10 sec: 75365.6, 60 sec: 80691.0, 300 sec: 79087.5). Total num frames: 416428032. Throughput: 0: 20105.5. Samples: 94094012. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 02:04:20,466][1157520] Avg episode reward: [(0, '53.487')] [2024-09-30 02:04:20,496][1157819] Updated weights for policy 0, policy_version 101668 (0.0006) [2024-09-30 02:04:21,048][1157819] Updated weights for policy 0, policy_version 101678 (0.0006) [2024-09-30 02:04:21,545][1157819] Updated weights for policy 0, policy_version 101688 (0.0006) [2024-09-30 02:04:22,106][1157819] Updated weights for policy 0, policy_version 101698 (0.0006) [2024-09-30 02:04:22,658][1157819] Updated weights for policy 0, policy_version 101708 (0.0006) [2024-09-30 02:04:23,199][1157819] Updated weights for policy 0, policy_version 101718 (0.0006) [2024-09-30 02:04:23,705][1157819] Updated weights for policy 0, policy_version 101728 (0.0006) [2024-09-30 02:04:24,208][1157819] Updated weights for policy 0, policy_version 101738 (0.0006) [2024-09-30 02:04:24,751][1157819] Updated weights for policy 0, policy_version 101748 (0.0006) [2024-09-30 02:04:25,266][1157819] Updated weights for policy 0, policy_version 101758 (0.0006) [2024-09-30 02:04:25,466][1157520] Fps is (10 sec: 77413.2, 60 sec: 80554.4, 300 sec: 79031.9). Total num frames: 416817152. Throughput: 0: 19981.4. Samples: 94151664. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 02:04:25,466][1157520] Avg episode reward: [(0, '55.710')] [2024-09-30 02:04:25,765][1157819] Updated weights for policy 0, policy_version 101768 (0.0006) [2024-09-30 02:04:26,268][1157819] Updated weights for policy 0, policy_version 101778 (0.0006) [2024-09-30 02:04:26,825][1157819] Updated weights for policy 0, policy_version 101788 (0.0006) [2024-09-30 02:04:27,351][1157819] Updated weights for policy 0, policy_version 101798 (0.0006) [2024-09-30 02:04:27,871][1157819] Updated weights for policy 0, policy_version 101808 (0.0006) [2024-09-30 02:04:28,406][1157819] Updated weights for policy 0, policy_version 101818 (0.0006) [2024-09-30 02:04:28,934][1157819] Updated weights for policy 0, policy_version 101828 (0.0006) [2024-09-30 02:04:29,456][1157819] Updated weights for policy 0, policy_version 101838 (0.0006) [2024-09-30 02:04:29,982][1157819] Updated weights for policy 0, policy_version 101848 (0.0006) [2024-09-30 02:04:30,466][1157520] Fps is (10 sec: 77414.5, 60 sec: 80008.4, 300 sec: 78962.5). Total num frames: 417202176. Throughput: 0: 19748.8. Samples: 94269004. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 02:04:30,466][1157520] Avg episode reward: [(0, '54.596')] [2024-09-30 02:04:30,539][1157819] Updated weights for policy 0, policy_version 101858 (0.0006) [2024-09-30 02:04:31,047][1157819] Updated weights for policy 0, policy_version 101868 (0.0006) [2024-09-30 02:04:31,594][1157819] Updated weights for policy 0, policy_version 101878 (0.0006) [2024-09-30 02:04:32,104][1157819] Updated weights for policy 0, policy_version 101888 (0.0006) [2024-09-30 02:04:32,647][1157819] Updated weights for policy 0, policy_version 101898 (0.0006) [2024-09-30 02:04:33,170][1157819] Updated weights for policy 0, policy_version 101908 (0.0006) [2024-09-30 02:04:33,706][1157819] Updated weights for policy 0, policy_version 101918 (0.0006) [2024-09-30 02:04:34,214][1157819] Updated weights for policy 0, policy_version 101928 (0.0006) [2024-09-30 02:04:34,740][1157819] Updated weights for policy 0, policy_version 101938 (0.0006) [2024-09-30 02:04:35,254][1157819] Updated weights for policy 0, policy_version 101948 (0.0006) [2024-09-30 02:04:35,466][1157520] Fps is (10 sec: 77823.9, 60 sec: 79735.2, 300 sec: 78837.5). Total num frames: 417595392. Throughput: 0: 19548.1. Samples: 94385408. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 02:04:35,466][1157520] Avg episode reward: [(0, '55.871')] [2024-09-30 02:04:35,761][1157819] Updated weights for policy 0, policy_version 101958 (0.0006) [2024-09-30 02:04:36,288][1157819] Updated weights for policy 0, policy_version 101968 (0.0006) [2024-09-30 02:04:36,800][1157819] Updated weights for policy 0, policy_version 101978 (0.0006) [2024-09-30 02:04:37,305][1157819] Updated weights for policy 0, policy_version 101988 (0.0006) [2024-09-30 02:04:37,843][1157819] Updated weights for policy 0, policy_version 101998 (0.0006) [2024-09-30 02:04:38,395][1157819] Updated weights for policy 0, policy_version 102008 (0.0006) [2024-09-30 02:04:38,891][1157819] Updated weights for policy 0, policy_version 102018 (0.0006) [2024-09-30 02:04:39,395][1157819] Updated weights for policy 0, policy_version 102028 (0.0006) [2024-09-30 02:04:39,899][1157819] Updated weights for policy 0, policy_version 102038 (0.0006) [2024-09-30 02:04:40,406][1157819] Updated weights for policy 0, policy_version 102048 (0.0006) [2024-09-30 02:04:40,466][1157520] Fps is (10 sec: 79053.6, 60 sec: 79325.9, 300 sec: 78768.1). Total num frames: 417992704. Throughput: 0: 19461.8. Samples: 94444616. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 02:04:40,466][1157520] Avg episode reward: [(0, '54.910')] [2024-09-30 02:04:40,943][1157819] Updated weights for policy 0, policy_version 102058 (0.0006) [2024-09-30 02:04:41,452][1157819] Updated weights for policy 0, policy_version 102068 (0.0006) [2024-09-30 02:04:41,953][1157819] Updated weights for policy 0, policy_version 102078 (0.0006) [2024-09-30 02:04:42,455][1157819] Updated weights for policy 0, policy_version 102088 (0.0006) [2024-09-30 02:04:42,943][1157736] Signal inference workers to stop experience collection... (6600 times) [2024-09-30 02:04:42,944][1157736] Signal inference workers to resume experience collection... (6600 times) [2024-09-30 02:04:42,948][1157819] InferenceWorker_p0-w0: stopping experience collection (6600 times) [2024-09-30 02:04:42,948][1157819] InferenceWorker_p0-w0: resuming experience collection (6600 times) [2024-09-30 02:04:42,958][1157819] Updated weights for policy 0, policy_version 102098 (0.0006) [2024-09-30 02:04:43,468][1157819] Updated weights for policy 0, policy_version 102108 (0.0006) [2024-09-30 02:04:43,992][1157819] Updated weights for policy 0, policy_version 102118 (0.0006) [2024-09-30 02:04:44,489][1157819] Updated weights for policy 0, policy_version 102128 (0.0006) [2024-09-30 02:04:45,002][1157819] Updated weights for policy 0, policy_version 102138 (0.0006) [2024-09-30 02:04:45,466][1157520] Fps is (10 sec: 79463.4, 60 sec: 78847.9, 300 sec: 78670.9). Total num frames: 418390016. Throughput: 0: 19473.1. Samples: 94564824. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 02:04:45,466][1157520] Avg episode reward: [(0, '54.868')] [2024-09-30 02:04:45,528][1157819] Updated weights for policy 0, policy_version 102148 (0.0006) [2024-09-30 02:04:45,995][1157819] Updated weights for policy 0, policy_version 102158 (0.0006) [2024-09-30 02:04:46,488][1157819] Updated weights for policy 0, policy_version 102168 (0.0006) [2024-09-30 02:04:46,976][1157819] Updated weights for policy 0, policy_version 102178 (0.0006) [2024-09-30 02:04:47,460][1157819] Updated weights for policy 0, policy_version 102188 (0.0006) [2024-09-30 02:04:47,932][1157819] Updated weights for policy 0, policy_version 102198 (0.0006) [2024-09-30 02:04:48,430][1157819] Updated weights for policy 0, policy_version 102208 (0.0006) [2024-09-30 02:04:48,922][1157819] Updated weights for policy 0, policy_version 102218 (0.0006) [2024-09-30 02:04:49,391][1157819] Updated weights for policy 0, policy_version 102228 (0.0006) [2024-09-30 02:04:49,880][1157819] Updated weights for policy 0, policy_version 102238 (0.0006) [2024-09-30 02:04:50,377][1157819] Updated weights for policy 0, policy_version 102248 (0.0006) [2024-09-30 02:04:50,466][1157520] Fps is (10 sec: 82330.2, 60 sec: 78984.6, 300 sec: 78684.9). Total num frames: 418816000. Throughput: 0: 19493.0. Samples: 94689704. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 02:04:50,466][1157520] Avg episode reward: [(0, '55.812')] [2024-09-30 02:04:50,834][1157819] Updated weights for policy 0, policy_version 102258 (0.0006) [2024-09-30 02:04:51,330][1157819] Updated weights for policy 0, policy_version 102268 (0.0006) [2024-09-30 02:04:51,847][1157819] Updated weights for policy 0, policy_version 102278 (0.0006) [2024-09-30 02:04:52,294][1157819] Updated weights for policy 0, policy_version 102288 (0.0006) [2024-09-30 02:04:52,776][1157819] Updated weights for policy 0, policy_version 102298 (0.0006) [2024-09-30 02:04:53,281][1157819] Updated weights for policy 0, policy_version 102308 (0.0005) [2024-09-30 02:04:53,770][1157819] Updated weights for policy 0, policy_version 102318 (0.0005) [2024-09-30 02:04:54,216][1157819] Updated weights for policy 0, policy_version 102328 (0.0006) [2024-09-30 02:04:54,736][1157819] Updated weights for policy 0, policy_version 102338 (0.0006) [2024-09-30 02:04:55,203][1157819] Updated weights for policy 0, policy_version 102348 (0.0006) [2024-09-30 02:04:55,466][1157520] Fps is (10 sec: 84788.2, 60 sec: 79189.3, 300 sec: 78684.8). Total num frames: 419237888. Throughput: 0: 19636.4. Samples: 94753128. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 02:04:55,466][1157520] Avg episode reward: [(0, '55.483')] [2024-09-30 02:04:55,675][1157819] Updated weights for policy 0, policy_version 102358 (0.0006) [2024-09-30 02:04:56,183][1157819] Updated weights for policy 0, policy_version 102368 (0.0006) [2024-09-30 02:04:56,673][1157819] Updated weights for policy 0, policy_version 102378 (0.0006) [2024-09-30 02:04:57,137][1157819] Updated weights for policy 0, policy_version 102388 (0.0006) [2024-09-30 02:04:57,635][1157819] Updated weights for policy 0, policy_version 102398 (0.0006) [2024-09-30 02:04:58,162][1157819] Updated weights for policy 0, policy_version 102408 (0.0006) [2024-09-30 02:04:58,672][1157819] Updated weights for policy 0, policy_version 102418 (0.0006) [2024-09-30 02:04:59,178][1157819] Updated weights for policy 0, policy_version 102428 (0.0006) [2024-09-30 02:04:59,688][1157819] Updated weights for policy 0, policy_version 102438 (0.0006) [2024-09-30 02:05:00,222][1157819] Updated weights for policy 0, policy_version 102448 (0.0006) [2024-09-30 02:05:00,466][1157520] Fps is (10 sec: 82739.2, 60 sec: 79189.4, 300 sec: 78643.2). Total num frames: 419643392. Throughput: 0: 19955.2. Samples: 94878332. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 02:05:00,466][1157520] Avg episode reward: [(0, '55.461')] [2024-09-30 02:05:00,737][1157819] Updated weights for policy 0, policy_version 102458 (0.0006) [2024-09-30 02:05:01,203][1157819] Updated weights for policy 0, policy_version 102468 (0.0006) [2024-09-30 02:05:01,679][1157819] Updated weights for policy 0, policy_version 102478 (0.0006) [2024-09-30 02:05:02,143][1157819] Updated weights for policy 0, policy_version 102488 (0.0006) [2024-09-30 02:05:02,639][1157819] Updated weights for policy 0, policy_version 102498 (0.0006) [2024-09-30 02:05:03,144][1157819] Updated weights for policy 0, policy_version 102508 (0.0006) [2024-09-30 02:05:03,694][1157819] Updated weights for policy 0, policy_version 102518 (0.0006) [2024-09-30 02:05:04,267][1157819] Updated weights for policy 0, policy_version 102528 (0.0006) [2024-09-30 02:05:04,796][1157819] Updated weights for policy 0, policy_version 102538 (0.0006) [2024-09-30 02:05:05,352][1157819] Updated weights for policy 0, policy_version 102548 (0.0006) [2024-09-30 02:05:05,466][1157520] Fps is (10 sec: 80690.3, 60 sec: 78984.4, 300 sec: 78754.2). Total num frames: 420044800. Throughput: 0: 20101.6. Samples: 94998584. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 02:05:05,466][1157520] Avg episode reward: [(0, '55.079')] [2024-09-30 02:05:05,886][1157819] Updated weights for policy 0, policy_version 102558 (0.0006) [2024-09-30 02:05:06,448][1157819] Updated weights for policy 0, policy_version 102568 (0.0006) [2024-09-30 02:05:06,922][1157819] Updated weights for policy 0, policy_version 102578 (0.0006) [2024-09-30 02:05:07,453][1157819] Updated weights for policy 0, policy_version 102588 (0.0006) [2024-09-30 02:05:07,961][1157819] Updated weights for policy 0, policy_version 102598 (0.0006) [2024-09-30 02:05:08,427][1157819] Updated weights for policy 0, policy_version 102608 (0.0006) [2024-09-30 02:05:08,942][1157819] Updated weights for policy 0, policy_version 102618 (0.0006) [2024-09-30 02:05:09,426][1157819] Updated weights for policy 0, policy_version 102628 (0.0006) [2024-09-30 02:05:09,922][1157819] Updated weights for policy 0, policy_version 102638 (0.0006) [2024-09-30 02:05:10,422][1157819] Updated weights for policy 0, policy_version 102648 (0.0006) [2024-09-30 02:05:10,466][1157520] Fps is (10 sec: 80690.1, 60 sec: 79598.8, 300 sec: 78768.1). Total num frames: 420450304. Throughput: 0: 20121.0. Samples: 95057108. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 02:05:10,466][1157520] Avg episode reward: [(0, '56.413')] [2024-09-30 02:05:10,919][1157819] Updated weights for policy 0, policy_version 102658 (0.0006) [2024-09-30 02:05:11,419][1157819] Updated weights for policy 0, policy_version 102668 (0.0006) [2024-09-30 02:05:11,895][1157819] Updated weights for policy 0, policy_version 102678 (0.0006) [2024-09-30 02:05:12,417][1157819] Updated weights for policy 0, policy_version 102688 (0.0006) [2024-09-30 02:05:12,936][1157819] Updated weights for policy 0, policy_version 102698 (0.0006) [2024-09-30 02:05:13,422][1157819] Updated weights for policy 0, policy_version 102708 (0.0006) [2024-09-30 02:05:13,920][1157819] Updated weights for policy 0, policy_version 102718 (0.0006) [2024-09-30 02:05:14,474][1157819] Updated weights for policy 0, policy_version 102728 (0.0006) [2024-09-30 02:05:14,975][1157819] Updated weights for policy 0, policy_version 102738 (0.0006) [2024-09-30 02:05:15,466][1157520] Fps is (10 sec: 80690.4, 60 sec: 80144.9, 300 sec: 78795.9). Total num frames: 420851712. Throughput: 0: 20244.8. Samples: 95180020. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 02:05:15,466][1157520] Avg episode reward: [(0, '53.341')] [2024-09-30 02:05:15,513][1157819] Updated weights for policy 0, policy_version 102748 (0.0006) [2024-09-30 02:05:16,005][1157819] Updated weights for policy 0, policy_version 102758 (0.0006) [2024-09-30 02:05:16,521][1157819] Updated weights for policy 0, policy_version 102768 (0.0006) [2024-09-30 02:05:16,800][1157736] Signal inference workers to stop experience collection... (6650 times) [2024-09-30 02:05:16,804][1157736] Signal inference workers to resume experience collection... (6650 times) [2024-09-30 02:05:16,806][1157819] InferenceWorker_p0-w0: stopping experience collection (6650 times) [2024-09-30 02:05:16,810][1157819] InferenceWorker_p0-w0: resuming experience collection (6650 times) [2024-09-30 02:05:17,020][1157819] Updated weights for policy 0, policy_version 102778 (0.0006) [2024-09-30 02:05:17,543][1157819] Updated weights for policy 0, policy_version 102788 (0.0006) [2024-09-30 02:05:18,062][1157819] Updated weights for policy 0, policy_version 102798 (0.0006) [2024-09-30 02:05:18,523][1157819] Updated weights for policy 0, policy_version 102808 (0.0006) [2024-09-30 02:05:19,001][1157819] Updated weights for policy 0, policy_version 102818 (0.0006) [2024-09-30 02:05:19,487][1157819] Updated weights for policy 0, policy_version 102828 (0.0006) [2024-09-30 02:05:19,942][1157819] Updated weights for policy 0, policy_version 102838 (0.0006) [2024-09-30 02:05:20,429][1157819] Updated weights for policy 0, policy_version 102848 (0.0006) [2024-09-30 02:05:20,466][1157520] Fps is (10 sec: 81511.3, 60 sec: 80623.1, 300 sec: 78934.8). Total num frames: 421265408. Throughput: 0: 20379.2. Samples: 95302468. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 02:05:20,466][1157520] Avg episode reward: [(0, '56.274')] [2024-09-30 02:05:20,913][1157819] Updated weights for policy 0, policy_version 102858 (0.0006) [2024-09-30 02:05:21,397][1157819] Updated weights for policy 0, policy_version 102868 (0.0006) [2024-09-30 02:05:21,861][1157819] Updated weights for policy 0, policy_version 102878 (0.0006) [2024-09-30 02:05:22,356][1157819] Updated weights for policy 0, policy_version 102888 (0.0006) [2024-09-30 02:05:22,888][1157819] Updated weights for policy 0, policy_version 102898 (0.0006) [2024-09-30 02:05:23,402][1157819] Updated weights for policy 0, policy_version 102908 (0.0006) [2024-09-30 02:05:23,897][1157819] Updated weights for policy 0, policy_version 102918 (0.0006) [2024-09-30 02:05:24,438][1157819] Updated weights for policy 0, policy_version 102928 (0.0006) [2024-09-30 02:05:24,988][1157819] Updated weights for policy 0, policy_version 102938 (0.0006) [2024-09-30 02:05:25,466][1157520] Fps is (10 sec: 81511.4, 60 sec: 80827.9, 300 sec: 78976.4). Total num frames: 421666816. Throughput: 0: 20469.5. Samples: 95365744. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 02:05:25,466][1157520] Avg episode reward: [(0, '55.572')] [2024-09-30 02:05:25,520][1157819] Updated weights for policy 0, policy_version 102948 (0.0006) [2024-09-30 02:05:26,041][1157819] Updated weights for policy 0, policy_version 102958 (0.0006) [2024-09-30 02:05:26,620][1157819] Updated weights for policy 0, policy_version 102968 (0.0006) [2024-09-30 02:05:27,155][1157819] Updated weights for policy 0, policy_version 102978 (0.0006) [2024-09-30 02:05:27,720][1157819] Updated weights for policy 0, policy_version 102988 (0.0006) [2024-09-30 02:05:28,286][1157819] Updated weights for policy 0, policy_version 102998 (0.0006) [2024-09-30 02:05:28,833][1157819] Updated weights for policy 0, policy_version 103008 (0.0006) [2024-09-30 02:05:29,368][1157819] Updated weights for policy 0, policy_version 103018 (0.0006) [2024-09-30 02:05:29,905][1157819] Updated weights for policy 0, policy_version 103028 (0.0006) [2024-09-30 02:05:30,466][1157520] Fps is (10 sec: 77413.5, 60 sec: 80623.0, 300 sec: 79045.8). Total num frames: 422039552. Throughput: 0: 20326.7. Samples: 95479524. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 02:05:30,466][1157520] Avg episode reward: [(0, '55.213')] [2024-09-30 02:05:30,470][1157819] Updated weights for policy 0, policy_version 103038 (0.0006) [2024-09-30 02:05:31,000][1157819] Updated weights for policy 0, policy_version 103048 (0.0006) [2024-09-30 02:05:31,557][1157819] Updated weights for policy 0, policy_version 103058 (0.0006) [2024-09-30 02:05:32,118][1157819] Updated weights for policy 0, policy_version 103068 (0.0006) [2024-09-30 02:05:32,634][1157819] Updated weights for policy 0, policy_version 103078 (0.0006) [2024-09-30 02:05:33,161][1157819] Updated weights for policy 0, policy_version 103088 (0.0006) [2024-09-30 02:05:33,726][1157819] Updated weights for policy 0, policy_version 103098 (0.0006) [2024-09-30 02:05:34,244][1157819] Updated weights for policy 0, policy_version 103108 (0.0006) [2024-09-30 02:05:34,788][1157819] Updated weights for policy 0, policy_version 103118 (0.0006) [2024-09-30 02:05:35,309][1157819] Updated weights for policy 0, policy_version 103128 (0.0006) [2024-09-30 02:05:35,466][1157520] Fps is (10 sec: 75366.2, 60 sec: 80418.3, 300 sec: 79115.3). Total num frames: 422420480. Throughput: 0: 20075.8. Samples: 95593120. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:05:35,466][1157520] Avg episode reward: [(0, '52.467')] [2024-09-30 02:05:35,888][1157819] Updated weights for policy 0, policy_version 103138 (0.0006) [2024-09-30 02:05:36,405][1157819] Updated weights for policy 0, policy_version 103148 (0.0006) [2024-09-30 02:05:36,941][1157819] Updated weights for policy 0, policy_version 103158 (0.0006) [2024-09-30 02:05:37,536][1157819] Updated weights for policy 0, policy_version 103168 (0.0006) [2024-09-30 02:05:38,047][1157819] Updated weights for policy 0, policy_version 103178 (0.0006) [2024-09-30 02:05:38,577][1157819] Updated weights for policy 0, policy_version 103188 (0.0006) [2024-09-30 02:05:39,137][1157819] Updated weights for policy 0, policy_version 103198 (0.0006) [2024-09-30 02:05:39,654][1157819] Updated weights for policy 0, policy_version 103208 (0.0006) [2024-09-30 02:05:40,195][1157819] Updated weights for policy 0, policy_version 103218 (0.0006) [2024-09-30 02:05:40,466][1157520] Fps is (10 sec: 75776.2, 60 sec: 80076.7, 300 sec: 79115.3). Total num frames: 422797312. Throughput: 0: 19913.1. Samples: 95649220. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:05:40,466][1157520] Avg episode reward: [(0, '53.298')] [2024-09-30 02:05:40,746][1157819] Updated weights for policy 0, policy_version 103228 (0.0006) [2024-09-30 02:05:41,309][1157819] Updated weights for policy 0, policy_version 103238 (0.0006) [2024-09-30 02:05:41,851][1157819] Updated weights for policy 0, policy_version 103248 (0.0006) [2024-09-30 02:05:42,408][1157819] Updated weights for policy 0, policy_version 103258 (0.0006) [2024-09-30 02:05:42,943][1157819] Updated weights for policy 0, policy_version 103268 (0.0006) [2024-09-30 02:05:43,501][1157819] Updated weights for policy 0, policy_version 103278 (0.0006) [2024-09-30 02:05:44,051][1157819] Updated weights for policy 0, policy_version 103288 (0.0006) [2024-09-30 02:05:44,628][1157819] Updated weights for policy 0, policy_version 103298 (0.0006) [2024-09-30 02:05:45,135][1157819] Updated weights for policy 0, policy_version 103308 (0.0006) [2024-09-30 02:05:45,466][1157520] Fps is (10 sec: 74956.7, 60 sec: 79667.2, 300 sec: 79115.2). Total num frames: 423170048. Throughput: 0: 19639.7. Samples: 95762120. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:05:45,466][1157520] Avg episode reward: [(0, '53.594')] [2024-09-30 02:05:45,705][1157819] Updated weights for policy 0, policy_version 103318 (0.0006) [2024-09-30 02:05:46,271][1157819] Updated weights for policy 0, policy_version 103328 (0.0006) [2024-09-30 02:05:46,832][1157819] Updated weights for policy 0, policy_version 103338 (0.0006) [2024-09-30 02:05:47,376][1157819] Updated weights for policy 0, policy_version 103348 (0.0006) [2024-09-30 02:05:47,891][1157819] Updated weights for policy 0, policy_version 103358 (0.0006) [2024-09-30 02:05:48,426][1157819] Updated weights for policy 0, policy_version 103368 (0.0006) [2024-09-30 02:05:48,978][1157819] Updated weights for policy 0, policy_version 103378 (0.0006) [2024-09-30 02:05:49,516][1157819] Updated weights for policy 0, policy_version 103388 (0.0006) [2024-09-30 02:05:50,050][1157819] Updated weights for policy 0, policy_version 103398 (0.0006) [2024-09-30 02:05:50,466][1157520] Fps is (10 sec: 75366.4, 60 sec: 78916.1, 300 sec: 79170.8). Total num frames: 423550976. Throughput: 0: 19468.5. Samples: 95874668. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:05:50,466][1157520] Avg episode reward: [(0, '56.836')] [2024-09-30 02:05:50,578][1157819] Updated weights for policy 0, policy_version 103408 (0.0006) [2024-09-30 02:05:51,114][1157819] Updated weights for policy 0, policy_version 103418 (0.0006) [2024-09-30 02:05:51,644][1157819] Updated weights for policy 0, policy_version 103428 (0.0006) [2024-09-30 02:05:52,171][1157819] Updated weights for policy 0, policy_version 103438 (0.0006) [2024-09-30 02:05:52,723][1157819] Updated weights for policy 0, policy_version 103448 (0.0006) [2024-09-30 02:05:53,226][1157819] Updated weights for policy 0, policy_version 103458 (0.0006) [2024-09-30 02:05:53,770][1157819] Updated weights for policy 0, policy_version 103468 (0.0006) [2024-09-30 02:05:54,321][1157819] Updated weights for policy 0, policy_version 103478 (0.0006) [2024-09-30 02:05:54,826][1157819] Updated weights for policy 0, policy_version 103488 (0.0006) [2024-09-30 02:05:55,320][1157819] Updated weights for policy 0, policy_version 103498 (0.0006) [2024-09-30 02:05:55,466][1157520] Fps is (10 sec: 76594.9, 60 sec: 78301.6, 300 sec: 79240.2). Total num frames: 423936000. Throughput: 0: 19446.5. Samples: 95932200. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:05:55,466][1157520] Avg episode reward: [(0, '56.356')] [2024-09-30 02:05:55,809][1157819] Updated weights for policy 0, policy_version 103508 (0.0006) [2024-09-30 02:05:56,311][1157819] Updated weights for policy 0, policy_version 103518 (0.0006) [2024-09-30 02:05:56,811][1157819] Updated weights for policy 0, policy_version 103528 (0.0006) [2024-09-30 02:05:57,202][1157736] Signal inference workers to stop experience collection... (6700 times) [2024-09-30 02:05:57,203][1157736] Signal inference workers to resume experience collection... (6700 times) [2024-09-30 02:05:57,207][1157819] InferenceWorker_p0-w0: stopping experience collection (6700 times) [2024-09-30 02:05:57,209][1157819] InferenceWorker_p0-w0: resuming experience collection (6700 times) [2024-09-30 02:05:57,347][1157819] Updated weights for policy 0, policy_version 103538 (0.0006) [2024-09-30 02:05:57,844][1157819] Updated weights for policy 0, policy_version 103548 (0.0006) [2024-09-30 02:05:58,327][1157819] Updated weights for policy 0, policy_version 103558 (0.0006) [2024-09-30 02:05:58,831][1157819] Updated weights for policy 0, policy_version 103568 (0.0006) [2024-09-30 02:05:59,327][1157819] Updated weights for policy 0, policy_version 103578 (0.0006) [2024-09-30 02:05:59,833][1157819] Updated weights for policy 0, policy_version 103588 (0.0006) [2024-09-30 02:06:00,354][1157819] Updated weights for policy 0, policy_version 103598 (0.0006) [2024-09-30 02:06:00,466][1157520] Fps is (10 sec: 79462.4, 60 sec: 78370.0, 300 sec: 79309.7). Total num frames: 424345600. Throughput: 0: 19393.1. Samples: 96052708. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:06:00,466][1157520] Avg episode reward: [(0, '55.188')] [2024-09-30 02:06:00,869][1157819] Updated weights for policy 0, policy_version 103608 (0.0006) [2024-09-30 02:06:01,382][1157819] Updated weights for policy 0, policy_version 103618 (0.0006) [2024-09-30 02:06:01,855][1157819] Updated weights for policy 0, policy_version 103628 (0.0006) [2024-09-30 02:06:02,346][1157819] Updated weights for policy 0, policy_version 103638 (0.0006) [2024-09-30 02:06:02,858][1157819] Updated weights for policy 0, policy_version 103648 (0.0006) [2024-09-30 02:06:03,380][1157819] Updated weights for policy 0, policy_version 103658 (0.0006) [2024-09-30 02:06:03,866][1157819] Updated weights for policy 0, policy_version 103668 (0.0006) [2024-09-30 02:06:04,337][1157819] Updated weights for policy 0, policy_version 103678 (0.0006) [2024-09-30 02:06:04,839][1157819] Updated weights for policy 0, policy_version 103688 (0.0006) [2024-09-30 02:06:05,316][1157819] Updated weights for policy 0, policy_version 103698 (0.0006) [2024-09-30 02:06:05,466][1157520] Fps is (10 sec: 81921.3, 60 sec: 78506.8, 300 sec: 79420.7). Total num frames: 424755200. Throughput: 0: 19400.2. Samples: 96175476. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:06:05,466][1157520] Avg episode reward: [(0, '52.584')] [2024-09-30 02:06:05,470][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000103701_424759296.pth... [2024-09-30 02:06:05,526][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000099061_405753856.pth [2024-09-30 02:06:05,790][1157819] Updated weights for policy 0, policy_version 103708 (0.0006) [2024-09-30 02:06:06,262][1157819] Updated weights for policy 0, policy_version 103718 (0.0006) [2024-09-30 02:06:06,765][1157819] Updated weights for policy 0, policy_version 103728 (0.0006) [2024-09-30 02:06:07,250][1157819] Updated weights for policy 0, policy_version 103738 (0.0006) [2024-09-30 02:06:07,770][1157819] Updated weights for policy 0, policy_version 103748 (0.0006) [2024-09-30 02:06:08,300][1157819] Updated weights for policy 0, policy_version 103758 (0.0006) [2024-09-30 02:06:08,790][1157819] Updated weights for policy 0, policy_version 103768 (0.0006) [2024-09-30 02:06:09,368][1157819] Updated weights for policy 0, policy_version 103778 (0.0006) [2024-09-30 02:06:09,931][1157819] Updated weights for policy 0, policy_version 103788 (0.0006) [2024-09-30 02:06:10,438][1157819] Updated weights for policy 0, policy_version 103798 (0.0006) [2024-09-30 02:06:10,466][1157520] Fps is (10 sec: 81100.6, 60 sec: 78438.4, 300 sec: 79365.2). Total num frames: 425156608. Throughput: 0: 19404.0. Samples: 96238924. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:06:10,466][1157520] Avg episode reward: [(0, '54.159')] [2024-09-30 02:06:11,026][1157819] Updated weights for policy 0, policy_version 103808 (0.0006) [2024-09-30 02:06:11,581][1157819] Updated weights for policy 0, policy_version 103818 (0.0006) [2024-09-30 02:06:12,082][1157819] Updated weights for policy 0, policy_version 103828 (0.0006) [2024-09-30 02:06:12,680][1157819] Updated weights for policy 0, policy_version 103838 (0.0006) [2024-09-30 02:06:13,190][1157819] Updated weights for policy 0, policy_version 103848 (0.0006) [2024-09-30 02:06:13,763][1157819] Updated weights for policy 0, policy_version 103858 (0.0006) [2024-09-30 02:06:14,329][1157819] Updated weights for policy 0, policy_version 103868 (0.0006) [2024-09-30 02:06:14,908][1157819] Updated weights for policy 0, policy_version 103878 (0.0006) [2024-09-30 02:06:15,466][1157520] Fps is (10 sec: 76594.3, 60 sec: 77824.1, 300 sec: 79212.4). Total num frames: 425521152. Throughput: 0: 19363.5. Samples: 96350880. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:06:15,466][1157520] Avg episode reward: [(0, '55.191')] [2024-09-30 02:06:15,485][1157819] Updated weights for policy 0, policy_version 103888 (0.0006) [2024-09-30 02:06:16,016][1157819] Updated weights for policy 0, policy_version 103898 (0.0006) [2024-09-30 02:06:16,573][1157819] Updated weights for policy 0, policy_version 103908 (0.0006) [2024-09-30 02:06:17,102][1157819] Updated weights for policy 0, policy_version 103918 (0.0006) [2024-09-30 02:06:17,644][1157819] Updated weights for policy 0, policy_version 103928 (0.0006) [2024-09-30 02:06:18,236][1157819] Updated weights for policy 0, policy_version 103938 (0.0006) [2024-09-30 02:06:18,784][1157819] Updated weights for policy 0, policy_version 103948 (0.0006) [2024-09-30 02:06:19,302][1157819] Updated weights for policy 0, policy_version 103958 (0.0006) [2024-09-30 02:06:19,799][1157819] Updated weights for policy 0, policy_version 103968 (0.0006) [2024-09-30 02:06:20,316][1157819] Updated weights for policy 0, policy_version 103978 (0.0006) [2024-09-30 02:06:20,466][1157520] Fps is (10 sec: 74548.2, 60 sec: 77277.9, 300 sec: 79184.7). Total num frames: 425902080. Throughput: 0: 19333.0. Samples: 96463104. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:06:20,466][1157520] Avg episode reward: [(0, '54.727')] [2024-09-30 02:06:20,825][1157819] Updated weights for policy 0, policy_version 103988 (0.0006) [2024-09-30 02:06:21,352][1157819] Updated weights for policy 0, policy_version 103998 (0.0006) [2024-09-30 02:06:21,857][1157819] Updated weights for policy 0, policy_version 104008 (0.0006) [2024-09-30 02:06:22,361][1157819] Updated weights for policy 0, policy_version 104018 (0.0006) [2024-09-30 02:06:22,883][1157819] Updated weights for policy 0, policy_version 104028 (0.0006) [2024-09-30 02:06:23,421][1157819] Updated weights for policy 0, policy_version 104038 (0.0006) [2024-09-30 02:06:23,952][1157819] Updated weights for policy 0, policy_version 104048 (0.0006) [2024-09-30 02:06:24,526][1157819] Updated weights for policy 0, policy_version 104058 (0.0006) [2024-09-30 02:06:25,072][1157819] Updated weights for policy 0, policy_version 104068 (0.0006) [2024-09-30 02:06:25,466][1157520] Fps is (10 sec: 76596.1, 60 sec: 77004.9, 300 sec: 79170.8). Total num frames: 426287104. Throughput: 0: 19401.8. Samples: 96522300. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:06:25,466][1157520] Avg episode reward: [(0, '54.275')] [2024-09-30 02:06:25,669][1157819] Updated weights for policy 0, policy_version 104078 (0.0007) [2024-09-30 02:06:26,214][1157819] Updated weights for policy 0, policy_version 104088 (0.0006) [2024-09-30 02:06:26,750][1157819] Updated weights for policy 0, policy_version 104098 (0.0006) [2024-09-30 02:06:26,966][1157736] Signal inference workers to stop experience collection... (6750 times) [2024-09-30 02:06:26,967][1157736] Signal inference workers to resume experience collection... (6750 times) [2024-09-30 02:06:26,972][1157819] InferenceWorker_p0-w0: stopping experience collection (6750 times) [2024-09-30 02:06:26,972][1157819] InferenceWorker_p0-w0: resuming experience collection (6750 times) [2024-09-30 02:06:27,277][1157819] Updated weights for policy 0, policy_version 104108 (0.0006) [2024-09-30 02:06:27,830][1157819] Updated weights for policy 0, policy_version 104118 (0.0006) [2024-09-30 02:06:28,363][1157819] Updated weights for policy 0, policy_version 104128 (0.0006) [2024-09-30 02:06:28,911][1157819] Updated weights for policy 0, policy_version 104138 (0.0006) [2024-09-30 02:06:29,489][1157819] Updated weights for policy 0, policy_version 104148 (0.0006) [2024-09-30 02:06:30,039][1157819] Updated weights for policy 0, policy_version 104158 (0.0006) [2024-09-30 02:06:30,466][1157520] Fps is (10 sec: 75776.0, 60 sec: 77005.0, 300 sec: 79087.5). Total num frames: 426659840. Throughput: 0: 19391.6. Samples: 96634740. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:06:30,466][1157520] Avg episode reward: [(0, '55.160')] [2024-09-30 02:06:30,574][1157819] Updated weights for policy 0, policy_version 104168 (0.0006) [2024-09-30 02:06:31,126][1157819] Updated weights for policy 0, policy_version 104178 (0.0006) [2024-09-30 02:06:31,661][1157819] Updated weights for policy 0, policy_version 104188 (0.0006) [2024-09-30 02:06:32,217][1157819] Updated weights for policy 0, policy_version 104198 (0.0006) [2024-09-30 02:06:32,770][1157819] Updated weights for policy 0, policy_version 104208 (0.0006) [2024-09-30 02:06:33,299][1157819] Updated weights for policy 0, policy_version 104218 (0.0006) [2024-09-30 02:06:33,842][1157819] Updated weights for policy 0, policy_version 104228 (0.0006) [2024-09-30 02:06:34,391][1157819] Updated weights for policy 0, policy_version 104238 (0.0006) [2024-09-30 02:06:34,927][1157819] Updated weights for policy 0, policy_version 104248 (0.0006) [2024-09-30 02:06:35,462][1157819] Updated weights for policy 0, policy_version 104258 (0.0006) [2024-09-30 02:06:35,466][1157520] Fps is (10 sec: 75366.4, 60 sec: 77004.9, 300 sec: 78962.5). Total num frames: 427040768. Throughput: 0: 19394.6. Samples: 96747424. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:06:35,466][1157520] Avg episode reward: [(0, '53.900')] [2024-09-30 02:06:35,973][1157819] Updated weights for policy 0, policy_version 104268 (0.0006) [2024-09-30 02:06:36,509][1157819] Updated weights for policy 0, policy_version 104278 (0.0006) [2024-09-30 02:06:37,105][1157819] Updated weights for policy 0, policy_version 104288 (0.0006) [2024-09-30 02:06:37,645][1157819] Updated weights for policy 0, policy_version 104298 (0.0006) [2024-09-30 02:06:38,162][1157819] Updated weights for policy 0, policy_version 104308 (0.0006) [2024-09-30 02:06:38,682][1157819] Updated weights for policy 0, policy_version 104318 (0.0006) [2024-09-30 02:06:39,211][1157819] Updated weights for policy 0, policy_version 104328 (0.0006) [2024-09-30 02:06:39,728][1157819] Updated weights for policy 0, policy_version 104338 (0.0006) [2024-09-30 02:06:40,238][1157819] Updated weights for policy 0, policy_version 104348 (0.0006) [2024-09-30 02:06:40,466][1157520] Fps is (10 sec: 76595.0, 60 sec: 77141.4, 300 sec: 78851.5). Total num frames: 427425792. Throughput: 0: 19374.7. Samples: 96804060. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:06:40,466][1157520] Avg episode reward: [(0, '54.110')] [2024-09-30 02:06:40,768][1157819] Updated weights for policy 0, policy_version 104358 (0.0006) [2024-09-30 02:06:41,286][1157819] Updated weights for policy 0, policy_version 104368 (0.0006) [2024-09-30 02:06:41,791][1157819] Updated weights for policy 0, policy_version 104378 (0.0006) [2024-09-30 02:06:42,324][1157819] Updated weights for policy 0, policy_version 104388 (0.0007) [2024-09-30 02:06:42,881][1157819] Updated weights for policy 0, policy_version 104398 (0.0007) [2024-09-30 02:06:43,452][1157819] Updated weights for policy 0, policy_version 104408 (0.0006) [2024-09-30 02:06:43,990][1157819] Updated weights for policy 0, policy_version 104418 (0.0006) [2024-09-30 02:06:44,557][1157819] Updated weights for policy 0, policy_version 104428 (0.0006) [2024-09-30 02:06:45,115][1157819] Updated weights for policy 0, policy_version 104438 (0.0006) [2024-09-30 02:06:45,466][1157520] Fps is (10 sec: 76185.9, 60 sec: 77209.8, 300 sec: 78740.4). Total num frames: 427802624. Throughput: 0: 19277.3. Samples: 96920184. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:06:45,466][1157520] Avg episode reward: [(0, '54.360')] [2024-09-30 02:06:45,629][1157819] Updated weights for policy 0, policy_version 104448 (0.0006) [2024-09-30 02:06:46,182][1157819] Updated weights for policy 0, policy_version 104458 (0.0006) [2024-09-30 02:06:46,746][1157819] Updated weights for policy 0, policy_version 104468 (0.0006) [2024-09-30 02:06:47,278][1157819] Updated weights for policy 0, policy_version 104478 (0.0006) [2024-09-30 02:06:47,803][1157819] Updated weights for policy 0, policy_version 104488 (0.0007) [2024-09-30 02:06:48,348][1157819] Updated weights for policy 0, policy_version 104498 (0.0007) [2024-09-30 02:06:48,910][1157819] Updated weights for policy 0, policy_version 104508 (0.0007) [2024-09-30 02:06:49,464][1157819] Updated weights for policy 0, policy_version 104518 (0.0006) [2024-09-30 02:06:50,000][1157819] Updated weights for policy 0, policy_version 104528 (0.0007) [2024-09-30 02:06:50,466][1157520] Fps is (10 sec: 74956.8, 60 sec: 77073.2, 300 sec: 78657.1). Total num frames: 428175360. Throughput: 0: 19047.6. Samples: 97032620. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:06:50,466][1157520] Avg episode reward: [(0, '53.767')] [2024-09-30 02:06:50,550][1157819] Updated weights for policy 0, policy_version 104538 (0.0006) [2024-09-30 02:06:51,117][1157819] Updated weights for policy 0, policy_version 104548 (0.0007) [2024-09-30 02:06:51,668][1157819] Updated weights for policy 0, policy_version 104558 (0.0006) [2024-09-30 02:06:52,210][1157819] Updated weights for policy 0, policy_version 104568 (0.0006) [2024-09-30 02:06:52,780][1157819] Updated weights for policy 0, policy_version 104578 (0.0006) [2024-09-30 02:06:53,325][1157819] Updated weights for policy 0, policy_version 104588 (0.0006) [2024-09-30 02:06:53,858][1157819] Updated weights for policy 0, policy_version 104598 (0.0006) [2024-09-30 02:06:54,420][1157819] Updated weights for policy 0, policy_version 104608 (0.0006) [2024-09-30 02:06:54,969][1157819] Updated weights for policy 0, policy_version 104618 (0.0006) [2024-09-30 02:06:55,466][1157520] Fps is (10 sec: 74547.0, 60 sec: 76868.5, 300 sec: 78573.8). Total num frames: 428548096. Throughput: 0: 18873.5. Samples: 97088228. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:06:55,466][1157520] Avg episode reward: [(0, '54.251')] [2024-09-30 02:06:55,503][1157819] Updated weights for policy 0, policy_version 104628 (0.0006) [2024-09-30 02:06:56,060][1157819] Updated weights for policy 0, policy_version 104638 (0.0006) [2024-09-30 02:06:56,122][1157736] Signal inference workers to stop experience collection... (6800 times) [2024-09-30 02:06:56,123][1157736] Signal inference workers to resume experience collection... (6800 times) [2024-09-30 02:06:56,128][1157819] InferenceWorker_p0-w0: stopping experience collection (6800 times) [2024-09-30 02:06:56,128][1157819] InferenceWorker_p0-w0: resuming experience collection (6800 times) [2024-09-30 02:06:56,621][1157819] Updated weights for policy 0, policy_version 104648 (0.0006) [2024-09-30 02:06:57,128][1157819] Updated weights for policy 0, policy_version 104658 (0.0006) [2024-09-30 02:06:57,629][1157819] Updated weights for policy 0, policy_version 104668 (0.0006) [2024-09-30 02:06:58,140][1157819] Updated weights for policy 0, policy_version 104678 (0.0007) [2024-09-30 02:06:58,668][1157819] Updated weights for policy 0, policy_version 104688 (0.0006) [2024-09-30 02:06:59,198][1157819] Updated weights for policy 0, policy_version 104698 (0.0006) [2024-09-30 02:06:59,694][1157819] Updated weights for policy 0, policy_version 104708 (0.0006) [2024-09-30 02:07:00,196][1157819] Updated weights for policy 0, policy_version 104718 (0.0006) [2024-09-30 02:07:00,466][1157520] Fps is (10 sec: 77004.9, 60 sec: 76663.6, 300 sec: 78615.4). Total num frames: 428945408. Throughput: 0: 18932.7. Samples: 97202848. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:07:00,466][1157520] Avg episode reward: [(0, '54.527')] [2024-09-30 02:07:00,739][1157819] Updated weights for policy 0, policy_version 104728 (0.0006) [2024-09-30 02:07:01,227][1157819] Updated weights for policy 0, policy_version 104738 (0.0006) [2024-09-30 02:07:01,720][1157819] Updated weights for policy 0, policy_version 104748 (0.0006) [2024-09-30 02:07:02,245][1157819] Updated weights for policy 0, policy_version 104758 (0.0006) [2024-09-30 02:07:02,766][1157819] Updated weights for policy 0, policy_version 104768 (0.0006) [2024-09-30 02:07:03,283][1157819] Updated weights for policy 0, policy_version 104778 (0.0006) [2024-09-30 02:07:03,808][1157819] Updated weights for policy 0, policy_version 104788 (0.0006) [2024-09-30 02:07:04,340][1157819] Updated weights for policy 0, policy_version 104798 (0.0006) [2024-09-30 02:07:04,947][1157819] Updated weights for policy 0, policy_version 104808 (0.0006) [2024-09-30 02:07:05,466][1157520] Fps is (10 sec: 78233.6, 60 sec: 76253.9, 300 sec: 78587.7). Total num frames: 429330432. Throughput: 0: 19054.0. Samples: 97320536. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:07:05,466][1157520] Avg episode reward: [(0, '55.657')] [2024-09-30 02:07:05,480][1157819] Updated weights for policy 0, policy_version 104818 (0.0006) [2024-09-30 02:07:06,056][1157819] Updated weights for policy 0, policy_version 104828 (0.0006) [2024-09-30 02:07:06,592][1157819] Updated weights for policy 0, policy_version 104838 (0.0006) [2024-09-30 02:07:07,099][1157819] Updated weights for policy 0, policy_version 104848 (0.0006) [2024-09-30 02:07:07,602][1157819] Updated weights for policy 0, policy_version 104858 (0.0006) [2024-09-30 02:07:08,112][1157819] Updated weights for policy 0, policy_version 104868 (0.0006) [2024-09-30 02:07:08,604][1157819] Updated weights for policy 0, policy_version 104878 (0.0006) [2024-09-30 02:07:09,101][1157819] Updated weights for policy 0, policy_version 104888 (0.0006) [2024-09-30 02:07:09,603][1157819] Updated weights for policy 0, policy_version 104898 (0.0006) [2024-09-30 02:07:10,112][1157819] Updated weights for policy 0, policy_version 104908 (0.0006) [2024-09-30 02:07:10,466][1157520] Fps is (10 sec: 78642.8, 60 sec: 76254.0, 300 sec: 78546.0). Total num frames: 429731840. Throughput: 0: 19027.5. Samples: 97378536. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:07:10,466][1157520] Avg episode reward: [(0, '57.032')] [2024-09-30 02:07:10,636][1157819] Updated weights for policy 0, policy_version 104918 (0.0006) [2024-09-30 02:07:11,150][1157819] Updated weights for policy 0, policy_version 104928 (0.0006) [2024-09-30 02:07:11,683][1157819] Updated weights for policy 0, policy_version 104938 (0.0006) [2024-09-30 02:07:12,195][1157819] Updated weights for policy 0, policy_version 104948 (0.0006) [2024-09-30 02:07:12,693][1157819] Updated weights for policy 0, policy_version 104958 (0.0006) [2024-09-30 02:07:13,213][1157819] Updated weights for policy 0, policy_version 104968 (0.0006) [2024-09-30 02:07:13,734][1157819] Updated weights for policy 0, policy_version 104978 (0.0006) [2024-09-30 02:07:14,234][1157819] Updated weights for policy 0, policy_version 104988 (0.0006) [2024-09-30 02:07:14,745][1157819] Updated weights for policy 0, policy_version 104998 (0.0006) [2024-09-30 02:07:15,273][1157819] Updated weights for policy 0, policy_version 105008 (0.0006) [2024-09-30 02:07:15,466][1157520] Fps is (10 sec: 79461.9, 60 sec: 76731.8, 300 sec: 78559.9). Total num frames: 430125056. Throughput: 0: 19199.3. Samples: 97498708. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:07:15,466][1157520] Avg episode reward: [(0, '56.180')] [2024-09-30 02:07:15,796][1157819] Updated weights for policy 0, policy_version 105018 (0.0006) [2024-09-30 02:07:16,277][1157819] Updated weights for policy 0, policy_version 105028 (0.0006) [2024-09-30 02:07:16,795][1157819] Updated weights for policy 0, policy_version 105038 (0.0006) [2024-09-30 02:07:17,330][1157819] Updated weights for policy 0, policy_version 105048 (0.0006) [2024-09-30 02:07:17,847][1157819] Updated weights for policy 0, policy_version 105058 (0.0006) [2024-09-30 02:07:18,348][1157819] Updated weights for policy 0, policy_version 105068 (0.0006) [2024-09-30 02:07:18,842][1157819] Updated weights for policy 0, policy_version 105078 (0.0006) [2024-09-30 02:07:19,359][1157819] Updated weights for policy 0, policy_version 105088 (0.0006) [2024-09-30 02:07:19,839][1157819] Updated weights for policy 0, policy_version 105098 (0.0006) [2024-09-30 02:07:20,330][1157819] Updated weights for policy 0, policy_version 105108 (0.0006) [2024-09-30 02:07:20,466][1157520] Fps is (10 sec: 79871.7, 60 sec: 77141.2, 300 sec: 78615.4). Total num frames: 430530560. Throughput: 0: 19374.7. Samples: 97619288. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:07:20,466][1157520] Avg episode reward: [(0, '53.449')] [2024-09-30 02:07:20,801][1157819] Updated weights for policy 0, policy_version 105118 (0.0006) [2024-09-30 02:07:21,349][1157819] Updated weights for policy 0, policy_version 105128 (0.0006) [2024-09-30 02:07:21,836][1157819] Updated weights for policy 0, policy_version 105138 (0.0006) [2024-09-30 02:07:22,362][1157819] Updated weights for policy 0, policy_version 105148 (0.0006) [2024-09-30 02:07:22,878][1157819] Updated weights for policy 0, policy_version 105158 (0.0006) [2024-09-30 02:07:23,393][1157819] Updated weights for policy 0, policy_version 105168 (0.0006) [2024-09-30 02:07:23,896][1157819] Updated weights for policy 0, policy_version 105178 (0.0006) [2024-09-30 02:07:24,417][1157819] Updated weights for policy 0, policy_version 105188 (0.0006) [2024-09-30 02:07:24,905][1157819] Updated weights for policy 0, policy_version 105198 (0.0006) [2024-09-30 02:07:25,466][1157520] Fps is (10 sec: 80281.3, 60 sec: 77346.0, 300 sec: 78684.8). Total num frames: 430927872. Throughput: 0: 19466.7. Samples: 97680064. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:07:25,466][1157520] Avg episode reward: [(0, '55.804')] [2024-09-30 02:07:25,477][1157819] Updated weights for policy 0, policy_version 105208 (0.0006) [2024-09-30 02:07:25,980][1157819] Updated weights for policy 0, policy_version 105218 (0.0006) [2024-09-30 02:07:26,479][1157819] Updated weights for policy 0, policy_version 105228 (0.0006) [2024-09-30 02:07:26,981][1157819] Updated weights for policy 0, policy_version 105238 (0.0006) [2024-09-30 02:07:27,500][1157819] Updated weights for policy 0, policy_version 105248 (0.0006) [2024-09-30 02:07:28,063][1157819] Updated weights for policy 0, policy_version 105258 (0.0006) [2024-09-30 02:07:28,555][1157819] Updated weights for policy 0, policy_version 105268 (0.0006) [2024-09-30 02:07:28,721][1157736] Signal inference workers to stop experience collection... (6850 times) [2024-09-30 02:07:28,723][1157736] Signal inference workers to resume experience collection... (6850 times) [2024-09-30 02:07:28,727][1157819] InferenceWorker_p0-w0: stopping experience collection (6850 times) [2024-09-30 02:07:28,729][1157819] InferenceWorker_p0-w0: resuming experience collection (6850 times) [2024-09-30 02:07:29,078][1157819] Updated weights for policy 0, policy_version 105278 (0.0006) [2024-09-30 02:07:29,601][1157819] Updated weights for policy 0, policy_version 105288 (0.0006) [2024-09-30 02:07:30,171][1157819] Updated weights for policy 0, policy_version 105298 (0.0006) [2024-09-30 02:07:30,466][1157520] Fps is (10 sec: 79052.9, 60 sec: 77687.4, 300 sec: 78782.0). Total num frames: 431321088. Throughput: 0: 19532.0. Samples: 97799128. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:07:30,466][1157520] Avg episode reward: [(0, '53.516')] [2024-09-30 02:07:30,678][1157819] Updated weights for policy 0, policy_version 105308 (0.0006) [2024-09-30 02:07:31,186][1157819] Updated weights for policy 0, policy_version 105318 (0.0006) [2024-09-30 02:07:31,683][1157819] Updated weights for policy 0, policy_version 105328 (0.0006) [2024-09-30 02:07:32,202][1157819] Updated weights for policy 0, policy_version 105338 (0.0006) [2024-09-30 02:07:32,739][1157819] Updated weights for policy 0, policy_version 105348 (0.0006) [2024-09-30 02:07:33,249][1157819] Updated weights for policy 0, policy_version 105358 (0.0006) [2024-09-30 02:07:33,782][1157819] Updated weights for policy 0, policy_version 105368 (0.0006) [2024-09-30 02:07:34,272][1157819] Updated weights for policy 0, policy_version 105378 (0.0006) [2024-09-30 02:07:34,802][1157819] Updated weights for policy 0, policy_version 105388 (0.0006) [2024-09-30 02:07:35,294][1157819] Updated weights for policy 0, policy_version 105398 (0.0006) [2024-09-30 02:07:35,466][1157520] Fps is (10 sec: 79462.2, 60 sec: 78028.7, 300 sec: 78865.3). Total num frames: 431722496. Throughput: 0: 19665.9. Samples: 97917588. Policy #0 lag: (min: 0.0, avg: 1.6, max: 6.0) [2024-09-30 02:07:35,466][1157520] Avg episode reward: [(0, '52.521')] [2024-09-30 02:07:35,792][1157819] Updated weights for policy 0, policy_version 105408 (0.0006) [2024-09-30 02:07:36,337][1157819] Updated weights for policy 0, policy_version 105418 (0.0006) [2024-09-30 02:07:36,826][1157819] Updated weights for policy 0, policy_version 105428 (0.0006) [2024-09-30 02:07:37,342][1157819] Updated weights for policy 0, policy_version 105438 (0.0006) [2024-09-30 02:07:37,842][1157819] Updated weights for policy 0, policy_version 105448 (0.0006) [2024-09-30 02:07:38,381][1157819] Updated weights for policy 0, policy_version 105458 (0.0006) [2024-09-30 02:07:38,907][1157819] Updated weights for policy 0, policy_version 105468 (0.0006) [2024-09-30 02:07:39,436][1157819] Updated weights for policy 0, policy_version 105478 (0.0006) [2024-09-30 02:07:39,930][1157819] Updated weights for policy 0, policy_version 105488 (0.0006) [2024-09-30 02:07:40,448][1157819] Updated weights for policy 0, policy_version 105498 (0.0006) [2024-09-30 02:07:40,466][1157520] Fps is (10 sec: 79871.6, 60 sec: 78233.5, 300 sec: 78920.9). Total num frames: 432119808. Throughput: 0: 19756.4. Samples: 97977268. Policy #0 lag: (min: 0.0, avg: 1.6, max: 6.0) [2024-09-30 02:07:40,466][1157520] Avg episode reward: [(0, '53.891')] [2024-09-30 02:07:40,937][1157819] Updated weights for policy 0, policy_version 105508 (0.0006) [2024-09-30 02:07:41,479][1157819] Updated weights for policy 0, policy_version 105518 (0.0006) [2024-09-30 02:07:41,988][1157819] Updated weights for policy 0, policy_version 105528 (0.0006) [2024-09-30 02:07:42,503][1157819] Updated weights for policy 0, policy_version 105538 (0.0006) [2024-09-30 02:07:42,992][1157819] Updated weights for policy 0, policy_version 105548 (0.0006) [2024-09-30 02:07:43,523][1157819] Updated weights for policy 0, policy_version 105558 (0.0006) [2024-09-30 02:07:44,029][1157819] Updated weights for policy 0, policy_version 105568 (0.0006) [2024-09-30 02:07:44,554][1157819] Updated weights for policy 0, policy_version 105578 (0.0006) [2024-09-30 02:07:45,061][1157819] Updated weights for policy 0, policy_version 105588 (0.0006) [2024-09-30 02:07:45,466][1157520] Fps is (10 sec: 79872.0, 60 sec: 78643.0, 300 sec: 78976.4). Total num frames: 432521216. Throughput: 0: 19866.6. Samples: 98096848. Policy #0 lag: (min: 0.0, avg: 1.6, max: 6.0) [2024-09-30 02:07:45,466][1157520] Avg episode reward: [(0, '52.712')] [2024-09-30 02:07:45,578][1157819] Updated weights for policy 0, policy_version 105598 (0.0006) [2024-09-30 02:07:46,079][1157819] Updated weights for policy 0, policy_version 105608 (0.0006) [2024-09-30 02:07:46,578][1157819] Updated weights for policy 0, policy_version 105618 (0.0006) [2024-09-30 02:07:47,082][1157819] Updated weights for policy 0, policy_version 105628 (0.0006) [2024-09-30 02:07:47,609][1157819] Updated weights for policy 0, policy_version 105638 (0.0006) [2024-09-30 02:07:48,122][1157819] Updated weights for policy 0, policy_version 105648 (0.0006) [2024-09-30 02:07:48,651][1157819] Updated weights for policy 0, policy_version 105658 (0.0006) [2024-09-30 02:07:49,164][1157819] Updated weights for policy 0, policy_version 105668 (0.0006) [2024-09-30 02:07:49,714][1157819] Updated weights for policy 0, policy_version 105678 (0.0006) [2024-09-30 02:07:50,229][1157819] Updated weights for policy 0, policy_version 105688 (0.0006) [2024-09-30 02:07:50,466][1157520] Fps is (10 sec: 79461.7, 60 sec: 78984.3, 300 sec: 78948.6). Total num frames: 432914432. Throughput: 0: 19900.4. Samples: 98216056. Policy #0 lag: (min: 0.0, avg: 1.6, max: 6.0) [2024-09-30 02:07:50,466][1157520] Avg episode reward: [(0, '53.634')] [2024-09-30 02:07:50,750][1157819] Updated weights for policy 0, policy_version 105698 (0.0006) [2024-09-30 02:07:51,260][1157819] Updated weights for policy 0, policy_version 105708 (0.0006) [2024-09-30 02:07:51,774][1157819] Updated weights for policy 0, policy_version 105718 (0.0006) [2024-09-30 02:07:52,300][1157819] Updated weights for policy 0, policy_version 105728 (0.0006) [2024-09-30 02:07:52,832][1157819] Updated weights for policy 0, policy_version 105738 (0.0006) [2024-09-30 02:07:53,341][1157819] Updated weights for policy 0, policy_version 105748 (0.0006) [2024-09-30 02:07:53,872][1157819] Updated weights for policy 0, policy_version 105758 (0.0006) [2024-09-30 02:07:54,381][1157819] Updated weights for policy 0, policy_version 105768 (0.0006) [2024-09-30 02:07:54,937][1157819] Updated weights for policy 0, policy_version 105778 (0.0006) [2024-09-30 02:07:55,466][1157520] Fps is (10 sec: 78233.1, 60 sec: 79257.4, 300 sec: 78865.3). Total num frames: 433303552. Throughput: 0: 19920.7. Samples: 98274968. Policy #0 lag: (min: 0.0, avg: 1.6, max: 6.0) [2024-09-30 02:07:55,466][1157520] Avg episode reward: [(0, '53.024')] [2024-09-30 02:07:55,468][1157819] Updated weights for policy 0, policy_version 105788 (0.0006) [2024-09-30 02:07:55,966][1157819] Updated weights for policy 0, policy_version 105798 (0.0006) [2024-09-30 02:07:56,470][1157819] Updated weights for policy 0, policy_version 105808 (0.0006) [2024-09-30 02:07:57,039][1157819] Updated weights for policy 0, policy_version 105818 (0.0006) [2024-09-30 02:07:57,540][1157819] Updated weights for policy 0, policy_version 105828 (0.0006) [2024-09-30 02:07:58,074][1157819] Updated weights for policy 0, policy_version 105838 (0.0006) [2024-09-30 02:07:58,581][1157819] Updated weights for policy 0, policy_version 105848 (0.0006) [2024-09-30 02:07:59,096][1157819] Updated weights for policy 0, policy_version 105858 (0.0006) [2024-09-30 02:07:59,612][1157819] Updated weights for policy 0, policy_version 105868 (0.0006) [2024-09-30 02:08:00,145][1157819] Updated weights for policy 0, policy_version 105878 (0.0006) [2024-09-30 02:08:00,466][1157520] Fps is (10 sec: 78644.2, 60 sec: 79257.5, 300 sec: 78851.5). Total num frames: 433700864. Throughput: 0: 19860.8. Samples: 98392444. Policy #0 lag: (min: 0.0, avg: 1.6, max: 6.0) [2024-09-30 02:08:00,466][1157520] Avg episode reward: [(0, '53.795')] [2024-09-30 02:08:00,672][1157819] Updated weights for policy 0, policy_version 105888 (0.0006) [2024-09-30 02:08:01,187][1157819] Updated weights for policy 0, policy_version 105898 (0.0006) [2024-09-30 02:08:01,678][1157819] Updated weights for policy 0, policy_version 105908 (0.0006) [2024-09-30 02:08:02,211][1157819] Updated weights for policy 0, policy_version 105918 (0.0006) [2024-09-30 02:08:02,738][1157819] Updated weights for policy 0, policy_version 105928 (0.0006) [2024-09-30 02:08:03,288][1157819] Updated weights for policy 0, policy_version 105938 (0.0006) [2024-09-30 02:08:03,795][1157819] Updated weights for policy 0, policy_version 105948 (0.0006) [2024-09-30 02:08:04,325][1157819] Updated weights for policy 0, policy_version 105958 (0.0006) [2024-09-30 02:08:04,831][1157819] Updated weights for policy 0, policy_version 105968 (0.0006) [2024-09-30 02:08:05,357][1157819] Updated weights for policy 0, policy_version 105978 (0.0006) [2024-09-30 02:08:05,466][1157520] Fps is (10 sec: 79053.2, 60 sec: 79394.0, 300 sec: 78865.3). Total num frames: 434094080. Throughput: 0: 19799.9. Samples: 98510284. Policy #0 lag: (min: 0.0, avg: 1.6, max: 6.0) [2024-09-30 02:08:05,466][1157520] Avg episode reward: [(0, '55.405')] [2024-09-30 02:08:05,471][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000105980_434094080.pth... [2024-09-30 02:08:05,519][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000101393_415305728.pth [2024-09-30 02:08:05,871][1157819] Updated weights for policy 0, policy_version 105988 (0.0006) [2024-09-30 02:08:06,383][1157819] Updated weights for policy 0, policy_version 105998 (0.0006) [2024-09-30 02:08:06,911][1157819] Updated weights for policy 0, policy_version 106008 (0.0006) [2024-09-30 02:08:07,415][1157819] Updated weights for policy 0, policy_version 106018 (0.0006) [2024-09-30 02:08:07,936][1157819] Updated weights for policy 0, policy_version 106028 (0.0006) [2024-09-30 02:08:08,472][1157819] Updated weights for policy 0, policy_version 106038 (0.0006) [2024-09-30 02:08:08,985][1157819] Updated weights for policy 0, policy_version 106048 (0.0006) [2024-09-30 02:08:09,496][1157819] Updated weights for policy 0, policy_version 106058 (0.0006) [2024-09-30 02:08:10,001][1157819] Updated weights for policy 0, policy_version 106068 (0.0006) [2024-09-30 02:08:10,466][1157520] Fps is (10 sec: 79052.7, 60 sec: 79325.8, 300 sec: 78948.6). Total num frames: 434491392. Throughput: 0: 19765.7. Samples: 98569520. Policy #0 lag: (min: 0.0, avg: 1.6, max: 6.0) [2024-09-30 02:08:10,466][1157520] Avg episode reward: [(0, '55.759')] [2024-09-30 02:08:10,488][1157819] Updated weights for policy 0, policy_version 106078 (0.0006) [2024-09-30 02:08:11,033][1157819] Updated weights for policy 0, policy_version 106088 (0.0006) [2024-09-30 02:08:11,534][1157819] Updated weights for policy 0, policy_version 106098 (0.0006) [2024-09-30 02:08:12,048][1157819] Updated weights for policy 0, policy_version 106108 (0.0006) [2024-09-30 02:08:12,565][1157819] Updated weights for policy 0, policy_version 106118 (0.0006) [2024-09-30 02:08:13,107][1157819] Updated weights for policy 0, policy_version 106128 (0.0006) [2024-09-30 02:08:13,612][1157819] Updated weights for policy 0, policy_version 106138 (0.0006) [2024-09-30 02:08:14,125][1157819] Updated weights for policy 0, policy_version 106148 (0.0006) [2024-09-30 02:08:14,672][1157819] Updated weights for policy 0, policy_version 106158 (0.0006) [2024-09-30 02:08:15,195][1157819] Updated weights for policy 0, policy_version 106168 (0.0006) [2024-09-30 02:08:15,466][1157520] Fps is (10 sec: 78642.9, 60 sec: 79257.5, 300 sec: 78962.5). Total num frames: 434880512. Throughput: 0: 19762.1. Samples: 98688424. Policy #0 lag: (min: 0.0, avg: 1.6, max: 6.0) [2024-09-30 02:08:15,466][1157520] Avg episode reward: [(0, '54.007')] [2024-09-30 02:08:15,727][1157819] Updated weights for policy 0, policy_version 106178 (0.0006) [2024-09-30 02:08:16,213][1157819] Updated weights for policy 0, policy_version 106188 (0.0006) [2024-09-30 02:08:16,715][1157819] Updated weights for policy 0, policy_version 106198 (0.0006) [2024-09-30 02:08:17,224][1157819] Updated weights for policy 0, policy_version 106208 (0.0006) [2024-09-30 02:08:17,710][1157819] Updated weights for policy 0, policy_version 106218 (0.0006) [2024-09-30 02:08:18,224][1157819] Updated weights for policy 0, policy_version 106228 (0.0006) [2024-09-30 02:08:18,741][1157819] Updated weights for policy 0, policy_version 106238 (0.0006) [2024-09-30 02:08:19,288][1157819] Updated weights for policy 0, policy_version 106248 (0.0006) [2024-09-30 02:08:19,791][1157819] Updated weights for policy 0, policy_version 106258 (0.0006) [2024-09-30 02:08:20,293][1157819] Updated weights for policy 0, policy_version 106268 (0.0006) [2024-09-30 02:08:20,466][1157520] Fps is (10 sec: 79463.2, 60 sec: 79257.7, 300 sec: 78990.3). Total num frames: 435286016. Throughput: 0: 19790.6. Samples: 98808164. Policy #0 lag: (min: 0.0, avg: 1.6, max: 6.0) [2024-09-30 02:08:20,466][1157520] Avg episode reward: [(0, '54.497')] [2024-09-30 02:08:20,815][1157819] Updated weights for policy 0, policy_version 106278 (0.0006) [2024-09-30 02:08:21,354][1157819] Updated weights for policy 0, policy_version 106288 (0.0006) [2024-09-30 02:08:21,841][1157819] Updated weights for policy 0, policy_version 106298 (0.0006) [2024-09-30 02:08:22,361][1157819] Updated weights for policy 0, policy_version 106308 (0.0006) [2024-09-30 02:08:22,912][1157819] Updated weights for policy 0, policy_version 106318 (0.0006) [2024-09-30 02:08:23,082][1157736] Signal inference workers to stop experience collection... (6900 times) [2024-09-30 02:08:23,085][1157819] InferenceWorker_p0-w0: stopping experience collection (6900 times) [2024-09-30 02:08:23,088][1157736] Signal inference workers to resume experience collection... (6900 times) [2024-09-30 02:08:23,088][1157819] InferenceWorker_p0-w0: resuming experience collection (6900 times) [2024-09-30 02:08:23,437][1157819] Updated weights for policy 0, policy_version 106328 (0.0006) [2024-09-30 02:08:23,934][1157819] Updated weights for policy 0, policy_version 106338 (0.0006) [2024-09-30 02:08:24,430][1157819] Updated weights for policy 0, policy_version 106348 (0.0006) [2024-09-30 02:08:24,932][1157819] Updated weights for policy 0, policy_version 106358 (0.0006) [2024-09-30 02:08:25,464][1157819] Updated weights for policy 0, policy_version 106368 (0.0006) [2024-09-30 02:08:25,466][1157520] Fps is (10 sec: 80282.9, 60 sec: 79257.7, 300 sec: 78920.9). Total num frames: 435683328. Throughput: 0: 19781.2. Samples: 98867420. Policy #0 lag: (min: 0.0, avg: 1.6, max: 6.0) [2024-09-30 02:08:25,466][1157520] Avg episode reward: [(0, '54.779')] [2024-09-30 02:08:25,967][1157819] Updated weights for policy 0, policy_version 106378 (0.0006) [2024-09-30 02:08:26,473][1157819] Updated weights for policy 0, policy_version 106388 (0.0006) [2024-09-30 02:08:27,026][1157819] Updated weights for policy 0, policy_version 106398 (0.0006) [2024-09-30 02:08:27,514][1157819] Updated weights for policy 0, policy_version 106408 (0.0006) [2024-09-30 02:08:28,036][1157819] Updated weights for policy 0, policy_version 106418 (0.0006) [2024-09-30 02:08:28,510][1157819] Updated weights for policy 0, policy_version 106428 (0.0006) [2024-09-30 02:08:29,006][1157819] Updated weights for policy 0, policy_version 106438 (0.0006) [2024-09-30 02:08:29,510][1157819] Updated weights for policy 0, policy_version 106448 (0.0006) [2024-09-30 02:08:29,986][1157819] Updated weights for policy 0, policy_version 106458 (0.0006) [2024-09-30 02:08:30,466][1157520] Fps is (10 sec: 80281.5, 60 sec: 79462.5, 300 sec: 78907.0). Total num frames: 436088832. Throughput: 0: 19798.0. Samples: 98987756. Policy #0 lag: (min: 0.0, avg: 1.6, max: 6.0) [2024-09-30 02:08:30,466][1157520] Avg episode reward: [(0, '55.674')] [2024-09-30 02:08:30,490][1157819] Updated weights for policy 0, policy_version 106468 (0.0006) [2024-09-30 02:08:30,985][1157819] Updated weights for policy 0, policy_version 106478 (0.0006) [2024-09-30 02:08:31,471][1157819] Updated weights for policy 0, policy_version 106488 (0.0006) [2024-09-30 02:08:31,966][1157819] Updated weights for policy 0, policy_version 106498 (0.0006) [2024-09-30 02:08:32,476][1157819] Updated weights for policy 0, policy_version 106508 (0.0006) [2024-09-30 02:08:33,029][1157819] Updated weights for policy 0, policy_version 106518 (0.0006) [2024-09-30 02:08:33,530][1157819] Updated weights for policy 0, policy_version 106528 (0.0006) [2024-09-30 02:08:34,068][1157819] Updated weights for policy 0, policy_version 106538 (0.0006) [2024-09-30 02:08:34,554][1157819] Updated weights for policy 0, policy_version 106548 (0.0006) [2024-09-30 02:08:35,103][1157819] Updated weights for policy 0, policy_version 106558 (0.0006) [2024-09-30 02:08:35,466][1157520] Fps is (10 sec: 80691.2, 60 sec: 79462.5, 300 sec: 78837.6). Total num frames: 436490240. Throughput: 0: 19858.1. Samples: 99109668. Policy #0 lag: (min: 0.0, avg: 1.6, max: 6.0) [2024-09-30 02:08:35,466][1157520] Avg episode reward: [(0, '53.902')] [2024-09-30 02:08:35,647][1157819] Updated weights for policy 0, policy_version 106568 (0.0006) [2024-09-30 02:08:36,168][1157819] Updated weights for policy 0, policy_version 106578 (0.0006) [2024-09-30 02:08:36,702][1157819] Updated weights for policy 0, policy_version 106588 (0.0006) [2024-09-30 02:08:37,215][1157819] Updated weights for policy 0, policy_version 106598 (0.0006) [2024-09-30 02:08:37,816][1157819] Updated weights for policy 0, policy_version 106608 (0.0006) [2024-09-30 02:08:38,419][1157819] Updated weights for policy 0, policy_version 106618 (0.0007) [2024-09-30 02:08:38,946][1157819] Updated weights for policy 0, policy_version 106628 (0.0007) [2024-09-30 02:08:39,499][1157819] Updated weights for policy 0, policy_version 106638 (0.0007) [2024-09-30 02:08:40,095][1157819] Updated weights for policy 0, policy_version 106648 (0.0007) [2024-09-30 02:08:40,466][1157520] Fps is (10 sec: 76594.0, 60 sec: 78916.2, 300 sec: 78629.3). Total num frames: 436854784. Throughput: 0: 19797.6. Samples: 99165860. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 02:08:40,466][1157520] Avg episode reward: [(0, '52.967')] [2024-09-30 02:08:40,671][1157819] Updated weights for policy 0, policy_version 106658 (0.0007) [2024-09-30 02:08:41,286][1157819] Updated weights for policy 0, policy_version 106668 (0.0007) [2024-09-30 02:08:41,826][1157819] Updated weights for policy 0, policy_version 106678 (0.0006) [2024-09-30 02:08:42,425][1157819] Updated weights for policy 0, policy_version 106688 (0.0006) [2024-09-30 02:08:42,996][1157819] Updated weights for policy 0, policy_version 106698 (0.0006) [2024-09-30 02:08:43,569][1157819] Updated weights for policy 0, policy_version 106708 (0.0006) [2024-09-30 02:08:44,065][1157819] Updated weights for policy 0, policy_version 106718 (0.0006) [2024-09-30 02:08:44,572][1157819] Updated weights for policy 0, policy_version 106728 (0.0006) [2024-09-30 02:08:45,091][1157819] Updated weights for policy 0, policy_version 106738 (0.0006) [2024-09-30 02:08:45,466][1157520] Fps is (10 sec: 73727.1, 60 sec: 78438.4, 300 sec: 78476.6). Total num frames: 437227520. Throughput: 0: 19581.8. Samples: 99273624. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 02:08:45,466][1157520] Avg episode reward: [(0, '56.092')] [2024-09-30 02:08:45,603][1157819] Updated weights for policy 0, policy_version 106748 (0.0006) [2024-09-30 02:08:46,102][1157819] Updated weights for policy 0, policy_version 106758 (0.0006) [2024-09-30 02:08:46,643][1157819] Updated weights for policy 0, policy_version 106768 (0.0006) [2024-09-30 02:08:47,196][1157819] Updated weights for policy 0, policy_version 106778 (0.0006) [2024-09-30 02:08:47,729][1157819] Updated weights for policy 0, policy_version 106788 (0.0006) [2024-09-30 02:08:48,304][1157819] Updated weights for policy 0, policy_version 106798 (0.0006) [2024-09-30 02:08:48,839][1157819] Updated weights for policy 0, policy_version 106808 (0.0006) [2024-09-30 02:08:49,364][1157819] Updated weights for policy 0, policy_version 106818 (0.0006) [2024-09-30 02:08:49,913][1157819] Updated weights for policy 0, policy_version 106828 (0.0006) [2024-09-30 02:08:50,438][1157819] Updated weights for policy 0, policy_version 106838 (0.0006) [2024-09-30 02:08:50,466][1157520] Fps is (10 sec: 75366.4, 60 sec: 78233.7, 300 sec: 78379.4). Total num frames: 437608448. Throughput: 0: 19536.8. Samples: 99389440. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 02:08:50,466][1157520] Avg episode reward: [(0, '51.993')] [2024-09-30 02:08:51,010][1157819] Updated weights for policy 0, policy_version 106848 (0.0006) [2024-09-30 02:08:51,568][1157819] Updated weights for policy 0, policy_version 106858 (0.0006) [2024-09-30 02:08:52,121][1157819] Updated weights for policy 0, policy_version 106868 (0.0006) [2024-09-30 02:08:52,692][1157819] Updated weights for policy 0, policy_version 106878 (0.0006) [2024-09-30 02:08:53,205][1157819] Updated weights for policy 0, policy_version 106888 (0.0006) [2024-09-30 02:08:53,751][1157819] Updated weights for policy 0, policy_version 106898 (0.0006) [2024-09-30 02:08:54,308][1157819] Updated weights for policy 0, policy_version 106908 (0.0006) [2024-09-30 02:08:54,869][1157819] Updated weights for policy 0, policy_version 106918 (0.0006) [2024-09-30 02:08:55,422][1157819] Updated weights for policy 0, policy_version 106928 (0.0006) [2024-09-30 02:08:55,466][1157520] Fps is (10 sec: 75365.7, 60 sec: 77960.5, 300 sec: 78268.2). Total num frames: 437981184. Throughput: 0: 19464.2. Samples: 99445412. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 02:08:55,466][1157520] Avg episode reward: [(0, '55.595')] [2024-09-30 02:08:55,956][1157819] Updated weights for policy 0, policy_version 106938 (0.0006) [2024-09-30 02:08:56,518][1157819] Updated weights for policy 0, policy_version 106948 (0.0006) [2024-09-30 02:08:57,048][1157819] Updated weights for policy 0, policy_version 106958 (0.0006) [2024-09-30 02:08:57,606][1157819] Updated weights for policy 0, policy_version 106968 (0.0006) [2024-09-30 02:08:58,156][1157819] Updated weights for policy 0, policy_version 106978 (0.0006) [2024-09-30 02:08:58,675][1157819] Updated weights for policy 0, policy_version 106988 (0.0006) [2024-09-30 02:08:59,210][1157819] Updated weights for policy 0, policy_version 106998 (0.0006) [2024-09-30 02:08:59,792][1157819] Updated weights for policy 0, policy_version 107008 (0.0006) [2024-09-30 02:09:00,293][1157819] Updated weights for policy 0, policy_version 107018 (0.0006) [2024-09-30 02:09:00,466][1157520] Fps is (10 sec: 74957.4, 60 sec: 77619.2, 300 sec: 78143.3). Total num frames: 438358016. Throughput: 0: 19317.1. Samples: 99557692. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 02:09:00,466][1157520] Avg episode reward: [(0, '53.966')] [2024-09-30 02:09:00,602][1157736] Signal inference workers to stop experience collection... (6950 times) [2024-09-30 02:09:00,602][1157736] Signal inference workers to resume experience collection... (6950 times) [2024-09-30 02:09:00,605][1157819] InferenceWorker_p0-w0: stopping experience collection (6950 times) [2024-09-30 02:09:00,606][1157819] InferenceWorker_p0-w0: resuming experience collection (6950 times) [2024-09-30 02:09:00,840][1157819] Updated weights for policy 0, policy_version 107028 (0.0006) [2024-09-30 02:09:01,337][1157819] Updated weights for policy 0, policy_version 107038 (0.0006) [2024-09-30 02:09:01,844][1157819] Updated weights for policy 0, policy_version 107048 (0.0006) [2024-09-30 02:09:02,418][1157819] Updated weights for policy 0, policy_version 107058 (0.0006) [2024-09-30 02:09:02,937][1157819] Updated weights for policy 0, policy_version 107068 (0.0006) [2024-09-30 02:09:03,437][1157819] Updated weights for policy 0, policy_version 107078 (0.0006) [2024-09-30 02:09:03,946][1157819] Updated weights for policy 0, policy_version 107088 (0.0006) [2024-09-30 02:09:04,467][1157819] Updated weights for policy 0, policy_version 107098 (0.0006) [2024-09-30 02:09:04,983][1157819] Updated weights for policy 0, policy_version 107108 (0.0006) [2024-09-30 02:09:05,466][1157520] Fps is (10 sec: 77005.7, 60 sec: 77619.2, 300 sec: 78226.6). Total num frames: 438751232. Throughput: 0: 19257.6. Samples: 99674760. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 02:09:05,466][1157520] Avg episode reward: [(0, '55.955')] [2024-09-30 02:09:05,483][1157819] Updated weights for policy 0, policy_version 107118 (0.0006) [2024-09-30 02:09:06,003][1157819] Updated weights for policy 0, policy_version 107128 (0.0006) [2024-09-30 02:09:06,528][1157819] Updated weights for policy 0, policy_version 107138 (0.0006) [2024-09-30 02:09:07,035][1157819] Updated weights for policy 0, policy_version 107148 (0.0006) [2024-09-30 02:09:07,525][1157819] Updated weights for policy 0, policy_version 107158 (0.0006) [2024-09-30 02:09:08,025][1157819] Updated weights for policy 0, policy_version 107168 (0.0006) [2024-09-30 02:09:08,566][1157819] Updated weights for policy 0, policy_version 107178 (0.0006) [2024-09-30 02:09:09,060][1157819] Updated weights for policy 0, policy_version 107188 (0.0007) [2024-09-30 02:09:09,560][1157819] Updated weights for policy 0, policy_version 107198 (0.0006) [2024-09-30 02:09:10,055][1157819] Updated weights for policy 0, policy_version 107208 (0.0006) [2024-09-30 02:09:10,466][1157520] Fps is (10 sec: 79871.5, 60 sec: 77755.7, 300 sec: 78351.6). Total num frames: 439156736. Throughput: 0: 19273.9. Samples: 99734748. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 02:09:10,466][1157520] Avg episode reward: [(0, '52.953')] [2024-09-30 02:09:10,539][1157819] Updated weights for policy 0, policy_version 107218 (0.0006) [2024-09-30 02:09:11,033][1157819] Updated weights for policy 0, policy_version 107228 (0.0006) [2024-09-30 02:09:11,608][1157819] Updated weights for policy 0, policy_version 107238 (0.0006) [2024-09-30 02:09:12,183][1157819] Updated weights for policy 0, policy_version 107248 (0.0007) [2024-09-30 02:09:12,673][1157819] Updated weights for policy 0, policy_version 107258 (0.0006) [2024-09-30 02:09:13,215][1157819] Updated weights for policy 0, policy_version 107268 (0.0006) [2024-09-30 02:09:13,775][1157819] Updated weights for policy 0, policy_version 107278 (0.0006) [2024-09-30 02:09:14,271][1157819] Updated weights for policy 0, policy_version 107288 (0.0007) [2024-09-30 02:09:14,819][1157819] Updated weights for policy 0, policy_version 107298 (0.0007) [2024-09-30 02:09:15,364][1157819] Updated weights for policy 0, policy_version 107308 (0.0007) [2024-09-30 02:09:15,466][1157520] Fps is (10 sec: 79052.7, 60 sec: 77687.5, 300 sec: 78351.6). Total num frames: 439541760. Throughput: 0: 19227.4. Samples: 99852992. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 02:09:15,466][1157520] Avg episode reward: [(0, '54.857')] [2024-09-30 02:09:15,887][1157819] Updated weights for policy 0, policy_version 107318 (0.0007) [2024-09-30 02:09:16,462][1157819] Updated weights for policy 0, policy_version 107328 (0.0006) [2024-09-30 02:09:17,053][1157819] Updated weights for policy 0, policy_version 107338 (0.0006) [2024-09-30 02:09:17,632][1157819] Updated weights for policy 0, policy_version 107348 (0.0006) [2024-09-30 02:09:18,179][1157819] Updated weights for policy 0, policy_version 107358 (0.0006) [2024-09-30 02:09:18,733][1157819] Updated weights for policy 0, policy_version 107368 (0.0006) [2024-09-30 02:09:19,259][1157819] Updated weights for policy 0, policy_version 107378 (0.0006) [2024-09-30 02:09:19,847][1157819] Updated weights for policy 0, policy_version 107388 (0.0006) [2024-09-30 02:09:20,377][1157819] Updated weights for policy 0, policy_version 107398 (0.0006) [2024-09-30 02:09:20,466][1157520] Fps is (10 sec: 74957.2, 60 sec: 77004.7, 300 sec: 78268.3). Total num frames: 439906304. Throughput: 0: 18999.8. Samples: 99964660. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 02:09:20,466][1157520] Avg episode reward: [(0, '53.297')] [2024-09-30 02:09:20,881][1157819] Updated weights for policy 0, policy_version 107408 (0.0006) [2024-09-30 02:09:21,424][1157819] Updated weights for policy 0, policy_version 107418 (0.0006) [2024-09-30 02:09:21,896][1157819] Updated weights for policy 0, policy_version 107428 (0.0006) [2024-09-30 02:09:22,394][1157819] Updated weights for policy 0, policy_version 107438 (0.0006) [2024-09-30 02:09:22,883][1157819] Updated weights for policy 0, policy_version 107448 (0.0006) [2024-09-30 02:09:23,381][1157819] Updated weights for policy 0, policy_version 107458 (0.0006) [2024-09-30 02:09:23,886][1157819] Updated weights for policy 0, policy_version 107468 (0.0006) [2024-09-30 02:09:24,455][1157819] Updated weights for policy 0, policy_version 107478 (0.0006) [2024-09-30 02:09:24,999][1157819] Updated weights for policy 0, policy_version 107488 (0.0006) [2024-09-30 02:09:25,466][1157520] Fps is (10 sec: 76185.7, 60 sec: 77004.7, 300 sec: 78310.0). Total num frames: 440303616. Throughput: 0: 19079.5. Samples: 100024436. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 02:09:25,466][1157520] Avg episode reward: [(0, '54.521')] [2024-09-30 02:09:25,542][1157819] Updated weights for policy 0, policy_version 107498 (0.0006) [2024-09-30 02:09:26,073][1157819] Updated weights for policy 0, policy_version 107508 (0.0006) [2024-09-30 02:09:26,626][1157819] Updated weights for policy 0, policy_version 107518 (0.0006) [2024-09-30 02:09:27,204][1157819] Updated weights for policy 0, policy_version 107528 (0.0006) [2024-09-30 02:09:27,716][1157819] Updated weights for policy 0, policy_version 107538 (0.0006) [2024-09-30 02:09:28,260][1157819] Updated weights for policy 0, policy_version 107548 (0.0006) [2024-09-30 02:09:28,803][1157819] Updated weights for policy 0, policy_version 107558 (0.0006) [2024-09-30 02:09:29,354][1157819] Updated weights for policy 0, policy_version 107568 (0.0006) [2024-09-30 02:09:29,896][1157819] Updated weights for policy 0, policy_version 107578 (0.0006) [2024-09-30 02:09:30,409][1157736] Signal inference workers to stop experience collection... (7000 times) [2024-09-30 02:09:30,411][1157819] InferenceWorker_p0-w0: stopping experience collection (7000 times) [2024-09-30 02:09:30,419][1157736] Signal inference workers to resume experience collection... (7000 times) [2024-09-30 02:09:30,420][1157819] InferenceWorker_p0-w0: resuming experience collection (7000 times) [2024-09-30 02:09:30,440][1157819] Updated weights for policy 0, policy_version 107588 (0.0007) [2024-09-30 02:09:30,466][1157520] Fps is (10 sec: 77414.6, 60 sec: 76526.9, 300 sec: 78254.5). Total num frames: 440680448. Throughput: 0: 19234.5. Samples: 100139176. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 02:09:30,466][1157520] Avg episode reward: [(0, '52.226')] [2024-09-30 02:09:31,070][1157819] Updated weights for policy 0, policy_version 107598 (0.0006) [2024-09-30 02:09:31,703][1157819] Updated weights for policy 0, policy_version 107608 (0.0006) [2024-09-30 02:09:32,356][1157819] Updated weights for policy 0, policy_version 107618 (0.0006) [2024-09-30 02:09:32,925][1157819] Updated weights for policy 0, policy_version 107628 (0.0006) [2024-09-30 02:09:33,551][1157819] Updated weights for policy 0, policy_version 107638 (0.0006) [2024-09-30 02:09:34,144][1157819] Updated weights for policy 0, policy_version 107648 (0.0006) [2024-09-30 02:09:34,742][1157819] Updated weights for policy 0, policy_version 107658 (0.0006) [2024-09-30 02:09:35,285][1157819] Updated weights for policy 0, policy_version 107668 (0.0006) [2024-09-30 02:09:35,466][1157520] Fps is (10 sec: 71680.8, 60 sec: 75503.0, 300 sec: 78060.1). Total num frames: 441020416. Throughput: 0: 18970.1. Samples: 100243092. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 02:09:35,466][1157520] Avg episode reward: [(0, '55.653')] [2024-09-30 02:09:35,815][1157819] Updated weights for policy 0, policy_version 107678 (0.0006) [2024-09-30 02:09:36,365][1157819] Updated weights for policy 0, policy_version 107688 (0.0006) [2024-09-30 02:09:36,926][1157819] Updated weights for policy 0, policy_version 107698 (0.0006) [2024-09-30 02:09:37,462][1157819] Updated weights for policy 0, policy_version 107708 (0.0006) [2024-09-30 02:09:37,994][1157819] Updated weights for policy 0, policy_version 107718 (0.0006) [2024-09-30 02:09:38,587][1157819] Updated weights for policy 0, policy_version 107728 (0.0006) [2024-09-30 02:09:39,086][1157819] Updated weights for policy 0, policy_version 107738 (0.0006) [2024-09-30 02:09:39,617][1157819] Updated weights for policy 0, policy_version 107748 (0.0006) [2024-09-30 02:09:40,171][1157819] Updated weights for policy 0, policy_version 107758 (0.0006) [2024-09-30 02:09:40,466][1157520] Fps is (10 sec: 71680.4, 60 sec: 75707.9, 300 sec: 77990.6). Total num frames: 441397248. Throughput: 0: 18974.5. Samples: 100299260. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 02:09:40,466][1157520] Avg episode reward: [(0, '55.712')] [2024-09-30 02:09:40,714][1157819] Updated weights for policy 0, policy_version 107768 (0.0006) [2024-09-30 02:09:41,295][1157819] Updated weights for policy 0, policy_version 107778 (0.0006) [2024-09-30 02:09:41,859][1157819] Updated weights for policy 0, policy_version 107788 (0.0007) [2024-09-30 02:09:42,412][1157819] Updated weights for policy 0, policy_version 107798 (0.0006) [2024-09-30 02:09:42,994][1157819] Updated weights for policy 0, policy_version 107808 (0.0006) [2024-09-30 02:09:43,537][1157819] Updated weights for policy 0, policy_version 107818 (0.0006) [2024-09-30 02:09:44,095][1157819] Updated weights for policy 0, policy_version 107828 (0.0006) [2024-09-30 02:09:44,659][1157819] Updated weights for policy 0, policy_version 107838 (0.0006) [2024-09-30 02:09:45,185][1157819] Updated weights for policy 0, policy_version 107848 (0.0006) [2024-09-30 02:09:45,466][1157520] Fps is (10 sec: 74547.2, 60 sec: 75639.6, 300 sec: 77796.2). Total num frames: 441765888. Throughput: 0: 18944.5. Samples: 100410192. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 02:09:45,466][1157520] Avg episode reward: [(0, '56.472')] [2024-09-30 02:09:45,710][1157819] Updated weights for policy 0, policy_version 107858 (0.0006) [2024-09-30 02:09:46,275][1157819] Updated weights for policy 0, policy_version 107868 (0.0006) [2024-09-30 02:09:46,815][1157819] Updated weights for policy 0, policy_version 107878 (0.0006) [2024-09-30 02:09:47,327][1157819] Updated weights for policy 0, policy_version 107888 (0.0006) [2024-09-30 02:09:47,830][1157819] Updated weights for policy 0, policy_version 107898 (0.0006) [2024-09-30 02:09:48,345][1157819] Updated weights for policy 0, policy_version 107908 (0.0006) [2024-09-30 02:09:48,890][1157819] Updated weights for policy 0, policy_version 107918 (0.0007) [2024-09-30 02:09:49,433][1157819] Updated weights for policy 0, policy_version 107928 (0.0007) [2024-09-30 02:09:49,992][1157819] Updated weights for policy 0, policy_version 107938 (0.0006) [2024-09-30 02:09:50,466][1157520] Fps is (10 sec: 75366.5, 60 sec: 75707.9, 300 sec: 77671.3). Total num frames: 442150912. Throughput: 0: 18893.3. Samples: 100524956. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 02:09:50,466][1157520] Avg episode reward: [(0, '54.385')] [2024-09-30 02:09:50,524][1157819] Updated weights for policy 0, policy_version 107948 (0.0006) [2024-09-30 02:09:51,060][1157819] Updated weights for policy 0, policy_version 107958 (0.0006) [2024-09-30 02:09:51,571][1157819] Updated weights for policy 0, policy_version 107968 (0.0006) [2024-09-30 02:09:52,081][1157819] Updated weights for policy 0, policy_version 107978 (0.0006) [2024-09-30 02:09:52,587][1157819] Updated weights for policy 0, policy_version 107988 (0.0006) [2024-09-30 02:09:53,152][1157819] Updated weights for policy 0, policy_version 107998 (0.0006) [2024-09-30 02:09:53,700][1157819] Updated weights for policy 0, policy_version 108008 (0.0006) [2024-09-30 02:09:54,254][1157819] Updated weights for policy 0, policy_version 108018 (0.0006) [2024-09-30 02:09:54,789][1157819] Updated weights for policy 0, policy_version 108028 (0.0006) [2024-09-30 02:09:55,280][1157819] Updated weights for policy 0, policy_version 108038 (0.0006) [2024-09-30 02:09:55,466][1157520] Fps is (10 sec: 77004.9, 60 sec: 75912.8, 300 sec: 77601.8). Total num frames: 442535936. Throughput: 0: 18845.5. Samples: 100582792. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 02:09:55,466][1157520] Avg episode reward: [(0, '55.609')] [2024-09-30 02:09:55,818][1157819] Updated weights for policy 0, policy_version 108048 (0.0006) [2024-09-30 02:09:56,017][1157736] Signal inference workers to stop experience collection... (7050 times) [2024-09-30 02:09:56,017][1157736] Signal inference workers to resume experience collection... (7050 times) [2024-09-30 02:09:56,021][1157819] InferenceWorker_p0-w0: stopping experience collection (7050 times) [2024-09-30 02:09:56,021][1157819] InferenceWorker_p0-w0: resuming experience collection (7050 times) [2024-09-30 02:09:56,324][1157819] Updated weights for policy 0, policy_version 108058 (0.0006) [2024-09-30 02:09:56,831][1157819] Updated weights for policy 0, policy_version 108068 (0.0006) [2024-09-30 02:09:57,356][1157819] Updated weights for policy 0, policy_version 108078 (0.0006) [2024-09-30 02:09:57,870][1157819] Updated weights for policy 0, policy_version 108088 (0.0006) [2024-09-30 02:09:58,409][1157819] Updated weights for policy 0, policy_version 108098 (0.0006) [2024-09-30 02:09:58,914][1157819] Updated weights for policy 0, policy_version 108108 (0.0006) [2024-09-30 02:09:59,431][1157819] Updated weights for policy 0, policy_version 108118 (0.0006) [2024-09-30 02:09:59,941][1157819] Updated weights for policy 0, policy_version 108128 (0.0006) [2024-09-30 02:10:00,435][1157819] Updated weights for policy 0, policy_version 108138 (0.0006) [2024-09-30 02:10:00,466][1157520] Fps is (10 sec: 78233.6, 60 sec: 76254.0, 300 sec: 77588.0). Total num frames: 442933248. Throughput: 0: 18824.9. Samples: 100700108. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 02:10:00,466][1157520] Avg episode reward: [(0, '53.736')] [2024-09-30 02:10:00,952][1157819] Updated weights for policy 0, policy_version 108148 (0.0006) [2024-09-30 02:10:01,479][1157819] Updated weights for policy 0, policy_version 108158 (0.0006) [2024-09-30 02:10:02,007][1157819] Updated weights for policy 0, policy_version 108168 (0.0006) [2024-09-30 02:10:02,507][1157819] Updated weights for policy 0, policy_version 108178 (0.0006) [2024-09-30 02:10:02,998][1157819] Updated weights for policy 0, policy_version 108188 (0.0006) [2024-09-30 02:10:03,505][1157819] Updated weights for policy 0, policy_version 108198 (0.0006) [2024-09-30 02:10:03,999][1157819] Updated weights for policy 0, policy_version 108208 (0.0006) [2024-09-30 02:10:04,497][1157819] Updated weights for policy 0, policy_version 108218 (0.0006) [2024-09-30 02:10:04,980][1157819] Updated weights for policy 0, policy_version 108228 (0.0006) [2024-09-30 02:10:05,466][1157520] Fps is (10 sec: 80281.3, 60 sec: 76458.8, 300 sec: 77588.0). Total num frames: 443338752. Throughput: 0: 19039.2. Samples: 100821424. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 02:10:05,466][1157520] Avg episode reward: [(0, '53.749')] [2024-09-30 02:10:05,469][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000108237_443338752.pth... [2024-09-30 02:10:05,517][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000103701_424759296.pth [2024-09-30 02:10:05,536][1157819] Updated weights for policy 0, policy_version 108238 (0.0006) [2024-09-30 02:10:06,082][1157819] Updated weights for policy 0, policy_version 108248 (0.0006) [2024-09-30 02:10:06,588][1157819] Updated weights for policy 0, policy_version 108258 (0.0006) [2024-09-30 02:10:07,096][1157819] Updated weights for policy 0, policy_version 108268 (0.0006) [2024-09-30 02:10:07,609][1157819] Updated weights for policy 0, policy_version 108278 (0.0006) [2024-09-30 02:10:08,109][1157819] Updated weights for policy 0, policy_version 108288 (0.0006) [2024-09-30 02:10:08,616][1157819] Updated weights for policy 0, policy_version 108298 (0.0006) [2024-09-30 02:10:09,113][1157819] Updated weights for policy 0, policy_version 108308 (0.0006) [2024-09-30 02:10:09,615][1157819] Updated weights for policy 0, policy_version 108318 (0.0006) [2024-09-30 02:10:10,122][1157819] Updated weights for policy 0, policy_version 108328 (0.0006) [2024-09-30 02:10:10,466][1157520] Fps is (10 sec: 80690.8, 60 sec: 76390.5, 300 sec: 77588.0). Total num frames: 443740160. Throughput: 0: 19014.8. Samples: 100880100. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 02:10:10,466][1157520] Avg episode reward: [(0, '55.144')] [2024-09-30 02:10:10,623][1157819] Updated weights for policy 0, policy_version 108338 (0.0006) [2024-09-30 02:10:11,114][1157819] Updated weights for policy 0, policy_version 108348 (0.0006) [2024-09-30 02:10:11,632][1157819] Updated weights for policy 0, policy_version 108358 (0.0006) [2024-09-30 02:10:12,127][1157819] Updated weights for policy 0, policy_version 108368 (0.0006) [2024-09-30 02:10:12,629][1157819] Updated weights for policy 0, policy_version 108378 (0.0006) [2024-09-30 02:10:13,132][1157819] Updated weights for policy 0, policy_version 108388 (0.0006) [2024-09-30 02:10:13,682][1157819] Updated weights for policy 0, policy_version 108398 (0.0006) [2024-09-30 02:10:14,174][1157819] Updated weights for policy 0, policy_version 108408 (0.0006) [2024-09-30 02:10:14,718][1157819] Updated weights for policy 0, policy_version 108418 (0.0006) [2024-09-30 02:10:15,224][1157819] Updated weights for policy 0, policy_version 108428 (0.0006) [2024-09-30 02:10:15,466][1157520] Fps is (10 sec: 79872.2, 60 sec: 76595.4, 300 sec: 77532.4). Total num frames: 444137472. Throughput: 0: 19171.1. Samples: 101001876. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 02:10:15,466][1157520] Avg episode reward: [(0, '51.967')] [2024-09-30 02:10:15,742][1157819] Updated weights for policy 0, policy_version 108438 (0.0006) [2024-09-30 02:10:16,242][1157819] Updated weights for policy 0, policy_version 108448 (0.0006) [2024-09-30 02:10:16,771][1157819] Updated weights for policy 0, policy_version 108458 (0.0006) [2024-09-30 02:10:17,312][1157819] Updated weights for policy 0, policy_version 108468 (0.0006) [2024-09-30 02:10:17,837][1157819] Updated weights for policy 0, policy_version 108478 (0.0006) [2024-09-30 02:10:18,353][1157819] Updated weights for policy 0, policy_version 108488 (0.0006) [2024-09-30 02:10:18,847][1157819] Updated weights for policy 0, policy_version 108498 (0.0006) [2024-09-30 02:10:19,367][1157819] Updated weights for policy 0, policy_version 108508 (0.0006) [2024-09-30 02:10:19,891][1157819] Updated weights for policy 0, policy_version 108518 (0.0006) [2024-09-30 02:10:20,391][1157819] Updated weights for policy 0, policy_version 108528 (0.0006) [2024-09-30 02:10:20,466][1157520] Fps is (10 sec: 79462.8, 60 sec: 77141.5, 300 sec: 77518.6). Total num frames: 444534784. Throughput: 0: 19500.7. Samples: 101120624. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 02:10:20,466][1157520] Avg episode reward: [(0, '54.016')] [2024-09-30 02:10:20,885][1157819] Updated weights for policy 0, policy_version 108538 (0.0006) [2024-09-30 02:10:21,418][1157819] Updated weights for policy 0, policy_version 108548 (0.0006) [2024-09-30 02:10:21,945][1157819] Updated weights for policy 0, policy_version 108558 (0.0006) [2024-09-30 02:10:22,495][1157819] Updated weights for policy 0, policy_version 108568 (0.0006) [2024-09-30 02:10:23,030][1157819] Updated weights for policy 0, policy_version 108578 (0.0006) [2024-09-30 02:10:23,568][1157819] Updated weights for policy 0, policy_version 108588 (0.0006) [2024-09-30 02:10:24,078][1157819] Updated weights for policy 0, policy_version 108598 (0.0006) [2024-09-30 02:10:24,637][1157819] Updated weights for policy 0, policy_version 108608 (0.0006) [2024-09-30 02:10:25,224][1157819] Updated weights for policy 0, policy_version 108618 (0.0006) [2024-09-30 02:10:25,466][1157520] Fps is (10 sec: 77823.3, 60 sec: 76868.3, 300 sec: 77546.3). Total num frames: 444915712. Throughput: 0: 19567.3. Samples: 101179788. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 02:10:25,466][1157520] Avg episode reward: [(0, '50.836')] [2024-09-30 02:10:25,759][1157819] Updated weights for policy 0, policy_version 108628 (0.0006) [2024-09-30 02:10:26,320][1157819] Updated weights for policy 0, policy_version 108638 (0.0006) [2024-09-30 02:10:26,831][1157819] Updated weights for policy 0, policy_version 108648 (0.0006) [2024-09-30 02:10:27,392][1157819] Updated weights for policy 0, policy_version 108658 (0.0006) [2024-09-30 02:10:27,964][1157819] Updated weights for policy 0, policy_version 108668 (0.0006) [2024-09-30 02:10:28,534][1157819] Updated weights for policy 0, policy_version 108678 (0.0006) [2024-09-30 02:10:29,065][1157819] Updated weights for policy 0, policy_version 108688 (0.0006) [2024-09-30 02:10:29,260][1157736] Signal inference workers to stop experience collection... (7100 times) [2024-09-30 02:10:29,262][1157736] Signal inference workers to resume experience collection... (7100 times) [2024-09-30 02:10:29,262][1157819] InferenceWorker_p0-w0: stopping experience collection (7100 times) [2024-09-30 02:10:29,266][1157819] InferenceWorker_p0-w0: resuming experience collection (7100 times) [2024-09-30 02:10:29,608][1157819] Updated weights for policy 0, policy_version 108698 (0.0006) [2024-09-30 02:10:30,168][1157819] Updated weights for policy 0, policy_version 108708 (0.0006) [2024-09-30 02:10:30,466][1157520] Fps is (10 sec: 75365.8, 60 sec: 76800.0, 300 sec: 77518.5). Total num frames: 445288448. Throughput: 0: 19576.4. Samples: 101291132. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 02:10:30,466][1157520] Avg episode reward: [(0, '56.218')] [2024-09-30 02:10:30,676][1157819] Updated weights for policy 0, policy_version 108718 (0.0006) [2024-09-30 02:10:31,194][1157819] Updated weights for policy 0, policy_version 108728 (0.0006) [2024-09-30 02:10:31,692][1157819] Updated weights for policy 0, policy_version 108738 (0.0006) [2024-09-30 02:10:32,199][1157819] Updated weights for policy 0, policy_version 108748 (0.0006) [2024-09-30 02:10:32,705][1157819] Updated weights for policy 0, policy_version 108758 (0.0006) [2024-09-30 02:10:33,236][1157819] Updated weights for policy 0, policy_version 108768 (0.0006) [2024-09-30 02:10:33,738][1157819] Updated weights for policy 0, policy_version 108778 (0.0006) [2024-09-30 02:10:34,265][1157819] Updated weights for policy 0, policy_version 108788 (0.0006) [2024-09-30 02:10:34,779][1157819] Updated weights for policy 0, policy_version 108798 (0.0006) [2024-09-30 02:10:35,301][1157819] Updated weights for policy 0, policy_version 108808 (0.0006) [2024-09-30 02:10:35,466][1157520] Fps is (10 sec: 77414.3, 60 sec: 77823.9, 300 sec: 77601.8). Total num frames: 445689856. Throughput: 0: 19660.0. Samples: 101409656. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 02:10:35,466][1157520] Avg episode reward: [(0, '53.034')] [2024-09-30 02:10:35,808][1157819] Updated weights for policy 0, policy_version 108818 (0.0006) [2024-09-30 02:10:36,311][1157819] Updated weights for policy 0, policy_version 108828 (0.0006) [2024-09-30 02:10:36,820][1157819] Updated weights for policy 0, policy_version 108838 (0.0006) [2024-09-30 02:10:37,367][1157819] Updated weights for policy 0, policy_version 108848 (0.0006) [2024-09-30 02:10:37,897][1157819] Updated weights for policy 0, policy_version 108858 (0.0006) [2024-09-30 02:10:38,455][1157819] Updated weights for policy 0, policy_version 108868 (0.0006) [2024-09-30 02:10:38,992][1157819] Updated weights for policy 0, policy_version 108878 (0.0006) [2024-09-30 02:10:39,531][1157819] Updated weights for policy 0, policy_version 108888 (0.0006) [2024-09-30 02:10:40,065][1157819] Updated weights for policy 0, policy_version 108898 (0.0006) [2024-09-30 02:10:40,466][1157520] Fps is (10 sec: 78643.0, 60 sec: 77960.4, 300 sec: 77643.5). Total num frames: 446074880. Throughput: 0: 19683.3. Samples: 101468544. Policy #0 lag: (min: 0.0, avg: 2.2, max: 4.0) [2024-09-30 02:10:40,466][1157520] Avg episode reward: [(0, '54.343')] [2024-09-30 02:10:40,600][1157819] Updated weights for policy 0, policy_version 108908 (0.0006) [2024-09-30 02:10:41,118][1157819] Updated weights for policy 0, policy_version 108918 (0.0006) [2024-09-30 02:10:41,635][1157819] Updated weights for policy 0, policy_version 108928 (0.0006) [2024-09-30 02:10:42,167][1157819] Updated weights for policy 0, policy_version 108938 (0.0006) [2024-09-30 02:10:42,652][1157819] Updated weights for policy 0, policy_version 108948 (0.0006) [2024-09-30 02:10:43,144][1157819] Updated weights for policy 0, policy_version 108958 (0.0006) [2024-09-30 02:10:43,654][1157819] Updated weights for policy 0, policy_version 108968 (0.0006) [2024-09-30 02:10:44,153][1157819] Updated weights for policy 0, policy_version 108978 (0.0006) [2024-09-30 02:10:44,688][1157819] Updated weights for policy 0, policy_version 108988 (0.0006) [2024-09-30 02:10:45,231][1157819] Updated weights for policy 0, policy_version 108998 (0.0006) [2024-09-30 02:10:45,466][1157520] Fps is (10 sec: 78642.2, 60 sec: 78506.4, 300 sec: 77712.9). Total num frames: 446476288. Throughput: 0: 19681.9. Samples: 101585796. Policy #0 lag: (min: 0.0, avg: 2.2, max: 4.0) [2024-09-30 02:10:45,466][1157520] Avg episode reward: [(0, '54.351')] [2024-09-30 02:10:45,727][1157819] Updated weights for policy 0, policy_version 109008 (0.0006) [2024-09-30 02:10:46,250][1157819] Updated weights for policy 0, policy_version 109018 (0.0006) [2024-09-30 02:10:46,742][1157819] Updated weights for policy 0, policy_version 109028 (0.0006) [2024-09-30 02:10:47,257][1157819] Updated weights for policy 0, policy_version 109038 (0.0006) [2024-09-30 02:10:47,768][1157819] Updated weights for policy 0, policy_version 109048 (0.0006) [2024-09-30 02:10:48,311][1157819] Updated weights for policy 0, policy_version 109058 (0.0006) [2024-09-30 02:10:48,848][1157819] Updated weights for policy 0, policy_version 109068 (0.0006) [2024-09-30 02:10:49,373][1157819] Updated weights for policy 0, policy_version 109078 (0.0006) [2024-09-30 02:10:49,886][1157819] Updated weights for policy 0, policy_version 109088 (0.0006) [2024-09-30 02:10:50,378][1157819] Updated weights for policy 0, policy_version 109098 (0.0006) [2024-09-30 02:10:50,466][1157520] Fps is (10 sec: 79462.5, 60 sec: 78643.1, 300 sec: 77740.7). Total num frames: 446869504. Throughput: 0: 19624.0. Samples: 101704504. Policy #0 lag: (min: 0.0, avg: 2.2, max: 4.0) [2024-09-30 02:10:50,466][1157520] Avg episode reward: [(0, '56.537')] [2024-09-30 02:10:50,896][1157819] Updated weights for policy 0, policy_version 109108 (0.0006) [2024-09-30 02:10:51,405][1157819] Updated weights for policy 0, policy_version 109118 (0.0006) [2024-09-30 02:10:51,955][1157819] Updated weights for policy 0, policy_version 109128 (0.0006) [2024-09-30 02:10:52,448][1157819] Updated weights for policy 0, policy_version 109138 (0.0006) [2024-09-30 02:10:52,951][1157819] Updated weights for policy 0, policy_version 109148 (0.0006) [2024-09-30 02:10:53,470][1157819] Updated weights for policy 0, policy_version 109158 (0.0006) [2024-09-30 02:10:53,546][1157736] Signal inference workers to stop experience collection... (7150 times) [2024-09-30 02:10:53,547][1157736] Signal inference workers to resume experience collection... (7150 times) [2024-09-30 02:10:53,551][1157819] InferenceWorker_p0-w0: stopping experience collection (7150 times) [2024-09-30 02:10:53,551][1157819] InferenceWorker_p0-w0: resuming experience collection (7150 times) [2024-09-30 02:10:54,016][1157819] Updated weights for policy 0, policy_version 109168 (0.0006) [2024-09-30 02:10:54,512][1157819] Updated weights for policy 0, policy_version 109178 (0.0006) [2024-09-30 02:10:54,992][1157819] Updated weights for policy 0, policy_version 109188 (0.0006) [2024-09-30 02:10:55,466][1157520] Fps is (10 sec: 79463.5, 60 sec: 78916.1, 300 sec: 77712.9). Total num frames: 447270912. Throughput: 0: 19644.0. Samples: 101764080. Policy #0 lag: (min: 0.0, avg: 2.2, max: 4.0) [2024-09-30 02:10:55,466][1157520] Avg episode reward: [(0, '55.828')] [2024-09-30 02:10:55,496][1157819] Updated weights for policy 0, policy_version 109198 (0.0006) [2024-09-30 02:10:55,967][1157819] Updated weights for policy 0, policy_version 109208 (0.0006) [2024-09-30 02:10:56,436][1157819] Updated weights for policy 0, policy_version 109218 (0.0006) [2024-09-30 02:10:56,917][1157819] Updated weights for policy 0, policy_version 109228 (0.0006) [2024-09-30 02:10:57,422][1157819] Updated weights for policy 0, policy_version 109238 (0.0006) [2024-09-30 02:10:57,897][1157819] Updated weights for policy 0, policy_version 109248 (0.0006) [2024-09-30 02:10:58,398][1157819] Updated weights for policy 0, policy_version 109258 (0.0006) [2024-09-30 02:10:58,922][1157819] Updated weights for policy 0, policy_version 109268 (0.0006) [2024-09-30 02:10:59,417][1157819] Updated weights for policy 0, policy_version 109278 (0.0006) [2024-09-30 02:10:59,936][1157819] Updated weights for policy 0, policy_version 109288 (0.0006) [2024-09-30 02:11:00,453][1157819] Updated weights for policy 0, policy_version 109298 (0.0006) [2024-09-30 02:11:00,466][1157520] Fps is (10 sec: 81510.4, 60 sec: 79189.2, 300 sec: 77726.8). Total num frames: 447684608. Throughput: 0: 19689.2. Samples: 101887892. Policy #0 lag: (min: 0.0, avg: 2.2, max: 4.0) [2024-09-30 02:11:00,466][1157520] Avg episode reward: [(0, '52.345')] [2024-09-30 02:11:01,047][1157819] Updated weights for policy 0, policy_version 109308 (0.0006) [2024-09-30 02:11:01,599][1157819] Updated weights for policy 0, policy_version 109318 (0.0006) [2024-09-30 02:11:02,179][1157819] Updated weights for policy 0, policy_version 109328 (0.0006) [2024-09-30 02:11:02,709][1157819] Updated weights for policy 0, policy_version 109338 (0.0006) [2024-09-30 02:11:03,246][1157819] Updated weights for policy 0, policy_version 109348 (0.0006) [2024-09-30 02:11:03,784][1157819] Updated weights for policy 0, policy_version 109358 (0.0006) [2024-09-30 02:11:04,313][1157819] Updated weights for policy 0, policy_version 109368 (0.0006) [2024-09-30 02:11:04,880][1157819] Updated weights for policy 0, policy_version 109378 (0.0006) [2024-09-30 02:11:05,441][1157819] Updated weights for policy 0, policy_version 109388 (0.0006) [2024-09-30 02:11:05,466][1157520] Fps is (10 sec: 78643.0, 60 sec: 78643.1, 300 sec: 77629.6). Total num frames: 448057344. Throughput: 0: 19576.7. Samples: 102001576. Policy #0 lag: (min: 0.0, avg: 2.2, max: 4.0) [2024-09-30 02:11:05,466][1157520] Avg episode reward: [(0, '54.306')] [2024-09-30 02:11:05,951][1157819] Updated weights for policy 0, policy_version 109398 (0.0006) [2024-09-30 02:11:06,444][1157819] Updated weights for policy 0, policy_version 109408 (0.0006) [2024-09-30 02:11:06,944][1157819] Updated weights for policy 0, policy_version 109418 (0.0006) [2024-09-30 02:11:07,457][1157819] Updated weights for policy 0, policy_version 109428 (0.0006) [2024-09-30 02:11:08,030][1157819] Updated weights for policy 0, policy_version 109438 (0.0006) [2024-09-30 02:11:08,540][1157819] Updated weights for policy 0, policy_version 109448 (0.0006) [2024-09-30 02:11:09,045][1157819] Updated weights for policy 0, policy_version 109458 (0.0006) [2024-09-30 02:11:09,555][1157819] Updated weights for policy 0, policy_version 109468 (0.0006) [2024-09-30 02:11:10,074][1157819] Updated weights for policy 0, policy_version 109478 (0.0006) [2024-09-30 02:11:10,466][1157520] Fps is (10 sec: 76595.1, 60 sec: 78506.6, 300 sec: 77726.8). Total num frames: 448450560. Throughput: 0: 19563.7. Samples: 102060156. Policy #0 lag: (min: 0.0, avg: 2.2, max: 4.0) [2024-09-30 02:11:10,466][1157520] Avg episode reward: [(0, '55.190')] [2024-09-30 02:11:10,605][1157819] Updated weights for policy 0, policy_version 109488 (0.0006) [2024-09-30 02:11:11,150][1157819] Updated weights for policy 0, policy_version 109498 (0.0006) [2024-09-30 02:11:11,671][1157819] Updated weights for policy 0, policy_version 109508 (0.0006) [2024-09-30 02:11:12,164][1157819] Updated weights for policy 0, policy_version 109518 (0.0006) [2024-09-30 02:11:12,658][1157819] Updated weights for policy 0, policy_version 109528 (0.0006) [2024-09-30 02:11:13,158][1157819] Updated weights for policy 0, policy_version 109538 (0.0006) [2024-09-30 02:11:13,671][1157819] Updated weights for policy 0, policy_version 109548 (0.0006) [2024-09-30 02:11:14,185][1157819] Updated weights for policy 0, policy_version 109558 (0.0006) [2024-09-30 02:11:14,694][1157819] Updated weights for policy 0, policy_version 109568 (0.0006) [2024-09-30 02:11:15,207][1157819] Updated weights for policy 0, policy_version 109578 (0.0006) [2024-09-30 02:11:15,466][1157520] Fps is (10 sec: 79052.9, 60 sec: 78506.5, 300 sec: 77782.3). Total num frames: 448847872. Throughput: 0: 19735.0. Samples: 102179208. Policy #0 lag: (min: 0.0, avg: 2.2, max: 4.0) [2024-09-30 02:11:15,466][1157520] Avg episode reward: [(0, '55.270')] [2024-09-30 02:11:15,727][1157819] Updated weights for policy 0, policy_version 109588 (0.0006) [2024-09-30 02:11:16,238][1157819] Updated weights for policy 0, policy_version 109598 (0.0006) [2024-09-30 02:11:16,764][1157819] Updated weights for policy 0, policy_version 109608 (0.0006) [2024-09-30 02:11:17,272][1157819] Updated weights for policy 0, policy_version 109618 (0.0006) [2024-09-30 02:11:17,787][1157819] Updated weights for policy 0, policy_version 109628 (0.0006) [2024-09-30 02:11:18,281][1157819] Updated weights for policy 0, policy_version 109638 (0.0006) [2024-09-30 02:11:18,775][1157819] Updated weights for policy 0, policy_version 109648 (0.0006) [2024-09-30 02:11:19,277][1157819] Updated weights for policy 0, policy_version 109658 (0.0006) [2024-09-30 02:11:19,793][1157819] Updated weights for policy 0, policy_version 109668 (0.0006) [2024-09-30 02:11:20,286][1157819] Updated weights for policy 0, policy_version 109678 (0.0006) [2024-09-30 02:11:20,466][1157520] Fps is (10 sec: 80691.6, 60 sec: 78711.4, 300 sec: 77865.6). Total num frames: 449257472. Throughput: 0: 19787.8. Samples: 102300108. Policy #0 lag: (min: 0.0, avg: 2.2, max: 4.0) [2024-09-30 02:11:20,466][1157520] Avg episode reward: [(0, '55.900')] [2024-09-30 02:11:20,773][1157819] Updated weights for policy 0, policy_version 109688 (0.0006) [2024-09-30 02:11:21,285][1157819] Updated weights for policy 0, policy_version 109698 (0.0006) [2024-09-30 02:11:21,783][1157819] Updated weights for policy 0, policy_version 109708 (0.0006) [2024-09-30 02:11:22,289][1157819] Updated weights for policy 0, policy_version 109718 (0.0006) [2024-09-30 02:11:22,819][1157819] Updated weights for policy 0, policy_version 109728 (0.0006) [2024-09-30 02:11:23,359][1157819] Updated weights for policy 0, policy_version 109738 (0.0006) [2024-09-30 02:11:23,867][1157819] Updated weights for policy 0, policy_version 109748 (0.0006) [2024-09-30 02:11:24,352][1157819] Updated weights for policy 0, policy_version 109758 (0.0006) [2024-09-30 02:11:24,924][1157819] Updated weights for policy 0, policy_version 109768 (0.0006) [2024-09-30 02:11:25,433][1157819] Updated weights for policy 0, policy_version 109778 (0.0006) [2024-09-30 02:11:25,466][1157520] Fps is (10 sec: 80281.3, 60 sec: 78916.2, 300 sec: 77935.0). Total num frames: 449650688. Throughput: 0: 19825.0. Samples: 102360668. Policy #0 lag: (min: 0.0, avg: 2.2, max: 4.0) [2024-09-30 02:11:25,466][1157520] Avg episode reward: [(0, '54.675')] [2024-09-30 02:11:25,987][1157819] Updated weights for policy 0, policy_version 109788 (0.0006) [2024-09-30 02:11:26,550][1157819] Updated weights for policy 0, policy_version 109798 (0.0006) [2024-09-30 02:11:26,822][1157736] Signal inference workers to stop experience collection... (7200 times) [2024-09-30 02:11:26,823][1157736] Signal inference workers to resume experience collection... (7200 times) [2024-09-30 02:11:26,826][1157819] InferenceWorker_p0-w0: stopping experience collection (7200 times) [2024-09-30 02:11:26,829][1157819] InferenceWorker_p0-w0: resuming experience collection (7200 times) [2024-09-30 02:11:27,112][1157819] Updated weights for policy 0, policy_version 109808 (0.0006) [2024-09-30 02:11:27,618][1157819] Updated weights for policy 0, policy_version 109818 (0.0006) [2024-09-30 02:11:28,198][1157819] Updated weights for policy 0, policy_version 109828 (0.0006) [2024-09-30 02:11:28,811][1157819] Updated weights for policy 0, policy_version 109838 (0.0006) [2024-09-30 02:11:29,422][1157819] Updated weights for policy 0, policy_version 109848 (0.0006) [2024-09-30 02:11:30,081][1157819] Updated weights for policy 0, policy_version 109858 (0.0006) [2024-09-30 02:11:30,466][1157520] Fps is (10 sec: 74546.9, 60 sec: 78574.9, 300 sec: 77837.9). Total num frames: 450002944. Throughput: 0: 19732.8. Samples: 102473768. Policy #0 lag: (min: 0.0, avg: 2.2, max: 4.0) [2024-09-30 02:11:30,466][1157520] Avg episode reward: [(0, '53.979')] [2024-09-30 02:11:30,661][1157819] Updated weights for policy 0, policy_version 109868 (0.0006) [2024-09-30 02:11:31,219][1157819] Updated weights for policy 0, policy_version 109878 (0.0006) [2024-09-30 02:11:31,776][1157819] Updated weights for policy 0, policy_version 109888 (0.0006) [2024-09-30 02:11:32,338][1157819] Updated weights for policy 0, policy_version 109898 (0.0006) [2024-09-30 02:11:32,893][1157819] Updated weights for policy 0, policy_version 109908 (0.0006) [2024-09-30 02:11:33,409][1157819] Updated weights for policy 0, policy_version 109918 (0.0006) [2024-09-30 02:11:33,989][1157819] Updated weights for policy 0, policy_version 109928 (0.0006) [2024-09-30 02:11:34,531][1157819] Updated weights for policy 0, policy_version 109938 (0.0006) [2024-09-30 02:11:35,119][1157819] Updated weights for policy 0, policy_version 109948 (0.0006) [2024-09-30 02:11:35,466][1157520] Fps is (10 sec: 72089.7, 60 sec: 78028.8, 300 sec: 77782.3). Total num frames: 450371584. Throughput: 0: 19477.3. Samples: 102580984. Policy #0 lag: (min: 0.0, avg: 2.2, max: 4.0) [2024-09-30 02:11:35,466][1157520] Avg episode reward: [(0, '55.600')] [2024-09-30 02:11:35,661][1157819] Updated weights for policy 0, policy_version 109958 (0.0006) [2024-09-30 02:11:36,180][1157819] Updated weights for policy 0, policy_version 109968 (0.0006) [2024-09-30 02:11:36,745][1157819] Updated weights for policy 0, policy_version 109978 (0.0006) [2024-09-30 02:11:37,301][1157819] Updated weights for policy 0, policy_version 109988 (0.0006) [2024-09-30 02:11:37,854][1157819] Updated weights for policy 0, policy_version 109998 (0.0006) [2024-09-30 02:11:38,437][1157819] Updated weights for policy 0, policy_version 110008 (0.0006) [2024-09-30 02:11:38,982][1157819] Updated weights for policy 0, policy_version 110018 (0.0006) [2024-09-30 02:11:39,534][1157819] Updated weights for policy 0, policy_version 110028 (0.0006) [2024-09-30 02:11:40,081][1157819] Updated weights for policy 0, policy_version 110038 (0.0006) [2024-09-30 02:11:40,466][1157520] Fps is (10 sec: 73727.7, 60 sec: 77755.7, 300 sec: 77754.5). Total num frames: 450740224. Throughput: 0: 19385.4. Samples: 102636424. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 02:11:40,466][1157520] Avg episode reward: [(0, '53.619')] [2024-09-30 02:11:40,664][1157819] Updated weights for policy 0, policy_version 110048 (0.0006) [2024-09-30 02:11:41,238][1157819] Updated weights for policy 0, policy_version 110058 (0.0006) [2024-09-30 02:11:41,795][1157819] Updated weights for policy 0, policy_version 110068 (0.0006) [2024-09-30 02:11:42,327][1157819] Updated weights for policy 0, policy_version 110078 (0.0006) [2024-09-30 02:11:42,835][1157819] Updated weights for policy 0, policy_version 110088 (0.0006) [2024-09-30 02:11:43,395][1157819] Updated weights for policy 0, policy_version 110098 (0.0006) [2024-09-30 02:11:43,946][1157819] Updated weights for policy 0, policy_version 110108 (0.0006) [2024-09-30 02:11:44,536][1157819] Updated weights for policy 0, policy_version 110118 (0.0006) [2024-09-30 02:11:45,092][1157819] Updated weights for policy 0, policy_version 110128 (0.0006) [2024-09-30 02:11:45,466][1157520] Fps is (10 sec: 73318.0, 60 sec: 77141.4, 300 sec: 77726.8). Total num frames: 451104768. Throughput: 0: 19101.8. Samples: 102747476. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 02:11:45,466][1157520] Avg episode reward: [(0, '54.781')] [2024-09-30 02:11:45,685][1157819] Updated weights for policy 0, policy_version 110138 (0.0006) [2024-09-30 02:11:46,298][1157819] Updated weights for policy 0, policy_version 110148 (0.0006) [2024-09-30 02:11:46,812][1157819] Updated weights for policy 0, policy_version 110158 (0.0006) [2024-09-30 02:11:47,364][1157819] Updated weights for policy 0, policy_version 110168 (0.0006) [2024-09-30 02:11:47,937][1157819] Updated weights for policy 0, policy_version 110178 (0.0006) [2024-09-30 02:11:48,482][1157819] Updated weights for policy 0, policy_version 110188 (0.0006) [2024-09-30 02:11:49,071][1157819] Updated weights for policy 0, policy_version 110198 (0.0006) [2024-09-30 02:11:49,625][1157819] Updated weights for policy 0, policy_version 110208 (0.0006) [2024-09-30 02:11:50,164][1157819] Updated weights for policy 0, policy_version 110218 (0.0006) [2024-09-30 02:11:50,466][1157520] Fps is (10 sec: 73319.3, 60 sec: 76731.8, 300 sec: 77712.9). Total num frames: 451473408. Throughput: 0: 18990.4. Samples: 102856140. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 02:11:50,466][1157520] Avg episode reward: [(0, '54.079')] [2024-09-30 02:11:50,800][1157819] Updated weights for policy 0, policy_version 110228 (0.0006) [2024-09-30 02:11:51,406][1157819] Updated weights for policy 0, policy_version 110238 (0.0006) [2024-09-30 02:11:51,958][1157819] Updated weights for policy 0, policy_version 110248 (0.0006) [2024-09-30 02:11:52,514][1157819] Updated weights for policy 0, policy_version 110258 (0.0006) [2024-09-30 02:11:53,061][1157819] Updated weights for policy 0, policy_version 110268 (0.0006) [2024-09-30 02:11:53,611][1157819] Updated weights for policy 0, policy_version 110278 (0.0006) [2024-09-30 02:11:53,765][1157736] Signal inference workers to stop experience collection... (7250 times) [2024-09-30 02:11:53,766][1157736] Signal inference workers to resume experience collection... (7250 times) [2024-09-30 02:11:53,770][1157819] InferenceWorker_p0-w0: stopping experience collection (7250 times) [2024-09-30 02:11:53,770][1157819] InferenceWorker_p0-w0: resuming experience collection (7250 times) [2024-09-30 02:11:54,116][1157819] Updated weights for policy 0, policy_version 110288 (0.0006) [2024-09-30 02:11:54,726][1157819] Updated weights for policy 0, policy_version 110298 (0.0006) [2024-09-30 02:11:55,272][1157819] Updated weights for policy 0, policy_version 110308 (0.0006) [2024-09-30 02:11:55,466][1157520] Fps is (10 sec: 72499.6, 60 sec: 75980.8, 300 sec: 77574.0). Total num frames: 451829760. Throughput: 0: 18875.6. Samples: 102909560. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 02:11:55,466][1157520] Avg episode reward: [(0, '54.658')] [2024-09-30 02:11:55,875][1157819] Updated weights for policy 0, policy_version 110318 (0.0006) [2024-09-30 02:11:56,506][1157819] Updated weights for policy 0, policy_version 110328 (0.0006) [2024-09-30 02:11:57,077][1157819] Updated weights for policy 0, policy_version 110338 (0.0006) [2024-09-30 02:11:57,682][1157819] Updated weights for policy 0, policy_version 110348 (0.0007) [2024-09-30 02:11:58,285][1157819] Updated weights for policy 0, policy_version 110358 (0.0006) [2024-09-30 02:11:58,864][1157819] Updated weights for policy 0, policy_version 110368 (0.0006) [2024-09-30 02:11:59,448][1157819] Updated weights for policy 0, policy_version 110378 (0.0006) [2024-09-30 02:12:00,034][1157819] Updated weights for policy 0, policy_version 110388 (0.0007) [2024-09-30 02:12:00,466][1157520] Fps is (10 sec: 70450.7, 60 sec: 74888.6, 300 sec: 77449.1). Total num frames: 452177920. Throughput: 0: 18580.8. Samples: 103015344. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 02:12:00,466][1157520] Avg episode reward: [(0, '55.068')] [2024-09-30 02:12:00,624][1157819] Updated weights for policy 0, policy_version 110398 (0.0006) [2024-09-30 02:12:01,228][1157819] Updated weights for policy 0, policy_version 110408 (0.0006) [2024-09-30 02:12:01,843][1157819] Updated weights for policy 0, policy_version 110418 (0.0006) [2024-09-30 02:12:02,400][1157819] Updated weights for policy 0, policy_version 110428 (0.0006) [2024-09-30 02:12:02,895][1157819] Updated weights for policy 0, policy_version 110438 (0.0006) [2024-09-30 02:12:03,375][1157819] Updated weights for policy 0, policy_version 110448 (0.0006) [2024-09-30 02:12:03,890][1157819] Updated weights for policy 0, policy_version 110458 (0.0006) [2024-09-30 02:12:04,382][1157819] Updated weights for policy 0, policy_version 110468 (0.0006) [2024-09-30 02:12:04,886][1157819] Updated weights for policy 0, policy_version 110478 (0.0006) [2024-09-30 02:12:05,381][1157819] Updated weights for policy 0, policy_version 110488 (0.0006) [2024-09-30 02:12:05,466][1157520] Fps is (10 sec: 73728.1, 60 sec: 75161.6, 300 sec: 77407.4). Total num frames: 452567040. Throughput: 0: 18389.7. Samples: 103127644. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 02:12:05,466][1157520] Avg episode reward: [(0, '55.926')] [2024-09-30 02:12:05,472][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000110490_452567040.pth... [2024-09-30 02:12:05,517][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000105980_434094080.pth [2024-09-30 02:12:05,877][1157819] Updated weights for policy 0, policy_version 110498 (0.0006) [2024-09-30 02:12:06,383][1157819] Updated weights for policy 0, policy_version 110508 (0.0006) [2024-09-30 02:12:06,898][1157819] Updated weights for policy 0, policy_version 110518 (0.0006) [2024-09-30 02:12:07,395][1157819] Updated weights for policy 0, policy_version 110528 (0.0007) [2024-09-30 02:12:07,899][1157819] Updated weights for policy 0, policy_version 110538 (0.0006) [2024-09-30 02:12:08,458][1157819] Updated weights for policy 0, policy_version 110548 (0.0006) [2024-09-30 02:12:09,019][1157819] Updated weights for policy 0, policy_version 110558 (0.0006) [2024-09-30 02:12:09,542][1157819] Updated weights for policy 0, policy_version 110568 (0.0006) [2024-09-30 02:12:10,036][1157819] Updated weights for policy 0, policy_version 110578 (0.0006) [2024-09-30 02:12:10,466][1157520] Fps is (10 sec: 78234.3, 60 sec: 75161.7, 300 sec: 77407.5). Total num frames: 452960256. Throughput: 0: 18380.8. Samples: 103187800. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 02:12:10,466][1157520] Avg episode reward: [(0, '55.351')] [2024-09-30 02:12:10,550][1157819] Updated weights for policy 0, policy_version 110588 (0.0006) [2024-09-30 02:12:11,020][1157819] Updated weights for policy 0, policy_version 110598 (0.0006) [2024-09-30 02:12:11,515][1157819] Updated weights for policy 0, policy_version 110608 (0.0006) [2024-09-30 02:12:11,869][1157736] Signal inference workers to stop experience collection... (7300 times) [2024-09-30 02:12:11,872][1157736] Signal inference workers to resume experience collection... (7300 times) [2024-09-30 02:12:11,873][1157819] InferenceWorker_p0-w0: stopping experience collection (7300 times) [2024-09-30 02:12:11,877][1157819] InferenceWorker_p0-w0: resuming experience collection (7300 times) [2024-09-30 02:12:11,999][1157819] Updated weights for policy 0, policy_version 110618 (0.0006) [2024-09-30 02:12:12,476][1157819] Updated weights for policy 0, policy_version 110628 (0.0006) [2024-09-30 02:12:12,975][1157819] Updated weights for policy 0, policy_version 110638 (0.0006) [2024-09-30 02:12:13,454][1157819] Updated weights for policy 0, policy_version 110648 (0.0006) [2024-09-30 02:12:13,946][1157819] Updated weights for policy 0, policy_version 110658 (0.0006) [2024-09-30 02:12:14,442][1157819] Updated weights for policy 0, policy_version 110668 (0.0007) [2024-09-30 02:12:14,940][1157819] Updated weights for policy 0, policy_version 110678 (0.0007) [2024-09-30 02:12:15,430][1157819] Updated weights for policy 0, policy_version 110688 (0.0006) [2024-09-30 02:12:15,466][1157520] Fps is (10 sec: 81101.6, 60 sec: 75503.1, 300 sec: 77449.1). Total num frames: 453378048. Throughput: 0: 18591.7. Samples: 103310392. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 02:12:15,466][1157520] Avg episode reward: [(0, '55.825')] [2024-09-30 02:12:15,929][1157819] Updated weights for policy 0, policy_version 110698 (0.0007) [2024-09-30 02:12:16,418][1157819] Updated weights for policy 0, policy_version 110708 (0.0006) [2024-09-30 02:12:16,904][1157819] Updated weights for policy 0, policy_version 110718 (0.0007) [2024-09-30 02:12:17,433][1157819] Updated weights for policy 0, policy_version 110728 (0.0006) [2024-09-30 02:12:17,950][1157819] Updated weights for policy 0, policy_version 110738 (0.0007) [2024-09-30 02:12:18,439][1157819] Updated weights for policy 0, policy_version 110748 (0.0006) [2024-09-30 02:12:18,918][1157819] Updated weights for policy 0, policy_version 110758 (0.0006) [2024-09-30 02:12:19,422][1157819] Updated weights for policy 0, policy_version 110768 (0.0006) [2024-09-30 02:12:19,910][1157819] Updated weights for policy 0, policy_version 110778 (0.0006) [2024-09-30 02:12:20,405][1157819] Updated weights for policy 0, policy_version 110788 (0.0006) [2024-09-30 02:12:20,466][1157520] Fps is (10 sec: 82739.1, 60 sec: 75503.0, 300 sec: 77490.8). Total num frames: 453787648. Throughput: 0: 18961.8. Samples: 103434264. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 02:12:20,466][1157520] Avg episode reward: [(0, '52.864')] [2024-09-30 02:12:20,885][1157819] Updated weights for policy 0, policy_version 110798 (0.0006) [2024-09-30 02:12:21,397][1157819] Updated weights for policy 0, policy_version 110808 (0.0006) [2024-09-30 02:12:21,887][1157819] Updated weights for policy 0, policy_version 110818 (0.0006) [2024-09-30 02:12:22,390][1157819] Updated weights for policy 0, policy_version 110828 (0.0006) [2024-09-30 02:12:22,914][1157819] Updated weights for policy 0, policy_version 110838 (0.0006) [2024-09-30 02:12:23,470][1157819] Updated weights for policy 0, policy_version 110848 (0.0006) [2024-09-30 02:12:24,014][1157819] Updated weights for policy 0, policy_version 110858 (0.0006) [2024-09-30 02:12:24,528][1157819] Updated weights for policy 0, policy_version 110868 (0.0006) [2024-09-30 02:12:25,070][1157819] Updated weights for policy 0, policy_version 110878 (0.0006) [2024-09-30 02:12:25,466][1157520] Fps is (10 sec: 80689.9, 60 sec: 75571.2, 300 sec: 77504.6). Total num frames: 454184960. Throughput: 0: 19091.7. Samples: 103495552. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 02:12:25,466][1157520] Avg episode reward: [(0, '56.427')] [2024-09-30 02:12:25,638][1157819] Updated weights for policy 0, policy_version 110888 (0.0006) [2024-09-30 02:12:26,171][1157819] Updated weights for policy 0, policy_version 110898 (0.0006) [2024-09-30 02:12:26,698][1157819] Updated weights for policy 0, policy_version 110908 (0.0006) [2024-09-30 02:12:27,240][1157819] Updated weights for policy 0, policy_version 110918 (0.0006) [2024-09-30 02:12:27,751][1157819] Updated weights for policy 0, policy_version 110928 (0.0006) [2024-09-30 02:12:28,339][1157819] Updated weights for policy 0, policy_version 110938 (0.0006) [2024-09-30 02:12:28,860][1157819] Updated weights for policy 0, policy_version 110948 (0.0006) [2024-09-30 02:12:29,412][1157819] Updated weights for policy 0, policy_version 110958 (0.0006) [2024-09-30 02:12:29,986][1157819] Updated weights for policy 0, policy_version 110968 (0.0006) [2024-09-30 02:12:30,466][1157520] Fps is (10 sec: 77004.1, 60 sec: 75912.5, 300 sec: 77407.5). Total num frames: 454557696. Throughput: 0: 19152.6. Samples: 103609340. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 02:12:30,466][1157520] Avg episode reward: [(0, '53.929')] [2024-09-30 02:12:30,502][1157819] Updated weights for policy 0, policy_version 110978 (0.0006) [2024-09-30 02:12:31,026][1157819] Updated weights for policy 0, policy_version 110988 (0.0006) [2024-09-30 02:12:31,554][1157819] Updated weights for policy 0, policy_version 110998 (0.0006) [2024-09-30 02:12:32,142][1157819] Updated weights for policy 0, policy_version 111008 (0.0006) [2024-09-30 02:12:32,691][1157819] Updated weights for policy 0, policy_version 111018 (0.0006) [2024-09-30 02:12:33,237][1157819] Updated weights for policy 0, policy_version 111028 (0.0006) [2024-09-30 02:12:33,804][1157819] Updated weights for policy 0, policy_version 111038 (0.0006) [2024-09-30 02:12:34,366][1157819] Updated weights for policy 0, policy_version 111048 (0.0006) [2024-09-30 02:12:34,906][1157819] Updated weights for policy 0, policy_version 111058 (0.0006) [2024-09-30 02:12:35,466][1157520] Fps is (10 sec: 74547.6, 60 sec: 75980.8, 300 sec: 77324.2). Total num frames: 454930432. Throughput: 0: 19225.6. Samples: 103721292. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 02:12:35,466][1157520] Avg episode reward: [(0, '54.245')] [2024-09-30 02:12:35,490][1157819] Updated weights for policy 0, policy_version 111068 (0.0006) [2024-09-30 02:12:36,023][1157819] Updated weights for policy 0, policy_version 111078 (0.0006) [2024-09-30 02:12:36,601][1157819] Updated weights for policy 0, policy_version 111088 (0.0006) [2024-09-30 02:12:37,183][1157819] Updated weights for policy 0, policy_version 111098 (0.0006) [2024-09-30 02:12:37,696][1157819] Updated weights for policy 0, policy_version 111108 (0.0006) [2024-09-30 02:12:38,270][1157819] Updated weights for policy 0, policy_version 111118 (0.0006) [2024-09-30 02:12:38,835][1157819] Updated weights for policy 0, policy_version 111128 (0.0006) [2024-09-30 02:12:39,332][1157819] Updated weights for policy 0, policy_version 111138 (0.0006) [2024-09-30 02:12:39,886][1157819] Updated weights for policy 0, policy_version 111148 (0.0006) [2024-09-30 02:12:40,449][1157819] Updated weights for policy 0, policy_version 111158 (0.0006) [2024-09-30 02:12:40,466][1157520] Fps is (10 sec: 74547.2, 60 sec: 76049.1, 300 sec: 77227.0). Total num frames: 455303168. Throughput: 0: 19250.6. Samples: 103775836. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:12:40,466][1157520] Avg episode reward: [(0, '54.422')] [2024-09-30 02:12:41,002][1157819] Updated weights for policy 0, policy_version 111168 (0.0006) [2024-09-30 02:12:41,563][1157819] Updated weights for policy 0, policy_version 111178 (0.0006) [2024-09-30 02:12:42,148][1157819] Updated weights for policy 0, policy_version 111188 (0.0006) [2024-09-30 02:12:42,700][1157819] Updated weights for policy 0, policy_version 111198 (0.0006) [2024-09-30 02:12:43,281][1157819] Updated weights for policy 0, policy_version 111208 (0.0006) [2024-09-30 02:12:43,847][1157819] Updated weights for policy 0, policy_version 111218 (0.0006) [2024-09-30 02:12:44,419][1157819] Updated weights for policy 0, policy_version 111228 (0.0006) [2024-09-30 02:12:44,950][1157819] Updated weights for policy 0, policy_version 111238 (0.0006) [2024-09-30 02:12:45,466][1157520] Fps is (10 sec: 73728.2, 60 sec: 76049.2, 300 sec: 77129.8). Total num frames: 455667712. Throughput: 0: 19357.4. Samples: 103886428. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:12:45,466][1157520] Avg episode reward: [(0, '55.525')] [2024-09-30 02:12:45,497][1157819] Updated weights for policy 0, policy_version 111248 (0.0006) [2024-09-30 02:12:46,070][1157819] Updated weights for policy 0, policy_version 111258 (0.0006) [2024-09-30 02:12:46,610][1157819] Updated weights for policy 0, policy_version 111268 (0.0006) [2024-09-30 02:12:47,173][1157819] Updated weights for policy 0, policy_version 111278 (0.0006) [2024-09-30 02:12:47,678][1157819] Updated weights for policy 0, policy_version 111288 (0.0006) [2024-09-30 02:12:48,203][1157819] Updated weights for policy 0, policy_version 111298 (0.0006) [2024-09-30 02:12:48,800][1157819] Updated weights for policy 0, policy_version 111308 (0.0006) [2024-09-30 02:12:49,393][1157819] Updated weights for policy 0, policy_version 111318 (0.0006) [2024-09-30 02:12:49,957][1157819] Updated weights for policy 0, policy_version 111328 (0.0006) [2024-09-30 02:12:50,466][1157520] Fps is (10 sec: 72909.1, 60 sec: 75980.7, 300 sec: 77046.5). Total num frames: 456032256. Throughput: 0: 19307.8. Samples: 103996496. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:12:50,466][1157520] Avg episode reward: [(0, '55.986')] [2024-09-30 02:12:50,572][1157819] Updated weights for policy 0, policy_version 111338 (0.0006) [2024-09-30 02:12:51,204][1157819] Updated weights for policy 0, policy_version 111348 (0.0006) [2024-09-30 02:12:51,825][1157819] Updated weights for policy 0, policy_version 111358 (0.0006) [2024-09-30 02:12:52,452][1157819] Updated weights for policy 0, policy_version 111368 (0.0006) [2024-09-30 02:12:53,022][1157819] Updated weights for policy 0, policy_version 111378 (0.0006) [2024-09-30 02:12:53,667][1157819] Updated weights for policy 0, policy_version 111388 (0.0006) [2024-09-30 02:12:54,269][1157819] Updated weights for policy 0, policy_version 111398 (0.0006) [2024-09-30 02:12:54,791][1157819] Updated weights for policy 0, policy_version 111408 (0.0006) [2024-09-30 02:12:55,325][1157819] Updated weights for policy 0, policy_version 111418 (0.0006) [2024-09-30 02:12:55,466][1157520] Fps is (10 sec: 70859.8, 60 sec: 75775.9, 300 sec: 76865.9). Total num frames: 456376320. Throughput: 0: 19086.0. Samples: 104046676. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:12:55,466][1157520] Avg episode reward: [(0, '55.786')] [2024-09-30 02:12:55,842][1157736] Signal inference workers to stop experience collection... (7350 times) [2024-09-30 02:12:55,844][1157819] InferenceWorker_p0-w0: stopping experience collection (7350 times) [2024-09-30 02:12:55,847][1157736] Signal inference workers to resume experience collection... (7350 times) [2024-09-30 02:12:55,848][1157819] InferenceWorker_p0-w0: resuming experience collection (7350 times) [2024-09-30 02:12:55,863][1157819] Updated weights for policy 0, policy_version 111428 (0.0006) [2024-09-30 02:12:56,429][1157819] Updated weights for policy 0, policy_version 111438 (0.0006) [2024-09-30 02:12:56,966][1157819] Updated weights for policy 0, policy_version 111448 (0.0006) [2024-09-30 02:12:57,528][1157819] Updated weights for policy 0, policy_version 111458 (0.0006) [2024-09-30 02:12:58,073][1157819] Updated weights for policy 0, policy_version 111468 (0.0006) [2024-09-30 02:12:58,616][1157819] Updated weights for policy 0, policy_version 111478 (0.0006) [2024-09-30 02:12:59,174][1157819] Updated weights for policy 0, policy_version 111488 (0.0007) [2024-09-30 02:12:59,697][1157819] Updated weights for policy 0, policy_version 111498 (0.0006) [2024-09-30 02:13:00,292][1157819] Updated weights for policy 0, policy_version 111508 (0.0006) [2024-09-30 02:13:00,466][1157520] Fps is (10 sec: 71679.8, 60 sec: 76185.6, 300 sec: 76796.5). Total num frames: 456749056. Throughput: 0: 18809.7. Samples: 104156828. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:13:00,466][1157520] Avg episode reward: [(0, '54.749')] [2024-09-30 02:13:00,828][1157819] Updated weights for policy 0, policy_version 111518 (0.0006) [2024-09-30 02:13:01,393][1157819] Updated weights for policy 0, policy_version 111528 (0.0006) [2024-09-30 02:13:01,982][1157819] Updated weights for policy 0, policy_version 111538 (0.0006) [2024-09-30 02:13:02,553][1157819] Updated weights for policy 0, policy_version 111548 (0.0006) [2024-09-30 02:13:03,119][1157819] Updated weights for policy 0, policy_version 111558 (0.0006) [2024-09-30 02:13:03,707][1157819] Updated weights for policy 0, policy_version 111568 (0.0006) [2024-09-30 02:13:04,254][1157819] Updated weights for policy 0, policy_version 111578 (0.0006) [2024-09-30 02:13:04,862][1157819] Updated weights for policy 0, policy_version 111588 (0.0006) [2024-09-30 02:13:05,417][1157819] Updated weights for policy 0, policy_version 111598 (0.0006) [2024-09-30 02:13:05,466][1157520] Fps is (10 sec: 73318.8, 60 sec: 75707.7, 300 sec: 76671.5). Total num frames: 457109504. Throughput: 0: 18465.6. Samples: 104265220. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:13:05,466][1157520] Avg episode reward: [(0, '56.347')] [2024-09-30 02:13:05,995][1157819] Updated weights for policy 0, policy_version 111608 (0.0006) [2024-09-30 02:13:06,546][1157819] Updated weights for policy 0, policy_version 111618 (0.0006) [2024-09-30 02:13:07,116][1157819] Updated weights for policy 0, policy_version 111628 (0.0006) [2024-09-30 02:13:07,657][1157819] Updated weights for policy 0, policy_version 111638 (0.0006) [2024-09-30 02:13:08,182][1157819] Updated weights for policy 0, policy_version 111648 (0.0006) [2024-09-30 02:13:08,740][1157819] Updated weights for policy 0, policy_version 111658 (0.0006) [2024-09-30 02:13:09,276][1157819] Updated weights for policy 0, policy_version 111668 (0.0006) [2024-09-30 02:13:09,810][1157819] Updated weights for policy 0, policy_version 111678 (0.0006) [2024-09-30 02:13:10,355][1157819] Updated weights for policy 0, policy_version 111688 (0.0006) [2024-09-30 02:13:10,466][1157520] Fps is (10 sec: 72908.6, 60 sec: 75298.0, 300 sec: 76602.2). Total num frames: 457478144. Throughput: 0: 18321.1. Samples: 104320000. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:13:10,466][1157520] Avg episode reward: [(0, '52.563')] [2024-09-30 02:13:10,901][1157819] Updated weights for policy 0, policy_version 111698 (0.0006) [2024-09-30 02:13:11,464][1157819] Updated weights for policy 0, policy_version 111708 (0.0006) [2024-09-30 02:13:12,022][1157819] Updated weights for policy 0, policy_version 111718 (0.0006) [2024-09-30 02:13:12,537][1157819] Updated weights for policy 0, policy_version 111728 (0.0006) [2024-09-30 02:13:13,073][1157819] Updated weights for policy 0, policy_version 111738 (0.0006) [2024-09-30 02:13:13,614][1157819] Updated weights for policy 0, policy_version 111748 (0.0006) [2024-09-30 02:13:14,145][1157819] Updated weights for policy 0, policy_version 111758 (0.0006) [2024-09-30 02:13:14,676][1157819] Updated weights for policy 0, policy_version 111768 (0.0006) [2024-09-30 02:13:15,247][1157819] Updated weights for policy 0, policy_version 111778 (0.0006) [2024-09-30 02:13:15,466][1157520] Fps is (10 sec: 74957.5, 60 sec: 74683.6, 300 sec: 76518.8). Total num frames: 457859072. Throughput: 0: 18312.1. Samples: 104433384. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:13:15,466][1157520] Avg episode reward: [(0, '54.307')] [2024-09-30 02:13:15,775][1157819] Updated weights for policy 0, policy_version 111788 (0.0006) [2024-09-30 02:13:16,316][1157819] Updated weights for policy 0, policy_version 111798 (0.0006) [2024-09-30 02:13:16,891][1157819] Updated weights for policy 0, policy_version 111808 (0.0007) [2024-09-30 02:13:17,428][1157819] Updated weights for policy 0, policy_version 111818 (0.0006) [2024-09-30 02:13:17,978][1157819] Updated weights for policy 0, policy_version 111828 (0.0006) [2024-09-30 02:13:18,527][1157819] Updated weights for policy 0, policy_version 111838 (0.0006) [2024-09-30 02:13:18,581][1157736] Signal inference workers to stop experience collection... (7400 times) [2024-09-30 02:13:18,581][1157736] Signal inference workers to resume experience collection... (7400 times) [2024-09-30 02:13:18,585][1157819] InferenceWorker_p0-w0: stopping experience collection (7400 times) [2024-09-30 02:13:18,585][1157819] InferenceWorker_p0-w0: resuming experience collection (7400 times) [2024-09-30 02:13:19,051][1157819] Updated weights for policy 0, policy_version 111848 (0.0006) [2024-09-30 02:13:19,613][1157819] Updated weights for policy 0, policy_version 111858 (0.0006) [2024-09-30 02:13:20,128][1157819] Updated weights for policy 0, policy_version 111868 (0.0006) [2024-09-30 02:13:20,466][1157520] Fps is (10 sec: 75366.5, 60 sec: 74069.2, 300 sec: 76435.5). Total num frames: 458231808. Throughput: 0: 18327.9. Samples: 104546048. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:13:20,466][1157520] Avg episode reward: [(0, '54.827')] [2024-09-30 02:13:20,699][1157819] Updated weights for policy 0, policy_version 111878 (0.0006) [2024-09-30 02:13:21,241][1157819] Updated weights for policy 0, policy_version 111888 (0.0006) [2024-09-30 02:13:21,826][1157819] Updated weights for policy 0, policy_version 111898 (0.0006) [2024-09-30 02:13:22,397][1157819] Updated weights for policy 0, policy_version 111908 (0.0006) [2024-09-30 02:13:22,907][1157819] Updated weights for policy 0, policy_version 111918 (0.0006) [2024-09-30 02:13:23,479][1157819] Updated weights for policy 0, policy_version 111928 (0.0006) [2024-09-30 02:13:24,010][1157819] Updated weights for policy 0, policy_version 111938 (0.0006) [2024-09-30 02:13:24,559][1157819] Updated weights for policy 0, policy_version 111948 (0.0006) [2024-09-30 02:13:25,122][1157819] Updated weights for policy 0, policy_version 111958 (0.0006) [2024-09-30 02:13:25,466][1157520] Fps is (10 sec: 74547.0, 60 sec: 73659.8, 300 sec: 76324.4). Total num frames: 458604544. Throughput: 0: 18354.0. Samples: 104601768. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:13:25,466][1157520] Avg episode reward: [(0, '53.573')] [2024-09-30 02:13:25,674][1157819] Updated weights for policy 0, policy_version 111968 (0.0006) [2024-09-30 02:13:26,213][1157819] Updated weights for policy 0, policy_version 111978 (0.0006) [2024-09-30 02:13:26,788][1157819] Updated weights for policy 0, policy_version 111988 (0.0006) [2024-09-30 02:13:27,347][1157819] Updated weights for policy 0, policy_version 111998 (0.0006) [2024-09-30 02:13:27,899][1157819] Updated weights for policy 0, policy_version 112008 (0.0006) [2024-09-30 02:13:28,432][1157819] Updated weights for policy 0, policy_version 112018 (0.0006) [2024-09-30 02:13:28,986][1157819] Updated weights for policy 0, policy_version 112028 (0.0006) [2024-09-30 02:13:29,534][1157819] Updated weights for policy 0, policy_version 112038 (0.0006) [2024-09-30 02:13:30,068][1157819] Updated weights for policy 0, policy_version 112048 (0.0006) [2024-09-30 02:13:30,466][1157520] Fps is (10 sec: 74137.7, 60 sec: 73591.5, 300 sec: 76213.4). Total num frames: 458973184. Throughput: 0: 18370.6. Samples: 104713104. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:13:30,466][1157520] Avg episode reward: [(0, '56.481')] [2024-09-30 02:13:30,614][1157819] Updated weights for policy 0, policy_version 112058 (0.0006) [2024-09-30 02:13:31,193][1157819] Updated weights for policy 0, policy_version 112068 (0.0006) [2024-09-30 02:13:31,784][1157819] Updated weights for policy 0, policy_version 112078 (0.0006) [2024-09-30 02:13:32,369][1157819] Updated weights for policy 0, policy_version 112088 (0.0006) [2024-09-30 02:13:32,995][1157819] Updated weights for policy 0, policy_version 112098 (0.0006) [2024-09-30 02:13:33,578][1157819] Updated weights for policy 0, policy_version 112108 (0.0006) [2024-09-30 02:13:34,144][1157819] Updated weights for policy 0, policy_version 112118 (0.0006) [2024-09-30 02:13:34,708][1157819] Updated weights for policy 0, policy_version 112128 (0.0006) [2024-09-30 02:13:35,275][1157819] Updated weights for policy 0, policy_version 112138 (0.0006) [2024-09-30 02:13:35,466][1157520] Fps is (10 sec: 72499.0, 60 sec: 73318.4, 300 sec: 76185.6). Total num frames: 459329536. Throughput: 0: 18312.8. Samples: 104820572. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:13:35,466][1157520] Avg episode reward: [(0, '55.062')] [2024-09-30 02:13:35,837][1157819] Updated weights for policy 0, policy_version 112148 (0.0006) [2024-09-30 02:13:36,428][1157819] Updated weights for policy 0, policy_version 112158 (0.0006) [2024-09-30 02:13:36,946][1157819] Updated weights for policy 0, policy_version 112168 (0.0007) [2024-09-30 02:13:37,491][1157819] Updated weights for policy 0, policy_version 112178 (0.0006) [2024-09-30 02:13:38,073][1157819] Updated weights for policy 0, policy_version 112188 (0.0007) [2024-09-30 02:13:38,601][1157819] Updated weights for policy 0, policy_version 112198 (0.0006) [2024-09-30 02:13:39,138][1157819] Updated weights for policy 0, policy_version 112208 (0.0006) [2024-09-30 02:13:39,725][1157819] Updated weights for policy 0, policy_version 112218 (0.0006) [2024-09-30 02:13:40,348][1157819] Updated weights for policy 0, policy_version 112228 (0.0006) [2024-09-30 02:13:40,466][1157520] Fps is (10 sec: 72089.4, 60 sec: 73181.9, 300 sec: 76157.8). Total num frames: 459694080. Throughput: 0: 18424.8. Samples: 104875788. Policy #0 lag: (min: 0.0, avg: 1.7, max: 5.0) [2024-09-30 02:13:40,466][1157520] Avg episode reward: [(0, '55.585')] [2024-09-30 02:13:40,921][1157819] Updated weights for policy 0, policy_version 112238 (0.0006) [2024-09-30 02:13:41,503][1157819] Updated weights for policy 0, policy_version 112248 (0.0006) [2024-09-30 02:13:42,080][1157819] Updated weights for policy 0, policy_version 112258 (0.0006) [2024-09-30 02:13:42,577][1157819] Updated weights for policy 0, policy_version 112268 (0.0006) [2024-09-30 02:13:43,129][1157819] Updated weights for policy 0, policy_version 112278 (0.0006) [2024-09-30 02:13:43,651][1157819] Updated weights for policy 0, policy_version 112288 (0.0006) [2024-09-30 02:13:44,312][1157819] Updated weights for policy 0, policy_version 112298 (0.0006) [2024-09-30 02:13:44,658][1157736] Signal inference workers to stop experience collection... (7450 times) [2024-09-30 02:13:44,658][1157736] Signal inference workers to resume experience collection... (7450 times) [2024-09-30 02:13:44,663][1157819] InferenceWorker_p0-w0: stopping experience collection (7450 times) [2024-09-30 02:13:44,663][1157819] InferenceWorker_p0-w0: resuming experience collection (7450 times) [2024-09-30 02:13:44,851][1157819] Updated weights for policy 0, policy_version 112308 (0.0006) [2024-09-30 02:13:45,466][1157520] Fps is (10 sec: 72089.5, 60 sec: 73045.3, 300 sec: 76074.5). Total num frames: 460050432. Throughput: 0: 18400.9. Samples: 104984868. Policy #0 lag: (min: 0.0, avg: 1.7, max: 5.0) [2024-09-30 02:13:45,466][1157520] Avg episode reward: [(0, '55.113')] [2024-09-30 02:13:45,476][1157819] Updated weights for policy 0, policy_version 112318 (0.0006) [2024-09-30 02:13:46,061][1157819] Updated weights for policy 0, policy_version 112328 (0.0006) [2024-09-30 02:13:46,654][1157819] Updated weights for policy 0, policy_version 112338 (0.0006) [2024-09-30 02:13:47,265][1157819] Updated weights for policy 0, policy_version 112348 (0.0006) [2024-09-30 02:13:47,848][1157819] Updated weights for policy 0, policy_version 112358 (0.0006) [2024-09-30 02:13:48,416][1157819] Updated weights for policy 0, policy_version 112368 (0.0006) [2024-09-30 02:13:48,979][1157819] Updated weights for policy 0, policy_version 112378 (0.0006) [2024-09-30 02:13:49,568][1157819] Updated weights for policy 0, policy_version 112388 (0.0006) [2024-09-30 02:13:50,161][1157819] Updated weights for policy 0, policy_version 112398 (0.0006) [2024-09-30 02:13:50,466][1157520] Fps is (10 sec: 70861.1, 60 sec: 72840.5, 300 sec: 76005.1). Total num frames: 460402688. Throughput: 0: 18306.9. Samples: 105089028. Policy #0 lag: (min: 0.0, avg: 1.7, max: 5.0) [2024-09-30 02:13:50,466][1157520] Avg episode reward: [(0, '54.547')] [2024-09-30 02:13:50,737][1157819] Updated weights for policy 0, policy_version 112408 (0.0006) [2024-09-30 02:13:51,320][1157819] Updated weights for policy 0, policy_version 112418 (0.0006) [2024-09-30 02:13:51,906][1157819] Updated weights for policy 0, policy_version 112428 (0.0006) [2024-09-30 02:13:52,507][1157819] Updated weights for policy 0, policy_version 112438 (0.0006) [2024-09-30 02:13:53,089][1157819] Updated weights for policy 0, policy_version 112448 (0.0006) [2024-09-30 02:13:53,675][1157819] Updated weights for policy 0, policy_version 112458 (0.0006) [2024-09-30 02:13:54,230][1157819] Updated weights for policy 0, policy_version 112468 (0.0006) [2024-09-30 02:13:54,752][1157819] Updated weights for policy 0, policy_version 112478 (0.0006) [2024-09-30 02:13:55,308][1157819] Updated weights for policy 0, policy_version 112488 (0.0006) [2024-09-30 02:13:55,466][1157520] Fps is (10 sec: 70860.9, 60 sec: 73045.5, 300 sec: 75935.7). Total num frames: 460759040. Throughput: 0: 18263.6. Samples: 105141860. Policy #0 lag: (min: 0.0, avg: 1.7, max: 5.0) [2024-09-30 02:13:55,466][1157520] Avg episode reward: [(0, '56.061')] [2024-09-30 02:13:55,836][1157819] Updated weights for policy 0, policy_version 112498 (0.0006) [2024-09-30 02:13:56,414][1157819] Updated weights for policy 0, policy_version 112508 (0.0006) [2024-09-30 02:13:56,999][1157819] Updated weights for policy 0, policy_version 112518 (0.0006) [2024-09-30 02:13:57,586][1157819] Updated weights for policy 0, policy_version 112528 (0.0006) [2024-09-30 02:13:58,176][1157819] Updated weights for policy 0, policy_version 112538 (0.0006) [2024-09-30 02:13:58,768][1157819] Updated weights for policy 0, policy_version 112548 (0.0006) [2024-09-30 02:13:59,345][1157819] Updated weights for policy 0, policy_version 112558 (0.0006) [2024-09-30 02:13:59,952][1157819] Updated weights for policy 0, policy_version 112568 (0.0006) [2024-09-30 02:14:00,466][1157520] Fps is (10 sec: 70860.2, 60 sec: 72703.9, 300 sec: 75796.8). Total num frames: 461111296. Throughput: 0: 18131.3. Samples: 105249292. Policy #0 lag: (min: 0.0, avg: 1.7, max: 5.0) [2024-09-30 02:14:00,466][1157520] Avg episode reward: [(0, '55.659')] [2024-09-30 02:14:00,504][1157819] Updated weights for policy 0, policy_version 112578 (0.0006) [2024-09-30 02:14:01,060][1157819] Updated weights for policy 0, policy_version 112588 (0.0006) [2024-09-30 02:14:01,641][1157819] Updated weights for policy 0, policy_version 112598 (0.0006) [2024-09-30 02:14:02,195][1157819] Updated weights for policy 0, policy_version 112608 (0.0006) [2024-09-30 02:14:02,685][1157819] Updated weights for policy 0, policy_version 112618 (0.0006) [2024-09-30 02:14:03,230][1157819] Updated weights for policy 0, policy_version 112628 (0.0006) [2024-09-30 02:14:03,757][1157819] Updated weights for policy 0, policy_version 112638 (0.0006) [2024-09-30 02:14:04,259][1157819] Updated weights for policy 0, policy_version 112648 (0.0006) [2024-09-30 02:14:04,766][1157819] Updated weights for policy 0, policy_version 112658 (0.0006) [2024-09-30 02:14:05,267][1157819] Updated weights for policy 0, policy_version 112668 (0.0006) [2024-09-30 02:14:05,466][1157520] Fps is (10 sec: 74137.3, 60 sec: 73181.9, 300 sec: 75741.3). Total num frames: 461500416. Throughput: 0: 18135.3. Samples: 105362136. Policy #0 lag: (min: 0.0, avg: 1.7, max: 5.0) [2024-09-30 02:14:05,466][1157520] Avg episode reward: [(0, '52.882')] [2024-09-30 02:14:05,472][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000112671_461500416.pth... [2024-09-30 02:14:05,542][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000108237_443338752.pth [2024-09-30 02:14:05,842][1157819] Updated weights for policy 0, policy_version 112678 (0.0006) [2024-09-30 02:14:06,405][1157819] Updated weights for policy 0, policy_version 112688 (0.0006) [2024-09-30 02:14:06,938][1157819] Updated weights for policy 0, policy_version 112698 (0.0006) [2024-09-30 02:14:07,464][1157819] Updated weights for policy 0, policy_version 112708 (0.0006) [2024-09-30 02:14:07,974][1157819] Updated weights for policy 0, policy_version 112718 (0.0006) [2024-09-30 02:14:08,489][1157819] Updated weights for policy 0, policy_version 112728 (0.0006) [2024-09-30 02:14:09,075][1157819] Updated weights for policy 0, policy_version 112738 (0.0006) [2024-09-30 02:14:09,575][1157819] Updated weights for policy 0, policy_version 112748 (0.0006) [2024-09-30 02:14:10,102][1157819] Updated weights for policy 0, policy_version 112758 (0.0006) [2024-09-30 02:14:10,466][1157520] Fps is (10 sec: 77004.8, 60 sec: 73386.6, 300 sec: 75727.4). Total num frames: 461881344. Throughput: 0: 18171.1. Samples: 105419468. Policy #0 lag: (min: 0.0, avg: 1.7, max: 5.0) [2024-09-30 02:14:10,466][1157520] Avg episode reward: [(0, '54.108')] [2024-09-30 02:14:10,614][1157819] Updated weights for policy 0, policy_version 112768 (0.0006) [2024-09-30 02:14:11,126][1157819] Updated weights for policy 0, policy_version 112778 (0.0006) [2024-09-30 02:14:11,608][1157736] Signal inference workers to stop experience collection... (7500 times) [2024-09-30 02:14:11,608][1157736] Signal inference workers to resume experience collection... (7500 times) [2024-09-30 02:14:11,611][1157819] InferenceWorker_p0-w0: stopping experience collection (7500 times) [2024-09-30 02:14:11,614][1157819] InferenceWorker_p0-w0: resuming experience collection (7500 times) [2024-09-30 02:14:11,627][1157819] Updated weights for policy 0, policy_version 112788 (0.0006) [2024-09-30 02:14:12,178][1157819] Updated weights for policy 0, policy_version 112798 (0.0006) [2024-09-30 02:14:12,644][1157819] Updated weights for policy 0, policy_version 112808 (0.0006) [2024-09-30 02:14:13,166][1157819] Updated weights for policy 0, policy_version 112818 (0.0006) [2024-09-30 02:14:13,680][1157819] Updated weights for policy 0, policy_version 112828 (0.0006) [2024-09-30 02:14:14,173][1157819] Updated weights for policy 0, policy_version 112838 (0.0006) [2024-09-30 02:14:14,657][1157819] Updated weights for policy 0, policy_version 112848 (0.0006) [2024-09-30 02:14:15,173][1157819] Updated weights for policy 0, policy_version 112858 (0.0006) [2024-09-30 02:14:15,466][1157520] Fps is (10 sec: 79052.2, 60 sec: 73864.3, 300 sec: 75880.1). Total num frames: 462290944. Throughput: 0: 18343.2. Samples: 105538552. Policy #0 lag: (min: 0.0, avg: 1.7, max: 5.0) [2024-09-30 02:14:15,466][1157520] Avg episode reward: [(0, '55.968')] [2024-09-30 02:14:15,656][1157819] Updated weights for policy 0, policy_version 112868 (0.0006) [2024-09-30 02:14:16,132][1157819] Updated weights for policy 0, policy_version 112878 (0.0006) [2024-09-30 02:14:16,617][1157819] Updated weights for policy 0, policy_version 112888 (0.0006) [2024-09-30 02:14:17,077][1157819] Updated weights for policy 0, policy_version 112898 (0.0006) [2024-09-30 02:14:17,579][1157819] Updated weights for policy 0, policy_version 112908 (0.0006) [2024-09-30 02:14:18,049][1157819] Updated weights for policy 0, policy_version 112918 (0.0006) [2024-09-30 02:14:18,524][1157819] Updated weights for policy 0, policy_version 112928 (0.0006) [2024-09-30 02:14:19,034][1157819] Updated weights for policy 0, policy_version 112938 (0.0006) [2024-09-30 02:14:19,544][1157819] Updated weights for policy 0, policy_version 112948 (0.0006) [2024-09-30 02:14:20,086][1157819] Updated weights for policy 0, policy_version 112958 (0.0006) [2024-09-30 02:14:20,466][1157520] Fps is (10 sec: 82329.8, 60 sec: 74547.1, 300 sec: 75935.7). Total num frames: 462704640. Throughput: 0: 18730.7. Samples: 105663456. Policy #0 lag: (min: 0.0, avg: 1.7, max: 5.0) [2024-09-30 02:14:20,466][1157520] Avg episode reward: [(0, '54.190')] [2024-09-30 02:14:20,621][1157819] Updated weights for policy 0, policy_version 112968 (0.0006) [2024-09-30 02:14:21,156][1157819] Updated weights for policy 0, policy_version 112978 (0.0006) [2024-09-30 02:14:21,679][1157819] Updated weights for policy 0, policy_version 112988 (0.0006) [2024-09-30 02:14:22,222][1157819] Updated weights for policy 0, policy_version 112998 (0.0006) [2024-09-30 02:14:22,776][1157819] Updated weights for policy 0, policy_version 113008 (0.0006) [2024-09-30 02:14:23,308][1157819] Updated weights for policy 0, policy_version 113018 (0.0006) [2024-09-30 02:14:23,893][1157819] Updated weights for policy 0, policy_version 113028 (0.0006) [2024-09-30 02:14:24,427][1157819] Updated weights for policy 0, policy_version 113038 (0.0006) [2024-09-30 02:14:24,973][1157819] Updated weights for policy 0, policy_version 113048 (0.0006) [2024-09-30 02:14:25,466][1157520] Fps is (10 sec: 78642.9, 60 sec: 74547.0, 300 sec: 75921.7). Total num frames: 463077376. Throughput: 0: 18767.7. Samples: 105720336. Policy #0 lag: (min: 0.0, avg: 1.7, max: 5.0) [2024-09-30 02:14:25,466][1157520] Avg episode reward: [(0, '54.580')] [2024-09-30 02:14:25,499][1157819] Updated weights for policy 0, policy_version 113058 (0.0007) [2024-09-30 02:14:26,082][1157819] Updated weights for policy 0, policy_version 113068 (0.0006) [2024-09-30 02:14:26,589][1157819] Updated weights for policy 0, policy_version 113078 (0.0006) [2024-09-30 02:14:27,105][1157819] Updated weights for policy 0, policy_version 113088 (0.0006) [2024-09-30 02:14:27,674][1157819] Updated weights for policy 0, policy_version 113098 (0.0006) [2024-09-30 02:14:28,220][1157819] Updated weights for policy 0, policy_version 113108 (0.0006) [2024-09-30 02:14:28,719][1157819] Updated weights for policy 0, policy_version 113118 (0.0006) [2024-09-30 02:14:29,231][1157819] Updated weights for policy 0, policy_version 113128 (0.0007) [2024-09-30 02:14:29,770][1157819] Updated weights for policy 0, policy_version 113138 (0.0007) [2024-09-30 02:14:30,359][1157819] Updated weights for policy 0, policy_version 113148 (0.0007) [2024-09-30 02:14:30,466][1157520] Fps is (10 sec: 75776.3, 60 sec: 74820.3, 300 sec: 76074.5). Total num frames: 463462400. Throughput: 0: 18872.3. Samples: 105834120. Policy #0 lag: (min: 0.0, avg: 1.7, max: 5.0) [2024-09-30 02:14:30,466][1157520] Avg episode reward: [(0, '54.731')] [2024-09-30 02:14:30,601][1157736] Signal inference workers to stop experience collection... (7550 times) [2024-09-30 02:14:30,603][1157736] Signal inference workers to resume experience collection... (7550 times) [2024-09-30 02:14:30,606][1157819] InferenceWorker_p0-w0: stopping experience collection (7550 times) [2024-09-30 02:14:30,608][1157819] InferenceWorker_p0-w0: resuming experience collection (7550 times) [2024-09-30 02:14:30,907][1157819] Updated weights for policy 0, policy_version 113158 (0.0007) [2024-09-30 02:14:31,508][1157819] Updated weights for policy 0, policy_version 113168 (0.0007) [2024-09-30 02:14:32,073][1157819] Updated weights for policy 0, policy_version 113178 (0.0006) [2024-09-30 02:14:32,674][1157819] Updated weights for policy 0, policy_version 113188 (0.0006) [2024-09-30 02:14:33,266][1157819] Updated weights for policy 0, policy_version 113198 (0.0006) [2024-09-30 02:14:33,859][1157819] Updated weights for policy 0, policy_version 113208 (0.0006) [2024-09-30 02:14:34,463][1157819] Updated weights for policy 0, policy_version 113218 (0.0006) [2024-09-30 02:14:35,083][1157819] Updated weights for policy 0, policy_version 113228 (0.0006) [2024-09-30 02:14:35,466][1157520] Fps is (10 sec: 72908.4, 60 sec: 74615.2, 300 sec: 75963.4). Total num frames: 463806464. Throughput: 0: 18919.1. Samples: 105940392. Policy #0 lag: (min: 0.0, avg: 1.7, max: 5.0) [2024-09-30 02:14:35,466][1157520] Avg episode reward: [(0, '54.252')] [2024-09-30 02:14:35,708][1157819] Updated weights for policy 0, policy_version 113238 (0.0006) [2024-09-30 02:14:36,294][1157819] Updated weights for policy 0, policy_version 113248 (0.0006) [2024-09-30 02:14:36,833][1157819] Updated weights for policy 0, policy_version 113258 (0.0006) [2024-09-30 02:14:37,400][1157819] Updated weights for policy 0, policy_version 113268 (0.0006) [2024-09-30 02:14:37,930][1157819] Updated weights for policy 0, policy_version 113278 (0.0006) [2024-09-30 02:14:38,444][1157819] Updated weights for policy 0, policy_version 113288 (0.0006) [2024-09-30 02:14:39,026][1157819] Updated weights for policy 0, policy_version 113298 (0.0006) [2024-09-30 02:14:39,548][1157819] Updated weights for policy 0, policy_version 113308 (0.0006) [2024-09-30 02:14:40,113][1157819] Updated weights for policy 0, policy_version 113318 (0.0006) [2024-09-30 02:14:40,466][1157520] Fps is (10 sec: 71269.9, 60 sec: 74683.7, 300 sec: 75963.4). Total num frames: 464175104. Throughput: 0: 18940.0. Samples: 105994160. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:14:40,466][1157520] Avg episode reward: [(0, '53.690')] [2024-09-30 02:14:40,666][1157819] Updated weights for policy 0, policy_version 113328 (0.0006) [2024-09-30 02:14:41,213][1157819] Updated weights for policy 0, policy_version 113338 (0.0006) [2024-09-30 02:14:41,752][1157819] Updated weights for policy 0, policy_version 113348 (0.0006) [2024-09-30 02:14:42,255][1157819] Updated weights for policy 0, policy_version 113358 (0.0006) [2024-09-30 02:14:42,846][1157819] Updated weights for policy 0, policy_version 113368 (0.0006) [2024-09-30 02:14:43,360][1157819] Updated weights for policy 0, policy_version 113378 (0.0006) [2024-09-30 02:14:43,924][1157819] Updated weights for policy 0, policy_version 113388 (0.0006) [2024-09-30 02:14:44,488][1157819] Updated weights for policy 0, policy_version 113398 (0.0006) [2024-09-30 02:14:45,062][1157819] Updated weights for policy 0, policy_version 113408 (0.0006) [2024-09-30 02:14:45,466][1157520] Fps is (10 sec: 74138.0, 60 sec: 74956.6, 300 sec: 75921.7). Total num frames: 464547840. Throughput: 0: 19052.6. Samples: 106106660. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:14:45,466][1157520] Avg episode reward: [(0, '55.544')] [2024-09-30 02:14:45,609][1157819] Updated weights for policy 0, policy_version 113418 (0.0006) [2024-09-30 02:14:46,109][1157819] Updated weights for policy 0, policy_version 113428 (0.0006) [2024-09-30 02:14:46,654][1157819] Updated weights for policy 0, policy_version 113438 (0.0006) [2024-09-30 02:14:47,209][1157819] Updated weights for policy 0, policy_version 113448 (0.0006) [2024-09-30 02:14:47,739][1157819] Updated weights for policy 0, policy_version 113458 (0.0006) [2024-09-30 02:14:48,242][1157819] Updated weights for policy 0, policy_version 113468 (0.0006) [2024-09-30 02:14:48,756][1157819] Updated weights for policy 0, policy_version 113478 (0.0006) [2024-09-30 02:14:49,270][1157819] Updated weights for policy 0, policy_version 113488 (0.0006) [2024-09-30 02:14:49,818][1157819] Updated weights for policy 0, policy_version 113498 (0.0006) [2024-09-30 02:14:50,379][1157819] Updated weights for policy 0, policy_version 113508 (0.0006) [2024-09-30 02:14:50,466][1157520] Fps is (10 sec: 76185.1, 60 sec: 75571.0, 300 sec: 75935.6). Total num frames: 464936960. Throughput: 0: 19096.8. Samples: 106221492. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:14:50,466][1157520] Avg episode reward: [(0, '55.036')] [2024-09-30 02:14:50,881][1157819] Updated weights for policy 0, policy_version 113518 (0.0006) [2024-09-30 02:14:51,381][1157819] Updated weights for policy 0, policy_version 113528 (0.0006) [2024-09-30 02:14:51,917][1157819] Updated weights for policy 0, policy_version 113538 (0.0006) [2024-09-30 02:14:52,111][1157736] Signal inference workers to stop experience collection... (7600 times) [2024-09-30 02:14:52,111][1157736] Signal inference workers to resume experience collection... (7600 times) [2024-09-30 02:14:52,115][1157819] InferenceWorker_p0-w0: stopping experience collection (7600 times) [2024-09-30 02:14:52,115][1157819] InferenceWorker_p0-w0: resuming experience collection (7600 times) [2024-09-30 02:14:52,450][1157819] Updated weights for policy 0, policy_version 113548 (0.0006) [2024-09-30 02:14:52,973][1157819] Updated weights for policy 0, policy_version 113558 (0.0006) [2024-09-30 02:14:53,484][1157819] Updated weights for policy 0, policy_version 113568 (0.0006) [2024-09-30 02:14:54,023][1157819] Updated weights for policy 0, policy_version 113578 (0.0006) [2024-09-30 02:14:54,537][1157819] Updated weights for policy 0, policy_version 113588 (0.0006) [2024-09-30 02:14:55,081][1157819] Updated weights for policy 0, policy_version 113598 (0.0006) [2024-09-30 02:14:55,466][1157520] Fps is (10 sec: 77824.0, 60 sec: 76117.1, 300 sec: 75907.8). Total num frames: 465326080. Throughput: 0: 19119.4. Samples: 106279844. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:14:55,466][1157520] Avg episode reward: [(0, '54.935')] [2024-09-30 02:14:55,583][1157819] Updated weights for policy 0, policy_version 113608 (0.0006) [2024-09-30 02:14:56,086][1157819] Updated weights for policy 0, policy_version 113618 (0.0006) [2024-09-30 02:14:56,627][1157819] Updated weights for policy 0, policy_version 113628 (0.0006) [2024-09-30 02:14:57,154][1157819] Updated weights for policy 0, policy_version 113638 (0.0006) [2024-09-30 02:14:57,674][1157819] Updated weights for policy 0, policy_version 113648 (0.0006) [2024-09-30 02:14:58,191][1157819] Updated weights for policy 0, policy_version 113658 (0.0006) [2024-09-30 02:14:58,726][1157819] Updated weights for policy 0, policy_version 113668 (0.0006) [2024-09-30 02:14:59,227][1157819] Updated weights for policy 0, policy_version 113678 (0.0006) [2024-09-30 02:14:59,773][1157819] Updated weights for policy 0, policy_version 113688 (0.0006) [2024-09-30 02:15:00,271][1157819] Updated weights for policy 0, policy_version 113698 (0.0006) [2024-09-30 02:15:00,466][1157520] Fps is (10 sec: 78234.4, 60 sec: 76800.0, 300 sec: 75866.2). Total num frames: 465719296. Throughput: 0: 19086.8. Samples: 106397456. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:15:00,466][1157520] Avg episode reward: [(0, '55.486')] [2024-09-30 02:15:00,795][1157819] Updated weights for policy 0, policy_version 113708 (0.0006) [2024-09-30 02:15:01,320][1157819] Updated weights for policy 0, policy_version 113718 (0.0006) [2024-09-30 02:15:01,878][1157819] Updated weights for policy 0, policy_version 113728 (0.0006) [2024-09-30 02:15:02,388][1157819] Updated weights for policy 0, policy_version 113738 (0.0006) [2024-09-30 02:15:02,898][1157819] Updated weights for policy 0, policy_version 113748 (0.0006) [2024-09-30 02:15:03,430][1157819] Updated weights for policy 0, policy_version 113758 (0.0006) [2024-09-30 02:15:03,950][1157819] Updated weights for policy 0, policy_version 113768 (0.0006) [2024-09-30 02:15:04,480][1157819] Updated weights for policy 0, policy_version 113778 (0.0006) [2024-09-30 02:15:04,991][1157819] Updated weights for policy 0, policy_version 113788 (0.0006) [2024-09-30 02:15:05,466][1157520] Fps is (10 sec: 78644.4, 60 sec: 76868.3, 300 sec: 75838.5). Total num frames: 466112512. Throughput: 0: 18922.2. Samples: 106514956. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:15:05,466][1157520] Avg episode reward: [(0, '55.433')] [2024-09-30 02:15:05,502][1157819] Updated weights for policy 0, policy_version 113798 (0.0006) [2024-09-30 02:15:06,042][1157819] Updated weights for policy 0, policy_version 113808 (0.0006) [2024-09-30 02:15:06,569][1157819] Updated weights for policy 0, policy_version 113818 (0.0006) [2024-09-30 02:15:07,066][1157819] Updated weights for policy 0, policy_version 113828 (0.0006) [2024-09-30 02:15:07,603][1157819] Updated weights for policy 0, policy_version 113838 (0.0006) [2024-09-30 02:15:08,134][1157819] Updated weights for policy 0, policy_version 113848 (0.0006) [2024-09-30 02:15:08,641][1157819] Updated weights for policy 0, policy_version 113858 (0.0006) [2024-09-30 02:15:09,159][1157819] Updated weights for policy 0, policy_version 113868 (0.0006) [2024-09-30 02:15:09,694][1157819] Updated weights for policy 0, policy_version 113878 (0.0006) [2024-09-30 02:15:10,179][1157819] Updated weights for policy 0, policy_version 113888 (0.0006) [2024-09-30 02:15:10,466][1157520] Fps is (10 sec: 78643.3, 60 sec: 77073.1, 300 sec: 75824.6). Total num frames: 466505728. Throughput: 0: 18960.0. Samples: 106573532. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:15:10,466][1157520] Avg episode reward: [(0, '57.045')] [2024-09-30 02:15:10,709][1157819] Updated weights for policy 0, policy_version 113898 (0.0006) [2024-09-30 02:15:11,259][1157819] Updated weights for policy 0, policy_version 113908 (0.0006) [2024-09-30 02:15:11,759][1157819] Updated weights for policy 0, policy_version 113918 (0.0006) [2024-09-30 02:15:12,275][1157819] Updated weights for policy 0, policy_version 113928 (0.0006) [2024-09-30 02:15:12,806][1157819] Updated weights for policy 0, policy_version 113938 (0.0006) [2024-09-30 02:15:13,361][1157819] Updated weights for policy 0, policy_version 113948 (0.0006) [2024-09-30 02:15:13,876][1157819] Updated weights for policy 0, policy_version 113958 (0.0006) [2024-09-30 02:15:14,380][1157819] Updated weights for policy 0, policy_version 113968 (0.0006) [2024-09-30 02:15:14,949][1157819] Updated weights for policy 0, policy_version 113978 (0.0006) [2024-09-30 02:15:15,466][1157520] Fps is (10 sec: 77824.9, 60 sec: 76663.8, 300 sec: 75782.9). Total num frames: 466890752. Throughput: 0: 19062.2. Samples: 106691916. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:15:15,466][1157520] Avg episode reward: [(0, '53.871')] [2024-09-30 02:15:15,489][1157819] Updated weights for policy 0, policy_version 113988 (0.0006) [2024-09-30 02:15:16,014][1157819] Updated weights for policy 0, policy_version 113998 (0.0006) [2024-09-30 02:15:16,522][1157819] Updated weights for policy 0, policy_version 114008 (0.0006) [2024-09-30 02:15:17,084][1157819] Updated weights for policy 0, policy_version 114018 (0.0006) [2024-09-30 02:15:17,607][1157819] Updated weights for policy 0, policy_version 114028 (0.0006) [2024-09-30 02:15:18,137][1157819] Updated weights for policy 0, policy_version 114038 (0.0006) [2024-09-30 02:15:18,648][1157819] Updated weights for policy 0, policy_version 114048 (0.0006) [2024-09-30 02:15:19,168][1157819] Updated weights for policy 0, policy_version 114058 (0.0006) [2024-09-30 02:15:19,666][1157819] Updated weights for policy 0, policy_version 114068 (0.0006) [2024-09-30 02:15:20,169][1157819] Updated weights for policy 0, policy_version 114078 (0.0006) [2024-09-30 02:15:20,466][1157520] Fps is (10 sec: 77415.2, 60 sec: 76254.0, 300 sec: 75810.7). Total num frames: 467279872. Throughput: 0: 19270.8. Samples: 106807572. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:15:20,466][1157520] Avg episode reward: [(0, '55.814')] [2024-09-30 02:15:20,723][1157819] Updated weights for policy 0, policy_version 114088 (0.0006) [2024-09-30 02:15:21,230][1157819] Updated weights for policy 0, policy_version 114098 (0.0006) [2024-09-30 02:15:21,748][1157819] Updated weights for policy 0, policy_version 114108 (0.0006) [2024-09-30 02:15:22,254][1157819] Updated weights for policy 0, policy_version 114118 (0.0006) [2024-09-30 02:15:22,807][1157819] Updated weights for policy 0, policy_version 114128 (0.0006) [2024-09-30 02:15:23,317][1157819] Updated weights for policy 0, policy_version 114138 (0.0006) [2024-09-30 02:15:23,815][1157819] Updated weights for policy 0, policy_version 114148 (0.0006) [2024-09-30 02:15:24,354][1157819] Updated weights for policy 0, policy_version 114158 (0.0006) [2024-09-30 02:15:24,875][1157819] Updated weights for policy 0, policy_version 114168 (0.0006) [2024-09-30 02:15:25,417][1157819] Updated weights for policy 0, policy_version 114178 (0.0006) [2024-09-30 02:15:25,466][1157520] Fps is (10 sec: 78643.2, 60 sec: 76663.8, 300 sec: 75894.0). Total num frames: 467677184. Throughput: 0: 19401.0. Samples: 106867204. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:15:25,466][1157520] Avg episode reward: [(0, '54.489')] [2024-09-30 02:15:25,915][1157819] Updated weights for policy 0, policy_version 114188 (0.0006) [2024-09-30 02:15:26,403][1157819] Updated weights for policy 0, policy_version 114198 (0.0005) [2024-09-30 02:15:26,886][1157819] Updated weights for policy 0, policy_version 114208 (0.0006) [2024-09-30 02:15:27,406][1157819] Updated weights for policy 0, policy_version 114218 (0.0006) [2024-09-30 02:15:27,872][1157819] Updated weights for policy 0, policy_version 114228 (0.0006) [2024-09-30 02:15:28,376][1157819] Updated weights for policy 0, policy_version 114238 (0.0006) [2024-09-30 02:15:28,862][1157819] Updated weights for policy 0, policy_version 114248 (0.0006) [2024-09-30 02:15:29,350][1157819] Updated weights for policy 0, policy_version 114258 (0.0006) [2024-09-30 02:15:29,882][1157819] Updated weights for policy 0, policy_version 114268 (0.0006) [2024-09-30 02:15:30,410][1157819] Updated weights for policy 0, policy_version 114278 (0.0006) [2024-09-30 02:15:30,466][1157520] Fps is (10 sec: 80691.1, 60 sec: 77073.2, 300 sec: 75921.8). Total num frames: 468086784. Throughput: 0: 19592.1. Samples: 106988300. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:15:30,466][1157520] Avg episode reward: [(0, '56.070')] [2024-09-30 02:15:30,927][1157819] Updated weights for policy 0, policy_version 114288 (0.0006) [2024-09-30 02:15:31,411][1157819] Updated weights for policy 0, policy_version 114298 (0.0006) [2024-09-30 02:15:31,908][1157819] Updated weights for policy 0, policy_version 114308 (0.0006) [2024-09-30 02:15:32,431][1157819] Updated weights for policy 0, policy_version 114318 (0.0006) [2024-09-30 02:15:32,935][1157819] Updated weights for policy 0, policy_version 114328 (0.0006) [2024-09-30 02:15:33,487][1157819] Updated weights for policy 0, policy_version 114338 (0.0006) [2024-09-30 02:15:33,994][1157819] Updated weights for policy 0, policy_version 114348 (0.0006) [2024-09-30 02:15:34,491][1157819] Updated weights for policy 0, policy_version 114358 (0.0006) [2024-09-30 02:15:35,027][1157819] Updated weights for policy 0, policy_version 114368 (0.0006) [2024-09-30 02:15:35,466][1157520] Fps is (10 sec: 80691.1, 60 sec: 77960.9, 300 sec: 75963.5). Total num frames: 468484096. Throughput: 0: 19703.5. Samples: 107108144. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:15:35,466][1157520] Avg episode reward: [(0, '56.525')] [2024-09-30 02:15:35,530][1157819] Updated weights for policy 0, policy_version 114378 (0.0006) [2024-09-30 02:15:36,024][1157819] Updated weights for policy 0, policy_version 114388 (0.0006) [2024-09-30 02:15:36,585][1157819] Updated weights for policy 0, policy_version 114398 (0.0006) [2024-09-30 02:15:37,109][1157819] Updated weights for policy 0, policy_version 114408 (0.0006) [2024-09-30 02:15:37,607][1157819] Updated weights for policy 0, policy_version 114418 (0.0006) [2024-09-30 02:15:38,115][1157819] Updated weights for policy 0, policy_version 114428 (0.0006) [2024-09-30 02:15:38,617][1157819] Updated weights for policy 0, policy_version 114438 (0.0006) [2024-09-30 02:15:38,938][1157736] Signal inference workers to stop experience collection... (7650 times) [2024-09-30 02:15:38,938][1157736] Signal inference workers to resume experience collection... (7650 times) [2024-09-30 02:15:38,942][1157819] InferenceWorker_p0-w0: stopping experience collection (7650 times) [2024-09-30 02:15:38,942][1157819] InferenceWorker_p0-w0: resuming experience collection (7650 times) [2024-09-30 02:15:39,109][1157819] Updated weights for policy 0, policy_version 114448 (0.0006) [2024-09-30 02:15:39,611][1157819] Updated weights for policy 0, policy_version 114458 (0.0006) [2024-09-30 02:15:40,133][1157819] Updated weights for policy 0, policy_version 114468 (0.0006) [2024-09-30 02:15:40,466][1157520] Fps is (10 sec: 79872.1, 60 sec: 78506.9, 300 sec: 75963.5). Total num frames: 468885504. Throughput: 0: 19735.4. Samples: 107167932. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:15:40,466][1157520] Avg episode reward: [(0, '57.548')] [2024-09-30 02:15:40,628][1157819] Updated weights for policy 0, policy_version 114478 (0.0006) [2024-09-30 02:15:41,123][1157819] Updated weights for policy 0, policy_version 114488 (0.0006) [2024-09-30 02:15:41,593][1157819] Updated weights for policy 0, policy_version 114498 (0.0006) [2024-09-30 02:15:42,099][1157819] Updated weights for policy 0, policy_version 114508 (0.0006) [2024-09-30 02:15:42,591][1157819] Updated weights for policy 0, policy_version 114518 (0.0006) [2024-09-30 02:15:43,074][1157819] Updated weights for policy 0, policy_version 114528 (0.0006) [2024-09-30 02:15:43,576][1157819] Updated weights for policy 0, policy_version 114538 (0.0006) [2024-09-30 02:15:44,043][1157819] Updated weights for policy 0, policy_version 114548 (0.0006) [2024-09-30 02:15:44,549][1157819] Updated weights for policy 0, policy_version 114558 (0.0006) [2024-09-30 02:15:45,046][1157819] Updated weights for policy 0, policy_version 114568 (0.0006) [2024-09-30 02:15:45,466][1157520] Fps is (10 sec: 81919.9, 60 sec: 79257.9, 300 sec: 76046.8). Total num frames: 469303296. Throughput: 0: 19869.9. Samples: 107291600. Policy #0 lag: (min: 0.0, avg: 1.7, max: 5.0) [2024-09-30 02:15:45,466][1157520] Avg episode reward: [(0, '55.492')] [2024-09-30 02:15:45,527][1157819] Updated weights for policy 0, policy_version 114578 (0.0006) [2024-09-30 02:15:46,032][1157819] Updated weights for policy 0, policy_version 114588 (0.0006) [2024-09-30 02:15:46,519][1157819] Updated weights for policy 0, policy_version 114598 (0.0006) [2024-09-30 02:15:47,010][1157819] Updated weights for policy 0, policy_version 114608 (0.0006) [2024-09-30 02:15:47,500][1157819] Updated weights for policy 0, policy_version 114618 (0.0006) [2024-09-30 02:15:48,003][1157819] Updated weights for policy 0, policy_version 114628 (0.0006) [2024-09-30 02:15:48,494][1157819] Updated weights for policy 0, policy_version 114638 (0.0006) [2024-09-30 02:15:48,974][1157819] Updated weights for policy 0, policy_version 114648 (0.0006) [2024-09-30 02:15:49,473][1157819] Updated weights for policy 0, policy_version 114658 (0.0006) [2024-09-30 02:15:49,957][1157819] Updated weights for policy 0, policy_version 114668 (0.0006) [2024-09-30 02:15:50,461][1157819] Updated weights for policy 0, policy_version 114678 (0.0006) [2024-09-30 02:15:50,466][1157520] Fps is (10 sec: 83558.4, 60 sec: 79735.8, 300 sec: 76102.3). Total num frames: 469721088. Throughput: 0: 20034.5. Samples: 107416508. Policy #0 lag: (min: 0.0, avg: 1.7, max: 5.0) [2024-09-30 02:15:50,466][1157520] Avg episode reward: [(0, '53.235')] [2024-09-30 02:15:50,974][1157819] Updated weights for policy 0, policy_version 114688 (0.0006) [2024-09-30 02:15:51,484][1157819] Updated weights for policy 0, policy_version 114698 (0.0006) [2024-09-30 02:15:51,976][1157819] Updated weights for policy 0, policy_version 114708 (0.0006) [2024-09-30 02:15:52,507][1157819] Updated weights for policy 0, policy_version 114718 (0.0006) [2024-09-30 02:15:53,018][1157819] Updated weights for policy 0, policy_version 114728 (0.0006) [2024-09-30 02:15:53,534][1157819] Updated weights for policy 0, policy_version 114738 (0.0006) [2024-09-30 02:15:54,034][1157819] Updated weights for policy 0, policy_version 114748 (0.0006) [2024-09-30 02:15:54,559][1157819] Updated weights for policy 0, policy_version 114758 (0.0006) [2024-09-30 02:15:55,038][1157819] Updated weights for policy 0, policy_version 114768 (0.0006) [2024-09-30 02:15:55,466][1157520] Fps is (10 sec: 81920.2, 60 sec: 79940.6, 300 sec: 76060.7). Total num frames: 470122496. Throughput: 0: 20074.2. Samples: 107476868. Policy #0 lag: (min: 0.0, avg: 1.7, max: 5.0) [2024-09-30 02:15:55,466][1157520] Avg episode reward: [(0, '55.509')] [2024-09-30 02:15:55,561][1157819] Updated weights for policy 0, policy_version 114778 (0.0006) [2024-09-30 02:15:56,082][1157819] Updated weights for policy 0, policy_version 114788 (0.0006) [2024-09-30 02:15:56,583][1157819] Updated weights for policy 0, policy_version 114798 (0.0006) [2024-09-30 02:15:57,069][1157819] Updated weights for policy 0, policy_version 114808 (0.0006) [2024-09-30 02:15:57,575][1157819] Updated weights for policy 0, policy_version 114818 (0.0006) [2024-09-30 02:15:58,073][1157819] Updated weights for policy 0, policy_version 114828 (0.0006) [2024-09-30 02:15:58,552][1157819] Updated weights for policy 0, policy_version 114838 (0.0006) [2024-09-30 02:15:59,065][1157819] Updated weights for policy 0, policy_version 114848 (0.0006) [2024-09-30 02:15:59,555][1157819] Updated weights for policy 0, policy_version 114858 (0.0006) [2024-09-30 02:16:00,055][1157819] Updated weights for policy 0, policy_version 114868 (0.0006) [2024-09-30 02:16:00,466][1157520] Fps is (10 sec: 81100.8, 60 sec: 80213.5, 300 sec: 76185.6). Total num frames: 470532096. Throughput: 0: 20156.2. Samples: 107598944. Policy #0 lag: (min: 0.0, avg: 1.7, max: 5.0) [2024-09-30 02:16:00,466][1157520] Avg episode reward: [(0, '54.104')] [2024-09-30 02:16:00,558][1157819] Updated weights for policy 0, policy_version 114878 (0.0006) [2024-09-30 02:16:01,054][1157819] Updated weights for policy 0, policy_version 114888 (0.0006) [2024-09-30 02:16:01,538][1157819] Updated weights for policy 0, policy_version 114898 (0.0006) [2024-09-30 02:16:02,073][1157819] Updated weights for policy 0, policy_version 114908 (0.0006) [2024-09-30 02:16:02,521][1157819] Updated weights for policy 0, policy_version 114918 (0.0006) [2024-09-30 02:16:02,997][1157819] Updated weights for policy 0, policy_version 114928 (0.0006) [2024-09-30 02:16:03,524][1157819] Updated weights for policy 0, policy_version 114938 (0.0006) [2024-09-30 02:16:03,993][1157819] Updated weights for policy 0, policy_version 114948 (0.0006) [2024-09-30 02:16:04,484][1157819] Updated weights for policy 0, policy_version 114958 (0.0006) [2024-09-30 02:16:05,029][1157819] Updated weights for policy 0, policy_version 114968 (0.0006) [2024-09-30 02:16:05,466][1157520] Fps is (10 sec: 82329.0, 60 sec: 80554.7, 300 sec: 76255.0). Total num frames: 470945792. Throughput: 0: 20340.1. Samples: 107722880. Policy #0 lag: (min: 0.0, avg: 1.7, max: 5.0) [2024-09-30 02:16:05,466][1157520] Avg episode reward: [(0, '56.365')] [2024-09-30 02:16:05,474][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000114978_470949888.pth... [2024-09-30 02:16:05,476][1157819] Updated weights for policy 0, policy_version 114978 (0.0006) [2024-09-30 02:16:05,517][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000110490_452567040.pth [2024-09-30 02:16:05,965][1157819] Updated weights for policy 0, policy_version 114988 (0.0006) [2024-09-30 02:16:06,495][1157819] Updated weights for policy 0, policy_version 114998 (0.0006) [2024-09-30 02:16:06,982][1157819] Updated weights for policy 0, policy_version 115008 (0.0006) [2024-09-30 02:16:07,470][1157819] Updated weights for policy 0, policy_version 115018 (0.0006) [2024-09-30 02:16:07,968][1157819] Updated weights for policy 0, policy_version 115028 (0.0006) [2024-09-30 02:16:08,447][1157819] Updated weights for policy 0, policy_version 115038 (0.0006) [2024-09-30 02:16:08,930][1157819] Updated weights for policy 0, policy_version 115048 (0.0006) [2024-09-30 02:16:09,433][1157819] Updated weights for policy 0, policy_version 115058 (0.0006) [2024-09-30 02:16:09,902][1157819] Updated weights for policy 0, policy_version 115068 (0.0006) [2024-09-30 02:16:10,382][1157819] Updated weights for policy 0, policy_version 115078 (0.0006) [2024-09-30 02:16:10,466][1157520] Fps is (10 sec: 83148.9, 60 sec: 80964.4, 300 sec: 76324.5). Total num frames: 471363584. Throughput: 0: 20383.6. Samples: 107784464. Policy #0 lag: (min: 0.0, avg: 1.7, max: 5.0) [2024-09-30 02:16:10,466][1157520] Avg episode reward: [(0, '55.467')] [2024-09-30 02:16:10,906][1157819] Updated weights for policy 0, policy_version 115088 (0.0006) [2024-09-30 02:16:11,357][1157819] Updated weights for policy 0, policy_version 115098 (0.0006) [2024-09-30 02:16:11,831][1157819] Updated weights for policy 0, policy_version 115108 (0.0006) [2024-09-30 02:16:12,340][1157819] Updated weights for policy 0, policy_version 115118 (0.0006) [2024-09-30 02:16:12,811][1157819] Updated weights for policy 0, policy_version 115128 (0.0006) [2024-09-30 02:16:13,272][1157819] Updated weights for policy 0, policy_version 115138 (0.0006) [2024-09-30 02:16:13,746][1157819] Updated weights for policy 0, policy_version 115148 (0.0006) [2024-09-30 02:16:14,209][1157819] Updated weights for policy 0, policy_version 115158 (0.0006) [2024-09-30 02:16:14,737][1157819] Updated weights for policy 0, policy_version 115168 (0.0006) [2024-09-30 02:16:15,215][1157819] Updated weights for policy 0, policy_version 115178 (0.0006) [2024-09-30 02:16:15,466][1157520] Fps is (10 sec: 84378.2, 60 sec: 81646.9, 300 sec: 76380.0). Total num frames: 471789568. Throughput: 0: 20532.6. Samples: 107912268. Policy #0 lag: (min: 0.0, avg: 1.7, max: 5.0) [2024-09-30 02:16:15,466][1157520] Avg episode reward: [(0, '55.537')] [2024-09-30 02:16:15,678][1157819] Updated weights for policy 0, policy_version 115188 (0.0006) [2024-09-30 02:16:16,206][1157819] Updated weights for policy 0, policy_version 115198 (0.0006) [2024-09-30 02:16:16,670][1157819] Updated weights for policy 0, policy_version 115208 (0.0006) [2024-09-30 02:16:17,163][1157819] Updated weights for policy 0, policy_version 115218 (0.0006) [2024-09-30 02:16:17,666][1157819] Updated weights for policy 0, policy_version 115228 (0.0006) [2024-09-30 02:16:18,142][1157819] Updated weights for policy 0, policy_version 115238 (0.0006) [2024-09-30 02:16:18,645][1157819] Updated weights for policy 0, policy_version 115248 (0.0006) [2024-09-30 02:16:19,130][1157819] Updated weights for policy 0, policy_version 115258 (0.0006) [2024-09-30 02:16:19,620][1157819] Updated weights for policy 0, policy_version 115268 (0.0006) [2024-09-30 02:16:20,126][1157819] Updated weights for policy 0, policy_version 115278 (0.0006) [2024-09-30 02:16:20,466][1157520] Fps is (10 sec: 84377.4, 60 sec: 82124.8, 300 sec: 76463.3). Total num frames: 472207360. Throughput: 0: 20656.4. Samples: 108037680. Policy #0 lag: (min: 0.0, avg: 1.7, max: 5.0) [2024-09-30 02:16:20,466][1157520] Avg episode reward: [(0, '54.285')] [2024-09-30 02:16:20,609][1157819] Updated weights for policy 0, policy_version 115288 (0.0006) [2024-09-30 02:16:21,086][1157819] Updated weights for policy 0, policy_version 115298 (0.0006) [2024-09-30 02:16:21,605][1157819] Updated weights for policy 0, policy_version 115308 (0.0006) [2024-09-30 02:16:22,071][1157819] Updated weights for policy 0, policy_version 115318 (0.0006) [2024-09-30 02:16:22,552][1157819] Updated weights for policy 0, policy_version 115328 (0.0006) [2024-09-30 02:16:23,059][1157819] Updated weights for policy 0, policy_version 115338 (0.0006) [2024-09-30 02:16:23,546][1157819] Updated weights for policy 0, policy_version 115348 (0.0006) [2024-09-30 02:16:24,034][1157819] Updated weights for policy 0, policy_version 115358 (0.0006) [2024-09-30 02:16:24,539][1157819] Updated weights for policy 0, policy_version 115368 (0.0006) [2024-09-30 02:16:25,019][1157819] Updated weights for policy 0, policy_version 115378 (0.0006) [2024-09-30 02:16:25,466][1157520] Fps is (10 sec: 83148.7, 60 sec: 82397.8, 300 sec: 76671.6). Total num frames: 472621056. Throughput: 0: 20721.9. Samples: 108100416. Policy #0 lag: (min: 0.0, avg: 1.7, max: 5.0) [2024-09-30 02:16:25,466][1157520] Avg episode reward: [(0, '55.699')] [2024-09-30 02:16:25,513][1157819] Updated weights for policy 0, policy_version 115388 (0.0006) [2024-09-30 02:16:26,011][1157819] Updated weights for policy 0, policy_version 115398 (0.0006) [2024-09-30 02:16:26,493][1157819] Updated weights for policy 0, policy_version 115408 (0.0006) [2024-09-30 02:16:27,013][1157819] Updated weights for policy 0, policy_version 115418 (0.0006) [2024-09-30 02:16:27,537][1157819] Updated weights for policy 0, policy_version 115428 (0.0006) [2024-09-30 02:16:28,022][1157819] Updated weights for policy 0, policy_version 115438 (0.0006) [2024-09-30 02:16:28,547][1157819] Updated weights for policy 0, policy_version 115448 (0.0006) [2024-09-30 02:16:29,037][1157819] Updated weights for policy 0, policy_version 115458 (0.0006) [2024-09-30 02:16:29,540][1157819] Updated weights for policy 0, policy_version 115468 (0.0006) [2024-09-30 02:16:30,032][1157819] Updated weights for policy 0, policy_version 115478 (0.0006) [2024-09-30 02:16:30,466][1157520] Fps is (10 sec: 82329.9, 60 sec: 82397.9, 300 sec: 76810.5). Total num frames: 473030656. Throughput: 0: 20712.4. Samples: 108223656. Policy #0 lag: (min: 0.0, avg: 1.7, max: 5.0) [2024-09-30 02:16:30,466][1157520] Avg episode reward: [(0, '55.097')] [2024-09-30 02:16:30,527][1157819] Updated weights for policy 0, policy_version 115488 (0.0006) [2024-09-30 02:16:31,018][1157819] Updated weights for policy 0, policy_version 115498 (0.0006) [2024-09-30 02:16:31,498][1157819] Updated weights for policy 0, policy_version 115508 (0.0006) [2024-09-30 02:16:31,984][1157736] Signal inference workers to stop experience collection... (7700 times) [2024-09-30 02:16:31,984][1157736] Signal inference workers to resume experience collection... (7700 times) [2024-09-30 02:16:31,990][1157819] InferenceWorker_p0-w0: stopping experience collection (7700 times) [2024-09-30 02:16:31,990][1157819] InferenceWorker_p0-w0: resuming experience collection (7700 times) [2024-09-30 02:16:31,998][1157819] Updated weights for policy 0, policy_version 115518 (0.0006) [2024-09-30 02:16:32,512][1157819] Updated weights for policy 0, policy_version 115528 (0.0006) [2024-09-30 02:16:32,982][1157819] Updated weights for policy 0, policy_version 115538 (0.0006) [2024-09-30 02:16:33,480][1157819] Updated weights for policy 0, policy_version 115548 (0.0006) [2024-09-30 02:16:33,982][1157819] Updated weights for policy 0, policy_version 115558 (0.0006) [2024-09-30 02:16:34,468][1157819] Updated weights for policy 0, policy_version 115568 (0.0006) [2024-09-30 02:16:34,957][1157819] Updated weights for policy 0, policy_version 115578 (0.0006) [2024-09-30 02:16:35,455][1157819] Updated weights for policy 0, policy_version 115588 (0.0006) [2024-09-30 02:16:35,466][1157520] Fps is (10 sec: 82739.2, 60 sec: 82739.2, 300 sec: 76977.1). Total num frames: 473448448. Throughput: 0: 20704.3. Samples: 108348204. Policy #0 lag: (min: 0.0, avg: 1.7, max: 5.0) [2024-09-30 02:16:35,466][1157520] Avg episode reward: [(0, '55.445')] [2024-09-30 02:16:35,958][1157819] Updated weights for policy 0, policy_version 115598 (0.0006) [2024-09-30 02:16:36,450][1157819] Updated weights for policy 0, policy_version 115608 (0.0006) [2024-09-30 02:16:36,938][1157819] Updated weights for policy 0, policy_version 115618 (0.0006) [2024-09-30 02:16:37,437][1157819] Updated weights for policy 0, policy_version 115628 (0.0006) [2024-09-30 02:16:37,932][1157819] Updated weights for policy 0, policy_version 115638 (0.0006) [2024-09-30 02:16:38,421][1157819] Updated weights for policy 0, policy_version 115648 (0.0006) [2024-09-30 02:16:38,915][1157819] Updated weights for policy 0, policy_version 115658 (0.0006) [2024-09-30 02:16:39,409][1157819] Updated weights for policy 0, policy_version 115668 (0.0006) [2024-09-30 02:16:39,900][1157819] Updated weights for policy 0, policy_version 115678 (0.0006) [2024-09-30 02:16:40,402][1157819] Updated weights for policy 0, policy_version 115688 (0.0006) [2024-09-30 02:16:40,466][1157520] Fps is (10 sec: 83148.5, 60 sec: 82944.0, 300 sec: 77143.7). Total num frames: 473862144. Throughput: 0: 20744.4. Samples: 108410364. Policy #0 lag: (min: 0.0, avg: 1.7, max: 5.0) [2024-09-30 02:16:40,466][1157520] Avg episode reward: [(0, '52.180')] [2024-09-30 02:16:40,896][1157819] Updated weights for policy 0, policy_version 115698 (0.0006) [2024-09-30 02:16:41,371][1157819] Updated weights for policy 0, policy_version 115708 (0.0006) [2024-09-30 02:16:41,875][1157819] Updated weights for policy 0, policy_version 115718 (0.0006) [2024-09-30 02:16:42,409][1157819] Updated weights for policy 0, policy_version 115728 (0.0006) [2024-09-30 02:16:42,947][1157819] Updated weights for policy 0, policy_version 115738 (0.0006) [2024-09-30 02:16:43,439][1157819] Updated weights for policy 0, policy_version 115748 (0.0006) [2024-09-30 02:16:43,951][1157819] Updated weights for policy 0, policy_version 115758 (0.0006) [2024-09-30 02:16:44,483][1157819] Updated weights for policy 0, policy_version 115768 (0.0006) [2024-09-30 02:16:45,040][1157819] Updated weights for policy 0, policy_version 115778 (0.0006) [2024-09-30 02:16:45,466][1157520] Fps is (10 sec: 81100.6, 60 sec: 82602.6, 300 sec: 77240.8). Total num frames: 474259456. Throughput: 0: 20748.9. Samples: 108532644. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 02:16:45,466][1157520] Avg episode reward: [(0, '55.540')] [2024-09-30 02:16:45,527][1157819] Updated weights for policy 0, policy_version 115788 (0.0006) [2024-09-30 02:16:46,026][1157819] Updated weights for policy 0, policy_version 115798 (0.0006) [2024-09-30 02:16:46,518][1157819] Updated weights for policy 0, policy_version 115808 (0.0006) [2024-09-30 02:16:47,017][1157819] Updated weights for policy 0, policy_version 115818 (0.0006) [2024-09-30 02:16:47,509][1157819] Updated weights for policy 0, policy_version 115828 (0.0006) [2024-09-30 02:16:48,003][1157819] Updated weights for policy 0, policy_version 115838 (0.0006) [2024-09-30 02:16:48,505][1157819] Updated weights for policy 0, policy_version 115848 (0.0006) [2024-09-30 02:16:48,988][1157819] Updated weights for policy 0, policy_version 115858 (0.0006) [2024-09-30 02:16:49,484][1157819] Updated weights for policy 0, policy_version 115868 (0.0006) [2024-09-30 02:16:49,980][1157819] Updated weights for policy 0, policy_version 115878 (0.0006) [2024-09-30 02:16:50,466][1157520] Fps is (10 sec: 81510.4, 60 sec: 82602.6, 300 sec: 77449.1). Total num frames: 474677248. Throughput: 0: 20712.2. Samples: 108654928. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 02:16:50,466][1157819] Updated weights for policy 0, policy_version 115888 (0.0006) [2024-09-30 02:16:50,466][1157520] Avg episode reward: [(0, '54.916')] [2024-09-30 02:16:50,966][1157819] Updated weights for policy 0, policy_version 115898 (0.0006) [2024-09-30 02:16:51,463][1157819] Updated weights for policy 0, policy_version 115908 (0.0006) [2024-09-30 02:16:51,949][1157819] Updated weights for policy 0, policy_version 115918 (0.0006) [2024-09-30 02:16:52,443][1157819] Updated weights for policy 0, policy_version 115928 (0.0006) [2024-09-30 02:16:52,933][1157819] Updated weights for policy 0, policy_version 115938 (0.0006) [2024-09-30 02:16:53,412][1157819] Updated weights for policy 0, policy_version 115948 (0.0006) [2024-09-30 02:16:53,919][1157819] Updated weights for policy 0, policy_version 115958 (0.0006) [2024-09-30 02:16:54,426][1157819] Updated weights for policy 0, policy_version 115968 (0.0006) [2024-09-30 02:16:54,958][1157819] Updated weights for policy 0, policy_version 115978 (0.0006) [2024-09-30 02:16:55,459][1157819] Updated weights for policy 0, policy_version 115988 (0.0006) [2024-09-30 02:16:55,466][1157520] Fps is (10 sec: 82738.9, 60 sec: 82739.1, 300 sec: 77657.4). Total num frames: 475086848. Throughput: 0: 20731.2. Samples: 108717372. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 02:16:55,466][1157520] Avg episode reward: [(0, '53.611')] [2024-09-30 02:16:55,965][1157819] Updated weights for policy 0, policy_version 115998 (0.0006) [2024-09-30 02:16:56,482][1157819] Updated weights for policy 0, policy_version 116008 (0.0006) [2024-09-30 02:16:56,938][1157819] Updated weights for policy 0, policy_version 116018 (0.0006) [2024-09-30 02:16:57,426][1157819] Updated weights for policy 0, policy_version 116028 (0.0006) [2024-09-30 02:16:57,893][1157819] Updated weights for policy 0, policy_version 116038 (0.0006) [2024-09-30 02:16:58,391][1157819] Updated weights for policy 0, policy_version 116048 (0.0006) [2024-09-30 02:16:58,888][1157819] Updated weights for policy 0, policy_version 116058 (0.0006) [2024-09-30 02:16:59,369][1157819] Updated weights for policy 0, policy_version 116068 (0.0006) [2024-09-30 02:16:59,840][1157819] Updated weights for policy 0, policy_version 116078 (0.0006) [2024-09-30 02:17:00,333][1157819] Updated weights for policy 0, policy_version 116088 (0.0006) [2024-09-30 02:17:00,466][1157520] Fps is (10 sec: 82738.2, 60 sec: 82875.5, 300 sec: 77754.6). Total num frames: 475504640. Throughput: 0: 20648.1. Samples: 108841436. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 02:17:00,466][1157520] Avg episode reward: [(0, '51.312')] [2024-09-30 02:17:00,812][1157819] Updated weights for policy 0, policy_version 116098 (0.0006) [2024-09-30 02:17:01,305][1157819] Updated weights for policy 0, policy_version 116108 (0.0006) [2024-09-30 02:17:01,757][1157819] Updated weights for policy 0, policy_version 116118 (0.0006) [2024-09-30 02:17:02,228][1157819] Updated weights for policy 0, policy_version 116128 (0.0006) [2024-09-30 02:17:02,660][1157819] Updated weights for policy 0, policy_version 116138 (0.0006) [2024-09-30 02:17:03,132][1157819] Updated weights for policy 0, policy_version 116148 (0.0006) [2024-09-30 02:17:03,609][1157819] Updated weights for policy 0, policy_version 116158 (0.0006) [2024-09-30 02:17:04,080][1157819] Updated weights for policy 0, policy_version 116168 (0.0006) [2024-09-30 02:17:04,511][1157819] Updated weights for policy 0, policy_version 116178 (0.0006) [2024-09-30 02:17:04,984][1157819] Updated weights for policy 0, policy_version 116188 (0.0006) [2024-09-30 02:17:05,466][1157520] Fps is (10 sec: 85605.3, 60 sec: 83285.1, 300 sec: 77907.2). Total num frames: 475942912. Throughput: 0: 20764.0. Samples: 108972064. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 02:17:05,466][1157520] Avg episode reward: [(0, '55.210')] [2024-09-30 02:17:05,475][1157819] Updated weights for policy 0, policy_version 116198 (0.0006) [2024-09-30 02:17:05,926][1157819] Updated weights for policy 0, policy_version 116208 (0.0006) [2024-09-30 02:17:06,356][1157819] Updated weights for policy 0, policy_version 116218 (0.0006) [2024-09-30 02:17:06,835][1157819] Updated weights for policy 0, policy_version 116228 (0.0006) [2024-09-30 02:17:07,289][1157819] Updated weights for policy 0, policy_version 116238 (0.0006) [2024-09-30 02:17:07,714][1157819] Updated weights for policy 0, policy_version 116248 (0.0006) [2024-09-30 02:17:08,222][1157819] Updated weights for policy 0, policy_version 116258 (0.0006) [2024-09-30 02:17:08,715][1157819] Updated weights for policy 0, policy_version 116268 (0.0006) [2024-09-30 02:17:09,223][1157819] Updated weights for policy 0, policy_version 116278 (0.0006) [2024-09-30 02:17:09,721][1157819] Updated weights for policy 0, policy_version 116288 (0.0006) [2024-09-30 02:17:10,239][1157819] Updated weights for policy 0, policy_version 116298 (0.0006) [2024-09-30 02:17:10,466][1157520] Fps is (10 sec: 86836.6, 60 sec: 83490.1, 300 sec: 77949.0). Total num frames: 476372992. Throughput: 0: 20860.6. Samples: 109039140. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 02:17:10,466][1157520] Avg episode reward: [(0, '57.170')] [2024-09-30 02:17:10,776][1157819] Updated weights for policy 0, policy_version 116308 (0.0006) [2024-09-30 02:17:10,928][1157736] Signal inference workers to stop experience collection... (7750 times) [2024-09-30 02:17:10,928][1157736] Signal inference workers to resume experience collection... (7750 times) [2024-09-30 02:17:10,931][1157819] InferenceWorker_p0-w0: stopping experience collection (7750 times) [2024-09-30 02:17:10,932][1157819] InferenceWorker_p0-w0: resuming experience collection (7750 times) [2024-09-30 02:17:11,266][1157819] Updated weights for policy 0, policy_version 116318 (0.0006) [2024-09-30 02:17:11,779][1157819] Updated weights for policy 0, policy_version 116328 (0.0006) [2024-09-30 02:17:12,289][1157819] Updated weights for policy 0, policy_version 116338 (0.0006) [2024-09-30 02:17:12,824][1157819] Updated weights for policy 0, policy_version 116348 (0.0006) [2024-09-30 02:17:13,328][1157819] Updated weights for policy 0, policy_version 116358 (0.0006) [2024-09-30 02:17:13,830][1157819] Updated weights for policy 0, policy_version 116368 (0.0006) [2024-09-30 02:17:14,326][1157819] Updated weights for policy 0, policy_version 116378 (0.0006) [2024-09-30 02:17:14,822][1157819] Updated weights for policy 0, policy_version 116388 (0.0006) [2024-09-30 02:17:15,315][1157819] Updated weights for policy 0, policy_version 116398 (0.0006) [2024-09-30 02:17:15,466][1157520] Fps is (10 sec: 83150.5, 60 sec: 83080.5, 300 sec: 77921.2). Total num frames: 476774400. Throughput: 0: 20811.1. Samples: 109160156. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 02:17:15,466][1157520] Avg episode reward: [(0, '53.388')] [2024-09-30 02:17:15,867][1157819] Updated weights for policy 0, policy_version 116408 (0.0006) [2024-09-30 02:17:16,395][1157819] Updated weights for policy 0, policy_version 116418 (0.0006) [2024-09-30 02:17:16,891][1157819] Updated weights for policy 0, policy_version 116428 (0.0007) [2024-09-30 02:17:17,391][1157819] Updated weights for policy 0, policy_version 116438 (0.0006) [2024-09-30 02:17:17,990][1157819] Updated weights for policy 0, policy_version 116448 (0.0006) [2024-09-30 02:17:18,490][1157819] Updated weights for policy 0, policy_version 116458 (0.0006) [2024-09-30 02:17:18,990][1157819] Updated weights for policy 0, policy_version 116468 (0.0006) [2024-09-30 02:17:19,512][1157819] Updated weights for policy 0, policy_version 116478 (0.0006) [2024-09-30 02:17:20,009][1157819] Updated weights for policy 0, policy_version 116488 (0.0006) [2024-09-30 02:17:20,466][1157520] Fps is (10 sec: 79872.1, 60 sec: 82739.3, 300 sec: 77921.2). Total num frames: 477171712. Throughput: 0: 20684.5. Samples: 109279004. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 02:17:20,466][1157520] Avg episode reward: [(0, '56.295')] [2024-09-30 02:17:20,494][1157819] Updated weights for policy 0, policy_version 116498 (0.0006) [2024-09-30 02:17:20,982][1157819] Updated weights for policy 0, policy_version 116508 (0.0006) [2024-09-30 02:17:21,475][1157819] Updated weights for policy 0, policy_version 116518 (0.0006) [2024-09-30 02:17:21,952][1157819] Updated weights for policy 0, policy_version 116528 (0.0006) [2024-09-30 02:17:22,411][1157819] Updated weights for policy 0, policy_version 116538 (0.0006) [2024-09-30 02:17:22,892][1157819] Updated weights for policy 0, policy_version 116548 (0.0006) [2024-09-30 02:17:23,352][1157819] Updated weights for policy 0, policy_version 116558 (0.0006) [2024-09-30 02:17:23,803][1157819] Updated weights for policy 0, policy_version 116568 (0.0006) [2024-09-30 02:17:24,272][1157819] Updated weights for policy 0, policy_version 116578 (0.0006) [2024-09-30 02:17:24,738][1157819] Updated weights for policy 0, policy_version 116588 (0.0006) [2024-09-30 02:17:25,218][1157819] Updated weights for policy 0, policy_version 116598 (0.0006) [2024-09-30 02:17:25,466][1157520] Fps is (10 sec: 83149.1, 60 sec: 83080.6, 300 sec: 78129.5). Total num frames: 477605888. Throughput: 0: 20726.2. Samples: 109343044. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 02:17:25,466][1157520] Avg episode reward: [(0, '54.135')] [2024-09-30 02:17:25,674][1157819] Updated weights for policy 0, policy_version 116608 (0.0006) [2024-09-30 02:17:26,120][1157819] Updated weights for policy 0, policy_version 116618 (0.0006) [2024-09-30 02:17:26,594][1157819] Updated weights for policy 0, policy_version 116628 (0.0006) [2024-09-30 02:17:27,082][1157819] Updated weights for policy 0, policy_version 116638 (0.0006) [2024-09-30 02:17:27,517][1157819] Updated weights for policy 0, policy_version 116648 (0.0006) [2024-09-30 02:17:27,973][1157819] Updated weights for policy 0, policy_version 116658 (0.0006) [2024-09-30 02:17:28,446][1157819] Updated weights for policy 0, policy_version 116668 (0.0006) [2024-09-30 02:17:28,920][1157819] Updated weights for policy 0, policy_version 116678 (0.0006) [2024-09-30 02:17:29,390][1157819] Updated weights for policy 0, policy_version 116688 (0.0006) [2024-09-30 02:17:29,845][1157819] Updated weights for policy 0, policy_version 116698 (0.0006) [2024-09-30 02:17:30,280][1157819] Updated weights for policy 0, policy_version 116708 (0.0006) [2024-09-30 02:17:30,466][1157520] Fps is (10 sec: 87654.1, 60 sec: 83626.6, 300 sec: 78365.5). Total num frames: 478048256. Throughput: 0: 20951.8. Samples: 109475472. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 02:17:30,466][1157520] Avg episode reward: [(0, '57.094')] [2024-09-30 02:17:30,767][1157819] Updated weights for policy 0, policy_version 116718 (0.0006) [2024-09-30 02:17:31,259][1157819] Updated weights for policy 0, policy_version 116728 (0.0006) [2024-09-30 02:17:31,751][1157819] Updated weights for policy 0, policy_version 116738 (0.0006) [2024-09-30 02:17:32,224][1157736] Signal inference workers to stop experience collection... (7800 times) [2024-09-30 02:17:32,228][1157819] InferenceWorker_p0-w0: stopping experience collection (7800 times) [2024-09-30 02:17:32,233][1157736] Signal inference workers to resume experience collection... (7800 times) [2024-09-30 02:17:32,233][1157819] InferenceWorker_p0-w0: resuming experience collection (7800 times) [2024-09-30 02:17:32,247][1157819] Updated weights for policy 0, policy_version 116748 (0.0006) [2024-09-30 02:17:32,806][1157819] Updated weights for policy 0, policy_version 116758 (0.0006) [2024-09-30 02:17:33,361][1157819] Updated weights for policy 0, policy_version 116768 (0.0006) [2024-09-30 02:17:33,865][1157819] Updated weights for policy 0, policy_version 116778 (0.0006) [2024-09-30 02:17:34,368][1157819] Updated weights for policy 0, policy_version 116788 (0.0006) [2024-09-30 02:17:34,915][1157819] Updated weights for policy 0, policy_version 116798 (0.0006) [2024-09-30 02:17:35,451][1157819] Updated weights for policy 0, policy_version 116808 (0.0006) [2024-09-30 02:17:35,466][1157520] Fps is (10 sec: 83967.3, 60 sec: 83285.3, 300 sec: 78448.8). Total num frames: 478445568. Throughput: 0: 20969.5. Samples: 109598556. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 02:17:35,466][1157520] Avg episode reward: [(0, '56.142')] [2024-09-30 02:17:36,002][1157819] Updated weights for policy 0, policy_version 116818 (0.0006) [2024-09-30 02:17:36,571][1157819] Updated weights for policy 0, policy_version 116828 (0.0006) [2024-09-30 02:17:37,079][1157819] Updated weights for policy 0, policy_version 116838 (0.0006) [2024-09-30 02:17:37,600][1157819] Updated weights for policy 0, policy_version 116848 (0.0006) [2024-09-30 02:17:38,126][1157819] Updated weights for policy 0, policy_version 116858 (0.0006) [2024-09-30 02:17:38,660][1157819] Updated weights for policy 0, policy_version 116868 (0.0006) [2024-09-30 02:17:39,223][1157819] Updated weights for policy 0, policy_version 116878 (0.0006) [2024-09-30 02:17:39,748][1157819] Updated weights for policy 0, policy_version 116888 (0.0006) [2024-09-30 02:17:40,284][1157819] Updated weights for policy 0, policy_version 116898 (0.0006) [2024-09-30 02:17:40,466][1157520] Fps is (10 sec: 77823.7, 60 sec: 82739.2, 300 sec: 78504.4). Total num frames: 478826496. Throughput: 0: 20853.5. Samples: 109655780. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 02:17:40,466][1157520] Avg episode reward: [(0, '56.936')] [2024-09-30 02:17:40,851][1157819] Updated weights for policy 0, policy_version 116908 (0.0006) [2024-09-30 02:17:41,354][1157819] Updated weights for policy 0, policy_version 116918 (0.0006) [2024-09-30 02:17:41,898][1157819] Updated weights for policy 0, policy_version 116928 (0.0006) [2024-09-30 02:17:42,415][1157819] Updated weights for policy 0, policy_version 116938 (0.0006) [2024-09-30 02:17:42,916][1157819] Updated weights for policy 0, policy_version 116948 (0.0006) [2024-09-30 02:17:43,428][1157819] Updated weights for policy 0, policy_version 116958 (0.0006) [2024-09-30 02:17:43,944][1157819] Updated weights for policy 0, policy_version 116968 (0.0006) [2024-09-30 02:17:44,427][1157819] Updated weights for policy 0, policy_version 116978 (0.0006) [2024-09-30 02:17:44,924][1157819] Updated weights for policy 0, policy_version 116988 (0.0006) [2024-09-30 02:17:45,416][1157819] Updated weights for policy 0, policy_version 116998 (0.0006) [2024-09-30 02:17:45,466][1157520] Fps is (10 sec: 78233.8, 60 sec: 82807.5, 300 sec: 78629.3). Total num frames: 479227904. Throughput: 0: 20684.9. Samples: 109772256. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 02:17:45,466][1157520] Avg episode reward: [(0, '56.562')] [2024-09-30 02:17:45,879][1157819] Updated weights for policy 0, policy_version 117008 (0.0006) [2024-09-30 02:17:46,362][1157819] Updated weights for policy 0, policy_version 117018 (0.0006) [2024-09-30 02:17:46,853][1157819] Updated weights for policy 0, policy_version 117028 (0.0006) [2024-09-30 02:17:47,336][1157819] Updated weights for policy 0, policy_version 117038 (0.0006) [2024-09-30 02:17:47,829][1157819] Updated weights for policy 0, policy_version 117048 (0.0006) [2024-09-30 02:17:48,283][1157819] Updated weights for policy 0, policy_version 117058 (0.0006) [2024-09-30 02:17:48,777][1157819] Updated weights for policy 0, policy_version 117068 (0.0006) [2024-09-30 02:17:49,292][1157819] Updated weights for policy 0, policy_version 117078 (0.0006) [2024-09-30 02:17:49,763][1157819] Updated weights for policy 0, policy_version 117088 (0.0006) [2024-09-30 02:17:50,251][1157819] Updated weights for policy 0, policy_version 117098 (0.0006) [2024-09-30 02:17:50,466][1157520] Fps is (10 sec: 81920.3, 60 sec: 82807.5, 300 sec: 78879.3). Total num frames: 479645696. Throughput: 0: 20585.4. Samples: 109898404. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 02:17:50,466][1157520] Avg episode reward: [(0, '56.398')] [2024-09-30 02:17:50,753][1157819] Updated weights for policy 0, policy_version 117108 (0.0006) [2024-09-30 02:17:51,234][1157819] Updated weights for policy 0, policy_version 117118 (0.0006) [2024-09-30 02:17:51,733][1157819] Updated weights for policy 0, policy_version 117128 (0.0006) [2024-09-30 02:17:52,216][1157819] Updated weights for policy 0, policy_version 117138 (0.0006) [2024-09-30 02:17:52,693][1157819] Updated weights for policy 0, policy_version 117148 (0.0006) [2024-09-30 02:17:53,219][1157819] Updated weights for policy 0, policy_version 117158 (0.0006) [2024-09-30 02:17:53,723][1157819] Updated weights for policy 0, policy_version 117168 (0.0006) [2024-09-30 02:17:54,227][1157819] Updated weights for policy 0, policy_version 117178 (0.0006) [2024-09-30 02:17:54,719][1157819] Updated weights for policy 0, policy_version 117188 (0.0006) [2024-09-30 02:17:55,252][1157819] Updated weights for policy 0, policy_version 117198 (0.0006) [2024-09-30 02:17:55,466][1157520] Fps is (10 sec: 83148.1, 60 sec: 82875.7, 300 sec: 79018.1). Total num frames: 480059392. Throughput: 0: 20492.9. Samples: 109961324. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 02:17:55,466][1157520] Avg episode reward: [(0, '54.239')] [2024-09-30 02:17:55,768][1157819] Updated weights for policy 0, policy_version 117208 (0.0006) [2024-09-30 02:17:56,265][1157819] Updated weights for policy 0, policy_version 117218 (0.0006) [2024-09-30 02:17:56,757][1157819] Updated weights for policy 0, policy_version 117228 (0.0006) [2024-09-30 02:17:57,261][1157819] Updated weights for policy 0, policy_version 117238 (0.0006) [2024-09-30 02:17:57,776][1157819] Updated weights for policy 0, policy_version 117248 (0.0006) [2024-09-30 02:17:58,291][1157819] Updated weights for policy 0, policy_version 117258 (0.0006) [2024-09-30 02:17:58,803][1157819] Updated weights for policy 0, policy_version 117268 (0.0006) [2024-09-30 02:17:59,293][1157819] Updated weights for policy 0, policy_version 117278 (0.0006) [2024-09-30 02:17:59,850][1157819] Updated weights for policy 0, policy_version 117288 (0.0006) [2024-09-30 02:18:00,377][1157819] Updated weights for policy 0, policy_version 117298 (0.0006) [2024-09-30 02:18:00,466][1157520] Fps is (10 sec: 81509.3, 60 sec: 82602.6, 300 sec: 79156.9). Total num frames: 480460800. Throughput: 0: 20486.2. Samples: 110082036. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 02:18:00,466][1157520] Avg episode reward: [(0, '54.668')] [2024-09-30 02:18:00,876][1157819] Updated weights for policy 0, policy_version 117308 (0.0006) [2024-09-30 02:18:01,387][1157819] Updated weights for policy 0, policy_version 117318 (0.0006) [2024-09-30 02:18:01,926][1157819] Updated weights for policy 0, policy_version 117328 (0.0006) [2024-09-30 02:18:02,460][1157819] Updated weights for policy 0, policy_version 117338 (0.0006) [2024-09-30 02:18:02,961][1157819] Updated weights for policy 0, policy_version 117348 (0.0006) [2024-09-30 02:18:03,488][1157819] Updated weights for policy 0, policy_version 117358 (0.0006) [2024-09-30 02:18:04,009][1157819] Updated weights for policy 0, policy_version 117368 (0.0006) [2024-09-30 02:18:04,541][1157819] Updated weights for policy 0, policy_version 117378 (0.0006) [2024-09-30 02:18:05,067][1157819] Updated weights for policy 0, policy_version 117388 (0.0006) [2024-09-30 02:18:05,466][1157520] Fps is (10 sec: 79053.0, 60 sec: 81783.6, 300 sec: 79226.4). Total num frames: 480849920. Throughput: 0: 20464.5. Samples: 110199908. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 02:18:05,466][1157520] Avg episode reward: [(0, '56.483')] [2024-09-30 02:18:05,471][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000117395_480849920.pth... [2024-09-30 02:18:05,501][1157736] Signal inference workers to stop experience collection... (7850 times) [2024-09-30 02:18:05,503][1157819] InferenceWorker_p0-w0: stopping experience collection (7850 times) [2024-09-30 02:18:05,533][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000112671_461500416.pth [2024-09-30 02:18:05,551][1157736] Signal inference workers to resume experience collection... (7850 times) [2024-09-30 02:18:05,551][1157819] InferenceWorker_p0-w0: resuming experience collection (7850 times) [2024-09-30 02:18:05,614][1157819] Updated weights for policy 0, policy_version 117398 (0.0006) [2024-09-30 02:18:06,205][1157819] Updated weights for policy 0, policy_version 117408 (0.0006) [2024-09-30 02:18:06,747][1157819] Updated weights for policy 0, policy_version 117418 (0.0006) [2024-09-30 02:18:07,324][1157819] Updated weights for policy 0, policy_version 117428 (0.0006) [2024-09-30 02:18:07,857][1157819] Updated weights for policy 0, policy_version 117438 (0.0006) [2024-09-30 02:18:08,424][1157819] Updated weights for policy 0, policy_version 117448 (0.0006) [2024-09-30 02:18:08,963][1157819] Updated weights for policy 0, policy_version 117458 (0.0006) [2024-09-30 02:18:09,524][1157819] Updated weights for policy 0, policy_version 117468 (0.0006) [2024-09-30 02:18:10,100][1157819] Updated weights for policy 0, policy_version 117478 (0.0006) [2024-09-30 02:18:10,466][1157520] Fps is (10 sec: 75777.1, 60 sec: 80759.4, 300 sec: 79184.7). Total num frames: 481218560. Throughput: 0: 20267.5. Samples: 110255084. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 02:18:10,466][1157520] Avg episode reward: [(0, '55.994')] [2024-09-30 02:18:10,591][1157819] Updated weights for policy 0, policy_version 117488 (0.0006) [2024-09-30 02:18:11,169][1157819] Updated weights for policy 0, policy_version 117498 (0.0006) [2024-09-30 02:18:11,694][1157819] Updated weights for policy 0, policy_version 117508 (0.0006) [2024-09-30 02:18:12,232][1157819] Updated weights for policy 0, policy_version 117518 (0.0006) [2024-09-30 02:18:12,779][1157819] Updated weights for policy 0, policy_version 117528 (0.0006) [2024-09-30 02:18:13,319][1157819] Updated weights for policy 0, policy_version 117538 (0.0006) [2024-09-30 02:18:13,911][1157819] Updated weights for policy 0, policy_version 117548 (0.0006) [2024-09-30 02:18:14,448][1157819] Updated weights for policy 0, policy_version 117558 (0.0006) [2024-09-30 02:18:15,020][1157819] Updated weights for policy 0, policy_version 117568 (0.0006) [2024-09-30 02:18:15,466][1157520] Fps is (10 sec: 74138.5, 60 sec: 80281.6, 300 sec: 79184.7). Total num frames: 481591296. Throughput: 0: 19814.8. Samples: 110367136. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 02:18:15,466][1157520] Avg episode reward: [(0, '54.242')] [2024-09-30 02:18:15,550][1157819] Updated weights for policy 0, policy_version 117578 (0.0006) [2024-09-30 02:18:16,128][1157819] Updated weights for policy 0, policy_version 117588 (0.0006) [2024-09-30 02:18:16,683][1157819] Updated weights for policy 0, policy_version 117598 (0.0006) [2024-09-30 02:18:17,243][1157819] Updated weights for policy 0, policy_version 117608 (0.0006) [2024-09-30 02:18:17,813][1157819] Updated weights for policy 0, policy_version 117618 (0.0006) [2024-09-30 02:18:18,378][1157819] Updated weights for policy 0, policy_version 117628 (0.0006) [2024-09-30 02:18:18,936][1157819] Updated weights for policy 0, policy_version 117638 (0.0006) [2024-09-30 02:18:19,506][1157819] Updated weights for policy 0, policy_version 117648 (0.0006) [2024-09-30 02:18:20,045][1157819] Updated weights for policy 0, policy_version 117658 (0.0006) [2024-09-30 02:18:20,466][1157520] Fps is (10 sec: 73728.0, 60 sec: 79735.4, 300 sec: 79157.0). Total num frames: 481955840. Throughput: 0: 19526.3. Samples: 110477240. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 02:18:20,466][1157520] Avg episode reward: [(0, '55.425')] [2024-09-30 02:18:20,552][1157819] Updated weights for policy 0, policy_version 117668 (0.0006) [2024-09-30 02:18:21,134][1157819] Updated weights for policy 0, policy_version 117678 (0.0006) [2024-09-30 02:18:21,654][1157819] Updated weights for policy 0, policy_version 117688 (0.0006) [2024-09-30 02:18:22,229][1157819] Updated weights for policy 0, policy_version 117698 (0.0006) [2024-09-30 02:18:22,760][1157819] Updated weights for policy 0, policy_version 117708 (0.0006) [2024-09-30 02:18:23,304][1157819] Updated weights for policy 0, policy_version 117718 (0.0006) [2024-09-30 02:18:23,852][1157819] Updated weights for policy 0, policy_version 117728 (0.0006) [2024-09-30 02:18:24,378][1157819] Updated weights for policy 0, policy_version 117738 (0.0006) [2024-09-30 02:18:24,911][1157819] Updated weights for policy 0, policy_version 117748 (0.0006) [2024-09-30 02:18:25,449][1157819] Updated weights for policy 0, policy_version 117758 (0.0006) [2024-09-30 02:18:25,466][1157520] Fps is (10 sec: 74547.0, 60 sec: 78848.0, 300 sec: 79198.6). Total num frames: 482336768. Throughput: 0: 19506.4. Samples: 110533568. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 02:18:25,466][1157520] Avg episode reward: [(0, '54.387')] [2024-09-30 02:18:26,017][1157819] Updated weights for policy 0, policy_version 117768 (0.0006) [2024-09-30 02:18:26,547][1157819] Updated weights for policy 0, policy_version 117778 (0.0006) [2024-09-30 02:18:27,075][1157819] Updated weights for policy 0, policy_version 117788 (0.0006) [2024-09-30 02:18:27,626][1157819] Updated weights for policy 0, policy_version 117798 (0.0006) [2024-09-30 02:18:28,173][1157819] Updated weights for policy 0, policy_version 117808 (0.0006) [2024-09-30 02:18:28,700][1157819] Updated weights for policy 0, policy_version 117818 (0.0006) [2024-09-30 02:18:29,256][1157819] Updated weights for policy 0, policy_version 117828 (0.0006) [2024-09-30 02:18:29,790][1157819] Updated weights for policy 0, policy_version 117838 (0.0006) [2024-09-30 02:18:30,365][1157819] Updated weights for policy 0, policy_version 117848 (0.0006) [2024-09-30 02:18:30,466][1157520] Fps is (10 sec: 75776.2, 60 sec: 77755.8, 300 sec: 79268.1). Total num frames: 482713600. Throughput: 0: 19440.4. Samples: 110647072. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 02:18:30,466][1157520] Avg episode reward: [(0, '55.422')] [2024-09-30 02:18:30,884][1157819] Updated weights for policy 0, policy_version 117858 (0.0006) [2024-09-30 02:18:31,421][1157819] Updated weights for policy 0, policy_version 117868 (0.0006) [2024-09-30 02:18:32,012][1157819] Updated weights for policy 0, policy_version 117878 (0.0006) [2024-09-30 02:18:32,554][1157819] Updated weights for policy 0, policy_version 117888 (0.0006) [2024-09-30 02:18:33,047][1157819] Updated weights for policy 0, policy_version 117898 (0.0006) [2024-09-30 02:18:33,579][1157819] Updated weights for policy 0, policy_version 117908 (0.0006) [2024-09-30 02:18:34,158][1157819] Updated weights for policy 0, policy_version 117918 (0.0006) [2024-09-30 02:18:34,711][1157819] Updated weights for policy 0, policy_version 117928 (0.0006) [2024-09-30 02:18:34,956][1157736] Signal inference workers to stop experience collection... (7900 times) [2024-09-30 02:18:34,960][1157736] Signal inference workers to resume experience collection... (7900 times) [2024-09-30 02:18:34,964][1157819] InferenceWorker_p0-w0: stopping experience collection (7900 times) [2024-09-30 02:18:34,966][1157819] InferenceWorker_p0-w0: resuming experience collection (7900 times) [2024-09-30 02:18:35,281][1157819] Updated weights for policy 0, policy_version 117938 (0.0006) [2024-09-30 02:18:35,466][1157520] Fps is (10 sec: 74956.9, 60 sec: 77346.2, 300 sec: 79295.8). Total num frames: 483086336. Throughput: 0: 19139.2. Samples: 110759668. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 02:18:35,466][1157520] Avg episode reward: [(0, '54.982')] [2024-09-30 02:18:35,805][1157819] Updated weights for policy 0, policy_version 117948 (0.0006) [2024-09-30 02:18:36,348][1157819] Updated weights for policy 0, policy_version 117958 (0.0006) [2024-09-30 02:18:36,879][1157819] Updated weights for policy 0, policy_version 117968 (0.0006) [2024-09-30 02:18:37,388][1157819] Updated weights for policy 0, policy_version 117978 (0.0006) [2024-09-30 02:18:37,910][1157819] Updated weights for policy 0, policy_version 117988 (0.0006) [2024-09-30 02:18:38,410][1157819] Updated weights for policy 0, policy_version 117998 (0.0006) [2024-09-30 02:18:38,946][1157819] Updated weights for policy 0, policy_version 118008 (0.0006) [2024-09-30 02:18:39,445][1157819] Updated weights for policy 0, policy_version 118018 (0.0006) [2024-09-30 02:18:39,967][1157819] Updated weights for policy 0, policy_version 118028 (0.0006) [2024-09-30 02:18:40,466][1157520] Fps is (10 sec: 76594.2, 60 sec: 77550.9, 300 sec: 79420.8). Total num frames: 483479552. Throughput: 0: 19023.7. Samples: 110817392. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 02:18:40,466][1157520] Avg episode reward: [(0, '56.082')] [2024-09-30 02:18:40,486][1157819] Updated weights for policy 0, policy_version 118038 (0.0006) [2024-09-30 02:18:41,012][1157819] Updated weights for policy 0, policy_version 118048 (0.0006) [2024-09-30 02:18:41,524][1157819] Updated weights for policy 0, policy_version 118058 (0.0006) [2024-09-30 02:18:42,046][1157819] Updated weights for policy 0, policy_version 118068 (0.0006) [2024-09-30 02:18:42,554][1157819] Updated weights for policy 0, policy_version 118078 (0.0006) [2024-09-30 02:18:43,037][1157819] Updated weights for policy 0, policy_version 118088 (0.0006) [2024-09-30 02:18:43,518][1157819] Updated weights for policy 0, policy_version 118098 (0.0007) [2024-09-30 02:18:44,060][1157819] Updated weights for policy 0, policy_version 118108 (0.0006) [2024-09-30 02:18:44,595][1157819] Updated weights for policy 0, policy_version 118118 (0.0006) [2024-09-30 02:18:45,140][1157819] Updated weights for policy 0, policy_version 118128 (0.0006) [2024-09-30 02:18:45,466][1157520] Fps is (10 sec: 79050.9, 60 sec: 77482.4, 300 sec: 79573.4). Total num frames: 483876864. Throughput: 0: 19007.5. Samples: 110937376. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:18:45,466][1157520] Avg episode reward: [(0, '54.348')] [2024-09-30 02:18:45,644][1157819] Updated weights for policy 0, policy_version 118138 (0.0006) [2024-09-30 02:18:46,206][1157819] Updated weights for policy 0, policy_version 118148 (0.0006) [2024-09-30 02:18:46,780][1157819] Updated weights for policy 0, policy_version 118158 (0.0007) [2024-09-30 02:18:47,358][1157819] Updated weights for policy 0, policy_version 118168 (0.0006) [2024-09-30 02:18:47,954][1157819] Updated weights for policy 0, policy_version 118178 (0.0006) [2024-09-30 02:18:48,564][1157819] Updated weights for policy 0, policy_version 118188 (0.0006) [2024-09-30 02:18:49,186][1157819] Updated weights for policy 0, policy_version 118198 (0.0006) [2024-09-30 02:18:49,830][1157819] Updated weights for policy 0, policy_version 118208 (0.0006) [2024-09-30 02:18:50,411][1157819] Updated weights for policy 0, policy_version 118218 (0.0006) [2024-09-30 02:18:50,466][1157520] Fps is (10 sec: 74546.9, 60 sec: 76322.0, 300 sec: 79545.7). Total num frames: 484225024. Throughput: 0: 18768.2. Samples: 111044476. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:18:50,466][1157520] Avg episode reward: [(0, '56.463')] [2024-09-30 02:18:50,990][1157819] Updated weights for policy 0, policy_version 118228 (0.0006) [2024-09-30 02:18:51,568][1157819] Updated weights for policy 0, policy_version 118238 (0.0006) [2024-09-30 02:18:52,139][1157819] Updated weights for policy 0, policy_version 118248 (0.0006) [2024-09-30 02:18:52,681][1157819] Updated weights for policy 0, policy_version 118258 (0.0006) [2024-09-30 02:18:53,221][1157819] Updated weights for policy 0, policy_version 118268 (0.0006) [2024-09-30 02:18:53,779][1157819] Updated weights for policy 0, policy_version 118278 (0.0006) [2024-09-30 02:18:54,327][1157819] Updated weights for policy 0, policy_version 118288 (0.0006) [2024-09-30 02:18:54,864][1157819] Updated weights for policy 0, policy_version 118298 (0.0006) [2024-09-30 02:18:55,401][1157819] Updated weights for policy 0, policy_version 118308 (0.0006) [2024-09-30 02:18:55,466][1157520] Fps is (10 sec: 71680.6, 60 sec: 75571.1, 300 sec: 79601.2). Total num frames: 484593664. Throughput: 0: 18740.2. Samples: 111098396. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:18:55,466][1157520] Avg episode reward: [(0, '54.821')] [2024-09-30 02:18:55,950][1157819] Updated weights for policy 0, policy_version 118318 (0.0006) [2024-09-30 02:18:56,451][1157819] Updated weights for policy 0, policy_version 118328 (0.0006) [2024-09-30 02:18:56,958][1157819] Updated weights for policy 0, policy_version 118338 (0.0006) [2024-09-30 02:18:57,471][1157819] Updated weights for policy 0, policy_version 118348 (0.0006) [2024-09-30 02:18:57,915][1157736] Signal inference workers to stop experience collection... (7950 times) [2024-09-30 02:18:57,915][1157736] Signal inference workers to resume experience collection... (7950 times) [2024-09-30 02:18:57,918][1157819] InferenceWorker_p0-w0: stopping experience collection (7950 times) [2024-09-30 02:18:57,919][1157819] InferenceWorker_p0-w0: resuming experience collection (7950 times) [2024-09-30 02:18:58,002][1157819] Updated weights for policy 0, policy_version 118358 (0.0006) [2024-09-30 02:18:58,528][1157819] Updated weights for policy 0, policy_version 118368 (0.0006) [2024-09-30 02:18:59,040][1157819] Updated weights for policy 0, policy_version 118378 (0.0006) [2024-09-30 02:18:59,538][1157819] Updated weights for policy 0, policy_version 118388 (0.0006) [2024-09-30 02:19:00,047][1157819] Updated weights for policy 0, policy_version 118398 (0.0006) [2024-09-30 02:19:00,466][1157520] Fps is (10 sec: 76595.0, 60 sec: 75502.9, 300 sec: 79629.0). Total num frames: 484990976. Throughput: 0: 18822.5. Samples: 111214152. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:19:00,466][1157520] Avg episode reward: [(0, '53.278')] [2024-09-30 02:19:00,555][1157819] Updated weights for policy 0, policy_version 118408 (0.0006) [2024-09-30 02:19:01,053][1157819] Updated weights for policy 0, policy_version 118418 (0.0006) [2024-09-30 02:19:01,547][1157819] Updated weights for policy 0, policy_version 118428 (0.0006) [2024-09-30 02:19:02,050][1157819] Updated weights for policy 0, policy_version 118438 (0.0006) [2024-09-30 02:19:02,530][1157819] Updated weights for policy 0, policy_version 118448 (0.0006) [2024-09-30 02:19:03,022][1157819] Updated weights for policy 0, policy_version 118458 (0.0006) [2024-09-30 02:19:03,543][1157819] Updated weights for policy 0, policy_version 118468 (0.0006) [2024-09-30 02:19:04,061][1157819] Updated weights for policy 0, policy_version 118478 (0.0006) [2024-09-30 02:19:04,604][1157819] Updated weights for policy 0, policy_version 118488 (0.0006) [2024-09-30 02:19:05,124][1157819] Updated weights for policy 0, policy_version 118498 (0.0006) [2024-09-30 02:19:05,466][1157520] Fps is (10 sec: 79872.8, 60 sec: 75707.8, 300 sec: 79698.5). Total num frames: 485392384. Throughput: 0: 19074.5. Samples: 111335596. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:19:05,466][1157520] Avg episode reward: [(0, '53.763')] [2024-09-30 02:19:05,633][1157819] Updated weights for policy 0, policy_version 118508 (0.0006) [2024-09-30 02:19:06,103][1157819] Updated weights for policy 0, policy_version 118518 (0.0006) [2024-09-30 02:19:06,541][1157819] Updated weights for policy 0, policy_version 118528 (0.0006) [2024-09-30 02:19:06,978][1157819] Updated weights for policy 0, policy_version 118538 (0.0006) [2024-09-30 02:19:07,411][1157819] Updated weights for policy 0, policy_version 118548 (0.0006) [2024-09-30 02:19:07,853][1157819] Updated weights for policy 0, policy_version 118558 (0.0006) [2024-09-30 02:19:08,308][1157819] Updated weights for policy 0, policy_version 118568 (0.0006) [2024-09-30 02:19:08,733][1157819] Updated weights for policy 0, policy_version 118578 (0.0006) [2024-09-30 02:19:09,185][1157819] Updated weights for policy 0, policy_version 118588 (0.0006) [2024-09-30 02:19:09,607][1157819] Updated weights for policy 0, policy_version 118598 (0.0006) [2024-09-30 02:19:10,071][1157819] Updated weights for policy 0, policy_version 118608 (0.0006) [2024-09-30 02:19:10,466][1157520] Fps is (10 sec: 86427.1, 60 sec: 77277.9, 300 sec: 79879.0). Total num frames: 485855232. Throughput: 0: 19291.6. Samples: 111401688. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:19:10,466][1157520] Avg episode reward: [(0, '53.345')] [2024-09-30 02:19:10,488][1157819] Updated weights for policy 0, policy_version 118618 (0.0006) [2024-09-30 02:19:10,941][1157819] Updated weights for policy 0, policy_version 118628 (0.0006) [2024-09-30 02:19:11,403][1157819] Updated weights for policy 0, policy_version 118638 (0.0006) [2024-09-30 02:19:11,877][1157819] Updated weights for policy 0, policy_version 118648 (0.0006) [2024-09-30 02:19:12,366][1157819] Updated weights for policy 0, policy_version 118658 (0.0006) [2024-09-30 02:19:12,821][1157819] Updated weights for policy 0, policy_version 118668 (0.0006) [2024-09-30 02:19:13,285][1157819] Updated weights for policy 0, policy_version 118678 (0.0006) [2024-09-30 02:19:13,786][1157819] Updated weights for policy 0, policy_version 118688 (0.0006) [2024-09-30 02:19:14,263][1157819] Updated weights for policy 0, policy_version 118698 (0.0006) [2024-09-30 02:19:14,755][1157819] Updated weights for policy 0, policy_version 118708 (0.0006) [2024-09-30 02:19:15,234][1157819] Updated weights for policy 0, policy_version 118718 (0.0006) [2024-09-30 02:19:15,466][1157520] Fps is (10 sec: 89293.4, 60 sec: 78233.6, 300 sec: 79934.5). Total num frames: 486285312. Throughput: 0: 19773.0. Samples: 111536856. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:19:15,466][1157520] Avg episode reward: [(0, '56.953')] [2024-09-30 02:19:15,707][1157819] Updated weights for policy 0, policy_version 118728 (0.0006) [2024-09-30 02:19:16,180][1157819] Updated weights for policy 0, policy_version 118738 (0.0006) [2024-09-30 02:19:16,676][1157819] Updated weights for policy 0, policy_version 118748 (0.0006) [2024-09-30 02:19:17,177][1157819] Updated weights for policy 0, policy_version 118758 (0.0006) [2024-09-30 02:19:17,680][1157819] Updated weights for policy 0, policy_version 118768 (0.0006) [2024-09-30 02:19:18,188][1157819] Updated weights for policy 0, policy_version 118778 (0.0006) [2024-09-30 02:19:18,699][1157819] Updated weights for policy 0, policy_version 118788 (0.0006) [2024-09-30 02:19:19,153][1157819] Updated weights for policy 0, policy_version 118798 (0.0006) [2024-09-30 02:19:19,639][1157819] Updated weights for policy 0, policy_version 118808 (0.0006) [2024-09-30 02:19:20,098][1157819] Updated weights for policy 0, policy_version 118818 (0.0006) [2024-09-30 02:19:20,466][1157520] Fps is (10 sec: 85196.9, 60 sec: 79189.4, 300 sec: 80101.2). Total num frames: 486707200. Throughput: 0: 20059.6. Samples: 111662348. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:19:20,466][1157520] Avg episode reward: [(0, '54.638')] [2024-09-30 02:19:20,615][1157819] Updated weights for policy 0, policy_version 118828 (0.0006) [2024-09-30 02:19:21,126][1157819] Updated weights for policy 0, policy_version 118838 (0.0006) [2024-09-30 02:19:21,621][1157819] Updated weights for policy 0, policy_version 118848 (0.0006) [2024-09-30 02:19:22,121][1157819] Updated weights for policy 0, policy_version 118858 (0.0006) [2024-09-30 02:19:22,624][1157819] Updated weights for policy 0, policy_version 118868 (0.0006) [2024-09-30 02:19:23,111][1157819] Updated weights for policy 0, policy_version 118878 (0.0006) [2024-09-30 02:19:23,626][1157819] Updated weights for policy 0, policy_version 118888 (0.0006) [2024-09-30 02:19:24,133][1157819] Updated weights for policy 0, policy_version 118898 (0.0006) [2024-09-30 02:19:24,620][1157819] Updated weights for policy 0, policy_version 118908 (0.0006) [2024-09-30 02:19:25,126][1157819] Updated weights for policy 0, policy_version 118918 (0.0006) [2024-09-30 02:19:25,466][1157520] Fps is (10 sec: 82739.1, 60 sec: 79598.9, 300 sec: 80170.5). Total num frames: 487112704. Throughput: 0: 20157.3. Samples: 111724468. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:19:25,466][1157520] Avg episode reward: [(0, '58.601')] [2024-09-30 02:19:25,478][1157736] Saving new best policy, reward=58.601! [2024-09-30 02:19:25,646][1157819] Updated weights for policy 0, policy_version 118928 (0.0006) [2024-09-30 02:19:25,647][1157736] Signal inference workers to stop experience collection... (8000 times) [2024-09-30 02:19:25,648][1157736] Signal inference workers to resume experience collection... (8000 times) [2024-09-30 02:19:25,653][1157819] InferenceWorker_p0-w0: stopping experience collection (8000 times) [2024-09-30 02:19:25,653][1157819] InferenceWorker_p0-w0: resuming experience collection (8000 times) [2024-09-30 02:19:26,134][1157819] Updated weights for policy 0, policy_version 118938 (0.0006) [2024-09-30 02:19:26,618][1157819] Updated weights for policy 0, policy_version 118948 (0.0006) [2024-09-30 02:19:27,160][1157819] Updated weights for policy 0, policy_version 118958 (0.0006) [2024-09-30 02:19:27,659][1157819] Updated weights for policy 0, policy_version 118968 (0.0006) [2024-09-30 02:19:28,166][1157819] Updated weights for policy 0, policy_version 118978 (0.0006) [2024-09-30 02:19:28,669][1157819] Updated weights for policy 0, policy_version 118988 (0.0006) [2024-09-30 02:19:29,242][1157819] Updated weights for policy 0, policy_version 118998 (0.0006) [2024-09-30 02:19:29,815][1157819] Updated weights for policy 0, policy_version 119008 (0.0006) [2024-09-30 02:19:30,278][1157819] Updated weights for policy 0, policy_version 119018 (0.0006) [2024-09-30 02:19:30,466][1157520] Fps is (10 sec: 80689.7, 60 sec: 80008.3, 300 sec: 80365.0). Total num frames: 487514112. Throughput: 0: 20184.2. Samples: 111845664. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:19:30,466][1157520] Avg episode reward: [(0, '52.472')] [2024-09-30 02:19:30,772][1157819] Updated weights for policy 0, policy_version 119028 (0.0006) [2024-09-30 02:19:31,294][1157819] Updated weights for policy 0, policy_version 119038 (0.0006) [2024-09-30 02:19:31,773][1157819] Updated weights for policy 0, policy_version 119048 (0.0006) [2024-09-30 02:19:32,225][1157819] Updated weights for policy 0, policy_version 119058 (0.0006) [2024-09-30 02:19:32,723][1157819] Updated weights for policy 0, policy_version 119068 (0.0006) [2024-09-30 02:19:33,221][1157819] Updated weights for policy 0, policy_version 119078 (0.0006) [2024-09-30 02:19:33,729][1157819] Updated weights for policy 0, policy_version 119088 (0.0006) [2024-09-30 02:19:34,216][1157819] Updated weights for policy 0, policy_version 119098 (0.0006) [2024-09-30 02:19:34,729][1157819] Updated weights for policy 0, policy_version 119108 (0.0006) [2024-09-30 02:19:35,222][1157819] Updated weights for policy 0, policy_version 119118 (0.0006) [2024-09-30 02:19:35,466][1157520] Fps is (10 sec: 81509.5, 60 sec: 80691.1, 300 sec: 80517.7). Total num frames: 487927808. Throughput: 0: 20521.2. Samples: 111967928. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:19:35,466][1157520] Avg episode reward: [(0, '55.043')] [2024-09-30 02:19:35,723][1157819] Updated weights for policy 0, policy_version 119128 (0.0006) [2024-09-30 02:19:36,212][1157819] Updated weights for policy 0, policy_version 119138 (0.0006) [2024-09-30 02:19:36,703][1157819] Updated weights for policy 0, policy_version 119148 (0.0006) [2024-09-30 02:19:37,204][1157819] Updated weights for policy 0, policy_version 119158 (0.0006) [2024-09-30 02:19:37,718][1157819] Updated weights for policy 0, policy_version 119168 (0.0006) [2024-09-30 02:19:38,258][1157819] Updated weights for policy 0, policy_version 119178 (0.0006) [2024-09-30 02:19:38,771][1157819] Updated weights for policy 0, policy_version 119188 (0.0006) [2024-09-30 02:19:39,249][1157819] Updated weights for policy 0, policy_version 119198 (0.0006) [2024-09-30 02:19:39,757][1157819] Updated weights for policy 0, policy_version 119208 (0.0006) [2024-09-30 02:19:40,238][1157819] Updated weights for policy 0, policy_version 119218 (0.0006) [2024-09-30 02:19:40,466][1157520] Fps is (10 sec: 81919.9, 60 sec: 80895.9, 300 sec: 80628.7). Total num frames: 488333312. Throughput: 0: 20691.5. Samples: 112029516. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:19:40,466][1157520] Avg episode reward: [(0, '54.436')] [2024-09-30 02:19:40,704][1157819] Updated weights for policy 0, policy_version 119228 (0.0006) [2024-09-30 02:19:41,205][1157819] Updated weights for policy 0, policy_version 119238 (0.0006) [2024-09-30 02:19:41,712][1157819] Updated weights for policy 0, policy_version 119248 (0.0006) [2024-09-30 02:19:42,212][1157819] Updated weights for policy 0, policy_version 119258 (0.0006) [2024-09-30 02:19:42,683][1157819] Updated weights for policy 0, policy_version 119268 (0.0006) [2024-09-30 02:19:43,176][1157819] Updated weights for policy 0, policy_version 119278 (0.0006) [2024-09-30 02:19:43,680][1157819] Updated weights for policy 0, policy_version 119288 (0.0006) [2024-09-30 02:19:44,176][1157819] Updated weights for policy 0, policy_version 119298 (0.0006) [2024-09-30 02:19:44,678][1157819] Updated weights for policy 0, policy_version 119308 (0.0006) [2024-09-30 02:19:45,170][1157819] Updated weights for policy 0, policy_version 119318 (0.0006) [2024-09-30 02:19:45,466][1157520] Fps is (10 sec: 81920.1, 60 sec: 81169.3, 300 sec: 80712.1). Total num frames: 488747008. Throughput: 0: 20861.5. Samples: 112152916. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:19:45,466][1157520] Avg episode reward: [(0, '52.115')] [2024-09-30 02:19:45,669][1157819] Updated weights for policy 0, policy_version 119328 (0.0006) [2024-09-30 02:19:46,162][1157819] Updated weights for policy 0, policy_version 119338 (0.0006) [2024-09-30 02:19:46,653][1157819] Updated weights for policy 0, policy_version 119348 (0.0006) [2024-09-30 02:19:47,138][1157819] Updated weights for policy 0, policy_version 119358 (0.0006) [2024-09-30 02:19:47,642][1157819] Updated weights for policy 0, policy_version 119368 (0.0006) [2024-09-30 02:19:48,133][1157819] Updated weights for policy 0, policy_version 119378 (0.0006) [2024-09-30 02:19:48,599][1157819] Updated weights for policy 0, policy_version 119388 (0.0006) [2024-09-30 02:19:49,089][1157819] Updated weights for policy 0, policy_version 119398 (0.0006) [2024-09-30 02:19:49,594][1157819] Updated weights for policy 0, policy_version 119408 (0.0006) [2024-09-30 02:19:50,086][1157819] Updated weights for policy 0, policy_version 119418 (0.0006) [2024-09-30 02:19:50,466][1157520] Fps is (10 sec: 83149.6, 60 sec: 82329.7, 300 sec: 80809.3). Total num frames: 489164800. Throughput: 0: 20941.3. Samples: 112277956. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 02:19:50,466][1157520] Avg episode reward: [(0, '55.447')] [2024-09-30 02:19:50,550][1157819] Updated weights for policy 0, policy_version 119428 (0.0006) [2024-09-30 02:19:51,055][1157819] Updated weights for policy 0, policy_version 119438 (0.0006) [2024-09-30 02:19:51,561][1157819] Updated weights for policy 0, policy_version 119448 (0.0006) [2024-09-30 02:19:52,038][1157819] Updated weights for policy 0, policy_version 119458 (0.0006) [2024-09-30 02:19:52,506][1157819] Updated weights for policy 0, policy_version 119468 (0.0006) [2024-09-30 02:19:52,988][1157819] Updated weights for policy 0, policy_version 119478 (0.0006) [2024-09-30 02:19:53,425][1157819] Updated weights for policy 0, policy_version 119488 (0.0006) [2024-09-30 02:19:53,861][1157819] Updated weights for policy 0, policy_version 119498 (0.0006) [2024-09-30 02:19:54,357][1157819] Updated weights for policy 0, policy_version 119508 (0.0006) [2024-09-30 02:19:54,823][1157819] Updated weights for policy 0, policy_version 119518 (0.0006) [2024-09-30 02:19:55,325][1157819] Updated weights for policy 0, policy_version 119528 (0.0006) [2024-09-30 02:19:55,466][1157520] Fps is (10 sec: 85196.9, 60 sec: 83422.0, 300 sec: 80948.1). Total num frames: 489598976. Throughput: 0: 20884.6. Samples: 112341496. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 02:19:55,466][1157520] Avg episode reward: [(0, '55.307')] [2024-09-30 02:19:55,770][1157819] Updated weights for policy 0, policy_version 119538 (0.0006) [2024-09-30 02:19:56,301][1157819] Updated weights for policy 0, policy_version 119548 (0.0006) [2024-09-30 02:19:56,816][1157819] Updated weights for policy 0, policy_version 119558 (0.0006) [2024-09-30 02:19:57,393][1157819] Updated weights for policy 0, policy_version 119568 (0.0006) [2024-09-30 02:19:57,912][1157819] Updated weights for policy 0, policy_version 119578 (0.0006) [2024-09-30 02:19:58,464][1157819] Updated weights for policy 0, policy_version 119588 (0.0006) [2024-09-30 02:19:59,031][1157819] Updated weights for policy 0, policy_version 119598 (0.0006) [2024-09-30 02:19:59,583][1157819] Updated weights for policy 0, policy_version 119608 (0.0006) [2024-09-30 02:19:59,668][1157736] Signal inference workers to stop experience collection... (8050 times) [2024-09-30 02:19:59,668][1157736] Signal inference workers to resume experience collection... (8050 times) [2024-09-30 02:19:59,671][1157819] InferenceWorker_p0-w0: stopping experience collection (8050 times) [2024-09-30 02:19:59,672][1157819] InferenceWorker_p0-w0: resuming experience collection (8050 times) [2024-09-30 02:20:00,090][1157819] Updated weights for policy 0, policy_version 119618 (0.0006) [2024-09-30 02:20:00,466][1157520] Fps is (10 sec: 81510.3, 60 sec: 83148.9, 300 sec: 80906.4). Total num frames: 489979904. Throughput: 0: 20603.6. Samples: 112464020. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 02:20:00,466][1157520] Avg episode reward: [(0, '56.344')] [2024-09-30 02:20:00,643][1157819] Updated weights for policy 0, policy_version 119628 (0.0006) [2024-09-30 02:20:01,155][1157819] Updated weights for policy 0, policy_version 119638 (0.0006) [2024-09-30 02:20:01,657][1157819] Updated weights for policy 0, policy_version 119648 (0.0006) [2024-09-30 02:20:02,161][1157819] Updated weights for policy 0, policy_version 119658 (0.0006) [2024-09-30 02:20:02,676][1157819] Updated weights for policy 0, policy_version 119668 (0.0006) [2024-09-30 02:20:03,188][1157819] Updated weights for policy 0, policy_version 119678 (0.0006) [2024-09-30 02:20:03,679][1157819] Updated weights for policy 0, policy_version 119688 (0.0006) [2024-09-30 02:20:04,170][1157819] Updated weights for policy 0, policy_version 119698 (0.0006) [2024-09-30 02:20:04,681][1157819] Updated weights for policy 0, policy_version 119708 (0.0006) [2024-09-30 02:20:05,190][1157819] Updated weights for policy 0, policy_version 119718 (0.0006) [2024-09-30 02:20:05,466][1157520] Fps is (10 sec: 78643.8, 60 sec: 83217.1, 300 sec: 80948.1). Total num frames: 490385408. Throughput: 0: 20467.5. Samples: 112583388. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 02:20:05,466][1157520] Avg episode reward: [(0, '55.104')] [2024-09-30 02:20:05,469][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000119723_490385408.pth... [2024-09-30 02:20:05,535][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000114978_470949888.pth [2024-09-30 02:20:05,740][1157819] Updated weights for policy 0, policy_version 119728 (0.0006) [2024-09-30 02:20:06,271][1157819] Updated weights for policy 0, policy_version 119738 (0.0006) [2024-09-30 02:20:06,795][1157819] Updated weights for policy 0, policy_version 119748 (0.0006) [2024-09-30 02:20:07,307][1157819] Updated weights for policy 0, policy_version 119758 (0.0006) [2024-09-30 02:20:07,802][1157819] Updated weights for policy 0, policy_version 119768 (0.0006) [2024-09-30 02:20:08,293][1157819] Updated weights for policy 0, policy_version 119778 (0.0006) [2024-09-30 02:20:08,794][1157819] Updated weights for policy 0, policy_version 119788 (0.0006) [2024-09-30 02:20:09,279][1157819] Updated weights for policy 0, policy_version 119798 (0.0006) [2024-09-30 02:20:09,781][1157819] Updated weights for policy 0, policy_version 119808 (0.0006) [2024-09-30 02:20:10,271][1157819] Updated weights for policy 0, policy_version 119818 (0.0006) [2024-09-30 02:20:10,466][1157520] Fps is (10 sec: 80691.3, 60 sec: 82192.9, 300 sec: 81003.6). Total num frames: 490786816. Throughput: 0: 20394.9. Samples: 112642240. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 02:20:10,466][1157520] Avg episode reward: [(0, '55.036')] [2024-09-30 02:20:10,800][1157819] Updated weights for policy 0, policy_version 119828 (0.0006) [2024-09-30 02:20:11,306][1157819] Updated weights for policy 0, policy_version 119838 (0.0006) [2024-09-30 02:20:11,812][1157819] Updated weights for policy 0, policy_version 119848 (0.0006) [2024-09-30 02:20:12,338][1157819] Updated weights for policy 0, policy_version 119858 (0.0006) [2024-09-30 02:20:12,838][1157819] Updated weights for policy 0, policy_version 119868 (0.0006) [2024-09-30 02:20:13,341][1157819] Updated weights for policy 0, policy_version 119878 (0.0006) [2024-09-30 02:20:13,836][1157819] Updated weights for policy 0, policy_version 119888 (0.0006) [2024-09-30 02:20:14,339][1157819] Updated weights for policy 0, policy_version 119898 (0.0006) [2024-09-30 02:20:14,837][1157819] Updated weights for policy 0, policy_version 119908 (0.0006) [2024-09-30 02:20:15,359][1157819] Updated weights for policy 0, policy_version 119918 (0.0006) [2024-09-30 02:20:15,466][1157520] Fps is (10 sec: 80280.8, 60 sec: 81715.0, 300 sec: 81045.2). Total num frames: 491188224. Throughput: 0: 20407.5. Samples: 112764000. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 02:20:15,466][1157520] Avg episode reward: [(0, '55.415')] [2024-09-30 02:20:15,869][1157819] Updated weights for policy 0, policy_version 119928 (0.0006) [2024-09-30 02:20:16,420][1157819] Updated weights for policy 0, policy_version 119938 (0.0006) [2024-09-30 02:20:16,906][1157819] Updated weights for policy 0, policy_version 119948 (0.0006) [2024-09-30 02:20:17,427][1157819] Updated weights for policy 0, policy_version 119958 (0.0006) [2024-09-30 02:20:17,931][1157819] Updated weights for policy 0, policy_version 119968 (0.0006) [2024-09-30 02:20:18,453][1157819] Updated weights for policy 0, policy_version 119978 (0.0006) [2024-09-30 02:20:18,929][1157819] Updated weights for policy 0, policy_version 119988 (0.0006) [2024-09-30 02:20:19,415][1157819] Updated weights for policy 0, policy_version 119998 (0.0006) [2024-09-30 02:20:19,862][1157819] Updated weights for policy 0, policy_version 120008 (0.0006) [2024-09-30 02:20:20,305][1157819] Updated weights for policy 0, policy_version 120018 (0.0006) [2024-09-30 02:20:20,466][1157520] Fps is (10 sec: 81920.1, 60 sec: 81646.8, 300 sec: 81114.7). Total num frames: 491606016. Throughput: 0: 20411.8. Samples: 112886456. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 02:20:20,466][1157520] Avg episode reward: [(0, '54.429')] [2024-09-30 02:20:20,787][1157819] Updated weights for policy 0, policy_version 120028 (0.0006) [2024-09-30 02:20:21,247][1157819] Updated weights for policy 0, policy_version 120038 (0.0006) [2024-09-30 02:20:21,710][1157819] Updated weights for policy 0, policy_version 120048 (0.0006) [2024-09-30 02:20:22,190][1157819] Updated weights for policy 0, policy_version 120058 (0.0006) [2024-09-30 02:20:22,644][1157819] Updated weights for policy 0, policy_version 120068 (0.0006) [2024-09-30 02:20:23,134][1157819] Updated weights for policy 0, policy_version 120078 (0.0006) [2024-09-30 02:20:23,604][1157819] Updated weights for policy 0, policy_version 120088 (0.0006) [2024-09-30 02:20:24,092][1157819] Updated weights for policy 0, policy_version 120098 (0.0006) [2024-09-30 02:20:24,587][1157819] Updated weights for policy 0, policy_version 120108 (0.0006) [2024-09-30 02:20:25,080][1157819] Updated weights for policy 0, policy_version 120118 (0.0006) [2024-09-30 02:20:25,249][1157736] Signal inference workers to stop experience collection... (8100 times) [2024-09-30 02:20:25,250][1157736] Signal inference workers to resume experience collection... (8100 times) [2024-09-30 02:20:25,253][1157819] InferenceWorker_p0-w0: stopping experience collection (8100 times) [2024-09-30 02:20:25,253][1157819] InferenceWorker_p0-w0: resuming experience collection (8100 times) [2024-09-30 02:20:25,466][1157520] Fps is (10 sec: 84786.9, 60 sec: 82056.3, 300 sec: 81184.1). Total num frames: 492036096. Throughput: 0: 20513.6. Samples: 112952628. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 02:20:25,466][1157520] Avg episode reward: [(0, '55.335')] [2024-09-30 02:20:25,583][1157819] Updated weights for policy 0, policy_version 120128 (0.0006) [2024-09-30 02:20:26,082][1157819] Updated weights for policy 0, policy_version 120138 (0.0006) [2024-09-30 02:20:26,567][1157819] Updated weights for policy 0, policy_version 120148 (0.0006) [2024-09-30 02:20:27,053][1157819] Updated weights for policy 0, policy_version 120158 (0.0006) [2024-09-30 02:20:27,533][1157819] Updated weights for policy 0, policy_version 120168 (0.0006) [2024-09-30 02:20:28,040][1157819] Updated weights for policy 0, policy_version 120178 (0.0006) [2024-09-30 02:20:28,533][1157819] Updated weights for policy 0, policy_version 120188 (0.0006) [2024-09-30 02:20:29,045][1157819] Updated weights for policy 0, policy_version 120198 (0.0006) [2024-09-30 02:20:29,537][1157819] Updated weights for policy 0, policy_version 120208 (0.0006) [2024-09-30 02:20:30,057][1157819] Updated weights for policy 0, policy_version 120218 (0.0006) [2024-09-30 02:20:30,466][1157520] Fps is (10 sec: 83558.0, 60 sec: 82124.9, 300 sec: 81211.9). Total num frames: 492441600. Throughput: 0: 20551.9. Samples: 113077752. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 02:20:30,466][1157520] Avg episode reward: [(0, '57.167')] [2024-09-30 02:20:30,586][1157819] Updated weights for policy 0, policy_version 120228 (0.0006) [2024-09-30 02:20:31,163][1157819] Updated weights for policy 0, policy_version 120238 (0.0006) [2024-09-30 02:20:31,662][1157819] Updated weights for policy 0, policy_version 120248 (0.0006) [2024-09-30 02:20:32,191][1157819] Updated weights for policy 0, policy_version 120258 (0.0006) [2024-09-30 02:20:32,752][1157819] Updated weights for policy 0, policy_version 120268 (0.0006) [2024-09-30 02:20:33,324][1157819] Updated weights for policy 0, policy_version 120278 (0.0007) [2024-09-30 02:20:33,877][1157819] Updated weights for policy 0, policy_version 120288 (0.0006) [2024-09-30 02:20:34,375][1157819] Updated weights for policy 0, policy_version 120298 (0.0006) [2024-09-30 02:20:34,897][1157819] Updated weights for policy 0, policy_version 120308 (0.0006) [2024-09-30 02:20:35,426][1157819] Updated weights for policy 0, policy_version 120318 (0.0006) [2024-09-30 02:20:35,466][1157520] Fps is (10 sec: 79053.2, 60 sec: 81647.0, 300 sec: 81156.3). Total num frames: 492826624. Throughput: 0: 20342.6. Samples: 113193372. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 02:20:35,466][1157520] Avg episode reward: [(0, '54.128')] [2024-09-30 02:20:35,923][1157819] Updated weights for policy 0, policy_version 120328 (0.0006) [2024-09-30 02:20:36,419][1157819] Updated weights for policy 0, policy_version 120338 (0.0006) [2024-09-30 02:20:36,940][1157819] Updated weights for policy 0, policy_version 120348 (0.0006) [2024-09-30 02:20:37,439][1157819] Updated weights for policy 0, policy_version 120358 (0.0006) [2024-09-30 02:20:37,967][1157819] Updated weights for policy 0, policy_version 120368 (0.0006) [2024-09-30 02:20:38,491][1157819] Updated weights for policy 0, policy_version 120378 (0.0006) [2024-09-30 02:20:38,976][1157819] Updated weights for policy 0, policy_version 120388 (0.0006) [2024-09-30 02:20:39,479][1157819] Updated weights for policy 0, policy_version 120398 (0.0006) [2024-09-30 02:20:40,007][1157819] Updated weights for policy 0, policy_version 120408 (0.0006) [2024-09-30 02:20:40,466][1157520] Fps is (10 sec: 78643.4, 60 sec: 81578.8, 300 sec: 81100.8). Total num frames: 493228032. Throughput: 0: 20264.6. Samples: 113253404. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 02:20:40,466][1157520] Avg episode reward: [(0, '56.048')] [2024-09-30 02:20:40,505][1157819] Updated weights for policy 0, policy_version 120418 (0.0006) [2024-09-30 02:20:40,998][1157819] Updated weights for policy 0, policy_version 120428 (0.0006) [2024-09-30 02:20:41,512][1157819] Updated weights for policy 0, policy_version 120438 (0.0006) [2024-09-30 02:20:42,035][1157819] Updated weights for policy 0, policy_version 120448 (0.0006) [2024-09-30 02:20:42,550][1157819] Updated weights for policy 0, policy_version 120458 (0.0006) [2024-09-30 02:20:43,083][1157819] Updated weights for policy 0, policy_version 120468 (0.0006) [2024-09-30 02:20:43,609][1157819] Updated weights for policy 0, policy_version 120478 (0.0006) [2024-09-30 02:20:44,156][1157819] Updated weights for policy 0, policy_version 120488 (0.0006) [2024-09-30 02:20:44,799][1157819] Updated weights for policy 0, policy_version 120498 (0.0006) [2024-09-30 02:20:45,393][1157819] Updated weights for policy 0, policy_version 120508 (0.0007) [2024-09-30 02:20:45,466][1157520] Fps is (10 sec: 78233.7, 60 sec: 81032.6, 300 sec: 80975.8). Total num frames: 493608960. Throughput: 0: 20188.6. Samples: 113372508. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 02:20:45,466][1157520] Avg episode reward: [(0, '55.039')] [2024-09-30 02:20:45,975][1157819] Updated weights for policy 0, policy_version 120518 (0.0006) [2024-09-30 02:20:46,550][1157819] Updated weights for policy 0, policy_version 120528 (0.0006) [2024-09-30 02:20:47,085][1157819] Updated weights for policy 0, policy_version 120538 (0.0006) [2024-09-30 02:20:47,589][1157819] Updated weights for policy 0, policy_version 120548 (0.0006) [2024-09-30 02:20:47,728][1157736] Signal inference workers to stop experience collection... (8150 times) [2024-09-30 02:20:47,732][1157736] Signal inference workers to resume experience collection... (8150 times) [2024-09-30 02:20:47,733][1157819] InferenceWorker_p0-w0: stopping experience collection (8150 times) [2024-09-30 02:20:47,736][1157819] InferenceWorker_p0-w0: resuming experience collection (8150 times) [2024-09-30 02:20:48,124][1157819] Updated weights for policy 0, policy_version 120558 (0.0006) [2024-09-30 02:20:48,716][1157819] Updated weights for policy 0, policy_version 120568 (0.0006) [2024-09-30 02:20:49,239][1157819] Updated weights for policy 0, policy_version 120578 (0.0006) [2024-09-30 02:20:49,832][1157819] Updated weights for policy 0, policy_version 120588 (0.0006) [2024-09-30 02:20:50,343][1157819] Updated weights for policy 0, policy_version 120598 (0.0006) [2024-09-30 02:20:50,466][1157520] Fps is (10 sec: 74546.9, 60 sec: 80145.0, 300 sec: 80850.8). Total num frames: 493973504. Throughput: 0: 19966.4. Samples: 113481880. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 02:20:50,466][1157520] Avg episode reward: [(0, '54.584')] [2024-09-30 02:20:50,893][1157819] Updated weights for policy 0, policy_version 120608 (0.0006) [2024-09-30 02:20:51,429][1157819] Updated weights for policy 0, policy_version 120618 (0.0006) [2024-09-30 02:20:51,925][1157819] Updated weights for policy 0, policy_version 120628 (0.0006) [2024-09-30 02:20:52,463][1157819] Updated weights for policy 0, policy_version 120638 (0.0006) [2024-09-30 02:20:52,961][1157819] Updated weights for policy 0, policy_version 120648 (0.0006) [2024-09-30 02:20:53,498][1157819] Updated weights for policy 0, policy_version 120658 (0.0006) [2024-09-30 02:20:54,014][1157819] Updated weights for policy 0, policy_version 120668 (0.0006) [2024-09-30 02:20:54,515][1157819] Updated weights for policy 0, policy_version 120678 (0.0006) [2024-09-30 02:20:55,023][1157819] Updated weights for policy 0, policy_version 120688 (0.0006) [2024-09-30 02:20:55,466][1157520] Fps is (10 sec: 75775.9, 60 sec: 79462.4, 300 sec: 80795.3). Total num frames: 494366720. Throughput: 0: 19949.4. Samples: 113539964. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 02:20:55,466][1157520] Avg episode reward: [(0, '54.713')] [2024-09-30 02:20:55,532][1157819] Updated weights for policy 0, policy_version 120698 (0.0006) [2024-09-30 02:20:56,101][1157819] Updated weights for policy 0, policy_version 120708 (0.0006) [2024-09-30 02:20:56,640][1157819] Updated weights for policy 0, policy_version 120718 (0.0006) [2024-09-30 02:20:57,133][1157819] Updated weights for policy 0, policy_version 120728 (0.0006) [2024-09-30 02:20:57,654][1157819] Updated weights for policy 0, policy_version 120738 (0.0006) [2024-09-30 02:20:58,162][1157819] Updated weights for policy 0, policy_version 120748 (0.0006) [2024-09-30 02:20:58,670][1157819] Updated weights for policy 0, policy_version 120758 (0.0006) [2024-09-30 02:20:59,193][1157819] Updated weights for policy 0, policy_version 120768 (0.0006) [2024-09-30 02:20:59,712][1157819] Updated weights for policy 0, policy_version 120778 (0.0006) [2024-09-30 02:21:00,206][1157819] Updated weights for policy 0, policy_version 120788 (0.0006) [2024-09-30 02:21:00,466][1157520] Fps is (10 sec: 79462.9, 60 sec: 79803.8, 300 sec: 80753.7). Total num frames: 494768128. Throughput: 0: 19875.8. Samples: 113658408. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 02:21:00,466][1157520] Avg episode reward: [(0, '55.762')] [2024-09-30 02:21:00,711][1157819] Updated weights for policy 0, policy_version 120798 (0.0006) [2024-09-30 02:21:01,198][1157819] Updated weights for policy 0, policy_version 120808 (0.0006) [2024-09-30 02:21:01,723][1157819] Updated weights for policy 0, policy_version 120818 (0.0006) [2024-09-30 02:21:02,227][1157819] Updated weights for policy 0, policy_version 120828 (0.0006) [2024-09-30 02:21:02,731][1157819] Updated weights for policy 0, policy_version 120838 (0.0006) [2024-09-30 02:21:03,270][1157819] Updated weights for policy 0, policy_version 120848 (0.0006) [2024-09-30 02:21:03,545][1157736] Signal inference workers to stop experience collection... (8200 times) [2024-09-30 02:21:03,545][1157736] Signal inference workers to resume experience collection... (8200 times) [2024-09-30 02:21:03,549][1157819] InferenceWorker_p0-w0: stopping experience collection (8200 times) [2024-09-30 02:21:03,549][1157819] InferenceWorker_p0-w0: resuming experience collection (8200 times) [2024-09-30 02:21:03,843][1157819] Updated weights for policy 0, policy_version 120858 (0.0006) [2024-09-30 02:21:04,379][1157819] Updated weights for policy 0, policy_version 120868 (0.0006) [2024-09-30 02:21:04,898][1157819] Updated weights for policy 0, policy_version 120878 (0.0006) [2024-09-30 02:21:05,451][1157819] Updated weights for policy 0, policy_version 120888 (0.0006) [2024-09-30 02:21:05,466][1157520] Fps is (10 sec: 79052.2, 60 sec: 79530.5, 300 sec: 80656.4). Total num frames: 495157248. Throughput: 0: 19776.9. Samples: 113776420. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 02:21:05,466][1157520] Avg episode reward: [(0, '55.559')] [2024-09-30 02:21:06,010][1157819] Updated weights for policy 0, policy_version 120898 (0.0006) [2024-09-30 02:21:06,541][1157819] Updated weights for policy 0, policy_version 120908 (0.0006) [2024-09-30 02:21:07,043][1157819] Updated weights for policy 0, policy_version 120918 (0.0006) [2024-09-30 02:21:07,563][1157819] Updated weights for policy 0, policy_version 120928 (0.0006) [2024-09-30 02:21:08,133][1157819] Updated weights for policy 0, policy_version 120938 (0.0006) [2024-09-30 02:21:08,702][1157819] Updated weights for policy 0, policy_version 120948 (0.0006) [2024-09-30 02:21:09,244][1157819] Updated weights for policy 0, policy_version 120958 (0.0006) [2024-09-30 02:21:09,817][1157819] Updated weights for policy 0, policy_version 120968 (0.0006) [2024-09-30 02:21:10,379][1157819] Updated weights for policy 0, policy_version 120978 (0.0006) [2024-09-30 02:21:10,466][1157520] Fps is (10 sec: 76185.6, 60 sec: 79052.8, 300 sec: 80476.0). Total num frames: 495529984. Throughput: 0: 19590.1. Samples: 113834180. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 02:21:10,466][1157520] Avg episode reward: [(0, '53.469')] [2024-09-30 02:21:10,958][1157819] Updated weights for policy 0, policy_version 120988 (0.0006) [2024-09-30 02:21:11,517][1157819] Updated weights for policy 0, policy_version 120998 (0.0006) [2024-09-30 02:21:12,089][1157819] Updated weights for policy 0, policy_version 121008 (0.0006) [2024-09-30 02:21:12,633][1157819] Updated weights for policy 0, policy_version 121018 (0.0006) [2024-09-30 02:21:13,143][1157819] Updated weights for policy 0, policy_version 121028 (0.0006) [2024-09-30 02:21:13,716][1157819] Updated weights for policy 0, policy_version 121038 (0.0006) [2024-09-30 02:21:14,283][1157819] Updated weights for policy 0, policy_version 121048 (0.0006) [2024-09-30 02:21:14,833][1157819] Updated weights for policy 0, policy_version 121058 (0.0006) [2024-09-30 02:21:15,381][1157819] Updated weights for policy 0, policy_version 121068 (0.0006) [2024-09-30 02:21:15,466][1157520] Fps is (10 sec: 74138.0, 60 sec: 78506.7, 300 sec: 80309.3). Total num frames: 495898624. Throughput: 0: 19252.6. Samples: 113944120. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 02:21:15,466][1157520] Avg episode reward: [(0, '54.448')] [2024-09-30 02:21:15,898][1157819] Updated weights for policy 0, policy_version 121078 (0.0006) [2024-09-30 02:21:16,479][1157819] Updated weights for policy 0, policy_version 121088 (0.0006) [2024-09-30 02:21:16,994][1157819] Updated weights for policy 0, policy_version 121098 (0.0006) [2024-09-30 02:21:17,556][1157819] Updated weights for policy 0, policy_version 121108 (0.0006) [2024-09-30 02:21:18,089][1157819] Updated weights for policy 0, policy_version 121118 (0.0006) [2024-09-30 02:21:18,617][1157819] Updated weights for policy 0, policy_version 121128 (0.0006) [2024-09-30 02:21:19,160][1157819] Updated weights for policy 0, policy_version 121138 (0.0006) [2024-09-30 02:21:19,705][1157819] Updated weights for policy 0, policy_version 121148 (0.0006) [2024-09-30 02:21:20,232][1157819] Updated weights for policy 0, policy_version 121158 (0.0006) [2024-09-30 02:21:20,466][1157520] Fps is (10 sec: 74547.2, 60 sec: 77824.0, 300 sec: 80184.4). Total num frames: 496275456. Throughput: 0: 19194.9. Samples: 114057144. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 02:21:20,466][1157520] Avg episode reward: [(0, '54.945')] [2024-09-30 02:21:20,776][1157819] Updated weights for policy 0, policy_version 121168 (0.0006) [2024-09-30 02:21:21,325][1157819] Updated weights for policy 0, policy_version 121178 (0.0006) [2024-09-30 02:21:21,844][1157819] Updated weights for policy 0, policy_version 121188 (0.0006) [2024-09-30 02:21:22,366][1157819] Updated weights for policy 0, policy_version 121198 (0.0006) [2024-09-30 02:21:22,923][1157819] Updated weights for policy 0, policy_version 121208 (0.0006) [2024-09-30 02:21:23,463][1157819] Updated weights for policy 0, policy_version 121218 (0.0006) [2024-09-30 02:21:24,024][1157819] Updated weights for policy 0, policy_version 121228 (0.0006) [2024-09-30 02:21:24,596][1157819] Updated weights for policy 0, policy_version 121238 (0.0006) [2024-09-30 02:21:25,094][1157819] Updated weights for policy 0, policy_version 121248 (0.0006) [2024-09-30 02:21:25,466][1157520] Fps is (10 sec: 75775.1, 60 sec: 77004.7, 300 sec: 80087.1). Total num frames: 496656384. Throughput: 0: 19127.7. Samples: 114114152. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 02:21:25,466][1157520] Avg episode reward: [(0, '55.057')] [2024-09-30 02:21:25,624][1157819] Updated weights for policy 0, policy_version 121258 (0.0006) [2024-09-30 02:21:26,166][1157819] Updated weights for policy 0, policy_version 121268 (0.0006) [2024-09-30 02:21:26,757][1157819] Updated weights for policy 0, policy_version 121278 (0.0006) [2024-09-30 02:21:27,291][1157819] Updated weights for policy 0, policy_version 121288 (0.0006) [2024-09-30 02:21:27,866][1157819] Updated weights for policy 0, policy_version 121298 (0.0006) [2024-09-30 02:21:28,396][1157819] Updated weights for policy 0, policy_version 121308 (0.0006) [2024-09-30 02:21:28,951][1157819] Updated weights for policy 0, policy_version 121318 (0.0006) [2024-09-30 02:21:29,493][1157819] Updated weights for policy 0, policy_version 121328 (0.0006) [2024-09-30 02:21:30,050][1157819] Updated weights for policy 0, policy_version 121338 (0.0006) [2024-09-30 02:21:30,466][1157520] Fps is (10 sec: 75366.3, 60 sec: 76458.7, 300 sec: 79934.5). Total num frames: 497029120. Throughput: 0: 18981.5. Samples: 114226676. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 02:21:30,466][1157520] Avg episode reward: [(0, '54.343')] [2024-09-30 02:21:30,587][1157819] Updated weights for policy 0, policy_version 121348 (0.0006) [2024-09-30 02:21:31,141][1157819] Updated weights for policy 0, policy_version 121358 (0.0006) [2024-09-30 02:21:31,684][1157819] Updated weights for policy 0, policy_version 121368 (0.0006) [2024-09-30 02:21:32,278][1157819] Updated weights for policy 0, policy_version 121378 (0.0006) [2024-09-30 02:21:32,280][1157736] Signal inference workers to stop experience collection... (8250 times) [2024-09-30 02:21:32,280][1157736] Signal inference workers to resume experience collection... (8250 times) [2024-09-30 02:21:32,285][1157819] InferenceWorker_p0-w0: stopping experience collection (8250 times) [2024-09-30 02:21:32,285][1157819] InferenceWorker_p0-w0: resuming experience collection (8250 times) [2024-09-30 02:21:32,787][1157819] Updated weights for policy 0, policy_version 121388 (0.0006) [2024-09-30 02:21:33,316][1157819] Updated weights for policy 0, policy_version 121398 (0.0006) [2024-09-30 02:21:33,873][1157819] Updated weights for policy 0, policy_version 121408 (0.0006) [2024-09-30 02:21:34,444][1157819] Updated weights for policy 0, policy_version 121418 (0.0006) [2024-09-30 02:21:34,986][1157819] Updated weights for policy 0, policy_version 121428 (0.0006) [2024-09-30 02:21:35,466][1157520] Fps is (10 sec: 74548.1, 60 sec: 76253.9, 300 sec: 79795.6). Total num frames: 497401856. Throughput: 0: 19038.6. Samples: 114338616. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 02:21:35,466][1157520] Avg episode reward: [(0, '55.874')] [2024-09-30 02:21:35,544][1157819] Updated weights for policy 0, policy_version 121438 (0.0006) [2024-09-30 02:21:36,093][1157819] Updated weights for policy 0, policy_version 121448 (0.0006) [2024-09-30 02:21:36,616][1157819] Updated weights for policy 0, policy_version 121458 (0.0006) [2024-09-30 02:21:37,133][1157819] Updated weights for policy 0, policy_version 121468 (0.0006) [2024-09-30 02:21:37,705][1157819] Updated weights for policy 0, policy_version 121478 (0.0006) [2024-09-30 02:21:38,255][1157819] Updated weights for policy 0, policy_version 121488 (0.0006) [2024-09-30 02:21:38,789][1157819] Updated weights for policy 0, policy_version 121498 (0.0006) [2024-09-30 02:21:39,273][1157819] Updated weights for policy 0, policy_version 121508 (0.0006) [2024-09-30 02:21:39,777][1157819] Updated weights for policy 0, policy_version 121518 (0.0006) [2024-09-30 02:21:40,285][1157819] Updated weights for policy 0, policy_version 121528 (0.0006) [2024-09-30 02:21:40,466][1157520] Fps is (10 sec: 75775.5, 60 sec: 75980.7, 300 sec: 79754.0). Total num frames: 497786880. Throughput: 0: 18990.0. Samples: 114394516. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 02:21:40,466][1157520] Avg episode reward: [(0, '56.456')] [2024-09-30 02:21:40,813][1157819] Updated weights for policy 0, policy_version 121538 (0.0006) [2024-09-30 02:21:41,336][1157819] Updated weights for policy 0, policy_version 121548 (0.0006) [2024-09-30 02:21:41,868][1157819] Updated weights for policy 0, policy_version 121558 (0.0006) [2024-09-30 02:21:42,351][1157819] Updated weights for policy 0, policy_version 121568 (0.0006) [2024-09-30 02:21:42,885][1157819] Updated weights for policy 0, policy_version 121578 (0.0006) [2024-09-30 02:21:43,409][1157819] Updated weights for policy 0, policy_version 121588 (0.0006) [2024-09-30 02:21:43,901][1157819] Updated weights for policy 0, policy_version 121598 (0.0006) [2024-09-30 02:21:44,413][1157819] Updated weights for policy 0, policy_version 121608 (0.0006) [2024-09-30 02:21:44,909][1157819] Updated weights for policy 0, policy_version 121618 (0.0006) [2024-09-30 02:21:45,395][1157819] Updated weights for policy 0, policy_version 121628 (0.0006) [2024-09-30 02:21:45,466][1157520] Fps is (10 sec: 79051.7, 60 sec: 76390.2, 300 sec: 79712.3). Total num frames: 498192384. Throughput: 0: 19010.8. Samples: 114513896. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 02:21:45,466][1157520] Avg episode reward: [(0, '56.043')] [2024-09-30 02:21:45,891][1157819] Updated weights for policy 0, policy_version 121638 (0.0006) [2024-09-30 02:21:46,381][1157819] Updated weights for policy 0, policy_version 121648 (0.0006) [2024-09-30 02:21:46,873][1157819] Updated weights for policy 0, policy_version 121658 (0.0006) [2024-09-30 02:21:47,365][1157819] Updated weights for policy 0, policy_version 121668 (0.0006) [2024-09-30 02:21:47,853][1157819] Updated weights for policy 0, policy_version 121678 (0.0006) [2024-09-30 02:21:48,348][1157819] Updated weights for policy 0, policy_version 121688 (0.0006) [2024-09-30 02:21:48,845][1157819] Updated weights for policy 0, policy_version 121698 (0.0006) [2024-09-30 02:21:49,336][1157819] Updated weights for policy 0, policy_version 121708 (0.0006) [2024-09-30 02:21:49,833][1157819] Updated weights for policy 0, policy_version 121718 (0.0006) [2024-09-30 02:21:50,334][1157819] Updated weights for policy 0, policy_version 121728 (0.0007) [2024-09-30 02:21:50,466][1157520] Fps is (10 sec: 82329.9, 60 sec: 77277.9, 300 sec: 79740.1). Total num frames: 498610176. Throughput: 0: 19160.6. Samples: 114638644. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:21:50,466][1157520] Avg episode reward: [(0, '55.604')] [2024-09-30 02:21:50,825][1157819] Updated weights for policy 0, policy_version 121738 (0.0006) [2024-09-30 02:21:51,290][1157819] Updated weights for policy 0, policy_version 121748 (0.0006) [2024-09-30 02:21:51,792][1157819] Updated weights for policy 0, policy_version 121758 (0.0006) [2024-09-30 02:21:52,300][1157819] Updated weights for policy 0, policy_version 121768 (0.0006) [2024-09-30 02:21:52,792][1157819] Updated weights for policy 0, policy_version 121778 (0.0006) [2024-09-30 02:21:53,306][1157819] Updated weights for policy 0, policy_version 121788 (0.0006) [2024-09-30 02:21:53,807][1157819] Updated weights for policy 0, policy_version 121798 (0.0006) [2024-09-30 02:21:54,305][1157819] Updated weights for policy 0, policy_version 121808 (0.0006) [2024-09-30 02:21:54,844][1157819] Updated weights for policy 0, policy_version 121818 (0.0006) [2024-09-30 02:21:55,380][1157819] Updated weights for policy 0, policy_version 121828 (0.0006) [2024-09-30 02:21:55,466][1157520] Fps is (10 sec: 82330.8, 60 sec: 77482.7, 300 sec: 79698.4). Total num frames: 499015680. Throughput: 0: 19253.2. Samples: 114700576. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:21:55,466][1157520] Avg episode reward: [(0, '54.756')] [2024-09-30 02:21:55,891][1157819] Updated weights for policy 0, policy_version 121838 (0.0006) [2024-09-30 02:21:56,410][1157819] Updated weights for policy 0, policy_version 121848 (0.0006) [2024-09-30 02:21:56,913][1157819] Updated weights for policy 0, policy_version 121858 (0.0006) [2024-09-30 02:21:57,400][1157819] Updated weights for policy 0, policy_version 121868 (0.0006) [2024-09-30 02:21:57,962][1157819] Updated weights for policy 0, policy_version 121878 (0.0006) [2024-09-30 02:21:58,472][1157819] Updated weights for policy 0, policy_version 121888 (0.0006) [2024-09-30 02:21:59,013][1157819] Updated weights for policy 0, policy_version 121898 (0.0006) [2024-09-30 02:21:59,015][1157736] Signal inference workers to stop experience collection... (8300 times) [2024-09-30 02:21:59,015][1157736] Signal inference workers to resume experience collection... (8300 times) [2024-09-30 02:21:59,021][1157819] InferenceWorker_p0-w0: stopping experience collection (8300 times) [2024-09-30 02:21:59,021][1157819] InferenceWorker_p0-w0: resuming experience collection (8300 times) [2024-09-30 02:21:59,506][1157819] Updated weights for policy 0, policy_version 121908 (0.0006) [2024-09-30 02:22:00,006][1157819] Updated weights for policy 0, policy_version 121918 (0.0006) [2024-09-30 02:22:00,466][1157520] Fps is (10 sec: 79872.3, 60 sec: 77346.1, 300 sec: 79545.7). Total num frames: 499408896. Throughput: 0: 19451.4. Samples: 114819432. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:22:00,466][1157520] Avg episode reward: [(0, '54.897')] [2024-09-30 02:22:00,547][1157819] Updated weights for policy 0, policy_version 121928 (0.0006) [2024-09-30 02:22:01,041][1157819] Updated weights for policy 0, policy_version 121938 (0.0006) [2024-09-30 02:22:01,570][1157819] Updated weights for policy 0, policy_version 121948 (0.0006) [2024-09-30 02:22:02,069][1157819] Updated weights for policy 0, policy_version 121958 (0.0006) [2024-09-30 02:22:02,591][1157819] Updated weights for policy 0, policy_version 121968 (0.0006) [2024-09-30 02:22:03,092][1157819] Updated weights for policy 0, policy_version 121978 (0.0006) [2024-09-30 02:22:03,595][1157819] Updated weights for policy 0, policy_version 121988 (0.0006) [2024-09-30 02:22:04,135][1157819] Updated weights for policy 0, policy_version 121998 (0.0006) [2024-09-30 02:22:04,665][1157819] Updated weights for policy 0, policy_version 122008 (0.0006) [2024-09-30 02:22:05,182][1157819] Updated weights for policy 0, policy_version 122018 (0.0006) [2024-09-30 02:22:05,466][1157520] Fps is (10 sec: 79052.8, 60 sec: 77482.7, 300 sec: 79434.6). Total num frames: 499806208. Throughput: 0: 19595.4. Samples: 114938936. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:22:05,466][1157520] Avg episode reward: [(0, '54.669')] [2024-09-30 02:22:05,471][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000122023_499806208.pth... [2024-09-30 02:22:05,517][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000117395_480849920.pth [2024-09-30 02:22:05,754][1157819] Updated weights for policy 0, policy_version 122028 (0.0006) [2024-09-30 02:22:06,292][1157819] Updated weights for policy 0, policy_version 122038 (0.0006) [2024-09-30 02:22:06,786][1157819] Updated weights for policy 0, policy_version 122048 (0.0006) [2024-09-30 02:22:07,281][1157819] Updated weights for policy 0, policy_version 122058 (0.0006) [2024-09-30 02:22:07,799][1157819] Updated weights for policy 0, policy_version 122068 (0.0006) [2024-09-30 02:22:08,356][1157819] Updated weights for policy 0, policy_version 122078 (0.0006) [2024-09-30 02:22:08,858][1157819] Updated weights for policy 0, policy_version 122088 (0.0006) [2024-09-30 02:22:09,360][1157819] Updated weights for policy 0, policy_version 122098 (0.0006) [2024-09-30 02:22:09,864][1157819] Updated weights for policy 0, policy_version 122108 (0.0006) [2024-09-30 02:22:10,362][1157819] Updated weights for policy 0, policy_version 122118 (0.0006) [2024-09-30 02:22:10,466][1157520] Fps is (10 sec: 79052.5, 60 sec: 77823.9, 300 sec: 79406.8). Total num frames: 500199424. Throughput: 0: 19618.5. Samples: 114996984. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:22:10,466][1157520] Avg episode reward: [(0, '54.495')] [2024-09-30 02:22:10,866][1157819] Updated weights for policy 0, policy_version 122128 (0.0006) [2024-09-30 02:22:11,389][1157819] Updated weights for policy 0, policy_version 122138 (0.0006) [2024-09-30 02:22:11,942][1157819] Updated weights for policy 0, policy_version 122148 (0.0006) [2024-09-30 02:22:12,456][1157819] Updated weights for policy 0, policy_version 122158 (0.0006) [2024-09-30 02:22:12,955][1157819] Updated weights for policy 0, policy_version 122168 (0.0006) [2024-09-30 02:22:13,451][1157819] Updated weights for policy 0, policy_version 122178 (0.0006) [2024-09-30 02:22:13,967][1157819] Updated weights for policy 0, policy_version 122188 (0.0006) [2024-09-30 02:22:14,478][1157819] Updated weights for policy 0, policy_version 122198 (0.0006) [2024-09-30 02:22:15,017][1157819] Updated weights for policy 0, policy_version 122208 (0.0006) [2024-09-30 02:22:15,466][1157520] Fps is (10 sec: 79052.9, 60 sec: 78301.9, 300 sec: 79406.8). Total num frames: 500596736. Throughput: 0: 19776.8. Samples: 115116632. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:22:15,466][1157520] Avg episode reward: [(0, '55.154')] [2024-09-30 02:22:15,543][1157819] Updated weights for policy 0, policy_version 122218 (0.0006) [2024-09-30 02:22:16,055][1157819] Updated weights for policy 0, policy_version 122228 (0.0006) [2024-09-30 02:22:16,553][1157819] Updated weights for policy 0, policy_version 122238 (0.0006) [2024-09-30 02:22:17,067][1157819] Updated weights for policy 0, policy_version 122248 (0.0006) [2024-09-30 02:22:17,569][1157819] Updated weights for policy 0, policy_version 122258 (0.0006) [2024-09-30 02:22:18,063][1157819] Updated weights for policy 0, policy_version 122268 (0.0006) [2024-09-30 02:22:18,580][1157819] Updated weights for policy 0, policy_version 122278 (0.0006) [2024-09-30 02:22:19,108][1157819] Updated weights for policy 0, policy_version 122288 (0.0006) [2024-09-30 02:22:19,635][1157819] Updated weights for policy 0, policy_version 122298 (0.0006) [2024-09-30 02:22:20,141][1157819] Updated weights for policy 0, policy_version 122308 (0.0006) [2024-09-30 02:22:20,466][1157520] Fps is (10 sec: 79871.5, 60 sec: 78711.3, 300 sec: 79295.7). Total num frames: 500998144. Throughput: 0: 19953.6. Samples: 115236528. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:22:20,466][1157520] Avg episode reward: [(0, '55.363')] [2024-09-30 02:22:20,650][1157819] Updated weights for policy 0, policy_version 122318 (0.0006) [2024-09-30 02:22:21,179][1157819] Updated weights for policy 0, policy_version 122328 (0.0006) [2024-09-30 02:22:21,694][1157819] Updated weights for policy 0, policy_version 122338 (0.0006) [2024-09-30 02:22:22,271][1157819] Updated weights for policy 0, policy_version 122348 (0.0006) [2024-09-30 02:22:22,807][1157819] Updated weights for policy 0, policy_version 122358 (0.0006) [2024-09-30 02:22:23,358][1157819] Updated weights for policy 0, policy_version 122368 (0.0006) [2024-09-30 02:22:23,921][1157819] Updated weights for policy 0, policy_version 122378 (0.0006) [2024-09-30 02:22:24,459][1157819] Updated weights for policy 0, policy_version 122388 (0.0006) [2024-09-30 02:22:24,981][1157819] Updated weights for policy 0, policy_version 122398 (0.0006) [2024-09-30 02:22:25,466][1157520] Fps is (10 sec: 77824.2, 60 sec: 78643.4, 300 sec: 79073.6). Total num frames: 501374976. Throughput: 0: 19995.1. Samples: 115294292. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:22:25,466][1157520] Avg episode reward: [(0, '54.780')] [2024-09-30 02:22:25,533][1157819] Updated weights for policy 0, policy_version 122408 (0.0006) [2024-09-30 02:22:26,097][1157819] Updated weights for policy 0, policy_version 122418 (0.0006) [2024-09-30 02:22:26,628][1157819] Updated weights for policy 0, policy_version 122428 (0.0006) [2024-09-30 02:22:27,202][1157819] Updated weights for policy 0, policy_version 122438 (0.0006) [2024-09-30 02:22:27,728][1157819] Updated weights for policy 0, policy_version 122448 (0.0006) [2024-09-30 02:22:28,260][1157819] Updated weights for policy 0, policy_version 122458 (0.0006) [2024-09-30 02:22:28,825][1157819] Updated weights for policy 0, policy_version 122468 (0.0006) [2024-09-30 02:22:29,338][1157819] Updated weights for policy 0, policy_version 122478 (0.0006) [2024-09-30 02:22:29,883][1157819] Updated weights for policy 0, policy_version 122488 (0.0006) [2024-09-30 02:22:30,432][1157819] Updated weights for policy 0, policy_version 122498 (0.0006) [2024-09-30 02:22:30,466][1157520] Fps is (10 sec: 75366.7, 60 sec: 78711.4, 300 sec: 79004.2). Total num frames: 501751808. Throughput: 0: 19842.8. Samples: 115406820. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:22:30,466][1157520] Avg episode reward: [(0, '55.595')] [2024-09-30 02:22:30,971][1157819] Updated weights for policy 0, policy_version 122508 (0.0006) [2024-09-30 02:22:31,491][1157819] Updated weights for policy 0, policy_version 122518 (0.0006) [2024-09-30 02:22:32,078][1157819] Updated weights for policy 0, policy_version 122528 (0.0006) [2024-09-30 02:22:32,575][1157819] Updated weights for policy 0, policy_version 122538 (0.0006) [2024-09-30 02:22:33,084][1157819] Updated weights for policy 0, policy_version 122548 (0.0006) [2024-09-30 02:22:33,649][1157819] Updated weights for policy 0, policy_version 122558 (0.0006) [2024-09-30 02:22:34,191][1157819] Updated weights for policy 0, policy_version 122568 (0.0006) [2024-09-30 02:22:34,523][1157736] Signal inference workers to stop experience collection... (8350 times) [2024-09-30 02:22:34,527][1157736] Signal inference workers to resume experience collection... (8350 times) [2024-09-30 02:22:34,528][1157819] InferenceWorker_p0-w0: stopping experience collection (8350 times) [2024-09-30 02:22:34,531][1157819] InferenceWorker_p0-w0: resuming experience collection (8350 times) [2024-09-30 02:22:34,703][1157819] Updated weights for policy 0, policy_version 122578 (0.0006) [2024-09-30 02:22:35,272][1157819] Updated weights for policy 0, policy_version 122588 (0.0006) [2024-09-30 02:22:35,466][1157520] Fps is (10 sec: 75775.6, 60 sec: 78848.0, 300 sec: 79004.2). Total num frames: 502132736. Throughput: 0: 19610.1. Samples: 115521100. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:22:35,466][1157520] Avg episode reward: [(0, '56.759')] [2024-09-30 02:22:35,798][1157819] Updated weights for policy 0, policy_version 122598 (0.0006) [2024-09-30 02:22:36,335][1157819] Updated weights for policy 0, policy_version 122608 (0.0006) [2024-09-30 02:22:36,860][1157819] Updated weights for policy 0, policy_version 122618 (0.0006) [2024-09-30 02:22:37,376][1157819] Updated weights for policy 0, policy_version 122628 (0.0006) [2024-09-30 02:22:37,876][1157819] Updated weights for policy 0, policy_version 122638 (0.0006) [2024-09-30 02:22:38,400][1157819] Updated weights for policy 0, policy_version 122648 (0.0006) [2024-09-30 02:22:38,910][1157819] Updated weights for policy 0, policy_version 122658 (0.0006) [2024-09-30 02:22:39,460][1157819] Updated weights for policy 0, policy_version 122668 (0.0006) [2024-09-30 02:22:39,956][1157819] Updated weights for policy 0, policy_version 122678 (0.0006) [2024-09-30 02:22:40,466][1157520] Fps is (10 sec: 77415.2, 60 sec: 78984.7, 300 sec: 78976.4). Total num frames: 502525952. Throughput: 0: 19529.5. Samples: 115579400. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:22:40,466][1157520] Avg episode reward: [(0, '55.196')] [2024-09-30 02:22:40,479][1157819] Updated weights for policy 0, policy_version 122688 (0.0006) [2024-09-30 02:22:40,980][1157819] Updated weights for policy 0, policy_version 122698 (0.0006) [2024-09-30 02:22:41,474][1157819] Updated weights for policy 0, policy_version 122708 (0.0006) [2024-09-30 02:22:42,030][1157819] Updated weights for policy 0, policy_version 122718 (0.0006) [2024-09-30 02:22:42,555][1157819] Updated weights for policy 0, policy_version 122728 (0.0006) [2024-09-30 02:22:43,058][1157819] Updated weights for policy 0, policy_version 122738 (0.0006) [2024-09-30 02:22:43,562][1157819] Updated weights for policy 0, policy_version 122748 (0.0006) [2024-09-30 02:22:44,101][1157819] Updated weights for policy 0, policy_version 122758 (0.0006) [2024-09-30 02:22:44,667][1157819] Updated weights for policy 0, policy_version 122768 (0.0006) [2024-09-30 02:22:45,174][1157819] Updated weights for policy 0, policy_version 122778 (0.0006) [2024-09-30 02:22:45,466][1157520] Fps is (10 sec: 78642.0, 60 sec: 78779.7, 300 sec: 78893.1). Total num frames: 502919168. Throughput: 0: 19518.0. Samples: 115697744. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:22:45,466][1157520] Avg episode reward: [(0, '57.298')] [2024-09-30 02:22:45,673][1157819] Updated weights for policy 0, policy_version 122788 (0.0006) [2024-09-30 02:22:46,175][1157819] Updated weights for policy 0, policy_version 122798 (0.0006) [2024-09-30 02:22:46,693][1157819] Updated weights for policy 0, policy_version 122808 (0.0006) [2024-09-30 02:22:47,231][1157819] Updated weights for policy 0, policy_version 122818 (0.0006) [2024-09-30 02:22:47,762][1157819] Updated weights for policy 0, policy_version 122828 (0.0006) [2024-09-30 02:22:48,283][1157819] Updated weights for policy 0, policy_version 122838 (0.0006) [2024-09-30 02:22:48,804][1157819] Updated weights for policy 0, policy_version 122848 (0.0006) [2024-09-30 02:22:49,321][1157819] Updated weights for policy 0, policy_version 122858 (0.0006) [2024-09-30 02:22:49,846][1157819] Updated weights for policy 0, policy_version 122868 (0.0006) [2024-09-30 02:22:50,345][1157819] Updated weights for policy 0, policy_version 122878 (0.0006) [2024-09-30 02:22:50,466][1157520] Fps is (10 sec: 79052.2, 60 sec: 78438.4, 300 sec: 78837.6). Total num frames: 503316480. Throughput: 0: 19487.3. Samples: 115815864. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:22:50,466][1157520] Avg episode reward: [(0, '56.498')] [2024-09-30 02:22:50,843][1157819] Updated weights for policy 0, policy_version 122888 (0.0006) [2024-09-30 02:22:51,353][1157819] Updated weights for policy 0, policy_version 122898 (0.0006) [2024-09-30 02:22:51,887][1157819] Updated weights for policy 0, policy_version 122908 (0.0006) [2024-09-30 02:22:52,415][1157819] Updated weights for policy 0, policy_version 122918 (0.0006) [2024-09-30 02:22:52,910][1157819] Updated weights for policy 0, policy_version 122928 (0.0006) [2024-09-30 02:22:53,421][1157819] Updated weights for policy 0, policy_version 122938 (0.0006) [2024-09-30 02:22:53,937][1157819] Updated weights for policy 0, policy_version 122948 (0.0006) [2024-09-30 02:22:54,442][1157819] Updated weights for policy 0, policy_version 122958 (0.0006) [2024-09-30 02:22:54,934][1157819] Updated weights for policy 0, policy_version 122968 (0.0006) [2024-09-30 02:22:55,452][1157819] Updated weights for policy 0, policy_version 122978 (0.0006) [2024-09-30 02:22:55,466][1157520] Fps is (10 sec: 79872.9, 60 sec: 78370.1, 300 sec: 78837.6). Total num frames: 503717888. Throughput: 0: 19534.1. Samples: 115876020. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:22:55,466][1157520] Avg episode reward: [(0, '53.732')] [2024-09-30 02:22:55,955][1157819] Updated weights for policy 0, policy_version 122988 (0.0006) [2024-09-30 02:22:56,494][1157819] Updated weights for policy 0, policy_version 122998 (0.0006) [2024-09-30 02:22:57,030][1157819] Updated weights for policy 0, policy_version 123008 (0.0006) [2024-09-30 02:22:57,527][1157819] Updated weights for policy 0, policy_version 123018 (0.0006) [2024-09-30 02:22:58,029][1157819] Updated weights for policy 0, policy_version 123028 (0.0006) [2024-09-30 02:22:58,532][1157819] Updated weights for policy 0, policy_version 123038 (0.0006) [2024-09-30 02:22:59,035][1157819] Updated weights for policy 0, policy_version 123048 (0.0006) [2024-09-30 02:22:59,533][1157819] Updated weights for policy 0, policy_version 123058 (0.0006) [2024-09-30 02:23:00,166][1157819] Updated weights for policy 0, policy_version 123068 (0.0006) [2024-09-30 02:23:00,466][1157520] Fps is (10 sec: 78643.3, 60 sec: 78233.6, 300 sec: 78823.7). Total num frames: 504102912. Throughput: 0: 19536.1. Samples: 115995756. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:23:00,466][1157520] Avg episode reward: [(0, '56.137')] [2024-09-30 02:23:00,794][1157819] Updated weights for policy 0, policy_version 123078 (0.0006) [2024-09-30 02:23:01,360][1157819] Updated weights for policy 0, policy_version 123088 (0.0006) [2024-09-30 02:23:01,940][1157819] Updated weights for policy 0, policy_version 123098 (0.0006) [2024-09-30 02:23:02,521][1157819] Updated weights for policy 0, policy_version 123108 (0.0006) [2024-09-30 02:23:03,046][1157819] Updated weights for policy 0, policy_version 123118 (0.0006) [2024-09-30 02:23:03,595][1157819] Updated weights for policy 0, policy_version 123128 (0.0006) [2024-09-30 02:23:04,147][1157819] Updated weights for policy 0, policy_version 123138 (0.0006) [2024-09-30 02:23:04,691][1157819] Updated weights for policy 0, policy_version 123148 (0.0006) [2024-09-30 02:23:05,265][1157819] Updated weights for policy 0, policy_version 123158 (0.0006) [2024-09-30 02:23:05,466][1157520] Fps is (10 sec: 74957.3, 60 sec: 77687.5, 300 sec: 78809.8). Total num frames: 504467456. Throughput: 0: 19299.8. Samples: 116105016. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:23:05,466][1157520] Avg episode reward: [(0, '54.670')] [2024-09-30 02:23:05,797][1157819] Updated weights for policy 0, policy_version 123168 (0.0006) [2024-09-30 02:23:06,340][1157819] Updated weights for policy 0, policy_version 123178 (0.0006) [2024-09-30 02:23:06,902][1157819] Updated weights for policy 0, policy_version 123188 (0.0006) [2024-09-30 02:23:07,434][1157819] Updated weights for policy 0, policy_version 123198 (0.0006) [2024-09-30 02:23:08,003][1157819] Updated weights for policy 0, policy_version 123208 (0.0006) [2024-09-30 02:23:08,556][1157819] Updated weights for policy 0, policy_version 123218 (0.0006) [2024-09-30 02:23:09,128][1157819] Updated weights for policy 0, policy_version 123228 (0.0006) [2024-09-30 02:23:09,632][1157819] Updated weights for policy 0, policy_version 123238 (0.0006) [2024-09-30 02:23:10,204][1157819] Updated weights for policy 0, policy_version 123248 (0.0006) [2024-09-30 02:23:10,466][1157520] Fps is (10 sec: 74137.2, 60 sec: 77414.4, 300 sec: 78823.7). Total num frames: 504844288. Throughput: 0: 19257.4. Samples: 116160876. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:23:10,466][1157520] Avg episode reward: [(0, '56.788')] [2024-09-30 02:23:10,735][1157819] Updated weights for policy 0, policy_version 123258 (0.0006) [2024-09-30 02:23:11,285][1157819] Updated weights for policy 0, policy_version 123268 (0.0006) [2024-09-30 02:23:11,802][1157819] Updated weights for policy 0, policy_version 123278 (0.0006) [2024-09-30 02:23:12,330][1157819] Updated weights for policy 0, policy_version 123288 (0.0006) [2024-09-30 02:23:12,828][1157819] Updated weights for policy 0, policy_version 123298 (0.0006) [2024-09-30 02:23:13,356][1157819] Updated weights for policy 0, policy_version 123308 (0.0006) [2024-09-30 02:23:13,875][1157819] Updated weights for policy 0, policy_version 123318 (0.0006) [2024-09-30 02:23:14,375][1157819] Updated weights for policy 0, policy_version 123328 (0.0006) [2024-09-30 02:23:14,873][1157819] Updated weights for policy 0, policy_version 123338 (0.0006) [2024-09-30 02:23:15,367][1157819] Updated weights for policy 0, policy_version 123348 (0.0006) [2024-09-30 02:23:15,466][1157520] Fps is (10 sec: 77004.6, 60 sec: 77346.1, 300 sec: 78920.9). Total num frames: 505237504. Throughput: 0: 19319.4. Samples: 116276192. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:23:15,466][1157520] Avg episode reward: [(0, '53.873')] [2024-09-30 02:23:15,862][1157819] Updated weights for policy 0, policy_version 123358 (0.0006) [2024-09-30 02:23:16,369][1157819] Updated weights for policy 0, policy_version 123368 (0.0006) [2024-09-30 02:23:16,902][1157819] Updated weights for policy 0, policy_version 123378 (0.0006) [2024-09-30 02:23:17,439][1157819] Updated weights for policy 0, policy_version 123388 (0.0006) [2024-09-30 02:23:17,945][1157819] Updated weights for policy 0, policy_version 123398 (0.0006) [2024-09-30 02:23:18,453][1157819] Updated weights for policy 0, policy_version 123408 (0.0006) [2024-09-30 02:23:18,953][1157819] Updated weights for policy 0, policy_version 123418 (0.0006) [2024-09-30 02:23:19,451][1157819] Updated weights for policy 0, policy_version 123428 (0.0006) [2024-09-30 02:23:19,947][1157819] Updated weights for policy 0, policy_version 123438 (0.0006) [2024-09-30 02:23:20,466][1157520] Fps is (10 sec: 79462.7, 60 sec: 77346.2, 300 sec: 78990.3). Total num frames: 505638912. Throughput: 0: 19459.5. Samples: 116396776. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:23:20,466][1157520] Avg episode reward: [(0, '55.409')] [2024-09-30 02:23:20,474][1157819] Updated weights for policy 0, policy_version 123448 (0.0006) [2024-09-30 02:23:20,972][1157819] Updated weights for policy 0, policy_version 123458 (0.0006) [2024-09-30 02:23:21,468][1157819] Updated weights for policy 0, policy_version 123468 (0.0006) [2024-09-30 02:23:22,028][1157819] Updated weights for policy 0, policy_version 123478 (0.0006) [2024-09-30 02:23:22,534][1157819] Updated weights for policy 0, policy_version 123488 (0.0006) [2024-09-30 02:23:23,046][1157819] Updated weights for policy 0, policy_version 123498 (0.0006) [2024-09-30 02:23:23,562][1157819] Updated weights for policy 0, policy_version 123508 (0.0006) [2024-09-30 02:23:24,082][1157819] Updated weights for policy 0, policy_version 123518 (0.0006) [2024-09-30 02:23:24,580][1157819] Updated weights for policy 0, policy_version 123528 (0.0006) [2024-09-30 02:23:25,101][1157819] Updated weights for policy 0, policy_version 123538 (0.0006) [2024-09-30 02:23:25,466][1157520] Fps is (10 sec: 80281.9, 60 sec: 77755.7, 300 sec: 79073.6). Total num frames: 506040320. Throughput: 0: 19497.5. Samples: 116456788. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:23:25,466][1157520] Avg episode reward: [(0, '52.873')] [2024-09-30 02:23:25,611][1157819] Updated weights for policy 0, policy_version 123548 (0.0006) [2024-09-30 02:23:26,145][1157819] Updated weights for policy 0, policy_version 123558 (0.0006) [2024-09-30 02:23:26,663][1157819] Updated weights for policy 0, policy_version 123568 (0.0006) [2024-09-30 02:23:27,172][1157819] Updated weights for policy 0, policy_version 123578 (0.0006) [2024-09-30 02:23:27,711][1157819] Updated weights for policy 0, policy_version 123588 (0.0006) [2024-09-30 02:23:28,259][1157819] Updated weights for policy 0, policy_version 123598 (0.0006) [2024-09-30 02:23:28,759][1157819] Updated weights for policy 0, policy_version 123608 (0.0006) [2024-09-30 02:23:29,279][1157819] Updated weights for policy 0, policy_version 123618 (0.0006) [2024-09-30 02:23:29,788][1157819] Updated weights for policy 0, policy_version 123628 (0.0006) [2024-09-30 02:23:30,342][1157819] Updated weights for policy 0, policy_version 123638 (0.0006) [2024-09-30 02:23:30,466][1157520] Fps is (10 sec: 79462.6, 60 sec: 78028.9, 300 sec: 79143.0). Total num frames: 506433536. Throughput: 0: 19496.8. Samples: 116575096. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:23:30,466][1157520] Avg episode reward: [(0, '55.140')] [2024-09-30 02:23:30,852][1157819] Updated weights for policy 0, policy_version 123648 (0.0006) [2024-09-30 02:23:31,375][1157819] Updated weights for policy 0, policy_version 123658 (0.0006) [2024-09-30 02:23:31,879][1157819] Updated weights for policy 0, policy_version 123668 (0.0006) [2024-09-30 02:23:32,432][1157819] Updated weights for policy 0, policy_version 123678 (0.0006) [2024-09-30 02:23:32,945][1157819] Updated weights for policy 0, policy_version 123688 (0.0006) [2024-09-30 02:23:33,436][1157819] Updated weights for policy 0, policy_version 123698 (0.0006) [2024-09-30 02:23:33,965][1157819] Updated weights for policy 0, policy_version 123708 (0.0006) [2024-09-30 02:23:34,489][1157819] Updated weights for policy 0, policy_version 123718 (0.0006) [2024-09-30 02:23:35,023][1157819] Updated weights for policy 0, policy_version 123728 (0.0006) [2024-09-30 02:23:35,466][1157520] Fps is (10 sec: 78232.7, 60 sec: 78165.2, 300 sec: 79129.1). Total num frames: 506822656. Throughput: 0: 19494.5. Samples: 116693120. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:23:35,466][1157520] Avg episode reward: [(0, '56.812')] [2024-09-30 02:23:35,533][1157819] Updated weights for policy 0, policy_version 123738 (0.0006) [2024-09-30 02:23:36,066][1157819] Updated weights for policy 0, policy_version 123748 (0.0006) [2024-09-30 02:23:36,184][1157736] Signal inference workers to stop experience collection... (8400 times) [2024-09-30 02:23:36,185][1157736] Signal inference workers to resume experience collection... (8400 times) [2024-09-30 02:23:36,190][1157819] InferenceWorker_p0-w0: stopping experience collection (8400 times) [2024-09-30 02:23:36,190][1157819] InferenceWorker_p0-w0: resuming experience collection (8400 times) [2024-09-30 02:23:36,573][1157819] Updated weights for policy 0, policy_version 123758 (0.0006) [2024-09-30 02:23:37,098][1157819] Updated weights for policy 0, policy_version 123768 (0.0006) [2024-09-30 02:23:37,637][1157819] Updated weights for policy 0, policy_version 123778 (0.0006) [2024-09-30 02:23:38,145][1157819] Updated weights for policy 0, policy_version 123788 (0.0006) [2024-09-30 02:23:38,683][1157819] Updated weights for policy 0, policy_version 123798 (0.0006) [2024-09-30 02:23:39,217][1157819] Updated weights for policy 0, policy_version 123808 (0.0006) [2024-09-30 02:23:39,734][1157819] Updated weights for policy 0, policy_version 123818 (0.0006) [2024-09-30 02:23:40,263][1157819] Updated weights for policy 0, policy_version 123828 (0.0006) [2024-09-30 02:23:40,466][1157520] Fps is (10 sec: 77824.1, 60 sec: 78097.0, 300 sec: 79101.4). Total num frames: 507211776. Throughput: 0: 19459.4. Samples: 116751692. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:23:40,466][1157520] Avg episode reward: [(0, '55.921')] [2024-09-30 02:23:40,818][1157819] Updated weights for policy 0, policy_version 123838 (0.0006) [2024-09-30 02:23:41,337][1157819] Updated weights for policy 0, policy_version 123848 (0.0006) [2024-09-30 02:23:41,871][1157819] Updated weights for policy 0, policy_version 123858 (0.0006) [2024-09-30 02:23:42,407][1157819] Updated weights for policy 0, policy_version 123868 (0.0006) [2024-09-30 02:23:43,008][1157819] Updated weights for policy 0, policy_version 123878 (0.0006) [2024-09-30 02:23:43,573][1157819] Updated weights for policy 0, policy_version 123888 (0.0006) [2024-09-30 02:23:44,121][1157819] Updated weights for policy 0, policy_version 123898 (0.0006) [2024-09-30 02:23:44,697][1157819] Updated weights for policy 0, policy_version 123908 (0.0006) [2024-09-30 02:23:45,239][1157819] Updated weights for policy 0, policy_version 123918 (0.0006) [2024-09-30 02:23:45,466][1157520] Fps is (10 sec: 75775.6, 60 sec: 77687.5, 300 sec: 79170.8). Total num frames: 507580416. Throughput: 0: 19318.7. Samples: 116865100. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:23:45,466][1157520] Avg episode reward: [(0, '53.798')] [2024-09-30 02:23:45,801][1157819] Updated weights for policy 0, policy_version 123928 (0.0006) [2024-09-30 02:23:46,350][1157819] Updated weights for policy 0, policy_version 123938 (0.0006) [2024-09-30 02:23:46,973][1157819] Updated weights for policy 0, policy_version 123948 (0.0006) [2024-09-30 02:23:47,537][1157819] Updated weights for policy 0, policy_version 123958 (0.0006) [2024-09-30 02:23:48,099][1157819] Updated weights for policy 0, policy_version 123968 (0.0006) [2024-09-30 02:23:48,685][1157819] Updated weights for policy 0, policy_version 123978 (0.0006) [2024-09-30 02:23:49,252][1157819] Updated weights for policy 0, policy_version 123988 (0.0006) [2024-09-30 02:23:49,852][1157819] Updated weights for policy 0, policy_version 123998 (0.0006) [2024-09-30 02:23:50,425][1157819] Updated weights for policy 0, policy_version 124008 (0.0006) [2024-09-30 02:23:50,466][1157520] Fps is (10 sec: 72499.0, 60 sec: 77004.8, 300 sec: 79129.2). Total num frames: 507936768. Throughput: 0: 19285.4. Samples: 116972860. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:23:50,466][1157520] Avg episode reward: [(0, '54.831')] [2024-09-30 02:23:51,030][1157819] Updated weights for policy 0, policy_version 124018 (0.0006) [2024-09-30 02:23:51,619][1157819] Updated weights for policy 0, policy_version 124028 (0.0006) [2024-09-30 02:23:52,163][1157819] Updated weights for policy 0, policy_version 124038 (0.0006) [2024-09-30 02:23:52,760][1157819] Updated weights for policy 0, policy_version 124048 (0.0006) [2024-09-30 02:23:53,339][1157819] Updated weights for policy 0, policy_version 124058 (0.0006) [2024-09-30 02:23:53,891][1157819] Updated weights for policy 0, policy_version 124068 (0.0006) [2024-09-30 02:23:54,537][1157819] Updated weights for policy 0, policy_version 124078 (0.0007) [2024-09-30 02:23:55,118][1157819] Updated weights for policy 0, policy_version 124088 (0.0006) [2024-09-30 02:23:55,466][1157520] Fps is (10 sec: 70452.0, 60 sec: 76117.4, 300 sec: 78962.6). Total num frames: 508284928. Throughput: 0: 19213.1. Samples: 117025464. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:23:55,466][1157520] Avg episode reward: [(0, '56.355')] [2024-09-30 02:23:55,762][1157819] Updated weights for policy 0, policy_version 124098 (0.0006) [2024-09-30 02:23:56,366][1157819] Updated weights for policy 0, policy_version 124108 (0.0006) [2024-09-30 02:23:56,914][1157819] Updated weights for policy 0, policy_version 124118 (0.0006) [2024-09-30 02:23:57,510][1157819] Updated weights for policy 0, policy_version 124128 (0.0006) [2024-09-30 02:23:58,025][1157819] Updated weights for policy 0, policy_version 124138 (0.0006) [2024-09-30 02:23:58,533][1157819] Updated weights for policy 0, policy_version 124148 (0.0006) [2024-09-30 02:23:59,106][1157819] Updated weights for policy 0, policy_version 124158 (0.0006) [2024-09-30 02:23:59,633][1157819] Updated weights for policy 0, policy_version 124168 (0.0007) [2024-09-30 02:24:00,223][1157819] Updated weights for policy 0, policy_version 124178 (0.0006) [2024-09-30 02:24:00,466][1157520] Fps is (10 sec: 70859.9, 60 sec: 75707.6, 300 sec: 78823.7). Total num frames: 508645376. Throughput: 0: 19019.3. Samples: 117132064. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:24:00,466][1157520] Avg episode reward: [(0, '55.704')] [2024-09-30 02:24:00,817][1157819] Updated weights for policy 0, policy_version 124188 (0.0007) [2024-09-30 02:24:01,406][1157819] Updated weights for policy 0, policy_version 124198 (0.0006) [2024-09-30 02:24:01,983][1157819] Updated weights for policy 0, policy_version 124208 (0.0006) [2024-09-30 02:24:02,554][1157819] Updated weights for policy 0, policy_version 124218 (0.0006) [2024-09-30 02:24:03,145][1157819] Updated weights for policy 0, policy_version 124228 (0.0006) [2024-09-30 02:24:03,727][1157819] Updated weights for policy 0, policy_version 124238 (0.0006) [2024-09-30 02:24:04,276][1157819] Updated weights for policy 0, policy_version 124248 (0.0006) [2024-09-30 02:24:04,870][1157819] Updated weights for policy 0, policy_version 124258 (0.0006) [2024-09-30 02:24:05,447][1157819] Updated weights for policy 0, policy_version 124268 (0.0006) [2024-09-30 02:24:05,466][1157520] Fps is (10 sec: 71678.9, 60 sec: 75571.0, 300 sec: 78462.6). Total num frames: 509001728. Throughput: 0: 18712.2. Samples: 117238828. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:24:05,466][1157520] Avg episode reward: [(0, '57.396')] [2024-09-30 02:24:05,475][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000124269_509005824.pth... [2024-09-30 02:24:05,532][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000119723_490385408.pth [2024-09-30 02:24:05,940][1157819] Updated weights for policy 0, policy_version 124278 (0.0006) [2024-09-30 02:24:06,476][1157819] Updated weights for policy 0, policy_version 124288 (0.0006) [2024-09-30 02:24:06,972][1157819] Updated weights for policy 0, policy_version 124298 (0.0006) [2024-09-30 02:24:07,428][1157819] Updated weights for policy 0, policy_version 124308 (0.0006) [2024-09-30 02:24:07,918][1157819] Updated weights for policy 0, policy_version 124318 (0.0006) [2024-09-30 02:24:08,416][1157819] Updated weights for policy 0, policy_version 124328 (0.0006) [2024-09-30 02:24:08,932][1157819] Updated weights for policy 0, policy_version 124338 (0.0006) [2024-09-30 02:24:09,516][1157819] Updated weights for policy 0, policy_version 124348 (0.0006) [2024-09-30 02:24:10,112][1157819] Updated weights for policy 0, policy_version 124358 (0.0006) [2024-09-30 02:24:10,466][1157520] Fps is (10 sec: 74957.3, 60 sec: 75844.2, 300 sec: 78337.7). Total num frames: 509394944. Throughput: 0: 18710.3. Samples: 117298752. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:24:10,466][1157520] Avg episode reward: [(0, '57.132')] [2024-09-30 02:24:10,695][1157819] Updated weights for policy 0, policy_version 124368 (0.0006) [2024-09-30 02:24:11,244][1157819] Updated weights for policy 0, policy_version 124378 (0.0006) [2024-09-30 02:24:11,864][1157819] Updated weights for policy 0, policy_version 124388 (0.0006) [2024-09-30 02:24:12,414][1157819] Updated weights for policy 0, policy_version 124398 (0.0006) [2024-09-30 02:24:12,987][1157819] Updated weights for policy 0, policy_version 124408 (0.0006) [2024-09-30 02:24:13,540][1157819] Updated weights for policy 0, policy_version 124418 (0.0006) [2024-09-30 02:24:14,105][1157819] Updated weights for policy 0, policy_version 124428 (0.0006) [2024-09-30 02:24:14,618][1157819] Updated weights for policy 0, policy_version 124438 (0.0006) [2024-09-30 02:24:15,178][1157819] Updated weights for policy 0, policy_version 124448 (0.0005) [2024-09-30 02:24:15,466][1157520] Fps is (10 sec: 75776.2, 60 sec: 75366.2, 300 sec: 78143.3). Total num frames: 509759488. Throughput: 0: 18527.7. Samples: 117408848. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:24:15,466][1157520] Avg episode reward: [(0, '54.792')] [2024-09-30 02:24:15,728][1157819] Updated weights for policy 0, policy_version 124458 (0.0006) [2024-09-30 02:24:16,281][1157819] Updated weights for policy 0, policy_version 124468 (0.0006) [2024-09-30 02:24:16,813][1157819] Updated weights for policy 0, policy_version 124478 (0.0006) [2024-09-30 02:24:17,328][1157819] Updated weights for policy 0, policy_version 124488 (0.0006) [2024-09-30 02:24:17,902][1157819] Updated weights for policy 0, policy_version 124498 (0.0006) [2024-09-30 02:24:18,413][1157819] Updated weights for policy 0, policy_version 124508 (0.0006) [2024-09-30 02:24:18,951][1157819] Updated weights for policy 0, policy_version 124518 (0.0006) [2024-09-30 02:24:19,534][1157819] Updated weights for policy 0, policy_version 124528 (0.0006) [2024-09-30 02:24:19,533][1157736] Signal inference workers to stop experience collection... (8450 times) [2024-09-30 02:24:19,534][1157736] Signal inference workers to resume experience collection... (8450 times) [2024-09-30 02:24:19,540][1157819] InferenceWorker_p0-w0: stopping experience collection (8450 times) [2024-09-30 02:24:19,540][1157819] InferenceWorker_p0-w0: resuming experience collection (8450 times) [2024-09-30 02:24:20,076][1157819] Updated weights for policy 0, policy_version 124538 (0.0006) [2024-09-30 02:24:20,466][1157520] Fps is (10 sec: 74137.5, 60 sec: 74956.7, 300 sec: 78046.1). Total num frames: 510136320. Throughput: 0: 18412.5. Samples: 117521680. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:24:20,466][1157520] Avg episode reward: [(0, '55.698')] [2024-09-30 02:24:20,607][1157819] Updated weights for policy 0, policy_version 124548 (0.0006) [2024-09-30 02:24:21,163][1157819] Updated weights for policy 0, policy_version 124558 (0.0006) [2024-09-30 02:24:21,724][1157819] Updated weights for policy 0, policy_version 124568 (0.0006) [2024-09-30 02:24:22,242][1157819] Updated weights for policy 0, policy_version 124578 (0.0006) [2024-09-30 02:24:22,801][1157819] Updated weights for policy 0, policy_version 124588 (0.0006) [2024-09-30 02:24:23,330][1157819] Updated weights for policy 0, policy_version 124598 (0.0006) [2024-09-30 02:24:23,878][1157819] Updated weights for policy 0, policy_version 124608 (0.0006) [2024-09-30 02:24:24,419][1157819] Updated weights for policy 0, policy_version 124618 (0.0006) [2024-09-30 02:24:24,989][1157819] Updated weights for policy 0, policy_version 124628 (0.0006) [2024-09-30 02:24:25,466][1157520] Fps is (10 sec: 74957.4, 60 sec: 74478.8, 300 sec: 77949.0). Total num frames: 510509056. Throughput: 0: 18359.5. Samples: 117577872. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:24:25,466][1157520] Avg episode reward: [(0, '55.342')] [2024-09-30 02:24:25,537][1157819] Updated weights for policy 0, policy_version 124638 (0.0006) [2024-09-30 02:24:26,038][1157819] Updated weights for policy 0, policy_version 124648 (0.0006) [2024-09-30 02:24:26,615][1157819] Updated weights for policy 0, policy_version 124658 (0.0006) [2024-09-30 02:24:27,121][1157819] Updated weights for policy 0, policy_version 124668 (0.0005) [2024-09-30 02:24:27,628][1157819] Updated weights for policy 0, policy_version 124678 (0.0006) [2024-09-30 02:24:28,139][1157819] Updated weights for policy 0, policy_version 124688 (0.0006) [2024-09-30 02:24:28,674][1157819] Updated weights for policy 0, policy_version 124698 (0.0006) [2024-09-30 02:24:29,197][1157819] Updated weights for policy 0, policy_version 124708 (0.0006) [2024-09-30 02:24:29,741][1157819] Updated weights for policy 0, policy_version 124718 (0.0006) [2024-09-30 02:24:30,235][1157819] Updated weights for policy 0, policy_version 124728 (0.0006) [2024-09-30 02:24:30,466][1157520] Fps is (10 sec: 76595.5, 60 sec: 74478.9, 300 sec: 77879.5). Total num frames: 510902272. Throughput: 0: 18395.9. Samples: 117692912. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:24:30,466][1157520] Avg episode reward: [(0, '56.501')] [2024-09-30 02:24:30,759][1157819] Updated weights for policy 0, policy_version 124738 (0.0006) [2024-09-30 02:24:31,276][1157819] Updated weights for policy 0, policy_version 124748 (0.0006) [2024-09-30 02:24:31,799][1157819] Updated weights for policy 0, policy_version 124758 (0.0006) [2024-09-30 02:24:32,355][1157819] Updated weights for policy 0, policy_version 124768 (0.0006) [2024-09-30 02:24:32,862][1157819] Updated weights for policy 0, policy_version 124778 (0.0006) [2024-09-30 02:24:33,380][1157819] Updated weights for policy 0, policy_version 124788 (0.0006) [2024-09-30 02:24:33,958][1157819] Updated weights for policy 0, policy_version 124798 (0.0006) [2024-09-30 02:24:34,499][1157819] Updated weights for policy 0, policy_version 124808 (0.0006) [2024-09-30 02:24:35,069][1157819] Updated weights for policy 0, policy_version 124818 (0.0006) [2024-09-30 02:24:35,466][1157520] Fps is (10 sec: 77414.8, 60 sec: 74342.5, 300 sec: 77796.2). Total num frames: 511283200. Throughput: 0: 18576.1. Samples: 117808784. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:24:35,466][1157520] Avg episode reward: [(0, '56.347')] [2024-09-30 02:24:35,606][1157819] Updated weights for policy 0, policy_version 124828 (0.0006) [2024-09-30 02:24:36,132][1157819] Updated weights for policy 0, policy_version 124838 (0.0006) [2024-09-30 02:24:36,678][1157819] Updated weights for policy 0, policy_version 124848 (0.0006) [2024-09-30 02:24:37,234][1157819] Updated weights for policy 0, policy_version 124858 (0.0006) [2024-09-30 02:24:37,771][1157819] Updated weights for policy 0, policy_version 124868 (0.0006) [2024-09-30 02:24:38,332][1157819] Updated weights for policy 0, policy_version 124878 (0.0006) [2024-09-30 02:24:38,852][1157819] Updated weights for policy 0, policy_version 124888 (0.0006) [2024-09-30 02:24:39,408][1157819] Updated weights for policy 0, policy_version 124898 (0.0006) [2024-09-30 02:24:39,949][1157819] Updated weights for policy 0, policy_version 124908 (0.0006) [2024-09-30 02:24:40,466][1157520] Fps is (10 sec: 75366.8, 60 sec: 74069.3, 300 sec: 77657.4). Total num frames: 511655936. Throughput: 0: 18655.4. Samples: 117864956. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:24:40,466][1157520] Avg episode reward: [(0, '55.343')] [2024-09-30 02:24:40,518][1157819] Updated weights for policy 0, policy_version 124918 (0.0006) [2024-09-30 02:24:41,082][1157819] Updated weights for policy 0, policy_version 124928 (0.0006) [2024-09-30 02:24:41,694][1157819] Updated weights for policy 0, policy_version 124938 (0.0006) [2024-09-30 02:24:42,264][1157819] Updated weights for policy 0, policy_version 124948 (0.0006) [2024-09-30 02:24:42,854][1157819] Updated weights for policy 0, policy_version 124958 (0.0006) [2024-09-30 02:24:43,450][1157819] Updated weights for policy 0, policy_version 124968 (0.0006) [2024-09-30 02:24:44,032][1157819] Updated weights for policy 0, policy_version 124978 (0.0006) [2024-09-30 02:24:44,613][1157819] Updated weights for policy 0, policy_version 124988 (0.0006) [2024-09-30 02:24:45,209][1157819] Updated weights for policy 0, policy_version 124998 (0.0006) [2024-09-30 02:24:45,466][1157520] Fps is (10 sec: 72499.4, 60 sec: 73796.5, 300 sec: 77435.2). Total num frames: 512008192. Throughput: 0: 18694.7. Samples: 117973324. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:24:45,466][1157520] Avg episode reward: [(0, '55.112')] [2024-09-30 02:24:45,774][1157819] Updated weights for policy 0, policy_version 125008 (0.0006) [2024-09-30 02:24:46,300][1157819] Updated weights for policy 0, policy_version 125018 (0.0006) [2024-09-30 02:24:46,836][1157819] Updated weights for policy 0, policy_version 125028 (0.0006) [2024-09-30 02:24:47,339][1157819] Updated weights for policy 0, policy_version 125038 (0.0006) [2024-09-30 02:24:47,867][1157819] Updated weights for policy 0, policy_version 125048 (0.0006) [2024-09-30 02:24:48,374][1157819] Updated weights for policy 0, policy_version 125058 (0.0006) [2024-09-30 02:24:48,884][1157819] Updated weights for policy 0, policy_version 125068 (0.0006) [2024-09-30 02:24:49,392][1157819] Updated weights for policy 0, policy_version 125078 (0.0005) [2024-09-30 02:24:49,933][1157819] Updated weights for policy 0, policy_version 125088 (0.0006) [2024-09-30 02:24:50,466][1157520] Fps is (10 sec: 74137.6, 60 sec: 74342.4, 300 sec: 77282.5). Total num frames: 512397312. Throughput: 0: 18854.5. Samples: 118087276. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:24:50,466][1157520] Avg episode reward: [(0, '55.219')] [2024-09-30 02:24:50,474][1157819] Updated weights for policy 0, policy_version 125098 (0.0006) [2024-09-30 02:24:50,977][1157819] Updated weights for policy 0, policy_version 125108 (0.0006) [2024-09-30 02:24:51,479][1157819] Updated weights for policy 0, policy_version 125118 (0.0006) [2024-09-30 02:24:52,006][1157819] Updated weights for policy 0, policy_version 125128 (0.0006) [2024-09-30 02:24:52,548][1157819] Updated weights for policy 0, policy_version 125138 (0.0006) [2024-09-30 02:24:53,075][1157819] Updated weights for policy 0, policy_version 125148 (0.0006) [2024-09-30 02:24:53,601][1157819] Updated weights for policy 0, policy_version 125158 (0.0006) [2024-09-30 02:24:54,163][1157819] Updated weights for policy 0, policy_version 125168 (0.0006) [2024-09-30 02:24:54,704][1157819] Updated weights for policy 0, policy_version 125178 (0.0006) [2024-09-30 02:24:54,912][1157736] Signal inference workers to stop experience collection... (8500 times) [2024-09-30 02:24:54,915][1157819] InferenceWorker_p0-w0: stopping experience collection (8500 times) [2024-09-30 02:24:54,924][1157736] Signal inference workers to resume experience collection... (8500 times) [2024-09-30 02:24:54,924][1157819] InferenceWorker_p0-w0: resuming experience collection (8500 times) [2024-09-30 02:24:55,410][1157819] Updated weights for policy 0, policy_version 125188 (0.0006) [2024-09-30 02:24:55,466][1157520] Fps is (10 sec: 76185.2, 60 sec: 74752.0, 300 sec: 77254.7). Total num frames: 512770048. Throughput: 0: 18834.8. Samples: 118146316. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 02:24:55,466][1157520] Avg episode reward: [(0, '55.592')] [2024-09-30 02:24:56,120][1157819] Updated weights for policy 0, policy_version 125198 (0.0006) [2024-09-30 02:24:56,756][1157819] Updated weights for policy 0, policy_version 125208 (0.0006) [2024-09-30 02:24:57,390][1157819] Updated weights for policy 0, policy_version 125218 (0.0006) [2024-09-30 02:24:58,033][1157819] Updated weights for policy 0, policy_version 125228 (0.0006) [2024-09-30 02:24:58,639][1157819] Updated weights for policy 0, policy_version 125238 (0.0006) [2024-09-30 02:24:59,293][1157819] Updated weights for policy 0, policy_version 125248 (0.0006) [2024-09-30 02:24:59,931][1157819] Updated weights for policy 0, policy_version 125258 (0.0006) [2024-09-30 02:25:00,466][1157520] Fps is (10 sec: 69222.3, 60 sec: 74069.5, 300 sec: 76963.1). Total num frames: 513089536. Throughput: 0: 18598.2. Samples: 118245764. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 02:25:00,466][1157520] Avg episode reward: [(0, '54.373')] [2024-09-30 02:25:00,558][1157819] Updated weights for policy 0, policy_version 125268 (0.0006) [2024-09-30 02:25:01,167][1157819] Updated weights for policy 0, policy_version 125278 (0.0006) [2024-09-30 02:25:01,672][1157819] Updated weights for policy 0, policy_version 125288 (0.0006) [2024-09-30 02:25:02,174][1157819] Updated weights for policy 0, policy_version 125298 (0.0006) [2024-09-30 02:25:02,673][1157819] Updated weights for policy 0, policy_version 125308 (0.0006) [2024-09-30 02:25:03,192][1157819] Updated weights for policy 0, policy_version 125318 (0.0006) [2024-09-30 02:25:03,681][1157819] Updated weights for policy 0, policy_version 125328 (0.0006) [2024-09-30 02:25:04,180][1157819] Updated weights for policy 0, policy_version 125338 (0.0006) [2024-09-30 02:25:04,683][1157819] Updated weights for policy 0, policy_version 125348 (0.0006) [2024-09-30 02:25:05,187][1157819] Updated weights for policy 0, policy_version 125358 (0.0006) [2024-09-30 02:25:05,466][1157520] Fps is (10 sec: 71681.1, 60 sec: 74752.3, 300 sec: 76949.3). Total num frames: 513486848. Throughput: 0: 18597.8. Samples: 118358576. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 02:25:05,466][1157520] Avg episode reward: [(0, '55.336')] [2024-09-30 02:25:05,698][1157819] Updated weights for policy 0, policy_version 125368 (0.0006) [2024-09-30 02:25:06,199][1157819] Updated weights for policy 0, policy_version 125378 (0.0006) [2024-09-30 02:25:06,686][1157819] Updated weights for policy 0, policy_version 125388 (0.0006) [2024-09-30 02:25:07,182][1157819] Updated weights for policy 0, policy_version 125398 (0.0006) [2024-09-30 02:25:07,740][1157819] Updated weights for policy 0, policy_version 125408 (0.0006) [2024-09-30 02:25:08,259][1157819] Updated weights for policy 0, policy_version 125418 (0.0006) [2024-09-30 02:25:08,748][1157819] Updated weights for policy 0, policy_version 125428 (0.0006) [2024-09-30 02:25:09,260][1157819] Updated weights for policy 0, policy_version 125438 (0.0006) [2024-09-30 02:25:09,800][1157819] Updated weights for policy 0, policy_version 125448 (0.0006) [2024-09-30 02:25:10,305][1157819] Updated weights for policy 0, policy_version 125458 (0.0006) [2024-09-30 02:25:10,466][1157520] Fps is (10 sec: 79871.9, 60 sec: 74888.6, 300 sec: 76949.3). Total num frames: 513888256. Throughput: 0: 18695.1. Samples: 118419152. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 02:25:10,466][1157520] Avg episode reward: [(0, '55.274')] [2024-09-30 02:25:10,804][1157819] Updated weights for policy 0, policy_version 125468 (0.0006) [2024-09-30 02:25:11,334][1157819] Updated weights for policy 0, policy_version 125478 (0.0006) [2024-09-30 02:25:11,849][1157819] Updated weights for policy 0, policy_version 125488 (0.0006) [2024-09-30 02:25:12,353][1157819] Updated weights for policy 0, policy_version 125498 (0.0006) [2024-09-30 02:25:12,867][1157819] Updated weights for policy 0, policy_version 125508 (0.0006) [2024-09-30 02:25:13,376][1157819] Updated weights for policy 0, policy_version 125518 (0.0006) [2024-09-30 02:25:13,900][1157819] Updated weights for policy 0, policy_version 125528 (0.0006) [2024-09-30 02:25:14,389][1157819] Updated weights for policy 0, policy_version 125538 (0.0006) [2024-09-30 02:25:14,917][1157819] Updated weights for policy 0, policy_version 125548 (0.0006) [2024-09-30 02:25:15,439][1157819] Updated weights for policy 0, policy_version 125558 (0.0006) [2024-09-30 02:25:15,466][1157520] Fps is (10 sec: 80280.3, 60 sec: 75503.0, 300 sec: 76893.7). Total num frames: 514289664. Throughput: 0: 18796.9. Samples: 118538772. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 02:25:15,466][1157520] Avg episode reward: [(0, '56.460')] [2024-09-30 02:25:15,945][1157819] Updated weights for policy 0, policy_version 125568 (0.0006) [2024-09-30 02:25:16,497][1157819] Updated weights for policy 0, policy_version 125578 (0.0006) [2024-09-30 02:25:17,015][1157819] Updated weights for policy 0, policy_version 125588 (0.0006) [2024-09-30 02:25:17,500][1157819] Updated weights for policy 0, policy_version 125598 (0.0006) [2024-09-30 02:25:18,024][1157819] Updated weights for policy 0, policy_version 125608 (0.0006) [2024-09-30 02:25:18,538][1157819] Updated weights for policy 0, policy_version 125618 (0.0006) [2024-09-30 02:25:19,045][1157819] Updated weights for policy 0, policy_version 125628 (0.0006) [2024-09-30 02:25:19,599][1157819] Updated weights for policy 0, policy_version 125638 (0.0006) [2024-09-30 02:25:20,110][1157819] Updated weights for policy 0, policy_version 125648 (0.0006) [2024-09-30 02:25:20,421][1157736] Signal inference workers to stop experience collection... (8550 times) [2024-09-30 02:25:20,423][1157736] Signal inference workers to resume experience collection... (8550 times) [2024-09-30 02:25:20,427][1157819] InferenceWorker_p0-w0: stopping experience collection (8550 times) [2024-09-30 02:25:20,429][1157819] InferenceWorker_p0-w0: resuming experience collection (8550 times) [2024-09-30 02:25:20,466][1157520] Fps is (10 sec: 79462.5, 60 sec: 75776.1, 300 sec: 76768.8). Total num frames: 514682880. Throughput: 0: 18862.0. Samples: 118657572. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 02:25:20,466][1157520] Avg episode reward: [(0, '56.283')] [2024-09-30 02:25:20,603][1157819] Updated weights for policy 0, policy_version 125658 (0.0006) [2024-09-30 02:25:21,112][1157819] Updated weights for policy 0, policy_version 125668 (0.0006) [2024-09-30 02:25:21,620][1157819] Updated weights for policy 0, policy_version 125678 (0.0006) [2024-09-30 02:25:22,099][1157819] Updated weights for policy 0, policy_version 125688 (0.0006) [2024-09-30 02:25:22,585][1157819] Updated weights for policy 0, policy_version 125698 (0.0006) [2024-09-30 02:25:23,068][1157819] Updated weights for policy 0, policy_version 125708 (0.0006) [2024-09-30 02:25:23,555][1157819] Updated weights for policy 0, policy_version 125718 (0.0006) [2024-09-30 02:25:24,039][1157819] Updated weights for policy 0, policy_version 125728 (0.0006) [2024-09-30 02:25:24,534][1157819] Updated weights for policy 0, policy_version 125738 (0.0006) [2024-09-30 02:25:25,022][1157819] Updated weights for policy 0, policy_version 125748 (0.0006) [2024-09-30 02:25:25,466][1157520] Fps is (10 sec: 81101.1, 60 sec: 76527.0, 300 sec: 76810.4). Total num frames: 515100672. Throughput: 0: 18976.8. Samples: 118718912. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 02:25:25,466][1157520] Avg episode reward: [(0, '56.000')] [2024-09-30 02:25:25,525][1157819] Updated weights for policy 0, policy_version 125758 (0.0006) [2024-09-30 02:25:26,023][1157819] Updated weights for policy 0, policy_version 125768 (0.0006) [2024-09-30 02:25:26,502][1157819] Updated weights for policy 0, policy_version 125778 (0.0006) [2024-09-30 02:25:26,966][1157819] Updated weights for policy 0, policy_version 125788 (0.0006) [2024-09-30 02:25:27,457][1157819] Updated weights for policy 0, policy_version 125798 (0.0006) [2024-09-30 02:25:27,961][1157819] Updated weights for policy 0, policy_version 125808 (0.0006) [2024-09-30 02:25:28,440][1157819] Updated weights for policy 0, policy_version 125818 (0.0006) [2024-09-30 02:25:28,933][1157819] Updated weights for policy 0, policy_version 125828 (0.0006) [2024-09-30 02:25:29,395][1157819] Updated weights for policy 0, policy_version 125838 (0.0006) [2024-09-30 02:25:29,941][1157819] Updated weights for policy 0, policy_version 125848 (0.0006) [2024-09-30 02:25:30,464][1157819] Updated weights for policy 0, policy_version 125858 (0.0006) [2024-09-30 02:25:30,466][1157520] Fps is (10 sec: 83148.9, 60 sec: 76868.3, 300 sec: 76907.6). Total num frames: 515514368. Throughput: 0: 19373.0. Samples: 118845108. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 02:25:30,466][1157520] Avg episode reward: [(0, '54.858')] [2024-09-30 02:25:30,992][1157819] Updated weights for policy 0, policy_version 125868 (0.0006) [2024-09-30 02:25:31,520][1157819] Updated weights for policy 0, policy_version 125878 (0.0006) [2024-09-30 02:25:32,038][1157819] Updated weights for policy 0, policy_version 125888 (0.0006) [2024-09-30 02:25:32,590][1157819] Updated weights for policy 0, policy_version 125898 (0.0006) [2024-09-30 02:25:33,180][1157819] Updated weights for policy 0, policy_version 125908 (0.0006) [2024-09-30 02:25:33,738][1157819] Updated weights for policy 0, policy_version 125918 (0.0006) [2024-09-30 02:25:34,355][1157819] Updated weights for policy 0, policy_version 125928 (0.0006) [2024-09-30 02:25:34,888][1157819] Updated weights for policy 0, policy_version 125938 (0.0006) [2024-09-30 02:25:35,466][1157520] Fps is (10 sec: 77824.1, 60 sec: 76595.2, 300 sec: 76782.6). Total num frames: 515878912. Throughput: 0: 19358.7. Samples: 118958416. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 02:25:35,466][1157520] Avg episode reward: [(0, '56.815')] [2024-09-30 02:25:35,538][1157819] Updated weights for policy 0, policy_version 125948 (0.0006) [2024-09-30 02:25:36,112][1157819] Updated weights for policy 0, policy_version 125958 (0.0006) [2024-09-30 02:25:36,682][1157819] Updated weights for policy 0, policy_version 125968 (0.0006) [2024-09-30 02:25:37,256][1157819] Updated weights for policy 0, policy_version 125978 (0.0006) [2024-09-30 02:25:37,840][1157819] Updated weights for policy 0, policy_version 125988 (0.0006) [2024-09-30 02:25:38,389][1157819] Updated weights for policy 0, policy_version 125998 (0.0006) [2024-09-30 02:25:38,986][1157819] Updated weights for policy 0, policy_version 126008 (0.0006) [2024-09-30 02:25:39,531][1157819] Updated weights for policy 0, policy_version 126018 (0.0006) [2024-09-30 02:25:40,115][1157819] Updated weights for policy 0, policy_version 126028 (0.0006) [2024-09-30 02:25:40,466][1157520] Fps is (10 sec: 71679.4, 60 sec: 76253.8, 300 sec: 76685.4). Total num frames: 516231168. Throughput: 0: 19215.2. Samples: 119011000. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 02:25:40,466][1157520] Avg episode reward: [(0, '54.170')] [2024-09-30 02:25:40,679][1157819] Updated weights for policy 0, policy_version 126038 (0.0006) [2024-09-30 02:25:41,254][1157819] Updated weights for policy 0, policy_version 126048 (0.0006) [2024-09-30 02:25:41,808][1157819] Updated weights for policy 0, policy_version 126058 (0.0006) [2024-09-30 02:25:42,373][1157819] Updated weights for policy 0, policy_version 126068 (0.0006) [2024-09-30 02:25:42,966][1157819] Updated weights for policy 0, policy_version 126078 (0.0006) [2024-09-30 02:25:43,549][1157819] Updated weights for policy 0, policy_version 126088 (0.0006) [2024-09-30 02:25:44,138][1157819] Updated weights for policy 0, policy_version 126098 (0.0006) [2024-09-30 02:25:44,697][1157819] Updated weights for policy 0, policy_version 126108 (0.0006) [2024-09-30 02:25:45,270][1157819] Updated weights for policy 0, policy_version 126118 (0.0006) [2024-09-30 02:25:45,466][1157520] Fps is (10 sec: 70860.5, 60 sec: 76322.0, 300 sec: 76657.7). Total num frames: 516587520. Throughput: 0: 19397.7. Samples: 119118660. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 02:25:45,466][1157520] Avg episode reward: [(0, '55.079')] [2024-09-30 02:25:45,779][1157736] Signal inference workers to stop experience collection... (8600 times) [2024-09-30 02:25:45,780][1157736] Signal inference workers to resume experience collection... (8600 times) [2024-09-30 02:25:45,783][1157819] InferenceWorker_p0-w0: stopping experience collection (8600 times) [2024-09-30 02:25:45,785][1157819] InferenceWorker_p0-w0: resuming experience collection (8600 times) [2024-09-30 02:25:45,826][1157819] Updated weights for policy 0, policy_version 126128 (0.0007) [2024-09-30 02:25:46,438][1157819] Updated weights for policy 0, policy_version 126138 (0.0006) [2024-09-30 02:25:47,030][1157819] Updated weights for policy 0, policy_version 126148 (0.0006) [2024-09-30 02:25:47,565][1157819] Updated weights for policy 0, policy_version 126158 (0.0006) [2024-09-30 02:25:48,176][1157819] Updated weights for policy 0, policy_version 126168 (0.0006) [2024-09-30 02:25:48,751][1157819] Updated weights for policy 0, policy_version 126178 (0.0006) [2024-09-30 02:25:49,344][1157819] Updated weights for policy 0, policy_version 126188 (0.0006) [2024-09-30 02:25:49,909][1157819] Updated weights for policy 0, policy_version 126198 (0.0006) [2024-09-30 02:25:50,466][1157520] Fps is (10 sec: 71270.8, 60 sec: 75776.0, 300 sec: 76532.7). Total num frames: 516943872. Throughput: 0: 19241.2. Samples: 119224432. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 02:25:50,466][1157520] Avg episode reward: [(0, '53.857')] [2024-09-30 02:25:50,506][1157819] Updated weights for policy 0, policy_version 126208 (0.0006) [2024-09-30 02:25:51,118][1157819] Updated weights for policy 0, policy_version 126218 (0.0006) [2024-09-30 02:25:51,699][1157819] Updated weights for policy 0, policy_version 126228 (0.0006) [2024-09-30 02:25:52,222][1157819] Updated weights for policy 0, policy_version 126238 (0.0006) [2024-09-30 02:25:52,771][1157819] Updated weights for policy 0, policy_version 126248 (0.0006) [2024-09-30 02:25:53,354][1157819] Updated weights for policy 0, policy_version 126258 (0.0006) [2024-09-30 02:25:53,920][1157819] Updated weights for policy 0, policy_version 126268 (0.0006) [2024-09-30 02:25:54,506][1157819] Updated weights for policy 0, policy_version 126278 (0.0006) [2024-09-30 02:25:55,027][1157819] Updated weights for policy 0, policy_version 126288 (0.0006) [2024-09-30 02:25:55,466][1157520] Fps is (10 sec: 71680.0, 60 sec: 75571.2, 300 sec: 76393.8). Total num frames: 517304320. Throughput: 0: 19098.8. Samples: 119278600. Policy #0 lag: (min: 0.0, avg: 2.0, max: 6.0) [2024-09-30 02:25:55,466][1157520] Avg episode reward: [(0, '56.188')] [2024-09-30 02:25:55,608][1157819] Updated weights for policy 0, policy_version 126298 (0.0006) [2024-09-30 02:25:56,156][1157819] Updated weights for policy 0, policy_version 126308 (0.0006) [2024-09-30 02:25:56,709][1157819] Updated weights for policy 0, policy_version 126318 (0.0006) [2024-09-30 02:25:57,264][1157819] Updated weights for policy 0, policy_version 126328 (0.0006) [2024-09-30 02:25:57,835][1157819] Updated weights for policy 0, policy_version 126338 (0.0006) [2024-09-30 02:25:58,400][1157819] Updated weights for policy 0, policy_version 126348 (0.0006) [2024-09-30 02:25:58,940][1157819] Updated weights for policy 0, policy_version 126358 (0.0006) [2024-09-30 02:25:59,487][1157819] Updated weights for policy 0, policy_version 126368 (0.0006) [2024-09-30 02:26:00,054][1157819] Updated weights for policy 0, policy_version 126378 (0.0006) [2024-09-30 02:26:00,466][1157520] Fps is (10 sec: 72908.7, 60 sec: 76390.4, 300 sec: 76324.5). Total num frames: 517672960. Throughput: 0: 18866.5. Samples: 119387764. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 02:26:00,466][1157520] Avg episode reward: [(0, '52.374')] [2024-09-30 02:26:00,594][1157819] Updated weights for policy 0, policy_version 126388 (0.0006) [2024-09-30 02:26:01,097][1157819] Updated weights for policy 0, policy_version 126398 (0.0006) [2024-09-30 02:26:01,641][1157819] Updated weights for policy 0, policy_version 126408 (0.0006) [2024-09-30 02:26:02,156][1157819] Updated weights for policy 0, policy_version 126418 (0.0006) [2024-09-30 02:26:02,680][1157819] Updated weights for policy 0, policy_version 126428 (0.0007) [2024-09-30 02:26:03,176][1157819] Updated weights for policy 0, policy_version 126438 (0.0006) [2024-09-30 02:26:03,672][1157819] Updated weights for policy 0, policy_version 126448 (0.0006) [2024-09-30 02:26:04,177][1157819] Updated weights for policy 0, policy_version 126458 (0.0006) [2024-09-30 02:26:04,728][1157819] Updated weights for policy 0, policy_version 126468 (0.0006) [2024-09-30 02:26:05,261][1157819] Updated weights for policy 0, policy_version 126478 (0.0006) [2024-09-30 02:26:05,466][1157520] Fps is (10 sec: 76594.8, 60 sec: 76390.1, 300 sec: 76407.7). Total num frames: 518070272. Throughput: 0: 18827.8. Samples: 119504824. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 02:26:05,466][1157520] Avg episode reward: [(0, '57.176')] [2024-09-30 02:26:05,487][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000126483_518074368.pth... [2024-09-30 02:26:05,548][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000122023_499806208.pth [2024-09-30 02:26:05,736][1157819] Updated weights for policy 0, policy_version 126488 (0.0006) [2024-09-30 02:26:06,231][1157819] Updated weights for policy 0, policy_version 126498 (0.0006) [2024-09-30 02:26:06,715][1157819] Updated weights for policy 0, policy_version 126508 (0.0006) [2024-09-30 02:26:07,220][1157819] Updated weights for policy 0, policy_version 126518 (0.0006) [2024-09-30 02:26:07,774][1157819] Updated weights for policy 0, policy_version 126528 (0.0006) [2024-09-30 02:26:08,368][1157819] Updated weights for policy 0, policy_version 126538 (0.0006) [2024-09-30 02:26:08,959][1157819] Updated weights for policy 0, policy_version 126548 (0.0006) [2024-09-30 02:26:09,571][1157819] Updated weights for policy 0, policy_version 126558 (0.0006) [2024-09-30 02:26:10,174][1157819] Updated weights for policy 0, policy_version 126568 (0.0006) [2024-09-30 02:26:10,466][1157520] Fps is (10 sec: 77005.1, 60 sec: 75912.6, 300 sec: 76421.6). Total num frames: 518443008. Throughput: 0: 18795.5. Samples: 119564708. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 02:26:10,466][1157520] Avg episode reward: [(0, '55.021')] [2024-09-30 02:26:10,712][1157819] Updated weights for policy 0, policy_version 126578 (0.0006) [2024-09-30 02:26:11,275][1157819] Updated weights for policy 0, policy_version 126588 (0.0006) [2024-09-30 02:26:11,824][1157819] Updated weights for policy 0, policy_version 126598 (0.0006) [2024-09-30 02:26:12,401][1157819] Updated weights for policy 0, policy_version 126608 (0.0006) [2024-09-30 02:26:12,961][1157819] Updated weights for policy 0, policy_version 126618 (0.0006) [2024-09-30 02:26:13,469][1157736] Signal inference workers to stop experience collection... (8650 times) [2024-09-30 02:26:13,469][1157736] Signal inference workers to resume experience collection... (8650 times) [2024-09-30 02:26:13,475][1157819] InferenceWorker_p0-w0: stopping experience collection (8650 times) [2024-09-30 02:26:13,475][1157819] InferenceWorker_p0-w0: resuming experience collection (8650 times) [2024-09-30 02:26:13,484][1157819] Updated weights for policy 0, policy_version 126628 (0.0006) [2024-09-30 02:26:14,092][1157819] Updated weights for policy 0, policy_version 126638 (0.0006) [2024-09-30 02:26:14,619][1157819] Updated weights for policy 0, policy_version 126648 (0.0006) [2024-09-30 02:26:15,175][1157819] Updated weights for policy 0, policy_version 126658 (0.0006) [2024-09-30 02:26:15,466][1157520] Fps is (10 sec: 74137.3, 60 sec: 75366.3, 300 sec: 76393.8). Total num frames: 518811648. Throughput: 0: 18378.4. Samples: 119672140. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 02:26:15,466][1157520] Avg episode reward: [(0, '54.083')] [2024-09-30 02:26:15,709][1157819] Updated weights for policy 0, policy_version 126668 (0.0006) [2024-09-30 02:26:16,279][1157819] Updated weights for policy 0, policy_version 126678 (0.0006) [2024-09-30 02:26:16,821][1157819] Updated weights for policy 0, policy_version 126688 (0.0006) [2024-09-30 02:26:17,377][1157819] Updated weights for policy 0, policy_version 126698 (0.0006) [2024-09-30 02:26:17,939][1157819] Updated weights for policy 0, policy_version 126708 (0.0006) [2024-09-30 02:26:18,468][1157819] Updated weights for policy 0, policy_version 126718 (0.0006) [2024-09-30 02:26:19,041][1157819] Updated weights for policy 0, policy_version 126728 (0.0006) [2024-09-30 02:26:19,549][1157819] Updated weights for policy 0, policy_version 126738 (0.0006) [2024-09-30 02:26:20,069][1157819] Updated weights for policy 0, policy_version 126748 (0.0006) [2024-09-30 02:26:20,466][1157520] Fps is (10 sec: 74136.9, 60 sec: 75025.0, 300 sec: 76366.1). Total num frames: 519184384. Throughput: 0: 18350.1. Samples: 119784172. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 02:26:20,466][1157520] Avg episode reward: [(0, '55.233')] [2024-09-30 02:26:20,658][1157819] Updated weights for policy 0, policy_version 126758 (0.0006) [2024-09-30 02:26:21,170][1157819] Updated weights for policy 0, policy_version 126768 (0.0006) [2024-09-30 02:26:21,746][1157819] Updated weights for policy 0, policy_version 126778 (0.0006) [2024-09-30 02:26:22,254][1157819] Updated weights for policy 0, policy_version 126788 (0.0006) [2024-09-30 02:26:22,751][1157819] Updated weights for policy 0, policy_version 126798 (0.0006) [2024-09-30 02:26:23,266][1157819] Updated weights for policy 0, policy_version 126808 (0.0006) [2024-09-30 02:26:23,764][1157819] Updated weights for policy 0, policy_version 126818 (0.0006) [2024-09-30 02:26:24,279][1157819] Updated weights for policy 0, policy_version 126828 (0.0006) [2024-09-30 02:26:24,805][1157819] Updated weights for policy 0, policy_version 126838 (0.0006) [2024-09-30 02:26:25,352][1157819] Updated weights for policy 0, policy_version 126848 (0.0006) [2024-09-30 02:26:25,466][1157520] Fps is (10 sec: 77005.9, 60 sec: 74683.8, 300 sec: 76449.4). Total num frames: 519581696. Throughput: 0: 18464.6. Samples: 119841908. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 02:26:25,466][1157520] Avg episode reward: [(0, '56.657')] [2024-09-30 02:26:25,838][1157819] Updated weights for policy 0, policy_version 126858 (0.0006) [2024-09-30 02:26:26,365][1157819] Updated weights for policy 0, policy_version 126868 (0.0006) [2024-09-30 02:26:26,859][1157819] Updated weights for policy 0, policy_version 126878 (0.0006) [2024-09-30 02:26:27,396][1157819] Updated weights for policy 0, policy_version 126888 (0.0006) [2024-09-30 02:26:27,916][1157819] Updated weights for policy 0, policy_version 126898 (0.0006) [2024-09-30 02:26:28,441][1157819] Updated weights for policy 0, policy_version 126908 (0.0006) [2024-09-30 02:26:28,918][1157819] Updated weights for policy 0, policy_version 126918 (0.0006) [2024-09-30 02:26:29,434][1157819] Updated weights for policy 0, policy_version 126928 (0.0006) [2024-09-30 02:26:29,965][1157819] Updated weights for policy 0, policy_version 126938 (0.0006) [2024-09-30 02:26:30,466][1157520] Fps is (10 sec: 79053.3, 60 sec: 74342.4, 300 sec: 76518.8). Total num frames: 519974912. Throughput: 0: 18722.1. Samples: 119961152. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 02:26:30,466][1157520] Avg episode reward: [(0, '56.170')] [2024-09-30 02:26:30,515][1157819] Updated weights for policy 0, policy_version 126948 (0.0006) [2024-09-30 02:26:31,019][1157819] Updated weights for policy 0, policy_version 126958 (0.0006) [2024-09-30 02:26:31,516][1157819] Updated weights for policy 0, policy_version 126968 (0.0006) [2024-09-30 02:26:32,040][1157819] Updated weights for policy 0, policy_version 126978 (0.0006) [2024-09-30 02:26:32,573][1157819] Updated weights for policy 0, policy_version 126988 (0.0006) [2024-09-30 02:26:33,105][1157819] Updated weights for policy 0, policy_version 126998 (0.0006) [2024-09-30 02:26:33,621][1157819] Updated weights for policy 0, policy_version 127008 (0.0006) [2024-09-30 02:26:34,156][1157819] Updated weights for policy 0, policy_version 127018 (0.0006) [2024-09-30 02:26:34,681][1157819] Updated weights for policy 0, policy_version 127028 (0.0006) [2024-09-30 02:26:35,193][1157819] Updated weights for policy 0, policy_version 127038 (0.0006) [2024-09-30 02:26:35,466][1157520] Fps is (10 sec: 78642.1, 60 sec: 74820.1, 300 sec: 76546.6). Total num frames: 520368128. Throughput: 0: 18987.6. Samples: 120078876. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 02:26:35,466][1157520] Avg episode reward: [(0, '54.800')] [2024-09-30 02:26:35,707][1157819] Updated weights for policy 0, policy_version 127048 (0.0006) [2024-09-30 02:26:36,255][1157819] Updated weights for policy 0, policy_version 127058 (0.0006) [2024-09-30 02:26:36,789][1157819] Updated weights for policy 0, policy_version 127068 (0.0006) [2024-09-30 02:26:37,315][1157819] Updated weights for policy 0, policy_version 127078 (0.0006) [2024-09-30 02:26:37,854][1157819] Updated weights for policy 0, policy_version 127088 (0.0006) [2024-09-30 02:26:38,352][1157819] Updated weights for policy 0, policy_version 127098 (0.0006) [2024-09-30 02:26:38,857][1157819] Updated weights for policy 0, policy_version 127108 (0.0006) [2024-09-30 02:26:39,348][1157819] Updated weights for policy 0, policy_version 127118 (0.0006) [2024-09-30 02:26:39,911][1157819] Updated weights for policy 0, policy_version 127128 (0.0006) [2024-09-30 02:26:40,423][1157819] Updated weights for policy 0, policy_version 127138 (0.0006) [2024-09-30 02:26:40,466][1157520] Fps is (10 sec: 78643.4, 60 sec: 75503.0, 300 sec: 76505.0). Total num frames: 520761344. Throughput: 0: 19080.9. Samples: 120137240. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 02:26:40,466][1157520] Avg episode reward: [(0, '56.710')] [2024-09-30 02:26:40,903][1157819] Updated weights for policy 0, policy_version 127148 (0.0006) [2024-09-30 02:26:41,386][1157819] Updated weights for policy 0, policy_version 127158 (0.0006) [2024-09-30 02:26:41,926][1157819] Updated weights for policy 0, policy_version 127168 (0.0006) [2024-09-30 02:26:42,408][1157819] Updated weights for policy 0, policy_version 127178 (0.0006) [2024-09-30 02:26:42,879][1157819] Updated weights for policy 0, policy_version 127188 (0.0006) [2024-09-30 02:26:43,378][1157819] Updated weights for policy 0, policy_version 127198 (0.0006) [2024-09-30 02:26:43,873][1157819] Updated weights for policy 0, policy_version 127208 (0.0006) [2024-09-30 02:26:44,362][1157819] Updated weights for policy 0, policy_version 127218 (0.0006) [2024-09-30 02:26:44,856][1157819] Updated weights for policy 0, policy_version 127228 (0.0006) [2024-09-30 02:26:45,347][1157819] Updated weights for policy 0, policy_version 127238 (0.0006) [2024-09-30 02:26:45,466][1157520] Fps is (10 sec: 80692.5, 60 sec: 76458.8, 300 sec: 76491.1). Total num frames: 521175040. Throughput: 0: 19363.3. Samples: 120259112. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 02:26:45,466][1157520] Avg episode reward: [(0, '55.411')] [2024-09-30 02:26:45,814][1157819] Updated weights for policy 0, policy_version 127248 (0.0006) [2024-09-30 02:26:46,311][1157819] Updated weights for policy 0, policy_version 127258 (0.0006) [2024-09-30 02:26:46,803][1157819] Updated weights for policy 0, policy_version 127268 (0.0006) [2024-09-30 02:26:47,298][1157819] Updated weights for policy 0, policy_version 127278 (0.0006) [2024-09-30 02:26:47,767][1157819] Updated weights for policy 0, policy_version 127288 (0.0006) [2024-09-30 02:26:48,316][1157819] Updated weights for policy 0, policy_version 127298 (0.0006) [2024-09-30 02:26:48,845][1157819] Updated weights for policy 0, policy_version 127308 (0.0006) [2024-09-30 02:26:49,398][1157819] Updated weights for policy 0, policy_version 127318 (0.0006) [2024-09-30 02:26:49,931][1157819] Updated weights for policy 0, policy_version 127328 (0.0006) [2024-09-30 02:26:50,466][1157520] Fps is (10 sec: 81100.7, 60 sec: 77141.4, 300 sec: 76463.3). Total num frames: 521572352. Throughput: 0: 19472.8. Samples: 120381100. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 02:26:50,466][1157520] Avg episode reward: [(0, '57.411')] [2024-09-30 02:26:50,483][1157819] Updated weights for policy 0, policy_version 127338 (0.0006) [2024-09-30 02:26:51,031][1157819] Updated weights for policy 0, policy_version 127348 (0.0006) [2024-09-30 02:26:51,569][1157819] Updated weights for policy 0, policy_version 127358 (0.0006) [2024-09-30 02:26:52,172][1157819] Updated weights for policy 0, policy_version 127368 (0.0006) [2024-09-30 02:26:52,691][1157819] Updated weights for policy 0, policy_version 127378 (0.0006) [2024-09-30 02:26:53,232][1157819] Updated weights for policy 0, policy_version 127388 (0.0006) [2024-09-30 02:26:53,499][1157736] Signal inference workers to stop experience collection... (8700 times) [2024-09-30 02:26:53,500][1157736] Signal inference workers to resume experience collection... (8700 times) [2024-09-30 02:26:53,503][1157819] InferenceWorker_p0-w0: stopping experience collection (8700 times) [2024-09-30 02:26:53,503][1157819] InferenceWorker_p0-w0: resuming experience collection (8700 times) [2024-09-30 02:26:53,797][1157819] Updated weights for policy 0, policy_version 127398 (0.0006) [2024-09-30 02:26:54,329][1157819] Updated weights for policy 0, policy_version 127408 (0.0006) [2024-09-30 02:26:54,870][1157819] Updated weights for policy 0, policy_version 127418 (0.0006) [2024-09-30 02:26:55,444][1157819] Updated weights for policy 0, policy_version 127428 (0.0006) [2024-09-30 02:26:55,466][1157520] Fps is (10 sec: 77003.9, 60 sec: 77346.1, 300 sec: 76393.8). Total num frames: 521945088. Throughput: 0: 19374.7. Samples: 120436572. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 02:26:55,466][1157520] Avg episode reward: [(0, '56.547')] [2024-09-30 02:26:55,968][1157819] Updated weights for policy 0, policy_version 127438 (0.0006) [2024-09-30 02:26:56,477][1157819] Updated weights for policy 0, policy_version 127448 (0.0006) [2024-09-30 02:26:56,993][1157819] Updated weights for policy 0, policy_version 127458 (0.0006) [2024-09-30 02:26:57,565][1157819] Updated weights for policy 0, policy_version 127468 (0.0006) [2024-09-30 02:26:58,107][1157819] Updated weights for policy 0, policy_version 127478 (0.0006) [2024-09-30 02:26:58,673][1157819] Updated weights for policy 0, policy_version 127488 (0.0006) [2024-09-30 02:26:59,215][1157819] Updated weights for policy 0, policy_version 127498 (0.0007) [2024-09-30 02:26:59,719][1157819] Updated weights for policy 0, policy_version 127508 (0.0006) [2024-09-30 02:27:00,278][1157819] Updated weights for policy 0, policy_version 127518 (0.0006) [2024-09-30 02:27:00,466][1157520] Fps is (10 sec: 75365.5, 60 sec: 77550.8, 300 sec: 76338.3). Total num frames: 522326016. Throughput: 0: 19512.2. Samples: 120550188. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 02:27:00,466][1157520] Avg episode reward: [(0, '55.576')] [2024-09-30 02:27:00,830][1157819] Updated weights for policy 0, policy_version 127528 (0.0006) [2024-09-30 02:27:01,386][1157819] Updated weights for policy 0, policy_version 127538 (0.0006) [2024-09-30 02:27:01,897][1157819] Updated weights for policy 0, policy_version 127548 (0.0006) [2024-09-30 02:27:02,448][1157819] Updated weights for policy 0, policy_version 127558 (0.0006) [2024-09-30 02:27:03,002][1157819] Updated weights for policy 0, policy_version 127568 (0.0006) [2024-09-30 02:27:03,527][1157819] Updated weights for policy 0, policy_version 127578 (0.0006) [2024-09-30 02:27:04,063][1157819] Updated weights for policy 0, policy_version 127588 (0.0006) [2024-09-30 02:27:04,590][1157819] Updated weights for policy 0, policy_version 127598 (0.0006) [2024-09-30 02:27:05,111][1157819] Updated weights for policy 0, policy_version 127608 (0.0006) [2024-09-30 02:27:05,466][1157520] Fps is (10 sec: 75776.7, 60 sec: 77209.7, 300 sec: 76282.8). Total num frames: 522702848. Throughput: 0: 19550.2. Samples: 120663932. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 02:27:05,466][1157520] Avg episode reward: [(0, '55.697')] [2024-09-30 02:27:05,700][1157819] Updated weights for policy 0, policy_version 127618 (0.0006) [2024-09-30 02:27:06,221][1157819] Updated weights for policy 0, policy_version 127628 (0.0006) [2024-09-30 02:27:06,747][1157819] Updated weights for policy 0, policy_version 127638 (0.0006) [2024-09-30 02:27:07,336][1157819] Updated weights for policy 0, policy_version 127648 (0.0006) [2024-09-30 02:27:07,855][1157819] Updated weights for policy 0, policy_version 127658 (0.0006) [2024-09-30 02:27:08,427][1157819] Updated weights for policy 0, policy_version 127668 (0.0006) [2024-09-30 02:27:08,975][1157819] Updated weights for policy 0, policy_version 127678 (0.0006) [2024-09-30 02:27:09,507][1157819] Updated weights for policy 0, policy_version 127688 (0.0006) [2024-09-30 02:27:10,072][1157819] Updated weights for policy 0, policy_version 127698 (0.0006) [2024-09-30 02:27:10,466][1157520] Fps is (10 sec: 75367.0, 60 sec: 77277.8, 300 sec: 76213.4). Total num frames: 523079680. Throughput: 0: 19513.3. Samples: 120720008. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 02:27:10,466][1157520] Avg episode reward: [(0, '56.867')] [2024-09-30 02:27:10,608][1157819] Updated weights for policy 0, policy_version 127708 (0.0006) [2024-09-30 02:27:11,166][1157819] Updated weights for policy 0, policy_version 127718 (0.0006) [2024-09-30 02:27:11,703][1157819] Updated weights for policy 0, policy_version 127728 (0.0006) [2024-09-30 02:27:12,228][1157819] Updated weights for policy 0, policy_version 127738 (0.0006) [2024-09-30 02:27:12,781][1157819] Updated weights for policy 0, policy_version 127748 (0.0006) [2024-09-30 02:27:13,303][1157819] Updated weights for policy 0, policy_version 127758 (0.0006) [2024-09-30 02:27:13,848][1157819] Updated weights for policy 0, policy_version 127768 (0.0006) [2024-09-30 02:27:14,427][1157819] Updated weights for policy 0, policy_version 127778 (0.0006) [2024-09-30 02:27:14,981][1157819] Updated weights for policy 0, policy_version 127788 (0.0006) [2024-09-30 02:27:15,466][1157520] Fps is (10 sec: 74956.8, 60 sec: 77346.3, 300 sec: 76116.2). Total num frames: 523452416. Throughput: 0: 19375.0. Samples: 120833028. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 02:27:15,466][1157520] Avg episode reward: [(0, '55.081')] [2024-09-30 02:27:15,582][1157819] Updated weights for policy 0, policy_version 127798 (0.0006) [2024-09-30 02:27:16,134][1157819] Updated weights for policy 0, policy_version 127808 (0.0006) [2024-09-30 02:27:16,719][1157819] Updated weights for policy 0, policy_version 127818 (0.0006) [2024-09-30 02:27:17,242][1157819] Updated weights for policy 0, policy_version 127828 (0.0006) [2024-09-30 02:27:17,845][1157819] Updated weights for policy 0, policy_version 127838 (0.0006) [2024-09-30 02:27:18,383][1157819] Updated weights for policy 0, policy_version 127848 (0.0006) [2024-09-30 02:27:18,963][1157819] Updated weights for policy 0, policy_version 127858 (0.0006) [2024-09-30 02:27:19,505][1157819] Updated weights for policy 0, policy_version 127868 (0.0006) [2024-09-30 02:27:20,031][1157819] Updated weights for policy 0, policy_version 127878 (0.0006) [2024-09-30 02:27:20,466][1157520] Fps is (10 sec: 73728.2, 60 sec: 77209.7, 300 sec: 76074.5). Total num frames: 523816960. Throughput: 0: 19188.8. Samples: 120942368. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 02:27:20,466][1157520] Avg episode reward: [(0, '55.837')] [2024-09-30 02:27:20,570][1157819] Updated weights for policy 0, policy_version 127888 (0.0006) [2024-09-30 02:27:21,145][1157819] Updated weights for policy 0, policy_version 127898 (0.0006) [2024-09-30 02:27:21,669][1157819] Updated weights for policy 0, policy_version 127908 (0.0006) [2024-09-30 02:27:22,225][1157819] Updated weights for policy 0, policy_version 127918 (0.0006) [2024-09-30 02:27:22,758][1157819] Updated weights for policy 0, policy_version 127928 (0.0006) [2024-09-30 02:27:23,270][1157819] Updated weights for policy 0, policy_version 127938 (0.0006) [2024-09-30 02:27:23,779][1157819] Updated weights for policy 0, policy_version 127948 (0.0006) [2024-09-30 02:27:24,292][1157819] Updated weights for policy 0, policy_version 127958 (0.0006) [2024-09-30 02:27:24,790][1157819] Updated weights for policy 0, policy_version 127968 (0.0006) [2024-09-30 02:27:25,339][1157819] Updated weights for policy 0, policy_version 127978 (0.0006) [2024-09-30 02:27:25,466][1157520] Fps is (10 sec: 75366.4, 60 sec: 77073.0, 300 sec: 76116.2). Total num frames: 524206080. Throughput: 0: 19149.6. Samples: 120998972. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 02:27:25,466][1157520] Avg episode reward: [(0, '56.082')] [2024-09-30 02:27:25,843][1157819] Updated weights for policy 0, policy_version 127988 (0.0006) [2024-09-30 02:27:26,358][1157819] Updated weights for policy 0, policy_version 127998 (0.0006) [2024-09-30 02:27:26,893][1157819] Updated weights for policy 0, policy_version 128008 (0.0006) [2024-09-30 02:27:27,391][1157819] Updated weights for policy 0, policy_version 128018 (0.0006) [2024-09-30 02:27:27,939][1157819] Updated weights for policy 0, policy_version 128028 (0.0006) [2024-09-30 02:27:28,468][1157819] Updated weights for policy 0, policy_version 128038 (0.0006) [2024-09-30 02:27:28,963][1157819] Updated weights for policy 0, policy_version 128048 (0.0006) [2024-09-30 02:27:29,500][1157819] Updated weights for policy 0, policy_version 128058 (0.0006) [2024-09-30 02:27:30,041][1157819] Updated weights for policy 0, policy_version 128068 (0.0006) [2024-09-30 02:27:30,466][1157520] Fps is (10 sec: 77823.7, 60 sec: 77004.8, 300 sec: 76143.9). Total num frames: 524595200. Throughput: 0: 19074.3. Samples: 121117456. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 02:27:30,466][1157520] Avg episode reward: [(0, '56.381')] [2024-09-30 02:27:30,550][1157819] Updated weights for policy 0, policy_version 128078 (0.0006) [2024-09-30 02:27:31,062][1157819] Updated weights for policy 0, policy_version 128088 (0.0006) [2024-09-30 02:27:31,625][1157819] Updated weights for policy 0, policy_version 128098 (0.0006) [2024-09-30 02:27:32,152][1157819] Updated weights for policy 0, policy_version 128108 (0.0006) [2024-09-30 02:27:32,655][1157819] Updated weights for policy 0, policy_version 128118 (0.0006) [2024-09-30 02:27:33,165][1157819] Updated weights for policy 0, policy_version 128128 (0.0006) [2024-09-30 02:27:33,363][1157736] Signal inference workers to stop experience collection... (8750 times) [2024-09-30 02:27:33,364][1157736] Signal inference workers to resume experience collection... (8750 times) [2024-09-30 02:27:33,370][1157819] InferenceWorker_p0-w0: stopping experience collection (8750 times) [2024-09-30 02:27:33,370][1157819] InferenceWorker_p0-w0: resuming experience collection (8750 times) [2024-09-30 02:27:33,688][1157819] Updated weights for policy 0, policy_version 128138 (0.0006) [2024-09-30 02:27:34,232][1157819] Updated weights for policy 0, policy_version 128148 (0.0006) [2024-09-30 02:27:34,768][1157819] Updated weights for policy 0, policy_version 128158 (0.0006) [2024-09-30 02:27:35,342][1157819] Updated weights for policy 0, policy_version 128168 (0.0006) [2024-09-30 02:27:35,466][1157520] Fps is (10 sec: 78233.5, 60 sec: 77005.0, 300 sec: 76143.9). Total num frames: 524988416. Throughput: 0: 18957.9. Samples: 121234208. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 02:27:35,466][1157520] Avg episode reward: [(0, '56.909')] [2024-09-30 02:27:35,833][1157819] Updated weights for policy 0, policy_version 128178 (0.0006) [2024-09-30 02:27:36,325][1157819] Updated weights for policy 0, policy_version 128188 (0.0006) [2024-09-30 02:27:36,848][1157819] Updated weights for policy 0, policy_version 128198 (0.0006) [2024-09-30 02:27:37,377][1157819] Updated weights for policy 0, policy_version 128208 (0.0006) [2024-09-30 02:27:37,873][1157819] Updated weights for policy 0, policy_version 128218 (0.0006) [2024-09-30 02:27:38,418][1157819] Updated weights for policy 0, policy_version 128228 (0.0006) [2024-09-30 02:27:38,917][1157819] Updated weights for policy 0, policy_version 128238 (0.0006) [2024-09-30 02:27:39,460][1157819] Updated weights for policy 0, policy_version 128248 (0.0006) [2024-09-30 02:27:39,978][1157819] Updated weights for policy 0, policy_version 128258 (0.0006) [2024-09-30 02:27:40,466][1157520] Fps is (10 sec: 78643.5, 60 sec: 77004.8, 300 sec: 76144.0). Total num frames: 525381632. Throughput: 0: 19033.3. Samples: 121293068. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 02:27:40,466][1157520] Avg episode reward: [(0, '57.358')] [2024-09-30 02:27:40,470][1157819] Updated weights for policy 0, policy_version 128268 (0.0006) [2024-09-30 02:27:40,963][1157819] Updated weights for policy 0, policy_version 128278 (0.0006) [2024-09-30 02:27:41,518][1157819] Updated weights for policy 0, policy_version 128288 (0.0006) [2024-09-30 02:27:42,067][1157819] Updated weights for policy 0, policy_version 128298 (0.0006) [2024-09-30 02:27:42,588][1157819] Updated weights for policy 0, policy_version 128308 (0.0006) [2024-09-30 02:27:43,102][1157819] Updated weights for policy 0, policy_version 128318 (0.0006) [2024-09-30 02:27:43,628][1157819] Updated weights for policy 0, policy_version 128328 (0.0006) [2024-09-30 02:27:44,142][1157819] Updated weights for policy 0, policy_version 128338 (0.0006) [2024-09-30 02:27:44,685][1157819] Updated weights for policy 0, policy_version 128348 (0.0006) [2024-09-30 02:27:45,139][1157819] Updated weights for policy 0, policy_version 128358 (0.0006) [2024-09-30 02:27:45,466][1157520] Fps is (10 sec: 78642.8, 60 sec: 76663.3, 300 sec: 76130.0). Total num frames: 525774848. Throughput: 0: 19129.5. Samples: 121411016. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 02:27:45,466][1157520] Avg episode reward: [(0, '55.863')] [2024-09-30 02:27:45,692][1157819] Updated weights for policy 0, policy_version 128368 (0.0006) [2024-09-30 02:27:46,206][1157819] Updated weights for policy 0, policy_version 128378 (0.0006) [2024-09-30 02:27:46,703][1157819] Updated weights for policy 0, policy_version 128388 (0.0006) [2024-09-30 02:27:47,241][1157819] Updated weights for policy 0, policy_version 128398 (0.0006) [2024-09-30 02:27:47,739][1157819] Updated weights for policy 0, policy_version 128408 (0.0006) [2024-09-30 02:27:48,263][1157819] Updated weights for policy 0, policy_version 128418 (0.0006) [2024-09-30 02:27:48,770][1157819] Updated weights for policy 0, policy_version 128428 (0.0006) [2024-09-30 02:27:49,347][1157819] Updated weights for policy 0, policy_version 128438 (0.0006) [2024-09-30 02:27:49,829][1157819] Updated weights for policy 0, policy_version 128448 (0.0006) [2024-09-30 02:27:50,335][1157819] Updated weights for policy 0, policy_version 128458 (0.0006) [2024-09-30 02:27:50,466][1157520] Fps is (10 sec: 78643.3, 60 sec: 76595.2, 300 sec: 76102.3). Total num frames: 526168064. Throughput: 0: 19242.6. Samples: 121529848. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 02:27:50,466][1157520] Avg episode reward: [(0, '56.640')] [2024-09-30 02:27:50,876][1157819] Updated weights for policy 0, policy_version 128468 (0.0006) [2024-09-30 02:27:51,381][1157819] Updated weights for policy 0, policy_version 128478 (0.0006) [2024-09-30 02:27:51,908][1157819] Updated weights for policy 0, policy_version 128488 (0.0006) [2024-09-30 02:27:52,431][1157819] Updated weights for policy 0, policy_version 128498 (0.0006) [2024-09-30 02:27:52,974][1157819] Updated weights for policy 0, policy_version 128508 (0.0006) [2024-09-30 02:27:53,492][1157819] Updated weights for policy 0, policy_version 128518 (0.0006) [2024-09-30 02:27:54,025][1157819] Updated weights for policy 0, policy_version 128528 (0.0006) [2024-09-30 02:27:54,539][1157819] Updated weights for policy 0, policy_version 128538 (0.0006) [2024-09-30 02:27:55,102][1157819] Updated weights for policy 0, policy_version 128548 (0.0007) [2024-09-30 02:27:55,466][1157520] Fps is (10 sec: 77824.4, 60 sec: 76800.1, 300 sec: 76102.3). Total num frames: 526553088. Throughput: 0: 19307.3. Samples: 121588836. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 02:27:55,466][1157520] Avg episode reward: [(0, '54.246')] [2024-09-30 02:27:55,647][1157819] Updated weights for policy 0, policy_version 128558 (0.0006) [2024-09-30 02:27:56,211][1157819] Updated weights for policy 0, policy_version 128568 (0.0006) [2024-09-30 02:27:56,774][1157819] Updated weights for policy 0, policy_version 128578 (0.0006) [2024-09-30 02:27:57,337][1157819] Updated weights for policy 0, policy_version 128588 (0.0006) [2024-09-30 02:27:57,876][1157819] Updated weights for policy 0, policy_version 128598 (0.0006) [2024-09-30 02:27:58,429][1157819] Updated weights for policy 0, policy_version 128608 (0.0006) [2024-09-30 02:27:58,972][1157819] Updated weights for policy 0, policy_version 128618 (0.0006) [2024-09-30 02:27:59,506][1157819] Updated weights for policy 0, policy_version 128628 (0.0006) [2024-09-30 02:28:00,068][1157819] Updated weights for policy 0, policy_version 128638 (0.0006) [2024-09-30 02:28:00,466][1157520] Fps is (10 sec: 76184.9, 60 sec: 76731.8, 300 sec: 76143.9). Total num frames: 526929920. Throughput: 0: 19303.7. Samples: 121701696. Policy #0 lag: (min: 1.0, avg: 2.4, max: 6.0) [2024-09-30 02:28:00,466][1157520] Avg episode reward: [(0, '56.795')] [2024-09-30 02:28:00,625][1157819] Updated weights for policy 0, policy_version 128648 (0.0006) [2024-09-30 02:28:01,189][1157819] Updated weights for policy 0, policy_version 128658 (0.0006) [2024-09-30 02:28:01,801][1157819] Updated weights for policy 0, policy_version 128668 (0.0006) [2024-09-30 02:28:02,468][1157819] Updated weights for policy 0, policy_version 128678 (0.0006) [2024-09-30 02:28:03,046][1157819] Updated weights for policy 0, policy_version 128688 (0.0006) [2024-09-30 02:28:03,659][1157819] Updated weights for policy 0, policy_version 128698 (0.0006) [2024-09-30 02:28:04,238][1157819] Updated weights for policy 0, policy_version 128708 (0.0006) [2024-09-30 02:28:04,812][1157819] Updated weights for policy 0, policy_version 128718 (0.0006) [2024-09-30 02:28:05,401][1157819] Updated weights for policy 0, policy_version 128728 (0.0006) [2024-09-30 02:28:05,466][1157520] Fps is (10 sec: 72089.2, 60 sec: 76185.5, 300 sec: 76032.9). Total num frames: 527273984. Throughput: 0: 19210.5. Samples: 121806840. Policy #0 lag: (min: 1.0, avg: 2.4, max: 6.0) [2024-09-30 02:28:05,466][1157520] Avg episode reward: [(0, '55.605')] [2024-09-30 02:28:05,473][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000128729_527273984.pth... [2024-09-30 02:28:05,536][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000124269_509005824.pth [2024-09-30 02:28:06,021][1157819] Updated weights for policy 0, policy_version 128738 (0.0006) [2024-09-30 02:28:06,621][1157819] Updated weights for policy 0, policy_version 128748 (0.0006) [2024-09-30 02:28:07,242][1157819] Updated weights for policy 0, policy_version 128758 (0.0006) [2024-09-30 02:28:07,843][1157819] Updated weights for policy 0, policy_version 128768 (0.0006) [2024-09-30 02:28:08,507][1157819] Updated weights for policy 0, policy_version 128778 (0.0006) [2024-09-30 02:28:09,090][1157819] Updated weights for policy 0, policy_version 128788 (0.0006) [2024-09-30 02:28:09,357][1157736] Signal inference workers to stop experience collection... (8800 times) [2024-09-30 02:28:09,358][1157736] Signal inference workers to resume experience collection... (8800 times) [2024-09-30 02:28:09,363][1157819] InferenceWorker_p0-w0: stopping experience collection (8800 times) [2024-09-30 02:28:09,363][1157819] InferenceWorker_p0-w0: resuming experience collection (8800 times) [2024-09-30 02:28:09,685][1157819] Updated weights for policy 0, policy_version 128798 (0.0006) [2024-09-30 02:28:10,298][1157819] Updated weights for policy 0, policy_version 128808 (0.0006) [2024-09-30 02:28:10,466][1157520] Fps is (10 sec: 67584.6, 60 sec: 75434.7, 300 sec: 75824.6). Total num frames: 527605760. Throughput: 0: 19074.9. Samples: 121857340. Policy #0 lag: (min: 1.0, avg: 2.4, max: 6.0) [2024-09-30 02:28:10,466][1157520] Avg episode reward: [(0, '56.073')] [2024-09-30 02:28:10,900][1157819] Updated weights for policy 0, policy_version 128818 (0.0006) [2024-09-30 02:28:11,464][1157819] Updated weights for policy 0, policy_version 128828 (0.0006) [2024-09-30 02:28:12,023][1157819] Updated weights for policy 0, policy_version 128838 (0.0006) [2024-09-30 02:28:12,633][1157819] Updated weights for policy 0, policy_version 128848 (0.0006) [2024-09-30 02:28:13,238][1157819] Updated weights for policy 0, policy_version 128858 (0.0006) [2024-09-30 02:28:13,795][1157819] Updated weights for policy 0, policy_version 128868 (0.0006) [2024-09-30 02:28:14,394][1157819] Updated weights for policy 0, policy_version 128878 (0.0006) [2024-09-30 02:28:14,968][1157819] Updated weights for policy 0, policy_version 128888 (0.0006) [2024-09-30 02:28:15,466][1157520] Fps is (10 sec: 68403.6, 60 sec: 75093.3, 300 sec: 75658.0). Total num frames: 527958016. Throughput: 0: 18740.6. Samples: 121960784. Policy #0 lag: (min: 1.0, avg: 2.4, max: 6.0) [2024-09-30 02:28:15,466][1157520] Avg episode reward: [(0, '56.740')] [2024-09-30 02:28:15,553][1157819] Updated weights for policy 0, policy_version 128898 (0.0006) [2024-09-30 02:28:16,143][1157819] Updated weights for policy 0, policy_version 128908 (0.0006) [2024-09-30 02:28:16,714][1157819] Updated weights for policy 0, policy_version 128918 (0.0006) [2024-09-30 02:28:17,282][1157819] Updated weights for policy 0, policy_version 128928 (0.0006) [2024-09-30 02:28:17,866][1157819] Updated weights for policy 0, policy_version 128938 (0.0006) [2024-09-30 02:28:18,482][1157819] Updated weights for policy 0, policy_version 128948 (0.0006) [2024-09-30 02:28:19,062][1157819] Updated weights for policy 0, policy_version 128958 (0.0006) [2024-09-30 02:28:19,678][1157819] Updated weights for policy 0, policy_version 128968 (0.0006) [2024-09-30 02:28:20,282][1157819] Updated weights for policy 0, policy_version 128978 (0.0006) [2024-09-30 02:28:20,466][1157520] Fps is (10 sec: 70041.6, 60 sec: 74820.3, 300 sec: 75477.5). Total num frames: 528306176. Throughput: 0: 18471.6. Samples: 122065428. Policy #0 lag: (min: 1.0, avg: 2.4, max: 6.0) [2024-09-30 02:28:20,466][1157520] Avg episode reward: [(0, '54.625')] [2024-09-30 02:28:20,842][1157819] Updated weights for policy 0, policy_version 128988 (0.0006) [2024-09-30 02:28:21,448][1157819] Updated weights for policy 0, policy_version 128998 (0.0006) [2024-09-30 02:28:22,042][1157819] Updated weights for policy 0, policy_version 129008 (0.0006) [2024-09-30 02:28:22,637][1157819] Updated weights for policy 0, policy_version 129018 (0.0006) [2024-09-30 02:28:23,215][1157819] Updated weights for policy 0, policy_version 129028 (0.0006) [2024-09-30 02:28:23,821][1157819] Updated weights for policy 0, policy_version 129038 (0.0006) [2024-09-30 02:28:24,414][1157819] Updated weights for policy 0, policy_version 129048 (0.0006) [2024-09-30 02:28:25,038][1157819] Updated weights for policy 0, policy_version 129058 (0.0006) [2024-09-30 02:28:25,466][1157520] Fps is (10 sec: 69222.1, 60 sec: 74069.3, 300 sec: 75310.8). Total num frames: 528650240. Throughput: 0: 18314.4. Samples: 122117216. Policy #0 lag: (min: 1.0, avg: 2.4, max: 6.0) [2024-09-30 02:28:25,466][1157520] Avg episode reward: [(0, '57.908')] [2024-09-30 02:28:25,609][1157819] Updated weights for policy 0, policy_version 129068 (0.0006) [2024-09-30 02:28:26,226][1157819] Updated weights for policy 0, policy_version 129078 (0.0006) [2024-09-30 02:28:26,771][1157819] Updated weights for policy 0, policy_version 129088 (0.0006) [2024-09-30 02:28:27,306][1157819] Updated weights for policy 0, policy_version 129098 (0.0006) [2024-09-30 02:28:27,866][1157819] Updated weights for policy 0, policy_version 129108 (0.0006) [2024-09-30 02:28:28,416][1157819] Updated weights for policy 0, policy_version 129118 (0.0006) [2024-09-30 02:28:28,954][1157819] Updated weights for policy 0, policy_version 129128 (0.0006) [2024-09-30 02:28:29,535][1157819] Updated weights for policy 0, policy_version 129138 (0.0006) [2024-09-30 02:28:30,076][1157819] Updated weights for policy 0, policy_version 129148 (0.0006) [2024-09-30 02:28:30,466][1157520] Fps is (10 sec: 71269.9, 60 sec: 73728.0, 300 sec: 75241.4). Total num frames: 529018880. Throughput: 0: 18064.7. Samples: 122223928. Policy #0 lag: (min: 1.0, avg: 2.4, max: 6.0) [2024-09-30 02:28:30,466][1157520] Avg episode reward: [(0, '54.232')] [2024-09-30 02:28:30,606][1157819] Updated weights for policy 0, policy_version 129158 (0.0006) [2024-09-30 02:28:31,191][1157819] Updated weights for policy 0, policy_version 129168 (0.0006) [2024-09-30 02:28:31,767][1157819] Updated weights for policy 0, policy_version 129178 (0.0007) [2024-09-30 02:28:32,352][1157819] Updated weights for policy 0, policy_version 129188 (0.0006) [2024-09-30 02:28:32,964][1157819] Updated weights for policy 0, policy_version 129198 (0.0006) [2024-09-30 02:28:33,520][1157819] Updated weights for policy 0, policy_version 129208 (0.0006) [2024-09-30 02:28:34,121][1157819] Updated weights for policy 0, policy_version 129218 (0.0006) [2024-09-30 02:28:34,685][1157819] Updated weights for policy 0, policy_version 129228 (0.0006) [2024-09-30 02:28:35,205][1157819] Updated weights for policy 0, policy_version 129238 (0.0006) [2024-09-30 02:28:35,466][1157520] Fps is (10 sec: 72499.4, 60 sec: 73113.6, 300 sec: 75130.3). Total num frames: 529375232. Throughput: 0: 17822.2. Samples: 122331848. Policy #0 lag: (min: 1.0, avg: 2.4, max: 6.0) [2024-09-30 02:28:35,466][1157520] Avg episode reward: [(0, '55.157')] [2024-09-30 02:28:35,530][1157736] Signal inference workers to stop experience collection... (8850 times) [2024-09-30 02:28:35,530][1157736] Signal inference workers to resume experience collection... (8850 times) [2024-09-30 02:28:35,534][1157819] InferenceWorker_p0-w0: stopping experience collection (8850 times) [2024-09-30 02:28:35,534][1157819] InferenceWorker_p0-w0: resuming experience collection (8850 times) [2024-09-30 02:28:35,739][1157819] Updated weights for policy 0, policy_version 129248 (0.0006) [2024-09-30 02:28:36,403][1157819] Updated weights for policy 0, policy_version 129258 (0.0006) [2024-09-30 02:28:37,004][1157819] Updated weights for policy 0, policy_version 129268 (0.0006) [2024-09-30 02:28:37,645][1157819] Updated weights for policy 0, policy_version 129278 (0.0006) [2024-09-30 02:28:38,304][1157819] Updated weights for policy 0, policy_version 129288 (0.0006) [2024-09-30 02:28:38,903][1157819] Updated weights for policy 0, policy_version 129298 (0.0006) [2024-09-30 02:28:39,504][1157819] Updated weights for policy 0, policy_version 129308 (0.0006) [2024-09-30 02:28:40,110][1157819] Updated weights for policy 0, policy_version 129318 (0.0006) [2024-09-30 02:28:40,466][1157520] Fps is (10 sec: 68812.7, 60 sec: 72089.5, 300 sec: 75005.4). Total num frames: 529707008. Throughput: 0: 17651.5. Samples: 122383156. Policy #0 lag: (min: 1.0, avg: 2.4, max: 6.0) [2024-09-30 02:28:40,466][1157520] Avg episode reward: [(0, '56.086')] [2024-09-30 02:28:40,728][1157819] Updated weights for policy 0, policy_version 129328 (0.0006) [2024-09-30 02:28:41,342][1157819] Updated weights for policy 0, policy_version 129338 (0.0006) [2024-09-30 02:28:41,948][1157819] Updated weights for policy 0, policy_version 129348 (0.0006) [2024-09-30 02:28:42,618][1157819] Updated weights for policy 0, policy_version 129358 (0.0006) [2024-09-30 02:28:43,203][1157819] Updated weights for policy 0, policy_version 129368 (0.0006) [2024-09-30 02:28:43,839][1157819] Updated weights for policy 0, policy_version 129378 (0.0006) [2024-09-30 02:28:44,439][1157819] Updated weights for policy 0, policy_version 129388 (0.0006) [2024-09-30 02:28:45,036][1157819] Updated weights for policy 0, policy_version 129398 (0.0006) [2024-09-30 02:28:45,466][1157520] Fps is (10 sec: 66354.5, 60 sec: 71065.5, 300 sec: 74922.1). Total num frames: 530038784. Throughput: 0: 17343.9. Samples: 122482172. Policy #0 lag: (min: 1.0, avg: 2.4, max: 6.0) [2024-09-30 02:28:45,466][1157520] Avg episode reward: [(0, '53.636')] [2024-09-30 02:28:45,664][1157819] Updated weights for policy 0, policy_version 129408 (0.0006) [2024-09-30 02:28:46,230][1157819] Updated weights for policy 0, policy_version 129418 (0.0006) [2024-09-30 02:28:46,850][1157819] Updated weights for policy 0, policy_version 129428 (0.0006) [2024-09-30 02:28:47,463][1157819] Updated weights for policy 0, policy_version 129438 (0.0006) [2024-09-30 02:28:48,047][1157819] Updated weights for policy 0, policy_version 129448 (0.0006) [2024-09-30 02:28:48,651][1157819] Updated weights for policy 0, policy_version 129458 (0.0006) [2024-09-30 02:28:49,227][1157819] Updated weights for policy 0, policy_version 129468 (0.0006) [2024-09-30 02:28:49,769][1157819] Updated weights for policy 0, policy_version 129478 (0.0006) [2024-09-30 02:28:50,313][1157819] Updated weights for policy 0, policy_version 129488 (0.0006) [2024-09-30 02:28:50,466][1157520] Fps is (10 sec: 68403.6, 60 sec: 70382.9, 300 sec: 74936.0). Total num frames: 530391040. Throughput: 0: 17312.9. Samples: 122585920. Policy #0 lag: (min: 1.0, avg: 2.4, max: 6.0) [2024-09-30 02:28:50,466][1157520] Avg episode reward: [(0, '53.518')] [2024-09-30 02:28:50,889][1157819] Updated weights for policy 0, policy_version 129498 (0.0006) [2024-09-30 02:28:51,421][1157819] Updated weights for policy 0, policy_version 129508 (0.0006) [2024-09-30 02:28:51,954][1157819] Updated weights for policy 0, policy_version 129518 (0.0006) [2024-09-30 02:28:52,464][1157819] Updated weights for policy 0, policy_version 129528 (0.0006) [2024-09-30 02:28:52,999][1157819] Updated weights for policy 0, policy_version 129538 (0.0007) [2024-09-30 02:28:53,494][1157819] Updated weights for policy 0, policy_version 129548 (0.0006) [2024-09-30 02:28:54,016][1157819] Updated weights for policy 0, policy_version 129558 (0.0006) [2024-09-30 02:28:54,536][1157819] Updated weights for policy 0, policy_version 129568 (0.0006) [2024-09-30 02:28:55,060][1157819] Updated weights for policy 0, policy_version 129578 (0.0006) [2024-09-30 02:28:55,466][1157520] Fps is (10 sec: 74138.4, 60 sec: 70451.2, 300 sec: 75033.2). Total num frames: 530780160. Throughput: 0: 17466.8. Samples: 122643348. Policy #0 lag: (min: 1.0, avg: 2.4, max: 6.0) [2024-09-30 02:28:55,466][1157520] Avg episode reward: [(0, '53.622')] [2024-09-30 02:28:55,587][1157819] Updated weights for policy 0, policy_version 129588 (0.0006) [2024-09-30 02:28:56,110][1157819] Updated weights for policy 0, policy_version 129598 (0.0006) [2024-09-30 02:28:56,678][1157819] Updated weights for policy 0, policy_version 129608 (0.0006) [2024-09-30 02:28:57,206][1157819] Updated weights for policy 0, policy_version 129618 (0.0007) [2024-09-30 02:28:57,716][1157819] Updated weights for policy 0, policy_version 129628 (0.0006) [2024-09-30 02:28:58,229][1157819] Updated weights for policy 0, policy_version 129638 (0.0006) [2024-09-30 02:28:58,743][1157819] Updated weights for policy 0, policy_version 129648 (0.0006) [2024-09-30 02:28:59,261][1157819] Updated weights for policy 0, policy_version 129658 (0.0006) [2024-09-30 02:28:59,811][1157819] Updated weights for policy 0, policy_version 129668 (0.0006) [2024-09-30 02:29:00,309][1157819] Updated weights for policy 0, policy_version 129678 (0.0006) [2024-09-30 02:29:00,466][1157520] Fps is (10 sec: 77823.8, 60 sec: 70656.0, 300 sec: 75144.3). Total num frames: 531169280. Throughput: 0: 17766.1. Samples: 122760260. Policy #0 lag: (min: 1.0, avg: 2.4, max: 6.0) [2024-09-30 02:29:00,466][1157520] Avg episode reward: [(0, '54.775')] [2024-09-30 02:29:00,845][1157819] Updated weights for policy 0, policy_version 129688 (0.0006) [2024-09-30 02:29:01,364][1157819] Updated weights for policy 0, policy_version 129698 (0.0006) [2024-09-30 02:29:01,900][1157819] Updated weights for policy 0, policy_version 129708 (0.0006) [2024-09-30 02:29:02,452][1157819] Updated weights for policy 0, policy_version 129718 (0.0006) [2024-09-30 02:29:02,970][1157819] Updated weights for policy 0, policy_version 129728 (0.0006) [2024-09-30 02:29:03,496][1157819] Updated weights for policy 0, policy_version 129738 (0.0006) [2024-09-30 02:29:03,798][1157736] Signal inference workers to stop experience collection... (8900 times) [2024-09-30 02:29:03,803][1157819] InferenceWorker_p0-w0: stopping experience collection (8900 times) [2024-09-30 02:29:03,808][1157736] Signal inference workers to resume experience collection... (8900 times) [2024-09-30 02:29:03,808][1157819] InferenceWorker_p0-w0: resuming experience collection (8900 times) [2024-09-30 02:29:04,065][1157819] Updated weights for policy 0, policy_version 129748 (0.0007) [2024-09-30 02:29:04,644][1157819] Updated weights for policy 0, policy_version 129758 (0.0006) [2024-09-30 02:29:05,195][1157819] Updated weights for policy 0, policy_version 129768 (0.0006) [2024-09-30 02:29:05,466][1157520] Fps is (10 sec: 77003.8, 60 sec: 71270.3, 300 sec: 75102.6). Total num frames: 531550208. Throughput: 0: 17994.2. Samples: 122875168. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 02:29:05,466][1157520] Avg episode reward: [(0, '53.694')] [2024-09-30 02:29:05,732][1157819] Updated weights for policy 0, policy_version 129778 (0.0006) [2024-09-30 02:29:06,249][1157819] Updated weights for policy 0, policy_version 129788 (0.0006) [2024-09-30 02:29:06,747][1157819] Updated weights for policy 0, policy_version 129798 (0.0006) [2024-09-30 02:29:07,249][1157819] Updated weights for policy 0, policy_version 129808 (0.0006) [2024-09-30 02:29:07,785][1157819] Updated weights for policy 0, policy_version 129818 (0.0006) [2024-09-30 02:29:08,353][1157819] Updated weights for policy 0, policy_version 129828 (0.0006) [2024-09-30 02:29:08,907][1157819] Updated weights for policy 0, policy_version 129838 (0.0006) [2024-09-30 02:29:09,457][1157819] Updated weights for policy 0, policy_version 129848 (0.0006) [2024-09-30 02:29:09,982][1157819] Updated weights for policy 0, policy_version 129858 (0.0006) [2024-09-30 02:29:10,466][1157520] Fps is (10 sec: 76595.1, 60 sec: 72157.8, 300 sec: 75172.0). Total num frames: 531935232. Throughput: 0: 18136.5. Samples: 122933360. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 02:29:10,466][1157520] Avg episode reward: [(0, '54.103')] [2024-09-30 02:29:10,506][1157819] Updated weights for policy 0, policy_version 129868 (0.0006) [2024-09-30 02:29:11,010][1157819] Updated weights for policy 0, policy_version 129878 (0.0006) [2024-09-30 02:29:11,534][1157819] Updated weights for policy 0, policy_version 129888 (0.0006) [2024-09-30 02:29:12,032][1157819] Updated weights for policy 0, policy_version 129898 (0.0006) [2024-09-30 02:29:12,551][1157819] Updated weights for policy 0, policy_version 129908 (0.0006) [2024-09-30 02:29:13,098][1157819] Updated weights for policy 0, policy_version 129918 (0.0006) [2024-09-30 02:29:13,601][1157819] Updated weights for policy 0, policy_version 129928 (0.0006) [2024-09-30 02:29:14,134][1157819] Updated weights for policy 0, policy_version 129938 (0.0006) [2024-09-30 02:29:14,635][1157819] Updated weights for policy 0, policy_version 129948 (0.0006) [2024-09-30 02:29:15,153][1157819] Updated weights for policy 0, policy_version 129958 (0.0006) [2024-09-30 02:29:15,466][1157520] Fps is (10 sec: 78234.9, 60 sec: 72908.8, 300 sec: 75241.5). Total num frames: 532332544. Throughput: 0: 18353.7. Samples: 123049844. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 02:29:15,466][1157520] Avg episode reward: [(0, '54.822')] [2024-09-30 02:29:15,688][1157819] Updated weights for policy 0, policy_version 129968 (0.0006) [2024-09-30 02:29:16,214][1157819] Updated weights for policy 0, policy_version 129978 (0.0006) [2024-09-30 02:29:16,709][1157819] Updated weights for policy 0, policy_version 129988 (0.0006) [2024-09-30 02:29:17,208][1157819] Updated weights for policy 0, policy_version 129998 (0.0006) [2024-09-30 02:29:17,704][1157819] Updated weights for policy 0, policy_version 130008 (0.0006) [2024-09-30 02:29:18,202][1157819] Updated weights for policy 0, policy_version 130018 (0.0006) [2024-09-30 02:29:18,681][1157819] Updated weights for policy 0, policy_version 130028 (0.0006) [2024-09-30 02:29:19,176][1157819] Updated weights for policy 0, policy_version 130038 (0.0006) [2024-09-30 02:29:19,675][1157819] Updated weights for policy 0, policy_version 130048 (0.0006) [2024-09-30 02:29:20,165][1157819] Updated weights for policy 0, policy_version 130058 (0.0006) [2024-09-30 02:29:20,466][1157520] Fps is (10 sec: 80282.3, 60 sec: 73864.6, 300 sec: 75352.5). Total num frames: 532738048. Throughput: 0: 18657.1. Samples: 123171416. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 02:29:20,466][1157520] Avg episode reward: [(0, '55.173')] [2024-09-30 02:29:20,661][1157819] Updated weights for policy 0, policy_version 130068 (0.0006) [2024-09-30 02:29:21,152][1157819] Updated weights for policy 0, policy_version 130078 (0.0006) [2024-09-30 02:29:21,643][1157819] Updated weights for policy 0, policy_version 130088 (0.0006) [2024-09-30 02:29:22,126][1157819] Updated weights for policy 0, policy_version 130098 (0.0006) [2024-09-30 02:29:22,625][1157819] Updated weights for policy 0, policy_version 130108 (0.0006) [2024-09-30 02:29:23,112][1157819] Updated weights for policy 0, policy_version 130118 (0.0006) [2024-09-30 02:29:23,633][1157819] Updated weights for policy 0, policy_version 130128 (0.0006) [2024-09-30 02:29:24,166][1157819] Updated weights for policy 0, policy_version 130138 (0.0006) [2024-09-30 02:29:24,667][1157819] Updated weights for policy 0, policy_version 130148 (0.0006) [2024-09-30 02:29:25,214][1157819] Updated weights for policy 0, policy_version 130158 (0.0006) [2024-09-30 02:29:25,466][1157520] Fps is (10 sec: 81509.8, 60 sec: 74956.8, 300 sec: 75408.1). Total num frames: 533147648. Throughput: 0: 18908.3. Samples: 123234028. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 02:29:25,466][1157520] Avg episode reward: [(0, '56.593')] [2024-09-30 02:29:25,733][1157819] Updated weights for policy 0, policy_version 130168 (0.0006) [2024-09-30 02:29:26,233][1157819] Updated weights for policy 0, policy_version 130178 (0.0006) [2024-09-30 02:29:26,764][1157819] Updated weights for policy 0, policy_version 130188 (0.0006) [2024-09-30 02:29:27,261][1157819] Updated weights for policy 0, policy_version 130198 (0.0006) [2024-09-30 02:29:27,790][1157819] Updated weights for policy 0, policy_version 130208 (0.0006) [2024-09-30 02:29:28,427][1157819] Updated weights for policy 0, policy_version 130218 (0.0006) [2024-09-30 02:29:29,024][1157819] Updated weights for policy 0, policy_version 130228 (0.0006) [2024-09-30 02:29:29,657][1157819] Updated weights for policy 0, policy_version 130238 (0.0006) [2024-09-30 02:29:30,240][1157819] Updated weights for policy 0, policy_version 130248 (0.0006) [2024-09-30 02:29:30,466][1157520] Fps is (10 sec: 77004.7, 60 sec: 74820.4, 300 sec: 75338.6). Total num frames: 533508096. Throughput: 0: 19269.7. Samples: 123349308. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 02:29:30,466][1157520] Avg episode reward: [(0, '56.265')] [2024-09-30 02:29:30,826][1157819] Updated weights for policy 0, policy_version 130258 (0.0006) [2024-09-30 02:29:31,413][1157819] Updated weights for policy 0, policy_version 130268 (0.0006) [2024-09-30 02:29:32,008][1157819] Updated weights for policy 0, policy_version 130278 (0.0006) [2024-09-30 02:29:32,616][1157819] Updated weights for policy 0, policy_version 130288 (0.0006) [2024-09-30 02:29:33,133][1157819] Updated weights for policy 0, policy_version 130298 (0.0006) [2024-09-30 02:29:33,695][1157819] Updated weights for policy 0, policy_version 130308 (0.0006) [2024-09-30 02:29:34,251][1157819] Updated weights for policy 0, policy_version 130318 (0.0006) [2024-09-30 02:29:34,764][1157819] Updated weights for policy 0, policy_version 130328 (0.0006) [2024-09-30 02:29:35,292][1157819] Updated weights for policy 0, policy_version 130338 (0.0006) [2024-09-30 02:29:35,466][1157520] Fps is (10 sec: 72908.7, 60 sec: 75025.0, 300 sec: 75324.7). Total num frames: 533876736. Throughput: 0: 19339.0. Samples: 123456176. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 02:29:35,466][1157520] Avg episode reward: [(0, '54.679')] [2024-09-30 02:29:35,826][1157819] Updated weights for policy 0, policy_version 130348 (0.0006) [2024-09-30 02:29:36,380][1157819] Updated weights for policy 0, policy_version 130358 (0.0006) [2024-09-30 02:29:36,873][1157736] Signal inference workers to stop experience collection... (8950 times) [2024-09-30 02:29:36,877][1157736] Signal inference workers to resume experience collection... (8950 times) [2024-09-30 02:29:36,877][1157819] InferenceWorker_p0-w0: stopping experience collection (8950 times) [2024-09-30 02:29:36,880][1157819] InferenceWorker_p0-w0: resuming experience collection (8950 times) [2024-09-30 02:29:36,896][1157819] Updated weights for policy 0, policy_version 130368 (0.0006) [2024-09-30 02:29:37,447][1157819] Updated weights for policy 0, policy_version 130378 (0.0006) [2024-09-30 02:29:37,986][1157819] Updated weights for policy 0, policy_version 130388 (0.0006) [2024-09-30 02:29:38,487][1157819] Updated weights for policy 0, policy_version 130398 (0.0006) [2024-09-30 02:29:39,018][1157819] Updated weights for policy 0, policy_version 130408 (0.0006) [2024-09-30 02:29:39,536][1157819] Updated weights for policy 0, policy_version 130418 (0.0006) [2024-09-30 02:29:40,036][1157819] Updated weights for policy 0, policy_version 130428 (0.0006) [2024-09-30 02:29:40,466][1157520] Fps is (10 sec: 75775.6, 60 sec: 75980.8, 300 sec: 75449.7). Total num frames: 534265856. Throughput: 0: 19335.3. Samples: 123513436. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 02:29:40,466][1157520] Avg episode reward: [(0, '57.014')] [2024-09-30 02:29:40,563][1157819] Updated weights for policy 0, policy_version 130438 (0.0006) [2024-09-30 02:29:41,072][1157819] Updated weights for policy 0, policy_version 130448 (0.0006) [2024-09-30 02:29:41,597][1157819] Updated weights for policy 0, policy_version 130458 (0.0006) [2024-09-30 02:29:42,115][1157819] Updated weights for policy 0, policy_version 130468 (0.0006) [2024-09-30 02:29:42,612][1157819] Updated weights for policy 0, policy_version 130478 (0.0006) [2024-09-30 02:29:43,125][1157819] Updated weights for policy 0, policy_version 130488 (0.0006) [2024-09-30 02:29:43,647][1157819] Updated weights for policy 0, policy_version 130498 (0.0006) [2024-09-30 02:29:44,205][1157819] Updated weights for policy 0, policy_version 130508 (0.0006) [2024-09-30 02:29:44,747][1157819] Updated weights for policy 0, policy_version 130518 (0.0006) [2024-09-30 02:29:45,249][1157819] Updated weights for policy 0, policy_version 130528 (0.0006) [2024-09-30 02:29:45,466][1157520] Fps is (10 sec: 78234.5, 60 sec: 77005.0, 300 sec: 75463.6). Total num frames: 534659072. Throughput: 0: 19383.7. Samples: 123632524. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 02:29:45,466][1157520] Avg episode reward: [(0, '53.833')] [2024-09-30 02:29:45,762][1157819] Updated weights for policy 0, policy_version 130538 (0.0006) [2024-09-30 02:29:46,293][1157819] Updated weights for policy 0, policy_version 130548 (0.0006) [2024-09-30 02:29:46,822][1157819] Updated weights for policy 0, policy_version 130558 (0.0006) [2024-09-30 02:29:47,362][1157819] Updated weights for policy 0, policy_version 130568 (0.0006) [2024-09-30 02:29:47,891][1157819] Updated weights for policy 0, policy_version 130578 (0.0006) [2024-09-30 02:29:48,433][1157819] Updated weights for policy 0, policy_version 130588 (0.0006) [2024-09-30 02:29:48,951][1157819] Updated weights for policy 0, policy_version 130598 (0.0006) [2024-09-30 02:29:49,479][1157819] Updated weights for policy 0, policy_version 130608 (0.0006) [2024-09-30 02:29:50,022][1157819] Updated weights for policy 0, policy_version 130618 (0.0006) [2024-09-30 02:29:50,466][1157520] Fps is (10 sec: 77824.4, 60 sec: 77551.0, 300 sec: 75505.3). Total num frames: 535044096. Throughput: 0: 19400.9. Samples: 123748204. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 02:29:50,466][1157520] Avg episode reward: [(0, '53.639')] [2024-09-30 02:29:50,573][1157819] Updated weights for policy 0, policy_version 130628 (0.0006) [2024-09-30 02:29:51,098][1157819] Updated weights for policy 0, policy_version 130638 (0.0006) [2024-09-30 02:29:51,643][1157819] Updated weights for policy 0, policy_version 130648 (0.0006) [2024-09-30 02:29:52,221][1157819] Updated weights for policy 0, policy_version 130658 (0.0006) [2024-09-30 02:29:52,757][1157819] Updated weights for policy 0, policy_version 130668 (0.0006) [2024-09-30 02:29:53,325][1157819] Updated weights for policy 0, policy_version 130678 (0.0006) [2024-09-30 02:29:53,853][1157819] Updated weights for policy 0, policy_version 130688 (0.0006) [2024-09-30 02:29:54,379][1157819] Updated weights for policy 0, policy_version 130698 (0.0006) [2024-09-30 02:29:54,919][1157819] Updated weights for policy 0, policy_version 130708 (0.0006) [2024-09-30 02:29:55,466][1157520] Fps is (10 sec: 76185.0, 60 sec: 77346.1, 300 sec: 75699.6). Total num frames: 535420928. Throughput: 0: 19362.7. Samples: 123804680. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 02:29:55,466][1157520] Avg episode reward: [(0, '53.919')] [2024-09-30 02:29:55,467][1157819] Updated weights for policy 0, policy_version 130718 (0.0006) [2024-09-30 02:29:55,987][1157819] Updated weights for policy 0, policy_version 130728 (0.0006) [2024-09-30 02:29:56,497][1157819] Updated weights for policy 0, policy_version 130738 (0.0006) [2024-09-30 02:29:57,072][1157819] Updated weights for policy 0, policy_version 130748 (0.0006) [2024-09-30 02:29:57,597][1157819] Updated weights for policy 0, policy_version 130758 (0.0006) [2024-09-30 02:29:58,120][1157819] Updated weights for policy 0, policy_version 130768 (0.0006) [2024-09-30 02:29:58,656][1157819] Updated weights for policy 0, policy_version 130778 (0.0006) [2024-09-30 02:29:59,189][1157819] Updated weights for policy 0, policy_version 130788 (0.0006) [2024-09-30 02:29:59,698][1157819] Updated weights for policy 0, policy_version 130798 (0.0006) [2024-09-30 02:30:00,224][1157819] Updated weights for policy 0, policy_version 130808 (0.0006) [2024-09-30 02:30:00,466][1157520] Fps is (10 sec: 76185.7, 60 sec: 77278.0, 300 sec: 75658.0). Total num frames: 535805952. Throughput: 0: 19321.3. Samples: 123919304. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 02:30:00,466][1157520] Avg episode reward: [(0, '57.173')] [2024-09-30 02:30:00,776][1157819] Updated weights for policy 0, policy_version 130818 (0.0006) [2024-09-30 02:30:01,303][1157819] Updated weights for policy 0, policy_version 130828 (0.0006) [2024-09-30 02:30:01,808][1157819] Updated weights for policy 0, policy_version 130838 (0.0006) [2024-09-30 02:30:02,345][1157819] Updated weights for policy 0, policy_version 130848 (0.0006) [2024-09-30 02:30:02,854][1157819] Updated weights for policy 0, policy_version 130858 (0.0006) [2024-09-30 02:30:03,347][1157819] Updated weights for policy 0, policy_version 130868 (0.0006) [2024-09-30 02:30:03,843][1157819] Updated weights for policy 0, policy_version 130878 (0.0006) [2024-09-30 02:30:04,375][1157819] Updated weights for policy 0, policy_version 130888 (0.0006) [2024-09-30 02:30:04,896][1157819] Updated weights for policy 0, policy_version 130898 (0.0006) [2024-09-30 02:30:05,387][1157819] Updated weights for policy 0, policy_version 130908 (0.0006) [2024-09-30 02:30:05,466][1157520] Fps is (10 sec: 78233.7, 60 sec: 77551.1, 300 sec: 75644.1). Total num frames: 536203264. Throughput: 0: 19252.2. Samples: 124037764. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 02:30:05,466][1157520] Avg episode reward: [(0, '56.831')] [2024-09-30 02:30:05,480][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000130910_536207360.pth... [2024-09-30 02:30:05,525][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000126483_518074368.pth [2024-09-30 02:30:05,560][1157736] Signal inference workers to stop experience collection... (9000 times) [2024-09-30 02:30:05,561][1157736] Signal inference workers to resume experience collection... (9000 times) [2024-09-30 02:30:05,565][1157819] InferenceWorker_p0-w0: stopping experience collection (9000 times) [2024-09-30 02:30:05,568][1157819] InferenceWorker_p0-w0: resuming experience collection (9000 times) [2024-09-30 02:30:05,886][1157819] Updated weights for policy 0, policy_version 130918 (0.0006) [2024-09-30 02:30:06,420][1157819] Updated weights for policy 0, policy_version 130928 (0.0006) [2024-09-30 02:30:06,926][1157819] Updated weights for policy 0, policy_version 130938 (0.0006) [2024-09-30 02:30:07,431][1157819] Updated weights for policy 0, policy_version 130948 (0.0006) [2024-09-30 02:30:07,936][1157819] Updated weights for policy 0, policy_version 130958 (0.0006) [2024-09-30 02:30:08,475][1157819] Updated weights for policy 0, policy_version 130968 (0.0006) [2024-09-30 02:30:08,981][1157819] Updated weights for policy 0, policy_version 130978 (0.0006) [2024-09-30 02:30:09,505][1157819] Updated weights for policy 0, policy_version 130988 (0.0006) [2024-09-30 02:30:10,054][1157819] Updated weights for policy 0, policy_version 130998 (0.0006) [2024-09-30 02:30:10,466][1157520] Fps is (10 sec: 79052.6, 60 sec: 77687.6, 300 sec: 75616.3). Total num frames: 536596480. Throughput: 0: 19192.0. Samples: 124097668. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 02:30:10,466][1157520] Avg episode reward: [(0, '54.470')] [2024-09-30 02:30:10,607][1157819] Updated weights for policy 0, policy_version 131008 (0.0006) [2024-09-30 02:30:11,174][1157819] Updated weights for policy 0, policy_version 131018 (0.0006) [2024-09-30 02:30:11,736][1157819] Updated weights for policy 0, policy_version 131028 (0.0006) [2024-09-30 02:30:12,275][1157819] Updated weights for policy 0, policy_version 131038 (0.0006) [2024-09-30 02:30:12,838][1157819] Updated weights for policy 0, policy_version 131048 (0.0006) [2024-09-30 02:30:13,369][1157819] Updated weights for policy 0, policy_version 131058 (0.0006) [2024-09-30 02:30:13,916][1157819] Updated weights for policy 0, policy_version 131068 (0.0006) [2024-09-30 02:30:14,473][1157819] Updated weights for policy 0, policy_version 131078 (0.0006) [2024-09-30 02:30:15,004][1157819] Updated weights for policy 0, policy_version 131088 (0.0006) [2024-09-30 02:30:15,466][1157520] Fps is (10 sec: 76595.4, 60 sec: 77277.8, 300 sec: 75546.9). Total num frames: 536969216. Throughput: 0: 19153.6. Samples: 124211220. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 02:30:15,466][1157520] Avg episode reward: [(0, '56.712')] [2024-09-30 02:30:15,569][1157819] Updated weights for policy 0, policy_version 131098 (0.0006) [2024-09-30 02:30:16,130][1157819] Updated weights for policy 0, policy_version 131108 (0.0006) [2024-09-30 02:30:16,654][1157819] Updated weights for policy 0, policy_version 131118 (0.0006) [2024-09-30 02:30:17,183][1157819] Updated weights for policy 0, policy_version 131128 (0.0006) [2024-09-30 02:30:17,707][1157819] Updated weights for policy 0, policy_version 131138 (0.0006) [2024-09-30 02:30:18,263][1157819] Updated weights for policy 0, policy_version 131148 (0.0006) [2024-09-30 02:30:18,799][1157819] Updated weights for policy 0, policy_version 131158 (0.0006) [2024-09-30 02:30:19,336][1157819] Updated weights for policy 0, policy_version 131168 (0.0006) [2024-09-30 02:30:19,898][1157819] Updated weights for policy 0, policy_version 131178 (0.0006) [2024-09-30 02:30:20,431][1157819] Updated weights for policy 0, policy_version 131188 (0.0006) [2024-09-30 02:30:20,466][1157520] Fps is (10 sec: 75366.0, 60 sec: 76868.2, 300 sec: 75421.9). Total num frames: 537350144. Throughput: 0: 19295.3. Samples: 124324464. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 02:30:20,466][1157520] Avg episode reward: [(0, '54.433')] [2024-09-30 02:30:20,947][1157819] Updated weights for policy 0, policy_version 131198 (0.0006) [2024-09-30 02:30:21,513][1157819] Updated weights for policy 0, policy_version 131208 (0.0006) [2024-09-30 02:30:22,040][1157819] Updated weights for policy 0, policy_version 131218 (0.0006) [2024-09-30 02:30:22,563][1157819] Updated weights for policy 0, policy_version 131228 (0.0006) [2024-09-30 02:30:23,065][1157819] Updated weights for policy 0, policy_version 131238 (0.0006) [2024-09-30 02:30:23,644][1157819] Updated weights for policy 0, policy_version 131248 (0.0006) [2024-09-30 02:30:24,170][1157819] Updated weights for policy 0, policy_version 131258 (0.0006) [2024-09-30 02:30:24,694][1157819] Updated weights for policy 0, policy_version 131268 (0.0006) [2024-09-30 02:30:25,244][1157819] Updated weights for policy 0, policy_version 131278 (0.0006) [2024-09-30 02:30:25,466][1157520] Fps is (10 sec: 76185.6, 60 sec: 76390.5, 300 sec: 75310.9). Total num frames: 537731072. Throughput: 0: 19298.4. Samples: 124381864. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 02:30:25,466][1157520] Avg episode reward: [(0, '54.013')] [2024-09-30 02:30:25,785][1157819] Updated weights for policy 0, policy_version 131288 (0.0006) [2024-09-30 02:30:26,358][1157819] Updated weights for policy 0, policy_version 131298 (0.0006) [2024-09-30 02:30:26,907][1157819] Updated weights for policy 0, policy_version 131308 (0.0006) [2024-09-30 02:30:27,416][1157819] Updated weights for policy 0, policy_version 131318 (0.0006) [2024-09-30 02:30:27,921][1157819] Updated weights for policy 0, policy_version 131328 (0.0006) [2024-09-30 02:30:28,512][1157819] Updated weights for policy 0, policy_version 131338 (0.0006) [2024-09-30 02:30:29,001][1157819] Updated weights for policy 0, policy_version 131348 (0.0006) [2024-09-30 02:30:29,508][1157819] Updated weights for policy 0, policy_version 131358 (0.0006) [2024-09-30 02:30:30,023][1157819] Updated weights for policy 0, policy_version 131368 (0.0006) [2024-09-30 02:30:30,466][1157520] Fps is (10 sec: 76595.4, 60 sec: 76800.0, 300 sec: 75380.3). Total num frames: 538116096. Throughput: 0: 19194.3. Samples: 124496268. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 02:30:30,466][1157520] Avg episode reward: [(0, '55.831')] [2024-09-30 02:30:30,545][1157819] Updated weights for policy 0, policy_version 131378 (0.0006) [2024-09-30 02:30:31,062][1157819] Updated weights for policy 0, policy_version 131388 (0.0006) [2024-09-30 02:30:31,575][1157819] Updated weights for policy 0, policy_version 131398 (0.0006) [2024-09-30 02:30:32,127][1157819] Updated weights for policy 0, policy_version 131408 (0.0006) [2024-09-30 02:30:32,627][1157819] Updated weights for policy 0, policy_version 131418 (0.0006) [2024-09-30 02:30:33,144][1157819] Updated weights for policy 0, policy_version 131428 (0.0006) [2024-09-30 02:30:33,672][1157819] Updated weights for policy 0, policy_version 131438 (0.0006) [2024-09-30 02:30:34,194][1157819] Updated weights for policy 0, policy_version 131448 (0.0006) [2024-09-30 02:30:34,702][1157819] Updated weights for policy 0, policy_version 131458 (0.0006) [2024-09-30 02:30:35,196][1157819] Updated weights for policy 0, policy_version 131468 (0.0006) [2024-09-30 02:30:35,466][1157520] Fps is (10 sec: 77824.0, 60 sec: 77209.7, 300 sec: 75519.1). Total num frames: 538509312. Throughput: 0: 19260.4. Samples: 124614924. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 02:30:35,466][1157520] Avg episode reward: [(0, '54.629')] [2024-09-30 02:30:35,713][1157819] Updated weights for policy 0, policy_version 131478 (0.0006) [2024-09-30 02:30:36,223][1157819] Updated weights for policy 0, policy_version 131488 (0.0006) [2024-09-30 02:30:36,740][1157819] Updated weights for policy 0, policy_version 131498 (0.0006) [2024-09-30 02:30:37,267][1157819] Updated weights for policy 0, policy_version 131508 (0.0006) [2024-09-30 02:30:37,755][1157819] Updated weights for policy 0, policy_version 131518 (0.0006) [2024-09-30 02:30:38,253][1157819] Updated weights for policy 0, policy_version 131528 (0.0006) [2024-09-30 02:30:38,747][1157819] Updated weights for policy 0, policy_version 131538 (0.0006) [2024-09-30 02:30:39,241][1157819] Updated weights for policy 0, policy_version 131548 (0.0006) [2024-09-30 02:30:39,725][1157819] Updated weights for policy 0, policy_version 131558 (0.0006) [2024-09-30 02:30:40,200][1157819] Updated weights for policy 0, policy_version 131568 (0.0006) [2024-09-30 02:30:40,466][1157520] Fps is (10 sec: 80282.6, 60 sec: 77551.1, 300 sec: 75699.7). Total num frames: 538918912. Throughput: 0: 19348.8. Samples: 124675372. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 02:30:40,466][1157520] Avg episode reward: [(0, '55.063')] [2024-09-30 02:30:40,780][1157819] Updated weights for policy 0, policy_version 131578 (0.0006) [2024-09-30 02:30:41,271][1157819] Updated weights for policy 0, policy_version 131588 (0.0007) [2024-09-30 02:30:41,768][1157819] Updated weights for policy 0, policy_version 131598 (0.0006) [2024-09-30 02:30:42,261][1157736] Signal inference workers to stop experience collection... (9050 times) [2024-09-30 02:30:42,262][1157736] Signal inference workers to resume experience collection... (9050 times) [2024-09-30 02:30:42,265][1157819] InferenceWorker_p0-w0: stopping experience collection (9050 times) [2024-09-30 02:30:42,268][1157819] InferenceWorker_p0-w0: resuming experience collection (9050 times) [2024-09-30 02:30:42,276][1157819] Updated weights for policy 0, policy_version 131608 (0.0006) [2024-09-30 02:30:42,820][1157819] Updated weights for policy 0, policy_version 131618 (0.0006) [2024-09-30 02:30:43,330][1157819] Updated weights for policy 0, policy_version 131628 (0.0006) [2024-09-30 02:30:43,872][1157819] Updated weights for policy 0, policy_version 131638 (0.0006) [2024-09-30 02:30:44,389][1157819] Updated weights for policy 0, policy_version 131648 (0.0006) [2024-09-30 02:30:44,929][1157819] Updated weights for policy 0, policy_version 131658 (0.0006) [2024-09-30 02:30:45,445][1157819] Updated weights for policy 0, policy_version 131668 (0.0006) [2024-09-30 02:30:45,466][1157520] Fps is (10 sec: 80282.4, 60 sec: 77551.0, 300 sec: 75824.6). Total num frames: 539312128. Throughput: 0: 19475.7. Samples: 124795708. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 02:30:45,466][1157520] Avg episode reward: [(0, '55.245')] [2024-09-30 02:30:45,962][1157819] Updated weights for policy 0, policy_version 131678 (0.0006) [2024-09-30 02:30:46,535][1157819] Updated weights for policy 0, policy_version 131688 (0.0006) [2024-09-30 02:30:47,034][1157819] Updated weights for policy 0, policy_version 131698 (0.0006) [2024-09-30 02:30:47,577][1157819] Updated weights for policy 0, policy_version 131708 (0.0007) [2024-09-30 02:30:48,129][1157819] Updated weights for policy 0, policy_version 131718 (0.0007) [2024-09-30 02:30:48,696][1157819] Updated weights for policy 0, policy_version 131728 (0.0007) [2024-09-30 02:30:49,255][1157819] Updated weights for policy 0, policy_version 131738 (0.0007) [2024-09-30 02:30:49,835][1157819] Updated weights for policy 0, policy_version 131748 (0.0007) [2024-09-30 02:30:50,375][1157819] Updated weights for policy 0, policy_version 131758 (0.0006) [2024-09-30 02:30:50,466][1157520] Fps is (10 sec: 76595.2, 60 sec: 77346.3, 300 sec: 75866.3). Total num frames: 539684864. Throughput: 0: 19366.9. Samples: 124909272. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 02:30:50,466][1157520] Avg episode reward: [(0, '54.143')] [2024-09-30 02:30:50,919][1157819] Updated weights for policy 0, policy_version 131768 (0.0006) [2024-09-30 02:30:51,476][1157819] Updated weights for policy 0, policy_version 131778 (0.0006) [2024-09-30 02:30:51,992][1157819] Updated weights for policy 0, policy_version 131788 (0.0006) [2024-09-30 02:30:52,533][1157819] Updated weights for policy 0, policy_version 131798 (0.0006) [2024-09-30 02:30:53,108][1157819] Updated weights for policy 0, policy_version 131808 (0.0006) [2024-09-30 02:30:53,650][1157819] Updated weights for policy 0, policy_version 131818 (0.0006) [2024-09-30 02:30:54,226][1157819] Updated weights for policy 0, policy_version 131828 (0.0006) [2024-09-30 02:30:54,743][1157819] Updated weights for policy 0, policy_version 131838 (0.0006) [2024-09-30 02:30:55,275][1157819] Updated weights for policy 0, policy_version 131848 (0.0007) [2024-09-30 02:30:55,466][1157520] Fps is (10 sec: 74955.6, 60 sec: 77346.1, 300 sec: 75894.0). Total num frames: 540061696. Throughput: 0: 19282.8. Samples: 124965396. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 02:30:55,466][1157520] Avg episode reward: [(0, '56.563')] [2024-09-30 02:30:55,852][1157819] Updated weights for policy 0, policy_version 131858 (0.0006) [2024-09-30 02:30:56,413][1157819] Updated weights for policy 0, policy_version 131868 (0.0006) [2024-09-30 02:30:56,990][1157819] Updated weights for policy 0, policy_version 131878 (0.0006) [2024-09-30 02:30:57,544][1157819] Updated weights for policy 0, policy_version 131888 (0.0006) [2024-09-30 02:30:58,150][1157819] Updated weights for policy 0, policy_version 131898 (0.0006) [2024-09-30 02:30:58,715][1157819] Updated weights for policy 0, policy_version 131908 (0.0006) [2024-09-30 02:30:59,295][1157819] Updated weights for policy 0, policy_version 131918 (0.0006) [2024-09-30 02:30:59,839][1157819] Updated weights for policy 0, policy_version 131928 (0.0006) [2024-09-30 02:31:00,430][1157819] Updated weights for policy 0, policy_version 131938 (0.0006) [2024-09-30 02:31:00,466][1157520] Fps is (10 sec: 73727.2, 60 sec: 76936.5, 300 sec: 75769.1). Total num frames: 540422144. Throughput: 0: 19188.1. Samples: 125074684. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 02:31:00,466][1157520] Avg episode reward: [(0, '54.626')] [2024-09-30 02:31:00,959][1157819] Updated weights for policy 0, policy_version 131948 (0.0006) [2024-09-30 02:31:01,524][1157819] Updated weights for policy 0, policy_version 131958 (0.0006) [2024-09-30 02:31:02,076][1157819] Updated weights for policy 0, policy_version 131968 (0.0006) [2024-09-30 02:31:02,682][1157819] Updated weights for policy 0, policy_version 131978 (0.0006) [2024-09-30 02:31:03,224][1157819] Updated weights for policy 0, policy_version 131988 (0.0006) [2024-09-30 02:31:03,827][1157819] Updated weights for policy 0, policy_version 131998 (0.0006) [2024-09-30 02:31:04,382][1157819] Updated weights for policy 0, policy_version 132008 (0.0006) [2024-09-30 02:31:05,012][1157819] Updated weights for policy 0, policy_version 132018 (0.0006) [2024-09-30 02:31:05,466][1157520] Fps is (10 sec: 71679.9, 60 sec: 76253.8, 300 sec: 75713.5). Total num frames: 540778496. Throughput: 0: 19073.4. Samples: 125182768. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 02:31:05,466][1157520] Avg episode reward: [(0, '58.005')] [2024-09-30 02:31:05,568][1157819] Updated weights for policy 0, policy_version 132028 (0.0006) [2024-09-30 02:31:06,158][1157819] Updated weights for policy 0, policy_version 132038 (0.0006) [2024-09-30 02:31:06,697][1157819] Updated weights for policy 0, policy_version 132048 (0.0006) [2024-09-30 02:31:07,233][1157819] Updated weights for policy 0, policy_version 132058 (0.0006) [2024-09-30 02:31:07,770][1157819] Updated weights for policy 0, policy_version 132068 (0.0006) [2024-09-30 02:31:08,304][1157819] Updated weights for policy 0, policy_version 132078 (0.0006) [2024-09-30 02:31:08,865][1157819] Updated weights for policy 0, policy_version 132088 (0.0006) [2024-09-30 02:31:09,414][1157819] Updated weights for policy 0, policy_version 132098 (0.0006) [2024-09-30 02:31:09,949][1157819] Updated weights for policy 0, policy_version 132108 (0.0006) [2024-09-30 02:31:10,466][1157520] Fps is (10 sec: 72908.7, 60 sec: 75912.5, 300 sec: 75727.4). Total num frames: 541151232. Throughput: 0: 19019.5. Samples: 125237740. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 02:31:10,466][1157520] Avg episode reward: [(0, '55.386')] [2024-09-30 02:31:10,472][1157819] Updated weights for policy 0, policy_version 132118 (0.0006) [2024-09-30 02:31:11,000][1157819] Updated weights for policy 0, policy_version 132128 (0.0006) [2024-09-30 02:31:11,546][1157819] Updated weights for policy 0, policy_version 132138 (0.0006) [2024-09-30 02:31:12,093][1157819] Updated weights for policy 0, policy_version 132148 (0.0006) [2024-09-30 02:31:12,649][1157819] Updated weights for policy 0, policy_version 132158 (0.0006) [2024-09-30 02:31:13,213][1157819] Updated weights for policy 0, policy_version 132168 (0.0006) [2024-09-30 02:31:13,779][1157819] Updated weights for policy 0, policy_version 132178 (0.0006) [2024-09-30 02:31:14,284][1157819] Updated weights for policy 0, policy_version 132188 (0.0006) [2024-09-30 02:31:14,786][1157819] Updated weights for policy 0, policy_version 132198 (0.0006) [2024-09-30 02:31:15,303][1157819] Updated weights for policy 0, policy_version 132208 (0.0006) [2024-09-30 02:31:15,466][1157520] Fps is (10 sec: 75366.9, 60 sec: 76049.1, 300 sec: 75755.2). Total num frames: 541532160. Throughput: 0: 18998.1. Samples: 125351184. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 02:31:15,466][1157520] Avg episode reward: [(0, '53.554')] [2024-09-30 02:31:15,801][1157819] Updated weights for policy 0, policy_version 132218 (0.0006) [2024-09-30 02:31:16,398][1157819] Updated weights for policy 0, policy_version 132228 (0.0006) [2024-09-30 02:31:16,912][1157819] Updated weights for policy 0, policy_version 132238 (0.0006) [2024-09-30 02:31:17,411][1157819] Updated weights for policy 0, policy_version 132248 (0.0006) [2024-09-30 02:31:17,960][1157819] Updated weights for policy 0, policy_version 132258 (0.0006) [2024-09-30 02:31:18,181][1157736] Signal inference workers to stop experience collection... (9100 times) [2024-09-30 02:31:18,185][1157819] InferenceWorker_p0-w0: stopping experience collection (9100 times) [2024-09-30 02:31:18,186][1157736] Signal inference workers to resume experience collection... (9100 times) [2024-09-30 02:31:18,190][1157819] InferenceWorker_p0-w0: resuming experience collection (9100 times) [2024-09-30 02:31:18,511][1157819] Updated weights for policy 0, policy_version 132268 (0.0006) [2024-09-30 02:31:19,072][1157819] Updated weights for policy 0, policy_version 132278 (0.0006) [2024-09-30 02:31:19,593][1157819] Updated weights for policy 0, policy_version 132288 (0.0006) [2024-09-30 02:31:20,130][1157819] Updated weights for policy 0, policy_version 132298 (0.0006) [2024-09-30 02:31:20,466][1157520] Fps is (10 sec: 76595.3, 60 sec: 76117.4, 300 sec: 75713.5). Total num frames: 541917184. Throughput: 0: 18925.7. Samples: 125466580. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 02:31:20,466][1157520] Avg episode reward: [(0, '57.551')] [2024-09-30 02:31:20,665][1157819] Updated weights for policy 0, policy_version 132308 (0.0006) [2024-09-30 02:31:21,172][1157819] Updated weights for policy 0, policy_version 132318 (0.0007) [2024-09-30 02:31:21,713][1157819] Updated weights for policy 0, policy_version 132328 (0.0006) [2024-09-30 02:31:22,232][1157819] Updated weights for policy 0, policy_version 132338 (0.0006) [2024-09-30 02:31:22,772][1157819] Updated weights for policy 0, policy_version 132348 (0.0006) [2024-09-30 02:31:23,378][1157819] Updated weights for policy 0, policy_version 132358 (0.0006) [2024-09-30 02:31:23,968][1157819] Updated weights for policy 0, policy_version 132368 (0.0006) [2024-09-30 02:31:24,546][1157819] Updated weights for policy 0, policy_version 132378 (0.0006) [2024-09-30 02:31:25,089][1157819] Updated weights for policy 0, policy_version 132388 (0.0006) [2024-09-30 02:31:25,466][1157520] Fps is (10 sec: 74955.7, 60 sec: 75844.1, 300 sec: 75616.3). Total num frames: 542281728. Throughput: 0: 18870.0. Samples: 125524528. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 02:31:25,466][1157520] Avg episode reward: [(0, '55.248')] [2024-09-30 02:31:25,709][1157819] Updated weights for policy 0, policy_version 132398 (0.0006) [2024-09-30 02:31:26,295][1157819] Updated weights for policy 0, policy_version 132408 (0.0006) [2024-09-30 02:31:26,839][1157819] Updated weights for policy 0, policy_version 132418 (0.0006) [2024-09-30 02:31:27,359][1157819] Updated weights for policy 0, policy_version 132428 (0.0006) [2024-09-30 02:31:27,922][1157819] Updated weights for policy 0, policy_version 132438 (0.0006) [2024-09-30 02:31:28,468][1157819] Updated weights for policy 0, policy_version 132448 (0.0006) [2024-09-30 02:31:29,026][1157819] Updated weights for policy 0, policy_version 132458 (0.0006) [2024-09-30 02:31:29,547][1157819] Updated weights for policy 0, policy_version 132468 (0.0006) [2024-09-30 02:31:30,058][1157819] Updated weights for policy 0, policy_version 132478 (0.0006) [2024-09-30 02:31:30,466][1157520] Fps is (10 sec: 74138.2, 60 sec: 75707.9, 300 sec: 75560.8). Total num frames: 542658560. Throughput: 0: 18595.6. Samples: 125632508. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 02:31:30,466][1157520] Avg episode reward: [(0, '57.575')] [2024-09-30 02:31:30,540][1157819] Updated weights for policy 0, policy_version 132488 (0.0006) [2024-09-30 02:31:31,047][1157819] Updated weights for policy 0, policy_version 132498 (0.0006) [2024-09-30 02:31:31,561][1157819] Updated weights for policy 0, policy_version 132508 (0.0006) [2024-09-30 02:31:32,062][1157819] Updated weights for policy 0, policy_version 132518 (0.0006) [2024-09-30 02:31:32,572][1157819] Updated weights for policy 0, policy_version 132528 (0.0006) [2024-09-30 02:31:33,081][1157819] Updated weights for policy 0, policy_version 132538 (0.0006) [2024-09-30 02:31:33,650][1157819] Updated weights for policy 0, policy_version 132548 (0.0006) [2024-09-30 02:31:34,196][1157819] Updated weights for policy 0, policy_version 132558 (0.0006) [2024-09-30 02:31:34,779][1157819] Updated weights for policy 0, policy_version 132568 (0.0006) [2024-09-30 02:31:35,323][1157819] Updated weights for policy 0, policy_version 132578 (0.0006) [2024-09-30 02:31:35,466][1157520] Fps is (10 sec: 76597.0, 60 sec: 75639.6, 300 sec: 75546.9). Total num frames: 543047680. Throughput: 0: 18680.3. Samples: 125749884. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 02:31:35,466][1157520] Avg episode reward: [(0, '55.828')] [2024-09-30 02:31:35,925][1157819] Updated weights for policy 0, policy_version 132588 (0.0006) [2024-09-30 02:31:36,527][1157819] Updated weights for policy 0, policy_version 132598 (0.0006) [2024-09-30 02:31:37,118][1157819] Updated weights for policy 0, policy_version 132608 (0.0007) [2024-09-30 02:31:37,728][1157819] Updated weights for policy 0, policy_version 132618 (0.0007) [2024-09-30 02:31:38,274][1157819] Updated weights for policy 0, policy_version 132628 (0.0007) [2024-09-30 02:31:38,748][1157736] Signal inference workers to stop experience collection... (9150 times) [2024-09-30 02:31:38,749][1157736] Signal inference workers to resume experience collection... (9150 times) [2024-09-30 02:31:38,754][1157819] InferenceWorker_p0-w0: stopping experience collection (9150 times) [2024-09-30 02:31:38,754][1157819] InferenceWorker_p0-w0: resuming experience collection (9150 times) [2024-09-30 02:31:38,933][1157819] Updated weights for policy 0, policy_version 132638 (0.0006) [2024-09-30 02:31:39,555][1157819] Updated weights for policy 0, policy_version 132648 (0.0006) [2024-09-30 02:31:40,141][1157819] Updated weights for policy 0, policy_version 132658 (0.0006) [2024-09-30 02:31:40,466][1157520] Fps is (10 sec: 72908.4, 60 sec: 74478.8, 300 sec: 75297.0). Total num frames: 543387648. Throughput: 0: 18598.0. Samples: 125802304. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 02:31:40,466][1157520] Avg episode reward: [(0, '56.224')] [2024-09-30 02:31:40,716][1157819] Updated weights for policy 0, policy_version 132668 (0.0006) [2024-09-30 02:31:41,322][1157819] Updated weights for policy 0, policy_version 132678 (0.0006) [2024-09-30 02:31:41,838][1157819] Updated weights for policy 0, policy_version 132688 (0.0006) [2024-09-30 02:31:42,404][1157819] Updated weights for policy 0, policy_version 132698 (0.0006) [2024-09-30 02:31:42,941][1157819] Updated weights for policy 0, policy_version 132708 (0.0006) [2024-09-30 02:31:43,490][1157819] Updated weights for policy 0, policy_version 132718 (0.0006) [2024-09-30 02:31:44,081][1157819] Updated weights for policy 0, policy_version 132728 (0.0006) [2024-09-30 02:31:44,657][1157819] Updated weights for policy 0, policy_version 132738 (0.0006) [2024-09-30 02:31:45,187][1157819] Updated weights for policy 0, policy_version 132748 (0.0006) [2024-09-30 02:31:45,466][1157520] Fps is (10 sec: 70859.2, 60 sec: 74069.0, 300 sec: 75199.7). Total num frames: 543756288. Throughput: 0: 18522.6. Samples: 125908204. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 02:31:45,466][1157520] Avg episode reward: [(0, '56.526')] [2024-09-30 02:31:45,718][1157819] Updated weights for policy 0, policy_version 132758 (0.0006) [2024-09-30 02:31:46,218][1157819] Updated weights for policy 0, policy_version 132768 (0.0006) [2024-09-30 02:31:46,728][1157819] Updated weights for policy 0, policy_version 132778 (0.0006) [2024-09-30 02:31:47,237][1157819] Updated weights for policy 0, policy_version 132788 (0.0006) [2024-09-30 02:31:47,788][1157819] Updated weights for policy 0, policy_version 132798 (0.0006) [2024-09-30 02:31:48,314][1157819] Updated weights for policy 0, policy_version 132808 (0.0006) [2024-09-30 02:31:48,815][1157819] Updated weights for policy 0, policy_version 132818 (0.0006) [2024-09-30 02:31:49,355][1157819] Updated weights for policy 0, policy_version 132828 (0.0006) [2024-09-30 02:31:49,856][1157819] Updated weights for policy 0, policy_version 132838 (0.0006) [2024-09-30 02:31:50,389][1157819] Updated weights for policy 0, policy_version 132848 (0.0006) [2024-09-30 02:31:50,466][1157520] Fps is (10 sec: 75776.1, 60 sec: 74342.3, 300 sec: 75255.4). Total num frames: 544145408. Throughput: 0: 18703.3. Samples: 126024416. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 02:31:50,466][1157520] Avg episode reward: [(0, '54.927')] [2024-09-30 02:31:50,923][1157819] Updated weights for policy 0, policy_version 132858 (0.0006) [2024-09-30 02:31:51,466][1157819] Updated weights for policy 0, policy_version 132868 (0.0006) [2024-09-30 02:31:51,970][1157819] Updated weights for policy 0, policy_version 132878 (0.0006) [2024-09-30 02:31:52,485][1157819] Updated weights for policy 0, policy_version 132888 (0.0006) [2024-09-30 02:31:52,986][1157819] Updated weights for policy 0, policy_version 132898 (0.0006) [2024-09-30 02:31:53,495][1157819] Updated weights for policy 0, policy_version 132908 (0.0006) [2024-09-30 02:31:54,038][1157819] Updated weights for policy 0, policy_version 132918 (0.0006) [2024-09-30 02:31:54,590][1157819] Updated weights for policy 0, policy_version 132928 (0.0006) [2024-09-30 02:31:55,108][1157819] Updated weights for policy 0, policy_version 132938 (0.0006) [2024-09-30 02:31:55,466][1157520] Fps is (10 sec: 78643.8, 60 sec: 74683.7, 300 sec: 75310.9). Total num frames: 544542720. Throughput: 0: 18794.3. Samples: 126083484. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 02:31:55,466][1157520] Avg episode reward: [(0, '52.313')] [2024-09-30 02:31:55,601][1157819] Updated weights for policy 0, policy_version 132948 (0.0006) [2024-09-30 02:31:56,119][1157819] Updated weights for policy 0, policy_version 132958 (0.0006) [2024-09-30 02:31:56,631][1157819] Updated weights for policy 0, policy_version 132968 (0.0006) [2024-09-30 02:31:57,157][1157819] Updated weights for policy 0, policy_version 132978 (0.0006) [2024-09-30 02:31:57,679][1157819] Updated weights for policy 0, policy_version 132988 (0.0006) [2024-09-30 02:31:58,240][1157819] Updated weights for policy 0, policy_version 132998 (0.0006) [2024-09-30 02:31:58,741][1157819] Updated weights for policy 0, policy_version 133008 (0.0006) [2024-09-30 02:31:59,264][1157819] Updated weights for policy 0, policy_version 133018 (0.0006) [2024-09-30 02:31:59,785][1157819] Updated weights for policy 0, policy_version 133028 (0.0006) [2024-09-30 02:32:00,348][1157819] Updated weights for policy 0, policy_version 133038 (0.0006) [2024-09-30 02:32:00,466][1157520] Fps is (10 sec: 78642.9, 60 sec: 75161.6, 300 sec: 75352.5). Total num frames: 544931840. Throughput: 0: 18888.5. Samples: 126201168. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 02:32:00,466][1157520] Avg episode reward: [(0, '55.663')] [2024-09-30 02:32:00,938][1157819] Updated weights for policy 0, policy_version 133048 (0.0006) [2024-09-30 02:32:01,497][1157819] Updated weights for policy 0, policy_version 133058 (0.0006) [2024-09-30 02:32:02,048][1157819] Updated weights for policy 0, policy_version 133068 (0.0006) [2024-09-30 02:32:02,573][1157819] Updated weights for policy 0, policy_version 133078 (0.0006) [2024-09-30 02:32:03,137][1157819] Updated weights for policy 0, policy_version 133088 (0.0006) [2024-09-30 02:32:03,709][1157819] Updated weights for policy 0, policy_version 133098 (0.0006) [2024-09-30 02:32:04,238][1157819] Updated weights for policy 0, policy_version 133108 (0.0006) [2024-09-30 02:32:04,780][1157819] Updated weights for policy 0, policy_version 133118 (0.0006) [2024-09-30 02:32:05,337][1157819] Updated weights for policy 0, policy_version 133128 (0.0006) [2024-09-30 02:32:05,466][1157520] Fps is (10 sec: 75367.0, 60 sec: 75298.2, 300 sec: 75310.9). Total num frames: 545296384. Throughput: 0: 18793.1. Samples: 126312268. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 02:32:05,466][1157520] Avg episode reward: [(0, '56.497')] [2024-09-30 02:32:05,472][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000133129_545296384.pth... [2024-09-30 02:32:05,521][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000128729_527273984.pth [2024-09-30 02:32:05,912][1157819] Updated weights for policy 0, policy_version 133138 (0.0006) [2024-09-30 02:32:06,466][1157819] Updated weights for policy 0, policy_version 133148 (0.0006) [2024-09-30 02:32:07,035][1157819] Updated weights for policy 0, policy_version 133158 (0.0006) [2024-09-30 02:32:07,566][1157819] Updated weights for policy 0, policy_version 133168 (0.0006) [2024-09-30 02:32:08,101][1157819] Updated weights for policy 0, policy_version 133178 (0.0006) [2024-09-30 02:32:08,605][1157819] Updated weights for policy 0, policy_version 133188 (0.0006) [2024-09-30 02:32:09,172][1157819] Updated weights for policy 0, policy_version 133198 (0.0006) [2024-09-30 02:32:09,335][1157736] Signal inference workers to stop experience collection... (9200 times) [2024-09-30 02:32:09,335][1157736] Signal inference workers to resume experience collection... (9200 times) [2024-09-30 02:32:09,341][1157819] InferenceWorker_p0-w0: stopping experience collection (9200 times) [2024-09-30 02:32:09,341][1157819] InferenceWorker_p0-w0: resuming experience collection (9200 times) [2024-09-30 02:32:09,725][1157819] Updated weights for policy 0, policy_version 133208 (0.0006) [2024-09-30 02:32:10,306][1157819] Updated weights for policy 0, policy_version 133218 (0.0006) [2024-09-30 02:32:10,466][1157520] Fps is (10 sec: 73727.6, 60 sec: 75298.1, 300 sec: 75310.9). Total num frames: 545669120. Throughput: 0: 18752.0. Samples: 126368368. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:32:10,466][1157520] Avg episode reward: [(0, '55.017')] [2024-09-30 02:32:10,833][1157819] Updated weights for policy 0, policy_version 133228 (0.0006) [2024-09-30 02:32:11,409][1157819] Updated weights for policy 0, policy_version 133238 (0.0007) [2024-09-30 02:32:11,960][1157819] Updated weights for policy 0, policy_version 133248 (0.0007) [2024-09-30 02:32:12,516][1157819] Updated weights for policy 0, policy_version 133258 (0.0006) [2024-09-30 02:32:13,092][1157819] Updated weights for policy 0, policy_version 133268 (0.0006) [2024-09-30 02:32:13,663][1157819] Updated weights for policy 0, policy_version 133278 (0.0006) [2024-09-30 02:32:14,225][1157819] Updated weights for policy 0, policy_version 133288 (0.0006) [2024-09-30 02:32:14,799][1157819] Updated weights for policy 0, policy_version 133298 (0.0006) [2024-09-30 02:32:15,356][1157819] Updated weights for policy 0, policy_version 133308 (0.0006) [2024-09-30 02:32:15,466][1157520] Fps is (10 sec: 73727.8, 60 sec: 75025.1, 300 sec: 75310.9). Total num frames: 546033664. Throughput: 0: 18807.1. Samples: 126478828. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:32:15,466][1157520] Avg episode reward: [(0, '56.925')] [2024-09-30 02:32:15,946][1157819] Updated weights for policy 0, policy_version 133318 (0.0006) [2024-09-30 02:32:16,486][1157819] Updated weights for policy 0, policy_version 133328 (0.0006) [2024-09-30 02:32:17,079][1157819] Updated weights for policy 0, policy_version 133338 (0.0006) [2024-09-30 02:32:17,649][1157819] Updated weights for policy 0, policy_version 133348 (0.0006) [2024-09-30 02:32:18,198][1157819] Updated weights for policy 0, policy_version 133358 (0.0006) [2024-09-30 02:32:18,757][1157819] Updated weights for policy 0, policy_version 133368 (0.0006) [2024-09-30 02:32:19,316][1157819] Updated weights for policy 0, policy_version 133378 (0.0006) [2024-09-30 02:32:19,886][1157819] Updated weights for policy 0, policy_version 133388 (0.0006) [2024-09-30 02:32:20,466][1157520] Fps is (10 sec: 72499.3, 60 sec: 74615.4, 300 sec: 75213.7). Total num frames: 546394112. Throughput: 0: 18619.7. Samples: 126587772. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:32:20,466][1157520] Avg episode reward: [(0, '56.593')] [2024-09-30 02:32:20,487][1157819] Updated weights for policy 0, policy_version 133398 (0.0006) [2024-09-30 02:32:21,033][1157819] Updated weights for policy 0, policy_version 133408 (0.0006) [2024-09-30 02:32:21,610][1157819] Updated weights for policy 0, policy_version 133418 (0.0006) [2024-09-30 02:32:22,218][1157819] Updated weights for policy 0, policy_version 133428 (0.0006) [2024-09-30 02:32:22,792][1157819] Updated weights for policy 0, policy_version 133438 (0.0006) [2024-09-30 02:32:23,400][1157819] Updated weights for policy 0, policy_version 133448 (0.0006) [2024-09-30 02:32:23,977][1157819] Updated weights for policy 0, policy_version 133458 (0.0006) [2024-09-30 02:32:24,550][1157819] Updated weights for policy 0, policy_version 133468 (0.0006) [2024-09-30 02:32:25,123][1157819] Updated weights for policy 0, policy_version 133478 (0.0006) [2024-09-30 02:32:25,466][1157520] Fps is (10 sec: 71270.4, 60 sec: 74410.8, 300 sec: 75088.7). Total num frames: 546746368. Throughput: 0: 18622.1. Samples: 126640300. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:32:25,466][1157520] Avg episode reward: [(0, '53.617')] [2024-09-30 02:32:25,719][1157819] Updated weights for policy 0, policy_version 133488 (0.0006) [2024-09-30 02:32:26,290][1157819] Updated weights for policy 0, policy_version 133498 (0.0006) [2024-09-30 02:32:26,893][1157819] Updated weights for policy 0, policy_version 133508 (0.0006) [2024-09-30 02:32:27,458][1157819] Updated weights for policy 0, policy_version 133518 (0.0006) [2024-09-30 02:32:28,081][1157819] Updated weights for policy 0, policy_version 133528 (0.0006) [2024-09-30 02:32:28,641][1157819] Updated weights for policy 0, policy_version 133538 (0.0006) [2024-09-30 02:32:29,185][1157819] Updated weights for policy 0, policy_version 133548 (0.0006) [2024-09-30 02:32:29,764][1157819] Updated weights for policy 0, policy_version 133558 (0.0006) [2024-09-30 02:32:30,284][1157819] Updated weights for policy 0, policy_version 133568 (0.0006) [2024-09-30 02:32:30,466][1157520] Fps is (10 sec: 70860.5, 60 sec: 74069.1, 300 sec: 74963.7). Total num frames: 547102720. Throughput: 0: 18610.5. Samples: 126745676. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:32:30,466][1157520] Avg episode reward: [(0, '53.052')] [2024-09-30 02:32:30,857][1157819] Updated weights for policy 0, policy_version 133578 (0.0006) [2024-09-30 02:32:31,429][1157819] Updated weights for policy 0, policy_version 133588 (0.0006) [2024-09-30 02:32:31,961][1157819] Updated weights for policy 0, policy_version 133598 (0.0006) [2024-09-30 02:32:32,506][1157819] Updated weights for policy 0, policy_version 133608 (0.0006) [2024-09-30 02:32:33,080][1157819] Updated weights for policy 0, policy_version 133618 (0.0006) [2024-09-30 02:32:33,615][1157819] Updated weights for policy 0, policy_version 133628 (0.0006) [2024-09-30 02:32:34,239][1157819] Updated weights for policy 0, policy_version 133638 (0.0006) [2024-09-30 02:32:34,770][1157819] Updated weights for policy 0, policy_version 133648 (0.0006) [2024-09-30 02:32:35,393][1157819] Updated weights for policy 0, policy_version 133658 (0.0006) [2024-09-30 02:32:35,466][1157520] Fps is (10 sec: 72499.3, 60 sec: 73727.9, 300 sec: 74880.4). Total num frames: 547471360. Throughput: 0: 18471.9. Samples: 126855652. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:32:35,466][1157520] Avg episode reward: [(0, '54.957')] [2024-09-30 02:32:35,958][1157819] Updated weights for policy 0, policy_version 133668 (0.0006) [2024-09-30 02:32:36,547][1157819] Updated weights for policy 0, policy_version 133678 (0.0006) [2024-09-30 02:32:37,106][1157819] Updated weights for policy 0, policy_version 133688 (0.0006) [2024-09-30 02:32:37,693][1157819] Updated weights for policy 0, policy_version 133698 (0.0006) [2024-09-30 02:32:38,298][1157819] Updated weights for policy 0, policy_version 133708 (0.0006) [2024-09-30 02:32:38,875][1157819] Updated weights for policy 0, policy_version 133718 (0.0006) [2024-09-30 02:32:39,473][1157819] Updated weights for policy 0, policy_version 133728 (0.0006) [2024-09-30 02:32:40,068][1157819] Updated weights for policy 0, policy_version 133738 (0.0006) [2024-09-30 02:32:40,466][1157520] Fps is (10 sec: 71680.4, 60 sec: 73864.5, 300 sec: 74727.7). Total num frames: 547819520. Throughput: 0: 18322.8. Samples: 126908008. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:32:40,466][1157520] Avg episode reward: [(0, '52.562')] [2024-09-30 02:32:40,676][1157819] Updated weights for policy 0, policy_version 133748 (0.0007) [2024-09-30 02:32:41,249][1157819] Updated weights for policy 0, policy_version 133758 (0.0007) [2024-09-30 02:32:41,835][1157819] Updated weights for policy 0, policy_version 133768 (0.0006) [2024-09-30 02:32:42,453][1157819] Updated weights for policy 0, policy_version 133778 (0.0006) [2024-09-30 02:32:43,023][1157819] Updated weights for policy 0, policy_version 133788 (0.0006) [2024-09-30 02:32:43,597][1157819] Updated weights for policy 0, policy_version 133798 (0.0006) [2024-09-30 02:32:44,127][1157819] Updated weights for policy 0, policy_version 133808 (0.0006) [2024-09-30 02:32:44,682][1157819] Updated weights for policy 0, policy_version 133818 (0.0006) [2024-09-30 02:32:45,244][1157819] Updated weights for policy 0, policy_version 133828 (0.0006) [2024-09-30 02:32:45,466][1157520] Fps is (10 sec: 70041.5, 60 sec: 73591.6, 300 sec: 74588.8). Total num frames: 548171776. Throughput: 0: 18036.3. Samples: 127012804. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:32:45,466][1157520] Avg episode reward: [(0, '54.142')] [2024-09-30 02:32:45,791][1157819] Updated weights for policy 0, policy_version 133838 (0.0006) [2024-09-30 02:32:46,353][1157819] Updated weights for policy 0, policy_version 133848 (0.0006) [2024-09-30 02:32:46,910][1157819] Updated weights for policy 0, policy_version 133858 (0.0006) [2024-09-30 02:32:47,461][1157819] Updated weights for policy 0, policy_version 133868 (0.0006) [2024-09-30 02:32:48,026][1157819] Updated weights for policy 0, policy_version 133878 (0.0006) [2024-09-30 02:32:48,581][1157819] Updated weights for policy 0, policy_version 133888 (0.0006) [2024-09-30 02:32:49,156][1157819] Updated weights for policy 0, policy_version 133898 (0.0007) [2024-09-30 02:32:49,707][1157819] Updated weights for policy 0, policy_version 133908 (0.0006) [2024-09-30 02:32:50,280][1157819] Updated weights for policy 0, policy_version 133918 (0.0006) [2024-09-30 02:32:50,466][1157520] Fps is (10 sec: 72089.4, 60 sec: 73250.0, 300 sec: 74533.3). Total num frames: 548540416. Throughput: 0: 18025.2. Samples: 127123404. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:32:50,466][1157520] Avg episode reward: [(0, '55.977')] [2024-09-30 02:32:50,838][1157819] Updated weights for policy 0, policy_version 133928 (0.0006) [2024-09-30 02:32:51,374][1157819] Updated weights for policy 0, policy_version 133938 (0.0007) [2024-09-30 02:32:51,972][1157819] Updated weights for policy 0, policy_version 133948 (0.0006) [2024-09-30 02:32:52,524][1157819] Updated weights for policy 0, policy_version 133958 (0.0007) [2024-09-30 02:32:53,097][1157819] Updated weights for policy 0, policy_version 133968 (0.0007) [2024-09-30 02:32:53,679][1157819] Updated weights for policy 0, policy_version 133978 (0.0006) [2024-09-30 02:32:54,259][1157819] Updated weights for policy 0, policy_version 133988 (0.0006) [2024-09-30 02:32:54,844][1157819] Updated weights for policy 0, policy_version 133998 (0.0006) [2024-09-30 02:32:55,432][1157819] Updated weights for policy 0, policy_version 134008 (0.0006) [2024-09-30 02:32:55,466][1157520] Fps is (10 sec: 72498.5, 60 sec: 72567.4, 300 sec: 74463.9). Total num frames: 548896768. Throughput: 0: 17986.2. Samples: 127177748. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:32:55,466][1157520] Avg episode reward: [(0, '54.143')] [2024-09-30 02:32:56,026][1157819] Updated weights for policy 0, policy_version 134018 (0.0006) [2024-09-30 02:32:56,148][1157736] Signal inference workers to stop experience collection... (9250 times) [2024-09-30 02:32:56,149][1157736] Signal inference workers to resume experience collection... (9250 times) [2024-09-30 02:32:56,152][1157819] InferenceWorker_p0-w0: stopping experience collection (9250 times) [2024-09-30 02:32:56,153][1157819] InferenceWorker_p0-w0: resuming experience collection (9250 times) [2024-09-30 02:32:56,621][1157819] Updated weights for policy 0, policy_version 134028 (0.0006) [2024-09-30 02:32:57,200][1157819] Updated weights for policy 0, policy_version 134038 (0.0006) [2024-09-30 02:32:57,794][1157819] Updated weights for policy 0, policy_version 134048 (0.0006) [2024-09-30 02:32:58,355][1157819] Updated weights for policy 0, policy_version 134058 (0.0006) [2024-09-30 02:32:58,948][1157819] Updated weights for policy 0, policy_version 134068 (0.0006) [2024-09-30 02:32:59,501][1157819] Updated weights for policy 0, policy_version 134078 (0.0006) [2024-09-30 02:33:00,077][1157819] Updated weights for policy 0, policy_version 134088 (0.0006) [2024-09-30 02:33:00,466][1157520] Fps is (10 sec: 70861.1, 60 sec: 71953.1, 300 sec: 74491.7). Total num frames: 549249024. Throughput: 0: 17873.1. Samples: 127283116. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:33:00,466][1157520] Avg episode reward: [(0, '54.787')] [2024-09-30 02:33:00,647][1157819] Updated weights for policy 0, policy_version 134098 (0.0006) [2024-09-30 02:33:01,211][1157819] Updated weights for policy 0, policy_version 134108 (0.0006) [2024-09-30 02:33:01,785][1157819] Updated weights for policy 0, policy_version 134118 (0.0006) [2024-09-30 02:33:02,358][1157819] Updated weights for policy 0, policy_version 134128 (0.0006) [2024-09-30 02:33:03,010][1157819] Updated weights for policy 0, policy_version 134138 (0.0006) [2024-09-30 02:33:03,599][1157819] Updated weights for policy 0, policy_version 134148 (0.0006) [2024-09-30 02:33:04,195][1157819] Updated weights for policy 0, policy_version 134158 (0.0006) [2024-09-30 02:33:04,789][1157819] Updated weights for policy 0, policy_version 134168 (0.0006) [2024-09-30 02:33:05,404][1157819] Updated weights for policy 0, policy_version 134178 (0.0006) [2024-09-30 02:33:05,466][1157520] Fps is (10 sec: 69632.7, 60 sec: 71611.7, 300 sec: 74533.3). Total num frames: 549593088. Throughput: 0: 17782.8. Samples: 127388000. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:33:05,466][1157520] Avg episode reward: [(0, '53.820')] [2024-09-30 02:33:05,966][1157819] Updated weights for policy 0, policy_version 134188 (0.0006) [2024-09-30 02:33:06,515][1157819] Updated weights for policy 0, policy_version 134198 (0.0006) [2024-09-30 02:33:07,061][1157819] Updated weights for policy 0, policy_version 134208 (0.0006) [2024-09-30 02:33:07,584][1157819] Updated weights for policy 0, policy_version 134218 (0.0006) [2024-09-30 02:33:08,121][1157819] Updated weights for policy 0, policy_version 134228 (0.0006) [2024-09-30 02:33:08,667][1157819] Updated weights for policy 0, policy_version 134238 (0.0006) [2024-09-30 02:33:09,197][1157819] Updated weights for policy 0, policy_version 134248 (0.0006) [2024-09-30 02:33:09,734][1157819] Updated weights for policy 0, policy_version 134258 (0.0006) [2024-09-30 02:33:10,300][1157819] Updated weights for policy 0, policy_version 134268 (0.0006) [2024-09-30 02:33:10,466][1157520] Fps is (10 sec: 72499.0, 60 sec: 71748.3, 300 sec: 74630.5). Total num frames: 549974016. Throughput: 0: 17830.8. Samples: 127442688. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:33:10,466][1157520] Avg episode reward: [(0, '52.724')] [2024-09-30 02:33:10,816][1157819] Updated weights for policy 0, policy_version 134278 (0.0006) [2024-09-30 02:33:11,376][1157819] Updated weights for policy 0, policy_version 134288 (0.0006) [2024-09-30 02:33:11,909][1157819] Updated weights for policy 0, policy_version 134298 (0.0006) [2024-09-30 02:33:12,436][1157819] Updated weights for policy 0, policy_version 134308 (0.0006) [2024-09-30 02:33:13,013][1157819] Updated weights for policy 0, policy_version 134318 (0.0006) [2024-09-30 02:33:13,546][1157819] Updated weights for policy 0, policy_version 134328 (0.0006) [2024-09-30 02:33:14,098][1157819] Updated weights for policy 0, policy_version 134338 (0.0006) [2024-09-30 02:33:14,676][1157819] Updated weights for policy 0, policy_version 134348 (0.0006) [2024-09-30 02:33:15,224][1157819] Updated weights for policy 0, policy_version 134358 (0.0006) [2024-09-30 02:33:15,466][1157520] Fps is (10 sec: 74955.9, 60 sec: 71816.4, 300 sec: 74699.9). Total num frames: 550342656. Throughput: 0: 18010.8. Samples: 127556164. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 02:33:15,466][1157520] Avg episode reward: [(0, '52.637')] [2024-09-30 02:33:15,791][1157819] Updated weights for policy 0, policy_version 134368 (0.0006) [2024-09-30 02:33:16,343][1157819] Updated weights for policy 0, policy_version 134378 (0.0006) [2024-09-30 02:33:16,896][1157819] Updated weights for policy 0, policy_version 134388 (0.0006) [2024-09-30 02:33:17,417][1157819] Updated weights for policy 0, policy_version 134398 (0.0006) [2024-09-30 02:33:17,964][1157819] Updated weights for policy 0, policy_version 134408 (0.0006) [2024-09-30 02:33:18,469][1157819] Updated weights for policy 0, policy_version 134418 (0.0006) [2024-09-30 02:33:18,989][1157819] Updated weights for policy 0, policy_version 134428 (0.0006) [2024-09-30 02:33:19,541][1157819] Updated weights for policy 0, policy_version 134438 (0.0006) [2024-09-30 02:33:20,052][1157819] Updated weights for policy 0, policy_version 134448 (0.0006) [2024-09-30 02:33:20,466][1157520] Fps is (10 sec: 75365.7, 60 sec: 72226.0, 300 sec: 74838.8). Total num frames: 550727680. Throughput: 0: 18079.6. Samples: 127669236. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 02:33:20,466][1157520] Avg episode reward: [(0, '55.448')] [2024-09-30 02:33:20,609][1157819] Updated weights for policy 0, policy_version 134458 (0.0006) [2024-09-30 02:33:21,144][1157819] Updated weights for policy 0, policy_version 134468 (0.0006) [2024-09-30 02:33:21,664][1157819] Updated weights for policy 0, policy_version 134478 (0.0006) [2024-09-30 02:33:22,175][1157819] Updated weights for policy 0, policy_version 134488 (0.0006) [2024-09-30 02:33:22,695][1157819] Updated weights for policy 0, policy_version 134498 (0.0006) [2024-09-30 02:33:23,201][1157819] Updated weights for policy 0, policy_version 134508 (0.0006) [2024-09-30 02:33:23,761][1157819] Updated weights for policy 0, policy_version 134518 (0.0006) [2024-09-30 02:33:24,305][1157819] Updated weights for policy 0, policy_version 134528 (0.0006) [2024-09-30 02:33:24,797][1157819] Updated weights for policy 0, policy_version 134538 (0.0006) [2024-09-30 02:33:25,323][1157819] Updated weights for policy 0, policy_version 134548 (0.0006) [2024-09-30 02:33:25,466][1157520] Fps is (10 sec: 77825.2, 60 sec: 72908.8, 300 sec: 74922.1). Total num frames: 551120896. Throughput: 0: 18210.1. Samples: 127727464. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 02:33:25,466][1157520] Avg episode reward: [(0, '55.640')] [2024-09-30 02:33:25,830][1157819] Updated weights for policy 0, policy_version 134558 (0.0006) [2024-09-30 02:33:26,325][1157819] Updated weights for policy 0, policy_version 134568 (0.0006) [2024-09-30 02:33:26,842][1157819] Updated weights for policy 0, policy_version 134578 (0.0006) [2024-09-30 02:33:27,391][1157819] Updated weights for policy 0, policy_version 134588 (0.0006) [2024-09-30 02:33:27,930][1157819] Updated weights for policy 0, policy_version 134598 (0.0006) [2024-09-30 02:33:28,417][1157819] Updated weights for policy 0, policy_version 134608 (0.0006) [2024-09-30 02:33:28,942][1157819] Updated weights for policy 0, policy_version 134618 (0.0006) [2024-09-30 02:33:29,438][1157819] Updated weights for policy 0, policy_version 134628 (0.0006) [2024-09-30 02:33:29,684][1157736] Signal inference workers to stop experience collection... (9300 times) [2024-09-30 02:33:29,685][1157736] Signal inference workers to resume experience collection... (9300 times) [2024-09-30 02:33:29,690][1157819] InferenceWorker_p0-w0: stopping experience collection (9300 times) [2024-09-30 02:33:29,690][1157819] InferenceWorker_p0-w0: resuming experience collection (9300 times) [2024-09-30 02:33:29,944][1157819] Updated weights for policy 0, policy_version 134638 (0.0006) [2024-09-30 02:33:30,466][1157520] Fps is (10 sec: 78643.7, 60 sec: 73523.2, 300 sec: 75047.0). Total num frames: 551514112. Throughput: 0: 18515.2. Samples: 127845988. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 02:33:30,466][1157520] Avg episode reward: [(0, '54.371')] [2024-09-30 02:33:30,495][1157819] Updated weights for policy 0, policy_version 134648 (0.0006) [2024-09-30 02:33:30,997][1157819] Updated weights for policy 0, policy_version 134658 (0.0006) [2024-09-30 02:33:31,543][1157819] Updated weights for policy 0, policy_version 134668 (0.0006) [2024-09-30 02:33:32,042][1157819] Updated weights for policy 0, policy_version 134678 (0.0006) [2024-09-30 02:33:32,577][1157819] Updated weights for policy 0, policy_version 134688 (0.0006) [2024-09-30 02:33:33,108][1157819] Updated weights for policy 0, policy_version 134698 (0.0006) [2024-09-30 02:33:33,658][1157819] Updated weights for policy 0, policy_version 134708 (0.0006) [2024-09-30 02:33:34,148][1157819] Updated weights for policy 0, policy_version 134718 (0.0006) [2024-09-30 02:33:34,661][1157819] Updated weights for policy 0, policy_version 134728 (0.0006) [2024-09-30 02:33:35,266][1157819] Updated weights for policy 0, policy_version 134738 (0.0007) [2024-09-30 02:33:35,466][1157520] Fps is (10 sec: 77823.0, 60 sec: 73796.1, 300 sec: 75227.5). Total num frames: 551899136. Throughput: 0: 18671.1. Samples: 127963604. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 02:33:35,466][1157520] Avg episode reward: [(0, '55.795')] [2024-09-30 02:33:35,832][1157819] Updated weights for policy 0, policy_version 134748 (0.0006) [2024-09-30 02:33:36,468][1157819] Updated weights for policy 0, policy_version 134758 (0.0006) [2024-09-30 02:33:37,056][1157819] Updated weights for policy 0, policy_version 134768 (0.0006) [2024-09-30 02:33:37,666][1157819] Updated weights for policy 0, policy_version 134778 (0.0006) [2024-09-30 02:33:38,335][1157819] Updated weights for policy 0, policy_version 134788 (0.0007) [2024-09-30 02:33:38,948][1157819] Updated weights for policy 0, policy_version 134798 (0.0007) [2024-09-30 02:33:39,612][1157819] Updated weights for policy 0, policy_version 134808 (0.0006) [2024-09-30 02:33:40,271][1157819] Updated weights for policy 0, policy_version 134818 (0.0006) [2024-09-30 02:33:40,466][1157520] Fps is (10 sec: 71270.2, 60 sec: 73454.8, 300 sec: 75213.7). Total num frames: 552226816. Throughput: 0: 18591.8. Samples: 128014380. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 02:33:40,466][1157520] Avg episode reward: [(0, '55.994')] [2024-09-30 02:33:40,872][1157819] Updated weights for policy 0, policy_version 134828 (0.0006) [2024-09-30 02:33:41,429][1157819] Updated weights for policy 0, policy_version 134838 (0.0006) [2024-09-30 02:33:42,045][1157819] Updated weights for policy 0, policy_version 134848 (0.0006) [2024-09-30 02:33:42,664][1157819] Updated weights for policy 0, policy_version 134858 (0.0006) [2024-09-30 02:33:43,272][1157819] Updated weights for policy 0, policy_version 134868 (0.0006) [2024-09-30 02:33:43,836][1157819] Updated weights for policy 0, policy_version 134878 (0.0006) [2024-09-30 02:33:44,436][1157819] Updated weights for policy 0, policy_version 134888 (0.0006) [2024-09-30 02:33:45,046][1157819] Updated weights for policy 0, policy_version 134898 (0.0007) [2024-09-30 02:33:45,466][1157520] Fps is (10 sec: 67175.0, 60 sec: 73318.4, 300 sec: 75185.9). Total num frames: 552570880. Throughput: 0: 18464.5. Samples: 128114020. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 02:33:45,466][1157520] Avg episode reward: [(0, '57.017')] [2024-09-30 02:33:45,627][1157819] Updated weights for policy 0, policy_version 134908 (0.0007) [2024-09-30 02:33:46,228][1157819] Updated weights for policy 0, policy_version 134918 (0.0006) [2024-09-30 02:33:46,818][1157819] Updated weights for policy 0, policy_version 134928 (0.0006) [2024-09-30 02:33:47,370][1157819] Updated weights for policy 0, policy_version 134938 (0.0006) [2024-09-30 02:33:47,994][1157819] Updated weights for policy 0, policy_version 134948 (0.0006) [2024-09-30 02:33:48,573][1157819] Updated weights for policy 0, policy_version 134958 (0.0006) [2024-09-30 02:33:49,166][1157819] Updated weights for policy 0, policy_version 134968 (0.0006) [2024-09-30 02:33:49,748][1157819] Updated weights for policy 0, policy_version 134978 (0.0007) [2024-09-30 02:33:50,352][1157819] Updated weights for policy 0, policy_version 134988 (0.0007) [2024-09-30 02:33:50,466][1157520] Fps is (10 sec: 69223.1, 60 sec: 72977.1, 300 sec: 75047.1). Total num frames: 552919040. Throughput: 0: 18456.8. Samples: 128218556. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 02:33:50,466][1157520] Avg episode reward: [(0, '55.763')] [2024-09-30 02:33:50,893][1157819] Updated weights for policy 0, policy_version 134998 (0.0006) [2024-09-30 02:33:51,393][1157819] Updated weights for policy 0, policy_version 135008 (0.0006) [2024-09-30 02:33:51,962][1157819] Updated weights for policy 0, policy_version 135018 (0.0006) [2024-09-30 02:33:52,460][1157819] Updated weights for policy 0, policy_version 135028 (0.0006) [2024-09-30 02:33:53,014][1157819] Updated weights for policy 0, policy_version 135038 (0.0006) [2024-09-30 02:33:53,545][1157819] Updated weights for policy 0, policy_version 135048 (0.0006) [2024-09-30 02:33:54,096][1157819] Updated weights for policy 0, policy_version 135058 (0.0006) [2024-09-30 02:33:54,651][1157819] Updated weights for policy 0, policy_version 135068 (0.0006) [2024-09-30 02:33:55,218][1157819] Updated weights for policy 0, policy_version 135078 (0.0006) [2024-09-30 02:33:55,466][1157520] Fps is (10 sec: 72499.3, 60 sec: 73318.5, 300 sec: 75005.4). Total num frames: 553295872. Throughput: 0: 18487.4. Samples: 128274620. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 02:33:55,466][1157520] Avg episode reward: [(0, '56.113')] [2024-09-30 02:33:55,779][1157819] Updated weights for policy 0, policy_version 135088 (0.0006) [2024-09-30 02:33:56,341][1157819] Updated weights for policy 0, policy_version 135098 (0.0006) [2024-09-30 02:33:56,874][1157819] Updated weights for policy 0, policy_version 135108 (0.0006) [2024-09-30 02:33:57,427][1157819] Updated weights for policy 0, policy_version 135118 (0.0006) [2024-09-30 02:33:57,967][1157819] Updated weights for policy 0, policy_version 135128 (0.0006) [2024-09-30 02:33:58,532][1157819] Updated weights for policy 0, policy_version 135138 (0.0006) [2024-09-30 02:33:59,061][1157819] Updated weights for policy 0, policy_version 135148 (0.0006) [2024-09-30 02:33:59,655][1157819] Updated weights for policy 0, policy_version 135158 (0.0006) [2024-09-30 02:34:00,203][1157819] Updated weights for policy 0, policy_version 135168 (0.0006) [2024-09-30 02:34:00,466][1157520] Fps is (10 sec: 74547.5, 60 sec: 73591.5, 300 sec: 74963.8). Total num frames: 553664512. Throughput: 0: 18440.3. Samples: 128385972. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 02:34:00,466][1157520] Avg episode reward: [(0, '54.128')] [2024-09-30 02:34:00,757][1157819] Updated weights for policy 0, policy_version 135178 (0.0006) [2024-09-30 02:34:01,337][1157819] Updated weights for policy 0, policy_version 135188 (0.0006) [2024-09-30 02:34:01,911][1157819] Updated weights for policy 0, policy_version 135198 (0.0006) [2024-09-30 02:34:02,491][1157819] Updated weights for policy 0, policy_version 135208 (0.0006) [2024-09-30 02:34:03,067][1157819] Updated weights for policy 0, policy_version 135218 (0.0006) [2024-09-30 02:34:03,609][1157819] Updated weights for policy 0, policy_version 135228 (0.0006) [2024-09-30 02:34:04,168][1157819] Updated weights for policy 0, policy_version 135238 (0.0006) [2024-09-30 02:34:04,721][1157819] Updated weights for policy 0, policy_version 135248 (0.0006) [2024-09-30 02:34:05,270][1157819] Updated weights for policy 0, policy_version 135258 (0.0006) [2024-09-30 02:34:05,466][1157520] Fps is (10 sec: 72908.5, 60 sec: 73864.5, 300 sec: 74880.4). Total num frames: 554024960. Throughput: 0: 18349.7. Samples: 128494972. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 02:34:05,466][1157520] Avg episode reward: [(0, '55.606')] [2024-09-30 02:34:05,478][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000135261_554029056.pth... [2024-09-30 02:34:05,528][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000130910_536207360.pth [2024-09-30 02:34:05,878][1157819] Updated weights for policy 0, policy_version 135268 (0.0006) [2024-09-30 02:34:06,469][1157819] Updated weights for policy 0, policy_version 135278 (0.0006) [2024-09-30 02:34:07,071][1157819] Updated weights for policy 0, policy_version 135288 (0.0006) [2024-09-30 02:34:07,652][1157819] Updated weights for policy 0, policy_version 135298 (0.0006) [2024-09-30 02:34:08,238][1157819] Updated weights for policy 0, policy_version 135308 (0.0006) [2024-09-30 02:34:08,827][1157819] Updated weights for policy 0, policy_version 135318 (0.0006) [2024-09-30 02:34:09,426][1157819] Updated weights for policy 0, policy_version 135328 (0.0006) [2024-09-30 02:34:10,019][1157819] Updated weights for policy 0, policy_version 135338 (0.0006) [2024-09-30 02:34:10,466][1157520] Fps is (10 sec: 70860.5, 60 sec: 73318.5, 300 sec: 74713.8). Total num frames: 554373120. Throughput: 0: 18214.0. Samples: 128547092. Policy #0 lag: (min: 0.0, avg: 1.9, max: 5.0) [2024-09-30 02:34:10,466][1157520] Avg episode reward: [(0, '55.066')] [2024-09-30 02:34:10,583][1157819] Updated weights for policy 0, policy_version 135348 (0.0006) [2024-09-30 02:34:11,145][1157819] Updated weights for policy 0, policy_version 135358 (0.0006) [2024-09-30 02:34:11,706][1157819] Updated weights for policy 0, policy_version 135368 (0.0006) [2024-09-30 02:34:12,247][1157819] Updated weights for policy 0, policy_version 135378 (0.0006) [2024-09-30 02:34:12,820][1157819] Updated weights for policy 0, policy_version 135388 (0.0006) [2024-09-30 02:34:13,373][1157819] Updated weights for policy 0, policy_version 135398 (0.0006) [2024-09-30 02:34:13,953][1157819] Updated weights for policy 0, policy_version 135408 (0.0006) [2024-09-30 02:34:14,540][1157819] Updated weights for policy 0, policy_version 135418 (0.0006) [2024-09-30 02:34:15,129][1157819] Updated weights for policy 0, policy_version 135428 (0.0006) [2024-09-30 02:34:15,466][1157520] Fps is (10 sec: 70859.9, 60 sec: 73181.8, 300 sec: 74561.0). Total num frames: 554733568. Throughput: 0: 17972.5. Samples: 128654752. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 02:34:15,466][1157520] Avg episode reward: [(0, '55.724')] [2024-09-30 02:34:15,745][1157819] Updated weights for policy 0, policy_version 135438 (0.0006) [2024-09-30 02:34:16,327][1157819] Updated weights for policy 0, policy_version 135448 (0.0006) [2024-09-30 02:34:16,893][1157819] Updated weights for policy 0, policy_version 135458 (0.0006) [2024-09-30 02:34:17,498][1157819] Updated weights for policy 0, policy_version 135468 (0.0006) [2024-09-30 02:34:18,094][1157819] Updated weights for policy 0, policy_version 135478 (0.0006) [2024-09-30 02:34:18,688][1157819] Updated weights for policy 0, policy_version 135488 (0.0006) [2024-09-30 02:34:19,275][1157736] Signal inference workers to stop experience collection... (9350 times) [2024-09-30 02:34:19,276][1157736] Signal inference workers to resume experience collection... (9350 times) [2024-09-30 02:34:19,280][1157819] InferenceWorker_p0-w0: stopping experience collection (9350 times) [2024-09-30 02:34:19,282][1157819] InferenceWorker_p0-w0: resuming experience collection (9350 times) [2024-09-30 02:34:19,291][1157819] Updated weights for policy 0, policy_version 135498 (0.0006) [2024-09-30 02:34:19,856][1157819] Updated weights for policy 0, policy_version 135508 (0.0006) [2024-09-30 02:34:20,458][1157819] Updated weights for policy 0, policy_version 135518 (0.0006) [2024-09-30 02:34:20,466][1157520] Fps is (10 sec: 70860.5, 60 sec: 72567.6, 300 sec: 74352.8). Total num frames: 555081728. Throughput: 0: 17664.7. Samples: 128758512. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 02:34:20,466][1157520] Avg episode reward: [(0, '54.319')] [2024-09-30 02:34:20,986][1157819] Updated weights for policy 0, policy_version 135528 (0.0006) [2024-09-30 02:34:21,575][1157819] Updated weights for policy 0, policy_version 135538 (0.0006) [2024-09-30 02:34:22,099][1157819] Updated weights for policy 0, policy_version 135548 (0.0006) [2024-09-30 02:34:22,596][1157819] Updated weights for policy 0, policy_version 135558 (0.0006) [2024-09-30 02:34:23,112][1157819] Updated weights for policy 0, policy_version 135568 (0.0006) [2024-09-30 02:34:23,679][1157819] Updated weights for policy 0, policy_version 135578 (0.0006) [2024-09-30 02:34:24,195][1157819] Updated weights for policy 0, policy_version 135588 (0.0006) [2024-09-30 02:34:24,699][1157819] Updated weights for policy 0, policy_version 135598 (0.0006) [2024-09-30 02:34:25,213][1157819] Updated weights for policy 0, policy_version 135608 (0.0006) [2024-09-30 02:34:25,466][1157520] Fps is (10 sec: 73319.0, 60 sec: 72430.8, 300 sec: 74436.1). Total num frames: 555466752. Throughput: 0: 17787.3. Samples: 128814808. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 02:34:25,466][1157520] Avg episode reward: [(0, '55.509')] [2024-09-30 02:34:25,746][1157819] Updated weights for policy 0, policy_version 135618 (0.0006) [2024-09-30 02:34:26,258][1157819] Updated weights for policy 0, policy_version 135628 (0.0006) [2024-09-30 02:34:26,743][1157819] Updated weights for policy 0, policy_version 135638 (0.0006) [2024-09-30 02:34:27,266][1157819] Updated weights for policy 0, policy_version 135648 (0.0006) [2024-09-30 02:34:27,788][1157819] Updated weights for policy 0, policy_version 135658 (0.0006) [2024-09-30 02:34:28,310][1157819] Updated weights for policy 0, policy_version 135668 (0.0006) [2024-09-30 02:34:28,830][1157819] Updated weights for policy 0, policy_version 135678 (0.0006) [2024-09-30 02:34:29,312][1157819] Updated weights for policy 0, policy_version 135688 (0.0006) [2024-09-30 02:34:29,833][1157819] Updated weights for policy 0, policy_version 135698 (0.0006) [2024-09-30 02:34:30,362][1157819] Updated weights for policy 0, policy_version 135708 (0.0006) [2024-09-30 02:34:30,466][1157520] Fps is (10 sec: 78643.2, 60 sec: 72567.5, 300 sec: 74547.2). Total num frames: 555868160. Throughput: 0: 18217.9. Samples: 128933824. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 02:34:30,466][1157520] Avg episode reward: [(0, '57.142')] [2024-09-30 02:34:30,866][1157819] Updated weights for policy 0, policy_version 135718 (0.0006) [2024-09-30 02:34:31,368][1157819] Updated weights for policy 0, policy_version 135728 (0.0006) [2024-09-30 02:34:31,857][1157819] Updated weights for policy 0, policy_version 135738 (0.0006) [2024-09-30 02:34:32,368][1157819] Updated weights for policy 0, policy_version 135748 (0.0006) [2024-09-30 02:34:32,848][1157819] Updated weights for policy 0, policy_version 135758 (0.0006) [2024-09-30 02:34:33,337][1157819] Updated weights for policy 0, policy_version 135768 (0.0006) [2024-09-30 02:34:33,852][1157819] Updated weights for policy 0, policy_version 135778 (0.0006) [2024-09-30 02:34:34,388][1157819] Updated weights for policy 0, policy_version 135788 (0.0006) [2024-09-30 02:34:34,887][1157819] Updated weights for policy 0, policy_version 135798 (0.0006) [2024-09-30 02:34:35,404][1157819] Updated weights for policy 0, policy_version 135808 (0.0006) [2024-09-30 02:34:35,466][1157520] Fps is (10 sec: 80691.9, 60 sec: 72908.9, 300 sec: 74602.8). Total num frames: 556273664. Throughput: 0: 18590.0. Samples: 129055108. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 02:34:35,466][1157520] Avg episode reward: [(0, '54.963')] [2024-09-30 02:34:35,918][1157819] Updated weights for policy 0, policy_version 135818 (0.0006) [2024-09-30 02:34:36,441][1157819] Updated weights for policy 0, policy_version 135828 (0.0006) [2024-09-30 02:34:36,940][1157819] Updated weights for policy 0, policy_version 135838 (0.0006) [2024-09-30 02:34:37,465][1157819] Updated weights for policy 0, policy_version 135848 (0.0006) [2024-09-30 02:34:37,962][1157819] Updated weights for policy 0, policy_version 135858 (0.0006) [2024-09-30 02:34:38,481][1157819] Updated weights for policy 0, policy_version 135868 (0.0006) [2024-09-30 02:34:38,986][1157819] Updated weights for policy 0, policy_version 135878 (0.0006) [2024-09-30 02:34:39,476][1157819] Updated weights for policy 0, policy_version 135888 (0.0006) [2024-09-30 02:34:40,054][1157819] Updated weights for policy 0, policy_version 135898 (0.0006) [2024-09-30 02:34:40,466][1157520] Fps is (10 sec: 80282.1, 60 sec: 74069.5, 300 sec: 74616.6). Total num frames: 556670976. Throughput: 0: 18663.5. Samples: 129114476. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 02:34:40,466][1157520] Avg episode reward: [(0, '55.462')] [2024-09-30 02:34:40,616][1157819] Updated weights for policy 0, policy_version 135908 (0.0006) [2024-09-30 02:34:41,144][1157819] Updated weights for policy 0, policy_version 135918 (0.0006) [2024-09-30 02:34:41,714][1157819] Updated weights for policy 0, policy_version 135928 (0.0006) [2024-09-30 02:34:42,274][1157819] Updated weights for policy 0, policy_version 135938 (0.0006) [2024-09-30 02:34:42,779][1157819] Updated weights for policy 0, policy_version 135948 (0.0007) [2024-09-30 02:34:43,322][1157819] Updated weights for policy 0, policy_version 135958 (0.0006) [2024-09-30 02:34:43,845][1157819] Updated weights for policy 0, policy_version 135968 (0.0006) [2024-09-30 02:34:44,374][1157819] Updated weights for policy 0, policy_version 135978 (0.0006) [2024-09-30 02:34:44,887][1157819] Updated weights for policy 0, policy_version 135988 (0.0006) [2024-09-30 02:34:45,449][1157819] Updated weights for policy 0, policy_version 135998 (0.0007) [2024-09-30 02:34:45,466][1157520] Fps is (10 sec: 77414.4, 60 sec: 74615.5, 300 sec: 74588.9). Total num frames: 557047808. Throughput: 0: 18754.8. Samples: 129229940. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 02:34:45,466][1157520] Avg episode reward: [(0, '55.151')] [2024-09-30 02:34:45,973][1157819] Updated weights for policy 0, policy_version 136008 (0.0006) [2024-09-30 02:34:46,521][1157819] Updated weights for policy 0, policy_version 136018 (0.0007) [2024-09-30 02:34:47,018][1157819] Updated weights for policy 0, policy_version 136028 (0.0006) [2024-09-30 02:34:47,554][1157819] Updated weights for policy 0, policy_version 136038 (0.0006) [2024-09-30 02:34:48,105][1157819] Updated weights for policy 0, policy_version 136048 (0.0006) [2024-09-30 02:34:48,657][1157819] Updated weights for policy 0, policy_version 136058 (0.0006) [2024-09-30 02:34:49,164][1157819] Updated weights for policy 0, policy_version 136068 (0.0007) [2024-09-30 02:34:49,670][1157819] Updated weights for policy 0, policy_version 136078 (0.0006) [2024-09-30 02:34:50,178][1157819] Updated weights for policy 0, policy_version 136088 (0.0006) [2024-09-30 02:34:50,466][1157520] Fps is (10 sec: 76185.7, 60 sec: 75229.9, 300 sec: 74616.6). Total num frames: 557432832. Throughput: 0: 18908.4. Samples: 129345848. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 02:34:50,466][1157520] Avg episode reward: [(0, '57.491')] [2024-09-30 02:34:50,692][1157819] Updated weights for policy 0, policy_version 136098 (0.0006) [2024-09-30 02:34:51,234][1157819] Updated weights for policy 0, policy_version 136108 (0.0006) [2024-09-30 02:34:51,773][1157819] Updated weights for policy 0, policy_version 136118 (0.0006) [2024-09-30 02:34:52,285][1157819] Updated weights for policy 0, policy_version 136128 (0.0006) [2024-09-30 02:34:52,802][1157819] Updated weights for policy 0, policy_version 136138 (0.0006) [2024-09-30 02:34:53,303][1157819] Updated weights for policy 0, policy_version 136148 (0.0006) [2024-09-30 02:34:53,800][1157819] Updated weights for policy 0, policy_version 136158 (0.0006) [2024-09-30 02:34:54,340][1157819] Updated weights for policy 0, policy_version 136168 (0.0006) [2024-09-30 02:34:54,900][1157819] Updated weights for policy 0, policy_version 136178 (0.0006) [2024-09-30 02:34:55,408][1157819] Updated weights for policy 0, policy_version 136188 (0.0006) [2024-09-30 02:34:55,466][1157520] Fps is (10 sec: 78233.8, 60 sec: 75571.2, 300 sec: 74658.3). Total num frames: 557830144. Throughput: 0: 19070.8. Samples: 129405276. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 02:34:55,466][1157520] Avg episode reward: [(0, '54.591')] [2024-09-30 02:34:55,903][1157819] Updated weights for policy 0, policy_version 136198 (0.0006) [2024-09-30 02:34:56,416][1157819] Updated weights for policy 0, policy_version 136208 (0.0007) [2024-09-30 02:34:56,935][1157819] Updated weights for policy 0, policy_version 136218 (0.0006) [2024-09-30 02:34:57,434][1157819] Updated weights for policy 0, policy_version 136228 (0.0006) [2024-09-30 02:34:57,973][1157819] Updated weights for policy 0, policy_version 136238 (0.0007) [2024-09-30 02:34:58,532][1157819] Updated weights for policy 0, policy_version 136248 (0.0006) [2024-09-30 02:34:59,177][1157819] Updated weights for policy 0, policy_version 136258 (0.0006) [2024-09-30 02:34:59,790][1157819] Updated weights for policy 0, policy_version 136268 (0.0007) [2024-09-30 02:35:00,367][1157819] Updated weights for policy 0, policy_version 136278 (0.0006) [2024-09-30 02:35:00,466][1157520] Fps is (10 sec: 76594.9, 60 sec: 75571.1, 300 sec: 74561.1). Total num frames: 558198784. Throughput: 0: 19258.9. Samples: 129521400. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 02:35:00,466][1157520] Avg episode reward: [(0, '55.195')] [2024-09-30 02:35:00,962][1157819] Updated weights for policy 0, policy_version 136288 (0.0007) [2024-09-30 02:35:01,549][1157819] Updated weights for policy 0, policy_version 136298 (0.0007) [2024-09-30 02:35:02,109][1157819] Updated weights for policy 0, policy_version 136308 (0.0007) [2024-09-30 02:35:02,668][1157819] Updated weights for policy 0, policy_version 136318 (0.0007) [2024-09-30 02:35:03,225][1157819] Updated weights for policy 0, policy_version 136328 (0.0007) [2024-09-30 02:35:03,793][1157819] Updated weights for policy 0, policy_version 136338 (0.0007) [2024-09-30 02:35:04,348][1157819] Updated weights for policy 0, policy_version 136348 (0.0006) [2024-09-30 02:35:04,932][1157819] Updated weights for policy 0, policy_version 136358 (0.0007) [2024-09-30 02:35:05,466][1157520] Fps is (10 sec: 72908.7, 60 sec: 75571.3, 300 sec: 74450.0). Total num frames: 558559232. Throughput: 0: 19322.7. Samples: 129628032. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 02:35:05,466][1157520] Avg episode reward: [(0, '56.419')] [2024-09-30 02:35:05,479][1157819] Updated weights for policy 0, policy_version 136368 (0.0006) [2024-09-30 02:35:05,979][1157819] Updated weights for policy 0, policy_version 136378 (0.0006) [2024-09-30 02:35:06,492][1157819] Updated weights for policy 0, policy_version 136388 (0.0007) [2024-09-30 02:35:07,038][1157819] Updated weights for policy 0, policy_version 136398 (0.0007) [2024-09-30 02:35:07,295][1157736] Signal inference workers to stop experience collection... (9400 times) [2024-09-30 02:35:07,295][1157736] Signal inference workers to resume experience collection... (9400 times) [2024-09-30 02:35:07,299][1157819] InferenceWorker_p0-w0: stopping experience collection (9400 times) [2024-09-30 02:35:07,299][1157819] InferenceWorker_p0-w0: resuming experience collection (9400 times) [2024-09-30 02:35:07,586][1157819] Updated weights for policy 0, policy_version 136408 (0.0006) [2024-09-30 02:35:08,120][1157819] Updated weights for policy 0, policy_version 136418 (0.0007) [2024-09-30 02:35:08,665][1157819] Updated weights for policy 0, policy_version 136428 (0.0007) [2024-09-30 02:35:09,208][1157819] Updated weights for policy 0, policy_version 136438 (0.0007) [2024-09-30 02:35:09,741][1157819] Updated weights for policy 0, policy_version 136448 (0.0007) [2024-09-30 02:35:10,282][1157819] Updated weights for policy 0, policy_version 136458 (0.0007) [2024-09-30 02:35:10,466][1157520] Fps is (10 sec: 74547.3, 60 sec: 76185.6, 300 sec: 74491.7). Total num frames: 558944256. Throughput: 0: 19347.8. Samples: 129685456. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 02:35:10,466][1157520] Avg episode reward: [(0, '57.659')] [2024-09-30 02:35:10,805][1157819] Updated weights for policy 0, policy_version 136468 (0.0007) [2024-09-30 02:35:11,358][1157819] Updated weights for policy 0, policy_version 136478 (0.0007) [2024-09-30 02:35:11,897][1157819] Updated weights for policy 0, policy_version 136488 (0.0007) [2024-09-30 02:35:12,441][1157819] Updated weights for policy 0, policy_version 136498 (0.0007) [2024-09-30 02:35:12,981][1157819] Updated weights for policy 0, policy_version 136508 (0.0007) [2024-09-30 02:35:13,524][1157819] Updated weights for policy 0, policy_version 136518 (0.0006) [2024-09-30 02:35:14,033][1157819] Updated weights for policy 0, policy_version 136528 (0.0006) [2024-09-30 02:35:14,545][1157819] Updated weights for policy 0, policy_version 136538 (0.0006) [2024-09-30 02:35:15,047][1157819] Updated weights for policy 0, policy_version 136548 (0.0006) [2024-09-30 02:35:15,466][1157520] Fps is (10 sec: 77415.1, 60 sec: 76663.8, 300 sec: 74519.5). Total num frames: 559333376. Throughput: 0: 19241.7. Samples: 129799700. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 02:35:15,466][1157520] Avg episode reward: [(0, '55.721')] [2024-09-30 02:35:15,524][1157819] Updated weights for policy 0, policy_version 136558 (0.0006) [2024-09-30 02:35:16,020][1157819] Updated weights for policy 0, policy_version 136568 (0.0006) [2024-09-30 02:35:16,532][1157819] Updated weights for policy 0, policy_version 136578 (0.0006) [2024-09-30 02:35:17,024][1157819] Updated weights for policy 0, policy_version 136588 (0.0006) [2024-09-30 02:35:17,538][1157819] Updated weights for policy 0, policy_version 136598 (0.0006) [2024-09-30 02:35:18,052][1157819] Updated weights for policy 0, policy_version 136608 (0.0006) [2024-09-30 02:35:18,567][1157819] Updated weights for policy 0, policy_version 136618 (0.0006) [2024-09-30 02:35:19,067][1157819] Updated weights for policy 0, policy_version 136628 (0.0006) [2024-09-30 02:35:19,578][1157819] Updated weights for policy 0, policy_version 136638 (0.0006) [2024-09-30 02:35:20,151][1157819] Updated weights for policy 0, policy_version 136648 (0.0006) [2024-09-30 02:35:20,466][1157520] Fps is (10 sec: 78643.1, 60 sec: 77482.7, 300 sec: 74575.0). Total num frames: 559730688. Throughput: 0: 19237.9. Samples: 129920812. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 02:35:20,466][1157520] Avg episode reward: [(0, '54.973')] [2024-09-30 02:35:20,705][1157819] Updated weights for policy 0, policy_version 136658 (0.0006) [2024-09-30 02:35:21,244][1157819] Updated weights for policy 0, policy_version 136668 (0.0006) [2024-09-30 02:35:21,820][1157819] Updated weights for policy 0, policy_version 136678 (0.0006) [2024-09-30 02:35:22,331][1157819] Updated weights for policy 0, policy_version 136688 (0.0006) [2024-09-30 02:35:22,859][1157819] Updated weights for policy 0, policy_version 136698 (0.0006) [2024-09-30 02:35:23,397][1157819] Updated weights for policy 0, policy_version 136708 (0.0006) [2024-09-30 02:35:23,961][1157819] Updated weights for policy 0, policy_version 136718 (0.0006) [2024-09-30 02:35:24,505][1157819] Updated weights for policy 0, policy_version 136728 (0.0006) [2024-09-30 02:35:25,043][1157819] Updated weights for policy 0, policy_version 136738 (0.0006) [2024-09-30 02:35:25,466][1157520] Fps is (10 sec: 77413.5, 60 sec: 77346.2, 300 sec: 74547.2). Total num frames: 560107520. Throughput: 0: 19172.1. Samples: 129977220. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 02:35:25,466][1157520] Avg episode reward: [(0, '55.588')] [2024-09-30 02:35:25,602][1157819] Updated weights for policy 0, policy_version 136748 (0.0006) [2024-09-30 02:35:26,116][1157819] Updated weights for policy 0, policy_version 136758 (0.0006) [2024-09-30 02:35:26,671][1157819] Updated weights for policy 0, policy_version 136768 (0.0006) [2024-09-30 02:35:27,228][1157819] Updated weights for policy 0, policy_version 136778 (0.0006) [2024-09-30 02:35:27,781][1157819] Updated weights for policy 0, policy_version 136788 (0.0006) [2024-09-30 02:35:28,264][1157819] Updated weights for policy 0, policy_version 136798 (0.0006) [2024-09-30 02:35:28,783][1157819] Updated weights for policy 0, policy_version 136808 (0.0006) [2024-09-30 02:35:29,315][1157819] Updated weights for policy 0, policy_version 136818 (0.0006) [2024-09-30 02:35:29,830][1157819] Updated weights for policy 0, policy_version 136828 (0.0006) [2024-09-30 02:35:30,342][1157819] Updated weights for policy 0, policy_version 136838 (0.0006) [2024-09-30 02:35:30,466][1157520] Fps is (10 sec: 76595.0, 60 sec: 77141.3, 300 sec: 74533.3). Total num frames: 560496640. Throughput: 0: 19138.6. Samples: 130091176. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 02:35:30,466][1157520] Avg episode reward: [(0, '55.306')] [2024-09-30 02:35:30,859][1157819] Updated weights for policy 0, policy_version 136848 (0.0006) [2024-09-30 02:35:31,379][1157819] Updated weights for policy 0, policy_version 136858 (0.0007) [2024-09-30 02:35:31,926][1157819] Updated weights for policy 0, policy_version 136868 (0.0006) [2024-09-30 02:35:32,444][1157819] Updated weights for policy 0, policy_version 136878 (0.0006) [2024-09-30 02:35:32,946][1157819] Updated weights for policy 0, policy_version 136888 (0.0006) [2024-09-30 02:35:33,456][1157819] Updated weights for policy 0, policy_version 136898 (0.0006) [2024-09-30 02:35:34,009][1157819] Updated weights for policy 0, policy_version 136908 (0.0007) [2024-09-30 02:35:34,527][1157819] Updated weights for policy 0, policy_version 136918 (0.0007) [2024-09-30 02:35:35,051][1157819] Updated weights for policy 0, policy_version 136928 (0.0007) [2024-09-30 02:35:35,466][1157520] Fps is (10 sec: 77824.2, 60 sec: 76868.3, 300 sec: 74463.9). Total num frames: 560885760. Throughput: 0: 19183.3. Samples: 130209096. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 02:35:35,466][1157520] Avg episode reward: [(0, '56.114')] [2024-09-30 02:35:35,618][1157819] Updated weights for policy 0, policy_version 136938 (0.0007) [2024-09-30 02:35:36,176][1157819] Updated weights for policy 0, policy_version 136948 (0.0006) [2024-09-30 02:35:36,697][1157819] Updated weights for policy 0, policy_version 136958 (0.0006) [2024-09-30 02:35:37,230][1157819] Updated weights for policy 0, policy_version 136968 (0.0006) [2024-09-30 02:35:37,734][1157819] Updated weights for policy 0, policy_version 136978 (0.0006) [2024-09-30 02:35:38,252][1157819] Updated weights for policy 0, policy_version 136988 (0.0006) [2024-09-30 02:35:38,793][1157819] Updated weights for policy 0, policy_version 136998 (0.0006) [2024-09-30 02:35:39,305][1157819] Updated weights for policy 0, policy_version 137008 (0.0006) [2024-09-30 02:35:39,814][1157819] Updated weights for policy 0, policy_version 137018 (0.0006) [2024-09-30 02:35:39,970][1157736] Signal inference workers to stop experience collection... (9450 times) [2024-09-30 02:35:39,974][1157819] InferenceWorker_p0-w0: stopping experience collection (9450 times) [2024-09-30 02:35:39,975][1157736] Signal inference workers to resume experience collection... (9450 times) [2024-09-30 02:35:39,979][1157819] InferenceWorker_p0-w0: resuming experience collection (9450 times) [2024-09-30 02:35:40,318][1157819] Updated weights for policy 0, policy_version 137028 (0.0006) [2024-09-30 02:35:40,466][1157520] Fps is (10 sec: 77823.6, 60 sec: 76731.6, 300 sec: 74450.0). Total num frames: 561274880. Throughput: 0: 19133.8. Samples: 130266300. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 02:35:40,466][1157520] Avg episode reward: [(0, '54.888')] [2024-09-30 02:35:40,849][1157819] Updated weights for policy 0, policy_version 137038 (0.0006) [2024-09-30 02:35:41,417][1157819] Updated weights for policy 0, policy_version 137048 (0.0006) [2024-09-30 02:35:41,914][1157819] Updated weights for policy 0, policy_version 137058 (0.0006) [2024-09-30 02:35:42,448][1157819] Updated weights for policy 0, policy_version 137068 (0.0006) [2024-09-30 02:35:42,935][1157819] Updated weights for policy 0, policy_version 137078 (0.0006) [2024-09-30 02:35:43,379][1157819] Updated weights for policy 0, policy_version 137088 (0.0006) [2024-09-30 02:35:43,915][1157819] Updated weights for policy 0, policy_version 137098 (0.0006) [2024-09-30 02:35:44,406][1157819] Updated weights for policy 0, policy_version 137108 (0.0006) [2024-09-30 02:35:44,852][1157819] Updated weights for policy 0, policy_version 137118 (0.0006) [2024-09-30 02:35:45,301][1157819] Updated weights for policy 0, policy_version 137128 (0.0006) [2024-09-30 02:35:45,466][1157520] Fps is (10 sec: 80282.4, 60 sec: 77346.3, 300 sec: 74588.9). Total num frames: 561688576. Throughput: 0: 19211.9. Samples: 130385936. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 02:35:45,466][1157520] Avg episode reward: [(0, '55.566')] [2024-09-30 02:35:45,731][1157819] Updated weights for policy 0, policy_version 137138 (0.0006) [2024-09-30 02:35:46,184][1157819] Updated weights for policy 0, policy_version 137148 (0.0006) [2024-09-30 02:35:46,604][1157819] Updated weights for policy 0, policy_version 137158 (0.0006) [2024-09-30 02:35:47,048][1157819] Updated weights for policy 0, policy_version 137168 (0.0006) [2024-09-30 02:35:47,496][1157819] Updated weights for policy 0, policy_version 137178 (0.0006) [2024-09-30 02:35:47,943][1157819] Updated weights for policy 0, policy_version 137188 (0.0006) [2024-09-30 02:35:48,372][1157819] Updated weights for policy 0, policy_version 137198 (0.0006) [2024-09-30 02:35:48,816][1157819] Updated weights for policy 0, policy_version 137208 (0.0006) [2024-09-30 02:35:49,241][1157819] Updated weights for policy 0, policy_version 137218 (0.0006) [2024-09-30 02:35:49,689][1157819] Updated weights for policy 0, policy_version 137228 (0.0006) [2024-09-30 02:35:50,110][1157819] Updated weights for policy 0, policy_version 137238 (0.0006) [2024-09-30 02:35:50,466][1157520] Fps is (10 sec: 88475.2, 60 sec: 78779.8, 300 sec: 74908.2). Total num frames: 562159616. Throughput: 0: 19915.4. Samples: 130524224. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 02:35:50,466][1157520] Avg episode reward: [(0, '56.787')] [2024-09-30 02:35:50,564][1157819] Updated weights for policy 0, policy_version 137248 (0.0006) [2024-09-30 02:35:50,966][1157819] Updated weights for policy 0, policy_version 137258 (0.0006) [2024-09-30 02:35:51,428][1157819] Updated weights for policy 0, policy_version 137268 (0.0006) [2024-09-30 02:35:51,817][1157819] Updated weights for policy 0, policy_version 137278 (0.0006) [2024-09-30 02:35:52,292][1157819] Updated weights for policy 0, policy_version 137288 (0.0006) [2024-09-30 02:35:52,687][1157819] Updated weights for policy 0, policy_version 137298 (0.0006) [2024-09-30 02:35:53,165][1157819] Updated weights for policy 0, policy_version 137308 (0.0006) [2024-09-30 02:35:53,583][1157819] Updated weights for policy 0, policy_version 137318 (0.0006) [2024-09-30 02:35:54,034][1157819] Updated weights for policy 0, policy_version 137328 (0.0006) [2024-09-30 02:35:54,516][1157819] Updated weights for policy 0, policy_version 137338 (0.0006) [2024-09-30 02:35:55,020][1157819] Updated weights for policy 0, policy_version 137348 (0.0006) [2024-09-30 02:35:55,466][1157520] Fps is (10 sec: 92158.7, 60 sec: 79667.1, 300 sec: 75213.7). Total num frames: 562610176. Throughput: 0: 20209.7. Samples: 130594892. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 02:35:55,466][1157520] Avg episode reward: [(0, '54.376')] [2024-09-30 02:35:55,510][1157819] Updated weights for policy 0, policy_version 137358 (0.0007) [2024-09-30 02:35:55,516][1157736] Signal inference workers to stop experience collection... (9500 times) [2024-09-30 02:35:55,516][1157736] Signal inference workers to resume experience collection... (9500 times) [2024-09-30 02:35:55,520][1157819] InferenceWorker_p0-w0: stopping experience collection (9500 times) [2024-09-30 02:35:55,520][1157819] InferenceWorker_p0-w0: resuming experience collection (9500 times) [2024-09-30 02:35:55,972][1157819] Updated weights for policy 0, policy_version 137368 (0.0006) [2024-09-30 02:35:56,409][1157819] Updated weights for policy 0, policy_version 137378 (0.0006) [2024-09-30 02:35:56,882][1157819] Updated weights for policy 0, policy_version 137388 (0.0006) [2024-09-30 02:35:57,364][1157819] Updated weights for policy 0, policy_version 137398 (0.0006) [2024-09-30 02:35:57,839][1157819] Updated weights for policy 0, policy_version 137408 (0.0006) [2024-09-30 02:35:58,264][1157819] Updated weights for policy 0, policy_version 137418 (0.0006) [2024-09-30 02:35:58,758][1157819] Updated weights for policy 0, policy_version 137428 (0.0006) [2024-09-30 02:35:59,262][1157819] Updated weights for policy 0, policy_version 137438 (0.0006) [2024-09-30 02:35:59,752][1157819] Updated weights for policy 0, policy_version 137448 (0.0006) [2024-09-30 02:36:00,237][1157819] Updated weights for policy 0, policy_version 137458 (0.0006) [2024-09-30 02:36:00,466][1157520] Fps is (10 sec: 88472.7, 60 sec: 80759.5, 300 sec: 75477.5). Total num frames: 563044352. Throughput: 0: 20603.4. Samples: 130726856. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 02:36:00,466][1157520] Avg episode reward: [(0, '55.276')] [2024-09-30 02:36:00,746][1157819] Updated weights for policy 0, policy_version 137468 (0.0006) [2024-09-30 02:36:01,234][1157819] Updated weights for policy 0, policy_version 137478 (0.0006) [2024-09-30 02:36:01,751][1157819] Updated weights for policy 0, policy_version 137488 (0.0006) [2024-09-30 02:36:02,165][1157819] Updated weights for policy 0, policy_version 137498 (0.0006) [2024-09-30 02:36:02,677][1157819] Updated weights for policy 0, policy_version 137508 (0.0006) [2024-09-30 02:36:03,210][1157819] Updated weights for policy 0, policy_version 137518 (0.0006) [2024-09-30 02:36:03,754][1157819] Updated weights for policy 0, policy_version 137528 (0.0006) [2024-09-30 02:36:04,269][1157819] Updated weights for policy 0, policy_version 137538 (0.0006) [2024-09-30 02:36:04,806][1157819] Updated weights for policy 0, policy_version 137548 (0.0006) [2024-09-30 02:36:05,315][1157819] Updated weights for policy 0, policy_version 137558 (0.0006) [2024-09-30 02:36:05,466][1157520] Fps is (10 sec: 83558.6, 60 sec: 81442.1, 300 sec: 75574.7). Total num frames: 563445760. Throughput: 0: 20635.0. Samples: 130849388. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 02:36:05,466][1157520] Avg episode reward: [(0, '56.879')] [2024-09-30 02:36:05,488][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000137561_563449856.pth... [2024-09-30 02:36:05,546][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000133129_545296384.pth [2024-09-30 02:36:05,816][1157819] Updated weights for policy 0, policy_version 137568 (0.0006) [2024-09-30 02:36:06,307][1157819] Updated weights for policy 0, policy_version 137578 (0.0006) [2024-09-30 02:36:06,807][1157819] Updated weights for policy 0, policy_version 137588 (0.0006) [2024-09-30 02:36:07,273][1157819] Updated weights for policy 0, policy_version 137598 (0.0006) [2024-09-30 02:36:07,803][1157819] Updated weights for policy 0, policy_version 137608 (0.0006) [2024-09-30 02:36:08,283][1157819] Updated weights for policy 0, policy_version 137618 (0.0006) [2024-09-30 02:36:08,774][1157819] Updated weights for policy 0, policy_version 137628 (0.0006) [2024-09-30 02:36:09,259][1157819] Updated weights for policy 0, policy_version 137638 (0.0006) [2024-09-30 02:36:09,711][1157819] Updated weights for policy 0, policy_version 137648 (0.0006) [2024-09-30 02:36:10,213][1157819] Updated weights for policy 0, policy_version 137658 (0.0006) [2024-09-30 02:36:10,466][1157520] Fps is (10 sec: 82329.7, 60 sec: 82056.5, 300 sec: 75713.5). Total num frames: 563867648. Throughput: 0: 20736.4. Samples: 130910356. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 02:36:10,466][1157520] Avg episode reward: [(0, '53.718')] [2024-09-30 02:36:10,653][1157819] Updated weights for policy 0, policy_version 137668 (0.0006) [2024-09-30 02:36:11,145][1157819] Updated weights for policy 0, policy_version 137678 (0.0006) [2024-09-30 02:36:11,651][1157819] Updated weights for policy 0, policy_version 137688 (0.0006) [2024-09-30 02:36:12,148][1157819] Updated weights for policy 0, policy_version 137698 (0.0006) [2024-09-30 02:36:12,580][1157736] Signal inference workers to stop experience collection... (9550 times) [2024-09-30 02:36:12,581][1157736] Signal inference workers to resume experience collection... (9550 times) [2024-09-30 02:36:12,586][1157819] InferenceWorker_p0-w0: stopping experience collection (9550 times) [2024-09-30 02:36:12,586][1157819] InferenceWorker_p0-w0: resuming experience collection (9550 times) [2024-09-30 02:36:12,647][1157819] Updated weights for policy 0, policy_version 137708 (0.0006) [2024-09-30 02:36:13,163][1157819] Updated weights for policy 0, policy_version 137718 (0.0006) [2024-09-30 02:36:13,653][1157819] Updated weights for policy 0, policy_version 137728 (0.0006) [2024-09-30 02:36:14,158][1157819] Updated weights for policy 0, policy_version 137738 (0.0006) [2024-09-30 02:36:14,652][1157819] Updated weights for policy 0, policy_version 137748 (0.0006) [2024-09-30 02:36:15,147][1157819] Updated weights for policy 0, policy_version 137758 (0.0006) [2024-09-30 02:36:15,466][1157520] Fps is (10 sec: 83558.4, 60 sec: 82466.0, 300 sec: 75810.7). Total num frames: 564281344. Throughput: 0: 21000.6. Samples: 131036204. Policy #0 lag: (min: 0.0, avg: 2.1, max: 4.0) [2024-09-30 02:36:15,466][1157520] Avg episode reward: [(0, '54.659')] [2024-09-30 02:36:15,649][1157819] Updated weights for policy 0, policy_version 137768 (0.0006) [2024-09-30 02:36:16,136][1157819] Updated weights for policy 0, policy_version 137778 (0.0006) [2024-09-30 02:36:16,612][1157819] Updated weights for policy 0, policy_version 137788 (0.0006) [2024-09-30 02:36:17,111][1157819] Updated weights for policy 0, policy_version 137798 (0.0006) [2024-09-30 02:36:17,616][1157819] Updated weights for policy 0, policy_version 137808 (0.0006) [2024-09-30 02:36:18,114][1157819] Updated weights for policy 0, policy_version 137818 (0.0006) [2024-09-30 02:36:18,615][1157819] Updated weights for policy 0, policy_version 137828 (0.0006) [2024-09-30 02:36:19,122][1157819] Updated weights for policy 0, policy_version 137838 (0.0006) [2024-09-30 02:36:19,621][1157819] Updated weights for policy 0, policy_version 137848 (0.0006) [2024-09-30 02:36:20,125][1157819] Updated weights for policy 0, policy_version 137858 (0.0006) [2024-09-30 02:36:20,466][1157520] Fps is (10 sec: 82739.1, 60 sec: 82739.2, 300 sec: 75977.4). Total num frames: 564695040. Throughput: 0: 21128.2. Samples: 131159864. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:36:20,466][1157520] Avg episode reward: [(0, '54.671')] [2024-09-30 02:36:20,612][1157819] Updated weights for policy 0, policy_version 137868 (0.0006) [2024-09-30 02:36:21,106][1157819] Updated weights for policy 0, policy_version 137878 (0.0006) [2024-09-30 02:36:21,607][1157819] Updated weights for policy 0, policy_version 137888 (0.0006) [2024-09-30 02:36:22,131][1157819] Updated weights for policy 0, policy_version 137898 (0.0006) [2024-09-30 02:36:22,638][1157819] Updated weights for policy 0, policy_version 137908 (0.0006) [2024-09-30 02:36:23,172][1157819] Updated weights for policy 0, policy_version 137918 (0.0006) [2024-09-30 02:36:23,702][1157819] Updated weights for policy 0, policy_version 137928 (0.0006) [2024-09-30 02:36:24,230][1157819] Updated weights for policy 0, policy_version 137938 (0.0006) [2024-09-30 02:36:24,755][1157819] Updated weights for policy 0, policy_version 137948 (0.0006) [2024-09-30 02:36:25,258][1157819] Updated weights for policy 0, policy_version 137958 (0.0006) [2024-09-30 02:36:25,466][1157520] Fps is (10 sec: 81100.9, 60 sec: 83080.5, 300 sec: 76046.7). Total num frames: 565092352. Throughput: 0: 21202.1. Samples: 131220392. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:36:25,466][1157520] Avg episode reward: [(0, '55.178')] [2024-09-30 02:36:25,773][1157819] Updated weights for policy 0, policy_version 137968 (0.0006) [2024-09-30 02:36:26,308][1157819] Updated weights for policy 0, policy_version 137978 (0.0006) [2024-09-30 02:36:26,833][1157819] Updated weights for policy 0, policy_version 137988 (0.0006) [2024-09-30 02:36:27,319][1157819] Updated weights for policy 0, policy_version 137998 (0.0006) [2024-09-30 02:36:27,858][1157819] Updated weights for policy 0, policy_version 138008 (0.0006) [2024-09-30 02:36:28,367][1157819] Updated weights for policy 0, policy_version 138018 (0.0006) [2024-09-30 02:36:28,895][1157819] Updated weights for policy 0, policy_version 138028 (0.0006) [2024-09-30 02:36:29,395][1157819] Updated weights for policy 0, policy_version 138038 (0.0006) [2024-09-30 02:36:29,916][1157819] Updated weights for policy 0, policy_version 138048 (0.0006) [2024-09-30 02:36:30,434][1157819] Updated weights for policy 0, policy_version 138058 (0.0006) [2024-09-30 02:36:30,466][1157520] Fps is (10 sec: 79462.6, 60 sec: 83217.1, 300 sec: 76074.5). Total num frames: 565489664. Throughput: 0: 21172.9. Samples: 131338716. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:36:30,466][1157520] Avg episode reward: [(0, '54.143')] [2024-09-30 02:36:30,945][1157819] Updated weights for policy 0, policy_version 138068 (0.0006) [2024-09-30 02:36:31,470][1157819] Updated weights for policy 0, policy_version 138078 (0.0006) [2024-09-30 02:36:32,006][1157819] Updated weights for policy 0, policy_version 138088 (0.0006) [2024-09-30 02:36:32,524][1157819] Updated weights for policy 0, policy_version 138098 (0.0006) [2024-09-30 02:36:33,055][1157819] Updated weights for policy 0, policy_version 138108 (0.0006) [2024-09-30 02:36:33,544][1157819] Updated weights for policy 0, policy_version 138118 (0.0006) [2024-09-30 02:36:34,050][1157819] Updated weights for policy 0, policy_version 138128 (0.0006) [2024-09-30 02:36:34,606][1157819] Updated weights for policy 0, policy_version 138138 (0.0006) [2024-09-30 02:36:35,127][1157819] Updated weights for policy 0, policy_version 138148 (0.0006) [2024-09-30 02:36:35,466][1157520] Fps is (10 sec: 78643.1, 60 sec: 83217.0, 300 sec: 76241.1). Total num frames: 565878784. Throughput: 0: 20728.5. Samples: 131457008. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:36:35,466][1157520] Avg episode reward: [(0, '55.173')] [2024-09-30 02:36:35,628][1157819] Updated weights for policy 0, policy_version 138158 (0.0006) [2024-09-30 02:36:36,179][1157819] Updated weights for policy 0, policy_version 138168 (0.0006) [2024-09-30 02:36:36,692][1157819] Updated weights for policy 0, policy_version 138178 (0.0006) [2024-09-30 02:36:37,205][1157819] Updated weights for policy 0, policy_version 138188 (0.0006) [2024-09-30 02:36:37,745][1157819] Updated weights for policy 0, policy_version 138198 (0.0006) [2024-09-30 02:36:38,291][1157819] Updated weights for policy 0, policy_version 138208 (0.0006) [2024-09-30 02:36:38,794][1157819] Updated weights for policy 0, policy_version 138218 (0.0006) [2024-09-30 02:36:39,327][1157819] Updated weights for policy 0, policy_version 138228 (0.0006) [2024-09-30 02:36:39,852][1157819] Updated weights for policy 0, policy_version 138238 (0.0006) [2024-09-30 02:36:40,426][1157819] Updated weights for policy 0, policy_version 138248 (0.0006) [2024-09-30 02:36:40,466][1157520] Fps is (10 sec: 77823.5, 60 sec: 83217.1, 300 sec: 76310.6). Total num frames: 566267904. Throughput: 0: 20458.2. Samples: 131515512. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:36:40,466][1157520] Avg episode reward: [(0, '54.984')] [2024-09-30 02:36:40,900][1157819] Updated weights for policy 0, policy_version 138258 (0.0006) [2024-09-30 02:36:41,439][1157819] Updated weights for policy 0, policy_version 138268 (0.0006) [2024-09-30 02:36:41,967][1157819] Updated weights for policy 0, policy_version 138278 (0.0006) [2024-09-30 02:36:42,501][1157819] Updated weights for policy 0, policy_version 138288 (0.0006) [2024-09-30 02:36:43,005][1157819] Updated weights for policy 0, policy_version 138298 (0.0006) [2024-09-30 02:36:43,525][1157819] Updated weights for policy 0, policy_version 138308 (0.0006) [2024-09-30 02:36:44,057][1157819] Updated weights for policy 0, policy_version 138318 (0.0006) [2024-09-30 02:36:44,575][1157819] Updated weights for policy 0, policy_version 138328 (0.0006) [2024-09-30 02:36:45,086][1157819] Updated weights for policy 0, policy_version 138338 (0.0006) [2024-09-30 02:36:45,466][1157520] Fps is (10 sec: 77824.2, 60 sec: 82807.3, 300 sec: 76310.5). Total num frames: 566657024. Throughput: 0: 20121.0. Samples: 131632300. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:36:45,466][1157520] Avg episode reward: [(0, '55.000')] [2024-09-30 02:36:45,652][1157819] Updated weights for policy 0, policy_version 138348 (0.0006) [2024-09-30 02:36:46,203][1157819] Updated weights for policy 0, policy_version 138358 (0.0006) [2024-09-30 02:36:46,754][1157819] Updated weights for policy 0, policy_version 138368 (0.0006) [2024-09-30 02:36:47,270][1157819] Updated weights for policy 0, policy_version 138378 (0.0006) [2024-09-30 02:36:47,790][1157819] Updated weights for policy 0, policy_version 138388 (0.0006) [2024-09-30 02:36:48,320][1157819] Updated weights for policy 0, policy_version 138398 (0.0006) [2024-09-30 02:36:48,861][1157819] Updated weights for policy 0, policy_version 138408 (0.0006) [2024-09-30 02:36:49,437][1157819] Updated weights for policy 0, policy_version 138418 (0.0006) [2024-09-30 02:36:50,035][1157819] Updated weights for policy 0, policy_version 138428 (0.0006) [2024-09-30 02:36:50,466][1157520] Fps is (10 sec: 76185.3, 60 sec: 81168.8, 300 sec: 76227.3). Total num frames: 567029760. Throughput: 0: 19931.1. Samples: 131746288. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:36:50,466][1157520] Avg episode reward: [(0, '54.315')] [2024-09-30 02:36:50,578][1157819] Updated weights for policy 0, policy_version 138438 (0.0006) [2024-09-30 02:36:51,133][1157819] Updated weights for policy 0, policy_version 138448 (0.0006) [2024-09-30 02:36:51,697][1157819] Updated weights for policy 0, policy_version 138458 (0.0006) [2024-09-30 02:36:52,277][1157819] Updated weights for policy 0, policy_version 138468 (0.0006) [2024-09-30 02:36:52,694][1157736] Signal inference workers to stop experience collection... (9600 times) [2024-09-30 02:36:52,695][1157736] Signal inference workers to resume experience collection... (9600 times) [2024-09-30 02:36:52,698][1157819] InferenceWorker_p0-w0: stopping experience collection (9600 times) [2024-09-30 02:36:52,698][1157819] InferenceWorker_p0-w0: resuming experience collection (9600 times) [2024-09-30 02:36:52,883][1157819] Updated weights for policy 0, policy_version 138478 (0.0006) [2024-09-30 02:36:53,451][1157819] Updated weights for policy 0, policy_version 138488 (0.0006) [2024-09-30 02:36:54,019][1157819] Updated weights for policy 0, policy_version 138498 (0.0006) [2024-09-30 02:36:54,606][1157819] Updated weights for policy 0, policy_version 138508 (0.0006) [2024-09-30 02:36:55,154][1157819] Updated weights for policy 0, policy_version 138518 (0.0006) [2024-09-30 02:36:55,466][1157520] Fps is (10 sec: 73317.5, 60 sec: 79667.1, 300 sec: 76130.0). Total num frames: 567390208. Throughput: 0: 19778.7. Samples: 131800400. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:36:55,466][1157520] Avg episode reward: [(0, '54.561')] [2024-09-30 02:36:55,689][1157819] Updated weights for policy 0, policy_version 138528 (0.0006) [2024-09-30 02:36:56,228][1157819] Updated weights for policy 0, policy_version 138538 (0.0006) [2024-09-30 02:36:56,736][1157819] Updated weights for policy 0, policy_version 138548 (0.0006) [2024-09-30 02:36:57,280][1157819] Updated weights for policy 0, policy_version 138558 (0.0006) [2024-09-30 02:36:57,823][1157819] Updated weights for policy 0, policy_version 138568 (0.0006) [2024-09-30 02:36:58,437][1157819] Updated weights for policy 0, policy_version 138578 (0.0006) [2024-09-30 02:36:59,053][1157819] Updated weights for policy 0, policy_version 138588 (0.0006) [2024-09-30 02:36:59,660][1157819] Updated weights for policy 0, policy_version 138598 (0.0006) [2024-09-30 02:37:00,263][1157819] Updated weights for policy 0, policy_version 138608 (0.0006) [2024-09-30 02:37:00,466][1157520] Fps is (10 sec: 72090.4, 60 sec: 78438.4, 300 sec: 76116.2). Total num frames: 567750656. Throughput: 0: 19424.7. Samples: 131910316. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:37:00,466][1157520] Avg episode reward: [(0, '52.572')] [2024-09-30 02:37:00,859][1157819] Updated weights for policy 0, policy_version 138618 (0.0006) [2024-09-30 02:37:01,451][1157819] Updated weights for policy 0, policy_version 138628 (0.0007) [2024-09-30 02:37:02,093][1157819] Updated weights for policy 0, policy_version 138638 (0.0006) [2024-09-30 02:37:02,722][1157819] Updated weights for policy 0, policy_version 138648 (0.0006) [2024-09-30 02:37:03,333][1157819] Updated weights for policy 0, policy_version 138658 (0.0006) [2024-09-30 02:37:03,934][1157819] Updated weights for policy 0, policy_version 138668 (0.0006) [2024-09-30 02:37:04,555][1157819] Updated weights for policy 0, policy_version 138678 (0.0005) [2024-09-30 02:37:05,141][1157819] Updated weights for policy 0, policy_version 138688 (0.0006) [2024-09-30 02:37:05,466][1157520] Fps is (10 sec: 69633.0, 60 sec: 77346.2, 300 sec: 75991.2). Total num frames: 568086528. Throughput: 0: 18909.7. Samples: 132010800. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:37:05,466][1157520] Avg episode reward: [(0, '56.388')] [2024-09-30 02:37:05,726][1157819] Updated weights for policy 0, policy_version 138698 (0.0006) [2024-09-30 02:37:06,318][1157819] Updated weights for policy 0, policy_version 138708 (0.0006) [2024-09-30 02:37:06,910][1157819] Updated weights for policy 0, policy_version 138718 (0.0006) [2024-09-30 02:37:07,449][1157819] Updated weights for policy 0, policy_version 138728 (0.0006) [2024-09-30 02:37:07,965][1157819] Updated weights for policy 0, policy_version 138738 (0.0006) [2024-09-30 02:37:08,459][1157819] Updated weights for policy 0, policy_version 138748 (0.0006) [2024-09-30 02:37:08,967][1157819] Updated weights for policy 0, policy_version 138758 (0.0006) [2024-09-30 02:37:09,470][1157819] Updated weights for policy 0, policy_version 138768 (0.0006) [2024-09-30 02:37:10,059][1157819] Updated weights for policy 0, policy_version 138778 (0.0006) [2024-09-30 02:37:10,466][1157520] Fps is (10 sec: 71680.4, 60 sec: 76663.5, 300 sec: 76046.8). Total num frames: 568467456. Throughput: 0: 18764.1. Samples: 132064776. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:37:10,466][1157520] Avg episode reward: [(0, '54.674')] [2024-09-30 02:37:10,579][1157819] Updated weights for policy 0, policy_version 138788 (0.0006) [2024-09-30 02:37:11,157][1157819] Updated weights for policy 0, policy_version 138798 (0.0006) [2024-09-30 02:37:11,709][1157819] Updated weights for policy 0, policy_version 138808 (0.0006) [2024-09-30 02:37:12,300][1157819] Updated weights for policy 0, policy_version 138818 (0.0006) [2024-09-30 02:37:12,825][1157819] Updated weights for policy 0, policy_version 138828 (0.0006) [2024-09-30 02:37:13,362][1157819] Updated weights for policy 0, policy_version 138838 (0.0006) [2024-09-30 02:37:13,955][1157819] Updated weights for policy 0, policy_version 138848 (0.0006) [2024-09-30 02:37:14,513][1157819] Updated weights for policy 0, policy_version 138858 (0.0006) [2024-09-30 02:37:15,036][1157819] Updated weights for policy 0, policy_version 138868 (0.0006) [2024-09-30 02:37:15,466][1157520] Fps is (10 sec: 74957.4, 60 sec: 75912.7, 300 sec: 76074.6). Total num frames: 568836096. Throughput: 0: 18637.4. Samples: 132177400. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:37:15,466][1157520] Avg episode reward: [(0, '55.407')] [2024-09-30 02:37:15,556][1157819] Updated weights for policy 0, policy_version 138878 (0.0006) [2024-09-30 02:37:16,111][1157819] Updated weights for policy 0, policy_version 138888 (0.0006) [2024-09-30 02:37:16,285][1157736] Signal inference workers to stop experience collection... (9650 times) [2024-09-30 02:37:16,286][1157736] Signal inference workers to resume experience collection... (9650 times) [2024-09-30 02:37:16,289][1157819] InferenceWorker_p0-w0: stopping experience collection (9650 times) [2024-09-30 02:37:16,292][1157819] InferenceWorker_p0-w0: resuming experience collection (9650 times) [2024-09-30 02:37:16,623][1157819] Updated weights for policy 0, policy_version 138898 (0.0006) [2024-09-30 02:37:17,129][1157819] Updated weights for policy 0, policy_version 138908 (0.0006) [2024-09-30 02:37:17,635][1157819] Updated weights for policy 0, policy_version 138918 (0.0006) [2024-09-30 02:37:18,186][1157819] Updated weights for policy 0, policy_version 138928 (0.0006) [2024-09-30 02:37:18,694][1157819] Updated weights for policy 0, policy_version 138938 (0.0006) [2024-09-30 02:37:19,200][1157819] Updated weights for policy 0, policy_version 138948 (0.0006) [2024-09-30 02:37:19,672][1157819] Updated weights for policy 0, policy_version 138958 (0.0006) [2024-09-30 02:37:20,205][1157819] Updated weights for policy 0, policy_version 138968 (0.0006) [2024-09-30 02:37:20,466][1157520] Fps is (10 sec: 76594.6, 60 sec: 75639.5, 300 sec: 76227.3). Total num frames: 569233408. Throughput: 0: 18624.6. Samples: 132295116. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:37:20,466][1157520] Avg episode reward: [(0, '54.485')] [2024-09-30 02:37:20,718][1157819] Updated weights for policy 0, policy_version 138978 (0.0006) [2024-09-30 02:37:21,221][1157819] Updated weights for policy 0, policy_version 138988 (0.0006) [2024-09-30 02:37:21,726][1157819] Updated weights for policy 0, policy_version 138998 (0.0006) [2024-09-30 02:37:22,263][1157819] Updated weights for policy 0, policy_version 139008 (0.0006) [2024-09-30 02:37:22,798][1157819] Updated weights for policy 0, policy_version 139018 (0.0006) [2024-09-30 02:37:23,315][1157819] Updated weights for policy 0, policy_version 139028 (0.0006) [2024-09-30 02:37:23,814][1157819] Updated weights for policy 0, policy_version 139038 (0.0006) [2024-09-30 02:37:24,348][1157819] Updated weights for policy 0, policy_version 139048 (0.0006) [2024-09-30 02:37:24,869][1157819] Updated weights for policy 0, policy_version 139058 (0.0006) [2024-09-30 02:37:25,398][1157819] Updated weights for policy 0, policy_version 139068 (0.0006) [2024-09-30 02:37:25,466][1157520] Fps is (10 sec: 79051.6, 60 sec: 75571.1, 300 sec: 76352.2). Total num frames: 569626624. Throughput: 0: 18643.3. Samples: 132354460. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:37:25,466][1157520] Avg episode reward: [(0, '55.676')] [2024-09-30 02:37:25,901][1157819] Updated weights for policy 0, policy_version 139078 (0.0006) [2024-09-30 02:37:26,437][1157819] Updated weights for policy 0, policy_version 139088 (0.0006) [2024-09-30 02:37:26,939][1157819] Updated weights for policy 0, policy_version 139098 (0.0006) [2024-09-30 02:37:27,440][1157819] Updated weights for policy 0, policy_version 139108 (0.0006) [2024-09-30 02:37:27,982][1157819] Updated weights for policy 0, policy_version 139118 (0.0006) [2024-09-30 02:37:28,514][1157819] Updated weights for policy 0, policy_version 139128 (0.0006) [2024-09-30 02:37:28,995][1157819] Updated weights for policy 0, policy_version 139138 (0.0006) [2024-09-30 02:37:29,535][1157819] Updated weights for policy 0, policy_version 139148 (0.0006) [2024-09-30 02:37:30,076][1157819] Updated weights for policy 0, policy_version 139158 (0.0006) [2024-09-30 02:37:30,466][1157520] Fps is (10 sec: 79052.8, 60 sec: 75571.2, 300 sec: 76449.4). Total num frames: 570023936. Throughput: 0: 18679.7. Samples: 132472884. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:37:30,466][1157520] Avg episode reward: [(0, '56.036')] [2024-09-30 02:37:30,574][1157819] Updated weights for policy 0, policy_version 139168 (0.0006) [2024-09-30 02:37:31,070][1157819] Updated weights for policy 0, policy_version 139178 (0.0006) [2024-09-30 02:37:31,592][1157819] Updated weights for policy 0, policy_version 139188 (0.0006) [2024-09-30 02:37:32,093][1157819] Updated weights for policy 0, policy_version 139198 (0.0006) [2024-09-30 02:37:32,633][1157819] Updated weights for policy 0, policy_version 139208 (0.0006) [2024-09-30 02:37:33,169][1157819] Updated weights for policy 0, policy_version 139218 (0.0006) [2024-09-30 02:37:33,663][1157819] Updated weights for policy 0, policy_version 139228 (0.0006) [2024-09-30 02:37:34,179][1157819] Updated weights for policy 0, policy_version 139238 (0.0005) [2024-09-30 02:37:34,678][1157819] Updated weights for policy 0, policy_version 139248 (0.0006) [2024-09-30 02:37:35,210][1157819] Updated weights for policy 0, policy_version 139258 (0.0006) [2024-09-30 02:37:35,466][1157520] Fps is (10 sec: 79052.0, 60 sec: 75639.3, 300 sec: 76602.1). Total num frames: 570417152. Throughput: 0: 18788.9. Samples: 132591788. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:37:35,466][1157520] Avg episode reward: [(0, '55.465')] [2024-09-30 02:37:35,734][1157819] Updated weights for policy 0, policy_version 139268 (0.0006) [2024-09-30 02:37:36,260][1157819] Updated weights for policy 0, policy_version 139278 (0.0006) [2024-09-30 02:37:36,752][1157819] Updated weights for policy 0, policy_version 139288 (0.0006) [2024-09-30 02:37:37,242][1157819] Updated weights for policy 0, policy_version 139298 (0.0006) [2024-09-30 02:37:37,723][1157819] Updated weights for policy 0, policy_version 139308 (0.0006) [2024-09-30 02:37:38,232][1157819] Updated weights for policy 0, policy_version 139318 (0.0006) [2024-09-30 02:37:38,714][1157819] Updated weights for policy 0, policy_version 139328 (0.0006) [2024-09-30 02:37:39,210][1157819] Updated weights for policy 0, policy_version 139338 (0.0006) [2024-09-30 02:37:39,690][1157819] Updated weights for policy 0, policy_version 139348 (0.0006) [2024-09-30 02:37:40,185][1157819] Updated weights for policy 0, policy_version 139358 (0.0006) [2024-09-30 02:37:40,466][1157520] Fps is (10 sec: 80690.6, 60 sec: 76049.0, 300 sec: 76810.4). Total num frames: 570830848. Throughput: 0: 18938.9. Samples: 132652648. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:37:40,466][1157520] Avg episode reward: [(0, '55.133')] [2024-09-30 02:37:40,646][1157819] Updated weights for policy 0, policy_version 139368 (0.0006) [2024-09-30 02:37:41,147][1157819] Updated weights for policy 0, policy_version 139378 (0.0006) [2024-09-30 02:37:41,638][1157819] Updated weights for policy 0, policy_version 139388 (0.0006) [2024-09-30 02:37:42,119][1157819] Updated weights for policy 0, policy_version 139398 (0.0006) [2024-09-30 02:37:42,582][1157819] Updated weights for policy 0, policy_version 139408 (0.0006) [2024-09-30 02:37:43,108][1157819] Updated weights for policy 0, policy_version 139418 (0.0006) [2024-09-30 02:37:43,626][1157819] Updated weights for policy 0, policy_version 139428 (0.0006) [2024-09-30 02:37:44,117][1157819] Updated weights for policy 0, policy_version 139438 (0.0006) [2024-09-30 02:37:44,568][1157819] Updated weights for policy 0, policy_version 139448 (0.0006) [2024-09-30 02:37:45,067][1157819] Updated weights for policy 0, policy_version 139458 (0.0006) [2024-09-30 02:37:45,466][1157520] Fps is (10 sec: 83148.8, 60 sec: 76526.7, 300 sec: 76977.0). Total num frames: 571248640. Throughput: 0: 19279.7. Samples: 132777904. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:37:45,466][1157520] Avg episode reward: [(0, '56.441')] [2024-09-30 02:37:45,548][1157819] Updated weights for policy 0, policy_version 139468 (0.0006) [2024-09-30 02:37:45,989][1157819] Updated weights for policy 0, policy_version 139478 (0.0006) [2024-09-30 02:37:46,497][1157819] Updated weights for policy 0, policy_version 139488 (0.0006) [2024-09-30 02:37:46,986][1157819] Updated weights for policy 0, policy_version 139498 (0.0006) [2024-09-30 02:37:47,478][1157819] Updated weights for policy 0, policy_version 139508 (0.0006) [2024-09-30 02:37:47,949][1157819] Updated weights for policy 0, policy_version 139518 (0.0006) [2024-09-30 02:37:48,444][1157819] Updated weights for policy 0, policy_version 139528 (0.0006) [2024-09-30 02:37:48,932][1157819] Updated weights for policy 0, policy_version 139538 (0.0006) [2024-09-30 02:37:49,427][1157819] Updated weights for policy 0, policy_version 139548 (0.0006) [2024-09-30 02:37:49,882][1157819] Updated weights for policy 0, policy_version 139558 (0.0006) [2024-09-30 02:37:50,339][1157819] Updated weights for policy 0, policy_version 139568 (0.0006) [2024-09-30 02:37:50,466][1157520] Fps is (10 sec: 84788.0, 60 sec: 77482.8, 300 sec: 77227.0). Total num frames: 571678720. Throughput: 0: 19880.7. Samples: 132905432. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:37:50,466][1157520] Avg episode reward: [(0, '55.276')] [2024-09-30 02:37:50,828][1157819] Updated weights for policy 0, policy_version 139578 (0.0006) [2024-09-30 02:37:51,321][1157819] Updated weights for policy 0, policy_version 139588 (0.0006) [2024-09-30 02:37:51,785][1157819] Updated weights for policy 0, policy_version 139598 (0.0006) [2024-09-30 02:37:52,250][1157819] Updated weights for policy 0, policy_version 139608 (0.0006) [2024-09-30 02:37:52,751][1157819] Updated weights for policy 0, policy_version 139618 (0.0006) [2024-09-30 02:37:53,201][1157819] Updated weights for policy 0, policy_version 139628 (0.0006) [2024-09-30 02:37:53,685][1157819] Updated weights for policy 0, policy_version 139638 (0.0006) [2024-09-30 02:37:54,164][1157819] Updated weights for policy 0, policy_version 139648 (0.0006) [2024-09-30 02:37:54,631][1157819] Updated weights for policy 0, policy_version 139658 (0.0006) [2024-09-30 02:37:55,113][1157819] Updated weights for policy 0, policy_version 139668 (0.0006) [2024-09-30 02:37:55,466][1157520] Fps is (10 sec: 86016.0, 60 sec: 78643.2, 300 sec: 77490.7). Total num frames: 572108800. Throughput: 0: 20114.3. Samples: 132969924. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:37:55,466][1157520] Avg episode reward: [(0, '55.792')] [2024-09-30 02:37:55,599][1157819] Updated weights for policy 0, policy_version 139678 (0.0006) [2024-09-30 02:37:56,090][1157819] Updated weights for policy 0, policy_version 139688 (0.0006) [2024-09-30 02:37:56,602][1157819] Updated weights for policy 0, policy_version 139698 (0.0006) [2024-09-30 02:37:57,056][1157819] Updated weights for policy 0, policy_version 139708 (0.0006) [2024-09-30 02:37:57,551][1157819] Updated weights for policy 0, policy_version 139718 (0.0006) [2024-09-30 02:37:58,089][1157819] Updated weights for policy 0, policy_version 139728 (0.0006) [2024-09-30 02:37:58,548][1157819] Updated weights for policy 0, policy_version 139738 (0.0006) [2024-09-30 02:37:59,030][1157819] Updated weights for policy 0, policy_version 139748 (0.0006) [2024-09-30 02:37:59,522][1157819] Updated weights for policy 0, policy_version 139758 (0.0006) [2024-09-30 02:38:00,026][1157819] Updated weights for policy 0, policy_version 139768 (0.0006) [2024-09-30 02:38:00,466][1157520] Fps is (10 sec: 84786.4, 60 sec: 79598.8, 300 sec: 77740.7). Total num frames: 572526592. Throughput: 0: 20423.5. Samples: 133096460. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:38:00,466][1157520] Avg episode reward: [(0, '55.198')] [2024-09-30 02:38:00,495][1157819] Updated weights for policy 0, policy_version 139778 (0.0006) [2024-09-30 02:38:00,985][1157819] Updated weights for policy 0, policy_version 139788 (0.0006) [2024-09-30 02:38:01,427][1157736] Signal inference workers to stop experience collection... (9700 times) [2024-09-30 02:38:01,432][1157819] InferenceWorker_p0-w0: stopping experience collection (9700 times) [2024-09-30 02:38:01,431][1157736] Signal inference workers to resume experience collection... (9700 times) [2024-09-30 02:38:01,436][1157819] InferenceWorker_p0-w0: resuming experience collection (9700 times) [2024-09-30 02:38:01,501][1157819] Updated weights for policy 0, policy_version 139798 (0.0006) [2024-09-30 02:38:02,017][1157819] Updated weights for policy 0, policy_version 139808 (0.0006) [2024-09-30 02:38:02,553][1157819] Updated weights for policy 0, policy_version 139818 (0.0006) [2024-09-30 02:38:03,049][1157819] Updated weights for policy 0, policy_version 139828 (0.0006) [2024-09-30 02:38:03,548][1157819] Updated weights for policy 0, policy_version 139838 (0.0006) [2024-09-30 02:38:04,051][1157819] Updated weights for policy 0, policy_version 139848 (0.0006) [2024-09-30 02:38:04,571][1157819] Updated weights for policy 0, policy_version 139858 (0.0006) [2024-09-30 02:38:05,055][1157819] Updated weights for policy 0, policy_version 139868 (0.0006) [2024-09-30 02:38:05,466][1157520] Fps is (10 sec: 81511.5, 60 sec: 80622.9, 300 sec: 77796.2). Total num frames: 572923904. Throughput: 0: 20529.4. Samples: 133218940. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:38:05,466][1157520] Avg episode reward: [(0, '53.197')] [2024-09-30 02:38:05,484][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000139875_572928000.pth... [2024-09-30 02:38:05,530][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000135261_554029056.pth [2024-09-30 02:38:05,622][1157819] Updated weights for policy 0, policy_version 139878 (0.0006) [2024-09-30 02:38:06,320][1157819] Updated weights for policy 0, policy_version 139888 (0.0006) [2024-09-30 02:38:06,952][1157819] Updated weights for policy 0, policy_version 139898 (0.0006) [2024-09-30 02:38:07,551][1157819] Updated weights for policy 0, policy_version 139908 (0.0006) [2024-09-30 02:38:08,209][1157819] Updated weights for policy 0, policy_version 139918 (0.0006) [2024-09-30 02:38:08,787][1157819] Updated weights for policy 0, policy_version 139928 (0.0006) [2024-09-30 02:38:09,386][1157819] Updated weights for policy 0, policy_version 139938 (0.0006) [2024-09-30 02:38:10,002][1157819] Updated weights for policy 0, policy_version 139948 (0.0006) [2024-09-30 02:38:10,466][1157520] Fps is (10 sec: 73318.6, 60 sec: 79871.8, 300 sec: 77685.2). Total num frames: 573259776. Throughput: 0: 20335.0. Samples: 133269536. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:38:10,466][1157520] Avg episode reward: [(0, '56.142')] [2024-09-30 02:38:10,587][1157819] Updated weights for policy 0, policy_version 139958 (0.0006) [2024-09-30 02:38:11,159][1157819] Updated weights for policy 0, policy_version 139968 (0.0006) [2024-09-30 02:38:11,690][1157819] Updated weights for policy 0, policy_version 139978 (0.0006) [2024-09-30 02:38:12,242][1157819] Updated weights for policy 0, policy_version 139988 (0.0006) [2024-09-30 02:38:12,806][1157819] Updated weights for policy 0, policy_version 139998 (0.0006) [2024-09-30 02:38:13,364][1157819] Updated weights for policy 0, policy_version 140008 (0.0006) [2024-09-30 02:38:13,932][1157819] Updated weights for policy 0, policy_version 140018 (0.0006) [2024-09-30 02:38:14,455][1157819] Updated weights for policy 0, policy_version 140028 (0.0006) [2024-09-30 02:38:15,020][1157819] Updated weights for policy 0, policy_version 140038 (0.0006) [2024-09-30 02:38:15,466][1157520] Fps is (10 sec: 70451.3, 60 sec: 79871.8, 300 sec: 77629.6). Total num frames: 573628416. Throughput: 0: 20071.3. Samples: 133376092. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:38:15,466][1157520] Avg episode reward: [(0, '56.659')] [2024-09-30 02:38:15,541][1157819] Updated weights for policy 0, policy_version 140048 (0.0006) [2024-09-30 02:38:16,107][1157819] Updated weights for policy 0, policy_version 140058 (0.0006) [2024-09-30 02:38:16,641][1157819] Updated weights for policy 0, policy_version 140068 (0.0006) [2024-09-30 02:38:17,183][1157819] Updated weights for policy 0, policy_version 140078 (0.0006) [2024-09-30 02:38:17,729][1157819] Updated weights for policy 0, policy_version 140088 (0.0006) [2024-09-30 02:38:18,248][1157819] Updated weights for policy 0, policy_version 140098 (0.0006) [2024-09-30 02:38:18,755][1157819] Updated weights for policy 0, policy_version 140108 (0.0006) [2024-09-30 02:38:19,261][1157819] Updated weights for policy 0, policy_version 140118 (0.0006) [2024-09-30 02:38:19,750][1157819] Updated weights for policy 0, policy_version 140128 (0.0006) [2024-09-30 02:38:20,249][1157819] Updated weights for policy 0, policy_version 140138 (0.0006) [2024-09-30 02:38:20,466][1157520] Fps is (10 sec: 76185.9, 60 sec: 79803.7, 300 sec: 77629.6). Total num frames: 574021632. Throughput: 0: 20005.6. Samples: 133492036. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 02:38:20,466][1157520] Avg episode reward: [(0, '54.529')] [2024-09-30 02:38:20,731][1157819] Updated weights for policy 0, policy_version 140148 (0.0006) [2024-09-30 02:38:21,218][1157819] Updated weights for policy 0, policy_version 140158 (0.0006) [2024-09-30 02:38:21,716][1157819] Updated weights for policy 0, policy_version 140168 (0.0006) [2024-09-30 02:38:22,204][1157819] Updated weights for policy 0, policy_version 140178 (0.0006) [2024-09-30 02:38:22,693][1157819] Updated weights for policy 0, policy_version 140188 (0.0006) [2024-09-30 02:38:23,181][1157819] Updated weights for policy 0, policy_version 140198 (0.0006) [2024-09-30 02:38:23,671][1157819] Updated weights for policy 0, policy_version 140208 (0.0006) [2024-09-30 02:38:24,163][1157819] Updated weights for policy 0, policy_version 140218 (0.0006) [2024-09-30 02:38:24,644][1157819] Updated weights for policy 0, policy_version 140228 (0.0006) [2024-09-30 02:38:25,126][1157819] Updated weights for policy 0, policy_version 140238 (0.0006) [2024-09-30 02:38:25,466][1157520] Fps is (10 sec: 81510.1, 60 sec: 80281.6, 300 sec: 77726.8). Total num frames: 574443520. Throughput: 0: 20044.1. Samples: 133554632. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 02:38:25,466][1157520] Avg episode reward: [(0, '54.654')] [2024-09-30 02:38:25,611][1157819] Updated weights for policy 0, policy_version 140248 (0.0006) [2024-09-30 02:38:26,086][1157819] Updated weights for policy 0, policy_version 140258 (0.0006) [2024-09-30 02:38:26,592][1157819] Updated weights for policy 0, policy_version 140268 (0.0006) [2024-09-30 02:38:27,087][1157819] Updated weights for policy 0, policy_version 140278 (0.0006) [2024-09-30 02:38:27,640][1157819] Updated weights for policy 0, policy_version 140288 (0.0006) [2024-09-30 02:38:28,162][1157819] Updated weights for policy 0, policy_version 140298 (0.0006) [2024-09-30 02:38:28,750][1157819] Updated weights for policy 0, policy_version 140308 (0.0006) [2024-09-30 02:38:29,136][1157736] Signal inference workers to stop experience collection... (9750 times) [2024-09-30 02:38:29,137][1157736] Signal inference workers to resume experience collection... (9750 times) [2024-09-30 02:38:29,141][1157819] InferenceWorker_p0-w0: stopping experience collection (9750 times) [2024-09-30 02:38:29,141][1157819] InferenceWorker_p0-w0: resuming experience collection (9750 times) [2024-09-30 02:38:29,262][1157819] Updated weights for policy 0, policy_version 140318 (0.0006) [2024-09-30 02:38:29,762][1157819] Updated weights for policy 0, policy_version 140328 (0.0006) [2024-09-30 02:38:30,276][1157819] Updated weights for policy 0, policy_version 140338 (0.0006) [2024-09-30 02:38:30,466][1157520] Fps is (10 sec: 81100.8, 60 sec: 80145.1, 300 sec: 77740.7). Total num frames: 574832640. Throughput: 0: 19981.0. Samples: 133677048. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 02:38:30,466][1157520] Avg episode reward: [(0, '55.340')] [2024-09-30 02:38:30,840][1157819] Updated weights for policy 0, policy_version 140348 (0.0006) [2024-09-30 02:38:31,364][1157819] Updated weights for policy 0, policy_version 140358 (0.0006) [2024-09-30 02:38:31,892][1157819] Updated weights for policy 0, policy_version 140368 (0.0006) [2024-09-30 02:38:32,445][1157819] Updated weights for policy 0, policy_version 140378 (0.0006) [2024-09-30 02:38:32,990][1157819] Updated weights for policy 0, policy_version 140388 (0.0006) [2024-09-30 02:38:33,500][1157819] Updated weights for policy 0, policy_version 140398 (0.0006) [2024-09-30 02:38:34,035][1157819] Updated weights for policy 0, policy_version 140408 (0.0006) [2024-09-30 02:38:34,572][1157819] Updated weights for policy 0, policy_version 140418 (0.0007) [2024-09-30 02:38:35,177][1157819] Updated weights for policy 0, policy_version 140428 (0.0007) [2024-09-30 02:38:35,466][1157520] Fps is (10 sec: 76595.0, 60 sec: 79872.1, 300 sec: 77907.3). Total num frames: 575209472. Throughput: 0: 19696.5. Samples: 133791776. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 02:38:35,466][1157520] Avg episode reward: [(0, '56.070')] [2024-09-30 02:38:35,743][1157819] Updated weights for policy 0, policy_version 140438 (0.0007) [2024-09-30 02:38:36,321][1157819] Updated weights for policy 0, policy_version 140448 (0.0006) [2024-09-30 02:38:36,919][1157819] Updated weights for policy 0, policy_version 140458 (0.0006) [2024-09-30 02:38:37,559][1157819] Updated weights for policy 0, policy_version 140468 (0.0006) [2024-09-30 02:38:38,166][1157819] Updated weights for policy 0, policy_version 140478 (0.0006) [2024-09-30 02:38:38,739][1157819] Updated weights for policy 0, policy_version 140488 (0.0006) [2024-09-30 02:38:39,321][1157819] Updated weights for policy 0, policy_version 140498 (0.0006) [2024-09-30 02:38:39,908][1157819] Updated weights for policy 0, policy_version 140508 (0.0006) [2024-09-30 02:38:40,466][1157520] Fps is (10 sec: 72499.0, 60 sec: 78779.8, 300 sec: 77921.2). Total num frames: 575557632. Throughput: 0: 19409.3. Samples: 133843340. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 02:38:40,466][1157520] Avg episode reward: [(0, '54.121')] [2024-09-30 02:38:40,491][1157819] Updated weights for policy 0, policy_version 140518 (0.0006) [2024-09-30 02:38:41,053][1157819] Updated weights for policy 0, policy_version 140528 (0.0006) [2024-09-30 02:38:41,626][1157819] Updated weights for policy 0, policy_version 140538 (0.0006) [2024-09-30 02:38:42,220][1157819] Updated weights for policy 0, policy_version 140548 (0.0006) [2024-09-30 02:38:42,773][1157819] Updated weights for policy 0, policy_version 140558 (0.0006) [2024-09-30 02:38:43,375][1157819] Updated weights for policy 0, policy_version 140568 (0.0006) [2024-09-30 02:38:43,958][1157819] Updated weights for policy 0, policy_version 140578 (0.0006) [2024-09-30 02:38:44,570][1157819] Updated weights for policy 0, policy_version 140588 (0.0006) [2024-09-30 02:38:45,115][1157819] Updated weights for policy 0, policy_version 140598 (0.0006) [2024-09-30 02:38:45,466][1157520] Fps is (10 sec: 70451.4, 60 sec: 77755.9, 300 sec: 77948.9). Total num frames: 575913984. Throughput: 0: 18944.3. Samples: 133948952. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 02:38:45,466][1157520] Avg episode reward: [(0, '54.706')] [2024-09-30 02:38:45,648][1157819] Updated weights for policy 0, policy_version 140608 (0.0006) [2024-09-30 02:38:46,177][1157819] Updated weights for policy 0, policy_version 140618 (0.0006) [2024-09-30 02:38:46,712][1157819] Updated weights for policy 0, policy_version 140628 (0.0006) [2024-09-30 02:38:47,287][1157819] Updated weights for policy 0, policy_version 140638 (0.0007) [2024-09-30 02:38:47,866][1157819] Updated weights for policy 0, policy_version 140648 (0.0007) [2024-09-30 02:38:48,459][1157819] Updated weights for policy 0, policy_version 140658 (0.0006) [2024-09-30 02:38:49,073][1157819] Updated weights for policy 0, policy_version 140668 (0.0006) [2024-09-30 02:38:49,641][1157819] Updated weights for policy 0, policy_version 140678 (0.0006) [2024-09-30 02:38:50,249][1157819] Updated weights for policy 0, policy_version 140688 (0.0006) [2024-09-30 02:38:50,466][1157520] Fps is (10 sec: 71680.3, 60 sec: 76595.2, 300 sec: 77893.4). Total num frames: 576274432. Throughput: 0: 18618.7. Samples: 134056780. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 02:38:50,466][1157520] Avg episode reward: [(0, '55.290')] [2024-09-30 02:38:50,761][1157819] Updated weights for policy 0, policy_version 140698 (0.0006) [2024-09-30 02:38:51,278][1157819] Updated weights for policy 0, policy_version 140708 (0.0006) [2024-09-30 02:38:51,838][1157819] Updated weights for policy 0, policy_version 140718 (0.0006) [2024-09-30 02:38:52,501][1157819] Updated weights for policy 0, policy_version 140728 (0.0006) [2024-09-30 02:38:53,102][1157819] Updated weights for policy 0, policy_version 140738 (0.0006) [2024-09-30 02:38:53,668][1157819] Updated weights for policy 0, policy_version 140748 (0.0006) [2024-09-30 02:38:53,800][1157736] Signal inference workers to stop experience collection... (9800 times) [2024-09-30 02:38:53,801][1157736] Signal inference workers to resume experience collection... (9800 times) [2024-09-30 02:38:53,806][1157819] InferenceWorker_p0-w0: stopping experience collection (9800 times) [2024-09-30 02:38:53,806][1157819] InferenceWorker_p0-w0: resuming experience collection (9800 times) [2024-09-30 02:38:54,316][1157819] Updated weights for policy 0, policy_version 140758 (0.0006) [2024-09-30 02:38:54,958][1157819] Updated weights for policy 0, policy_version 140768 (0.0006) [2024-09-30 02:38:55,466][1157520] Fps is (10 sec: 70860.9, 60 sec: 75230.0, 300 sec: 77824.0). Total num frames: 576622592. Throughput: 0: 18689.0. Samples: 134110540. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 02:38:55,466][1157520] Avg episode reward: [(0, '54.111')] [2024-09-30 02:38:55,543][1157819] Updated weights for policy 0, policy_version 140778 (0.0006) [2024-09-30 02:38:56,129][1157819] Updated weights for policy 0, policy_version 140788 (0.0006) [2024-09-30 02:38:56,757][1157819] Updated weights for policy 0, policy_version 140798 (0.0006) [2024-09-30 02:38:57,343][1157819] Updated weights for policy 0, policy_version 140808 (0.0006) [2024-09-30 02:38:57,932][1157819] Updated weights for policy 0, policy_version 140818 (0.0006) [2024-09-30 02:38:58,524][1157819] Updated weights for policy 0, policy_version 140828 (0.0006) [2024-09-30 02:38:59,136][1157819] Updated weights for policy 0, policy_version 140838 (0.0006) [2024-09-30 02:38:59,758][1157819] Updated weights for policy 0, policy_version 140848 (0.0006) [2024-09-30 02:39:00,326][1157819] Updated weights for policy 0, policy_version 140858 (0.0006) [2024-09-30 02:39:00,466][1157520] Fps is (10 sec: 68812.8, 60 sec: 73932.9, 300 sec: 77754.6). Total num frames: 576962560. Throughput: 0: 18582.1. Samples: 134212284. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 02:39:00,466][1157520] Avg episode reward: [(0, '55.882')] [2024-09-30 02:39:00,913][1157819] Updated weights for policy 0, policy_version 140868 (0.0006) [2024-09-30 02:39:01,503][1157819] Updated weights for policy 0, policy_version 140878 (0.0006) [2024-09-30 02:39:02,095][1157819] Updated weights for policy 0, policy_version 140888 (0.0006) [2024-09-30 02:39:02,715][1157819] Updated weights for policy 0, policy_version 140898 (0.0006) [2024-09-30 02:39:03,294][1157819] Updated weights for policy 0, policy_version 140908 (0.0006) [2024-09-30 02:39:03,911][1157819] Updated weights for policy 0, policy_version 140918 (0.0006) [2024-09-30 02:39:04,525][1157819] Updated weights for policy 0, policy_version 140928 (0.0006) [2024-09-30 02:39:05,075][1157819] Updated weights for policy 0, policy_version 140938 (0.0006) [2024-09-30 02:39:05,466][1157520] Fps is (10 sec: 68402.5, 60 sec: 73045.2, 300 sec: 77740.6). Total num frames: 577306624. Throughput: 0: 18291.8. Samples: 134315168. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 02:39:05,466][1157520] Avg episode reward: [(0, '56.034')] [2024-09-30 02:39:05,703][1157819] Updated weights for policy 0, policy_version 140948 (0.0006) [2024-09-30 02:39:06,289][1157819] Updated weights for policy 0, policy_version 140958 (0.0006) [2024-09-30 02:39:06,862][1157819] Updated weights for policy 0, policy_version 140968 (0.0006) [2024-09-30 02:39:07,463][1157819] Updated weights for policy 0, policy_version 140978 (0.0006) [2024-09-30 02:39:08,043][1157819] Updated weights for policy 0, policy_version 140988 (0.0006) [2024-09-30 02:39:08,604][1157819] Updated weights for policy 0, policy_version 140998 (0.0006) [2024-09-30 02:39:09,197][1157819] Updated weights for policy 0, policy_version 141008 (0.0006) [2024-09-30 02:39:09,787][1157819] Updated weights for policy 0, policy_version 141018 (0.0006) [2024-09-30 02:39:10,370][1157819] Updated weights for policy 0, policy_version 141028 (0.0006) [2024-09-30 02:39:10,466][1157520] Fps is (10 sec: 69222.4, 60 sec: 73250.2, 300 sec: 77699.1). Total num frames: 577654784. Throughput: 0: 18062.4. Samples: 134367440. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 02:39:10,466][1157520] Avg episode reward: [(0, '55.152')] [2024-09-30 02:39:10,985][1157819] Updated weights for policy 0, policy_version 141038 (0.0006) [2024-09-30 02:39:11,553][1157819] Updated weights for policy 0, policy_version 141048 (0.0006) [2024-09-30 02:39:12,142][1157819] Updated weights for policy 0, policy_version 141058 (0.0006) [2024-09-30 02:39:12,778][1157819] Updated weights for policy 0, policy_version 141068 (0.0006) [2024-09-30 02:39:13,351][1157819] Updated weights for policy 0, policy_version 141078 (0.0006) [2024-09-30 02:39:13,950][1157819] Updated weights for policy 0, policy_version 141088 (0.0006) [2024-09-30 02:39:14,553][1157819] Updated weights for policy 0, policy_version 141098 (0.0006) [2024-09-30 02:39:15,117][1157819] Updated weights for policy 0, policy_version 141108 (0.0006) [2024-09-30 02:39:15,466][1157520] Fps is (10 sec: 69632.8, 60 sec: 72908.8, 300 sec: 77699.0). Total num frames: 578002944. Throughput: 0: 17654.7. Samples: 134471508. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 02:39:15,466][1157520] Avg episode reward: [(0, '57.436')] [2024-09-30 02:39:15,710][1157819] Updated weights for policy 0, policy_version 141118 (0.0006) [2024-09-30 02:39:16,285][1157819] Updated weights for policy 0, policy_version 141128 (0.0006) [2024-09-30 02:39:16,858][1157819] Updated weights for policy 0, policy_version 141138 (0.0006) [2024-09-30 02:39:17,454][1157819] Updated weights for policy 0, policy_version 141148 (0.0006) [2024-09-30 02:39:18,038][1157819] Updated weights for policy 0, policy_version 141158 (0.0006) [2024-09-30 02:39:18,615][1157819] Updated weights for policy 0, policy_version 141168 (0.0006) [2024-09-30 02:39:19,179][1157819] Updated weights for policy 0, policy_version 141178 (0.0006) [2024-09-30 02:39:19,769][1157819] Updated weights for policy 0, policy_version 141188 (0.0007) [2024-09-30 02:39:20,324][1157819] Updated weights for policy 0, policy_version 141198 (0.0006) [2024-09-30 02:39:20,466][1157520] Fps is (10 sec: 70041.7, 60 sec: 72226.2, 300 sec: 77588.0). Total num frames: 578355200. Throughput: 0: 17454.1. Samples: 134577208. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 02:39:20,466][1157520] Avg episode reward: [(0, '55.384')] [2024-09-30 02:39:20,932][1157819] Updated weights for policy 0, policy_version 141208 (0.0006) [2024-09-30 02:39:21,501][1157819] Updated weights for policy 0, policy_version 141218 (0.0006) [2024-09-30 02:39:22,149][1157819] Updated weights for policy 0, policy_version 141228 (0.0006) [2024-09-30 02:39:22,732][1157819] Updated weights for policy 0, policy_version 141238 (0.0006) [2024-09-30 02:39:23,304][1157819] Updated weights for policy 0, policy_version 141248 (0.0006) [2024-09-30 02:39:23,850][1157819] Updated weights for policy 0, policy_version 141258 (0.0006) [2024-09-30 02:39:24,418][1157819] Updated weights for policy 0, policy_version 141268 (0.0006) [2024-09-30 02:39:25,000][1157819] Updated weights for policy 0, policy_version 141278 (0.0006) [2024-09-30 02:39:25,466][1157520] Fps is (10 sec: 70451.3, 60 sec: 71065.7, 300 sec: 77421.3). Total num frames: 578707456. Throughput: 0: 17463.6. Samples: 134629204. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 02:39:25,466][1157520] Avg episode reward: [(0, '54.256')] [2024-09-30 02:39:25,567][1157819] Updated weights for policy 0, policy_version 141288 (0.0006) [2024-09-30 02:39:26,185][1157819] Updated weights for policy 0, policy_version 141298 (0.0006) [2024-09-30 02:39:26,739][1157819] Updated weights for policy 0, policy_version 141308 (0.0006) [2024-09-30 02:39:27,336][1157819] Updated weights for policy 0, policy_version 141318 (0.0006) [2024-09-30 02:39:27,931][1157819] Updated weights for policy 0, policy_version 141328 (0.0006) [2024-09-30 02:39:28,487][1157819] Updated weights for policy 0, policy_version 141338 (0.0006) [2024-09-30 02:39:29,014][1157819] Updated weights for policy 0, policy_version 141348 (0.0006) [2024-09-30 02:39:29,541][1157819] Updated weights for policy 0, policy_version 141358 (0.0006) [2024-09-30 02:39:30,095][1157819] Updated weights for policy 0, policy_version 141368 (0.0006) [2024-09-30 02:39:30,466][1157520] Fps is (10 sec: 71270.4, 60 sec: 70587.8, 300 sec: 77268.6). Total num frames: 579067904. Throughput: 0: 17490.7. Samples: 134736032. Policy #0 lag: (min: 0.0, avg: 3.0, max: 6.0) [2024-09-30 02:39:30,466][1157520] Avg episode reward: [(0, '53.816')] [2024-09-30 02:39:30,627][1157736] Signal inference workers to stop experience collection... (9850 times) [2024-09-30 02:39:30,632][1157736] Signal inference workers to resume experience collection... (9850 times) [2024-09-30 02:39:30,632][1157819] InferenceWorker_p0-w0: stopping experience collection (9850 times) [2024-09-30 02:39:30,636][1157819] InferenceWorker_p0-w0: resuming experience collection (9850 times) [2024-09-30 02:39:30,650][1157819] Updated weights for policy 0, policy_version 141378 (0.0007) [2024-09-30 02:39:31,235][1157819] Updated weights for policy 0, policy_version 141388 (0.0006) [2024-09-30 02:39:31,782][1157819] Updated weights for policy 0, policy_version 141398 (0.0006) [2024-09-30 02:39:32,366][1157819] Updated weights for policy 0, policy_version 141408 (0.0006) [2024-09-30 02:39:32,957][1157819] Updated weights for policy 0, policy_version 141418 (0.0006) [2024-09-30 02:39:33,525][1157819] Updated weights for policy 0, policy_version 141428 (0.0006) [2024-09-30 02:39:34,071][1157819] Updated weights for policy 0, policy_version 141438 (0.0006) [2024-09-30 02:39:34,674][1157819] Updated weights for policy 0, policy_version 141448 (0.0006) [2024-09-30 02:39:35,267][1157819] Updated weights for policy 0, policy_version 141458 (0.0006) [2024-09-30 02:39:35,466][1157520] Fps is (10 sec: 71270.1, 60 sec: 70178.2, 300 sec: 77115.9). Total num frames: 579420160. Throughput: 0: 17513.7. Samples: 134844896. Policy #0 lag: (min: 0.0, avg: 3.0, max: 6.0) [2024-09-30 02:39:35,466][1157520] Avg episode reward: [(0, '56.118')] [2024-09-30 02:39:35,871][1157819] Updated weights for policy 0, policy_version 141468 (0.0006) [2024-09-30 02:39:36,432][1157819] Updated weights for policy 0, policy_version 141478 (0.0006) [2024-09-30 02:39:37,018][1157819] Updated weights for policy 0, policy_version 141488 (0.0006) [2024-09-30 02:39:37,532][1157819] Updated weights for policy 0, policy_version 141498 (0.0006) [2024-09-30 02:39:38,042][1157819] Updated weights for policy 0, policy_version 141508 (0.0006) [2024-09-30 02:39:38,602][1157819] Updated weights for policy 0, policy_version 141518 (0.0006) [2024-09-30 02:39:39,172][1157819] Updated weights for policy 0, policy_version 141528 (0.0006) [2024-09-30 02:39:39,693][1157819] Updated weights for policy 0, policy_version 141538 (0.0006) [2024-09-30 02:39:40,185][1157819] Updated weights for policy 0, policy_version 141548 (0.0006) [2024-09-30 02:39:40,466][1157520] Fps is (10 sec: 73318.5, 60 sec: 70724.3, 300 sec: 77129.8). Total num frames: 579801088. Throughput: 0: 17521.1. Samples: 134898988. Policy #0 lag: (min: 0.0, avg: 3.0, max: 6.0) [2024-09-30 02:39:40,466][1157520] Avg episode reward: [(0, '54.647')] [2024-09-30 02:39:40,686][1157819] Updated weights for policy 0, policy_version 141558 (0.0006) [2024-09-30 02:39:41,212][1157819] Updated weights for policy 0, policy_version 141568 (0.0006) [2024-09-30 02:39:41,745][1157819] Updated weights for policy 0, policy_version 141578 (0.0006) [2024-09-30 02:39:42,308][1157819] Updated weights for policy 0, policy_version 141588 (0.0006) [2024-09-30 02:39:42,835][1157819] Updated weights for policy 0, policy_version 141598 (0.0006) [2024-09-30 02:39:43,364][1157819] Updated weights for policy 0, policy_version 141608 (0.0006) [2024-09-30 02:39:43,920][1157819] Updated weights for policy 0, policy_version 141618 (0.0006) [2024-09-30 02:39:44,445][1157819] Updated weights for policy 0, policy_version 141628 (0.0006) [2024-09-30 02:39:44,979][1157819] Updated weights for policy 0, policy_version 141638 (0.0006) [2024-09-30 02:39:45,466][1157520] Fps is (10 sec: 76186.0, 60 sec: 71133.9, 300 sec: 77115.9). Total num frames: 580182016. Throughput: 0: 17828.9. Samples: 135014584. Policy #0 lag: (min: 0.0, avg: 3.0, max: 6.0) [2024-09-30 02:39:45,466][1157520] Avg episode reward: [(0, '53.656')] [2024-09-30 02:39:45,525][1157819] Updated weights for policy 0, policy_version 141648 (0.0006) [2024-09-30 02:39:46,060][1157819] Updated weights for policy 0, policy_version 141658 (0.0006) [2024-09-30 02:39:46,601][1157819] Updated weights for policy 0, policy_version 141668 (0.0006) [2024-09-30 02:39:47,122][1157819] Updated weights for policy 0, policy_version 141678 (0.0006) [2024-09-30 02:39:47,624][1157819] Updated weights for policy 0, policy_version 141688 (0.0006) [2024-09-30 02:39:48,133][1157819] Updated weights for policy 0, policy_version 141698 (0.0006) [2024-09-30 02:39:48,663][1157819] Updated weights for policy 0, policy_version 141708 (0.0006) [2024-09-30 02:39:49,245][1157819] Updated weights for policy 0, policy_version 141718 (0.0006) [2024-09-30 02:39:49,802][1157819] Updated weights for policy 0, policy_version 141728 (0.0006) [2024-09-30 02:39:50,332][1157819] Updated weights for policy 0, policy_version 141738 (0.0006) [2024-09-30 02:39:50,466][1157520] Fps is (10 sec: 77004.8, 60 sec: 71611.7, 300 sec: 77088.1). Total num frames: 580571136. Throughput: 0: 18105.3. Samples: 135129904. Policy #0 lag: (min: 0.0, avg: 3.0, max: 6.0) [2024-09-30 02:39:50,466][1157520] Avg episode reward: [(0, '55.014')] [2024-09-30 02:39:50,869][1157819] Updated weights for policy 0, policy_version 141748 (0.0006) [2024-09-30 02:39:51,391][1157819] Updated weights for policy 0, policy_version 141758 (0.0006) [2024-09-30 02:39:51,898][1157736] Signal inference workers to stop experience collection... (9900 times) [2024-09-30 02:39:51,898][1157736] Signal inference workers to resume experience collection... (9900 times) [2024-09-30 02:39:51,904][1157819] InferenceWorker_p0-w0: stopping experience collection (9900 times) [2024-09-30 02:39:51,904][1157819] InferenceWorker_p0-w0: resuming experience collection (9900 times) [2024-09-30 02:39:51,919][1157819] Updated weights for policy 0, policy_version 141768 (0.0006) [2024-09-30 02:39:52,493][1157819] Updated weights for policy 0, policy_version 141778 (0.0006) [2024-09-30 02:39:53,060][1157819] Updated weights for policy 0, policy_version 141788 (0.0006) [2024-09-30 02:39:53,651][1157819] Updated weights for policy 0, policy_version 141798 (0.0006) [2024-09-30 02:39:54,219][1157819] Updated weights for policy 0, policy_version 141808 (0.0006) [2024-09-30 02:39:54,831][1157819] Updated weights for policy 0, policy_version 141818 (0.0006) [2024-09-30 02:39:55,383][1157819] Updated weights for policy 0, policy_version 141828 (0.0006) [2024-09-30 02:39:55,466][1157520] Fps is (10 sec: 74956.7, 60 sec: 71816.6, 300 sec: 77060.3). Total num frames: 580931584. Throughput: 0: 18187.9. Samples: 135185896. Policy #0 lag: (min: 0.0, avg: 3.0, max: 6.0) [2024-09-30 02:39:55,466][1157520] Avg episode reward: [(0, '55.682')] [2024-09-30 02:39:55,969][1157819] Updated weights for policy 0, policy_version 141838 (0.0006) [2024-09-30 02:39:56,544][1157819] Updated weights for policy 0, policy_version 141848 (0.0006) [2024-09-30 02:39:57,136][1157819] Updated weights for policy 0, policy_version 141858 (0.0006) [2024-09-30 02:39:57,700][1157819] Updated weights for policy 0, policy_version 141868 (0.0006) [2024-09-30 02:39:58,211][1157819] Updated weights for policy 0, policy_version 141878 (0.0007) [2024-09-30 02:39:58,765][1157819] Updated weights for policy 0, policy_version 141888 (0.0006) [2024-09-30 02:39:59,319][1157819] Updated weights for policy 0, policy_version 141898 (0.0006) [2024-09-30 02:39:59,876][1157819] Updated weights for policy 0, policy_version 141908 (0.0006) [2024-09-30 02:40:00,422][1157819] Updated weights for policy 0, policy_version 141918 (0.0006) [2024-09-30 02:40:00,466][1157520] Fps is (10 sec: 72908.2, 60 sec: 72294.3, 300 sec: 77088.1). Total num frames: 581300224. Throughput: 0: 18266.3. Samples: 135293492. Policy #0 lag: (min: 0.0, avg: 3.0, max: 6.0) [2024-09-30 02:40:00,466][1157520] Avg episode reward: [(0, '55.250')] [2024-09-30 02:40:00,939][1157819] Updated weights for policy 0, policy_version 141928 (0.0006) [2024-09-30 02:40:01,481][1157819] Updated weights for policy 0, policy_version 141938 (0.0006) [2024-09-30 02:40:02,018][1157819] Updated weights for policy 0, policy_version 141948 (0.0006) [2024-09-30 02:40:02,534][1157819] Updated weights for policy 0, policy_version 141958 (0.0005) [2024-09-30 02:40:03,031][1157819] Updated weights for policy 0, policy_version 141968 (0.0005) [2024-09-30 02:40:03,508][1157819] Updated weights for policy 0, policy_version 141978 (0.0006) [2024-09-30 02:40:03,997][1157819] Updated weights for policy 0, policy_version 141988 (0.0006) [2024-09-30 02:40:04,497][1157819] Updated weights for policy 0, policy_version 141998 (0.0006) [2024-09-30 02:40:04,974][1157819] Updated weights for policy 0, policy_version 142008 (0.0006) [2024-09-30 02:40:05,461][1157819] Updated weights for policy 0, policy_version 142018 (0.0006) [2024-09-30 02:40:05,466][1157520] Fps is (10 sec: 77415.1, 60 sec: 73318.7, 300 sec: 77157.5). Total num frames: 581705728. Throughput: 0: 18551.9. Samples: 135412044. Policy #0 lag: (min: 0.0, avg: 3.0, max: 6.0) [2024-09-30 02:40:05,466][1157520] Avg episode reward: [(0, '54.230')] [2024-09-30 02:40:05,469][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000142018_581705728.pth... [2024-09-30 02:40:05,513][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000137561_563449856.pth [2024-09-30 02:40:05,964][1157819] Updated weights for policy 0, policy_version 142028 (0.0006) [2024-09-30 02:40:06,445][1157819] Updated weights for policy 0, policy_version 142038 (0.0006) [2024-09-30 02:40:06,960][1157819] Updated weights for policy 0, policy_version 142048 (0.0006) [2024-09-30 02:40:07,476][1157819] Updated weights for policy 0, policy_version 142058 (0.0007) [2024-09-30 02:40:07,993][1157819] Updated weights for policy 0, policy_version 142068 (0.0007) [2024-09-30 02:40:08,523][1157819] Updated weights for policy 0, policy_version 142078 (0.0006) [2024-09-30 02:40:09,014][1157819] Updated weights for policy 0, policy_version 142088 (0.0006) [2024-09-30 02:40:09,522][1157819] Updated weights for policy 0, policy_version 142098 (0.0006) [2024-09-30 02:40:10,051][1157819] Updated weights for policy 0, policy_version 142108 (0.0006) [2024-09-30 02:40:10,466][1157520] Fps is (10 sec: 80281.5, 60 sec: 74137.5, 300 sec: 77185.3). Total num frames: 582103040. Throughput: 0: 18761.9. Samples: 135473492. Policy #0 lag: (min: 0.0, avg: 3.0, max: 6.0) [2024-09-30 02:40:10,466][1157520] Avg episode reward: [(0, '55.399')] [2024-09-30 02:40:10,580][1157819] Updated weights for policy 0, policy_version 142118 (0.0006) [2024-09-30 02:40:11,138][1157819] Updated weights for policy 0, policy_version 142128 (0.0006) [2024-09-30 02:40:11,700][1157819] Updated weights for policy 0, policy_version 142138 (0.0006) [2024-09-30 02:40:12,258][1157819] Updated weights for policy 0, policy_version 142148 (0.0006) [2024-09-30 02:40:12,819][1157819] Updated weights for policy 0, policy_version 142158 (0.0006) [2024-09-30 02:40:13,371][1157819] Updated weights for policy 0, policy_version 142168 (0.0006) [2024-09-30 02:40:13,933][1157819] Updated weights for policy 0, policy_version 142178 (0.0006) [2024-09-30 02:40:14,494][1157819] Updated weights for policy 0, policy_version 142188 (0.0006) [2024-09-30 02:40:15,046][1157819] Updated weights for policy 0, policy_version 142198 (0.0006) [2024-09-30 02:40:15,466][1157520] Fps is (10 sec: 76594.4, 60 sec: 74478.9, 300 sec: 77088.1). Total num frames: 582471680. Throughput: 0: 18926.7. Samples: 135587736. Policy #0 lag: (min: 0.0, avg: 3.0, max: 6.0) [2024-09-30 02:40:15,466][1157520] Avg episode reward: [(0, '55.840')] [2024-09-30 02:40:15,601][1157819] Updated weights for policy 0, policy_version 142208 (0.0006) [2024-09-30 02:40:16,175][1157819] Updated weights for policy 0, policy_version 142218 (0.0006) [2024-09-30 02:40:16,716][1157819] Updated weights for policy 0, policy_version 142228 (0.0006) [2024-09-30 02:40:17,277][1157819] Updated weights for policy 0, policy_version 142238 (0.0006) [2024-09-30 02:40:17,843][1157819] Updated weights for policy 0, policy_version 142248 (0.0006) [2024-09-30 02:40:18,358][1157819] Updated weights for policy 0, policy_version 142258 (0.0006) [2024-09-30 02:40:18,909][1157819] Updated weights for policy 0, policy_version 142268 (0.0006) [2024-09-30 02:40:19,479][1157819] Updated weights for policy 0, policy_version 142278 (0.0006) [2024-09-30 02:40:20,048][1157819] Updated weights for policy 0, policy_version 142288 (0.0006) [2024-09-30 02:40:20,466][1157520] Fps is (10 sec: 73728.0, 60 sec: 74751.9, 300 sec: 77060.3). Total num frames: 582840320. Throughput: 0: 18961.8. Samples: 135698176. Policy #0 lag: (min: 0.0, avg: 3.0, max: 6.0) [2024-09-30 02:40:20,466][1157520] Avg episode reward: [(0, '56.679')] [2024-09-30 02:40:20,607][1157819] Updated weights for policy 0, policy_version 142298 (0.0006) [2024-09-30 02:40:21,107][1157819] Updated weights for policy 0, policy_version 142308 (0.0006) [2024-09-30 02:40:21,615][1157819] Updated weights for policy 0, policy_version 142318 (0.0006) [2024-09-30 02:40:22,130][1157819] Updated weights for policy 0, policy_version 142328 (0.0006) [2024-09-30 02:40:22,633][1157819] Updated weights for policy 0, policy_version 142338 (0.0006) [2024-09-30 02:40:23,148][1157819] Updated weights for policy 0, policy_version 142348 (0.0006) [2024-09-30 02:40:23,688][1157819] Updated weights for policy 0, policy_version 142358 (0.0006) [2024-09-30 02:40:24,214][1157819] Updated weights for policy 0, policy_version 142368 (0.0006) [2024-09-30 02:40:24,744][1157819] Updated weights for policy 0, policy_version 142378 (0.0006) [2024-09-30 02:40:25,286][1157819] Updated weights for policy 0, policy_version 142388 (0.0006) [2024-09-30 02:40:25,466][1157520] Fps is (10 sec: 76185.5, 60 sec: 75434.7, 300 sec: 77074.2). Total num frames: 583233536. Throughput: 0: 19074.8. Samples: 135757356. Policy #0 lag: (min: 0.0, avg: 3.0, max: 6.0) [2024-09-30 02:40:25,466][1157520] Avg episode reward: [(0, '55.094')] [2024-09-30 02:40:25,838][1157819] Updated weights for policy 0, policy_version 142398 (0.0006) [2024-09-30 02:40:26,362][1157819] Updated weights for policy 0, policy_version 142408 (0.0006) [2024-09-30 02:40:26,884][1157819] Updated weights for policy 0, policy_version 142418 (0.0006) [2024-09-30 02:40:27,447][1157819] Updated weights for policy 0, policy_version 142428 (0.0006) [2024-09-30 02:40:27,998][1157819] Updated weights for policy 0, policy_version 142438 (0.0006) [2024-09-30 02:40:28,522][1157819] Updated weights for policy 0, policy_version 142448 (0.0006) [2024-09-30 02:40:29,065][1157819] Updated weights for policy 0, policy_version 142458 (0.0006) [2024-09-30 02:40:29,634][1157819] Updated weights for policy 0, policy_version 142468 (0.0006) [2024-09-30 02:40:30,166][1157819] Updated weights for policy 0, policy_version 142478 (0.0006) [2024-09-30 02:40:30,466][1157520] Fps is (10 sec: 77004.9, 60 sec: 75707.6, 300 sec: 77032.6). Total num frames: 583610368. Throughput: 0: 19048.1. Samples: 135871748. Policy #0 lag: (min: 0.0, avg: 1.5, max: 4.0) [2024-09-30 02:40:30,466][1157520] Avg episode reward: [(0, '54.644')] [2024-09-30 02:40:30,716][1157819] Updated weights for policy 0, policy_version 142488 (0.0006) [2024-09-30 02:40:31,275][1157819] Updated weights for policy 0, policy_version 142498 (0.0006) [2024-09-30 02:40:31,788][1157819] Updated weights for policy 0, policy_version 142508 (0.0006) [2024-09-30 02:40:32,297][1157819] Updated weights for policy 0, policy_version 142518 (0.0006) [2024-09-30 02:40:32,821][1157819] Updated weights for policy 0, policy_version 142528 (0.0006) [2024-09-30 02:40:33,372][1157819] Updated weights for policy 0, policy_version 142538 (0.0006) [2024-09-30 02:40:33,919][1157819] Updated weights for policy 0, policy_version 142548 (0.0006) [2024-09-30 02:40:34,433][1157819] Updated weights for policy 0, policy_version 142558 (0.0006) [2024-09-30 02:40:34,991][1157819] Updated weights for policy 0, policy_version 142568 (0.0006) [2024-09-30 02:40:35,466][1157520] Fps is (10 sec: 75775.2, 60 sec: 76185.5, 300 sec: 77004.8). Total num frames: 583991296. Throughput: 0: 19024.0. Samples: 135985988. Policy #0 lag: (min: 0.0, avg: 1.5, max: 4.0) [2024-09-30 02:40:35,466][1157520] Avg episode reward: [(0, '54.975')] [2024-09-30 02:40:35,494][1157819] Updated weights for policy 0, policy_version 142578 (0.0005) [2024-09-30 02:40:35,742][1157736] Signal inference workers to stop experience collection... (9950 times) [2024-09-30 02:40:35,742][1157736] Signal inference workers to resume experience collection... (9950 times) [2024-09-30 02:40:35,746][1157819] InferenceWorker_p0-w0: stopping experience collection (9950 times) [2024-09-30 02:40:35,746][1157819] InferenceWorker_p0-w0: resuming experience collection (9950 times) [2024-09-30 02:40:36,023][1157819] Updated weights for policy 0, policy_version 142588 (0.0006) [2024-09-30 02:40:36,595][1157819] Updated weights for policy 0, policy_version 142598 (0.0006) [2024-09-30 02:40:37,116][1157819] Updated weights for policy 0, policy_version 142608 (0.0005) [2024-09-30 02:40:37,634][1157819] Updated weights for policy 0, policy_version 142618 (0.0006) [2024-09-30 02:40:38,166][1157819] Updated weights for policy 0, policy_version 142628 (0.0006) [2024-09-30 02:40:38,707][1157819] Updated weights for policy 0, policy_version 142638 (0.0006) [2024-09-30 02:40:39,238][1157819] Updated weights for policy 0, policy_version 142648 (0.0006) [2024-09-30 02:40:39,806][1157819] Updated weights for policy 0, policy_version 142658 (0.0006) [2024-09-30 02:40:40,330][1157819] Updated weights for policy 0, policy_version 142668 (0.0006) [2024-09-30 02:40:40,466][1157520] Fps is (10 sec: 76595.5, 60 sec: 76253.8, 300 sec: 76907.6). Total num frames: 584376320. Throughput: 0: 19062.0. Samples: 136043688. Policy #0 lag: (min: 0.0, avg: 1.5, max: 4.0) [2024-09-30 02:40:40,466][1157520] Avg episode reward: [(0, '56.919')] [2024-09-30 02:40:40,850][1157819] Updated weights for policy 0, policy_version 142678 (0.0006) [2024-09-30 02:40:41,383][1157819] Updated weights for policy 0, policy_version 142688 (0.0006) [2024-09-30 02:40:41,915][1157819] Updated weights for policy 0, policy_version 142698 (0.0006) [2024-09-30 02:40:42,438][1157819] Updated weights for policy 0, policy_version 142708 (0.0006) [2024-09-30 02:40:42,954][1157819] Updated weights for policy 0, policy_version 142718 (0.0006) [2024-09-30 02:40:43,453][1157819] Updated weights for policy 0, policy_version 142728 (0.0006) [2024-09-30 02:40:43,974][1157819] Updated weights for policy 0, policy_version 142738 (0.0006) [2024-09-30 02:40:44,485][1157819] Updated weights for policy 0, policy_version 142748 (0.0006) [2024-09-30 02:40:45,019][1157819] Updated weights for policy 0, policy_version 142758 (0.0006) [2024-09-30 02:40:45,462][1157819] Updated weights for policy 0, policy_version 142768 (0.0006) [2024-09-30 02:40:45,466][1157520] Fps is (10 sec: 78644.3, 60 sec: 76595.2, 300 sec: 76671.5). Total num frames: 584777728. Throughput: 0: 19253.9. Samples: 136159916. Policy #0 lag: (min: 0.0, avg: 1.5, max: 4.0) [2024-09-30 02:40:45,466][1157520] Avg episode reward: [(0, '56.553')] [2024-09-30 02:40:45,946][1157819] Updated weights for policy 0, policy_version 142778 (0.0006) [2024-09-30 02:40:46,445][1157819] Updated weights for policy 0, policy_version 142788 (0.0006) [2024-09-30 02:40:46,928][1157819] Updated weights for policy 0, policy_version 142798 (0.0006) [2024-09-30 02:40:47,408][1157819] Updated weights for policy 0, policy_version 142808 (0.0006) [2024-09-30 02:40:47,904][1157819] Updated weights for policy 0, policy_version 142818 (0.0006) [2024-09-30 02:40:48,377][1157819] Updated weights for policy 0, policy_version 142828 (0.0006) [2024-09-30 02:40:48,851][1157819] Updated weights for policy 0, policy_version 142838 (0.0006) [2024-09-30 02:40:49,333][1157819] Updated weights for policy 0, policy_version 142848 (0.0006) [2024-09-30 02:40:49,822][1157819] Updated weights for policy 0, policy_version 142858 (0.0006) [2024-09-30 02:40:50,272][1157819] Updated weights for policy 0, policy_version 142868 (0.0006) [2024-09-30 02:40:50,466][1157520] Fps is (10 sec: 82329.6, 60 sec: 77141.3, 300 sec: 76574.4). Total num frames: 585199616. Throughput: 0: 19421.2. Samples: 136286000. Policy #0 lag: (min: 0.0, avg: 1.5, max: 4.0) [2024-09-30 02:40:50,466][1157520] Avg episode reward: [(0, '54.708')] [2024-09-30 02:40:50,761][1157819] Updated weights for policy 0, policy_version 142878 (0.0006) [2024-09-30 02:40:51,246][1157819] Updated weights for policy 0, policy_version 142888 (0.0006) [2024-09-30 02:40:51,733][1157819] Updated weights for policy 0, policy_version 142898 (0.0006) [2024-09-30 02:40:52,226][1157819] Updated weights for policy 0, policy_version 142908 (0.0006) [2024-09-30 02:40:52,698][1157819] Updated weights for policy 0, policy_version 142918 (0.0006) [2024-09-30 02:40:53,172][1157819] Updated weights for policy 0, policy_version 142928 (0.0006) [2024-09-30 02:40:53,665][1157819] Updated weights for policy 0, policy_version 142938 (0.0006) [2024-09-30 02:40:54,128][1157819] Updated weights for policy 0, policy_version 142948 (0.0006) [2024-09-30 02:40:54,581][1157819] Updated weights for policy 0, policy_version 142958 (0.0006) [2024-09-30 02:40:55,024][1157819] Updated weights for policy 0, policy_version 142968 (0.0006) [2024-09-30 02:40:55,466][1157520] Fps is (10 sec: 85607.0, 60 sec: 78370.3, 300 sec: 76574.4). Total num frames: 585633792. Throughput: 0: 19476.3. Samples: 136349924. Policy #0 lag: (min: 0.0, avg: 1.5, max: 4.0) [2024-09-30 02:40:55,466][1157520] Avg episode reward: [(0, '54.172')] [2024-09-30 02:40:55,497][1157819] Updated weights for policy 0, policy_version 142978 (0.0006) [2024-09-30 02:40:55,922][1157819] Updated weights for policy 0, policy_version 142988 (0.0006) [2024-09-30 02:40:56,401][1157819] Updated weights for policy 0, policy_version 142998 (0.0006) [2024-09-30 02:40:56,869][1157819] Updated weights for policy 0, policy_version 143008 (0.0006) [2024-09-30 02:40:57,359][1157819] Updated weights for policy 0, policy_version 143018 (0.0006) [2024-09-30 02:40:57,846][1157819] Updated weights for policy 0, policy_version 143028 (0.0007) [2024-09-30 02:40:58,348][1157819] Updated weights for policy 0, policy_version 143038 (0.0006) [2024-09-30 02:40:58,852][1157819] Updated weights for policy 0, policy_version 143048 (0.0006) [2024-09-30 02:40:59,363][1157819] Updated weights for policy 0, policy_version 143058 (0.0006) [2024-09-30 02:40:59,873][1157819] Updated weights for policy 0, policy_version 143068 (0.0006) [2024-09-30 02:41:00,385][1157819] Updated weights for policy 0, policy_version 143078 (0.0006) [2024-09-30 02:41:00,466][1157520] Fps is (10 sec: 85196.7, 60 sec: 79189.4, 300 sec: 76629.9). Total num frames: 586051584. Throughput: 0: 19831.1. Samples: 136480136. Policy #0 lag: (min: 0.0, avg: 1.5, max: 4.0) [2024-09-30 02:41:00,466][1157520] Avg episode reward: [(0, '54.394')] [2024-09-30 02:41:00,903][1157819] Updated weights for policy 0, policy_version 143088 (0.0006) [2024-09-30 02:41:01,404][1157819] Updated weights for policy 0, policy_version 143098 (0.0006) [2024-09-30 02:41:01,912][1157819] Updated weights for policy 0, policy_version 143108 (0.0006) [2024-09-30 02:41:02,432][1157819] Updated weights for policy 0, policy_version 143118 (0.0006) [2024-09-30 02:41:02,971][1157819] Updated weights for policy 0, policy_version 143128 (0.0006) [2024-09-30 02:41:03,504][1157819] Updated weights for policy 0, policy_version 143138 (0.0006) [2024-09-30 02:41:04,013][1157819] Updated weights for policy 0, policy_version 143148 (0.0006) [2024-09-30 02:41:04,529][1157819] Updated weights for policy 0, policy_version 143158 (0.0007) [2024-09-30 02:41:05,041][1157819] Updated weights for policy 0, policy_version 143168 (0.0006) [2024-09-30 02:41:05,466][1157520] Fps is (10 sec: 81508.7, 60 sec: 79052.5, 300 sec: 76546.6). Total num frames: 586448896. Throughput: 0: 20022.5. Samples: 136599188. Policy #0 lag: (min: 0.0, avg: 1.5, max: 4.0) [2024-09-30 02:41:05,466][1157520] Avg episode reward: [(0, '55.327')] [2024-09-30 02:41:05,549][1157819] Updated weights for policy 0, policy_version 143178 (0.0006) [2024-09-30 02:41:06,069][1157819] Updated weights for policy 0, policy_version 143188 (0.0006) [2024-09-30 02:41:06,590][1157819] Updated weights for policy 0, policy_version 143198 (0.0006) [2024-09-30 02:41:07,118][1157819] Updated weights for policy 0, policy_version 143208 (0.0006) [2024-09-30 02:41:07,658][1157819] Updated weights for policy 0, policy_version 143218 (0.0006) [2024-09-30 02:41:08,168][1157819] Updated weights for policy 0, policy_version 143228 (0.0006) [2024-09-30 02:41:08,690][1157819] Updated weights for policy 0, policy_version 143238 (0.0006) [2024-09-30 02:41:09,201][1157819] Updated weights for policy 0, policy_version 143248 (0.0006) [2024-09-30 02:41:09,718][1157819] Updated weights for policy 0, policy_version 143258 (0.0006) [2024-09-30 02:41:10,272][1157819] Updated weights for policy 0, policy_version 143268 (0.0006) [2024-09-30 02:41:10,466][1157520] Fps is (10 sec: 78643.2, 60 sec: 78916.3, 300 sec: 76463.3). Total num frames: 586838016. Throughput: 0: 20016.4. Samples: 136658096. Policy #0 lag: (min: 0.0, avg: 1.5, max: 4.0) [2024-09-30 02:41:10,466][1157520] Avg episode reward: [(0, '56.414')] [2024-09-30 02:41:10,767][1157819] Updated weights for policy 0, policy_version 143278 (0.0005) [2024-09-30 02:41:11,261][1157819] Updated weights for policy 0, policy_version 143288 (0.0006) [2024-09-30 02:41:11,774][1157819] Updated weights for policy 0, policy_version 143298 (0.0006) [2024-09-30 02:41:12,346][1157819] Updated weights for policy 0, policy_version 143308 (0.0006) [2024-09-30 02:41:12,829][1157819] Updated weights for policy 0, policy_version 143318 (0.0006) [2024-09-30 02:41:13,321][1157819] Updated weights for policy 0, policy_version 143328 (0.0006) [2024-09-30 02:41:13,871][1157819] Updated weights for policy 0, policy_version 143338 (0.0006) [2024-09-30 02:41:14,368][1157819] Updated weights for policy 0, policy_version 143348 (0.0006) [2024-09-30 02:41:14,879][1157819] Updated weights for policy 0, policy_version 143358 (0.0006) [2024-09-30 02:41:15,432][1157819] Updated weights for policy 0, policy_version 143368 (0.0006) [2024-09-30 02:41:15,466][1157520] Fps is (10 sec: 79053.6, 60 sec: 79462.4, 300 sec: 76421.6). Total num frames: 587239424. Throughput: 0: 20116.1. Samples: 136776972. Policy #0 lag: (min: 0.0, avg: 1.5, max: 4.0) [2024-09-30 02:41:15,466][1157520] Avg episode reward: [(0, '53.754')] [2024-09-30 02:41:15,982][1157819] Updated weights for policy 0, policy_version 143378 (0.0006) [2024-09-30 02:41:16,498][1157819] Updated weights for policy 0, policy_version 143388 (0.0006) [2024-09-30 02:41:17,023][1157819] Updated weights for policy 0, policy_version 143398 (0.0006) [2024-09-30 02:41:17,612][1157819] Updated weights for policy 0, policy_version 143408 (0.0006) [2024-09-30 02:41:18,138][1157819] Updated weights for policy 0, policy_version 143418 (0.0006) [2024-09-30 02:41:18,730][1157819] Updated weights for policy 0, policy_version 143428 (0.0006) [2024-09-30 02:41:19,297][1157819] Updated weights for policy 0, policy_version 143438 (0.0006) [2024-09-30 02:41:19,813][1157819] Updated weights for policy 0, policy_version 143448 (0.0006) [2024-09-30 02:41:20,370][1157819] Updated weights for policy 0, policy_version 143458 (0.0006) [2024-09-30 02:41:20,466][1157520] Fps is (10 sec: 77004.9, 60 sec: 79462.5, 300 sec: 76324.4). Total num frames: 587608064. Throughput: 0: 20090.2. Samples: 136890044. Policy #0 lag: (min: 0.0, avg: 1.5, max: 4.0) [2024-09-30 02:41:20,466][1157520] Avg episode reward: [(0, '58.851')] [2024-09-30 02:41:20,467][1157736] Saving new best policy, reward=58.851! [2024-09-30 02:41:21,008][1157819] Updated weights for policy 0, policy_version 143468 (0.0006) [2024-09-30 02:41:21,598][1157819] Updated weights for policy 0, policy_version 143478 (0.0006) [2024-09-30 02:41:22,196][1157819] Updated weights for policy 0, policy_version 143488 (0.0006) [2024-09-30 02:41:22,777][1157819] Updated weights for policy 0, policy_version 143498 (0.0006) [2024-09-30 02:41:23,372][1157819] Updated weights for policy 0, policy_version 143508 (0.0006) [2024-09-30 02:41:23,947][1157819] Updated weights for policy 0, policy_version 143518 (0.0006) [2024-09-30 02:41:24,440][1157819] Updated weights for policy 0, policy_version 143528 (0.0006) [2024-09-30 02:41:24,529][1157736] Signal inference workers to stop experience collection... (10000 times) [2024-09-30 02:41:24,532][1157819] InferenceWorker_p0-w0: stopping experience collection (10000 times) [2024-09-30 02:41:24,536][1157736] Signal inference workers to resume experience collection... (10000 times) [2024-09-30 02:41:24,536][1157819] InferenceWorker_p0-w0: resuming experience collection (10000 times) [2024-09-30 02:41:25,015][1157819] Updated weights for policy 0, policy_version 143538 (0.0007) [2024-09-30 02:41:25,466][1157520] Fps is (10 sec: 72909.0, 60 sec: 78916.3, 300 sec: 76199.5). Total num frames: 587968512. Throughput: 0: 19969.1. Samples: 136942296. Policy #0 lag: (min: 0.0, avg: 1.5, max: 4.0) [2024-09-30 02:41:25,466][1157520] Avg episode reward: [(0, '56.547')] [2024-09-30 02:41:25,510][1157819] Updated weights for policy 0, policy_version 143548 (0.0006) [2024-09-30 02:41:26,028][1157819] Updated weights for policy 0, policy_version 143558 (0.0006) [2024-09-30 02:41:26,585][1157819] Updated weights for policy 0, policy_version 143568 (0.0006) [2024-09-30 02:41:27,140][1157819] Updated weights for policy 0, policy_version 143578 (0.0006) [2024-09-30 02:41:27,711][1157819] Updated weights for policy 0, policy_version 143588 (0.0006) [2024-09-30 02:41:28,289][1157819] Updated weights for policy 0, policy_version 143598 (0.0006) [2024-09-30 02:41:28,858][1157819] Updated weights for policy 0, policy_version 143608 (0.0006) [2024-09-30 02:41:29,449][1157819] Updated weights for policy 0, policy_version 143618 (0.0006) [2024-09-30 02:41:30,038][1157819] Updated weights for policy 0, policy_version 143628 (0.0006) [2024-09-30 02:41:30,466][1157520] Fps is (10 sec: 72088.8, 60 sec: 78643.1, 300 sec: 76102.3). Total num frames: 588328960. Throughput: 0: 19852.8. Samples: 137053296. Policy #0 lag: (min: 0.0, avg: 1.5, max: 4.0) [2024-09-30 02:41:30,466][1157520] Avg episode reward: [(0, '54.936')] [2024-09-30 02:41:30,607][1157819] Updated weights for policy 0, policy_version 143638 (0.0006) [2024-09-30 02:41:31,149][1157819] Updated weights for policy 0, policy_version 143648 (0.0006) [2024-09-30 02:41:31,741][1157819] Updated weights for policy 0, policy_version 143658 (0.0006) [2024-09-30 02:41:32,299][1157819] Updated weights for policy 0, policy_version 143668 (0.0006) [2024-09-30 02:41:32,880][1157819] Updated weights for policy 0, policy_version 143678 (0.0006) [2024-09-30 02:41:33,473][1157819] Updated weights for policy 0, policy_version 143688 (0.0006) [2024-09-30 02:41:34,010][1157819] Updated weights for policy 0, policy_version 143698 (0.0006) [2024-09-30 02:41:34,549][1157819] Updated weights for policy 0, policy_version 143708 (0.0006) [2024-09-30 02:41:35,101][1157819] Updated weights for policy 0, policy_version 143718 (0.0006) [2024-09-30 02:41:35,466][1157520] Fps is (10 sec: 72908.6, 60 sec: 78438.5, 300 sec: 76032.9). Total num frames: 588697600. Throughput: 0: 19451.2. Samples: 137161304. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 02:41:35,466][1157520] Avg episode reward: [(0, '53.096')] [2024-09-30 02:41:35,660][1157819] Updated weights for policy 0, policy_version 143728 (0.0006) [2024-09-30 02:41:36,227][1157819] Updated weights for policy 0, policy_version 143738 (0.0006) [2024-09-30 02:41:36,718][1157819] Updated weights for policy 0, policy_version 143748 (0.0006) [2024-09-30 02:41:37,304][1157819] Updated weights for policy 0, policy_version 143758 (0.0006) [2024-09-30 02:41:37,846][1157819] Updated weights for policy 0, policy_version 143768 (0.0006) [2024-09-30 02:41:38,352][1157819] Updated weights for policy 0, policy_version 143778 (0.0006) [2024-09-30 02:41:38,940][1157819] Updated weights for policy 0, policy_version 143788 (0.0006) [2024-09-30 02:41:39,459][1157819] Updated weights for policy 0, policy_version 143798 (0.0006) [2024-09-30 02:41:40,000][1157819] Updated weights for policy 0, policy_version 143808 (0.0006) [2024-09-30 02:41:40,466][1157520] Fps is (10 sec: 73729.0, 60 sec: 78165.4, 300 sec: 75963.4). Total num frames: 589066240. Throughput: 0: 19276.4. Samples: 137217364. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 02:41:40,466][1157520] Avg episode reward: [(0, '55.188')] [2024-09-30 02:41:40,573][1157819] Updated weights for policy 0, policy_version 143818 (0.0006) [2024-09-30 02:41:41,114][1157819] Updated weights for policy 0, policy_version 143828 (0.0006) [2024-09-30 02:41:41,692][1157819] Updated weights for policy 0, policy_version 143838 (0.0006) [2024-09-30 02:41:42,283][1157819] Updated weights for policy 0, policy_version 143848 (0.0006) [2024-09-30 02:41:42,821][1157819] Updated weights for policy 0, policy_version 143858 (0.0006) [2024-09-30 02:41:43,352][1157819] Updated weights for policy 0, policy_version 143868 (0.0006) [2024-09-30 02:41:43,915][1157819] Updated weights for policy 0, policy_version 143878 (0.0006) [2024-09-30 02:41:44,445][1157819] Updated weights for policy 0, policy_version 143888 (0.0006) [2024-09-30 02:41:45,036][1157819] Updated weights for policy 0, policy_version 143898 (0.0006) [2024-09-30 02:41:45,466][1157520] Fps is (10 sec: 74137.7, 60 sec: 77687.4, 300 sec: 75963.5). Total num frames: 589438976. Throughput: 0: 18855.0. Samples: 137328612. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 02:41:45,466][1157520] Avg episode reward: [(0, '55.964')] [2024-09-30 02:41:45,564][1157819] Updated weights for policy 0, policy_version 143908 (0.0006) [2024-09-30 02:41:46,143][1157819] Updated weights for policy 0, policy_version 143918 (0.0006) [2024-09-30 02:41:46,669][1157819] Updated weights for policy 0, policy_version 143928 (0.0006) [2024-09-30 02:41:47,175][1157819] Updated weights for policy 0, policy_version 143938 (0.0006) [2024-09-30 02:41:47,724][1157819] Updated weights for policy 0, policy_version 143948 (0.0006) [2024-09-30 02:41:48,261][1157819] Updated weights for policy 0, policy_version 143958 (0.0006) [2024-09-30 02:41:48,815][1157819] Updated weights for policy 0, policy_version 143968 (0.0006) [2024-09-30 02:41:49,384][1157819] Updated weights for policy 0, policy_version 143978 (0.0006) [2024-09-30 02:41:49,894][1157819] Updated weights for policy 0, policy_version 143988 (0.0006) [2024-09-30 02:41:50,445][1157819] Updated weights for policy 0, policy_version 143998 (0.0006) [2024-09-30 02:41:50,466][1157520] Fps is (10 sec: 74957.0, 60 sec: 76936.6, 300 sec: 76019.0). Total num frames: 589815808. Throughput: 0: 18717.4. Samples: 137441468. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 02:41:50,466][1157520] Avg episode reward: [(0, '54.749')] [2024-09-30 02:41:50,976][1157819] Updated weights for policy 0, policy_version 144008 (0.0006) [2024-09-30 02:41:51,531][1157819] Updated weights for policy 0, policy_version 144018 (0.0006) [2024-09-30 02:41:52,074][1157819] Updated weights for policy 0, policy_version 144028 (0.0006) [2024-09-30 02:41:52,596][1157819] Updated weights for policy 0, policy_version 144038 (0.0006) [2024-09-30 02:41:53,181][1157819] Updated weights for policy 0, policy_version 144048 (0.0006) [2024-09-30 02:41:53,824][1157819] Updated weights for policy 0, policy_version 144058 (0.0006) [2024-09-30 02:41:54,321][1157819] Updated weights for policy 0, policy_version 144068 (0.0006) [2024-09-30 02:41:54,900][1157819] Updated weights for policy 0, policy_version 144078 (0.0006) [2024-09-30 02:41:55,426][1157819] Updated weights for policy 0, policy_version 144088 (0.0006) [2024-09-30 02:41:55,466][1157520] Fps is (10 sec: 74957.1, 60 sec: 75912.5, 300 sec: 76060.6). Total num frames: 590188544. Throughput: 0: 18662.5. Samples: 137497908. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 02:41:55,466][1157520] Avg episode reward: [(0, '56.631')] [2024-09-30 02:41:55,980][1157819] Updated weights for policy 0, policy_version 144098 (0.0006) [2024-09-30 02:41:56,555][1157819] Updated weights for policy 0, policy_version 144108 (0.0006) [2024-09-30 02:41:57,071][1157819] Updated weights for policy 0, policy_version 144118 (0.0006) [2024-09-30 02:41:57,621][1157819] Updated weights for policy 0, policy_version 144128 (0.0006) [2024-09-30 02:41:58,127][1157819] Updated weights for policy 0, policy_version 144138 (0.0006) [2024-09-30 02:41:58,646][1157819] Updated weights for policy 0, policy_version 144148 (0.0006) [2024-09-30 02:41:59,145][1157819] Updated weights for policy 0, policy_version 144158 (0.0006) [2024-09-30 02:41:59,645][1157819] Updated weights for policy 0, policy_version 144168 (0.0006) [2024-09-30 02:42:00,142][1157819] Updated weights for policy 0, policy_version 144178 (0.0006) [2024-09-30 02:42:00,466][1157520] Fps is (10 sec: 76185.2, 60 sec: 75434.7, 300 sec: 76241.1). Total num frames: 590577664. Throughput: 0: 18527.8. Samples: 137610724. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 02:42:00,466][1157520] Avg episode reward: [(0, '57.033')] [2024-09-30 02:42:00,645][1157819] Updated weights for policy 0, policy_version 144188 (0.0006) [2024-09-30 02:42:01,160][1157819] Updated weights for policy 0, policy_version 144198 (0.0006) [2024-09-30 02:42:01,674][1157819] Updated weights for policy 0, policy_version 144208 (0.0006) [2024-09-30 02:42:02,186][1157819] Updated weights for policy 0, policy_version 144218 (0.0006) [2024-09-30 02:42:02,704][1157819] Updated weights for policy 0, policy_version 144228 (0.0006) [2024-09-30 02:42:03,222][1157819] Updated weights for policy 0, policy_version 144238 (0.0006) [2024-09-30 02:42:03,729][1157819] Updated weights for policy 0, policy_version 144248 (0.0006) [2024-09-30 02:42:04,223][1157819] Updated weights for policy 0, policy_version 144258 (0.0006) [2024-09-30 02:42:04,752][1157819] Updated weights for policy 0, policy_version 144268 (0.0006) [2024-09-30 02:42:05,242][1157819] Updated weights for policy 0, policy_version 144278 (0.0006) [2024-09-30 02:42:05,466][1157520] Fps is (10 sec: 79051.4, 60 sec: 75502.9, 300 sec: 76310.5). Total num frames: 590979072. Throughput: 0: 18695.7. Samples: 137731352. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 02:42:05,466][1157520] Avg episode reward: [(0, '56.462')] [2024-09-30 02:42:05,472][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000144282_590979072.pth... [2024-09-30 02:42:05,518][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000139875_572928000.pth [2024-09-30 02:42:05,544][1157736] Signal inference workers to stop experience collection... (10050 times) [2024-09-30 02:42:05,544][1157736] Signal inference workers to resume experience collection... (10050 times) [2024-09-30 02:42:05,549][1157819] InferenceWorker_p0-w0: stopping experience collection (10050 times) [2024-09-30 02:42:05,549][1157819] InferenceWorker_p0-w0: resuming experience collection (10050 times) [2024-09-30 02:42:05,765][1157819] Updated weights for policy 0, policy_version 144288 (0.0006) [2024-09-30 02:42:06,265][1157819] Updated weights for policy 0, policy_version 144298 (0.0006) [2024-09-30 02:42:06,823][1157819] Updated weights for policy 0, policy_version 144308 (0.0006) [2024-09-30 02:42:07,307][1157819] Updated weights for policy 0, policy_version 144318 (0.0006) [2024-09-30 02:42:07,827][1157819] Updated weights for policy 0, policy_version 144328 (0.0006) [2024-09-30 02:42:08,333][1157819] Updated weights for policy 0, policy_version 144338 (0.0006) [2024-09-30 02:42:08,827][1157819] Updated weights for policy 0, policy_version 144348 (0.0006) [2024-09-30 02:42:09,336][1157819] Updated weights for policy 0, policy_version 144358 (0.0006) [2024-09-30 02:42:09,849][1157819] Updated weights for policy 0, policy_version 144368 (0.0006) [2024-09-30 02:42:10,294][1157819] Updated weights for policy 0, policy_version 144378 (0.0006) [2024-09-30 02:42:10,466][1157520] Fps is (10 sec: 80691.5, 60 sec: 75776.1, 300 sec: 76435.5). Total num frames: 591384576. Throughput: 0: 18867.5. Samples: 137791332. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 02:42:10,466][1157520] Avg episode reward: [(0, '55.015')] [2024-09-30 02:42:10,791][1157819] Updated weights for policy 0, policy_version 144388 (0.0006) [2024-09-30 02:42:11,277][1157819] Updated weights for policy 0, policy_version 144398 (0.0006) [2024-09-30 02:42:11,761][1157819] Updated weights for policy 0, policy_version 144408 (0.0006) [2024-09-30 02:42:12,254][1157819] Updated weights for policy 0, policy_version 144418 (0.0006) [2024-09-30 02:42:12,728][1157819] Updated weights for policy 0, policy_version 144428 (0.0005) [2024-09-30 02:42:13,194][1157819] Updated weights for policy 0, policy_version 144438 (0.0006) [2024-09-30 02:42:13,694][1157819] Updated weights for policy 0, policy_version 144448 (0.0006) [2024-09-30 02:42:14,175][1157819] Updated weights for policy 0, policy_version 144458 (0.0006) [2024-09-30 02:42:14,663][1157819] Updated weights for policy 0, policy_version 144468 (0.0006) [2024-09-30 02:42:15,160][1157819] Updated weights for policy 0, policy_version 144478 (0.0006) [2024-09-30 02:42:15,466][1157520] Fps is (10 sec: 82740.5, 60 sec: 76117.4, 300 sec: 76518.8). Total num frames: 591806464. Throughput: 0: 19189.2. Samples: 137916808. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 02:42:15,466][1157520] Avg episode reward: [(0, '57.039')] [2024-09-30 02:42:15,643][1157819] Updated weights for policy 0, policy_version 144488 (0.0006) [2024-09-30 02:42:16,113][1157819] Updated weights for policy 0, policy_version 144498 (0.0006) [2024-09-30 02:42:16,613][1157819] Updated weights for policy 0, policy_version 144508 (0.0006) [2024-09-30 02:42:17,094][1157819] Updated weights for policy 0, policy_version 144518 (0.0006) [2024-09-30 02:42:17,587][1157819] Updated weights for policy 0, policy_version 144528 (0.0006) [2024-09-30 02:42:18,095][1157819] Updated weights for policy 0, policy_version 144538 (0.0006) [2024-09-30 02:42:18,588][1157819] Updated weights for policy 0, policy_version 144548 (0.0006) [2024-09-30 02:42:19,075][1157819] Updated weights for policy 0, policy_version 144558 (0.0006) [2024-09-30 02:42:19,566][1157819] Updated weights for policy 0, policy_version 144568 (0.0006) [2024-09-30 02:42:20,050][1157819] Updated weights for policy 0, policy_version 144578 (0.0006) [2024-09-30 02:42:20,466][1157520] Fps is (10 sec: 83968.2, 60 sec: 76936.6, 300 sec: 76602.2). Total num frames: 592224256. Throughput: 0: 19574.4. Samples: 138042152. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 02:42:20,466][1157520] Avg episode reward: [(0, '54.733')] [2024-09-30 02:42:20,546][1157819] Updated weights for policy 0, policy_version 144588 (0.0006) [2024-09-30 02:42:21,042][1157819] Updated weights for policy 0, policy_version 144598 (0.0006) [2024-09-30 02:42:21,475][1157819] Updated weights for policy 0, policy_version 144608 (0.0006) [2024-09-30 02:42:21,969][1157819] Updated weights for policy 0, policy_version 144618 (0.0006) [2024-09-30 02:42:22,473][1157819] Updated weights for policy 0, policy_version 144628 (0.0006) [2024-09-30 02:42:22,938][1157819] Updated weights for policy 0, policy_version 144638 (0.0006) [2024-09-30 02:42:23,424][1157819] Updated weights for policy 0, policy_version 144648 (0.0006) [2024-09-30 02:42:23,904][1157819] Updated weights for policy 0, policy_version 144658 (0.0006) [2024-09-30 02:42:24,400][1157819] Updated weights for policy 0, policy_version 144668 (0.0005) [2024-09-30 02:42:24,904][1157819] Updated weights for policy 0, policy_version 144678 (0.0006) [2024-09-30 02:42:25,371][1157819] Updated weights for policy 0, policy_version 144688 (0.0006) [2024-09-30 02:42:25,466][1157520] Fps is (10 sec: 83968.0, 60 sec: 77960.5, 300 sec: 76685.4). Total num frames: 592646144. Throughput: 0: 19738.0. Samples: 138105576. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 02:42:25,466][1157520] Avg episode reward: [(0, '55.096')] [2024-09-30 02:42:25,866][1157819] Updated weights for policy 0, policy_version 144698 (0.0006) [2024-09-30 02:42:26,322][1157819] Updated weights for policy 0, policy_version 144708 (0.0006) [2024-09-30 02:42:26,762][1157819] Updated weights for policy 0, policy_version 144718 (0.0006) [2024-09-30 02:42:27,216][1157819] Updated weights for policy 0, policy_version 144728 (0.0006) [2024-09-30 02:42:27,707][1157819] Updated weights for policy 0, policy_version 144738 (0.0006) [2024-09-30 02:42:28,185][1157819] Updated weights for policy 0, policy_version 144748 (0.0006) [2024-09-30 02:42:28,609][1157819] Updated weights for policy 0, policy_version 144758 (0.0006) [2024-09-30 02:42:29,078][1157819] Updated weights for policy 0, policy_version 144768 (0.0006) [2024-09-30 02:42:29,558][1157819] Updated weights for policy 0, policy_version 144778 (0.0006) [2024-09-30 02:42:30,030][1157819] Updated weights for policy 0, policy_version 144788 (0.0006) [2024-09-30 02:42:30,466][1157520] Fps is (10 sec: 86426.2, 60 sec: 79326.2, 300 sec: 76852.1). Total num frames: 593088512. Throughput: 0: 20157.0. Samples: 138235676. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 02:42:30,466][1157520] Avg episode reward: [(0, '55.229')] [2024-09-30 02:42:30,478][1157819] Updated weights for policy 0, policy_version 144798 (0.0006) [2024-09-30 02:42:30,954][1157819] Updated weights for policy 0, policy_version 144808 (0.0006) [2024-09-30 02:42:31,400][1157819] Updated weights for policy 0, policy_version 144818 (0.0006) [2024-09-30 02:42:31,880][1157819] Updated weights for policy 0, policy_version 144828 (0.0006) [2024-09-30 02:42:32,327][1157819] Updated weights for policy 0, policy_version 144838 (0.0006) [2024-09-30 02:42:32,812][1157819] Updated weights for policy 0, policy_version 144848 (0.0006) [2024-09-30 02:42:33,279][1157819] Updated weights for policy 0, policy_version 144858 (0.0006) [2024-09-30 02:42:33,739][1157819] Updated weights for policy 0, policy_version 144868 (0.0006) [2024-09-30 02:42:34,190][1157819] Updated weights for policy 0, policy_version 144878 (0.0006) [2024-09-30 02:42:34,652][1157819] Updated weights for policy 0, policy_version 144888 (0.0006) [2024-09-30 02:42:35,106][1157819] Updated weights for policy 0, policy_version 144898 (0.0006) [2024-09-30 02:42:35,466][1157520] Fps is (10 sec: 88884.1, 60 sec: 80623.1, 300 sec: 76963.2). Total num frames: 593534976. Throughput: 0: 20597.5. Samples: 138368356. Policy #0 lag: (min: 0.0, avg: 2.6, max: 7.0) [2024-09-30 02:42:35,466][1157520] Avg episode reward: [(0, '55.437')] [2024-09-30 02:42:35,541][1157819] Updated weights for policy 0, policy_version 144908 (0.0006) [2024-09-30 02:42:35,978][1157819] Updated weights for policy 0, policy_version 144918 (0.0006) [2024-09-30 02:42:36,472][1157819] Updated weights for policy 0, policy_version 144928 (0.0006) [2024-09-30 02:42:36,957][1157819] Updated weights for policy 0, policy_version 144938 (0.0006) [2024-09-30 02:42:37,417][1157819] Updated weights for policy 0, policy_version 144948 (0.0006) [2024-09-30 02:42:37,901][1157819] Updated weights for policy 0, policy_version 144958 (0.0006) [2024-09-30 02:42:38,397][1157819] Updated weights for policy 0, policy_version 144968 (0.0006) [2024-09-30 02:42:38,890][1157819] Updated weights for policy 0, policy_version 144978 (0.0006) [2024-09-30 02:42:39,381][1157819] Updated weights for policy 0, policy_version 144988 (0.0006) [2024-09-30 02:42:39,846][1157819] Updated weights for policy 0, policy_version 144998 (0.0006) [2024-09-30 02:42:40,328][1157819] Updated weights for policy 0, policy_version 145008 (0.0006) [2024-09-30 02:42:40,466][1157520] Fps is (10 sec: 87654.5, 60 sec: 81647.1, 300 sec: 77004.9). Total num frames: 593965056. Throughput: 0: 20811.7. Samples: 138434432. Policy #0 lag: (min: 0.0, avg: 2.6, max: 7.0) [2024-09-30 02:42:40,466][1157520] Avg episode reward: [(0, '56.489')] [2024-09-30 02:42:40,816][1157819] Updated weights for policy 0, policy_version 145018 (0.0006) [2024-09-30 02:42:41,306][1157819] Updated weights for policy 0, policy_version 145028 (0.0006) [2024-09-30 02:42:41,800][1157819] Updated weights for policy 0, policy_version 145038 (0.0006) [2024-09-30 02:42:42,284][1157819] Updated weights for policy 0, policy_version 145048 (0.0006) [2024-09-30 02:42:42,752][1157819] Updated weights for policy 0, policy_version 145058 (0.0006) [2024-09-30 02:42:43,250][1157819] Updated weights for policy 0, policy_version 145068 (0.0006) [2024-09-30 02:42:43,745][1157819] Updated weights for policy 0, policy_version 145078 (0.0006) [2024-09-30 02:42:44,238][1157819] Updated weights for policy 0, policy_version 145088 (0.0006) [2024-09-30 02:42:44,717][1157819] Updated weights for policy 0, policy_version 145098 (0.0006) [2024-09-30 02:42:45,195][1157819] Updated weights for policy 0, policy_version 145108 (0.0006) [2024-09-30 02:42:45,466][1157520] Fps is (10 sec: 84786.7, 60 sec: 82398.0, 300 sec: 76963.1). Total num frames: 594382848. Throughput: 0: 21115.4. Samples: 138560916. Policy #0 lag: (min: 0.0, avg: 2.6, max: 7.0) [2024-09-30 02:42:45,466][1157520] Avg episode reward: [(0, '54.793')] [2024-09-30 02:42:45,674][1157819] Updated weights for policy 0, policy_version 145118 (0.0006) [2024-09-30 02:42:46,158][1157819] Updated weights for policy 0, policy_version 145128 (0.0006) [2024-09-30 02:42:46,643][1157819] Updated weights for policy 0, policy_version 145138 (0.0006) [2024-09-30 02:42:47,129][1157819] Updated weights for policy 0, policy_version 145148 (0.0006) [2024-09-30 02:42:47,615][1157819] Updated weights for policy 0, policy_version 145158 (0.0006) [2024-09-30 02:42:48,137][1157819] Updated weights for policy 0, policy_version 145168 (0.0006) [2024-09-30 02:42:48,593][1157819] Updated weights for policy 0, policy_version 145178 (0.0006) [2024-09-30 02:42:49,083][1157819] Updated weights for policy 0, policy_version 145188 (0.0006) [2024-09-30 02:42:49,596][1157819] Updated weights for policy 0, policy_version 145198 (0.0006) [2024-09-30 02:42:50,093][1157819] Updated weights for policy 0, policy_version 145208 (0.0006) [2024-09-30 02:42:50,466][1157520] Fps is (10 sec: 83148.6, 60 sec: 83012.4, 300 sec: 76907.7). Total num frames: 594796544. Throughput: 0: 21228.1. Samples: 138686612. Policy #0 lag: (min: 0.0, avg: 2.6, max: 7.0) [2024-09-30 02:42:50,466][1157520] Avg episode reward: [(0, '54.427')] [2024-09-30 02:42:50,610][1157819] Updated weights for policy 0, policy_version 145218 (0.0006) [2024-09-30 02:42:51,118][1157819] Updated weights for policy 0, policy_version 145228 (0.0006) [2024-09-30 02:42:51,656][1157819] Updated weights for policy 0, policy_version 145238 (0.0006) [2024-09-30 02:42:52,179][1157819] Updated weights for policy 0, policy_version 145248 (0.0006) [2024-09-30 02:42:52,694][1157819] Updated weights for policy 0, policy_version 145258 (0.0006) [2024-09-30 02:42:53,193][1157819] Updated weights for policy 0, policy_version 145268 (0.0006) [2024-09-30 02:42:53,696][1157819] Updated weights for policy 0, policy_version 145278 (0.0006) [2024-09-30 02:42:53,770][1157736] Signal inference workers to stop experience collection... (10100 times) [2024-09-30 02:42:53,771][1157736] Signal inference workers to resume experience collection... (10100 times) [2024-09-30 02:42:53,775][1157819] InferenceWorker_p0-w0: stopping experience collection (10100 times) [2024-09-30 02:42:53,776][1157819] InferenceWorker_p0-w0: resuming experience collection (10100 times) [2024-09-30 02:42:54,218][1157819] Updated weights for policy 0, policy_version 145288 (0.0006) [2024-09-30 02:42:54,737][1157819] Updated weights for policy 0, policy_version 145298 (0.0006) [2024-09-30 02:42:55,240][1157819] Updated weights for policy 0, policy_version 145308 (0.0006) [2024-09-30 02:42:55,466][1157520] Fps is (10 sec: 81510.8, 60 sec: 83490.2, 300 sec: 76852.1). Total num frames: 595197952. Throughput: 0: 21223.4. Samples: 138746384. Policy #0 lag: (min: 0.0, avg: 2.6, max: 7.0) [2024-09-30 02:42:55,466][1157520] Avg episode reward: [(0, '56.791')] [2024-09-30 02:42:55,756][1157819] Updated weights for policy 0, policy_version 145318 (0.0006) [2024-09-30 02:42:56,296][1157819] Updated weights for policy 0, policy_version 145328 (0.0006) [2024-09-30 02:42:56,817][1157819] Updated weights for policy 0, policy_version 145338 (0.0006) [2024-09-30 02:42:57,338][1157819] Updated weights for policy 0, policy_version 145348 (0.0006) [2024-09-30 02:42:57,842][1157819] Updated weights for policy 0, policy_version 145358 (0.0006) [2024-09-30 02:42:58,359][1157819] Updated weights for policy 0, policy_version 145368 (0.0006) [2024-09-30 02:42:58,863][1157819] Updated weights for policy 0, policy_version 145378 (0.0006) [2024-09-30 02:42:59,374][1157819] Updated weights for policy 0, policy_version 145388 (0.0006) [2024-09-30 02:42:59,919][1157819] Updated weights for policy 0, policy_version 145398 (0.0006) [2024-09-30 02:43:00,416][1157819] Updated weights for policy 0, policy_version 145408 (0.0006) [2024-09-30 02:43:00,466][1157520] Fps is (10 sec: 79872.2, 60 sec: 83626.9, 300 sec: 76852.1). Total num frames: 595595264. Throughput: 0: 21080.1. Samples: 138865408. Policy #0 lag: (min: 0.0, avg: 2.6, max: 7.0) [2024-09-30 02:43:00,466][1157520] Avg episode reward: [(0, '52.088')] [2024-09-30 02:43:00,911][1157819] Updated weights for policy 0, policy_version 145418 (0.0006) [2024-09-30 02:43:01,412][1157819] Updated weights for policy 0, policy_version 145428 (0.0006) [2024-09-30 02:43:01,928][1157819] Updated weights for policy 0, policy_version 145438 (0.0006) [2024-09-30 02:43:02,520][1157819] Updated weights for policy 0, policy_version 145448 (0.0007) [2024-09-30 02:43:03,080][1157819] Updated weights for policy 0, policy_version 145458 (0.0006) [2024-09-30 02:43:03,617][1157819] Updated weights for policy 0, policy_version 145468 (0.0006) [2024-09-30 02:43:04,170][1157819] Updated weights for policy 0, policy_version 145478 (0.0006) [2024-09-30 02:43:04,694][1157819] Updated weights for policy 0, policy_version 145488 (0.0006) [2024-09-30 02:43:05,235][1157819] Updated weights for policy 0, policy_version 145498 (0.0006) [2024-09-30 02:43:05,466][1157520] Fps is (10 sec: 77413.7, 60 sec: 83217.3, 300 sec: 76990.9). Total num frames: 595972096. Throughput: 0: 20871.0. Samples: 138981348. Policy #0 lag: (min: 0.0, avg: 2.6, max: 7.0) [2024-09-30 02:43:05,466][1157520] Avg episode reward: [(0, '56.967')] [2024-09-30 02:43:05,792][1157819] Updated weights for policy 0, policy_version 145508 (0.0006) [2024-09-30 02:43:06,308][1157819] Updated weights for policy 0, policy_version 145518 (0.0006) [2024-09-30 02:43:06,836][1157819] Updated weights for policy 0, policy_version 145528 (0.0006) [2024-09-30 02:43:07,414][1157819] Updated weights for policy 0, policy_version 145538 (0.0006) [2024-09-30 02:43:07,921][1157819] Updated weights for policy 0, policy_version 145548 (0.0006) [2024-09-30 02:43:08,440][1157819] Updated weights for policy 0, policy_version 145558 (0.0006) [2024-09-30 02:43:09,047][1157819] Updated weights for policy 0, policy_version 145568 (0.0006) [2024-09-30 02:43:09,556][1157819] Updated weights for policy 0, policy_version 145578 (0.0006) [2024-09-30 02:43:10,092][1157819] Updated weights for policy 0, policy_version 145588 (0.0006) [2024-09-30 02:43:10,466][1157520] Fps is (10 sec: 75775.2, 60 sec: 82807.5, 300 sec: 77032.6). Total num frames: 596353024. Throughput: 0: 20724.6. Samples: 139038184. Policy #0 lag: (min: 0.0, avg: 2.6, max: 7.0) [2024-09-30 02:43:10,466][1157520] Avg episode reward: [(0, '56.663')] [2024-09-30 02:43:10,638][1157819] Updated weights for policy 0, policy_version 145598 (0.0006) [2024-09-30 02:43:11,189][1157819] Updated weights for policy 0, policy_version 145608 (0.0006) [2024-09-30 02:43:11,753][1157819] Updated weights for policy 0, policy_version 145618 (0.0006) [2024-09-30 02:43:12,268][1157819] Updated weights for policy 0, policy_version 145628 (0.0006) [2024-09-30 02:43:12,807][1157819] Updated weights for policy 0, policy_version 145638 (0.0006) [2024-09-30 02:43:13,393][1157819] Updated weights for policy 0, policy_version 145648 (0.0007) [2024-09-30 02:43:13,964][1157819] Updated weights for policy 0, policy_version 145658 (0.0006) [2024-09-30 02:43:14,534][1157819] Updated weights for policy 0, policy_version 145668 (0.0006) [2024-09-30 02:43:15,086][1157819] Updated weights for policy 0, policy_version 145678 (0.0006) [2024-09-30 02:43:15,466][1157520] Fps is (10 sec: 75366.4, 60 sec: 81988.3, 300 sec: 76963.1). Total num frames: 596725760. Throughput: 0: 20334.8. Samples: 139150744. Policy #0 lag: (min: 0.0, avg: 2.6, max: 7.0) [2024-09-30 02:43:15,466][1157520] Avg episode reward: [(0, '54.390')] [2024-09-30 02:43:15,629][1157819] Updated weights for policy 0, policy_version 145688 (0.0006) [2024-09-30 02:43:16,154][1157819] Updated weights for policy 0, policy_version 145698 (0.0006) [2024-09-30 02:43:16,689][1157819] Updated weights for policy 0, policy_version 145708 (0.0006) [2024-09-30 02:43:17,194][1157819] Updated weights for policy 0, policy_version 145718 (0.0006) [2024-09-30 02:43:17,700][1157819] Updated weights for policy 0, policy_version 145728 (0.0005) [2024-09-30 02:43:18,243][1157819] Updated weights for policy 0, policy_version 145738 (0.0006) [2024-09-30 02:43:18,777][1157819] Updated weights for policy 0, policy_version 145748 (0.0006) [2024-09-30 02:43:19,332][1157819] Updated weights for policy 0, policy_version 145758 (0.0006) [2024-09-30 02:43:19,817][1157819] Updated weights for policy 0, policy_version 145768 (0.0006) [2024-09-30 02:43:20,313][1157819] Updated weights for policy 0, policy_version 145778 (0.0006) [2024-09-30 02:43:20,466][1157520] Fps is (10 sec: 76185.6, 60 sec: 81510.4, 300 sec: 76852.1). Total num frames: 597114880. Throughput: 0: 19951.5. Samples: 139266176. Policy #0 lag: (min: 0.0, avg: 2.6, max: 7.0) [2024-09-30 02:43:20,466][1157520] Avg episode reward: [(0, '53.279')] [2024-09-30 02:43:20,851][1157819] Updated weights for policy 0, policy_version 145788 (0.0006) [2024-09-30 02:43:21,380][1157819] Updated weights for policy 0, policy_version 145798 (0.0006) [2024-09-30 02:43:21,953][1157819] Updated weights for policy 0, policy_version 145808 (0.0006) [2024-09-30 02:43:22,488][1157819] Updated weights for policy 0, policy_version 145818 (0.0006) [2024-09-30 02:43:23,014][1157819] Updated weights for policy 0, policy_version 145828 (0.0006) [2024-09-30 02:43:23,582][1157819] Updated weights for policy 0, policy_version 145838 (0.0006) [2024-09-30 02:43:23,982][1157736] Signal inference workers to stop experience collection... (10150 times) [2024-09-30 02:43:23,983][1157736] Signal inference workers to resume experience collection... (10150 times) [2024-09-30 02:43:23,988][1157819] InferenceWorker_p0-w0: stopping experience collection (10150 times) [2024-09-30 02:43:23,988][1157819] InferenceWorker_p0-w0: resuming experience collection (10150 times) [2024-09-30 02:43:24,129][1157819] Updated weights for policy 0, policy_version 145848 (0.0006) [2024-09-30 02:43:24,667][1157819] Updated weights for policy 0, policy_version 145858 (0.0006) [2024-09-30 02:43:25,182][1157819] Updated weights for policy 0, policy_version 145868 (0.0006) [2024-09-30 02:43:25,466][1157520] Fps is (10 sec: 76595.0, 60 sec: 80759.4, 300 sec: 76810.4). Total num frames: 597491712. Throughput: 0: 19754.2. Samples: 139323372. Policy #0 lag: (min: 0.0, avg: 2.6, max: 7.0) [2024-09-30 02:43:25,466][1157520] Avg episode reward: [(0, '54.225')] [2024-09-30 02:43:25,750][1157819] Updated weights for policy 0, policy_version 145878 (0.0006) [2024-09-30 02:43:26,313][1157819] Updated weights for policy 0, policy_version 145888 (0.0006) [2024-09-30 02:43:26,827][1157819] Updated weights for policy 0, policy_version 145898 (0.0006) [2024-09-30 02:43:27,368][1157819] Updated weights for policy 0, policy_version 145908 (0.0006) [2024-09-30 02:43:27,905][1157819] Updated weights for policy 0, policy_version 145918 (0.0006) [2024-09-30 02:43:28,419][1157819] Updated weights for policy 0, policy_version 145928 (0.0006) [2024-09-30 02:43:28,949][1157819] Updated weights for policy 0, policy_version 145938 (0.0006) [2024-09-30 02:43:29,451][1157819] Updated weights for policy 0, policy_version 145948 (0.0006) [2024-09-30 02:43:29,966][1157819] Updated weights for policy 0, policy_version 145958 (0.0006) [2024-09-30 02:43:30,466][1157520] Fps is (10 sec: 76595.0, 60 sec: 79871.9, 300 sec: 76852.1). Total num frames: 597880832. Throughput: 0: 19478.9. Samples: 139437468. Policy #0 lag: (min: 0.0, avg: 2.6, max: 7.0) [2024-09-30 02:43:30,466][1157520] Avg episode reward: [(0, '56.985')] [2024-09-30 02:43:30,513][1157819] Updated weights for policy 0, policy_version 145968 (0.0006) [2024-09-30 02:43:31,025][1157819] Updated weights for policy 0, policy_version 145978 (0.0006) [2024-09-30 02:43:31,579][1157819] Updated weights for policy 0, policy_version 145988 (0.0006) [2024-09-30 02:43:32,075][1157819] Updated weights for policy 0, policy_version 145998 (0.0006) [2024-09-30 02:43:32,581][1157819] Updated weights for policy 0, policy_version 146008 (0.0006) [2024-09-30 02:43:33,109][1157819] Updated weights for policy 0, policy_version 146018 (0.0006) [2024-09-30 02:43:33,617][1157819] Updated weights for policy 0, policy_version 146028 (0.0006) [2024-09-30 02:43:34,116][1157819] Updated weights for policy 0, policy_version 146038 (0.0006) [2024-09-30 02:43:34,655][1157819] Updated weights for policy 0, policy_version 146048 (0.0006) [2024-09-30 02:43:35,175][1157819] Updated weights for policy 0, policy_version 146058 (0.0006) [2024-09-30 02:43:35,466][1157520] Fps is (10 sec: 78233.6, 60 sec: 78984.4, 300 sec: 77004.8). Total num frames: 598274048. Throughput: 0: 19315.8. Samples: 139555824. Policy #0 lag: (min: 0.0, avg: 2.6, max: 7.0) [2024-09-30 02:43:35,466][1157520] Avg episode reward: [(0, '53.742')] [2024-09-30 02:43:35,688][1157819] Updated weights for policy 0, policy_version 146068 (0.0006) [2024-09-30 02:43:36,207][1157819] Updated weights for policy 0, policy_version 146078 (0.0006) [2024-09-30 02:43:36,733][1157819] Updated weights for policy 0, policy_version 146088 (0.0006) [2024-09-30 02:43:37,262][1157819] Updated weights for policy 0, policy_version 146098 (0.0006) [2024-09-30 02:43:37,797][1157819] Updated weights for policy 0, policy_version 146108 (0.0006) [2024-09-30 02:43:38,330][1157819] Updated weights for policy 0, policy_version 146118 (0.0006) [2024-09-30 02:43:38,816][1157819] Updated weights for policy 0, policy_version 146128 (0.0006) [2024-09-30 02:43:39,346][1157819] Updated weights for policy 0, policy_version 146138 (0.0006) [2024-09-30 02:43:39,861][1157819] Updated weights for policy 0, policy_version 146148 (0.0006) [2024-09-30 02:43:40,405][1157819] Updated weights for policy 0, policy_version 146158 (0.0006) [2024-09-30 02:43:40,466][1157520] Fps is (10 sec: 78233.3, 60 sec: 78301.7, 300 sec: 77115.9). Total num frames: 598663168. Throughput: 0: 19292.9. Samples: 139614568. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:43:40,466][1157520] Avg episode reward: [(0, '54.258')] [2024-09-30 02:43:40,948][1157819] Updated weights for policy 0, policy_version 146168 (0.0006) [2024-09-30 02:43:41,442][1157819] Updated weights for policy 0, policy_version 146178 (0.0006) [2024-09-30 02:43:41,964][1157819] Updated weights for policy 0, policy_version 146188 (0.0006) [2024-09-30 02:43:42,494][1157819] Updated weights for policy 0, policy_version 146198 (0.0006) [2024-09-30 02:43:43,009][1157819] Updated weights for policy 0, policy_version 146208 (0.0006) [2024-09-30 02:43:43,539][1157819] Updated weights for policy 0, policy_version 146218 (0.0006) [2024-09-30 02:43:44,064][1157819] Updated weights for policy 0, policy_version 146228 (0.0006) [2024-09-30 02:43:44,560][1157819] Updated weights for policy 0, policy_version 146238 (0.0006) [2024-09-30 02:43:45,069][1157819] Updated weights for policy 0, policy_version 146248 (0.0006) [2024-09-30 02:43:45,466][1157520] Fps is (10 sec: 78232.8, 60 sec: 77892.0, 300 sec: 77226.9). Total num frames: 599056384. Throughput: 0: 19262.3. Samples: 139732216. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:43:45,466][1157520] Avg episode reward: [(0, '56.340')] [2024-09-30 02:43:45,599][1157819] Updated weights for policy 0, policy_version 146258 (0.0006) [2024-09-30 02:43:46,140][1157819] Updated weights for policy 0, policy_version 146268 (0.0006) [2024-09-30 02:43:46,666][1157819] Updated weights for policy 0, policy_version 146278 (0.0006) [2024-09-30 02:43:47,172][1157819] Updated weights for policy 0, policy_version 146288 (0.0006) [2024-09-30 02:43:47,693][1157819] Updated weights for policy 0, policy_version 146298 (0.0006) [2024-09-30 02:43:48,209][1157819] Updated weights for policy 0, policy_version 146308 (0.0006) [2024-09-30 02:43:48,747][1157819] Updated weights for policy 0, policy_version 146318 (0.0006) [2024-09-30 02:43:49,271][1157819] Updated weights for policy 0, policy_version 146328 (0.0006) [2024-09-30 02:43:49,767][1157819] Updated weights for policy 0, policy_version 146338 (0.0006) [2024-09-30 02:43:50,296][1157819] Updated weights for policy 0, policy_version 146348 (0.0006) [2024-09-30 02:43:50,466][1157520] Fps is (10 sec: 79053.4, 60 sec: 77619.1, 300 sec: 77393.6). Total num frames: 599453696. Throughput: 0: 19304.9. Samples: 139850068. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:43:50,466][1157520] Avg episode reward: [(0, '52.564')] [2024-09-30 02:43:50,829][1157819] Updated weights for policy 0, policy_version 146358 (0.0006) [2024-09-30 02:43:51,354][1157819] Updated weights for policy 0, policy_version 146368 (0.0006) [2024-09-30 02:43:51,889][1157819] Updated weights for policy 0, policy_version 146378 (0.0006) [2024-09-30 02:43:52,403][1157819] Updated weights for policy 0, policy_version 146388 (0.0006) [2024-09-30 02:43:52,934][1157819] Updated weights for policy 0, policy_version 146398 (0.0006) [2024-09-30 02:43:53,436][1157819] Updated weights for policy 0, policy_version 146408 (0.0006) [2024-09-30 02:43:53,964][1157819] Updated weights for policy 0, policy_version 146418 (0.0006) [2024-09-30 02:43:54,486][1157819] Updated weights for policy 0, policy_version 146428 (0.0006) [2024-09-30 02:43:55,020][1157819] Updated weights for policy 0, policy_version 146438 (0.0006) [2024-09-30 02:43:55,466][1157520] Fps is (10 sec: 79054.0, 60 sec: 77482.6, 300 sec: 77574.1). Total num frames: 599846912. Throughput: 0: 19347.2. Samples: 139908808. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:43:55,466][1157520] Avg episode reward: [(0, '54.315')] [2024-09-30 02:43:55,514][1157819] Updated weights for policy 0, policy_version 146448 (0.0006) [2024-09-30 02:43:56,036][1157819] Updated weights for policy 0, policy_version 146458 (0.0006) [2024-09-30 02:43:56,586][1157819] Updated weights for policy 0, policy_version 146468 (0.0006) [2024-09-30 02:43:57,101][1157819] Updated weights for policy 0, policy_version 146478 (0.0006) [2024-09-30 02:43:57,642][1157819] Updated weights for policy 0, policy_version 146488 (0.0006) [2024-09-30 02:43:58,194][1157819] Updated weights for policy 0, policy_version 146498 (0.0006) [2024-09-30 02:43:58,772][1157819] Updated weights for policy 0, policy_version 146508 (0.0006) [2024-09-30 02:43:59,287][1157819] Updated weights for policy 0, policy_version 146518 (0.0006) [2024-09-30 02:43:59,842][1157819] Updated weights for policy 0, policy_version 146528 (0.0006) [2024-09-30 02:44:00,398][1157819] Updated weights for policy 0, policy_version 146538 (0.0006) [2024-09-30 02:44:00,466][1157520] Fps is (10 sec: 77004.7, 60 sec: 77141.2, 300 sec: 77685.2). Total num frames: 600223744. Throughput: 0: 19422.4. Samples: 140024752. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:44:00,466][1157520] Avg episode reward: [(0, '53.774')] [2024-09-30 02:44:00,911][1157819] Updated weights for policy 0, policy_version 146548 (0.0006) [2024-09-30 02:44:01,409][1157819] Updated weights for policy 0, policy_version 146558 (0.0006) [2024-09-30 02:44:01,942][1157819] Updated weights for policy 0, policy_version 146568 (0.0006) [2024-09-30 02:44:02,435][1157819] Updated weights for policy 0, policy_version 146578 (0.0006) [2024-09-30 02:44:02,946][1157819] Updated weights for policy 0, policy_version 146588 (0.0006) [2024-09-30 02:44:03,479][1157819] Updated weights for policy 0, policy_version 146598 (0.0006) [2024-09-30 02:44:04,022][1157819] Updated weights for policy 0, policy_version 146608 (0.0006) [2024-09-30 02:44:04,504][1157819] Updated weights for policy 0, policy_version 146618 (0.0006) [2024-09-30 02:44:05,002][1157819] Updated weights for policy 0, policy_version 146628 (0.0006) [2024-09-30 02:44:05,466][1157520] Fps is (10 sec: 77413.7, 60 sec: 77482.6, 300 sec: 77851.8). Total num frames: 600621056. Throughput: 0: 19476.1. Samples: 140142600. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:44:05,466][1157520] Avg episode reward: [(0, '54.472')] [2024-09-30 02:44:05,472][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000146636_600621056.pth... [2024-09-30 02:44:05,518][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000142018_581705728.pth [2024-09-30 02:44:05,547][1157819] Updated weights for policy 0, policy_version 146638 (0.0006) [2024-09-30 02:44:06,204][1157819] Updated weights for policy 0, policy_version 146648 (0.0006) [2024-09-30 02:44:06,858][1157819] Updated weights for policy 0, policy_version 146658 (0.0006) [2024-09-30 02:44:07,505][1157819] Updated weights for policy 0, policy_version 146668 (0.0006) [2024-09-30 02:44:08,121][1157819] Updated weights for policy 0, policy_version 146678 (0.0006) [2024-09-30 02:44:08,726][1157819] Updated weights for policy 0, policy_version 146688 (0.0006) [2024-09-30 02:44:09,289][1157819] Updated weights for policy 0, policy_version 146698 (0.0006) [2024-09-30 02:44:09,850][1157819] Updated weights for policy 0, policy_version 146708 (0.0006) [2024-09-30 02:44:10,415][1157819] Updated weights for policy 0, policy_version 146718 (0.0006) [2024-09-30 02:44:10,466][1157520] Fps is (10 sec: 73318.3, 60 sec: 76731.7, 300 sec: 77810.1). Total num frames: 600956928. Throughput: 0: 19320.5. Samples: 140192796. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:44:10,466][1157520] Avg episode reward: [(0, '54.315')] [2024-09-30 02:44:11,038][1157819] Updated weights for policy 0, policy_version 146728 (0.0006) [2024-09-30 02:44:11,592][1157819] Updated weights for policy 0, policy_version 146738 (0.0006) [2024-09-30 02:44:12,181][1157819] Updated weights for policy 0, policy_version 146748 (0.0006) [2024-09-30 02:44:12,778][1157819] Updated weights for policy 0, policy_version 146758 (0.0006) [2024-09-30 02:44:13,346][1157819] Updated weights for policy 0, policy_version 146768 (0.0006) [2024-09-30 02:44:13,898][1157819] Updated weights for policy 0, policy_version 146778 (0.0006) [2024-09-30 02:44:14,489][1157819] Updated weights for policy 0, policy_version 146788 (0.0006) [2024-09-30 02:44:15,075][1157819] Updated weights for policy 0, policy_version 146798 (0.0006) [2024-09-30 02:44:15,466][1157520] Fps is (10 sec: 68813.2, 60 sec: 76390.4, 300 sec: 77810.1). Total num frames: 601309184. Throughput: 0: 19134.9. Samples: 140298540. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:44:15,466][1157520] Avg episode reward: [(0, '53.739')] [2024-09-30 02:44:15,678][1157819] Updated weights for policy 0, policy_version 146808 (0.0006) [2024-09-30 02:44:16,286][1157819] Updated weights for policy 0, policy_version 146818 (0.0006) [2024-09-30 02:44:16,809][1157819] Updated weights for policy 0, policy_version 146828 (0.0006) [2024-09-30 02:44:17,398][1157819] Updated weights for policy 0, policy_version 146838 (0.0006) [2024-09-30 02:44:17,919][1157819] Updated weights for policy 0, policy_version 146848 (0.0006) [2024-09-30 02:44:18,457][1157819] Updated weights for policy 0, policy_version 146858 (0.0006) [2024-09-30 02:44:19,005][1157819] Updated weights for policy 0, policy_version 146868 (0.0006) [2024-09-30 02:44:19,524][1157819] Updated weights for policy 0, policy_version 146878 (0.0006) [2024-09-30 02:44:20,014][1157819] Updated weights for policy 0, policy_version 146888 (0.0006) [2024-09-30 02:44:20,466][1157520] Fps is (10 sec: 72909.5, 60 sec: 76185.7, 300 sec: 77893.5). Total num frames: 601686016. Throughput: 0: 18950.5. Samples: 140408596. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:44:20,466][1157520] Avg episode reward: [(0, '56.344')] [2024-09-30 02:44:20,522][1157819] Updated weights for policy 0, policy_version 146898 (0.0006) [2024-09-30 02:44:21,061][1157819] Updated weights for policy 0, policy_version 146908 (0.0006) [2024-09-30 02:44:21,554][1157819] Updated weights for policy 0, policy_version 146918 (0.0006) [2024-09-30 02:44:22,041][1157819] Updated weights for policy 0, policy_version 146928 (0.0006) [2024-09-30 02:44:22,574][1157819] Updated weights for policy 0, policy_version 146938 (0.0006) [2024-09-30 02:44:23,112][1157819] Updated weights for policy 0, policy_version 146948 (0.0006) [2024-09-30 02:44:23,611][1157819] Updated weights for policy 0, policy_version 146958 (0.0006) [2024-09-30 02:44:24,107][1157819] Updated weights for policy 0, policy_version 146968 (0.0006) [2024-09-30 02:44:24,590][1157736] Signal inference workers to stop experience collection... (10200 times) [2024-09-30 02:44:24,591][1157736] Signal inference workers to resume experience collection... (10200 times) [2024-09-30 02:44:24,595][1157819] InferenceWorker_p0-w0: stopping experience collection (10200 times) [2024-09-30 02:44:24,595][1157819] InferenceWorker_p0-w0: resuming experience collection (10200 times) [2024-09-30 02:44:24,602][1157819] Updated weights for policy 0, policy_version 146978 (0.0006) [2024-09-30 02:44:25,098][1157819] Updated weights for policy 0, policy_version 146988 (0.0006) [2024-09-30 02:44:25,466][1157520] Fps is (10 sec: 78234.4, 60 sec: 76663.6, 300 sec: 78046.2). Total num frames: 602091520. Throughput: 0: 18977.4. Samples: 140468548. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:44:25,466][1157520] Avg episode reward: [(0, '53.412')] [2024-09-30 02:44:25,577][1157819] Updated weights for policy 0, policy_version 146998 (0.0006) [2024-09-30 02:44:26,068][1157819] Updated weights for policy 0, policy_version 147008 (0.0006) [2024-09-30 02:44:26,572][1157819] Updated weights for policy 0, policy_version 147018 (0.0006) [2024-09-30 02:44:27,074][1157819] Updated weights for policy 0, policy_version 147028 (0.0006) [2024-09-30 02:44:27,584][1157819] Updated weights for policy 0, policy_version 147038 (0.0006) [2024-09-30 02:44:28,092][1157819] Updated weights for policy 0, policy_version 147048 (0.0006) [2024-09-30 02:44:28,591][1157819] Updated weights for policy 0, policy_version 147058 (0.0006) [2024-09-30 02:44:29,105][1157819] Updated weights for policy 0, policy_version 147068 (0.0006) [2024-09-30 02:44:29,657][1157819] Updated weights for policy 0, policy_version 147078 (0.0006) [2024-09-30 02:44:30,219][1157819] Updated weights for policy 0, policy_version 147088 (0.0006) [2024-09-30 02:44:30,466][1157520] Fps is (10 sec: 80691.1, 60 sec: 76868.4, 300 sec: 78212.8). Total num frames: 602492928. Throughput: 0: 19085.4. Samples: 140591056. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:44:30,466][1157520] Avg episode reward: [(0, '56.021')] [2024-09-30 02:44:30,718][1157819] Updated weights for policy 0, policy_version 147098 (0.0006) [2024-09-30 02:44:31,208][1157819] Updated weights for policy 0, policy_version 147108 (0.0006) [2024-09-30 02:44:31,716][1157819] Updated weights for policy 0, policy_version 147118 (0.0006) [2024-09-30 02:44:32,240][1157819] Updated weights for policy 0, policy_version 147128 (0.0006) [2024-09-30 02:44:32,788][1157819] Updated weights for policy 0, policy_version 147138 (0.0006) [2024-09-30 02:44:33,312][1157819] Updated weights for policy 0, policy_version 147148 (0.0006) [2024-09-30 02:44:33,807][1157819] Updated weights for policy 0, policy_version 147158 (0.0006) [2024-09-30 02:44:34,328][1157819] Updated weights for policy 0, policy_version 147168 (0.0006) [2024-09-30 02:44:34,849][1157819] Updated weights for policy 0, policy_version 147178 (0.0006) [2024-09-30 02:44:35,400][1157819] Updated weights for policy 0, policy_version 147188 (0.0006) [2024-09-30 02:44:35,466][1157520] Fps is (10 sec: 79872.1, 60 sec: 76936.7, 300 sec: 78268.3). Total num frames: 602890240. Throughput: 0: 19088.2. Samples: 140709036. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:44:35,466][1157520] Avg episode reward: [(0, '56.898')] [2024-09-30 02:44:35,917][1157819] Updated weights for policy 0, policy_version 147198 (0.0006) [2024-09-30 02:44:36,407][1157819] Updated weights for policy 0, policy_version 147208 (0.0006) [2024-09-30 02:44:36,892][1157819] Updated weights for policy 0, policy_version 147218 (0.0006) [2024-09-30 02:44:37,401][1157819] Updated weights for policy 0, policy_version 147228 (0.0006) [2024-09-30 02:44:37,895][1157819] Updated weights for policy 0, policy_version 147238 (0.0006) [2024-09-30 02:44:38,368][1157819] Updated weights for policy 0, policy_version 147248 (0.0006) [2024-09-30 02:44:38,865][1157819] Updated weights for policy 0, policy_version 147258 (0.0006) [2024-09-30 02:44:39,360][1157819] Updated weights for policy 0, policy_version 147268 (0.0006) [2024-09-30 02:44:39,835][1157819] Updated weights for policy 0, policy_version 147278 (0.0006) [2024-09-30 02:44:40,329][1157819] Updated weights for policy 0, policy_version 147288 (0.0006) [2024-09-30 02:44:40,466][1157520] Fps is (10 sec: 80691.1, 60 sec: 77278.0, 300 sec: 78365.5). Total num frames: 603299840. Throughput: 0: 19128.0. Samples: 140769568. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 02:44:40,466][1157520] Avg episode reward: [(0, '56.493')] [2024-09-30 02:44:40,817][1157819] Updated weights for policy 0, policy_version 147298 (0.0006) [2024-09-30 02:44:41,302][1157819] Updated weights for policy 0, policy_version 147308 (0.0006) [2024-09-30 02:44:41,801][1157819] Updated weights for policy 0, policy_version 147318 (0.0006) [2024-09-30 02:44:42,297][1157819] Updated weights for policy 0, policy_version 147328 (0.0006) [2024-09-30 02:44:42,788][1157819] Updated weights for policy 0, policy_version 147338 (0.0006) [2024-09-30 02:44:43,269][1157819] Updated weights for policy 0, policy_version 147348 (0.0006) [2024-09-30 02:44:43,766][1157819] Updated weights for policy 0, policy_version 147358 (0.0006) [2024-09-30 02:44:44,269][1157819] Updated weights for policy 0, policy_version 147368 (0.0006) [2024-09-30 02:44:44,727][1157819] Updated weights for policy 0, policy_version 147378 (0.0006) [2024-09-30 02:44:45,240][1157819] Updated weights for policy 0, policy_version 147388 (0.0006) [2024-09-30 02:44:45,466][1157520] Fps is (10 sec: 82738.9, 60 sec: 77687.7, 300 sec: 78462.7). Total num frames: 603717632. Throughput: 0: 19337.2. Samples: 140894924. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 02:44:45,466][1157520] Avg episode reward: [(0, '54.655')] [2024-09-30 02:44:45,710][1157819] Updated weights for policy 0, policy_version 147398 (0.0006) [2024-09-30 02:44:46,161][1157819] Updated weights for policy 0, policy_version 147408 (0.0006) [2024-09-30 02:44:46,639][1157819] Updated weights for policy 0, policy_version 147418 (0.0006) [2024-09-30 02:44:47,110][1157819] Updated weights for policy 0, policy_version 147428 (0.0006) [2024-09-30 02:44:47,562][1157819] Updated weights for policy 0, policy_version 147438 (0.0006) [2024-09-30 02:44:48,049][1157819] Updated weights for policy 0, policy_version 147448 (0.0006) [2024-09-30 02:44:48,548][1157819] Updated weights for policy 0, policy_version 147458 (0.0006) [2024-09-30 02:44:49,028][1157819] Updated weights for policy 0, policy_version 147468 (0.0006) [2024-09-30 02:44:49,491][1157819] Updated weights for policy 0, policy_version 147478 (0.0006) [2024-09-30 02:44:49,990][1157819] Updated weights for policy 0, policy_version 147488 (0.0006) [2024-09-30 02:44:50,453][1157819] Updated weights for policy 0, policy_version 147498 (0.0006) [2024-09-30 02:44:50,466][1157520] Fps is (10 sec: 85197.0, 60 sec: 78302.0, 300 sec: 78712.7). Total num frames: 604151808. Throughput: 0: 19579.1. Samples: 141023656. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 02:44:50,466][1157520] Avg episode reward: [(0, '54.404')] [2024-09-30 02:44:50,971][1157819] Updated weights for policy 0, policy_version 147508 (0.0006) [2024-09-30 02:44:51,443][1157819] Updated weights for policy 0, policy_version 147518 (0.0006) [2024-09-30 02:44:51,941][1157819] Updated weights for policy 0, policy_version 147528 (0.0006) [2024-09-30 02:44:52,442][1157819] Updated weights for policy 0, policy_version 147538 (0.0006) [2024-09-30 02:44:52,922][1157819] Updated weights for policy 0, policy_version 147548 (0.0006) [2024-09-30 02:44:53,411][1157819] Updated weights for policy 0, policy_version 147558 (0.0006) [2024-09-30 02:44:53,923][1157819] Updated weights for policy 0, policy_version 147568 (0.0006) [2024-09-30 02:44:54,391][1157819] Updated weights for policy 0, policy_version 147578 (0.0006) [2024-09-30 02:44:54,877][1157819] Updated weights for policy 0, policy_version 147588 (0.0006) [2024-09-30 02:44:55,392][1157819] Updated weights for policy 0, policy_version 147598 (0.0006) [2024-09-30 02:44:55,466][1157520] Fps is (10 sec: 84787.4, 60 sec: 78643.3, 300 sec: 78865.4). Total num frames: 604565504. Throughput: 0: 19851.1. Samples: 141086092. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 02:44:55,466][1157520] Avg episode reward: [(0, '56.156')] [2024-09-30 02:44:55,871][1157819] Updated weights for policy 0, policy_version 147608 (0.0006) [2024-09-30 02:44:56,360][1157819] Updated weights for policy 0, policy_version 147618 (0.0006) [2024-09-30 02:44:56,843][1157819] Updated weights for policy 0, policy_version 147628 (0.0006) [2024-09-30 02:44:57,363][1157819] Updated weights for policy 0, policy_version 147638 (0.0006) [2024-09-30 02:44:57,821][1157819] Updated weights for policy 0, policy_version 147648 (0.0006) [2024-09-30 02:44:58,312][1157819] Updated weights for policy 0, policy_version 147658 (0.0006) [2024-09-30 02:44:58,800][1157819] Updated weights for policy 0, policy_version 147668 (0.0006) [2024-09-30 02:44:59,290][1157819] Updated weights for policy 0, policy_version 147678 (0.0006) [2024-09-30 02:44:59,762][1157819] Updated weights for policy 0, policy_version 147688 (0.0006) [2024-09-30 02:45:00,235][1157819] Updated weights for policy 0, policy_version 147698 (0.0006) [2024-09-30 02:45:00,466][1157520] Fps is (10 sec: 83558.5, 60 sec: 79394.3, 300 sec: 78920.9). Total num frames: 604987392. Throughput: 0: 20295.8. Samples: 141211848. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 02:45:00,466][1157520] Avg episode reward: [(0, '55.206')] [2024-09-30 02:45:00,758][1157819] Updated weights for policy 0, policy_version 147708 (0.0006) [2024-09-30 02:45:01,218][1157819] Updated weights for policy 0, policy_version 147718 (0.0006) [2024-09-30 02:45:01,707][1157819] Updated weights for policy 0, policy_version 147728 (0.0006) [2024-09-30 02:45:02,202][1157819] Updated weights for policy 0, policy_version 147738 (0.0006) [2024-09-30 02:45:02,656][1157819] Updated weights for policy 0, policy_version 147748 (0.0006) [2024-09-30 02:45:03,190][1157819] Updated weights for policy 0, policy_version 147758 (0.0006) [2024-09-30 02:45:03,692][1157819] Updated weights for policy 0, policy_version 147768 (0.0006) [2024-09-30 02:45:04,188][1157819] Updated weights for policy 0, policy_version 147778 (0.0006) [2024-09-30 02:45:04,703][1157819] Updated weights for policy 0, policy_version 147788 (0.0006) [2024-09-30 02:45:05,213][1157819] Updated weights for policy 0, policy_version 147798 (0.0006) [2024-09-30 02:45:05,466][1157520] Fps is (10 sec: 83148.7, 60 sec: 79599.1, 300 sec: 78962.6). Total num frames: 605396992. Throughput: 0: 20616.6. Samples: 141336344. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 02:45:05,466][1157520] Avg episode reward: [(0, '54.988')] [2024-09-30 02:45:05,721][1157819] Updated weights for policy 0, policy_version 147808 (0.0006) [2024-09-30 02:45:06,264][1157819] Updated weights for policy 0, policy_version 147818 (0.0006) [2024-09-30 02:45:06,780][1157819] Updated weights for policy 0, policy_version 147828 (0.0006) [2024-09-30 02:45:07,296][1157819] Updated weights for policy 0, policy_version 147838 (0.0006) [2024-09-30 02:45:07,820][1157819] Updated weights for policy 0, policy_version 147848 (0.0006) [2024-09-30 02:45:08,309][1157819] Updated weights for policy 0, policy_version 147858 (0.0006) [2024-09-30 02:45:08,812][1157819] Updated weights for policy 0, policy_version 147868 (0.0006) [2024-09-30 02:45:09,337][1157819] Updated weights for policy 0, policy_version 147878 (0.0006) [2024-09-30 02:45:09,871][1157819] Updated weights for policy 0, policy_version 147888 (0.0006) [2024-09-30 02:45:10,375][1157819] Updated weights for policy 0, policy_version 147898 (0.0006) [2024-09-30 02:45:10,466][1157520] Fps is (10 sec: 80690.6, 60 sec: 80623.0, 300 sec: 79059.8). Total num frames: 605794304. Throughput: 0: 20609.0. Samples: 141395952. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 02:45:10,466][1157520] Avg episode reward: [(0, '54.567')] [2024-09-30 02:45:10,876][1157819] Updated weights for policy 0, policy_version 147908 (0.0006) [2024-09-30 02:45:11,379][1157819] Updated weights for policy 0, policy_version 147918 (0.0006) [2024-09-30 02:45:11,891][1157819] Updated weights for policy 0, policy_version 147928 (0.0006) [2024-09-30 02:45:12,389][1157819] Updated weights for policy 0, policy_version 147938 (0.0006) [2024-09-30 02:45:12,909][1157819] Updated weights for policy 0, policy_version 147948 (0.0006) [2024-09-30 02:45:13,452][1157819] Updated weights for policy 0, policy_version 147958 (0.0006) [2024-09-30 02:45:13,947][1157819] Updated weights for policy 0, policy_version 147968 (0.0006) [2024-09-30 02:45:14,446][1157819] Updated weights for policy 0, policy_version 147978 (0.0006) [2024-09-30 02:45:14,968][1157819] Updated weights for policy 0, policy_version 147988 (0.0006) [2024-09-30 02:45:15,466][1157520] Fps is (10 sec: 79462.4, 60 sec: 81374.0, 300 sec: 79157.0). Total num frames: 606191616. Throughput: 0: 20555.3. Samples: 141516044. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 02:45:15,466][1157520] Avg episode reward: [(0, '56.299')] [2024-09-30 02:45:15,508][1157819] Updated weights for policy 0, policy_version 147998 (0.0006) [2024-09-30 02:45:16,028][1157819] Updated weights for policy 0, policy_version 148008 (0.0006) [2024-09-30 02:45:16,543][1157819] Updated weights for policy 0, policy_version 148018 (0.0006) [2024-09-30 02:45:17,046][1157819] Updated weights for policy 0, policy_version 148028 (0.0006) [2024-09-30 02:45:17,585][1157819] Updated weights for policy 0, policy_version 148038 (0.0006) [2024-09-30 02:45:18,124][1157819] Updated weights for policy 0, policy_version 148048 (0.0006) [2024-09-30 02:45:18,604][1157819] Updated weights for policy 0, policy_version 148058 (0.0006) [2024-09-30 02:45:19,120][1157819] Updated weights for policy 0, policy_version 148068 (0.0006) [2024-09-30 02:45:19,643][1157819] Updated weights for policy 0, policy_version 148078 (0.0006) [2024-09-30 02:45:20,176][1157819] Updated weights for policy 0, policy_version 148088 (0.0006) [2024-09-30 02:45:20,466][1157520] Fps is (10 sec: 79463.0, 60 sec: 81715.2, 300 sec: 79170.9). Total num frames: 606588928. Throughput: 0: 20566.0. Samples: 141634504. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 02:45:20,466][1157520] Avg episode reward: [(0, '55.797')] [2024-09-30 02:45:20,679][1157819] Updated weights for policy 0, policy_version 148098 (0.0006) [2024-09-30 02:45:21,212][1157819] Updated weights for policy 0, policy_version 148108 (0.0006) [2024-09-30 02:45:21,731][1157819] Updated weights for policy 0, policy_version 148118 (0.0006) [2024-09-30 02:45:22,253][1157819] Updated weights for policy 0, policy_version 148128 (0.0006) [2024-09-30 02:45:22,532][1157736] Signal inference workers to stop experience collection... (10250 times) [2024-09-30 02:45:22,533][1157736] Signal inference workers to resume experience collection... (10250 times) [2024-09-30 02:45:22,536][1157819] InferenceWorker_p0-w0: stopping experience collection (10250 times) [2024-09-30 02:45:22,536][1157819] InferenceWorker_p0-w0: resuming experience collection (10250 times) [2024-09-30 02:45:22,781][1157819] Updated weights for policy 0, policy_version 148138 (0.0006) [2024-09-30 02:45:23,306][1157819] Updated weights for policy 0, policy_version 148148 (0.0006) [2024-09-30 02:45:23,813][1157819] Updated weights for policy 0, policy_version 148158 (0.0006) [2024-09-30 02:45:24,325][1157819] Updated weights for policy 0, policy_version 148168 (0.0006) [2024-09-30 02:45:24,864][1157819] Updated weights for policy 0, policy_version 148178 (0.0006) [2024-09-30 02:45:25,391][1157819] Updated weights for policy 0, policy_version 148188 (0.0006) [2024-09-30 02:45:25,466][1157520] Fps is (10 sec: 79052.8, 60 sec: 81510.4, 300 sec: 79226.4). Total num frames: 606982144. Throughput: 0: 20533.1. Samples: 141693556. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 02:45:25,466][1157520] Avg episode reward: [(0, '54.027')] [2024-09-30 02:45:25,929][1157819] Updated weights for policy 0, policy_version 148198 (0.0006) [2024-09-30 02:45:26,445][1157819] Updated weights for policy 0, policy_version 148208 (0.0006) [2024-09-30 02:45:26,973][1157819] Updated weights for policy 0, policy_version 148218 (0.0006) [2024-09-30 02:45:27,523][1157819] Updated weights for policy 0, policy_version 148228 (0.0006) [2024-09-30 02:45:28,032][1157819] Updated weights for policy 0, policy_version 148238 (0.0007) [2024-09-30 02:45:28,598][1157819] Updated weights for policy 0, policy_version 148248 (0.0006) [2024-09-30 02:45:29,091][1157819] Updated weights for policy 0, policy_version 148258 (0.0006) [2024-09-30 02:45:29,635][1157819] Updated weights for policy 0, policy_version 148268 (0.0006) [2024-09-30 02:45:30,172][1157819] Updated weights for policy 0, policy_version 148278 (0.0006) [2024-09-30 02:45:30,466][1157520] Fps is (10 sec: 77823.8, 60 sec: 81237.3, 300 sec: 79240.3). Total num frames: 607367168. Throughput: 0: 20330.1. Samples: 141809780. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 02:45:30,466][1157520] Avg episode reward: [(0, '56.265')] [2024-09-30 02:45:30,699][1157819] Updated weights for policy 0, policy_version 148288 (0.0007) [2024-09-30 02:45:31,228][1157819] Updated weights for policy 0, policy_version 148298 (0.0006) [2024-09-30 02:45:31,748][1157819] Updated weights for policy 0, policy_version 148308 (0.0006) [2024-09-30 02:45:32,253][1157819] Updated weights for policy 0, policy_version 148318 (0.0006) [2024-09-30 02:45:32,782][1157819] Updated weights for policy 0, policy_version 148328 (0.0006) [2024-09-30 02:45:33,283][1157819] Updated weights for policy 0, policy_version 148338 (0.0007) [2024-09-30 02:45:33,861][1157819] Updated weights for policy 0, policy_version 148348 (0.0006) [2024-09-30 02:45:34,411][1157819] Updated weights for policy 0, policy_version 148358 (0.0006) [2024-09-30 02:45:34,952][1157819] Updated weights for policy 0, policy_version 148368 (0.0006) [2024-09-30 02:45:35,466][1157520] Fps is (10 sec: 76593.9, 60 sec: 80964.0, 300 sec: 79226.3). Total num frames: 607748096. Throughput: 0: 20041.1. Samples: 141925508. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 02:45:35,466][1157520] Avg episode reward: [(0, '56.273')] [2024-09-30 02:45:35,534][1157819] Updated weights for policy 0, policy_version 148378 (0.0007) [2024-09-30 02:45:36,086][1157819] Updated weights for policy 0, policy_version 148388 (0.0007) [2024-09-30 02:45:36,667][1157819] Updated weights for policy 0, policy_version 148398 (0.0006) [2024-09-30 02:45:37,225][1157819] Updated weights for policy 0, policy_version 148408 (0.0006) [2024-09-30 02:45:37,775][1157819] Updated weights for policy 0, policy_version 148418 (0.0006) [2024-09-30 02:45:38,342][1157819] Updated weights for policy 0, policy_version 148428 (0.0006) [2024-09-30 02:45:38,912][1157819] Updated weights for policy 0, policy_version 148438 (0.0007) [2024-09-30 02:45:39,459][1157819] Updated weights for policy 0, policy_version 148448 (0.0006) [2024-09-30 02:45:40,034][1157819] Updated weights for policy 0, policy_version 148458 (0.0006) [2024-09-30 02:45:40,466][1157520] Fps is (10 sec: 74546.7, 60 sec: 80213.3, 300 sec: 79101.4). Total num frames: 608112640. Throughput: 0: 19865.1. Samples: 141980024. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 02:45:40,466][1157520] Avg episode reward: [(0, '55.927')] [2024-09-30 02:45:40,598][1157819] Updated weights for policy 0, policy_version 148468 (0.0006) [2024-09-30 02:45:41,158][1157819] Updated weights for policy 0, policy_version 148478 (0.0006) [2024-09-30 02:45:41,715][1157819] Updated weights for policy 0, policy_version 148488 (0.0006) [2024-09-30 02:45:42,278][1157819] Updated weights for policy 0, policy_version 148498 (0.0006) [2024-09-30 02:45:42,831][1157819] Updated weights for policy 0, policy_version 148508 (0.0007) [2024-09-30 02:45:43,355][1157819] Updated weights for policy 0, policy_version 148518 (0.0007) [2024-09-30 02:45:43,934][1157819] Updated weights for policy 0, policy_version 148528 (0.0007) [2024-09-30 02:45:44,498][1157819] Updated weights for policy 0, policy_version 148538 (0.0006) [2024-09-30 02:45:45,051][1157819] Updated weights for policy 0, policy_version 148548 (0.0006) [2024-09-30 02:45:45,466][1157520] Fps is (10 sec: 73727.6, 60 sec: 79462.1, 300 sec: 78934.7). Total num frames: 608485376. Throughput: 0: 19512.2. Samples: 142089904. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 02:45:45,466][1157520] Avg episode reward: [(0, '55.639')] [2024-09-30 02:45:45,594][1157819] Updated weights for policy 0, policy_version 148558 (0.0006) [2024-09-30 02:45:46,128][1157819] Updated weights for policy 0, policy_version 148568 (0.0006) [2024-09-30 02:45:46,673][1157819] Updated weights for policy 0, policy_version 148578 (0.0006) [2024-09-30 02:45:47,254][1157819] Updated weights for policy 0, policy_version 148588 (0.0006) [2024-09-30 02:45:47,748][1157819] Updated weights for policy 0, policy_version 148598 (0.0006) [2024-09-30 02:45:48,295][1157819] Updated weights for policy 0, policy_version 148608 (0.0006) [2024-09-30 02:45:48,882][1157819] Updated weights for policy 0, policy_version 148618 (0.0006) [2024-09-30 02:45:49,401][1157819] Updated weights for policy 0, policy_version 148628 (0.0006) [2024-09-30 02:45:49,969][1157819] Updated weights for policy 0, policy_version 148638 (0.0006) [2024-09-30 02:45:50,466][1157520] Fps is (10 sec: 74546.9, 60 sec: 78438.3, 300 sec: 78726.5). Total num frames: 608858112. Throughput: 0: 19234.0. Samples: 142201876. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 02:45:50,466][1157520] Avg episode reward: [(0, '56.343')] [2024-09-30 02:45:50,531][1157819] Updated weights for policy 0, policy_version 148648 (0.0006) [2024-09-30 02:45:51,068][1157819] Updated weights for policy 0, policy_version 148658 (0.0006) [2024-09-30 02:45:51,627][1157819] Updated weights for policy 0, policy_version 148668 (0.0006) [2024-09-30 02:45:52,172][1157819] Updated weights for policy 0, policy_version 148678 (0.0006) [2024-09-30 02:45:52,717][1157819] Updated weights for policy 0, policy_version 148688 (0.0006) [2024-09-30 02:45:53,233][1157819] Updated weights for policy 0, policy_version 148698 (0.0006) [2024-09-30 02:45:53,765][1157819] Updated weights for policy 0, policy_version 148708 (0.0006) [2024-09-30 02:45:54,300][1157819] Updated weights for policy 0, policy_version 148718 (0.0006) [2024-09-30 02:45:54,816][1157819] Updated weights for policy 0, policy_version 148728 (0.0006) [2024-09-30 02:45:55,309][1157819] Updated weights for policy 0, policy_version 148738 (0.0006) [2024-09-30 02:45:55,466][1157520] Fps is (10 sec: 75777.0, 60 sec: 77960.4, 300 sec: 78615.4). Total num frames: 609243136. Throughput: 0: 19146.8. Samples: 142257560. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 02:45:55,466][1157520] Avg episode reward: [(0, '56.383')] [2024-09-30 02:45:55,795][1157819] Updated weights for policy 0, policy_version 148748 (0.0006) [2024-09-30 02:45:56,321][1157819] Updated weights for policy 0, policy_version 148758 (0.0006) [2024-09-30 02:45:56,832][1157819] Updated weights for policy 0, policy_version 148768 (0.0006) [2024-09-30 02:45:57,362][1157819] Updated weights for policy 0, policy_version 148778 (0.0006) [2024-09-30 02:45:57,861][1157819] Updated weights for policy 0, policy_version 148788 (0.0006) [2024-09-30 02:45:58,372][1157819] Updated weights for policy 0, policy_version 148798 (0.0006) [2024-09-30 02:45:58,875][1157819] Updated weights for policy 0, policy_version 148808 (0.0006) [2024-09-30 02:45:59,417][1157819] Updated weights for policy 0, policy_version 148818 (0.0006) [2024-09-30 02:45:59,928][1157819] Updated weights for policy 0, policy_version 148828 (0.0006) [2024-09-30 02:46:00,429][1157819] Updated weights for policy 0, policy_version 148838 (0.0006) [2024-09-30 02:46:00,466][1157520] Fps is (10 sec: 78233.6, 60 sec: 77550.8, 300 sec: 78615.5). Total num frames: 609640448. Throughput: 0: 19139.2. Samples: 142377308. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 02:46:00,466][1157520] Avg episode reward: [(0, '55.572')] [2024-09-30 02:46:00,938][1157819] Updated weights for policy 0, policy_version 148848 (0.0006) [2024-09-30 02:46:01,459][1157819] Updated weights for policy 0, policy_version 148858 (0.0006) [2024-09-30 02:46:01,965][1157819] Updated weights for policy 0, policy_version 148868 (0.0006) [2024-09-30 02:46:02,499][1157819] Updated weights for policy 0, policy_version 148878 (0.0006) [2024-09-30 02:46:03,041][1157819] Updated weights for policy 0, policy_version 148888 (0.0006) [2024-09-30 02:46:03,538][1157819] Updated weights for policy 0, policy_version 148898 (0.0006) [2024-09-30 02:46:04,044][1157819] Updated weights for policy 0, policy_version 148908 (0.0006) [2024-09-30 02:46:04,533][1157819] Updated weights for policy 0, policy_version 148918 (0.0006) [2024-09-30 02:46:05,036][1157819] Updated weights for policy 0, policy_version 148928 (0.0006) [2024-09-30 02:46:05,466][1157520] Fps is (10 sec: 79461.5, 60 sec: 77345.9, 300 sec: 78643.2). Total num frames: 610037760. Throughput: 0: 19156.5. Samples: 142496552. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 02:46:05,466][1157520] Avg episode reward: [(0, '55.490')] [2024-09-30 02:46:05,472][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000148935_610037760.pth... [2024-09-30 02:46:05,519][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000144282_590979072.pth [2024-09-30 02:46:05,607][1157819] Updated weights for policy 0, policy_version 148938 (0.0006) [2024-09-30 02:46:06,174][1157819] Updated weights for policy 0, policy_version 148948 (0.0006) [2024-09-30 02:46:06,741][1157819] Updated weights for policy 0, policy_version 148958 (0.0006) [2024-09-30 02:46:07,323][1157819] Updated weights for policy 0, policy_version 148968 (0.0006) [2024-09-30 02:46:07,844][1157819] Updated weights for policy 0, policy_version 148978 (0.0006) [2024-09-30 02:46:08,434][1157819] Updated weights for policy 0, policy_version 148988 (0.0006) [2024-09-30 02:46:08,972][1157819] Updated weights for policy 0, policy_version 148998 (0.0006) [2024-09-30 02:46:09,533][1157819] Updated weights for policy 0, policy_version 149008 (0.0006) [2024-09-30 02:46:10,104][1157819] Updated weights for policy 0, policy_version 149018 (0.0006) [2024-09-30 02:46:10,466][1157520] Fps is (10 sec: 76185.6, 60 sec: 76799.9, 300 sec: 78518.2). Total num frames: 610402304. Throughput: 0: 19072.9. Samples: 142551836. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 02:46:10,466][1157520] Avg episode reward: [(0, '56.639')] [2024-09-30 02:46:10,658][1157819] Updated weights for policy 0, policy_version 149028 (0.0006) [2024-09-30 02:46:11,234][1157819] Updated weights for policy 0, policy_version 149038 (0.0006) [2024-09-30 02:46:11,776][1157819] Updated weights for policy 0, policy_version 149048 (0.0006) [2024-09-30 02:46:12,329][1157819] Updated weights for policy 0, policy_version 149058 (0.0006) [2024-09-30 02:46:12,851][1157819] Updated weights for policy 0, policy_version 149068 (0.0006) [2024-09-30 02:46:13,401][1157819] Updated weights for policy 0, policy_version 149078 (0.0006) [2024-09-30 02:46:13,961][1157819] Updated weights for policy 0, policy_version 149088 (0.0006) [2024-09-30 02:46:14,486][1157819] Updated weights for policy 0, policy_version 149098 (0.0006) [2024-09-30 02:46:15,065][1157819] Updated weights for policy 0, policy_version 149108 (0.0006) [2024-09-30 02:46:15,466][1157520] Fps is (10 sec: 74138.6, 60 sec: 76458.6, 300 sec: 78546.0). Total num frames: 610779136. Throughput: 0: 18961.5. Samples: 142663048. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 02:46:15,466][1157520] Avg episode reward: [(0, '55.806')] [2024-09-30 02:46:15,586][1157819] Updated weights for policy 0, policy_version 149118 (0.0006) [2024-09-30 02:46:16,114][1157819] Updated weights for policy 0, policy_version 149128 (0.0006) [2024-09-30 02:46:16,667][1157819] Updated weights for policy 0, policy_version 149138 (0.0006) [2024-09-30 02:46:17,236][1157819] Updated weights for policy 0, policy_version 149148 (0.0006) [2024-09-30 02:46:17,805][1157819] Updated weights for policy 0, policy_version 149158 (0.0006) [2024-09-30 02:46:18,365][1157819] Updated weights for policy 0, policy_version 149168 (0.0006) [2024-09-30 02:46:18,893][1157819] Updated weights for policy 0, policy_version 149178 (0.0006) [2024-09-30 02:46:19,390][1157819] Updated weights for policy 0, policy_version 149188 (0.0006) [2024-09-30 02:46:19,893][1157819] Updated weights for policy 0, policy_version 149198 (0.0006) [2024-09-30 02:46:20,415][1157819] Updated weights for policy 0, policy_version 149208 (0.0006) [2024-09-30 02:46:20,466][1157520] Fps is (10 sec: 75775.7, 60 sec: 76185.4, 300 sec: 78615.4). Total num frames: 611160064. Throughput: 0: 18906.9. Samples: 142776320. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 02:46:20,466][1157520] Avg episode reward: [(0, '53.880')] [2024-09-30 02:46:20,906][1157819] Updated weights for policy 0, policy_version 149218 (0.0006) [2024-09-30 02:46:21,408][1157819] Updated weights for policy 0, policy_version 149228 (0.0006) [2024-09-30 02:46:21,910][1157819] Updated weights for policy 0, policy_version 149238 (0.0006) [2024-09-30 02:46:22,408][1157819] Updated weights for policy 0, policy_version 149248 (0.0006) [2024-09-30 02:46:22,957][1157819] Updated weights for policy 0, policy_version 149258 (0.0006) [2024-09-30 02:46:23,450][1157819] Updated weights for policy 0, policy_version 149268 (0.0006) [2024-09-30 02:46:23,965][1157819] Updated weights for policy 0, policy_version 149278 (0.0006) [2024-09-30 02:46:24,460][1157819] Updated weights for policy 0, policy_version 149288 (0.0006) [2024-09-30 02:46:24,962][1157819] Updated weights for policy 0, policy_version 149298 (0.0006) [2024-09-30 02:46:25,466][1157520] Fps is (10 sec: 78233.4, 60 sec: 76322.0, 300 sec: 78754.3). Total num frames: 611561472. Throughput: 0: 19040.7. Samples: 142836856. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 02:46:25,466][1157520] Avg episode reward: [(0, '54.971')] [2024-09-30 02:46:25,473][1157819] Updated weights for policy 0, policy_version 149308 (0.0006) [2024-09-30 02:46:26,002][1157819] Updated weights for policy 0, policy_version 149318 (0.0006) [2024-09-30 02:46:26,560][1157819] Updated weights for policy 0, policy_version 149328 (0.0006) [2024-09-30 02:46:27,116][1157819] Updated weights for policy 0, policy_version 149338 (0.0006) [2024-09-30 02:46:27,605][1157819] Updated weights for policy 0, policy_version 149348 (0.0006) [2024-09-30 02:46:28,099][1157819] Updated weights for policy 0, policy_version 149358 (0.0006) [2024-09-30 02:46:28,601][1157819] Updated weights for policy 0, policy_version 149368 (0.0006) [2024-09-30 02:46:29,108][1157819] Updated weights for policy 0, policy_version 149378 (0.0006) [2024-09-30 02:46:29,594][1157819] Updated weights for policy 0, policy_version 149388 (0.0006) [2024-09-30 02:46:30,098][1157819] Updated weights for policy 0, policy_version 149398 (0.0006) [2024-09-30 02:46:30,466][1157520] Fps is (10 sec: 80282.1, 60 sec: 76595.1, 300 sec: 78865.4). Total num frames: 611962880. Throughput: 0: 19262.9. Samples: 142956732. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 02:46:30,466][1157520] Avg episode reward: [(0, '53.810')] [2024-09-30 02:46:30,587][1157819] Updated weights for policy 0, policy_version 149408 (0.0006) [2024-09-30 02:46:31,083][1157819] Updated weights for policy 0, policy_version 149418 (0.0006) [2024-09-30 02:46:31,610][1157819] Updated weights for policy 0, policy_version 149428 (0.0006) [2024-09-30 02:46:32,167][1157819] Updated weights for policy 0, policy_version 149438 (0.0006) [2024-09-30 02:46:32,693][1157819] Updated weights for policy 0, policy_version 149448 (0.0006) [2024-09-30 02:46:33,236][1157819] Updated weights for policy 0, policy_version 149458 (0.0006) [2024-09-30 02:46:33,786][1157819] Updated weights for policy 0, policy_version 149468 (0.0006) [2024-09-30 02:46:34,302][1157819] Updated weights for policy 0, policy_version 149478 (0.0006) [2024-09-30 02:46:34,861][1157819] Updated weights for policy 0, policy_version 149488 (0.0006) [2024-09-30 02:46:35,425][1157819] Updated weights for policy 0, policy_version 149498 (0.0006) [2024-09-30 02:46:35,466][1157520] Fps is (10 sec: 78643.5, 60 sec: 76663.6, 300 sec: 78920.9). Total num frames: 612347904. Throughput: 0: 19383.3. Samples: 143074124. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 02:46:35,466][1157520] Avg episode reward: [(0, '56.797')] [2024-09-30 02:46:36,006][1157819] Updated weights for policy 0, policy_version 149508 (0.0006) [2024-09-30 02:46:36,558][1157819] Updated weights for policy 0, policy_version 149518 (0.0006) [2024-09-30 02:46:37,058][1157819] Updated weights for policy 0, policy_version 149528 (0.0006) [2024-09-30 02:46:37,573][1157819] Updated weights for policy 0, policy_version 149538 (0.0006) [2024-09-30 02:46:38,096][1157819] Updated weights for policy 0, policy_version 149548 (0.0006) [2024-09-30 02:46:38,622][1157819] Updated weights for policy 0, policy_version 149558 (0.0006) [2024-09-30 02:46:39,133][1157819] Updated weights for policy 0, policy_version 149568 (0.0006) [2024-09-30 02:46:39,653][1157819] Updated weights for policy 0, policy_version 149578 (0.0006) [2024-09-30 02:46:40,178][1157819] Updated weights for policy 0, policy_version 149588 (0.0006) [2024-09-30 02:46:40,466][1157520] Fps is (10 sec: 77004.8, 60 sec: 77004.8, 300 sec: 78962.6). Total num frames: 612732928. Throughput: 0: 19405.1. Samples: 143130788. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 02:46:40,466][1157520] Avg episode reward: [(0, '55.018')] [2024-09-30 02:46:40,508][1157736] Signal inference workers to stop experience collection... (10300 times) [2024-09-30 02:46:40,509][1157736] Signal inference workers to resume experience collection... (10300 times) [2024-09-30 02:46:40,512][1157819] InferenceWorker_p0-w0: stopping experience collection (10300 times) [2024-09-30 02:46:40,512][1157819] InferenceWorker_p0-w0: resuming experience collection (10300 times) [2024-09-30 02:46:40,692][1157819] Updated weights for policy 0, policy_version 149598 (0.0006) [2024-09-30 02:46:41,291][1157819] Updated weights for policy 0, policy_version 149608 (0.0006) [2024-09-30 02:46:41,818][1157819] Updated weights for policy 0, policy_version 149618 (0.0006) [2024-09-30 02:46:42,368][1157819] Updated weights for policy 0, policy_version 149628 (0.0006) [2024-09-30 02:46:42,915][1157819] Updated weights for policy 0, policy_version 149638 (0.0006) [2024-09-30 02:46:43,459][1157819] Updated weights for policy 0, policy_version 149648 (0.0006) [2024-09-30 02:46:44,012][1157819] Updated weights for policy 0, policy_version 149658 (0.0006) [2024-09-30 02:46:44,531][1157819] Updated weights for policy 0, policy_version 149668 (0.0006) [2024-09-30 02:46:45,103][1157819] Updated weights for policy 0, policy_version 149678 (0.0006) [2024-09-30 02:46:45,466][1157520] Fps is (10 sec: 75775.9, 60 sec: 77005.0, 300 sec: 78948.7). Total num frames: 613105664. Throughput: 0: 19294.8. Samples: 143245576. Policy #0 lag: (min: 0.0, avg: 2.0, max: 5.0) [2024-09-30 02:46:45,466][1157520] Avg episode reward: [(0, '54.320')] [2024-09-30 02:46:45,671][1157819] Updated weights for policy 0, policy_version 149688 (0.0006) [2024-09-30 02:46:46,224][1157819] Updated weights for policy 0, policy_version 149698 (0.0006) [2024-09-30 02:46:46,768][1157819] Updated weights for policy 0, policy_version 149708 (0.0006) [2024-09-30 02:46:47,315][1157819] Updated weights for policy 0, policy_version 149718 (0.0006) [2024-09-30 02:46:47,850][1157819] Updated weights for policy 0, policy_version 149728 (0.0006) [2024-09-30 02:46:48,384][1157819] Updated weights for policy 0, policy_version 149738 (0.0006) [2024-09-30 02:46:48,961][1157819] Updated weights for policy 0, policy_version 149748 (0.0006) [2024-09-30 02:46:49,497][1157819] Updated weights for policy 0, policy_version 149758 (0.0006) [2024-09-30 02:46:50,028][1157819] Updated weights for policy 0, policy_version 149768 (0.0006) [2024-09-30 02:46:50,466][1157520] Fps is (10 sec: 74547.2, 60 sec: 77004.8, 300 sec: 78948.7). Total num frames: 613478400. Throughput: 0: 19133.9. Samples: 143357576. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:46:50,466][1157520] Avg episode reward: [(0, '52.995')] [2024-09-30 02:46:50,580][1157819] Updated weights for policy 0, policy_version 149778 (0.0006) [2024-09-30 02:46:51,101][1157819] Updated weights for policy 0, policy_version 149788 (0.0006) [2024-09-30 02:46:51,598][1157819] Updated weights for policy 0, policy_version 149798 (0.0006) [2024-09-30 02:46:52,112][1157819] Updated weights for policy 0, policy_version 149808 (0.0006) [2024-09-30 02:46:52,633][1157819] Updated weights for policy 0, policy_version 149818 (0.0006) [2024-09-30 02:46:53,163][1157819] Updated weights for policy 0, policy_version 149828 (0.0006) [2024-09-30 02:46:53,665][1157819] Updated weights for policy 0, policy_version 149838 (0.0006) [2024-09-30 02:46:54,170][1157819] Updated weights for policy 0, policy_version 149848 (0.0006) [2024-09-30 02:46:54,675][1157819] Updated weights for policy 0, policy_version 149858 (0.0006) [2024-09-30 02:46:55,224][1157819] Updated weights for policy 0, policy_version 149868 (0.0007) [2024-09-30 02:46:55,466][1157520] Fps is (10 sec: 77004.8, 60 sec: 77209.6, 300 sec: 78976.4). Total num frames: 613875712. Throughput: 0: 19218.5. Samples: 143416668. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:46:55,466][1157520] Avg episode reward: [(0, '56.829')] [2024-09-30 02:46:55,721][1157819] Updated weights for policy 0, policy_version 149878 (0.0006) [2024-09-30 02:46:56,228][1157819] Updated weights for policy 0, policy_version 149888 (0.0006) [2024-09-30 02:46:56,766][1157819] Updated weights for policy 0, policy_version 149898 (0.0006) [2024-09-30 02:46:57,253][1157819] Updated weights for policy 0, policy_version 149908 (0.0006) [2024-09-30 02:46:57,763][1157819] Updated weights for policy 0, policy_version 149918 (0.0006) [2024-09-30 02:46:58,273][1157819] Updated weights for policy 0, policy_version 149928 (0.0006) [2024-09-30 02:46:58,806][1157819] Updated weights for policy 0, policy_version 149938 (0.0006) [2024-09-30 02:46:59,319][1157819] Updated weights for policy 0, policy_version 149948 (0.0006) [2024-09-30 02:46:59,835][1157819] Updated weights for policy 0, policy_version 149958 (0.0006) [2024-09-30 02:47:00,353][1157819] Updated weights for policy 0, policy_version 149968 (0.0006) [2024-09-30 02:47:00,466][1157520] Fps is (10 sec: 79872.1, 60 sec: 77277.9, 300 sec: 78976.5). Total num frames: 614277120. Throughput: 0: 19400.9. Samples: 143536088. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:47:00,466][1157520] Avg episode reward: [(0, '53.198')] [2024-09-30 02:47:00,862][1157819] Updated weights for policy 0, policy_version 149978 (0.0006) [2024-09-30 02:47:01,410][1157819] Updated weights for policy 0, policy_version 149988 (0.0006) [2024-09-30 02:47:01,930][1157819] Updated weights for policy 0, policy_version 149998 (0.0006) [2024-09-30 02:47:02,442][1157819] Updated weights for policy 0, policy_version 150008 (0.0006) [2024-09-30 02:47:02,949][1157819] Updated weights for policy 0, policy_version 150018 (0.0006) [2024-09-30 02:47:03,458][1157819] Updated weights for policy 0, policy_version 150028 (0.0006) [2024-09-30 02:47:03,975][1157819] Updated weights for policy 0, policy_version 150038 (0.0006) [2024-09-30 02:47:04,498][1157819] Updated weights for policy 0, policy_version 150048 (0.0006) [2024-09-30 02:47:05,018][1157819] Updated weights for policy 0, policy_version 150058 (0.0006) [2024-09-30 02:47:05,466][1157520] Fps is (10 sec: 79462.3, 60 sec: 77209.8, 300 sec: 78934.8). Total num frames: 614670336. Throughput: 0: 19515.6. Samples: 143654520. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:47:05,466][1157520] Avg episode reward: [(0, '55.839')] [2024-09-30 02:47:05,547][1157819] Updated weights for policy 0, policy_version 150068 (0.0006) [2024-09-30 02:47:06,057][1157819] Updated weights for policy 0, policy_version 150078 (0.0006) [2024-09-30 02:47:06,590][1157819] Updated weights for policy 0, policy_version 150088 (0.0006) [2024-09-30 02:47:07,119][1157819] Updated weights for policy 0, policy_version 150098 (0.0006) [2024-09-30 02:47:07,629][1157819] Updated weights for policy 0, policy_version 150108 (0.0006) [2024-09-30 02:47:08,142][1157819] Updated weights for policy 0, policy_version 150118 (0.0006) [2024-09-30 02:47:08,641][1157819] Updated weights for policy 0, policy_version 150128 (0.0006) [2024-09-30 02:47:09,139][1157819] Updated weights for policy 0, policy_version 150138 (0.0006) [2024-09-30 02:47:09,659][1157819] Updated weights for policy 0, policy_version 150148 (0.0006) [2024-09-30 02:47:10,186][1157819] Updated weights for policy 0, policy_version 150158 (0.0006) [2024-09-30 02:47:10,466][1157520] Fps is (10 sec: 79052.8, 60 sec: 77755.8, 300 sec: 78851.5). Total num frames: 615067648. Throughput: 0: 19487.9. Samples: 143713812. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:47:10,466][1157520] Avg episode reward: [(0, '55.493')] [2024-09-30 02:47:10,703][1157819] Updated weights for policy 0, policy_version 150168 (0.0006) [2024-09-30 02:47:11,204][1157819] Updated weights for policy 0, policy_version 150178 (0.0006) [2024-09-30 02:47:11,739][1157819] Updated weights for policy 0, policy_version 150188 (0.0006) [2024-09-30 02:47:12,271][1157819] Updated weights for policy 0, policy_version 150198 (0.0006) [2024-09-30 02:47:12,765][1157819] Updated weights for policy 0, policy_version 150208 (0.0006) [2024-09-30 02:47:13,261][1157819] Updated weights for policy 0, policy_version 150218 (0.0006) [2024-09-30 02:47:13,788][1157819] Updated weights for policy 0, policy_version 150228 (0.0006) [2024-09-30 02:47:14,315][1157819] Updated weights for policy 0, policy_version 150238 (0.0006) [2024-09-30 02:47:14,821][1157819] Updated weights for policy 0, policy_version 150248 (0.0006) [2024-09-30 02:47:15,315][1157819] Updated weights for policy 0, policy_version 150258 (0.0006) [2024-09-30 02:47:15,466][1157520] Fps is (10 sec: 79462.5, 60 sec: 78097.1, 300 sec: 78782.0). Total num frames: 615464960. Throughput: 0: 19481.5. Samples: 143833400. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:47:15,466][1157520] Avg episode reward: [(0, '53.529')] [2024-09-30 02:47:15,825][1157819] Updated weights for policy 0, policy_version 150268 (0.0006) [2024-09-30 02:47:16,346][1157819] Updated weights for policy 0, policy_version 150278 (0.0006) [2024-09-30 02:47:16,851][1157819] Updated weights for policy 0, policy_version 150288 (0.0006) [2024-09-30 02:47:17,375][1157819] Updated weights for policy 0, policy_version 150298 (0.0006) [2024-09-30 02:47:17,928][1157819] Updated weights for policy 0, policy_version 150308 (0.0006) [2024-09-30 02:47:18,421][1157819] Updated weights for policy 0, policy_version 150318 (0.0006) [2024-09-30 02:47:18,950][1157819] Updated weights for policy 0, policy_version 150328 (0.0006) [2024-09-30 02:47:19,478][1157819] Updated weights for policy 0, policy_version 150338 (0.0006) [2024-09-30 02:47:19,966][1157819] Updated weights for policy 0, policy_version 150348 (0.0006) [2024-09-30 02:47:20,466][1157520] Fps is (10 sec: 79462.3, 60 sec: 78370.2, 300 sec: 78698.7). Total num frames: 615862272. Throughput: 0: 19514.0. Samples: 143952256. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:47:20,466][1157520] Avg episode reward: [(0, '55.577')] [2024-09-30 02:47:20,488][1157819] Updated weights for policy 0, policy_version 150358 (0.0006) [2024-09-30 02:47:21,045][1157819] Updated weights for policy 0, policy_version 150368 (0.0006) [2024-09-30 02:47:21,559][1157819] Updated weights for policy 0, policy_version 150378 (0.0006) [2024-09-30 02:47:22,078][1157819] Updated weights for policy 0, policy_version 150388 (0.0006) [2024-09-30 02:47:22,561][1157819] Updated weights for policy 0, policy_version 150398 (0.0006) [2024-09-30 02:47:23,115][1157819] Updated weights for policy 0, policy_version 150408 (0.0006) [2024-09-30 02:47:23,664][1157819] Updated weights for policy 0, policy_version 150418 (0.0006) [2024-09-30 02:47:24,151][1157819] Updated weights for policy 0, policy_version 150428 (0.0006) [2024-09-30 02:47:24,657][1157819] Updated weights for policy 0, policy_version 150438 (0.0006) [2024-09-30 02:47:25,194][1157819] Updated weights for policy 0, policy_version 150448 (0.0006) [2024-09-30 02:47:25,466][1157520] Fps is (10 sec: 79052.7, 60 sec: 78233.6, 300 sec: 78532.1). Total num frames: 616255488. Throughput: 0: 19564.8. Samples: 144011204. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:47:25,466][1157520] Avg episode reward: [(0, '55.523')] [2024-09-30 02:47:25,753][1157819] Updated weights for policy 0, policy_version 150458 (0.0006) [2024-09-30 02:47:26,295][1157819] Updated weights for policy 0, policy_version 150468 (0.0006) [2024-09-30 02:47:26,865][1157819] Updated weights for policy 0, policy_version 150478 (0.0006) [2024-09-30 02:47:27,377][1157819] Updated weights for policy 0, policy_version 150488 (0.0006) [2024-09-30 02:47:27,895][1157819] Updated weights for policy 0, policy_version 150498 (0.0006) [2024-09-30 02:47:28,437][1157819] Updated weights for policy 0, policy_version 150508 (0.0006) [2024-09-30 02:47:29,012][1157819] Updated weights for policy 0, policy_version 150518 (0.0006) [2024-09-30 02:47:29,545][1157819] Updated weights for policy 0, policy_version 150528 (0.0006) [2024-09-30 02:47:30,101][1157819] Updated weights for policy 0, policy_version 150538 (0.0006) [2024-09-30 02:47:30,466][1157520] Fps is (10 sec: 76595.2, 60 sec: 77755.7, 300 sec: 78282.2). Total num frames: 616628224. Throughput: 0: 19573.3. Samples: 144126372. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:47:30,466][1157520] Avg episode reward: [(0, '53.766')] [2024-09-30 02:47:30,638][1157819] Updated weights for policy 0, policy_version 150548 (0.0006) [2024-09-30 02:47:31,127][1157819] Updated weights for policy 0, policy_version 150558 (0.0006) [2024-09-30 02:47:31,627][1157819] Updated weights for policy 0, policy_version 150568 (0.0006) [2024-09-30 02:47:32,140][1157819] Updated weights for policy 0, policy_version 150578 (0.0006) [2024-09-30 02:47:32,676][1157819] Updated weights for policy 0, policy_version 150588 (0.0006) [2024-09-30 02:47:33,182][1157819] Updated weights for policy 0, policy_version 150598 (0.0006) [2024-09-30 02:47:33,695][1157819] Updated weights for policy 0, policy_version 150608 (0.0006) [2024-09-30 02:47:34,218][1157819] Updated weights for policy 0, policy_version 150618 (0.0006) [2024-09-30 02:47:34,751][1157819] Updated weights for policy 0, policy_version 150628 (0.0006) [2024-09-30 02:47:35,286][1157819] Updated weights for policy 0, policy_version 150638 (0.0006) [2024-09-30 02:47:35,466][1157520] Fps is (10 sec: 77004.8, 60 sec: 77960.5, 300 sec: 78171.1). Total num frames: 617025536. Throughput: 0: 19692.4. Samples: 144243736. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:47:35,466][1157520] Avg episode reward: [(0, '54.799')] [2024-09-30 02:47:35,826][1157819] Updated weights for policy 0, policy_version 150648 (0.0006) [2024-09-30 02:47:36,334][1157819] Updated weights for policy 0, policy_version 150658 (0.0006) [2024-09-30 02:47:36,825][1157819] Updated weights for policy 0, policy_version 150668 (0.0006) [2024-09-30 02:47:37,316][1157819] Updated weights for policy 0, policy_version 150678 (0.0006) [2024-09-30 02:47:37,867][1157819] Updated weights for policy 0, policy_version 150688 (0.0006) [2024-09-30 02:47:38,364][1157819] Updated weights for policy 0, policy_version 150698 (0.0006) [2024-09-30 02:47:38,859][1157819] Updated weights for policy 0, policy_version 150708 (0.0006) [2024-09-30 02:47:39,383][1157819] Updated weights for policy 0, policy_version 150718 (0.0006) [2024-09-30 02:47:39,917][1157819] Updated weights for policy 0, policy_version 150728 (0.0006) [2024-09-30 02:47:40,466][1157520] Fps is (10 sec: 79052.7, 60 sec: 78097.1, 300 sec: 78087.8). Total num frames: 617418752. Throughput: 0: 19709.2. Samples: 144303584. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:47:40,466][1157520] Avg episode reward: [(0, '52.579')] [2024-09-30 02:47:40,482][1157736] Signal inference workers to stop experience collection... (10350 times) [2024-09-30 02:47:40,485][1157736] Signal inference workers to resume experience collection... (10350 times) [2024-09-30 02:47:40,486][1157819] Updated weights for policy 0, policy_version 150738 (0.0006) [2024-09-30 02:47:40,489][1157819] InferenceWorker_p0-w0: stopping experience collection (10350 times) [2024-09-30 02:47:40,491][1157819] InferenceWorker_p0-w0: resuming experience collection (10350 times) [2024-09-30 02:47:40,995][1157819] Updated weights for policy 0, policy_version 150748 (0.0006) [2024-09-30 02:47:41,533][1157819] Updated weights for policy 0, policy_version 150758 (0.0006) [2024-09-30 02:47:42,114][1157819] Updated weights for policy 0, policy_version 150768 (0.0006) [2024-09-30 02:47:42,652][1157819] Updated weights for policy 0, policy_version 150778 (0.0006) [2024-09-30 02:47:43,201][1157819] Updated weights for policy 0, policy_version 150788 (0.0006) [2024-09-30 02:47:43,747][1157819] Updated weights for policy 0, policy_version 150798 (0.0006) [2024-09-30 02:47:44,305][1157819] Updated weights for policy 0, policy_version 150808 (0.0006) [2024-09-30 02:47:44,855][1157819] Updated weights for policy 0, policy_version 150818 (0.0006) [2024-09-30 02:47:45,366][1157819] Updated weights for policy 0, policy_version 150828 (0.0006) [2024-09-30 02:47:45,466][1157520] Fps is (10 sec: 77004.7, 60 sec: 78165.3, 300 sec: 77962.8). Total num frames: 617795584. Throughput: 0: 19597.8. Samples: 144417992. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:47:45,466][1157520] Avg episode reward: [(0, '54.108')] [2024-09-30 02:47:45,869][1157819] Updated weights for policy 0, policy_version 150838 (0.0006) [2024-09-30 02:47:46,370][1157819] Updated weights for policy 0, policy_version 150848 (0.0006) [2024-09-30 02:47:46,883][1157819] Updated weights for policy 0, policy_version 150858 (0.0006) [2024-09-30 02:47:47,344][1157819] Updated weights for policy 0, policy_version 150868 (0.0006) [2024-09-30 02:47:47,849][1157819] Updated weights for policy 0, policy_version 150878 (0.0006) [2024-09-30 02:47:48,342][1157819] Updated weights for policy 0, policy_version 150888 (0.0006) [2024-09-30 02:47:48,852][1157819] Updated weights for policy 0, policy_version 150898 (0.0006) [2024-09-30 02:47:49,349][1157819] Updated weights for policy 0, policy_version 150908 (0.0006) [2024-09-30 02:47:49,807][1157819] Updated weights for policy 0, policy_version 150918 (0.0006) [2024-09-30 02:47:50,266][1157819] Updated weights for policy 0, policy_version 150928 (0.0006) [2024-09-30 02:47:50,466][1157520] Fps is (10 sec: 79872.3, 60 sec: 78984.6, 300 sec: 78032.3). Total num frames: 618217472. Throughput: 0: 19676.8. Samples: 144539976. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 02:47:50,466][1157520] Avg episode reward: [(0, '56.715')] [2024-09-30 02:47:50,717][1157819] Updated weights for policy 0, policy_version 150938 (0.0006) [2024-09-30 02:47:51,208][1157819] Updated weights for policy 0, policy_version 150948 (0.0006) [2024-09-30 02:47:51,710][1157819] Updated weights for policy 0, policy_version 150958 (0.0006) [2024-09-30 02:47:52,168][1157819] Updated weights for policy 0, policy_version 150968 (0.0006) [2024-09-30 02:47:52,645][1157819] Updated weights for policy 0, policy_version 150978 (0.0006) [2024-09-30 02:47:53,140][1157819] Updated weights for policy 0, policy_version 150988 (0.0006) [2024-09-30 02:47:53,638][1157819] Updated weights for policy 0, policy_version 150998 (0.0006) [2024-09-30 02:47:54,151][1157819] Updated weights for policy 0, policy_version 151008 (0.0006) [2024-09-30 02:47:54,646][1157819] Updated weights for policy 0, policy_version 151018 (0.0006) [2024-09-30 02:47:55,146][1157819] Updated weights for policy 0, policy_version 151028 (0.0006) [2024-09-30 02:47:55,466][1157520] Fps is (10 sec: 83968.0, 60 sec: 79325.8, 300 sec: 78101.7). Total num frames: 618635264. Throughput: 0: 19785.0. Samples: 144604136. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:47:55,466][1157520] Avg episode reward: [(0, '55.008')] [2024-09-30 02:47:55,663][1157819] Updated weights for policy 0, policy_version 151038 (0.0006) [2024-09-30 02:47:56,196][1157819] Updated weights for policy 0, policy_version 151048 (0.0006) [2024-09-30 02:47:56,746][1157819] Updated weights for policy 0, policy_version 151058 (0.0006) [2024-09-30 02:47:57,294][1157819] Updated weights for policy 0, policy_version 151068 (0.0006) [2024-09-30 02:47:57,831][1157819] Updated weights for policy 0, policy_version 151078 (0.0007) [2024-09-30 02:47:58,374][1157819] Updated weights for policy 0, policy_version 151088 (0.0006) [2024-09-30 02:47:58,936][1157819] Updated weights for policy 0, policy_version 151098 (0.0007) [2024-09-30 02:47:59,482][1157819] Updated weights for policy 0, policy_version 151108 (0.0006) [2024-09-30 02:47:59,973][1157819] Updated weights for policy 0, policy_version 151118 (0.0006) [2024-09-30 02:48:00,466][1157520] Fps is (10 sec: 79871.6, 60 sec: 78984.5, 300 sec: 78115.6). Total num frames: 619016192. Throughput: 0: 19741.0. Samples: 144721744. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:48:00,466][1157520] Avg episode reward: [(0, '56.143')] [2024-09-30 02:48:00,507][1157819] Updated weights for policy 0, policy_version 151128 (0.0006) [2024-09-30 02:48:00,713][1157736] Signal inference workers to stop experience collection... (10400 times) [2024-09-30 02:48:00,714][1157736] Signal inference workers to resume experience collection... (10400 times) [2024-09-30 02:48:00,718][1157819] InferenceWorker_p0-w0: stopping experience collection (10400 times) [2024-09-30 02:48:00,720][1157819] InferenceWorker_p0-w0: resuming experience collection (10400 times) [2024-09-30 02:48:01,021][1157819] Updated weights for policy 0, policy_version 151138 (0.0007) [2024-09-30 02:48:01,540][1157819] Updated weights for policy 0, policy_version 151148 (0.0006) [2024-09-30 02:48:02,035][1157819] Updated weights for policy 0, policy_version 151158 (0.0006) [2024-09-30 02:48:02,526][1157819] Updated weights for policy 0, policy_version 151168 (0.0006) [2024-09-30 02:48:03,026][1157819] Updated weights for policy 0, policy_version 151178 (0.0006) [2024-09-30 02:48:03,515][1157819] Updated weights for policy 0, policy_version 151188 (0.0006) [2024-09-30 02:48:04,015][1157819] Updated weights for policy 0, policy_version 151198 (0.0006) [2024-09-30 02:48:04,514][1157819] Updated weights for policy 0, policy_version 151208 (0.0006) [2024-09-30 02:48:05,000][1157819] Updated weights for policy 0, policy_version 151218 (0.0006) [2024-09-30 02:48:05,466][1157520] Fps is (10 sec: 79051.9, 60 sec: 79257.4, 300 sec: 78212.7). Total num frames: 619425792. Throughput: 0: 19786.4. Samples: 144842648. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:48:05,466][1157520] Avg episode reward: [(0, '55.858')] [2024-09-30 02:48:05,472][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000151227_619425792.pth... [2024-09-30 02:48:05,526][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000146636_600621056.pth [2024-09-30 02:48:05,547][1157819] Updated weights for policy 0, policy_version 151228 (0.0006) [2024-09-30 02:48:06,010][1157819] Updated weights for policy 0, policy_version 151238 (0.0006) [2024-09-30 02:48:06,538][1157819] Updated weights for policy 0, policy_version 151248 (0.0006) [2024-09-30 02:48:07,032][1157819] Updated weights for policy 0, policy_version 151258 (0.0006) [2024-09-30 02:48:07,557][1157819] Updated weights for policy 0, policy_version 151268 (0.0006) [2024-09-30 02:48:08,068][1157819] Updated weights for policy 0, policy_version 151278 (0.0006) [2024-09-30 02:48:08,612][1157819] Updated weights for policy 0, policy_version 151288 (0.0006) [2024-09-30 02:48:09,145][1157819] Updated weights for policy 0, policy_version 151298 (0.0006) [2024-09-30 02:48:09,673][1157819] Updated weights for policy 0, policy_version 151308 (0.0006) [2024-09-30 02:48:10,195][1157819] Updated weights for policy 0, policy_version 151318 (0.0006) [2024-09-30 02:48:10,466][1157520] Fps is (10 sec: 79871.6, 60 sec: 79120.9, 300 sec: 78268.3). Total num frames: 619814912. Throughput: 0: 19815.7. Samples: 144902912. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:48:10,466][1157520] Avg episode reward: [(0, '52.838')] [2024-09-30 02:48:10,755][1157819] Updated weights for policy 0, policy_version 151328 (0.0006) [2024-09-30 02:48:11,260][1157819] Updated weights for policy 0, policy_version 151338 (0.0006) [2024-09-30 02:48:11,786][1157819] Updated weights for policy 0, policy_version 151348 (0.0006) [2024-09-30 02:48:12,280][1157819] Updated weights for policy 0, policy_version 151358 (0.0006) [2024-09-30 02:48:12,775][1157819] Updated weights for policy 0, policy_version 151368 (0.0006) [2024-09-30 02:48:13,306][1157819] Updated weights for policy 0, policy_version 151378 (0.0006) [2024-09-30 02:48:13,823][1157819] Updated weights for policy 0, policy_version 151388 (0.0006) [2024-09-30 02:48:14,320][1157819] Updated weights for policy 0, policy_version 151398 (0.0006) [2024-09-30 02:48:14,838][1157819] Updated weights for policy 0, policy_version 151408 (0.0006) [2024-09-30 02:48:15,359][1157819] Updated weights for policy 0, policy_version 151418 (0.0006) [2024-09-30 02:48:15,466][1157520] Fps is (10 sec: 79053.7, 60 sec: 79189.3, 300 sec: 78310.0). Total num frames: 620216320. Throughput: 0: 19869.0. Samples: 145020480. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:48:15,466][1157520] Avg episode reward: [(0, '54.440')] [2024-09-30 02:48:15,853][1157819] Updated weights for policy 0, policy_version 151428 (0.0006) [2024-09-30 02:48:16,357][1157819] Updated weights for policy 0, policy_version 151438 (0.0006) [2024-09-30 02:48:16,857][1157819] Updated weights for policy 0, policy_version 151448 (0.0006) [2024-09-30 02:48:17,361][1157819] Updated weights for policy 0, policy_version 151458 (0.0006) [2024-09-30 02:48:17,847][1157819] Updated weights for policy 0, policy_version 151468 (0.0006) [2024-09-30 02:48:18,268][1157819] Updated weights for policy 0, policy_version 151478 (0.0006) [2024-09-30 02:48:18,771][1157819] Updated weights for policy 0, policy_version 151488 (0.0006) [2024-09-30 02:48:19,280][1157819] Updated weights for policy 0, policy_version 151498 (0.0006) [2024-09-30 02:48:19,762][1157819] Updated weights for policy 0, policy_version 151508 (0.0006) [2024-09-30 02:48:20,268][1157819] Updated weights for policy 0, policy_version 151518 (0.0006) [2024-09-30 02:48:20,466][1157520] Fps is (10 sec: 81920.8, 60 sec: 79530.7, 300 sec: 78448.8). Total num frames: 620634112. Throughput: 0: 20010.7. Samples: 145144216. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:48:20,466][1157520] Avg episode reward: [(0, '53.720')] [2024-09-30 02:48:20,767][1157819] Updated weights for policy 0, policy_version 151528 (0.0006) [2024-09-30 02:48:21,266][1157819] Updated weights for policy 0, policy_version 151538 (0.0006) [2024-09-30 02:48:21,757][1157819] Updated weights for policy 0, policy_version 151548 (0.0006) [2024-09-30 02:48:22,261][1157819] Updated weights for policy 0, policy_version 151558 (0.0006) [2024-09-30 02:48:22,770][1157819] Updated weights for policy 0, policy_version 151568 (0.0006) [2024-09-30 02:48:23,255][1157819] Updated weights for policy 0, policy_version 151578 (0.0006) [2024-09-30 02:48:23,757][1157819] Updated weights for policy 0, policy_version 151588 (0.0006) [2024-09-30 02:48:24,260][1157819] Updated weights for policy 0, policy_version 151598 (0.0006) [2024-09-30 02:48:24,738][1157819] Updated weights for policy 0, policy_version 151608 (0.0006) [2024-09-30 02:48:25,219][1157819] Updated weights for policy 0, policy_version 151618 (0.0006) [2024-09-30 02:48:25,466][1157520] Fps is (10 sec: 82738.8, 60 sec: 79803.7, 300 sec: 78518.2). Total num frames: 621043712. Throughput: 0: 20059.2. Samples: 145206248. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:48:25,466][1157520] Avg episode reward: [(0, '57.233')] [2024-09-30 02:48:25,745][1157819] Updated weights for policy 0, policy_version 151628 (0.0006) [2024-09-30 02:48:26,229][1157819] Updated weights for policy 0, policy_version 151638 (0.0006) [2024-09-30 02:48:26,704][1157819] Updated weights for policy 0, policy_version 151648 (0.0006) [2024-09-30 02:48:27,228][1157819] Updated weights for policy 0, policy_version 151658 (0.0006) [2024-09-30 02:48:27,694][1157819] Updated weights for policy 0, policy_version 151668 (0.0006) [2024-09-30 02:48:28,171][1157819] Updated weights for policy 0, policy_version 151678 (0.0006) [2024-09-30 02:48:28,635][1157819] Updated weights for policy 0, policy_version 151688 (0.0006) [2024-09-30 02:48:29,104][1157819] Updated weights for policy 0, policy_version 151698 (0.0006) [2024-09-30 02:48:29,591][1157819] Updated weights for policy 0, policy_version 151708 (0.0006) [2024-09-30 02:48:30,081][1157819] Updated weights for policy 0, policy_version 151718 (0.0006) [2024-09-30 02:48:30,466][1157520] Fps is (10 sec: 83148.8, 60 sec: 80622.9, 300 sec: 78615.4). Total num frames: 621465600. Throughput: 0: 20303.0. Samples: 145331624. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:48:30,466][1157520] Avg episode reward: [(0, '54.739')] [2024-09-30 02:48:30,562][1157819] Updated weights for policy 0, policy_version 151728 (0.0006) [2024-09-30 02:48:31,055][1157819] Updated weights for policy 0, policy_version 151738 (0.0006) [2024-09-30 02:48:31,565][1157819] Updated weights for policy 0, policy_version 151748 (0.0006) [2024-09-30 02:48:32,045][1157819] Updated weights for policy 0, policy_version 151758 (0.0006) [2024-09-30 02:48:32,532][1157819] Updated weights for policy 0, policy_version 151768 (0.0006) [2024-09-30 02:48:33,030][1157819] Updated weights for policy 0, policy_version 151778 (0.0006) [2024-09-30 02:48:33,518][1157819] Updated weights for policy 0, policy_version 151788 (0.0006) [2024-09-30 02:48:34,031][1157819] Updated weights for policy 0, policy_version 151798 (0.0006) [2024-09-30 02:48:34,534][1157819] Updated weights for policy 0, policy_version 151808 (0.0006) [2024-09-30 02:48:35,092][1157819] Updated weights for policy 0, policy_version 151818 (0.0006) [2024-09-30 02:48:35,466][1157520] Fps is (10 sec: 83149.3, 60 sec: 80827.7, 300 sec: 78684.9). Total num frames: 621875200. Throughput: 0: 20356.1. Samples: 145456000. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:48:35,466][1157520] Avg episode reward: [(0, '55.989')] [2024-09-30 02:48:35,613][1157819] Updated weights for policy 0, policy_version 151828 (0.0006) [2024-09-30 02:48:36,119][1157819] Updated weights for policy 0, policy_version 151838 (0.0006) [2024-09-30 02:48:36,620][1157819] Updated weights for policy 0, policy_version 151848 (0.0006) [2024-09-30 02:48:37,126][1157819] Updated weights for policy 0, policy_version 151858 (0.0006) [2024-09-30 02:48:37,614][1157819] Updated weights for policy 0, policy_version 151868 (0.0006) [2024-09-30 02:48:38,108][1157819] Updated weights for policy 0, policy_version 151878 (0.0006) [2024-09-30 02:48:38,607][1157819] Updated weights for policy 0, policy_version 151888 (0.0006) [2024-09-30 02:48:39,137][1157819] Updated weights for policy 0, policy_version 151898 (0.0006) [2024-09-30 02:48:39,668][1157819] Updated weights for policy 0, policy_version 151908 (0.0006) [2024-09-30 02:48:40,169][1157819] Updated weights for policy 0, policy_version 151918 (0.0006) [2024-09-30 02:48:40,466][1157520] Fps is (10 sec: 81100.9, 60 sec: 80964.3, 300 sec: 78712.7). Total num frames: 622276608. Throughput: 0: 20270.9. Samples: 145516324. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:48:40,466][1157520] Avg episode reward: [(0, '55.853')] [2024-09-30 02:48:40,671][1157819] Updated weights for policy 0, policy_version 151928 (0.0006) [2024-09-30 02:48:41,192][1157819] Updated weights for policy 0, policy_version 151938 (0.0006) [2024-09-30 02:48:41,693][1157819] Updated weights for policy 0, policy_version 151948 (0.0006) [2024-09-30 02:48:42,210][1157819] Updated weights for policy 0, policy_version 151958 (0.0006) [2024-09-30 02:48:42,735][1157819] Updated weights for policy 0, policy_version 151968 (0.0006) [2024-09-30 02:48:43,255][1157819] Updated weights for policy 0, policy_version 151978 (0.0006) [2024-09-30 02:48:43,786][1157819] Updated weights for policy 0, policy_version 151988 (0.0006) [2024-09-30 02:48:44,307][1157819] Updated weights for policy 0, policy_version 151998 (0.0006) [2024-09-30 02:48:44,806][1157819] Updated weights for policy 0, policy_version 152008 (0.0006) [2024-09-30 02:48:45,298][1157819] Updated weights for policy 0, policy_version 152018 (0.0006) [2024-09-30 02:48:45,466][1157520] Fps is (10 sec: 80281.6, 60 sec: 81373.9, 300 sec: 78726.5). Total num frames: 622678016. Throughput: 0: 20314.4. Samples: 145635892. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:48:45,466][1157520] Avg episode reward: [(0, '55.600')] [2024-09-30 02:48:45,803][1157819] Updated weights for policy 0, policy_version 152028 (0.0006) [2024-09-30 02:48:46,299][1157819] Updated weights for policy 0, policy_version 152038 (0.0006) [2024-09-30 02:48:46,789][1157819] Updated weights for policy 0, policy_version 152048 (0.0006) [2024-09-30 02:48:47,287][1157819] Updated weights for policy 0, policy_version 152058 (0.0006) [2024-09-30 02:48:47,792][1157819] Updated weights for policy 0, policy_version 152068 (0.0006) [2024-09-30 02:48:48,285][1157819] Updated weights for policy 0, policy_version 152078 (0.0006) [2024-09-30 02:48:48,797][1157819] Updated weights for policy 0, policy_version 152088 (0.0006) [2024-09-30 02:48:49,302][1157819] Updated weights for policy 0, policy_version 152098 (0.0006) [2024-09-30 02:48:49,799][1157819] Updated weights for policy 0, policy_version 152108 (0.0006) [2024-09-30 02:48:50,299][1157819] Updated weights for policy 0, policy_version 152118 (0.0006) [2024-09-30 02:48:50,466][1157520] Fps is (10 sec: 81100.7, 60 sec: 81169.0, 300 sec: 78782.0). Total num frames: 623087616. Throughput: 0: 20348.3. Samples: 145758320. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:48:50,466][1157520] Avg episode reward: [(0, '54.583')] [2024-09-30 02:48:50,777][1157819] Updated weights for policy 0, policy_version 152128 (0.0006) [2024-09-30 02:48:51,286][1157819] Updated weights for policy 0, policy_version 152138 (0.0006) [2024-09-30 02:48:51,783][1157819] Updated weights for policy 0, policy_version 152148 (0.0006) [2024-09-30 02:48:51,909][1157736] Signal inference workers to stop experience collection... (10450 times) [2024-09-30 02:48:51,910][1157736] Signal inference workers to resume experience collection... (10450 times) [2024-09-30 02:48:51,914][1157819] InferenceWorker_p0-w0: stopping experience collection (10450 times) [2024-09-30 02:48:51,914][1157819] InferenceWorker_p0-w0: resuming experience collection (10450 times) [2024-09-30 02:48:52,271][1157819] Updated weights for policy 0, policy_version 152158 (0.0006) [2024-09-30 02:48:52,768][1157819] Updated weights for policy 0, policy_version 152168 (0.0006) [2024-09-30 02:48:53,262][1157819] Updated weights for policy 0, policy_version 152178 (0.0006) [2024-09-30 02:48:53,749][1157819] Updated weights for policy 0, policy_version 152188 (0.0006) [2024-09-30 02:48:54,245][1157819] Updated weights for policy 0, policy_version 152198 (0.0006) [2024-09-30 02:48:54,747][1157819] Updated weights for policy 0, policy_version 152208 (0.0006) [2024-09-30 02:48:55,226][1157819] Updated weights for policy 0, policy_version 152218 (0.0006) [2024-09-30 02:48:55,466][1157520] Fps is (10 sec: 82329.4, 60 sec: 81100.8, 300 sec: 78907.0). Total num frames: 623501312. Throughput: 0: 20388.0. Samples: 145820372. Policy #0 lag: (min: 1.0, avg: 2.2, max: 6.0) [2024-09-30 02:48:55,466][1157520] Avg episode reward: [(0, '55.121')] [2024-09-30 02:48:55,706][1157819] Updated weights for policy 0, policy_version 152228 (0.0006) [2024-09-30 02:48:56,210][1157819] Updated weights for policy 0, policy_version 152238 (0.0006) [2024-09-30 02:48:56,696][1157819] Updated weights for policy 0, policy_version 152248 (0.0006) [2024-09-30 02:48:57,196][1157819] Updated weights for policy 0, policy_version 152258 (0.0006) [2024-09-30 02:48:57,702][1157819] Updated weights for policy 0, policy_version 152268 (0.0006) [2024-09-30 02:48:58,196][1157819] Updated weights for policy 0, policy_version 152278 (0.0006) [2024-09-30 02:48:58,681][1157819] Updated weights for policy 0, policy_version 152288 (0.0006) [2024-09-30 02:48:59,185][1157819] Updated weights for policy 0, policy_version 152298 (0.0006) [2024-09-30 02:48:59,683][1157819] Updated weights for policy 0, policy_version 152308 (0.0006) [2024-09-30 02:49:00,177][1157819] Updated weights for policy 0, policy_version 152318 (0.0006) [2024-09-30 02:49:00,466][1157520] Fps is (10 sec: 82739.1, 60 sec: 81647.0, 300 sec: 78962.6). Total num frames: 623915008. Throughput: 0: 20542.2. Samples: 145944880. Policy #0 lag: (min: 1.0, avg: 2.2, max: 6.0) [2024-09-30 02:49:00,466][1157520] Avg episode reward: [(0, '54.261')] [2024-09-30 02:49:00,682][1157819] Updated weights for policy 0, policy_version 152328 (0.0006) [2024-09-30 02:49:01,175][1157819] Updated weights for policy 0, policy_version 152338 (0.0006) [2024-09-30 02:49:01,634][1157819] Updated weights for policy 0, policy_version 152348 (0.0006) [2024-09-30 02:49:02,131][1157819] Updated weights for policy 0, policy_version 152358 (0.0006) [2024-09-30 02:49:02,599][1157819] Updated weights for policy 0, policy_version 152368 (0.0006) [2024-09-30 02:49:03,070][1157819] Updated weights for policy 0, policy_version 152378 (0.0006) [2024-09-30 02:49:03,567][1157819] Updated weights for policy 0, policy_version 152388 (0.0006) [2024-09-30 02:49:04,040][1157819] Updated weights for policy 0, policy_version 152398 (0.0006) [2024-09-30 02:49:04,532][1157819] Updated weights for policy 0, policy_version 152408 (0.0006) [2024-09-30 02:49:04,987][1157819] Updated weights for policy 0, policy_version 152418 (0.0006) [2024-09-30 02:49:05,466][1157520] Fps is (10 sec: 83558.6, 60 sec: 81851.9, 300 sec: 79254.1). Total num frames: 624336896. Throughput: 0: 20606.9. Samples: 146071528. Policy #0 lag: (min: 1.0, avg: 2.2, max: 6.0) [2024-09-30 02:49:05,466][1157520] Avg episode reward: [(0, '53.978')] [2024-09-30 02:49:05,487][1157819] Updated weights for policy 0, policy_version 152428 (0.0006) [2024-09-30 02:49:06,007][1157819] Updated weights for policy 0, policy_version 152438 (0.0006) [2024-09-30 02:49:06,522][1157819] Updated weights for policy 0, policy_version 152448 (0.0006) [2024-09-30 02:49:07,027][1157819] Updated weights for policy 0, policy_version 152458 (0.0006) [2024-09-30 02:49:07,569][1157819] Updated weights for policy 0, policy_version 152468 (0.0006) [2024-09-30 02:49:08,126][1157819] Updated weights for policy 0, policy_version 152478 (0.0006) [2024-09-30 02:49:08,658][1157819] Updated weights for policy 0, policy_version 152488 (0.0006) [2024-09-30 02:49:09,178][1157819] Updated weights for policy 0, policy_version 152498 (0.0006) [2024-09-30 02:49:09,706][1157819] Updated weights for policy 0, policy_version 152508 (0.0006) [2024-09-30 02:49:10,236][1157819] Updated weights for policy 0, policy_version 152518 (0.0007) [2024-09-30 02:49:10,466][1157520] Fps is (10 sec: 81510.4, 60 sec: 81920.1, 300 sec: 79393.0). Total num frames: 624730112. Throughput: 0: 20554.8. Samples: 146131212. Policy #0 lag: (min: 1.0, avg: 2.2, max: 6.0) [2024-09-30 02:49:10,466][1157520] Avg episode reward: [(0, '54.540')] [2024-09-30 02:49:10,706][1157819] Updated weights for policy 0, policy_version 152528 (0.0007) [2024-09-30 02:49:11,215][1157819] Updated weights for policy 0, policy_version 152538 (0.0007) [2024-09-30 02:49:11,718][1157819] Updated weights for policy 0, policy_version 152548 (0.0007) [2024-09-30 02:49:12,221][1157819] Updated weights for policy 0, policy_version 152558 (0.0006) [2024-09-30 02:49:12,729][1157819] Updated weights for policy 0, policy_version 152568 (0.0007) [2024-09-30 02:49:13,245][1157819] Updated weights for policy 0, policy_version 152578 (0.0006) [2024-09-30 02:49:13,751][1157819] Updated weights for policy 0, policy_version 152588 (0.0006) [2024-09-30 02:49:14,219][1157819] Updated weights for policy 0, policy_version 152598 (0.0006) [2024-09-30 02:49:14,709][1157819] Updated weights for policy 0, policy_version 152608 (0.0006) [2024-09-30 02:49:15,158][1157819] Updated weights for policy 0, policy_version 152618 (0.0006) [2024-09-30 02:49:15,466][1157520] Fps is (10 sec: 81100.7, 60 sec: 82193.1, 300 sec: 79531.8). Total num frames: 625147904. Throughput: 0: 20427.4. Samples: 146250856. Policy #0 lag: (min: 1.0, avg: 2.2, max: 6.0) [2024-09-30 02:49:15,466][1157520] Avg episode reward: [(0, '51.589')] [2024-09-30 02:49:15,649][1157819] Updated weights for policy 0, policy_version 152628 (0.0006) [2024-09-30 02:49:16,136][1157819] Updated weights for policy 0, policy_version 152638 (0.0006) [2024-09-30 02:49:16,646][1157819] Updated weights for policy 0, policy_version 152648 (0.0006) [2024-09-30 02:49:17,162][1157819] Updated weights for policy 0, policy_version 152658 (0.0006) [2024-09-30 02:49:17,665][1157819] Updated weights for policy 0, policy_version 152668 (0.0006) [2024-09-30 02:49:18,179][1157819] Updated weights for policy 0, policy_version 152678 (0.0006) [2024-09-30 02:49:18,701][1157819] Updated weights for policy 0, policy_version 152688 (0.0006) [2024-09-30 02:49:19,211][1157819] Updated weights for policy 0, policy_version 152698 (0.0006) [2024-09-30 02:49:19,696][1157819] Updated weights for policy 0, policy_version 152708 (0.0006) [2024-09-30 02:49:20,201][1157819] Updated weights for policy 0, policy_version 152718 (0.0006) [2024-09-30 02:49:20,466][1157520] Fps is (10 sec: 82328.9, 60 sec: 81988.1, 300 sec: 79531.8). Total num frames: 625553408. Throughput: 0: 20418.6. Samples: 146374840. Policy #0 lag: (min: 1.0, avg: 2.2, max: 6.0) [2024-09-30 02:49:20,466][1157520] Avg episode reward: [(0, '55.073')] [2024-09-30 02:49:20,669][1157819] Updated weights for policy 0, policy_version 152728 (0.0006) [2024-09-30 02:49:21,170][1157819] Updated weights for policy 0, policy_version 152738 (0.0006) [2024-09-30 02:49:21,642][1157819] Updated weights for policy 0, policy_version 152748 (0.0006) [2024-09-30 02:49:22,122][1157819] Updated weights for policy 0, policy_version 152758 (0.0006) [2024-09-30 02:49:22,638][1157819] Updated weights for policy 0, policy_version 152768 (0.0006) [2024-09-30 02:49:23,142][1157819] Updated weights for policy 0, policy_version 152778 (0.0006) [2024-09-30 02:49:23,633][1157819] Updated weights for policy 0, policy_version 152788 (0.0006) [2024-09-30 02:49:24,143][1157819] Updated weights for policy 0, policy_version 152798 (0.0006) [2024-09-30 02:49:24,662][1157819] Updated weights for policy 0, policy_version 152808 (0.0006) [2024-09-30 02:49:25,190][1157819] Updated weights for policy 0, policy_version 152818 (0.0006) [2024-09-30 02:49:25,466][1157520] Fps is (10 sec: 81510.0, 60 sec: 81988.3, 300 sec: 79559.5). Total num frames: 625963008. Throughput: 0: 20466.6. Samples: 146437324. Policy #0 lag: (min: 1.0, avg: 2.2, max: 6.0) [2024-09-30 02:49:25,466][1157520] Avg episode reward: [(0, '56.930')] [2024-09-30 02:49:25,712][1157819] Updated weights for policy 0, policy_version 152828 (0.0006) [2024-09-30 02:49:26,217][1157819] Updated weights for policy 0, policy_version 152838 (0.0006) [2024-09-30 02:49:26,735][1157819] Updated weights for policy 0, policy_version 152848 (0.0006) [2024-09-30 02:49:27,257][1157819] Updated weights for policy 0, policy_version 152858 (0.0006) [2024-09-30 02:49:27,775][1157819] Updated weights for policy 0, policy_version 152868 (0.0006) [2024-09-30 02:49:28,291][1157819] Updated weights for policy 0, policy_version 152878 (0.0006) [2024-09-30 02:49:28,804][1157819] Updated weights for policy 0, policy_version 152888 (0.0006) [2024-09-30 02:49:29,344][1157819] Updated weights for policy 0, policy_version 152898 (0.0006) [2024-09-30 02:49:29,843][1157819] Updated weights for policy 0, policy_version 152908 (0.0006) [2024-09-30 02:49:30,375][1157819] Updated weights for policy 0, policy_version 152918 (0.0006) [2024-09-30 02:49:30,466][1157520] Fps is (10 sec: 80282.3, 60 sec: 81510.4, 300 sec: 79545.7). Total num frames: 626356224. Throughput: 0: 20465.0. Samples: 146556816. Policy #0 lag: (min: 1.0, avg: 2.2, max: 6.0) [2024-09-30 02:49:30,466][1157520] Avg episode reward: [(0, '53.605')] [2024-09-30 02:49:30,878][1157819] Updated weights for policy 0, policy_version 152928 (0.0006) [2024-09-30 02:49:31,416][1157819] Updated weights for policy 0, policy_version 152938 (0.0006) [2024-09-30 02:49:31,957][1157819] Updated weights for policy 0, policy_version 152948 (0.0006) [2024-09-30 02:49:32,480][1157819] Updated weights for policy 0, policy_version 152958 (0.0006) [2024-09-30 02:49:33,034][1157819] Updated weights for policy 0, policy_version 152968 (0.0006) [2024-09-30 02:49:33,538][1157819] Updated weights for policy 0, policy_version 152978 (0.0006) [2024-09-30 02:49:34,051][1157819] Updated weights for policy 0, policy_version 152988 (0.0006) [2024-09-30 02:49:34,561][1157819] Updated weights for policy 0, policy_version 152998 (0.0006) [2024-09-30 02:49:35,066][1157819] Updated weights for policy 0, policy_version 153008 (0.0006) [2024-09-30 02:49:35,466][1157520] Fps is (10 sec: 78643.5, 60 sec: 81237.3, 300 sec: 79490.1). Total num frames: 626749440. Throughput: 0: 20359.3. Samples: 146674488. Policy #0 lag: (min: 1.0, avg: 2.2, max: 6.0) [2024-09-30 02:49:35,466][1157520] Avg episode reward: [(0, '56.912')] [2024-09-30 02:49:35,566][1157819] Updated weights for policy 0, policy_version 153018 (0.0006) [2024-09-30 02:49:36,088][1157819] Updated weights for policy 0, policy_version 153028 (0.0006) [2024-09-30 02:49:36,599][1157819] Updated weights for policy 0, policy_version 153038 (0.0006) [2024-09-30 02:49:37,081][1157819] Updated weights for policy 0, policy_version 153048 (0.0006) [2024-09-30 02:49:37,585][1157819] Updated weights for policy 0, policy_version 153058 (0.0006) [2024-09-30 02:49:38,072][1157819] Updated weights for policy 0, policy_version 153068 (0.0006) [2024-09-30 02:49:38,574][1157819] Updated weights for policy 0, policy_version 153078 (0.0006) [2024-09-30 02:49:39,076][1157819] Updated weights for policy 0, policy_version 153088 (0.0006) [2024-09-30 02:49:39,571][1157819] Updated weights for policy 0, policy_version 153098 (0.0006) [2024-09-30 02:49:40,066][1157819] Updated weights for policy 0, policy_version 153108 (0.0006) [2024-09-30 02:49:40,466][1157520] Fps is (10 sec: 80691.0, 60 sec: 81442.1, 300 sec: 79476.3). Total num frames: 627163136. Throughput: 0: 20337.3. Samples: 146735552. Policy #0 lag: (min: 1.0, avg: 2.2, max: 6.0) [2024-09-30 02:49:40,466][1157520] Avg episode reward: [(0, '53.298')] [2024-09-30 02:49:40,573][1157819] Updated weights for policy 0, policy_version 153118 (0.0006) [2024-09-30 02:49:41,082][1157819] Updated weights for policy 0, policy_version 153128 (0.0006) [2024-09-30 02:49:41,580][1157819] Updated weights for policy 0, policy_version 153138 (0.0006) [2024-09-30 02:49:42,095][1157819] Updated weights for policy 0, policy_version 153148 (0.0007) [2024-09-30 02:49:42,631][1157819] Updated weights for policy 0, policy_version 153158 (0.0006) [2024-09-30 02:49:43,143][1157819] Updated weights for policy 0, policy_version 153168 (0.0006) [2024-09-30 02:49:43,660][1157819] Updated weights for policy 0, policy_version 153178 (0.0006) [2024-09-30 02:49:44,261][1157819] Updated weights for policy 0, policy_version 153188 (0.0006) [2024-09-30 02:49:44,789][1157819] Updated weights for policy 0, policy_version 153198 (0.0006) [2024-09-30 02:49:45,280][1157736] Signal inference workers to stop experience collection... (10500 times) [2024-09-30 02:49:45,285][1157819] InferenceWorker_p0-w0: stopping experience collection (10500 times) [2024-09-30 02:49:45,285][1157736] Signal inference workers to resume experience collection... (10500 times) [2024-09-30 02:49:45,289][1157819] InferenceWorker_p0-w0: resuming experience collection (10500 times) [2024-09-30 02:49:45,333][1157819] Updated weights for policy 0, policy_version 153208 (0.0006) [2024-09-30 02:49:45,466][1157520] Fps is (10 sec: 79872.2, 60 sec: 81169.0, 300 sec: 79309.6). Total num frames: 627548160. Throughput: 0: 20246.6. Samples: 146855976. Policy #0 lag: (min: 1.0, avg: 2.2, max: 6.0) [2024-09-30 02:49:45,466][1157520] Avg episode reward: [(0, '54.779')] [2024-09-30 02:49:45,896][1157819] Updated weights for policy 0, policy_version 153218 (0.0006) [2024-09-30 02:49:46,442][1157819] Updated weights for policy 0, policy_version 153228 (0.0006) [2024-09-30 02:49:46,943][1157819] Updated weights for policy 0, policy_version 153238 (0.0006) [2024-09-30 02:49:47,440][1157819] Updated weights for policy 0, policy_version 153248 (0.0006) [2024-09-30 02:49:47,989][1157819] Updated weights for policy 0, policy_version 153258 (0.0007) [2024-09-30 02:49:48,505][1157819] Updated weights for policy 0, policy_version 153268 (0.0006) [2024-09-30 02:49:49,050][1157819] Updated weights for policy 0, policy_version 153278 (0.0006) [2024-09-30 02:49:49,564][1157819] Updated weights for policy 0, policy_version 153288 (0.0006) [2024-09-30 02:49:50,061][1157819] Updated weights for policy 0, policy_version 153298 (0.0006) [2024-09-30 02:49:50,466][1157520] Fps is (10 sec: 77414.0, 60 sec: 80827.6, 300 sec: 79226.3). Total num frames: 627937280. Throughput: 0: 20004.7. Samples: 146971740. Policy #0 lag: (min: 1.0, avg: 2.2, max: 6.0) [2024-09-30 02:49:50,466][1157520] Avg episode reward: [(0, '56.319')] [2024-09-30 02:49:50,569][1157819] Updated weights for policy 0, policy_version 153308 (0.0006) [2024-09-30 02:49:51,093][1157819] Updated weights for policy 0, policy_version 153318 (0.0006) [2024-09-30 02:49:51,601][1157819] Updated weights for policy 0, policy_version 153328 (0.0006) [2024-09-30 02:49:52,095][1157819] Updated weights for policy 0, policy_version 153338 (0.0006) [2024-09-30 02:49:52,659][1157819] Updated weights for policy 0, policy_version 153348 (0.0006) [2024-09-30 02:49:53,171][1157819] Updated weights for policy 0, policy_version 153358 (0.0006) [2024-09-30 02:49:53,693][1157819] Updated weights for policy 0, policy_version 153368 (0.0006) [2024-09-30 02:49:54,226][1157819] Updated weights for policy 0, policy_version 153378 (0.0006) [2024-09-30 02:49:54,796][1157819] Updated weights for policy 0, policy_version 153388 (0.0006) [2024-09-30 02:49:55,347][1157819] Updated weights for policy 0, policy_version 153398 (0.0006) [2024-09-30 02:49:55,466][1157520] Fps is (10 sec: 77414.3, 60 sec: 80349.9, 300 sec: 79101.4). Total num frames: 628322304. Throughput: 0: 19998.0. Samples: 147031124. Policy #0 lag: (min: 1.0, avg: 2.2, max: 6.0) [2024-09-30 02:49:55,466][1157520] Avg episode reward: [(0, '55.869')] [2024-09-30 02:49:55,914][1157819] Updated weights for policy 0, policy_version 153408 (0.0006) [2024-09-30 02:49:56,478][1157819] Updated weights for policy 0, policy_version 153418 (0.0006) [2024-09-30 02:49:56,982][1157819] Updated weights for policy 0, policy_version 153428 (0.0006) [2024-09-30 02:49:57,549][1157819] Updated weights for policy 0, policy_version 153438 (0.0006) [2024-09-30 02:49:58,114][1157819] Updated weights for policy 0, policy_version 153448 (0.0006) [2024-09-30 02:49:58,673][1157819] Updated weights for policy 0, policy_version 153458 (0.0006) [2024-09-30 02:49:59,192][1157819] Updated weights for policy 0, policy_version 153468 (0.0006) [2024-09-30 02:49:59,731][1157819] Updated weights for policy 0, policy_version 153478 (0.0006) [2024-09-30 02:50:00,283][1157819] Updated weights for policy 0, policy_version 153488 (0.0006) [2024-09-30 02:50:00,466][1157520] Fps is (10 sec: 76185.8, 60 sec: 79735.4, 300 sec: 78990.3). Total num frames: 628699136. Throughput: 0: 19835.9. Samples: 147143472. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 02:50:00,466][1157520] Avg episode reward: [(0, '51.563')] [2024-09-30 02:50:00,822][1157819] Updated weights for policy 0, policy_version 153498 (0.0006) [2024-09-30 02:50:01,351][1157819] Updated weights for policy 0, policy_version 153508 (0.0006) [2024-09-30 02:50:01,883][1157819] Updated weights for policy 0, policy_version 153518 (0.0006) [2024-09-30 02:50:02,418][1157819] Updated weights for policy 0, policy_version 153528 (0.0006) [2024-09-30 02:50:02,962][1157819] Updated weights for policy 0, policy_version 153538 (0.0006) [2024-09-30 02:50:03,504][1157819] Updated weights for policy 0, policy_version 153548 (0.0006) [2024-09-30 02:50:04,034][1157819] Updated weights for policy 0, policy_version 153558 (0.0006) [2024-09-30 02:50:04,576][1157819] Updated weights for policy 0, policy_version 153568 (0.0006) [2024-09-30 02:50:05,102][1157819] Updated weights for policy 0, policy_version 153578 (0.0006) [2024-09-30 02:50:05,466][1157520] Fps is (10 sec: 75775.9, 60 sec: 79052.8, 300 sec: 78934.8). Total num frames: 629080064. Throughput: 0: 19620.8. Samples: 147257776. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 02:50:05,466][1157520] Avg episode reward: [(0, '54.940')] [2024-09-30 02:50:05,472][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000153585_629084160.pth... [2024-09-30 02:50:05,517][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000148935_610037760.pth [2024-09-30 02:50:05,647][1157819] Updated weights for policy 0, policy_version 153588 (0.0006) [2024-09-30 02:50:06,176][1157819] Updated weights for policy 0, policy_version 153598 (0.0006) [2024-09-30 02:50:06,711][1157819] Updated weights for policy 0, policy_version 153608 (0.0006) [2024-09-30 02:50:07,280][1157819] Updated weights for policy 0, policy_version 153618 (0.0006) [2024-09-30 02:50:07,822][1157819] Updated weights for policy 0, policy_version 153628 (0.0006) [2024-09-30 02:50:08,400][1157819] Updated weights for policy 0, policy_version 153638 (0.0006) [2024-09-30 02:50:08,960][1157819] Updated weights for policy 0, policy_version 153648 (0.0006) [2024-09-30 02:50:09,508][1157819] Updated weights for policy 0, policy_version 153658 (0.0006) [2024-09-30 02:50:10,011][1157819] Updated weights for policy 0, policy_version 153668 (0.0006) [2024-09-30 02:50:10,466][1157520] Fps is (10 sec: 75775.9, 60 sec: 78779.7, 300 sec: 78865.3). Total num frames: 629456896. Throughput: 0: 19489.5. Samples: 147314352. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 02:50:10,466][1157520] Avg episode reward: [(0, '54.495')] [2024-09-30 02:50:10,506][1157819] Updated weights for policy 0, policy_version 153678 (0.0006) [2024-09-30 02:50:11,011][1157819] Updated weights for policy 0, policy_version 153688 (0.0006) [2024-09-30 02:50:11,512][1157819] Updated weights for policy 0, policy_version 153698 (0.0006) [2024-09-30 02:50:11,997][1157819] Updated weights for policy 0, policy_version 153708 (0.0006) [2024-09-30 02:50:12,487][1157819] Updated weights for policy 0, policy_version 153718 (0.0006) [2024-09-30 02:50:12,974][1157819] Updated weights for policy 0, policy_version 153728 (0.0006) [2024-09-30 02:50:13,466][1157819] Updated weights for policy 0, policy_version 153738 (0.0006) [2024-09-30 02:50:13,969][1157819] Updated weights for policy 0, policy_version 153748 (0.0006) [2024-09-30 02:50:14,463][1157819] Updated weights for policy 0, policy_version 153758 (0.0006) [2024-09-30 02:50:14,994][1157819] Updated weights for policy 0, policy_version 153768 (0.0006) [2024-09-30 02:50:15,466][1157520] Fps is (10 sec: 79052.8, 60 sec: 78711.4, 300 sec: 78920.8). Total num frames: 629870592. Throughput: 0: 19499.9. Samples: 147434312. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 02:50:15,466][1157520] Avg episode reward: [(0, '54.210')] [2024-09-30 02:50:15,479][1157819] Updated weights for policy 0, policy_version 153778 (0.0006) [2024-09-30 02:50:15,985][1157819] Updated weights for policy 0, policy_version 153788 (0.0006) [2024-09-30 02:50:16,481][1157819] Updated weights for policy 0, policy_version 153798 (0.0006) [2024-09-30 02:50:16,988][1157819] Updated weights for policy 0, policy_version 153808 (0.0006) [2024-09-30 02:50:17,550][1157819] Updated weights for policy 0, policy_version 153818 (0.0006) [2024-09-30 02:50:18,077][1157819] Updated weights for policy 0, policy_version 153828 (0.0006) [2024-09-30 02:50:18,584][1157819] Updated weights for policy 0, policy_version 153838 (0.0006) [2024-09-30 02:50:19,164][1157819] Updated weights for policy 0, policy_version 153848 (0.0006) [2024-09-30 02:50:19,830][1157819] Updated weights for policy 0, policy_version 153858 (0.0006) [2024-09-30 02:50:20,422][1157819] Updated weights for policy 0, policy_version 153868 (0.0006) [2024-09-30 02:50:20,466][1157520] Fps is (10 sec: 78642.9, 60 sec: 78165.3, 300 sec: 78851.4). Total num frames: 630243328. Throughput: 0: 19463.2. Samples: 147550332. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 02:50:20,466][1157520] Avg episode reward: [(0, '50.726')] [2024-09-30 02:50:20,940][1157819] Updated weights for policy 0, policy_version 153878 (0.0006) [2024-09-30 02:50:21,456][1157819] Updated weights for policy 0, policy_version 153888 (0.0006) [2024-09-30 02:50:21,957][1157819] Updated weights for policy 0, policy_version 153898 (0.0006) [2024-09-30 02:50:22,471][1157819] Updated weights for policy 0, policy_version 153908 (0.0006) [2024-09-30 02:50:22,998][1157819] Updated weights for policy 0, policy_version 153918 (0.0007) [2024-09-30 02:50:23,537][1157819] Updated weights for policy 0, policy_version 153928 (0.0006) [2024-09-30 02:50:24,033][1157819] Updated weights for policy 0, policy_version 153938 (0.0006) [2024-09-30 02:50:24,556][1157819] Updated weights for policy 0, policy_version 153948 (0.0006) [2024-09-30 02:50:25,142][1157819] Updated weights for policy 0, policy_version 153958 (0.0006) [2024-09-30 02:50:25,466][1157520] Fps is (10 sec: 76596.3, 60 sec: 77892.5, 300 sec: 78879.2). Total num frames: 630636544. Throughput: 0: 19382.4. Samples: 147607756. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 02:50:25,466][1157520] Avg episode reward: [(0, '53.914')] [2024-09-30 02:50:25,671][1157819] Updated weights for policy 0, policy_version 153968 (0.0006) [2024-09-30 02:50:26,175][1157819] Updated weights for policy 0, policy_version 153978 (0.0006) [2024-09-30 02:50:26,692][1157819] Updated weights for policy 0, policy_version 153988 (0.0006) [2024-09-30 02:50:27,245][1157819] Updated weights for policy 0, policy_version 153998 (0.0006) [2024-09-30 02:50:27,803][1157819] Updated weights for policy 0, policy_version 154008 (0.0006) [2024-09-30 02:50:28,331][1157819] Updated weights for policy 0, policy_version 154018 (0.0006) [2024-09-30 02:50:28,934][1157819] Updated weights for policy 0, policy_version 154028 (0.0006) [2024-09-30 02:50:29,447][1157819] Updated weights for policy 0, policy_version 154038 (0.0006) [2024-09-30 02:50:30,045][1157819] Updated weights for policy 0, policy_version 154048 (0.0006) [2024-09-30 02:50:30,466][1157520] Fps is (10 sec: 76186.3, 60 sec: 77482.7, 300 sec: 78837.6). Total num frames: 631005184. Throughput: 0: 19259.3. Samples: 147722644. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 02:50:30,466][1157520] Avg episode reward: [(0, '54.252')] [2024-09-30 02:50:30,636][1157819] Updated weights for policy 0, policy_version 154058 (0.0006) [2024-09-30 02:50:31,200][1157819] Updated weights for policy 0, policy_version 154068 (0.0006) [2024-09-30 02:50:31,812][1157819] Updated weights for policy 0, policy_version 154078 (0.0006) [2024-09-30 02:50:32,347][1157819] Updated weights for policy 0, policy_version 154088 (0.0006) [2024-09-30 02:50:32,967][1157819] Updated weights for policy 0, policy_version 154098 (0.0006) [2024-09-30 02:50:33,556][1157819] Updated weights for policy 0, policy_version 154108 (0.0006) [2024-09-30 02:50:34,111][1157819] Updated weights for policy 0, policy_version 154118 (0.0006) [2024-09-30 02:50:34,724][1157819] Updated weights for policy 0, policy_version 154128 (0.0006) [2024-09-30 02:50:35,217][1157736] Signal inference workers to stop experience collection... (10550 times) [2024-09-30 02:50:35,220][1157819] InferenceWorker_p0-w0: stopping experience collection (10550 times) [2024-09-30 02:50:35,222][1157736] Signal inference workers to resume experience collection... (10550 times) [2024-09-30 02:50:35,223][1157819] InferenceWorker_p0-w0: resuming experience collection (10550 times) [2024-09-30 02:50:35,326][1157819] Updated weights for policy 0, policy_version 154138 (0.0006) [2024-09-30 02:50:35,466][1157520] Fps is (10 sec: 72088.9, 60 sec: 76800.0, 300 sec: 78795.9). Total num frames: 631357440. Throughput: 0: 19025.8. Samples: 147827900. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 02:50:35,466][1157520] Avg episode reward: [(0, '56.916')] [2024-09-30 02:50:35,901][1157819] Updated weights for policy 0, policy_version 154148 (0.0006) [2024-09-30 02:50:36,493][1157819] Updated weights for policy 0, policy_version 154158 (0.0006) [2024-09-30 02:50:37,068][1157819] Updated weights for policy 0, policy_version 154168 (0.0006) [2024-09-30 02:50:37,670][1157819] Updated weights for policy 0, policy_version 154178 (0.0006) [2024-09-30 02:50:38,276][1157819] Updated weights for policy 0, policy_version 154188 (0.0006) [2024-09-30 02:50:38,940][1157819] Updated weights for policy 0, policy_version 154198 (0.0006) [2024-09-30 02:50:39,561][1157819] Updated weights for policy 0, policy_version 154208 (0.0006) [2024-09-30 02:50:40,171][1157819] Updated weights for policy 0, policy_version 154218 (0.0006) [2024-09-30 02:50:40,466][1157520] Fps is (10 sec: 68812.8, 60 sec: 75502.9, 300 sec: 78671.0). Total num frames: 631693312. Throughput: 0: 18873.4. Samples: 147880428. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 02:50:40,466][1157520] Avg episode reward: [(0, '56.542')] [2024-09-30 02:50:40,843][1157819] Updated weights for policy 0, policy_version 154228 (0.0006) [2024-09-30 02:50:41,481][1157819] Updated weights for policy 0, policy_version 154238 (0.0006) [2024-09-30 02:50:42,064][1157819] Updated weights for policy 0, policy_version 154248 (0.0006) [2024-09-30 02:50:42,619][1157819] Updated weights for policy 0, policy_version 154258 (0.0006) [2024-09-30 02:50:43,230][1157819] Updated weights for policy 0, policy_version 154268 (0.0006) [2024-09-30 02:50:43,755][1157819] Updated weights for policy 0, policy_version 154278 (0.0006) [2024-09-30 02:50:44,365][1157819] Updated weights for policy 0, policy_version 154288 (0.0006) [2024-09-30 02:50:44,865][1157819] Updated weights for policy 0, policy_version 154298 (0.0006) [2024-09-30 02:50:45,450][1157819] Updated weights for policy 0, policy_version 154308 (0.0006) [2024-09-30 02:50:45,466][1157520] Fps is (10 sec: 68812.6, 60 sec: 74956.8, 300 sec: 78601.5). Total num frames: 632045568. Throughput: 0: 18608.7. Samples: 147980864. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 02:50:45,466][1157520] Avg episode reward: [(0, '54.762')] [2024-09-30 02:50:45,981][1157819] Updated weights for policy 0, policy_version 154318 (0.0006) [2024-09-30 02:50:46,464][1157819] Updated weights for policy 0, policy_version 154328 (0.0006) [2024-09-30 02:50:46,956][1157819] Updated weights for policy 0, policy_version 154338 (0.0006) [2024-09-30 02:50:47,465][1157819] Updated weights for policy 0, policy_version 154348 (0.0006) [2024-09-30 02:50:48,020][1157819] Updated weights for policy 0, policy_version 154358 (0.0006) [2024-09-30 02:50:48,519][1157819] Updated weights for policy 0, policy_version 154368 (0.0006) [2024-09-30 02:50:49,006][1157819] Updated weights for policy 0, policy_version 154378 (0.0006) [2024-09-30 02:50:49,533][1157819] Updated weights for policy 0, policy_version 154388 (0.0006) [2024-09-30 02:50:50,038][1157819] Updated weights for policy 0, policy_version 154398 (0.0006) [2024-09-30 02:50:50,466][1157520] Fps is (10 sec: 75366.1, 60 sec: 75161.6, 300 sec: 78657.1). Total num frames: 632446976. Throughput: 0: 18683.7. Samples: 148098544. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 02:50:50,466][1157520] Avg episode reward: [(0, '56.175')] [2024-09-30 02:50:50,524][1157819] Updated weights for policy 0, policy_version 154408 (0.0005) [2024-09-30 02:50:51,045][1157819] Updated weights for policy 0, policy_version 154418 (0.0006) [2024-09-30 02:50:51,581][1157819] Updated weights for policy 0, policy_version 154428 (0.0006) [2024-09-30 02:50:52,082][1157819] Updated weights for policy 0, policy_version 154438 (0.0006) [2024-09-30 02:50:52,566][1157819] Updated weights for policy 0, policy_version 154448 (0.0006) [2024-09-30 02:50:53,119][1157819] Updated weights for policy 0, policy_version 154458 (0.0006) [2024-09-30 02:50:53,629][1157819] Updated weights for policy 0, policy_version 154468 (0.0006) [2024-09-30 02:50:54,125][1157819] Updated weights for policy 0, policy_version 154478 (0.0006) [2024-09-30 02:50:54,624][1157819] Updated weights for policy 0, policy_version 154488 (0.0006) [2024-09-30 02:50:55,132][1157819] Updated weights for policy 0, policy_version 154498 (0.0006) [2024-09-30 02:50:55,466][1157520] Fps is (10 sec: 80281.9, 60 sec: 75434.7, 300 sec: 78671.0). Total num frames: 632848384. Throughput: 0: 18765.3. Samples: 148158788. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 02:50:55,466][1157520] Avg episode reward: [(0, '55.980')] [2024-09-30 02:50:55,656][1157819] Updated weights for policy 0, policy_version 154508 (0.0006) [2024-09-30 02:50:56,171][1157819] Updated weights for policy 0, policy_version 154518 (0.0006) [2024-09-30 02:50:56,677][1157819] Updated weights for policy 0, policy_version 154528 (0.0005) [2024-09-30 02:50:57,156][1157819] Updated weights for policy 0, policy_version 154538 (0.0006) [2024-09-30 02:50:57,661][1157819] Updated weights for policy 0, policy_version 154548 (0.0006) [2024-09-30 02:50:58,208][1157819] Updated weights for policy 0, policy_version 154558 (0.0006) [2024-09-30 02:50:58,675][1157819] Updated weights for policy 0, policy_version 154568 (0.0006) [2024-09-30 02:50:59,164][1157819] Updated weights for policy 0, policy_version 154578 (0.0006) [2024-09-30 02:50:59,656][1157819] Updated weights for policy 0, policy_version 154588 (0.0006) [2024-09-30 02:51:00,146][1157819] Updated weights for policy 0, policy_version 154598 (0.0006) [2024-09-30 02:51:00,466][1157520] Fps is (10 sec: 81101.0, 60 sec: 75980.8, 300 sec: 78712.7). Total num frames: 633257984. Throughput: 0: 18790.4. Samples: 148279880. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 02:51:00,466][1157520] Avg episode reward: [(0, '56.453')] [2024-09-30 02:51:00,619][1157819] Updated weights for policy 0, policy_version 154608 (0.0006) [2024-09-30 02:51:01,069][1157819] Updated weights for policy 0, policy_version 154618 (0.0006) [2024-09-30 02:51:01,551][1157819] Updated weights for policy 0, policy_version 154628 (0.0006) [2024-09-30 02:51:02,050][1157819] Updated weights for policy 0, policy_version 154638 (0.0006) [2024-09-30 02:51:02,540][1157819] Updated weights for policy 0, policy_version 154648 (0.0006) [2024-09-30 02:51:03,023][1157819] Updated weights for policy 0, policy_version 154658 (0.0006) [2024-09-30 02:51:03,495][1157819] Updated weights for policy 0, policy_version 154668 (0.0005) [2024-09-30 02:51:04,001][1157819] Updated weights for policy 0, policy_version 154678 (0.0006) [2024-09-30 02:51:04,509][1157819] Updated weights for policy 0, policy_version 154688 (0.0006) [2024-09-30 02:51:04,953][1157819] Updated weights for policy 0, policy_version 154698 (0.0006) [2024-09-30 02:51:05,451][1157819] Updated weights for policy 0, policy_version 154708 (0.0006) [2024-09-30 02:51:05,466][1157520] Fps is (10 sec: 83558.2, 60 sec: 76731.8, 300 sec: 78920.9). Total num frames: 633683968. Throughput: 0: 19030.8. Samples: 148406716. Policy #0 lag: (min: 0.0, avg: 1.7, max: 6.0) [2024-09-30 02:51:05,466][1157520] Avg episode reward: [(0, '56.366')] [2024-09-30 02:51:05,986][1157819] Updated weights for policy 0, policy_version 154718 (0.0006) [2024-09-30 02:51:06,417][1157819] Updated weights for policy 0, policy_version 154728 (0.0006) [2024-09-30 02:51:06,912][1157819] Updated weights for policy 0, policy_version 154738 (0.0006) [2024-09-30 02:51:07,400][1157819] Updated weights for policy 0, policy_version 154748 (0.0006) [2024-09-30 02:51:07,899][1157819] Updated weights for policy 0, policy_version 154758 (0.0006) [2024-09-30 02:51:08,386][1157819] Updated weights for policy 0, policy_version 154768 (0.0005) [2024-09-30 02:51:08,872][1157819] Updated weights for policy 0, policy_version 154778 (0.0006) [2024-09-30 02:51:09,358][1157819] Updated weights for policy 0, policy_version 154788 (0.0006) [2024-09-30 02:51:09,840][1157819] Updated weights for policy 0, policy_version 154798 (0.0006) [2024-09-30 02:51:10,329][1157819] Updated weights for policy 0, policy_version 154808 (0.0006) [2024-09-30 02:51:10,466][1157520] Fps is (10 sec: 84377.4, 60 sec: 77414.4, 300 sec: 79059.7). Total num frames: 634101760. Throughput: 0: 19153.8. Samples: 148469680. Policy #0 lag: (min: 0.0, avg: 1.7, max: 6.0) [2024-09-30 02:51:10,466][1157520] Avg episode reward: [(0, '55.480')] [2024-09-30 02:51:10,839][1157819] Updated weights for policy 0, policy_version 154818 (0.0006) [2024-09-30 02:51:11,295][1157819] Updated weights for policy 0, policy_version 154828 (0.0006) [2024-09-30 02:51:11,786][1157819] Updated weights for policy 0, policy_version 154838 (0.0006) [2024-09-30 02:51:12,284][1157819] Updated weights for policy 0, policy_version 154848 (0.0006) [2024-09-30 02:51:12,768][1157819] Updated weights for policy 0, policy_version 154858 (0.0006) [2024-09-30 02:51:13,251][1157819] Updated weights for policy 0, policy_version 154868 (0.0006) [2024-09-30 02:51:13,754][1157819] Updated weights for policy 0, policy_version 154878 (0.0005) [2024-09-30 02:51:14,240][1157819] Updated weights for policy 0, policy_version 154888 (0.0006) [2024-09-30 02:51:14,729][1157819] Updated weights for policy 0, policy_version 154898 (0.0006) [2024-09-30 02:51:15,222][1157819] Updated weights for policy 0, policy_version 154908 (0.0006) [2024-09-30 02:51:15,466][1157520] Fps is (10 sec: 83968.1, 60 sec: 77551.0, 300 sec: 79198.6). Total num frames: 634523648. Throughput: 0: 19405.7. Samples: 148595900. Policy #0 lag: (min: 0.0, avg: 1.7, max: 6.0) [2024-09-30 02:51:15,466][1157520] Avg episode reward: [(0, '57.322')] [2024-09-30 02:51:15,725][1157819] Updated weights for policy 0, policy_version 154918 (0.0006) [2024-09-30 02:51:16,176][1157819] Updated weights for policy 0, policy_version 154928 (0.0006) [2024-09-30 02:51:16,671][1157819] Updated weights for policy 0, policy_version 154938 (0.0005) [2024-09-30 02:51:17,162][1157819] Updated weights for policy 0, policy_version 154948 (0.0006) [2024-09-30 02:51:17,266][1157736] Signal inference workers to stop experience collection... (10600 times) [2024-09-30 02:51:17,267][1157736] Signal inference workers to resume experience collection... (10600 times) [2024-09-30 02:51:17,271][1157819] InferenceWorker_p0-w0: stopping experience collection (10600 times) [2024-09-30 02:51:17,271][1157819] InferenceWorker_p0-w0: resuming experience collection (10600 times) [2024-09-30 02:51:17,618][1157819] Updated weights for policy 0, policy_version 154958 (0.0006) [2024-09-30 02:51:18,122][1157819] Updated weights for policy 0, policy_version 154968 (0.0006) [2024-09-30 02:51:18,618][1157819] Updated weights for policy 0, policy_version 154978 (0.0005) [2024-09-30 02:51:19,075][1157819] Updated weights for policy 0, policy_version 154988 (0.0005) [2024-09-30 02:51:19,563][1157819] Updated weights for policy 0, policy_version 154998 (0.0006) [2024-09-30 02:51:19,995][1157819] Updated weights for policy 0, policy_version 155008 (0.0006) [2024-09-30 02:51:20,466][1157520] Fps is (10 sec: 84787.3, 60 sec: 78438.5, 300 sec: 79281.9). Total num frames: 634949632. Throughput: 0: 19898.3. Samples: 148723324. Policy #0 lag: (min: 0.0, avg: 1.7, max: 6.0) [2024-09-30 02:51:20,466][1157520] Avg episode reward: [(0, '56.233')] [2024-09-30 02:51:20,483][1157819] Updated weights for policy 0, policy_version 155018 (0.0006) [2024-09-30 02:51:20,942][1157819] Updated weights for policy 0, policy_version 155028 (0.0006) [2024-09-30 02:51:21,388][1157819] Updated weights for policy 0, policy_version 155038 (0.0006) [2024-09-30 02:51:21,863][1157819] Updated weights for policy 0, policy_version 155048 (0.0006) [2024-09-30 02:51:22,320][1157819] Updated weights for policy 0, policy_version 155058 (0.0006) [2024-09-30 02:51:22,797][1157819] Updated weights for policy 0, policy_version 155068 (0.0006) [2024-09-30 02:51:23,252][1157819] Updated weights for policy 0, policy_version 155078 (0.0006) [2024-09-30 02:51:23,716][1157819] Updated weights for policy 0, policy_version 155088 (0.0006) [2024-09-30 02:51:24,178][1157819] Updated weights for policy 0, policy_version 155098 (0.0006) [2024-09-30 02:51:24,641][1157819] Updated weights for policy 0, policy_version 155108 (0.0006) [2024-09-30 02:51:25,087][1157819] Updated weights for policy 0, policy_version 155118 (0.0006) [2024-09-30 02:51:25,466][1157520] Fps is (10 sec: 86835.1, 60 sec: 79257.5, 300 sec: 79420.7). Total num frames: 635392000. Throughput: 0: 20206.9. Samples: 148789740. Policy #0 lag: (min: 0.0, avg: 1.7, max: 6.0) [2024-09-30 02:51:25,466][1157520] Avg episode reward: [(0, '53.191')] [2024-09-30 02:51:25,573][1157819] Updated weights for policy 0, policy_version 155128 (0.0006) [2024-09-30 02:51:26,068][1157819] Updated weights for policy 0, policy_version 155138 (0.0006) [2024-09-30 02:51:26,498][1157819] Updated weights for policy 0, policy_version 155148 (0.0006) [2024-09-30 02:51:26,958][1157819] Updated weights for policy 0, policy_version 155158 (0.0006) [2024-09-30 02:51:27,448][1157819] Updated weights for policy 0, policy_version 155168 (0.0006) [2024-09-30 02:51:27,954][1157819] Updated weights for policy 0, policy_version 155178 (0.0006) [2024-09-30 02:51:28,422][1157819] Updated weights for policy 0, policy_version 155188 (0.0006) [2024-09-30 02:51:28,908][1157819] Updated weights for policy 0, policy_version 155198 (0.0006) [2024-09-30 02:51:29,431][1157819] Updated weights for policy 0, policy_version 155208 (0.0006) [2024-09-30 02:51:29,898][1157819] Updated weights for policy 0, policy_version 155218 (0.0006) [2024-09-30 02:51:30,386][1157819] Updated weights for policy 0, policy_version 155228 (0.0006) [2024-09-30 02:51:30,466][1157520] Fps is (10 sec: 86834.9, 60 sec: 80213.2, 300 sec: 79559.6). Total num frames: 635817984. Throughput: 0: 20870.8. Samples: 148920052. Policy #0 lag: (min: 0.0, avg: 1.7, max: 6.0) [2024-09-30 02:51:30,466][1157520] Avg episode reward: [(0, '55.580')] [2024-09-30 02:51:30,887][1157819] Updated weights for policy 0, policy_version 155238 (0.0006) [2024-09-30 02:51:31,384][1157819] Updated weights for policy 0, policy_version 155248 (0.0006) [2024-09-30 02:51:31,903][1157819] Updated weights for policy 0, policy_version 155258 (0.0006) [2024-09-30 02:51:32,468][1157819] Updated weights for policy 0, policy_version 155268 (0.0006) [2024-09-30 02:51:32,981][1157819] Updated weights for policy 0, policy_version 155278 (0.0006) [2024-09-30 02:51:33,527][1157819] Updated weights for policy 0, policy_version 155288 (0.0006) [2024-09-30 02:51:34,072][1157819] Updated weights for policy 0, policy_version 155298 (0.0006) [2024-09-30 02:51:34,624][1157819] Updated weights for policy 0, policy_version 155308 (0.0006) [2024-09-30 02:51:35,166][1157819] Updated weights for policy 0, policy_version 155318 (0.0005) [2024-09-30 02:51:35,466][1157520] Fps is (10 sec: 81100.4, 60 sec: 80759.4, 300 sec: 79559.6). Total num frames: 636203008. Throughput: 0: 20886.7. Samples: 149038448. Policy #0 lag: (min: 0.0, avg: 1.7, max: 6.0) [2024-09-30 02:51:35,466][1157520] Avg episode reward: [(0, '55.962')] [2024-09-30 02:51:35,718][1157819] Updated weights for policy 0, policy_version 155328 (0.0006) [2024-09-30 02:51:36,257][1157819] Updated weights for policy 0, policy_version 155338 (0.0006) [2024-09-30 02:51:36,792][1157819] Updated weights for policy 0, policy_version 155348 (0.0006) [2024-09-30 02:51:37,358][1157819] Updated weights for policy 0, policy_version 155358 (0.0006) [2024-09-30 02:51:37,838][1157819] Updated weights for policy 0, policy_version 155368 (0.0006) [2024-09-30 02:51:38,325][1157819] Updated weights for policy 0, policy_version 155378 (0.0006) [2024-09-30 02:51:38,827][1157819] Updated weights for policy 0, policy_version 155388 (0.0006) [2024-09-30 02:51:39,336][1157819] Updated weights for policy 0, policy_version 155398 (0.0006) [2024-09-30 02:51:39,856][1157819] Updated weights for policy 0, policy_version 155408 (0.0006) [2024-09-30 02:51:40,345][1157819] Updated weights for policy 0, policy_version 155418 (0.0006) [2024-09-30 02:51:40,466][1157520] Fps is (10 sec: 78234.3, 60 sec: 81783.5, 300 sec: 79642.9). Total num frames: 636600320. Throughput: 0: 20822.2. Samples: 149095788. Policy #0 lag: (min: 0.0, avg: 1.7, max: 6.0) [2024-09-30 02:51:40,466][1157520] Avg episode reward: [(0, '54.954')] [2024-09-30 02:51:40,815][1157819] Updated weights for policy 0, policy_version 155428 (0.0006) [2024-09-30 02:51:41,298][1157819] Updated weights for policy 0, policy_version 155438 (0.0006) [2024-09-30 02:51:41,780][1157819] Updated weights for policy 0, policy_version 155448 (0.0006) [2024-09-30 02:51:42,261][1157819] Updated weights for policy 0, policy_version 155458 (0.0006) [2024-09-30 02:51:42,727][1157819] Updated weights for policy 0, policy_version 155468 (0.0006) [2024-09-30 02:51:43,216][1157819] Updated weights for policy 0, policy_version 155478 (0.0006) [2024-09-30 02:51:43,703][1157819] Updated weights for policy 0, policy_version 155488 (0.0006) [2024-09-30 02:51:44,185][1157819] Updated weights for policy 0, policy_version 155498 (0.0006) [2024-09-30 02:51:44,669][1157819] Updated weights for policy 0, policy_version 155508 (0.0006) [2024-09-30 02:51:45,141][1157819] Updated weights for policy 0, policy_version 155518 (0.0006) [2024-09-30 02:51:45,466][1157520] Fps is (10 sec: 82329.5, 60 sec: 83012.2, 300 sec: 79823.4). Total num frames: 637026304. Throughput: 0: 20930.4. Samples: 149221748. Policy #0 lag: (min: 0.0, avg: 1.7, max: 6.0) [2024-09-30 02:51:45,466][1157520] Avg episode reward: [(0, '56.249')] [2024-09-30 02:51:45,622][1157819] Updated weights for policy 0, policy_version 155528 (0.0006) [2024-09-30 02:51:46,105][1157819] Updated weights for policy 0, policy_version 155538 (0.0006) [2024-09-30 02:51:46,595][1157819] Updated weights for policy 0, policy_version 155548 (0.0006) [2024-09-30 02:51:47,075][1157819] Updated weights for policy 0, policy_version 155558 (0.0006) [2024-09-30 02:51:47,558][1157819] Updated weights for policy 0, policy_version 155568 (0.0006) [2024-09-30 02:51:48,054][1157819] Updated weights for policy 0, policy_version 155578 (0.0006) [2024-09-30 02:51:48,543][1157819] Updated weights for policy 0, policy_version 155588 (0.0006) [2024-09-30 02:51:49,032][1157819] Updated weights for policy 0, policy_version 155598 (0.0006) [2024-09-30 02:51:49,489][1157819] Updated weights for policy 0, policy_version 155608 (0.0006) [2024-09-30 02:51:49,964][1157819] Updated weights for policy 0, policy_version 155618 (0.0006) [2024-09-30 02:51:50,465][1157819] Updated weights for policy 0, policy_version 155628 (0.0006) [2024-09-30 02:51:50,466][1157520] Fps is (10 sec: 85196.3, 60 sec: 83421.9, 300 sec: 79920.6). Total num frames: 637452288. Throughput: 0: 20935.0. Samples: 149348792. Policy #0 lag: (min: 0.0, avg: 1.7, max: 6.0) [2024-09-30 02:51:50,466][1157520] Avg episode reward: [(0, '56.912')] [2024-09-30 02:51:50,933][1157819] Updated weights for policy 0, policy_version 155638 (0.0006) [2024-09-30 02:51:51,405][1157819] Updated weights for policy 0, policy_version 155648 (0.0006) [2024-09-30 02:51:51,900][1157819] Updated weights for policy 0, policy_version 155658 (0.0006) [2024-09-30 02:51:52,384][1157819] Updated weights for policy 0, policy_version 155668 (0.0006) [2024-09-30 02:51:52,869][1157819] Updated weights for policy 0, policy_version 155678 (0.0006) [2024-09-30 02:51:53,363][1157819] Updated weights for policy 0, policy_version 155688 (0.0006) [2024-09-30 02:51:53,827][1157819] Updated weights for policy 0, policy_version 155698 (0.0006) [2024-09-30 02:51:54,304][1157819] Updated weights for policy 0, policy_version 155708 (0.0006) [2024-09-30 02:51:54,799][1157819] Updated weights for policy 0, policy_version 155718 (0.0006) [2024-09-30 02:51:55,292][1157819] Updated weights for policy 0, policy_version 155728 (0.0006) [2024-09-30 02:51:55,466][1157520] Fps is (10 sec: 85197.3, 60 sec: 83831.4, 300 sec: 80003.9). Total num frames: 637878272. Throughput: 0: 20954.4. Samples: 149412628. Policy #0 lag: (min: 0.0, avg: 1.7, max: 6.0) [2024-09-30 02:51:55,466][1157520] Avg episode reward: [(0, '55.579')] [2024-09-30 02:51:55,782][1157819] Updated weights for policy 0, policy_version 155738 (0.0006) [2024-09-30 02:51:56,299][1157819] Updated weights for policy 0, policy_version 155748 (0.0006) [2024-09-30 02:51:56,852][1157819] Updated weights for policy 0, policy_version 155758 (0.0006) [2024-09-30 02:51:57,341][1157819] Updated weights for policy 0, policy_version 155768 (0.0006) [2024-09-30 02:51:57,876][1157819] Updated weights for policy 0, policy_version 155778 (0.0006) [2024-09-30 02:51:58,405][1157819] Updated weights for policy 0, policy_version 155788 (0.0006) [2024-09-30 02:51:58,922][1157819] Updated weights for policy 0, policy_version 155798 (0.0006) [2024-09-30 02:51:59,456][1157819] Updated weights for policy 0, policy_version 155808 (0.0006) [2024-09-30 02:51:59,977][1157819] Updated weights for policy 0, policy_version 155818 (0.0006) [2024-09-30 02:52:00,466][1157520] Fps is (10 sec: 81510.8, 60 sec: 83490.2, 300 sec: 79990.0). Total num frames: 638267392. Throughput: 0: 20858.8. Samples: 149534544. Policy #0 lag: (min: 0.0, avg: 1.7, max: 6.0) [2024-09-30 02:52:00,466][1157520] Avg episode reward: [(0, '56.511')] [2024-09-30 02:52:00,492][1157819] Updated weights for policy 0, policy_version 155828 (0.0006) [2024-09-30 02:52:01,007][1157819] Updated weights for policy 0, policy_version 155838 (0.0006) [2024-09-30 02:52:01,547][1157819] Updated weights for policy 0, policy_version 155848 (0.0006) [2024-09-30 02:52:02,083][1157819] Updated weights for policy 0, policy_version 155858 (0.0006) [2024-09-30 02:52:02,596][1157819] Updated weights for policy 0, policy_version 155868 (0.0006) [2024-09-30 02:52:03,038][1157736] Signal inference workers to stop experience collection... (10650 times) [2024-09-30 02:52:03,039][1157736] Signal inference workers to resume experience collection... (10650 times) [2024-09-30 02:52:03,043][1157819] InferenceWorker_p0-w0: stopping experience collection (10650 times) [2024-09-30 02:52:03,043][1157819] InferenceWorker_p0-w0: resuming experience collection (10650 times) [2024-09-30 02:52:03,115][1157819] Updated weights for policy 0, policy_version 155878 (0.0006) [2024-09-30 02:52:03,652][1157819] Updated weights for policy 0, policy_version 155888 (0.0006) [2024-09-30 02:52:04,160][1157819] Updated weights for policy 0, policy_version 155898 (0.0006) [2024-09-30 02:52:04,670][1157819] Updated weights for policy 0, policy_version 155908 (0.0006) [2024-09-30 02:52:05,171][1157819] Updated weights for policy 0, policy_version 155918 (0.0006) [2024-09-30 02:52:05,466][1157520] Fps is (10 sec: 78233.6, 60 sec: 82944.0, 300 sec: 79976.1). Total num frames: 638660608. Throughput: 0: 20644.4. Samples: 149652320. Policy #0 lag: (min: 0.0, avg: 1.7, max: 6.0) [2024-09-30 02:52:05,466][1157520] Avg episode reward: [(0, '55.332')] [2024-09-30 02:52:05,482][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000155924_638664704.pth... [2024-09-30 02:52:05,529][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000151227_619425792.pth [2024-09-30 02:52:05,723][1157819] Updated weights for policy 0, policy_version 155928 (0.0006) [2024-09-30 02:52:06,223][1157819] Updated weights for policy 0, policy_version 155938 (0.0006) [2024-09-30 02:52:06,718][1157819] Updated weights for policy 0, policy_version 155948 (0.0006) [2024-09-30 02:52:07,258][1157819] Updated weights for policy 0, policy_version 155958 (0.0006) [2024-09-30 02:52:07,800][1157819] Updated weights for policy 0, policy_version 155968 (0.0006) [2024-09-30 02:52:08,339][1157819] Updated weights for policy 0, policy_version 155978 (0.0006) [2024-09-30 02:52:08,826][1157819] Updated weights for policy 0, policy_version 155988 (0.0006) [2024-09-30 02:52:09,335][1157819] Updated weights for policy 0, policy_version 155998 (0.0006) [2024-09-30 02:52:09,902][1157819] Updated weights for policy 0, policy_version 156008 (0.0006) [2024-09-30 02:52:10,412][1157819] Updated weights for policy 0, policy_version 156018 (0.0006) [2024-09-30 02:52:10,466][1157520] Fps is (10 sec: 78642.9, 60 sec: 82534.4, 300 sec: 79962.2). Total num frames: 639053824. Throughput: 0: 20474.7. Samples: 149711100. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 02:52:10,466][1157520] Avg episode reward: [(0, '55.050')] [2024-09-30 02:52:10,938][1157819] Updated weights for policy 0, policy_version 156028 (0.0006) [2024-09-30 02:52:11,442][1157819] Updated weights for policy 0, policy_version 156038 (0.0005) [2024-09-30 02:52:11,957][1157819] Updated weights for policy 0, policy_version 156048 (0.0006) [2024-09-30 02:52:12,490][1157819] Updated weights for policy 0, policy_version 156058 (0.0006) [2024-09-30 02:52:12,999][1157819] Updated weights for policy 0, policy_version 156068 (0.0006) [2024-09-30 02:52:13,495][1157819] Updated weights for policy 0, policy_version 156078 (0.0006) [2024-09-30 02:52:13,982][1157819] Updated weights for policy 0, policy_version 156088 (0.0006) [2024-09-30 02:52:14,464][1157819] Updated weights for policy 0, policy_version 156098 (0.0006) [2024-09-30 02:52:14,951][1157819] Updated weights for policy 0, policy_version 156108 (0.0006) [2024-09-30 02:52:15,442][1157819] Updated weights for policy 0, policy_version 156118 (0.0006) [2024-09-30 02:52:15,466][1157520] Fps is (10 sec: 79871.8, 60 sec: 82261.3, 300 sec: 79990.0). Total num frames: 639459328. Throughput: 0: 20226.0. Samples: 149830220. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 02:52:15,466][1157520] Avg episode reward: [(0, '54.201')] [2024-09-30 02:52:15,920][1157819] Updated weights for policy 0, policy_version 156128 (0.0006) [2024-09-30 02:52:16,428][1157819] Updated weights for policy 0, policy_version 156138 (0.0006) [2024-09-30 02:52:16,898][1157819] Updated weights for policy 0, policy_version 156148 (0.0006) [2024-09-30 02:52:17,395][1157819] Updated weights for policy 0, policy_version 156158 (0.0006) [2024-09-30 02:52:17,892][1157819] Updated weights for policy 0, policy_version 156168 (0.0006) [2024-09-30 02:52:18,393][1157819] Updated weights for policy 0, policy_version 156178 (0.0006) [2024-09-30 02:52:18,883][1157819] Updated weights for policy 0, policy_version 156188 (0.0006) [2024-09-30 02:52:19,387][1157819] Updated weights for policy 0, policy_version 156198 (0.0005) [2024-09-30 02:52:19,889][1157819] Updated weights for policy 0, policy_version 156208 (0.0006) [2024-09-30 02:52:20,423][1157819] Updated weights for policy 0, policy_version 156218 (0.0006) [2024-09-30 02:52:20,466][1157520] Fps is (10 sec: 81920.4, 60 sec: 82056.6, 300 sec: 80059.5). Total num frames: 639873024. Throughput: 0: 20367.5. Samples: 149954984. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 02:52:20,466][1157520] Avg episode reward: [(0, '55.690')] [2024-09-30 02:52:20,925][1157819] Updated weights for policy 0, policy_version 156228 (0.0006) [2024-09-30 02:52:21,422][1157819] Updated weights for policy 0, policy_version 156238 (0.0006) [2024-09-30 02:52:21,966][1157819] Updated weights for policy 0, policy_version 156248 (0.0006) [2024-09-30 02:52:22,483][1157819] Updated weights for policy 0, policy_version 156258 (0.0006) [2024-09-30 02:52:22,972][1157819] Updated weights for policy 0, policy_version 156268 (0.0006) [2024-09-30 02:52:23,489][1157819] Updated weights for policy 0, policy_version 156278 (0.0006) [2024-09-30 02:52:24,005][1157819] Updated weights for policy 0, policy_version 156288 (0.0006) [2024-09-30 02:52:24,497][1157819] Updated weights for policy 0, policy_version 156298 (0.0006) [2024-09-30 02:52:24,995][1157819] Updated weights for policy 0, policy_version 156308 (0.0006) [2024-09-30 02:52:25,466][1157520] Fps is (10 sec: 81100.5, 60 sec: 81305.5, 300 sec: 80142.7). Total num frames: 640270336. Throughput: 0: 20422.3. Samples: 150014792. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 02:52:25,466][1157520] Avg episode reward: [(0, '57.508')] [2024-09-30 02:52:25,520][1157819] Updated weights for policy 0, policy_version 156318 (0.0006) [2024-09-30 02:52:26,046][1157819] Updated weights for policy 0, policy_version 156328 (0.0006) [2024-09-30 02:52:26,556][1157819] Updated weights for policy 0, policy_version 156338 (0.0006) [2024-09-30 02:52:27,072][1157819] Updated weights for policy 0, policy_version 156348 (0.0006) [2024-09-30 02:52:27,571][1157819] Updated weights for policy 0, policy_version 156358 (0.0006) [2024-09-30 02:52:28,078][1157819] Updated weights for policy 0, policy_version 156368 (0.0006) [2024-09-30 02:52:28,585][1157819] Updated weights for policy 0, policy_version 156378 (0.0006) [2024-09-30 02:52:29,074][1157819] Updated weights for policy 0, policy_version 156388 (0.0006) [2024-09-30 02:52:29,579][1157819] Updated weights for policy 0, policy_version 156398 (0.0006) [2024-09-30 02:52:30,081][1157819] Updated weights for policy 0, policy_version 156408 (0.0006) [2024-09-30 02:52:30,466][1157520] Fps is (10 sec: 80281.5, 60 sec: 80964.4, 300 sec: 80170.5). Total num frames: 640675840. Throughput: 0: 20303.2. Samples: 150135392. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 02:52:30,466][1157520] Avg episode reward: [(0, '56.041')] [2024-09-30 02:52:30,590][1157819] Updated weights for policy 0, policy_version 156418 (0.0006) [2024-09-30 02:52:31,101][1157819] Updated weights for policy 0, policy_version 156428 (0.0006) [2024-09-30 02:52:31,604][1157819] Updated weights for policy 0, policy_version 156438 (0.0006) [2024-09-30 02:52:32,107][1157819] Updated weights for policy 0, policy_version 156448 (0.0006) [2024-09-30 02:52:32,628][1157819] Updated weights for policy 0, policy_version 156458 (0.0006) [2024-09-30 02:52:33,099][1157819] Updated weights for policy 0, policy_version 156468 (0.0006) [2024-09-30 02:52:33,586][1157819] Updated weights for policy 0, policy_version 156478 (0.0006) [2024-09-30 02:52:34,109][1157819] Updated weights for policy 0, policy_version 156488 (0.0006) [2024-09-30 02:52:34,619][1157819] Updated weights for policy 0, policy_version 156498 (0.0006) [2024-09-30 02:52:35,113][1157819] Updated weights for policy 0, policy_version 156508 (0.0006) [2024-09-30 02:52:35,466][1157520] Fps is (10 sec: 81102.3, 60 sec: 81305.8, 300 sec: 80212.2). Total num frames: 641081344. Throughput: 0: 20189.9. Samples: 150257336. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 02:52:35,466][1157520] Avg episode reward: [(0, '56.750')] [2024-09-30 02:52:35,625][1157819] Updated weights for policy 0, policy_version 156518 (0.0006) [2024-09-30 02:52:36,152][1157819] Updated weights for policy 0, policy_version 156528 (0.0006) [2024-09-30 02:52:36,644][1157819] Updated weights for policy 0, policy_version 156538 (0.0006) [2024-09-30 02:52:37,139][1157819] Updated weights for policy 0, policy_version 156548 (0.0006) [2024-09-30 02:52:37,659][1157819] Updated weights for policy 0, policy_version 156558 (0.0006) [2024-09-30 02:52:38,167][1157819] Updated weights for policy 0, policy_version 156568 (0.0006) [2024-09-30 02:52:38,646][1157819] Updated weights for policy 0, policy_version 156578 (0.0006) [2024-09-30 02:52:39,138][1157819] Updated weights for policy 0, policy_version 156588 (0.0006) [2024-09-30 02:52:39,656][1157819] Updated weights for policy 0, policy_version 156598 (0.0006) [2024-09-30 02:52:40,190][1157819] Updated weights for policy 0, policy_version 156608 (0.0006) [2024-09-30 02:52:40,466][1157520] Fps is (10 sec: 81101.1, 60 sec: 81442.2, 300 sec: 80309.4). Total num frames: 641486848. Throughput: 0: 20121.9. Samples: 150318112. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 02:52:40,466][1157520] Avg episode reward: [(0, '54.967')] [2024-09-30 02:52:40,716][1157819] Updated weights for policy 0, policy_version 156618 (0.0006) [2024-09-30 02:52:41,216][1157819] Updated weights for policy 0, policy_version 156628 (0.0006) [2024-09-30 02:52:41,538][1157736] Signal inference workers to stop experience collection... (10700 times) [2024-09-30 02:52:41,539][1157736] Signal inference workers to resume experience collection... (10700 times) [2024-09-30 02:52:41,543][1157819] InferenceWorker_p0-w0: stopping experience collection (10700 times) [2024-09-30 02:52:41,545][1157819] InferenceWorker_p0-w0: resuming experience collection (10700 times) [2024-09-30 02:52:41,707][1157819] Updated weights for policy 0, policy_version 156638 (0.0006) [2024-09-30 02:52:42,211][1157819] Updated weights for policy 0, policy_version 156648 (0.0006) [2024-09-30 02:52:42,731][1157819] Updated weights for policy 0, policy_version 156658 (0.0006) [2024-09-30 02:52:43,250][1157819] Updated weights for policy 0, policy_version 156668 (0.0006) [2024-09-30 02:52:43,738][1157819] Updated weights for policy 0, policy_version 156678 (0.0006) [2024-09-30 02:52:44,282][1157819] Updated weights for policy 0, policy_version 156688 (0.0006) [2024-09-30 02:52:44,790][1157819] Updated weights for policy 0, policy_version 156698 (0.0006) [2024-09-30 02:52:45,296][1157819] Updated weights for policy 0, policy_version 156708 (0.0006) [2024-09-30 02:52:45,466][1157520] Fps is (10 sec: 80690.7, 60 sec: 81032.7, 300 sec: 80239.9). Total num frames: 641888256. Throughput: 0: 20094.9. Samples: 150438816. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 02:52:45,466][1157520] Avg episode reward: [(0, '55.987')] [2024-09-30 02:52:45,814][1157819] Updated weights for policy 0, policy_version 156718 (0.0006) [2024-09-30 02:52:46,328][1157819] Updated weights for policy 0, policy_version 156728 (0.0006) [2024-09-30 02:52:46,815][1157819] Updated weights for policy 0, policy_version 156738 (0.0006) [2024-09-30 02:52:47,323][1157819] Updated weights for policy 0, policy_version 156748 (0.0006) [2024-09-30 02:52:47,834][1157819] Updated weights for policy 0, policy_version 156758 (0.0006) [2024-09-30 02:52:48,324][1157819] Updated weights for policy 0, policy_version 156768 (0.0006) [2024-09-30 02:52:48,848][1157819] Updated weights for policy 0, policy_version 156778 (0.0006) [2024-09-30 02:52:49,348][1157819] Updated weights for policy 0, policy_version 156788 (0.0006) [2024-09-30 02:52:49,870][1157819] Updated weights for policy 0, policy_version 156798 (0.0006) [2024-09-30 02:52:50,358][1157819] Updated weights for policy 0, policy_version 156808 (0.0006) [2024-09-30 02:52:50,466][1157520] Fps is (10 sec: 80281.2, 60 sec: 80623.0, 300 sec: 80184.4). Total num frames: 642289664. Throughput: 0: 20170.2. Samples: 150559980. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 02:52:50,466][1157520] Avg episode reward: [(0, '54.240')] [2024-09-30 02:52:50,856][1157819] Updated weights for policy 0, policy_version 156818 (0.0006) [2024-09-30 02:52:51,366][1157819] Updated weights for policy 0, policy_version 156828 (0.0006) [2024-09-30 02:52:51,868][1157819] Updated weights for policy 0, policy_version 156838 (0.0006) [2024-09-30 02:52:52,385][1157819] Updated weights for policy 0, policy_version 156848 (0.0006) [2024-09-30 02:52:52,884][1157819] Updated weights for policy 0, policy_version 156858 (0.0006) [2024-09-30 02:52:53,373][1157819] Updated weights for policy 0, policy_version 156868 (0.0006) [2024-09-30 02:52:53,868][1157819] Updated weights for policy 0, policy_version 156878 (0.0006) [2024-09-30 02:52:54,396][1157819] Updated weights for policy 0, policy_version 156888 (0.0006) [2024-09-30 02:52:54,894][1157819] Updated weights for policy 0, policy_version 156898 (0.0006) [2024-09-30 02:52:55,394][1157819] Updated weights for policy 0, policy_version 156908 (0.0006) [2024-09-30 02:52:55,466][1157520] Fps is (10 sec: 81100.6, 60 sec: 80349.9, 300 sec: 80281.6). Total num frames: 642699264. Throughput: 0: 20215.2. Samples: 150620784. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 02:52:55,466][1157520] Avg episode reward: [(0, '54.549')] [2024-09-30 02:52:55,919][1157819] Updated weights for policy 0, policy_version 156918 (0.0006) [2024-09-30 02:52:56,439][1157819] Updated weights for policy 0, policy_version 156928 (0.0006) [2024-09-30 02:52:56,963][1157819] Updated weights for policy 0, policy_version 156938 (0.0006) [2024-09-30 02:52:57,486][1157819] Updated weights for policy 0, policy_version 156948 (0.0006) [2024-09-30 02:52:57,989][1157819] Updated weights for policy 0, policy_version 156958 (0.0006) [2024-09-30 02:52:58,516][1157819] Updated weights for policy 0, policy_version 156968 (0.0006) [2024-09-30 02:52:59,029][1157819] Updated weights for policy 0, policy_version 156978 (0.0006) [2024-09-30 02:52:59,545][1157819] Updated weights for policy 0, policy_version 156988 (0.0006) [2024-09-30 02:53:00,079][1157819] Updated weights for policy 0, policy_version 156998 (0.0006) [2024-09-30 02:53:00,466][1157520] Fps is (10 sec: 80281.6, 60 sec: 80418.1, 300 sec: 80226.1). Total num frames: 643092480. Throughput: 0: 20232.6. Samples: 150740684. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 02:53:00,466][1157520] Avg episode reward: [(0, '56.373')] [2024-09-30 02:53:00,576][1157819] Updated weights for policy 0, policy_version 157008 (0.0006) [2024-09-30 02:53:01,134][1157819] Updated weights for policy 0, policy_version 157018 (0.0006) [2024-09-30 02:53:01,646][1157819] Updated weights for policy 0, policy_version 157028 (0.0006) [2024-09-30 02:53:02,145][1157819] Updated weights for policy 0, policy_version 157038 (0.0006) [2024-09-30 02:53:02,690][1157819] Updated weights for policy 0, policy_version 157048 (0.0006) [2024-09-30 02:53:03,211][1157819] Updated weights for policy 0, policy_version 157058 (0.0006) [2024-09-30 02:53:03,748][1157819] Updated weights for policy 0, policy_version 157068 (0.0006) [2024-09-30 02:53:04,238][1157819] Updated weights for policy 0, policy_version 157078 (0.0006) [2024-09-30 02:53:04,755][1157819] Updated weights for policy 0, policy_version 157088 (0.0006) [2024-09-30 02:53:05,277][1157819] Updated weights for policy 0, policy_version 157098 (0.0006) [2024-09-30 02:53:05,466][1157520] Fps is (10 sec: 78643.0, 60 sec: 80418.2, 300 sec: 80240.0). Total num frames: 643485696. Throughput: 0: 20076.6. Samples: 150858432. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 02:53:05,466][1157520] Avg episode reward: [(0, '56.126')] [2024-09-30 02:53:05,778][1157819] Updated weights for policy 0, policy_version 157108 (0.0006) [2024-09-30 02:53:06,307][1157819] Updated weights for policy 0, policy_version 157118 (0.0006) [2024-09-30 02:53:06,831][1157819] Updated weights for policy 0, policy_version 157128 (0.0006) [2024-09-30 02:53:07,344][1157819] Updated weights for policy 0, policy_version 157138 (0.0006) [2024-09-30 02:53:07,847][1157819] Updated weights for policy 0, policy_version 157148 (0.0006) [2024-09-30 02:53:08,352][1157819] Updated weights for policy 0, policy_version 157158 (0.0006) [2024-09-30 02:53:08,892][1157819] Updated weights for policy 0, policy_version 157168 (0.0006) [2024-09-30 02:53:09,468][1157819] Updated weights for policy 0, policy_version 157178 (0.0006) [2024-09-30 02:53:10,013][1157819] Updated weights for policy 0, policy_version 157188 (0.0006) [2024-09-30 02:53:10,466][1157520] Fps is (10 sec: 78233.5, 60 sec: 80349.9, 300 sec: 80198.3). Total num frames: 643874816. Throughput: 0: 20076.2. Samples: 150918220. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:53:10,466][1157520] Avg episode reward: [(0, '54.984')] [2024-09-30 02:53:10,543][1157819] Updated weights for policy 0, policy_version 157198 (0.0006) [2024-09-30 02:53:11,113][1157819] Updated weights for policy 0, policy_version 157208 (0.0006) [2024-09-30 02:53:11,644][1157819] Updated weights for policy 0, policy_version 157218 (0.0006) [2024-09-30 02:53:12,177][1157819] Updated weights for policy 0, policy_version 157228 (0.0006) [2024-09-30 02:53:12,717][1157819] Updated weights for policy 0, policy_version 157238 (0.0006) [2024-09-30 02:53:13,257][1157819] Updated weights for policy 0, policy_version 157248 (0.0006) [2024-09-30 02:53:13,805][1157819] Updated weights for policy 0, policy_version 157258 (0.0006) [2024-09-30 02:53:14,341][1157819] Updated weights for policy 0, policy_version 157268 (0.0006) [2024-09-30 02:53:14,918][1157819] Updated weights for policy 0, policy_version 157278 (0.0006) [2024-09-30 02:53:15,427][1157819] Updated weights for policy 0, policy_version 157288 (0.0006) [2024-09-30 02:53:15,466][1157520] Fps is (10 sec: 76595.0, 60 sec: 79872.0, 300 sec: 80059.4). Total num frames: 644251648. Throughput: 0: 19919.8. Samples: 151031784. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:53:15,466][1157520] Avg episode reward: [(0, '52.777')] [2024-09-30 02:53:15,967][1157819] Updated weights for policy 0, policy_version 157298 (0.0006) [2024-09-30 02:53:16,534][1157819] Updated weights for policy 0, policy_version 157308 (0.0006) [2024-09-30 02:53:17,089][1157819] Updated weights for policy 0, policy_version 157318 (0.0006) [2024-09-30 02:53:17,655][1157819] Updated weights for policy 0, policy_version 157328 (0.0006) [2024-09-30 02:53:18,206][1157819] Updated weights for policy 0, policy_version 157338 (0.0006) [2024-09-30 02:53:18,711][1157819] Updated weights for policy 0, policy_version 157348 (0.0006) [2024-09-30 02:53:19,270][1157819] Updated weights for policy 0, policy_version 157358 (0.0006) [2024-09-30 02:53:19,827][1157819] Updated weights for policy 0, policy_version 157368 (0.0006) [2024-09-30 02:53:20,341][1157819] Updated weights for policy 0, policy_version 157378 (0.0006) [2024-09-30 02:53:20,466][1157520] Fps is (10 sec: 74956.9, 60 sec: 79189.3, 300 sec: 79934.5). Total num frames: 644624384. Throughput: 0: 19705.0. Samples: 151144064. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:53:20,466][1157520] Avg episode reward: [(0, '52.315')] [2024-09-30 02:53:20,918][1157819] Updated weights for policy 0, policy_version 157388 (0.0006) [2024-09-30 02:53:21,454][1157819] Updated weights for policy 0, policy_version 157398 (0.0006) [2024-09-30 02:53:21,939][1157819] Updated weights for policy 0, policy_version 157408 (0.0006) [2024-09-30 02:53:22,408][1157819] Updated weights for policy 0, policy_version 157418 (0.0006) [2024-09-30 02:53:22,924][1157819] Updated weights for policy 0, policy_version 157428 (0.0006) [2024-09-30 02:53:22,979][1157736] Signal inference workers to stop experience collection... (10750 times) [2024-09-30 02:53:22,980][1157736] Signal inference workers to resume experience collection... (10750 times) [2024-09-30 02:53:22,985][1157819] InferenceWorker_p0-w0: stopping experience collection (10750 times) [2024-09-30 02:53:22,985][1157819] InferenceWorker_p0-w0: resuming experience collection (10750 times) [2024-09-30 02:53:23,408][1157819] Updated weights for policy 0, policy_version 157438 (0.0006) [2024-09-30 02:53:23,915][1157819] Updated weights for policy 0, policy_version 157448 (0.0006) [2024-09-30 02:53:24,426][1157819] Updated weights for policy 0, policy_version 157458 (0.0006) [2024-09-30 02:53:24,942][1157819] Updated weights for policy 0, policy_version 157468 (0.0007) [2024-09-30 02:53:25,425][1157819] Updated weights for policy 0, policy_version 157478 (0.0006) [2024-09-30 02:53:25,466][1157520] Fps is (10 sec: 77824.0, 60 sec: 79325.9, 300 sec: 79878.9). Total num frames: 645029888. Throughput: 0: 19679.3. Samples: 151203684. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:53:25,466][1157520] Avg episode reward: [(0, '52.026')] [2024-09-30 02:53:25,952][1157819] Updated weights for policy 0, policy_version 157488 (0.0006) [2024-09-30 02:53:26,458][1157819] Updated weights for policy 0, policy_version 157498 (0.0006) [2024-09-30 02:53:26,976][1157819] Updated weights for policy 0, policy_version 157508 (0.0006) [2024-09-30 02:53:27,503][1157819] Updated weights for policy 0, policy_version 157518 (0.0006) [2024-09-30 02:53:28,029][1157819] Updated weights for policy 0, policy_version 157528 (0.0006) [2024-09-30 02:53:28,563][1157819] Updated weights for policy 0, policy_version 157538 (0.0006) [2024-09-30 02:53:29,090][1157819] Updated weights for policy 0, policy_version 157548 (0.0006) [2024-09-30 02:53:29,595][1157819] Updated weights for policy 0, policy_version 157558 (0.0006) [2024-09-30 02:53:30,185][1157819] Updated weights for policy 0, policy_version 157568 (0.0006) [2024-09-30 02:53:30,466][1157520] Fps is (10 sec: 79052.7, 60 sec: 78984.5, 300 sec: 79795.6). Total num frames: 645414912. Throughput: 0: 19653.1. Samples: 151323208. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:53:30,466][1157520] Avg episode reward: [(0, '52.811')] [2024-09-30 02:53:30,751][1157819] Updated weights for policy 0, policy_version 157578 (0.0006) [2024-09-30 02:53:31,340][1157819] Updated weights for policy 0, policy_version 157588 (0.0006) [2024-09-30 02:53:31,858][1157819] Updated weights for policy 0, policy_version 157598 (0.0006) [2024-09-30 02:53:32,384][1157819] Updated weights for policy 0, policy_version 157608 (0.0006) [2024-09-30 02:53:32,967][1157819] Updated weights for policy 0, policy_version 157618 (0.0006) [2024-09-30 02:53:33,537][1157819] Updated weights for policy 0, policy_version 157628 (0.0006) [2024-09-30 02:53:34,086][1157819] Updated weights for policy 0, policy_version 157638 (0.0006) [2024-09-30 02:53:34,675][1157819] Updated weights for policy 0, policy_version 157648 (0.0006) [2024-09-30 02:53:35,203][1157819] Updated weights for policy 0, policy_version 157658 (0.0006) [2024-09-30 02:53:35,466][1157520] Fps is (10 sec: 75366.6, 60 sec: 78370.0, 300 sec: 79684.5). Total num frames: 645783552. Throughput: 0: 19421.3. Samples: 151433940. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:53:35,466][1157520] Avg episode reward: [(0, '55.498')] [2024-09-30 02:53:35,747][1157819] Updated weights for policy 0, policy_version 157668 (0.0006) [2024-09-30 02:53:36,308][1157819] Updated weights for policy 0, policy_version 157678 (0.0006) [2024-09-30 02:53:36,854][1157819] Updated weights for policy 0, policy_version 157688 (0.0006) [2024-09-30 02:53:37,376][1157819] Updated weights for policy 0, policy_version 157698 (0.0006) [2024-09-30 02:53:37,903][1157819] Updated weights for policy 0, policy_version 157708 (0.0006) [2024-09-30 02:53:38,482][1157819] Updated weights for policy 0, policy_version 157718 (0.0006) [2024-09-30 02:53:39,025][1157819] Updated weights for policy 0, policy_version 157728 (0.0006) [2024-09-30 02:53:39,601][1157819] Updated weights for policy 0, policy_version 157738 (0.0006) [2024-09-30 02:53:40,125][1157819] Updated weights for policy 0, policy_version 157748 (0.0006) [2024-09-30 02:53:40,466][1157520] Fps is (10 sec: 74547.3, 60 sec: 77892.2, 300 sec: 79601.3). Total num frames: 646160384. Throughput: 0: 19319.4. Samples: 151490156. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:53:40,466][1157520] Avg episode reward: [(0, '52.373')] [2024-09-30 02:53:40,642][1157819] Updated weights for policy 0, policy_version 157758 (0.0006) [2024-09-30 02:53:41,229][1157819] Updated weights for policy 0, policy_version 157768 (0.0006) [2024-09-30 02:53:41,778][1157819] Updated weights for policy 0, policy_version 157778 (0.0006) [2024-09-30 02:53:42,288][1157819] Updated weights for policy 0, policy_version 157788 (0.0006) [2024-09-30 02:53:42,901][1157819] Updated weights for policy 0, policy_version 157798 (0.0006) [2024-09-30 02:53:43,429][1157819] Updated weights for policy 0, policy_version 157808 (0.0006) [2024-09-30 02:53:44,007][1157819] Updated weights for policy 0, policy_version 157818 (0.0006) [2024-09-30 02:53:44,568][1157819] Updated weights for policy 0, policy_version 157828 (0.0006) [2024-09-30 02:53:45,116][1157819] Updated weights for policy 0, policy_version 157838 (0.0006) [2024-09-30 02:53:45,466][1157520] Fps is (10 sec: 74547.1, 60 sec: 77346.1, 300 sec: 79462.4). Total num frames: 646529024. Throughput: 0: 19130.0. Samples: 151601536. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:53:45,466][1157520] Avg episode reward: [(0, '53.554')] [2024-09-30 02:53:45,631][1157819] Updated weights for policy 0, policy_version 157848 (0.0006) [2024-09-30 02:53:46,125][1157819] Updated weights for policy 0, policy_version 157858 (0.0006) [2024-09-30 02:53:46,628][1157819] Updated weights for policy 0, policy_version 157868 (0.0006) [2024-09-30 02:53:47,161][1157819] Updated weights for policy 0, policy_version 157878 (0.0006) [2024-09-30 02:53:47,653][1157819] Updated weights for policy 0, policy_version 157888 (0.0006) [2024-09-30 02:53:48,185][1157819] Updated weights for policy 0, policy_version 157898 (0.0006) [2024-09-30 02:53:48,689][1157819] Updated weights for policy 0, policy_version 157908 (0.0006) [2024-09-30 02:53:49,208][1157819] Updated weights for policy 0, policy_version 157918 (0.0006) [2024-09-30 02:53:49,751][1157819] Updated weights for policy 0, policy_version 157928 (0.0006) [2024-09-30 02:53:50,273][1157819] Updated weights for policy 0, policy_version 157938 (0.0006) [2024-09-30 02:53:50,466][1157520] Fps is (10 sec: 77004.9, 60 sec: 77346.2, 300 sec: 79420.8). Total num frames: 646930432. Throughput: 0: 19132.5. Samples: 151719392. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:53:50,466][1157520] Avg episode reward: [(0, '54.070')] [2024-09-30 02:53:50,765][1157819] Updated weights for policy 0, policy_version 157948 (0.0006) [2024-09-30 02:53:51,277][1157819] Updated weights for policy 0, policy_version 157958 (0.0006) [2024-09-30 02:53:51,785][1157819] Updated weights for policy 0, policy_version 157968 (0.0006) [2024-09-30 02:53:52,346][1157819] Updated weights for policy 0, policy_version 157978 (0.0006) [2024-09-30 02:53:52,848][1157819] Updated weights for policy 0, policy_version 157988 (0.0006) [2024-09-30 02:53:53,336][1157819] Updated weights for policy 0, policy_version 157998 (0.0006) [2024-09-30 02:53:53,836][1157819] Updated weights for policy 0, policy_version 158008 (0.0006) [2024-09-30 02:53:54,322][1157819] Updated weights for policy 0, policy_version 158018 (0.0006) [2024-09-30 02:53:54,824][1157819] Updated weights for policy 0, policy_version 158028 (0.0006) [2024-09-30 02:53:55,293][1157819] Updated weights for policy 0, policy_version 158038 (0.0006) [2024-09-30 02:53:55,466][1157520] Fps is (10 sec: 80691.6, 60 sec: 77277.9, 300 sec: 79393.0). Total num frames: 647335936. Throughput: 0: 19120.8. Samples: 151778656. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:53:55,466][1157520] Avg episode reward: [(0, '54.066')] [2024-09-30 02:53:55,797][1157819] Updated weights for policy 0, policy_version 158048 (0.0006) [2024-09-30 02:53:56,307][1157819] Updated weights for policy 0, policy_version 158058 (0.0006) [2024-09-30 02:53:56,805][1157819] Updated weights for policy 0, policy_version 158068 (0.0006) [2024-09-30 02:53:57,253][1157819] Updated weights for policy 0, policy_version 158078 (0.0006) [2024-09-30 02:53:57,720][1157819] Updated weights for policy 0, policy_version 158088 (0.0006) [2024-09-30 02:53:58,214][1157819] Updated weights for policy 0, policy_version 158098 (0.0006) [2024-09-30 02:53:58,717][1157819] Updated weights for policy 0, policy_version 158108 (0.0006) [2024-09-30 02:53:59,183][1157819] Updated weights for policy 0, policy_version 158118 (0.0006) [2024-09-30 02:53:59,689][1157819] Updated weights for policy 0, policy_version 158128 (0.0006) [2024-09-30 02:54:00,183][1157819] Updated weights for policy 0, policy_version 158138 (0.0006) [2024-09-30 02:54:00,466][1157520] Fps is (10 sec: 82329.5, 60 sec: 77687.5, 300 sec: 79379.1). Total num frames: 647753728. Throughput: 0: 19392.0. Samples: 151904424. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:54:00,466][1157520] Avg episode reward: [(0, '55.510')] [2024-09-30 02:54:00,663][1157819] Updated weights for policy 0, policy_version 158148 (0.0006) [2024-09-30 02:54:01,142][1157819] Updated weights for policy 0, policy_version 158158 (0.0006) [2024-09-30 02:54:01,635][1157819] Updated weights for policy 0, policy_version 158168 (0.0006) [2024-09-30 02:54:02,133][1157819] Updated weights for policy 0, policy_version 158178 (0.0006) [2024-09-30 02:54:02,624][1157819] Updated weights for policy 0, policy_version 158188 (0.0006) [2024-09-30 02:54:03,117][1157819] Updated weights for policy 0, policy_version 158198 (0.0006) [2024-09-30 02:54:03,615][1157819] Updated weights for policy 0, policy_version 158208 (0.0006) [2024-09-30 02:54:04,123][1157819] Updated weights for policy 0, policy_version 158218 (0.0006) [2024-09-30 02:54:04,605][1157819] Updated weights for policy 0, policy_version 158228 (0.0006) [2024-09-30 02:54:05,099][1157819] Updated weights for policy 0, policy_version 158238 (0.0006) [2024-09-30 02:54:05,466][1157520] Fps is (10 sec: 83557.8, 60 sec: 78097.0, 300 sec: 79462.4). Total num frames: 648171520. Throughput: 0: 19672.4. Samples: 152029324. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:54:05,466][1157520] Avg episode reward: [(0, '52.462')] [2024-09-30 02:54:05,482][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000158246_648175616.pth... [2024-09-30 02:54:05,534][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000153585_629084160.pth [2024-09-30 02:54:05,571][1157819] Updated weights for policy 0, policy_version 158248 (0.0006) [2024-09-30 02:54:06,051][1157819] Updated weights for policy 0, policy_version 158258 (0.0006) [2024-09-30 02:54:06,547][1157819] Updated weights for policy 0, policy_version 158268 (0.0006) [2024-09-30 02:54:07,054][1157819] Updated weights for policy 0, policy_version 158278 (0.0006) [2024-09-30 02:54:07,557][1157819] Updated weights for policy 0, policy_version 158288 (0.0006) [2024-09-30 02:54:08,068][1157819] Updated weights for policy 0, policy_version 158298 (0.0006) [2024-09-30 02:54:08,571][1157819] Updated weights for policy 0, policy_version 158308 (0.0006) [2024-09-30 02:54:09,067][1157736] Signal inference workers to stop experience collection... (10800 times) [2024-09-30 02:54:09,071][1157736] Signal inference workers to resume experience collection... (10800 times) [2024-09-30 02:54:09,075][1157819] InferenceWorker_p0-w0: stopping experience collection (10800 times) [2024-09-30 02:54:09,077][1157819] InferenceWorker_p0-w0: resuming experience collection (10800 times) [2024-09-30 02:54:09,085][1157819] Updated weights for policy 0, policy_version 158318 (0.0006) [2024-09-30 02:54:09,621][1157819] Updated weights for policy 0, policy_version 158328 (0.0006) [2024-09-30 02:54:10,164][1157819] Updated weights for policy 0, policy_version 158338 (0.0006) [2024-09-30 02:54:10,466][1157520] Fps is (10 sec: 81920.7, 60 sec: 78302.0, 300 sec: 79406.9). Total num frames: 648572928. Throughput: 0: 19732.5. Samples: 152091644. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:54:10,466][1157520] Avg episode reward: [(0, '55.287')] [2024-09-30 02:54:10,706][1157819] Updated weights for policy 0, policy_version 158348 (0.0006) [2024-09-30 02:54:11,205][1157819] Updated weights for policy 0, policy_version 158358 (0.0006) [2024-09-30 02:54:11,721][1157819] Updated weights for policy 0, policy_version 158368 (0.0006) [2024-09-30 02:54:12,247][1157819] Updated weights for policy 0, policy_version 158378 (0.0006) [2024-09-30 02:54:12,737][1157819] Updated weights for policy 0, policy_version 158388 (0.0006) [2024-09-30 02:54:13,211][1157819] Updated weights for policy 0, policy_version 158398 (0.0006) [2024-09-30 02:54:13,727][1157819] Updated weights for policy 0, policy_version 158408 (0.0006) [2024-09-30 02:54:14,220][1157819] Updated weights for policy 0, policy_version 158418 (0.0006) [2024-09-30 02:54:14,722][1157819] Updated weights for policy 0, policy_version 158428 (0.0006) [2024-09-30 02:54:15,209][1157819] Updated weights for policy 0, policy_version 158438 (0.0006) [2024-09-30 02:54:15,466][1157520] Fps is (10 sec: 81101.9, 60 sec: 78848.2, 300 sec: 79420.8). Total num frames: 648982528. Throughput: 0: 19731.9. Samples: 152211140. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 02:54:15,466][1157520] Avg episode reward: [(0, '55.670')] [2024-09-30 02:54:15,709][1157819] Updated weights for policy 0, policy_version 158448 (0.0006) [2024-09-30 02:54:16,207][1157819] Updated weights for policy 0, policy_version 158458 (0.0006) [2024-09-30 02:54:16,682][1157819] Updated weights for policy 0, policy_version 158468 (0.0006) [2024-09-30 02:54:17,191][1157819] Updated weights for policy 0, policy_version 158478 (0.0006) [2024-09-30 02:54:17,689][1157819] Updated weights for policy 0, policy_version 158488 (0.0006) [2024-09-30 02:54:18,194][1157819] Updated weights for policy 0, policy_version 158498 (0.0006) [2024-09-30 02:54:18,692][1157819] Updated weights for policy 0, policy_version 158508 (0.0006) [2024-09-30 02:54:19,206][1157819] Updated weights for policy 0, policy_version 158518 (0.0006) [2024-09-30 02:54:19,719][1157819] Updated weights for policy 0, policy_version 158528 (0.0006) [2024-09-30 02:54:20,235][1157819] Updated weights for policy 0, policy_version 158538 (0.0006) [2024-09-30 02:54:20,466][1157520] Fps is (10 sec: 81510.5, 60 sec: 79394.3, 300 sec: 79406.9). Total num frames: 649388032. Throughput: 0: 19999.2. Samples: 152333900. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 02:54:20,466][1157520] Avg episode reward: [(0, '54.522')] [2024-09-30 02:54:20,764][1157819] Updated weights for policy 0, policy_version 158548 (0.0006) [2024-09-30 02:54:21,278][1157819] Updated weights for policy 0, policy_version 158558 (0.0006) [2024-09-30 02:54:21,783][1157819] Updated weights for policy 0, policy_version 158568 (0.0006) [2024-09-30 02:54:22,323][1157819] Updated weights for policy 0, policy_version 158578 (0.0006) [2024-09-30 02:54:22,847][1157819] Updated weights for policy 0, policy_version 158588 (0.0006) [2024-09-30 02:54:23,366][1157819] Updated weights for policy 0, policy_version 158598 (0.0006) [2024-09-30 02:54:23,883][1157819] Updated weights for policy 0, policy_version 158608 (0.0006) [2024-09-30 02:54:24,394][1157819] Updated weights for policy 0, policy_version 158618 (0.0006) [2024-09-30 02:54:24,904][1157819] Updated weights for policy 0, policy_version 158628 (0.0006) [2024-09-30 02:54:25,436][1157819] Updated weights for policy 0, policy_version 158638 (0.0006) [2024-09-30 02:54:25,466][1157520] Fps is (10 sec: 79872.1, 60 sec: 79189.5, 300 sec: 79406.9). Total num frames: 649781248. Throughput: 0: 20065.5. Samples: 152393100. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 02:54:25,466][1157520] Avg episode reward: [(0, '55.561')] [2024-09-30 02:54:25,951][1157819] Updated weights for policy 0, policy_version 158648 (0.0006) [2024-09-30 02:54:26,475][1157819] Updated weights for policy 0, policy_version 158658 (0.0006) [2024-09-30 02:54:26,969][1157819] Updated weights for policy 0, policy_version 158668 (0.0006) [2024-09-30 02:54:27,493][1157819] Updated weights for policy 0, policy_version 158678 (0.0006) [2024-09-30 02:54:28,003][1157819] Updated weights for policy 0, policy_version 158688 (0.0006) [2024-09-30 02:54:28,550][1157819] Updated weights for policy 0, policy_version 158698 (0.0006) [2024-09-30 02:54:29,063][1157819] Updated weights for policy 0, policy_version 158708 (0.0006) [2024-09-30 02:54:29,568][1157819] Updated weights for policy 0, policy_version 158718 (0.0006) [2024-09-30 02:54:30,096][1157819] Updated weights for policy 0, policy_version 158728 (0.0006) [2024-09-30 02:54:30,466][1157520] Fps is (10 sec: 79052.9, 60 sec: 79394.3, 300 sec: 79420.8). Total num frames: 650178560. Throughput: 0: 20225.4. Samples: 152511676. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 02:54:30,466][1157520] Avg episode reward: [(0, '54.064')] [2024-09-30 02:54:30,631][1157819] Updated weights for policy 0, policy_version 158738 (0.0006) [2024-09-30 02:54:31,118][1157819] Updated weights for policy 0, policy_version 158748 (0.0006) [2024-09-30 02:54:31,675][1157819] Updated weights for policy 0, policy_version 158758 (0.0006) [2024-09-30 02:54:32,169][1157819] Updated weights for policy 0, policy_version 158768 (0.0006) [2024-09-30 02:54:32,683][1157819] Updated weights for policy 0, policy_version 158778 (0.0006) [2024-09-30 02:54:33,206][1157819] Updated weights for policy 0, policy_version 158788 (0.0006) [2024-09-30 02:54:33,729][1157819] Updated weights for policy 0, policy_version 158798 (0.0006) [2024-09-30 02:54:34,267][1157819] Updated weights for policy 0, policy_version 158808 (0.0006) [2024-09-30 02:54:34,773][1157819] Updated weights for policy 0, policy_version 158818 (0.0006) [2024-09-30 02:54:35,286][1157819] Updated weights for policy 0, policy_version 158828 (0.0006) [2024-09-30 02:54:35,466][1157520] Fps is (10 sec: 79052.7, 60 sec: 79803.9, 300 sec: 79351.4). Total num frames: 650571776. Throughput: 0: 20244.3. Samples: 152630384. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 02:54:35,466][1157520] Avg episode reward: [(0, '54.228')] [2024-09-30 02:54:35,789][1157819] Updated weights for policy 0, policy_version 158838 (0.0006) [2024-09-30 02:54:36,290][1157819] Updated weights for policy 0, policy_version 158848 (0.0006) [2024-09-30 02:54:36,795][1157819] Updated weights for policy 0, policy_version 158858 (0.0006) [2024-09-30 02:54:37,299][1157819] Updated weights for policy 0, policy_version 158868 (0.0006) [2024-09-30 02:54:37,791][1157819] Updated weights for policy 0, policy_version 158878 (0.0006) [2024-09-30 02:54:38,300][1157819] Updated weights for policy 0, policy_version 158888 (0.0006) [2024-09-30 02:54:38,773][1157819] Updated weights for policy 0, policy_version 158898 (0.0006) [2024-09-30 02:54:39,285][1157819] Updated weights for policy 0, policy_version 158908 (0.0006) [2024-09-30 02:54:39,779][1157819] Updated weights for policy 0, policy_version 158918 (0.0006) [2024-09-30 02:54:40,289][1157819] Updated weights for policy 0, policy_version 158928 (0.0006) [2024-09-30 02:54:40,466][1157520] Fps is (10 sec: 80281.0, 60 sec: 80349.9, 300 sec: 79434.7). Total num frames: 650981376. Throughput: 0: 20272.9. Samples: 152690936. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 02:54:40,466][1157520] Avg episode reward: [(0, '54.329')] [2024-09-30 02:54:40,798][1157819] Updated weights for policy 0, policy_version 158938 (0.0006) [2024-09-30 02:54:41,295][1157819] Updated weights for policy 0, policy_version 158948 (0.0006) [2024-09-30 02:54:41,839][1157819] Updated weights for policy 0, policy_version 158958 (0.0006) [2024-09-30 02:54:42,100][1157736] Signal inference workers to stop experience collection... (10850 times) [2024-09-30 02:54:42,101][1157736] Signal inference workers to resume experience collection... (10850 times) [2024-09-30 02:54:42,105][1157819] InferenceWorker_p0-w0: stopping experience collection (10850 times) [2024-09-30 02:54:42,105][1157819] InferenceWorker_p0-w0: resuming experience collection (10850 times) [2024-09-30 02:54:42,344][1157819] Updated weights for policy 0, policy_version 158968 (0.0006) [2024-09-30 02:54:42,844][1157819] Updated weights for policy 0, policy_version 158978 (0.0006) [2024-09-30 02:54:43,332][1157819] Updated weights for policy 0, policy_version 158988 (0.0006) [2024-09-30 02:54:43,819][1157819] Updated weights for policy 0, policy_version 158998 (0.0006) [2024-09-30 02:54:44,308][1157819] Updated weights for policy 0, policy_version 159008 (0.0006) [2024-09-30 02:54:44,803][1157819] Updated weights for policy 0, policy_version 159018 (0.0006) [2024-09-30 02:54:45,292][1157819] Updated weights for policy 0, policy_version 159028 (0.0006) [2024-09-30 02:54:45,466][1157520] Fps is (10 sec: 81920.1, 60 sec: 81032.7, 300 sec: 79504.1). Total num frames: 651390976. Throughput: 0: 20193.6. Samples: 152813136. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 02:54:45,466][1157520] Avg episode reward: [(0, '53.972')] [2024-09-30 02:54:45,783][1157819] Updated weights for policy 0, policy_version 159038 (0.0006) [2024-09-30 02:54:46,276][1157819] Updated weights for policy 0, policy_version 159048 (0.0006) [2024-09-30 02:54:46,762][1157819] Updated weights for policy 0, policy_version 159058 (0.0006) [2024-09-30 02:54:47,262][1157819] Updated weights for policy 0, policy_version 159068 (0.0006) [2024-09-30 02:54:47,748][1157819] Updated weights for policy 0, policy_version 159078 (0.0006) [2024-09-30 02:54:48,232][1157819] Updated weights for policy 0, policy_version 159088 (0.0006) [2024-09-30 02:54:48,735][1157819] Updated weights for policy 0, policy_version 159098 (0.0006) [2024-09-30 02:54:49,228][1157819] Updated weights for policy 0, policy_version 159108 (0.0006) [2024-09-30 02:54:49,723][1157819] Updated weights for policy 0, policy_version 159118 (0.0006) [2024-09-30 02:54:50,223][1157819] Updated weights for policy 0, policy_version 159128 (0.0006) [2024-09-30 02:54:50,466][1157520] Fps is (10 sec: 82330.1, 60 sec: 81237.5, 300 sec: 79601.3). Total num frames: 651804672. Throughput: 0: 20194.3. Samples: 152938064. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 02:54:50,466][1157520] Avg episode reward: [(0, '55.016')] [2024-09-30 02:54:50,710][1157819] Updated weights for policy 0, policy_version 159138 (0.0006) [2024-09-30 02:54:51,221][1157819] Updated weights for policy 0, policy_version 159148 (0.0006) [2024-09-30 02:54:51,699][1157819] Updated weights for policy 0, policy_version 159158 (0.0006) [2024-09-30 02:54:52,179][1157819] Updated weights for policy 0, policy_version 159168 (0.0006) [2024-09-30 02:54:52,694][1157819] Updated weights for policy 0, policy_version 159178 (0.0006) [2024-09-30 02:54:53,175][1157819] Updated weights for policy 0, policy_version 159188 (0.0006) [2024-09-30 02:54:53,662][1157819] Updated weights for policy 0, policy_version 159198 (0.0006) [2024-09-30 02:54:54,151][1157819] Updated weights for policy 0, policy_version 159208 (0.0006) [2024-09-30 02:54:54,636][1157819] Updated weights for policy 0, policy_version 159218 (0.0006) [2024-09-30 02:54:55,136][1157819] Updated weights for policy 0, policy_version 159228 (0.0006) [2024-09-30 02:54:55,466][1157520] Fps is (10 sec: 83558.4, 60 sec: 81510.5, 300 sec: 79754.0). Total num frames: 652226560. Throughput: 0: 20193.2. Samples: 153000336. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 02:54:55,466][1157520] Avg episode reward: [(0, '54.887')] [2024-09-30 02:54:55,613][1157819] Updated weights for policy 0, policy_version 159238 (0.0006) [2024-09-30 02:54:56,107][1157819] Updated weights for policy 0, policy_version 159248 (0.0006) [2024-09-30 02:54:56,587][1157819] Updated weights for policy 0, policy_version 159258 (0.0006) [2024-09-30 02:54:57,080][1157819] Updated weights for policy 0, policy_version 159268 (0.0006) [2024-09-30 02:54:57,567][1157819] Updated weights for policy 0, policy_version 159278 (0.0006) [2024-09-30 02:54:58,050][1157819] Updated weights for policy 0, policy_version 159288 (0.0006) [2024-09-30 02:54:58,541][1157819] Updated weights for policy 0, policy_version 159298 (0.0006) [2024-09-30 02:54:59,031][1157819] Updated weights for policy 0, policy_version 159308 (0.0006) [2024-09-30 02:54:59,504][1157819] Updated weights for policy 0, policy_version 159318 (0.0006) [2024-09-30 02:55:00,003][1157819] Updated weights for policy 0, policy_version 159328 (0.0006) [2024-09-30 02:55:00,466][1157520] Fps is (10 sec: 83968.1, 60 sec: 81510.6, 300 sec: 79879.0). Total num frames: 652644352. Throughput: 0: 20334.1. Samples: 153126176. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 02:55:00,466][1157520] Avg episode reward: [(0, '54.118')] [2024-09-30 02:55:00,467][1157819] Updated weights for policy 0, policy_version 159338 (0.0006) [2024-09-30 02:55:00,949][1157819] Updated weights for policy 0, policy_version 159348 (0.0006) [2024-09-30 02:55:01,413][1157819] Updated weights for policy 0, policy_version 159358 (0.0006) [2024-09-30 02:55:01,882][1157819] Updated weights for policy 0, policy_version 159368 (0.0006) [2024-09-30 02:55:02,331][1157819] Updated weights for policy 0, policy_version 159378 (0.0006) [2024-09-30 02:55:02,810][1157819] Updated weights for policy 0, policy_version 159388 (0.0006) [2024-09-30 02:55:03,292][1157819] Updated weights for policy 0, policy_version 159398 (0.0006) [2024-09-30 02:55:03,764][1157819] Updated weights for policy 0, policy_version 159408 (0.0006) [2024-09-30 02:55:04,233][1157819] Updated weights for policy 0, policy_version 159418 (0.0006) [2024-09-30 02:55:04,718][1157819] Updated weights for policy 0, policy_version 159428 (0.0006) [2024-09-30 02:55:05,212][1157819] Updated weights for policy 0, policy_version 159438 (0.0006) [2024-09-30 02:55:05,466][1157520] Fps is (10 sec: 85196.8, 60 sec: 81783.7, 300 sec: 80073.4). Total num frames: 653078528. Throughput: 0: 20480.7. Samples: 153255532. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 02:55:05,466][1157520] Avg episode reward: [(0, '56.959')] [2024-09-30 02:55:05,691][1157819] Updated weights for policy 0, policy_version 159448 (0.0006) [2024-09-30 02:55:06,172][1157819] Updated weights for policy 0, policy_version 159458 (0.0006) [2024-09-30 02:55:06,668][1157819] Updated weights for policy 0, policy_version 159468 (0.0006) [2024-09-30 02:55:07,156][1157819] Updated weights for policy 0, policy_version 159478 (0.0006) [2024-09-30 02:55:07,645][1157819] Updated weights for policy 0, policy_version 159488 (0.0006) [2024-09-30 02:55:08,128][1157819] Updated weights for policy 0, policy_version 159498 (0.0006) [2024-09-30 02:55:08,553][1157819] Updated weights for policy 0, policy_version 159508 (0.0006) [2024-09-30 02:55:09,019][1157819] Updated weights for policy 0, policy_version 159518 (0.0006) [2024-09-30 02:55:09,528][1157819] Updated weights for policy 0, policy_version 159528 (0.0006) [2024-09-30 02:55:10,003][1157819] Updated weights for policy 0, policy_version 159538 (0.0006) [2024-09-30 02:55:10,466][1157520] Fps is (10 sec: 85606.4, 60 sec: 82124.8, 300 sec: 80101.1). Total num frames: 653500416. Throughput: 0: 20580.4. Samples: 153319216. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 02:55:10,466][1157520] Avg episode reward: [(0, '53.985')] [2024-09-30 02:55:10,498][1157819] Updated weights for policy 0, policy_version 159548 (0.0006) [2024-09-30 02:55:11,009][1157819] Updated weights for policy 0, policy_version 159558 (0.0006) [2024-09-30 02:55:11,500][1157819] Updated weights for policy 0, policy_version 159568 (0.0006) [2024-09-30 02:55:11,961][1157819] Updated weights for policy 0, policy_version 159578 (0.0006) [2024-09-30 02:55:12,458][1157819] Updated weights for policy 0, policy_version 159588 (0.0006) [2024-09-30 02:55:12,959][1157819] Updated weights for policy 0, policy_version 159598 (0.0006) [2024-09-30 02:55:13,474][1157819] Updated weights for policy 0, policy_version 159608 (0.0006) [2024-09-30 02:55:14,003][1157819] Updated weights for policy 0, policy_version 159618 (0.0006) [2024-09-30 02:55:14,581][1157819] Updated weights for policy 0, policy_version 159628 (0.0006) [2024-09-30 02:55:15,125][1157819] Updated weights for policy 0, policy_version 159638 (0.0006) [2024-09-30 02:55:15,466][1157520] Fps is (10 sec: 82738.5, 60 sec: 82056.4, 300 sec: 80212.2). Total num frames: 653905920. Throughput: 0: 20726.7. Samples: 153444380. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 02:55:15,466][1157520] Avg episode reward: [(0, '57.247')] [2024-09-30 02:55:15,666][1157819] Updated weights for policy 0, policy_version 159648 (0.0006) [2024-09-30 02:55:16,189][1157819] Updated weights for policy 0, policy_version 159658 (0.0006) [2024-09-30 02:55:16,720][1157819] Updated weights for policy 0, policy_version 159668 (0.0006) [2024-09-30 02:55:17,276][1157819] Updated weights for policy 0, policy_version 159678 (0.0006) [2024-09-30 02:55:17,869][1157819] Updated weights for policy 0, policy_version 159688 (0.0006) [2024-09-30 02:55:18,449][1157819] Updated weights for policy 0, policy_version 159698 (0.0006) [2024-09-30 02:55:19,054][1157819] Updated weights for policy 0, policy_version 159708 (0.0006) [2024-09-30 02:55:19,675][1157819] Updated weights for policy 0, policy_version 159718 (0.0006) [2024-09-30 02:55:19,813][1157736] Signal inference workers to stop experience collection... (10900 times) [2024-09-30 02:55:19,814][1157736] Signal inference workers to resume experience collection... (10900 times) [2024-09-30 02:55:19,819][1157819] InferenceWorker_p0-w0: stopping experience collection (10900 times) [2024-09-30 02:55:19,819][1157819] InferenceWorker_p0-w0: resuming experience collection (10900 times) [2024-09-30 02:55:20,275][1157819] Updated weights for policy 0, policy_version 159728 (0.0006) [2024-09-30 02:55:20,466][1157520] Fps is (10 sec: 75774.0, 60 sec: 81168.7, 300 sec: 80073.3). Total num frames: 654258176. Throughput: 0: 20504.1. Samples: 153553072. Policy #0 lag: (min: 0.0, avg: 1.9, max: 4.0) [2024-09-30 02:55:20,466][1157520] Avg episode reward: [(0, '56.117')] [2024-09-30 02:55:20,835][1157819] Updated weights for policy 0, policy_version 159738 (0.0006) [2024-09-30 02:55:21,427][1157819] Updated weights for policy 0, policy_version 159748 (0.0006) [2024-09-30 02:55:22,067][1157819] Updated weights for policy 0, policy_version 159758 (0.0006) [2024-09-30 02:55:22,639][1157819] Updated weights for policy 0, policy_version 159768 (0.0006) [2024-09-30 02:55:23,205][1157819] Updated weights for policy 0, policy_version 159778 (0.0006) [2024-09-30 02:55:23,796][1157819] Updated weights for policy 0, policy_version 159788 (0.0006) [2024-09-30 02:55:24,393][1157819] Updated weights for policy 0, policy_version 159798 (0.0006) [2024-09-30 02:55:24,935][1157819] Updated weights for policy 0, policy_version 159808 (0.0006) [2024-09-30 02:55:25,466][1157520] Fps is (10 sec: 70041.5, 60 sec: 80418.0, 300 sec: 80003.9). Total num frames: 654606336. Throughput: 0: 20315.7. Samples: 153605144. Policy #0 lag: (min: 0.0, avg: 1.9, max: 4.0) [2024-09-30 02:55:25,466][1157520] Avg episode reward: [(0, '56.266')] [2024-09-30 02:55:25,536][1157819] Updated weights for policy 0, policy_version 159818 (0.0006) [2024-09-30 02:55:26,153][1157819] Updated weights for policy 0, policy_version 159828 (0.0006) [2024-09-30 02:55:26,708][1157819] Updated weights for policy 0, policy_version 159838 (0.0006) [2024-09-30 02:55:27,318][1157819] Updated weights for policy 0, policy_version 159848 (0.0006) [2024-09-30 02:55:27,875][1157819] Updated weights for policy 0, policy_version 159858 (0.0006) [2024-09-30 02:55:28,402][1157819] Updated weights for policy 0, policy_version 159868 (0.0006) [2024-09-30 02:55:28,926][1157819] Updated weights for policy 0, policy_version 159878 (0.0006) [2024-09-30 02:55:29,492][1157819] Updated weights for policy 0, policy_version 159888 (0.0006) [2024-09-30 02:55:30,045][1157819] Updated weights for policy 0, policy_version 159898 (0.0006) [2024-09-30 02:55:30,466][1157520] Fps is (10 sec: 71681.3, 60 sec: 79940.2, 300 sec: 80059.5). Total num frames: 654974976. Throughput: 0: 19974.4. Samples: 153711984. Policy #0 lag: (min: 0.0, avg: 1.9, max: 4.0) [2024-09-30 02:55:30,466][1157520] Avg episode reward: [(0, '56.337')] [2024-09-30 02:55:30,557][1157819] Updated weights for policy 0, policy_version 159908 (0.0006) [2024-09-30 02:55:31,104][1157819] Updated weights for policy 0, policy_version 159918 (0.0006) [2024-09-30 02:55:31,640][1157819] Updated weights for policy 0, policy_version 159928 (0.0006) [2024-09-30 02:55:32,221][1157819] Updated weights for policy 0, policy_version 159938 (0.0006) [2024-09-30 02:55:32,772][1157819] Updated weights for policy 0, policy_version 159948 (0.0006) [2024-09-30 02:55:33,348][1157819] Updated weights for policy 0, policy_version 159958 (0.0006) [2024-09-30 02:55:33,877][1157819] Updated weights for policy 0, policy_version 159968 (0.0006) [2024-09-30 02:55:34,434][1157819] Updated weights for policy 0, policy_version 159978 (0.0006) [2024-09-30 02:55:34,948][1157819] Updated weights for policy 0, policy_version 159988 (0.0006) [2024-09-30 02:55:35,466][1157520] Fps is (10 sec: 74138.3, 60 sec: 79598.9, 300 sec: 80184.4). Total num frames: 655347712. Throughput: 0: 19694.7. Samples: 153824324. Policy #0 lag: (min: 0.0, avg: 1.9, max: 4.0) [2024-09-30 02:55:35,466][1157520] Avg episode reward: [(0, '54.056')] [2024-09-30 02:55:35,483][1157819] Updated weights for policy 0, policy_version 159998 (0.0006) [2024-09-30 02:55:36,005][1157819] Updated weights for policy 0, policy_version 160008 (0.0006) [2024-09-30 02:55:36,511][1157819] Updated weights for policy 0, policy_version 160018 (0.0006) [2024-09-30 02:55:37,036][1157819] Updated weights for policy 0, policy_version 160028 (0.0006) [2024-09-30 02:55:37,541][1157819] Updated weights for policy 0, policy_version 160038 (0.0006) [2024-09-30 02:55:38,061][1157819] Updated weights for policy 0, policy_version 160048 (0.0006) [2024-09-30 02:55:38,660][1157819] Updated weights for policy 0, policy_version 160058 (0.0006) [2024-09-30 02:55:39,151][1157819] Updated weights for policy 0, policy_version 160068 (0.0006) [2024-09-30 02:55:39,703][1157819] Updated weights for policy 0, policy_version 160078 (0.0006) [2024-09-30 02:55:40,264][1157819] Updated weights for policy 0, policy_version 160088 (0.0006) [2024-09-30 02:55:40,466][1157520] Fps is (10 sec: 75776.0, 60 sec: 79189.3, 300 sec: 80295.5). Total num frames: 655732736. Throughput: 0: 19633.5. Samples: 153883844. Policy #0 lag: (min: 0.0, avg: 1.9, max: 4.0) [2024-09-30 02:55:40,466][1157520] Avg episode reward: [(0, '55.140')] [2024-09-30 02:55:40,801][1157819] Updated weights for policy 0, policy_version 160098 (0.0006) [2024-09-30 02:55:41,382][1157819] Updated weights for policy 0, policy_version 160108 (0.0006) [2024-09-30 02:55:41,914][1157819] Updated weights for policy 0, policy_version 160118 (0.0006) [2024-09-30 02:55:42,480][1157819] Updated weights for policy 0, policy_version 160128 (0.0006) [2024-09-30 02:55:43,044][1157819] Updated weights for policy 0, policy_version 160138 (0.0006) [2024-09-30 02:55:43,604][1157819] Updated weights for policy 0, policy_version 160148 (0.0006) [2024-09-30 02:55:44,136][1157819] Updated weights for policy 0, policy_version 160158 (0.0006) [2024-09-30 02:55:44,675][1157819] Updated weights for policy 0, policy_version 160168 (0.0006) [2024-09-30 02:55:45,197][1157819] Updated weights for policy 0, policy_version 160178 (0.0006) [2024-09-30 02:55:45,466][1157520] Fps is (10 sec: 75775.4, 60 sec: 78574.8, 300 sec: 80198.3). Total num frames: 656105472. Throughput: 0: 19313.1. Samples: 153995268. Policy #0 lag: (min: 0.0, avg: 1.9, max: 4.0) [2024-09-30 02:55:45,466][1157520] Avg episode reward: [(0, '55.646')] [2024-09-30 02:55:45,769][1157819] Updated weights for policy 0, policy_version 160188 (0.0006) [2024-09-30 02:55:46,294][1157819] Updated weights for policy 0, policy_version 160198 (0.0006) [2024-09-30 02:55:46,857][1157819] Updated weights for policy 0, policy_version 160208 (0.0006) [2024-09-30 02:55:47,405][1157819] Updated weights for policy 0, policy_version 160218 (0.0006) [2024-09-30 02:55:47,909][1157819] Updated weights for policy 0, policy_version 160228 (0.0006) [2024-09-30 02:55:48,448][1157819] Updated weights for policy 0, policy_version 160238 (0.0006) [2024-09-30 02:55:48,966][1157819] Updated weights for policy 0, policy_version 160248 (0.0006) [2024-09-30 02:55:49,478][1157819] Updated weights for policy 0, policy_version 160258 (0.0006) [2024-09-30 02:55:49,975][1157819] Updated weights for policy 0, policy_version 160268 (0.0006) [2024-09-30 02:55:50,466][1157520] Fps is (10 sec: 75776.0, 60 sec: 78097.0, 300 sec: 80142.8). Total num frames: 656490496. Throughput: 0: 18997.5. Samples: 154110420. Policy #0 lag: (min: 0.0, avg: 1.9, max: 4.0) [2024-09-30 02:55:50,466][1157520] Avg episode reward: [(0, '55.999')] [2024-09-30 02:55:50,512][1157819] Updated weights for policy 0, policy_version 160278 (0.0006) [2024-09-30 02:55:51,077][1157819] Updated weights for policy 0, policy_version 160288 (0.0006) [2024-09-30 02:55:51,631][1157819] Updated weights for policy 0, policy_version 160298 (0.0006) [2024-09-30 02:55:52,163][1157819] Updated weights for policy 0, policy_version 160308 (0.0006) [2024-09-30 02:55:52,683][1157819] Updated weights for policy 0, policy_version 160318 (0.0006) [2024-09-30 02:55:53,230][1157819] Updated weights for policy 0, policy_version 160328 (0.0006) [2024-09-30 02:55:53,778][1157819] Updated weights for policy 0, policy_version 160338 (0.0006) [2024-09-30 02:55:54,309][1157819] Updated weights for policy 0, policy_version 160348 (0.0006) [2024-09-30 02:55:54,820][1157819] Updated weights for policy 0, policy_version 160358 (0.0006) [2024-09-30 02:55:55,321][1157819] Updated weights for policy 0, policy_version 160368 (0.0006) [2024-09-30 02:55:55,466][1157520] Fps is (10 sec: 77004.7, 60 sec: 77482.6, 300 sec: 80059.5). Total num frames: 656875520. Throughput: 0: 18846.8. Samples: 154167324. Policy #0 lag: (min: 0.0, avg: 1.9, max: 4.0) [2024-09-30 02:55:55,466][1157520] Avg episode reward: [(0, '56.521')] [2024-09-30 02:55:55,853][1157819] Updated weights for policy 0, policy_version 160378 (0.0006) [2024-09-30 02:55:56,396][1157819] Updated weights for policy 0, policy_version 160388 (0.0006) [2024-09-30 02:55:56,890][1157819] Updated weights for policy 0, policy_version 160398 (0.0006) [2024-09-30 02:55:57,387][1157819] Updated weights for policy 0, policy_version 160408 (0.0006) [2024-09-30 02:55:57,917][1157819] Updated weights for policy 0, policy_version 160418 (0.0006) [2024-09-30 02:55:58,446][1157819] Updated weights for policy 0, policy_version 160428 (0.0006) [2024-09-30 02:55:58,513][1157736] Signal inference workers to stop experience collection... (10950 times) [2024-09-30 02:55:58,513][1157736] Signal inference workers to resume experience collection... (10950 times) [2024-09-30 02:55:58,519][1157819] InferenceWorker_p0-w0: stopping experience collection (10950 times) [2024-09-30 02:55:58,519][1157819] InferenceWorker_p0-w0: resuming experience collection (10950 times) [2024-09-30 02:55:58,982][1157819] Updated weights for policy 0, policy_version 160438 (0.0006) [2024-09-30 02:55:59,466][1157819] Updated weights for policy 0, policy_version 160448 (0.0006) [2024-09-30 02:55:59,987][1157819] Updated weights for policy 0, policy_version 160458 (0.0006) [2024-09-30 02:56:00,466][1157520] Fps is (10 sec: 77824.0, 60 sec: 77073.0, 300 sec: 79948.4). Total num frames: 657268736. Throughput: 0: 18679.6. Samples: 154284960. Policy #0 lag: (min: 0.0, avg: 1.9, max: 4.0) [2024-09-30 02:56:00,466][1157520] Avg episode reward: [(0, '55.107')] [2024-09-30 02:56:00,499][1157819] Updated weights for policy 0, policy_version 160468 (0.0006) [2024-09-30 02:56:01,016][1157819] Updated weights for policy 0, policy_version 160478 (0.0006) [2024-09-30 02:56:01,536][1157819] Updated weights for policy 0, policy_version 160488 (0.0006) [2024-09-30 02:56:02,040][1157819] Updated weights for policy 0, policy_version 160498 (0.0006) [2024-09-30 02:56:02,549][1157819] Updated weights for policy 0, policy_version 160508 (0.0006) [2024-09-30 02:56:03,033][1157819] Updated weights for policy 0, policy_version 160518 (0.0006) [2024-09-30 02:56:03,512][1157819] Updated weights for policy 0, policy_version 160528 (0.0006) [2024-09-30 02:56:04,019][1157819] Updated weights for policy 0, policy_version 160538 (0.0006) [2024-09-30 02:56:04,508][1157819] Updated weights for policy 0, policy_version 160548 (0.0006) [2024-09-30 02:56:04,993][1157819] Updated weights for policy 0, policy_version 160558 (0.0006) [2024-09-30 02:56:05,466][1157520] Fps is (10 sec: 80691.0, 60 sec: 76731.6, 300 sec: 79934.5). Total num frames: 657682432. Throughput: 0: 18978.3. Samples: 154407092. Policy #0 lag: (min: 0.0, avg: 1.9, max: 4.0) [2024-09-30 02:56:05,466][1157520] Avg episode reward: [(0, '55.420')] [2024-09-30 02:56:05,472][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000160567_657682432.pth... [2024-09-30 02:56:05,518][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000155924_638664704.pth [2024-09-30 02:56:05,537][1157819] Updated weights for policy 0, policy_version 160568 (0.0006) [2024-09-30 02:56:06,106][1157819] Updated weights for policy 0, policy_version 160578 (0.0006) [2024-09-30 02:56:06,690][1157819] Updated weights for policy 0, policy_version 160588 (0.0006) [2024-09-30 02:56:07,248][1157819] Updated weights for policy 0, policy_version 160598 (0.0006) [2024-09-30 02:56:07,847][1157819] Updated weights for policy 0, policy_version 160608 (0.0006) [2024-09-30 02:56:08,430][1157819] Updated weights for policy 0, policy_version 160618 (0.0006) [2024-09-30 02:56:08,939][1157819] Updated weights for policy 0, policy_version 160628 (0.0006) [2024-09-30 02:56:09,503][1157819] Updated weights for policy 0, policy_version 160638 (0.0006) [2024-09-30 02:56:10,048][1157819] Updated weights for policy 0, policy_version 160648 (0.0006) [2024-09-30 02:56:10,466][1157520] Fps is (10 sec: 77005.3, 60 sec: 75639.4, 300 sec: 79712.4). Total num frames: 658038784. Throughput: 0: 19028.7. Samples: 154461436. Policy #0 lag: (min: 0.0, avg: 1.9, max: 4.0) [2024-09-30 02:56:10,466][1157520] Avg episode reward: [(0, '54.240')] [2024-09-30 02:56:10,629][1157819] Updated weights for policy 0, policy_version 160658 (0.0006) [2024-09-30 02:56:11,171][1157819] Updated weights for policy 0, policy_version 160668 (0.0006) [2024-09-30 02:56:11,735][1157819] Updated weights for policy 0, policy_version 160678 (0.0006) [2024-09-30 02:56:12,279][1157819] Updated weights for policy 0, policy_version 160688 (0.0006) [2024-09-30 02:56:12,843][1157819] Updated weights for policy 0, policy_version 160698 (0.0006) [2024-09-30 02:56:13,404][1157819] Updated weights for policy 0, policy_version 160708 (0.0006) [2024-09-30 02:56:13,929][1157819] Updated weights for policy 0, policy_version 160718 (0.0006) [2024-09-30 02:56:14,546][1157819] Updated weights for policy 0, policy_version 160728 (0.0006) [2024-09-30 02:56:15,119][1157819] Updated weights for policy 0, policy_version 160738 (0.0006) [2024-09-30 02:56:15,466][1157520] Fps is (10 sec: 72089.6, 60 sec: 74956.8, 300 sec: 79504.1). Total num frames: 658403328. Throughput: 0: 19121.0. Samples: 154572428. Policy #0 lag: (min: 0.0, avg: 1.9, max: 4.0) [2024-09-30 02:56:15,466][1157520] Avg episode reward: [(0, '53.368')] [2024-09-30 02:56:15,709][1157819] Updated weights for policy 0, policy_version 160748 (0.0006) [2024-09-30 02:56:16,310][1157819] Updated weights for policy 0, policy_version 160758 (0.0006) [2024-09-30 02:56:16,877][1157819] Updated weights for policy 0, policy_version 160768 (0.0006) [2024-09-30 02:56:17,469][1157819] Updated weights for policy 0, policy_version 160778 (0.0006) [2024-09-30 02:56:18,055][1157819] Updated weights for policy 0, policy_version 160788 (0.0006) [2024-09-30 02:56:18,661][1157819] Updated weights for policy 0, policy_version 160798 (0.0006) [2024-09-30 02:56:19,247][1157819] Updated weights for policy 0, policy_version 160808 (0.0006) [2024-09-30 02:56:19,809][1157819] Updated weights for policy 0, policy_version 160818 (0.0006) [2024-09-30 02:56:20,376][1157819] Updated weights for policy 0, policy_version 160828 (0.0006) [2024-09-30 02:56:20,466][1157520] Fps is (10 sec: 71269.8, 60 sec: 74888.7, 300 sec: 79184.7). Total num frames: 658751488. Throughput: 0: 18950.4. Samples: 154677092. Policy #0 lag: (min: 0.0, avg: 1.9, max: 4.0) [2024-09-30 02:56:20,466][1157520] Avg episode reward: [(0, '55.754')] [2024-09-30 02:56:20,995][1157819] Updated weights for policy 0, policy_version 160838 (0.0006) [2024-09-30 02:56:21,596][1157819] Updated weights for policy 0, policy_version 160848 (0.0006) [2024-09-30 02:56:22,208][1157819] Updated weights for policy 0, policy_version 160858 (0.0006) [2024-09-30 02:56:22,848][1157819] Updated weights for policy 0, policy_version 160868 (0.0006) [2024-09-30 02:56:23,454][1157819] Updated weights for policy 0, policy_version 160878 (0.0006) [2024-09-30 02:56:24,080][1157819] Updated weights for policy 0, policy_version 160888 (0.0006) [2024-09-30 02:56:24,711][1157819] Updated weights for policy 0, policy_version 160898 (0.0006) [2024-09-30 02:56:25,296][1157819] Updated weights for policy 0, policy_version 160908 (0.0006) [2024-09-30 02:56:25,466][1157520] Fps is (10 sec: 68402.1, 60 sec: 74683.5, 300 sec: 78879.2). Total num frames: 659087360. Throughput: 0: 18762.3. Samples: 154728152. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:56:25,466][1157520] Avg episode reward: [(0, '56.101')] [2024-09-30 02:56:25,916][1157819] Updated weights for policy 0, policy_version 160918 (0.0006) [2024-09-30 02:56:26,544][1157819] Updated weights for policy 0, policy_version 160928 (0.0006) [2024-09-30 02:56:27,170][1157819] Updated weights for policy 0, policy_version 160938 (0.0006) [2024-09-30 02:56:27,777][1157819] Updated weights for policy 0, policy_version 160948 (0.0006) [2024-09-30 02:56:28,463][1157819] Updated weights for policy 0, policy_version 160958 (0.0006) [2024-09-30 02:56:29,066][1157819] Updated weights for policy 0, policy_version 160968 (0.0006) [2024-09-30 02:56:29,676][1157819] Updated weights for policy 0, policy_version 160978 (0.0006) [2024-09-30 02:56:30,330][1157819] Updated weights for policy 0, policy_version 160988 (0.0006) [2024-09-30 02:56:30,466][1157520] Fps is (10 sec: 66354.7, 60 sec: 74001.0, 300 sec: 78684.9). Total num frames: 659415040. Throughput: 0: 18477.8. Samples: 154826772. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:56:30,466][1157520] Avg episode reward: [(0, '55.417')] [2024-09-30 02:56:30,923][1157819] Updated weights for policy 0, policy_version 160998 (0.0006) [2024-09-30 02:56:31,496][1157819] Updated weights for policy 0, policy_version 161008 (0.0006) [2024-09-30 02:56:32,114][1157819] Updated weights for policy 0, policy_version 161018 (0.0006) [2024-09-30 02:56:32,756][1157819] Updated weights for policy 0, policy_version 161028 (0.0006) [2024-09-30 02:56:33,403][1157819] Updated weights for policy 0, policy_version 161038 (0.0006) [2024-09-30 02:56:33,991][1157819] Updated weights for policy 0, policy_version 161048 (0.0006) [2024-09-30 02:56:34,602][1157819] Updated weights for policy 0, policy_version 161058 (0.0006) [2024-09-30 02:56:35,192][1157819] Updated weights for policy 0, policy_version 161068 (0.0006) [2024-09-30 02:56:35,466][1157520] Fps is (10 sec: 66355.3, 60 sec: 73386.4, 300 sec: 78476.5). Total num frames: 659750912. Throughput: 0: 18140.6. Samples: 154926752. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:56:35,466][1157520] Avg episode reward: [(0, '54.772')] [2024-09-30 02:56:35,791][1157819] Updated weights for policy 0, policy_version 161078 (0.0006) [2024-09-30 02:56:36,195][1157736] Signal inference workers to stop experience collection... (11000 times) [2024-09-30 02:56:36,196][1157736] Signal inference workers to resume experience collection... (11000 times) [2024-09-30 02:56:36,199][1157819] InferenceWorker_p0-w0: stopping experience collection (11000 times) [2024-09-30 02:56:36,199][1157819] InferenceWorker_p0-w0: resuming experience collection (11000 times) [2024-09-30 02:56:36,420][1157819] Updated weights for policy 0, policy_version 161088 (0.0006) [2024-09-30 02:56:36,996][1157819] Updated weights for policy 0, policy_version 161098 (0.0006) [2024-09-30 02:56:37,522][1157819] Updated weights for policy 0, policy_version 161108 (0.0006) [2024-09-30 02:56:38,096][1157819] Updated weights for policy 0, policy_version 161118 (0.0006) [2024-09-30 02:56:38,619][1157819] Updated weights for policy 0, policy_version 161128 (0.0006) [2024-09-30 02:56:39,131][1157819] Updated weights for policy 0, policy_version 161138 (0.0006) [2024-09-30 02:56:39,656][1157819] Updated weights for policy 0, policy_version 161148 (0.0006) [2024-09-30 02:56:40,196][1157819] Updated weights for policy 0, policy_version 161158 (0.0006) [2024-09-30 02:56:40,466][1157520] Fps is (10 sec: 70860.6, 60 sec: 73181.7, 300 sec: 78296.1). Total num frames: 660123648. Throughput: 0: 18036.1. Samples: 154978952. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:56:40,466][1157520] Avg episode reward: [(0, '53.622')] [2024-09-30 02:56:40,698][1157819] Updated weights for policy 0, policy_version 161168 (0.0006) [2024-09-30 02:56:41,228][1157819] Updated weights for policy 0, policy_version 161178 (0.0006) [2024-09-30 02:56:41,749][1157819] Updated weights for policy 0, policy_version 161188 (0.0006) [2024-09-30 02:56:42,279][1157819] Updated weights for policy 0, policy_version 161198 (0.0006) [2024-09-30 02:56:42,818][1157819] Updated weights for policy 0, policy_version 161208 (0.0006) [2024-09-30 02:56:43,352][1157819] Updated weights for policy 0, policy_version 161218 (0.0006) [2024-09-30 02:56:43,861][1157819] Updated weights for policy 0, policy_version 161228 (0.0006) [2024-09-30 02:56:44,388][1157819] Updated weights for policy 0, policy_version 161238 (0.0006) [2024-09-30 02:56:44,906][1157819] Updated weights for policy 0, policy_version 161248 (0.0006) [2024-09-30 02:56:45,397][1157819] Updated weights for policy 0, policy_version 161258 (0.0006) [2024-09-30 02:56:45,466][1157520] Fps is (10 sec: 76597.1, 60 sec: 73523.3, 300 sec: 78185.0). Total num frames: 660516864. Throughput: 0: 18020.6. Samples: 155095888. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:56:45,466][1157520] Avg episode reward: [(0, '57.166')] [2024-09-30 02:56:45,910][1157819] Updated weights for policy 0, policy_version 161268 (0.0006) [2024-09-30 02:56:46,400][1157819] Updated weights for policy 0, policy_version 161278 (0.0006) [2024-09-30 02:56:46,898][1157819] Updated weights for policy 0, policy_version 161288 (0.0006) [2024-09-30 02:56:47,380][1157819] Updated weights for policy 0, policy_version 161298 (0.0006) [2024-09-30 02:56:47,871][1157819] Updated weights for policy 0, policy_version 161308 (0.0006) [2024-09-30 02:56:48,373][1157819] Updated weights for policy 0, policy_version 161318 (0.0006) [2024-09-30 02:56:48,838][1157819] Updated weights for policy 0, policy_version 161328 (0.0006) [2024-09-30 02:56:49,338][1157819] Updated weights for policy 0, policy_version 161338 (0.0006) [2024-09-30 02:56:49,775][1157819] Updated weights for policy 0, policy_version 161348 (0.0006) [2024-09-30 02:56:50,252][1157819] Updated weights for policy 0, policy_version 161358 (0.0006) [2024-09-30 02:56:50,466][1157520] Fps is (10 sec: 81511.9, 60 sec: 74137.7, 300 sec: 78171.2). Total num frames: 660938752. Throughput: 0: 18069.6. Samples: 155220224. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:56:50,466][1157520] Avg episode reward: [(0, '56.680')] [2024-09-30 02:56:50,748][1157819] Updated weights for policy 0, policy_version 161368 (0.0006) [2024-09-30 02:56:51,222][1157819] Updated weights for policy 0, policy_version 161378 (0.0006) [2024-09-30 02:56:51,734][1157819] Updated weights for policy 0, policy_version 161388 (0.0006) [2024-09-30 02:56:52,236][1157819] Updated weights for policy 0, policy_version 161398 (0.0006) [2024-09-30 02:56:52,730][1157819] Updated weights for policy 0, policy_version 161408 (0.0006) [2024-09-30 02:56:53,238][1157819] Updated weights for policy 0, policy_version 161418 (0.0006) [2024-09-30 02:56:53,762][1157819] Updated weights for policy 0, policy_version 161428 (0.0006) [2024-09-30 02:56:54,286][1157819] Updated weights for policy 0, policy_version 161438 (0.0006) [2024-09-30 02:56:54,826][1157819] Updated weights for policy 0, policy_version 161448 (0.0006) [2024-09-30 02:56:55,341][1157819] Updated weights for policy 0, policy_version 161458 (0.0006) [2024-09-30 02:56:55,466][1157520] Fps is (10 sec: 82329.5, 60 sec: 74410.8, 300 sec: 78212.8). Total num frames: 661340160. Throughput: 0: 18249.5. Samples: 155282664. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:56:55,466][1157520] Avg episode reward: [(0, '54.045')] [2024-09-30 02:56:55,862][1157819] Updated weights for policy 0, policy_version 161468 (0.0006) [2024-09-30 02:56:56,379][1157819] Updated weights for policy 0, policy_version 161478 (0.0006) [2024-09-30 02:56:56,876][1157819] Updated weights for policy 0, policy_version 161488 (0.0006) [2024-09-30 02:56:57,418][1157819] Updated weights for policy 0, policy_version 161498 (0.0006) [2024-09-30 02:56:57,923][1157819] Updated weights for policy 0, policy_version 161508 (0.0006) [2024-09-30 02:56:58,440][1157819] Updated weights for policy 0, policy_version 161518 (0.0006) [2024-09-30 02:56:58,965][1157819] Updated weights for policy 0, policy_version 161528 (0.0006) [2024-09-30 02:56:59,474][1157819] Updated weights for policy 0, policy_version 161538 (0.0006) [2024-09-30 02:56:59,977][1157819] Updated weights for policy 0, policy_version 161548 (0.0006) [2024-09-30 02:57:00,466][1157520] Fps is (10 sec: 79462.5, 60 sec: 74410.8, 300 sec: 78212.8). Total num frames: 661733376. Throughput: 0: 18421.6. Samples: 155401400. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:57:00,466][1157520] Avg episode reward: [(0, '52.789')] [2024-09-30 02:57:00,491][1157819] Updated weights for policy 0, policy_version 161558 (0.0006) [2024-09-30 02:57:00,990][1157819] Updated weights for policy 0, policy_version 161568 (0.0006) [2024-09-30 02:57:01,539][1157819] Updated weights for policy 0, policy_version 161578 (0.0006) [2024-09-30 02:57:02,112][1157736] Signal inference workers to stop experience collection... (11050 times) [2024-09-30 02:57:02,116][1157819] InferenceWorker_p0-w0: stopping experience collection (11050 times) [2024-09-30 02:57:02,117][1157736] Signal inference workers to resume experience collection... (11050 times) [2024-09-30 02:57:02,117][1157819] Updated weights for policy 0, policy_version 161588 (0.0006) [2024-09-30 02:57:02,121][1157819] InferenceWorker_p0-w0: resuming experience collection (11050 times) [2024-09-30 02:57:02,621][1157819] Updated weights for policy 0, policy_version 161598 (0.0006) [2024-09-30 02:57:03,155][1157819] Updated weights for policy 0, policy_version 161608 (0.0006) [2024-09-30 02:57:03,697][1157819] Updated weights for policy 0, policy_version 161618 (0.0006) [2024-09-30 02:57:04,260][1157819] Updated weights for policy 0, policy_version 161628 (0.0006) [2024-09-30 02:57:04,788][1157819] Updated weights for policy 0, policy_version 161638 (0.0006) [2024-09-30 02:57:05,353][1157819] Updated weights for policy 0, policy_version 161648 (0.0007) [2024-09-30 02:57:05,466][1157520] Fps is (10 sec: 77823.3, 60 sec: 73932.8, 300 sec: 78185.0). Total num frames: 662118400. Throughput: 0: 18674.2. Samples: 155517432. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:57:05,466][1157520] Avg episode reward: [(0, '55.184')] [2024-09-30 02:57:05,881][1157819] Updated weights for policy 0, policy_version 161658 (0.0006) [2024-09-30 02:57:06,424][1157819] Updated weights for policy 0, policy_version 161668 (0.0006) [2024-09-30 02:57:06,958][1157819] Updated weights for policy 0, policy_version 161678 (0.0007) [2024-09-30 02:57:07,474][1157819] Updated weights for policy 0, policy_version 161688 (0.0006) [2024-09-30 02:57:08,004][1157819] Updated weights for policy 0, policy_version 161698 (0.0006) [2024-09-30 02:57:08,521][1157819] Updated weights for policy 0, policy_version 161708 (0.0006) [2024-09-30 02:57:09,037][1157819] Updated weights for policy 0, policy_version 161718 (0.0006) [2024-09-30 02:57:09,566][1157819] Updated weights for policy 0, policy_version 161728 (0.0006) [2024-09-30 02:57:10,071][1157819] Updated weights for policy 0, policy_version 161738 (0.0006) [2024-09-30 02:57:10,466][1157520] Fps is (10 sec: 77414.2, 60 sec: 74478.9, 300 sec: 78129.5). Total num frames: 662507520. Throughput: 0: 18813.1. Samples: 155574736. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:57:10,466][1157520] Avg episode reward: [(0, '53.401')] [2024-09-30 02:57:10,565][1157819] Updated weights for policy 0, policy_version 161748 (0.0006) [2024-09-30 02:57:11,095][1157819] Updated weights for policy 0, policy_version 161758 (0.0006) [2024-09-30 02:57:11,602][1157819] Updated weights for policy 0, policy_version 161768 (0.0006) [2024-09-30 02:57:12,097][1157819] Updated weights for policy 0, policy_version 161778 (0.0006) [2024-09-30 02:57:12,593][1157819] Updated weights for policy 0, policy_version 161788 (0.0006) [2024-09-30 02:57:13,085][1157819] Updated weights for policy 0, policy_version 161798 (0.0006) [2024-09-30 02:57:13,584][1157819] Updated weights for policy 0, policy_version 161808 (0.0006) [2024-09-30 02:57:14,087][1157819] Updated weights for policy 0, policy_version 161818 (0.0006) [2024-09-30 02:57:14,623][1157819] Updated weights for policy 0, policy_version 161828 (0.0006) [2024-09-30 02:57:15,145][1157819] Updated weights for policy 0, policy_version 161838 (0.0006) [2024-09-30 02:57:15,466][1157520] Fps is (10 sec: 79053.3, 60 sec: 75093.4, 300 sec: 78087.8). Total num frames: 662908928. Throughput: 0: 19304.1. Samples: 155695452. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:57:15,466][1157520] Avg episode reward: [(0, '55.472')] [2024-09-30 02:57:15,686][1157819] Updated weights for policy 0, policy_version 161848 (0.0006) [2024-09-30 02:57:16,274][1157819] Updated weights for policy 0, policy_version 161858 (0.0006) [2024-09-30 02:57:16,875][1157819] Updated weights for policy 0, policy_version 161868 (0.0006) [2024-09-30 02:57:17,431][1157819] Updated weights for policy 0, policy_version 161878 (0.0006) [2024-09-30 02:57:18,000][1157819] Updated weights for policy 0, policy_version 161888 (0.0006) [2024-09-30 02:57:18,616][1157819] Updated weights for policy 0, policy_version 161898 (0.0006) [2024-09-30 02:57:19,127][1157819] Updated weights for policy 0, policy_version 161908 (0.0006) [2024-09-30 02:57:19,639][1157819] Updated weights for policy 0, policy_version 161918 (0.0006) [2024-09-30 02:57:20,185][1157819] Updated weights for policy 0, policy_version 161928 (0.0006) [2024-09-30 02:57:20,466][1157520] Fps is (10 sec: 77004.2, 60 sec: 75434.6, 300 sec: 77990.6). Total num frames: 663277568. Throughput: 0: 19544.9. Samples: 155806268. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:57:20,466][1157520] Avg episode reward: [(0, '55.054')] [2024-09-30 02:57:20,757][1157819] Updated weights for policy 0, policy_version 161938 (0.0006) [2024-09-30 02:57:21,303][1157819] Updated weights for policy 0, policy_version 161948 (0.0006) [2024-09-30 02:57:21,871][1157819] Updated weights for policy 0, policy_version 161958 (0.0006) [2024-09-30 02:57:22,405][1157819] Updated weights for policy 0, policy_version 161968 (0.0006) [2024-09-30 02:57:23,016][1157819] Updated weights for policy 0, policy_version 161978 (0.0006) [2024-09-30 02:57:23,549][1157819] Updated weights for policy 0, policy_version 161988 (0.0006) [2024-09-30 02:57:24,116][1157819] Updated weights for policy 0, policy_version 161998 (0.0006) [2024-09-30 02:57:24,743][1157819] Updated weights for policy 0, policy_version 162008 (0.0006) [2024-09-30 02:57:25,332][1157819] Updated weights for policy 0, policy_version 162018 (0.0006) [2024-09-30 02:57:25,466][1157520] Fps is (10 sec: 72498.4, 60 sec: 75776.2, 300 sec: 77824.0). Total num frames: 663633920. Throughput: 0: 19628.2. Samples: 155862220. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 02:57:25,466][1157520] Avg episode reward: [(0, '55.018')] [2024-09-30 02:57:25,909][1157819] Updated weights for policy 0, policy_version 162028 (0.0006) [2024-09-30 02:57:26,491][1157819] Updated weights for policy 0, policy_version 162038 (0.0006) [2024-09-30 02:57:27,073][1157819] Updated weights for policy 0, policy_version 162048 (0.0006) [2024-09-30 02:57:27,649][1157819] Updated weights for policy 0, policy_version 162058 (0.0006) [2024-09-30 02:57:28,235][1157819] Updated weights for policy 0, policy_version 162068 (0.0006) [2024-09-30 02:57:28,783][1157819] Updated weights for policy 0, policy_version 162078 (0.0006) [2024-09-30 02:57:29,360][1157819] Updated weights for policy 0, policy_version 162088 (0.0007) [2024-09-30 02:57:29,912][1157819] Updated weights for policy 0, policy_version 162098 (0.0006) [2024-09-30 02:57:30,466][1157520] Fps is (10 sec: 70860.9, 60 sec: 76185.7, 300 sec: 77643.5). Total num frames: 663986176. Throughput: 0: 19374.6. Samples: 155967748. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:57:30,466][1157520] Avg episode reward: [(0, '55.375')] [2024-09-30 02:57:30,543][1157819] Updated weights for policy 0, policy_version 162108 (0.0006) [2024-09-30 02:57:31,143][1157819] Updated weights for policy 0, policy_version 162118 (0.0006) [2024-09-30 02:57:31,770][1157819] Updated weights for policy 0, policy_version 162128 (0.0006) [2024-09-30 02:57:32,388][1157819] Updated weights for policy 0, policy_version 162138 (0.0006) [2024-09-30 02:57:33,005][1157819] Updated weights for policy 0, policy_version 162148 (0.0006) [2024-09-30 02:57:33,605][1157819] Updated weights for policy 0, policy_version 162158 (0.0006) [2024-09-30 02:57:34,135][1157819] Updated weights for policy 0, policy_version 162168 (0.0006) [2024-09-30 02:57:34,697][1157819] Updated weights for policy 0, policy_version 162178 (0.0006) [2024-09-30 02:57:35,199][1157819] Updated weights for policy 0, policy_version 162188 (0.0006) [2024-09-30 02:57:35,466][1157520] Fps is (10 sec: 70860.2, 60 sec: 76527.0, 300 sec: 77476.8). Total num frames: 664342528. Throughput: 0: 18932.0. Samples: 156072168. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:57:35,466][1157520] Avg episode reward: [(0, '53.493')] [2024-09-30 02:57:35,748][1157819] Updated weights for policy 0, policy_version 162198 (0.0006) [2024-09-30 02:57:36,257][1157819] Updated weights for policy 0, policy_version 162208 (0.0006) [2024-09-30 02:57:36,757][1157819] Updated weights for policy 0, policy_version 162218 (0.0006) [2024-09-30 02:57:37,287][1157819] Updated weights for policy 0, policy_version 162228 (0.0006) [2024-09-30 02:57:37,813][1157819] Updated weights for policy 0, policy_version 162238 (0.0006) [2024-09-30 02:57:38,345][1157819] Updated weights for policy 0, policy_version 162248 (0.0006) [2024-09-30 02:57:38,870][1157819] Updated weights for policy 0, policy_version 162258 (0.0006) [2024-09-30 02:57:39,362][1157819] Updated weights for policy 0, policy_version 162268 (0.0006) [2024-09-30 02:57:39,864][1157819] Updated weights for policy 0, policy_version 162278 (0.0006) [2024-09-30 02:57:40,330][1157736] Signal inference workers to stop experience collection... (11100 times) [2024-09-30 02:57:40,331][1157736] Signal inference workers to resume experience collection... (11100 times) [2024-09-30 02:57:40,337][1157819] InferenceWorker_p0-w0: stopping experience collection (11100 times) [2024-09-30 02:57:40,337][1157819] InferenceWorker_p0-w0: resuming experience collection (11100 times) [2024-09-30 02:57:40,412][1157819] Updated weights for policy 0, policy_version 162288 (0.0006) [2024-09-30 02:57:40,466][1157520] Fps is (10 sec: 74957.4, 60 sec: 76868.5, 300 sec: 77449.1). Total num frames: 664735744. Throughput: 0: 18855.9. Samples: 156131180. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:57:40,466][1157520] Avg episode reward: [(0, '55.549')] [2024-09-30 02:57:40,858][1157819] Updated weights for policy 0, policy_version 162298 (0.0006) [2024-09-30 02:57:41,395][1157819] Updated weights for policy 0, policy_version 162308 (0.0006) [2024-09-30 02:57:41,947][1157819] Updated weights for policy 0, policy_version 162318 (0.0006) [2024-09-30 02:57:42,447][1157819] Updated weights for policy 0, policy_version 162328 (0.0006) [2024-09-30 02:57:42,943][1157819] Updated weights for policy 0, policy_version 162338 (0.0006) [2024-09-30 02:57:43,465][1157819] Updated weights for policy 0, policy_version 162348 (0.0006) [2024-09-30 02:57:43,989][1157819] Updated weights for policy 0, policy_version 162358 (0.0006) [2024-09-30 02:57:44,506][1157819] Updated weights for policy 0, policy_version 162368 (0.0006) [2024-09-30 02:57:44,998][1157819] Updated weights for policy 0, policy_version 162378 (0.0006) [2024-09-30 02:57:45,466][1157520] Fps is (10 sec: 79053.6, 60 sec: 76936.4, 300 sec: 77435.2). Total num frames: 665133056. Throughput: 0: 18869.8. Samples: 156250544. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:57:45,466][1157520] Avg episode reward: [(0, '53.892')] [2024-09-30 02:57:45,492][1157819] Updated weights for policy 0, policy_version 162388 (0.0006) [2024-09-30 02:57:46,026][1157819] Updated weights for policy 0, policy_version 162398 (0.0006) [2024-09-30 02:57:46,562][1157819] Updated weights for policy 0, policy_version 162408 (0.0006) [2024-09-30 02:57:47,084][1157819] Updated weights for policy 0, policy_version 162418 (0.0006) [2024-09-30 02:57:47,593][1157819] Updated weights for policy 0, policy_version 162428 (0.0006) [2024-09-30 02:57:48,095][1157819] Updated weights for policy 0, policy_version 162438 (0.0006) [2024-09-30 02:57:48,612][1157819] Updated weights for policy 0, policy_version 162448 (0.0006) [2024-09-30 02:57:49,115][1157819] Updated weights for policy 0, policy_version 162458 (0.0006) [2024-09-30 02:57:49,629][1157819] Updated weights for policy 0, policy_version 162468 (0.0006) [2024-09-30 02:57:50,147][1157819] Updated weights for policy 0, policy_version 162478 (0.0006) [2024-09-30 02:57:50,466][1157520] Fps is (10 sec: 79461.9, 60 sec: 76526.8, 300 sec: 77393.6). Total num frames: 665530368. Throughput: 0: 18951.5. Samples: 156370248. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:57:50,466][1157520] Avg episode reward: [(0, '56.037')] [2024-09-30 02:57:50,672][1157819] Updated weights for policy 0, policy_version 162488 (0.0006) [2024-09-30 02:57:51,200][1157819] Updated weights for policy 0, policy_version 162498 (0.0006) [2024-09-30 02:57:51,698][1157819] Updated weights for policy 0, policy_version 162508 (0.0006) [2024-09-30 02:57:52,219][1157819] Updated weights for policy 0, policy_version 162518 (0.0006) [2024-09-30 02:57:52,771][1157819] Updated weights for policy 0, policy_version 162528 (0.0006) [2024-09-30 02:57:53,260][1157819] Updated weights for policy 0, policy_version 162538 (0.0006) [2024-09-30 02:57:53,776][1157819] Updated weights for policy 0, policy_version 162548 (0.0006) [2024-09-30 02:57:54,286][1157819] Updated weights for policy 0, policy_version 162558 (0.0006) [2024-09-30 02:57:54,813][1157819] Updated weights for policy 0, policy_version 162568 (0.0006) [2024-09-30 02:57:55,324][1157819] Updated weights for policy 0, policy_version 162578 (0.0006) [2024-09-30 02:57:55,466][1157520] Fps is (10 sec: 79461.7, 60 sec: 76458.4, 300 sec: 77407.4). Total num frames: 665927680. Throughput: 0: 18996.4. Samples: 156429576. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:57:55,466][1157520] Avg episode reward: [(0, '55.927')] [2024-09-30 02:57:55,859][1157819] Updated weights for policy 0, policy_version 162588 (0.0006) [2024-09-30 02:57:56,369][1157819] Updated weights for policy 0, policy_version 162598 (0.0006) [2024-09-30 02:57:56,863][1157819] Updated weights for policy 0, policy_version 162608 (0.0006) [2024-09-30 02:57:57,370][1157819] Updated weights for policy 0, policy_version 162618 (0.0006) [2024-09-30 02:57:57,897][1157819] Updated weights for policy 0, policy_version 162628 (0.0006) [2024-09-30 02:57:58,425][1157819] Updated weights for policy 0, policy_version 162638 (0.0006) [2024-09-30 02:57:58,920][1157819] Updated weights for policy 0, policy_version 162648 (0.0006) [2024-09-30 02:57:59,413][1157819] Updated weights for policy 0, policy_version 162658 (0.0006) [2024-09-30 02:57:59,994][1157819] Updated weights for policy 0, policy_version 162668 (0.0006) [2024-09-30 02:58:00,466][1157520] Fps is (10 sec: 79053.2, 60 sec: 76458.6, 300 sec: 77407.5). Total num frames: 666320896. Throughput: 0: 18957.6. Samples: 156548544. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:58:00,466][1157520] Avg episode reward: [(0, '55.085')] [2024-09-30 02:58:00,543][1157819] Updated weights for policy 0, policy_version 162678 (0.0006) [2024-09-30 02:58:01,044][1157819] Updated weights for policy 0, policy_version 162688 (0.0006) [2024-09-30 02:58:01,535][1157819] Updated weights for policy 0, policy_version 162698 (0.0006) [2024-09-30 02:58:02,099][1157819] Updated weights for policy 0, policy_version 162708 (0.0006) [2024-09-30 02:58:02,564][1157819] Updated weights for policy 0, policy_version 162718 (0.0006) [2024-09-30 02:58:03,053][1157819] Updated weights for policy 0, policy_version 162728 (0.0006) [2024-09-30 02:58:03,557][1157819] Updated weights for policy 0, policy_version 162738 (0.0006) [2024-09-30 02:58:04,035][1157819] Updated weights for policy 0, policy_version 162748 (0.0007) [2024-09-30 02:58:04,540][1157819] Updated weights for policy 0, policy_version 162758 (0.0007) [2024-09-30 02:58:05,049][1157819] Updated weights for policy 0, policy_version 162768 (0.0006) [2024-09-30 02:58:05,466][1157520] Fps is (10 sec: 80283.0, 60 sec: 76868.3, 300 sec: 77476.9). Total num frames: 666730496. Throughput: 0: 19177.8. Samples: 156669268. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:58:05,466][1157520] Avg episode reward: [(0, '56.694')] [2024-09-30 02:58:05,469][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000162776_666730496.pth... [2024-09-30 02:58:05,516][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000158246_648175616.pth [2024-09-30 02:58:05,556][1157819] Updated weights for policy 0, policy_version 162778 (0.0007) [2024-09-30 02:58:06,046][1157819] Updated weights for policy 0, policy_version 162788 (0.0006) [2024-09-30 02:58:06,558][1157819] Updated weights for policy 0, policy_version 162798 (0.0006) [2024-09-30 02:58:07,047][1157819] Updated weights for policy 0, policy_version 162808 (0.0006) [2024-09-30 02:58:07,554][1157819] Updated weights for policy 0, policy_version 162818 (0.0006) [2024-09-30 02:58:08,102][1157819] Updated weights for policy 0, policy_version 162828 (0.0007) [2024-09-30 02:58:08,647][1157819] Updated weights for policy 0, policy_version 162838 (0.0006) [2024-09-30 02:58:09,173][1157819] Updated weights for policy 0, policy_version 162848 (0.0006) [2024-09-30 02:58:09,762][1157819] Updated weights for policy 0, policy_version 162858 (0.0006) [2024-09-30 02:58:10,288][1157819] Updated weights for policy 0, policy_version 162868 (0.0007) [2024-09-30 02:58:10,466][1157520] Fps is (10 sec: 79871.3, 60 sec: 76868.1, 300 sec: 77518.5). Total num frames: 667119616. Throughput: 0: 19278.2. Samples: 156729740. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:58:10,466][1157520] Avg episode reward: [(0, '54.712')] [2024-09-30 02:58:10,803][1157819] Updated weights for policy 0, policy_version 162878 (0.0006) [2024-09-30 02:58:11,398][1157819] Updated weights for policy 0, policy_version 162888 (0.0006) [2024-09-30 02:58:11,902][1157819] Updated weights for policy 0, policy_version 162898 (0.0006) [2024-09-30 02:58:12,468][1157819] Updated weights for policy 0, policy_version 162908 (0.0006) [2024-09-30 02:58:12,854][1157736] Signal inference workers to stop experience collection... (11150 times) [2024-09-30 02:58:12,857][1157819] InferenceWorker_p0-w0: stopping experience collection (11150 times) [2024-09-30 02:58:12,860][1157736] Signal inference workers to resume experience collection... (11150 times) [2024-09-30 02:58:12,861][1157819] InferenceWorker_p0-w0: resuming experience collection (11150 times) [2024-09-30 02:58:13,015][1157819] Updated weights for policy 0, policy_version 162918 (0.0006) [2024-09-30 02:58:13,508][1157819] Updated weights for policy 0, policy_version 162928 (0.0006) [2024-09-30 02:58:14,119][1157819] Updated weights for policy 0, policy_version 162938 (0.0006) [2024-09-30 02:58:14,653][1157819] Updated weights for policy 0, policy_version 162948 (0.0006) [2024-09-30 02:58:15,242][1157819] Updated weights for policy 0, policy_version 162958 (0.0006) [2024-09-30 02:58:15,466][1157520] Fps is (10 sec: 75775.6, 60 sec: 76322.1, 300 sec: 77504.7). Total num frames: 667488256. Throughput: 0: 19442.8. Samples: 156842672. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:58:15,466][1157520] Avg episode reward: [(0, '54.760')] [2024-09-30 02:58:15,779][1157819] Updated weights for policy 0, policy_version 162968 (0.0006) [2024-09-30 02:58:16,378][1157819] Updated weights for policy 0, policy_version 162978 (0.0006) [2024-09-30 02:58:16,928][1157819] Updated weights for policy 0, policy_version 162988 (0.0006) [2024-09-30 02:58:17,475][1157819] Updated weights for policy 0, policy_version 162998 (0.0006) [2024-09-30 02:58:17,998][1157819] Updated weights for policy 0, policy_version 163008 (0.0006) [2024-09-30 02:58:18,551][1157819] Updated weights for policy 0, policy_version 163018 (0.0006) [2024-09-30 02:58:19,113][1157819] Updated weights for policy 0, policy_version 163028 (0.0006) [2024-09-30 02:58:19,636][1157819] Updated weights for policy 0, policy_version 163038 (0.0006) [2024-09-30 02:58:20,165][1157819] Updated weights for policy 0, policy_version 163048 (0.0006) [2024-09-30 02:58:20,466][1157520] Fps is (10 sec: 74548.1, 60 sec: 76458.8, 300 sec: 77407.5). Total num frames: 667865088. Throughput: 0: 19592.2. Samples: 156953812. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:58:20,466][1157520] Avg episode reward: [(0, '55.057')] [2024-09-30 02:58:20,735][1157819] Updated weights for policy 0, policy_version 163058 (0.0006) [2024-09-30 02:58:21,256][1157819] Updated weights for policy 0, policy_version 163068 (0.0006) [2024-09-30 02:58:21,781][1157819] Updated weights for policy 0, policy_version 163078 (0.0006) [2024-09-30 02:58:22,334][1157819] Updated weights for policy 0, policy_version 163088 (0.0006) [2024-09-30 02:58:22,868][1157819] Updated weights for policy 0, policy_version 163098 (0.0006) [2024-09-30 02:58:23,367][1157819] Updated weights for policy 0, policy_version 163108 (0.0006) [2024-09-30 02:58:23,862][1157819] Updated weights for policy 0, policy_version 163118 (0.0006) [2024-09-30 02:58:24,353][1157819] Updated weights for policy 0, policy_version 163128 (0.0006) [2024-09-30 02:58:24,854][1157819] Updated weights for policy 0, policy_version 163138 (0.0006) [2024-09-30 02:58:25,367][1157819] Updated weights for policy 0, policy_version 163148 (0.0006) [2024-09-30 02:58:25,466][1157520] Fps is (10 sec: 77005.4, 60 sec: 77073.2, 300 sec: 77435.2). Total num frames: 668258304. Throughput: 0: 19551.7. Samples: 157011008. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 02:58:25,466][1157520] Avg episode reward: [(0, '55.731')] [2024-09-30 02:58:25,867][1157819] Updated weights for policy 0, policy_version 163158 (0.0006) [2024-09-30 02:58:26,390][1157819] Updated weights for policy 0, policy_version 163168 (0.0006) [2024-09-30 02:58:26,940][1157819] Updated weights for policy 0, policy_version 163178 (0.0006) [2024-09-30 02:58:27,442][1157819] Updated weights for policy 0, policy_version 163188 (0.0006) [2024-09-30 02:58:27,945][1157819] Updated weights for policy 0, policy_version 163198 (0.0006) [2024-09-30 02:58:28,440][1157819] Updated weights for policy 0, policy_version 163208 (0.0006) [2024-09-30 02:58:28,941][1157819] Updated weights for policy 0, policy_version 163218 (0.0006) [2024-09-30 02:58:29,474][1157819] Updated weights for policy 0, policy_version 163228 (0.0006) [2024-09-30 02:58:29,962][1157819] Updated weights for policy 0, policy_version 163238 (0.0006) [2024-09-30 02:58:30,432][1157819] Updated weights for policy 0, policy_version 163248 (0.0006) [2024-09-30 02:58:30,466][1157520] Fps is (10 sec: 79871.4, 60 sec: 77960.6, 300 sec: 77560.2). Total num frames: 668663808. Throughput: 0: 19585.6. Samples: 157131896. Policy #0 lag: (min: 0.0, avg: 1.4, max: 5.0) [2024-09-30 02:58:30,466][1157520] Avg episode reward: [(0, '54.435')] [2024-09-30 02:58:30,923][1157819] Updated weights for policy 0, policy_version 163258 (0.0006) [2024-09-30 02:58:31,405][1157819] Updated weights for policy 0, policy_version 163268 (0.0006) [2024-09-30 02:58:31,904][1157819] Updated weights for policy 0, policy_version 163278 (0.0006) [2024-09-30 02:58:32,388][1157819] Updated weights for policy 0, policy_version 163288 (0.0006) [2024-09-30 02:58:32,837][1157819] Updated weights for policy 0, policy_version 163298 (0.0006) [2024-09-30 02:58:33,329][1157819] Updated weights for policy 0, policy_version 163308 (0.0006) [2024-09-30 02:58:33,843][1157819] Updated weights for policy 0, policy_version 163318 (0.0006) [2024-09-30 02:58:34,354][1157819] Updated weights for policy 0, policy_version 163328 (0.0006) [2024-09-30 02:58:34,859][1157819] Updated weights for policy 0, policy_version 163338 (0.0006) [2024-09-30 02:58:35,376][1157819] Updated weights for policy 0, policy_version 163348 (0.0006) [2024-09-30 02:58:35,466][1157520] Fps is (10 sec: 81920.2, 60 sec: 78916.6, 300 sec: 77685.2). Total num frames: 669077504. Throughput: 0: 19693.3. Samples: 157256444. Policy #0 lag: (min: 0.0, avg: 1.4, max: 5.0) [2024-09-30 02:58:35,466][1157520] Avg episode reward: [(0, '58.019')] [2024-09-30 02:58:35,878][1157819] Updated weights for policy 0, policy_version 163358 (0.0006) [2024-09-30 02:58:36,406][1157819] Updated weights for policy 0, policy_version 163368 (0.0006) [2024-09-30 02:58:36,922][1157819] Updated weights for policy 0, policy_version 163378 (0.0006) [2024-09-30 02:58:37,482][1157819] Updated weights for policy 0, policy_version 163388 (0.0006) [2024-09-30 02:58:37,980][1157819] Updated weights for policy 0, policy_version 163398 (0.0006) [2024-09-30 02:58:38,472][1157819] Updated weights for policy 0, policy_version 163408 (0.0006) [2024-09-30 02:58:38,967][1157819] Updated weights for policy 0, policy_version 163418 (0.0006) [2024-09-30 02:58:39,514][1157819] Updated weights for policy 0, policy_version 163428 (0.0006) [2024-09-30 02:58:40,050][1157819] Updated weights for policy 0, policy_version 163438 (0.0006) [2024-09-30 02:58:40,466][1157520] Fps is (10 sec: 81101.3, 60 sec: 78984.5, 300 sec: 77782.4). Total num frames: 669474816. Throughput: 0: 19690.9. Samples: 157315664. Policy #0 lag: (min: 0.0, avg: 1.4, max: 5.0) [2024-09-30 02:58:40,466][1157520] Avg episode reward: [(0, '55.449')] [2024-09-30 02:58:40,574][1157819] Updated weights for policy 0, policy_version 163448 (0.0006) [2024-09-30 02:58:41,070][1157819] Updated weights for policy 0, policy_version 163458 (0.0006) [2024-09-30 02:58:41,571][1157819] Updated weights for policy 0, policy_version 163468 (0.0006) [2024-09-30 02:58:42,070][1157819] Updated weights for policy 0, policy_version 163478 (0.0006) [2024-09-30 02:58:42,609][1157819] Updated weights for policy 0, policy_version 163488 (0.0006) [2024-09-30 02:58:43,110][1157819] Updated weights for policy 0, policy_version 163498 (0.0006) [2024-09-30 02:58:43,642][1157819] Updated weights for policy 0, policy_version 163508 (0.0006) [2024-09-30 02:58:44,166][1157819] Updated weights for policy 0, policy_version 163518 (0.0006) [2024-09-30 02:58:44,655][1157819] Updated weights for policy 0, policy_version 163528 (0.0006) [2024-09-30 02:58:45,184][1157819] Updated weights for policy 0, policy_version 163538 (0.0006) [2024-09-30 02:58:45,466][1157520] Fps is (10 sec: 79462.2, 60 sec: 78984.7, 300 sec: 77768.5). Total num frames: 669872128. Throughput: 0: 19699.2. Samples: 157435008. Policy #0 lag: (min: 0.0, avg: 1.4, max: 5.0) [2024-09-30 02:58:45,466][1157520] Avg episode reward: [(0, '56.402')] [2024-09-30 02:58:45,714][1157819] Updated weights for policy 0, policy_version 163548 (0.0006) [2024-09-30 02:58:46,249][1157819] Updated weights for policy 0, policy_version 163558 (0.0006) [2024-09-30 02:58:46,753][1157819] Updated weights for policy 0, policy_version 163568 (0.0006) [2024-09-30 02:58:47,251][1157819] Updated weights for policy 0, policy_version 163578 (0.0006) [2024-09-30 02:58:47,775][1157819] Updated weights for policy 0, policy_version 163588 (0.0006) [2024-09-30 02:58:48,294][1157819] Updated weights for policy 0, policy_version 163598 (0.0006) [2024-09-30 02:58:48,816][1157819] Updated weights for policy 0, policy_version 163608 (0.0006) [2024-09-30 02:58:49,338][1157819] Updated weights for policy 0, policy_version 163618 (0.0006) [2024-09-30 02:58:49,847][1157819] Updated weights for policy 0, policy_version 163628 (0.0006) [2024-09-30 02:58:50,348][1157819] Updated weights for policy 0, policy_version 163638 (0.0006) [2024-09-30 02:58:50,466][1157520] Fps is (10 sec: 79052.9, 60 sec: 78916.4, 300 sec: 77726.8). Total num frames: 670265344. Throughput: 0: 19655.9. Samples: 157553784. Policy #0 lag: (min: 0.0, avg: 1.4, max: 5.0) [2024-09-30 02:58:50,466][1157520] Avg episode reward: [(0, '55.165')] [2024-09-30 02:58:50,877][1157819] Updated weights for policy 0, policy_version 163648 (0.0006) [2024-09-30 02:58:51,403][1157819] Updated weights for policy 0, policy_version 163658 (0.0006) [2024-09-30 02:58:51,937][1157819] Updated weights for policy 0, policy_version 163668 (0.0006) [2024-09-30 02:58:52,455][1157819] Updated weights for policy 0, policy_version 163678 (0.0006) [2024-09-30 02:58:52,958][1157819] Updated weights for policy 0, policy_version 163688 (0.0006) [2024-09-30 02:58:53,462][1157819] Updated weights for policy 0, policy_version 163698 (0.0006) [2024-09-30 02:58:53,956][1157819] Updated weights for policy 0, policy_version 163708 (0.0006) [2024-09-30 02:58:54,472][1157819] Updated weights for policy 0, policy_version 163718 (0.0006) [2024-09-30 02:58:54,955][1157819] Updated weights for policy 0, policy_version 163728 (0.0006) [2024-09-30 02:58:55,441][1157819] Updated weights for policy 0, policy_version 163738 (0.0006) [2024-09-30 02:58:55,466][1157520] Fps is (10 sec: 79872.1, 60 sec: 79053.1, 300 sec: 77685.2). Total num frames: 670670848. Throughput: 0: 19630.4. Samples: 157613108. Policy #0 lag: (min: 0.0, avg: 1.4, max: 5.0) [2024-09-30 02:58:55,466][1157520] Avg episode reward: [(0, '53.563')] [2024-09-30 02:58:55,940][1157819] Updated weights for policy 0, policy_version 163748 (0.0006) [2024-09-30 02:58:56,441][1157819] Updated weights for policy 0, policy_version 163758 (0.0006) [2024-09-30 02:58:56,934][1157819] Updated weights for policy 0, policy_version 163768 (0.0006) [2024-09-30 02:58:57,420][1157819] Updated weights for policy 0, policy_version 163778 (0.0006) [2024-09-30 02:58:57,926][1157819] Updated weights for policy 0, policy_version 163788 (0.0006) [2024-09-30 02:58:58,421][1157819] Updated weights for policy 0, policy_version 163798 (0.0006) [2024-09-30 02:58:58,902][1157819] Updated weights for policy 0, policy_version 163808 (0.0006) [2024-09-30 02:58:59,410][1157819] Updated weights for policy 0, policy_version 163818 (0.0006) [2024-09-30 02:58:59,935][1157819] Updated weights for policy 0, policy_version 163828 (0.0006) [2024-09-30 02:59:00,436][1157819] Updated weights for policy 0, policy_version 163838 (0.0006) [2024-09-30 02:59:00,466][1157520] Fps is (10 sec: 81509.1, 60 sec: 79325.7, 300 sec: 77657.4). Total num frames: 671080448. Throughput: 0: 19871.3. Samples: 157736880. Policy #0 lag: (min: 0.0, avg: 1.4, max: 5.0) [2024-09-30 02:59:00,466][1157520] Avg episode reward: [(0, '55.323')] [2024-09-30 02:59:00,948][1157819] Updated weights for policy 0, policy_version 163848 (0.0006) [2024-09-30 02:59:01,216][1157736] Signal inference workers to stop experience collection... (11200 times) [2024-09-30 02:59:01,216][1157736] Signal inference workers to resume experience collection... (11200 times) [2024-09-30 02:59:01,222][1157819] InferenceWorker_p0-w0: stopping experience collection (11200 times) [2024-09-30 02:59:01,222][1157819] InferenceWorker_p0-w0: resuming experience collection (11200 times) [2024-09-30 02:59:01,511][1157819] Updated weights for policy 0, policy_version 163858 (0.0006) [2024-09-30 02:59:02,033][1157819] Updated weights for policy 0, policy_version 163868 (0.0006) [2024-09-30 02:59:02,545][1157819] Updated weights for policy 0, policy_version 163878 (0.0006) [2024-09-30 02:59:03,114][1157819] Updated weights for policy 0, policy_version 163888 (0.0006) [2024-09-30 02:59:03,643][1157819] Updated weights for policy 0, policy_version 163898 (0.0006) [2024-09-30 02:59:04,199][1157819] Updated weights for policy 0, policy_version 163908 (0.0006) [2024-09-30 02:59:04,761][1157819] Updated weights for policy 0, policy_version 163918 (0.0006) [2024-09-30 02:59:05,292][1157819] Updated weights for policy 0, policy_version 163928 (0.0006) [2024-09-30 02:59:05,466][1157520] Fps is (10 sec: 78642.3, 60 sec: 78779.6, 300 sec: 77574.0). Total num frames: 671457280. Throughput: 0: 19969.7. Samples: 157852452. Policy #0 lag: (min: 0.0, avg: 1.4, max: 5.0) [2024-09-30 02:59:05,466][1157520] Avg episode reward: [(0, '57.222')] [2024-09-30 02:59:05,865][1157819] Updated weights for policy 0, policy_version 163938 (0.0006) [2024-09-30 02:59:06,422][1157819] Updated weights for policy 0, policy_version 163948 (0.0006) [2024-09-30 02:59:06,947][1157819] Updated weights for policy 0, policy_version 163958 (0.0006) [2024-09-30 02:59:07,522][1157819] Updated weights for policy 0, policy_version 163968 (0.0006) [2024-09-30 02:59:08,062][1157819] Updated weights for policy 0, policy_version 163978 (0.0006) [2024-09-30 02:59:08,627][1157819] Updated weights for policy 0, policy_version 163988 (0.0006) [2024-09-30 02:59:09,207][1157819] Updated weights for policy 0, policy_version 163998 (0.0007) [2024-09-30 02:59:09,767][1157819] Updated weights for policy 0, policy_version 164008 (0.0006) [2024-09-30 02:59:10,352][1157819] Updated weights for policy 0, policy_version 164018 (0.0006) [2024-09-30 02:59:10,466][1157520] Fps is (10 sec: 74547.1, 60 sec: 78438.3, 300 sec: 77435.2). Total num frames: 671825920. Throughput: 0: 19932.9. Samples: 157907992. Policy #0 lag: (min: 0.0, avg: 1.4, max: 5.0) [2024-09-30 02:59:10,466][1157520] Avg episode reward: [(0, '57.473')] [2024-09-30 02:59:10,875][1157819] Updated weights for policy 0, policy_version 164028 (0.0006) [2024-09-30 02:59:11,449][1157819] Updated weights for policy 0, policy_version 164038 (0.0006) [2024-09-30 02:59:12,008][1157819] Updated weights for policy 0, policy_version 164048 (0.0006) [2024-09-30 02:59:12,557][1157819] Updated weights for policy 0, policy_version 164058 (0.0006) [2024-09-30 02:59:13,109][1157819] Updated weights for policy 0, policy_version 164068 (0.0006) [2024-09-30 02:59:13,681][1157819] Updated weights for policy 0, policy_version 164078 (0.0006) [2024-09-30 02:59:14,194][1157819] Updated weights for policy 0, policy_version 164088 (0.0006) [2024-09-30 02:59:14,727][1157819] Updated weights for policy 0, policy_version 164098 (0.0006) [2024-09-30 02:59:15,302][1157819] Updated weights for policy 0, policy_version 164108 (0.0006) [2024-09-30 02:59:15,466][1157520] Fps is (10 sec: 73727.2, 60 sec: 78438.2, 300 sec: 77310.2). Total num frames: 672194560. Throughput: 0: 19696.5. Samples: 158018240. Policy #0 lag: (min: 0.0, avg: 1.4, max: 5.0) [2024-09-30 02:59:15,466][1157520] Avg episode reward: [(0, '56.105')] [2024-09-30 02:59:15,833][1157819] Updated weights for policy 0, policy_version 164118 (0.0006) [2024-09-30 02:59:16,422][1157819] Updated weights for policy 0, policy_version 164128 (0.0006) [2024-09-30 02:59:16,945][1157819] Updated weights for policy 0, policy_version 164138 (0.0006) [2024-09-30 02:59:17,527][1157819] Updated weights for policy 0, policy_version 164148 (0.0006) [2024-09-30 02:59:18,058][1157819] Updated weights for policy 0, policy_version 164158 (0.0006) [2024-09-30 02:59:18,597][1157819] Updated weights for policy 0, policy_version 164168 (0.0006) [2024-09-30 02:59:19,181][1157819] Updated weights for policy 0, policy_version 164178 (0.0006) [2024-09-30 02:59:19,698][1157819] Updated weights for policy 0, policy_version 164188 (0.0006) [2024-09-30 02:59:20,268][1157819] Updated weights for policy 0, policy_version 164198 (0.0006) [2024-09-30 02:59:20,466][1157520] Fps is (10 sec: 74138.8, 60 sec: 78370.1, 300 sec: 77240.8). Total num frames: 672567296. Throughput: 0: 19407.0. Samples: 158129760. Policy #0 lag: (min: 0.0, avg: 1.4, max: 5.0) [2024-09-30 02:59:20,466][1157520] Avg episode reward: [(0, '55.108')] [2024-09-30 02:59:20,801][1157819] Updated weights for policy 0, policy_version 164208 (0.0006) [2024-09-30 02:59:21,330][1157819] Updated weights for policy 0, policy_version 164218 (0.0006) [2024-09-30 02:59:21,838][1157819] Updated weights for policy 0, policy_version 164228 (0.0006) [2024-09-30 02:59:22,339][1157819] Updated weights for policy 0, policy_version 164238 (0.0006) [2024-09-30 02:59:22,859][1157819] Updated weights for policy 0, policy_version 164248 (0.0006) [2024-09-30 02:59:23,389][1157819] Updated weights for policy 0, policy_version 164258 (0.0006) [2024-09-30 02:59:23,894][1157819] Updated weights for policy 0, policy_version 164268 (0.0006) [2024-09-30 02:59:24,414][1157819] Updated weights for policy 0, policy_version 164278 (0.0006) [2024-09-30 02:59:24,922][1157819] Updated weights for policy 0, policy_version 164288 (0.0006) [2024-09-30 02:59:25,449][1157819] Updated weights for policy 0, policy_version 164298 (0.0006) [2024-09-30 02:59:25,466][1157520] Fps is (10 sec: 77006.5, 60 sec: 78438.4, 300 sec: 77240.8). Total num frames: 672964608. Throughput: 0: 19391.5. Samples: 158188284. Policy #0 lag: (min: 0.0, avg: 1.4, max: 5.0) [2024-09-30 02:59:25,466][1157520] Avg episode reward: [(0, '55.497')] [2024-09-30 02:59:25,961][1157819] Updated weights for policy 0, policy_version 164308 (0.0006) [2024-09-30 02:59:26,466][1157819] Updated weights for policy 0, policy_version 164318 (0.0006) [2024-09-30 02:59:26,998][1157819] Updated weights for policy 0, policy_version 164328 (0.0006) [2024-09-30 02:59:27,519][1157819] Updated weights for policy 0, policy_version 164338 (0.0006) [2024-09-30 02:59:28,039][1157819] Updated weights for policy 0, policy_version 164348 (0.0006) [2024-09-30 02:59:28,558][1157819] Updated weights for policy 0, policy_version 164358 (0.0006) [2024-09-30 02:59:29,057][1157819] Updated weights for policy 0, policy_version 164368 (0.0006) [2024-09-30 02:59:29,557][1157819] Updated weights for policy 0, policy_version 164378 (0.0006) [2024-09-30 02:59:30,109][1157819] Updated weights for policy 0, policy_version 164388 (0.0006) [2024-09-30 02:59:30,466][1157520] Fps is (10 sec: 79461.7, 60 sec: 78301.8, 300 sec: 77254.7). Total num frames: 673361920. Throughput: 0: 19383.0. Samples: 158307244. Policy #0 lag: (min: 0.0, avg: 1.4, max: 5.0) [2024-09-30 02:59:30,466][1157520] Avg episode reward: [(0, '53.926')] [2024-09-30 02:59:30,604][1157819] Updated weights for policy 0, policy_version 164398 (0.0006) [2024-09-30 02:59:31,111][1157819] Updated weights for policy 0, policy_version 164408 (0.0006) [2024-09-30 02:59:31,617][1157819] Updated weights for policy 0, policy_version 164418 (0.0006) [2024-09-30 02:59:32,161][1157819] Updated weights for policy 0, policy_version 164428 (0.0006) [2024-09-30 02:59:32,700][1157819] Updated weights for policy 0, policy_version 164438 (0.0007) [2024-09-30 02:59:33,244][1157819] Updated weights for policy 0, policy_version 164448 (0.0006) [2024-09-30 02:59:33,788][1157819] Updated weights for policy 0, policy_version 164458 (0.0006) [2024-09-30 02:59:34,353][1157819] Updated weights for policy 0, policy_version 164468 (0.0006) [2024-09-30 02:59:34,882][1157819] Updated weights for policy 0, policy_version 164478 (0.0006) [2024-09-30 02:59:35,445][1157819] Updated weights for policy 0, policy_version 164488 (0.0006) [2024-09-30 02:59:35,466][1157520] Fps is (10 sec: 77822.7, 60 sec: 77755.5, 300 sec: 77157.5). Total num frames: 673742848. Throughput: 0: 19325.8. Samples: 158423448. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 02:59:35,466][1157520] Avg episode reward: [(0, '57.357')] [2024-09-30 02:59:35,984][1157819] Updated weights for policy 0, policy_version 164498 (0.0006) [2024-09-30 02:59:36,519][1157819] Updated weights for policy 0, policy_version 164508 (0.0006) [2024-09-30 02:59:37,075][1157819] Updated weights for policy 0, policy_version 164518 (0.0006) [2024-09-30 02:59:37,614][1157819] Updated weights for policy 0, policy_version 164528 (0.0006) [2024-09-30 02:59:38,173][1157819] Updated weights for policy 0, policy_version 164538 (0.0006) [2024-09-30 02:59:38,795][1157819] Updated weights for policy 0, policy_version 164548 (0.0007) [2024-09-30 02:59:39,396][1157819] Updated weights for policy 0, policy_version 164558 (0.0006) [2024-09-30 02:59:39,986][1157819] Updated weights for policy 0, policy_version 164568 (0.0006) [2024-09-30 02:59:40,466][1157520] Fps is (10 sec: 74137.3, 60 sec: 77141.2, 300 sec: 76990.9). Total num frames: 674103296. Throughput: 0: 19259.9. Samples: 158479804. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 02:59:40,466][1157520] Avg episode reward: [(0, '55.180')] [2024-09-30 02:59:40,584][1157819] Updated weights for policy 0, policy_version 164578 (0.0006) [2024-09-30 02:59:41,165][1157819] Updated weights for policy 0, policy_version 164588 (0.0006) [2024-09-30 02:59:41,755][1157819] Updated weights for policy 0, policy_version 164598 (0.0006) [2024-09-30 02:59:42,351][1157819] Updated weights for policy 0, policy_version 164608 (0.0006) [2024-09-30 02:59:42,925][1157819] Updated weights for policy 0, policy_version 164618 (0.0006) [2024-09-30 02:59:43,539][1157819] Updated weights for policy 0, policy_version 164628 (0.0006) [2024-09-30 02:59:44,081][1157819] Updated weights for policy 0, policy_version 164638 (0.0006) [2024-09-30 02:59:44,635][1157819] Updated weights for policy 0, policy_version 164648 (0.0006) [2024-09-30 02:59:45,176][1157819] Updated weights for policy 0, policy_version 164658 (0.0006) [2024-09-30 02:59:45,466][1157520] Fps is (10 sec: 71680.8, 60 sec: 76458.6, 300 sec: 76796.5). Total num frames: 674459648. Throughput: 0: 18829.0. Samples: 158584184. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 02:59:45,466][1157520] Avg episode reward: [(0, '56.321')] [2024-09-30 02:59:45,706][1157819] Updated weights for policy 0, policy_version 164668 (0.0006) [2024-09-30 02:59:46,236][1157819] Updated weights for policy 0, policy_version 164678 (0.0006) [2024-09-30 02:59:46,787][1157819] Updated weights for policy 0, policy_version 164688 (0.0006) [2024-09-30 02:59:47,304][1157819] Updated weights for policy 0, policy_version 164698 (0.0006) [2024-09-30 02:59:47,809][1157819] Updated weights for policy 0, policy_version 164708 (0.0006) [2024-09-30 02:59:48,342][1157819] Updated weights for policy 0, policy_version 164718 (0.0006) [2024-09-30 02:59:48,922][1157819] Updated weights for policy 0, policy_version 164728 (0.0006) [2024-09-30 02:59:49,466][1157819] Updated weights for policy 0, policy_version 164738 (0.0006) [2024-09-30 02:59:50,026][1157819] Updated weights for policy 0, policy_version 164748 (0.0006) [2024-09-30 02:59:50,466][1157520] Fps is (10 sec: 73729.0, 60 sec: 76253.9, 300 sec: 76657.7). Total num frames: 674840576. Throughput: 0: 18780.9. Samples: 158697588. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 02:59:50,466][1157520] Avg episode reward: [(0, '53.650')] [2024-09-30 02:59:50,557][1157819] Updated weights for policy 0, policy_version 164758 (0.0006) [2024-09-30 02:59:51,080][1157819] Updated weights for policy 0, policy_version 164768 (0.0006) [2024-09-30 02:59:51,619][1157819] Updated weights for policy 0, policy_version 164778 (0.0006) [2024-09-30 02:59:52,108][1157736] Signal inference workers to stop experience collection... (11250 times) [2024-09-30 02:59:52,109][1157736] Signal inference workers to resume experience collection... (11250 times) [2024-09-30 02:59:52,115][1157819] InferenceWorker_p0-w0: stopping experience collection (11250 times) [2024-09-30 02:59:52,115][1157819] InferenceWorker_p0-w0: resuming experience collection (11250 times) [2024-09-30 02:59:52,123][1157819] Updated weights for policy 0, policy_version 164788 (0.0006) [2024-09-30 02:59:52,648][1157819] Updated weights for policy 0, policy_version 164798 (0.0006) [2024-09-30 02:59:53,172][1157819] Updated weights for policy 0, policy_version 164808 (0.0006) [2024-09-30 02:59:53,720][1157819] Updated weights for policy 0, policy_version 164818 (0.0006) [2024-09-30 02:59:54,223][1157819] Updated weights for policy 0, policy_version 164828 (0.0006) [2024-09-30 02:59:54,749][1157819] Updated weights for policy 0, policy_version 164838 (0.0006) [2024-09-30 02:59:55,243][1157819] Updated weights for policy 0, policy_version 164848 (0.0006) [2024-09-30 02:59:55,466][1157520] Fps is (10 sec: 77415.0, 60 sec: 76049.1, 300 sec: 76574.4). Total num frames: 675233792. Throughput: 0: 18835.3. Samples: 158755576. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 02:59:55,466][1157520] Avg episode reward: [(0, '54.846')] [2024-09-30 02:59:55,764][1157819] Updated weights for policy 0, policy_version 164858 (0.0006) [2024-09-30 02:59:56,299][1157819] Updated weights for policy 0, policy_version 164868 (0.0006) [2024-09-30 02:59:56,816][1157819] Updated weights for policy 0, policy_version 164878 (0.0006) [2024-09-30 02:59:57,347][1157819] Updated weights for policy 0, policy_version 164888 (0.0006) [2024-09-30 02:59:57,880][1157819] Updated weights for policy 0, policy_version 164898 (0.0006) [2024-09-30 02:59:58,424][1157819] Updated weights for policy 0, policy_version 164908 (0.0006) [2024-09-30 02:59:58,991][1157819] Updated weights for policy 0, policy_version 164918 (0.0006) [2024-09-30 02:59:59,508][1157819] Updated weights for policy 0, policy_version 164928 (0.0006) [2024-09-30 03:00:00,091][1157819] Updated weights for policy 0, policy_version 164938 (0.0006) [2024-09-30 03:00:00,466][1157520] Fps is (10 sec: 77004.9, 60 sec: 75503.1, 300 sec: 76380.0). Total num frames: 675610624. Throughput: 0: 18981.3. Samples: 158872392. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 03:00:00,466][1157520] Avg episode reward: [(0, '52.669')] [2024-09-30 03:00:00,643][1157819] Updated weights for policy 0, policy_version 164948 (0.0006) [2024-09-30 03:00:01,212][1157819] Updated weights for policy 0, policy_version 164958 (0.0006) [2024-09-30 03:00:01,779][1157819] Updated weights for policy 0, policy_version 164968 (0.0006) [2024-09-30 03:00:02,267][1157819] Updated weights for policy 0, policy_version 164978 (0.0006) [2024-09-30 03:00:02,813][1157819] Updated weights for policy 0, policy_version 164988 (0.0006) [2024-09-30 03:00:03,302][1157819] Updated weights for policy 0, policy_version 164998 (0.0006) [2024-09-30 03:00:03,848][1157819] Updated weights for policy 0, policy_version 165008 (0.0006) [2024-09-30 03:00:04,365][1157819] Updated weights for policy 0, policy_version 165018 (0.0006) [2024-09-30 03:00:04,862][1157819] Updated weights for policy 0, policy_version 165028 (0.0006) [2024-09-30 03:00:05,354][1157819] Updated weights for policy 0, policy_version 165038 (0.0006) [2024-09-30 03:00:05,466][1157520] Fps is (10 sec: 77004.4, 60 sec: 75776.1, 300 sec: 76282.8). Total num frames: 676003840. Throughput: 0: 19056.2. Samples: 158987292. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 03:00:05,466][1157520] Avg episode reward: [(0, '55.566')] [2024-09-30 03:00:05,469][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000165040_676003840.pth... [2024-09-30 03:00:05,515][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000160567_657682432.pth [2024-09-30 03:00:05,915][1157819] Updated weights for policy 0, policy_version 165048 (0.0006) [2024-09-30 03:00:06,492][1157819] Updated weights for policy 0, policy_version 165058 (0.0006) [2024-09-30 03:00:07,081][1157819] Updated weights for policy 0, policy_version 165068 (0.0006) [2024-09-30 03:00:07,670][1157819] Updated weights for policy 0, policy_version 165078 (0.0006) [2024-09-30 03:00:08,261][1157819] Updated weights for policy 0, policy_version 165088 (0.0006) [2024-09-30 03:00:08,781][1157819] Updated weights for policy 0, policy_version 165098 (0.0006) [2024-09-30 03:00:09,228][1157819] Updated weights for policy 0, policy_version 165108 (0.0006) [2024-09-30 03:00:09,696][1157819] Updated weights for policy 0, policy_version 165118 (0.0006) [2024-09-30 03:00:10,126][1157819] Updated weights for policy 0, policy_version 165128 (0.0006) [2024-09-30 03:00:10,466][1157520] Fps is (10 sec: 78233.3, 60 sec: 76117.5, 300 sec: 76227.3). Total num frames: 676392960. Throughput: 0: 18971.6. Samples: 159042004. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 03:00:10,466][1157520] Avg episode reward: [(0, '55.686')] [2024-09-30 03:00:10,576][1157819] Updated weights for policy 0, policy_version 165138 (0.0006) [2024-09-30 03:00:11,021][1157819] Updated weights for policy 0, policy_version 165148 (0.0006) [2024-09-30 03:00:11,462][1157819] Updated weights for policy 0, policy_version 165158 (0.0006) [2024-09-30 03:00:11,945][1157819] Updated weights for policy 0, policy_version 165168 (0.0006) [2024-09-30 03:00:12,421][1157819] Updated weights for policy 0, policy_version 165178 (0.0007) [2024-09-30 03:00:12,921][1157819] Updated weights for policy 0, policy_version 165188 (0.0006) [2024-09-30 03:00:13,435][1157819] Updated weights for policy 0, policy_version 165198 (0.0006) [2024-09-30 03:00:13,935][1157819] Updated weights for policy 0, policy_version 165208 (0.0006) [2024-09-30 03:00:14,425][1157819] Updated weights for policy 0, policy_version 165218 (0.0006) [2024-09-30 03:00:14,941][1157819] Updated weights for policy 0, policy_version 165228 (0.0006) [2024-09-30 03:00:15,441][1157819] Updated weights for policy 0, policy_version 165238 (0.0007) [2024-09-30 03:00:15,466][1157520] Fps is (10 sec: 81099.8, 60 sec: 77004.9, 300 sec: 76463.3). Total num frames: 676814848. Throughput: 0: 19172.9. Samples: 159170028. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 03:00:15,466][1157520] Avg episode reward: [(0, '56.234')] [2024-09-30 03:00:15,987][1157819] Updated weights for policy 0, policy_version 165248 (0.0006) [2024-09-30 03:00:16,542][1157819] Updated weights for policy 0, policy_version 165258 (0.0007) [2024-09-30 03:00:17,105][1157819] Updated weights for policy 0, policy_version 165268 (0.0007) [2024-09-30 03:00:17,720][1157819] Updated weights for policy 0, policy_version 165278 (0.0006) [2024-09-30 03:00:18,349][1157819] Updated weights for policy 0, policy_version 165288 (0.0007) [2024-09-30 03:00:18,935][1157819] Updated weights for policy 0, policy_version 165298 (0.0006) [2024-09-30 03:00:19,527][1157819] Updated weights for policy 0, policy_version 165308 (0.0006) [2024-09-30 03:00:20,115][1157819] Updated weights for policy 0, policy_version 165318 (0.0006) [2024-09-30 03:00:20,466][1157520] Fps is (10 sec: 77413.2, 60 sec: 76663.2, 300 sec: 76477.2). Total num frames: 677167104. Throughput: 0: 19022.2. Samples: 159279448. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 03:00:20,466][1157520] Avg episode reward: [(0, '57.459')] [2024-09-30 03:00:20,708][1157819] Updated weights for policy 0, policy_version 165328 (0.0006) [2024-09-30 03:00:21,288][1157819] Updated weights for policy 0, policy_version 165338 (0.0007) [2024-09-30 03:00:21,908][1157819] Updated weights for policy 0, policy_version 165348 (0.0006) [2024-09-30 03:00:22,519][1157819] Updated weights for policy 0, policy_version 165358 (0.0006) [2024-09-30 03:00:23,123][1157819] Updated weights for policy 0, policy_version 165368 (0.0006) [2024-09-30 03:00:23,725][1157819] Updated weights for policy 0, policy_version 165378 (0.0006) [2024-09-30 03:00:24,289][1157819] Updated weights for policy 0, policy_version 165388 (0.0006) [2024-09-30 03:00:24,855][1157819] Updated weights for policy 0, policy_version 165398 (0.0006) [2024-09-30 03:00:25,466][1157520] Fps is (10 sec: 69223.4, 60 sec: 75707.7, 300 sec: 76380.0). Total num frames: 677507072. Throughput: 0: 18918.5. Samples: 159331136. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 03:00:25,466][1157520] Avg episode reward: [(0, '55.282')] [2024-09-30 03:00:25,468][1157819] Updated weights for policy 0, policy_version 165408 (0.0006) [2024-09-30 03:00:26,105][1157819] Updated weights for policy 0, policy_version 165418 (0.0006) [2024-09-30 03:00:26,651][1157819] Updated weights for policy 0, policy_version 165428 (0.0006) [2024-09-30 03:00:27,140][1157736] Signal inference workers to stop experience collection... (11300 times) [2024-09-30 03:00:27,142][1157736] Signal inference workers to resume experience collection... (11300 times) [2024-09-30 03:00:27,143][1157819] InferenceWorker_p0-w0: stopping experience collection (11300 times) [2024-09-30 03:00:27,146][1157819] InferenceWorker_p0-w0: resuming experience collection (11300 times) [2024-09-30 03:00:27,265][1157819] Updated weights for policy 0, policy_version 165438 (0.0006) [2024-09-30 03:00:27,788][1157819] Updated weights for policy 0, policy_version 165448 (0.0006) [2024-09-30 03:00:28,312][1157819] Updated weights for policy 0, policy_version 165458 (0.0006) [2024-09-30 03:00:28,838][1157819] Updated weights for policy 0, policy_version 165468 (0.0006) [2024-09-30 03:00:29,414][1157819] Updated weights for policy 0, policy_version 165478 (0.0006) [2024-09-30 03:00:29,924][1157819] Updated weights for policy 0, policy_version 165488 (0.0006) [2024-09-30 03:00:30,442][1157819] Updated weights for policy 0, policy_version 165498 (0.0006) [2024-09-30 03:00:30,466][1157520] Fps is (10 sec: 71271.7, 60 sec: 75298.3, 300 sec: 76380.0). Total num frames: 677879808. Throughput: 0: 18973.1. Samples: 159437972. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 03:00:30,466][1157520] Avg episode reward: [(0, '55.731')] [2024-09-30 03:00:30,953][1157819] Updated weights for policy 0, policy_version 165508 (0.0006) [2024-09-30 03:00:31,484][1157819] Updated weights for policy 0, policy_version 165518 (0.0006) [2024-09-30 03:00:32,018][1157819] Updated weights for policy 0, policy_version 165528 (0.0006) [2024-09-30 03:00:32,539][1157819] Updated weights for policy 0, policy_version 165538 (0.0006) [2024-09-30 03:00:33,123][1157819] Updated weights for policy 0, policy_version 165548 (0.0006) [2024-09-30 03:00:33,661][1157819] Updated weights for policy 0, policy_version 165558 (0.0006) [2024-09-30 03:00:34,176][1157819] Updated weights for policy 0, policy_version 165568 (0.0006) [2024-09-30 03:00:34,733][1157819] Updated weights for policy 0, policy_version 165578 (0.0006) [2024-09-30 03:00:35,269][1157819] Updated weights for policy 0, policy_version 165588 (0.0006) [2024-09-30 03:00:35,466][1157520] Fps is (10 sec: 75773.6, 60 sec: 75366.2, 300 sec: 76379.9). Total num frames: 678264832. Throughput: 0: 19012.2. Samples: 159553144. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 03:00:35,466][1157520] Avg episode reward: [(0, '53.928')] [2024-09-30 03:00:35,772][1157819] Updated weights for policy 0, policy_version 165598 (0.0006) [2024-09-30 03:00:36,269][1157819] Updated weights for policy 0, policy_version 165608 (0.0006) [2024-09-30 03:00:36,767][1157819] Updated weights for policy 0, policy_version 165618 (0.0006) [2024-09-30 03:00:37,316][1157819] Updated weights for policy 0, policy_version 165628 (0.0006) [2024-09-30 03:00:37,849][1157819] Updated weights for policy 0, policy_version 165638 (0.0006) [2024-09-30 03:00:38,377][1157819] Updated weights for policy 0, policy_version 165648 (0.0006) [2024-09-30 03:00:38,937][1157819] Updated weights for policy 0, policy_version 165658 (0.0006) [2024-09-30 03:00:39,463][1157819] Updated weights for policy 0, policy_version 165668 (0.0006) [2024-09-30 03:00:39,960][1157819] Updated weights for policy 0, policy_version 165678 (0.0006) [2024-09-30 03:00:40,460][1157819] Updated weights for policy 0, policy_version 165688 (0.0006) [2024-09-30 03:00:40,466][1157520] Fps is (10 sec: 77823.7, 60 sec: 75912.7, 300 sec: 76449.4). Total num frames: 678658048. Throughput: 0: 19036.1. Samples: 159612200. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 03:00:40,466][1157520] Avg episode reward: [(0, '55.015')] [2024-09-30 03:00:40,966][1157819] Updated weights for policy 0, policy_version 165698 (0.0006) [2024-09-30 03:00:41,480][1157819] Updated weights for policy 0, policy_version 165708 (0.0006) [2024-09-30 03:00:41,997][1157819] Updated weights for policy 0, policy_version 165718 (0.0006) [2024-09-30 03:00:42,531][1157819] Updated weights for policy 0, policy_version 165728 (0.0006) [2024-09-30 03:00:43,052][1157819] Updated weights for policy 0, policy_version 165738 (0.0006) [2024-09-30 03:00:43,552][1157819] Updated weights for policy 0, policy_version 165748 (0.0006) [2024-09-30 03:00:44,077][1157819] Updated weights for policy 0, policy_version 165758 (0.0006) [2024-09-30 03:00:44,603][1157819] Updated weights for policy 0, policy_version 165768 (0.0006) [2024-09-30 03:00:45,161][1157819] Updated weights for policy 0, policy_version 165778 (0.0006) [2024-09-30 03:00:45,466][1157520] Fps is (10 sec: 78645.6, 60 sec: 76527.0, 300 sec: 76477.2). Total num frames: 679051264. Throughput: 0: 19061.1. Samples: 159730144. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 03:00:45,466][1157520] Avg episode reward: [(0, '56.749')] [2024-09-30 03:00:45,669][1157819] Updated weights for policy 0, policy_version 165788 (0.0006) [2024-09-30 03:00:46,168][1157819] Updated weights for policy 0, policy_version 165798 (0.0006) [2024-09-30 03:00:46,692][1157819] Updated weights for policy 0, policy_version 165808 (0.0006) [2024-09-30 03:00:47,231][1157819] Updated weights for policy 0, policy_version 165818 (0.0006) [2024-09-30 03:00:47,762][1157819] Updated weights for policy 0, policy_version 165828 (0.0006) [2024-09-30 03:00:48,287][1157819] Updated weights for policy 0, policy_version 165838 (0.0006) [2024-09-30 03:00:48,775][1157819] Updated weights for policy 0, policy_version 165848 (0.0006) [2024-09-30 03:00:49,281][1157819] Updated weights for policy 0, policy_version 165858 (0.0006) [2024-09-30 03:00:49,732][1157819] Updated weights for policy 0, policy_version 165868 (0.0006) [2024-09-30 03:00:50,248][1157819] Updated weights for policy 0, policy_version 165878 (0.0006) [2024-09-30 03:00:50,466][1157520] Fps is (10 sec: 79462.7, 60 sec: 76868.3, 300 sec: 76532.7). Total num frames: 679452672. Throughput: 0: 19158.1. Samples: 159849404. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 03:00:50,466][1157520] Avg episode reward: [(0, '56.344')] [2024-09-30 03:00:50,759][1157819] Updated weights for policy 0, policy_version 165888 (0.0006) [2024-09-30 03:00:51,238][1157819] Updated weights for policy 0, policy_version 165898 (0.0006) [2024-09-30 03:00:51,771][1157819] Updated weights for policy 0, policy_version 165908 (0.0006) [2024-09-30 03:00:52,273][1157819] Updated weights for policy 0, policy_version 165918 (0.0006) [2024-09-30 03:00:52,788][1157819] Updated weights for policy 0, policy_version 165928 (0.0006) [2024-09-30 03:00:53,274][1157819] Updated weights for policy 0, policy_version 165938 (0.0006) [2024-09-30 03:00:53,823][1157819] Updated weights for policy 0, policy_version 165948 (0.0006) [2024-09-30 03:00:54,363][1157819] Updated weights for policy 0, policy_version 165958 (0.0006) [2024-09-30 03:00:54,865][1157819] Updated weights for policy 0, policy_version 165968 (0.0006) [2024-09-30 03:00:55,363][1157819] Updated weights for policy 0, policy_version 165978 (0.0006) [2024-09-30 03:00:55,466][1157520] Fps is (10 sec: 80281.9, 60 sec: 77004.8, 300 sec: 76560.5). Total num frames: 679854080. Throughput: 0: 19291.2. Samples: 159910108. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 03:00:55,466][1157520] Avg episode reward: [(0, '54.422')] [2024-09-30 03:00:55,891][1157819] Updated weights for policy 0, policy_version 165988 (0.0006) [2024-09-30 03:00:56,408][1157819] Updated weights for policy 0, policy_version 165998 (0.0006) [2024-09-30 03:00:56,909][1157819] Updated weights for policy 0, policy_version 166008 (0.0006) [2024-09-30 03:00:57,425][1157819] Updated weights for policy 0, policy_version 166018 (0.0006) [2024-09-30 03:00:57,971][1157819] Updated weights for policy 0, policy_version 166028 (0.0006) [2024-09-30 03:00:58,487][1157819] Updated weights for policy 0, policy_version 166038 (0.0006) [2024-09-30 03:00:58,981][1157819] Updated weights for policy 0, policy_version 166048 (0.0006) [2024-09-30 03:00:59,480][1157819] Updated weights for policy 0, policy_version 166058 (0.0006) [2024-09-30 03:01:00,008][1157819] Updated weights for policy 0, policy_version 166068 (0.0006) [2024-09-30 03:01:00,466][1157520] Fps is (10 sec: 79052.8, 60 sec: 77209.6, 300 sec: 76477.2). Total num frames: 680243200. Throughput: 0: 19091.7. Samples: 160029152. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 03:01:00,466][1157520] Avg episode reward: [(0, '55.552')] [2024-09-30 03:01:00,560][1157819] Updated weights for policy 0, policy_version 166078 (0.0006) [2024-09-30 03:01:01,074][1157819] Updated weights for policy 0, policy_version 166088 (0.0006) [2024-09-30 03:01:01,592][1157819] Updated weights for policy 0, policy_version 166098 (0.0006) [2024-09-30 03:01:02,094][1157819] Updated weights for policy 0, policy_version 166108 (0.0006) [2024-09-30 03:01:02,602][1157819] Updated weights for policy 0, policy_version 166118 (0.0006) [2024-09-30 03:01:03,101][1157819] Updated weights for policy 0, policy_version 166128 (0.0006) [2024-09-30 03:01:03,667][1157819] Updated weights for policy 0, policy_version 166138 (0.0006) [2024-09-30 03:01:04,202][1157819] Updated weights for policy 0, policy_version 166148 (0.0006) [2024-09-30 03:01:04,676][1157819] Updated weights for policy 0, policy_version 166158 (0.0006) [2024-09-30 03:01:05,165][1157819] Updated weights for policy 0, policy_version 166168 (0.0006) [2024-09-30 03:01:05,466][1157520] Fps is (10 sec: 79462.1, 60 sec: 77414.4, 300 sec: 76643.8). Total num frames: 680648704. Throughput: 0: 19305.4. Samples: 160148188. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 03:01:05,466][1157520] Avg episode reward: [(0, '55.415')] [2024-09-30 03:01:05,660][1157819] Updated weights for policy 0, policy_version 166178 (0.0006) [2024-09-30 03:01:06,129][1157819] Updated weights for policy 0, policy_version 166188 (0.0006) [2024-09-30 03:01:06,610][1157819] Updated weights for policy 0, policy_version 166198 (0.0006) [2024-09-30 03:01:07,110][1157819] Updated weights for policy 0, policy_version 166208 (0.0006) [2024-09-30 03:01:07,593][1157819] Updated weights for policy 0, policy_version 166218 (0.0006) [2024-09-30 03:01:08,092][1157819] Updated weights for policy 0, policy_version 166228 (0.0006) [2024-09-30 03:01:08,584][1157819] Updated weights for policy 0, policy_version 166238 (0.0006) [2024-09-30 03:01:09,059][1157819] Updated weights for policy 0, policy_version 166248 (0.0006) [2024-09-30 03:01:09,558][1157819] Updated weights for policy 0, policy_version 166258 (0.0006) [2024-09-30 03:01:10,056][1157819] Updated weights for policy 0, policy_version 166268 (0.0006) [2024-09-30 03:01:10,466][1157520] Fps is (10 sec: 82329.7, 60 sec: 77892.3, 300 sec: 76824.3). Total num frames: 681066496. Throughput: 0: 19556.5. Samples: 160211176. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 03:01:10,466][1157520] Avg episode reward: [(0, '54.672')] [2024-09-30 03:01:10,542][1157819] Updated weights for policy 0, policy_version 166278 (0.0006) [2024-09-30 03:01:11,037][1157819] Updated weights for policy 0, policy_version 166288 (0.0006) [2024-09-30 03:01:11,567][1157819] Updated weights for policy 0, policy_version 166298 (0.0006) [2024-09-30 03:01:11,646][1157736] Signal inference workers to stop experience collection... (11350 times) [2024-09-30 03:01:11,646][1157736] Signal inference workers to resume experience collection... (11350 times) [2024-09-30 03:01:11,651][1157819] InferenceWorker_p0-w0: stopping experience collection (11350 times) [2024-09-30 03:01:11,651][1157819] InferenceWorker_p0-w0: resuming experience collection (11350 times) [2024-09-30 03:01:12,069][1157819] Updated weights for policy 0, policy_version 166308 (0.0006) [2024-09-30 03:01:12,570][1157819] Updated weights for policy 0, policy_version 166318 (0.0006) [2024-09-30 03:01:13,152][1157819] Updated weights for policy 0, policy_version 166328 (0.0006) [2024-09-30 03:01:13,657][1157819] Updated weights for policy 0, policy_version 166338 (0.0006) [2024-09-30 03:01:14,157][1157819] Updated weights for policy 0, policy_version 166348 (0.0006) [2024-09-30 03:01:14,684][1157819] Updated weights for policy 0, policy_version 166358 (0.0006) [2024-09-30 03:01:15,216][1157819] Updated weights for policy 0, policy_version 166368 (0.0006) [2024-09-30 03:01:15,466][1157520] Fps is (10 sec: 81509.5, 60 sec: 77482.7, 300 sec: 76990.9). Total num frames: 681463808. Throughput: 0: 19884.7. Samples: 160332788. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 03:01:15,466][1157520] Avg episode reward: [(0, '52.941')] [2024-09-30 03:01:15,717][1157819] Updated weights for policy 0, policy_version 166378 (0.0006) [2024-09-30 03:01:16,264][1157819] Updated weights for policy 0, policy_version 166388 (0.0006) [2024-09-30 03:01:16,770][1157819] Updated weights for policy 0, policy_version 166398 (0.0006) [2024-09-30 03:01:17,274][1157819] Updated weights for policy 0, policy_version 166408 (0.0006) [2024-09-30 03:01:17,816][1157819] Updated weights for policy 0, policy_version 166418 (0.0006) [2024-09-30 03:01:18,345][1157819] Updated weights for policy 0, policy_version 166428 (0.0006) [2024-09-30 03:01:18,900][1157819] Updated weights for policy 0, policy_version 166438 (0.0006) [2024-09-30 03:01:19,458][1157819] Updated weights for policy 0, policy_version 166448 (0.0006) [2024-09-30 03:01:19,985][1157819] Updated weights for policy 0, policy_version 166458 (0.0006) [2024-09-30 03:01:20,466][1157520] Fps is (10 sec: 78233.5, 60 sec: 78029.0, 300 sec: 77157.6). Total num frames: 681848832. Throughput: 0: 19912.1. Samples: 160449184. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 03:01:20,466][1157520] Avg episode reward: [(0, '54.316')] [2024-09-30 03:01:20,529][1157819] Updated weights for policy 0, policy_version 166468 (0.0006) [2024-09-30 03:01:21,036][1157819] Updated weights for policy 0, policy_version 166478 (0.0006) [2024-09-30 03:01:21,548][1157819] Updated weights for policy 0, policy_version 166488 (0.0006) [2024-09-30 03:01:22,113][1157819] Updated weights for policy 0, policy_version 166498 (0.0006) [2024-09-30 03:01:22,672][1157819] Updated weights for policy 0, policy_version 166508 (0.0006) [2024-09-30 03:01:23,181][1157819] Updated weights for policy 0, policy_version 166518 (0.0006) [2024-09-30 03:01:23,725][1157819] Updated weights for policy 0, policy_version 166528 (0.0006) [2024-09-30 03:01:24,261][1157819] Updated weights for policy 0, policy_version 166538 (0.0006) [2024-09-30 03:01:24,825][1157819] Updated weights for policy 0, policy_version 166548 (0.0006) [2024-09-30 03:01:25,438][1157819] Updated weights for policy 0, policy_version 166558 (0.0006) [2024-09-30 03:01:25,466][1157520] Fps is (10 sec: 75776.0, 60 sec: 78574.8, 300 sec: 77310.3). Total num frames: 682221568. Throughput: 0: 19883.4. Samples: 160506956. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 03:01:25,466][1157520] Avg episode reward: [(0, '54.538')] [2024-09-30 03:01:26,136][1157819] Updated weights for policy 0, policy_version 166568 (0.0006) [2024-09-30 03:01:26,783][1157819] Updated weights for policy 0, policy_version 166578 (0.0006) [2024-09-30 03:01:27,359][1157819] Updated weights for policy 0, policy_version 166588 (0.0006) [2024-09-30 03:01:27,965][1157819] Updated weights for policy 0, policy_version 166598 (0.0006) [2024-09-30 03:01:28,533][1157819] Updated weights for policy 0, policy_version 166608 (0.0006) [2024-09-30 03:01:29,111][1157819] Updated weights for policy 0, policy_version 166618 (0.0006) [2024-09-30 03:01:29,693][1157819] Updated weights for policy 0, policy_version 166628 (0.0006) [2024-09-30 03:01:30,285][1157819] Updated weights for policy 0, policy_version 166638 (0.0006) [2024-09-30 03:01:30,466][1157520] Fps is (10 sec: 70860.1, 60 sec: 77960.4, 300 sec: 77310.3). Total num frames: 682557440. Throughput: 0: 19575.8. Samples: 160611056. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 03:01:30,466][1157520] Avg episode reward: [(0, '53.551')] [2024-09-30 03:01:30,844][1157819] Updated weights for policy 0, policy_version 166648 (0.0006) [2024-09-30 03:01:31,424][1157819] Updated weights for policy 0, policy_version 166658 (0.0006) [2024-09-30 03:01:31,951][1157819] Updated weights for policy 0, policy_version 166668 (0.0006) [2024-09-30 03:01:32,507][1157819] Updated weights for policy 0, policy_version 166678 (0.0006) [2024-09-30 03:01:33,083][1157819] Updated weights for policy 0, policy_version 166688 (0.0006) [2024-09-30 03:01:33,617][1157819] Updated weights for policy 0, policy_version 166698 (0.0006) [2024-09-30 03:01:34,158][1157819] Updated weights for policy 0, policy_version 166708 (0.0006) [2024-09-30 03:01:34,737][1157819] Updated weights for policy 0, policy_version 166718 (0.0006) [2024-09-30 03:01:35,265][1157819] Updated weights for policy 0, policy_version 166728 (0.0006) [2024-09-30 03:01:35,466][1157520] Fps is (10 sec: 70862.0, 60 sec: 77756.2, 300 sec: 77310.3). Total num frames: 682930176. Throughput: 0: 19356.3. Samples: 160720436. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 03:01:35,466][1157520] Avg episode reward: [(0, '55.947')] [2024-09-30 03:01:35,814][1157819] Updated weights for policy 0, policy_version 166738 (0.0006) [2024-09-30 03:01:36,324][1157819] Updated weights for policy 0, policy_version 166748 (0.0006) [2024-09-30 03:01:36,826][1157819] Updated weights for policy 0, policy_version 166758 (0.0006) [2024-09-30 03:01:37,359][1157819] Updated weights for policy 0, policy_version 166768 (0.0006) [2024-09-30 03:01:37,840][1157819] Updated weights for policy 0, policy_version 166778 (0.0006) [2024-09-30 03:01:38,209][1157736] Signal inference workers to stop experience collection... (11400 times) [2024-09-30 03:01:38,213][1157819] InferenceWorker_p0-w0: stopping experience collection (11400 times) [2024-09-30 03:01:38,213][1157736] Signal inference workers to resume experience collection... (11400 times) [2024-09-30 03:01:38,217][1157819] InferenceWorker_p0-w0: resuming experience collection (11400 times) [2024-09-30 03:01:38,329][1157819] Updated weights for policy 0, policy_version 166788 (0.0006) [2024-09-30 03:01:38,821][1157819] Updated weights for policy 0, policy_version 166798 (0.0006) [2024-09-30 03:01:39,340][1157819] Updated weights for policy 0, policy_version 166808 (0.0006) [2024-09-30 03:01:39,820][1157819] Updated weights for policy 0, policy_version 166818 (0.0006) [2024-09-30 03:01:40,373][1157819] Updated weights for policy 0, policy_version 166828 (0.0007) [2024-09-30 03:01:40,466][1157520] Fps is (10 sec: 77824.7, 60 sec: 77960.6, 300 sec: 77351.9). Total num frames: 683335680. Throughput: 0: 19327.6. Samples: 160779852. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 03:01:40,466][1157520] Avg episode reward: [(0, '54.795')] [2024-09-30 03:01:40,938][1157819] Updated weights for policy 0, policy_version 166838 (0.0007) [2024-09-30 03:01:41,455][1157819] Updated weights for policy 0, policy_version 166848 (0.0006) [2024-09-30 03:01:42,007][1157819] Updated weights for policy 0, policy_version 166858 (0.0006) [2024-09-30 03:01:42,521][1157819] Updated weights for policy 0, policy_version 166868 (0.0006) [2024-09-30 03:01:43,095][1157819] Updated weights for policy 0, policy_version 166878 (0.0006) [2024-09-30 03:01:43,642][1157819] Updated weights for policy 0, policy_version 166888 (0.0006) [2024-09-30 03:01:44,187][1157819] Updated weights for policy 0, policy_version 166898 (0.0006) [2024-09-30 03:01:44,739][1157819] Updated weights for policy 0, policy_version 166908 (0.0006) [2024-09-30 03:01:45,277][1157819] Updated weights for policy 0, policy_version 166918 (0.0006) [2024-09-30 03:01:45,466][1157520] Fps is (10 sec: 77823.6, 60 sec: 77619.2, 300 sec: 77185.3). Total num frames: 683708416. Throughput: 0: 19262.6. Samples: 160895968. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:01:45,466][1157520] Avg episode reward: [(0, '54.633')] [2024-09-30 03:01:45,804][1157819] Updated weights for policy 0, policy_version 166928 (0.0006) [2024-09-30 03:01:46,336][1157819] Updated weights for policy 0, policy_version 166938 (0.0006) [2024-09-30 03:01:46,856][1157819] Updated weights for policy 0, policy_version 166948 (0.0006) [2024-09-30 03:01:47,363][1157819] Updated weights for policy 0, policy_version 166958 (0.0006) [2024-09-30 03:01:47,863][1157819] Updated weights for policy 0, policy_version 166968 (0.0006) [2024-09-30 03:01:48,379][1157819] Updated weights for policy 0, policy_version 166978 (0.0006) [2024-09-30 03:01:48,893][1157819] Updated weights for policy 0, policy_version 166988 (0.0006) [2024-09-30 03:01:49,397][1157819] Updated weights for policy 0, policy_version 166998 (0.0006) [2024-09-30 03:01:49,844][1157819] Updated weights for policy 0, policy_version 167008 (0.0006) [2024-09-30 03:01:50,326][1157819] Updated weights for policy 0, policy_version 167018 (0.0006) [2024-09-30 03:01:50,466][1157520] Fps is (10 sec: 77824.4, 60 sec: 77687.5, 300 sec: 77199.2). Total num frames: 684113920. Throughput: 0: 19256.4. Samples: 161014724. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:01:50,466][1157520] Avg episode reward: [(0, '51.675')] [2024-09-30 03:01:50,810][1157819] Updated weights for policy 0, policy_version 167028 (0.0006) [2024-09-30 03:01:51,307][1157819] Updated weights for policy 0, policy_version 167038 (0.0006) [2024-09-30 03:01:51,797][1157819] Updated weights for policy 0, policy_version 167048 (0.0006) [2024-09-30 03:01:52,320][1157819] Updated weights for policy 0, policy_version 167058 (0.0006) [2024-09-30 03:01:52,785][1157819] Updated weights for policy 0, policy_version 167068 (0.0006) [2024-09-30 03:01:53,281][1157819] Updated weights for policy 0, policy_version 167078 (0.0006) [2024-09-30 03:01:53,792][1157819] Updated weights for policy 0, policy_version 167088 (0.0006) [2024-09-30 03:01:54,308][1157819] Updated weights for policy 0, policy_version 167098 (0.0006) [2024-09-30 03:01:54,807][1157819] Updated weights for policy 0, policy_version 167108 (0.0006) [2024-09-30 03:01:55,297][1157819] Updated weights for policy 0, policy_version 167118 (0.0006) [2024-09-30 03:01:55,466][1157520] Fps is (10 sec: 81920.1, 60 sec: 77892.2, 300 sec: 77268.6). Total num frames: 684527616. Throughput: 0: 19245.4. Samples: 161077220. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:01:55,466][1157520] Avg episode reward: [(0, '56.305')] [2024-09-30 03:01:55,814][1157819] Updated weights for policy 0, policy_version 167128 (0.0006) [2024-09-30 03:01:56,312][1157819] Updated weights for policy 0, policy_version 167138 (0.0006) [2024-09-30 03:01:56,818][1157819] Updated weights for policy 0, policy_version 167148 (0.0006) [2024-09-30 03:01:57,320][1157819] Updated weights for policy 0, policy_version 167158 (0.0006) [2024-09-30 03:01:57,838][1157819] Updated weights for policy 0, policy_version 167168 (0.0006) [2024-09-30 03:01:58,364][1157819] Updated weights for policy 0, policy_version 167178 (0.0006) [2024-09-30 03:01:58,917][1157819] Updated weights for policy 0, policy_version 167188 (0.0006) [2024-09-30 03:01:59,424][1157819] Updated weights for policy 0, policy_version 167198 (0.0006) [2024-09-30 03:01:59,926][1157819] Updated weights for policy 0, policy_version 167208 (0.0006) [2024-09-30 03:02:00,466][1157520] Fps is (10 sec: 80691.1, 60 sec: 77960.6, 300 sec: 77296.4). Total num frames: 684920832. Throughput: 0: 19233.8. Samples: 161198304. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:02:00,466][1157520] Avg episode reward: [(0, '56.187')] [2024-09-30 03:02:00,474][1157819] Updated weights for policy 0, policy_version 167218 (0.0006) [2024-09-30 03:02:00,986][1157819] Updated weights for policy 0, policy_version 167228 (0.0006) [2024-09-30 03:02:01,501][1157819] Updated weights for policy 0, policy_version 167238 (0.0006) [2024-09-30 03:02:02,035][1157819] Updated weights for policy 0, policy_version 167248 (0.0006) [2024-09-30 03:02:02,527][1157819] Updated weights for policy 0, policy_version 167258 (0.0006) [2024-09-30 03:02:03,005][1157819] Updated weights for policy 0, policy_version 167268 (0.0006) [2024-09-30 03:02:03,470][1157819] Updated weights for policy 0, policy_version 167278 (0.0006) [2024-09-30 03:02:03,940][1157819] Updated weights for policy 0, policy_version 167288 (0.0006) [2024-09-30 03:02:04,451][1157819] Updated weights for policy 0, policy_version 167298 (0.0006) [2024-09-30 03:02:04,948][1157819] Updated weights for policy 0, policy_version 167308 (0.0006) [2024-09-30 03:02:05,429][1157819] Updated weights for policy 0, policy_version 167318 (0.0006) [2024-09-30 03:02:05,466][1157520] Fps is (10 sec: 81101.3, 60 sec: 78165.4, 300 sec: 77393.6). Total num frames: 685338624. Throughput: 0: 19350.0. Samples: 161319932. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:02:05,466][1157520] Avg episode reward: [(0, '54.139')] [2024-09-30 03:02:05,469][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000167319_685338624.pth... [2024-09-30 03:02:05,513][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000162776_666730496.pth [2024-09-30 03:02:05,924][1157819] Updated weights for policy 0, policy_version 167328 (0.0006) [2024-09-30 03:02:06,425][1157819] Updated weights for policy 0, policy_version 167338 (0.0006) [2024-09-30 03:02:06,907][1157819] Updated weights for policy 0, policy_version 167348 (0.0006) [2024-09-30 03:02:07,412][1157819] Updated weights for policy 0, policy_version 167358 (0.0006) [2024-09-30 03:02:07,911][1157819] Updated weights for policy 0, policy_version 167368 (0.0006) [2024-09-30 03:02:08,373][1157819] Updated weights for policy 0, policy_version 167378 (0.0006) [2024-09-30 03:02:08,857][1157819] Updated weights for policy 0, policy_version 167388 (0.0006) [2024-09-30 03:02:09,354][1157819] Updated weights for policy 0, policy_version 167398 (0.0006) [2024-09-30 03:02:09,855][1157819] Updated weights for policy 0, policy_version 167408 (0.0006) [2024-09-30 03:02:10,348][1157819] Updated weights for policy 0, policy_version 167418 (0.0006) [2024-09-30 03:02:10,466][1157520] Fps is (10 sec: 83148.8, 60 sec: 78097.1, 300 sec: 77435.2). Total num frames: 685752320. Throughput: 0: 19457.3. Samples: 161382532. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:02:10,466][1157520] Avg episode reward: [(0, '58.358')] [2024-09-30 03:02:10,824][1157819] Updated weights for policy 0, policy_version 167428 (0.0006) [2024-09-30 03:02:11,320][1157819] Updated weights for policy 0, policy_version 167438 (0.0006) [2024-09-30 03:02:11,816][1157819] Updated weights for policy 0, policy_version 167448 (0.0006) [2024-09-30 03:02:12,297][1157819] Updated weights for policy 0, policy_version 167458 (0.0006) [2024-09-30 03:02:12,749][1157819] Updated weights for policy 0, policy_version 167468 (0.0006) [2024-09-30 03:02:13,227][1157819] Updated weights for policy 0, policy_version 167478 (0.0006) [2024-09-30 03:02:13,728][1157819] Updated weights for policy 0, policy_version 167488 (0.0006) [2024-09-30 03:02:14,197][1157819] Updated weights for policy 0, policy_version 167498 (0.0005) [2024-09-30 03:02:14,670][1157819] Updated weights for policy 0, policy_version 167508 (0.0005) [2024-09-30 03:02:15,170][1157819] Updated weights for policy 0, policy_version 167518 (0.0006) [2024-09-30 03:02:15,466][1157520] Fps is (10 sec: 83557.9, 60 sec: 78506.8, 300 sec: 77615.7). Total num frames: 686174208. Throughput: 0: 19950.9. Samples: 161508844. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:02:15,466][1157520] Avg episode reward: [(0, '56.171')] [2024-09-30 03:02:15,674][1157819] Updated weights for policy 0, policy_version 167528 (0.0005) [2024-09-30 03:02:16,175][1157819] Updated weights for policy 0, policy_version 167538 (0.0006) [2024-09-30 03:02:16,671][1157819] Updated weights for policy 0, policy_version 167548 (0.0006) [2024-09-30 03:02:17,184][1157819] Updated weights for policy 0, policy_version 167558 (0.0006) [2024-09-30 03:02:17,696][1157819] Updated weights for policy 0, policy_version 167568 (0.0006) [2024-09-30 03:02:18,192][1157819] Updated weights for policy 0, policy_version 167578 (0.0006) [2024-09-30 03:02:18,693][1157819] Updated weights for policy 0, policy_version 167588 (0.0006) [2024-09-30 03:02:19,218][1157819] Updated weights for policy 0, policy_version 167598 (0.0006) [2024-09-30 03:02:19,749][1157819] Updated weights for policy 0, policy_version 167608 (0.0006) [2024-09-30 03:02:20,238][1157819] Updated weights for policy 0, policy_version 167618 (0.0006) [2024-09-30 03:02:20,466][1157520] Fps is (10 sec: 82738.8, 60 sec: 78848.0, 300 sec: 77782.4). Total num frames: 686579712. Throughput: 0: 20250.0. Samples: 161631688. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:02:20,466][1157520] Avg episode reward: [(0, '57.679')] [2024-09-30 03:02:20,756][1157819] Updated weights for policy 0, policy_version 167628 (0.0006) [2024-09-30 03:02:21,274][1157819] Updated weights for policy 0, policy_version 167638 (0.0006) [2024-09-30 03:02:21,839][1157819] Updated weights for policy 0, policy_version 167648 (0.0006) [2024-09-30 03:02:22,350][1157819] Updated weights for policy 0, policy_version 167658 (0.0006) [2024-09-30 03:02:22,844][1157819] Updated weights for policy 0, policy_version 167668 (0.0006) [2024-09-30 03:02:23,328][1157819] Updated weights for policy 0, policy_version 167678 (0.0006) [2024-09-30 03:02:23,824][1157819] Updated weights for policy 0, policy_version 167688 (0.0006) [2024-09-30 03:02:24,312][1157819] Updated weights for policy 0, policy_version 167698 (0.0006) [2024-09-30 03:02:24,795][1157819] Updated weights for policy 0, policy_version 167708 (0.0005) [2024-09-30 03:02:25,276][1157819] Updated weights for policy 0, policy_version 167718 (0.0006) [2024-09-30 03:02:25,339][1157736] Signal inference workers to stop experience collection... (11450 times) [2024-09-30 03:02:25,344][1157819] InferenceWorker_p0-w0: stopping experience collection (11450 times) [2024-09-30 03:02:25,344][1157736] Signal inference workers to resume experience collection... (11450 times) [2024-09-30 03:02:25,348][1157819] InferenceWorker_p0-w0: resuming experience collection (11450 times) [2024-09-30 03:02:25,466][1157520] Fps is (10 sec: 81101.0, 60 sec: 79394.3, 300 sec: 77962.9). Total num frames: 686985216. Throughput: 0: 20244.8. Samples: 161690868. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:02:25,466][1157520] Avg episode reward: [(0, '53.964')] [2024-09-30 03:02:25,768][1157819] Updated weights for policy 0, policy_version 167728 (0.0006) [2024-09-30 03:02:26,254][1157819] Updated weights for policy 0, policy_version 167738 (0.0006) [2024-09-30 03:02:26,748][1157819] Updated weights for policy 0, policy_version 167748 (0.0006) [2024-09-30 03:02:27,240][1157819] Updated weights for policy 0, policy_version 167758 (0.0006) [2024-09-30 03:02:27,739][1157819] Updated weights for policy 0, policy_version 167768 (0.0006) [2024-09-30 03:02:28,233][1157819] Updated weights for policy 0, policy_version 167778 (0.0006) [2024-09-30 03:02:28,722][1157819] Updated weights for policy 0, policy_version 167788 (0.0006) [2024-09-30 03:02:29,204][1157819] Updated weights for policy 0, policy_version 167798 (0.0006) [2024-09-30 03:02:29,691][1157819] Updated weights for policy 0, policy_version 167808 (0.0006) [2024-09-30 03:02:30,190][1157819] Updated weights for policy 0, policy_version 167818 (0.0006) [2024-09-30 03:02:30,466][1157520] Fps is (10 sec: 82329.0, 60 sec: 80759.5, 300 sec: 78171.1). Total num frames: 687403008. Throughput: 0: 20448.5. Samples: 161816152. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:02:30,466][1157520] Avg episode reward: [(0, '54.767')] [2024-09-30 03:02:30,743][1157819] Updated weights for policy 0, policy_version 167828 (0.0006) [2024-09-30 03:02:31,268][1157819] Updated weights for policy 0, policy_version 167838 (0.0006) [2024-09-30 03:02:31,789][1157819] Updated weights for policy 0, policy_version 167848 (0.0006) [2024-09-30 03:02:32,289][1157819] Updated weights for policy 0, policy_version 167858 (0.0006) [2024-09-30 03:02:32,831][1157819] Updated weights for policy 0, policy_version 167868 (0.0006) [2024-09-30 03:02:33,347][1157819] Updated weights for policy 0, policy_version 167878 (0.0006) [2024-09-30 03:02:33,866][1157819] Updated weights for policy 0, policy_version 167888 (0.0006) [2024-09-30 03:02:34,394][1157819] Updated weights for policy 0, policy_version 167898 (0.0006) [2024-09-30 03:02:34,899][1157819] Updated weights for policy 0, policy_version 167908 (0.0006) [2024-09-30 03:02:35,402][1157819] Updated weights for policy 0, policy_version 167918 (0.0006) [2024-09-30 03:02:35,466][1157520] Fps is (10 sec: 81100.7, 60 sec: 81100.7, 300 sec: 78171.1). Total num frames: 687796224. Throughput: 0: 20465.3. Samples: 161935664. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:02:35,466][1157520] Avg episode reward: [(0, '55.130')] [2024-09-30 03:02:35,898][1157819] Updated weights for policy 0, policy_version 167928 (0.0006) [2024-09-30 03:02:36,397][1157819] Updated weights for policy 0, policy_version 167938 (0.0006) [2024-09-30 03:02:36,918][1157819] Updated weights for policy 0, policy_version 167948 (0.0006) [2024-09-30 03:02:37,453][1157819] Updated weights for policy 0, policy_version 167958 (0.0006) [2024-09-30 03:02:37,956][1157819] Updated weights for policy 0, policy_version 167968 (0.0006) [2024-09-30 03:02:38,484][1157819] Updated weights for policy 0, policy_version 167978 (0.0006) [2024-09-30 03:02:38,976][1157819] Updated weights for policy 0, policy_version 167988 (0.0006) [2024-09-30 03:02:39,495][1157819] Updated weights for policy 0, policy_version 167998 (0.0006) [2024-09-30 03:02:39,991][1157819] Updated weights for policy 0, policy_version 168008 (0.0006) [2024-09-30 03:02:40,466][1157520] Fps is (10 sec: 79053.7, 60 sec: 80964.3, 300 sec: 78171.2). Total num frames: 688193536. Throughput: 0: 20416.9. Samples: 161995980. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:02:40,466][1157520] Avg episode reward: [(0, '53.138')] [2024-09-30 03:02:40,518][1157819] Updated weights for policy 0, policy_version 168018 (0.0006) [2024-09-30 03:02:41,034][1157819] Updated weights for policy 0, policy_version 168028 (0.0006) [2024-09-30 03:02:41,572][1157819] Updated weights for policy 0, policy_version 168038 (0.0006) [2024-09-30 03:02:42,087][1157819] Updated weights for policy 0, policy_version 168048 (0.0006) [2024-09-30 03:02:42,614][1157819] Updated weights for policy 0, policy_version 168058 (0.0006) [2024-09-30 03:02:43,119][1157819] Updated weights for policy 0, policy_version 168068 (0.0006) [2024-09-30 03:02:43,634][1157819] Updated weights for policy 0, policy_version 168078 (0.0006) [2024-09-30 03:02:44,132][1157819] Updated weights for policy 0, policy_version 168088 (0.0006) [2024-09-30 03:02:44,651][1157819] Updated weights for policy 0, policy_version 168098 (0.0006) [2024-09-30 03:02:45,188][1157819] Updated weights for policy 0, policy_version 168108 (0.0006) [2024-09-30 03:02:45,466][1157520] Fps is (10 sec: 79461.4, 60 sec: 81373.7, 300 sec: 78171.1). Total num frames: 688590848. Throughput: 0: 20368.2. Samples: 162114876. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:02:45,466][1157520] Avg episode reward: [(0, '54.194')] [2024-09-30 03:02:45,707][1157819] Updated weights for policy 0, policy_version 168118 (0.0006) [2024-09-30 03:02:46,227][1157819] Updated weights for policy 0, policy_version 168128 (0.0006) [2024-09-30 03:02:46,786][1157819] Updated weights for policy 0, policy_version 168138 (0.0006) [2024-09-30 03:02:47,316][1157819] Updated weights for policy 0, policy_version 168148 (0.0006) [2024-09-30 03:02:47,858][1157819] Updated weights for policy 0, policy_version 168158 (0.0006) [2024-09-30 03:02:48,382][1157819] Updated weights for policy 0, policy_version 168168 (0.0006) [2024-09-30 03:02:48,930][1157819] Updated weights for policy 0, policy_version 168178 (0.0006) [2024-09-30 03:02:49,482][1157819] Updated weights for policy 0, policy_version 168188 (0.0006) [2024-09-30 03:02:50,017][1157819] Updated weights for policy 0, policy_version 168198 (0.0006) [2024-09-30 03:02:50,466][1157520] Fps is (10 sec: 77823.9, 60 sec: 80964.2, 300 sec: 78115.6). Total num frames: 688971776. Throughput: 0: 20225.0. Samples: 162230060. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:02:50,466][1157520] Avg episode reward: [(0, '55.276')] [2024-09-30 03:02:50,529][1157819] Updated weights for policy 0, policy_version 168208 (0.0006) [2024-09-30 03:02:51,069][1157819] Updated weights for policy 0, policy_version 168218 (0.0006) [2024-09-30 03:02:51,571][1157819] Updated weights for policy 0, policy_version 168228 (0.0006) [2024-09-30 03:02:52,109][1157819] Updated weights for policy 0, policy_version 168238 (0.0006) [2024-09-30 03:02:52,615][1157819] Updated weights for policy 0, policy_version 168248 (0.0006) [2024-09-30 03:02:53,115][1157819] Updated weights for policy 0, policy_version 168258 (0.0006) [2024-09-30 03:02:53,617][1157819] Updated weights for policy 0, policy_version 168268 (0.0006) [2024-09-30 03:02:54,113][1157819] Updated weights for policy 0, policy_version 168278 (0.0006) [2024-09-30 03:02:54,692][1157819] Updated weights for policy 0, policy_version 168288 (0.0006) [2024-09-30 03:02:55,209][1157819] Updated weights for policy 0, policy_version 168298 (0.0006) [2024-09-30 03:02:55,466][1157520] Fps is (10 sec: 77824.9, 60 sec: 80691.2, 300 sec: 78129.5). Total num frames: 689369088. Throughput: 0: 20154.8. Samples: 162289500. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:02:55,466][1157520] Avg episode reward: [(0, '53.023')] [2024-09-30 03:02:55,705][1157819] Updated weights for policy 0, policy_version 168308 (0.0006) [2024-09-30 03:02:56,219][1157819] Updated weights for policy 0, policy_version 168318 (0.0006) [2024-09-30 03:02:56,733][1157819] Updated weights for policy 0, policy_version 168328 (0.0006) [2024-09-30 03:02:57,256][1157819] Updated weights for policy 0, policy_version 168338 (0.0006) [2024-09-30 03:02:57,760][1157819] Updated weights for policy 0, policy_version 168348 (0.0006) [2024-09-30 03:02:58,301][1157819] Updated weights for policy 0, policy_version 168358 (0.0006) [2024-09-30 03:02:58,810][1157819] Updated weights for policy 0, policy_version 168368 (0.0006) [2024-09-30 03:02:59,321][1157819] Updated weights for policy 0, policy_version 168378 (0.0006) [2024-09-30 03:02:59,851][1157819] Updated weights for policy 0, policy_version 168388 (0.0006) [2024-09-30 03:03:00,384][1157819] Updated weights for policy 0, policy_version 168398 (0.0006) [2024-09-30 03:03:00,466][1157520] Fps is (10 sec: 79052.8, 60 sec: 80691.2, 300 sec: 78073.9). Total num frames: 689762304. Throughput: 0: 19989.9. Samples: 162408388. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:03:00,466][1157520] Avg episode reward: [(0, '54.757')] [2024-09-30 03:03:00,867][1157819] Updated weights for policy 0, policy_version 168408 (0.0006) [2024-09-30 03:03:01,393][1157819] Updated weights for policy 0, policy_version 168418 (0.0006) [2024-09-30 03:03:01,929][1157819] Updated weights for policy 0, policy_version 168428 (0.0006) [2024-09-30 03:03:02,456][1157819] Updated weights for policy 0, policy_version 168438 (0.0006) [2024-09-30 03:03:02,950][1157819] Updated weights for policy 0, policy_version 168448 (0.0006) [2024-09-30 03:03:03,460][1157819] Updated weights for policy 0, policy_version 168458 (0.0006) [2024-09-30 03:03:03,984][1157819] Updated weights for policy 0, policy_version 168468 (0.0006) [2024-09-30 03:03:04,477][1157819] Updated weights for policy 0, policy_version 168478 (0.0006) [2024-09-30 03:03:04,960][1157819] Updated weights for policy 0, policy_version 168488 (0.0006) [2024-09-30 03:03:05,450][1157819] Updated weights for policy 0, policy_version 168498 (0.0006) [2024-09-30 03:03:05,466][1157520] Fps is (10 sec: 79872.2, 60 sec: 80486.4, 300 sec: 78129.5). Total num frames: 690167808. Throughput: 0: 19910.2. Samples: 162527648. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:03:05,466][1157520] Avg episode reward: [(0, '55.705')] [2024-09-30 03:03:05,970][1157819] Updated weights for policy 0, policy_version 168508 (0.0006) [2024-09-30 03:03:06,442][1157819] Updated weights for policy 0, policy_version 168518 (0.0006) [2024-09-30 03:03:06,940][1157819] Updated weights for policy 0, policy_version 168528 (0.0006) [2024-09-30 03:03:07,444][1157819] Updated weights for policy 0, policy_version 168538 (0.0006) [2024-09-30 03:03:07,962][1157819] Updated weights for policy 0, policy_version 168548 (0.0006) [2024-09-30 03:03:08,436][1157819] Updated weights for policy 0, policy_version 168558 (0.0006) [2024-09-30 03:03:08,951][1157819] Updated weights for policy 0, policy_version 168568 (0.0006) [2024-09-30 03:03:09,287][1157736] Signal inference workers to stop experience collection... (11500 times) [2024-09-30 03:03:09,291][1157819] InferenceWorker_p0-w0: stopping experience collection (11500 times) [2024-09-30 03:03:09,291][1157736] Signal inference workers to resume experience collection... (11500 times) [2024-09-30 03:03:09,295][1157819] InferenceWorker_p0-w0: resuming experience collection (11500 times) [2024-09-30 03:03:09,462][1157819] Updated weights for policy 0, policy_version 168578 (0.0006) [2024-09-30 03:03:09,936][1157819] Updated weights for policy 0, policy_version 168588 (0.0006) [2024-09-30 03:03:10,423][1157819] Updated weights for policy 0, policy_version 168598 (0.0006) [2024-09-30 03:03:10,466][1157520] Fps is (10 sec: 81509.5, 60 sec: 80418.0, 300 sec: 78268.3). Total num frames: 690577408. Throughput: 0: 19977.8. Samples: 162589872. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:03:10,466][1157520] Avg episode reward: [(0, '55.075')] [2024-09-30 03:03:10,955][1157819] Updated weights for policy 0, policy_version 168608 (0.0006) [2024-09-30 03:03:11,497][1157819] Updated weights for policy 0, policy_version 168618 (0.0006) [2024-09-30 03:03:11,995][1157819] Updated weights for policy 0, policy_version 168628 (0.0006) [2024-09-30 03:03:12,501][1157819] Updated weights for policy 0, policy_version 168638 (0.0006) [2024-09-30 03:03:12,994][1157819] Updated weights for policy 0, policy_version 168648 (0.0006) [2024-09-30 03:03:13,530][1157819] Updated weights for policy 0, policy_version 168658 (0.0006) [2024-09-30 03:03:14,054][1157819] Updated weights for policy 0, policy_version 168668 (0.0006) [2024-09-30 03:03:14,588][1157819] Updated weights for policy 0, policy_version 168678 (0.0006) [2024-09-30 03:03:15,136][1157819] Updated weights for policy 0, policy_version 168688 (0.0006) [2024-09-30 03:03:15,466][1157520] Fps is (10 sec: 80280.8, 60 sec: 79940.2, 300 sec: 78323.8). Total num frames: 690970624. Throughput: 0: 19888.8. Samples: 162711148. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:03:15,466][1157520] Avg episode reward: [(0, '58.362')] [2024-09-30 03:03:15,679][1157819] Updated weights for policy 0, policy_version 168698 (0.0006) [2024-09-30 03:03:16,208][1157819] Updated weights for policy 0, policy_version 168708 (0.0006) [2024-09-30 03:03:16,768][1157819] Updated weights for policy 0, policy_version 168718 (0.0006) [2024-09-30 03:03:17,300][1157819] Updated weights for policy 0, policy_version 168728 (0.0006) [2024-09-30 03:03:17,862][1157819] Updated weights for policy 0, policy_version 168738 (0.0006) [2024-09-30 03:03:18,415][1157819] Updated weights for policy 0, policy_version 168748 (0.0006) [2024-09-30 03:03:18,959][1157819] Updated weights for policy 0, policy_version 168758 (0.0006) [2024-09-30 03:03:19,522][1157819] Updated weights for policy 0, policy_version 168768 (0.0006) [2024-09-30 03:03:20,057][1157819] Updated weights for policy 0, policy_version 168778 (0.0006) [2024-09-30 03:03:20,466][1157520] Fps is (10 sec: 76595.4, 60 sec: 79394.1, 300 sec: 78254.4). Total num frames: 691343360. Throughput: 0: 19731.8. Samples: 162823596. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:03:20,466][1157520] Avg episode reward: [(0, '56.777')] [2024-09-30 03:03:20,596][1157819] Updated weights for policy 0, policy_version 168788 (0.0006) [2024-09-30 03:03:21,148][1157819] Updated weights for policy 0, policy_version 168798 (0.0006) [2024-09-30 03:03:21,682][1157819] Updated weights for policy 0, policy_version 168808 (0.0006) [2024-09-30 03:03:22,243][1157819] Updated weights for policy 0, policy_version 168818 (0.0006) [2024-09-30 03:03:22,776][1157819] Updated weights for policy 0, policy_version 168828 (0.0006) [2024-09-30 03:03:23,295][1157819] Updated weights for policy 0, policy_version 168838 (0.0006) [2024-09-30 03:03:23,859][1157819] Updated weights for policy 0, policy_version 168848 (0.0006) [2024-09-30 03:03:24,376][1157819] Updated weights for policy 0, policy_version 168858 (0.0006) [2024-09-30 03:03:24,873][1157819] Updated weights for policy 0, policy_version 168868 (0.0006) [2024-09-30 03:03:25,375][1157819] Updated weights for policy 0, policy_version 168878 (0.0006) [2024-09-30 03:03:25,466][1157520] Fps is (10 sec: 75775.1, 60 sec: 79052.5, 300 sec: 78185.0). Total num frames: 691728384. Throughput: 0: 19640.2. Samples: 162879792. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:03:25,466][1157520] Avg episode reward: [(0, '56.095')] [2024-09-30 03:03:25,878][1157819] Updated weights for policy 0, policy_version 168888 (0.0006) [2024-09-30 03:03:26,397][1157819] Updated weights for policy 0, policy_version 168898 (0.0006) [2024-09-30 03:03:26,931][1157819] Updated weights for policy 0, policy_version 168908 (0.0006) [2024-09-30 03:03:27,456][1157819] Updated weights for policy 0, policy_version 168918 (0.0006) [2024-09-30 03:03:27,985][1157819] Updated weights for policy 0, policy_version 168928 (0.0006) [2024-09-30 03:03:28,476][1157819] Updated weights for policy 0, policy_version 168938 (0.0006) [2024-09-30 03:03:29,005][1157819] Updated weights for policy 0, policy_version 168948 (0.0006) [2024-09-30 03:03:29,487][1157819] Updated weights for policy 0, policy_version 168958 (0.0006) [2024-09-30 03:03:30,031][1157819] Updated weights for policy 0, policy_version 168968 (0.0006) [2024-09-30 03:03:30,466][1157520] Fps is (10 sec: 78233.2, 60 sec: 78711.4, 300 sec: 78129.4). Total num frames: 692125696. Throughput: 0: 19637.2. Samples: 162998552. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:03:30,466][1157520] Avg episode reward: [(0, '57.586')] [2024-09-30 03:03:30,576][1157819] Updated weights for policy 0, policy_version 168978 (0.0006) [2024-09-30 03:03:31,067][1157819] Updated weights for policy 0, policy_version 168988 (0.0006) [2024-09-30 03:03:31,565][1157819] Updated weights for policy 0, policy_version 168998 (0.0006) [2024-09-30 03:03:32,128][1157819] Updated weights for policy 0, policy_version 169008 (0.0006) [2024-09-30 03:03:32,674][1157819] Updated weights for policy 0, policy_version 169018 (0.0006) [2024-09-30 03:03:33,183][1157819] Updated weights for policy 0, policy_version 169028 (0.0006) [2024-09-30 03:03:33,692][1157819] Updated weights for policy 0, policy_version 169038 (0.0006) [2024-09-30 03:03:34,199][1157819] Updated weights for policy 0, policy_version 169048 (0.0006) [2024-09-30 03:03:34,766][1157819] Updated weights for policy 0, policy_version 169058 (0.0006) [2024-09-30 03:03:35,300][1157819] Updated weights for policy 0, policy_version 169068 (0.0006) [2024-09-30 03:03:35,466][1157520] Fps is (10 sec: 78643.7, 60 sec: 78643.0, 300 sec: 78101.6). Total num frames: 692514816. Throughput: 0: 19685.8. Samples: 163115924. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:03:35,466][1157520] Avg episode reward: [(0, '55.026')] [2024-09-30 03:03:35,807][1157819] Updated weights for policy 0, policy_version 169078 (0.0006) [2024-09-30 03:03:36,341][1157819] Updated weights for policy 0, policy_version 169088 (0.0006) [2024-09-30 03:03:36,851][1157819] Updated weights for policy 0, policy_version 169098 (0.0006) [2024-09-30 03:03:37,369][1157819] Updated weights for policy 0, policy_version 169108 (0.0006) [2024-09-30 03:03:37,897][1157819] Updated weights for policy 0, policy_version 169118 (0.0006) [2024-09-30 03:03:38,399][1157819] Updated weights for policy 0, policy_version 169128 (0.0006) [2024-09-30 03:03:38,948][1157819] Updated weights for policy 0, policy_version 169138 (0.0006) [2024-09-30 03:03:39,469][1157819] Updated weights for policy 0, policy_version 169148 (0.0006) [2024-09-30 03:03:39,999][1157819] Updated weights for policy 0, policy_version 169158 (0.0006) [2024-09-30 03:03:40,466][1157520] Fps is (10 sec: 78234.0, 60 sec: 78574.8, 300 sec: 78087.8). Total num frames: 692908032. Throughput: 0: 19672.4. Samples: 163174760. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:03:40,466][1157520] Avg episode reward: [(0, '57.618')] [2024-09-30 03:03:40,505][1157819] Updated weights for policy 0, policy_version 169168 (0.0006) [2024-09-30 03:03:41,032][1157819] Updated weights for policy 0, policy_version 169178 (0.0006) [2024-09-30 03:03:41,535][1157819] Updated weights for policy 0, policy_version 169188 (0.0006) [2024-09-30 03:03:42,022][1157819] Updated weights for policy 0, policy_version 169198 (0.0006) [2024-09-30 03:03:42,552][1157819] Updated weights for policy 0, policy_version 169208 (0.0005) [2024-09-30 03:03:43,011][1157819] Updated weights for policy 0, policy_version 169218 (0.0005) [2024-09-30 03:03:43,541][1157819] Updated weights for policy 0, policy_version 169228 (0.0006) [2024-09-30 03:03:44,041][1157819] Updated weights for policy 0, policy_version 169238 (0.0006) [2024-09-30 03:03:44,485][1157819] Updated weights for policy 0, policy_version 169248 (0.0005) [2024-09-30 03:03:44,746][1157736] Signal inference workers to stop experience collection... (11550 times) [2024-09-30 03:03:44,749][1157736] Signal inference workers to resume experience collection... (11550 times) [2024-09-30 03:03:44,749][1157819] InferenceWorker_p0-w0: stopping experience collection (11550 times) [2024-09-30 03:03:44,753][1157819] InferenceWorker_p0-w0: resuming experience collection (11550 times) [2024-09-30 03:03:44,986][1157819] Updated weights for policy 0, policy_version 169258 (0.0006) [2024-09-30 03:03:45,466][1157520] Fps is (10 sec: 80282.1, 60 sec: 78779.8, 300 sec: 78143.3). Total num frames: 693317632. Throughput: 0: 19705.6. Samples: 163295144. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:03:45,466][1157520] Avg episode reward: [(0, '55.828')] [2024-09-30 03:03:45,481][1157819] Updated weights for policy 0, policy_version 169268 (0.0006) [2024-09-30 03:03:45,921][1157819] Updated weights for policy 0, policy_version 169278 (0.0006) [2024-09-30 03:03:46,437][1157819] Updated weights for policy 0, policy_version 169288 (0.0006) [2024-09-30 03:03:46,942][1157819] Updated weights for policy 0, policy_version 169298 (0.0006) [2024-09-30 03:03:47,407][1157819] Updated weights for policy 0, policy_version 169308 (0.0006) [2024-09-30 03:03:47,907][1157819] Updated weights for policy 0, policy_version 169318 (0.0006) [2024-09-30 03:03:48,396][1157819] Updated weights for policy 0, policy_version 169328 (0.0006) [2024-09-30 03:03:48,890][1157819] Updated weights for policy 0, policy_version 169338 (0.0006) [2024-09-30 03:03:49,382][1157819] Updated weights for policy 0, policy_version 169348 (0.0006) [2024-09-30 03:03:49,845][1157819] Updated weights for policy 0, policy_version 169358 (0.0006) [2024-09-30 03:03:50,360][1157819] Updated weights for policy 0, policy_version 169368 (0.0006) [2024-09-30 03:03:50,466][1157520] Fps is (10 sec: 83148.8, 60 sec: 79462.3, 300 sec: 78198.9). Total num frames: 693739520. Throughput: 0: 19851.2. Samples: 163420952. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:03:50,466][1157520] Avg episode reward: [(0, '55.332')] [2024-09-30 03:03:50,837][1157819] Updated weights for policy 0, policy_version 169378 (0.0006) [2024-09-30 03:03:51,326][1157819] Updated weights for policy 0, policy_version 169388 (0.0006) [2024-09-30 03:03:51,827][1157819] Updated weights for policy 0, policy_version 169398 (0.0006) [2024-09-30 03:03:52,318][1157819] Updated weights for policy 0, policy_version 169408 (0.0006) [2024-09-30 03:03:52,815][1157819] Updated weights for policy 0, policy_version 169418 (0.0006) [2024-09-30 03:03:53,306][1157819] Updated weights for policy 0, policy_version 169428 (0.0006) [2024-09-30 03:03:53,793][1157819] Updated weights for policy 0, policy_version 169438 (0.0006) [2024-09-30 03:03:54,312][1157819] Updated weights for policy 0, policy_version 169448 (0.0006) [2024-09-30 03:03:54,777][1157819] Updated weights for policy 0, policy_version 169458 (0.0006) [2024-09-30 03:03:55,301][1157819] Updated weights for policy 0, policy_version 169468 (0.0006) [2024-09-30 03:03:55,466][1157520] Fps is (10 sec: 83558.0, 60 sec: 79735.3, 300 sec: 78212.8). Total num frames: 694153216. Throughput: 0: 19858.5. Samples: 163483504. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:03:55,466][1157520] Avg episode reward: [(0, '58.383')] [2024-09-30 03:03:55,783][1157819] Updated weights for policy 0, policy_version 169478 (0.0006) [2024-09-30 03:03:56,248][1157819] Updated weights for policy 0, policy_version 169488 (0.0006) [2024-09-30 03:03:56,777][1157819] Updated weights for policy 0, policy_version 169498 (0.0006) [2024-09-30 03:03:57,274][1157819] Updated weights for policy 0, policy_version 169508 (0.0006) [2024-09-30 03:03:57,774][1157819] Updated weights for policy 0, policy_version 169518 (0.0006) [2024-09-30 03:03:58,300][1157819] Updated weights for policy 0, policy_version 169528 (0.0006) [2024-09-30 03:03:58,821][1157819] Updated weights for policy 0, policy_version 169538 (0.0006) [2024-09-30 03:03:59,341][1157819] Updated weights for policy 0, policy_version 169548 (0.0006) [2024-09-30 03:03:59,850][1157819] Updated weights for policy 0, policy_version 169558 (0.0006) [2024-09-30 03:04:00,356][1157819] Updated weights for policy 0, policy_version 169568 (0.0006) [2024-09-30 03:04:00,466][1157520] Fps is (10 sec: 81510.6, 60 sec: 79871.9, 300 sec: 78296.1). Total num frames: 694554624. Throughput: 0: 19890.4. Samples: 163606216. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:04:00,466][1157520] Avg episode reward: [(0, '57.201')] [2024-09-30 03:04:00,895][1157819] Updated weights for policy 0, policy_version 169578 (0.0005) [2024-09-30 03:04:01,402][1157819] Updated weights for policy 0, policy_version 169588 (0.0006) [2024-09-30 03:04:01,924][1157819] Updated weights for policy 0, policy_version 169598 (0.0006) [2024-09-30 03:04:02,426][1157819] Updated weights for policy 0, policy_version 169608 (0.0006) [2024-09-30 03:04:02,932][1157819] Updated weights for policy 0, policy_version 169618 (0.0006) [2024-09-30 03:04:03,447][1157819] Updated weights for policy 0, policy_version 169628 (0.0006) [2024-09-30 03:04:03,956][1157819] Updated weights for policy 0, policy_version 169638 (0.0006) [2024-09-30 03:04:04,442][1157819] Updated weights for policy 0, policy_version 169648 (0.0005) [2024-09-30 03:04:04,916][1157819] Updated weights for policy 0, policy_version 169658 (0.0006) [2024-09-30 03:04:05,403][1157819] Updated weights for policy 0, policy_version 169668 (0.0006) [2024-09-30 03:04:05,466][1157520] Fps is (10 sec: 81100.8, 60 sec: 79940.1, 300 sec: 78434.9). Total num frames: 694964224. Throughput: 0: 20070.7. Samples: 163726780. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:04:05,466][1157520] Avg episode reward: [(0, '54.278')] [2024-09-30 03:04:05,472][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000169670_694968320.pth... [2024-09-30 03:04:05,517][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000165040_676003840.pth [2024-09-30 03:04:05,877][1157819] Updated weights for policy 0, policy_version 169678 (0.0006) [2024-09-30 03:04:06,332][1157819] Updated weights for policy 0, policy_version 169688 (0.0006) [2024-09-30 03:04:06,787][1157819] Updated weights for policy 0, policy_version 169698 (0.0006) [2024-09-30 03:04:07,272][1157819] Updated weights for policy 0, policy_version 169708 (0.0006) [2024-09-30 03:04:07,765][1157819] Updated weights for policy 0, policy_version 169718 (0.0006) [2024-09-30 03:04:08,256][1157819] Updated weights for policy 0, policy_version 169728 (0.0006) [2024-09-30 03:04:08,758][1157819] Updated weights for policy 0, policy_version 169738 (0.0006) [2024-09-30 03:04:09,245][1157819] Updated weights for policy 0, policy_version 169748 (0.0006) [2024-09-30 03:04:09,734][1157819] Updated weights for policy 0, policy_version 169758 (0.0006) [2024-09-30 03:04:10,211][1157819] Updated weights for policy 0, policy_version 169768 (0.0006) [2024-09-30 03:04:10,466][1157520] Fps is (10 sec: 83557.3, 60 sec: 80213.2, 300 sec: 78629.3). Total num frames: 695390208. Throughput: 0: 20264.1. Samples: 163791676. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:04:10,466][1157520] Avg episode reward: [(0, '56.703')] [2024-09-30 03:04:10,684][1157819] Updated weights for policy 0, policy_version 169778 (0.0006) [2024-09-30 03:04:11,193][1157819] Updated weights for policy 0, policy_version 169788 (0.0006) [2024-09-30 03:04:11,668][1157819] Updated weights for policy 0, policy_version 169798 (0.0006) [2024-09-30 03:04:12,160][1157819] Updated weights for policy 0, policy_version 169808 (0.0006) [2024-09-30 03:04:12,625][1157819] Updated weights for policy 0, policy_version 169818 (0.0006) [2024-09-30 03:04:13,152][1157819] Updated weights for policy 0, policy_version 169828 (0.0006) [2024-09-30 03:04:13,607][1157819] Updated weights for policy 0, policy_version 169838 (0.0006) [2024-09-30 03:04:14,148][1157819] Updated weights for policy 0, policy_version 169848 (0.0006) [2024-09-30 03:04:14,667][1157819] Updated weights for policy 0, policy_version 169858 (0.0006) [2024-09-30 03:04:15,185][1157819] Updated weights for policy 0, policy_version 169868 (0.0006) [2024-09-30 03:04:15,466][1157520] Fps is (10 sec: 83557.8, 60 sec: 80486.2, 300 sec: 78754.2). Total num frames: 695799808. Throughput: 0: 20416.8. Samples: 163917312. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:04:15,466][1157520] Avg episode reward: [(0, '54.021')] [2024-09-30 03:04:15,700][1157819] Updated weights for policy 0, policy_version 169878 (0.0006) [2024-09-30 03:04:16,210][1157819] Updated weights for policy 0, policy_version 169888 (0.0006) [2024-09-30 03:04:16,711][1157819] Updated weights for policy 0, policy_version 169898 (0.0006) [2024-09-30 03:04:17,198][1157819] Updated weights for policy 0, policy_version 169908 (0.0006) [2024-09-30 03:04:17,693][1157819] Updated weights for policy 0, policy_version 169918 (0.0006) [2024-09-30 03:04:18,209][1157819] Updated weights for policy 0, policy_version 169928 (0.0006) [2024-09-30 03:04:18,693][1157819] Updated weights for policy 0, policy_version 169938 (0.0006) [2024-09-30 03:04:19,191][1157819] Updated weights for policy 0, policy_version 169948 (0.0006) [2024-09-30 03:04:19,679][1157819] Updated weights for policy 0, policy_version 169958 (0.0006) [2024-09-30 03:04:20,182][1157819] Updated weights for policy 0, policy_version 169968 (0.0006) [2024-09-30 03:04:20,466][1157520] Fps is (10 sec: 81920.2, 60 sec: 81100.7, 300 sec: 78795.9). Total num frames: 696209408. Throughput: 0: 20515.0. Samples: 164039100. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:04:20,466][1157520] Avg episode reward: [(0, '56.422')] [2024-09-30 03:04:20,692][1157819] Updated weights for policy 0, policy_version 169978 (0.0006) [2024-09-30 03:04:21,184][1157819] Updated weights for policy 0, policy_version 169988 (0.0006) [2024-09-30 03:04:21,662][1157819] Updated weights for policy 0, policy_version 169998 (0.0006) [2024-09-30 03:04:22,162][1157819] Updated weights for policy 0, policy_version 170008 (0.0006) [2024-09-30 03:04:22,647][1157819] Updated weights for policy 0, policy_version 170018 (0.0006) [2024-09-30 03:04:23,157][1157819] Updated weights for policy 0, policy_version 170028 (0.0006) [2024-09-30 03:04:23,659][1157819] Updated weights for policy 0, policy_version 170038 (0.0006) [2024-09-30 03:04:24,156][1157819] Updated weights for policy 0, policy_version 170048 (0.0006) [2024-09-30 03:04:24,684][1157819] Updated weights for policy 0, policy_version 170058 (0.0006) [2024-09-30 03:04:25,181][1157819] Updated weights for policy 0, policy_version 170068 (0.0006) [2024-09-30 03:04:25,466][1157520] Fps is (10 sec: 81920.6, 60 sec: 81510.5, 300 sec: 78837.6). Total num frames: 696619008. Throughput: 0: 20587.7. Samples: 164101208. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:04:25,466][1157520] Avg episode reward: [(0, '56.006')] [2024-09-30 03:04:25,700][1157819] Updated weights for policy 0, policy_version 170078 (0.0006) [2024-09-30 03:04:26,195][1157819] Updated weights for policy 0, policy_version 170088 (0.0006) [2024-09-30 03:04:26,725][1157819] Updated weights for policy 0, policy_version 170098 (0.0006) [2024-09-30 03:04:27,283][1157819] Updated weights for policy 0, policy_version 170108 (0.0006) [2024-09-30 03:04:27,811][1157819] Updated weights for policy 0, policy_version 170118 (0.0006) [2024-09-30 03:04:28,305][1157819] Updated weights for policy 0, policy_version 170128 (0.0006) [2024-09-30 03:04:28,836][1157819] Updated weights for policy 0, policy_version 170138 (0.0006) [2024-09-30 03:04:29,334][1157819] Updated weights for policy 0, policy_version 170148 (0.0006) [2024-09-30 03:04:29,847][1157819] Updated weights for policy 0, policy_version 170158 (0.0006) [2024-09-30 03:04:30,360][1157819] Updated weights for policy 0, policy_version 170168 (0.0006) [2024-09-30 03:04:30,466][1157520] Fps is (10 sec: 80692.5, 60 sec: 81510.6, 300 sec: 78893.2). Total num frames: 697016320. Throughput: 0: 20565.8. Samples: 164220604. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:04:30,466][1157520] Avg episode reward: [(0, '56.786')] [2024-09-30 03:04:30,853][1157819] Updated weights for policy 0, policy_version 170178 (0.0006) [2024-09-30 03:04:31,357][1157819] Updated weights for policy 0, policy_version 170188 (0.0006) [2024-09-30 03:04:31,835][1157819] Updated weights for policy 0, policy_version 170198 (0.0006) [2024-09-30 03:04:32,332][1157819] Updated weights for policy 0, policy_version 170208 (0.0006) [2024-09-30 03:04:32,818][1157819] Updated weights for policy 0, policy_version 170218 (0.0006) [2024-09-30 03:04:33,299][1157819] Updated weights for policy 0, policy_version 170228 (0.0006) [2024-09-30 03:04:33,789][1157819] Updated weights for policy 0, policy_version 170238 (0.0006) [2024-09-30 03:04:34,274][1157819] Updated weights for policy 0, policy_version 170248 (0.0006) [2024-09-30 03:04:34,772][1157819] Updated weights for policy 0, policy_version 170258 (0.0006) [2024-09-30 03:04:34,945][1157736] Signal inference workers to stop experience collection... (11600 times) [2024-09-30 03:04:34,945][1157736] Signal inference workers to resume experience collection... (11600 times) [2024-09-30 03:04:34,949][1157819] InferenceWorker_p0-w0: stopping experience collection (11600 times) [2024-09-30 03:04:34,949][1157819] InferenceWorker_p0-w0: resuming experience collection (11600 times) [2024-09-30 03:04:35,256][1157819] Updated weights for policy 0, policy_version 170268 (0.0006) [2024-09-30 03:04:35,466][1157520] Fps is (10 sec: 81511.6, 60 sec: 81988.5, 300 sec: 79087.5). Total num frames: 697434112. Throughput: 0: 20522.6. Samples: 164344468. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:04:35,466][1157520] Avg episode reward: [(0, '56.088')] [2024-09-30 03:04:35,775][1157819] Updated weights for policy 0, policy_version 170278 (0.0006) [2024-09-30 03:04:36,239][1157819] Updated weights for policy 0, policy_version 170288 (0.0006) [2024-09-30 03:04:36,731][1157819] Updated weights for policy 0, policy_version 170298 (0.0006) [2024-09-30 03:04:37,235][1157819] Updated weights for policy 0, policy_version 170308 (0.0006) [2024-09-30 03:04:37,723][1157819] Updated weights for policy 0, policy_version 170318 (0.0006) [2024-09-30 03:04:38,215][1157819] Updated weights for policy 0, policy_version 170328 (0.0006) [2024-09-30 03:04:38,704][1157819] Updated weights for policy 0, policy_version 170338 (0.0006) [2024-09-30 03:04:39,199][1157819] Updated weights for policy 0, policy_version 170348 (0.0006) [2024-09-30 03:04:39,691][1157819] Updated weights for policy 0, policy_version 170358 (0.0005) [2024-09-30 03:04:40,182][1157819] Updated weights for policy 0, policy_version 170368 (0.0006) [2024-09-30 03:04:40,466][1157520] Fps is (10 sec: 83558.4, 60 sec: 82398.0, 300 sec: 79295.8). Total num frames: 697851904. Throughput: 0: 20520.3. Samples: 164406916. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:04:40,466][1157520] Avg episode reward: [(0, '55.224')] [2024-09-30 03:04:40,668][1157819] Updated weights for policy 0, policy_version 170378 (0.0005) [2024-09-30 03:04:41,109][1157819] Updated weights for policy 0, policy_version 170388 (0.0006) [2024-09-30 03:04:41,565][1157819] Updated weights for policy 0, policy_version 170398 (0.0006) [2024-09-30 03:04:42,041][1157819] Updated weights for policy 0, policy_version 170408 (0.0006) [2024-09-30 03:04:42,532][1157819] Updated weights for policy 0, policy_version 170418 (0.0006) [2024-09-30 03:04:43,002][1157819] Updated weights for policy 0, policy_version 170428 (0.0006) [2024-09-30 03:04:43,468][1157819] Updated weights for policy 0, policy_version 170438 (0.0006) [2024-09-30 03:04:43,915][1157819] Updated weights for policy 0, policy_version 170448 (0.0006) [2024-09-30 03:04:44,404][1157819] Updated weights for policy 0, policy_version 170458 (0.0006) [2024-09-30 03:04:44,879][1157819] Updated weights for policy 0, policy_version 170468 (0.0006) [2024-09-30 03:04:45,334][1157819] Updated weights for policy 0, policy_version 170478 (0.0006) [2024-09-30 03:04:45,466][1157520] Fps is (10 sec: 85196.8, 60 sec: 82807.6, 300 sec: 79476.3). Total num frames: 698286080. Throughput: 0: 20653.8. Samples: 164535636. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:04:45,466][1157520] Avg episode reward: [(0, '54.289')] [2024-09-30 03:04:45,817][1157819] Updated weights for policy 0, policy_version 170488 (0.0006) [2024-09-30 03:04:46,282][1157819] Updated weights for policy 0, policy_version 170498 (0.0006) [2024-09-30 03:04:46,739][1157819] Updated weights for policy 0, policy_version 170508 (0.0006) [2024-09-30 03:04:47,222][1157819] Updated weights for policy 0, policy_version 170518 (0.0006) [2024-09-30 03:04:47,665][1157819] Updated weights for policy 0, policy_version 170528 (0.0006) [2024-09-30 03:04:48,139][1157819] Updated weights for policy 0, policy_version 170538 (0.0006) [2024-09-30 03:04:48,628][1157819] Updated weights for policy 0, policy_version 170548 (0.0006) [2024-09-30 03:04:49,105][1157819] Updated weights for policy 0, policy_version 170558 (0.0006) [2024-09-30 03:04:49,552][1157819] Updated weights for policy 0, policy_version 170568 (0.0006) [2024-09-30 03:04:50,041][1157819] Updated weights for policy 0, policy_version 170578 (0.0006) [2024-09-30 03:04:50,466][1157520] Fps is (10 sec: 87244.8, 60 sec: 83080.6, 300 sec: 79629.0). Total num frames: 698724352. Throughput: 0: 20885.9. Samples: 164666644. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:04:50,466][1157520] Avg episode reward: [(0, '53.603')] [2024-09-30 03:04:50,513][1157819] Updated weights for policy 0, policy_version 170588 (0.0006) [2024-09-30 03:04:50,981][1157819] Updated weights for policy 0, policy_version 170598 (0.0006) [2024-09-30 03:04:51,483][1157819] Updated weights for policy 0, policy_version 170608 (0.0006) [2024-09-30 03:04:52,000][1157819] Updated weights for policy 0, policy_version 170618 (0.0006) [2024-09-30 03:04:52,495][1157819] Updated weights for policy 0, policy_version 170628 (0.0006) [2024-09-30 03:04:52,994][1157819] Updated weights for policy 0, policy_version 170638 (0.0006) [2024-09-30 03:04:53,500][1157819] Updated weights for policy 0, policy_version 170648 (0.0006) [2024-09-30 03:04:54,020][1157819] Updated weights for policy 0, policy_version 170658 (0.0006) [2024-09-30 03:04:54,538][1157819] Updated weights for policy 0, policy_version 170668 (0.0006) [2024-09-30 03:04:55,093][1157819] Updated weights for policy 0, policy_version 170678 (0.0006) [2024-09-30 03:04:55,466][1157520] Fps is (10 sec: 83967.0, 60 sec: 82875.8, 300 sec: 79712.3). Total num frames: 699125760. Throughput: 0: 20829.4. Samples: 164728996. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:04:55,466][1157520] Avg episode reward: [(0, '53.154')] [2024-09-30 03:04:55,583][1157819] Updated weights for policy 0, policy_version 170688 (0.0006) [2024-09-30 03:04:56,089][1157819] Updated weights for policy 0, policy_version 170698 (0.0006) [2024-09-30 03:04:56,603][1157819] Updated weights for policy 0, policy_version 170708 (0.0006) [2024-09-30 03:04:57,123][1157819] Updated weights for policy 0, policy_version 170718 (0.0006) [2024-09-30 03:04:57,629][1157819] Updated weights for policy 0, policy_version 170728 (0.0006) [2024-09-30 03:04:58,119][1157819] Updated weights for policy 0, policy_version 170738 (0.0006) [2024-09-30 03:04:58,638][1157819] Updated weights for policy 0, policy_version 170748 (0.0006) [2024-09-30 03:04:59,159][1157819] Updated weights for policy 0, policy_version 170758 (0.0006) [2024-09-30 03:04:59,673][1157819] Updated weights for policy 0, policy_version 170768 (0.0006) [2024-09-30 03:05:00,179][1157819] Updated weights for policy 0, policy_version 170778 (0.0006) [2024-09-30 03:05:00,466][1157520] Fps is (10 sec: 80280.7, 60 sec: 82875.7, 300 sec: 79740.1). Total num frames: 699527168. Throughput: 0: 20696.6. Samples: 164848656. Policy #0 lag: (min: 0.0, avg: 2.0, max: 4.0) [2024-09-30 03:05:00,466][1157520] Avg episode reward: [(0, '57.168')] [2024-09-30 03:05:00,674][1157819] Updated weights for policy 0, policy_version 170788 (0.0006) [2024-09-30 03:05:01,208][1157819] Updated weights for policy 0, policy_version 170798 (0.0006) [2024-09-30 03:05:01,689][1157819] Updated weights for policy 0, policy_version 170808 (0.0006) [2024-09-30 03:05:02,174][1157819] Updated weights for policy 0, policy_version 170818 (0.0006) [2024-09-30 03:05:02,630][1157819] Updated weights for policy 0, policy_version 170828 (0.0006) [2024-09-30 03:05:03,114][1157819] Updated weights for policy 0, policy_version 170838 (0.0006) [2024-09-30 03:05:03,615][1157819] Updated weights for policy 0, policy_version 170848 (0.0006) [2024-09-30 03:05:04,103][1157819] Updated weights for policy 0, policy_version 170858 (0.0006) [2024-09-30 03:05:04,593][1157819] Updated weights for policy 0, policy_version 170868 (0.0006) [2024-09-30 03:05:05,062][1157819] Updated weights for policy 0, policy_version 170878 (0.0006) [2024-09-30 03:05:05,466][1157520] Fps is (10 sec: 82329.7, 60 sec: 83080.6, 300 sec: 79851.2). Total num frames: 699949056. Throughput: 0: 20760.8. Samples: 164973336. Policy #0 lag: (min: 0.0, avg: 2.0, max: 4.0) [2024-09-30 03:05:05,466][1157520] Avg episode reward: [(0, '55.834')] [2024-09-30 03:05:05,527][1157819] Updated weights for policy 0, policy_version 170888 (0.0006) [2024-09-30 03:05:06,036][1157819] Updated weights for policy 0, policy_version 170898 (0.0006) [2024-09-30 03:05:06,422][1157736] Signal inference workers to stop experience collection... (11650 times) [2024-09-30 03:05:06,423][1157736] Signal inference workers to resume experience collection... (11650 times) [2024-09-30 03:05:06,427][1157819] InferenceWorker_p0-w0: stopping experience collection (11650 times) [2024-09-30 03:05:06,429][1157819] InferenceWorker_p0-w0: resuming experience collection (11650 times) [2024-09-30 03:05:06,546][1157819] Updated weights for policy 0, policy_version 170908 (0.0006) [2024-09-30 03:05:07,064][1157819] Updated weights for policy 0, policy_version 170918 (0.0006) [2024-09-30 03:05:07,564][1157819] Updated weights for policy 0, policy_version 170928 (0.0006) [2024-09-30 03:05:08,065][1157819] Updated weights for policy 0, policy_version 170938 (0.0006) [2024-09-30 03:05:08,554][1157819] Updated weights for policy 0, policy_version 170948 (0.0006) [2024-09-30 03:05:09,059][1157819] Updated weights for policy 0, policy_version 170958 (0.0006) [2024-09-30 03:05:09,605][1157819] Updated weights for policy 0, policy_version 170968 (0.0006) [2024-09-30 03:05:10,110][1157819] Updated weights for policy 0, policy_version 170978 (0.0006) [2024-09-30 03:05:10,466][1157520] Fps is (10 sec: 82329.8, 60 sec: 82671.1, 300 sec: 79781.8). Total num frames: 700350464. Throughput: 0: 20751.3. Samples: 165035016. Policy #0 lag: (min: 0.0, avg: 2.0, max: 4.0) [2024-09-30 03:05:10,466][1157520] Avg episode reward: [(0, '57.229')] [2024-09-30 03:05:10,634][1157819] Updated weights for policy 0, policy_version 170988 (0.0006) [2024-09-30 03:05:11,147][1157819] Updated weights for policy 0, policy_version 170998 (0.0006) [2024-09-30 03:05:11,668][1157819] Updated weights for policy 0, policy_version 171008 (0.0006) [2024-09-30 03:05:12,165][1157819] Updated weights for policy 0, policy_version 171018 (0.0006) [2024-09-30 03:05:12,665][1157819] Updated weights for policy 0, policy_version 171028 (0.0006) [2024-09-30 03:05:13,184][1157819] Updated weights for policy 0, policy_version 171038 (0.0006) [2024-09-30 03:05:13,708][1157819] Updated weights for policy 0, policy_version 171048 (0.0006) [2024-09-30 03:05:14,193][1157819] Updated weights for policy 0, policy_version 171058 (0.0006) [2024-09-30 03:05:14,696][1157819] Updated weights for policy 0, policy_version 171068 (0.0006) [2024-09-30 03:05:15,218][1157819] Updated weights for policy 0, policy_version 171078 (0.0006) [2024-09-30 03:05:15,466][1157520] Fps is (10 sec: 80281.5, 60 sec: 82534.5, 300 sec: 79948.4). Total num frames: 700751872. Throughput: 0: 20766.4. Samples: 165155096. Policy #0 lag: (min: 0.0, avg: 2.0, max: 4.0) [2024-09-30 03:05:15,466][1157520] Avg episode reward: [(0, '56.586')] [2024-09-30 03:05:15,743][1157819] Updated weights for policy 0, policy_version 171088 (0.0006) [2024-09-30 03:05:16,207][1157819] Updated weights for policy 0, policy_version 171098 (0.0006) [2024-09-30 03:05:16,691][1157819] Updated weights for policy 0, policy_version 171108 (0.0006) [2024-09-30 03:05:17,142][1157819] Updated weights for policy 0, policy_version 171118 (0.0006) [2024-09-30 03:05:17,631][1157819] Updated weights for policy 0, policy_version 171128 (0.0006) [2024-09-30 03:05:18,127][1157819] Updated weights for policy 0, policy_version 171138 (0.0006) [2024-09-30 03:05:18,626][1157819] Updated weights for policy 0, policy_version 171148 (0.0006) [2024-09-30 03:05:19,107][1157819] Updated weights for policy 0, policy_version 171158 (0.0006) [2024-09-30 03:05:19,594][1157819] Updated weights for policy 0, policy_version 171168 (0.0006) [2024-09-30 03:05:20,070][1157819] Updated weights for policy 0, policy_version 171178 (0.0006) [2024-09-30 03:05:20,466][1157520] Fps is (10 sec: 82739.3, 60 sec: 82807.6, 300 sec: 80239.9). Total num frames: 701177856. Throughput: 0: 20795.7. Samples: 165280276. Policy #0 lag: (min: 0.0, avg: 2.0, max: 4.0) [2024-09-30 03:05:20,466][1157520] Avg episode reward: [(0, '55.045')] [2024-09-30 03:05:20,556][1157819] Updated weights for policy 0, policy_version 171188 (0.0006) [2024-09-30 03:05:21,039][1157819] Updated weights for policy 0, policy_version 171198 (0.0006) [2024-09-30 03:05:21,542][1157819] Updated weights for policy 0, policy_version 171208 (0.0006) [2024-09-30 03:05:22,033][1157819] Updated weights for policy 0, policy_version 171218 (0.0005) [2024-09-30 03:05:22,517][1157819] Updated weights for policy 0, policy_version 171228 (0.0006) [2024-09-30 03:05:23,001][1157819] Updated weights for policy 0, policy_version 171238 (0.0006) [2024-09-30 03:05:23,465][1157819] Updated weights for policy 0, policy_version 171248 (0.0006) [2024-09-30 03:05:23,950][1157819] Updated weights for policy 0, policy_version 171258 (0.0006) [2024-09-30 03:05:24,450][1157819] Updated weights for policy 0, policy_version 171268 (0.0006) [2024-09-30 03:05:24,942][1157819] Updated weights for policy 0, policy_version 171278 (0.0006) [2024-09-30 03:05:25,421][1157819] Updated weights for policy 0, policy_version 171288 (0.0006) [2024-09-30 03:05:25,466][1157520] Fps is (10 sec: 84787.2, 60 sec: 83012.3, 300 sec: 80406.5). Total num frames: 701599744. Throughput: 0: 20809.0. Samples: 165343324. Policy #0 lag: (min: 0.0, avg: 2.0, max: 4.0) [2024-09-30 03:05:25,466][1157520] Avg episode reward: [(0, '57.131')] [2024-09-30 03:05:25,925][1157819] Updated weights for policy 0, policy_version 171298 (0.0006) [2024-09-30 03:05:26,373][1157819] Updated weights for policy 0, policy_version 171308 (0.0006) [2024-09-30 03:05:26,845][1157819] Updated weights for policy 0, policy_version 171318 (0.0006) [2024-09-30 03:05:27,307][1157819] Updated weights for policy 0, policy_version 171328 (0.0006) [2024-09-30 03:05:27,782][1157819] Updated weights for policy 0, policy_version 171338 (0.0006) [2024-09-30 03:05:28,263][1157819] Updated weights for policy 0, policy_version 171348 (0.0006) [2024-09-30 03:05:28,731][1157819] Updated weights for policy 0, policy_version 171358 (0.0006) [2024-09-30 03:05:29,209][1157819] Updated weights for policy 0, policy_version 171368 (0.0006) [2024-09-30 03:05:29,671][1157819] Updated weights for policy 0, policy_version 171378 (0.0006) [2024-09-30 03:05:30,135][1157819] Updated weights for policy 0, policy_version 171388 (0.0006) [2024-09-30 03:05:30,466][1157520] Fps is (10 sec: 85607.1, 60 sec: 83626.7, 300 sec: 80573.3). Total num frames: 702033920. Throughput: 0: 20804.3. Samples: 165471828. Policy #0 lag: (min: 0.0, avg: 2.0, max: 4.0) [2024-09-30 03:05:30,466][1157520] Avg episode reward: [(0, '56.426')] [2024-09-30 03:05:30,607][1157819] Updated weights for policy 0, policy_version 171398 (0.0006) [2024-09-30 03:05:31,076][1157819] Updated weights for policy 0, policy_version 171408 (0.0006) [2024-09-30 03:05:31,550][1157819] Updated weights for policy 0, policy_version 171418 (0.0006) [2024-09-30 03:05:32,017][1157819] Updated weights for policy 0, policy_version 171428 (0.0006) [2024-09-30 03:05:32,464][1157819] Updated weights for policy 0, policy_version 171438 (0.0006) [2024-09-30 03:05:32,935][1157819] Updated weights for policy 0, policy_version 171448 (0.0006) [2024-09-30 03:05:33,427][1157819] Updated weights for policy 0, policy_version 171458 (0.0006) [2024-09-30 03:05:33,889][1157819] Updated weights for policy 0, policy_version 171468 (0.0006) [2024-09-30 03:05:34,339][1157819] Updated weights for policy 0, policy_version 171478 (0.0006) [2024-09-30 03:05:34,809][1157819] Updated weights for policy 0, policy_version 171488 (0.0006) [2024-09-30 03:05:35,260][1157819] Updated weights for policy 0, policy_version 171498 (0.0006) [2024-09-30 03:05:35,466][1157520] Fps is (10 sec: 87245.7, 60 sec: 83968.0, 300 sec: 80725.9). Total num frames: 702472192. Throughput: 0: 20814.3. Samples: 165603288. Policy #0 lag: (min: 0.0, avg: 2.0, max: 4.0) [2024-09-30 03:05:35,466][1157520] Avg episode reward: [(0, '56.618')] [2024-09-30 03:05:35,754][1157819] Updated weights for policy 0, policy_version 171508 (0.0006) [2024-09-30 03:05:36,193][1157819] Updated weights for policy 0, policy_version 171518 (0.0006) [2024-09-30 03:05:36,679][1157819] Updated weights for policy 0, policy_version 171528 (0.0006) [2024-09-30 03:05:37,159][1157819] Updated weights for policy 0, policy_version 171538 (0.0006) [2024-09-30 03:05:37,649][1157819] Updated weights for policy 0, policy_version 171548 (0.0006) [2024-09-30 03:05:38,116][1157819] Updated weights for policy 0, policy_version 171558 (0.0006) [2024-09-30 03:05:38,630][1157819] Updated weights for policy 0, policy_version 171568 (0.0006) [2024-09-30 03:05:39,127][1157819] Updated weights for policy 0, policy_version 171578 (0.0006) [2024-09-30 03:05:39,613][1157819] Updated weights for policy 0, policy_version 171588 (0.0006) [2024-09-30 03:05:40,118][1157819] Updated weights for policy 0, policy_version 171598 (0.0006) [2024-09-30 03:05:40,466][1157520] Fps is (10 sec: 86015.3, 60 sec: 84036.2, 300 sec: 80823.1). Total num frames: 702894080. Throughput: 0: 20872.3. Samples: 165668248. Policy #0 lag: (min: 0.0, avg: 2.0, max: 4.0) [2024-09-30 03:05:40,466][1157520] Avg episode reward: [(0, '56.734')] [2024-09-30 03:05:40,615][1157819] Updated weights for policy 0, policy_version 171608 (0.0006) [2024-09-30 03:05:41,129][1157819] Updated weights for policy 0, policy_version 171618 (0.0006) [2024-09-30 03:05:41,709][1157819] Updated weights for policy 0, policy_version 171628 (0.0006) [2024-09-30 03:05:42,206][1157819] Updated weights for policy 0, policy_version 171638 (0.0006) [2024-09-30 03:05:42,698][1157819] Updated weights for policy 0, policy_version 171648 (0.0006) [2024-09-30 03:05:43,218][1157819] Updated weights for policy 0, policy_version 171658 (0.0006) [2024-09-30 03:05:43,704][1157819] Updated weights for policy 0, policy_version 171668 (0.0006) [2024-09-30 03:05:44,192][1157819] Updated weights for policy 0, policy_version 171678 (0.0007) [2024-09-30 03:05:44,676][1157819] Updated weights for policy 0, policy_version 171688 (0.0006) [2024-09-30 03:05:44,986][1157736] Signal inference workers to stop experience collection... (11700 times) [2024-09-30 03:05:44,989][1157736] Signal inference workers to resume experience collection... (11700 times) [2024-09-30 03:05:44,994][1157819] InferenceWorker_p0-w0: stopping experience collection (11700 times) [2024-09-30 03:05:44,996][1157819] InferenceWorker_p0-w0: resuming experience collection (11700 times) [2024-09-30 03:05:45,166][1157819] Updated weights for policy 0, policy_version 171698 (0.0006) [2024-09-30 03:05:45,466][1157520] Fps is (10 sec: 82738.3, 60 sec: 83558.2, 300 sec: 80837.0). Total num frames: 703299584. Throughput: 0: 20914.2. Samples: 165789796. Policy #0 lag: (min: 0.0, avg: 2.0, max: 4.0) [2024-09-30 03:05:45,466][1157520] Avg episode reward: [(0, '53.901')] [2024-09-30 03:05:45,659][1157819] Updated weights for policy 0, policy_version 171708 (0.0006) [2024-09-30 03:05:46,161][1157819] Updated weights for policy 0, policy_version 171718 (0.0006) [2024-09-30 03:05:46,656][1157819] Updated weights for policy 0, policy_version 171728 (0.0006) [2024-09-30 03:05:47,138][1157819] Updated weights for policy 0, policy_version 171738 (0.0006) [2024-09-30 03:05:47,635][1157819] Updated weights for policy 0, policy_version 171748 (0.0006) [2024-09-30 03:05:48,135][1157819] Updated weights for policy 0, policy_version 171758 (0.0006) [2024-09-30 03:05:48,619][1157819] Updated weights for policy 0, policy_version 171768 (0.0006) [2024-09-30 03:05:49,124][1157819] Updated weights for policy 0, policy_version 171778 (0.0006) [2024-09-30 03:05:49,623][1157819] Updated weights for policy 0, policy_version 171788 (0.0006) [2024-09-30 03:05:50,097][1157819] Updated weights for policy 0, policy_version 171798 (0.0006) [2024-09-30 03:05:50,466][1157520] Fps is (10 sec: 81919.9, 60 sec: 83148.7, 300 sec: 80878.6). Total num frames: 703713280. Throughput: 0: 20918.2. Samples: 165914656. Policy #0 lag: (min: 0.0, avg: 2.0, max: 4.0) [2024-09-30 03:05:50,466][1157520] Avg episode reward: [(0, '56.020')] [2024-09-30 03:05:50,594][1157819] Updated weights for policy 0, policy_version 171808 (0.0006) [2024-09-30 03:05:51,074][1157819] Updated weights for policy 0, policy_version 171818 (0.0006) [2024-09-30 03:05:51,577][1157819] Updated weights for policy 0, policy_version 171828 (0.0006) [2024-09-30 03:05:52,032][1157819] Updated weights for policy 0, policy_version 171838 (0.0006) [2024-09-30 03:05:52,539][1157819] Updated weights for policy 0, policy_version 171848 (0.0006) [2024-09-30 03:05:53,028][1157819] Updated weights for policy 0, policy_version 171858 (0.0006) [2024-09-30 03:05:53,485][1157819] Updated weights for policy 0, policy_version 171868 (0.0006) [2024-09-30 03:05:53,982][1157819] Updated weights for policy 0, policy_version 171878 (0.0006) [2024-09-30 03:05:54,470][1157819] Updated weights for policy 0, policy_version 171888 (0.0005) [2024-09-30 03:05:54,960][1157819] Updated weights for policy 0, policy_version 171898 (0.0006) [2024-09-30 03:05:55,450][1157819] Updated weights for policy 0, policy_version 171908 (0.0006) [2024-09-30 03:05:55,466][1157520] Fps is (10 sec: 83558.5, 60 sec: 83490.1, 300 sec: 80989.7). Total num frames: 704135168. Throughput: 0: 20951.5. Samples: 165977832. Policy #0 lag: (min: 0.0, avg: 2.0, max: 4.0) [2024-09-30 03:05:55,466][1157520] Avg episode reward: [(0, '55.543')] [2024-09-30 03:05:55,908][1157819] Updated weights for policy 0, policy_version 171918 (0.0006) [2024-09-30 03:05:56,407][1157819] Updated weights for policy 0, policy_version 171928 (0.0006) [2024-09-30 03:05:56,903][1157819] Updated weights for policy 0, policy_version 171938 (0.0006) [2024-09-30 03:05:57,402][1157819] Updated weights for policy 0, policy_version 171948 (0.0006) [2024-09-30 03:05:57,893][1157819] Updated weights for policy 0, policy_version 171958 (0.0006) [2024-09-30 03:05:58,401][1157819] Updated weights for policy 0, policy_version 171968 (0.0006) [2024-09-30 03:05:58,911][1157819] Updated weights for policy 0, policy_version 171978 (0.0006) [2024-09-30 03:05:59,430][1157819] Updated weights for policy 0, policy_version 171988 (0.0006) [2024-09-30 03:05:59,925][1157819] Updated weights for policy 0, policy_version 171998 (0.0006) [2024-09-30 03:06:00,466][1157520] Fps is (10 sec: 82739.3, 60 sec: 83558.5, 300 sec: 80989.7). Total num frames: 704540672. Throughput: 0: 21061.6. Samples: 166102868. Policy #0 lag: (min: 0.0, avg: 2.0, max: 4.0) [2024-09-30 03:06:00,466][1157520] Avg episode reward: [(0, '55.888')] [2024-09-30 03:06:00,472][1157819] Updated weights for policy 0, policy_version 172008 (0.0007) [2024-09-30 03:06:00,934][1157819] Updated weights for policy 0, policy_version 172018 (0.0006) [2024-09-30 03:06:01,448][1157819] Updated weights for policy 0, policy_version 172028 (0.0006) [2024-09-30 03:06:01,949][1157819] Updated weights for policy 0, policy_version 172038 (0.0006) [2024-09-30 03:06:02,447][1157819] Updated weights for policy 0, policy_version 172048 (0.0006) [2024-09-30 03:06:02,946][1157819] Updated weights for policy 0, policy_version 172058 (0.0007) [2024-09-30 03:06:03,462][1157819] Updated weights for policy 0, policy_version 172068 (0.0007) [2024-09-30 03:06:03,958][1157819] Updated weights for policy 0, policy_version 172078 (0.0006) [2024-09-30 03:06:04,445][1157819] Updated weights for policy 0, policy_version 172088 (0.0006) [2024-09-30 03:06:04,908][1157819] Updated weights for policy 0, policy_version 172098 (0.0006) [2024-09-30 03:06:05,413][1157819] Updated weights for policy 0, policy_version 172108 (0.0007) [2024-09-30 03:06:05,466][1157520] Fps is (10 sec: 81918.4, 60 sec: 83421.6, 300 sec: 80975.8). Total num frames: 704954368. Throughput: 0: 21003.3. Samples: 166225428. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 03:06:05,466][1157520] Avg episode reward: [(0, '55.531')] [2024-09-30 03:06:05,471][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000172108_704954368.pth... [2024-09-30 03:06:05,518][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000167319_685338624.pth [2024-09-30 03:06:05,995][1157819] Updated weights for policy 0, policy_version 172118 (0.0007) [2024-09-30 03:06:06,524][1157819] Updated weights for policy 0, policy_version 172128 (0.0006) [2024-09-30 03:06:07,073][1157819] Updated weights for policy 0, policy_version 172138 (0.0006) [2024-09-30 03:06:07,595][1157819] Updated weights for policy 0, policy_version 172148 (0.0006) [2024-09-30 03:06:08,125][1157819] Updated weights for policy 0, policy_version 172158 (0.0006) [2024-09-30 03:06:08,661][1157819] Updated weights for policy 0, policy_version 172168 (0.0006) [2024-09-30 03:06:09,203][1157819] Updated weights for policy 0, policy_version 172178 (0.0007) [2024-09-30 03:06:09,729][1157819] Updated weights for policy 0, policy_version 172188 (0.0006) [2024-09-30 03:06:10,242][1157819] Updated weights for policy 0, policy_version 172198 (0.0006) [2024-09-30 03:06:10,466][1157520] Fps is (10 sec: 79871.7, 60 sec: 83148.8, 300 sec: 80934.2). Total num frames: 705339392. Throughput: 0: 20886.6. Samples: 166283220. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 03:06:10,466][1157520] Avg episode reward: [(0, '55.822')] [2024-09-30 03:06:10,790][1157819] Updated weights for policy 0, policy_version 172208 (0.0007) [2024-09-30 03:06:11,306][1157819] Updated weights for policy 0, policy_version 172218 (0.0007) [2024-09-30 03:06:11,851][1157819] Updated weights for policy 0, policy_version 172228 (0.0006) [2024-09-30 03:06:12,427][1157819] Updated weights for policy 0, policy_version 172238 (0.0006) [2024-09-30 03:06:12,931][1157819] Updated weights for policy 0, policy_version 172248 (0.0006) [2024-09-30 03:06:13,440][1157819] Updated weights for policy 0, policy_version 172258 (0.0007) [2024-09-30 03:06:13,950][1157819] Updated weights for policy 0, policy_version 172268 (0.0006) [2024-09-30 03:06:14,470][1157819] Updated weights for policy 0, policy_version 172278 (0.0006) [2024-09-30 03:06:14,987][1157819] Updated weights for policy 0, policy_version 172288 (0.0006) [2024-09-30 03:06:15,466][1157520] Fps is (10 sec: 77006.1, 60 sec: 82875.7, 300 sec: 80934.1). Total num frames: 705724416. Throughput: 0: 20614.5. Samples: 166399484. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 03:06:15,466][1157520] Avg episode reward: [(0, '56.504')] [2024-09-30 03:06:15,541][1157819] Updated weights for policy 0, policy_version 172298 (0.0006) [2024-09-30 03:06:16,089][1157819] Updated weights for policy 0, policy_version 172308 (0.0006) [2024-09-30 03:06:16,606][1157819] Updated weights for policy 0, policy_version 172318 (0.0006) [2024-09-30 03:06:17,110][1157819] Updated weights for policy 0, policy_version 172328 (0.0006) [2024-09-30 03:06:17,622][1157819] Updated weights for policy 0, policy_version 172338 (0.0006) [2024-09-30 03:06:18,126][1157819] Updated weights for policy 0, policy_version 172348 (0.0006) [2024-09-30 03:06:18,673][1157819] Updated weights for policy 0, policy_version 172358 (0.0006) [2024-09-30 03:06:19,177][1157819] Updated weights for policy 0, policy_version 172368 (0.0006) [2024-09-30 03:06:19,692][1157819] Updated weights for policy 0, policy_version 172378 (0.0006) [2024-09-30 03:06:20,211][1157819] Updated weights for policy 0, policy_version 172388 (0.0006) [2024-09-30 03:06:20,466][1157520] Fps is (10 sec: 78233.4, 60 sec: 82397.8, 300 sec: 81017.5). Total num frames: 706121728. Throughput: 0: 20311.5. Samples: 166517308. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 03:06:20,466][1157520] Avg episode reward: [(0, '55.977')] [2024-09-30 03:06:20,700][1157819] Updated weights for policy 0, policy_version 172398 (0.0006) [2024-09-30 03:06:21,203][1157819] Updated weights for policy 0, policy_version 172408 (0.0006) [2024-09-30 03:06:21,718][1157819] Updated weights for policy 0, policy_version 172418 (0.0006) [2024-09-30 03:06:22,240][1157819] Updated weights for policy 0, policy_version 172428 (0.0006) [2024-09-30 03:06:22,760][1157819] Updated weights for policy 0, policy_version 172438 (0.0006) [2024-09-30 03:06:23,277][1157819] Updated weights for policy 0, policy_version 172448 (0.0006) [2024-09-30 03:06:23,504][1157736] Signal inference workers to stop experience collection... (11750 times) [2024-09-30 03:06:23,507][1157819] InferenceWorker_p0-w0: stopping experience collection (11750 times) [2024-09-30 03:06:23,513][1157736] Signal inference workers to resume experience collection... (11750 times) [2024-09-30 03:06:23,513][1157819] InferenceWorker_p0-w0: resuming experience collection (11750 times) [2024-09-30 03:06:23,804][1157819] Updated weights for policy 0, policy_version 172458 (0.0006) [2024-09-30 03:06:24,310][1157819] Updated weights for policy 0, policy_version 172468 (0.0006) [2024-09-30 03:06:24,866][1157819] Updated weights for policy 0, policy_version 172478 (0.0006) [2024-09-30 03:06:25,353][1157819] Updated weights for policy 0, policy_version 172488 (0.0006) [2024-09-30 03:06:25,466][1157520] Fps is (10 sec: 79052.7, 60 sec: 81920.0, 300 sec: 81211.9). Total num frames: 706514944. Throughput: 0: 20194.6. Samples: 166577008. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 03:06:25,466][1157520] Avg episode reward: [(0, '56.275')] [2024-09-30 03:06:25,870][1157819] Updated weights for policy 0, policy_version 172498 (0.0006) [2024-09-30 03:06:26,386][1157819] Updated weights for policy 0, policy_version 172508 (0.0006) [2024-09-30 03:06:26,902][1157819] Updated weights for policy 0, policy_version 172518 (0.0006) [2024-09-30 03:06:27,445][1157819] Updated weights for policy 0, policy_version 172528 (0.0006) [2024-09-30 03:06:27,954][1157819] Updated weights for policy 0, policy_version 172538 (0.0006) [2024-09-30 03:06:28,483][1157819] Updated weights for policy 0, policy_version 172548 (0.0006) [2024-09-30 03:06:28,984][1157819] Updated weights for policy 0, policy_version 172558 (0.0006) [2024-09-30 03:06:29,485][1157819] Updated weights for policy 0, policy_version 172568 (0.0006) [2024-09-30 03:06:30,026][1157819] Updated weights for policy 0, policy_version 172578 (0.0006) [2024-09-30 03:06:30,466][1157520] Fps is (10 sec: 79463.0, 60 sec: 81373.8, 300 sec: 81309.0). Total num frames: 706916352. Throughput: 0: 20126.1. Samples: 166695468. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 03:06:30,466][1157520] Avg episode reward: [(0, '56.572')] [2024-09-30 03:06:30,539][1157819] Updated weights for policy 0, policy_version 172588 (0.0006) [2024-09-30 03:06:31,036][1157819] Updated weights for policy 0, policy_version 172598 (0.0007) [2024-09-30 03:06:31,537][1157819] Updated weights for policy 0, policy_version 172608 (0.0006) [2024-09-30 03:06:31,996][1157819] Updated weights for policy 0, policy_version 172618 (0.0006) [2024-09-30 03:06:32,484][1157819] Updated weights for policy 0, policy_version 172628 (0.0006) [2024-09-30 03:06:32,985][1157819] Updated weights for policy 0, policy_version 172638 (0.0006) [2024-09-30 03:06:33,485][1157819] Updated weights for policy 0, policy_version 172648 (0.0006) [2024-09-30 03:06:33,976][1157819] Updated weights for policy 0, policy_version 172658 (0.0006) [2024-09-30 03:06:34,472][1157819] Updated weights for policy 0, policy_version 172668 (0.0006) [2024-09-30 03:06:34,957][1157819] Updated weights for policy 0, policy_version 172678 (0.0005) [2024-09-30 03:06:35,464][1157819] Updated weights for policy 0, policy_version 172688 (0.0006) [2024-09-30 03:06:35,466][1157520] Fps is (10 sec: 81511.3, 60 sec: 80964.2, 300 sec: 81336.8). Total num frames: 707330048. Throughput: 0: 20087.0. Samples: 166818568. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 03:06:35,466][1157520] Avg episode reward: [(0, '53.924')] [2024-09-30 03:06:35,947][1157819] Updated weights for policy 0, policy_version 172698 (0.0006) [2024-09-30 03:06:36,438][1157819] Updated weights for policy 0, policy_version 172708 (0.0006) [2024-09-30 03:06:36,937][1157819] Updated weights for policy 0, policy_version 172718 (0.0006) [2024-09-30 03:06:37,434][1157819] Updated weights for policy 0, policy_version 172728 (0.0006) [2024-09-30 03:06:37,931][1157819] Updated weights for policy 0, policy_version 172738 (0.0006) [2024-09-30 03:06:38,418][1157819] Updated weights for policy 0, policy_version 172748 (0.0006) [2024-09-30 03:06:38,903][1157819] Updated weights for policy 0, policy_version 172758 (0.0006) [2024-09-30 03:06:39,394][1157819] Updated weights for policy 0, policy_version 172768 (0.0006) [2024-09-30 03:06:39,866][1157819] Updated weights for policy 0, policy_version 172778 (0.0006) [2024-09-30 03:06:40,373][1157819] Updated weights for policy 0, policy_version 172788 (0.0006) [2024-09-30 03:06:40,466][1157520] Fps is (10 sec: 83149.2, 60 sec: 80896.1, 300 sec: 81489.6). Total num frames: 707747840. Throughput: 0: 20069.2. Samples: 166880944. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 03:06:40,466][1157520] Avg episode reward: [(0, '56.919')] [2024-09-30 03:06:40,839][1157819] Updated weights for policy 0, policy_version 172798 (0.0006) [2024-09-30 03:06:41,321][1157819] Updated weights for policy 0, policy_version 172808 (0.0006) [2024-09-30 03:06:41,810][1157819] Updated weights for policy 0, policy_version 172818 (0.0006) [2024-09-30 03:06:42,306][1157819] Updated weights for policy 0, policy_version 172828 (0.0006) [2024-09-30 03:06:42,816][1157819] Updated weights for policy 0, policy_version 172838 (0.0006) [2024-09-30 03:06:43,286][1157819] Updated weights for policy 0, policy_version 172848 (0.0006) [2024-09-30 03:06:43,779][1157819] Updated weights for policy 0, policy_version 172858 (0.0006) [2024-09-30 03:06:44,278][1157819] Updated weights for policy 0, policy_version 172868 (0.0006) [2024-09-30 03:06:44,770][1157819] Updated weights for policy 0, policy_version 172878 (0.0006) [2024-09-30 03:06:45,255][1157819] Updated weights for policy 0, policy_version 172888 (0.0006) [2024-09-30 03:06:45,466][1157520] Fps is (10 sec: 83149.0, 60 sec: 81032.7, 300 sec: 81517.3). Total num frames: 708161536. Throughput: 0: 20082.4. Samples: 167006576. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 03:06:45,466][1157520] Avg episode reward: [(0, '53.802')] [2024-09-30 03:06:45,755][1157819] Updated weights for policy 0, policy_version 172898 (0.0006) [2024-09-30 03:06:46,243][1157819] Updated weights for policy 0, policy_version 172908 (0.0006) [2024-09-30 03:06:46,718][1157819] Updated weights for policy 0, policy_version 172918 (0.0006) [2024-09-30 03:06:47,218][1157819] Updated weights for policy 0, policy_version 172928 (0.0006) [2024-09-30 03:06:47,807][1157819] Updated weights for policy 0, policy_version 172938 (0.0006) [2024-09-30 03:06:48,326][1157819] Updated weights for policy 0, policy_version 172948 (0.0006) [2024-09-30 03:06:48,861][1157819] Updated weights for policy 0, policy_version 172958 (0.0006) [2024-09-30 03:06:49,481][1157819] Updated weights for policy 0, policy_version 172968 (0.0006) [2024-09-30 03:06:50,019][1157819] Updated weights for policy 0, policy_version 172978 (0.0006) [2024-09-30 03:06:50,466][1157520] Fps is (10 sec: 79872.1, 60 sec: 80554.8, 300 sec: 81420.2). Total num frames: 708546560. Throughput: 0: 19997.9. Samples: 167125328. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 03:06:50,466][1157520] Avg episode reward: [(0, '54.273')] [2024-09-30 03:06:50,637][1157819] Updated weights for policy 0, policy_version 172988 (0.0007) [2024-09-30 03:06:51,164][1157819] Updated weights for policy 0, policy_version 172998 (0.0006) [2024-09-30 03:06:51,735][1157819] Updated weights for policy 0, policy_version 173008 (0.0006) [2024-09-30 03:06:52,271][1157819] Updated weights for policy 0, policy_version 173018 (0.0007) [2024-09-30 03:06:52,839][1157819] Updated weights for policy 0, policy_version 173028 (0.0007) [2024-09-30 03:06:53,407][1157819] Updated weights for policy 0, policy_version 173038 (0.0007) [2024-09-30 03:06:53,961][1157819] Updated weights for policy 0, policy_version 173048 (0.0006) [2024-09-30 03:06:54,547][1157819] Updated weights for policy 0, policy_version 173058 (0.0007) [2024-09-30 03:06:55,117][1157819] Updated weights for policy 0, policy_version 173068 (0.0006) [2024-09-30 03:06:55,466][1157520] Fps is (10 sec: 74956.8, 60 sec: 79599.1, 300 sec: 81323.0). Total num frames: 708911104. Throughput: 0: 19918.0. Samples: 167179528. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 03:06:55,466][1157520] Avg episode reward: [(0, '56.965')] [2024-09-30 03:06:55,670][1157819] Updated weights for policy 0, policy_version 173078 (0.0007) [2024-09-30 03:06:56,256][1157819] Updated weights for policy 0, policy_version 173088 (0.0006) [2024-09-30 03:06:56,834][1157819] Updated weights for policy 0, policy_version 173098 (0.0006) [2024-09-30 03:06:57,388][1157819] Updated weights for policy 0, policy_version 173108 (0.0006) [2024-09-30 03:06:57,972][1157819] Updated weights for policy 0, policy_version 173118 (0.0006) [2024-09-30 03:06:58,518][1157819] Updated weights for policy 0, policy_version 173128 (0.0006) [2024-09-30 03:06:59,083][1157819] Updated weights for policy 0, policy_version 173138 (0.0006) [2024-09-30 03:06:59,644][1157819] Updated weights for policy 0, policy_version 173148 (0.0006) [2024-09-30 03:07:00,210][1157819] Updated weights for policy 0, policy_version 173158 (0.0006) [2024-09-30 03:07:00,466][1157520] Fps is (10 sec: 72499.1, 60 sec: 78848.1, 300 sec: 81128.6). Total num frames: 709271552. Throughput: 0: 19747.1. Samples: 167288100. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 03:07:00,466][1157520] Avg episode reward: [(0, '55.375')] [2024-09-30 03:07:00,746][1157819] Updated weights for policy 0, policy_version 173168 (0.0006) [2024-09-30 03:07:01,297][1157819] Updated weights for policy 0, policy_version 173178 (0.0006) [2024-09-30 03:07:01,858][1157819] Updated weights for policy 0, policy_version 173188 (0.0006) [2024-09-30 03:07:02,418][1157819] Updated weights for policy 0, policy_version 173198 (0.0006) [2024-09-30 03:07:02,989][1157819] Updated weights for policy 0, policy_version 173208 (0.0006) [2024-09-30 03:07:03,544][1157819] Updated weights for policy 0, policy_version 173218 (0.0006) [2024-09-30 03:07:04,084][1157819] Updated weights for policy 0, policy_version 173228 (0.0006) [2024-09-30 03:07:04,672][1157819] Updated weights for policy 0, policy_version 173238 (0.0006) [2024-09-30 03:07:05,218][1157819] Updated weights for policy 0, policy_version 173248 (0.0006) [2024-09-30 03:07:05,466][1157520] Fps is (10 sec: 72499.3, 60 sec: 78029.2, 300 sec: 80962.0). Total num frames: 709636096. Throughput: 0: 19565.1. Samples: 167397736. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 03:07:05,466][1157520] Avg episode reward: [(0, '55.062')] [2024-09-30 03:07:05,796][1157819] Updated weights for policy 0, policy_version 173258 (0.0006) [2024-09-30 03:07:06,340][1157819] Updated weights for policy 0, policy_version 173268 (0.0006) [2024-09-30 03:07:06,911][1157819] Updated weights for policy 0, policy_version 173278 (0.0006) [2024-09-30 03:07:07,446][1157819] Updated weights for policy 0, policy_version 173288 (0.0006) [2024-09-30 03:07:08,015][1157819] Updated weights for policy 0, policy_version 173298 (0.0006) [2024-09-30 03:07:08,581][1157819] Updated weights for policy 0, policy_version 173308 (0.0006) [2024-09-30 03:07:09,152][1157819] Updated weights for policy 0, policy_version 173318 (0.0006) [2024-09-30 03:07:09,714][1157819] Updated weights for policy 0, policy_version 173328 (0.0007) [2024-09-30 03:07:10,281][1157819] Updated weights for policy 0, policy_version 173338 (0.0007) [2024-09-30 03:07:10,466][1157520] Fps is (10 sec: 73318.2, 60 sec: 77755.8, 300 sec: 80781.4). Total num frames: 710004736. Throughput: 0: 19463.1. Samples: 167452844. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 03:07:10,466][1157520] Avg episode reward: [(0, '55.479')] [2024-09-30 03:07:10,850][1157819] Updated weights for policy 0, policy_version 173348 (0.0006) [2024-09-30 03:07:11,416][1157819] Updated weights for policy 0, policy_version 173358 (0.0006) [2024-09-30 03:07:11,968][1157819] Updated weights for policy 0, policy_version 173368 (0.0006) [2024-09-30 03:07:12,536][1157819] Updated weights for policy 0, policy_version 173378 (0.0006) [2024-09-30 03:07:13,113][1157819] Updated weights for policy 0, policy_version 173388 (0.0006) [2024-09-30 03:07:13,686][1157819] Updated weights for policy 0, policy_version 173398 (0.0006) [2024-09-30 03:07:14,248][1157819] Updated weights for policy 0, policy_version 173408 (0.0006) [2024-09-30 03:07:14,813][1157819] Updated weights for policy 0, policy_version 173418 (0.0006) [2024-09-30 03:07:15,366][1157819] Updated weights for policy 0, policy_version 173428 (0.0006) [2024-09-30 03:07:15,466][1157520] Fps is (10 sec: 73318.2, 60 sec: 77414.6, 300 sec: 80642.6). Total num frames: 710369280. Throughput: 0: 19243.1. Samples: 167561408. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 03:07:15,466][1157520] Avg episode reward: [(0, '57.672')] [2024-09-30 03:07:15,926][1157819] Updated weights for policy 0, policy_version 173438 (0.0006) [2024-09-30 03:07:16,452][1157819] Updated weights for policy 0, policy_version 173448 (0.0006) [2024-09-30 03:07:17,034][1157819] Updated weights for policy 0, policy_version 173458 (0.0006) [2024-09-30 03:07:17,591][1157819] Updated weights for policy 0, policy_version 173468 (0.0006) [2024-09-30 03:07:18,099][1157819] Updated weights for policy 0, policy_version 173478 (0.0006) [2024-09-30 03:07:18,673][1157819] Updated weights for policy 0, policy_version 173488 (0.0006) [2024-09-30 03:07:19,231][1157819] Updated weights for policy 0, policy_version 173498 (0.0006) [2024-09-30 03:07:19,781][1157819] Updated weights for policy 0, policy_version 173508 (0.0006) [2024-09-30 03:07:20,367][1157819] Updated weights for policy 0, policy_version 173518 (0.0006) [2024-09-30 03:07:20,466][1157520] Fps is (10 sec: 73318.9, 60 sec: 76936.8, 300 sec: 80517.6). Total num frames: 710737920. Throughput: 0: 18968.2. Samples: 167672136. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 03:07:20,466][1157520] Avg episode reward: [(0, '56.840')] [2024-09-30 03:07:20,923][1157819] Updated weights for policy 0, policy_version 173528 (0.0006) [2024-09-30 03:07:21,434][1157819] Updated weights for policy 0, policy_version 173538 (0.0006) [2024-09-30 03:07:21,984][1157819] Updated weights for policy 0, policy_version 173548 (0.0006) [2024-09-30 03:07:22,509][1157819] Updated weights for policy 0, policy_version 173558 (0.0006) [2024-09-30 03:07:23,077][1157819] Updated weights for policy 0, policy_version 173568 (0.0006) [2024-09-30 03:07:23,636][1157819] Updated weights for policy 0, policy_version 173578 (0.0006) [2024-09-30 03:07:24,131][1157819] Updated weights for policy 0, policy_version 173588 (0.0006) [2024-09-30 03:07:24,639][1157819] Updated weights for policy 0, policy_version 173598 (0.0006) [2024-09-30 03:07:25,178][1157819] Updated weights for policy 0, policy_version 173608 (0.0006) [2024-09-30 03:07:25,466][1157520] Fps is (10 sec: 74956.7, 60 sec: 76731.9, 300 sec: 80392.7). Total num frames: 711118848. Throughput: 0: 18829.8. Samples: 167728284. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 03:07:25,466][1157520] Avg episode reward: [(0, '57.013')] [2024-09-30 03:07:25,710][1157819] Updated weights for policy 0, policy_version 173618 (0.0006) [2024-09-30 03:07:26,221][1157819] Updated weights for policy 0, policy_version 173628 (0.0006) [2024-09-30 03:07:26,726][1157819] Updated weights for policy 0, policy_version 173638 (0.0006) [2024-09-30 03:07:27,257][1157819] Updated weights for policy 0, policy_version 173648 (0.0006) [2024-09-30 03:07:27,800][1157819] Updated weights for policy 0, policy_version 173658 (0.0006) [2024-09-30 03:07:28,326][1157819] Updated weights for policy 0, policy_version 173668 (0.0006) [2024-09-30 03:07:28,791][1157819] Updated weights for policy 0, policy_version 173678 (0.0006) [2024-09-30 03:07:29,315][1157819] Updated weights for policy 0, policy_version 173688 (0.0006) [2024-09-30 03:07:29,816][1157819] Updated weights for policy 0, policy_version 173698 (0.0006) [2024-09-30 03:07:30,263][1157819] Updated weights for policy 0, policy_version 173708 (0.0006) [2024-09-30 03:07:30,466][1157520] Fps is (10 sec: 78233.4, 60 sec: 76731.8, 300 sec: 80420.5). Total num frames: 711520256. Throughput: 0: 18649.6. Samples: 167845808. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 03:07:30,466][1157520] Avg episode reward: [(0, '54.261')] [2024-09-30 03:07:30,771][1157819] Updated weights for policy 0, policy_version 173718 (0.0006) [2024-09-30 03:07:31,255][1157819] Updated weights for policy 0, policy_version 173728 (0.0006) [2024-09-30 03:07:31,753][1157819] Updated weights for policy 0, policy_version 173738 (0.0006) [2024-09-30 03:07:32,257][1157819] Updated weights for policy 0, policy_version 173748 (0.0006) [2024-09-30 03:07:32,755][1157819] Updated weights for policy 0, policy_version 173758 (0.0006) [2024-09-30 03:07:33,227][1157819] Updated weights for policy 0, policy_version 173768 (0.0006) [2024-09-30 03:07:33,749][1157819] Updated weights for policy 0, policy_version 173778 (0.0006) [2024-09-30 03:07:34,245][1157819] Updated weights for policy 0, policy_version 173788 (0.0006) [2024-09-30 03:07:34,724][1157819] Updated weights for policy 0, policy_version 173798 (0.0006) [2024-09-30 03:07:35,212][1157819] Updated weights for policy 0, policy_version 173808 (0.0006) [2024-09-30 03:07:35,466][1157520] Fps is (10 sec: 81510.3, 60 sec: 76731.7, 300 sec: 80476.0). Total num frames: 711933952. Throughput: 0: 18776.3. Samples: 167970260. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 03:07:35,466][1157520] Avg episode reward: [(0, '57.458')] [2024-09-30 03:07:35,707][1157819] Updated weights for policy 0, policy_version 173818 (0.0006) [2024-09-30 03:07:36,190][1157819] Updated weights for policy 0, policy_version 173828 (0.0006) [2024-09-30 03:07:36,700][1157819] Updated weights for policy 0, policy_version 173838 (0.0006) [2024-09-30 03:07:37,176][1157819] Updated weights for policy 0, policy_version 173848 (0.0006) [2024-09-30 03:07:37,677][1157819] Updated weights for policy 0, policy_version 173858 (0.0006) [2024-09-30 03:07:38,174][1157819] Updated weights for policy 0, policy_version 173868 (0.0006) [2024-09-30 03:07:38,708][1157819] Updated weights for policy 0, policy_version 173878 (0.0006) [2024-09-30 03:07:39,210][1157819] Updated weights for policy 0, policy_version 173888 (0.0006) [2024-09-30 03:07:39,741][1157819] Updated weights for policy 0, policy_version 173898 (0.0006) [2024-09-30 03:07:40,285][1157819] Updated weights for policy 0, policy_version 173908 (0.0006) [2024-09-30 03:07:40,466][1157520] Fps is (10 sec: 81919.3, 60 sec: 76526.9, 300 sec: 80503.8). Total num frames: 712339456. Throughput: 0: 18953.5. Samples: 168032436. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 03:07:40,466][1157520] Avg episode reward: [(0, '54.380')] [2024-09-30 03:07:40,834][1157819] Updated weights for policy 0, policy_version 173918 (0.0006) [2024-09-30 03:07:41,416][1157819] Updated weights for policy 0, policy_version 173928 (0.0006) [2024-09-30 03:07:41,980][1157819] Updated weights for policy 0, policy_version 173938 (0.0006) [2024-09-30 03:07:42,501][1157819] Updated weights for policy 0, policy_version 173948 (0.0007) [2024-09-30 03:07:43,055][1157819] Updated weights for policy 0, policy_version 173958 (0.0006) [2024-09-30 03:07:43,582][1157819] Updated weights for policy 0, policy_version 173968 (0.0006) [2024-09-30 03:07:44,092][1157819] Updated weights for policy 0, policy_version 173978 (0.0006) [2024-09-30 03:07:44,271][1157736] Signal inference workers to stop experience collection... (11800 times) [2024-09-30 03:07:44,272][1157736] Signal inference workers to resume experience collection... (11800 times) [2024-09-30 03:07:44,275][1157819] InferenceWorker_p0-w0: stopping experience collection (11800 times) [2024-09-30 03:07:44,275][1157819] InferenceWorker_p0-w0: resuming experience collection (11800 times) [2024-09-30 03:07:44,600][1157819] Updated weights for policy 0, policy_version 173988 (0.0007) [2024-09-30 03:07:45,127][1157819] Updated weights for policy 0, policy_version 173998 (0.0006) [2024-09-30 03:07:45,466][1157520] Fps is (10 sec: 78643.4, 60 sec: 75980.8, 300 sec: 80503.8). Total num frames: 712720384. Throughput: 0: 19096.4. Samples: 168147440. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 03:07:45,466][1157520] Avg episode reward: [(0, '54.105')] [2024-09-30 03:07:45,636][1157819] Updated weights for policy 0, policy_version 174008 (0.0006) [2024-09-30 03:07:46,156][1157819] Updated weights for policy 0, policy_version 174018 (0.0006) [2024-09-30 03:07:46,707][1157819] Updated weights for policy 0, policy_version 174028 (0.0006) [2024-09-30 03:07:47,223][1157819] Updated weights for policy 0, policy_version 174038 (0.0006) [2024-09-30 03:07:47,738][1157819] Updated weights for policy 0, policy_version 174048 (0.0006) [2024-09-30 03:07:48,221][1157819] Updated weights for policy 0, policy_version 174058 (0.0006) [2024-09-30 03:07:48,759][1157819] Updated weights for policy 0, policy_version 174068 (0.0006) [2024-09-30 03:07:49,316][1157819] Updated weights for policy 0, policy_version 174078 (0.0006) [2024-09-30 03:07:49,821][1157819] Updated weights for policy 0, policy_version 174088 (0.0006) [2024-09-30 03:07:50,386][1157819] Updated weights for policy 0, policy_version 174098 (0.0006) [2024-09-30 03:07:50,466][1157520] Fps is (10 sec: 77004.6, 60 sec: 76048.9, 300 sec: 80476.0). Total num frames: 713109504. Throughput: 0: 19285.2. Samples: 168265572. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 03:07:50,466][1157520] Avg episode reward: [(0, '54.226')] [2024-09-30 03:07:50,932][1157819] Updated weights for policy 0, policy_version 174108 (0.0006) [2024-09-30 03:07:51,478][1157819] Updated weights for policy 0, policy_version 174118 (0.0006) [2024-09-30 03:07:52,037][1157819] Updated weights for policy 0, policy_version 174128 (0.0006) [2024-09-30 03:07:52,586][1157819] Updated weights for policy 0, policy_version 174138 (0.0006) [2024-09-30 03:07:53,172][1157819] Updated weights for policy 0, policy_version 174148 (0.0006) [2024-09-30 03:07:53,676][1157819] Updated weights for policy 0, policy_version 174158 (0.0006) [2024-09-30 03:07:54,232][1157819] Updated weights for policy 0, policy_version 174168 (0.0006) [2024-09-30 03:07:54,769][1157819] Updated weights for policy 0, policy_version 174178 (0.0006) [2024-09-30 03:07:55,293][1157819] Updated weights for policy 0, policy_version 174188 (0.0006) [2024-09-30 03:07:55,466][1157520] Fps is (10 sec: 76184.6, 60 sec: 76185.4, 300 sec: 80406.5). Total num frames: 713482240. Throughput: 0: 19287.9. Samples: 168320800. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 03:07:55,466][1157520] Avg episode reward: [(0, '54.681')] [2024-09-30 03:07:55,891][1157819] Updated weights for policy 0, policy_version 174198 (0.0006) [2024-09-30 03:07:56,464][1157819] Updated weights for policy 0, policy_version 174208 (0.0006) [2024-09-30 03:07:57,044][1157819] Updated weights for policy 0, policy_version 174218 (0.0006) [2024-09-30 03:07:57,644][1157819] Updated weights for policy 0, policy_version 174228 (0.0006) [2024-09-30 03:07:58,225][1157819] Updated weights for policy 0, policy_version 174238 (0.0006) [2024-09-30 03:07:58,832][1157819] Updated weights for policy 0, policy_version 174248 (0.0006) [2024-09-30 03:07:59,436][1157819] Updated weights for policy 0, policy_version 174258 (0.0006) [2024-09-30 03:07:59,995][1157819] Updated weights for policy 0, policy_version 174268 (0.0006) [2024-09-30 03:08:00,466][1157520] Fps is (10 sec: 72499.6, 60 sec: 76049.0, 300 sec: 80226.0). Total num frames: 713834496. Throughput: 0: 19283.7. Samples: 168429176. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 03:08:00,466][1157520] Avg episode reward: [(0, '56.640')] [2024-09-30 03:08:00,607][1157819] Updated weights for policy 0, policy_version 174278 (0.0006) [2024-09-30 03:08:01,196][1157819] Updated weights for policy 0, policy_version 174288 (0.0006) [2024-09-30 03:08:01,794][1157819] Updated weights for policy 0, policy_version 174298 (0.0006) [2024-09-30 03:08:02,373][1157819] Updated weights for policy 0, policy_version 174308 (0.0006) [2024-09-30 03:08:02,955][1157819] Updated weights for policy 0, policy_version 174318 (0.0006) [2024-09-30 03:08:03,537][1157819] Updated weights for policy 0, policy_version 174328 (0.0006) [2024-09-30 03:08:04,127][1157819] Updated weights for policy 0, policy_version 174338 (0.0006) [2024-09-30 03:08:04,712][1157819] Updated weights for policy 0, policy_version 174348 (0.0006) [2024-09-30 03:08:05,296][1157819] Updated weights for policy 0, policy_version 174358 (0.0006) [2024-09-30 03:08:05,466][1157520] Fps is (10 sec: 69632.1, 60 sec: 75707.6, 300 sec: 80003.9). Total num frames: 714178560. Throughput: 0: 19134.7. Samples: 168533200. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 03:08:05,466][1157520] Avg episode reward: [(0, '53.688')] [2024-09-30 03:08:05,472][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000174360_714178560.pth... [2024-09-30 03:08:05,513][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000169670_694968320.pth [2024-09-30 03:08:05,883][1157819] Updated weights for policy 0, policy_version 174368 (0.0006) [2024-09-30 03:08:06,495][1157819] Updated weights for policy 0, policy_version 174378 (0.0006) [2024-09-30 03:08:07,048][1157819] Updated weights for policy 0, policy_version 174388 (0.0006) [2024-09-30 03:08:07,672][1157819] Updated weights for policy 0, policy_version 174398 (0.0006) [2024-09-30 03:08:08,273][1157819] Updated weights for policy 0, policy_version 174408 (0.0006) [2024-09-30 03:08:08,911][1157819] Updated weights for policy 0, policy_version 174418 (0.0006) [2024-09-30 03:08:09,496][1157819] Updated weights for policy 0, policy_version 174428 (0.0006) [2024-09-30 03:08:10,079][1157819] Updated weights for policy 0, policy_version 174438 (0.0006) [2024-09-30 03:08:10,466][1157520] Fps is (10 sec: 68812.7, 60 sec: 75298.1, 300 sec: 79837.3). Total num frames: 714522624. Throughput: 0: 19042.1. Samples: 168585180. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 03:08:10,466][1157520] Avg episode reward: [(0, '55.590')] [2024-09-30 03:08:10,646][1157819] Updated weights for policy 0, policy_version 174448 (0.0006) [2024-09-30 03:08:11,251][1157819] Updated weights for policy 0, policy_version 174458 (0.0006) [2024-09-30 03:08:11,831][1157819] Updated weights for policy 0, policy_version 174468 (0.0006) [2024-09-30 03:08:12,407][1157819] Updated weights for policy 0, policy_version 174478 (0.0006) [2024-09-30 03:08:13,011][1157819] Updated weights for policy 0, policy_version 174488 (0.0006) [2024-09-30 03:08:13,591][1157819] Updated weights for policy 0, policy_version 174498 (0.0006) [2024-09-30 03:08:14,177][1157819] Updated weights for policy 0, policy_version 174508 (0.0006) [2024-09-30 03:08:14,785][1157819] Updated weights for policy 0, policy_version 174518 (0.0006) [2024-09-30 03:08:15,391][1157819] Updated weights for policy 0, policy_version 174528 (0.0006) [2024-09-30 03:08:15,466][1157520] Fps is (10 sec: 68812.7, 60 sec: 74956.6, 300 sec: 79740.1). Total num frames: 714866688. Throughput: 0: 18735.1. Samples: 168688888. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:08:15,466][1157520] Avg episode reward: [(0, '55.820')] [2024-09-30 03:08:15,970][1157819] Updated weights for policy 0, policy_version 174538 (0.0006) [2024-09-30 03:08:16,582][1157819] Updated weights for policy 0, policy_version 174548 (0.0006) [2024-09-30 03:08:17,170][1157819] Updated weights for policy 0, policy_version 174558 (0.0006) [2024-09-30 03:08:17,755][1157819] Updated weights for policy 0, policy_version 174568 (0.0006) [2024-09-30 03:08:18,347][1157819] Updated weights for policy 0, policy_version 174578 (0.0006) [2024-09-30 03:08:18,929][1157819] Updated weights for policy 0, policy_version 174588 (0.0006) [2024-09-30 03:08:19,492][1157819] Updated weights for policy 0, policy_version 174598 (0.0006) [2024-09-30 03:08:20,069][1157819] Updated weights for policy 0, policy_version 174608 (0.0006) [2024-09-30 03:08:20,466][1157520] Fps is (10 sec: 70042.0, 60 sec: 74752.0, 300 sec: 79643.0). Total num frames: 715223040. Throughput: 0: 18292.7. Samples: 168793432. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:08:20,466][1157520] Avg episode reward: [(0, '55.948')] [2024-09-30 03:08:20,599][1157819] Updated weights for policy 0, policy_version 174618 (0.0006) [2024-09-30 03:08:21,188][1157819] Updated weights for policy 0, policy_version 174628 (0.0006) [2024-09-30 03:08:21,716][1157819] Updated weights for policy 0, policy_version 174638 (0.0006) [2024-09-30 03:08:22,287][1157819] Updated weights for policy 0, policy_version 174648 (0.0006) [2024-09-30 03:08:22,837][1157819] Updated weights for policy 0, policy_version 174658 (0.0006) [2024-09-30 03:08:23,369][1157819] Updated weights for policy 0, policy_version 174668 (0.0006) [2024-09-30 03:08:23,935][1157819] Updated weights for policy 0, policy_version 174678 (0.0006) [2024-09-30 03:08:24,488][1157819] Updated weights for policy 0, policy_version 174688 (0.0006) [2024-09-30 03:08:24,999][1157819] Updated weights for policy 0, policy_version 174698 (0.0006) [2024-09-30 03:08:25,466][1157520] Fps is (10 sec: 73319.4, 60 sec: 74683.8, 300 sec: 79573.5). Total num frames: 715599872. Throughput: 0: 18136.1. Samples: 168848560. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:08:25,466][1157520] Avg episode reward: [(0, '56.448')] [2024-09-30 03:08:25,518][1157819] Updated weights for policy 0, policy_version 174708 (0.0006) [2024-09-30 03:08:26,021][1157819] Updated weights for policy 0, policy_version 174718 (0.0006) [2024-09-30 03:08:26,501][1157819] Updated weights for policy 0, policy_version 174728 (0.0006) [2024-09-30 03:08:27,022][1157819] Updated weights for policy 0, policy_version 174738 (0.0006) [2024-09-30 03:08:27,521][1157819] Updated weights for policy 0, policy_version 174748 (0.0006) [2024-09-30 03:08:28,003][1157819] Updated weights for policy 0, policy_version 174758 (0.0006) [2024-09-30 03:08:28,500][1157819] Updated weights for policy 0, policy_version 174768 (0.0006) [2024-09-30 03:08:29,017][1157819] Updated weights for policy 0, policy_version 174778 (0.0006) [2024-09-30 03:08:29,500][1157819] Updated weights for policy 0, policy_version 174788 (0.0006) [2024-09-30 03:08:29,992][1157819] Updated weights for policy 0, policy_version 174798 (0.0006) [2024-09-30 03:08:30,466][1157520] Fps is (10 sec: 78643.3, 60 sec: 74820.3, 300 sec: 79642.9). Total num frames: 716009472. Throughput: 0: 18224.2. Samples: 168967528. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:08:30,466][1157520] Avg episode reward: [(0, '55.126')] [2024-09-30 03:08:30,475][1157819] Updated weights for policy 0, policy_version 174808 (0.0006) [2024-09-30 03:08:30,966][1157819] Updated weights for policy 0, policy_version 174818 (0.0006) [2024-09-30 03:08:31,442][1157819] Updated weights for policy 0, policy_version 174828 (0.0006) [2024-09-30 03:08:31,936][1157819] Updated weights for policy 0, policy_version 174838 (0.0006) [2024-09-30 03:08:32,444][1157819] Updated weights for policy 0, policy_version 174848 (0.0006) [2024-09-30 03:08:32,901][1157819] Updated weights for policy 0, policy_version 174858 (0.0006) [2024-09-30 03:08:33,386][1157819] Updated weights for policy 0, policy_version 174868 (0.0006) [2024-09-30 03:08:33,882][1157819] Updated weights for policy 0, policy_version 174878 (0.0006) [2024-09-30 03:08:34,378][1157819] Updated weights for policy 0, policy_version 174888 (0.0006) [2024-09-30 03:08:34,863][1157819] Updated weights for policy 0, policy_version 174898 (0.0006) [2024-09-30 03:08:35,347][1157819] Updated weights for policy 0, policy_version 174908 (0.0006) [2024-09-30 03:08:35,466][1157520] Fps is (10 sec: 83558.2, 60 sec: 75025.1, 300 sec: 79754.0). Total num frames: 716435456. Throughput: 0: 18415.6. Samples: 169094272. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:08:35,466][1157520] Avg episode reward: [(0, '56.106')] [2024-09-30 03:08:35,797][1157819] Updated weights for policy 0, policy_version 174918 (0.0006) [2024-09-30 03:08:36,300][1157819] Updated weights for policy 0, policy_version 174928 (0.0006) [2024-09-30 03:08:36,800][1157819] Updated weights for policy 0, policy_version 174938 (0.0006) [2024-09-30 03:08:37,287][1157819] Updated weights for policy 0, policy_version 174948 (0.0006) [2024-09-30 03:08:37,758][1157819] Updated weights for policy 0, policy_version 174958 (0.0006) [2024-09-30 03:08:38,267][1157819] Updated weights for policy 0, policy_version 174968 (0.0006) [2024-09-30 03:08:38,723][1157819] Updated weights for policy 0, policy_version 174978 (0.0006) [2024-09-30 03:08:39,194][1157736] Signal inference workers to stop experience collection... (11850 times) [2024-09-30 03:08:39,195][1157736] Signal inference workers to resume experience collection... (11850 times) [2024-09-30 03:08:39,199][1157819] InferenceWorker_p0-w0: stopping experience collection (11850 times) [2024-09-30 03:08:39,201][1157819] InferenceWorker_p0-w0: resuming experience collection (11850 times) [2024-09-30 03:08:39,211][1157819] Updated weights for policy 0, policy_version 174988 (0.0006) [2024-09-30 03:08:39,698][1157819] Updated weights for policy 0, policy_version 174998 (0.0006) [2024-09-30 03:08:40,210][1157819] Updated weights for policy 0, policy_version 175008 (0.0006) [2024-09-30 03:08:40,466][1157520] Fps is (10 sec: 84377.6, 60 sec: 75230.0, 300 sec: 79781.8). Total num frames: 716853248. Throughput: 0: 18595.6. Samples: 169157600. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:08:40,466][1157520] Avg episode reward: [(0, '56.123')] [2024-09-30 03:08:40,664][1157819] Updated weights for policy 0, policy_version 175018 (0.0006) [2024-09-30 03:08:41,171][1157819] Updated weights for policy 0, policy_version 175028 (0.0006) [2024-09-30 03:08:41,673][1157819] Updated weights for policy 0, policy_version 175038 (0.0006) [2024-09-30 03:08:42,148][1157819] Updated weights for policy 0, policy_version 175048 (0.0006) [2024-09-30 03:08:42,626][1157819] Updated weights for policy 0, policy_version 175058 (0.0006) [2024-09-30 03:08:43,115][1157819] Updated weights for policy 0, policy_version 175068 (0.0006) [2024-09-30 03:08:43,593][1157819] Updated weights for policy 0, policy_version 175078 (0.0006) [2024-09-30 03:08:44,094][1157819] Updated weights for policy 0, policy_version 175088 (0.0006) [2024-09-30 03:08:44,602][1157819] Updated weights for policy 0, policy_version 175098 (0.0006) [2024-09-30 03:08:45,155][1157819] Updated weights for policy 0, policy_version 175108 (0.0006) [2024-09-30 03:08:45,466][1157520] Fps is (10 sec: 82738.5, 60 sec: 75707.6, 300 sec: 79740.1). Total num frames: 717262848. Throughput: 0: 18987.6. Samples: 169283620. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:08:45,466][1157520] Avg episode reward: [(0, '54.789')] [2024-09-30 03:08:45,635][1157819] Updated weights for policy 0, policy_version 175118 (0.0006) [2024-09-30 03:08:46,132][1157819] Updated weights for policy 0, policy_version 175128 (0.0006) [2024-09-30 03:08:46,652][1157819] Updated weights for policy 0, policy_version 175138 (0.0006) [2024-09-30 03:08:47,203][1157819] Updated weights for policy 0, policy_version 175148 (0.0006) [2024-09-30 03:08:47,718][1157819] Updated weights for policy 0, policy_version 175158 (0.0006) [2024-09-30 03:08:48,221][1157819] Updated weights for policy 0, policy_version 175168 (0.0006) [2024-09-30 03:08:48,717][1157819] Updated weights for policy 0, policy_version 175178 (0.0006) [2024-09-30 03:08:49,259][1157819] Updated weights for policy 0, policy_version 175188 (0.0006) [2024-09-30 03:08:49,770][1157819] Updated weights for policy 0, policy_version 175198 (0.0006) [2024-09-30 03:08:50,292][1157819] Updated weights for policy 0, policy_version 175208 (0.0006) [2024-09-30 03:08:50,466][1157520] Fps is (10 sec: 81099.7, 60 sec: 75912.5, 300 sec: 79698.4). Total num frames: 717664256. Throughput: 0: 19326.1. Samples: 169402876. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:08:50,466][1157520] Avg episode reward: [(0, '55.620')] [2024-09-30 03:08:50,815][1157819] Updated weights for policy 0, policy_version 175218 (0.0006) [2024-09-30 03:08:51,347][1157819] Updated weights for policy 0, policy_version 175228 (0.0006) [2024-09-30 03:08:51,858][1157819] Updated weights for policy 0, policy_version 175238 (0.0006) [2024-09-30 03:08:52,372][1157819] Updated weights for policy 0, policy_version 175248 (0.0006) [2024-09-30 03:08:52,876][1157819] Updated weights for policy 0, policy_version 175258 (0.0006) [2024-09-30 03:08:53,410][1157819] Updated weights for policy 0, policy_version 175268 (0.0006) [2024-09-30 03:08:53,911][1157819] Updated weights for policy 0, policy_version 175278 (0.0006) [2024-09-30 03:08:54,438][1157819] Updated weights for policy 0, policy_version 175288 (0.0006) [2024-09-30 03:08:54,954][1157819] Updated weights for policy 0, policy_version 175298 (0.0006) [2024-09-30 03:08:55,459][1157819] Updated weights for policy 0, policy_version 175308 (0.0006) [2024-09-30 03:08:55,466][1157520] Fps is (10 sec: 79871.9, 60 sec: 76322.2, 300 sec: 79684.5). Total num frames: 718061568. Throughput: 0: 19491.4. Samples: 169462292. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:08:55,466][1157520] Avg episode reward: [(0, '55.130')] [2024-09-30 03:08:55,963][1157819] Updated weights for policy 0, policy_version 175318 (0.0006) [2024-09-30 03:08:56,485][1157819] Updated weights for policy 0, policy_version 175328 (0.0006) [2024-09-30 03:08:56,982][1157819] Updated weights for policy 0, policy_version 175338 (0.0006) [2024-09-30 03:08:57,494][1157819] Updated weights for policy 0, policy_version 175348 (0.0006) [2024-09-30 03:08:57,999][1157819] Updated weights for policy 0, policy_version 175358 (0.0006) [2024-09-30 03:08:58,511][1157819] Updated weights for policy 0, policy_version 175368 (0.0006) [2024-09-30 03:08:59,027][1157819] Updated weights for policy 0, policy_version 175378 (0.0006) [2024-09-30 03:08:59,529][1157819] Updated weights for policy 0, policy_version 175388 (0.0006) [2024-09-30 03:09:00,029][1157819] Updated weights for policy 0, policy_version 175398 (0.0006) [2024-09-30 03:09:00,466][1157520] Fps is (10 sec: 79872.5, 60 sec: 77141.3, 300 sec: 79656.8). Total num frames: 718462976. Throughput: 0: 19855.0. Samples: 169582364. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:09:00,466][1157520] Avg episode reward: [(0, '53.590')] [2024-09-30 03:09:00,556][1157819] Updated weights for policy 0, policy_version 175408 (0.0006) [2024-09-30 03:09:01,060][1157819] Updated weights for policy 0, policy_version 175418 (0.0006) [2024-09-30 03:09:01,559][1157819] Updated weights for policy 0, policy_version 175428 (0.0006) [2024-09-30 03:09:02,069][1157819] Updated weights for policy 0, policy_version 175438 (0.0006) [2024-09-30 03:09:02,590][1157819] Updated weights for policy 0, policy_version 175448 (0.0006) [2024-09-30 03:09:03,113][1157819] Updated weights for policy 0, policy_version 175458 (0.0006) [2024-09-30 03:09:03,611][1157819] Updated weights for policy 0, policy_version 175468 (0.0006) [2024-09-30 03:09:04,112][1157819] Updated weights for policy 0, policy_version 175478 (0.0006) [2024-09-30 03:09:04,615][1157819] Updated weights for policy 0, policy_version 175488 (0.0006) [2024-09-30 03:09:05,105][1157819] Updated weights for policy 0, policy_version 175498 (0.0006) [2024-09-30 03:09:05,466][1157520] Fps is (10 sec: 80281.2, 60 sec: 78097.0, 300 sec: 79573.5). Total num frames: 718864384. Throughput: 0: 20207.6. Samples: 169702776. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:09:05,466][1157520] Avg episode reward: [(0, '56.011')] [2024-09-30 03:09:05,645][1157819] Updated weights for policy 0, policy_version 175508 (0.0006) [2024-09-30 03:09:06,169][1157819] Updated weights for policy 0, policy_version 175518 (0.0006) [2024-09-30 03:09:06,705][1157819] Updated weights for policy 0, policy_version 175528 (0.0006) [2024-09-30 03:09:07,292][1157819] Updated weights for policy 0, policy_version 175538 (0.0006) [2024-09-30 03:09:07,794][1157819] Updated weights for policy 0, policy_version 175548 (0.0006) [2024-09-30 03:09:08,311][1157819] Updated weights for policy 0, policy_version 175558 (0.0006) [2024-09-30 03:09:08,898][1157819] Updated weights for policy 0, policy_version 175568 (0.0006) [2024-09-30 03:09:09,417][1157819] Updated weights for policy 0, policy_version 175578 (0.0006) [2024-09-30 03:09:09,911][1157819] Updated weights for policy 0, policy_version 175588 (0.0006) [2024-09-30 03:09:10,431][1157819] Updated weights for policy 0, policy_version 175598 (0.0006) [2024-09-30 03:09:10,466][1157520] Fps is (10 sec: 78642.7, 60 sec: 78779.7, 300 sec: 79490.2). Total num frames: 719249408. Throughput: 0: 20272.6. Samples: 169760832. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:09:10,466][1157520] Avg episode reward: [(0, '57.100')] [2024-09-30 03:09:10,934][1157819] Updated weights for policy 0, policy_version 175608 (0.0006) [2024-09-30 03:09:11,429][1157819] Updated weights for policy 0, policy_version 175618 (0.0006) [2024-09-30 03:09:11,906][1157819] Updated weights for policy 0, policy_version 175628 (0.0006) [2024-09-30 03:09:12,402][1157819] Updated weights for policy 0, policy_version 175638 (0.0006) [2024-09-30 03:09:12,904][1157819] Updated weights for policy 0, policy_version 175648 (0.0006) [2024-09-30 03:09:13,403][1157819] Updated weights for policy 0, policy_version 175658 (0.0006) [2024-09-30 03:09:13,926][1157819] Updated weights for policy 0, policy_version 175668 (0.0006) [2024-09-30 03:09:14,431][1157819] Updated weights for policy 0, policy_version 175678 (0.0006) [2024-09-30 03:09:14,958][1157819] Updated weights for policy 0, policy_version 175688 (0.0006) [2024-09-30 03:09:15,449][1157819] Updated weights for policy 0, policy_version 175698 (0.0006) [2024-09-30 03:09:15,466][1157520] Fps is (10 sec: 79462.5, 60 sec: 79872.0, 300 sec: 79490.2). Total num frames: 719659008. Throughput: 0: 20302.2. Samples: 169881132. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:09:15,466][1157520] Avg episode reward: [(0, '57.321')] [2024-09-30 03:09:15,965][1157819] Updated weights for policy 0, policy_version 175708 (0.0006) [2024-09-30 03:09:16,472][1157819] Updated weights for policy 0, policy_version 175718 (0.0006) [2024-09-30 03:09:16,997][1157819] Updated weights for policy 0, policy_version 175728 (0.0006) [2024-09-30 03:09:17,527][1157819] Updated weights for policy 0, policy_version 175738 (0.0006) [2024-09-30 03:09:18,064][1157819] Updated weights for policy 0, policy_version 175748 (0.0006) [2024-09-30 03:09:18,614][1157819] Updated weights for policy 0, policy_version 175758 (0.0006) [2024-09-30 03:09:19,128][1157819] Updated weights for policy 0, policy_version 175768 (0.0006) [2024-09-30 03:09:19,629][1157819] Updated weights for policy 0, policy_version 175778 (0.0006) [2024-09-30 03:09:20,130][1157819] Updated weights for policy 0, policy_version 175788 (0.0006) [2024-09-30 03:09:20,466][1157520] Fps is (10 sec: 80281.3, 60 sec: 80486.2, 300 sec: 79434.6). Total num frames: 720052224. Throughput: 0: 20127.8. Samples: 170000028. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 03:09:20,466][1157520] Avg episode reward: [(0, '54.872')] [2024-09-30 03:09:20,642][1157819] Updated weights for policy 0, policy_version 175798 (0.0006) [2024-09-30 03:09:21,140][1157819] Updated weights for policy 0, policy_version 175808 (0.0006) [2024-09-30 03:09:21,651][1157819] Updated weights for policy 0, policy_version 175818 (0.0006) [2024-09-30 03:09:22,165][1157819] Updated weights for policy 0, policy_version 175828 (0.0006) [2024-09-30 03:09:22,655][1157819] Updated weights for policy 0, policy_version 175838 (0.0006) [2024-09-30 03:09:23,151][1157819] Updated weights for policy 0, policy_version 175848 (0.0006) [2024-09-30 03:09:23,649][1157819] Updated weights for policy 0, policy_version 175858 (0.0006) [2024-09-30 03:09:24,121][1157736] Signal inference workers to stop experience collection... (11900 times) [2024-09-30 03:09:24,125][1157736] Signal inference workers to resume experience collection... (11900 times) [2024-09-30 03:09:24,126][1157819] InferenceWorker_p0-w0: stopping experience collection (11900 times) [2024-09-30 03:09:24,129][1157819] InferenceWorker_p0-w0: resuming experience collection (11900 times) [2024-09-30 03:09:24,139][1157819] Updated weights for policy 0, policy_version 175868 (0.0006) [2024-09-30 03:09:24,639][1157819] Updated weights for policy 0, policy_version 175878 (0.0006) [2024-09-30 03:09:25,174][1157819] Updated weights for policy 0, policy_version 175888 (0.0006) [2024-09-30 03:09:25,466][1157520] Fps is (10 sec: 80281.6, 60 sec: 81032.3, 300 sec: 79476.2). Total num frames: 720461824. Throughput: 0: 20070.3. Samples: 170060768. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 03:09:25,466][1157520] Avg episode reward: [(0, '54.489')] [2024-09-30 03:09:25,701][1157819] Updated weights for policy 0, policy_version 175898 (0.0006) [2024-09-30 03:09:26,199][1157819] Updated weights for policy 0, policy_version 175908 (0.0006) [2024-09-30 03:09:26,692][1157819] Updated weights for policy 0, policy_version 175918 (0.0006) [2024-09-30 03:09:27,187][1157819] Updated weights for policy 0, policy_version 175928 (0.0006) [2024-09-30 03:09:27,696][1157819] Updated weights for policy 0, policy_version 175938 (0.0006) [2024-09-30 03:09:28,203][1157819] Updated weights for policy 0, policy_version 175948 (0.0006) [2024-09-30 03:09:28,719][1157819] Updated weights for policy 0, policy_version 175958 (0.0006) [2024-09-30 03:09:29,263][1157819] Updated weights for policy 0, policy_version 175968 (0.0006) [2024-09-30 03:09:29,750][1157819] Updated weights for policy 0, policy_version 175978 (0.0006) [2024-09-30 03:09:30,253][1157819] Updated weights for policy 0, policy_version 175988 (0.0006) [2024-09-30 03:09:30,466][1157520] Fps is (10 sec: 80691.6, 60 sec: 80827.6, 300 sec: 79406.8). Total num frames: 720859136. Throughput: 0: 19967.2. Samples: 170182144. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 03:09:30,466][1157520] Avg episode reward: [(0, '56.606')] [2024-09-30 03:09:30,788][1157819] Updated weights for policy 0, policy_version 175998 (0.0006) [2024-09-30 03:09:31,318][1157819] Updated weights for policy 0, policy_version 176008 (0.0006) [2024-09-30 03:09:31,844][1157819] Updated weights for policy 0, policy_version 176018 (0.0006) [2024-09-30 03:09:32,334][1157819] Updated weights for policy 0, policy_version 176028 (0.0006) [2024-09-30 03:09:32,828][1157819] Updated weights for policy 0, policy_version 176038 (0.0006) [2024-09-30 03:09:33,348][1157819] Updated weights for policy 0, policy_version 176048 (0.0006) [2024-09-30 03:09:33,893][1157819] Updated weights for policy 0, policy_version 176058 (0.0006) [2024-09-30 03:09:34,396][1157819] Updated weights for policy 0, policy_version 176068 (0.0006) [2024-09-30 03:09:34,899][1157819] Updated weights for policy 0, policy_version 176078 (0.0006) [2024-09-30 03:09:35,401][1157819] Updated weights for policy 0, policy_version 176088 (0.0006) [2024-09-30 03:09:35,466][1157520] Fps is (10 sec: 79872.5, 60 sec: 80418.0, 300 sec: 79351.3). Total num frames: 721260544. Throughput: 0: 19974.5. Samples: 170301728. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 03:09:35,466][1157520] Avg episode reward: [(0, '56.348')] [2024-09-30 03:09:35,909][1157819] Updated weights for policy 0, policy_version 176098 (0.0006) [2024-09-30 03:09:36,441][1157819] Updated weights for policy 0, policy_version 176108 (0.0006) [2024-09-30 03:09:36,963][1157819] Updated weights for policy 0, policy_version 176118 (0.0006) [2024-09-30 03:09:37,444][1157819] Updated weights for policy 0, policy_version 176128 (0.0006) [2024-09-30 03:09:37,941][1157819] Updated weights for policy 0, policy_version 176138 (0.0006) [2024-09-30 03:09:38,441][1157819] Updated weights for policy 0, policy_version 176148 (0.0006) [2024-09-30 03:09:38,924][1157819] Updated weights for policy 0, policy_version 176158 (0.0006) [2024-09-30 03:09:39,426][1157819] Updated weights for policy 0, policy_version 176168 (0.0006) [2024-09-30 03:09:39,911][1157819] Updated weights for policy 0, policy_version 176178 (0.0006) [2024-09-30 03:09:40,383][1157819] Updated weights for policy 0, policy_version 176188 (0.0006) [2024-09-30 03:09:40,466][1157520] Fps is (10 sec: 81100.5, 60 sec: 80281.4, 300 sec: 79268.0). Total num frames: 721670144. Throughput: 0: 19989.8. Samples: 170361832. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 03:09:40,466][1157520] Avg episode reward: [(0, '57.482')] [2024-09-30 03:09:40,883][1157819] Updated weights for policy 0, policy_version 176198 (0.0006) [2024-09-30 03:09:41,373][1157819] Updated weights for policy 0, policy_version 176208 (0.0006) [2024-09-30 03:09:41,844][1157819] Updated weights for policy 0, policy_version 176218 (0.0006) [2024-09-30 03:09:42,325][1157819] Updated weights for policy 0, policy_version 176228 (0.0006) [2024-09-30 03:09:42,811][1157819] Updated weights for policy 0, policy_version 176238 (0.0006) [2024-09-30 03:09:43,343][1157819] Updated weights for policy 0, policy_version 176248 (0.0006) [2024-09-30 03:09:43,867][1157819] Updated weights for policy 0, policy_version 176258 (0.0006) [2024-09-30 03:09:44,366][1157819] Updated weights for policy 0, policy_version 176268 (0.0006) [2024-09-30 03:09:44,878][1157819] Updated weights for policy 0, policy_version 176278 (0.0006) [2024-09-30 03:09:45,363][1157819] Updated weights for policy 0, policy_version 176288 (0.0006) [2024-09-30 03:09:45,466][1157520] Fps is (10 sec: 82329.1, 60 sec: 80349.8, 300 sec: 79184.7). Total num frames: 722083840. Throughput: 0: 20100.8. Samples: 170486904. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 03:09:45,466][1157520] Avg episode reward: [(0, '54.893')] [2024-09-30 03:09:45,882][1157819] Updated weights for policy 0, policy_version 176298 (0.0006) [2024-09-30 03:09:46,414][1157819] Updated weights for policy 0, policy_version 176308 (0.0006) [2024-09-30 03:09:46,898][1157819] Updated weights for policy 0, policy_version 176318 (0.0006) [2024-09-30 03:09:47,431][1157819] Updated weights for policy 0, policy_version 176328 (0.0006) [2024-09-30 03:09:47,941][1157819] Updated weights for policy 0, policy_version 176338 (0.0006) [2024-09-30 03:09:48,465][1157819] Updated weights for policy 0, policy_version 176348 (0.0006) [2024-09-30 03:09:48,977][1157819] Updated weights for policy 0, policy_version 176358 (0.0006) [2024-09-30 03:09:49,517][1157819] Updated weights for policy 0, policy_version 176368 (0.0006) [2024-09-30 03:09:50,015][1157819] Updated weights for policy 0, policy_version 176378 (0.0006) [2024-09-30 03:09:50,466][1157520] Fps is (10 sec: 80691.9, 60 sec: 80213.4, 300 sec: 79156.9). Total num frames: 722477056. Throughput: 0: 20087.6. Samples: 170606716. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 03:09:50,466][1157520] Avg episode reward: [(0, '55.733')] [2024-09-30 03:09:50,511][1157819] Updated weights for policy 0, policy_version 176388 (0.0006) [2024-09-30 03:09:51,003][1157819] Updated weights for policy 0, policy_version 176398 (0.0006) [2024-09-30 03:09:51,518][1157819] Updated weights for policy 0, policy_version 176408 (0.0006) [2024-09-30 03:09:52,050][1157819] Updated weights for policy 0, policy_version 176418 (0.0007) [2024-09-30 03:09:52,573][1157819] Updated weights for policy 0, policy_version 176428 (0.0007) [2024-09-30 03:09:53,081][1157819] Updated weights for policy 0, policy_version 176438 (0.0006) [2024-09-30 03:09:53,591][1157819] Updated weights for policy 0, policy_version 176448 (0.0006) [2024-09-30 03:09:54,104][1157819] Updated weights for policy 0, policy_version 176458 (0.0006) [2024-09-30 03:09:54,602][1157819] Updated weights for policy 0, policy_version 176468 (0.0006) [2024-09-30 03:09:55,116][1157819] Updated weights for policy 0, policy_version 176478 (0.0006) [2024-09-30 03:09:55,466][1157520] Fps is (10 sec: 79462.3, 60 sec: 80281.5, 300 sec: 79156.9). Total num frames: 722878464. Throughput: 0: 20129.9. Samples: 170666680. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 03:09:55,466][1157520] Avg episode reward: [(0, '54.001')] [2024-09-30 03:09:55,631][1157819] Updated weights for policy 0, policy_version 176488 (0.0006) [2024-09-30 03:09:56,208][1157819] Updated weights for policy 0, policy_version 176498 (0.0006) [2024-09-30 03:09:56,761][1157819] Updated weights for policy 0, policy_version 176508 (0.0006) [2024-09-30 03:09:57,287][1157819] Updated weights for policy 0, policy_version 176518 (0.0006) [2024-09-30 03:09:57,813][1157819] Updated weights for policy 0, policy_version 176528 (0.0006) [2024-09-30 03:09:58,368][1157819] Updated weights for policy 0, policy_version 176538 (0.0006) [2024-09-30 03:09:58,912][1157819] Updated weights for policy 0, policy_version 176548 (0.0006) [2024-09-30 03:09:59,486][1157819] Updated weights for policy 0, policy_version 176558 (0.0006) [2024-09-30 03:10:00,024][1157819] Updated weights for policy 0, policy_version 176568 (0.0006) [2024-09-30 03:10:00,466][1157520] Fps is (10 sec: 77824.0, 60 sec: 79872.0, 300 sec: 79004.2). Total num frames: 723255296. Throughput: 0: 20031.9. Samples: 170782564. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 03:10:00,466][1157520] Avg episode reward: [(0, '55.484')] [2024-09-30 03:10:00,584][1157819] Updated weights for policy 0, policy_version 176578 (0.0006) [2024-09-30 03:10:01,117][1157819] Updated weights for policy 0, policy_version 176588 (0.0006) [2024-09-30 03:10:01,653][1157819] Updated weights for policy 0, policy_version 176598 (0.0006) [2024-09-30 03:10:02,176][1157819] Updated weights for policy 0, policy_version 176608 (0.0006) [2024-09-30 03:10:02,719][1157819] Updated weights for policy 0, policy_version 176618 (0.0006) [2024-09-30 03:10:03,256][1157819] Updated weights for policy 0, policy_version 176628 (0.0006) [2024-09-30 03:10:03,807][1157819] Updated weights for policy 0, policy_version 176638 (0.0006) [2024-09-30 03:10:04,342][1157819] Updated weights for policy 0, policy_version 176648 (0.0006) [2024-09-30 03:10:04,865][1157819] Updated weights for policy 0, policy_version 176658 (0.0006) [2024-09-30 03:10:05,408][1157819] Updated weights for policy 0, policy_version 176668 (0.0006) [2024-09-30 03:10:05,466][1157520] Fps is (10 sec: 75366.8, 60 sec: 79462.5, 300 sec: 78920.9). Total num frames: 723632128. Throughput: 0: 19913.2. Samples: 170896120. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 03:10:05,466][1157520] Avg episode reward: [(0, '55.959')] [2024-09-30 03:10:05,478][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000176669_723636224.pth... [2024-09-30 03:10:05,524][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000172108_704954368.pth [2024-09-30 03:10:05,946][1157819] Updated weights for policy 0, policy_version 176678 (0.0006) [2024-09-30 03:10:06,532][1157819] Updated weights for policy 0, policy_version 176688 (0.0006) [2024-09-30 03:10:07,059][1157819] Updated weights for policy 0, policy_version 176698 (0.0006) [2024-09-30 03:10:07,576][1157819] Updated weights for policy 0, policy_version 176708 (0.0006) [2024-09-30 03:10:08,074][1157819] Updated weights for policy 0, policy_version 176718 (0.0006) [2024-09-30 03:10:08,602][1157819] Updated weights for policy 0, policy_version 176728 (0.0006) [2024-09-30 03:10:09,112][1157819] Updated weights for policy 0, policy_version 176738 (0.0006) [2024-09-30 03:10:09,635][1157819] Updated weights for policy 0, policy_version 176748 (0.0006) [2024-09-30 03:10:10,135][1157819] Updated weights for policy 0, policy_version 176758 (0.0006) [2024-09-30 03:10:10,466][1157520] Fps is (10 sec: 77004.6, 60 sec: 79599.0, 300 sec: 78893.1). Total num frames: 724025344. Throughput: 0: 19829.9. Samples: 170953112. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 03:10:10,466][1157520] Avg episode reward: [(0, '54.838')] [2024-09-30 03:10:10,662][1157819] Updated weights for policy 0, policy_version 176768 (0.0006) [2024-09-30 03:10:11,159][1157819] Updated weights for policy 0, policy_version 176778 (0.0006) [2024-09-30 03:10:11,667][1157819] Updated weights for policy 0, policy_version 176788 (0.0006) [2024-09-30 03:10:12,187][1157819] Updated weights for policy 0, policy_version 176798 (0.0006) [2024-09-30 03:10:12,303][1157736] Signal inference workers to stop experience collection... (11950 times) [2024-09-30 03:10:12,303][1157736] Signal inference workers to resume experience collection... (11950 times) [2024-09-30 03:10:12,307][1157819] InferenceWorker_p0-w0: stopping experience collection (11950 times) [2024-09-30 03:10:12,307][1157819] InferenceWorker_p0-w0: resuming experience collection (11950 times) [2024-09-30 03:10:12,705][1157819] Updated weights for policy 0, policy_version 176808 (0.0006) [2024-09-30 03:10:13,203][1157819] Updated weights for policy 0, policy_version 176818 (0.0006) [2024-09-30 03:10:13,723][1157819] Updated weights for policy 0, policy_version 176828 (0.0006) [2024-09-30 03:10:14,260][1157819] Updated weights for policy 0, policy_version 176838 (0.0006) [2024-09-30 03:10:14,784][1157819] Updated weights for policy 0, policy_version 176848 (0.0006) [2024-09-30 03:10:15,297][1157819] Updated weights for policy 0, policy_version 176858 (0.0006) [2024-09-30 03:10:15,466][1157520] Fps is (10 sec: 79053.0, 60 sec: 79394.2, 300 sec: 78795.9). Total num frames: 724422656. Throughput: 0: 19791.0. Samples: 171072740. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 03:10:15,466][1157520] Avg episode reward: [(0, '55.843')] [2024-09-30 03:10:15,822][1157819] Updated weights for policy 0, policy_version 176868 (0.0006) [2024-09-30 03:10:16,322][1157819] Updated weights for policy 0, policy_version 176878 (0.0006) [2024-09-30 03:10:16,834][1157819] Updated weights for policy 0, policy_version 176888 (0.0006) [2024-09-30 03:10:17,365][1157819] Updated weights for policy 0, policy_version 176898 (0.0006) [2024-09-30 03:10:17,884][1157819] Updated weights for policy 0, policy_version 176908 (0.0006) [2024-09-30 03:10:18,427][1157819] Updated weights for policy 0, policy_version 176918 (0.0006) [2024-09-30 03:10:18,966][1157819] Updated weights for policy 0, policy_version 176928 (0.0006) [2024-09-30 03:10:19,504][1157819] Updated weights for policy 0, policy_version 176938 (0.0006) [2024-09-30 03:10:20,058][1157819] Updated weights for policy 0, policy_version 176948 (0.0006) [2024-09-30 03:10:20,466][1157520] Fps is (10 sec: 78233.1, 60 sec: 79257.6, 300 sec: 78671.0). Total num frames: 724807680. Throughput: 0: 19724.1. Samples: 171189316. Policy #0 lag: (min: 0.0, avg: 2.2, max: 6.0) [2024-09-30 03:10:20,466][1157520] Avg episode reward: [(0, '54.768')] [2024-09-30 03:10:20,581][1157819] Updated weights for policy 0, policy_version 176958 (0.0006) [2024-09-30 03:10:21,106][1157819] Updated weights for policy 0, policy_version 176968 (0.0006) [2024-09-30 03:10:21,655][1157819] Updated weights for policy 0, policy_version 176978 (0.0006) [2024-09-30 03:10:22,186][1157819] Updated weights for policy 0, policy_version 176988 (0.0006) [2024-09-30 03:10:22,673][1157819] Updated weights for policy 0, policy_version 176998 (0.0006) [2024-09-30 03:10:23,204][1157819] Updated weights for policy 0, policy_version 177008 (0.0006) [2024-09-30 03:10:23,730][1157819] Updated weights for policy 0, policy_version 177018 (0.0006) [2024-09-30 03:10:24,272][1157819] Updated weights for policy 0, policy_version 177028 (0.0006) [2024-09-30 03:10:24,757][1157819] Updated weights for policy 0, policy_version 177038 (0.0006) [2024-09-30 03:10:25,267][1157819] Updated weights for policy 0, policy_version 177048 (0.0006) [2024-09-30 03:10:25,466][1157520] Fps is (10 sec: 77823.0, 60 sec: 78984.4, 300 sec: 78532.1). Total num frames: 725200896. Throughput: 0: 19692.7. Samples: 171248004. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:10:25,466][1157520] Avg episode reward: [(0, '55.551')] [2024-09-30 03:10:25,810][1157819] Updated weights for policy 0, policy_version 177058 (0.0006) [2024-09-30 03:10:26,335][1157819] Updated weights for policy 0, policy_version 177068 (0.0006) [2024-09-30 03:10:26,833][1157819] Updated weights for policy 0, policy_version 177078 (0.0006) [2024-09-30 03:10:27,365][1157819] Updated weights for policy 0, policy_version 177088 (0.0006) [2024-09-30 03:10:27,889][1157819] Updated weights for policy 0, policy_version 177098 (0.0006) [2024-09-30 03:10:28,425][1157819] Updated weights for policy 0, policy_version 177108 (0.0006) [2024-09-30 03:10:28,924][1157819] Updated weights for policy 0, policy_version 177118 (0.0006) [2024-09-30 03:10:29,430][1157819] Updated weights for policy 0, policy_version 177128 (0.0006) [2024-09-30 03:10:29,946][1157819] Updated weights for policy 0, policy_version 177138 (0.0006) [2024-09-30 03:10:30,466][1157520] Fps is (10 sec: 78643.3, 60 sec: 78916.2, 300 sec: 78379.4). Total num frames: 725594112. Throughput: 0: 19535.0. Samples: 171365980. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:10:30,466][1157520] Avg episode reward: [(0, '56.921')] [2024-09-30 03:10:30,499][1157819] Updated weights for policy 0, policy_version 177148 (0.0006) [2024-09-30 03:10:30,984][1157819] Updated weights for policy 0, policy_version 177158 (0.0006) [2024-09-30 03:10:31,462][1157819] Updated weights for policy 0, policy_version 177168 (0.0006) [2024-09-30 03:10:32,014][1157819] Updated weights for policy 0, policy_version 177178 (0.0006) [2024-09-30 03:10:32,561][1157819] Updated weights for policy 0, policy_version 177188 (0.0006) [2024-09-30 03:10:33,112][1157819] Updated weights for policy 0, policy_version 177198 (0.0006) [2024-09-30 03:10:33,660][1157819] Updated weights for policy 0, policy_version 177208 (0.0006) [2024-09-30 03:10:34,221][1157819] Updated weights for policy 0, policy_version 177218 (0.0006) [2024-09-30 03:10:34,791][1157819] Updated weights for policy 0, policy_version 177228 (0.0006) [2024-09-30 03:10:35,324][1157819] Updated weights for policy 0, policy_version 177238 (0.0006) [2024-09-30 03:10:35,466][1157520] Fps is (10 sec: 77824.8, 60 sec: 78643.2, 300 sec: 78254.4). Total num frames: 725979136. Throughput: 0: 19458.3. Samples: 171482340. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:10:35,466][1157520] Avg episode reward: [(0, '56.529')] [2024-09-30 03:10:35,855][1157819] Updated weights for policy 0, policy_version 177248 (0.0006) [2024-09-30 03:10:36,404][1157819] Updated weights for policy 0, policy_version 177258 (0.0006) [2024-09-30 03:10:36,955][1157819] Updated weights for policy 0, policy_version 177268 (0.0006) [2024-09-30 03:10:37,496][1157819] Updated weights for policy 0, policy_version 177278 (0.0006) [2024-09-30 03:10:38,070][1157819] Updated weights for policy 0, policy_version 177288 (0.0006) [2024-09-30 03:10:38,573][1157819] Updated weights for policy 0, policy_version 177298 (0.0006) [2024-09-30 03:10:39,113][1157819] Updated weights for policy 0, policy_version 177308 (0.0006) [2024-09-30 03:10:39,695][1157819] Updated weights for policy 0, policy_version 177318 (0.0006) [2024-09-30 03:10:40,212][1157819] Updated weights for policy 0, policy_version 177328 (0.0006) [2024-09-30 03:10:40,466][1157520] Fps is (10 sec: 75366.2, 60 sec: 77960.5, 300 sec: 78129.5). Total num frames: 726347776. Throughput: 0: 19369.7. Samples: 171538316. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:10:40,466][1157520] Avg episode reward: [(0, '54.911')] [2024-09-30 03:10:40,791][1157819] Updated weights for policy 0, policy_version 177338 (0.0006) [2024-09-30 03:10:41,336][1157819] Updated weights for policy 0, policy_version 177348 (0.0006) [2024-09-30 03:10:41,906][1157819] Updated weights for policy 0, policy_version 177358 (0.0006) [2024-09-30 03:10:42,465][1157819] Updated weights for policy 0, policy_version 177368 (0.0006) [2024-09-30 03:10:43,026][1157819] Updated weights for policy 0, policy_version 177378 (0.0006) [2024-09-30 03:10:43,624][1157819] Updated weights for policy 0, policy_version 177388 (0.0006) [2024-09-30 03:10:44,177][1157819] Updated weights for policy 0, policy_version 177398 (0.0006) [2024-09-30 03:10:44,749][1157819] Updated weights for policy 0, policy_version 177408 (0.0006) [2024-09-30 03:10:45,276][1157819] Updated weights for policy 0, policy_version 177418 (0.0006) [2024-09-30 03:10:45,466][1157520] Fps is (10 sec: 73727.9, 60 sec: 77209.6, 300 sec: 77976.7). Total num frames: 726716416. Throughput: 0: 19236.0. Samples: 171648184. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:10:45,466][1157520] Avg episode reward: [(0, '55.533')] [2024-09-30 03:10:45,803][1157819] Updated weights for policy 0, policy_version 177428 (0.0006) [2024-09-30 03:10:46,313][1157819] Updated weights for policy 0, policy_version 177438 (0.0006) [2024-09-30 03:10:46,823][1157819] Updated weights for policy 0, policy_version 177448 (0.0006) [2024-09-30 03:10:47,359][1157819] Updated weights for policy 0, policy_version 177458 (0.0006) [2024-09-30 03:10:47,863][1157819] Updated weights for policy 0, policy_version 177468 (0.0006) [2024-09-30 03:10:48,398][1157819] Updated weights for policy 0, policy_version 177478 (0.0006) [2024-09-30 03:10:48,896][1157819] Updated weights for policy 0, policy_version 177488 (0.0006) [2024-09-30 03:10:49,455][1157819] Updated weights for policy 0, policy_version 177498 (0.0006) [2024-09-30 03:10:49,989][1157819] Updated weights for policy 0, policy_version 177508 (0.0006) [2024-09-30 03:10:50,466][1157520] Fps is (10 sec: 75776.4, 60 sec: 77141.3, 300 sec: 77865.7). Total num frames: 727105536. Throughput: 0: 19295.8. Samples: 171764432. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:10:50,466][1157520] Avg episode reward: [(0, '57.669')] [2024-09-30 03:10:50,506][1157819] Updated weights for policy 0, policy_version 177518 (0.0006) [2024-09-30 03:10:51,077][1157819] Updated weights for policy 0, policy_version 177528 (0.0006) [2024-09-30 03:10:51,632][1157819] Updated weights for policy 0, policy_version 177538 (0.0006) [2024-09-30 03:10:52,138][1157819] Updated weights for policy 0, policy_version 177548 (0.0006) [2024-09-30 03:10:52,704][1157819] Updated weights for policy 0, policy_version 177558 (0.0006) [2024-09-30 03:10:53,285][1157819] Updated weights for policy 0, policy_version 177568 (0.0006) [2024-09-30 03:10:53,830][1157819] Updated weights for policy 0, policy_version 177578 (0.0006) [2024-09-30 03:10:54,397][1157819] Updated weights for policy 0, policy_version 177588 (0.0006) [2024-09-30 03:10:54,950][1157819] Updated weights for policy 0, policy_version 177598 (0.0006) [2024-09-30 03:10:55,466][1157520] Fps is (10 sec: 76185.7, 60 sec: 76663.5, 300 sec: 77754.6). Total num frames: 727478272. Throughput: 0: 19275.6. Samples: 171820512. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:10:55,466][1157520] Avg episode reward: [(0, '55.021')] [2024-09-30 03:10:55,510][1157819] Updated weights for policy 0, policy_version 177608 (0.0006) [2024-09-30 03:10:56,080][1157819] Updated weights for policy 0, policy_version 177618 (0.0006) [2024-09-30 03:10:56,589][1157819] Updated weights for policy 0, policy_version 177628 (0.0006) [2024-09-30 03:10:57,125][1157819] Updated weights for policy 0, policy_version 177638 (0.0006) [2024-09-30 03:10:57,693][1157819] Updated weights for policy 0, policy_version 177648 (0.0006) [2024-09-30 03:10:58,205][1157819] Updated weights for policy 0, policy_version 177658 (0.0006) [2024-09-30 03:10:58,817][1157819] Updated weights for policy 0, policy_version 177668 (0.0006) [2024-09-30 03:10:59,333][1157819] Updated weights for policy 0, policy_version 177678 (0.0006) [2024-09-30 03:10:59,854][1157819] Updated weights for policy 0, policy_version 177688 (0.0006) [2024-09-30 03:11:00,446][1157819] Updated weights for policy 0, policy_version 177698 (0.0006) [2024-09-30 03:11:00,466][1157520] Fps is (10 sec: 74547.5, 60 sec: 76595.2, 300 sec: 77615.8). Total num frames: 727851008. Throughput: 0: 19098.0. Samples: 171932152. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:11:00,466][1157520] Avg episode reward: [(0, '55.593')] [2024-09-30 03:11:01,017][1157819] Updated weights for policy 0, policy_version 177708 (0.0006) [2024-09-30 03:11:01,614][1157819] Updated weights for policy 0, policy_version 177718 (0.0006) [2024-09-30 03:11:02,204][1157819] Updated weights for policy 0, policy_version 177728 (0.0006) [2024-09-30 03:11:02,766][1157819] Updated weights for policy 0, policy_version 177738 (0.0006) [2024-09-30 03:11:03,358][1157819] Updated weights for policy 0, policy_version 177748 (0.0006) [2024-09-30 03:11:03,971][1157819] Updated weights for policy 0, policy_version 177758 (0.0006) [2024-09-30 03:11:04,555][1157819] Updated weights for policy 0, policy_version 177768 (0.0006) [2024-09-30 03:11:05,162][1157819] Updated weights for policy 0, policy_version 177778 (0.0006) [2024-09-30 03:11:05,466][1157520] Fps is (10 sec: 72088.7, 60 sec: 76117.2, 300 sec: 77490.7). Total num frames: 728199168. Throughput: 0: 18870.2. Samples: 172038476. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:11:05,466][1157520] Avg episode reward: [(0, '55.842')] [2024-09-30 03:11:05,742][1157819] Updated weights for policy 0, policy_version 177788 (0.0006) [2024-09-30 03:11:06,305][1157819] Updated weights for policy 0, policy_version 177798 (0.0006) [2024-09-30 03:11:06,899][1157819] Updated weights for policy 0, policy_version 177808 (0.0006) [2024-09-30 03:11:07,496][1157819] Updated weights for policy 0, policy_version 177818 (0.0006) [2024-09-30 03:11:08,092][1157819] Updated weights for policy 0, policy_version 177828 (0.0006) [2024-09-30 03:11:08,628][1157819] Updated weights for policy 0, policy_version 177838 (0.0006) [2024-09-30 03:11:09,168][1157819] Updated weights for policy 0, policy_version 177848 (0.0006) [2024-09-30 03:11:09,749][1157819] Updated weights for policy 0, policy_version 177858 (0.0006) [2024-09-30 03:11:10,314][1157819] Updated weights for policy 0, policy_version 177868 (0.0006) [2024-09-30 03:11:10,466][1157520] Fps is (10 sec: 70451.2, 60 sec: 75502.9, 300 sec: 77393.6). Total num frames: 728555520. Throughput: 0: 18724.6. Samples: 172090608. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:11:10,466][1157520] Avg episode reward: [(0, '57.206')] [2024-09-30 03:11:10,843][1157819] Updated weights for policy 0, policy_version 177878 (0.0006) [2024-09-30 03:11:11,419][1157819] Updated weights for policy 0, policy_version 177888 (0.0006) [2024-09-30 03:11:11,981][1157819] Updated weights for policy 0, policy_version 177898 (0.0006) [2024-09-30 03:11:12,530][1157819] Updated weights for policy 0, policy_version 177908 (0.0006) [2024-09-30 03:11:13,111][1157819] Updated weights for policy 0, policy_version 177918 (0.0006) [2024-09-30 03:11:13,652][1157819] Updated weights for policy 0, policy_version 177928 (0.0006) [2024-09-30 03:11:14,243][1157819] Updated weights for policy 0, policy_version 177938 (0.0006) [2024-09-30 03:11:14,782][1157819] Updated weights for policy 0, policy_version 177948 (0.0006) [2024-09-30 03:11:15,350][1157819] Updated weights for policy 0, policy_version 177958 (0.0006) [2024-09-30 03:11:15,466][1157520] Fps is (10 sec: 72500.0, 60 sec: 75025.0, 300 sec: 77296.4). Total num frames: 728924160. Throughput: 0: 18544.7. Samples: 172200492. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:11:15,466][1157520] Avg episode reward: [(0, '54.287')] [2024-09-30 03:11:15,893][1157819] Updated weights for policy 0, policy_version 177968 (0.0006) [2024-09-30 03:11:16,466][1157819] Updated weights for policy 0, policy_version 177978 (0.0006) [2024-09-30 03:11:17,000][1157819] Updated weights for policy 0, policy_version 177988 (0.0006) [2024-09-30 03:11:17,547][1157819] Updated weights for policy 0, policy_version 177998 (0.0006) [2024-09-30 03:11:18,098][1157819] Updated weights for policy 0, policy_version 178008 (0.0006) [2024-09-30 03:11:18,613][1157819] Updated weights for policy 0, policy_version 178018 (0.0006) [2024-09-30 03:11:19,164][1157819] Updated weights for policy 0, policy_version 178028 (0.0006) [2024-09-30 03:11:19,722][1157819] Updated weights for policy 0, policy_version 178038 (0.0006) [2024-09-30 03:11:20,244][1157819] Updated weights for policy 0, policy_version 178048 (0.0006) [2024-09-30 03:11:20,358][1157736] Signal inference workers to stop experience collection... (12000 times) [2024-09-30 03:11:20,361][1157736] Signal inference workers to resume experience collection... (12000 times) [2024-09-30 03:11:20,364][1157819] InferenceWorker_p0-w0: stopping experience collection (12000 times) [2024-09-30 03:11:20,368][1157819] InferenceWorker_p0-w0: resuming experience collection (12000 times) [2024-09-30 03:11:20,466][1157520] Fps is (10 sec: 74546.6, 60 sec: 74888.5, 300 sec: 77240.8). Total num frames: 729300992. Throughput: 0: 18449.7. Samples: 172312576. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:11:20,466][1157520] Avg episode reward: [(0, '54.994')] [2024-09-30 03:11:20,768][1157819] Updated weights for policy 0, policy_version 178058 (0.0006) [2024-09-30 03:11:21,309][1157819] Updated weights for policy 0, policy_version 178068 (0.0006) [2024-09-30 03:11:21,841][1157819] Updated weights for policy 0, policy_version 178078 (0.0006) [2024-09-30 03:11:22,386][1157819] Updated weights for policy 0, policy_version 178088 (0.0006) [2024-09-30 03:11:22,914][1157819] Updated weights for policy 0, policy_version 178098 (0.0006) [2024-09-30 03:11:23,487][1157819] Updated weights for policy 0, policy_version 178108 (0.0006) [2024-09-30 03:11:24,009][1157819] Updated weights for policy 0, policy_version 178118 (0.0006) [2024-09-30 03:11:24,511][1157819] Updated weights for policy 0, policy_version 178128 (0.0006) [2024-09-30 03:11:25,036][1157819] Updated weights for policy 0, policy_version 178138 (0.0006) [2024-09-30 03:11:25,466][1157520] Fps is (10 sec: 76185.6, 60 sec: 74752.1, 300 sec: 77185.3). Total num frames: 729686016. Throughput: 0: 18467.1. Samples: 172369336. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:11:25,466][1157520] Avg episode reward: [(0, '56.447')] [2024-09-30 03:11:25,576][1157819] Updated weights for policy 0, policy_version 178148 (0.0006) [2024-09-30 03:11:26,087][1157819] Updated weights for policy 0, policy_version 178158 (0.0006) [2024-09-30 03:11:26,596][1157819] Updated weights for policy 0, policy_version 178168 (0.0006) [2024-09-30 03:11:27,150][1157819] Updated weights for policy 0, policy_version 178178 (0.0006) [2024-09-30 03:11:27,688][1157819] Updated weights for policy 0, policy_version 178188 (0.0006) [2024-09-30 03:11:28,221][1157819] Updated weights for policy 0, policy_version 178198 (0.0006) [2024-09-30 03:11:28,749][1157819] Updated weights for policy 0, policy_version 178208 (0.0006) [2024-09-30 03:11:29,270][1157819] Updated weights for policy 0, policy_version 178218 (0.0006) [2024-09-30 03:11:29,844][1157819] Updated weights for policy 0, policy_version 178228 (0.0006) [2024-09-30 03:11:30,404][1157819] Updated weights for policy 0, policy_version 178238 (0.0006) [2024-09-30 03:11:30,466][1157520] Fps is (10 sec: 76595.6, 60 sec: 74547.2, 300 sec: 77074.2). Total num frames: 730066944. Throughput: 0: 18609.2. Samples: 172485600. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 03:11:30,466][1157520] Avg episode reward: [(0, '56.148')] [2024-09-30 03:11:30,932][1157819] Updated weights for policy 0, policy_version 178248 (0.0006) [2024-09-30 03:11:31,495][1157819] Updated weights for policy 0, policy_version 178258 (0.0006) [2024-09-30 03:11:32,054][1157819] Updated weights for policy 0, policy_version 178268 (0.0006) [2024-09-30 03:11:32,610][1157819] Updated weights for policy 0, policy_version 178278 (0.0006) [2024-09-30 03:11:33,151][1157819] Updated weights for policy 0, policy_version 178288 (0.0006) [2024-09-30 03:11:33,709][1157819] Updated weights for policy 0, policy_version 178298 (0.0006) [2024-09-30 03:11:34,251][1157819] Updated weights for policy 0, policy_version 178308 (0.0006) [2024-09-30 03:11:34,788][1157819] Updated weights for policy 0, policy_version 178318 (0.0006) [2024-09-30 03:11:35,344][1157819] Updated weights for policy 0, policy_version 178328 (0.0006) [2024-09-30 03:11:35,466][1157520] Fps is (10 sec: 74957.0, 60 sec: 74274.2, 300 sec: 76907.6). Total num frames: 730435584. Throughput: 0: 18513.6. Samples: 172597544. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 03:11:35,466][1157520] Avg episode reward: [(0, '56.089')] [2024-09-30 03:11:35,892][1157819] Updated weights for policy 0, policy_version 178338 (0.0006) [2024-09-30 03:11:36,428][1157819] Updated weights for policy 0, policy_version 178348 (0.0006) [2024-09-30 03:11:36,964][1157819] Updated weights for policy 0, policy_version 178358 (0.0006) [2024-09-30 03:11:37,529][1157819] Updated weights for policy 0, policy_version 178368 (0.0006) [2024-09-30 03:11:38,079][1157819] Updated weights for policy 0, policy_version 178378 (0.0006) [2024-09-30 03:11:38,615][1157819] Updated weights for policy 0, policy_version 178388 (0.0006) [2024-09-30 03:11:39,167][1157819] Updated weights for policy 0, policy_version 178398 (0.0006) [2024-09-30 03:11:39,728][1157819] Updated weights for policy 0, policy_version 178408 (0.0006) [2024-09-30 03:11:40,294][1157819] Updated weights for policy 0, policy_version 178418 (0.0006) [2024-09-30 03:11:40,466][1157520] Fps is (10 sec: 74546.2, 60 sec: 74410.6, 300 sec: 76782.6). Total num frames: 730812416. Throughput: 0: 18509.5. Samples: 172653444. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 03:11:40,466][1157520] Avg episode reward: [(0, '54.698')] [2024-09-30 03:11:40,863][1157819] Updated weights for policy 0, policy_version 178428 (0.0006) [2024-09-30 03:11:41,433][1157819] Updated weights for policy 0, policy_version 178438 (0.0006) [2024-09-30 03:11:42,035][1157819] Updated weights for policy 0, policy_version 178448 (0.0006) [2024-09-30 03:11:42,612][1157819] Updated weights for policy 0, policy_version 178458 (0.0006) [2024-09-30 03:11:43,162][1157819] Updated weights for policy 0, policy_version 178468 (0.0006) [2024-09-30 03:11:43,694][1157819] Updated weights for policy 0, policy_version 178478 (0.0006) [2024-09-30 03:11:44,262][1157819] Updated weights for policy 0, policy_version 178488 (0.0006) [2024-09-30 03:11:44,823][1157819] Updated weights for policy 0, policy_version 178498 (0.0006) [2024-09-30 03:11:45,369][1157819] Updated weights for policy 0, policy_version 178508 (0.0006) [2024-09-30 03:11:45,466][1157520] Fps is (10 sec: 73726.8, 60 sec: 74274.0, 300 sec: 76699.3). Total num frames: 731172864. Throughput: 0: 18462.4. Samples: 172762964. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 03:11:45,466][1157520] Avg episode reward: [(0, '56.799')] [2024-09-30 03:11:45,926][1157819] Updated weights for policy 0, policy_version 178518 (0.0006) [2024-09-30 03:11:46,474][1157819] Updated weights for policy 0, policy_version 178528 (0.0006) [2024-09-30 03:11:47,044][1157819] Updated weights for policy 0, policy_version 178538 (0.0006) [2024-09-30 03:11:47,599][1157819] Updated weights for policy 0, policy_version 178548 (0.0006) [2024-09-30 03:11:48,126][1157819] Updated weights for policy 0, policy_version 178558 (0.0006) [2024-09-30 03:11:48,695][1157819] Updated weights for policy 0, policy_version 178568 (0.0006) [2024-09-30 03:11:49,263][1157819] Updated weights for policy 0, policy_version 178578 (0.0006) [2024-09-30 03:11:49,817][1157819] Updated weights for policy 0, policy_version 178588 (0.0006) [2024-09-30 03:11:50,364][1157819] Updated weights for policy 0, policy_version 178598 (0.0006) [2024-09-30 03:11:50,466][1157520] Fps is (10 sec: 73319.7, 60 sec: 74001.1, 300 sec: 76727.1). Total num frames: 731545600. Throughput: 0: 18560.6. Samples: 172873700. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 03:11:50,466][1157520] Avg episode reward: [(0, '54.542')] [2024-09-30 03:11:50,887][1157819] Updated weights for policy 0, policy_version 178608 (0.0006) [2024-09-30 03:11:51,436][1157819] Updated weights for policy 0, policy_version 178618 (0.0006) [2024-09-30 03:11:51,991][1157819] Updated weights for policy 0, policy_version 178628 (0.0006) [2024-09-30 03:11:52,531][1157819] Updated weights for policy 0, policy_version 178638 (0.0006) [2024-09-30 03:11:53,108][1157819] Updated weights for policy 0, policy_version 178648 (0.0006) [2024-09-30 03:11:53,636][1157819] Updated weights for policy 0, policy_version 178658 (0.0006) [2024-09-30 03:11:54,231][1157819] Updated weights for policy 0, policy_version 178668 (0.0006) [2024-09-30 03:11:54,725][1157819] Updated weights for policy 0, policy_version 178678 (0.0006) [2024-09-30 03:11:55,296][1157819] Updated weights for policy 0, policy_version 178688 (0.0006) [2024-09-30 03:11:55,466][1157520] Fps is (10 sec: 74548.5, 60 sec: 74001.1, 300 sec: 76768.7). Total num frames: 731918336. Throughput: 0: 18647.5. Samples: 172929744. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 03:11:55,466][1157520] Avg episode reward: [(0, '55.951')] [2024-09-30 03:11:55,846][1157819] Updated weights for policy 0, policy_version 178698 (0.0006) [2024-09-30 03:11:56,406][1157819] Updated weights for policy 0, policy_version 178708 (0.0006) [2024-09-30 03:11:56,959][1157819] Updated weights for policy 0, policy_version 178718 (0.0006) [2024-09-30 03:11:57,509][1157819] Updated weights for policy 0, policy_version 178728 (0.0006) [2024-09-30 03:11:58,047][1157819] Updated weights for policy 0, policy_version 178738 (0.0007) [2024-09-30 03:11:58,629][1157819] Updated weights for policy 0, policy_version 178748 (0.0006) [2024-09-30 03:11:59,172][1157819] Updated weights for policy 0, policy_version 178758 (0.0006) [2024-09-30 03:11:59,722][1157819] Updated weights for policy 0, policy_version 178768 (0.0006) [2024-09-30 03:12:00,277][1157819] Updated weights for policy 0, policy_version 178778 (0.0006) [2024-09-30 03:12:00,466][1157520] Fps is (10 sec: 74137.4, 60 sec: 73932.8, 300 sec: 76782.6). Total num frames: 732286976. Throughput: 0: 18681.8. Samples: 173041172. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 03:12:00,466][1157520] Avg episode reward: [(0, '55.077')] [2024-09-30 03:12:00,798][1157819] Updated weights for policy 0, policy_version 178788 (0.0006) [2024-09-30 03:12:01,302][1157819] Updated weights for policy 0, policy_version 178798 (0.0006) [2024-09-30 03:12:01,850][1157819] Updated weights for policy 0, policy_version 178808 (0.0006) [2024-09-30 03:12:02,404][1157819] Updated weights for policy 0, policy_version 178818 (0.0006) [2024-09-30 03:12:02,917][1157819] Updated weights for policy 0, policy_version 178828 (0.0006) [2024-09-30 03:12:03,418][1157819] Updated weights for policy 0, policy_version 178838 (0.0006) [2024-09-30 03:12:03,971][1157819] Updated weights for policy 0, policy_version 178848 (0.0006) [2024-09-30 03:12:04,499][1157819] Updated weights for policy 0, policy_version 178858 (0.0006) [2024-09-30 03:12:05,076][1157819] Updated weights for policy 0, policy_version 178868 (0.0006) [2024-09-30 03:12:05,466][1157520] Fps is (10 sec: 75366.1, 60 sec: 74547.3, 300 sec: 76838.2). Total num frames: 732672000. Throughput: 0: 18731.0. Samples: 173155472. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 03:12:05,466][1157520] Avg episode reward: [(0, '57.080')] [2024-09-30 03:12:05,472][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000178875_732672000.pth... [2024-09-30 03:12:05,518][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000174360_714178560.pth [2024-09-30 03:12:05,642][1157819] Updated weights for policy 0, policy_version 178878 (0.0006) [2024-09-30 03:12:06,204][1157819] Updated weights for policy 0, policy_version 178888 (0.0006) [2024-09-30 03:12:06,779][1157819] Updated weights for policy 0, policy_version 178898 (0.0006) [2024-09-30 03:12:07,328][1157819] Updated weights for policy 0, policy_version 178908 (0.0006) [2024-09-30 03:12:07,836][1157819] Updated weights for policy 0, policy_version 178918 (0.0006) [2024-09-30 03:12:08,364][1157819] Updated weights for policy 0, policy_version 178928 (0.0006) [2024-09-30 03:12:08,949][1157819] Updated weights for policy 0, policy_version 178938 (0.0006) [2024-09-30 03:12:09,477][1157819] Updated weights for policy 0, policy_version 178948 (0.0006) [2024-09-30 03:12:10,048][1157819] Updated weights for policy 0, policy_version 178958 (0.0006) [2024-09-30 03:12:10,466][1157520] Fps is (10 sec: 75367.0, 60 sec: 74752.1, 300 sec: 76852.1). Total num frames: 733040640. Throughput: 0: 18695.3. Samples: 173210624. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 03:12:10,466][1157520] Avg episode reward: [(0, '56.837')] [2024-09-30 03:12:10,612][1157819] Updated weights for policy 0, policy_version 178968 (0.0006) [2024-09-30 03:12:11,225][1157819] Updated weights for policy 0, policy_version 178978 (0.0006) [2024-09-30 03:12:11,781][1157819] Updated weights for policy 0, policy_version 178988 (0.0006) [2024-09-30 03:12:12,390][1157819] Updated weights for policy 0, policy_version 178998 (0.0006) [2024-09-30 03:12:12,972][1157819] Updated weights for policy 0, policy_version 179008 (0.0006) [2024-09-30 03:12:13,552][1157819] Updated weights for policy 0, policy_version 179018 (0.0006) [2024-09-30 03:12:14,192][1157819] Updated weights for policy 0, policy_version 179028 (0.0006) [2024-09-30 03:12:14,762][1157819] Updated weights for policy 0, policy_version 179038 (0.0006) [2024-09-30 03:12:15,346][1157819] Updated weights for policy 0, policy_version 179048 (0.0006) [2024-09-30 03:12:15,466][1157520] Fps is (10 sec: 71270.6, 60 sec: 74342.4, 300 sec: 76768.7). Total num frames: 733384704. Throughput: 0: 18497.2. Samples: 173317976. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 03:12:15,466][1157520] Avg episode reward: [(0, '56.639')] [2024-09-30 03:12:15,966][1157819] Updated weights for policy 0, policy_version 179058 (0.0006) [2024-09-30 03:12:16,560][1157819] Updated weights for policy 0, policy_version 179068 (0.0006) [2024-09-30 03:12:17,168][1157819] Updated weights for policy 0, policy_version 179078 (0.0006) [2024-09-30 03:12:17,792][1157819] Updated weights for policy 0, policy_version 179088 (0.0006) [2024-09-30 03:12:18,374][1157819] Updated weights for policy 0, policy_version 179098 (0.0006) [2024-09-30 03:12:18,966][1157819] Updated weights for policy 0, policy_version 179108 (0.0006) [2024-09-30 03:12:19,570][1157819] Updated weights for policy 0, policy_version 179118 (0.0006) [2024-09-30 03:12:20,171][1157819] Updated weights for policy 0, policy_version 179128 (0.0006) [2024-09-30 03:12:20,466][1157520] Fps is (10 sec: 68402.8, 60 sec: 73728.1, 300 sec: 76629.9). Total num frames: 733724672. Throughput: 0: 18289.2. Samples: 173420560. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 03:12:20,466][1157520] Avg episode reward: [(0, '55.460')] [2024-09-30 03:12:20,750][1157819] Updated weights for policy 0, policy_version 179138 (0.0006) [2024-09-30 03:12:21,333][1157819] Updated weights for policy 0, policy_version 179148 (0.0006) [2024-09-30 03:12:21,924][1157819] Updated weights for policy 0, policy_version 179158 (0.0006) [2024-09-30 03:12:22,537][1157819] Updated weights for policy 0, policy_version 179168 (0.0006) [2024-09-30 03:12:23,070][1157819] Updated weights for policy 0, policy_version 179178 (0.0006) [2024-09-30 03:12:23,633][1157819] Updated weights for policy 0, policy_version 179188 (0.0006) [2024-09-30 03:12:24,192][1157819] Updated weights for policy 0, policy_version 179198 (0.0006) [2024-09-30 03:12:24,731][1157819] Updated weights for policy 0, policy_version 179208 (0.0006) [2024-09-30 03:12:25,227][1157819] Updated weights for policy 0, policy_version 179218 (0.0006) [2024-09-30 03:12:25,466][1157520] Fps is (10 sec: 70450.8, 60 sec: 73386.6, 300 sec: 76504.9). Total num frames: 734089216. Throughput: 0: 18206.4. Samples: 173472732. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 03:12:25,466][1157520] Avg episode reward: [(0, '55.985')] [2024-09-30 03:12:25,663][1157736] Signal inference workers to stop experience collection... (12050 times) [2024-09-30 03:12:25,664][1157736] Signal inference workers to resume experience collection... (12050 times) [2024-09-30 03:12:25,667][1157819] InferenceWorker_p0-w0: stopping experience collection (12050 times) [2024-09-30 03:12:25,667][1157819] InferenceWorker_p0-w0: resuming experience collection (12050 times) [2024-09-30 03:12:25,761][1157819] Updated weights for policy 0, policy_version 179228 (0.0006) [2024-09-30 03:12:26,276][1157819] Updated weights for policy 0, policy_version 179238 (0.0006) [2024-09-30 03:12:26,791][1157819] Updated weights for policy 0, policy_version 179248 (0.0006) [2024-09-30 03:12:27,314][1157819] Updated weights for policy 0, policy_version 179258 (0.0006) [2024-09-30 03:12:27,838][1157819] Updated weights for policy 0, policy_version 179268 (0.0006) [2024-09-30 03:12:28,346][1157819] Updated weights for policy 0, policy_version 179278 (0.0006) [2024-09-30 03:12:28,863][1157819] Updated weights for policy 0, policy_version 179288 (0.0006) [2024-09-30 03:12:29,377][1157819] Updated weights for policy 0, policy_version 179298 (0.0006) [2024-09-30 03:12:29,910][1157819] Updated weights for policy 0, policy_version 179308 (0.0006) [2024-09-30 03:12:30,420][1157819] Updated weights for policy 0, policy_version 179318 (0.0006) [2024-09-30 03:12:30,466][1157520] Fps is (10 sec: 76595.3, 60 sec: 73728.1, 300 sec: 76463.3). Total num frames: 734490624. Throughput: 0: 18361.1. Samples: 173589208. Policy #0 lag: (min: 0.0, avg: 2.7, max: 7.0) [2024-09-30 03:12:30,466][1157520] Avg episode reward: [(0, '56.395')] [2024-09-30 03:12:30,936][1157819] Updated weights for policy 0, policy_version 179328 (0.0006) [2024-09-30 03:12:31,455][1157819] Updated weights for policy 0, policy_version 179338 (0.0006) [2024-09-30 03:12:31,955][1157819] Updated weights for policy 0, policy_version 179348 (0.0006) [2024-09-30 03:12:32,504][1157819] Updated weights for policy 0, policy_version 179358 (0.0006) [2024-09-30 03:12:33,022][1157819] Updated weights for policy 0, policy_version 179368 (0.0006) [2024-09-30 03:12:33,522][1157819] Updated weights for policy 0, policy_version 179378 (0.0006) [2024-09-30 03:12:34,049][1157819] Updated weights for policy 0, policy_version 179388 (0.0006) [2024-09-30 03:12:34,559][1157819] Updated weights for policy 0, policy_version 179398 (0.0006) [2024-09-30 03:12:35,100][1157819] Updated weights for policy 0, policy_version 179408 (0.0006) [2024-09-30 03:12:35,466][1157520] Fps is (10 sec: 79461.9, 60 sec: 74137.4, 300 sec: 76421.6). Total num frames: 734883840. Throughput: 0: 18533.4. Samples: 173707704. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 03:12:35,466][1157520] Avg episode reward: [(0, '55.522')] [2024-09-30 03:12:35,620][1157819] Updated weights for policy 0, policy_version 179418 (0.0006) [2024-09-30 03:12:36,134][1157819] Updated weights for policy 0, policy_version 179428 (0.0006) [2024-09-30 03:12:36,636][1157819] Updated weights for policy 0, policy_version 179438 (0.0006) [2024-09-30 03:12:37,189][1157819] Updated weights for policy 0, policy_version 179448 (0.0006) [2024-09-30 03:12:37,714][1157819] Updated weights for policy 0, policy_version 179458 (0.0006) [2024-09-30 03:12:38,242][1157819] Updated weights for policy 0, policy_version 179468 (0.0006) [2024-09-30 03:12:38,763][1157819] Updated weights for policy 0, policy_version 179478 (0.0006) [2024-09-30 03:12:39,288][1157819] Updated weights for policy 0, policy_version 179488 (0.0006) [2024-09-30 03:12:39,816][1157819] Updated weights for policy 0, policy_version 179498 (0.0006) [2024-09-30 03:12:40,352][1157819] Updated weights for policy 0, policy_version 179508 (0.0006) [2024-09-30 03:12:40,466][1157520] Fps is (10 sec: 78232.8, 60 sec: 74342.5, 300 sec: 76449.4). Total num frames: 735272960. Throughput: 0: 18593.4. Samples: 173766448. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 03:12:40,466][1157520] Avg episode reward: [(0, '56.266')] [2024-09-30 03:12:40,856][1157819] Updated weights for policy 0, policy_version 179518 (0.0006) [2024-09-30 03:12:41,401][1157819] Updated weights for policy 0, policy_version 179528 (0.0006) [2024-09-30 03:12:41,921][1157819] Updated weights for policy 0, policy_version 179538 (0.0006) [2024-09-30 03:12:42,414][1157819] Updated weights for policy 0, policy_version 179548 (0.0006) [2024-09-30 03:12:42,907][1157819] Updated weights for policy 0, policy_version 179558 (0.0006) [2024-09-30 03:12:43,446][1157819] Updated weights for policy 0, policy_version 179568 (0.0006) [2024-09-30 03:12:44,011][1157819] Updated weights for policy 0, policy_version 179578 (0.0006) [2024-09-30 03:12:44,502][1157819] Updated weights for policy 0, policy_version 179588 (0.0006) [2024-09-30 03:12:45,006][1157819] Updated weights for policy 0, policy_version 179598 (0.0006) [2024-09-30 03:12:45,466][1157520] Fps is (10 sec: 78234.0, 60 sec: 74888.6, 300 sec: 76463.3). Total num frames: 735666176. Throughput: 0: 18731.3. Samples: 173884084. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 03:12:45,466][1157520] Avg episode reward: [(0, '55.787')] [2024-09-30 03:12:45,525][1157819] Updated weights for policy 0, policy_version 179608 (0.0006) [2024-09-30 03:12:46,033][1157819] Updated weights for policy 0, policy_version 179618 (0.0006) [2024-09-30 03:12:46,557][1157819] Updated weights for policy 0, policy_version 179628 (0.0006) [2024-09-30 03:12:47,092][1157819] Updated weights for policy 0, policy_version 179638 (0.0006) [2024-09-30 03:12:47,587][1157819] Updated weights for policy 0, policy_version 179648 (0.0006) [2024-09-30 03:12:48,118][1157819] Updated weights for policy 0, policy_version 179658 (0.0006) [2024-09-30 03:12:48,671][1157819] Updated weights for policy 0, policy_version 179668 (0.0006) [2024-09-30 03:12:49,186][1157819] Updated weights for policy 0, policy_version 179678 (0.0006) [2024-09-30 03:12:49,686][1157819] Updated weights for policy 0, policy_version 179688 (0.0006) [2024-09-30 03:12:50,176][1157819] Updated weights for policy 0, policy_version 179698 (0.0006) [2024-09-30 03:12:50,466][1157520] Fps is (10 sec: 79052.2, 60 sec: 75297.9, 300 sec: 76546.6). Total num frames: 736063488. Throughput: 0: 18821.4. Samples: 174002436. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 03:12:50,466][1157520] Avg episode reward: [(0, '57.154')] [2024-09-30 03:12:50,711][1157819] Updated weights for policy 0, policy_version 179708 (0.0006) [2024-09-30 03:12:51,231][1157819] Updated weights for policy 0, policy_version 179718 (0.0006) [2024-09-30 03:12:51,759][1157819] Updated weights for policy 0, policy_version 179728 (0.0006) [2024-09-30 03:12:52,277][1157819] Updated weights for policy 0, policy_version 179738 (0.0006) [2024-09-30 03:12:52,787][1157819] Updated weights for policy 0, policy_version 179748 (0.0006) [2024-09-30 03:12:53,314][1157819] Updated weights for policy 0, policy_version 179758 (0.0006) [2024-09-30 03:12:53,845][1157819] Updated weights for policy 0, policy_version 179768 (0.0006) [2024-09-30 03:12:54,362][1157819] Updated weights for policy 0, policy_version 179778 (0.0006) [2024-09-30 03:12:54,851][1157819] Updated weights for policy 0, policy_version 179788 (0.0006) [2024-09-30 03:12:55,378][1157819] Updated weights for policy 0, policy_version 179798 (0.0006) [2024-09-30 03:12:55,466][1157520] Fps is (10 sec: 79053.2, 60 sec: 75639.4, 300 sec: 76685.4). Total num frames: 736456704. Throughput: 0: 18912.0. Samples: 174061668. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 03:12:55,466][1157520] Avg episode reward: [(0, '56.691')] [2024-09-30 03:12:55,926][1157819] Updated weights for policy 0, policy_version 179808 (0.0006) [2024-09-30 03:12:56,445][1157819] Updated weights for policy 0, policy_version 179818 (0.0006) [2024-09-30 03:12:56,947][1157819] Updated weights for policy 0, policy_version 179828 (0.0006) [2024-09-30 03:12:57,466][1157819] Updated weights for policy 0, policy_version 179838 (0.0006) [2024-09-30 03:12:58,000][1157819] Updated weights for policy 0, policy_version 179848 (0.0006) [2024-09-30 03:12:58,490][1157819] Updated weights for policy 0, policy_version 179858 (0.0006) [2024-09-30 03:12:59,014][1157819] Updated weights for policy 0, policy_version 179868 (0.0006) [2024-09-30 03:12:59,560][1157819] Updated weights for policy 0, policy_version 179878 (0.0006) [2024-09-30 03:13:00,057][1157819] Updated weights for policy 0, policy_version 179888 (0.0006) [2024-09-30 03:13:00,466][1157520] Fps is (10 sec: 79053.9, 60 sec: 76117.3, 300 sec: 76866.0). Total num frames: 736854016. Throughput: 0: 19163.2. Samples: 174180320. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 03:13:00,466][1157520] Avg episode reward: [(0, '55.293')] [2024-09-30 03:13:00,557][1157819] Updated weights for policy 0, policy_version 179898 (0.0006) [2024-09-30 03:13:01,069][1157819] Updated weights for policy 0, policy_version 179908 (0.0006) [2024-09-30 03:13:01,589][1157819] Updated weights for policy 0, policy_version 179918 (0.0006) [2024-09-30 03:13:02,082][1157819] Updated weights for policy 0, policy_version 179928 (0.0006) [2024-09-30 03:13:02,603][1157819] Updated weights for policy 0, policy_version 179938 (0.0006) [2024-09-30 03:13:03,164][1157819] Updated weights for policy 0, policy_version 179948 (0.0006) [2024-09-30 03:13:03,680][1157819] Updated weights for policy 0, policy_version 179958 (0.0006) [2024-09-30 03:13:04,200][1157819] Updated weights for policy 0, policy_version 179968 (0.0006) [2024-09-30 03:13:04,718][1157819] Updated weights for policy 0, policy_version 179978 (0.0006) [2024-09-30 03:13:05,252][1157819] Updated weights for policy 0, policy_version 179988 (0.0006) [2024-09-30 03:13:05,466][1157520] Fps is (10 sec: 79052.6, 60 sec: 76253.8, 300 sec: 77032.5). Total num frames: 737247232. Throughput: 0: 19513.5. Samples: 174298668. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 03:13:05,466][1157520] Avg episode reward: [(0, '53.902')] [2024-09-30 03:13:05,753][1157819] Updated weights for policy 0, policy_version 179998 (0.0006) [2024-09-30 03:13:06,305][1157819] Updated weights for policy 0, policy_version 180008 (0.0006) [2024-09-30 03:13:06,844][1157819] Updated weights for policy 0, policy_version 180018 (0.0006) [2024-09-30 03:13:07,350][1157819] Updated weights for policy 0, policy_version 180028 (0.0006) [2024-09-30 03:13:07,876][1157819] Updated weights for policy 0, policy_version 180038 (0.0006) [2024-09-30 03:13:08,388][1157819] Updated weights for policy 0, policy_version 180048 (0.0006) [2024-09-30 03:13:08,893][1157819] Updated weights for policy 0, policy_version 180058 (0.0006) [2024-09-30 03:13:09,408][1157819] Updated weights for policy 0, policy_version 180068 (0.0006) [2024-09-30 03:13:09,967][1157819] Updated weights for policy 0, policy_version 180078 (0.0006) [2024-09-30 03:13:10,464][1157819] Updated weights for policy 0, policy_version 180088 (0.0006) [2024-09-30 03:13:10,466][1157520] Fps is (10 sec: 78644.1, 60 sec: 76663.5, 300 sec: 77199.2). Total num frames: 737640448. Throughput: 0: 19657.0. Samples: 174357292. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 03:13:10,466][1157520] Avg episode reward: [(0, '53.222')] [2024-09-30 03:13:10,947][1157819] Updated weights for policy 0, policy_version 180098 (0.0006) [2024-09-30 03:13:11,458][1157819] Updated weights for policy 0, policy_version 180108 (0.0006) [2024-09-30 03:13:11,986][1157819] Updated weights for policy 0, policy_version 180118 (0.0006) [2024-09-30 03:13:12,479][1157819] Updated weights for policy 0, policy_version 180128 (0.0006) [2024-09-30 03:13:12,995][1157819] Updated weights for policy 0, policy_version 180138 (0.0006) [2024-09-30 03:13:13,538][1157819] Updated weights for policy 0, policy_version 180148 (0.0006) [2024-09-30 03:13:14,034][1157819] Updated weights for policy 0, policy_version 180158 (0.0006) [2024-09-30 03:13:14,546][1157819] Updated weights for policy 0, policy_version 180168 (0.0006) [2024-09-30 03:13:15,101][1157819] Updated weights for policy 0, policy_version 180178 (0.0006) [2024-09-30 03:13:15,466][1157520] Fps is (10 sec: 78643.5, 60 sec: 77482.7, 300 sec: 77324.1). Total num frames: 738033664. Throughput: 0: 19720.2. Samples: 174476620. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 03:13:15,466][1157520] Avg episode reward: [(0, '54.375')] [2024-09-30 03:13:15,653][1157819] Updated weights for policy 0, policy_version 180188 (0.0006) [2024-09-30 03:13:16,237][1157819] Updated weights for policy 0, policy_version 180198 (0.0006) [2024-09-30 03:13:16,794][1157819] Updated weights for policy 0, policy_version 180208 (0.0006) [2024-09-30 03:13:17,421][1157819] Updated weights for policy 0, policy_version 180218 (0.0006) [2024-09-30 03:13:18,007][1157819] Updated weights for policy 0, policy_version 180228 (0.0006) [2024-09-30 03:13:18,314][1157736] Signal inference workers to stop experience collection... (12100 times) [2024-09-30 03:13:18,314][1157736] Signal inference workers to resume experience collection... (12100 times) [2024-09-30 03:13:18,318][1157819] InferenceWorker_p0-w0: stopping experience collection (12100 times) [2024-09-30 03:13:18,320][1157819] InferenceWorker_p0-w0: resuming experience collection (12100 times) [2024-09-30 03:13:18,565][1157819] Updated weights for policy 0, policy_version 180238 (0.0006) [2024-09-30 03:13:19,155][1157819] Updated weights for policy 0, policy_version 180248 (0.0006) [2024-09-30 03:13:19,765][1157819] Updated weights for policy 0, policy_version 180258 (0.0006) [2024-09-30 03:13:20,292][1157819] Updated weights for policy 0, policy_version 180268 (0.0006) [2024-09-30 03:13:20,466][1157520] Fps is (10 sec: 74955.9, 60 sec: 77755.7, 300 sec: 77254.7). Total num frames: 738390016. Throughput: 0: 19496.3. Samples: 174585036. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 03:13:20,466][1157520] Avg episode reward: [(0, '56.992')] [2024-09-30 03:13:20,899][1157819] Updated weights for policy 0, policy_version 180278 (0.0006) [2024-09-30 03:13:21,433][1157819] Updated weights for policy 0, policy_version 180288 (0.0006) [2024-09-30 03:13:22,024][1157819] Updated weights for policy 0, policy_version 180298 (0.0006) [2024-09-30 03:13:22,542][1157819] Updated weights for policy 0, policy_version 180308 (0.0006) [2024-09-30 03:13:23,163][1157819] Updated weights for policy 0, policy_version 180318 (0.0006) [2024-09-30 03:13:23,752][1157819] Updated weights for policy 0, policy_version 180328 (0.0006) [2024-09-30 03:13:24,337][1157819] Updated weights for policy 0, policy_version 180338 (0.0006) [2024-09-30 03:13:24,919][1157819] Updated weights for policy 0, policy_version 180348 (0.0006) [2024-09-30 03:13:25,466][1157520] Fps is (10 sec: 70860.9, 60 sec: 77551.0, 300 sec: 77060.3). Total num frames: 738742272. Throughput: 0: 19396.7. Samples: 174639296. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 03:13:25,466][1157520] Avg episode reward: [(0, '57.613')] [2024-09-30 03:13:25,506][1157819] Updated weights for policy 0, policy_version 180358 (0.0006) [2024-09-30 03:13:26,070][1157819] Updated weights for policy 0, policy_version 180368 (0.0006) [2024-09-30 03:13:26,676][1157819] Updated weights for policy 0, policy_version 180378 (0.0006) [2024-09-30 03:13:27,242][1157819] Updated weights for policy 0, policy_version 180388 (0.0006) [2024-09-30 03:13:27,844][1157819] Updated weights for policy 0, policy_version 180398 (0.0006) [2024-09-30 03:13:28,419][1157819] Updated weights for policy 0, policy_version 180408 (0.0006) [2024-09-30 03:13:29,008][1157819] Updated weights for policy 0, policy_version 180418 (0.0006) [2024-09-30 03:13:29,606][1157819] Updated weights for policy 0, policy_version 180428 (0.0006) [2024-09-30 03:13:30,185][1157819] Updated weights for policy 0, policy_version 180438 (0.0006) [2024-09-30 03:13:30,466][1157520] Fps is (10 sec: 70042.0, 60 sec: 76663.5, 300 sec: 76796.5). Total num frames: 739090432. Throughput: 0: 19108.3. Samples: 174743956. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 03:13:30,466][1157520] Avg episode reward: [(0, '56.943')] [2024-09-30 03:13:30,782][1157819] Updated weights for policy 0, policy_version 180448 (0.0006) [2024-09-30 03:13:31,364][1157819] Updated weights for policy 0, policy_version 180458 (0.0006) [2024-09-30 03:13:31,899][1157819] Updated weights for policy 0, policy_version 180468 (0.0006) [2024-09-30 03:13:32,477][1157819] Updated weights for policy 0, policy_version 180478 (0.0006) [2024-09-30 03:13:33,058][1157819] Updated weights for policy 0, policy_version 180488 (0.0006) [2024-09-30 03:13:33,625][1157819] Updated weights for policy 0, policy_version 180498 (0.0006) [2024-09-30 03:13:34,183][1157819] Updated weights for policy 0, policy_version 180508 (0.0006) [2024-09-30 03:13:34,753][1157819] Updated weights for policy 0, policy_version 180518 (0.0006) [2024-09-30 03:13:35,306][1157819] Updated weights for policy 0, policy_version 180528 (0.0006) [2024-09-30 03:13:35,466][1157520] Fps is (10 sec: 70860.4, 60 sec: 76117.4, 300 sec: 76602.1). Total num frames: 739450880. Throughput: 0: 18858.3. Samples: 174851060. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 03:13:35,466][1157520] Avg episode reward: [(0, '55.588')] [2024-09-30 03:13:35,875][1157819] Updated weights for policy 0, policy_version 180538 (0.0006) [2024-09-30 03:13:36,449][1157819] Updated weights for policy 0, policy_version 180548 (0.0006) [2024-09-30 03:13:37,020][1157819] Updated weights for policy 0, policy_version 180558 (0.0006) [2024-09-30 03:13:37,547][1157819] Updated weights for policy 0, policy_version 180568 (0.0006) [2024-09-30 03:13:38,063][1157819] Updated weights for policy 0, policy_version 180578 (0.0006) [2024-09-30 03:13:38,605][1157819] Updated weights for policy 0, policy_version 180588 (0.0006) [2024-09-30 03:13:39,087][1157819] Updated weights for policy 0, policy_version 180598 (0.0006) [2024-09-30 03:13:39,611][1157819] Updated weights for policy 0, policy_version 180608 (0.0006) [2024-09-30 03:13:40,149][1157819] Updated weights for policy 0, policy_version 180618 (0.0006) [2024-09-30 03:13:40,466][1157520] Fps is (10 sec: 74547.3, 60 sec: 76049.2, 300 sec: 76518.8). Total num frames: 739835904. Throughput: 0: 18766.6. Samples: 174906164. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 03:13:40,466][1157520] Avg episode reward: [(0, '54.125')] [2024-09-30 03:13:40,673][1157819] Updated weights for policy 0, policy_version 180628 (0.0006) [2024-09-30 03:13:41,171][1157819] Updated weights for policy 0, policy_version 180638 (0.0006) [2024-09-30 03:13:41,728][1157819] Updated weights for policy 0, policy_version 180648 (0.0006) [2024-09-30 03:13:42,249][1157819] Updated weights for policy 0, policy_version 180658 (0.0006) [2024-09-30 03:13:42,739][1157819] Updated weights for policy 0, policy_version 180668 (0.0006) [2024-09-30 03:13:42,874][1157736] Signal inference workers to stop experience collection... (12150 times) [2024-09-30 03:13:42,876][1157819] InferenceWorker_p0-w0: stopping experience collection (12150 times) [2024-09-30 03:13:42,883][1157736] Signal inference workers to resume experience collection... (12150 times) [2024-09-30 03:13:42,884][1157819] InferenceWorker_p0-w0: resuming experience collection (12150 times) [2024-09-30 03:13:43,291][1157819] Updated weights for policy 0, policy_version 180678 (0.0006) [2024-09-30 03:13:43,848][1157819] Updated weights for policy 0, policy_version 180688 (0.0006) [2024-09-30 03:13:44,394][1157819] Updated weights for policy 0, policy_version 180698 (0.0006) [2024-09-30 03:13:44,951][1157819] Updated weights for policy 0, policy_version 180708 (0.0006) [2024-09-30 03:13:45,466][1157520] Fps is (10 sec: 76595.5, 60 sec: 75844.3, 300 sec: 76449.4). Total num frames: 740216832. Throughput: 0: 18739.0. Samples: 175023576. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 03:13:45,466][1157520] Avg episode reward: [(0, '55.914')] [2024-09-30 03:13:45,492][1157819] Updated weights for policy 0, policy_version 180718 (0.0006) [2024-09-30 03:13:46,031][1157819] Updated weights for policy 0, policy_version 180728 (0.0006) [2024-09-30 03:13:46,575][1157819] Updated weights for policy 0, policy_version 180738 (0.0006) [2024-09-30 03:13:47,134][1157819] Updated weights for policy 0, policy_version 180748 (0.0006) [2024-09-30 03:13:47,706][1157819] Updated weights for policy 0, policy_version 180758 (0.0006) [2024-09-30 03:13:48,266][1157819] Updated weights for policy 0, policy_version 180768 (0.0006) [2024-09-30 03:13:48,790][1157819] Updated weights for policy 0, policy_version 180778 (0.0006) [2024-09-30 03:13:49,344][1157819] Updated weights for policy 0, policy_version 180788 (0.0006) [2024-09-30 03:13:49,928][1157819] Updated weights for policy 0, policy_version 180798 (0.0006) [2024-09-30 03:13:50,452][1157819] Updated weights for policy 0, policy_version 180808 (0.0006) [2024-09-30 03:13:50,466][1157520] Fps is (10 sec: 75366.0, 60 sec: 75434.8, 300 sec: 76366.1). Total num frames: 740589568. Throughput: 0: 18587.8. Samples: 175135120. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 03:13:50,466][1157520] Avg episode reward: [(0, '56.554')] [2024-09-30 03:13:50,995][1157819] Updated weights for policy 0, policy_version 180818 (0.0006) [2024-09-30 03:13:51,505][1157819] Updated weights for policy 0, policy_version 180828 (0.0006) [2024-09-30 03:13:52,032][1157819] Updated weights for policy 0, policy_version 180838 (0.0006) [2024-09-30 03:13:52,583][1157819] Updated weights for policy 0, policy_version 180848 (0.0006) [2024-09-30 03:13:53,094][1157819] Updated weights for policy 0, policy_version 180858 (0.0006) [2024-09-30 03:13:53,602][1157819] Updated weights for policy 0, policy_version 180868 (0.0006) [2024-09-30 03:13:54,129][1157819] Updated weights for policy 0, policy_version 180878 (0.0006) [2024-09-30 03:13:54,704][1157819] Updated weights for policy 0, policy_version 180888 (0.0006) [2024-09-30 03:13:55,242][1157819] Updated weights for policy 0, policy_version 180898 (0.0006) [2024-09-30 03:13:55,466][1157520] Fps is (10 sec: 75776.3, 60 sec: 75298.2, 300 sec: 76310.6). Total num frames: 740974592. Throughput: 0: 18571.3. Samples: 175193000. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 03:13:55,466][1157520] Avg episode reward: [(0, '54.985')] [2024-09-30 03:13:55,770][1157819] Updated weights for policy 0, policy_version 180908 (0.0006) [2024-09-30 03:13:56,329][1157819] Updated weights for policy 0, policy_version 180918 (0.0006) [2024-09-30 03:13:56,881][1157819] Updated weights for policy 0, policy_version 180928 (0.0006) [2024-09-30 03:13:57,393][1157819] Updated weights for policy 0, policy_version 180938 (0.0006) [2024-09-30 03:13:57,911][1157819] Updated weights for policy 0, policy_version 180948 (0.0006) [2024-09-30 03:13:58,428][1157819] Updated weights for policy 0, policy_version 180958 (0.0006) [2024-09-30 03:13:58,956][1157819] Updated weights for policy 0, policy_version 180968 (0.0006) [2024-09-30 03:13:59,493][1157819] Updated weights for policy 0, policy_version 180978 (0.0006) [2024-09-30 03:13:59,998][1157819] Updated weights for policy 0, policy_version 180988 (0.0006) [2024-09-30 03:14:00,466][1157520] Fps is (10 sec: 77005.6, 60 sec: 75093.5, 300 sec: 76255.1). Total num frames: 741359616. Throughput: 0: 18472.9. Samples: 175307900. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 03:14:00,466][1157520] Avg episode reward: [(0, '55.371')] [2024-09-30 03:14:00,556][1157819] Updated weights for policy 0, policy_version 180998 (0.0006) [2024-09-30 03:14:01,108][1157819] Updated weights for policy 0, policy_version 181008 (0.0006) [2024-09-30 03:14:01,591][1157819] Updated weights for policy 0, policy_version 181018 (0.0006) [2024-09-30 03:14:02,142][1157819] Updated weights for policy 0, policy_version 181028 (0.0006) [2024-09-30 03:14:02,652][1157819] Updated weights for policy 0, policy_version 181038 (0.0006) [2024-09-30 03:14:03,161][1157819] Updated weights for policy 0, policy_version 181048 (0.0006) [2024-09-30 03:14:03,735][1157819] Updated weights for policy 0, policy_version 181058 (0.0006) [2024-09-30 03:14:04,236][1157819] Updated weights for policy 0, policy_version 181068 (0.0006) [2024-09-30 03:14:04,739][1157819] Updated weights for policy 0, policy_version 181078 (0.0006) [2024-09-30 03:14:05,277][1157819] Updated weights for policy 0, policy_version 181088 (0.0006) [2024-09-30 03:14:05,466][1157520] Fps is (10 sec: 77414.7, 60 sec: 75025.2, 300 sec: 76268.9). Total num frames: 741748736. Throughput: 0: 18657.7. Samples: 175424632. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 03:14:05,466][1157520] Avg episode reward: [(0, '55.518')] [2024-09-30 03:14:05,469][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000181091_741748736.pth... [2024-09-30 03:14:05,539][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000176669_723636224.pth [2024-09-30 03:14:05,859][1157819] Updated weights for policy 0, policy_version 181098 (0.0006) [2024-09-30 03:14:06,397][1157819] Updated weights for policy 0, policy_version 181108 (0.0006) [2024-09-30 03:14:06,945][1157819] Updated weights for policy 0, policy_version 181118 (0.0006) [2024-09-30 03:14:07,525][1157819] Updated weights for policy 0, policy_version 181128 (0.0006) [2024-09-30 03:14:08,076][1157819] Updated weights for policy 0, policy_version 181138 (0.0006) [2024-09-30 03:14:08,579][1157819] Updated weights for policy 0, policy_version 181148 (0.0006) [2024-09-30 03:14:09,143][1157819] Updated weights for policy 0, policy_version 181158 (0.0006) [2024-09-30 03:14:09,702][1157819] Updated weights for policy 0, policy_version 181168 (0.0006) [2024-09-30 03:14:10,082][1157736] Signal inference workers to stop experience collection... (12200 times) [2024-09-30 03:14:10,084][1157736] Signal inference workers to resume experience collection... (12200 times) [2024-09-30 03:14:10,084][1157819] InferenceWorker_p0-w0: stopping experience collection (12200 times) [2024-09-30 03:14:10,088][1157819] InferenceWorker_p0-w0: resuming experience collection (12200 times) [2024-09-30 03:14:10,279][1157819] Updated weights for policy 0, policy_version 181178 (0.0006) [2024-09-30 03:14:10,466][1157520] Fps is (10 sec: 76185.2, 60 sec: 74683.7, 300 sec: 76144.0). Total num frames: 742121472. Throughput: 0: 18689.0. Samples: 175480300. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 03:14:10,466][1157520] Avg episode reward: [(0, '56.829')] [2024-09-30 03:14:10,832][1157819] Updated weights for policy 0, policy_version 181188 (0.0006) [2024-09-30 03:14:11,362][1157819] Updated weights for policy 0, policy_version 181198 (0.0006) [2024-09-30 03:14:11,887][1157819] Updated weights for policy 0, policy_version 181208 (0.0006) [2024-09-30 03:14:12,431][1157819] Updated weights for policy 0, policy_version 181218 (0.0006) [2024-09-30 03:14:12,974][1157819] Updated weights for policy 0, policy_version 181228 (0.0006) [2024-09-30 03:14:13,520][1157819] Updated weights for policy 0, policy_version 181238 (0.0006) [2024-09-30 03:14:14,069][1157819] Updated weights for policy 0, policy_version 181248 (0.0006) [2024-09-30 03:14:14,612][1157819] Updated weights for policy 0, policy_version 181258 (0.0006) [2024-09-30 03:14:15,150][1157819] Updated weights for policy 0, policy_version 181268 (0.0006) [2024-09-30 03:14:15,466][1157520] Fps is (10 sec: 74546.5, 60 sec: 74342.4, 300 sec: 76074.5). Total num frames: 742494208. Throughput: 0: 18860.9. Samples: 175592696. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 03:14:15,466][1157520] Avg episode reward: [(0, '54.750')] [2024-09-30 03:14:15,701][1157819] Updated weights for policy 0, policy_version 181278 (0.0006) [2024-09-30 03:14:16,240][1157819] Updated weights for policy 0, policy_version 181288 (0.0006) [2024-09-30 03:14:16,759][1157819] Updated weights for policy 0, policy_version 181298 (0.0006) [2024-09-30 03:14:17,313][1157819] Updated weights for policy 0, policy_version 181308 (0.0006) [2024-09-30 03:14:17,881][1157819] Updated weights for policy 0, policy_version 181318 (0.0006) [2024-09-30 03:14:18,455][1157819] Updated weights for policy 0, policy_version 181328 (0.0006) [2024-09-30 03:14:18,977][1157819] Updated weights for policy 0, policy_version 181338 (0.0006) [2024-09-30 03:14:19,512][1157819] Updated weights for policy 0, policy_version 181348 (0.0006) [2024-09-30 03:14:20,059][1157819] Updated weights for policy 0, policy_version 181358 (0.0006) [2024-09-30 03:14:20,466][1157520] Fps is (10 sec: 74546.6, 60 sec: 74615.4, 300 sec: 75949.6). Total num frames: 742866944. Throughput: 0: 18985.1. Samples: 175705388. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 03:14:20,466][1157520] Avg episode reward: [(0, '55.183')] [2024-09-30 03:14:20,585][1157819] Updated weights for policy 0, policy_version 181368 (0.0006) [2024-09-30 03:14:21,133][1157819] Updated weights for policy 0, policy_version 181378 (0.0006) [2024-09-30 03:14:21,671][1157819] Updated weights for policy 0, policy_version 181388 (0.0006) [2024-09-30 03:14:22,220][1157819] Updated weights for policy 0, policy_version 181398 (0.0006) [2024-09-30 03:14:22,773][1157819] Updated weights for policy 0, policy_version 181408 (0.0006) [2024-09-30 03:14:23,303][1157819] Updated weights for policy 0, policy_version 181418 (0.0006) [2024-09-30 03:14:23,830][1157819] Updated weights for policy 0, policy_version 181428 (0.0006) [2024-09-30 03:14:24,382][1157819] Updated weights for policy 0, policy_version 181438 (0.0006) [2024-09-30 03:14:24,924][1157819] Updated weights for policy 0, policy_version 181448 (0.0006) [2024-09-30 03:14:25,466][1157520] Fps is (10 sec: 75366.0, 60 sec: 75093.2, 300 sec: 75894.0). Total num frames: 743247872. Throughput: 0: 19023.3. Samples: 175762216. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 03:14:25,466][1157520] Avg episode reward: [(0, '56.525')] [2024-09-30 03:14:25,494][1157819] Updated weights for policy 0, policy_version 181458 (0.0006) [2024-09-30 03:14:26,014][1157819] Updated weights for policy 0, policy_version 181468 (0.0006) [2024-09-30 03:14:26,570][1157819] Updated weights for policy 0, policy_version 181478 (0.0006) [2024-09-30 03:14:27,108][1157819] Updated weights for policy 0, policy_version 181488 (0.0006) [2024-09-30 03:14:27,685][1157819] Updated weights for policy 0, policy_version 181498 (0.0006) [2024-09-30 03:14:28,215][1157819] Updated weights for policy 0, policy_version 181508 (0.0006) [2024-09-30 03:14:28,770][1157819] Updated weights for policy 0, policy_version 181518 (0.0006) [2024-09-30 03:14:29,289][1157819] Updated weights for policy 0, policy_version 181528 (0.0006) [2024-09-30 03:14:29,828][1157819] Updated weights for policy 0, policy_version 181538 (0.0006) [2024-09-30 03:14:30,354][1157819] Updated weights for policy 0, policy_version 181548 (0.0006) [2024-09-30 03:14:30,466][1157520] Fps is (10 sec: 75776.3, 60 sec: 75571.2, 300 sec: 75810.7). Total num frames: 743624704. Throughput: 0: 18928.6. Samples: 175875364. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 03:14:30,466][1157520] Avg episode reward: [(0, '55.947')] [2024-09-30 03:14:30,916][1157819] Updated weights for policy 0, policy_version 181558 (0.0006) [2024-09-30 03:14:31,464][1157819] Updated weights for policy 0, policy_version 181568 (0.0006) [2024-09-30 03:14:32,017][1157819] Updated weights for policy 0, policy_version 181578 (0.0006) [2024-09-30 03:14:32,580][1157819] Updated weights for policy 0, policy_version 181588 (0.0006) [2024-09-30 03:14:33,115][1157819] Updated weights for policy 0, policy_version 181598 (0.0006) [2024-09-30 03:14:33,652][1157819] Updated weights for policy 0, policy_version 181608 (0.0006) [2024-09-30 03:14:34,197][1157819] Updated weights for policy 0, policy_version 181618 (0.0006) [2024-09-30 03:14:34,756][1157819] Updated weights for policy 0, policy_version 181628 (0.0006) [2024-09-30 03:14:35,280][1157819] Updated weights for policy 0, policy_version 181638 (0.0006) [2024-09-30 03:14:35,466][1157520] Fps is (10 sec: 75367.2, 60 sec: 75844.4, 300 sec: 75699.7). Total num frames: 744001536. Throughput: 0: 18946.4. Samples: 175987708. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 03:14:35,466][1157520] Avg episode reward: [(0, '56.375')] [2024-09-30 03:14:35,844][1157819] Updated weights for policy 0, policy_version 181648 (0.0006) [2024-09-30 03:14:36,394][1157819] Updated weights for policy 0, policy_version 181658 (0.0006) [2024-09-30 03:14:36,948][1157819] Updated weights for policy 0, policy_version 181668 (0.0006) [2024-09-30 03:14:37,536][1157819] Updated weights for policy 0, policy_version 181678 (0.0006) [2024-09-30 03:14:38,132][1157819] Updated weights for policy 0, policy_version 181688 (0.0006) [2024-09-30 03:14:38,655][1157819] Updated weights for policy 0, policy_version 181698 (0.0006) [2024-09-30 03:14:39,198][1157819] Updated weights for policy 0, policy_version 181708 (0.0006) [2024-09-30 03:14:39,781][1157819] Updated weights for policy 0, policy_version 181718 (0.0006) [2024-09-30 03:14:40,352][1157819] Updated weights for policy 0, policy_version 181728 (0.0006) [2024-09-30 03:14:40,466][1157520] Fps is (10 sec: 74137.0, 60 sec: 75502.8, 300 sec: 75533.0). Total num frames: 744366080. Throughput: 0: 18887.0. Samples: 176042916. Policy #0 lag: (min: 0.0, avg: 2.4, max: 7.0) [2024-09-30 03:14:40,466][1157520] Avg episode reward: [(0, '54.735')] [2024-09-30 03:14:40,948][1157819] Updated weights for policy 0, policy_version 181738 (0.0006) [2024-09-30 03:14:41,530][1157819] Updated weights for policy 0, policy_version 181748 (0.0006) [2024-09-30 03:14:42,100][1157819] Updated weights for policy 0, policy_version 181758 (0.0006) [2024-09-30 03:14:42,698][1157819] Updated weights for policy 0, policy_version 181768 (0.0006) [2024-09-30 03:14:43,291][1157819] Updated weights for policy 0, policy_version 181778 (0.0006) [2024-09-30 03:14:43,925][1157819] Updated weights for policy 0, policy_version 181788 (0.0006) [2024-09-30 03:14:44,509][1157819] Updated weights for policy 0, policy_version 181798 (0.0006) [2024-09-30 03:14:45,086][1157819] Updated weights for policy 0, policy_version 181808 (0.0006) [2024-09-30 03:14:45,466][1157520] Fps is (10 sec: 71269.1, 60 sec: 74956.6, 300 sec: 75380.2). Total num frames: 744714240. Throughput: 0: 18696.7. Samples: 176149256. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 03:14:45,466][1157520] Avg episode reward: [(0, '54.502')] [2024-09-30 03:14:45,625][1157819] Updated weights for policy 0, policy_version 181818 (0.0006) [2024-09-30 03:14:46,120][1157819] Updated weights for policy 0, policy_version 181828 (0.0006) [2024-09-30 03:14:46,629][1157819] Updated weights for policy 0, policy_version 181838 (0.0006) [2024-09-30 03:14:47,144][1157819] Updated weights for policy 0, policy_version 181848 (0.0006) [2024-09-30 03:14:47,648][1157819] Updated weights for policy 0, policy_version 181858 (0.0006) [2024-09-30 03:14:48,170][1157819] Updated weights for policy 0, policy_version 181868 (0.0006) [2024-09-30 03:14:48,713][1157819] Updated weights for policy 0, policy_version 181878 (0.0006) [2024-09-30 03:14:49,229][1157819] Updated weights for policy 0, policy_version 181888 (0.0006) [2024-09-30 03:14:49,747][1157819] Updated weights for policy 0, policy_version 181898 (0.0006) [2024-09-30 03:14:50,121][1157736] Signal inference workers to stop experience collection... (12250 times) [2024-09-30 03:14:50,125][1157736] Signal inference workers to resume experience collection... (12250 times) [2024-09-30 03:14:50,126][1157819] InferenceWorker_p0-w0: stopping experience collection (12250 times) [2024-09-30 03:14:50,129][1157819] InferenceWorker_p0-w0: resuming experience collection (12250 times) [2024-09-30 03:14:50,325][1157819] Updated weights for policy 0, policy_version 181908 (0.0006) [2024-09-30 03:14:50,466][1157520] Fps is (10 sec: 74138.0, 60 sec: 75298.1, 300 sec: 75352.5). Total num frames: 745107456. Throughput: 0: 18652.6. Samples: 176264000. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 03:14:50,466][1157520] Avg episode reward: [(0, '56.846')] [2024-09-30 03:14:50,867][1157819] Updated weights for policy 0, policy_version 181918 (0.0006) [2024-09-30 03:14:51,387][1157819] Updated weights for policy 0, policy_version 181928 (0.0006) [2024-09-30 03:14:51,915][1157819] Updated weights for policy 0, policy_version 181938 (0.0006) [2024-09-30 03:14:52,464][1157819] Updated weights for policy 0, policy_version 181948 (0.0006) [2024-09-30 03:14:53,078][1157819] Updated weights for policy 0, policy_version 181958 (0.0006) [2024-09-30 03:14:53,623][1157819] Updated weights for policy 0, policy_version 181968 (0.0006) [2024-09-30 03:14:54,304][1157819] Updated weights for policy 0, policy_version 181978 (0.0006) [2024-09-30 03:14:54,984][1157819] Updated weights for policy 0, policy_version 181988 (0.0006) [2024-09-30 03:14:55,466][1157520] Fps is (10 sec: 73729.0, 60 sec: 74615.4, 300 sec: 75241.4). Total num frames: 745451520. Throughput: 0: 18660.0. Samples: 176320000. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 03:14:55,466][1157520] Avg episode reward: [(0, '56.487')] [2024-09-30 03:14:55,622][1157819] Updated weights for policy 0, policy_version 181998 (0.0006) [2024-09-30 03:14:56,227][1157819] Updated weights for policy 0, policy_version 182008 (0.0006) [2024-09-30 03:14:56,882][1157819] Updated weights for policy 0, policy_version 182018 (0.0006) [2024-09-30 03:14:57,510][1157819] Updated weights for policy 0, policy_version 182028 (0.0006) [2024-09-30 03:14:58,163][1157819] Updated weights for policy 0, policy_version 182038 (0.0006) [2024-09-30 03:14:58,834][1157819] Updated weights for policy 0, policy_version 182048 (0.0006) [2024-09-30 03:14:59,485][1157819] Updated weights for policy 0, policy_version 182058 (0.0006) [2024-09-30 03:15:00,132][1157819] Updated weights for policy 0, policy_version 182068 (0.0006) [2024-09-30 03:15:00,466][1157520] Fps is (10 sec: 65945.6, 60 sec: 73454.8, 300 sec: 75033.2). Total num frames: 745766912. Throughput: 0: 18311.6. Samples: 176416716. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 03:15:00,466][1157520] Avg episode reward: [(0, '55.155')] [2024-09-30 03:15:00,761][1157819] Updated weights for policy 0, policy_version 182078 (0.0006) [2024-09-30 03:15:01,292][1157819] Updated weights for policy 0, policy_version 182088 (0.0006) [2024-09-30 03:15:01,809][1157819] Updated weights for policy 0, policy_version 182098 (0.0006) [2024-09-30 03:15:02,315][1157819] Updated weights for policy 0, policy_version 182108 (0.0006) [2024-09-30 03:15:02,845][1157819] Updated weights for policy 0, policy_version 182118 (0.0006) [2024-09-30 03:15:03,340][1157819] Updated weights for policy 0, policy_version 182128 (0.0006) [2024-09-30 03:15:03,893][1157819] Updated weights for policy 0, policy_version 182138 (0.0006) [2024-09-30 03:15:04,408][1157819] Updated weights for policy 0, policy_version 182148 (0.0006) [2024-09-30 03:15:04,914][1157819] Updated weights for policy 0, policy_version 182158 (0.0006) [2024-09-30 03:15:05,423][1157819] Updated weights for policy 0, policy_version 182168 (0.0006) [2024-09-30 03:15:05,466][1157520] Fps is (10 sec: 71270.8, 60 sec: 73591.4, 300 sec: 75047.1). Total num frames: 746164224. Throughput: 0: 18271.4. Samples: 176527600. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 03:15:05,466][1157520] Avg episode reward: [(0, '54.776')] [2024-09-30 03:15:05,937][1157819] Updated weights for policy 0, policy_version 182178 (0.0006) [2024-09-30 03:15:06,469][1157819] Updated weights for policy 0, policy_version 182188 (0.0006) [2024-09-30 03:15:06,979][1157819] Updated weights for policy 0, policy_version 182198 (0.0006) [2024-09-30 03:15:07,497][1157819] Updated weights for policy 0, policy_version 182208 (0.0006) [2024-09-30 03:15:08,026][1157819] Updated weights for policy 0, policy_version 182218 (0.0006) [2024-09-30 03:15:08,564][1157819] Updated weights for policy 0, policy_version 182228 (0.0006) [2024-09-30 03:15:09,070][1157819] Updated weights for policy 0, policy_version 182238 (0.0006) [2024-09-30 03:15:09,592][1157819] Updated weights for policy 0, policy_version 182248 (0.0006) [2024-09-30 03:15:10,156][1157819] Updated weights for policy 0, policy_version 182258 (0.0006) [2024-09-30 03:15:10,466][1157520] Fps is (10 sec: 78644.1, 60 sec: 73864.6, 300 sec: 75019.3). Total num frames: 746553344. Throughput: 0: 18321.0. Samples: 176586656. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 03:15:10,466][1157520] Avg episode reward: [(0, '54.221')] [2024-09-30 03:15:10,698][1157819] Updated weights for policy 0, policy_version 182268 (0.0006) [2024-09-30 03:15:11,293][1157819] Updated weights for policy 0, policy_version 182278 (0.0006) [2024-09-30 03:15:11,902][1157819] Updated weights for policy 0, policy_version 182288 (0.0006) [2024-09-30 03:15:11,926][1157736] Signal inference workers to stop experience collection... (12300 times) [2024-09-30 03:15:11,929][1157819] InferenceWorker_p0-w0: stopping experience collection (12300 times) [2024-09-30 03:15:11,935][1157736] Signal inference workers to resume experience collection... (12300 times) [2024-09-30 03:15:11,935][1157819] InferenceWorker_p0-w0: resuming experience collection (12300 times) [2024-09-30 03:15:12,436][1157819] Updated weights for policy 0, policy_version 182298 (0.0006) [2024-09-30 03:15:13,015][1157819] Updated weights for policy 0, policy_version 182308 (0.0006) [2024-09-30 03:15:13,595][1157819] Updated weights for policy 0, policy_version 182318 (0.0006) [2024-09-30 03:15:14,159][1157819] Updated weights for policy 0, policy_version 182328 (0.0006) [2024-09-30 03:15:14,725][1157819] Updated weights for policy 0, policy_version 182338 (0.0006) [2024-09-30 03:15:15,301][1157819] Updated weights for policy 0, policy_version 182348 (0.0006) [2024-09-30 03:15:15,466][1157520] Fps is (10 sec: 74138.0, 60 sec: 73523.3, 300 sec: 74908.2). Total num frames: 746905600. Throughput: 0: 18264.2. Samples: 176697252. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 03:15:15,466][1157520] Avg episode reward: [(0, '57.355')] [2024-09-30 03:15:15,874][1157819] Updated weights for policy 0, policy_version 182358 (0.0006) [2024-09-30 03:15:16,468][1157819] Updated weights for policy 0, policy_version 182368 (0.0006) [2024-09-30 03:15:17,009][1157819] Updated weights for policy 0, policy_version 182378 (0.0006) [2024-09-30 03:15:17,550][1157819] Updated weights for policy 0, policy_version 182388 (0.0006) [2024-09-30 03:15:18,113][1157819] Updated weights for policy 0, policy_version 182398 (0.0006) [2024-09-30 03:15:18,717][1157819] Updated weights for policy 0, policy_version 182408 (0.0006) [2024-09-30 03:15:19,247][1157819] Updated weights for policy 0, policy_version 182418 (0.0006) [2024-09-30 03:15:19,771][1157819] Updated weights for policy 0, policy_version 182428 (0.0006) [2024-09-30 03:15:20,390][1157819] Updated weights for policy 0, policy_version 182438 (0.0006) [2024-09-30 03:15:20,466][1157520] Fps is (10 sec: 71679.9, 60 sec: 73386.8, 300 sec: 74811.1). Total num frames: 747270144. Throughput: 0: 18183.7. Samples: 176805972. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 03:15:20,466][1157520] Avg episode reward: [(0, '56.950')] [2024-09-30 03:15:20,930][1157819] Updated weights for policy 0, policy_version 182448 (0.0006) [2024-09-30 03:15:21,469][1157819] Updated weights for policy 0, policy_version 182458 (0.0006) [2024-09-30 03:15:22,034][1157819] Updated weights for policy 0, policy_version 182468 (0.0006) [2024-09-30 03:15:22,595][1157819] Updated weights for policy 0, policy_version 182478 (0.0006) [2024-09-30 03:15:23,122][1157819] Updated weights for policy 0, policy_version 182488 (0.0006) [2024-09-30 03:15:23,652][1157819] Updated weights for policy 0, policy_version 182498 (0.0006) [2024-09-30 03:15:24,211][1157819] Updated weights for policy 0, policy_version 182508 (0.0006) [2024-09-30 03:15:24,719][1157819] Updated weights for policy 0, policy_version 182518 (0.0006) [2024-09-30 03:15:25,255][1157819] Updated weights for policy 0, policy_version 182528 (0.0006) [2024-09-30 03:15:25,466][1157520] Fps is (10 sec: 74137.6, 60 sec: 73318.6, 300 sec: 74755.5). Total num frames: 747646976. Throughput: 0: 18185.7. Samples: 176861268. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 03:15:25,466][1157520] Avg episode reward: [(0, '56.187')] [2024-09-30 03:15:25,778][1157819] Updated weights for policy 0, policy_version 182538 (0.0006) [2024-09-30 03:15:26,324][1157819] Updated weights for policy 0, policy_version 182548 (0.0006) [2024-09-30 03:15:26,831][1157819] Updated weights for policy 0, policy_version 182558 (0.0006) [2024-09-30 03:15:27,361][1157819] Updated weights for policy 0, policy_version 182568 (0.0006) [2024-09-30 03:15:27,895][1157819] Updated weights for policy 0, policy_version 182578 (0.0006) [2024-09-30 03:15:28,400][1157819] Updated weights for policy 0, policy_version 182588 (0.0006) [2024-09-30 03:15:28,891][1157819] Updated weights for policy 0, policy_version 182598 (0.0006) [2024-09-30 03:15:29,359][1157819] Updated weights for policy 0, policy_version 182608 (0.0006) [2024-09-30 03:15:29,849][1157819] Updated weights for policy 0, policy_version 182618 (0.0006) [2024-09-30 03:15:30,320][1157819] Updated weights for policy 0, policy_version 182628 (0.0006) [2024-09-30 03:15:30,466][1157520] Fps is (10 sec: 78233.5, 60 sec: 73796.4, 300 sec: 74824.9). Total num frames: 748052480. Throughput: 0: 18424.5. Samples: 176978356. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 03:15:30,466][1157520] Avg episode reward: [(0, '55.850')] [2024-09-30 03:15:30,822][1157819] Updated weights for policy 0, policy_version 182638 (0.0006) [2024-09-30 03:15:31,282][1157819] Updated weights for policy 0, policy_version 182648 (0.0006) [2024-09-30 03:15:31,801][1157819] Updated weights for policy 0, policy_version 182658 (0.0006) [2024-09-30 03:15:32,238][1157819] Updated weights for policy 0, policy_version 182668 (0.0006) [2024-09-30 03:15:32,806][1157819] Updated weights for policy 0, policy_version 182678 (0.0006) [2024-09-30 03:15:33,331][1157819] Updated weights for policy 0, policy_version 182688 (0.0006) [2024-09-30 03:15:33,844][1157819] Updated weights for policy 0, policy_version 182698 (0.0006) [2024-09-30 03:15:34,366][1157819] Updated weights for policy 0, policy_version 182708 (0.0006) [2024-09-30 03:15:34,874][1157819] Updated weights for policy 0, policy_version 182718 (0.0006) [2024-09-30 03:15:35,393][1157819] Updated weights for policy 0, policy_version 182728 (0.0006) [2024-09-30 03:15:35,466][1157520] Fps is (10 sec: 81100.8, 60 sec: 74274.2, 300 sec: 74949.9). Total num frames: 748457984. Throughput: 0: 18612.4. Samples: 177101556. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 03:15:35,466][1157520] Avg episode reward: [(0, '55.206')] [2024-09-30 03:15:35,937][1157819] Updated weights for policy 0, policy_version 182738 (0.0006) [2024-09-30 03:15:36,449][1157819] Updated weights for policy 0, policy_version 182748 (0.0006) [2024-09-30 03:15:37,014][1157819] Updated weights for policy 0, policy_version 182758 (0.0006) [2024-09-30 03:15:37,535][1157819] Updated weights for policy 0, policy_version 182768 (0.0006) [2024-09-30 03:15:37,837][1157736] Signal inference workers to stop experience collection... (12350 times) [2024-09-30 03:15:37,838][1157736] Signal inference workers to resume experience collection... (12350 times) [2024-09-30 03:15:37,841][1157819] InferenceWorker_p0-w0: stopping experience collection (12350 times) [2024-09-30 03:15:37,841][1157819] InferenceWorker_p0-w0: resuming experience collection (12350 times) [2024-09-30 03:15:38,065][1157819] Updated weights for policy 0, policy_version 182778 (0.0006) [2024-09-30 03:15:38,623][1157819] Updated weights for policy 0, policy_version 182788 (0.0006) [2024-09-30 03:15:39,156][1157819] Updated weights for policy 0, policy_version 182798 (0.0006) [2024-09-30 03:15:39,710][1157819] Updated weights for policy 0, policy_version 182808 (0.0006) [2024-09-30 03:15:40,259][1157819] Updated weights for policy 0, policy_version 182818 (0.0006) [2024-09-30 03:15:40,466][1157520] Fps is (10 sec: 78232.5, 60 sec: 74478.9, 300 sec: 74977.6). Total num frames: 748834816. Throughput: 0: 18648.0. Samples: 177159160. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 03:15:40,466][1157520] Avg episode reward: [(0, '57.313')] [2024-09-30 03:15:40,807][1157819] Updated weights for policy 0, policy_version 182828 (0.0006) [2024-09-30 03:15:41,350][1157819] Updated weights for policy 0, policy_version 182838 (0.0006) [2024-09-30 03:15:41,872][1157819] Updated weights for policy 0, policy_version 182848 (0.0006) [2024-09-30 03:15:42,407][1157819] Updated weights for policy 0, policy_version 182858 (0.0006) [2024-09-30 03:15:42,916][1157819] Updated weights for policy 0, policy_version 182868 (0.0006) [2024-09-30 03:15:43,442][1157819] Updated weights for policy 0, policy_version 182878 (0.0006) [2024-09-30 03:15:44,001][1157819] Updated weights for policy 0, policy_version 182888 (0.0006) [2024-09-30 03:15:44,554][1157819] Updated weights for policy 0, policy_version 182898 (0.0006) [2024-09-30 03:15:45,083][1157819] Updated weights for policy 0, policy_version 182908 (0.0006) [2024-09-30 03:15:45,466][1157520] Fps is (10 sec: 75774.3, 60 sec: 75025.1, 300 sec: 74949.8). Total num frames: 749215744. Throughput: 0: 19033.8. Samples: 177273240. Policy #0 lag: (min: 0.0, avg: 1.9, max: 6.0) [2024-09-30 03:15:45,466][1157520] Avg episode reward: [(0, '55.613')] [2024-09-30 03:15:45,620][1157819] Updated weights for policy 0, policy_version 182918 (0.0006) [2024-09-30 03:15:46,163][1157819] Updated weights for policy 0, policy_version 182928 (0.0006) [2024-09-30 03:15:46,694][1157819] Updated weights for policy 0, policy_version 182938 (0.0006) [2024-09-30 03:15:47,239][1157819] Updated weights for policy 0, policy_version 182948 (0.0006) [2024-09-30 03:15:47,754][1157819] Updated weights for policy 0, policy_version 182958 (0.0006) [2024-09-30 03:15:48,285][1157819] Updated weights for policy 0, policy_version 182968 (0.0006) [2024-09-30 03:15:48,844][1157819] Updated weights for policy 0, policy_version 182978 (0.0006) [2024-09-30 03:15:49,389][1157819] Updated weights for policy 0, policy_version 182988 (0.0006) [2024-09-30 03:15:49,912][1157819] Updated weights for policy 0, policy_version 182998 (0.0006) [2024-09-30 03:15:50,456][1157819] Updated weights for policy 0, policy_version 183008 (0.0006) [2024-09-30 03:15:50,466][1157520] Fps is (10 sec: 76595.4, 60 sec: 74888.5, 300 sec: 74991.5). Total num frames: 749600768. Throughput: 0: 19108.9. Samples: 177387504. Policy #0 lag: (min: 0.0, avg: 2.5, max: 5.0) [2024-09-30 03:15:50,466][1157520] Avg episode reward: [(0, '53.614')] [2024-09-30 03:15:51,025][1157819] Updated weights for policy 0, policy_version 183018 (0.0006) [2024-09-30 03:15:51,551][1157819] Updated weights for policy 0, policy_version 183028 (0.0006) [2024-09-30 03:15:52,040][1157819] Updated weights for policy 0, policy_version 183038 (0.0006) [2024-09-30 03:15:52,566][1157819] Updated weights for policy 0, policy_version 183048 (0.0006) [2024-09-30 03:15:53,079][1157819] Updated weights for policy 0, policy_version 183058 (0.0006) [2024-09-30 03:15:53,574][1157819] Updated weights for policy 0, policy_version 183068 (0.0006) [2024-09-30 03:15:54,079][1157819] Updated weights for policy 0, policy_version 183078 (0.0006) [2024-09-30 03:15:54,591][1157819] Updated weights for policy 0, policy_version 183088 (0.0006) [2024-09-30 03:15:55,108][1157819] Updated weights for policy 0, policy_version 183098 (0.0006) [2024-09-30 03:15:55,466][1157520] Fps is (10 sec: 78234.7, 60 sec: 75776.0, 300 sec: 75074.8). Total num frames: 749998080. Throughput: 0: 19092.8. Samples: 177445832. Policy #0 lag: (min: 0.0, avg: 2.5, max: 5.0) [2024-09-30 03:15:55,466][1157520] Avg episode reward: [(0, '54.686')] [2024-09-30 03:15:55,594][1157819] Updated weights for policy 0, policy_version 183108 (0.0006) [2024-09-30 03:15:56,099][1157819] Updated weights for policy 0, policy_version 183118 (0.0006) [2024-09-30 03:15:56,608][1157819] Updated weights for policy 0, policy_version 183128 (0.0006) [2024-09-30 03:15:57,099][1157819] Updated weights for policy 0, policy_version 183138 (0.0006) [2024-09-30 03:15:57,618][1157819] Updated weights for policy 0, policy_version 183148 (0.0006) [2024-09-30 03:15:58,143][1157819] Updated weights for policy 0, policy_version 183158 (0.0006) [2024-09-30 03:15:58,656][1157819] Updated weights for policy 0, policy_version 183168 (0.0006) [2024-09-30 03:15:59,141][1157819] Updated weights for policy 0, policy_version 183178 (0.0006) [2024-09-30 03:15:59,669][1157819] Updated weights for policy 0, policy_version 183188 (0.0006) [2024-09-30 03:16:00,152][1157819] Updated weights for policy 0, policy_version 183198 (0.0006) [2024-09-30 03:16:00,466][1157520] Fps is (10 sec: 79872.5, 60 sec: 77209.7, 300 sec: 75255.4). Total num frames: 750399488. Throughput: 0: 19320.7. Samples: 177566684. Policy #0 lag: (min: 0.0, avg: 2.5, max: 5.0) [2024-09-30 03:16:00,466][1157520] Avg episode reward: [(0, '53.360')] [2024-09-30 03:16:00,680][1157819] Updated weights for policy 0, policy_version 183208 (0.0006) [2024-09-30 03:16:01,242][1157819] Updated weights for policy 0, policy_version 183218 (0.0006) [2024-09-30 03:16:01,841][1157819] Updated weights for policy 0, policy_version 183228 (0.0006) [2024-09-30 03:16:02,424][1157819] Updated weights for policy 0, policy_version 183238 (0.0006) [2024-09-30 03:16:02,983][1157819] Updated weights for policy 0, policy_version 183248 (0.0006) [2024-09-30 03:16:03,524][1157819] Updated weights for policy 0, policy_version 183258 (0.0006) [2024-09-30 03:16:04,058][1157819] Updated weights for policy 0, policy_version 183268 (0.0006) [2024-09-30 03:16:04,572][1157819] Updated weights for policy 0, policy_version 183278 (0.0006) [2024-09-30 03:16:05,106][1157819] Updated weights for policy 0, policy_version 183288 (0.0006) [2024-09-30 03:16:05,466][1157520] Fps is (10 sec: 77414.9, 60 sec: 76800.0, 300 sec: 75310.9). Total num frames: 750772224. Throughput: 0: 19433.1. Samples: 177680464. Policy #0 lag: (min: 0.0, avg: 2.5, max: 5.0) [2024-09-30 03:16:05,466][1157520] Avg episode reward: [(0, '56.892')] [2024-09-30 03:16:05,470][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000183294_750772224.pth... [2024-09-30 03:16:05,538][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000178875_732672000.pth [2024-09-30 03:16:05,697][1157819] Updated weights for policy 0, policy_version 183298 (0.0006) [2024-09-30 03:16:06,291][1157819] Updated weights for policy 0, policy_version 183308 (0.0006) [2024-09-30 03:16:06,871][1157819] Updated weights for policy 0, policy_version 183318 (0.0006) [2024-09-30 03:16:07,434][1157819] Updated weights for policy 0, policy_version 183328 (0.0006) [2024-09-30 03:16:08,024][1157819] Updated weights for policy 0, policy_version 183338 (0.0006) [2024-09-30 03:16:08,609][1157819] Updated weights for policy 0, policy_version 183348 (0.0006) [2024-09-30 03:16:09,150][1157819] Updated weights for policy 0, policy_version 183358 (0.0006) [2024-09-30 03:16:09,758][1157819] Updated weights for policy 0, policy_version 183368 (0.0006) [2024-09-30 03:16:10,278][1157819] Updated weights for policy 0, policy_version 183378 (0.0006) [2024-09-30 03:16:10,466][1157520] Fps is (10 sec: 72909.4, 60 sec: 76253.9, 300 sec: 75269.2). Total num frames: 751128576. Throughput: 0: 19391.4. Samples: 177733880. Policy #0 lag: (min: 0.0, avg: 2.5, max: 5.0) [2024-09-30 03:16:10,466][1157520] Avg episode reward: [(0, '55.034')] [2024-09-30 03:16:10,828][1157819] Updated weights for policy 0, policy_version 183388 (0.0006) [2024-09-30 03:16:11,385][1157819] Updated weights for policy 0, policy_version 183398 (0.0006) [2024-09-30 03:16:11,895][1157819] Updated weights for policy 0, policy_version 183408 (0.0006) [2024-09-30 03:16:12,424][1157819] Updated weights for policy 0, policy_version 183418 (0.0006) [2024-09-30 03:16:12,957][1157819] Updated weights for policy 0, policy_version 183428 (0.0006) [2024-09-30 03:16:13,488][1157819] Updated weights for policy 0, policy_version 183438 (0.0006) [2024-09-30 03:16:13,990][1157819] Updated weights for policy 0, policy_version 183448 (0.0006) [2024-09-30 03:16:14,537][1157819] Updated weights for policy 0, policy_version 183458 (0.0006) [2024-09-30 03:16:15,059][1157819] Updated weights for policy 0, policy_version 183468 (0.0006) [2024-09-30 03:16:15,466][1157520] Fps is (10 sec: 74137.7, 60 sec: 76800.0, 300 sec: 75297.0). Total num frames: 751513600. Throughput: 0: 19279.6. Samples: 177845940. Policy #0 lag: (min: 0.0, avg: 2.5, max: 5.0) [2024-09-30 03:16:15,466][1157520] Avg episode reward: [(0, '55.378')] [2024-09-30 03:16:15,589][1157819] Updated weights for policy 0, policy_version 183478 (0.0006) [2024-09-30 03:16:16,102][1157819] Updated weights for policy 0, policy_version 183488 (0.0006) [2024-09-30 03:16:16,601][1157819] Updated weights for policy 0, policy_version 183498 (0.0006) [2024-09-30 03:16:17,127][1157819] Updated weights for policy 0, policy_version 183508 (0.0006) [2024-09-30 03:16:17,643][1157819] Updated weights for policy 0, policy_version 183518 (0.0006) [2024-09-30 03:16:18,169][1157819] Updated weights for policy 0, policy_version 183528 (0.0006) [2024-09-30 03:16:18,713][1157819] Updated weights for policy 0, policy_version 183538 (0.0006) [2024-09-30 03:16:19,229][1157819] Updated weights for policy 0, policy_version 183548 (0.0006) [2024-09-30 03:16:19,742][1157819] Updated weights for policy 0, policy_version 183558 (0.0006) [2024-09-30 03:16:20,238][1157819] Updated weights for policy 0, policy_version 183568 (0.0006) [2024-09-30 03:16:20,466][1157520] Fps is (10 sec: 77824.1, 60 sec: 77277.9, 300 sec: 75324.8). Total num frames: 751906816. Throughput: 0: 19172.8. Samples: 177964332. Policy #0 lag: (min: 0.0, avg: 2.5, max: 5.0) [2024-09-30 03:16:20,466][1157520] Avg episode reward: [(0, '56.432')] [2024-09-30 03:16:20,804][1157819] Updated weights for policy 0, policy_version 183578 (0.0006) [2024-09-30 03:16:21,337][1157819] Updated weights for policy 0, policy_version 183588 (0.0006) [2024-09-30 03:16:21,854][1157819] Updated weights for policy 0, policy_version 183598 (0.0006) [2024-09-30 03:16:22,021][1157736] Signal inference workers to stop experience collection... (12400 times) [2024-09-30 03:16:22,021][1157736] Signal inference workers to resume experience collection... (12400 times) [2024-09-30 03:16:22,026][1157819] InferenceWorker_p0-w0: stopping experience collection (12400 times) [2024-09-30 03:16:22,027][1157819] InferenceWorker_p0-w0: resuming experience collection (12400 times) [2024-09-30 03:16:22,366][1157819] Updated weights for policy 0, policy_version 183608 (0.0006) [2024-09-30 03:16:23,066][1157819] Updated weights for policy 0, policy_version 183618 (0.0006) [2024-09-30 03:16:23,664][1157819] Updated weights for policy 0, policy_version 183628 (0.0006) [2024-09-30 03:16:24,295][1157819] Updated weights for policy 0, policy_version 183638 (0.0006) [2024-09-30 03:16:24,938][1157819] Updated weights for policy 0, policy_version 183648 (0.0006) [2024-09-30 03:16:25,466][1157520] Fps is (10 sec: 74136.9, 60 sec: 76799.9, 300 sec: 75213.7). Total num frames: 752254976. Throughput: 0: 19142.7. Samples: 178020580. Policy #0 lag: (min: 0.0, avg: 2.5, max: 5.0) [2024-09-30 03:16:25,466][1157520] Avg episode reward: [(0, '54.708')] [2024-09-30 03:16:25,582][1157819] Updated weights for policy 0, policy_version 183658 (0.0006) [2024-09-30 03:16:26,201][1157819] Updated weights for policy 0, policy_version 183668 (0.0006) [2024-09-30 03:16:26,843][1157819] Updated weights for policy 0, policy_version 183678 (0.0006) [2024-09-30 03:16:27,472][1157819] Updated weights for policy 0, policy_version 183688 (0.0006) [2024-09-30 03:16:28,074][1157819] Updated weights for policy 0, policy_version 183698 (0.0006) [2024-09-30 03:16:28,711][1157819] Updated weights for policy 0, policy_version 183708 (0.0006) [2024-09-30 03:16:29,293][1157819] Updated weights for policy 0, policy_version 183718 (0.0006) [2024-09-30 03:16:29,852][1157819] Updated weights for policy 0, policy_version 183728 (0.0006) [2024-09-30 03:16:30,408][1157819] Updated weights for policy 0, policy_version 183738 (0.0006) [2024-09-30 03:16:30,466][1157520] Fps is (10 sec: 68402.3, 60 sec: 75639.4, 300 sec: 75102.6). Total num frames: 752590848. Throughput: 0: 18775.0. Samples: 178118112. Policy #0 lag: (min: 0.0, avg: 2.5, max: 5.0) [2024-09-30 03:16:30,466][1157520] Avg episode reward: [(0, '54.673')] [2024-09-30 03:16:30,991][1157819] Updated weights for policy 0, policy_version 183748 (0.0006) [2024-09-30 03:16:31,524][1157819] Updated weights for policy 0, policy_version 183758 (0.0006) [2024-09-30 03:16:32,055][1157819] Updated weights for policy 0, policy_version 183768 (0.0006) [2024-09-30 03:16:32,608][1157819] Updated weights for policy 0, policy_version 183778 (0.0006) [2024-09-30 03:16:33,124][1157819] Updated weights for policy 0, policy_version 183788 (0.0006) [2024-09-30 03:16:33,704][1157819] Updated weights for policy 0, policy_version 183798 (0.0006) [2024-09-30 03:16:34,243][1157819] Updated weights for policy 0, policy_version 183808 (0.0006) [2024-09-30 03:16:34,823][1157819] Updated weights for policy 0, policy_version 183818 (0.0006) [2024-09-30 03:16:35,354][1157819] Updated weights for policy 0, policy_version 183828 (0.0006) [2024-09-30 03:16:35,466][1157520] Fps is (10 sec: 70861.0, 60 sec: 75093.2, 300 sec: 75088.7). Total num frames: 752963584. Throughput: 0: 18708.0. Samples: 178229364. Policy #0 lag: (min: 0.0, avg: 2.5, max: 5.0) [2024-09-30 03:16:35,466][1157520] Avg episode reward: [(0, '54.349')] [2024-09-30 03:16:35,912][1157819] Updated weights for policy 0, policy_version 183838 (0.0006) [2024-09-30 03:16:36,480][1157819] Updated weights for policy 0, policy_version 183848 (0.0006) [2024-09-30 03:16:37,023][1157819] Updated weights for policy 0, policy_version 183858 (0.0006) [2024-09-30 03:16:37,579][1157819] Updated weights for policy 0, policy_version 183868 (0.0006) [2024-09-30 03:16:38,134][1157819] Updated weights for policy 0, policy_version 183878 (0.0006) [2024-09-30 03:16:38,673][1157819] Updated weights for policy 0, policy_version 183888 (0.0006) [2024-09-30 03:16:39,210][1157819] Updated weights for policy 0, policy_version 183898 (0.0006) [2024-09-30 03:16:39,764][1157819] Updated weights for policy 0, policy_version 183908 (0.0006) [2024-09-30 03:16:40,329][1157819] Updated weights for policy 0, policy_version 183918 (0.0006) [2024-09-30 03:16:40,466][1157520] Fps is (10 sec: 74547.2, 60 sec: 75025.1, 300 sec: 75130.4). Total num frames: 753336320. Throughput: 0: 18643.5. Samples: 178284788. Policy #0 lag: (min: 0.0, avg: 2.5, max: 5.0) [2024-09-30 03:16:40,466][1157520] Avg episode reward: [(0, '55.974')] [2024-09-30 03:16:40,937][1157819] Updated weights for policy 0, policy_version 183928 (0.0006) [2024-09-30 03:16:41,523][1157819] Updated weights for policy 0, policy_version 183938 (0.0006) [2024-09-30 03:16:42,081][1157819] Updated weights for policy 0, policy_version 183948 (0.0006) [2024-09-30 03:16:42,633][1157819] Updated weights for policy 0, policy_version 183958 (0.0006) [2024-09-30 03:16:43,228][1157819] Updated weights for policy 0, policy_version 183968 (0.0006) [2024-09-30 03:16:43,799][1157819] Updated weights for policy 0, policy_version 183978 (0.0006) [2024-09-30 03:16:44,380][1157819] Updated weights for policy 0, policy_version 183988 (0.0006) [2024-09-30 03:16:44,924][1157819] Updated weights for policy 0, policy_version 183998 (0.0006) [2024-09-30 03:16:45,189][1157736] Signal inference workers to stop experience collection... (12450 times) [2024-09-30 03:16:45,193][1157736] Signal inference workers to resume experience collection... (12450 times) [2024-09-30 03:16:45,196][1157819] InferenceWorker_p0-w0: stopping experience collection (12450 times) [2024-09-30 03:16:45,199][1157819] InferenceWorker_p0-w0: resuming experience collection (12450 times) [2024-09-30 03:16:45,466][1157520] Fps is (10 sec: 72908.4, 60 sec: 74615.6, 300 sec: 75074.8). Total num frames: 753692672. Throughput: 0: 18371.1. Samples: 178393384. Policy #0 lag: (min: 0.0, avg: 2.5, max: 5.0) [2024-09-30 03:16:45,466][1157520] Avg episode reward: [(0, '56.367')] [2024-09-30 03:16:45,501][1157819] Updated weights for policy 0, policy_version 184008 (0.0006) [2024-09-30 03:16:46,046][1157819] Updated weights for policy 0, policy_version 184018 (0.0006) [2024-09-30 03:16:46,653][1157819] Updated weights for policy 0, policy_version 184028 (0.0006) [2024-09-30 03:16:47,204][1157819] Updated weights for policy 0, policy_version 184038 (0.0006) [2024-09-30 03:16:47,789][1157819] Updated weights for policy 0, policy_version 184048 (0.0006) [2024-09-30 03:16:48,388][1157819] Updated weights for policy 0, policy_version 184058 (0.0006) [2024-09-30 03:16:49,056][1157819] Updated weights for policy 0, policy_version 184068 (0.0006) [2024-09-30 03:16:49,622][1157819] Updated weights for policy 0, policy_version 184078 (0.0006) [2024-09-30 03:16:50,305][1157819] Updated weights for policy 0, policy_version 184088 (0.0006) [2024-09-30 03:16:50,466][1157520] Fps is (10 sec: 70041.8, 60 sec: 73932.9, 300 sec: 74977.6). Total num frames: 754036736. Throughput: 0: 18177.7. Samples: 178498460. Policy #0 lag: (min: 0.0, avg: 2.5, max: 5.0) [2024-09-30 03:16:50,466][1157520] Avg episode reward: [(0, '55.951')] [2024-09-30 03:16:50,901][1157819] Updated weights for policy 0, policy_version 184098 (0.0006) [2024-09-30 03:16:51,505][1157819] Updated weights for policy 0, policy_version 184108 (0.0006) [2024-09-30 03:16:52,137][1157819] Updated weights for policy 0, policy_version 184118 (0.0006) [2024-09-30 03:16:52,752][1157819] Updated weights for policy 0, policy_version 184128 (0.0006) [2024-09-30 03:16:53,388][1157819] Updated weights for policy 0, policy_version 184138 (0.0006) [2024-09-30 03:16:53,995][1157819] Updated weights for policy 0, policy_version 184148 (0.0006) [2024-09-30 03:16:54,630][1157819] Updated weights for policy 0, policy_version 184158 (0.0006) [2024-09-30 03:16:55,248][1157819] Updated weights for policy 0, policy_version 184168 (0.0006) [2024-09-30 03:16:55,466][1157520] Fps is (10 sec: 67173.9, 60 sec: 72772.1, 300 sec: 74838.8). Total num frames: 754364416. Throughput: 0: 18082.5. Samples: 178547596. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 03:16:55,466][1157520] Avg episode reward: [(0, '55.842')] [2024-09-30 03:16:55,884][1157819] Updated weights for policy 0, policy_version 184178 (0.0006) [2024-09-30 03:16:56,485][1157819] Updated weights for policy 0, policy_version 184188 (0.0006) [2024-09-30 03:16:57,036][1157819] Updated weights for policy 0, policy_version 184198 (0.0006) [2024-09-30 03:16:57,582][1157819] Updated weights for policy 0, policy_version 184208 (0.0006) [2024-09-30 03:16:58,186][1157819] Updated weights for policy 0, policy_version 184218 (0.0006) [2024-09-30 03:16:58,716][1157819] Updated weights for policy 0, policy_version 184228 (0.0006) [2024-09-30 03:16:59,214][1157819] Updated weights for policy 0, policy_version 184238 (0.0006) [2024-09-30 03:16:59,779][1157819] Updated weights for policy 0, policy_version 184248 (0.0007) [2024-09-30 03:17:00,331][1157819] Updated weights for policy 0, policy_version 184258 (0.0006) [2024-09-30 03:17:00,466][1157520] Fps is (10 sec: 69222.0, 60 sec: 72157.8, 300 sec: 74769.4). Total num frames: 754728960. Throughput: 0: 17894.3. Samples: 178651184. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 03:17:00,466][1157520] Avg episode reward: [(0, '56.588')] [2024-09-30 03:17:00,853][1157819] Updated weights for policy 0, policy_version 184268 (0.0006) [2024-09-30 03:17:01,349][1157819] Updated weights for policy 0, policy_version 184278 (0.0006) [2024-09-30 03:17:01,849][1157819] Updated weights for policy 0, policy_version 184288 (0.0006) [2024-09-30 03:17:02,418][1157819] Updated weights for policy 0, policy_version 184298 (0.0006) [2024-09-30 03:17:02,962][1157819] Updated weights for policy 0, policy_version 184308 (0.0006) [2024-09-30 03:17:03,176][1157736] Signal inference workers to stop experience collection... (12500 times) [2024-09-30 03:17:03,177][1157736] Signal inference workers to resume experience collection... (12500 times) [2024-09-30 03:17:03,180][1157819] InferenceWorker_p0-w0: stopping experience collection (12500 times) [2024-09-30 03:17:03,180][1157819] InferenceWorker_p0-w0: resuming experience collection (12500 times) [2024-09-30 03:17:03,510][1157819] Updated weights for policy 0, policy_version 184318 (0.0006) [2024-09-30 03:17:04,086][1157819] Updated weights for policy 0, policy_version 184328 (0.0006) [2024-09-30 03:17:04,596][1157819] Updated weights for policy 0, policy_version 184338 (0.0006) [2024-09-30 03:17:05,130][1157819] Updated weights for policy 0, policy_version 184348 (0.0006) [2024-09-30 03:17:05,466][1157520] Fps is (10 sec: 74548.1, 60 sec: 72294.3, 300 sec: 74811.0). Total num frames: 755109888. Throughput: 0: 17799.5. Samples: 178765312. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 03:17:05,466][1157520] Avg episode reward: [(0, '54.568')] [2024-09-30 03:17:05,704][1157819] Updated weights for policy 0, policy_version 184358 (0.0006) [2024-09-30 03:17:06,227][1157819] Updated weights for policy 0, policy_version 184368 (0.0006) [2024-09-30 03:17:06,766][1157819] Updated weights for policy 0, policy_version 184378 (0.0006) [2024-09-30 03:17:07,337][1157819] Updated weights for policy 0, policy_version 184388 (0.0006) [2024-09-30 03:17:07,866][1157819] Updated weights for policy 0, policy_version 184398 (0.0006) [2024-09-30 03:17:08,432][1157819] Updated weights for policy 0, policy_version 184408 (0.0006) [2024-09-30 03:17:08,920][1157819] Updated weights for policy 0, policy_version 184418 (0.0006) [2024-09-30 03:17:09,424][1157819] Updated weights for policy 0, policy_version 184428 (0.0006) [2024-09-30 03:17:09,918][1157819] Updated weights for policy 0, policy_version 184438 (0.0006) [2024-09-30 03:17:10,420][1157819] Updated weights for policy 0, policy_version 184448 (0.0006) [2024-09-30 03:17:10,466][1157520] Fps is (10 sec: 77004.7, 60 sec: 72840.3, 300 sec: 74963.7). Total num frames: 755499008. Throughput: 0: 17810.2. Samples: 178822040. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 03:17:10,466][1157520] Avg episode reward: [(0, '55.744')] [2024-09-30 03:17:10,949][1157819] Updated weights for policy 0, policy_version 184458 (0.0006) [2024-09-30 03:17:11,488][1157819] Updated weights for policy 0, policy_version 184468 (0.0006) [2024-09-30 03:17:11,988][1157819] Updated weights for policy 0, policy_version 184478 (0.0006) [2024-09-30 03:17:12,504][1157819] Updated weights for policy 0, policy_version 184488 (0.0006) [2024-09-30 03:17:12,990][1157819] Updated weights for policy 0, policy_version 184498 (0.0006) [2024-09-30 03:17:13,482][1157819] Updated weights for policy 0, policy_version 184508 (0.0006) [2024-09-30 03:17:13,986][1157819] Updated weights for policy 0, policy_version 184518 (0.0006) [2024-09-30 03:17:14,477][1157819] Updated weights for policy 0, policy_version 184528 (0.0006) [2024-09-30 03:17:14,960][1157819] Updated weights for policy 0, policy_version 184538 (0.0006) [2024-09-30 03:17:15,453][1157819] Updated weights for policy 0, policy_version 184548 (0.0006) [2024-09-30 03:17:15,466][1157520] Fps is (10 sec: 79872.6, 60 sec: 73250.1, 300 sec: 75199.8). Total num frames: 755908608. Throughput: 0: 18322.4. Samples: 178942620. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 03:17:15,466][1157520] Avg episode reward: [(0, '54.279')] [2024-09-30 03:17:15,902][1157819] Updated weights for policy 0, policy_version 184558 (0.0006) [2024-09-30 03:17:16,409][1157819] Updated weights for policy 0, policy_version 184568 (0.0006) [2024-09-30 03:17:16,942][1157819] Updated weights for policy 0, policy_version 184578 (0.0006) [2024-09-30 03:17:17,473][1157819] Updated weights for policy 0, policy_version 184588 (0.0006) [2024-09-30 03:17:17,974][1157819] Updated weights for policy 0, policy_version 184598 (0.0006) [2024-09-30 03:17:18,470][1157819] Updated weights for policy 0, policy_version 184608 (0.0006) [2024-09-30 03:17:18,972][1157819] Updated weights for policy 0, policy_version 184618 (0.0006) [2024-09-30 03:17:19,532][1157819] Updated weights for policy 0, policy_version 184628 (0.0006) [2024-09-30 03:17:20,004][1157819] Updated weights for policy 0, policy_version 184638 (0.0006) [2024-09-30 03:17:20,466][1157520] Fps is (10 sec: 81511.1, 60 sec: 73454.8, 300 sec: 75338.7). Total num frames: 756314112. Throughput: 0: 18567.6. Samples: 179064908. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 03:17:20,466][1157520] Avg episode reward: [(0, '53.215')] [2024-09-30 03:17:20,509][1157819] Updated weights for policy 0, policy_version 184648 (0.0006) [2024-09-30 03:17:21,065][1157819] Updated weights for policy 0, policy_version 184658 (0.0006) [2024-09-30 03:17:21,587][1157819] Updated weights for policy 0, policy_version 184668 (0.0006) [2024-09-30 03:17:22,112][1157819] Updated weights for policy 0, policy_version 184678 (0.0006) [2024-09-30 03:17:22,622][1157819] Updated weights for policy 0, policy_version 184688 (0.0006) [2024-09-30 03:17:23,175][1157819] Updated weights for policy 0, policy_version 184698 (0.0006) [2024-09-30 03:17:23,709][1157819] Updated weights for policy 0, policy_version 184708 (0.0006) [2024-09-30 03:17:24,236][1157819] Updated weights for policy 0, policy_version 184718 (0.0006) [2024-09-30 03:17:24,771][1157819] Updated weights for policy 0, policy_version 184728 (0.0006) [2024-09-30 03:17:25,294][1157819] Updated weights for policy 0, policy_version 184738 (0.0006) [2024-09-30 03:17:25,466][1157520] Fps is (10 sec: 79052.3, 60 sec: 74069.4, 300 sec: 75283.1). Total num frames: 756699136. Throughput: 0: 18633.7. Samples: 179123304. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 03:17:25,466][1157520] Avg episode reward: [(0, '55.681')] [2024-09-30 03:17:25,818][1157819] Updated weights for policy 0, policy_version 184748 (0.0006) [2024-09-30 03:17:26,360][1157819] Updated weights for policy 0, policy_version 184758 (0.0006) [2024-09-30 03:17:26,957][1157819] Updated weights for policy 0, policy_version 184768 (0.0006) [2024-09-30 03:17:27,513][1157819] Updated weights for policy 0, policy_version 184778 (0.0006) [2024-09-30 03:17:28,091][1157819] Updated weights for policy 0, policy_version 184788 (0.0006) [2024-09-30 03:17:28,682][1157819] Updated weights for policy 0, policy_version 184798 (0.0006) [2024-09-30 03:17:29,248][1157819] Updated weights for policy 0, policy_version 184808 (0.0006) [2024-09-30 03:17:29,820][1157819] Updated weights for policy 0, policy_version 184818 (0.0006) [2024-09-30 03:17:30,158][1157736] Signal inference workers to stop experience collection... (12550 times) [2024-09-30 03:17:30,159][1157736] Signal inference workers to resume experience collection... (12550 times) [2024-09-30 03:17:30,164][1157819] InferenceWorker_p0-w0: stopping experience collection (12550 times) [2024-09-30 03:17:30,164][1157819] InferenceWorker_p0-w0: resuming experience collection (12550 times) [2024-09-30 03:17:30,388][1157819] Updated weights for policy 0, policy_version 184828 (0.0006) [2024-09-30 03:17:30,466][1157520] Fps is (10 sec: 74956.8, 60 sec: 74547.3, 300 sec: 75185.9). Total num frames: 757063680. Throughput: 0: 18710.6. Samples: 179235360. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 03:17:30,466][1157520] Avg episode reward: [(0, '56.382')] [2024-09-30 03:17:30,953][1157819] Updated weights for policy 0, policy_version 184838 (0.0006) [2024-09-30 03:17:31,522][1157819] Updated weights for policy 0, policy_version 184848 (0.0006) [2024-09-30 03:17:32,058][1157819] Updated weights for policy 0, policy_version 184858 (0.0006) [2024-09-30 03:17:32,568][1157819] Updated weights for policy 0, policy_version 184868 (0.0006) [2024-09-30 03:17:33,149][1157819] Updated weights for policy 0, policy_version 184878 (0.0006) [2024-09-30 03:17:33,696][1157819] Updated weights for policy 0, policy_version 184888 (0.0006) [2024-09-30 03:17:34,247][1157819] Updated weights for policy 0, policy_version 184898 (0.0006) [2024-09-30 03:17:34,748][1157819] Updated weights for policy 0, policy_version 184908 (0.0006) [2024-09-30 03:17:35,247][1157819] Updated weights for policy 0, policy_version 184918 (0.0006) [2024-09-30 03:17:35,466][1157520] Fps is (10 sec: 74137.9, 60 sec: 74615.5, 300 sec: 75144.3). Total num frames: 757440512. Throughput: 0: 18852.0. Samples: 179346800. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 03:17:35,466][1157520] Avg episode reward: [(0, '55.281')] [2024-09-30 03:17:35,761][1157819] Updated weights for policy 0, policy_version 184928 (0.0006) [2024-09-30 03:17:36,284][1157819] Updated weights for policy 0, policy_version 184938 (0.0006) [2024-09-30 03:17:36,799][1157819] Updated weights for policy 0, policy_version 184948 (0.0006) [2024-09-30 03:17:37,332][1157819] Updated weights for policy 0, policy_version 184958 (0.0006) [2024-09-30 03:17:37,818][1157819] Updated weights for policy 0, policy_version 184968 (0.0006) [2024-09-30 03:17:38,353][1157819] Updated weights for policy 0, policy_version 184978 (0.0006) [2024-09-30 03:17:38,878][1157819] Updated weights for policy 0, policy_version 184988 (0.0006) [2024-09-30 03:17:39,397][1157819] Updated weights for policy 0, policy_version 184998 (0.0006) [2024-09-30 03:17:39,925][1157819] Updated weights for policy 0, policy_version 185008 (0.0006) [2024-09-30 03:17:40,419][1157819] Updated weights for policy 0, policy_version 185018 (0.0006) [2024-09-30 03:17:40,466][1157520] Fps is (10 sec: 77005.2, 60 sec: 74956.9, 300 sec: 75144.3). Total num frames: 757833728. Throughput: 0: 19086.0. Samples: 179406464. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 03:17:40,466][1157520] Avg episode reward: [(0, '55.426')] [2024-09-30 03:17:40,918][1157819] Updated weights for policy 0, policy_version 185028 (0.0006) [2024-09-30 03:17:41,408][1157819] Updated weights for policy 0, policy_version 185038 (0.0006) [2024-09-30 03:17:41,930][1157819] Updated weights for policy 0, policy_version 185048 (0.0006) [2024-09-30 03:17:42,487][1157819] Updated weights for policy 0, policy_version 185058 (0.0006) [2024-09-30 03:17:43,062][1157819] Updated weights for policy 0, policy_version 185068 (0.0006) [2024-09-30 03:17:43,672][1157819] Updated weights for policy 0, policy_version 185078 (0.0006) [2024-09-30 03:17:44,196][1157819] Updated weights for policy 0, policy_version 185088 (0.0006) [2024-09-30 03:17:44,791][1157819] Updated weights for policy 0, policy_version 185098 (0.0006) [2024-09-30 03:17:45,328][1157819] Updated weights for policy 0, policy_version 185108 (0.0006) [2024-09-30 03:17:45,466][1157520] Fps is (10 sec: 77004.6, 60 sec: 75298.2, 300 sec: 75074.9). Total num frames: 758210560. Throughput: 0: 19360.3. Samples: 179522396. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 03:17:45,466][1157520] Avg episode reward: [(0, '55.480')] [2024-09-30 03:17:45,918][1157819] Updated weights for policy 0, policy_version 185118 (0.0006) [2024-09-30 03:17:46,506][1157819] Updated weights for policy 0, policy_version 185128 (0.0006) [2024-09-30 03:17:47,123][1157819] Updated weights for policy 0, policy_version 185138 (0.0006) [2024-09-30 03:17:47,710][1157819] Updated weights for policy 0, policy_version 185148 (0.0006) [2024-09-30 03:17:48,320][1157819] Updated weights for policy 0, policy_version 185158 (0.0006) [2024-09-30 03:17:48,953][1157819] Updated weights for policy 0, policy_version 185168 (0.0006) [2024-09-30 03:17:49,586][1157819] Updated weights for policy 0, policy_version 185178 (0.0006) [2024-09-30 03:17:50,188][1157819] Updated weights for policy 0, policy_version 185188 (0.0006) [2024-09-30 03:17:50,466][1157520] Fps is (10 sec: 71269.6, 60 sec: 75161.5, 300 sec: 74880.4). Total num frames: 758546432. Throughput: 0: 19118.2. Samples: 179625632. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 03:17:50,466][1157520] Avg episode reward: [(0, '57.735')] [2024-09-30 03:17:50,775][1157819] Updated weights for policy 0, policy_version 185198 (0.0006) [2024-09-30 03:17:51,348][1157819] Updated weights for policy 0, policy_version 185208 (0.0006) [2024-09-30 03:17:51,906][1157819] Updated weights for policy 0, policy_version 185218 (0.0006) [2024-09-30 03:17:52,492][1157819] Updated weights for policy 0, policy_version 185228 (0.0006) [2024-09-30 03:17:53,038][1157819] Updated weights for policy 0, policy_version 185238 (0.0006) [2024-09-30 03:17:53,566][1157819] Updated weights for policy 0, policy_version 185248 (0.0006) [2024-09-30 03:17:54,111][1157819] Updated weights for policy 0, policy_version 185258 (0.0006) [2024-09-30 03:17:54,630][1157819] Updated weights for policy 0, policy_version 185268 (0.0006) [2024-09-30 03:17:55,175][1157819] Updated weights for policy 0, policy_version 185278 (0.0006) [2024-09-30 03:17:55,466][1157520] Fps is (10 sec: 70860.8, 60 sec: 75912.7, 300 sec: 74797.1). Total num frames: 758919168. Throughput: 0: 19041.9. Samples: 179678924. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 03:17:55,466][1157520] Avg episode reward: [(0, '56.481')] [2024-09-30 03:17:55,715][1157819] Updated weights for policy 0, policy_version 185288 (0.0006) [2024-09-30 03:17:56,287][1157819] Updated weights for policy 0, policy_version 185298 (0.0006) [2024-09-30 03:17:56,801][1157819] Updated weights for policy 0, policy_version 185308 (0.0006) [2024-09-30 03:17:57,348][1157819] Updated weights for policy 0, policy_version 185318 (0.0006) [2024-09-30 03:17:57,889][1157819] Updated weights for policy 0, policy_version 185328 (0.0006) [2024-09-30 03:17:58,412][1157819] Updated weights for policy 0, policy_version 185338 (0.0006) [2024-09-30 03:17:58,969][1157819] Updated weights for policy 0, policy_version 185348 (0.0006) [2024-09-30 03:17:59,535][1157819] Updated weights for policy 0, policy_version 185358 (0.0006) [2024-09-30 03:18:00,054][1157819] Updated weights for policy 0, policy_version 185368 (0.0006) [2024-09-30 03:18:00,466][1157520] Fps is (10 sec: 74957.9, 60 sec: 76117.5, 300 sec: 74741.6). Total num frames: 759296000. Throughput: 0: 18898.2. Samples: 179793040. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 03:18:00,466][1157520] Avg episode reward: [(0, '58.228')] [2024-09-30 03:18:00,611][1157819] Updated weights for policy 0, policy_version 185378 (0.0006) [2024-09-30 03:18:00,908][1157736] Signal inference workers to stop experience collection... (12600 times) [2024-09-30 03:18:00,909][1157736] Signal inference workers to resume experience collection... (12600 times) [2024-09-30 03:18:00,914][1157819] InferenceWorker_p0-w0: stopping experience collection (12600 times) [2024-09-30 03:18:00,914][1157819] InferenceWorker_p0-w0: resuming experience collection (12600 times) [2024-09-30 03:18:01,140][1157819] Updated weights for policy 0, policy_version 185388 (0.0006) [2024-09-30 03:18:01,717][1157819] Updated weights for policy 0, policy_version 185398 (0.0006) [2024-09-30 03:18:02,240][1157819] Updated weights for policy 0, policy_version 185408 (0.0006) [2024-09-30 03:18:02,767][1157819] Updated weights for policy 0, policy_version 185418 (0.0006) [2024-09-30 03:18:03,337][1157819] Updated weights for policy 0, policy_version 185428 (0.0006) [2024-09-30 03:18:03,862][1157819] Updated weights for policy 0, policy_version 185438 (0.0006) [2024-09-30 03:18:04,408][1157819] Updated weights for policy 0, policy_version 185448 (0.0006) [2024-09-30 03:18:04,962][1157819] Updated weights for policy 0, policy_version 185458 (0.0006) [2024-09-30 03:18:05,466][1157520] Fps is (10 sec: 75366.2, 60 sec: 76049.0, 300 sec: 74686.0). Total num frames: 759672832. Throughput: 0: 18688.0. Samples: 179905868. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 03:18:05,466][1157520] Avg episode reward: [(0, '55.835')] [2024-09-30 03:18:05,469][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000185467_759672832.pth... [2024-09-30 03:18:05,533][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000181091_741748736.pth [2024-09-30 03:18:05,555][1157819] Updated weights for policy 0, policy_version 185468 (0.0006) [2024-09-30 03:18:06,035][1157819] Updated weights for policy 0, policy_version 185478 (0.0006) [2024-09-30 03:18:06,537][1157819] Updated weights for policy 0, policy_version 185488 (0.0006) [2024-09-30 03:18:07,074][1157819] Updated weights for policy 0, policy_version 185498 (0.0006) [2024-09-30 03:18:07,579][1157819] Updated weights for policy 0, policy_version 185508 (0.0006) [2024-09-30 03:18:08,099][1157819] Updated weights for policy 0, policy_version 185518 (0.0006) [2024-09-30 03:18:08,604][1157819] Updated weights for policy 0, policy_version 185528 (0.0006) [2024-09-30 03:18:09,129][1157819] Updated weights for policy 0, policy_version 185538 (0.0006) [2024-09-30 03:18:09,635][1157819] Updated weights for policy 0, policy_version 185548 (0.0006) [2024-09-30 03:18:10,155][1157819] Updated weights for policy 0, policy_version 185558 (0.0006) [2024-09-30 03:18:10,466][1157520] Fps is (10 sec: 77004.8, 60 sec: 76117.5, 300 sec: 74686.1). Total num frames: 760066048. Throughput: 0: 18689.1. Samples: 179964312. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 03:18:10,466][1157520] Avg episode reward: [(0, '56.658')] [2024-09-30 03:18:10,670][1157819] Updated weights for policy 0, policy_version 185568 (0.0006) [2024-09-30 03:18:11,181][1157819] Updated weights for policy 0, policy_version 185578 (0.0006) [2024-09-30 03:18:11,709][1157819] Updated weights for policy 0, policy_version 185588 (0.0006) [2024-09-30 03:18:12,200][1157819] Updated weights for policy 0, policy_version 185598 (0.0006) [2024-09-30 03:18:12,717][1157819] Updated weights for policy 0, policy_version 185608 (0.0006) [2024-09-30 03:18:13,264][1157819] Updated weights for policy 0, policy_version 185618 (0.0006) [2024-09-30 03:18:13,748][1157819] Updated weights for policy 0, policy_version 185628 (0.0006) [2024-09-30 03:18:14,248][1157819] Updated weights for policy 0, policy_version 185638 (0.0006) [2024-09-30 03:18:14,795][1157819] Updated weights for policy 0, policy_version 185648 (0.0006) [2024-09-30 03:18:15,322][1157819] Updated weights for policy 0, policy_version 185658 (0.0006) [2024-09-30 03:18:15,466][1157520] Fps is (10 sec: 79053.5, 60 sec: 75912.5, 300 sec: 74824.9). Total num frames: 760463360. Throughput: 0: 18858.2. Samples: 180083980. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 03:18:15,466][1157520] Avg episode reward: [(0, '57.368')] [2024-09-30 03:18:15,851][1157819] Updated weights for policy 0, policy_version 185668 (0.0006) [2024-09-30 03:18:16,426][1157819] Updated weights for policy 0, policy_version 185678 (0.0006) [2024-09-30 03:18:16,940][1157819] Updated weights for policy 0, policy_version 185688 (0.0006) [2024-09-30 03:18:17,436][1157819] Updated weights for policy 0, policy_version 185698 (0.0006) [2024-09-30 03:18:17,944][1157819] Updated weights for policy 0, policy_version 185708 (0.0006) [2024-09-30 03:18:18,442][1157819] Updated weights for policy 0, policy_version 185718 (0.0006) [2024-09-30 03:18:18,947][1157819] Updated weights for policy 0, policy_version 185728 (0.0006) [2024-09-30 03:18:19,459][1157819] Updated weights for policy 0, policy_version 185738 (0.0006) [2024-09-30 03:18:19,964][1157819] Updated weights for policy 0, policy_version 185748 (0.0006) [2024-09-30 03:18:20,466][1157520] Fps is (10 sec: 79462.3, 60 sec: 75776.1, 300 sec: 74977.7). Total num frames: 760860672. Throughput: 0: 19007.3. Samples: 180202128. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 03:18:20,466][1157520] Avg episode reward: [(0, '57.405')] [2024-09-30 03:18:20,480][1157819] Updated weights for policy 0, policy_version 185758 (0.0006) [2024-09-30 03:18:20,998][1157819] Updated weights for policy 0, policy_version 185768 (0.0006) [2024-09-30 03:18:21,509][1157819] Updated weights for policy 0, policy_version 185778 (0.0006) [2024-09-30 03:18:22,020][1157819] Updated weights for policy 0, policy_version 185788 (0.0006) [2024-09-30 03:18:22,527][1157819] Updated weights for policy 0, policy_version 185798 (0.0006) [2024-09-30 03:18:23,026][1157819] Updated weights for policy 0, policy_version 185808 (0.0006) [2024-09-30 03:18:23,539][1157819] Updated weights for policy 0, policy_version 185818 (0.0006) [2024-09-30 03:18:24,062][1157819] Updated weights for policy 0, policy_version 185828 (0.0006) [2024-09-30 03:18:24,600][1157819] Updated weights for policy 0, policy_version 185838 (0.0006) [2024-09-30 03:18:25,135][1157819] Updated weights for policy 0, policy_version 185848 (0.0006) [2024-09-30 03:18:25,466][1157520] Fps is (10 sec: 79461.7, 60 sec: 75980.8, 300 sec: 75144.2). Total num frames: 761257984. Throughput: 0: 19029.3. Samples: 180262784. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 03:18:25,466][1157520] Avg episode reward: [(0, '56.706')] [2024-09-30 03:18:25,664][1157819] Updated weights for policy 0, policy_version 185858 (0.0006) [2024-09-30 03:18:26,195][1157819] Updated weights for policy 0, policy_version 185868 (0.0006) [2024-09-30 03:18:26,747][1157819] Updated weights for policy 0, policy_version 185878 (0.0006) [2024-09-30 03:18:27,272][1157819] Updated weights for policy 0, policy_version 185888 (0.0006) [2024-09-30 03:18:27,809][1157819] Updated weights for policy 0, policy_version 185898 (0.0006) [2024-09-30 03:18:28,381][1157819] Updated weights for policy 0, policy_version 185908 (0.0006) [2024-09-30 03:18:28,915][1157819] Updated weights for policy 0, policy_version 185918 (0.0006) [2024-09-30 03:18:28,958][1157736] Signal inference workers to stop experience collection... (12650 times) [2024-09-30 03:18:28,959][1157736] Signal inference workers to resume experience collection... (12650 times) [2024-09-30 03:18:28,964][1157819] InferenceWorker_p0-w0: stopping experience collection (12650 times) [2024-09-30 03:18:28,964][1157819] InferenceWorker_p0-w0: resuming experience collection (12650 times) [2024-09-30 03:18:29,485][1157819] Updated weights for policy 0, policy_version 185928 (0.0006) [2024-09-30 03:18:30,037][1157819] Updated weights for policy 0, policy_version 185938 (0.0006) [2024-09-30 03:18:30,466][1157520] Fps is (10 sec: 77004.3, 60 sec: 76117.3, 300 sec: 75185.9). Total num frames: 761630720. Throughput: 0: 19011.7. Samples: 180377924. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 03:18:30,466][1157520] Avg episode reward: [(0, '56.009')] [2024-09-30 03:18:30,662][1157819] Updated weights for policy 0, policy_version 185948 (0.0006) [2024-09-30 03:18:31,209][1157819] Updated weights for policy 0, policy_version 185958 (0.0006) [2024-09-30 03:18:31,782][1157819] Updated weights for policy 0, policy_version 185968 (0.0006) [2024-09-30 03:18:32,368][1157819] Updated weights for policy 0, policy_version 185978 (0.0006) [2024-09-30 03:18:32,918][1157819] Updated weights for policy 0, policy_version 185988 (0.0006) [2024-09-30 03:18:33,468][1157819] Updated weights for policy 0, policy_version 185998 (0.0006) [2024-09-30 03:18:34,016][1157819] Updated weights for policy 0, policy_version 186008 (0.0006) [2024-09-30 03:18:34,534][1157819] Updated weights for policy 0, policy_version 186018 (0.0006) [2024-09-30 03:18:35,046][1157819] Updated weights for policy 0, policy_version 186028 (0.0006) [2024-09-30 03:18:35,466][1157520] Fps is (10 sec: 73727.0, 60 sec: 75912.3, 300 sec: 75116.4). Total num frames: 761995264. Throughput: 0: 19146.8. Samples: 180487240. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 03:18:35,466][1157520] Avg episode reward: [(0, '54.392')] [2024-09-30 03:18:35,658][1157819] Updated weights for policy 0, policy_version 186038 (0.0006) [2024-09-30 03:18:36,237][1157819] Updated weights for policy 0, policy_version 186048 (0.0006) [2024-09-30 03:18:36,818][1157819] Updated weights for policy 0, policy_version 186058 (0.0006) [2024-09-30 03:18:37,371][1157819] Updated weights for policy 0, policy_version 186068 (0.0006) [2024-09-30 03:18:37,950][1157819] Updated weights for policy 0, policy_version 186078 (0.0006) [2024-09-30 03:18:38,497][1157819] Updated weights for policy 0, policy_version 186088 (0.0006) [2024-09-30 03:18:39,092][1157819] Updated weights for policy 0, policy_version 186098 (0.0006) [2024-09-30 03:18:39,699][1157819] Updated weights for policy 0, policy_version 186108 (0.0006) [2024-09-30 03:18:40,239][1157819] Updated weights for policy 0, policy_version 186118 (0.0006) [2024-09-30 03:18:40,466][1157520] Fps is (10 sec: 72499.2, 60 sec: 75366.3, 300 sec: 75047.1). Total num frames: 762355712. Throughput: 0: 19157.2. Samples: 180541000. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 03:18:40,466][1157520] Avg episode reward: [(0, '56.196')] [2024-09-30 03:18:40,827][1157819] Updated weights for policy 0, policy_version 186128 (0.0006) [2024-09-30 03:18:41,401][1157819] Updated weights for policy 0, policy_version 186138 (0.0006) [2024-09-30 03:18:41,953][1157819] Updated weights for policy 0, policy_version 186148 (0.0006) [2024-09-30 03:18:42,529][1157819] Updated weights for policy 0, policy_version 186158 (0.0006) [2024-09-30 03:18:43,099][1157819] Updated weights for policy 0, policy_version 186168 (0.0006) [2024-09-30 03:18:43,693][1157819] Updated weights for policy 0, policy_version 186178 (0.0006) [2024-09-30 03:18:44,271][1157819] Updated weights for policy 0, policy_version 186188 (0.0006) [2024-09-30 03:18:44,852][1157819] Updated weights for policy 0, policy_version 186198 (0.0006) [2024-09-30 03:18:45,357][1157819] Updated weights for policy 0, policy_version 186208 (0.0006) [2024-09-30 03:18:45,466][1157520] Fps is (10 sec: 72091.2, 60 sec: 75093.4, 300 sec: 75005.4). Total num frames: 762716160. Throughput: 0: 19000.4. Samples: 180648060. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 03:18:45,466][1157520] Avg episode reward: [(0, '57.012')] [2024-09-30 03:18:45,895][1157819] Updated weights for policy 0, policy_version 186218 (0.0006) [2024-09-30 03:18:46,437][1157819] Updated weights for policy 0, policy_version 186228 (0.0006) [2024-09-30 03:18:46,959][1157819] Updated weights for policy 0, policy_version 186238 (0.0006) [2024-09-30 03:18:47,493][1157819] Updated weights for policy 0, policy_version 186248 (0.0006) [2024-09-30 03:18:48,026][1157819] Updated weights for policy 0, policy_version 186258 (0.0006) [2024-09-30 03:18:48,562][1157819] Updated weights for policy 0, policy_version 186268 (0.0006) [2024-09-30 03:18:49,103][1157819] Updated weights for policy 0, policy_version 186278 (0.0006) [2024-09-30 03:18:49,633][1157819] Updated weights for policy 0, policy_version 186288 (0.0006) [2024-09-30 03:18:50,197][1157819] Updated weights for policy 0, policy_version 186298 (0.0006) [2024-09-30 03:18:50,466][1157520] Fps is (10 sec: 74137.7, 60 sec: 75844.4, 300 sec: 74991.5). Total num frames: 763097088. Throughput: 0: 19016.9. Samples: 180761628. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 03:18:50,466][1157520] Avg episode reward: [(0, '53.709')] [2024-09-30 03:18:50,716][1157819] Updated weights for policy 0, policy_version 186308 (0.0006) [2024-09-30 03:18:51,240][1157819] Updated weights for policy 0, policy_version 186318 (0.0006) [2024-09-30 03:18:51,812][1157819] Updated weights for policy 0, policy_version 186328 (0.0006) [2024-09-30 03:18:52,354][1157819] Updated weights for policy 0, policy_version 186338 (0.0006) [2024-09-30 03:18:52,859][1157819] Updated weights for policy 0, policy_version 186348 (0.0006) [2024-09-30 03:18:53,343][1157819] Updated weights for policy 0, policy_version 186358 (0.0006) [2024-09-30 03:18:53,837][1157819] Updated weights for policy 0, policy_version 186368 (0.0006) [2024-09-30 03:18:54,330][1157819] Updated weights for policy 0, policy_version 186378 (0.0006) [2024-09-30 03:18:54,848][1157819] Updated weights for policy 0, policy_version 186388 (0.0006) [2024-09-30 03:18:55,348][1157819] Updated weights for policy 0, policy_version 186398 (0.0006) [2024-09-30 03:18:55,466][1157520] Fps is (10 sec: 77823.9, 60 sec: 76253.9, 300 sec: 75033.2). Total num frames: 763494400. Throughput: 0: 19004.7. Samples: 180819524. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 03:18:55,466][1157520] Avg episode reward: [(0, '54.704')] [2024-09-30 03:18:55,852][1157819] Updated weights for policy 0, policy_version 186408 (0.0006) [2024-09-30 03:18:56,360][1157819] Updated weights for policy 0, policy_version 186418 (0.0006) [2024-09-30 03:18:56,845][1157819] Updated weights for policy 0, policy_version 186428 (0.0006) [2024-09-30 03:18:57,338][1157819] Updated weights for policy 0, policy_version 186438 (0.0006) [2024-09-30 03:18:57,860][1157819] Updated weights for policy 0, policy_version 186448 (0.0006) [2024-09-30 03:18:58,358][1157819] Updated weights for policy 0, policy_version 186458 (0.0006) [2024-09-30 03:18:58,883][1157819] Updated weights for policy 0, policy_version 186468 (0.0006) [2024-09-30 03:18:59,394][1157819] Updated weights for policy 0, policy_version 186478 (0.0006) [2024-09-30 03:18:59,762][1157736] Signal inference workers to stop experience collection... (12700 times) [2024-09-30 03:18:59,763][1157736] Signal inference workers to resume experience collection... (12700 times) [2024-09-30 03:18:59,766][1157819] InferenceWorker_p0-w0: stopping experience collection (12700 times) [2024-09-30 03:18:59,769][1157819] InferenceWorker_p0-w0: resuming experience collection (12700 times) [2024-09-30 03:18:59,942][1157819] Updated weights for policy 0, policy_version 186488 (0.0006) [2024-09-30 03:19:00,466][1157520] Fps is (10 sec: 79462.7, 60 sec: 76595.2, 300 sec: 75060.9). Total num frames: 763891712. Throughput: 0: 19054.0. Samples: 180941412. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 03:19:00,466][1157520] Avg episode reward: [(0, '54.884')] [2024-09-30 03:19:00,503][1157819] Updated weights for policy 0, policy_version 186498 (0.0006) [2024-09-30 03:19:01,022][1157819] Updated weights for policy 0, policy_version 186508 (0.0006) [2024-09-30 03:19:01,556][1157819] Updated weights for policy 0, policy_version 186518 (0.0006) [2024-09-30 03:19:02,095][1157819] Updated weights for policy 0, policy_version 186528 (0.0006) [2024-09-30 03:19:02,636][1157819] Updated weights for policy 0, policy_version 186538 (0.0006) [2024-09-30 03:19:03,162][1157819] Updated weights for policy 0, policy_version 186548 (0.0006) [2024-09-30 03:19:03,742][1157819] Updated weights for policy 0, policy_version 186558 (0.0006) [2024-09-30 03:19:04,280][1157819] Updated weights for policy 0, policy_version 186568 (0.0006) [2024-09-30 03:19:04,846][1157819] Updated weights for policy 0, policy_version 186578 (0.0006) [2024-09-30 03:19:05,372][1157819] Updated weights for policy 0, policy_version 186588 (0.0006) [2024-09-30 03:19:05,466][1157520] Fps is (10 sec: 77414.5, 60 sec: 76595.3, 300 sec: 75074.8). Total num frames: 764268544. Throughput: 0: 18955.4. Samples: 181055120. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 03:19:05,466][1157520] Avg episode reward: [(0, '57.559')] [2024-09-30 03:19:05,913][1157819] Updated weights for policy 0, policy_version 186598 (0.0006) [2024-09-30 03:19:06,482][1157819] Updated weights for policy 0, policy_version 186608 (0.0006) [2024-09-30 03:19:07,053][1157819] Updated weights for policy 0, policy_version 186618 (0.0006) [2024-09-30 03:19:07,599][1157819] Updated weights for policy 0, policy_version 186628 (0.0006) [2024-09-30 03:19:08,149][1157819] Updated weights for policy 0, policy_version 186638 (0.0006) [2024-09-30 03:19:08,690][1157819] Updated weights for policy 0, policy_version 186648 (0.0006) [2024-09-30 03:19:09,237][1157819] Updated weights for policy 0, policy_version 186658 (0.0006) [2024-09-30 03:19:09,804][1157819] Updated weights for policy 0, policy_version 186668 (0.0006) [2024-09-30 03:19:10,380][1157819] Updated weights for policy 0, policy_version 186678 (0.0006) [2024-09-30 03:19:10,466][1157520] Fps is (10 sec: 74956.8, 60 sec: 76253.8, 300 sec: 75074.9). Total num frames: 764641280. Throughput: 0: 18842.4. Samples: 181110692. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 03:19:10,466][1157520] Avg episode reward: [(0, '55.563')] [2024-09-30 03:19:10,944][1157819] Updated weights for policy 0, policy_version 186688 (0.0006) [2024-09-30 03:19:11,436][1157819] Updated weights for policy 0, policy_version 186698 (0.0006) [2024-09-30 03:19:11,933][1157819] Updated weights for policy 0, policy_version 186708 (0.0006) [2024-09-30 03:19:12,436][1157819] Updated weights for policy 0, policy_version 186718 (0.0006) [2024-09-30 03:19:12,955][1157819] Updated weights for policy 0, policy_version 186728 (0.0006) [2024-09-30 03:19:13,467][1157819] Updated weights for policy 0, policy_version 186738 (0.0006) [2024-09-30 03:19:14,029][1157819] Updated weights for policy 0, policy_version 186748 (0.0006) [2024-09-30 03:19:14,546][1157819] Updated weights for policy 0, policy_version 186758 (0.0006) [2024-09-30 03:19:15,082][1157819] Updated weights for policy 0, policy_version 186768 (0.0006) [2024-09-30 03:19:15,466][1157520] Fps is (10 sec: 76184.5, 60 sec: 76117.2, 300 sec: 75130.4). Total num frames: 765030400. Throughput: 0: 18842.3. Samples: 181225828. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 03:19:15,466][1157520] Avg episode reward: [(0, '52.703')] [2024-09-30 03:19:15,620][1157819] Updated weights for policy 0, policy_version 186778 (0.0006) [2024-09-30 03:19:16,151][1157819] Updated weights for policy 0, policy_version 186788 (0.0006) [2024-09-30 03:19:16,673][1157819] Updated weights for policy 0, policy_version 186798 (0.0006) [2024-09-30 03:19:17,173][1157819] Updated weights for policy 0, policy_version 186808 (0.0006) [2024-09-30 03:19:17,706][1157819] Updated weights for policy 0, policy_version 186818 (0.0006) [2024-09-30 03:19:18,226][1157819] Updated weights for policy 0, policy_version 186828 (0.0006) [2024-09-30 03:19:18,732][1157819] Updated weights for policy 0, policy_version 186838 (0.0006) [2024-09-30 03:19:19,241][1157819] Updated weights for policy 0, policy_version 186848 (0.0006) [2024-09-30 03:19:19,772][1157819] Updated weights for policy 0, policy_version 186858 (0.0006) [2024-09-30 03:19:20,300][1157819] Updated weights for policy 0, policy_version 186868 (0.0006) [2024-09-30 03:19:20,466][1157520] Fps is (10 sec: 78233.1, 60 sec: 76049.0, 300 sec: 75172.0). Total num frames: 765423616. Throughput: 0: 19015.1. Samples: 181342916. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 03:19:20,466][1157520] Avg episode reward: [(0, '55.005')] [2024-09-30 03:19:20,804][1157819] Updated weights for policy 0, policy_version 186878 (0.0006) [2024-09-30 03:19:21,312][1157819] Updated weights for policy 0, policy_version 186888 (0.0006) [2024-09-30 03:19:21,828][1157819] Updated weights for policy 0, policy_version 186898 (0.0006) [2024-09-30 03:19:22,342][1157819] Updated weights for policy 0, policy_version 186908 (0.0006) [2024-09-30 03:19:22,937][1157819] Updated weights for policy 0, policy_version 186918 (0.0006) [2024-09-30 03:19:23,452][1157819] Updated weights for policy 0, policy_version 186928 (0.0006) [2024-09-30 03:19:24,023][1157819] Updated weights for policy 0, policy_version 186938 (0.0006) [2024-09-30 03:19:24,546][1157819] Updated weights for policy 0, policy_version 186948 (0.0006) [2024-09-30 03:19:24,760][1157736] Signal inference workers to stop experience collection... (12750 times) [2024-09-30 03:19:24,762][1157736] Signal inference workers to resume experience collection... (12750 times) [2024-09-30 03:19:24,762][1157819] InferenceWorker_p0-w0: stopping experience collection (12750 times) [2024-09-30 03:19:24,766][1157819] InferenceWorker_p0-w0: resuming experience collection (12750 times) [2024-09-30 03:19:25,089][1157819] Updated weights for policy 0, policy_version 186958 (0.0006) [2024-09-30 03:19:25,466][1157520] Fps is (10 sec: 77414.4, 60 sec: 75775.9, 300 sec: 75185.9). Total num frames: 765804544. Throughput: 0: 19125.4. Samples: 181401644. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 03:19:25,466][1157520] Avg episode reward: [(0, '57.212')] [2024-09-30 03:19:25,650][1157819] Updated weights for policy 0, policy_version 186968 (0.0006) [2024-09-30 03:19:26,180][1157819] Updated weights for policy 0, policy_version 186978 (0.0006) [2024-09-30 03:19:26,689][1157819] Updated weights for policy 0, policy_version 186988 (0.0006) [2024-09-30 03:19:27,215][1157819] Updated weights for policy 0, policy_version 186998 (0.0006) [2024-09-30 03:19:27,712][1157819] Updated weights for policy 0, policy_version 187008 (0.0006) [2024-09-30 03:19:28,226][1157819] Updated weights for policy 0, policy_version 187018 (0.0006) [2024-09-30 03:19:28,740][1157819] Updated weights for policy 0, policy_version 187028 (0.0006) [2024-09-30 03:19:29,244][1157819] Updated weights for policy 0, policy_version 187038 (0.0006) [2024-09-30 03:19:29,789][1157819] Updated weights for policy 0, policy_version 187048 (0.0006) [2024-09-30 03:19:30,326][1157819] Updated weights for policy 0, policy_version 187058 (0.0006) [2024-09-30 03:19:30,466][1157520] Fps is (10 sec: 77414.3, 60 sec: 76117.3, 300 sec: 75241.4). Total num frames: 766197760. Throughput: 0: 19322.3. Samples: 181517564. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 03:19:30,466][1157520] Avg episode reward: [(0, '55.900')] [2024-09-30 03:19:30,821][1157819] Updated weights for policy 0, policy_version 187068 (0.0006) [2024-09-30 03:19:31,328][1157819] Updated weights for policy 0, policy_version 187078 (0.0006) [2024-09-30 03:19:31,842][1157819] Updated weights for policy 0, policy_version 187088 (0.0006) [2024-09-30 03:19:32,366][1157819] Updated weights for policy 0, policy_version 187098 (0.0006) [2024-09-30 03:19:32,891][1157819] Updated weights for policy 0, policy_version 187108 (0.0006) [2024-09-30 03:19:33,424][1157819] Updated weights for policy 0, policy_version 187118 (0.0006) [2024-09-30 03:19:33,951][1157819] Updated weights for policy 0, policy_version 187128 (0.0006) [2024-09-30 03:19:34,447][1157819] Updated weights for policy 0, policy_version 187138 (0.0006) [2024-09-30 03:19:34,945][1157819] Updated weights for policy 0, policy_version 187148 (0.0006) [2024-09-30 03:19:35,466][1157520] Fps is (10 sec: 79052.0, 60 sec: 76663.5, 300 sec: 75352.5). Total num frames: 766595072. Throughput: 0: 19428.3. Samples: 181635904. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 03:19:35,466][1157520] Avg episode reward: [(0, '56.638')] [2024-09-30 03:19:35,486][1157819] Updated weights for policy 0, policy_version 187158 (0.0006) [2024-09-30 03:19:36,049][1157819] Updated weights for policy 0, policy_version 187168 (0.0006) [2024-09-30 03:19:36,534][1157819] Updated weights for policy 0, policy_version 187178 (0.0006) [2024-09-30 03:19:37,030][1157819] Updated weights for policy 0, policy_version 187188 (0.0006) [2024-09-30 03:19:37,541][1157819] Updated weights for policy 0, policy_version 187198 (0.0006) [2024-09-30 03:19:38,035][1157819] Updated weights for policy 0, policy_version 187208 (0.0006) [2024-09-30 03:19:38,554][1157819] Updated weights for policy 0, policy_version 187218 (0.0006) [2024-09-30 03:19:39,122][1157819] Updated weights for policy 0, policy_version 187228 (0.0006) [2024-09-30 03:19:39,636][1157819] Updated weights for policy 0, policy_version 187238 (0.0006) [2024-09-30 03:19:40,141][1157819] Updated weights for policy 0, policy_version 187248 (0.0006) [2024-09-30 03:19:40,466][1157520] Fps is (10 sec: 79462.0, 60 sec: 77277.8, 300 sec: 75519.2). Total num frames: 766992384. Throughput: 0: 19474.0. Samples: 181695856. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 03:19:40,466][1157520] Avg episode reward: [(0, '55.675')] [2024-09-30 03:19:40,678][1157819] Updated weights for policy 0, policy_version 187258 (0.0006) [2024-09-30 03:19:41,212][1157819] Updated weights for policy 0, policy_version 187268 (0.0006) [2024-09-30 03:19:41,717][1157819] Updated weights for policy 0, policy_version 187278 (0.0006) [2024-09-30 03:19:42,218][1157819] Updated weights for policy 0, policy_version 187288 (0.0006) [2024-09-30 03:19:42,768][1157819] Updated weights for policy 0, policy_version 187298 (0.0006) [2024-09-30 03:19:43,277][1157819] Updated weights for policy 0, policy_version 187308 (0.0006) [2024-09-30 03:19:43,813][1157819] Updated weights for policy 0, policy_version 187318 (0.0006) [2024-09-30 03:19:44,323][1157819] Updated weights for policy 0, policy_version 187328 (0.0006) [2024-09-30 03:19:44,882][1157819] Updated weights for policy 0, policy_version 187338 (0.0006) [2024-09-30 03:19:45,409][1157819] Updated weights for policy 0, policy_version 187348 (0.0006) [2024-09-30 03:19:45,466][1157520] Fps is (10 sec: 78233.4, 60 sec: 77687.1, 300 sec: 75491.3). Total num frames: 767377408. Throughput: 0: 19379.3. Samples: 181813484. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 03:19:45,466][1157520] Avg episode reward: [(0, '57.851')] [2024-09-30 03:19:45,949][1157819] Updated weights for policy 0, policy_version 187358 (0.0006) [2024-09-30 03:19:46,500][1157819] Updated weights for policy 0, policy_version 187368 (0.0006) [2024-09-30 03:19:47,034][1157819] Updated weights for policy 0, policy_version 187378 (0.0006) [2024-09-30 03:19:47,600][1157819] Updated weights for policy 0, policy_version 187388 (0.0006) [2024-09-30 03:19:48,125][1157819] Updated weights for policy 0, policy_version 187398 (0.0006) [2024-09-30 03:19:48,632][1157819] Updated weights for policy 0, policy_version 187408 (0.0006) [2024-09-30 03:19:49,180][1157819] Updated weights for policy 0, policy_version 187418 (0.0006) [2024-09-30 03:19:49,713][1157819] Updated weights for policy 0, policy_version 187428 (0.0006) [2024-09-30 03:19:50,275][1157819] Updated weights for policy 0, policy_version 187438 (0.0006) [2024-09-30 03:19:50,466][1157520] Fps is (10 sec: 76594.9, 60 sec: 77687.3, 300 sec: 75616.3). Total num frames: 767758336. Throughput: 0: 19382.6. Samples: 181927340. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 03:19:50,466][1157520] Avg episode reward: [(0, '55.074')] [2024-09-30 03:19:50,813][1157819] Updated weights for policy 0, policy_version 187448 (0.0006) [2024-09-30 03:19:51,322][1157819] Updated weights for policy 0, policy_version 187458 (0.0006) [2024-09-30 03:19:51,874][1157819] Updated weights for policy 0, policy_version 187468 (0.0006) [2024-09-30 03:19:52,427][1157819] Updated weights for policy 0, policy_version 187478 (0.0006) [2024-09-30 03:19:52,981][1157819] Updated weights for policy 0, policy_version 187488 (0.0006) [2024-09-30 03:19:53,538][1157819] Updated weights for policy 0, policy_version 187498 (0.0006) [2024-09-30 03:19:53,740][1157736] Signal inference workers to stop experience collection... (12800 times) [2024-09-30 03:19:53,741][1157736] Signal inference workers to resume experience collection... (12800 times) [2024-09-30 03:19:53,744][1157819] InferenceWorker_p0-w0: stopping experience collection (12800 times) [2024-09-30 03:19:53,744][1157819] InferenceWorker_p0-w0: resuming experience collection (12800 times) [2024-09-30 03:19:54,079][1157819] Updated weights for policy 0, policy_version 187508 (0.0006) [2024-09-30 03:19:54,618][1157819] Updated weights for policy 0, policy_version 187518 (0.0006) [2024-09-30 03:19:55,200][1157819] Updated weights for policy 0, policy_version 187528 (0.0006) [2024-09-30 03:19:55,466][1157520] Fps is (10 sec: 75367.4, 60 sec: 77277.7, 300 sec: 75810.7). Total num frames: 768131072. Throughput: 0: 19407.1. Samples: 181984012. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 03:19:55,466][1157520] Avg episode reward: [(0, '56.477')] [2024-09-30 03:19:55,748][1157819] Updated weights for policy 0, policy_version 187538 (0.0006) [2024-09-30 03:19:56,311][1157819] Updated weights for policy 0, policy_version 187548 (0.0006) [2024-09-30 03:19:56,878][1157819] Updated weights for policy 0, policy_version 187558 (0.0006) [2024-09-30 03:19:57,468][1157819] Updated weights for policy 0, policy_version 187568 (0.0006) [2024-09-30 03:19:58,068][1157819] Updated weights for policy 0, policy_version 187578 (0.0006) [2024-09-30 03:19:58,637][1157819] Updated weights for policy 0, policy_version 187588 (0.0006) [2024-09-30 03:19:59,204][1157819] Updated weights for policy 0, policy_version 187598 (0.0006) [2024-09-30 03:19:59,757][1157819] Updated weights for policy 0, policy_version 187608 (0.0006) [2024-09-30 03:20:00,285][1157819] Updated weights for policy 0, policy_version 187618 (0.0006) [2024-09-30 03:20:00,466][1157520] Fps is (10 sec: 73318.8, 60 sec: 76663.3, 300 sec: 75685.7). Total num frames: 768491520. Throughput: 0: 19265.2. Samples: 182092760. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 03:20:00,466][1157520] Avg episode reward: [(0, '54.879')] [2024-09-30 03:20:00,825][1157819] Updated weights for policy 0, policy_version 187628 (0.0006) [2024-09-30 03:20:01,316][1157819] Updated weights for policy 0, policy_version 187638 (0.0006) [2024-09-30 03:20:01,832][1157819] Updated weights for policy 0, policy_version 187648 (0.0006) [2024-09-30 03:20:02,348][1157819] Updated weights for policy 0, policy_version 187658 (0.0006) [2024-09-30 03:20:02,858][1157819] Updated weights for policy 0, policy_version 187668 (0.0006) [2024-09-30 03:20:03,374][1157819] Updated weights for policy 0, policy_version 187678 (0.0006) [2024-09-30 03:20:03,902][1157819] Updated weights for policy 0, policy_version 187688 (0.0006) [2024-09-30 03:20:04,481][1157819] Updated weights for policy 0, policy_version 187698 (0.0006) [2024-09-30 03:20:05,028][1157819] Updated weights for policy 0, policy_version 187708 (0.0006) [2024-09-30 03:20:05,466][1157520] Fps is (10 sec: 74956.0, 60 sec: 76868.0, 300 sec: 75685.7). Total num frames: 768880640. Throughput: 0: 19236.1. Samples: 182208544. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 03:20:05,466][1157520] Avg episode reward: [(0, '56.803')] [2024-09-30 03:20:05,473][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000187716_768884736.pth... [2024-09-30 03:20:05,522][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000183294_750772224.pth [2024-09-30 03:20:05,578][1157819] Updated weights for policy 0, policy_version 187718 (0.0006) [2024-09-30 03:20:06,105][1157819] Updated weights for policy 0, policy_version 187728 (0.0006) [2024-09-30 03:20:06,625][1157819] Updated weights for policy 0, policy_version 187738 (0.0006) [2024-09-30 03:20:07,180][1157819] Updated weights for policy 0, policy_version 187748 (0.0006) [2024-09-30 03:20:07,758][1157819] Updated weights for policy 0, policy_version 187758 (0.0006) [2024-09-30 03:20:08,296][1157819] Updated weights for policy 0, policy_version 187768 (0.0006) [2024-09-30 03:20:08,864][1157819] Updated weights for policy 0, policy_version 187778 (0.0006) [2024-09-30 03:20:09,390][1157819] Updated weights for policy 0, policy_version 187788 (0.0006) [2024-09-30 03:20:09,982][1157819] Updated weights for policy 0, policy_version 187798 (0.0006) [2024-09-30 03:20:10,466][1157520] Fps is (10 sec: 76185.2, 60 sec: 76868.1, 300 sec: 75755.1). Total num frames: 769253376. Throughput: 0: 19185.9. Samples: 182265012. Policy #0 lag: (min: 0.0, avg: 2.4, max: 6.0) [2024-09-30 03:20:10,466][1157520] Avg episode reward: [(0, '56.666')] [2024-09-30 03:20:10,551][1157819] Updated weights for policy 0, policy_version 187808 (0.0006) [2024-09-30 03:20:11,121][1157819] Updated weights for policy 0, policy_version 187818 (0.0006) [2024-09-30 03:20:11,681][1157819] Updated weights for policy 0, policy_version 187828 (0.0006) [2024-09-30 03:20:12,266][1157819] Updated weights for policy 0, policy_version 187838 (0.0006) [2024-09-30 03:20:12,850][1157819] Updated weights for policy 0, policy_version 187848 (0.0006) [2024-09-30 03:20:13,452][1157819] Updated weights for policy 0, policy_version 187858 (0.0006) [2024-09-30 03:20:14,091][1157819] Updated weights for policy 0, policy_version 187868 (0.0006) [2024-09-30 03:20:14,671][1157819] Updated weights for policy 0, policy_version 187878 (0.0006) [2024-09-30 03:20:15,244][1157819] Updated weights for policy 0, policy_version 187888 (0.0006) [2024-09-30 03:20:15,466][1157520] Fps is (10 sec: 72090.4, 60 sec: 76185.6, 300 sec: 75699.6). Total num frames: 769601536. Throughput: 0: 18979.9. Samples: 182371660. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:20:15,466][1157520] Avg episode reward: [(0, '55.125')] [2024-09-30 03:20:15,858][1157819] Updated weights for policy 0, policy_version 187898 (0.0006) [2024-09-30 03:20:16,473][1157819] Updated weights for policy 0, policy_version 187908 (0.0006) [2024-09-30 03:20:17,058][1157819] Updated weights for policy 0, policy_version 187918 (0.0006) [2024-09-30 03:20:17,623][1157819] Updated weights for policy 0, policy_version 187928 (0.0006) [2024-09-30 03:20:17,821][1157736] Signal inference workers to stop experience collection... (12850 times) [2024-09-30 03:20:17,822][1157736] Signal inference workers to resume experience collection... (12850 times) [2024-09-30 03:20:17,825][1157819] InferenceWorker_p0-w0: stopping experience collection (12850 times) [2024-09-30 03:20:17,827][1157819] InferenceWorker_p0-w0: resuming experience collection (12850 times) [2024-09-30 03:20:18,258][1157819] Updated weights for policy 0, policy_version 187938 (0.0006) [2024-09-30 03:20:18,845][1157819] Updated weights for policy 0, policy_version 187948 (0.0006) [2024-09-30 03:20:19,407][1157819] Updated weights for policy 0, policy_version 187958 (0.0006) [2024-09-30 03:20:20,019][1157819] Updated weights for policy 0, policy_version 187968 (0.0007) [2024-09-30 03:20:20,466][1157520] Fps is (10 sec: 69222.8, 60 sec: 75366.4, 300 sec: 75588.5). Total num frames: 769945600. Throughput: 0: 18662.2. Samples: 182475700. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:20:20,466][1157520] Avg episode reward: [(0, '56.103')] [2024-09-30 03:20:20,595][1157819] Updated weights for policy 0, policy_version 187978 (0.0006) [2024-09-30 03:20:21,183][1157819] Updated weights for policy 0, policy_version 187988 (0.0006) [2024-09-30 03:20:21,799][1157819] Updated weights for policy 0, policy_version 187998 (0.0006) [2024-09-30 03:20:22,388][1157819] Updated weights for policy 0, policy_version 188008 (0.0006) [2024-09-30 03:20:22,983][1157819] Updated weights for policy 0, policy_version 188018 (0.0006) [2024-09-30 03:20:23,575][1157819] Updated weights for policy 0, policy_version 188028 (0.0006) [2024-09-30 03:20:24,177][1157819] Updated weights for policy 0, policy_version 188038 (0.0006) [2024-09-30 03:20:24,755][1157819] Updated weights for policy 0, policy_version 188048 (0.0006) [2024-09-30 03:20:25,368][1157819] Updated weights for policy 0, policy_version 188058 (0.0006) [2024-09-30 03:20:25,466][1157520] Fps is (10 sec: 69221.8, 60 sec: 74820.2, 300 sec: 75394.1). Total num frames: 770293760. Throughput: 0: 18468.6. Samples: 182526944. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:20:25,466][1157520] Avg episode reward: [(0, '56.394')] [2024-09-30 03:20:25,929][1157819] Updated weights for policy 0, policy_version 188068 (0.0006) [2024-09-30 03:20:26,533][1157819] Updated weights for policy 0, policy_version 188078 (0.0006) [2024-09-30 03:20:27,092][1157819] Updated weights for policy 0, policy_version 188088 (0.0006) [2024-09-30 03:20:27,644][1157819] Updated weights for policy 0, policy_version 188098 (0.0006) [2024-09-30 03:20:28,200][1157819] Updated weights for policy 0, policy_version 188108 (0.0006) [2024-09-30 03:20:28,749][1157819] Updated weights for policy 0, policy_version 188118 (0.0006) [2024-09-30 03:20:29,306][1157819] Updated weights for policy 0, policy_version 188128 (0.0006) [2024-09-30 03:20:29,867][1157819] Updated weights for policy 0, policy_version 188138 (0.0006) [2024-09-30 03:20:30,431][1157819] Updated weights for policy 0, policy_version 188148 (0.0006) [2024-09-30 03:20:30,466][1157520] Fps is (10 sec: 70860.8, 60 sec: 74274.1, 300 sec: 75241.4). Total num frames: 770654208. Throughput: 0: 18225.0. Samples: 182633604. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:20:30,466][1157520] Avg episode reward: [(0, '57.273')] [2024-09-30 03:20:30,983][1157819] Updated weights for policy 0, policy_version 188158 (0.0006) [2024-09-30 03:20:31,538][1157819] Updated weights for policy 0, policy_version 188168 (0.0006) [2024-09-30 03:20:32,069][1157819] Updated weights for policy 0, policy_version 188178 (0.0006) [2024-09-30 03:20:32,640][1157819] Updated weights for policy 0, policy_version 188188 (0.0006) [2024-09-30 03:20:33,210][1157819] Updated weights for policy 0, policy_version 188198 (0.0006) [2024-09-30 03:20:33,784][1157819] Updated weights for policy 0, policy_version 188208 (0.0006) [2024-09-30 03:20:34,337][1157819] Updated weights for policy 0, policy_version 188218 (0.0006) [2024-09-30 03:20:34,882][1157819] Updated weights for policy 0, policy_version 188228 (0.0006) [2024-09-30 03:20:35,457][1157819] Updated weights for policy 0, policy_version 188238 (0.0006) [2024-09-30 03:20:35,466][1157520] Fps is (10 sec: 72909.7, 60 sec: 73796.4, 300 sec: 75213.7). Total num frames: 771022848. Throughput: 0: 18134.1. Samples: 182743372. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:20:35,466][1157520] Avg episode reward: [(0, '53.995')] [2024-09-30 03:20:35,985][1157819] Updated weights for policy 0, policy_version 188248 (0.0006) [2024-09-30 03:20:36,568][1157819] Updated weights for policy 0, policy_version 188258 (0.0006) [2024-09-30 03:20:37,150][1157819] Updated weights for policy 0, policy_version 188268 (0.0006) [2024-09-30 03:20:37,720][1157819] Updated weights for policy 0, policy_version 188278 (0.0006) [2024-09-30 03:20:38,212][1157819] Updated weights for policy 0, policy_version 188288 (0.0006) [2024-09-30 03:20:38,750][1157819] Updated weights for policy 0, policy_version 188298 (0.0006) [2024-09-30 03:20:39,257][1157819] Updated weights for policy 0, policy_version 188308 (0.0006) [2024-09-30 03:20:39,819][1157819] Updated weights for policy 0, policy_version 188318 (0.0006) [2024-09-30 03:20:40,393][1157819] Updated weights for policy 0, policy_version 188328 (0.0006) [2024-09-30 03:20:40,466][1157520] Fps is (10 sec: 74547.7, 60 sec: 73455.1, 300 sec: 75199.8). Total num frames: 771399680. Throughput: 0: 18094.4. Samples: 182798256. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:20:40,466][1157520] Avg episode reward: [(0, '55.320')] [2024-09-30 03:20:40,945][1157819] Updated weights for policy 0, policy_version 188338 (0.0006) [2024-09-30 03:20:41,522][1157819] Updated weights for policy 0, policy_version 188348 (0.0006) [2024-09-30 03:20:42,023][1157819] Updated weights for policy 0, policy_version 188358 (0.0006) [2024-09-30 03:20:42,590][1157819] Updated weights for policy 0, policy_version 188368 (0.0006) [2024-09-30 03:20:43,139][1157819] Updated weights for policy 0, policy_version 188378 (0.0006) [2024-09-30 03:20:43,703][1157819] Updated weights for policy 0, policy_version 188388 (0.0006) [2024-09-30 03:20:44,248][1157819] Updated weights for policy 0, policy_version 188398 (0.0006) [2024-09-30 03:20:44,768][1157819] Updated weights for policy 0, policy_version 188408 (0.0006) [2024-09-30 03:20:45,348][1157819] Updated weights for policy 0, policy_version 188418 (0.0006) [2024-09-30 03:20:45,466][1157520] Fps is (10 sec: 74137.4, 60 sec: 73113.8, 300 sec: 75130.4). Total num frames: 771764224. Throughput: 0: 18187.0. Samples: 182911176. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:20:45,466][1157520] Avg episode reward: [(0, '55.117')] [2024-09-30 03:20:45,882][1157819] Updated weights for policy 0, policy_version 188428 (0.0006) [2024-09-30 03:20:46,474][1157819] Updated weights for policy 0, policy_version 188438 (0.0006) [2024-09-30 03:20:47,021][1157819] Updated weights for policy 0, policy_version 188448 (0.0006) [2024-09-30 03:20:47,594][1157819] Updated weights for policy 0, policy_version 188458 (0.0006) [2024-09-30 03:20:48,136][1157819] Updated weights for policy 0, policy_version 188468 (0.0006) [2024-09-30 03:20:48,716][1157819] Updated weights for policy 0, policy_version 188478 (0.0006) [2024-09-30 03:20:49,242][1157819] Updated weights for policy 0, policy_version 188488 (0.0006) [2024-09-30 03:20:49,488][1157736] Signal inference workers to stop experience collection... (12900 times) [2024-09-30 03:20:49,489][1157736] Signal inference workers to resume experience collection... (12900 times) [2024-09-30 03:20:49,493][1157819] InferenceWorker_p0-w0: stopping experience collection (12900 times) [2024-09-30 03:20:49,495][1157819] InferenceWorker_p0-w0: resuming experience collection (12900 times) [2024-09-30 03:20:49,787][1157819] Updated weights for policy 0, policy_version 188498 (0.0006) [2024-09-30 03:20:50,365][1157819] Updated weights for policy 0, policy_version 188508 (0.0006) [2024-09-30 03:20:50,466][1157520] Fps is (10 sec: 73317.9, 60 sec: 72908.9, 300 sec: 75033.2). Total num frames: 772132864. Throughput: 0: 18066.2. Samples: 183021520. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:20:50,466][1157520] Avg episode reward: [(0, '55.123')] [2024-09-30 03:20:50,913][1157819] Updated weights for policy 0, policy_version 188518 (0.0006) [2024-09-30 03:20:51,485][1157819] Updated weights for policy 0, policy_version 188528 (0.0006) [2024-09-30 03:20:52,021][1157819] Updated weights for policy 0, policy_version 188538 (0.0006) [2024-09-30 03:20:52,583][1157819] Updated weights for policy 0, policy_version 188548 (0.0006) [2024-09-30 03:20:53,131][1157819] Updated weights for policy 0, policy_version 188558 (0.0006) [2024-09-30 03:20:53,676][1157819] Updated weights for policy 0, policy_version 188568 (0.0006) [2024-09-30 03:20:54,220][1157819] Updated weights for policy 0, policy_version 188578 (0.0006) [2024-09-30 03:20:54,795][1157819] Updated weights for policy 0, policy_version 188588 (0.0006) [2024-09-30 03:20:55,345][1157819] Updated weights for policy 0, policy_version 188598 (0.0006) [2024-09-30 03:20:55,466][1157520] Fps is (10 sec: 74137.7, 60 sec: 72908.8, 300 sec: 74936.0). Total num frames: 772505600. Throughput: 0: 18043.0. Samples: 183076948. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:20:55,466][1157520] Avg episode reward: [(0, '57.153')] [2024-09-30 03:20:55,875][1157819] Updated weights for policy 0, policy_version 188608 (0.0006) [2024-09-30 03:20:56,451][1157819] Updated weights for policy 0, policy_version 188618 (0.0006) [2024-09-30 03:20:56,986][1157819] Updated weights for policy 0, policy_version 188628 (0.0006) [2024-09-30 03:20:57,557][1157819] Updated weights for policy 0, policy_version 188638 (0.0006) [2024-09-30 03:20:58,090][1157819] Updated weights for policy 0, policy_version 188648 (0.0006) [2024-09-30 03:20:58,620][1157819] Updated weights for policy 0, policy_version 188658 (0.0006) [2024-09-30 03:20:59,168][1157819] Updated weights for policy 0, policy_version 188668 (0.0006) [2024-09-30 03:20:59,706][1157819] Updated weights for policy 0, policy_version 188678 (0.0006) [2024-09-30 03:21:00,251][1157819] Updated weights for policy 0, policy_version 188688 (0.0006) [2024-09-30 03:21:00,466][1157520] Fps is (10 sec: 74547.3, 60 sec: 73113.6, 300 sec: 74936.0). Total num frames: 772878336. Throughput: 0: 18162.3. Samples: 183188964. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:21:00,466][1157520] Avg episode reward: [(0, '56.374')] [2024-09-30 03:21:00,799][1157819] Updated weights for policy 0, policy_version 188698 (0.0006) [2024-09-30 03:21:01,374][1157819] Updated weights for policy 0, policy_version 188708 (0.0006) [2024-09-30 03:21:01,931][1157819] Updated weights for policy 0, policy_version 188718 (0.0006) [2024-09-30 03:21:02,487][1157819] Updated weights for policy 0, policy_version 188728 (0.0006) [2024-09-30 03:21:03,043][1157819] Updated weights for policy 0, policy_version 188738 (0.0006) [2024-09-30 03:21:03,615][1157819] Updated weights for policy 0, policy_version 188748 (0.0006) [2024-09-30 03:21:04,167][1157819] Updated weights for policy 0, policy_version 188758 (0.0006) [2024-09-30 03:21:04,741][1157819] Updated weights for policy 0, policy_version 188768 (0.0006) [2024-09-30 03:21:05,278][1157819] Updated weights for policy 0, policy_version 188778 (0.0006) [2024-09-30 03:21:05,466][1157520] Fps is (10 sec: 74137.3, 60 sec: 72772.4, 300 sec: 74977.6). Total num frames: 773246976. Throughput: 0: 18305.8. Samples: 183299464. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:21:05,466][1157520] Avg episode reward: [(0, '57.144')] [2024-09-30 03:21:05,846][1157819] Updated weights for policy 0, policy_version 188788 (0.0006) [2024-09-30 03:21:06,399][1157819] Updated weights for policy 0, policy_version 188798 (0.0006) [2024-09-30 03:21:06,960][1157819] Updated weights for policy 0, policy_version 188808 (0.0006) [2024-09-30 03:21:07,527][1157819] Updated weights for policy 0, policy_version 188818 (0.0006) [2024-09-30 03:21:08,095][1157819] Updated weights for policy 0, policy_version 188828 (0.0006) [2024-09-30 03:21:08,630][1157819] Updated weights for policy 0, policy_version 188838 (0.0006) [2024-09-30 03:21:09,152][1157819] Updated weights for policy 0, policy_version 188848 (0.0006) [2024-09-30 03:21:09,737][1157819] Updated weights for policy 0, policy_version 188858 (0.0006) [2024-09-30 03:21:10,262][1157819] Updated weights for policy 0, policy_version 188868 (0.0006) [2024-09-30 03:21:10,466][1157520] Fps is (10 sec: 73318.3, 60 sec: 72635.8, 300 sec: 74908.2). Total num frames: 773611520. Throughput: 0: 18392.1. Samples: 183354584. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:21:10,466][1157520] Avg episode reward: [(0, '57.194')] [2024-09-30 03:21:10,862][1157819] Updated weights for policy 0, policy_version 188878 (0.0006) [2024-09-30 03:21:11,405][1157819] Updated weights for policy 0, policy_version 188888 (0.0006) [2024-09-30 03:21:11,978][1157819] Updated weights for policy 0, policy_version 188898 (0.0006) [2024-09-30 03:21:12,523][1157819] Updated weights for policy 0, policy_version 188908 (0.0006) [2024-09-30 03:21:13,100][1157819] Updated weights for policy 0, policy_version 188918 (0.0006) [2024-09-30 03:21:13,639][1157819] Updated weights for policy 0, policy_version 188928 (0.0006) [2024-09-30 03:21:14,209][1157819] Updated weights for policy 0, policy_version 188938 (0.0006) [2024-09-30 03:21:14,765][1157819] Updated weights for policy 0, policy_version 188948 (0.0006) [2024-09-30 03:21:15,318][1157819] Updated weights for policy 0, policy_version 188958 (0.0006) [2024-09-30 03:21:15,466][1157520] Fps is (10 sec: 73318.6, 60 sec: 72977.1, 300 sec: 74824.9). Total num frames: 773980160. Throughput: 0: 18468.3. Samples: 183464676. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:21:15,466][1157520] Avg episode reward: [(0, '55.474')] [2024-09-30 03:21:15,891][1157819] Updated weights for policy 0, policy_version 188968 (0.0006) [2024-09-30 03:21:16,464][1157819] Updated weights for policy 0, policy_version 188978 (0.0006) [2024-09-30 03:21:17,031][1157819] Updated weights for policy 0, policy_version 188988 (0.0006) [2024-09-30 03:21:17,593][1157819] Updated weights for policy 0, policy_version 188998 (0.0006) [2024-09-30 03:21:18,138][1157819] Updated weights for policy 0, policy_version 189008 (0.0006) [2024-09-30 03:21:18,723][1157819] Updated weights for policy 0, policy_version 189018 (0.0006) [2024-09-30 03:21:19,230][1157819] Updated weights for policy 0, policy_version 189028 (0.0006) [2024-09-30 03:21:19,832][1157819] Updated weights for policy 0, policy_version 189038 (0.0006) [2024-09-30 03:21:20,348][1157819] Updated weights for policy 0, policy_version 189048 (0.0006) [2024-09-30 03:21:20,466][1157520] Fps is (10 sec: 72907.9, 60 sec: 73250.0, 300 sec: 74866.5). Total num frames: 774340608. Throughput: 0: 18465.0. Samples: 183574300. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:21:20,466][1157520] Avg episode reward: [(0, '57.126')] [2024-09-30 03:21:20,962][1157819] Updated weights for policy 0, policy_version 189058 (0.0006) [2024-09-30 03:21:21,469][1157819] Updated weights for policy 0, policy_version 189068 (0.0006) [2024-09-30 03:21:22,060][1157819] Updated weights for policy 0, policy_version 189078 (0.0006) [2024-09-30 03:21:22,617][1157819] Updated weights for policy 0, policy_version 189088 (0.0006) [2024-09-30 03:21:23,172][1157819] Updated weights for policy 0, policy_version 189098 (0.0006) [2024-09-30 03:21:23,747][1157819] Updated weights for policy 0, policy_version 189108 (0.0006) [2024-09-30 03:21:24,275][1157819] Updated weights for policy 0, policy_version 189118 (0.0006) [2024-09-30 03:21:24,823][1157819] Updated weights for policy 0, policy_version 189128 (0.0006) [2024-09-30 03:21:25,389][1157819] Updated weights for policy 0, policy_version 189138 (0.0006) [2024-09-30 03:21:25,466][1157520] Fps is (10 sec: 73727.8, 60 sec: 73728.1, 300 sec: 75005.4). Total num frames: 774717440. Throughput: 0: 18463.2. Samples: 183629104. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:21:25,466][1157520] Avg episode reward: [(0, '57.053')] [2024-09-30 03:21:25,938][1157819] Updated weights for policy 0, policy_version 189148 (0.0006) [2024-09-30 03:21:26,466][1157819] Updated weights for policy 0, policy_version 189158 (0.0006) [2024-09-30 03:21:27,011][1157819] Updated weights for policy 0, policy_version 189168 (0.0006) [2024-09-30 03:21:27,551][1157819] Updated weights for policy 0, policy_version 189178 (0.0006) [2024-09-30 03:21:28,128][1157819] Updated weights for policy 0, policy_version 189188 (0.0006) [2024-09-30 03:21:28,674][1157819] Updated weights for policy 0, policy_version 189198 (0.0006) [2024-09-30 03:21:29,220][1157819] Updated weights for policy 0, policy_version 189208 (0.0006) [2024-09-30 03:21:29,736][1157819] Updated weights for policy 0, policy_version 189218 (0.0006) [2024-09-30 03:21:30,284][1157819] Updated weights for policy 0, policy_version 189228 (0.0006) [2024-09-30 03:21:30,466][1157520] Fps is (10 sec: 74547.5, 60 sec: 73864.4, 300 sec: 74991.5). Total num frames: 775086080. Throughput: 0: 18439.0. Samples: 183740932. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:21:30,466][1157520] Avg episode reward: [(0, '55.143')] [2024-09-30 03:21:30,852][1157819] Updated weights for policy 0, policy_version 189238 (0.0006) [2024-09-30 03:21:31,423][1157736] Signal inference workers to stop experience collection... (12950 times) [2024-09-30 03:21:31,424][1157736] Signal inference workers to resume experience collection... (12950 times) [2024-09-30 03:21:31,428][1157819] InferenceWorker_p0-w0: stopping experience collection (12950 times) [2024-09-30 03:21:31,428][1157819] InferenceWorker_p0-w0: resuming experience collection (12950 times) [2024-09-30 03:21:31,438][1157819] Updated weights for policy 0, policy_version 189248 (0.0006) [2024-09-30 03:21:32,015][1157819] Updated weights for policy 0, policy_version 189258 (0.0006) [2024-09-30 03:21:32,608][1157819] Updated weights for policy 0, policy_version 189268 (0.0006) [2024-09-30 03:21:33,190][1157819] Updated weights for policy 0, policy_version 189278 (0.0006) [2024-09-30 03:21:33,777][1157819] Updated weights for policy 0, policy_version 189288 (0.0006) [2024-09-30 03:21:34,380][1157819] Updated weights for policy 0, policy_version 189298 (0.0006) [2024-09-30 03:21:34,946][1157819] Updated weights for policy 0, policy_version 189308 (0.0006) [2024-09-30 03:21:35,466][1157520] Fps is (10 sec: 72088.8, 60 sec: 73591.3, 300 sec: 74922.0). Total num frames: 775438336. Throughput: 0: 18374.5. Samples: 183848376. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:21:35,466][1157520] Avg episode reward: [(0, '56.620')] [2024-09-30 03:21:35,540][1157819] Updated weights for policy 0, policy_version 189318 (0.0006) [2024-09-30 03:21:36,119][1157819] Updated weights for policy 0, policy_version 189328 (0.0006) [2024-09-30 03:21:36,687][1157819] Updated weights for policy 0, policy_version 189338 (0.0006) [2024-09-30 03:21:37,277][1157819] Updated weights for policy 0, policy_version 189348 (0.0006) [2024-09-30 03:21:37,853][1157819] Updated weights for policy 0, policy_version 189358 (0.0006) [2024-09-30 03:21:38,441][1157819] Updated weights for policy 0, policy_version 189368 (0.0006) [2024-09-30 03:21:39,037][1157819] Updated weights for policy 0, policy_version 189378 (0.0006) [2024-09-30 03:21:39,634][1157819] Updated weights for policy 0, policy_version 189388 (0.0006) [2024-09-30 03:21:40,181][1157819] Updated weights for policy 0, policy_version 189398 (0.0006) [2024-09-30 03:21:40,466][1157520] Fps is (10 sec: 70451.6, 60 sec: 73181.8, 300 sec: 74908.2). Total num frames: 775790592. Throughput: 0: 18320.4. Samples: 183901368. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:21:40,466][1157520] Avg episode reward: [(0, '56.530')] [2024-09-30 03:21:40,774][1157819] Updated weights for policy 0, policy_version 189408 (0.0006) [2024-09-30 03:21:41,353][1157819] Updated weights for policy 0, policy_version 189418 (0.0006) [2024-09-30 03:21:41,953][1157819] Updated weights for policy 0, policy_version 189428 (0.0006) [2024-09-30 03:21:42,516][1157819] Updated weights for policy 0, policy_version 189438 (0.0006) [2024-09-30 03:21:43,120][1157819] Updated weights for policy 0, policy_version 189448 (0.0006) [2024-09-30 03:21:43,658][1157819] Updated weights for policy 0, policy_version 189458 (0.0006) [2024-09-30 03:21:44,168][1157819] Updated weights for policy 0, policy_version 189468 (0.0006) [2024-09-30 03:21:44,718][1157819] Updated weights for policy 0, policy_version 189478 (0.0006) [2024-09-30 03:21:45,260][1157819] Updated weights for policy 0, policy_version 189488 (0.0006) [2024-09-30 03:21:45,466][1157520] Fps is (10 sec: 71680.0, 60 sec: 73181.7, 300 sec: 74977.6). Total num frames: 776155136. Throughput: 0: 18189.2. Samples: 184007480. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:21:45,466][1157520] Avg episode reward: [(0, '56.415')] [2024-09-30 03:21:45,827][1157819] Updated weights for policy 0, policy_version 189498 (0.0006) [2024-09-30 03:21:46,343][1157819] Updated weights for policy 0, policy_version 189508 (0.0006) [2024-09-30 03:21:46,863][1157819] Updated weights for policy 0, policy_version 189518 (0.0006) [2024-09-30 03:21:47,383][1157819] Updated weights for policy 0, policy_version 189528 (0.0006) [2024-09-30 03:21:47,916][1157819] Updated weights for policy 0, policy_version 189538 (0.0006) [2024-09-30 03:21:48,442][1157819] Updated weights for policy 0, policy_version 189548 (0.0006) [2024-09-30 03:21:48,959][1157819] Updated weights for policy 0, policy_version 189558 (0.0006) [2024-09-30 03:21:49,461][1157819] Updated weights for policy 0, policy_version 189568 (0.0006) [2024-09-30 03:21:49,988][1157819] Updated weights for policy 0, policy_version 189578 (0.0006) [2024-09-30 03:21:50,466][1157520] Fps is (10 sec: 75775.3, 60 sec: 73591.3, 300 sec: 75199.8). Total num frames: 776548352. Throughput: 0: 18323.4. Samples: 184124016. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:21:50,466][1157520] Avg episode reward: [(0, '54.089')] [2024-09-30 03:21:50,506][1157819] Updated weights for policy 0, policy_version 189588 (0.0006) [2024-09-30 03:21:51,015][1157819] Updated weights for policy 0, policy_version 189598 (0.0006) [2024-09-30 03:21:51,560][1157819] Updated weights for policy 0, policy_version 189608 (0.0006) [2024-09-30 03:21:52,055][1157819] Updated weights for policy 0, policy_version 189618 (0.0006) [2024-09-30 03:21:52,571][1157819] Updated weights for policy 0, policy_version 189628 (0.0006) [2024-09-30 03:21:53,084][1157819] Updated weights for policy 0, policy_version 189638 (0.0006) [2024-09-30 03:21:53,647][1157819] Updated weights for policy 0, policy_version 189648 (0.0006) [2024-09-30 03:21:54,161][1157819] Updated weights for policy 0, policy_version 189658 (0.0006) [2024-09-30 03:21:54,678][1157819] Updated weights for policy 0, policy_version 189668 (0.0006) [2024-09-30 03:21:55,189][1157819] Updated weights for policy 0, policy_version 189678 (0.0006) [2024-09-30 03:21:55,466][1157520] Fps is (10 sec: 78644.6, 60 sec: 73932.8, 300 sec: 75297.0). Total num frames: 776941568. Throughput: 0: 18407.6. Samples: 184182924. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:21:55,466][1157520] Avg episode reward: [(0, '55.385')] [2024-09-30 03:21:55,715][1157819] Updated weights for policy 0, policy_version 189688 (0.0006) [2024-09-30 03:21:56,211][1157819] Updated weights for policy 0, policy_version 189698 (0.0006) [2024-09-30 03:21:56,720][1157819] Updated weights for policy 0, policy_version 189708 (0.0006) [2024-09-30 03:21:57,268][1157819] Updated weights for policy 0, policy_version 189718 (0.0006) [2024-09-30 03:21:57,809][1157819] Updated weights for policy 0, policy_version 189728 (0.0006) [2024-09-30 03:21:58,365][1157819] Updated weights for policy 0, policy_version 189738 (0.0006) [2024-09-30 03:21:58,928][1157819] Updated weights for policy 0, policy_version 189748 (0.0006) [2024-09-30 03:21:59,518][1157819] Updated weights for policy 0, policy_version 189758 (0.0006) [2024-09-30 03:22:00,069][1157819] Updated weights for policy 0, policy_version 189768 (0.0006) [2024-09-30 03:22:00,466][1157520] Fps is (10 sec: 76596.3, 60 sec: 73932.8, 300 sec: 75269.2). Total num frames: 777314304. Throughput: 0: 18554.8. Samples: 184299640. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:22:00,466][1157520] Avg episode reward: [(0, '55.617')] [2024-09-30 03:22:00,661][1157819] Updated weights for policy 0, policy_version 189778 (0.0007) [2024-09-30 03:22:01,174][1157819] Updated weights for policy 0, policy_version 189788 (0.0006) [2024-09-30 03:22:01,799][1157819] Updated weights for policy 0, policy_version 189798 (0.0006) [2024-09-30 03:22:02,309][1157819] Updated weights for policy 0, policy_version 189808 (0.0006) [2024-09-30 03:22:02,893][1157819] Updated weights for policy 0, policy_version 189818 (0.0006) [2024-09-30 03:22:03,413][1157819] Updated weights for policy 0, policy_version 189828 (0.0006) [2024-09-30 03:22:03,931][1157819] Updated weights for policy 0, policy_version 189838 (0.0006) [2024-09-30 03:22:04,460][1157819] Updated weights for policy 0, policy_version 189848 (0.0006) [2024-09-30 03:22:05,003][1157819] Updated weights for policy 0, policy_version 189858 (0.0006) [2024-09-30 03:22:05,466][1157520] Fps is (10 sec: 75365.2, 60 sec: 74137.5, 300 sec: 75241.4). Total num frames: 777695232. Throughput: 0: 18584.3. Samples: 184410592. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:22:05,466][1157520] Avg episode reward: [(0, '53.892')] [2024-09-30 03:22:05,472][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000189867_777695232.pth... [2024-09-30 03:22:05,516][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000185467_759672832.pth [2024-09-30 03:22:05,537][1157819] Updated weights for policy 0, policy_version 189868 (0.0006) [2024-09-30 03:22:06,008][1157819] Updated weights for policy 0, policy_version 189878 (0.0006) [2024-09-30 03:22:06,532][1157819] Updated weights for policy 0, policy_version 189888 (0.0006) [2024-09-30 03:22:07,045][1157819] Updated weights for policy 0, policy_version 189898 (0.0006) [2024-09-30 03:22:07,542][1157819] Updated weights for policy 0, policy_version 189908 (0.0006) [2024-09-30 03:22:08,031][1157819] Updated weights for policy 0, policy_version 189918 (0.0006) [2024-09-30 03:22:08,525][1157819] Updated weights for policy 0, policy_version 189928 (0.0006) [2024-09-30 03:22:09,027][1157819] Updated weights for policy 0, policy_version 189938 (0.0006) [2024-09-30 03:22:09,536][1157819] Updated weights for policy 0, policy_version 189948 (0.0006) [2024-09-30 03:22:10,019][1157819] Updated weights for policy 0, policy_version 189958 (0.0007) [2024-09-30 03:22:10,466][1157520] Fps is (10 sec: 79052.3, 60 sec: 74888.5, 300 sec: 75241.4). Total num frames: 778104832. Throughput: 0: 18704.0. Samples: 184470784. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:22:10,466][1157520] Avg episode reward: [(0, '55.925')] [2024-09-30 03:22:10,506][1157819] Updated weights for policy 0, policy_version 189968 (0.0007) [2024-09-30 03:22:10,988][1157819] Updated weights for policy 0, policy_version 189978 (0.0006) [2024-09-30 03:22:11,526][1157819] Updated weights for policy 0, policy_version 189988 (0.0006) [2024-09-30 03:22:11,680][1157736] Signal inference workers to stop experience collection... (13000 times) [2024-09-30 03:22:11,680][1157736] Signal inference workers to resume experience collection... (13000 times) [2024-09-30 03:22:11,684][1157819] InferenceWorker_p0-w0: stopping experience collection (13000 times) [2024-09-30 03:22:11,684][1157819] InferenceWorker_p0-w0: resuming experience collection (13000 times) [2024-09-30 03:22:12,031][1157819] Updated weights for policy 0, policy_version 189998 (0.0006) [2024-09-30 03:22:12,593][1157819] Updated weights for policy 0, policy_version 190008 (0.0006) [2024-09-30 03:22:13,114][1157819] Updated weights for policy 0, policy_version 190018 (0.0006) [2024-09-30 03:22:13,637][1157819] Updated weights for policy 0, policy_version 190028 (0.0006) [2024-09-30 03:22:14,192][1157819] Updated weights for policy 0, policy_version 190038 (0.0006) [2024-09-30 03:22:14,707][1157819] Updated weights for policy 0, policy_version 190048 (0.0006) [2024-09-30 03:22:15,255][1157819] Updated weights for policy 0, policy_version 190058 (0.0006) [2024-09-30 03:22:15,466][1157520] Fps is (10 sec: 79463.4, 60 sec: 75161.6, 300 sec: 75172.0). Total num frames: 778489856. Throughput: 0: 18898.4. Samples: 184591360. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:22:15,466][1157520] Avg episode reward: [(0, '57.574')] [2024-09-30 03:22:15,799][1157819] Updated weights for policy 0, policy_version 190068 (0.0006) [2024-09-30 03:22:16,333][1157819] Updated weights for policy 0, policy_version 190078 (0.0006) [2024-09-30 03:22:16,919][1157819] Updated weights for policy 0, policy_version 190088 (0.0006) [2024-09-30 03:22:17,517][1157819] Updated weights for policy 0, policy_version 190098 (0.0006) [2024-09-30 03:22:18,069][1157819] Updated weights for policy 0, policy_version 190108 (0.0006) [2024-09-30 03:22:18,645][1157819] Updated weights for policy 0, policy_version 190118 (0.0006) [2024-09-30 03:22:19,240][1157819] Updated weights for policy 0, policy_version 190128 (0.0006) [2024-09-30 03:22:19,786][1157819] Updated weights for policy 0, policy_version 190138 (0.0006) [2024-09-30 03:22:20,324][1157819] Updated weights for policy 0, policy_version 190148 (0.0006) [2024-09-30 03:22:20,466][1157520] Fps is (10 sec: 74956.9, 60 sec: 75230.0, 300 sec: 75102.6). Total num frames: 778854400. Throughput: 0: 18968.3. Samples: 184701948. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:22:20,466][1157520] Avg episode reward: [(0, '54.821')] [2024-09-30 03:22:20,914][1157819] Updated weights for policy 0, policy_version 190158 (0.0006) [2024-09-30 03:22:21,470][1157819] Updated weights for policy 0, policy_version 190168 (0.0006) [2024-09-30 03:22:22,041][1157819] Updated weights for policy 0, policy_version 190178 (0.0006) [2024-09-30 03:22:22,610][1157819] Updated weights for policy 0, policy_version 190188 (0.0006) [2024-09-30 03:22:23,190][1157819] Updated weights for policy 0, policy_version 190198 (0.0006) [2024-09-30 03:22:23,754][1157819] Updated weights for policy 0, policy_version 190208 (0.0006) [2024-09-30 03:22:24,391][1157819] Updated weights for policy 0, policy_version 190218 (0.0006) [2024-09-30 03:22:25,026][1157819] Updated weights for policy 0, policy_version 190228 (0.0006) [2024-09-30 03:22:25,466][1157520] Fps is (10 sec: 71270.3, 60 sec: 74752.0, 300 sec: 75047.0). Total num frames: 779202560. Throughput: 0: 18983.3. Samples: 184755616. Policy #0 lag: (min: 1.0, avg: 2.3, max: 6.0) [2024-09-30 03:22:25,466][1157520] Avg episode reward: [(0, '55.586')] [2024-09-30 03:22:25,615][1157819] Updated weights for policy 0, policy_version 190238 (0.0006) [2024-09-30 03:22:26,202][1157819] Updated weights for policy 0, policy_version 190248 (0.0006) [2024-09-30 03:22:26,785][1157819] Updated weights for policy 0, policy_version 190258 (0.0006) [2024-09-30 03:22:27,339][1157819] Updated weights for policy 0, policy_version 190268 (0.0006) [2024-09-30 03:22:27,943][1157819] Updated weights for policy 0, policy_version 190278 (0.0006) [2024-09-30 03:22:28,519][1157819] Updated weights for policy 0, policy_version 190288 (0.0006) [2024-09-30 03:22:29,076][1157819] Updated weights for policy 0, policy_version 190298 (0.0006) [2024-09-30 03:22:29,604][1157819] Updated weights for policy 0, policy_version 190308 (0.0006) [2024-09-30 03:22:30,201][1157819] Updated weights for policy 0, policy_version 190318 (0.0006) [2024-09-30 03:22:30,466][1157520] Fps is (10 sec: 70451.4, 60 sec: 74547.3, 300 sec: 74977.6). Total num frames: 779558912. Throughput: 0: 18952.2. Samples: 184860324. Policy #0 lag: (min: 1.0, avg: 2.3, max: 6.0) [2024-09-30 03:22:30,466][1157520] Avg episode reward: [(0, '56.755')] [2024-09-30 03:22:30,878][1157819] Updated weights for policy 0, policy_version 190328 (0.0006) [2024-09-30 03:22:31,542][1157819] Updated weights for policy 0, policy_version 190338 (0.0006) [2024-09-30 03:22:32,171][1157819] Updated weights for policy 0, policy_version 190348 (0.0006) [2024-09-30 03:22:32,788][1157819] Updated weights for policy 0, policy_version 190358 (0.0006) [2024-09-30 03:22:33,430][1157819] Updated weights for policy 0, policy_version 190368 (0.0006) [2024-09-30 03:22:34,047][1157819] Updated weights for policy 0, policy_version 190378 (0.0006) [2024-09-30 03:22:34,668][1157819] Updated weights for policy 0, policy_version 190388 (0.0006) [2024-09-30 03:22:35,307][1157819] Updated weights for policy 0, policy_version 190398 (0.0006) [2024-09-30 03:22:35,466][1157520] Fps is (10 sec: 67584.1, 60 sec: 74001.3, 300 sec: 74727.7). Total num frames: 779878400. Throughput: 0: 18559.1. Samples: 184959172. Policy #0 lag: (min: 1.0, avg: 2.3, max: 6.0) [2024-09-30 03:22:35,466][1157520] Avg episode reward: [(0, '57.392')] [2024-09-30 03:22:35,937][1157819] Updated weights for policy 0, policy_version 190408 (0.0006) [2024-09-30 03:22:36,570][1157819] Updated weights for policy 0, policy_version 190418 (0.0006) [2024-09-30 03:22:37,177][1157819] Updated weights for policy 0, policy_version 190428 (0.0006) [2024-09-30 03:22:37,835][1157819] Updated weights for policy 0, policy_version 190438 (0.0006) [2024-09-30 03:22:38,228][1157736] Signal inference workers to stop experience collection... (13050 times) [2024-09-30 03:22:38,229][1157736] Signal inference workers to resume experience collection... (13050 times) [2024-09-30 03:22:38,233][1157819] InferenceWorker_p0-w0: stopping experience collection (13050 times) [2024-09-30 03:22:38,233][1157819] InferenceWorker_p0-w0: resuming experience collection (13050 times) [2024-09-30 03:22:38,459][1157819] Updated weights for policy 0, policy_version 190448 (0.0006) [2024-09-30 03:22:39,080][1157819] Updated weights for policy 0, policy_version 190458 (0.0006) [2024-09-30 03:22:39,716][1157819] Updated weights for policy 0, policy_version 190468 (0.0006) [2024-09-30 03:22:40,363][1157819] Updated weights for policy 0, policy_version 190478 (0.0006) [2024-09-30 03:22:40,466][1157520] Fps is (10 sec: 64306.4, 60 sec: 73523.1, 300 sec: 74547.2). Total num frames: 780201984. Throughput: 0: 18329.4. Samples: 185007748. Policy #0 lag: (min: 1.0, avg: 2.3, max: 6.0) [2024-09-30 03:22:40,466][1157520] Avg episode reward: [(0, '53.949')] [2024-09-30 03:22:40,973][1157819] Updated weights for policy 0, policy_version 190488 (0.0006) [2024-09-30 03:22:41,627][1157819] Updated weights for policy 0, policy_version 190498 (0.0006) [2024-09-30 03:22:42,262][1157819] Updated weights for policy 0, policy_version 190508 (0.0006) [2024-09-30 03:22:42,908][1157819] Updated weights for policy 0, policy_version 190518 (0.0006) [2024-09-30 03:22:43,570][1157819] Updated weights for policy 0, policy_version 190528 (0.0006) [2024-09-30 03:22:44,232][1157819] Updated weights for policy 0, policy_version 190538 (0.0006) [2024-09-30 03:22:44,825][1157819] Updated weights for policy 0, policy_version 190548 (0.0006) [2024-09-30 03:22:45,458][1157819] Updated weights for policy 0, policy_version 190558 (0.0006) [2024-09-30 03:22:45,466][1157520] Fps is (10 sec: 64716.9, 60 sec: 72840.7, 300 sec: 74505.6). Total num frames: 780525568. Throughput: 0: 17885.4. Samples: 185104484. Policy #0 lag: (min: 1.0, avg: 2.3, max: 6.0) [2024-09-30 03:22:45,466][1157520] Avg episode reward: [(0, '57.205')] [2024-09-30 03:22:46,122][1157819] Updated weights for policy 0, policy_version 190568 (0.0006) [2024-09-30 03:22:46,730][1157819] Updated weights for policy 0, policy_version 190578 (0.0006) [2024-09-30 03:22:47,372][1157819] Updated weights for policy 0, policy_version 190588 (0.0006) [2024-09-30 03:22:48,009][1157819] Updated weights for policy 0, policy_version 190598 (0.0007) [2024-09-30 03:22:48,619][1157819] Updated weights for policy 0, policy_version 190608 (0.0006) [2024-09-30 03:22:49,274][1157819] Updated weights for policy 0, policy_version 190618 (0.0006) [2024-09-30 03:22:49,898][1157819] Updated weights for policy 0, policy_version 190628 (0.0006) [2024-09-30 03:22:50,466][1157520] Fps is (10 sec: 64717.8, 60 sec: 71680.2, 300 sec: 74338.9). Total num frames: 780849152. Throughput: 0: 17568.6. Samples: 185201176. Policy #0 lag: (min: 1.0, avg: 2.3, max: 6.0) [2024-09-30 03:22:50,466][1157520] Avg episode reward: [(0, '56.096')] [2024-09-30 03:22:50,521][1157819] Updated weights for policy 0, policy_version 190638 (0.0006) [2024-09-30 03:22:51,138][1157819] Updated weights for policy 0, policy_version 190648 (0.0006) [2024-09-30 03:22:51,710][1157819] Updated weights for policy 0, policy_version 190658 (0.0006) [2024-09-30 03:22:52,282][1157819] Updated weights for policy 0, policy_version 190668 (0.0006) [2024-09-30 03:22:52,876][1157819] Updated weights for policy 0, policy_version 190678 (0.0006) [2024-09-30 03:22:53,453][1157819] Updated weights for policy 0, policy_version 190688 (0.0006) [2024-09-30 03:22:54,064][1157819] Updated weights for policy 0, policy_version 190698 (0.0006) [2024-09-30 03:22:54,653][1157819] Updated weights for policy 0, policy_version 190708 (0.0006) [2024-09-30 03:22:55,219][1157819] Updated weights for policy 0, policy_version 190718 (0.0006) [2024-09-30 03:22:55,466][1157520] Fps is (10 sec: 66765.0, 60 sec: 70860.8, 300 sec: 74227.8). Total num frames: 781193216. Throughput: 0: 17374.1. Samples: 185252616. Policy #0 lag: (min: 1.0, avg: 2.3, max: 6.0) [2024-09-30 03:22:55,466][1157520] Avg episode reward: [(0, '55.660')] [2024-09-30 03:22:55,834][1157819] Updated weights for policy 0, policy_version 190728 (0.0006) [2024-09-30 03:22:56,483][1157819] Updated weights for policy 0, policy_version 190738 (0.0006) [2024-09-30 03:22:57,083][1157819] Updated weights for policy 0, policy_version 190748 (0.0006) [2024-09-30 03:22:57,709][1157819] Updated weights for policy 0, policy_version 190758 (0.0006) [2024-09-30 03:22:58,400][1157819] Updated weights for policy 0, policy_version 190768 (0.0006) [2024-09-30 03:22:59,039][1157819] Updated weights for policy 0, policy_version 190778 (0.0006) [2024-09-30 03:22:59,686][1157819] Updated weights for policy 0, policy_version 190788 (0.0006) [2024-09-30 03:23:00,317][1157819] Updated weights for policy 0, policy_version 190798 (0.0006) [2024-09-30 03:23:00,466][1157520] Fps is (10 sec: 66764.7, 60 sec: 70041.6, 300 sec: 74047.4). Total num frames: 781516800. Throughput: 0: 16920.9. Samples: 185352800. Policy #0 lag: (min: 1.0, avg: 2.3, max: 6.0) [2024-09-30 03:23:00,466][1157520] Avg episode reward: [(0, '56.434')] [2024-09-30 03:23:00,948][1157819] Updated weights for policy 0, policy_version 190808 (0.0006) [2024-09-30 03:23:01,563][1157819] Updated weights for policy 0, policy_version 190818 (0.0006) [2024-09-30 03:23:02,229][1157819] Updated weights for policy 0, policy_version 190828 (0.0006) [2024-09-30 03:23:02,903][1157819] Updated weights for policy 0, policy_version 190838 (0.0006) [2024-09-30 03:23:03,508][1157819] Updated weights for policy 0, policy_version 190848 (0.0006) [2024-09-30 03:23:04,106][1157819] Updated weights for policy 0, policy_version 190858 (0.0006) [2024-09-30 03:23:04,675][1157819] Updated weights for policy 0, policy_version 190868 (0.0006) [2024-09-30 03:23:05,290][1157819] Updated weights for policy 0, policy_version 190878 (0.0006) [2024-09-30 03:23:05,466][1157520] Fps is (10 sec: 65536.0, 60 sec: 69222.6, 300 sec: 73839.1). Total num frames: 781848576. Throughput: 0: 16642.3. Samples: 185450852. Policy #0 lag: (min: 1.0, avg: 2.3, max: 6.0) [2024-09-30 03:23:05,466][1157520] Avg episode reward: [(0, '56.808')] [2024-09-30 03:23:05,814][1157819] Updated weights for policy 0, policy_version 190888 (0.0006) [2024-09-30 03:23:06,415][1157819] Updated weights for policy 0, policy_version 190898 (0.0006) [2024-09-30 03:23:06,973][1157819] Updated weights for policy 0, policy_version 190908 (0.0006) [2024-09-30 03:23:07,556][1157819] Updated weights for policy 0, policy_version 190918 (0.0006) [2024-09-30 03:23:08,104][1157819] Updated weights for policy 0, policy_version 190928 (0.0006) [2024-09-30 03:23:08,697][1157819] Updated weights for policy 0, policy_version 190938 (0.0006) [2024-09-30 03:23:09,296][1157819] Updated weights for policy 0, policy_version 190948 (0.0006) [2024-09-30 03:23:09,657][1157736] Signal inference workers to stop experience collection... (13100 times) [2024-09-30 03:23:09,659][1157736] Signal inference workers to resume experience collection... (13100 times) [2024-09-30 03:23:09,662][1157819] InferenceWorker_p0-w0: stopping experience collection (13100 times) [2024-09-30 03:23:09,665][1157819] InferenceWorker_p0-w0: resuming experience collection (13100 times) [2024-09-30 03:23:09,879][1157819] Updated weights for policy 0, policy_version 190958 (0.0006) [2024-09-30 03:23:10,454][1157819] Updated weights for policy 0, policy_version 190968 (0.0006) [2024-09-30 03:23:10,466][1157520] Fps is (10 sec: 68812.9, 60 sec: 68335.0, 300 sec: 73700.2). Total num frames: 782204928. Throughput: 0: 16635.8. Samples: 185504224. Policy #0 lag: (min: 1.0, avg: 2.3, max: 6.0) [2024-09-30 03:23:10,466][1157520] Avg episode reward: [(0, '53.358')] [2024-09-30 03:23:11,042][1157819] Updated weights for policy 0, policy_version 190978 (0.0006) [2024-09-30 03:23:11,615][1157819] Updated weights for policy 0, policy_version 190988 (0.0006) [2024-09-30 03:23:12,178][1157819] Updated weights for policy 0, policy_version 190998 (0.0006) [2024-09-30 03:23:12,770][1157819] Updated weights for policy 0, policy_version 191008 (0.0006) [2024-09-30 03:23:13,312][1157819] Updated weights for policy 0, policy_version 191018 (0.0006) [2024-09-30 03:23:13,896][1157819] Updated weights for policy 0, policy_version 191028 (0.0006) [2024-09-30 03:23:14,447][1157819] Updated weights for policy 0, policy_version 191038 (0.0006) [2024-09-30 03:23:15,016][1157819] Updated weights for policy 0, policy_version 191048 (0.0006) [2024-09-30 03:23:15,466][1157520] Fps is (10 sec: 71680.5, 60 sec: 67925.5, 300 sec: 73575.3). Total num frames: 782565376. Throughput: 0: 16685.5. Samples: 185611168. Policy #0 lag: (min: 1.0, avg: 2.3, max: 6.0) [2024-09-30 03:23:15,466][1157520] Avg episode reward: [(0, '55.072')] [2024-09-30 03:23:15,567][1157819] Updated weights for policy 0, policy_version 191058 (0.0006) [2024-09-30 03:23:16,107][1157819] Updated weights for policy 0, policy_version 191068 (0.0006) [2024-09-30 03:23:16,621][1157819] Updated weights for policy 0, policy_version 191078 (0.0006) [2024-09-30 03:23:17,172][1157819] Updated weights for policy 0, policy_version 191088 (0.0006) [2024-09-30 03:23:17,721][1157819] Updated weights for policy 0, policy_version 191098 (0.0006) [2024-09-30 03:23:18,259][1157819] Updated weights for policy 0, policy_version 191108 (0.0006) [2024-09-30 03:23:18,803][1157819] Updated weights for policy 0, policy_version 191118 (0.0006) [2024-09-30 03:23:19,339][1157819] Updated weights for policy 0, policy_version 191128 (0.0006) [2024-09-30 03:23:19,867][1157819] Updated weights for policy 0, policy_version 191138 (0.0006) [2024-09-30 03:23:20,416][1157819] Updated weights for policy 0, policy_version 191148 (0.0006) [2024-09-30 03:23:20,466][1157520] Fps is (10 sec: 73728.2, 60 sec: 68130.2, 300 sec: 73505.9). Total num frames: 782942208. Throughput: 0: 16987.4. Samples: 185723604. Policy #0 lag: (min: 1.0, avg: 2.3, max: 6.0) [2024-09-30 03:23:20,466][1157520] Avg episode reward: [(0, '56.228')] [2024-09-30 03:23:20,993][1157819] Updated weights for policy 0, policy_version 191158 (0.0006) [2024-09-30 03:23:21,505][1157819] Updated weights for policy 0, policy_version 191168 (0.0006) [2024-09-30 03:23:22,049][1157819] Updated weights for policy 0, policy_version 191178 (0.0006) [2024-09-30 03:23:22,605][1157819] Updated weights for policy 0, policy_version 191188 (0.0006) [2024-09-30 03:23:23,136][1157819] Updated weights for policy 0, policy_version 191198 (0.0006) [2024-09-30 03:23:23,653][1157819] Updated weights for policy 0, policy_version 191208 (0.0006) [2024-09-30 03:23:24,232][1157819] Updated weights for policy 0, policy_version 191218 (0.0006) [2024-09-30 03:23:24,749][1157819] Updated weights for policy 0, policy_version 191228 (0.0006) [2024-09-30 03:23:25,263][1157819] Updated weights for policy 0, policy_version 191238 (0.0006) [2024-09-30 03:23:25,466][1157520] Fps is (10 sec: 75776.0, 60 sec: 68676.4, 300 sec: 73533.6). Total num frames: 783323136. Throughput: 0: 17168.1. Samples: 185780308. Policy #0 lag: (min: 1.0, avg: 2.3, max: 6.0) [2024-09-30 03:23:25,466][1157520] Avg episode reward: [(0, '58.019')] [2024-09-30 03:23:25,789][1157819] Updated weights for policy 0, policy_version 191248 (0.0006) [2024-09-30 03:23:26,317][1157819] Updated weights for policy 0, policy_version 191258 (0.0006) [2024-09-30 03:23:26,869][1157819] Updated weights for policy 0, policy_version 191268 (0.0006) [2024-09-30 03:23:27,463][1157819] Updated weights for policy 0, policy_version 191278 (0.0006) [2024-09-30 03:23:28,018][1157819] Updated weights for policy 0, policy_version 191288 (0.0006) [2024-09-30 03:23:28,601][1157819] Updated weights for policy 0, policy_version 191298 (0.0006) [2024-09-30 03:23:29,169][1157819] Updated weights for policy 0, policy_version 191308 (0.0006) [2024-09-30 03:23:29,726][1157736] Signal inference workers to stop experience collection... (13150 times) [2024-09-30 03:23:29,727][1157736] Signal inference workers to resume experience collection... (13150 times) [2024-09-30 03:23:29,728][1157819] Updated weights for policy 0, policy_version 191318 (0.0006) [2024-09-30 03:23:29,731][1157819] InferenceWorker_p0-w0: stopping experience collection (13150 times) [2024-09-30 03:23:29,734][1157819] InferenceWorker_p0-w0: resuming experience collection (13150 times) [2024-09-30 03:23:30,299][1157819] Updated weights for policy 0, policy_version 191328 (0.0006) [2024-09-30 03:23:30,466][1157520] Fps is (10 sec: 74956.4, 60 sec: 68881.1, 300 sec: 73547.5). Total num frames: 783691776. Throughput: 0: 17521.3. Samples: 185892944. Policy #0 lag: (min: 1.0, avg: 2.3, max: 6.0) [2024-09-30 03:23:30,466][1157520] Avg episode reward: [(0, '56.736')] [2024-09-30 03:23:30,884][1157819] Updated weights for policy 0, policy_version 191338 (0.0006) [2024-09-30 03:23:31,456][1157819] Updated weights for policy 0, policy_version 191348 (0.0006) [2024-09-30 03:23:31,976][1157819] Updated weights for policy 0, policy_version 191358 (0.0006) [2024-09-30 03:23:32,604][1157819] Updated weights for policy 0, policy_version 191368 (0.0006) [2024-09-30 03:23:33,154][1157819] Updated weights for policy 0, policy_version 191378 (0.0006) [2024-09-30 03:23:33,665][1157819] Updated weights for policy 0, policy_version 191388 (0.0006) [2024-09-30 03:23:34,177][1157819] Updated weights for policy 0, policy_version 191398 (0.0006) [2024-09-30 03:23:34,715][1157819] Updated weights for policy 0, policy_version 191408 (0.0006) [2024-09-30 03:23:35,235][1157819] Updated weights for policy 0, policy_version 191418 (0.0006) [2024-09-30 03:23:35,466][1157520] Fps is (10 sec: 74137.4, 60 sec: 69768.6, 300 sec: 73589.2). Total num frames: 784064512. Throughput: 0: 17828.6. Samples: 186003464. Policy #0 lag: (min: 1.0, avg: 2.5, max: 6.0) [2024-09-30 03:23:35,466][1157520] Avg episode reward: [(0, '56.573')] [2024-09-30 03:23:35,732][1157819] Updated weights for policy 0, policy_version 191428 (0.0006) [2024-09-30 03:23:36,276][1157819] Updated weights for policy 0, policy_version 191438 (0.0006) [2024-09-30 03:23:36,786][1157819] Updated weights for policy 0, policy_version 191448 (0.0006) [2024-09-30 03:23:37,318][1157819] Updated weights for policy 0, policy_version 191458 (0.0006) [2024-09-30 03:23:37,829][1157819] Updated weights for policy 0, policy_version 191468 (0.0006) [2024-09-30 03:23:38,402][1157819] Updated weights for policy 0, policy_version 191478 (0.0006) [2024-09-30 03:23:38,924][1157819] Updated weights for policy 0, policy_version 191488 (0.0006) [2024-09-30 03:23:39,541][1157819] Updated weights for policy 0, policy_version 191498 (0.0006) [2024-09-30 03:23:40,112][1157819] Updated weights for policy 0, policy_version 191508 (0.0006) [2024-09-30 03:23:40,466][1157520] Fps is (10 sec: 74957.0, 60 sec: 70656.2, 300 sec: 73644.7). Total num frames: 784441344. Throughput: 0: 17994.4. Samples: 186062364. Policy #0 lag: (min: 1.0, avg: 2.5, max: 6.0) [2024-09-30 03:23:40,466][1157520] Avg episode reward: [(0, '56.395')] [2024-09-30 03:23:40,675][1157819] Updated weights for policy 0, policy_version 191518 (0.0006) [2024-09-30 03:23:41,243][1157819] Updated weights for policy 0, policy_version 191528 (0.0007) [2024-09-30 03:23:41,832][1157819] Updated weights for policy 0, policy_version 191538 (0.0006) [2024-09-30 03:23:42,350][1157819] Updated weights for policy 0, policy_version 191548 (0.0006) [2024-09-30 03:23:42,873][1157819] Updated weights for policy 0, policy_version 191558 (0.0006) [2024-09-30 03:23:43,415][1157819] Updated weights for policy 0, policy_version 191568 (0.0006) [2024-09-30 03:23:43,968][1157819] Updated weights for policy 0, policy_version 191578 (0.0006) [2024-09-30 03:23:44,479][1157819] Updated weights for policy 0, policy_version 191588 (0.0006) [2024-09-30 03:23:44,989][1157819] Updated weights for policy 0, policy_version 191598 (0.0006) [2024-09-30 03:23:45,466][1157520] Fps is (10 sec: 75366.7, 60 sec: 71543.6, 300 sec: 73630.8). Total num frames: 784818176. Throughput: 0: 18216.4. Samples: 186172536. Policy #0 lag: (min: 1.0, avg: 2.5, max: 6.0) [2024-09-30 03:23:45,466][1157520] Avg episode reward: [(0, '54.481')] [2024-09-30 03:23:45,558][1157819] Updated weights for policy 0, policy_version 191608 (0.0006) [2024-09-30 03:23:46,054][1157819] Updated weights for policy 0, policy_version 191618 (0.0006) [2024-09-30 03:23:46,565][1157819] Updated weights for policy 0, policy_version 191628 (0.0006) [2024-09-30 03:23:47,090][1157819] Updated weights for policy 0, policy_version 191638 (0.0006) [2024-09-30 03:23:47,595][1157819] Updated weights for policy 0, policy_version 191648 (0.0006) [2024-09-30 03:23:48,080][1157819] Updated weights for policy 0, policy_version 191658 (0.0006) [2024-09-30 03:23:48,592][1157819] Updated weights for policy 0, policy_version 191668 (0.0006) [2024-09-30 03:23:49,104][1157819] Updated weights for policy 0, policy_version 191678 (0.0006) [2024-09-30 03:23:49,623][1157819] Updated weights for policy 0, policy_version 191688 (0.0006) [2024-09-30 03:23:50,130][1157819] Updated weights for policy 0, policy_version 191698 (0.0006) [2024-09-30 03:23:50,466][1157520] Fps is (10 sec: 77415.1, 60 sec: 72772.4, 300 sec: 73630.8). Total num frames: 785215488. Throughput: 0: 18688.3. Samples: 186291824. Policy #0 lag: (min: 1.0, avg: 2.5, max: 6.0) [2024-09-30 03:23:50,466][1157520] Avg episode reward: [(0, '57.526')] [2024-09-30 03:23:50,678][1157819] Updated weights for policy 0, policy_version 191708 (0.0006) [2024-09-30 03:23:51,178][1157819] Updated weights for policy 0, policy_version 191718 (0.0006) [2024-09-30 03:23:51,752][1157819] Updated weights for policy 0, policy_version 191728 (0.0006) [2024-09-30 03:23:52,307][1157819] Updated weights for policy 0, policy_version 191738 (0.0006) [2024-09-30 03:23:52,867][1157819] Updated weights for policy 0, policy_version 191748 (0.0006) [2024-09-30 03:23:53,407][1157819] Updated weights for policy 0, policy_version 191758 (0.0006) [2024-09-30 03:23:53,924][1157819] Updated weights for policy 0, policy_version 191768 (0.0006) [2024-09-30 03:23:54,514][1157819] Updated weights for policy 0, policy_version 191778 (0.0006) [2024-09-30 03:23:55,040][1157819] Updated weights for policy 0, policy_version 191788 (0.0006) [2024-09-30 03:23:55,466][1157520] Fps is (10 sec: 77412.5, 60 sec: 73318.2, 300 sec: 73561.3). Total num frames: 785592320. Throughput: 0: 18769.3. Samples: 186348844. Policy #0 lag: (min: 1.0, avg: 2.5, max: 6.0) [2024-09-30 03:23:55,466][1157520] Avg episode reward: [(0, '55.718')] [2024-09-30 03:23:55,578][1157819] Updated weights for policy 0, policy_version 191798 (0.0006) [2024-09-30 03:23:56,134][1157819] Updated weights for policy 0, policy_version 191808 (0.0006) [2024-09-30 03:23:56,662][1157819] Updated weights for policy 0, policy_version 191818 (0.0006) [2024-09-30 03:23:57,146][1157819] Updated weights for policy 0, policy_version 191828 (0.0006) [2024-09-30 03:23:57,551][1157736] Signal inference workers to stop experience collection... (13200 times) [2024-09-30 03:23:57,553][1157736] Signal inference workers to resume experience collection... (13200 times) [2024-09-30 03:23:57,557][1157819] InferenceWorker_p0-w0: stopping experience collection (13200 times) [2024-09-30 03:23:57,559][1157819] InferenceWorker_p0-w0: resuming experience collection (13200 times) [2024-09-30 03:23:57,655][1157819] Updated weights for policy 0, policy_version 191838 (0.0006) [2024-09-30 03:23:58,215][1157819] Updated weights for policy 0, policy_version 191848 (0.0006) [2024-09-30 03:23:58,733][1157819] Updated weights for policy 0, policy_version 191858 (0.0006) [2024-09-30 03:23:59,225][1157819] Updated weights for policy 0, policy_version 191868 (0.0006) [2024-09-30 03:23:59,732][1157819] Updated weights for policy 0, policy_version 191878 (0.0006) [2024-09-30 03:24:00,240][1157819] Updated weights for policy 0, policy_version 191888 (0.0006) [2024-09-30 03:24:00,466][1157520] Fps is (10 sec: 77414.4, 60 sec: 74547.3, 300 sec: 73630.8). Total num frames: 785989632. Throughput: 0: 18952.9. Samples: 186464048. Policy #0 lag: (min: 1.0, avg: 2.5, max: 6.0) [2024-09-30 03:24:00,466][1157520] Avg episode reward: [(0, '55.387')] [2024-09-30 03:24:00,770][1157819] Updated weights for policy 0, policy_version 191898 (0.0006) [2024-09-30 03:24:01,262][1157819] Updated weights for policy 0, policy_version 191908 (0.0006) [2024-09-30 03:24:01,765][1157819] Updated weights for policy 0, policy_version 191918 (0.0006) [2024-09-30 03:24:02,275][1157819] Updated weights for policy 0, policy_version 191928 (0.0006) [2024-09-30 03:24:02,814][1157819] Updated weights for policy 0, policy_version 191938 (0.0006) [2024-09-30 03:24:03,312][1157819] Updated weights for policy 0, policy_version 191948 (0.0006) [2024-09-30 03:24:03,816][1157819] Updated weights for policy 0, policy_version 191958 (0.0006) [2024-09-30 03:24:04,324][1157819] Updated weights for policy 0, policy_version 191968 (0.0006) [2024-09-30 03:24:04,867][1157819] Updated weights for policy 0, policy_version 191978 (0.0006) [2024-09-30 03:24:05,378][1157819] Updated weights for policy 0, policy_version 191988 (0.0006) [2024-09-30 03:24:05,466][1157520] Fps is (10 sec: 79873.8, 60 sec: 75707.8, 300 sec: 73728.0). Total num frames: 786391040. Throughput: 0: 19128.5. Samples: 186584388. Policy #0 lag: (min: 1.0, avg: 2.5, max: 6.0) [2024-09-30 03:24:05,466][1157520] Avg episode reward: [(0, '55.179')] [2024-09-30 03:24:05,470][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000191990_786391040.pth... [2024-09-30 03:24:05,527][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000187716_768884736.pth [2024-09-30 03:24:05,923][1157819] Updated weights for policy 0, policy_version 191998 (0.0006) [2024-09-30 03:24:06,450][1157819] Updated weights for policy 0, policy_version 192008 (0.0006) [2024-09-30 03:24:06,986][1157819] Updated weights for policy 0, policy_version 192018 (0.0006) [2024-09-30 03:24:07,558][1157819] Updated weights for policy 0, policy_version 192028 (0.0006) [2024-09-30 03:24:08,126][1157819] Updated weights for policy 0, policy_version 192038 (0.0006) [2024-09-30 03:24:08,682][1157819] Updated weights for policy 0, policy_version 192048 (0.0006) [2024-09-30 03:24:09,186][1157819] Updated weights for policy 0, policy_version 192058 (0.0006) [2024-09-30 03:24:09,717][1157819] Updated weights for policy 0, policy_version 192068 (0.0006) [2024-09-30 03:24:10,298][1157819] Updated weights for policy 0, policy_version 192078 (0.0006) [2024-09-30 03:24:10,466][1157520] Fps is (10 sec: 77004.8, 60 sec: 75912.6, 300 sec: 73658.6). Total num frames: 786759680. Throughput: 0: 19122.7. Samples: 186640828. Policy #0 lag: (min: 1.0, avg: 2.5, max: 6.0) [2024-09-30 03:24:10,466][1157520] Avg episode reward: [(0, '55.021')] [2024-09-30 03:24:10,823][1157819] Updated weights for policy 0, policy_version 192088 (0.0006) [2024-09-30 03:24:11,397][1157819] Updated weights for policy 0, policy_version 192098 (0.0006) [2024-09-30 03:24:11,942][1157819] Updated weights for policy 0, policy_version 192108 (0.0006) [2024-09-30 03:24:12,520][1157819] Updated weights for policy 0, policy_version 192118 (0.0006) [2024-09-30 03:24:13,051][1157819] Updated weights for policy 0, policy_version 192128 (0.0006) [2024-09-30 03:24:13,617][1157819] Updated weights for policy 0, policy_version 192138 (0.0006) [2024-09-30 03:24:14,144][1157819] Updated weights for policy 0, policy_version 192148 (0.0006) [2024-09-30 03:24:14,678][1157819] Updated weights for policy 0, policy_version 192158 (0.0006) [2024-09-30 03:24:15,196][1157819] Updated weights for policy 0, policy_version 192168 (0.0006) [2024-09-30 03:24:15,466][1157520] Fps is (10 sec: 74957.0, 60 sec: 76253.9, 300 sec: 73616.9). Total num frames: 787140608. Throughput: 0: 19109.6. Samples: 186752872. Policy #0 lag: (min: 1.0, avg: 2.5, max: 6.0) [2024-09-30 03:24:15,466][1157520] Avg episode reward: [(0, '53.773')] [2024-09-30 03:24:15,757][1157819] Updated weights for policy 0, policy_version 192178 (0.0006) [2024-09-30 03:24:16,302][1157819] Updated weights for policy 0, policy_version 192188 (0.0006) [2024-09-30 03:24:16,867][1157819] Updated weights for policy 0, policy_version 192198 (0.0006) [2024-09-30 03:24:17,393][1157819] Updated weights for policy 0, policy_version 192208 (0.0006) [2024-09-30 03:24:17,943][1157819] Updated weights for policy 0, policy_version 192218 (0.0006) [2024-09-30 03:24:18,564][1157819] Updated weights for policy 0, policy_version 192228 (0.0006) [2024-09-30 03:24:19,143][1157819] Updated weights for policy 0, policy_version 192238 (0.0006) [2024-09-30 03:24:19,679][1157819] Updated weights for policy 0, policy_version 192248 (0.0006) [2024-09-30 03:24:20,221][1157819] Updated weights for policy 0, policy_version 192258 (0.0006) [2024-09-30 03:24:20,466][1157520] Fps is (10 sec: 74136.8, 60 sec: 75980.7, 300 sec: 73547.5). Total num frames: 787501056. Throughput: 0: 19123.5. Samples: 186864024. Policy #0 lag: (min: 1.0, avg: 2.5, max: 6.0) [2024-09-30 03:24:20,466][1157520] Avg episode reward: [(0, '57.158')] [2024-09-30 03:24:20,882][1157819] Updated weights for policy 0, policy_version 192268 (0.0006) [2024-09-30 03:24:21,444][1157819] Updated weights for policy 0, policy_version 192278 (0.0006) [2024-09-30 03:24:22,007][1157819] Updated weights for policy 0, policy_version 192288 (0.0006) [2024-09-30 03:24:22,622][1157819] Updated weights for policy 0, policy_version 192298 (0.0006) [2024-09-30 03:24:23,183][1157819] Updated weights for policy 0, policy_version 192308 (0.0006) [2024-09-30 03:24:23,779][1157819] Updated weights for policy 0, policy_version 192318 (0.0006) [2024-09-30 03:24:24,334][1157819] Updated weights for policy 0, policy_version 192328 (0.0006) [2024-09-30 03:24:24,932][1157819] Updated weights for policy 0, policy_version 192338 (0.0006) [2024-09-30 03:24:25,260][1157736] Signal inference workers to stop experience collection... (13250 times) [2024-09-30 03:24:25,260][1157736] Signal inference workers to resume experience collection... (13250 times) [2024-09-30 03:24:25,263][1157819] InferenceWorker_p0-w0: stopping experience collection (13250 times) [2024-09-30 03:24:25,264][1157819] InferenceWorker_p0-w0: resuming experience collection (13250 times) [2024-09-30 03:24:25,455][1157819] Updated weights for policy 0, policy_version 192348 (0.0006) [2024-09-30 03:24:25,466][1157520] Fps is (10 sec: 71679.9, 60 sec: 75571.2, 300 sec: 73422.6). Total num frames: 787857408. Throughput: 0: 18989.7. Samples: 186916900. Policy #0 lag: (min: 1.0, avg: 2.5, max: 6.0) [2024-09-30 03:24:25,466][1157520] Avg episode reward: [(0, '54.153')] [2024-09-30 03:24:25,988][1157819] Updated weights for policy 0, policy_version 192358 (0.0006) [2024-09-30 03:24:26,538][1157819] Updated weights for policy 0, policy_version 192368 (0.0006) [2024-09-30 03:24:27,052][1157819] Updated weights for policy 0, policy_version 192378 (0.0006) [2024-09-30 03:24:27,605][1157819] Updated weights for policy 0, policy_version 192388 (0.0006) [2024-09-30 03:24:28,120][1157819] Updated weights for policy 0, policy_version 192398 (0.0006) [2024-09-30 03:24:28,645][1157819] Updated weights for policy 0, policy_version 192408 (0.0006) [2024-09-30 03:24:29,217][1157819] Updated weights for policy 0, policy_version 192418 (0.0006) [2024-09-30 03:24:29,813][1157819] Updated weights for policy 0, policy_version 192428 (0.0006) [2024-09-30 03:24:30,390][1157819] Updated weights for policy 0, policy_version 192438 (0.0006) [2024-09-30 03:24:30,466][1157520] Fps is (10 sec: 72909.6, 60 sec: 75639.6, 300 sec: 73339.3). Total num frames: 788230144. Throughput: 0: 19015.5. Samples: 187028232. Policy #0 lag: (min: 1.0, avg: 2.5, max: 6.0) [2024-09-30 03:24:30,466][1157520] Avg episode reward: [(0, '56.664')] [2024-09-30 03:24:30,968][1157819] Updated weights for policy 0, policy_version 192448 (0.0006) [2024-09-30 03:24:31,558][1157819] Updated weights for policy 0, policy_version 192458 (0.0006) [2024-09-30 03:24:32,085][1157819] Updated weights for policy 0, policy_version 192468 (0.0006) [2024-09-30 03:24:32,622][1157819] Updated weights for policy 0, policy_version 192478 (0.0006) [2024-09-30 03:24:33,160][1157819] Updated weights for policy 0, policy_version 192488 (0.0006) [2024-09-30 03:24:33,687][1157819] Updated weights for policy 0, policy_version 192498 (0.0006) [2024-09-30 03:24:34,276][1157819] Updated weights for policy 0, policy_version 192508 (0.0007) [2024-09-30 03:24:34,807][1157819] Updated weights for policy 0, policy_version 192518 (0.0006) [2024-09-30 03:24:35,322][1157819] Updated weights for policy 0, policy_version 192528 (0.0006) [2024-09-30 03:24:35,466][1157520] Fps is (10 sec: 74547.2, 60 sec: 75639.5, 300 sec: 73255.9). Total num frames: 788602880. Throughput: 0: 18806.8. Samples: 187138132. Policy #0 lag: (min: 1.0, avg: 2.5, max: 6.0) [2024-09-30 03:24:35,466][1157520] Avg episode reward: [(0, '55.105')] [2024-09-30 03:24:35,870][1157819] Updated weights for policy 0, policy_version 192538 (0.0006) [2024-09-30 03:24:36,381][1157819] Updated weights for policy 0, policy_version 192548 (0.0007) [2024-09-30 03:24:36,896][1157819] Updated weights for policy 0, policy_version 192558 (0.0007) [2024-09-30 03:24:37,412][1157819] Updated weights for policy 0, policy_version 192568 (0.0007) [2024-09-30 03:24:37,954][1157819] Updated weights for policy 0, policy_version 192578 (0.0006) [2024-09-30 03:24:38,469][1157819] Updated weights for policy 0, policy_version 192588 (0.0006) [2024-09-30 03:24:39,000][1157819] Updated weights for policy 0, policy_version 192598 (0.0006) [2024-09-30 03:24:39,591][1157819] Updated weights for policy 0, policy_version 192608 (0.0007) [2024-09-30 03:24:40,172][1157819] Updated weights for policy 0, policy_version 192618 (0.0006) [2024-09-30 03:24:40,466][1157520] Fps is (10 sec: 75366.2, 60 sec: 75707.8, 300 sec: 73242.1). Total num frames: 788983808. Throughput: 0: 18841.5. Samples: 187196708. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 03:24:40,466][1157520] Avg episode reward: [(0, '57.153')] [2024-09-30 03:24:40,698][1157819] Updated weights for policy 0, policy_version 192628 (0.0006) [2024-09-30 03:24:41,291][1157819] Updated weights for policy 0, policy_version 192638 (0.0006) [2024-09-30 03:24:41,789][1157819] Updated weights for policy 0, policy_version 192648 (0.0006) [2024-09-30 03:24:42,286][1157819] Updated weights for policy 0, policy_version 192658 (0.0006) [2024-09-30 03:24:42,862][1157819] Updated weights for policy 0, policy_version 192668 (0.0006) [2024-09-30 03:24:43,423][1157819] Updated weights for policy 0, policy_version 192678 (0.0006) [2024-09-30 03:24:43,972][1157819] Updated weights for policy 0, policy_version 192688 (0.0006) [2024-09-30 03:24:44,541][1157819] Updated weights for policy 0, policy_version 192698 (0.0006) [2024-09-30 03:24:45,051][1157819] Updated weights for policy 0, policy_version 192708 (0.0006) [2024-09-30 03:24:45,466][1157520] Fps is (10 sec: 75776.1, 60 sec: 75707.7, 300 sec: 73228.2). Total num frames: 789360640. Throughput: 0: 18776.3. Samples: 187308984. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 03:24:45,466][1157520] Avg episode reward: [(0, '58.438')] [2024-09-30 03:24:45,606][1157819] Updated weights for policy 0, policy_version 192718 (0.0007) [2024-09-30 03:24:46,114][1157819] Updated weights for policy 0, policy_version 192728 (0.0006) [2024-09-30 03:24:46,628][1157819] Updated weights for policy 0, policy_version 192738 (0.0006) [2024-09-30 03:24:47,150][1157819] Updated weights for policy 0, policy_version 192748 (0.0006) [2024-09-30 03:24:47,675][1157819] Updated weights for policy 0, policy_version 192758 (0.0006) [2024-09-30 03:24:48,255][1157819] Updated weights for policy 0, policy_version 192768 (0.0007) [2024-09-30 03:24:48,752][1157819] Updated weights for policy 0, policy_version 192778 (0.0006) [2024-09-30 03:24:49,270][1157819] Updated weights for policy 0, policy_version 192788 (0.0006) [2024-09-30 03:24:49,780][1157819] Updated weights for policy 0, policy_version 192798 (0.0006) [2024-09-30 03:24:50,353][1157819] Updated weights for policy 0, policy_version 192808 (0.0006) [2024-09-30 03:24:50,466][1157520] Fps is (10 sec: 77005.1, 60 sec: 75639.5, 300 sec: 73297.6). Total num frames: 789753856. Throughput: 0: 18677.4. Samples: 187424872. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 03:24:50,466][1157520] Avg episode reward: [(0, '51.929')] [2024-09-30 03:24:50,862][1157819] Updated weights for policy 0, policy_version 192818 (0.0006) [2024-09-30 03:24:51,375][1157819] Updated weights for policy 0, policy_version 192828 (0.0006) [2024-09-30 03:24:51,876][1157819] Updated weights for policy 0, policy_version 192838 (0.0006) [2024-09-30 03:24:52,406][1157819] Updated weights for policy 0, policy_version 192848 (0.0006) [2024-09-30 03:24:52,955][1157819] Updated weights for policy 0, policy_version 192858 (0.0006) [2024-09-30 03:24:53,492][1157819] Updated weights for policy 0, policy_version 192868 (0.0006) [2024-09-30 03:24:54,009][1157819] Updated weights for policy 0, policy_version 192878 (0.0006) [2024-09-30 03:24:54,550][1157819] Updated weights for policy 0, policy_version 192888 (0.0006) [2024-09-30 03:24:55,147][1157819] Updated weights for policy 0, policy_version 192898 (0.0006) [2024-09-30 03:24:55,444][1157736] Signal inference workers to stop experience collection... (13300 times) [2024-09-30 03:24:55,448][1157736] Signal inference workers to resume experience collection... (13300 times) [2024-09-30 03:24:55,449][1157819] InferenceWorker_p0-w0: stopping experience collection (13300 times) [2024-09-30 03:24:55,451][1157819] InferenceWorker_p0-w0: resuming experience collection (13300 times) [2024-09-30 03:24:55,466][1157520] Fps is (10 sec: 77003.8, 60 sec: 75639.6, 300 sec: 73353.1). Total num frames: 790130688. Throughput: 0: 18731.4. Samples: 187483744. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 03:24:55,466][1157520] Avg episode reward: [(0, '55.231')] [2024-09-30 03:24:55,705][1157819] Updated weights for policy 0, policy_version 192908 (0.0006) [2024-09-30 03:24:56,267][1157819] Updated weights for policy 0, policy_version 192918 (0.0006) [2024-09-30 03:24:56,856][1157819] Updated weights for policy 0, policy_version 192928 (0.0006) [2024-09-30 03:24:57,450][1157819] Updated weights for policy 0, policy_version 192938 (0.0006) [2024-09-30 03:24:58,034][1157819] Updated weights for policy 0, policy_version 192948 (0.0006) [2024-09-30 03:24:58,646][1157819] Updated weights for policy 0, policy_version 192958 (0.0006) [2024-09-30 03:24:59,235][1157819] Updated weights for policy 0, policy_version 192968 (0.0006) [2024-09-30 03:24:59,813][1157819] Updated weights for policy 0, policy_version 192978 (0.0006) [2024-09-30 03:25:00,417][1157819] Updated weights for policy 0, policy_version 192988 (0.0006) [2024-09-30 03:25:00,466][1157520] Fps is (10 sec: 72908.2, 60 sec: 74888.4, 300 sec: 73228.2). Total num frames: 790482944. Throughput: 0: 18638.6. Samples: 187591612. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 03:25:00,466][1157520] Avg episode reward: [(0, '56.606')] [2024-09-30 03:25:00,976][1157819] Updated weights for policy 0, policy_version 192998 (0.0006) [2024-09-30 03:25:01,535][1157819] Updated weights for policy 0, policy_version 193008 (0.0006) [2024-09-30 03:25:02,040][1157819] Updated weights for policy 0, policy_version 193018 (0.0006) [2024-09-30 03:25:02,536][1157819] Updated weights for policy 0, policy_version 193028 (0.0006) [2024-09-30 03:25:03,041][1157819] Updated weights for policy 0, policy_version 193038 (0.0006) [2024-09-30 03:25:03,567][1157819] Updated weights for policy 0, policy_version 193048 (0.0006) [2024-09-30 03:25:04,096][1157819] Updated weights for policy 0, policy_version 193058 (0.0006) [2024-09-30 03:25:04,597][1157819] Updated weights for policy 0, policy_version 193068 (0.0006) [2024-09-30 03:25:05,111][1157819] Updated weights for policy 0, policy_version 193078 (0.0006) [2024-09-30 03:25:05,466][1157520] Fps is (10 sec: 74136.9, 60 sec: 74683.5, 300 sec: 73283.7). Total num frames: 790872064. Throughput: 0: 18690.0. Samples: 187705076. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 03:25:05,466][1157520] Avg episode reward: [(0, '57.842')] [2024-09-30 03:25:05,667][1157819] Updated weights for policy 0, policy_version 193088 (0.0006) [2024-09-30 03:25:06,191][1157819] Updated weights for policy 0, policy_version 193098 (0.0006) [2024-09-30 03:25:06,693][1157819] Updated weights for policy 0, policy_version 193108 (0.0006) [2024-09-30 03:25:07,201][1157819] Updated weights for policy 0, policy_version 193118 (0.0006) [2024-09-30 03:25:07,713][1157819] Updated weights for policy 0, policy_version 193128 (0.0006) [2024-09-30 03:25:08,256][1157819] Updated weights for policy 0, policy_version 193138 (0.0006) [2024-09-30 03:25:08,800][1157819] Updated weights for policy 0, policy_version 193148 (0.0006) [2024-09-30 03:25:09,320][1157819] Updated weights for policy 0, policy_version 193158 (0.0006) [2024-09-30 03:25:09,827][1157819] Updated weights for policy 0, policy_version 193168 (0.0006) [2024-09-30 03:25:10,338][1157819] Updated weights for policy 0, policy_version 193178 (0.0006) [2024-09-30 03:25:10,466][1157520] Fps is (10 sec: 78233.5, 60 sec: 75093.2, 300 sec: 73436.4). Total num frames: 791265280. Throughput: 0: 18830.2. Samples: 187764260. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 03:25:10,466][1157520] Avg episode reward: [(0, '57.585')] [2024-09-30 03:25:10,876][1157819] Updated weights for policy 0, policy_version 193188 (0.0006) [2024-09-30 03:25:11,413][1157819] Updated weights for policy 0, policy_version 193198 (0.0006) [2024-09-30 03:25:11,940][1157819] Updated weights for policy 0, policy_version 193208 (0.0006) [2024-09-30 03:25:12,517][1157819] Updated weights for policy 0, policy_version 193218 (0.0006) [2024-09-30 03:25:13,069][1157819] Updated weights for policy 0, policy_version 193228 (0.0006) [2024-09-30 03:25:13,583][1157819] Updated weights for policy 0, policy_version 193238 (0.0006) [2024-09-30 03:25:14,119][1157819] Updated weights for policy 0, policy_version 193248 (0.0006) [2024-09-30 03:25:14,696][1157819] Updated weights for policy 0, policy_version 193258 (0.0006) [2024-09-30 03:25:15,239][1157819] Updated weights for policy 0, policy_version 193268 (0.0006) [2024-09-30 03:25:15,466][1157520] Fps is (10 sec: 76596.2, 60 sec: 74956.7, 300 sec: 73533.6). Total num frames: 791638016. Throughput: 0: 18923.5. Samples: 187879792. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 03:25:15,466][1157520] Avg episode reward: [(0, '55.337')] [2024-09-30 03:25:15,779][1157819] Updated weights for policy 0, policy_version 193278 (0.0006) [2024-09-30 03:25:16,341][1157819] Updated weights for policy 0, policy_version 193288 (0.0006) [2024-09-30 03:25:16,879][1157819] Updated weights for policy 0, policy_version 193298 (0.0006) [2024-09-30 03:25:17,432][1157819] Updated weights for policy 0, policy_version 193308 (0.0006) [2024-09-30 03:25:17,976][1157819] Updated weights for policy 0, policy_version 193318 (0.0006) [2024-09-30 03:25:18,552][1157819] Updated weights for policy 0, policy_version 193328 (0.0006) [2024-09-30 03:25:19,099][1157819] Updated weights for policy 0, policy_version 193338 (0.0006) [2024-09-30 03:25:19,665][1157819] Updated weights for policy 0, policy_version 193348 (0.0006) [2024-09-30 03:25:20,199][1157819] Updated weights for policy 0, policy_version 193358 (0.0006) [2024-09-30 03:25:20,466][1157520] Fps is (10 sec: 74547.1, 60 sec: 75161.6, 300 sec: 73617.0). Total num frames: 792010752. Throughput: 0: 18954.7. Samples: 187991096. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 03:25:20,466][1157520] Avg episode reward: [(0, '56.991')] [2024-09-30 03:25:20,736][1157819] Updated weights for policy 0, policy_version 193368 (0.0006) [2024-09-30 03:25:21,250][1157819] Updated weights for policy 0, policy_version 193378 (0.0006) [2024-09-30 03:25:21,814][1157819] Updated weights for policy 0, policy_version 193388 (0.0006) [2024-09-30 03:25:22,377][1157819] Updated weights for policy 0, policy_version 193398 (0.0006) [2024-09-30 03:25:22,988][1157819] Updated weights for policy 0, policy_version 193408 (0.0006) [2024-09-30 03:25:23,560][1157819] Updated weights for policy 0, policy_version 193418 (0.0006) [2024-09-30 03:25:24,141][1157819] Updated weights for policy 0, policy_version 193428 (0.0006) [2024-09-30 03:25:24,671][1157819] Updated weights for policy 0, policy_version 193438 (0.0006) [2024-09-30 03:25:25,304][1157819] Updated weights for policy 0, policy_version 193448 (0.0006) [2024-09-30 03:25:25,466][1157520] Fps is (10 sec: 73318.2, 60 sec: 75229.7, 300 sec: 73616.9). Total num frames: 792371200. Throughput: 0: 18889.7. Samples: 188046748. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 03:25:25,466][1157520] Avg episode reward: [(0, '53.700')] [2024-09-30 03:25:25,785][1157736] Signal inference workers to stop experience collection... (13350 times) [2024-09-30 03:25:25,786][1157736] Signal inference workers to resume experience collection... (13350 times) [2024-09-30 03:25:25,789][1157819] InferenceWorker_p0-w0: stopping experience collection (13350 times) [2024-09-30 03:25:25,789][1157819] InferenceWorker_p0-w0: resuming experience collection (13350 times) [2024-09-30 03:25:25,869][1157819] Updated weights for policy 0, policy_version 193458 (0.0006) [2024-09-30 03:25:26,418][1157819] Updated weights for policy 0, policy_version 193468 (0.0006) [2024-09-30 03:25:26,956][1157819] Updated weights for policy 0, policy_version 193478 (0.0006) [2024-09-30 03:25:27,558][1157819] Updated weights for policy 0, policy_version 193488 (0.0006) [2024-09-30 03:25:28,179][1157819] Updated weights for policy 0, policy_version 193498 (0.0006) [2024-09-30 03:25:28,750][1157819] Updated weights for policy 0, policy_version 193508 (0.0006) [2024-09-30 03:25:29,309][1157819] Updated weights for policy 0, policy_version 193518 (0.0006) [2024-09-30 03:25:29,919][1157819] Updated weights for policy 0, policy_version 193528 (0.0006) [2024-09-30 03:25:30,466][1157520] Fps is (10 sec: 71679.3, 60 sec: 74956.6, 300 sec: 73575.2). Total num frames: 792727552. Throughput: 0: 18754.9. Samples: 188152960. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 03:25:30,466][1157520] Avg episode reward: [(0, '55.724')] [2024-09-30 03:25:30,490][1157819] Updated weights for policy 0, policy_version 193538 (0.0006) [2024-09-30 03:25:31,046][1157819] Updated weights for policy 0, policy_version 193548 (0.0006) [2024-09-30 03:25:31,691][1157819] Updated weights for policy 0, policy_version 193558 (0.0006) [2024-09-30 03:25:32,251][1157819] Updated weights for policy 0, policy_version 193568 (0.0006) [2024-09-30 03:25:32,810][1157819] Updated weights for policy 0, policy_version 193578 (0.0006) [2024-09-30 03:25:33,351][1157819] Updated weights for policy 0, policy_version 193588 (0.0006) [2024-09-30 03:25:33,922][1157819] Updated weights for policy 0, policy_version 193598 (0.0007) [2024-09-30 03:25:34,477][1157819] Updated weights for policy 0, policy_version 193608 (0.0006) [2024-09-30 03:25:35,074][1157819] Updated weights for policy 0, policy_version 193618 (0.0006) [2024-09-30 03:25:35,466][1157520] Fps is (10 sec: 71679.7, 60 sec: 74751.8, 300 sec: 73519.7). Total num frames: 793088000. Throughput: 0: 18561.2. Samples: 188260128. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 03:25:35,466][1157520] Avg episode reward: [(0, '56.078')] [2024-09-30 03:25:35,608][1157819] Updated weights for policy 0, policy_version 193628 (0.0006) [2024-09-30 03:25:36,159][1157819] Updated weights for policy 0, policy_version 193638 (0.0006) [2024-09-30 03:25:36,655][1157819] Updated weights for policy 0, policy_version 193648 (0.0006) [2024-09-30 03:25:37,197][1157819] Updated weights for policy 0, policy_version 193658 (0.0006) [2024-09-30 03:25:37,738][1157819] Updated weights for policy 0, policy_version 193668 (0.0006) [2024-09-30 03:25:38,223][1157819] Updated weights for policy 0, policy_version 193678 (0.0006) [2024-09-30 03:25:38,722][1157819] Updated weights for policy 0, policy_version 193688 (0.0006) [2024-09-30 03:25:39,235][1157819] Updated weights for policy 0, policy_version 193698 (0.0006) [2024-09-30 03:25:39,746][1157819] Updated weights for policy 0, policy_version 193708 (0.0006) [2024-09-30 03:25:40,268][1157819] Updated weights for policy 0, policy_version 193718 (0.0006) [2024-09-30 03:25:40,466][1157520] Fps is (10 sec: 75777.1, 60 sec: 75025.0, 300 sec: 73630.8). Total num frames: 793485312. Throughput: 0: 18532.4. Samples: 188317700. Policy #0 lag: (min: 0.0, avg: 2.3, max: 7.0) [2024-09-30 03:25:40,466][1157520] Avg episode reward: [(0, '54.470')] [2024-09-30 03:25:40,767][1157819] Updated weights for policy 0, policy_version 193728 (0.0006) [2024-09-30 03:25:41,279][1157819] Updated weights for policy 0, policy_version 193738 (0.0006) [2024-09-30 03:25:41,795][1157819] Updated weights for policy 0, policy_version 193748 (0.0006) [2024-09-30 03:25:42,291][1157819] Updated weights for policy 0, policy_version 193758 (0.0006) [2024-09-30 03:25:42,791][1157819] Updated weights for policy 0, policy_version 193768 (0.0006) [2024-09-30 03:25:43,327][1157819] Updated weights for policy 0, policy_version 193778 (0.0006) [2024-09-30 03:25:43,835][1157819] Updated weights for policy 0, policy_version 193788 (0.0006) [2024-09-30 03:25:44,347][1157819] Updated weights for policy 0, policy_version 193798 (0.0007) [2024-09-30 03:25:44,401][1157736] Signal inference workers to stop experience collection... (13400 times) [2024-09-30 03:25:44,404][1157819] InferenceWorker_p0-w0: stopping experience collection (13400 times) [2024-09-30 03:25:44,412][1157736] Signal inference workers to resume experience collection... (13400 times) [2024-09-30 03:25:44,412][1157819] InferenceWorker_p0-w0: resuming experience collection (13400 times) [2024-09-30 03:25:44,887][1157819] Updated weights for policy 0, policy_version 193808 (0.0007) [2024-09-30 03:25:45,389][1157819] Updated weights for policy 0, policy_version 193818 (0.0006) [2024-09-30 03:25:45,466][1157520] Fps is (10 sec: 79463.5, 60 sec: 75366.4, 300 sec: 73728.0). Total num frames: 793882624. Throughput: 0: 18809.1. Samples: 188438020. Policy #0 lag: (min: 0.0, avg: 1.9, max: 4.0) [2024-09-30 03:25:45,466][1157520] Avg episode reward: [(0, '57.790')] [2024-09-30 03:25:45,881][1157819] Updated weights for policy 0, policy_version 193828 (0.0006) [2024-09-30 03:25:46,366][1157819] Updated weights for policy 0, policy_version 193838 (0.0006) [2024-09-30 03:25:46,911][1157819] Updated weights for policy 0, policy_version 193848 (0.0006) [2024-09-30 03:25:47,425][1157819] Updated weights for policy 0, policy_version 193858 (0.0007) [2024-09-30 03:25:47,928][1157819] Updated weights for policy 0, policy_version 193868 (0.0006) [2024-09-30 03:25:48,438][1157819] Updated weights for policy 0, policy_version 193878 (0.0006) [2024-09-30 03:25:48,971][1157819] Updated weights for policy 0, policy_version 193888 (0.0006) [2024-09-30 03:25:49,489][1157819] Updated weights for policy 0, policy_version 193898 (0.0006) [2024-09-30 03:25:49,992][1157819] Updated weights for policy 0, policy_version 193908 (0.0006) [2024-09-30 03:25:50,466][1157520] Fps is (10 sec: 79872.6, 60 sec: 75502.9, 300 sec: 73825.2). Total num frames: 794284032. Throughput: 0: 18943.2. Samples: 188557516. Policy #0 lag: (min: 0.0, avg: 1.9, max: 4.0) [2024-09-30 03:25:50,466][1157520] Avg episode reward: [(0, '57.599')] [2024-09-30 03:25:50,523][1157819] Updated weights for policy 0, policy_version 193918 (0.0006) [2024-09-30 03:25:51,027][1157819] Updated weights for policy 0, policy_version 193928 (0.0006) [2024-09-30 03:25:51,537][1157819] Updated weights for policy 0, policy_version 193938 (0.0006) [2024-09-30 03:25:52,067][1157819] Updated weights for policy 0, policy_version 193948 (0.0006) [2024-09-30 03:25:52,582][1157819] Updated weights for policy 0, policy_version 193958 (0.0006) [2024-09-30 03:25:53,076][1157819] Updated weights for policy 0, policy_version 193968 (0.0006) [2024-09-30 03:25:53,594][1157819] Updated weights for policy 0, policy_version 193978 (0.0006) [2024-09-30 03:25:54,124][1157819] Updated weights for policy 0, policy_version 193988 (0.0006) [2024-09-30 03:25:54,616][1157819] Updated weights for policy 0, policy_version 193998 (0.0006) [2024-09-30 03:25:55,152][1157819] Updated weights for policy 0, policy_version 194008 (0.0006) [2024-09-30 03:25:55,466][1157520] Fps is (10 sec: 79872.0, 60 sec: 75844.4, 300 sec: 73908.5). Total num frames: 794681344. Throughput: 0: 18952.1. Samples: 188617104. Policy #0 lag: (min: 0.0, avg: 1.9, max: 4.0) [2024-09-30 03:25:55,466][1157520] Avg episode reward: [(0, '55.654')] [2024-09-30 03:25:55,650][1157819] Updated weights for policy 0, policy_version 194018 (0.0006) [2024-09-30 03:25:56,156][1157819] Updated weights for policy 0, policy_version 194028 (0.0006) [2024-09-30 03:25:56,677][1157819] Updated weights for policy 0, policy_version 194038 (0.0006) [2024-09-30 03:25:57,184][1157819] Updated weights for policy 0, policy_version 194048 (0.0006) [2024-09-30 03:25:57,694][1157819] Updated weights for policy 0, policy_version 194058 (0.0006) [2024-09-30 03:25:58,236][1157819] Updated weights for policy 0, policy_version 194068 (0.0006) [2024-09-30 03:25:58,756][1157819] Updated weights for policy 0, policy_version 194078 (0.0006) [2024-09-30 03:25:59,276][1157819] Updated weights for policy 0, policy_version 194088 (0.0006) [2024-09-30 03:25:59,818][1157819] Updated weights for policy 0, policy_version 194098 (0.0006) [2024-09-30 03:26:00,367][1157819] Updated weights for policy 0, policy_version 194108 (0.0006) [2024-09-30 03:26:00,466][1157520] Fps is (10 sec: 78642.9, 60 sec: 76458.7, 300 sec: 73978.0). Total num frames: 795070464. Throughput: 0: 19040.4. Samples: 188736608. Policy #0 lag: (min: 0.0, avg: 1.9, max: 4.0) [2024-09-30 03:26:00,466][1157520] Avg episode reward: [(0, '56.834')] [2024-09-30 03:26:00,883][1157819] Updated weights for policy 0, policy_version 194118 (0.0006) [2024-09-30 03:26:01,395][1157819] Updated weights for policy 0, policy_version 194128 (0.0006) [2024-09-30 03:26:01,921][1157819] Updated weights for policy 0, policy_version 194138 (0.0006) [2024-09-30 03:26:02,447][1157819] Updated weights for policy 0, policy_version 194148 (0.0006) [2024-09-30 03:26:02,978][1157819] Updated weights for policy 0, policy_version 194158 (0.0006) [2024-09-30 03:26:03,502][1157819] Updated weights for policy 0, policy_version 194168 (0.0006) [2024-09-30 03:26:04,016][1157819] Updated weights for policy 0, policy_version 194178 (0.0006) [2024-09-30 03:26:04,579][1157819] Updated weights for policy 0, policy_version 194188 (0.0006) [2024-09-30 03:26:05,141][1157819] Updated weights for policy 0, policy_version 194198 (0.0006) [2024-09-30 03:26:05,466][1157520] Fps is (10 sec: 77823.8, 60 sec: 76458.9, 300 sec: 74061.2). Total num frames: 795459584. Throughput: 0: 19138.6. Samples: 188852332. Policy #0 lag: (min: 0.0, avg: 1.9, max: 4.0) [2024-09-30 03:26:05,466][1157520] Avg episode reward: [(0, '56.627')] [2024-09-30 03:26:05,488][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000194205_795463680.pth... [2024-09-30 03:26:05,542][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000189867_777695232.pth [2024-09-30 03:26:05,647][1157819] Updated weights for policy 0, policy_version 194208 (0.0006) [2024-09-30 03:26:06,185][1157819] Updated weights for policy 0, policy_version 194218 (0.0006) [2024-09-30 03:26:06,744][1157819] Updated weights for policy 0, policy_version 194228 (0.0006) [2024-09-30 03:26:07,295][1157819] Updated weights for policy 0, policy_version 194238 (0.0006) [2024-09-30 03:26:07,848][1157819] Updated weights for policy 0, policy_version 194248 (0.0006) [2024-09-30 03:26:08,412][1157819] Updated weights for policy 0, policy_version 194258 (0.0006) [2024-09-30 03:26:08,989][1157819] Updated weights for policy 0, policy_version 194268 (0.0006) [2024-09-30 03:26:09,503][1157819] Updated weights for policy 0, policy_version 194278 (0.0007) [2024-09-30 03:26:10,057][1157819] Updated weights for policy 0, policy_version 194288 (0.0006) [2024-09-30 03:26:10,466][1157520] Fps is (10 sec: 76185.7, 60 sec: 76117.4, 300 sec: 74075.2). Total num frames: 795832320. Throughput: 0: 19158.9. Samples: 188908896. Policy #0 lag: (min: 0.0, avg: 1.9, max: 4.0) [2024-09-30 03:26:10,466][1157520] Avg episode reward: [(0, '54.676')] [2024-09-30 03:26:10,593][1157819] Updated weights for policy 0, policy_version 194298 (0.0006) [2024-09-30 03:26:11,138][1157819] Updated weights for policy 0, policy_version 194308 (0.0006) [2024-09-30 03:26:11,562][1157736] Signal inference workers to stop experience collection... (13450 times) [2024-09-30 03:26:11,562][1157736] Signal inference workers to resume experience collection... (13450 times) [2024-09-30 03:26:11,567][1157819] InferenceWorker_p0-w0: stopping experience collection (13450 times) [2024-09-30 03:26:11,567][1157819] InferenceWorker_p0-w0: resuming experience collection (13450 times) [2024-09-30 03:26:11,702][1157819] Updated weights for policy 0, policy_version 194318 (0.0006) [2024-09-30 03:26:12,257][1157819] Updated weights for policy 0, policy_version 194328 (0.0006) [2024-09-30 03:26:12,834][1157819] Updated weights for policy 0, policy_version 194338 (0.0006) [2024-09-30 03:26:13,351][1157819] Updated weights for policy 0, policy_version 194348 (0.0006) [2024-09-30 03:26:13,906][1157819] Updated weights for policy 0, policy_version 194358 (0.0006) [2024-09-30 03:26:14,456][1157819] Updated weights for policy 0, policy_version 194368 (0.0006) [2024-09-30 03:26:14,965][1157819] Updated weights for policy 0, policy_version 194378 (0.0006) [2024-09-30 03:26:15,462][1157819] Updated weights for policy 0, policy_version 194388 (0.0006) [2024-09-30 03:26:15,466][1157520] Fps is (10 sec: 75366.0, 60 sec: 76253.9, 300 sec: 74144.6). Total num frames: 796213248. Throughput: 0: 19275.5. Samples: 189020356. Policy #0 lag: (min: 0.0, avg: 1.9, max: 4.0) [2024-09-30 03:26:15,466][1157520] Avg episode reward: [(0, '54.876')] [2024-09-30 03:26:15,973][1157819] Updated weights for policy 0, policy_version 194398 (0.0006) [2024-09-30 03:26:16,467][1157819] Updated weights for policy 0, policy_version 194408 (0.0006) [2024-09-30 03:26:17,004][1157819] Updated weights for policy 0, policy_version 194418 (0.0006) [2024-09-30 03:26:17,502][1157819] Updated weights for policy 0, policy_version 194428 (0.0006) [2024-09-30 03:26:18,021][1157819] Updated weights for policy 0, policy_version 194438 (0.0006) [2024-09-30 03:26:18,517][1157819] Updated weights for policy 0, policy_version 194448 (0.0006) [2024-09-30 03:26:19,067][1157819] Updated weights for policy 0, policy_version 194458 (0.0006) [2024-09-30 03:26:19,573][1157819] Updated weights for policy 0, policy_version 194468 (0.0006) [2024-09-30 03:26:20,071][1157819] Updated weights for policy 0, policy_version 194478 (0.0006) [2024-09-30 03:26:20,466][1157520] Fps is (10 sec: 77823.4, 60 sec: 76663.5, 300 sec: 74214.0). Total num frames: 796610560. Throughput: 0: 19539.7. Samples: 189139412. Policy #0 lag: (min: 0.0, avg: 1.9, max: 4.0) [2024-09-30 03:26:20,466][1157520] Avg episode reward: [(0, '55.896')] [2024-09-30 03:26:20,606][1157819] Updated weights for policy 0, policy_version 194488 (0.0006) [2024-09-30 03:26:21,145][1157819] Updated weights for policy 0, policy_version 194498 (0.0006) [2024-09-30 03:26:21,672][1157819] Updated weights for policy 0, policy_version 194508 (0.0006) [2024-09-30 03:26:22,199][1157819] Updated weights for policy 0, policy_version 194518 (0.0006) [2024-09-30 03:26:22,719][1157819] Updated weights for policy 0, policy_version 194528 (0.0006) [2024-09-30 03:26:23,232][1157819] Updated weights for policy 0, policy_version 194538 (0.0006) [2024-09-30 03:26:23,740][1157819] Updated weights for policy 0, policy_version 194548 (0.0006) [2024-09-30 03:26:24,230][1157819] Updated weights for policy 0, policy_version 194558 (0.0006) [2024-09-30 03:26:24,731][1157819] Updated weights for policy 0, policy_version 194568 (0.0006) [2024-09-30 03:26:25,228][1157819] Updated weights for policy 0, policy_version 194578 (0.0006) [2024-09-30 03:26:25,466][1157520] Fps is (10 sec: 79462.3, 60 sec: 77277.9, 300 sec: 74311.2). Total num frames: 797007872. Throughput: 0: 19557.7. Samples: 189197800. Policy #0 lag: (min: 0.0, avg: 1.9, max: 4.0) [2024-09-30 03:26:25,466][1157520] Avg episode reward: [(0, '56.159')] [2024-09-30 03:26:25,716][1157819] Updated weights for policy 0, policy_version 194588 (0.0006) [2024-09-30 03:26:26,201][1157819] Updated weights for policy 0, policy_version 194598 (0.0006) [2024-09-30 03:26:26,720][1157819] Updated weights for policy 0, policy_version 194608 (0.0006) [2024-09-30 03:26:27,218][1157819] Updated weights for policy 0, policy_version 194618 (0.0006) [2024-09-30 03:26:27,695][1157819] Updated weights for policy 0, policy_version 194628 (0.0006) [2024-09-30 03:26:28,189][1157819] Updated weights for policy 0, policy_version 194638 (0.0006) [2024-09-30 03:26:28,661][1157819] Updated weights for policy 0, policy_version 194648 (0.0006) [2024-09-30 03:26:29,200][1157819] Updated weights for policy 0, policy_version 194658 (0.0006) [2024-09-30 03:26:29,706][1157819] Updated weights for policy 0, policy_version 194668 (0.0006) [2024-09-30 03:26:30,206][1157819] Updated weights for policy 0, policy_version 194678 (0.0006) [2024-09-30 03:26:30,466][1157520] Fps is (10 sec: 80691.2, 60 sec: 78165.5, 300 sec: 74505.6). Total num frames: 797417472. Throughput: 0: 19639.3. Samples: 189321788. Policy #0 lag: (min: 0.0, avg: 1.9, max: 4.0) [2024-09-30 03:26:30,466][1157520] Avg episode reward: [(0, '54.854')] [2024-09-30 03:26:30,737][1157819] Updated weights for policy 0, policy_version 194688 (0.0006) [2024-09-30 03:26:31,265][1157819] Updated weights for policy 0, policy_version 194698 (0.0006) [2024-09-30 03:26:31,798][1157819] Updated weights for policy 0, policy_version 194708 (0.0006) [2024-09-30 03:26:32,318][1157819] Updated weights for policy 0, policy_version 194718 (0.0006) [2024-09-30 03:26:32,810][1157819] Updated weights for policy 0, policy_version 194728 (0.0006) [2024-09-30 03:26:33,331][1157819] Updated weights for policy 0, policy_version 194738 (0.0006) [2024-09-30 03:26:33,857][1157819] Updated weights for policy 0, policy_version 194748 (0.0006) [2024-09-30 03:26:34,380][1157819] Updated weights for policy 0, policy_version 194758 (0.0006) [2024-09-30 03:26:34,928][1157819] Updated weights for policy 0, policy_version 194768 (0.0006) [2024-09-30 03:26:35,466][1157520] Fps is (10 sec: 79872.0, 60 sec: 78643.3, 300 sec: 74630.5). Total num frames: 797806592. Throughput: 0: 19612.2. Samples: 189440068. Policy #0 lag: (min: 0.0, avg: 1.9, max: 4.0) [2024-09-30 03:26:35,466][1157520] Avg episode reward: [(0, '55.712')] [2024-09-30 03:26:35,477][1157819] Updated weights for policy 0, policy_version 194778 (0.0006) [2024-09-30 03:26:36,021][1157819] Updated weights for policy 0, policy_version 194788 (0.0006) [2024-09-30 03:26:36,537][1157819] Updated weights for policy 0, policy_version 194798 (0.0006) [2024-09-30 03:26:37,077][1157819] Updated weights for policy 0, policy_version 194808 (0.0006) [2024-09-30 03:26:37,605][1157819] Updated weights for policy 0, policy_version 194818 (0.0006) [2024-09-30 03:26:38,162][1157819] Updated weights for policy 0, policy_version 194828 (0.0006) [2024-09-30 03:26:38,730][1157819] Updated weights for policy 0, policy_version 194838 (0.0006) [2024-09-30 03:26:39,253][1157819] Updated weights for policy 0, policy_version 194848 (0.0006) [2024-09-30 03:26:39,803][1157819] Updated weights for policy 0, policy_version 194858 (0.0006) [2024-09-30 03:26:40,341][1157819] Updated weights for policy 0, policy_version 194868 (0.0006) [2024-09-30 03:26:40,466][1157520] Fps is (10 sec: 76595.3, 60 sec: 78301.8, 300 sec: 74672.2). Total num frames: 798183424. Throughput: 0: 19549.0. Samples: 189496808. Policy #0 lag: (min: 0.0, avg: 1.9, max: 4.0) [2024-09-30 03:26:40,466][1157520] Avg episode reward: [(0, '56.340')] [2024-09-30 03:26:40,924][1157819] Updated weights for policy 0, policy_version 194878 (0.0006) [2024-09-30 03:26:41,546][1157819] Updated weights for policy 0, policy_version 194888 (0.0007) [2024-09-30 03:26:42,118][1157819] Updated weights for policy 0, policy_version 194898 (0.0006) [2024-09-30 03:26:42,725][1157819] Updated weights for policy 0, policy_version 194908 (0.0006) [2024-09-30 03:26:43,280][1157819] Updated weights for policy 0, policy_version 194918 (0.0006) [2024-09-30 03:26:43,889][1157819] Updated weights for policy 0, policy_version 194928 (0.0006) [2024-09-30 03:26:44,457][1157819] Updated weights for policy 0, policy_version 194938 (0.0006) [2024-09-30 03:26:45,035][1157819] Updated weights for policy 0, policy_version 194948 (0.0006) [2024-09-30 03:26:45,466][1157520] Fps is (10 sec: 72909.0, 60 sec: 77550.9, 300 sec: 74533.3). Total num frames: 798535680. Throughput: 0: 19296.4. Samples: 189604948. Policy #0 lag: (min: 0.0, avg: 1.9, max: 4.0) [2024-09-30 03:26:45,466][1157520] Avg episode reward: [(0, '54.520')] [2024-09-30 03:26:45,597][1157819] Updated weights for policy 0, policy_version 194958 (0.0006) [2024-09-30 03:26:46,168][1157819] Updated weights for policy 0, policy_version 194968 (0.0006) [2024-09-30 03:26:46,763][1157819] Updated weights for policy 0, policy_version 194978 (0.0006) [2024-09-30 03:26:47,355][1157819] Updated weights for policy 0, policy_version 194988 (0.0006) [2024-09-30 03:26:47,932][1157819] Updated weights for policy 0, policy_version 194998 (0.0006) [2024-09-30 03:26:48,047][1157736] Signal inference workers to stop experience collection... (13500 times) [2024-09-30 03:26:48,048][1157736] Signal inference workers to resume experience collection... (13500 times) [2024-09-30 03:26:48,052][1157819] InferenceWorker_p0-w0: stopping experience collection (13500 times) [2024-09-30 03:26:48,054][1157819] InferenceWorker_p0-w0: resuming experience collection (13500 times) [2024-09-30 03:26:48,531][1157819] Updated weights for policy 0, policy_version 195008 (0.0006) [2024-09-30 03:26:49,115][1157819] Updated weights for policy 0, policy_version 195018 (0.0006) [2024-09-30 03:26:49,695][1157819] Updated weights for policy 0, policy_version 195028 (0.0006) [2024-09-30 03:26:50,257][1157819] Updated weights for policy 0, policy_version 195038 (0.0006) [2024-09-30 03:26:50,466][1157520] Fps is (10 sec: 70451.0, 60 sec: 76731.6, 300 sec: 74394.5). Total num frames: 798887936. Throughput: 0: 19078.2. Samples: 189710852. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:26:50,466][1157520] Avg episode reward: [(0, '56.395')] [2024-09-30 03:26:50,800][1157819] Updated weights for policy 0, policy_version 195048 (0.0006) [2024-09-30 03:26:51,364][1157819] Updated weights for policy 0, policy_version 195058 (0.0006) [2024-09-30 03:26:51,936][1157819] Updated weights for policy 0, policy_version 195068 (0.0006) [2024-09-30 03:26:52,562][1157819] Updated weights for policy 0, policy_version 195078 (0.0006) [2024-09-30 03:26:53,118][1157819] Updated weights for policy 0, policy_version 195088 (0.0006) [2024-09-30 03:26:53,716][1157819] Updated weights for policy 0, policy_version 195098 (0.0006) [2024-09-30 03:26:54,315][1157819] Updated weights for policy 0, policy_version 195108 (0.0006) [2024-09-30 03:26:54,897][1157819] Updated weights for policy 0, policy_version 195118 (0.0006) [2024-09-30 03:26:55,466][1157520] Fps is (10 sec: 70451.0, 60 sec: 75980.7, 300 sec: 74325.0). Total num frames: 799240192. Throughput: 0: 19009.3. Samples: 189764316. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:26:55,466][1157520] Avg episode reward: [(0, '56.202')] [2024-09-30 03:26:55,472][1157819] Updated weights for policy 0, policy_version 195128 (0.0006) [2024-09-30 03:26:56,073][1157819] Updated weights for policy 0, policy_version 195138 (0.0006) [2024-09-30 03:26:56,645][1157819] Updated weights for policy 0, policy_version 195148 (0.0006) [2024-09-30 03:26:57,220][1157819] Updated weights for policy 0, policy_version 195158 (0.0006) [2024-09-30 03:26:57,843][1157819] Updated weights for policy 0, policy_version 195168 (0.0006) [2024-09-30 03:26:58,394][1157819] Updated weights for policy 0, policy_version 195178 (0.0006) [2024-09-30 03:26:58,950][1157819] Updated weights for policy 0, policy_version 195188 (0.0006) [2024-09-30 03:26:59,511][1157819] Updated weights for policy 0, policy_version 195198 (0.0006) [2024-09-30 03:27:00,049][1157819] Updated weights for policy 0, policy_version 195208 (0.0006) [2024-09-30 03:27:00,466][1157520] Fps is (10 sec: 70860.9, 60 sec: 75434.6, 300 sec: 74241.8). Total num frames: 799596544. Throughput: 0: 18865.3. Samples: 189869292. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:27:00,466][1157520] Avg episode reward: [(0, '55.928')] [2024-09-30 03:27:00,605][1157819] Updated weights for policy 0, policy_version 195218 (0.0006) [2024-09-30 03:27:01,147][1157819] Updated weights for policy 0, policy_version 195228 (0.0006) [2024-09-30 03:27:01,702][1157819] Updated weights for policy 0, policy_version 195238 (0.0006) [2024-09-30 03:27:02,246][1157819] Updated weights for policy 0, policy_version 195248 (0.0006) [2024-09-30 03:27:02,832][1157819] Updated weights for policy 0, policy_version 195258 (0.0006) [2024-09-30 03:27:03,367][1157819] Updated weights for policy 0, policy_version 195268 (0.0006) [2024-09-30 03:27:03,966][1157819] Updated weights for policy 0, policy_version 195278 (0.0006) [2024-09-30 03:27:04,555][1157819] Updated weights for policy 0, policy_version 195288 (0.0006) [2024-09-30 03:27:05,145][1157819] Updated weights for policy 0, policy_version 195298 (0.0006) [2024-09-30 03:27:05,466][1157520] Fps is (10 sec: 72089.8, 60 sec: 75025.0, 300 sec: 74089.0). Total num frames: 799961088. Throughput: 0: 18658.7. Samples: 189979052. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:27:05,466][1157520] Avg episode reward: [(0, '56.681')] [2024-09-30 03:27:05,724][1157819] Updated weights for policy 0, policy_version 195308 (0.0006) [2024-09-30 03:27:06,269][1157819] Updated weights for policy 0, policy_version 195318 (0.0006) [2024-09-30 03:27:06,783][1157819] Updated weights for policy 0, policy_version 195328 (0.0006) [2024-09-30 03:27:07,310][1157819] Updated weights for policy 0, policy_version 195338 (0.0006) [2024-09-30 03:27:07,821][1157819] Updated weights for policy 0, policy_version 195348 (0.0006) [2024-09-30 03:27:08,347][1157819] Updated weights for policy 0, policy_version 195358 (0.0006) [2024-09-30 03:27:08,843][1157819] Updated weights for policy 0, policy_version 195368 (0.0006) [2024-09-30 03:27:09,374][1157819] Updated weights for policy 0, policy_version 195378 (0.0006) [2024-09-30 03:27:09,889][1157819] Updated weights for policy 0, policy_version 195388 (0.0006) [2024-09-30 03:27:10,439][1157819] Updated weights for policy 0, policy_version 195398 (0.0006) [2024-09-30 03:27:10,466][1157520] Fps is (10 sec: 75366.0, 60 sec: 75298.0, 300 sec: 74102.9). Total num frames: 800350208. Throughput: 0: 18620.9. Samples: 190035740. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:27:10,466][1157520] Avg episode reward: [(0, '58.183')] [2024-09-30 03:27:10,951][1157819] Updated weights for policy 0, policy_version 195408 (0.0006) [2024-09-30 03:27:11,463][1157819] Updated weights for policy 0, policy_version 195418 (0.0006) [2024-09-30 03:27:11,971][1157819] Updated weights for policy 0, policy_version 195428 (0.0006) [2024-09-30 03:27:12,521][1157819] Updated weights for policy 0, policy_version 195438 (0.0006) [2024-09-30 03:27:13,061][1157819] Updated weights for policy 0, policy_version 195448 (0.0006) [2024-09-30 03:27:13,559][1157819] Updated weights for policy 0, policy_version 195458 (0.0006) [2024-09-30 03:27:14,079][1157819] Updated weights for policy 0, policy_version 195468 (0.0006) [2024-09-30 03:27:14,633][1157819] Updated weights for policy 0, policy_version 195478 (0.0006) [2024-09-30 03:27:15,214][1157819] Updated weights for policy 0, policy_version 195488 (0.0006) [2024-09-30 03:27:15,466][1157520] Fps is (10 sec: 77414.3, 60 sec: 75366.4, 300 sec: 74172.3). Total num frames: 800735232. Throughput: 0: 18482.0. Samples: 190153480. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:27:15,466][1157520] Avg episode reward: [(0, '56.389')] [2024-09-30 03:27:15,758][1157819] Updated weights for policy 0, policy_version 195498 (0.0006) [2024-09-30 03:27:16,333][1157819] Updated weights for policy 0, policy_version 195508 (0.0006) [2024-09-30 03:27:16,877][1157819] Updated weights for policy 0, policy_version 195518 (0.0006) [2024-09-30 03:27:17,408][1157819] Updated weights for policy 0, policy_version 195528 (0.0006) [2024-09-30 03:27:17,949][1157819] Updated weights for policy 0, policy_version 195538 (0.0006) [2024-09-30 03:27:18,508][1157819] Updated weights for policy 0, policy_version 195548 (0.0006) [2024-09-30 03:27:19,040][1157819] Updated weights for policy 0, policy_version 195558 (0.0006) [2024-09-30 03:27:19,625][1157819] Updated weights for policy 0, policy_version 195568 (0.0006) [2024-09-30 03:27:20,168][1157819] Updated weights for policy 0, policy_version 195578 (0.0006) [2024-09-30 03:27:20,466][1157520] Fps is (10 sec: 75776.3, 60 sec: 74956.8, 300 sec: 74255.6). Total num frames: 801107968. Throughput: 0: 18329.2. Samples: 190264880. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:27:20,466][1157520] Avg episode reward: [(0, '56.874')] [2024-09-30 03:27:20,679][1157819] Updated weights for policy 0, policy_version 195588 (0.0006) [2024-09-30 03:27:21,249][1157819] Updated weights for policy 0, policy_version 195598 (0.0006) [2024-09-30 03:27:21,802][1157819] Updated weights for policy 0, policy_version 195608 (0.0006) [2024-09-30 03:27:22,360][1157819] Updated weights for policy 0, policy_version 195618 (0.0006) [2024-09-30 03:27:22,924][1157819] Updated weights for policy 0, policy_version 195628 (0.0006) [2024-09-30 03:27:23,487][1157819] Updated weights for policy 0, policy_version 195638 (0.0006) [2024-09-30 03:27:24,091][1157819] Updated weights for policy 0, policy_version 195648 (0.0006) [2024-09-30 03:27:24,674][1157819] Updated weights for policy 0, policy_version 195658 (0.0006) [2024-09-30 03:27:24,729][1157736] Signal inference workers to stop experience collection... (13550 times) [2024-09-30 03:27:24,731][1157736] Signal inference workers to resume experience collection... (13550 times) [2024-09-30 03:27:24,731][1157819] InferenceWorker_p0-w0: stopping experience collection (13550 times) [2024-09-30 03:27:24,734][1157819] InferenceWorker_p0-w0: resuming experience collection (13550 times) [2024-09-30 03:27:25,269][1157819] Updated weights for policy 0, policy_version 195668 (0.0007) [2024-09-30 03:27:25,466][1157520] Fps is (10 sec: 72907.4, 60 sec: 74273.9, 300 sec: 74255.6). Total num frames: 801464320. Throughput: 0: 18305.6. Samples: 190320564. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:27:25,466][1157520] Avg episode reward: [(0, '57.112')] [2024-09-30 03:27:25,886][1157819] Updated weights for policy 0, policy_version 195678 (0.0006) [2024-09-30 03:27:26,474][1157819] Updated weights for policy 0, policy_version 195688 (0.0006) [2024-09-30 03:27:27,064][1157819] Updated weights for policy 0, policy_version 195698 (0.0006) [2024-09-30 03:27:27,700][1157819] Updated weights for policy 0, policy_version 195708 (0.0006) [2024-09-30 03:27:28,246][1157819] Updated weights for policy 0, policy_version 195718 (0.0006) [2024-09-30 03:27:28,834][1157819] Updated weights for policy 0, policy_version 195728 (0.0006) [2024-09-30 03:27:29,457][1157819] Updated weights for policy 0, policy_version 195738 (0.0006) [2024-09-30 03:27:30,082][1157819] Updated weights for policy 0, policy_version 195748 (0.0006) [2024-09-30 03:27:30,466][1157520] Fps is (10 sec: 70041.5, 60 sec: 73181.8, 300 sec: 74338.9). Total num frames: 801808384. Throughput: 0: 18207.9. Samples: 190424304. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:27:30,466][1157520] Avg episode reward: [(0, '56.416')] [2024-09-30 03:27:30,709][1157819] Updated weights for policy 0, policy_version 195758 (0.0006) [2024-09-30 03:27:31,279][1157819] Updated weights for policy 0, policy_version 195768 (0.0006) [2024-09-30 03:27:31,895][1157819] Updated weights for policy 0, policy_version 195778 (0.0006) [2024-09-30 03:27:32,489][1157819] Updated weights for policy 0, policy_version 195788 (0.0006) [2024-09-30 03:27:33,081][1157819] Updated weights for policy 0, policy_version 195798 (0.0006) [2024-09-30 03:27:33,642][1157819] Updated weights for policy 0, policy_version 195808 (0.0006) [2024-09-30 03:27:34,216][1157819] Updated weights for policy 0, policy_version 195818 (0.0006) [2024-09-30 03:27:34,765][1157819] Updated weights for policy 0, policy_version 195828 (0.0006) [2024-09-30 03:27:35,336][1157819] Updated weights for policy 0, policy_version 195838 (0.0006) [2024-09-30 03:27:35,466][1157520] Fps is (10 sec: 69633.3, 60 sec: 72567.5, 300 sec: 74436.2). Total num frames: 802160640. Throughput: 0: 18176.1. Samples: 190528776. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:27:35,466][1157520] Avg episode reward: [(0, '56.562')] [2024-09-30 03:27:35,885][1157819] Updated weights for policy 0, policy_version 195848 (0.0006) [2024-09-30 03:27:36,462][1157819] Updated weights for policy 0, policy_version 195858 (0.0007) [2024-09-30 03:27:36,962][1157819] Updated weights for policy 0, policy_version 195868 (0.0006) [2024-09-30 03:27:37,504][1157819] Updated weights for policy 0, policy_version 195878 (0.0006) [2024-09-30 03:27:38,075][1157819] Updated weights for policy 0, policy_version 195888 (0.0006) [2024-09-30 03:27:38,580][1157819] Updated weights for policy 0, policy_version 195898 (0.0006) [2024-09-30 03:27:39,092][1157819] Updated weights for policy 0, policy_version 195908 (0.0006) [2024-09-30 03:27:39,587][1157819] Updated weights for policy 0, policy_version 195918 (0.0006) [2024-09-30 03:27:40,145][1157819] Updated weights for policy 0, policy_version 195928 (0.0006) [2024-09-30 03:27:40,466][1157520] Fps is (10 sec: 73727.4, 60 sec: 72703.9, 300 sec: 74644.4). Total num frames: 802545664. Throughput: 0: 18218.2. Samples: 190584136. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:27:40,466][1157520] Avg episode reward: [(0, '57.167')] [2024-09-30 03:27:40,672][1157819] Updated weights for policy 0, policy_version 195938 (0.0006) [2024-09-30 03:27:41,184][1157819] Updated weights for policy 0, policy_version 195948 (0.0006) [2024-09-30 03:27:41,702][1157819] Updated weights for policy 0, policy_version 195958 (0.0006) [2024-09-30 03:27:42,240][1157819] Updated weights for policy 0, policy_version 195968 (0.0006) [2024-09-30 03:27:42,777][1157819] Updated weights for policy 0, policy_version 195978 (0.0006) [2024-09-30 03:27:43,311][1157819] Updated weights for policy 0, policy_version 195988 (0.0006) [2024-09-30 03:27:43,826][1157819] Updated weights for policy 0, policy_version 195998 (0.0006) [2024-09-30 03:27:44,336][1157819] Updated weights for policy 0, policy_version 196008 (0.0006) [2024-09-30 03:27:44,872][1157819] Updated weights for policy 0, policy_version 196018 (0.0006) [2024-09-30 03:27:45,377][1157819] Updated weights for policy 0, policy_version 196028 (0.0006) [2024-09-30 03:27:45,466][1157520] Fps is (10 sec: 77414.1, 60 sec: 73318.3, 300 sec: 74866.5). Total num frames: 802934784. Throughput: 0: 18491.3. Samples: 190701400. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:27:45,466][1157520] Avg episode reward: [(0, '57.045')] [2024-09-30 03:27:45,888][1157819] Updated weights for policy 0, policy_version 196038 (0.0006) [2024-09-30 03:27:46,400][1157819] Updated weights for policy 0, policy_version 196048 (0.0006) [2024-09-30 03:27:46,988][1157819] Updated weights for policy 0, policy_version 196058 (0.0006) [2024-09-30 03:27:47,546][1157819] Updated weights for policy 0, policy_version 196068 (0.0007) [2024-09-30 03:27:48,125][1157819] Updated weights for policy 0, policy_version 196078 (0.0006) [2024-09-30 03:27:48,707][1157819] Updated weights for policy 0, policy_version 196088 (0.0006) [2024-09-30 03:27:49,263][1157819] Updated weights for policy 0, policy_version 196098 (0.0007) [2024-09-30 03:27:49,821][1157819] Updated weights for policy 0, policy_version 196108 (0.0006) [2024-09-30 03:27:50,373][1157819] Updated weights for policy 0, policy_version 196118 (0.0006) [2024-09-30 03:27:50,466][1157520] Fps is (10 sec: 75367.0, 60 sec: 73523.2, 300 sec: 74936.0). Total num frames: 803299328. Throughput: 0: 18552.6. Samples: 190813920. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:27:50,466][1157520] Avg episode reward: [(0, '56.918')] [2024-09-30 03:27:50,941][1157819] Updated weights for policy 0, policy_version 196128 (0.0007) [2024-09-30 03:27:51,522][1157736] Signal inference workers to stop experience collection... (13600 times) [2024-09-30 03:27:51,523][1157736] Signal inference workers to resume experience collection... (13600 times) [2024-09-30 03:27:51,526][1157819] InferenceWorker_p0-w0: stopping experience collection (13600 times) [2024-09-30 03:27:51,526][1157819] InferenceWorker_p0-w0: resuming experience collection (13600 times) [2024-09-30 03:27:51,542][1157819] Updated weights for policy 0, policy_version 196138 (0.0006) [2024-09-30 03:27:52,111][1157819] Updated weights for policy 0, policy_version 196148 (0.0006) [2024-09-30 03:27:52,696][1157819] Updated weights for policy 0, policy_version 196158 (0.0006) [2024-09-30 03:27:53,273][1157819] Updated weights for policy 0, policy_version 196168 (0.0006) [2024-09-30 03:27:53,875][1157819] Updated weights for policy 0, policy_version 196178 (0.0006) [2024-09-30 03:27:54,493][1157819] Updated weights for policy 0, policy_version 196188 (0.0006) [2024-09-30 03:27:55,089][1157819] Updated weights for policy 0, policy_version 196198 (0.0006) [2024-09-30 03:27:55,466][1157520] Fps is (10 sec: 71680.3, 60 sec: 73523.2, 300 sec: 75033.2). Total num frames: 803651584. Throughput: 0: 18485.6. Samples: 190867592. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:27:55,466][1157520] Avg episode reward: [(0, '57.359')] [2024-09-30 03:27:55,682][1157819] Updated weights for policy 0, policy_version 196208 (0.0006) [2024-09-30 03:27:56,269][1157819] Updated weights for policy 0, policy_version 196218 (0.0006) [2024-09-30 03:27:56,816][1157819] Updated weights for policy 0, policy_version 196228 (0.0006) [2024-09-30 03:27:57,393][1157819] Updated weights for policy 0, policy_version 196238 (0.0007) [2024-09-30 03:27:57,943][1157819] Updated weights for policy 0, policy_version 196248 (0.0007) [2024-09-30 03:27:58,536][1157819] Updated weights for policy 0, policy_version 196258 (0.0006) [2024-09-30 03:27:59,113][1157819] Updated weights for policy 0, policy_version 196268 (0.0006) [2024-09-30 03:27:59,710][1157819] Updated weights for policy 0, policy_version 196278 (0.0007) [2024-09-30 03:28:00,307][1157819] Updated weights for policy 0, policy_version 196288 (0.0006) [2024-09-30 03:28:00,466][1157520] Fps is (10 sec: 70451.2, 60 sec: 73454.9, 300 sec: 75102.6). Total num frames: 804003840. Throughput: 0: 18199.6. Samples: 190972464. Policy #0 lag: (min: 1.0, avg: 2.7, max: 6.0) [2024-09-30 03:28:00,466][1157520] Avg episode reward: [(0, '57.671')] [2024-09-30 03:28:00,882][1157819] Updated weights for policy 0, policy_version 196298 (0.0006) [2024-09-30 03:28:01,454][1157819] Updated weights for policy 0, policy_version 196308 (0.0006) [2024-09-30 03:28:02,029][1157819] Updated weights for policy 0, policy_version 196318 (0.0006) [2024-09-30 03:28:02,621][1157819] Updated weights for policy 0, policy_version 196328 (0.0006) [2024-09-30 03:28:03,243][1157819] Updated weights for policy 0, policy_version 196338 (0.0006) [2024-09-30 03:28:03,788][1157819] Updated weights for policy 0, policy_version 196348 (0.0006) [2024-09-30 03:28:04,373][1157819] Updated weights for policy 0, policy_version 196358 (0.0007) [2024-09-30 03:28:04,919][1157819] Updated weights for policy 0, policy_version 196368 (0.0006) [2024-09-30 03:28:05,466][1157520] Fps is (10 sec: 70860.7, 60 sec: 73318.3, 300 sec: 75102.6). Total num frames: 804360192. Throughput: 0: 18081.0. Samples: 191078524. Policy #0 lag: (min: 1.0, avg: 2.7, max: 6.0) [2024-09-30 03:28:05,466][1157520] Avg episode reward: [(0, '55.161')] [2024-09-30 03:28:05,470][1157819] Updated weights for policy 0, policy_version 196378 (0.0006) [2024-09-30 03:28:05,490][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000196379_804368384.pth... [2024-09-30 03:28:05,544][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000191990_786391040.pth [2024-09-30 03:28:05,968][1157819] Updated weights for policy 0, policy_version 196388 (0.0006) [2024-09-30 03:28:06,484][1157819] Updated weights for policy 0, policy_version 196398 (0.0006) [2024-09-30 03:28:07,062][1157819] Updated weights for policy 0, policy_version 196408 (0.0006) [2024-09-30 03:28:07,566][1157819] Updated weights for policy 0, policy_version 196418 (0.0006) [2024-09-30 03:28:08,149][1157819] Updated weights for policy 0, policy_version 196428 (0.0007) [2024-09-30 03:28:08,734][1157819] Updated weights for policy 0, policy_version 196438 (0.0006) [2024-09-30 03:28:09,315][1157819] Updated weights for policy 0, policy_version 196448 (0.0006) [2024-09-30 03:28:09,889][1157819] Updated weights for policy 0, policy_version 196458 (0.0006) [2024-09-30 03:28:10,466][1157520] Fps is (10 sec: 72499.4, 60 sec: 72977.1, 300 sec: 75130.3). Total num frames: 804728832. Throughput: 0: 18119.9. Samples: 191135956. Policy #0 lag: (min: 1.0, avg: 2.7, max: 6.0) [2024-09-30 03:28:10,466][1157520] Avg episode reward: [(0, '56.700')] [2024-09-30 03:28:10,469][1157819] Updated weights for policy 0, policy_version 196468 (0.0006) [2024-09-30 03:28:11,052][1157819] Updated weights for policy 0, policy_version 196478 (0.0006) [2024-09-30 03:28:11,677][1157819] Updated weights for policy 0, policy_version 196488 (0.0007) [2024-09-30 03:28:12,250][1157819] Updated weights for policy 0, policy_version 196498 (0.0007) [2024-09-30 03:28:12,809][1157819] Updated weights for policy 0, policy_version 196508 (0.0007) [2024-09-30 03:28:13,363][1157819] Updated weights for policy 0, policy_version 196518 (0.0006) [2024-09-30 03:28:13,928][1157819] Updated weights for policy 0, policy_version 196528 (0.0007) [2024-09-30 03:28:14,567][1157819] Updated weights for policy 0, policy_version 196538 (0.0007) [2024-09-30 03:28:15,135][1157819] Updated weights for policy 0, policy_version 196548 (0.0007) [2024-09-30 03:28:15,466][1157520] Fps is (10 sec: 72089.4, 60 sec: 72430.9, 300 sec: 75047.0). Total num frames: 805081088. Throughput: 0: 18164.2. Samples: 191241696. Policy #0 lag: (min: 1.0, avg: 2.7, max: 6.0) [2024-09-30 03:28:15,466][1157520] Avg episode reward: [(0, '56.545')] [2024-09-30 03:28:15,738][1157819] Updated weights for policy 0, policy_version 196558 (0.0007) [2024-09-30 03:28:16,355][1157819] Updated weights for policy 0, policy_version 196568 (0.0007) [2024-09-30 03:28:16,958][1157819] Updated weights for policy 0, policy_version 196578 (0.0006) [2024-09-30 03:28:17,515][1157736] Signal inference workers to stop experience collection... (13650 times) [2024-09-30 03:28:17,516][1157736] Signal inference workers to resume experience collection... (13650 times) [2024-09-30 03:28:17,520][1157819] InferenceWorker_p0-w0: stopping experience collection (13650 times) [2024-09-30 03:28:17,522][1157819] InferenceWorker_p0-w0: resuming experience collection (13650 times) [2024-09-30 03:28:17,538][1157819] Updated weights for policy 0, policy_version 196588 (0.0007) [2024-09-30 03:28:18,130][1157819] Updated weights for policy 0, policy_version 196598 (0.0006) [2024-09-30 03:28:18,747][1157819] Updated weights for policy 0, policy_version 196608 (0.0007) [2024-09-30 03:28:19,323][1157819] Updated weights for policy 0, policy_version 196618 (0.0007) [2024-09-30 03:28:19,890][1157819] Updated weights for policy 0, policy_version 196628 (0.0006) [2024-09-30 03:28:20,466][1157520] Fps is (10 sec: 69631.8, 60 sec: 71953.0, 300 sec: 74922.1). Total num frames: 805425152. Throughput: 0: 18145.6. Samples: 191345328. Policy #0 lag: (min: 1.0, avg: 2.7, max: 6.0) [2024-09-30 03:28:20,466][1157520] Avg episode reward: [(0, '57.763')] [2024-09-30 03:28:20,486][1157819] Updated weights for policy 0, policy_version 196638 (0.0007) [2024-09-30 03:28:21,054][1157819] Updated weights for policy 0, policy_version 196648 (0.0006) [2024-09-30 03:28:21,677][1157819] Updated weights for policy 0, policy_version 196658 (0.0006) [2024-09-30 03:28:22,230][1157819] Updated weights for policy 0, policy_version 196668 (0.0006) [2024-09-30 03:28:22,818][1157819] Updated weights for policy 0, policy_version 196678 (0.0007) [2024-09-30 03:28:23,408][1157819] Updated weights for policy 0, policy_version 196688 (0.0007) [2024-09-30 03:28:23,999][1157819] Updated weights for policy 0, policy_version 196698 (0.0006) [2024-09-30 03:28:24,596][1157819] Updated weights for policy 0, policy_version 196708 (0.0006) [2024-09-30 03:28:25,186][1157819] Updated weights for policy 0, policy_version 196718 (0.0006) [2024-09-30 03:28:25,466][1157520] Fps is (10 sec: 69222.5, 60 sec: 71816.7, 300 sec: 74852.7). Total num frames: 805773312. Throughput: 0: 18081.4. Samples: 191397800. Policy #0 lag: (min: 1.0, avg: 2.7, max: 6.0) [2024-09-30 03:28:25,466][1157520] Avg episode reward: [(0, '55.953')] [2024-09-30 03:28:25,754][1157819] Updated weights for policy 0, policy_version 196728 (0.0006) [2024-09-30 03:28:26,340][1157819] Updated weights for policy 0, policy_version 196738 (0.0006) [2024-09-30 03:28:26,916][1157819] Updated weights for policy 0, policy_version 196748 (0.0006) [2024-09-30 03:28:27,525][1157819] Updated weights for policy 0, policy_version 196758 (0.0006) [2024-09-30 03:28:28,134][1157819] Updated weights for policy 0, policy_version 196768 (0.0006) [2024-09-30 03:28:28,688][1157819] Updated weights for policy 0, policy_version 196778 (0.0006) [2024-09-30 03:28:29,315][1157819] Updated weights for policy 0, policy_version 196788 (0.0006) [2024-09-30 03:28:29,872][1157819] Updated weights for policy 0, policy_version 196798 (0.0006) [2024-09-30 03:28:30,466][1157520] Fps is (10 sec: 69632.1, 60 sec: 71884.8, 300 sec: 74769.3). Total num frames: 806121472. Throughput: 0: 17793.4. Samples: 191502100. Policy #0 lag: (min: 1.0, avg: 2.7, max: 6.0) [2024-09-30 03:28:30,466][1157520] Avg episode reward: [(0, '54.487')] [2024-09-30 03:28:30,487][1157819] Updated weights for policy 0, policy_version 196808 (0.0006) [2024-09-30 03:28:31,037][1157819] Updated weights for policy 0, policy_version 196818 (0.0006) [2024-09-30 03:28:31,639][1157819] Updated weights for policy 0, policy_version 196828 (0.0006) [2024-09-30 03:28:32,247][1157819] Updated weights for policy 0, policy_version 196838 (0.0006) [2024-09-30 03:28:32,822][1157819] Updated weights for policy 0, policy_version 196848 (0.0006) [2024-09-30 03:28:33,422][1157819] Updated weights for policy 0, policy_version 196858 (0.0006) [2024-09-30 03:28:34,041][1157819] Updated weights for policy 0, policy_version 196868 (0.0006) [2024-09-30 03:28:34,631][1157819] Updated weights for policy 0, policy_version 196878 (0.0006) [2024-09-30 03:28:35,213][1157819] Updated weights for policy 0, policy_version 196888 (0.0006) [2024-09-30 03:28:35,466][1157520] Fps is (10 sec: 69632.0, 60 sec: 71816.5, 300 sec: 74672.1). Total num frames: 806469632. Throughput: 0: 17605.7. Samples: 191606176. Policy #0 lag: (min: 1.0, avg: 2.7, max: 6.0) [2024-09-30 03:28:35,466][1157520] Avg episode reward: [(0, '56.061')] [2024-09-30 03:28:35,814][1157819] Updated weights for policy 0, policy_version 196898 (0.0006) [2024-09-30 03:28:36,405][1157819] Updated weights for policy 0, policy_version 196908 (0.0006) [2024-09-30 03:28:36,979][1157819] Updated weights for policy 0, policy_version 196918 (0.0006) [2024-09-30 03:28:37,591][1157819] Updated weights for policy 0, policy_version 196928 (0.0006) [2024-09-30 03:28:38,183][1157819] Updated weights for policy 0, policy_version 196938 (0.0006) [2024-09-30 03:28:38,779][1157819] Updated weights for policy 0, policy_version 196948 (0.0006) [2024-09-30 03:28:39,371][1157819] Updated weights for policy 0, policy_version 196958 (0.0006) [2024-09-30 03:28:39,945][1157819] Updated weights for policy 0, policy_version 196968 (0.0006) [2024-09-30 03:28:40,466][1157520] Fps is (10 sec: 69631.9, 60 sec: 71202.2, 300 sec: 74574.9). Total num frames: 806817792. Throughput: 0: 17553.6. Samples: 191657504. Policy #0 lag: (min: 1.0, avg: 2.7, max: 6.0) [2024-09-30 03:28:40,466][1157520] Avg episode reward: [(0, '56.656')] [2024-09-30 03:28:40,482][1157819] Updated weights for policy 0, policy_version 196978 (0.0006) [2024-09-30 03:28:41,053][1157819] Updated weights for policy 0, policy_version 196988 (0.0006) [2024-09-30 03:28:41,597][1157819] Updated weights for policy 0, policy_version 196998 (0.0006) [2024-09-30 03:28:42,155][1157819] Updated weights for policy 0, policy_version 197008 (0.0006) [2024-09-30 03:28:42,725][1157819] Updated weights for policy 0, policy_version 197018 (0.0006) [2024-09-30 03:28:43,263][1157819] Updated weights for policy 0, policy_version 197028 (0.0006) [2024-09-30 03:28:43,808][1157819] Updated weights for policy 0, policy_version 197038 (0.0006) [2024-09-30 03:28:44,334][1157819] Updated weights for policy 0, policy_version 197048 (0.0006) [2024-09-30 03:28:44,898][1157819] Updated weights for policy 0, policy_version 197058 (0.0006) [2024-09-30 03:28:45,458][1157819] Updated weights for policy 0, policy_version 197068 (0.0006) [2024-09-30 03:28:45,466][1157520] Fps is (10 sec: 72089.4, 60 sec: 70929.0, 300 sec: 74491.6). Total num frames: 807190528. Throughput: 0: 17650.3. Samples: 191766728. Policy #0 lag: (min: 1.0, avg: 2.7, max: 6.0) [2024-09-30 03:28:45,466][1157520] Avg episode reward: [(0, '53.856')] [2024-09-30 03:28:45,991][1157819] Updated weights for policy 0, policy_version 197078 (0.0006) [2024-09-30 03:28:46,566][1157819] Updated weights for policy 0, policy_version 197088 (0.0006) [2024-09-30 03:28:47,100][1157819] Updated weights for policy 0, policy_version 197098 (0.0006) [2024-09-30 03:28:47,629][1157819] Updated weights for policy 0, policy_version 197108 (0.0006) [2024-09-30 03:28:48,198][1157819] Updated weights for policy 0, policy_version 197118 (0.0006) [2024-09-30 03:28:48,735][1157819] Updated weights for policy 0, policy_version 197128 (0.0006) [2024-09-30 03:28:49,244][1157736] Signal inference workers to stop experience collection... (13700 times) [2024-09-30 03:28:49,244][1157736] Signal inference workers to resume experience collection... (13700 times) [2024-09-30 03:28:49,248][1157819] InferenceWorker_p0-w0: stopping experience collection (13700 times) [2024-09-30 03:28:49,248][1157819] InferenceWorker_p0-w0: resuming experience collection (13700 times) [2024-09-30 03:28:49,264][1157819] Updated weights for policy 0, policy_version 197138 (0.0006) [2024-09-30 03:28:49,882][1157819] Updated weights for policy 0, policy_version 197148 (0.0006) [2024-09-30 03:28:50,456][1157819] Updated weights for policy 0, policy_version 197158 (0.0006) [2024-09-30 03:28:50,466][1157520] Fps is (10 sec: 74137.7, 60 sec: 70997.3, 300 sec: 74463.9). Total num frames: 807559168. Throughput: 0: 17763.2. Samples: 191877868. Policy #0 lag: (min: 1.0, avg: 2.7, max: 6.0) [2024-09-30 03:28:50,466][1157520] Avg episode reward: [(0, '55.900')] [2024-09-30 03:28:51,040][1157819] Updated weights for policy 0, policy_version 197168 (0.0006) [2024-09-30 03:28:51,618][1157819] Updated weights for policy 0, policy_version 197178 (0.0006) [2024-09-30 03:28:52,182][1157819] Updated weights for policy 0, policy_version 197188 (0.0006) [2024-09-30 03:28:52,774][1157819] Updated weights for policy 0, policy_version 197198 (0.0006) [2024-09-30 03:28:53,328][1157819] Updated weights for policy 0, policy_version 197208 (0.0006) [2024-09-30 03:28:53,896][1157819] Updated weights for policy 0, policy_version 197218 (0.0006) [2024-09-30 03:28:54,466][1157819] Updated weights for policy 0, policy_version 197228 (0.0006) [2024-09-30 03:28:55,052][1157819] Updated weights for policy 0, policy_version 197238 (0.0006) [2024-09-30 03:28:55,466][1157520] Fps is (10 sec: 72089.6, 60 sec: 70997.3, 300 sec: 74311.1). Total num frames: 807911424. Throughput: 0: 17673.7. Samples: 191931272. Policy #0 lag: (min: 1.0, avg: 2.7, max: 6.0) [2024-09-30 03:28:55,466][1157520] Avg episode reward: [(0, '55.479')] [2024-09-30 03:28:55,634][1157819] Updated weights for policy 0, policy_version 197248 (0.0006) [2024-09-30 03:28:56,210][1157819] Updated weights for policy 0, policy_version 197258 (0.0006) [2024-09-30 03:28:56,793][1157819] Updated weights for policy 0, policy_version 197268 (0.0006) [2024-09-30 03:28:57,387][1157819] Updated weights for policy 0, policy_version 197278 (0.0006) [2024-09-30 03:28:57,998][1157819] Updated weights for policy 0, policy_version 197288 (0.0006) [2024-09-30 03:28:58,558][1157819] Updated weights for policy 0, policy_version 197298 (0.0006) [2024-09-30 03:28:59,142][1157819] Updated weights for policy 0, policy_version 197308 (0.0006) [2024-09-30 03:28:59,713][1157819] Updated weights for policy 0, policy_version 197318 (0.0006) [2024-09-30 03:29:00,266][1157819] Updated weights for policy 0, policy_version 197328 (0.0006) [2024-09-30 03:29:00,466][1157520] Fps is (10 sec: 70860.8, 60 sec: 71065.6, 300 sec: 74158.4). Total num frames: 808267776. Throughput: 0: 17681.8. Samples: 192037376. Policy #0 lag: (min: 1.0, avg: 2.7, max: 6.0) [2024-09-30 03:29:00,466][1157520] Avg episode reward: [(0, '53.849')] [2024-09-30 03:29:00,789][1157819] Updated weights for policy 0, policy_version 197338 (0.0006) [2024-09-30 03:29:01,336][1157819] Updated weights for policy 0, policy_version 197348 (0.0006) [2024-09-30 03:29:01,965][1157819] Updated weights for policy 0, policy_version 197358 (0.0007) [2024-09-30 03:29:02,621][1157819] Updated weights for policy 0, policy_version 197368 (0.0006) [2024-09-30 03:29:03,267][1157819] Updated weights for policy 0, policy_version 197378 (0.0007) [2024-09-30 03:29:03,897][1157819] Updated weights for policy 0, policy_version 197388 (0.0006) [2024-09-30 03:29:04,505][1157819] Updated weights for policy 0, policy_version 197398 (0.0006) [2024-09-30 03:29:05,113][1157819] Updated weights for policy 0, policy_version 197408 (0.0006) [2024-09-30 03:29:05,466][1157520] Fps is (10 sec: 69221.7, 60 sec: 70724.1, 300 sec: 74047.3). Total num frames: 808603648. Throughput: 0: 17664.9. Samples: 192140252. Policy #0 lag: (min: 1.0, avg: 2.5, max: 5.0) [2024-09-30 03:29:05,466][1157520] Avg episode reward: [(0, '56.363')] [2024-09-30 03:29:05,781][1157819] Updated weights for policy 0, policy_version 197418 (0.0006) [2024-09-30 03:29:06,389][1157819] Updated weights for policy 0, policy_version 197428 (0.0006) [2024-09-30 03:29:07,032][1157819] Updated weights for policy 0, policy_version 197438 (0.0006) [2024-09-30 03:29:07,619][1157819] Updated weights for policy 0, policy_version 197448 (0.0006) [2024-09-30 03:29:08,296][1157819] Updated weights for policy 0, policy_version 197458 (0.0006) [2024-09-30 03:29:08,941][1157819] Updated weights for policy 0, policy_version 197468 (0.0006) [2024-09-30 03:29:09,572][1157819] Updated weights for policy 0, policy_version 197478 (0.0006) [2024-09-30 03:29:10,195][1157819] Updated weights for policy 0, policy_version 197488 (0.0006) [2024-09-30 03:29:10,466][1157520] Fps is (10 sec: 65945.2, 60 sec: 69973.2, 300 sec: 73852.9). Total num frames: 808927232. Throughput: 0: 17587.5. Samples: 192189240. Policy #0 lag: (min: 1.0, avg: 2.5, max: 5.0) [2024-09-30 03:29:10,466][1157520] Avg episode reward: [(0, '55.354')] [2024-09-30 03:29:10,855][1157819] Updated weights for policy 0, policy_version 197498 (0.0006) [2024-09-30 03:29:11,461][1157819] Updated weights for policy 0, policy_version 197508 (0.0006) [2024-09-30 03:29:12,038][1157819] Updated weights for policy 0, policy_version 197518 (0.0006) [2024-09-30 03:29:12,605][1157819] Updated weights for policy 0, policy_version 197528 (0.0006) [2024-09-30 03:29:13,182][1157819] Updated weights for policy 0, policy_version 197538 (0.0006) [2024-09-30 03:29:13,783][1157819] Updated weights for policy 0, policy_version 197548 (0.0006) [2024-09-30 03:29:14,351][1157819] Updated weights for policy 0, policy_version 197558 (0.0006) [2024-09-30 03:29:14,907][1157819] Updated weights for policy 0, policy_version 197568 (0.0006) [2024-09-30 03:29:15,466][1157520] Fps is (10 sec: 66764.8, 60 sec: 69836.7, 300 sec: 73797.4). Total num frames: 809271296. Throughput: 0: 17498.6. Samples: 192289540. Policy #0 lag: (min: 1.0, avg: 2.5, max: 5.0) [2024-09-30 03:29:15,466][1157520] Avg episode reward: [(0, '56.235')] [2024-09-30 03:29:15,501][1157819] Updated weights for policy 0, policy_version 197578 (0.0006) [2024-09-30 03:29:16,072][1157819] Updated weights for policy 0, policy_version 197588 (0.0006) [2024-09-30 03:29:16,650][1157819] Updated weights for policy 0, policy_version 197598 (0.0006) [2024-09-30 03:29:17,256][1157819] Updated weights for policy 0, policy_version 197608 (0.0006) [2024-09-30 03:29:17,836][1157819] Updated weights for policy 0, policy_version 197618 (0.0006) [2024-09-30 03:29:18,422][1157819] Updated weights for policy 0, policy_version 197628 (0.0006) [2024-09-30 03:29:19,004][1157819] Updated weights for policy 0, policy_version 197638 (0.0006) [2024-09-30 03:29:19,369][1157736] Signal inference workers to stop experience collection... (13750 times) [2024-09-30 03:29:19,370][1157736] Signal inference workers to resume experience collection... (13750 times) [2024-09-30 03:29:19,375][1157819] InferenceWorker_p0-w0: stopping experience collection (13750 times) [2024-09-30 03:29:19,375][1157819] InferenceWorker_p0-w0: resuming experience collection (13750 times) [2024-09-30 03:29:19,555][1157819] Updated weights for policy 0, policy_version 197648 (0.0006) [2024-09-30 03:29:20,130][1157819] Updated weights for policy 0, policy_version 197658 (0.0006) [2024-09-30 03:29:20,466][1157520] Fps is (10 sec: 70451.2, 60 sec: 70109.8, 300 sec: 73811.3). Total num frames: 809631744. Throughput: 0: 17540.6. Samples: 192395504. Policy #0 lag: (min: 1.0, avg: 2.5, max: 5.0) [2024-09-30 03:29:20,466][1157520] Avg episode reward: [(0, '56.364')] [2024-09-30 03:29:20,673][1157819] Updated weights for policy 0, policy_version 197668 (0.0006) [2024-09-30 03:29:21,240][1157819] Updated weights for policy 0, policy_version 197678 (0.0006) [2024-09-30 03:29:21,786][1157819] Updated weights for policy 0, policy_version 197688 (0.0006) [2024-09-30 03:29:22,323][1157819] Updated weights for policy 0, policy_version 197698 (0.0006) [2024-09-30 03:29:22,906][1157819] Updated weights for policy 0, policy_version 197708 (0.0006) [2024-09-30 03:29:23,428][1157819] Updated weights for policy 0, policy_version 197718 (0.0006) [2024-09-30 03:29:23,974][1157819] Updated weights for policy 0, policy_version 197728 (0.0006) [2024-09-30 03:29:24,493][1157819] Updated weights for policy 0, policy_version 197738 (0.0006) [2024-09-30 03:29:24,994][1157819] Updated weights for policy 0, policy_version 197748 (0.0006) [2024-09-30 03:29:25,466][1157520] Fps is (10 sec: 73728.9, 60 sec: 70587.7, 300 sec: 73825.2). Total num frames: 810008576. Throughput: 0: 17640.9. Samples: 192451344. Policy #0 lag: (min: 1.0, avg: 2.5, max: 5.0) [2024-09-30 03:29:25,466][1157520] Avg episode reward: [(0, '54.889')] [2024-09-30 03:29:25,559][1157819] Updated weights for policy 0, policy_version 197758 (0.0006) [2024-09-30 03:29:26,150][1157819] Updated weights for policy 0, policy_version 197768 (0.0006) [2024-09-30 03:29:26,782][1157819] Updated weights for policy 0, policy_version 197778 (0.0006) [2024-09-30 03:29:27,364][1157819] Updated weights for policy 0, policy_version 197788 (0.0006) [2024-09-30 03:29:27,945][1157819] Updated weights for policy 0, policy_version 197798 (0.0006) [2024-09-30 03:29:28,502][1157819] Updated weights for policy 0, policy_version 197808 (0.0006) [2024-09-30 03:29:29,111][1157819] Updated weights for policy 0, policy_version 197818 (0.0006) [2024-09-30 03:29:29,681][1157819] Updated weights for policy 0, policy_version 197828 (0.0006) [2024-09-30 03:29:30,278][1157819] Updated weights for policy 0, policy_version 197838 (0.0006) [2024-09-30 03:29:30,466][1157520] Fps is (10 sec: 72499.5, 60 sec: 70587.7, 300 sec: 73741.9). Total num frames: 810356736. Throughput: 0: 17631.8. Samples: 192560156. Policy #0 lag: (min: 1.0, avg: 2.5, max: 5.0) [2024-09-30 03:29:30,466][1157520] Avg episode reward: [(0, '55.991')] [2024-09-30 03:29:30,866][1157819] Updated weights for policy 0, policy_version 197848 (0.0006) [2024-09-30 03:29:31,488][1157819] Updated weights for policy 0, policy_version 197858 (0.0006) [2024-09-30 03:29:32,046][1157819] Updated weights for policy 0, policy_version 197868 (0.0006) [2024-09-30 03:29:32,645][1157819] Updated weights for policy 0, policy_version 197878 (0.0006) [2024-09-30 03:29:33,192][1157819] Updated weights for policy 0, policy_version 197888 (0.0006) [2024-09-30 03:29:33,795][1157819] Updated weights for policy 0, policy_version 197898 (0.0006) [2024-09-30 03:29:34,383][1157819] Updated weights for policy 0, policy_version 197908 (0.0006) [2024-09-30 03:29:34,950][1157819] Updated weights for policy 0, policy_version 197918 (0.0006) [2024-09-30 03:29:35,466][1157520] Fps is (10 sec: 70041.0, 60 sec: 70655.9, 300 sec: 73644.6). Total num frames: 810708992. Throughput: 0: 17497.0. Samples: 192665236. Policy #0 lag: (min: 1.0, avg: 2.5, max: 5.0) [2024-09-30 03:29:35,466][1157520] Avg episode reward: [(0, '55.812')] [2024-09-30 03:29:35,497][1157819] Updated weights for policy 0, policy_version 197928 (0.0006) [2024-09-30 03:29:36,065][1157819] Updated weights for policy 0, policy_version 197938 (0.0006) [2024-09-30 03:29:36,624][1157819] Updated weights for policy 0, policy_version 197948 (0.0006) [2024-09-30 03:29:37,182][1157819] Updated weights for policy 0, policy_version 197958 (0.0006) [2024-09-30 03:29:37,709][1157819] Updated weights for policy 0, policy_version 197968 (0.0006) [2024-09-30 03:29:38,284][1157819] Updated weights for policy 0, policy_version 197978 (0.0006) [2024-09-30 03:29:38,839][1157819] Updated weights for policy 0, policy_version 197988 (0.0006) [2024-09-30 03:29:39,356][1157819] Updated weights for policy 0, policy_version 197998 (0.0006) [2024-09-30 03:29:39,905][1157819] Updated weights for policy 0, policy_version 198008 (0.0006) [2024-09-30 03:29:40,466][1157520] Fps is (10 sec: 72089.7, 60 sec: 70997.3, 300 sec: 73616.9). Total num frames: 811077632. Throughput: 0: 17545.1. Samples: 192720800. Policy #0 lag: (min: 1.0, avg: 2.5, max: 5.0) [2024-09-30 03:29:40,466][1157520] Avg episode reward: [(0, '56.212')] [2024-09-30 03:29:40,490][1157819] Updated weights for policy 0, policy_version 198018 (0.0006) [2024-09-30 03:29:41,044][1157819] Updated weights for policy 0, policy_version 198028 (0.0007) [2024-09-30 03:29:41,581][1157819] Updated weights for policy 0, policy_version 198038 (0.0007) [2024-09-30 03:29:42,176][1157819] Updated weights for policy 0, policy_version 198048 (0.0007) [2024-09-30 03:29:42,763][1157819] Updated weights for policy 0, policy_version 198058 (0.0007) [2024-09-30 03:29:43,367][1157819] Updated weights for policy 0, policy_version 198068 (0.0007) [2024-09-30 03:29:43,952][1157819] Updated weights for policy 0, policy_version 198078 (0.0006) [2024-09-30 03:29:44,548][1157819] Updated weights for policy 0, policy_version 198088 (0.0006) [2024-09-30 03:29:45,161][1157819] Updated weights for policy 0, policy_version 198098 (0.0007) [2024-09-30 03:29:45,466][1157520] Fps is (10 sec: 72090.4, 60 sec: 70656.0, 300 sec: 73478.0). Total num frames: 811429888. Throughput: 0: 17598.7. Samples: 192829316. Policy #0 lag: (min: 1.0, avg: 2.5, max: 5.0) [2024-09-30 03:29:45,466][1157520] Avg episode reward: [(0, '57.646')] [2024-09-30 03:29:45,759][1157819] Updated weights for policy 0, policy_version 198108 (0.0006) [2024-09-30 03:29:46,403][1157819] Updated weights for policy 0, policy_version 198118 (0.0006) [2024-09-30 03:29:47,000][1157819] Updated weights for policy 0, policy_version 198128 (0.0006) [2024-09-30 03:29:47,603][1157819] Updated weights for policy 0, policy_version 198138 (0.0006) [2024-09-30 03:29:48,179][1157819] Updated weights for policy 0, policy_version 198148 (0.0006) [2024-09-30 03:29:48,807][1157819] Updated weights for policy 0, policy_version 198158 (0.0006) [2024-09-30 03:29:49,365][1157819] Updated weights for policy 0, policy_version 198168 (0.0006) [2024-09-30 03:29:49,970][1157819] Updated weights for policy 0, policy_version 198178 (0.0006) [2024-09-30 03:29:50,466][1157520] Fps is (10 sec: 69222.4, 60 sec: 70178.1, 300 sec: 73353.1). Total num frames: 811769856. Throughput: 0: 17584.3. Samples: 192931544. Policy #0 lag: (min: 1.0, avg: 2.5, max: 5.0) [2024-09-30 03:29:50,466][1157520] Avg episode reward: [(0, '55.202')] [2024-09-30 03:29:50,537][1157819] Updated weights for policy 0, policy_version 198188 (0.0006) [2024-09-30 03:29:51,125][1157736] Signal inference workers to stop experience collection... (13800 times) [2024-09-30 03:29:51,127][1157736] Signal inference workers to resume experience collection... (13800 times) [2024-09-30 03:29:51,128][1157819] Updated weights for policy 0, policy_version 198198 (0.0006) [2024-09-30 03:29:51,131][1157819] InferenceWorker_p0-w0: stopping experience collection (13800 times) [2024-09-30 03:29:51,134][1157819] InferenceWorker_p0-w0: resuming experience collection (13800 times) [2024-09-30 03:29:51,661][1157819] Updated weights for policy 0, policy_version 198208 (0.0006) [2024-09-30 03:29:52,205][1157819] Updated weights for policy 0, policy_version 198218 (0.0006) [2024-09-30 03:29:52,765][1157819] Updated weights for policy 0, policy_version 198228 (0.0006) [2024-09-30 03:29:53,302][1157819] Updated weights for policy 0, policy_version 198238 (0.0006) [2024-09-30 03:29:53,850][1157819] Updated weights for policy 0, policy_version 198248 (0.0006) [2024-09-30 03:29:54,368][1157819] Updated weights for policy 0, policy_version 198258 (0.0006) [2024-09-30 03:29:54,943][1157819] Updated weights for policy 0, policy_version 198268 (0.0006) [2024-09-30 03:29:55,466][1157520] Fps is (10 sec: 71270.3, 60 sec: 70519.5, 300 sec: 73422.5). Total num frames: 812142592. Throughput: 0: 17701.8. Samples: 192985820. Policy #0 lag: (min: 1.0, avg: 2.5, max: 5.0) [2024-09-30 03:29:55,466][1157520] Avg episode reward: [(0, '54.924')] [2024-09-30 03:29:55,479][1157819] Updated weights for policy 0, policy_version 198278 (0.0006) [2024-09-30 03:29:56,024][1157819] Updated weights for policy 0, policy_version 198288 (0.0006) [2024-09-30 03:29:56,590][1157819] Updated weights for policy 0, policy_version 198298 (0.0006) [2024-09-30 03:29:57,143][1157819] Updated weights for policy 0, policy_version 198308 (0.0006) [2024-09-30 03:29:57,669][1157819] Updated weights for policy 0, policy_version 198318 (0.0006) [2024-09-30 03:29:58,189][1157819] Updated weights for policy 0, policy_version 198328 (0.0006) [2024-09-30 03:29:58,708][1157819] Updated weights for policy 0, policy_version 198338 (0.0006) [2024-09-30 03:29:59,284][1157819] Updated weights for policy 0, policy_version 198348 (0.0006) [2024-09-30 03:29:59,810][1157819] Updated weights for policy 0, policy_version 198358 (0.0006) [2024-09-30 03:30:00,309][1157819] Updated weights for policy 0, policy_version 198368 (0.0006) [2024-09-30 03:30:00,466][1157520] Fps is (10 sec: 75365.6, 60 sec: 70928.9, 300 sec: 73394.8). Total num frames: 812523520. Throughput: 0: 17997.2. Samples: 193099412. Policy #0 lag: (min: 1.0, avg: 2.5, max: 5.0) [2024-09-30 03:30:00,466][1157520] Avg episode reward: [(0, '57.116')] [2024-09-30 03:30:00,819][1157819] Updated weights for policy 0, policy_version 198378 (0.0006) [2024-09-30 03:30:01,353][1157819] Updated weights for policy 0, policy_version 198388 (0.0006) [2024-09-30 03:30:01,910][1157819] Updated weights for policy 0, policy_version 198398 (0.0006) [2024-09-30 03:30:02,433][1157819] Updated weights for policy 0, policy_version 198408 (0.0006) [2024-09-30 03:30:02,965][1157819] Updated weights for policy 0, policy_version 198418 (0.0006) [2024-09-30 03:30:03,467][1157819] Updated weights for policy 0, policy_version 198428 (0.0006) [2024-09-30 03:30:03,983][1157819] Updated weights for policy 0, policy_version 198438 (0.0006) [2024-09-30 03:30:04,503][1157819] Updated weights for policy 0, policy_version 198448 (0.0006) [2024-09-30 03:30:05,043][1157819] Updated weights for policy 0, policy_version 198458 (0.0006) [2024-09-30 03:30:05,466][1157520] Fps is (10 sec: 77414.3, 60 sec: 71884.9, 300 sec: 73394.7). Total num frames: 812916736. Throughput: 0: 18245.3. Samples: 193216544. Policy #0 lag: (min: 1.0, avg: 2.5, max: 5.0) [2024-09-30 03:30:05,466][1157520] Avg episode reward: [(0, '54.144')] [2024-09-30 03:30:05,482][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000198467_812920832.pth... [2024-09-30 03:30:05,542][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000194205_795463680.pth [2024-09-30 03:30:05,562][1157819] Updated weights for policy 0, policy_version 198468 (0.0006) [2024-09-30 03:30:06,074][1157819] Updated weights for policy 0, policy_version 198478 (0.0006) [2024-09-30 03:30:06,595][1157819] Updated weights for policy 0, policy_version 198488 (0.0006) [2024-09-30 03:30:07,099][1157819] Updated weights for policy 0, policy_version 198498 (0.0006) [2024-09-30 03:30:07,598][1157819] Updated weights for policy 0, policy_version 198508 (0.0006) [2024-09-30 03:30:08,101][1157819] Updated weights for policy 0, policy_version 198518 (0.0006) [2024-09-30 03:30:08,562][1157819] Updated weights for policy 0, policy_version 198528 (0.0006) [2024-09-30 03:30:09,047][1157819] Updated weights for policy 0, policy_version 198538 (0.0006) [2024-09-30 03:30:09,528][1157819] Updated weights for policy 0, policy_version 198548 (0.0006) [2024-09-30 03:30:09,976][1157819] Updated weights for policy 0, policy_version 198558 (0.0006) [2024-09-30 03:30:10,411][1157819] Updated weights for policy 0, policy_version 198568 (0.0006) [2024-09-30 03:30:10,466][1157520] Fps is (10 sec: 81102.4, 60 sec: 73455.1, 300 sec: 73547.5). Total num frames: 813334528. Throughput: 0: 18328.0. Samples: 193276100. Policy #0 lag: (min: 0.0, avg: 2.6, max: 4.0) [2024-09-30 03:30:10,466][1157520] Avg episode reward: [(0, '55.264')] [2024-09-30 03:30:10,876][1157819] Updated weights for policy 0, policy_version 198578 (0.0006) [2024-09-30 03:30:11,346][1157819] Updated weights for policy 0, policy_version 198588 (0.0006) [2024-09-30 03:30:11,821][1157819] Updated weights for policy 0, policy_version 198598 (0.0006) [2024-09-30 03:30:12,257][1157819] Updated weights for policy 0, policy_version 198608 (0.0006) [2024-09-30 03:30:12,707][1157819] Updated weights for policy 0, policy_version 198618 (0.0006) [2024-09-30 03:30:13,161][1157819] Updated weights for policy 0, policy_version 198628 (0.0006) [2024-09-30 03:30:13,593][1157819] Updated weights for policy 0, policy_version 198638 (0.0006) [2024-09-30 03:30:14,066][1157819] Updated weights for policy 0, policy_version 198648 (0.0006) [2024-09-30 03:30:14,513][1157819] Updated weights for policy 0, policy_version 198658 (0.0006) [2024-09-30 03:30:14,943][1157819] Updated weights for policy 0, policy_version 198668 (0.0006) [2024-09-30 03:30:15,405][1157819] Updated weights for policy 0, policy_version 198678 (0.0006) [2024-09-30 03:30:15,466][1157520] Fps is (10 sec: 87246.0, 60 sec: 75298.5, 300 sec: 73825.2). Total num frames: 813789184. Throughput: 0: 18872.4. Samples: 193409412. Policy #0 lag: (min: 0.0, avg: 2.6, max: 4.0) [2024-09-30 03:30:15,466][1157520] Avg episode reward: [(0, '55.788')] [2024-09-30 03:30:15,839][1157819] Updated weights for policy 0, policy_version 198688 (0.0006) [2024-09-30 03:30:16,288][1157819] Updated weights for policy 0, policy_version 198698 (0.0006) [2024-09-30 03:30:16,752][1157819] Updated weights for policy 0, policy_version 198708 (0.0006) [2024-09-30 03:30:17,207][1157819] Updated weights for policy 0, policy_version 198718 (0.0006) [2024-09-30 03:30:17,636][1157819] Updated weights for policy 0, policy_version 198728 (0.0006) [2024-09-30 03:30:18,086][1157819] Updated weights for policy 0, policy_version 198738 (0.0006) [2024-09-30 03:30:18,562][1157819] Updated weights for policy 0, policy_version 198748 (0.0006) [2024-09-30 03:30:19,046][1157819] Updated weights for policy 0, policy_version 198758 (0.0006) [2024-09-30 03:30:19,512][1157819] Updated weights for policy 0, policy_version 198768 (0.0006) [2024-09-30 03:30:19,953][1157819] Updated weights for policy 0, policy_version 198778 (0.0006) [2024-09-30 03:30:20,418][1157819] Updated weights for policy 0, policy_version 198788 (0.0006) [2024-09-30 03:30:20,466][1157520] Fps is (10 sec: 90521.8, 60 sec: 76800.2, 300 sec: 74130.7). Total num frames: 814239744. Throughput: 0: 19547.2. Samples: 193544856. Policy #0 lag: (min: 0.0, avg: 2.6, max: 4.0) [2024-09-30 03:30:20,466][1157520] Avg episode reward: [(0, '57.216')] [2024-09-30 03:30:20,858][1157819] Updated weights for policy 0, policy_version 198798 (0.0006) [2024-09-30 03:30:21,315][1157819] Updated weights for policy 0, policy_version 198808 (0.0006) [2024-09-30 03:30:21,756][1157819] Updated weights for policy 0, policy_version 198818 (0.0006) [2024-09-30 03:30:22,218][1157819] Updated weights for policy 0, policy_version 198828 (0.0006) [2024-09-30 03:30:22,662][1157819] Updated weights for policy 0, policy_version 198838 (0.0006) [2024-09-30 03:30:23,106][1157819] Updated weights for policy 0, policy_version 198848 (0.0006) [2024-09-30 03:30:23,559][1157819] Updated weights for policy 0, policy_version 198858 (0.0006) [2024-09-30 03:30:24,021][1157819] Updated weights for policy 0, policy_version 198868 (0.0006) [2024-09-30 03:30:24,506][1157819] Updated weights for policy 0, policy_version 198878 (0.0006) [2024-09-30 03:30:24,999][1157819] Updated weights for policy 0, policy_version 198888 (0.0006) [2024-09-30 03:30:25,445][1157819] Updated weights for policy 0, policy_version 198898 (0.0006) [2024-09-30 03:30:25,466][1157520] Fps is (10 sec: 89700.9, 60 sec: 77960.5, 300 sec: 74436.1). Total num frames: 814686208. Throughput: 0: 19827.3. Samples: 193613028. Policy #0 lag: (min: 0.0, avg: 2.6, max: 4.0) [2024-09-30 03:30:25,466][1157520] Avg episode reward: [(0, '56.861')] [2024-09-30 03:30:25,912][1157819] Updated weights for policy 0, policy_version 198908 (0.0006) [2024-09-30 03:30:26,401][1157819] Updated weights for policy 0, policy_version 198918 (0.0006) [2024-09-30 03:30:26,903][1157819] Updated weights for policy 0, policy_version 198928 (0.0006) [2024-09-30 03:30:27,397][1157819] Updated weights for policy 0, policy_version 198938 (0.0006) [2024-09-30 03:30:27,911][1157819] Updated weights for policy 0, policy_version 198948 (0.0006) [2024-09-30 03:30:28,379][1157819] Updated weights for policy 0, policy_version 198958 (0.0006) [2024-09-30 03:30:28,868][1157819] Updated weights for policy 0, policy_version 198968 (0.0006) [2024-09-30 03:30:29,352][1157819] Updated weights for policy 0, policy_version 198978 (0.0006) [2024-09-30 03:30:29,874][1157819] Updated weights for policy 0, policy_version 198988 (0.0006) [2024-09-30 03:30:30,342][1157819] Updated weights for policy 0, policy_version 198998 (0.0006) [2024-09-30 03:30:30,466][1157520] Fps is (10 sec: 86014.5, 60 sec: 79052.7, 300 sec: 74616.6). Total num frames: 815099904. Throughput: 0: 20268.4. Samples: 193741396. Policy #0 lag: (min: 0.0, avg: 2.6, max: 4.0) [2024-09-30 03:30:30,466][1157520] Avg episode reward: [(0, '55.135')] [2024-09-30 03:30:30,839][1157819] Updated weights for policy 0, policy_version 199008 (0.0006) [2024-09-30 03:30:31,341][1157819] Updated weights for policy 0, policy_version 199018 (0.0006) [2024-09-30 03:30:31,803][1157819] Updated weights for policy 0, policy_version 199028 (0.0006) [2024-09-30 03:30:32,067][1157736] Signal inference workers to stop experience collection... (13850 times) [2024-09-30 03:30:32,068][1157736] Signal inference workers to resume experience collection... (13850 times) [2024-09-30 03:30:32,071][1157819] InferenceWorker_p0-w0: stopping experience collection (13850 times) [2024-09-30 03:30:32,072][1157819] InferenceWorker_p0-w0: resuming experience collection (13850 times) [2024-09-30 03:30:32,288][1157819] Updated weights for policy 0, policy_version 199038 (0.0006) [2024-09-30 03:30:32,776][1157819] Updated weights for policy 0, policy_version 199048 (0.0006) [2024-09-30 03:30:33,237][1157819] Updated weights for policy 0, policy_version 199058 (0.0006) [2024-09-30 03:30:33,717][1157819] Updated weights for policy 0, policy_version 199068 (0.0006) [2024-09-30 03:30:34,194][1157819] Updated weights for policy 0, policy_version 199078 (0.0006) [2024-09-30 03:30:34,649][1157819] Updated weights for policy 0, policy_version 199088 (0.0006) [2024-09-30 03:30:35,135][1157819] Updated weights for policy 0, policy_version 199098 (0.0006) [2024-09-30 03:30:35,466][1157520] Fps is (10 sec: 84786.7, 60 sec: 80418.1, 300 sec: 74741.5). Total num frames: 815534080. Throughput: 0: 20828.4. Samples: 193868824. Policy #0 lag: (min: 0.0, avg: 2.6, max: 4.0) [2024-09-30 03:30:35,466][1157520] Avg episode reward: [(0, '56.656')] [2024-09-30 03:30:35,609][1157819] Updated weights for policy 0, policy_version 199108 (0.0006) [2024-09-30 03:30:36,081][1157819] Updated weights for policy 0, policy_version 199118 (0.0006) [2024-09-30 03:30:36,561][1157819] Updated weights for policy 0, policy_version 199128 (0.0006) [2024-09-30 03:30:37,053][1157819] Updated weights for policy 0, policy_version 199138 (0.0006) [2024-09-30 03:30:37,532][1157819] Updated weights for policy 0, policy_version 199148 (0.0006) [2024-09-30 03:30:38,028][1157819] Updated weights for policy 0, policy_version 199158 (0.0006) [2024-09-30 03:30:38,488][1157819] Updated weights for policy 0, policy_version 199168 (0.0006) [2024-09-30 03:30:38,974][1157819] Updated weights for policy 0, policy_version 199178 (0.0006) [2024-09-30 03:30:39,462][1157819] Updated weights for policy 0, policy_version 199188 (0.0006) [2024-09-30 03:30:39,922][1157819] Updated weights for policy 0, policy_version 199198 (0.0006) [2024-09-30 03:30:40,399][1157819] Updated weights for policy 0, policy_version 199208 (0.0006) [2024-09-30 03:30:40,466][1157520] Fps is (10 sec: 86427.2, 60 sec: 81442.3, 300 sec: 74852.7). Total num frames: 815964160. Throughput: 0: 21053.4. Samples: 193933220. Policy #0 lag: (min: 0.0, avg: 2.6, max: 4.0) [2024-09-30 03:30:40,466][1157520] Avg episode reward: [(0, '55.987')] [2024-09-30 03:30:40,819][1157819] Updated weights for policy 0, policy_version 199218 (0.0006) [2024-09-30 03:30:41,283][1157819] Updated weights for policy 0, policy_version 199228 (0.0006) [2024-09-30 03:30:41,721][1157819] Updated weights for policy 0, policy_version 199238 (0.0006) [2024-09-30 03:30:42,178][1157819] Updated weights for policy 0, policy_version 199248 (0.0006) [2024-09-30 03:30:42,637][1157819] Updated weights for policy 0, policy_version 199258 (0.0006) [2024-09-30 03:30:43,085][1157819] Updated weights for policy 0, policy_version 199268 (0.0006) [2024-09-30 03:30:43,521][1157819] Updated weights for policy 0, policy_version 199278 (0.0006) [2024-09-30 03:30:43,987][1157819] Updated weights for policy 0, policy_version 199288 (0.0006) [2024-09-30 03:30:44,415][1157819] Updated weights for policy 0, policy_version 199298 (0.0006) [2024-09-30 03:30:44,863][1157819] Updated weights for policy 0, policy_version 199308 (0.0006) [2024-09-30 03:30:45,349][1157819] Updated weights for policy 0, policy_version 199318 (0.0006) [2024-09-30 03:30:45,466][1157520] Fps is (10 sec: 88475.5, 60 sec: 83148.9, 300 sec: 75033.2). Total num frames: 816418816. Throughput: 0: 21494.0. Samples: 194066636. Policy #0 lag: (min: 0.0, avg: 2.6, max: 4.0) [2024-09-30 03:30:45,466][1157520] Avg episode reward: [(0, '55.336')] [2024-09-30 03:30:45,774][1157819] Updated weights for policy 0, policy_version 199328 (0.0006) [2024-09-30 03:30:46,217][1157819] Updated weights for policy 0, policy_version 199338 (0.0006) [2024-09-30 03:30:46,677][1157819] Updated weights for policy 0, policy_version 199348 (0.0006) [2024-09-30 03:30:47,144][1157819] Updated weights for policy 0, policy_version 199358 (0.0006) [2024-09-30 03:30:47,561][1157819] Updated weights for policy 0, policy_version 199368 (0.0006) [2024-09-30 03:30:48,021][1157819] Updated weights for policy 0, policy_version 199378 (0.0006) [2024-09-30 03:30:48,500][1157819] Updated weights for policy 0, policy_version 199388 (0.0006) [2024-09-30 03:30:48,922][1157819] Updated weights for policy 0, policy_version 199398 (0.0006) [2024-09-30 03:30:49,400][1157819] Updated weights for policy 0, policy_version 199408 (0.0006) [2024-09-30 03:30:49,816][1157819] Updated weights for policy 0, policy_version 199418 (0.0006) [2024-09-30 03:30:50,274][1157819] Updated weights for policy 0, policy_version 199428 (0.0006) [2024-09-30 03:30:50,466][1157520] Fps is (10 sec: 90931.1, 60 sec: 85060.4, 300 sec: 75227.6). Total num frames: 816873472. Throughput: 0: 21925.0. Samples: 194203168. Policy #0 lag: (min: 0.0, avg: 2.6, max: 4.0) [2024-09-30 03:30:50,466][1157520] Avg episode reward: [(0, '55.364')] [2024-09-30 03:30:50,724][1157819] Updated weights for policy 0, policy_version 199438 (0.0006) [2024-09-30 03:30:51,170][1157819] Updated weights for policy 0, policy_version 199448 (0.0006) [2024-09-30 03:30:51,653][1157819] Updated weights for policy 0, policy_version 199458 (0.0006) [2024-09-30 03:30:52,144][1157819] Updated weights for policy 0, policy_version 199468 (0.0006) [2024-09-30 03:30:52,574][1157819] Updated weights for policy 0, policy_version 199478 (0.0006) [2024-09-30 03:30:53,050][1157819] Updated weights for policy 0, policy_version 199488 (0.0006) [2024-09-30 03:30:53,540][1157819] Updated weights for policy 0, policy_version 199498 (0.0006) [2024-09-30 03:30:54,019][1157819] Updated weights for policy 0, policy_version 199508 (0.0006) [2024-09-30 03:30:54,525][1157819] Updated weights for policy 0, policy_version 199518 (0.0006) [2024-09-30 03:30:55,044][1157819] Updated weights for policy 0, policy_version 199528 (0.0006) [2024-09-30 03:30:55,466][1157520] Fps is (10 sec: 88064.0, 60 sec: 85947.9, 300 sec: 75352.5). Total num frames: 817299456. Throughput: 0: 22076.3. Samples: 194269532. Policy #0 lag: (min: 0.0, avg: 2.6, max: 4.0) [2024-09-30 03:30:55,466][1157520] Avg episode reward: [(0, '55.941')] [2024-09-30 03:30:55,538][1157819] Updated weights for policy 0, policy_version 199538 (0.0006) [2024-09-30 03:30:56,145][1157819] Updated weights for policy 0, policy_version 199548 (0.0006) [2024-09-30 03:30:56,699][1157819] Updated weights for policy 0, policy_version 199558 (0.0006) [2024-09-30 03:30:57,302][1157819] Updated weights for policy 0, policy_version 199568 (0.0006) [2024-09-30 03:30:57,885][1157819] Updated weights for policy 0, policy_version 199578 (0.0006) [2024-09-30 03:30:58,479][1157819] Updated weights for policy 0, policy_version 199588 (0.0006) [2024-09-30 03:30:59,067][1157819] Updated weights for policy 0, policy_version 199598 (0.0006) [2024-09-30 03:30:59,656][1157819] Updated weights for policy 0, policy_version 199608 (0.0006) [2024-09-30 03:31:00,277][1157819] Updated weights for policy 0, policy_version 199618 (0.0006) [2024-09-30 03:31:00,466][1157520] Fps is (10 sec: 77413.3, 60 sec: 85401.7, 300 sec: 75213.6). Total num frames: 817647616. Throughput: 0: 21652.0. Samples: 194383756. Policy #0 lag: (min: 0.0, avg: 2.6, max: 4.0) [2024-09-30 03:31:00,466][1157520] Avg episode reward: [(0, '56.838')] [2024-09-30 03:31:00,847][1157819] Updated weights for policy 0, policy_version 199628 (0.0006) [2024-09-30 03:31:01,446][1157819] Updated weights for policy 0, policy_version 199638 (0.0006) [2024-09-30 03:31:02,086][1157819] Updated weights for policy 0, policy_version 199648 (0.0006) [2024-09-30 03:31:02,353][1157736] Signal inference workers to stop experience collection... (13900 times) [2024-09-30 03:31:02,357][1157819] InferenceWorker_p0-w0: stopping experience collection (13900 times) [2024-09-30 03:31:02,362][1157736] Signal inference workers to resume experience collection... (13900 times) [2024-09-30 03:31:02,363][1157819] InferenceWorker_p0-w0: resuming experience collection (13900 times) [2024-09-30 03:31:02,792][1157819] Updated weights for policy 0, policy_version 199658 (0.0006) [2024-09-30 03:31:03,400][1157819] Updated weights for policy 0, policy_version 199668 (0.0006) [2024-09-30 03:31:03,937][1157819] Updated weights for policy 0, policy_version 199678 (0.0006) [2024-09-30 03:31:04,511][1157819] Updated weights for policy 0, policy_version 199688 (0.0006) [2024-09-30 03:31:05,069][1157819] Updated weights for policy 0, policy_version 199698 (0.0006) [2024-09-30 03:31:05,466][1157520] Fps is (10 sec: 68812.2, 60 sec: 84514.2, 300 sec: 75102.6). Total num frames: 817987584. Throughput: 0: 20902.3. Samples: 194485464. Policy #0 lag: (min: 0.0, avg: 2.6, max: 4.0) [2024-09-30 03:31:05,466][1157520] Avg episode reward: [(0, '54.773')] [2024-09-30 03:31:05,623][1157819] Updated weights for policy 0, policy_version 199708 (0.0006) [2024-09-30 03:31:06,178][1157819] Updated weights for policy 0, policy_version 199718 (0.0006) [2024-09-30 03:31:06,735][1157819] Updated weights for policy 0, policy_version 199728 (0.0006) [2024-09-30 03:31:07,312][1157819] Updated weights for policy 0, policy_version 199738 (0.0006) [2024-09-30 03:31:07,873][1157819] Updated weights for policy 0, policy_version 199748 (0.0006) [2024-09-30 03:31:08,417][1157819] Updated weights for policy 0, policy_version 199758 (0.0006) [2024-09-30 03:31:08,968][1157819] Updated weights for policy 0, policy_version 199768 (0.0006) [2024-09-30 03:31:09,497][1157819] Updated weights for policy 0, policy_version 199778 (0.0006) [2024-09-30 03:31:10,029][1157819] Updated weights for policy 0, policy_version 199788 (0.0006) [2024-09-30 03:31:10,466][1157520] Fps is (10 sec: 71270.6, 60 sec: 83763.1, 300 sec: 75074.8). Total num frames: 818360320. Throughput: 0: 20618.2. Samples: 194540844. Policy #0 lag: (min: 0.0, avg: 2.6, max: 4.0) [2024-09-30 03:31:10,466][1157520] Avg episode reward: [(0, '55.706')] [2024-09-30 03:31:10,592][1157819] Updated weights for policy 0, policy_version 199798 (0.0006) [2024-09-30 03:31:11,104][1157819] Updated weights for policy 0, policy_version 199808 (0.0006) [2024-09-30 03:31:11,703][1157819] Updated weights for policy 0, policy_version 199818 (0.0006) [2024-09-30 03:31:12,258][1157819] Updated weights for policy 0, policy_version 199828 (0.0006) [2024-09-30 03:31:12,834][1157819] Updated weights for policy 0, policy_version 199838 (0.0006) [2024-09-30 03:31:13,415][1157819] Updated weights for policy 0, policy_version 199848 (0.0006) [2024-09-30 03:31:13,991][1157819] Updated weights for policy 0, policy_version 199858 (0.0006) [2024-09-30 03:31:14,587][1157819] Updated weights for policy 0, policy_version 199868 (0.0006) [2024-09-30 03:31:15,157][1157819] Updated weights for policy 0, policy_version 199878 (0.0006) [2024-09-30 03:31:15,466][1157520] Fps is (10 sec: 73318.4, 60 sec: 82192.9, 300 sec: 74949.8). Total num frames: 818720768. Throughput: 0: 20203.5. Samples: 194650552. Policy #0 lag: (min: 0.0, avg: 2.6, max: 4.0) [2024-09-30 03:31:15,466][1157520] Avg episode reward: [(0, '54.724')] [2024-09-30 03:31:15,774][1157819] Updated weights for policy 0, policy_version 199888 (0.0006) [2024-09-30 03:31:16,368][1157819] Updated weights for policy 0, policy_version 199898 (0.0006) [2024-09-30 03:31:16,959][1157819] Updated weights for policy 0, policy_version 199908 (0.0006) [2024-09-30 03:31:17,535][1157819] Updated weights for policy 0, policy_version 199918 (0.0006) [2024-09-30 03:31:18,128][1157819] Updated weights for policy 0, policy_version 199928 (0.0006) [2024-09-30 03:31:18,722][1157819] Updated weights for policy 0, policy_version 199938 (0.0006) [2024-09-30 03:31:19,296][1157819] Updated weights for policy 0, policy_version 199948 (0.0006) [2024-09-30 03:31:19,934][1157819] Updated weights for policy 0, policy_version 199958 (0.0006) [2024-09-30 03:31:20,466][1157520] Fps is (10 sec: 70451.1, 60 sec: 80417.9, 300 sec: 74769.4). Total num frames: 819064832. Throughput: 0: 19696.2. Samples: 194755152. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 03:31:20,466][1157520] Avg episode reward: [(0, '55.096')] [2024-09-30 03:31:20,471][1157819] Updated weights for policy 0, policy_version 199968 (0.0006) [2024-09-30 03:31:21,111][1157819] Updated weights for policy 0, policy_version 199978 (0.0006) [2024-09-30 03:31:21,673][1157819] Updated weights for policy 0, policy_version 199988 (0.0006) [2024-09-30 03:31:22,281][1157819] Updated weights for policy 0, policy_version 199998 (0.0006) [2024-09-30 03:31:22,853][1157819] Updated weights for policy 0, policy_version 200008 (0.0006) [2024-09-30 03:31:23,448][1157819] Updated weights for policy 0, policy_version 200018 (0.0006) [2024-09-30 03:31:24,035][1157819] Updated weights for policy 0, policy_version 200028 (0.0006) [2024-09-30 03:31:24,595][1157819] Updated weights for policy 0, policy_version 200038 (0.0006) [2024-09-30 03:31:25,133][1157819] Updated weights for policy 0, policy_version 200048 (0.0006) [2024-09-30 03:31:25,466][1157520] Fps is (10 sec: 70040.8, 60 sec: 78916.2, 300 sec: 74588.8). Total num frames: 819421184. Throughput: 0: 19416.1. Samples: 194806948. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 03:31:25,466][1157520] Avg episode reward: [(0, '55.462')] [2024-09-30 03:31:25,681][1157819] Updated weights for policy 0, policy_version 200058 (0.0006) [2024-09-30 03:31:26,220][1157819] Updated weights for policy 0, policy_version 200068 (0.0006) [2024-09-30 03:31:26,752][1157819] Updated weights for policy 0, policy_version 200078 (0.0006) [2024-09-30 03:31:27,329][1157819] Updated weights for policy 0, policy_version 200088 (0.0006) [2024-09-30 03:31:27,840][1157819] Updated weights for policy 0, policy_version 200098 (0.0006) [2024-09-30 03:31:28,389][1157819] Updated weights for policy 0, policy_version 200108 (0.0006) [2024-09-30 03:31:28,938][1157819] Updated weights for policy 0, policy_version 200118 (0.0006) [2024-09-30 03:31:29,473][1157819] Updated weights for policy 0, policy_version 200128 (0.0006) [2024-09-30 03:31:29,999][1157819] Updated weights for policy 0, policy_version 200138 (0.0006) [2024-09-30 03:31:30,466][1157520] Fps is (10 sec: 73318.4, 60 sec: 78301.9, 300 sec: 74547.2). Total num frames: 819798016. Throughput: 0: 18922.7. Samples: 194918160. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 03:31:30,466][1157520] Avg episode reward: [(0, '55.819')] [2024-09-30 03:31:30,533][1157819] Updated weights for policy 0, policy_version 200148 (0.0006) [2024-09-30 03:31:31,095][1157819] Updated weights for policy 0, policy_version 200158 (0.0006) [2024-09-30 03:31:31,654][1157819] Updated weights for policy 0, policy_version 200168 (0.0006) [2024-09-30 03:31:32,195][1157819] Updated weights for policy 0, policy_version 200178 (0.0006) [2024-09-30 03:31:32,704][1157819] Updated weights for policy 0, policy_version 200188 (0.0006) [2024-09-30 03:31:33,272][1157819] Updated weights for policy 0, policy_version 200198 (0.0006) [2024-09-30 03:31:33,810][1157819] Updated weights for policy 0, policy_version 200208 (0.0006) [2024-09-30 03:31:34,369][1157819] Updated weights for policy 0, policy_version 200218 (0.0006) [2024-09-30 03:31:34,885][1157819] Updated weights for policy 0, policy_version 200228 (0.0006) [2024-09-30 03:31:35,433][1157819] Updated weights for policy 0, policy_version 200238 (0.0006) [2024-09-30 03:31:35,466][1157520] Fps is (10 sec: 75367.1, 60 sec: 77346.3, 300 sec: 74547.2). Total num frames: 820174848. Throughput: 0: 18408.9. Samples: 195031572. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 03:31:35,466][1157520] Avg episode reward: [(0, '55.655')] [2024-09-30 03:31:35,993][1157819] Updated weights for policy 0, policy_version 200248 (0.0006) [2024-09-30 03:31:36,509][1157819] Updated weights for policy 0, policy_version 200258 (0.0006) [2024-09-30 03:31:37,059][1157819] Updated weights for policy 0, policy_version 200268 (0.0006) [2024-09-30 03:31:37,587][1157819] Updated weights for policy 0, policy_version 200278 (0.0006) [2024-09-30 03:31:38,137][1157819] Updated weights for policy 0, policy_version 200288 (0.0006) [2024-09-30 03:31:38,698][1157819] Updated weights for policy 0, policy_version 200298 (0.0006) [2024-09-30 03:31:39,206][1157819] Updated weights for policy 0, policy_version 200308 (0.0006) [2024-09-30 03:31:39,772][1157819] Updated weights for policy 0, policy_version 200318 (0.0006) [2024-09-30 03:31:40,307][1157819] Updated weights for policy 0, policy_version 200328 (0.0006) [2024-09-30 03:31:40,466][1157520] Fps is (10 sec: 75366.7, 60 sec: 76458.5, 300 sec: 74630.5). Total num frames: 820551680. Throughput: 0: 18191.6. Samples: 195088156. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 03:31:40,466][1157520] Avg episode reward: [(0, '56.349')] [2024-09-30 03:31:40,845][1157819] Updated weights for policy 0, policy_version 200338 (0.0006) [2024-09-30 03:31:41,385][1157819] Updated weights for policy 0, policy_version 200348 (0.0006) [2024-09-30 03:31:41,949][1157819] Updated weights for policy 0, policy_version 200358 (0.0006) [2024-09-30 03:31:42,485][1157819] Updated weights for policy 0, policy_version 200368 (0.0006) [2024-09-30 03:31:43,008][1157819] Updated weights for policy 0, policy_version 200378 (0.0006) [2024-09-30 03:31:43,591][1157819] Updated weights for policy 0, policy_version 200388 (0.0006) [2024-09-30 03:31:44,125][1157819] Updated weights for policy 0, policy_version 200398 (0.0006) [2024-09-30 03:31:44,663][1157819] Updated weights for policy 0, policy_version 200408 (0.0006) [2024-09-30 03:31:45,144][1157736] Signal inference workers to stop experience collection... (13950 times) [2024-09-30 03:31:45,144][1157736] Signal inference workers to resume experience collection... (13950 times) [2024-09-30 03:31:45,147][1157819] InferenceWorker_p0-w0: stopping experience collection (13950 times) [2024-09-30 03:31:45,150][1157819] InferenceWorker_p0-w0: resuming experience collection (13950 times) [2024-09-30 03:31:45,196][1157819] Updated weights for policy 0, policy_version 200418 (0.0006) [2024-09-30 03:31:45,466][1157520] Fps is (10 sec: 75366.7, 60 sec: 75161.5, 300 sec: 74713.8). Total num frames: 820928512. Throughput: 0: 18160.4. Samples: 195200972. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 03:31:45,466][1157520] Avg episode reward: [(0, '57.772')] [2024-09-30 03:31:45,798][1157819] Updated weights for policy 0, policy_version 200428 (0.0006) [2024-09-30 03:31:46,373][1157819] Updated weights for policy 0, policy_version 200438 (0.0006) [2024-09-30 03:31:46,960][1157819] Updated weights for policy 0, policy_version 200448 (0.0006) [2024-09-30 03:31:47,534][1157819] Updated weights for policy 0, policy_version 200458 (0.0006) [2024-09-30 03:31:48,146][1157819] Updated weights for policy 0, policy_version 200468 (0.0006) [2024-09-30 03:31:48,723][1157819] Updated weights for policy 0, policy_version 200478 (0.0006) [2024-09-30 03:31:49,309][1157819] Updated weights for policy 0, policy_version 200488 (0.0006) [2024-09-30 03:31:49,922][1157819] Updated weights for policy 0, policy_version 200498 (0.0006) [2024-09-30 03:31:50,466][1157520] Fps is (10 sec: 72499.3, 60 sec: 73386.6, 300 sec: 74699.9). Total num frames: 821276672. Throughput: 0: 18287.8. Samples: 195308416. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 03:31:50,466][1157520] Avg episode reward: [(0, '57.847')] [2024-09-30 03:31:50,468][1157819] Updated weights for policy 0, policy_version 200508 (0.0006) [2024-09-30 03:31:51,054][1157819] Updated weights for policy 0, policy_version 200518 (0.0006) [2024-09-30 03:31:51,656][1157819] Updated weights for policy 0, policy_version 200528 (0.0006) [2024-09-30 03:31:52,234][1157819] Updated weights for policy 0, policy_version 200538 (0.0006) [2024-09-30 03:31:52,807][1157819] Updated weights for policy 0, policy_version 200548 (0.0006) [2024-09-30 03:31:53,419][1157819] Updated weights for policy 0, policy_version 200558 (0.0006) [2024-09-30 03:31:53,979][1157819] Updated weights for policy 0, policy_version 200568 (0.0006) [2024-09-30 03:31:54,562][1157819] Updated weights for policy 0, policy_version 200578 (0.0006) [2024-09-30 03:31:55,170][1157819] Updated weights for policy 0, policy_version 200588 (0.0006) [2024-09-30 03:31:55,466][1157520] Fps is (10 sec: 70041.4, 60 sec: 72157.7, 300 sec: 74686.0). Total num frames: 821628928. Throughput: 0: 18222.2. Samples: 195360844. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 03:31:55,466][1157520] Avg episode reward: [(0, '57.341')] [2024-09-30 03:31:55,742][1157819] Updated weights for policy 0, policy_version 200598 (0.0006) [2024-09-30 03:31:56,306][1157819] Updated weights for policy 0, policy_version 200608 (0.0006) [2024-09-30 03:31:56,896][1157819] Updated weights for policy 0, policy_version 200618 (0.0006) [2024-09-30 03:31:57,459][1157819] Updated weights for policy 0, policy_version 200628 (0.0006) [2024-09-30 03:31:58,049][1157819] Updated weights for policy 0, policy_version 200638 (0.0006) [2024-09-30 03:31:58,632][1157819] Updated weights for policy 0, policy_version 200648 (0.0006) [2024-09-30 03:31:59,218][1157819] Updated weights for policy 0, policy_version 200658 (0.0006) [2024-09-30 03:31:59,772][1157819] Updated weights for policy 0, policy_version 200668 (0.0006) [2024-09-30 03:32:00,369][1157819] Updated weights for policy 0, policy_version 200678 (0.0006) [2024-09-30 03:32:00,466][1157520] Fps is (10 sec: 70860.7, 60 sec: 72294.5, 300 sec: 74658.3). Total num frames: 821985280. Throughput: 0: 18128.4. Samples: 195466332. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 03:32:00,466][1157520] Avg episode reward: [(0, '58.111')] [2024-09-30 03:32:00,877][1157819] Updated weights for policy 0, policy_version 200688 (0.0006) [2024-09-30 03:32:01,425][1157819] Updated weights for policy 0, policy_version 200698 (0.0006) [2024-09-30 03:32:01,986][1157819] Updated weights for policy 0, policy_version 200708 (0.0006) [2024-09-30 03:32:02,551][1157819] Updated weights for policy 0, policy_version 200718 (0.0006) [2024-09-30 03:32:03,087][1157819] Updated weights for policy 0, policy_version 200728 (0.0006) [2024-09-30 03:32:03,623][1157819] Updated weights for policy 0, policy_version 200738 (0.0006) [2024-09-30 03:32:04,168][1157819] Updated weights for policy 0, policy_version 200748 (0.0006) [2024-09-30 03:32:04,714][1157819] Updated weights for policy 0, policy_version 200758 (0.0006) [2024-09-30 03:32:05,254][1157819] Updated weights for policy 0, policy_version 200768 (0.0006) [2024-09-30 03:32:05,466][1157520] Fps is (10 sec: 72908.8, 60 sec: 72840.5, 300 sec: 74602.7). Total num frames: 822358016. Throughput: 0: 18278.8. Samples: 195577700. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 03:32:05,466][1157520] Avg episode reward: [(0, '57.486')] [2024-09-30 03:32:05,476][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000200772_822362112.pth... [2024-09-30 03:32:05,532][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000196379_804368384.pth [2024-09-30 03:32:05,790][1157819] Updated weights for policy 0, policy_version 200778 (0.0006) [2024-09-30 03:32:06,296][1157819] Updated weights for policy 0, policy_version 200788 (0.0006) [2024-09-30 03:32:06,778][1157819] Updated weights for policy 0, policy_version 200798 (0.0006) [2024-09-30 03:32:07,296][1157819] Updated weights for policy 0, policy_version 200808 (0.0006) [2024-09-30 03:32:07,803][1157819] Updated weights for policy 0, policy_version 200818 (0.0006) [2024-09-30 03:32:08,332][1157819] Updated weights for policy 0, policy_version 200828 (0.0006) [2024-09-30 03:32:08,856][1157819] Updated weights for policy 0, policy_version 200838 (0.0006) [2024-09-30 03:32:09,379][1157819] Updated weights for policy 0, policy_version 200848 (0.0006) [2024-09-30 03:32:09,872][1157819] Updated weights for policy 0, policy_version 200858 (0.0006) [2024-09-30 03:32:10,388][1157819] Updated weights for policy 0, policy_version 200868 (0.0006) [2024-09-30 03:32:10,466][1157520] Fps is (10 sec: 77415.2, 60 sec: 73318.5, 300 sec: 74658.3). Total num frames: 822759424. Throughput: 0: 18446.9. Samples: 195637056. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 03:32:10,466][1157520] Avg episode reward: [(0, '56.587')] [2024-09-30 03:32:10,916][1157819] Updated weights for policy 0, policy_version 200878 (0.0006) [2024-09-30 03:32:11,444][1157819] Updated weights for policy 0, policy_version 200888 (0.0006) [2024-09-30 03:32:11,948][1157819] Updated weights for policy 0, policy_version 200898 (0.0006) [2024-09-30 03:32:12,448][1157819] Updated weights for policy 0, policy_version 200908 (0.0006) [2024-09-30 03:32:12,977][1157819] Updated weights for policy 0, policy_version 200918 (0.0006) [2024-09-30 03:32:13,471][1157819] Updated weights for policy 0, policy_version 200928 (0.0006) [2024-09-30 03:32:14,000][1157819] Updated weights for policy 0, policy_version 200938 (0.0006) [2024-09-30 03:32:14,515][1157819] Updated weights for policy 0, policy_version 200948 (0.0006) [2024-09-30 03:32:15,011][1157819] Updated weights for policy 0, policy_version 200958 (0.0006) [2024-09-30 03:32:15,466][1157520] Fps is (10 sec: 79872.8, 60 sec: 73932.9, 300 sec: 74741.6). Total num frames: 823156736. Throughput: 0: 18626.7. Samples: 195756360. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 03:32:15,466][1157520] Avg episode reward: [(0, '54.856')] [2024-09-30 03:32:15,512][1157819] Updated weights for policy 0, policy_version 200968 (0.0006) [2024-09-30 03:32:16,071][1157819] Updated weights for policy 0, policy_version 200978 (0.0006) [2024-09-30 03:32:16,601][1157819] Updated weights for policy 0, policy_version 200988 (0.0006) [2024-09-30 03:32:17,099][1157819] Updated weights for policy 0, policy_version 200998 (0.0006) [2024-09-30 03:32:17,616][1157819] Updated weights for policy 0, policy_version 201008 (0.0006) [2024-09-30 03:32:18,117][1157819] Updated weights for policy 0, policy_version 201018 (0.0006) [2024-09-30 03:32:18,623][1157819] Updated weights for policy 0, policy_version 201028 (0.0006) [2024-09-30 03:32:19,128][1157819] Updated weights for policy 0, policy_version 201038 (0.0006) [2024-09-30 03:32:19,635][1157819] Updated weights for policy 0, policy_version 201048 (0.0006) [2024-09-30 03:32:20,145][1157819] Updated weights for policy 0, policy_version 201058 (0.0006) [2024-09-30 03:32:20,466][1157520] Fps is (10 sec: 79872.1, 60 sec: 74888.7, 300 sec: 74894.4). Total num frames: 823558144. Throughput: 0: 18772.9. Samples: 195876348. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 03:32:20,466][1157520] Avg episode reward: [(0, '56.705')] [2024-09-30 03:32:20,652][1157819] Updated weights for policy 0, policy_version 201068 (0.0006) [2024-09-30 03:32:21,184][1157819] Updated weights for policy 0, policy_version 201078 (0.0006) [2024-09-30 03:32:21,689][1157819] Updated weights for policy 0, policy_version 201088 (0.0006) [2024-09-30 03:32:22,191][1157819] Updated weights for policy 0, policy_version 201098 (0.0006) [2024-09-30 03:32:22,745][1157819] Updated weights for policy 0, policy_version 201108 (0.0006) [2024-09-30 03:32:23,315][1157819] Updated weights for policy 0, policy_version 201118 (0.0006) [2024-09-30 03:32:23,848][1157819] Updated weights for policy 0, policy_version 201128 (0.0006) [2024-09-30 03:32:24,362][1157819] Updated weights for policy 0, policy_version 201138 (0.0006) [2024-09-30 03:32:24,877][1157819] Updated weights for policy 0, policy_version 201148 (0.0006) [2024-09-30 03:32:25,327][1157736] Signal inference workers to stop experience collection... (14000 times) [2024-09-30 03:32:25,328][1157736] Signal inference workers to resume experience collection... (14000 times) [2024-09-30 03:32:25,333][1157819] InferenceWorker_p0-w0: stopping experience collection (14000 times) [2024-09-30 03:32:25,333][1157819] InferenceWorker_p0-w0: resuming experience collection (14000 times) [2024-09-30 03:32:25,405][1157819] Updated weights for policy 0, policy_version 201158 (0.0006) [2024-09-30 03:32:25,466][1157520] Fps is (10 sec: 79052.4, 60 sec: 75434.9, 300 sec: 75047.1). Total num frames: 823947264. Throughput: 0: 18819.7. Samples: 195935044. Policy #0 lag: (min: 0.0, avg: 2.3, max: 4.0) [2024-09-30 03:32:25,466][1157520] Avg episode reward: [(0, '56.689')] [2024-09-30 03:32:25,931][1157819] Updated weights for policy 0, policy_version 201168 (0.0006) [2024-09-30 03:32:26,468][1157819] Updated weights for policy 0, policy_version 201178 (0.0006) [2024-09-30 03:32:26,989][1157819] Updated weights for policy 0, policy_version 201188 (0.0006) [2024-09-30 03:32:27,506][1157819] Updated weights for policy 0, policy_version 201198 (0.0006) [2024-09-30 03:32:28,036][1157819] Updated weights for policy 0, policy_version 201208 (0.0006) [2024-09-30 03:32:28,576][1157819] Updated weights for policy 0, policy_version 201218 (0.0006) [2024-09-30 03:32:29,096][1157819] Updated weights for policy 0, policy_version 201228 (0.0006) [2024-09-30 03:32:29,633][1157819] Updated weights for policy 0, policy_version 201238 (0.0006) [2024-09-30 03:32:30,137][1157819] Updated weights for policy 0, policy_version 201248 (0.0006) [2024-09-30 03:32:30,466][1157520] Fps is (10 sec: 77824.0, 60 sec: 75639.7, 300 sec: 75172.0). Total num frames: 824336384. Throughput: 0: 18896.4. Samples: 196051308. Policy #0 lag: (min: 0.0, avg: 2.3, max: 4.0) [2024-09-30 03:32:30,466][1157520] Avg episode reward: [(0, '55.721')] [2024-09-30 03:32:30,667][1157819] Updated weights for policy 0, policy_version 201258 (0.0006) [2024-09-30 03:32:31,207][1157819] Updated weights for policy 0, policy_version 201268 (0.0006) [2024-09-30 03:32:31,738][1157819] Updated weights for policy 0, policy_version 201278 (0.0006) [2024-09-30 03:32:32,257][1157819] Updated weights for policy 0, policy_version 201288 (0.0006) [2024-09-30 03:32:32,786][1157819] Updated weights for policy 0, policy_version 201298 (0.0006) [2024-09-30 03:32:33,300][1157819] Updated weights for policy 0, policy_version 201308 (0.0006) [2024-09-30 03:32:33,839][1157819] Updated weights for policy 0, policy_version 201318 (0.0006) [2024-09-30 03:32:34,378][1157819] Updated weights for policy 0, policy_version 201328 (0.0006) [2024-09-30 03:32:34,908][1157819] Updated weights for policy 0, policy_version 201338 (0.0006) [2024-09-30 03:32:35,433][1157819] Updated weights for policy 0, policy_version 201348 (0.0006) [2024-09-30 03:32:35,466][1157520] Fps is (10 sec: 77414.2, 60 sec: 75776.0, 300 sec: 75172.0). Total num frames: 824721408. Throughput: 0: 19097.7. Samples: 196167812. Policy #0 lag: (min: 0.0, avg: 2.3, max: 4.0) [2024-09-30 03:32:35,466][1157520] Avg episode reward: [(0, '55.956')] [2024-09-30 03:32:35,970][1157819] Updated weights for policy 0, policy_version 201358 (0.0006) [2024-09-30 03:32:36,482][1157819] Updated weights for policy 0, policy_version 201368 (0.0006) [2024-09-30 03:32:37,022][1157819] Updated weights for policy 0, policy_version 201378 (0.0006) [2024-09-30 03:32:37,595][1157819] Updated weights for policy 0, policy_version 201388 (0.0006) [2024-09-30 03:32:38,098][1157819] Updated weights for policy 0, policy_version 201398 (0.0006) [2024-09-30 03:32:38,611][1157819] Updated weights for policy 0, policy_version 201408 (0.0006) [2024-09-30 03:32:39,136][1157819] Updated weights for policy 0, policy_version 201418 (0.0006) [2024-09-30 03:32:39,661][1157819] Updated weights for policy 0, policy_version 201428 (0.0006) [2024-09-30 03:32:40,239][1157819] Updated weights for policy 0, policy_version 201438 (0.0006) [2024-09-30 03:32:40,466][1157520] Fps is (10 sec: 77004.2, 60 sec: 75912.6, 300 sec: 75158.1). Total num frames: 825106432. Throughput: 0: 19218.0. Samples: 196225652. Policy #0 lag: (min: 0.0, avg: 2.3, max: 4.0) [2024-09-30 03:32:40,466][1157520] Avg episode reward: [(0, '56.690')] [2024-09-30 03:32:40,791][1157819] Updated weights for policy 0, policy_version 201448 (0.0006) [2024-09-30 03:32:41,365][1157819] Updated weights for policy 0, policy_version 201458 (0.0006) [2024-09-30 03:32:41,882][1157819] Updated weights for policy 0, policy_version 201468 (0.0006) [2024-09-30 03:32:42,456][1157819] Updated weights for policy 0, policy_version 201478 (0.0006) [2024-09-30 03:32:43,031][1157819] Updated weights for policy 0, policy_version 201488 (0.0006) [2024-09-30 03:32:43,589][1157819] Updated weights for policy 0, policy_version 201498 (0.0006) [2024-09-30 03:32:44,163][1157819] Updated weights for policy 0, policy_version 201508 (0.0007) [2024-09-30 03:32:44,719][1157819] Updated weights for policy 0, policy_version 201518 (0.0006) [2024-09-30 03:32:45,246][1157819] Updated weights for policy 0, policy_version 201528 (0.0006) [2024-09-30 03:32:45,466][1157520] Fps is (10 sec: 74956.7, 60 sec: 75707.7, 300 sec: 75158.1). Total num frames: 825470976. Throughput: 0: 19365.6. Samples: 196337784. Policy #0 lag: (min: 0.0, avg: 2.3, max: 4.0) [2024-09-30 03:32:45,466][1157520] Avg episode reward: [(0, '56.608')] [2024-09-30 03:32:45,817][1157819] Updated weights for policy 0, policy_version 201538 (0.0006) [2024-09-30 03:32:46,369][1157819] Updated weights for policy 0, policy_version 201548 (0.0006) [2024-09-30 03:32:46,971][1157819] Updated weights for policy 0, policy_version 201558 (0.0006) [2024-09-30 03:32:47,487][1157819] Updated weights for policy 0, policy_version 201568 (0.0006) [2024-09-30 03:32:48,020][1157819] Updated weights for policy 0, policy_version 201578 (0.0006) [2024-09-30 03:32:48,602][1157819] Updated weights for policy 0, policy_version 201588 (0.0006) [2024-09-30 03:32:49,137][1157819] Updated weights for policy 0, policy_version 201598 (0.0006) [2024-09-30 03:32:49,654][1157819] Updated weights for policy 0, policy_version 201608 (0.0006) [2024-09-30 03:32:50,238][1157819] Updated weights for policy 0, policy_version 201618 (0.0006) [2024-09-30 03:32:50,466][1157520] Fps is (10 sec: 73318.1, 60 sec: 76049.0, 300 sec: 75213.7). Total num frames: 825839616. Throughput: 0: 19351.1. Samples: 196448500. Policy #0 lag: (min: 0.0, avg: 2.3, max: 4.0) [2024-09-30 03:32:50,466][1157520] Avg episode reward: [(0, '55.239')] [2024-09-30 03:32:50,834][1157819] Updated weights for policy 0, policy_version 201628 (0.0006) [2024-09-30 03:32:51,387][1157819] Updated weights for policy 0, policy_version 201638 (0.0006) [2024-09-30 03:32:51,936][1157819] Updated weights for policy 0, policy_version 201648 (0.0006) [2024-09-30 03:32:52,476][1157819] Updated weights for policy 0, policy_version 201658 (0.0006) [2024-09-30 03:32:53,011][1157819] Updated weights for policy 0, policy_version 201668 (0.0006) [2024-09-30 03:32:53,531][1157819] Updated weights for policy 0, policy_version 201678 (0.0006) [2024-09-30 03:32:54,090][1157819] Updated weights for policy 0, policy_version 201688 (0.0006) [2024-09-30 03:32:54,645][1157819] Updated weights for policy 0, policy_version 201698 (0.0006) [2024-09-30 03:32:55,157][1157819] Updated weights for policy 0, policy_version 201708 (0.0006) [2024-09-30 03:32:55,466][1157520] Fps is (10 sec: 74547.2, 60 sec: 76458.7, 300 sec: 75297.0). Total num frames: 826216448. Throughput: 0: 19261.6. Samples: 196503828. Policy #0 lag: (min: 0.0, avg: 2.3, max: 4.0) [2024-09-30 03:32:55,466][1157520] Avg episode reward: [(0, '55.620')] [2024-09-30 03:32:55,703][1157819] Updated weights for policy 0, policy_version 201718 (0.0006) [2024-09-30 03:32:56,268][1157819] Updated weights for policy 0, policy_version 201728 (0.0006) [2024-09-30 03:32:56,815][1157819] Updated weights for policy 0, policy_version 201738 (0.0006) [2024-09-30 03:32:57,406][1157819] Updated weights for policy 0, policy_version 201748 (0.0006) [2024-09-30 03:32:57,962][1157819] Updated weights for policy 0, policy_version 201758 (0.0006) [2024-09-30 03:32:58,496][1157819] Updated weights for policy 0, policy_version 201768 (0.0006) [2024-09-30 03:32:59,074][1157819] Updated weights for policy 0, policy_version 201778 (0.0006) [2024-09-30 03:32:59,625][1157819] Updated weights for policy 0, policy_version 201788 (0.0006) [2024-09-30 03:33:00,141][1157819] Updated weights for policy 0, policy_version 201798 (0.0006) [2024-09-30 03:33:00,466][1157520] Fps is (10 sec: 74547.2, 60 sec: 76663.5, 300 sec: 75338.6). Total num frames: 826585088. Throughput: 0: 19093.9. Samples: 196615588. Policy #0 lag: (min: 0.0, avg: 2.3, max: 4.0) [2024-09-30 03:33:00,466][1157520] Avg episode reward: [(0, '54.273')] [2024-09-30 03:33:00,725][1157819] Updated weights for policy 0, policy_version 201808 (0.0006) [2024-09-30 03:33:01,279][1157819] Updated weights for policy 0, policy_version 201818 (0.0006) [2024-09-30 03:33:01,828][1157819] Updated weights for policy 0, policy_version 201828 (0.0006) [2024-09-30 03:33:02,342][1157819] Updated weights for policy 0, policy_version 201838 (0.0006) [2024-09-30 03:33:02,904][1157819] Updated weights for policy 0, policy_version 201848 (0.0006) [2024-09-30 03:33:03,452][1157819] Updated weights for policy 0, policy_version 201858 (0.0006) [2024-09-30 03:33:04,007][1157819] Updated weights for policy 0, policy_version 201868 (0.0006) [2024-09-30 03:33:04,579][1157819] Updated weights for policy 0, policy_version 201878 (0.0006) [2024-09-30 03:33:05,150][1157819] Updated weights for policy 0, policy_version 201888 (0.0006) [2024-09-30 03:33:05,466][1157520] Fps is (10 sec: 73727.9, 60 sec: 76595.2, 300 sec: 75338.6). Total num frames: 826953728. Throughput: 0: 18908.7. Samples: 196727244. Policy #0 lag: (min: 0.0, avg: 2.3, max: 4.0) [2024-09-30 03:33:05,466][1157520] Avg episode reward: [(0, '55.567')] [2024-09-30 03:33:05,734][1157819] Updated weights for policy 0, policy_version 201898 (0.0006) [2024-09-30 03:33:06,350][1157819] Updated weights for policy 0, policy_version 201908 (0.0006) [2024-09-30 03:33:06,968][1157819] Updated weights for policy 0, policy_version 201918 (0.0006) [2024-09-30 03:33:07,561][1157819] Updated weights for policy 0, policy_version 201928 (0.0006) [2024-09-30 03:33:08,123][1157819] Updated weights for policy 0, policy_version 201938 (0.0006) [2024-09-30 03:33:08,741][1157819] Updated weights for policy 0, policy_version 201948 (0.0006) [2024-09-30 03:33:09,283][1157819] Updated weights for policy 0, policy_version 201958 (0.0006) [2024-09-30 03:33:09,895][1157819] Updated weights for policy 0, policy_version 201968 (0.0006) [2024-09-30 03:33:10,466][1157520] Fps is (10 sec: 71270.8, 60 sec: 75639.4, 300 sec: 75310.9). Total num frames: 827297792. Throughput: 0: 18752.8. Samples: 196778920. Policy #0 lag: (min: 0.0, avg: 2.3, max: 4.0) [2024-09-30 03:33:10,466][1157520] Avg episode reward: [(0, '55.313')] [2024-09-30 03:33:10,499][1157819] Updated weights for policy 0, policy_version 201978 (0.0006) [2024-09-30 03:33:10,844][1157736] Signal inference workers to stop experience collection... (14050 times) [2024-09-30 03:33:10,847][1157819] InferenceWorker_p0-w0: stopping experience collection (14050 times) [2024-09-30 03:33:10,853][1157736] Signal inference workers to resume experience collection... (14050 times) [2024-09-30 03:33:10,854][1157819] InferenceWorker_p0-w0: resuming experience collection (14050 times) [2024-09-30 03:33:11,065][1157819] Updated weights for policy 0, policy_version 201988 (0.0006) [2024-09-30 03:33:11,675][1157819] Updated weights for policy 0, policy_version 201998 (0.0006) [2024-09-30 03:33:12,316][1157819] Updated weights for policy 0, policy_version 202008 (0.0006) [2024-09-30 03:33:12,880][1157819] Updated weights for policy 0, policy_version 202018 (0.0006) [2024-09-30 03:33:13,485][1157819] Updated weights for policy 0, policy_version 202028 (0.0006) [2024-09-30 03:33:14,091][1157819] Updated weights for policy 0, policy_version 202038 (0.0007) [2024-09-30 03:33:14,679][1157819] Updated weights for policy 0, policy_version 202048 (0.0006) [2024-09-30 03:33:15,298][1157819] Updated weights for policy 0, policy_version 202058 (0.0006) [2024-09-30 03:33:15,466][1157520] Fps is (10 sec: 68402.9, 60 sec: 74683.6, 300 sec: 75297.0). Total num frames: 827637760. Throughput: 0: 18459.7. Samples: 196881996. Policy #0 lag: (min: 0.0, avg: 2.3, max: 4.0) [2024-09-30 03:33:15,466][1157520] Avg episode reward: [(0, '56.802')] [2024-09-30 03:33:15,891][1157819] Updated weights for policy 0, policy_version 202068 (0.0006) [2024-09-30 03:33:16,470][1157819] Updated weights for policy 0, policy_version 202078 (0.0006) [2024-09-30 03:33:17,085][1157819] Updated weights for policy 0, policy_version 202088 (0.0007) [2024-09-30 03:33:17,658][1157819] Updated weights for policy 0, policy_version 202098 (0.0006) [2024-09-30 03:33:18,252][1157819] Updated weights for policy 0, policy_version 202108 (0.0006) [2024-09-30 03:33:18,840][1157819] Updated weights for policy 0, policy_version 202118 (0.0006) [2024-09-30 03:33:19,444][1157819] Updated weights for policy 0, policy_version 202128 (0.0006) [2024-09-30 03:33:20,042][1157819] Updated weights for policy 0, policy_version 202138 (0.0006) [2024-09-30 03:33:20,466][1157520] Fps is (10 sec: 68812.3, 60 sec: 73796.1, 300 sec: 75297.0). Total num frames: 827985920. Throughput: 0: 18163.4. Samples: 196985164. Policy #0 lag: (min: 0.0, avg: 2.3, max: 4.0) [2024-09-30 03:33:20,466][1157520] Avg episode reward: [(0, '57.249')] [2024-09-30 03:33:20,635][1157819] Updated weights for policy 0, policy_version 202148 (0.0006) [2024-09-30 03:33:21,239][1157819] Updated weights for policy 0, policy_version 202158 (0.0006) [2024-09-30 03:33:21,855][1157819] Updated weights for policy 0, policy_version 202168 (0.0006) [2024-09-30 03:33:22,453][1157819] Updated weights for policy 0, policy_version 202178 (0.0006) [2024-09-30 03:33:23,063][1157819] Updated weights for policy 0, policy_version 202188 (0.0006) [2024-09-30 03:33:23,674][1157819] Updated weights for policy 0, policy_version 202198 (0.0006) [2024-09-30 03:33:24,265][1157819] Updated weights for policy 0, policy_version 202208 (0.0006) [2024-09-30 03:33:24,871][1157819] Updated weights for policy 0, policy_version 202218 (0.0006) [2024-09-30 03:33:25,466][1157520] Fps is (10 sec: 68403.5, 60 sec: 72908.7, 300 sec: 75255.3). Total num frames: 828321792. Throughput: 0: 18021.6. Samples: 197036624. Policy #0 lag: (min: 0.0, avg: 2.3, max: 4.0) [2024-09-30 03:33:25,466][1157520] Avg episode reward: [(0, '56.511')] [2024-09-30 03:33:25,502][1157819] Updated weights for policy 0, policy_version 202228 (0.0006) [2024-09-30 03:33:26,157][1157819] Updated weights for policy 0, policy_version 202238 (0.0006) [2024-09-30 03:33:26,770][1157819] Updated weights for policy 0, policy_version 202248 (0.0006) [2024-09-30 03:33:27,419][1157819] Updated weights for policy 0, policy_version 202258 (0.0006) [2024-09-30 03:33:28,004][1157819] Updated weights for policy 0, policy_version 202268 (0.0006) [2024-09-30 03:33:28,667][1157819] Updated weights for policy 0, policy_version 202278 (0.0006) [2024-09-30 03:33:28,766][1157736] Signal inference workers to stop experience collection... (14100 times) [2024-09-30 03:33:28,767][1157736] Signal inference workers to resume experience collection... (14100 times) [2024-09-30 03:33:28,770][1157819] InferenceWorker_p0-w0: stopping experience collection (14100 times) [2024-09-30 03:33:28,772][1157819] InferenceWorker_p0-w0: resuming experience collection (14100 times) [2024-09-30 03:33:29,310][1157819] Updated weights for policy 0, policy_version 202288 (0.0006) [2024-09-30 03:33:29,972][1157819] Updated weights for policy 0, policy_version 202298 (0.0006) [2024-09-30 03:33:30,466][1157520] Fps is (10 sec: 65536.1, 60 sec: 71748.1, 300 sec: 75158.1). Total num frames: 828641280. Throughput: 0: 17705.1. Samples: 197134512. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 03:33:30,466][1157520] Avg episode reward: [(0, '57.971')] [2024-09-30 03:33:30,599][1157819] Updated weights for policy 0, policy_version 202308 (0.0006) [2024-09-30 03:33:31,224][1157819] Updated weights for policy 0, policy_version 202318 (0.0006) [2024-09-30 03:33:31,891][1157819] Updated weights for policy 0, policy_version 202328 (0.0006) [2024-09-30 03:33:32,483][1157819] Updated weights for policy 0, policy_version 202338 (0.0006) [2024-09-30 03:33:33,046][1157819] Updated weights for policy 0, policy_version 202348 (0.0006) [2024-09-30 03:33:33,685][1157819] Updated weights for policy 0, policy_version 202358 (0.0006) [2024-09-30 03:33:34,274][1157819] Updated weights for policy 0, policy_version 202368 (0.0006) [2024-09-30 03:33:34,856][1157819] Updated weights for policy 0, policy_version 202378 (0.0007) [2024-09-30 03:33:35,466][1157520] Fps is (10 sec: 65536.1, 60 sec: 70929.1, 300 sec: 75116.5). Total num frames: 828977152. Throughput: 0: 17446.7. Samples: 197233600. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 03:33:35,466][1157520] Avg episode reward: [(0, '55.966')] [2024-09-30 03:33:35,483][1157819] Updated weights for policy 0, policy_version 202388 (0.0007) [2024-09-30 03:33:36,094][1157819] Updated weights for policy 0, policy_version 202398 (0.0006) [2024-09-30 03:33:36,700][1157819] Updated weights for policy 0, policy_version 202408 (0.0007) [2024-09-30 03:33:37,289][1157819] Updated weights for policy 0, policy_version 202418 (0.0006) [2024-09-30 03:33:37,866][1157819] Updated weights for policy 0, policy_version 202428 (0.0006) [2024-09-30 03:33:38,488][1157819] Updated weights for policy 0, policy_version 202438 (0.0006) [2024-09-30 03:33:39,078][1157819] Updated weights for policy 0, policy_version 202448 (0.0006) [2024-09-30 03:33:39,646][1157819] Updated weights for policy 0, policy_version 202458 (0.0006) [2024-09-30 03:33:40,260][1157819] Updated weights for policy 0, policy_version 202468 (0.0006) [2024-09-30 03:33:40,466][1157520] Fps is (10 sec: 67993.9, 60 sec: 70246.4, 300 sec: 75019.3). Total num frames: 829321216. Throughput: 0: 17347.2. Samples: 197284452. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 03:33:40,466][1157520] Avg episode reward: [(0, '59.158')] [2024-09-30 03:33:40,484][1157736] Saving new best policy, reward=59.158! [2024-09-30 03:33:40,824][1157819] Updated weights for policy 0, policy_version 202478 (0.0006) [2024-09-30 03:33:41,406][1157819] Updated weights for policy 0, policy_version 202488 (0.0006) [2024-09-30 03:33:41,934][1157819] Updated weights for policy 0, policy_version 202498 (0.0006) [2024-09-30 03:33:42,511][1157819] Updated weights for policy 0, policy_version 202508 (0.0006) [2024-09-30 03:33:43,075][1157819] Updated weights for policy 0, policy_version 202518 (0.0006) [2024-09-30 03:33:43,639][1157819] Updated weights for policy 0, policy_version 202528 (0.0006) [2024-09-30 03:33:44,235][1157819] Updated weights for policy 0, policy_version 202538 (0.0006) [2024-09-30 03:33:44,758][1157819] Updated weights for policy 0, policy_version 202548 (0.0006) [2024-09-30 03:33:45,310][1157819] Updated weights for policy 0, policy_version 202558 (0.0006) [2024-09-30 03:33:45,466][1157520] Fps is (10 sec: 70860.5, 60 sec: 70246.3, 300 sec: 75005.4). Total num frames: 829685760. Throughput: 0: 17236.4. Samples: 197391228. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 03:33:45,466][1157520] Avg episode reward: [(0, '56.183')] [2024-09-30 03:33:45,862][1157819] Updated weights for policy 0, policy_version 202568 (0.0006) [2024-09-30 03:33:46,415][1157819] Updated weights for policy 0, policy_version 202578 (0.0006) [2024-09-30 03:33:46,948][1157819] Updated weights for policy 0, policy_version 202588 (0.0006) [2024-09-30 03:33:47,512][1157819] Updated weights for policy 0, policy_version 202598 (0.0006) [2024-09-30 03:33:48,060][1157819] Updated weights for policy 0, policy_version 202608 (0.0006) [2024-09-30 03:33:48,588][1157819] Updated weights for policy 0, policy_version 202618 (0.0006) [2024-09-30 03:33:49,111][1157819] Updated weights for policy 0, policy_version 202628 (0.0006) [2024-09-30 03:33:49,694][1157819] Updated weights for policy 0, policy_version 202638 (0.0006) [2024-09-30 03:33:50,214][1157819] Updated weights for policy 0, policy_version 202648 (0.0006) [2024-09-30 03:33:50,466][1157520] Fps is (10 sec: 74137.2, 60 sec: 70382.9, 300 sec: 75088.7). Total num frames: 830062592. Throughput: 0: 17240.9. Samples: 197503084. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 03:33:50,466][1157520] Avg episode reward: [(0, '58.562')] [2024-09-30 03:33:50,731][1157819] Updated weights for policy 0, policy_version 202658 (0.0006) [2024-09-30 03:33:51,235][1157819] Updated weights for policy 0, policy_version 202668 (0.0006) [2024-09-30 03:33:51,751][1157819] Updated weights for policy 0, policy_version 202678 (0.0006) [2024-09-30 03:33:52,295][1157819] Updated weights for policy 0, policy_version 202688 (0.0006) [2024-09-30 03:33:52,794][1157819] Updated weights for policy 0, policy_version 202698 (0.0006) [2024-09-30 03:33:53,328][1157819] Updated weights for policy 0, policy_version 202708 (0.0006) [2024-09-30 03:33:53,823][1157819] Updated weights for policy 0, policy_version 202718 (0.0006) [2024-09-30 03:33:54,377][1157819] Updated weights for policy 0, policy_version 202728 (0.0006) [2024-09-30 03:33:54,887][1157819] Updated weights for policy 0, policy_version 202738 (0.0006) [2024-09-30 03:33:55,402][1157819] Updated weights for policy 0, policy_version 202748 (0.0006) [2024-09-30 03:33:55,466][1157520] Fps is (10 sec: 77414.5, 60 sec: 70724.2, 300 sec: 75227.5). Total num frames: 830459904. Throughput: 0: 17410.4. Samples: 197562388. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 03:33:55,466][1157520] Avg episode reward: [(0, '56.485')] [2024-09-30 03:33:55,921][1157819] Updated weights for policy 0, policy_version 202758 (0.0006) [2024-09-30 03:33:56,426][1157819] Updated weights for policy 0, policy_version 202768 (0.0006) [2024-09-30 03:33:56,948][1157819] Updated weights for policy 0, policy_version 202778 (0.0006) [2024-09-30 03:33:57,484][1157819] Updated weights for policy 0, policy_version 202788 (0.0006) [2024-09-30 03:33:57,999][1157819] Updated weights for policy 0, policy_version 202798 (0.0006) [2024-09-30 03:33:58,517][1157819] Updated weights for policy 0, policy_version 202808 (0.0006) [2024-09-30 03:33:59,054][1157819] Updated weights for policy 0, policy_version 202818 (0.0006) [2024-09-30 03:33:59,568][1157819] Updated weights for policy 0, policy_version 202828 (0.0006) [2024-09-30 03:34:00,079][1157819] Updated weights for policy 0, policy_version 202838 (0.0006) [2024-09-30 03:34:00,466][1157520] Fps is (10 sec: 79053.0, 60 sec: 71133.9, 300 sec: 75422.0). Total num frames: 830853120. Throughput: 0: 17748.4. Samples: 197680672. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 03:34:00,466][1157520] Avg episode reward: [(0, '56.315')] [2024-09-30 03:34:00,613][1157819] Updated weights for policy 0, policy_version 202848 (0.0006) [2024-09-30 03:34:01,119][1157819] Updated weights for policy 0, policy_version 202858 (0.0006) [2024-09-30 03:34:01,616][1157819] Updated weights for policy 0, policy_version 202868 (0.0006) [2024-09-30 03:34:02,191][1157819] Updated weights for policy 0, policy_version 202878 (0.0006) [2024-09-30 03:34:02,688][1157819] Updated weights for policy 0, policy_version 202888 (0.0006) [2024-09-30 03:34:03,197][1157819] Updated weights for policy 0, policy_version 202898 (0.0006) [2024-09-30 03:34:03,706][1157819] Updated weights for policy 0, policy_version 202908 (0.0006) [2024-09-30 03:34:04,244][1157819] Updated weights for policy 0, policy_version 202918 (0.0006) [2024-09-30 03:34:04,760][1157819] Updated weights for policy 0, policy_version 202928 (0.0006) [2024-09-30 03:34:05,253][1157819] Updated weights for policy 0, policy_version 202938 (0.0006) [2024-09-30 03:34:05,466][1157520] Fps is (10 sec: 78643.0, 60 sec: 71543.4, 300 sec: 75658.0). Total num frames: 831246336. Throughput: 0: 18085.6. Samples: 197799016. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 03:34:05,466][1157520] Avg episode reward: [(0, '56.291')] [2024-09-30 03:34:05,475][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000202942_831250432.pth... [2024-09-30 03:34:05,523][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000198467_812920832.pth [2024-09-30 03:34:05,780][1157819] Updated weights for policy 0, policy_version 202948 (0.0006) [2024-09-30 03:34:06,310][1157819] Updated weights for policy 0, policy_version 202958 (0.0006) [2024-09-30 03:34:06,809][1157819] Updated weights for policy 0, policy_version 202968 (0.0006) [2024-09-30 03:34:07,321][1157819] Updated weights for policy 0, policy_version 202978 (0.0006) [2024-09-30 03:34:07,836][1157819] Updated weights for policy 0, policy_version 202988 (0.0006) [2024-09-30 03:34:08,364][1157819] Updated weights for policy 0, policy_version 202998 (0.0006) [2024-09-30 03:34:08,877][1157819] Updated weights for policy 0, policy_version 203008 (0.0006) [2024-09-30 03:34:09,367][1157819] Updated weights for policy 0, policy_version 203018 (0.0006) [2024-09-30 03:34:09,850][1157819] Updated weights for policy 0, policy_version 203028 (0.0006) [2024-09-30 03:34:10,376][1157819] Updated weights for policy 0, policy_version 203038 (0.0006) [2024-09-30 03:34:10,466][1157520] Fps is (10 sec: 79462.6, 60 sec: 72499.2, 300 sec: 75852.4). Total num frames: 831647744. Throughput: 0: 18267.8. Samples: 197858676. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 03:34:10,466][1157520] Avg episode reward: [(0, '56.982')] [2024-09-30 03:34:10,941][1157819] Updated weights for policy 0, policy_version 203048 (0.0006) [2024-09-30 03:34:11,443][1157819] Updated weights for policy 0, policy_version 203058 (0.0006) [2024-09-30 03:34:11,939][1157819] Updated weights for policy 0, policy_version 203068 (0.0006) [2024-09-30 03:34:12,472][1157819] Updated weights for policy 0, policy_version 203078 (0.0006) [2024-09-30 03:34:12,966][1157819] Updated weights for policy 0, policy_version 203088 (0.0006) [2024-09-30 03:34:13,425][1157819] Updated weights for policy 0, policy_version 203098 (0.0006) [2024-09-30 03:34:13,951][1157819] Updated weights for policy 0, policy_version 203108 (0.0006) [2024-09-30 03:34:14,437][1157819] Updated weights for policy 0, policy_version 203118 (0.0006) [2024-09-30 03:34:14,919][1157819] Updated weights for policy 0, policy_version 203128 (0.0006) [2024-09-30 03:34:15,419][1157819] Updated weights for policy 0, policy_version 203138 (0.0006) [2024-09-30 03:34:15,466][1157520] Fps is (10 sec: 81101.3, 60 sec: 73659.8, 300 sec: 76019.0). Total num frames: 832057344. Throughput: 0: 18777.7. Samples: 197979508. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 03:34:15,466][1157520] Avg episode reward: [(0, '56.534')] [2024-09-30 03:34:15,910][1157819] Updated weights for policy 0, policy_version 203148 (0.0006) [2024-09-30 03:34:16,389][1157819] Updated weights for policy 0, policy_version 203158 (0.0006) [2024-09-30 03:34:16,892][1157819] Updated weights for policy 0, policy_version 203168 (0.0006) [2024-09-30 03:34:17,439][1157819] Updated weights for policy 0, policy_version 203178 (0.0006) [2024-09-30 03:34:17,985][1157819] Updated weights for policy 0, policy_version 203188 (0.0006) [2024-09-30 03:34:18,509][1157819] Updated weights for policy 0, policy_version 203198 (0.0006) [2024-09-30 03:34:19,032][1157819] Updated weights for policy 0, policy_version 203208 (0.0006) [2024-09-30 03:34:19,589][1157819] Updated weights for policy 0, policy_version 203218 (0.0006) [2024-09-30 03:34:20,146][1157819] Updated weights for policy 0, policy_version 203228 (0.0006) [2024-09-30 03:34:20,466][1157520] Fps is (10 sec: 79871.6, 60 sec: 74342.4, 300 sec: 76060.6). Total num frames: 832446464. Throughput: 0: 19222.8. Samples: 198098624. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 03:34:20,466][1157520] Avg episode reward: [(0, '55.303')] [2024-09-30 03:34:20,705][1157819] Updated weights for policy 0, policy_version 203238 (0.0006) [2024-09-30 03:34:21,277][1157819] Updated weights for policy 0, policy_version 203248 (0.0006) [2024-09-30 03:34:21,777][1157819] Updated weights for policy 0, policy_version 203258 (0.0006) [2024-09-30 03:34:22,375][1157819] Updated weights for policy 0, policy_version 203268 (0.0006) [2024-09-30 03:34:22,916][1157819] Updated weights for policy 0, policy_version 203278 (0.0006) [2024-09-30 03:34:23,520][1157819] Updated weights for policy 0, policy_version 203288 (0.0006) [2024-09-30 03:34:24,096][1157819] Updated weights for policy 0, policy_version 203298 (0.0006) [2024-09-30 03:34:24,676][1157819] Updated weights for policy 0, policy_version 203308 (0.0006) [2024-09-30 03:34:25,282][1157819] Updated weights for policy 0, policy_version 203318 (0.0006) [2024-09-30 03:34:25,466][1157520] Fps is (10 sec: 74546.8, 60 sec: 74683.7, 300 sec: 76088.4). Total num frames: 832802816. Throughput: 0: 19332.8. Samples: 198154428. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 03:34:25,466][1157520] Avg episode reward: [(0, '56.264')] [2024-09-30 03:34:25,853][1157819] Updated weights for policy 0, policy_version 203328 (0.0006) [2024-09-30 03:34:26,445][1157819] Updated weights for policy 0, policy_version 203338 (0.0006) [2024-09-30 03:34:27,029][1157819] Updated weights for policy 0, policy_version 203348 (0.0006) [2024-09-30 03:34:27,637][1157819] Updated weights for policy 0, policy_version 203358 (0.0006) [2024-09-30 03:34:28,246][1157819] Updated weights for policy 0, policy_version 203368 (0.0006) [2024-09-30 03:34:28,426][1157736] Signal inference workers to stop experience collection... (14150 times) [2024-09-30 03:34:28,430][1157736] Signal inference workers to resume experience collection... (14150 times) [2024-09-30 03:34:28,431][1157819] InferenceWorker_p0-w0: stopping experience collection (14150 times) [2024-09-30 03:34:28,434][1157819] InferenceWorker_p0-w0: resuming experience collection (14150 times) [2024-09-30 03:34:28,799][1157819] Updated weights for policy 0, policy_version 203378 (0.0006) [2024-09-30 03:34:29,391][1157819] Updated weights for policy 0, policy_version 203388 (0.0006) [2024-09-30 03:34:29,945][1157819] Updated weights for policy 0, policy_version 203398 (0.0006) [2024-09-30 03:34:30,466][1157520] Fps is (10 sec: 70451.2, 60 sec: 75161.6, 300 sec: 76074.6). Total num frames: 833150976. Throughput: 0: 19276.6. Samples: 198258676. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 03:34:30,466][1157520] Avg episode reward: [(0, '57.952')] [2024-09-30 03:34:30,557][1157819] Updated weights for policy 0, policy_version 203408 (0.0006) [2024-09-30 03:34:31,170][1157819] Updated weights for policy 0, policy_version 203418 (0.0006) [2024-09-30 03:34:31,700][1157819] Updated weights for policy 0, policy_version 203428 (0.0006) [2024-09-30 03:34:32,264][1157819] Updated weights for policy 0, policy_version 203438 (0.0006) [2024-09-30 03:34:32,830][1157819] Updated weights for policy 0, policy_version 203448 (0.0006) [2024-09-30 03:34:33,433][1157819] Updated weights for policy 0, policy_version 203458 (0.0006) [2024-09-30 03:34:34,008][1157819] Updated weights for policy 0, policy_version 203468 (0.0006) [2024-09-30 03:34:34,600][1157819] Updated weights for policy 0, policy_version 203478 (0.0006) [2024-09-30 03:34:35,172][1157819] Updated weights for policy 0, policy_version 203488 (0.0006) [2024-09-30 03:34:35,466][1157520] Fps is (10 sec: 70041.6, 60 sec: 75434.6, 300 sec: 76019.0). Total num frames: 833503232. Throughput: 0: 19155.7. Samples: 198365092. Policy #0 lag: (min: 0.0, avg: 2.7, max: 6.0) [2024-09-30 03:34:35,466][1157520] Avg episode reward: [(0, '58.051')] [2024-09-30 03:34:35,763][1157819] Updated weights for policy 0, policy_version 203498 (0.0006) [2024-09-30 03:34:36,338][1157819] Updated weights for policy 0, policy_version 203508 (0.0006) [2024-09-30 03:34:36,942][1157819] Updated weights for policy 0, policy_version 203518 (0.0006) [2024-09-30 03:34:37,531][1157819] Updated weights for policy 0, policy_version 203528 (0.0006) [2024-09-30 03:34:38,063][1157819] Updated weights for policy 0, policy_version 203538 (0.0006) [2024-09-30 03:34:38,579][1157819] Updated weights for policy 0, policy_version 203548 (0.0006) [2024-09-30 03:34:39,085][1157819] Updated weights for policy 0, policy_version 203558 (0.0006) [2024-09-30 03:34:39,624][1157819] Updated weights for policy 0, policy_version 203568 (0.0006) [2024-09-30 03:34:40,167][1157819] Updated weights for policy 0, policy_version 203578 (0.0006) [2024-09-30 03:34:40,466][1157520] Fps is (10 sec: 72909.2, 60 sec: 75980.8, 300 sec: 76102.3). Total num frames: 833880064. Throughput: 0: 19004.9. Samples: 198417608. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 03:34:40,466][1157520] Avg episode reward: [(0, '56.746')] [2024-09-30 03:34:40,699][1157819] Updated weights for policy 0, policy_version 203588 (0.0006) [2024-09-30 03:34:41,214][1157819] Updated weights for policy 0, policy_version 203598 (0.0006) [2024-09-30 03:34:41,710][1157819] Updated weights for policy 0, policy_version 203608 (0.0006) [2024-09-30 03:34:42,250][1157819] Updated weights for policy 0, policy_version 203618 (0.0006) [2024-09-30 03:34:42,757][1157819] Updated weights for policy 0, policy_version 203628 (0.0006) [2024-09-30 03:34:43,353][1157819] Updated weights for policy 0, policy_version 203638 (0.0006) [2024-09-30 03:34:43,871][1157819] Updated weights for policy 0, policy_version 203648 (0.0006) [2024-09-30 03:34:44,414][1157819] Updated weights for policy 0, policy_version 203658 (0.0006) [2024-09-30 03:34:45,013][1157819] Updated weights for policy 0, policy_version 203668 (0.0006) [2024-09-30 03:34:45,466][1157520] Fps is (10 sec: 74956.8, 60 sec: 76117.3, 300 sec: 76213.4). Total num frames: 834252800. Throughput: 0: 18960.1. Samples: 198533876. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 03:34:45,466][1157520] Avg episode reward: [(0, '52.505')] [2024-09-30 03:34:45,574][1157819] Updated weights for policy 0, policy_version 203678 (0.0006) [2024-09-30 03:34:46,122][1157819] Updated weights for policy 0, policy_version 203688 (0.0006) [2024-09-30 03:34:46,623][1157819] Updated weights for policy 0, policy_version 203698 (0.0006) [2024-09-30 03:34:47,169][1157819] Updated weights for policy 0, policy_version 203708 (0.0006) [2024-09-30 03:34:47,727][1157819] Updated weights for policy 0, policy_version 203718 (0.0006) [2024-09-30 03:34:48,252][1157819] Updated weights for policy 0, policy_version 203728 (0.0006) [2024-09-30 03:34:48,859][1157819] Updated weights for policy 0, policy_version 203738 (0.0006) [2024-09-30 03:34:49,368][1157819] Updated weights for policy 0, policy_version 203748 (0.0006) [2024-09-30 03:34:49,940][1157819] Updated weights for policy 0, policy_version 203758 (0.0006) [2024-09-30 03:34:50,466][1157520] Fps is (10 sec: 74956.4, 60 sec: 76117.3, 300 sec: 76227.3). Total num frames: 834629632. Throughput: 0: 18810.1. Samples: 198645468. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 03:34:50,466][1157520] Avg episode reward: [(0, '57.326')] [2024-09-30 03:34:50,485][1157819] Updated weights for policy 0, policy_version 203768 (0.0006) [2024-09-30 03:34:51,018][1157819] Updated weights for policy 0, policy_version 203778 (0.0006) [2024-09-30 03:34:51,591][1157819] Updated weights for policy 0, policy_version 203788 (0.0006) [2024-09-30 03:34:52,116][1157819] Updated weights for policy 0, policy_version 203798 (0.0006) [2024-09-30 03:34:52,668][1157819] Updated weights for policy 0, policy_version 203808 (0.0006) [2024-09-30 03:34:53,217][1157819] Updated weights for policy 0, policy_version 203818 (0.0006) [2024-09-30 03:34:53,741][1157819] Updated weights for policy 0, policy_version 203828 (0.0006) [2024-09-30 03:34:54,315][1157819] Updated weights for policy 0, policy_version 203838 (0.0006) [2024-09-30 03:34:54,840][1157819] Updated weights for policy 0, policy_version 203848 (0.0006) [2024-09-30 03:34:55,403][1157819] Updated weights for policy 0, policy_version 203858 (0.0006) [2024-09-30 03:34:55,466][1157520] Fps is (10 sec: 74956.8, 60 sec: 75707.7, 300 sec: 76199.5). Total num frames: 835002368. Throughput: 0: 18727.5. Samples: 198701416. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 03:34:55,466][1157520] Avg episode reward: [(0, '55.839')] [2024-09-30 03:34:55,961][1157819] Updated weights for policy 0, policy_version 203868 (0.0006) [2024-09-30 03:34:56,538][1157819] Updated weights for policy 0, policy_version 203878 (0.0006) [2024-09-30 03:34:57,064][1157819] Updated weights for policy 0, policy_version 203888 (0.0006) [2024-09-30 03:34:57,597][1157819] Updated weights for policy 0, policy_version 203898 (0.0006) [2024-09-30 03:34:58,182][1157819] Updated weights for policy 0, policy_version 203908 (0.0006) [2024-09-30 03:34:58,699][1157819] Updated weights for policy 0, policy_version 203918 (0.0006) [2024-09-30 03:34:59,255][1157819] Updated weights for policy 0, policy_version 203928 (0.0006) [2024-09-30 03:34:59,805][1157819] Updated weights for policy 0, policy_version 203938 (0.0006) [2024-09-30 03:35:00,310][1157819] Updated weights for policy 0, policy_version 203948 (0.0006) [2024-09-30 03:35:00,466][1157520] Fps is (10 sec: 74956.8, 60 sec: 75434.6, 300 sec: 76144.0). Total num frames: 835379200. Throughput: 0: 18530.1. Samples: 198813364. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 03:35:00,466][1157520] Avg episode reward: [(0, '57.616')] [2024-09-30 03:35:00,886][1157819] Updated weights for policy 0, policy_version 203958 (0.0006) [2024-09-30 03:35:01,430][1157819] Updated weights for policy 0, policy_version 203968 (0.0006) [2024-09-30 03:35:01,912][1157819] Updated weights for policy 0, policy_version 203978 (0.0006) [2024-09-30 03:35:02,430][1157819] Updated weights for policy 0, policy_version 203988 (0.0006) [2024-09-30 03:35:02,977][1157819] Updated weights for policy 0, policy_version 203998 (0.0006) [2024-09-30 03:35:03,519][1157819] Updated weights for policy 0, policy_version 204008 (0.0006) [2024-09-30 03:35:04,042][1157819] Updated weights for policy 0, policy_version 204018 (0.0006) [2024-09-30 03:35:04,613][1157819] Updated weights for policy 0, policy_version 204028 (0.0006) [2024-09-30 03:35:05,143][1157819] Updated weights for policy 0, policy_version 204038 (0.0006) [2024-09-30 03:35:05,466][1157520] Fps is (10 sec: 76186.0, 60 sec: 75298.2, 300 sec: 76032.8). Total num frames: 835764224. Throughput: 0: 18435.7. Samples: 198928228. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 03:35:05,466][1157520] Avg episode reward: [(0, '53.233')] [2024-09-30 03:35:05,696][1157819] Updated weights for policy 0, policy_version 204048 (0.0007) [2024-09-30 03:35:06,292][1157819] Updated weights for policy 0, policy_version 204058 (0.0007) [2024-09-30 03:35:06,853][1157819] Updated weights for policy 0, policy_version 204068 (0.0007) [2024-09-30 03:35:07,390][1157819] Updated weights for policy 0, policy_version 204078 (0.0006) [2024-09-30 03:35:07,963][1157819] Updated weights for policy 0, policy_version 204088 (0.0006) [2024-09-30 03:35:08,533][1157819] Updated weights for policy 0, policy_version 204098 (0.0006) [2024-09-30 03:35:09,055][1157819] Updated weights for policy 0, policy_version 204108 (0.0006) [2024-09-30 03:35:09,565][1157819] Updated weights for policy 0, policy_version 204118 (0.0006) [2024-09-30 03:35:10,071][1157819] Updated weights for policy 0, policy_version 204128 (0.0006) [2024-09-30 03:35:10,466][1157520] Fps is (10 sec: 75776.9, 60 sec: 74820.4, 300 sec: 75755.2). Total num frames: 836136960. Throughput: 0: 18408.6. Samples: 198982812. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 03:35:10,466][1157520] Avg episode reward: [(0, '57.639')] [2024-09-30 03:35:10,586][1157819] Updated weights for policy 0, policy_version 204138 (0.0006) [2024-09-30 03:35:11,113][1157819] Updated weights for policy 0, policy_version 204148 (0.0006) [2024-09-30 03:35:11,608][1157819] Updated weights for policy 0, policy_version 204158 (0.0006) [2024-09-30 03:35:12,181][1157819] Updated weights for policy 0, policy_version 204168 (0.0006) [2024-09-30 03:35:12,691][1157819] Updated weights for policy 0, policy_version 204178 (0.0006) [2024-09-30 03:35:13,180][1157819] Updated weights for policy 0, policy_version 204188 (0.0006) [2024-09-30 03:35:13,385][1157736] Signal inference workers to stop experience collection... (14200 times) [2024-09-30 03:35:13,386][1157736] Signal inference workers to resume experience collection... (14200 times) [2024-09-30 03:35:13,390][1157819] InferenceWorker_p0-w0: stopping experience collection (14200 times) [2024-09-30 03:35:13,390][1157819] InferenceWorker_p0-w0: resuming experience collection (14200 times) [2024-09-30 03:35:13,702][1157819] Updated weights for policy 0, policy_version 204198 (0.0006) [2024-09-30 03:35:14,229][1157819] Updated weights for policy 0, policy_version 204208 (0.0006) [2024-09-30 03:35:14,804][1157819] Updated weights for policy 0, policy_version 204218 (0.0006) [2024-09-30 03:35:15,322][1157819] Updated weights for policy 0, policy_version 204228 (0.0006) [2024-09-30 03:35:15,466][1157520] Fps is (10 sec: 76185.1, 60 sec: 74478.9, 300 sec: 75546.9). Total num frames: 836526080. Throughput: 0: 18707.5. Samples: 199100512. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 03:35:15,466][1157520] Avg episode reward: [(0, '56.347')] [2024-09-30 03:35:15,856][1157819] Updated weights for policy 0, policy_version 204238 (0.0006) [2024-09-30 03:35:16,402][1157819] Updated weights for policy 0, policy_version 204248 (0.0006) [2024-09-30 03:35:16,962][1157819] Updated weights for policy 0, policy_version 204258 (0.0006) [2024-09-30 03:35:17,467][1157819] Updated weights for policy 0, policy_version 204268 (0.0006) [2024-09-30 03:35:17,957][1157819] Updated weights for policy 0, policy_version 204278 (0.0006) [2024-09-30 03:35:18,448][1157819] Updated weights for policy 0, policy_version 204288 (0.0006) [2024-09-30 03:35:18,989][1157819] Updated weights for policy 0, policy_version 204298 (0.0006) [2024-09-30 03:35:19,515][1157819] Updated weights for policy 0, policy_version 204308 (0.0006) [2024-09-30 03:35:20,062][1157819] Updated weights for policy 0, policy_version 204318 (0.0006) [2024-09-30 03:35:20,466][1157520] Fps is (10 sec: 77822.7, 60 sec: 74478.9, 300 sec: 75352.5). Total num frames: 836915200. Throughput: 0: 18928.0. Samples: 199216852. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 03:35:20,466][1157520] Avg episode reward: [(0, '54.614')] [2024-09-30 03:35:20,578][1157819] Updated weights for policy 0, policy_version 204328 (0.0006) [2024-09-30 03:35:21,095][1157819] Updated weights for policy 0, policy_version 204338 (0.0006) [2024-09-30 03:35:21,629][1157819] Updated weights for policy 0, policy_version 204348 (0.0006) [2024-09-30 03:35:22,181][1157819] Updated weights for policy 0, policy_version 204358 (0.0006) [2024-09-30 03:35:22,723][1157819] Updated weights for policy 0, policy_version 204368 (0.0006) [2024-09-30 03:35:23,279][1157819] Updated weights for policy 0, policy_version 204378 (0.0006) [2024-09-30 03:35:23,820][1157819] Updated weights for policy 0, policy_version 204388 (0.0006) [2024-09-30 03:35:24,354][1157819] Updated weights for policy 0, policy_version 204398 (0.0006) [2024-09-30 03:35:24,888][1157819] Updated weights for policy 0, policy_version 204408 (0.0006) [2024-09-30 03:35:25,430][1157819] Updated weights for policy 0, policy_version 204418 (0.0006) [2024-09-30 03:35:25,466][1157520] Fps is (10 sec: 77005.2, 60 sec: 74888.6, 300 sec: 75241.5). Total num frames: 837296128. Throughput: 0: 19026.9. Samples: 199273820. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 03:35:25,466][1157520] Avg episode reward: [(0, '56.412')] [2024-09-30 03:35:25,986][1157819] Updated weights for policy 0, policy_version 204428 (0.0006) [2024-09-30 03:35:26,543][1157819] Updated weights for policy 0, policy_version 204438 (0.0006) [2024-09-30 03:35:27,053][1157819] Updated weights for policy 0, policy_version 204448 (0.0006) [2024-09-30 03:35:27,592][1157819] Updated weights for policy 0, policy_version 204458 (0.0006) [2024-09-30 03:35:28,153][1157819] Updated weights for policy 0, policy_version 204468 (0.0006) [2024-09-30 03:35:28,710][1157819] Updated weights for policy 0, policy_version 204478 (0.0006) [2024-09-30 03:35:29,257][1157819] Updated weights for policy 0, policy_version 204488 (0.0006) [2024-09-30 03:35:29,796][1157819] Updated weights for policy 0, policy_version 204498 (0.0006) [2024-09-30 03:35:30,346][1157819] Updated weights for policy 0, policy_version 204508 (0.0006) [2024-09-30 03:35:30,466][1157520] Fps is (10 sec: 75776.4, 60 sec: 75366.4, 300 sec: 75047.1). Total num frames: 837672960. Throughput: 0: 18959.8. Samples: 199387068. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 03:35:30,466][1157520] Avg episode reward: [(0, '54.187')] [2024-09-30 03:35:30,866][1157819] Updated weights for policy 0, policy_version 204518 (0.0006) [2024-09-30 03:35:31,425][1157819] Updated weights for policy 0, policy_version 204528 (0.0006) [2024-09-30 03:35:31,965][1157819] Updated weights for policy 0, policy_version 204538 (0.0006) [2024-09-30 03:35:32,532][1157819] Updated weights for policy 0, policy_version 204548 (0.0006) [2024-09-30 03:35:33,089][1157819] Updated weights for policy 0, policy_version 204558 (0.0006) [2024-09-30 03:35:33,614][1157819] Updated weights for policy 0, policy_version 204568 (0.0006) [2024-09-30 03:35:34,167][1157819] Updated weights for policy 0, policy_version 204578 (0.0006) [2024-09-30 03:35:34,688][1157819] Updated weights for policy 0, policy_version 204588 (0.0006) [2024-09-30 03:35:35,239][1157819] Updated weights for policy 0, policy_version 204598 (0.0006) [2024-09-30 03:35:35,466][1157520] Fps is (10 sec: 75366.5, 60 sec: 75776.1, 300 sec: 74866.5). Total num frames: 838049792. Throughput: 0: 18982.5. Samples: 199499680. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 03:35:35,466][1157520] Avg episode reward: [(0, '56.227')] [2024-09-30 03:35:35,803][1157819] Updated weights for policy 0, policy_version 204608 (0.0006) [2024-09-30 03:35:36,332][1157819] Updated weights for policy 0, policy_version 204618 (0.0006) [2024-09-30 03:35:36,864][1157819] Updated weights for policy 0, policy_version 204628 (0.0006) [2024-09-30 03:35:37,366][1157819] Updated weights for policy 0, policy_version 204638 (0.0006) [2024-09-30 03:35:37,876][1157819] Updated weights for policy 0, policy_version 204648 (0.0006) [2024-09-30 03:35:38,402][1157819] Updated weights for policy 0, policy_version 204658 (0.0006) [2024-09-30 03:35:38,927][1157819] Updated weights for policy 0, policy_version 204668 (0.0006) [2024-09-30 03:35:39,447][1157819] Updated weights for policy 0, policy_version 204678 (0.0006) [2024-09-30 03:35:40,007][1157819] Updated weights for policy 0, policy_version 204688 (0.0006) [2024-09-30 03:35:40,466][1157520] Fps is (10 sec: 76595.5, 60 sec: 75980.8, 300 sec: 74644.4). Total num frames: 838438912. Throughput: 0: 19021.7. Samples: 199557392. Policy #0 lag: (min: 0.0, avg: 1.8, max: 5.0) [2024-09-30 03:35:40,466][1157520] Avg episode reward: [(0, '57.348')] [2024-09-30 03:35:40,516][1157819] Updated weights for policy 0, policy_version 204698 (0.0006) [2024-09-30 03:35:40,995][1157819] Updated weights for policy 0, policy_version 204708 (0.0006) [2024-09-30 03:35:41,508][1157819] Updated weights for policy 0, policy_version 204718 (0.0006) [2024-09-30 03:35:42,045][1157819] Updated weights for policy 0, policy_version 204728 (0.0006) [2024-09-30 03:35:42,592][1157819] Updated weights for policy 0, policy_version 204738 (0.0006) [2024-09-30 03:35:43,093][1157819] Updated weights for policy 0, policy_version 204748 (0.0006) [2024-09-30 03:35:43,597][1157819] Updated weights for policy 0, policy_version 204758 (0.0006) [2024-09-30 03:35:44,107][1157819] Updated weights for policy 0, policy_version 204768 (0.0006) [2024-09-30 03:35:44,647][1157819] Updated weights for policy 0, policy_version 204778 (0.0006) [2024-09-30 03:35:45,155][1157819] Updated weights for policy 0, policy_version 204788 (0.0006) [2024-09-30 03:35:45,466][1157520] Fps is (10 sec: 78233.2, 60 sec: 76322.1, 300 sec: 74436.1). Total num frames: 838832128. Throughput: 0: 19166.8. Samples: 199675872. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:35:45,466][1157520] Avg episode reward: [(0, '53.150')] [2024-09-30 03:35:45,680][1157819] Updated weights for policy 0, policy_version 204798 (0.0006) [2024-09-30 03:35:46,189][1157819] Updated weights for policy 0, policy_version 204808 (0.0006) [2024-09-30 03:35:46,671][1157819] Updated weights for policy 0, policy_version 204818 (0.0006) [2024-09-30 03:35:47,171][1157819] Updated weights for policy 0, policy_version 204828 (0.0006) [2024-09-30 03:35:47,684][1157819] Updated weights for policy 0, policy_version 204838 (0.0006) [2024-09-30 03:35:48,126][1157819] Updated weights for policy 0, policy_version 204848 (0.0006) [2024-09-30 03:35:48,620][1157819] Updated weights for policy 0, policy_version 204858 (0.0006) [2024-09-30 03:35:49,183][1157819] Updated weights for policy 0, policy_version 204868 (0.0006) [2024-09-30 03:35:49,694][1157819] Updated weights for policy 0, policy_version 204878 (0.0006) [2024-09-30 03:35:50,187][1157819] Updated weights for policy 0, policy_version 204888 (0.0006) [2024-09-30 03:35:50,466][1157520] Fps is (10 sec: 80281.0, 60 sec: 76868.2, 300 sec: 74380.5). Total num frames: 839241728. Throughput: 0: 19312.0. Samples: 199797268. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:35:50,466][1157520] Avg episode reward: [(0, '54.777')] [2024-09-30 03:35:50,723][1157819] Updated weights for policy 0, policy_version 204898 (0.0006) [2024-09-30 03:35:51,215][1157819] Updated weights for policy 0, policy_version 204908 (0.0006) [2024-09-30 03:35:51,750][1157819] Updated weights for policy 0, policy_version 204918 (0.0006) [2024-09-30 03:35:52,251][1157819] Updated weights for policy 0, policy_version 204928 (0.0006) [2024-09-30 03:35:52,790][1157819] Updated weights for policy 0, policy_version 204938 (0.0006) [2024-09-30 03:35:53,318][1157819] Updated weights for policy 0, policy_version 204948 (0.0006) [2024-09-30 03:35:53,809][1157819] Updated weights for policy 0, policy_version 204958 (0.0006) [2024-09-30 03:35:54,353][1157819] Updated weights for policy 0, policy_version 204968 (0.0006) [2024-09-30 03:35:54,760][1157736] Signal inference workers to stop experience collection... (14250 times) [2024-09-30 03:35:54,761][1157736] Signal inference workers to resume experience collection... (14250 times) [2024-09-30 03:35:54,766][1157819] InferenceWorker_p0-w0: stopping experience collection (14250 times) [2024-09-30 03:35:54,766][1157819] InferenceWorker_p0-w0: resuming experience collection (14250 times) [2024-09-30 03:35:54,897][1157819] Updated weights for policy 0, policy_version 204978 (0.0006) [2024-09-30 03:35:55,429][1157819] Updated weights for policy 0, policy_version 204988 (0.0006) [2024-09-30 03:35:55,466][1157520] Fps is (10 sec: 79873.0, 60 sec: 77141.5, 300 sec: 74519.5). Total num frames: 839630848. Throughput: 0: 19417.9. Samples: 199856616. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:35:55,466][1157520] Avg episode reward: [(0, '56.723')] [2024-09-30 03:35:56,023][1157819] Updated weights for policy 0, policy_version 204998 (0.0006) [2024-09-30 03:35:56,539][1157819] Updated weights for policy 0, policy_version 205008 (0.0006) [2024-09-30 03:35:57,106][1157819] Updated weights for policy 0, policy_version 205018 (0.0006) [2024-09-30 03:35:57,659][1157819] Updated weights for policy 0, policy_version 205028 (0.0006) [2024-09-30 03:35:58,229][1157819] Updated weights for policy 0, policy_version 205038 (0.0006) [2024-09-30 03:35:58,757][1157819] Updated weights for policy 0, policy_version 205048 (0.0006) [2024-09-30 03:35:59,303][1157819] Updated weights for policy 0, policy_version 205058 (0.0006) [2024-09-30 03:35:59,888][1157819] Updated weights for policy 0, policy_version 205068 (0.0006) [2024-09-30 03:36:00,421][1157819] Updated weights for policy 0, policy_version 205078 (0.0006) [2024-09-30 03:36:00,466][1157520] Fps is (10 sec: 75777.2, 60 sec: 77005.0, 300 sec: 74616.6). Total num frames: 839999488. Throughput: 0: 19314.2. Samples: 199969648. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:36:00,466][1157520] Avg episode reward: [(0, '55.434')] [2024-09-30 03:36:00,951][1157819] Updated weights for policy 0, policy_version 205088 (0.0006) [2024-09-30 03:36:01,474][1157819] Updated weights for policy 0, policy_version 205098 (0.0006) [2024-09-30 03:36:02,029][1157819] Updated weights for policy 0, policy_version 205108 (0.0006) [2024-09-30 03:36:02,563][1157819] Updated weights for policy 0, policy_version 205118 (0.0006) [2024-09-30 03:36:03,119][1157819] Updated weights for policy 0, policy_version 205128 (0.0006) [2024-09-30 03:36:03,627][1157819] Updated weights for policy 0, policy_version 205138 (0.0006) [2024-09-30 03:36:04,144][1157819] Updated weights for policy 0, policy_version 205148 (0.0006) [2024-09-30 03:36:04,666][1157819] Updated weights for policy 0, policy_version 205158 (0.0006) [2024-09-30 03:36:05,197][1157819] Updated weights for policy 0, policy_version 205168 (0.0006) [2024-09-30 03:36:05,466][1157520] Fps is (10 sec: 75366.4, 60 sec: 77004.9, 300 sec: 74658.3). Total num frames: 840384512. Throughput: 0: 19271.4. Samples: 200084064. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:36:05,466][1157520] Avg episode reward: [(0, '55.234')] [2024-09-30 03:36:05,485][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000205173_840388608.pth... [2024-09-30 03:36:05,542][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000200772_822362112.pth [2024-09-30 03:36:05,720][1157819] Updated weights for policy 0, policy_version 205178 (0.0006) [2024-09-30 03:36:06,239][1157819] Updated weights for policy 0, policy_version 205188 (0.0006) [2024-09-30 03:36:06,801][1157819] Updated weights for policy 0, policy_version 205198 (0.0008) [2024-09-30 03:36:07,348][1157819] Updated weights for policy 0, policy_version 205208 (0.0007) [2024-09-30 03:36:07,916][1157819] Updated weights for policy 0, policy_version 205218 (0.0006) [2024-09-30 03:36:08,529][1157819] Updated weights for policy 0, policy_version 205228 (0.0007) [2024-09-30 03:36:09,100][1157819] Updated weights for policy 0, policy_version 205238 (0.0006) [2024-09-30 03:36:09,701][1157819] Updated weights for policy 0, policy_version 205248 (0.0007) [2024-09-30 03:36:10,337][1157819] Updated weights for policy 0, policy_version 205258 (0.0006) [2024-09-30 03:36:10,466][1157520] Fps is (10 sec: 74136.8, 60 sec: 76731.6, 300 sec: 74644.4). Total num frames: 840740864. Throughput: 0: 19273.5. Samples: 200141128. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:36:10,466][1157520] Avg episode reward: [(0, '55.940')] [2024-09-30 03:36:10,954][1157819] Updated weights for policy 0, policy_version 205268 (0.0006) [2024-09-30 03:36:11,570][1157819] Updated weights for policy 0, policy_version 205278 (0.0006) [2024-09-30 03:36:12,135][1157819] Updated weights for policy 0, policy_version 205288 (0.0006) [2024-09-30 03:36:12,713][1157819] Updated weights for policy 0, policy_version 205298 (0.0006) [2024-09-30 03:36:13,224][1157819] Updated weights for policy 0, policy_version 205308 (0.0006) [2024-09-30 03:36:13,769][1157819] Updated weights for policy 0, policy_version 205318 (0.0006) [2024-09-30 03:36:14,294][1157819] Updated weights for policy 0, policy_version 205328 (0.0006) [2024-09-30 03:36:14,843][1157819] Updated weights for policy 0, policy_version 205338 (0.0006) [2024-09-30 03:36:15,413][1157819] Updated weights for policy 0, policy_version 205348 (0.0006) [2024-09-30 03:36:15,466][1157520] Fps is (10 sec: 72498.3, 60 sec: 76390.4, 300 sec: 74727.7). Total num frames: 841109504. Throughput: 0: 19090.7. Samples: 200246148. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:36:15,466][1157520] Avg episode reward: [(0, '55.237')] [2024-09-30 03:36:15,941][1157819] Updated weights for policy 0, policy_version 205358 (0.0006) [2024-09-30 03:36:16,468][1157819] Updated weights for policy 0, policy_version 205368 (0.0006) [2024-09-30 03:36:16,967][1157819] Updated weights for policy 0, policy_version 205378 (0.0006) [2024-09-30 03:36:17,492][1157819] Updated weights for policy 0, policy_version 205388 (0.0006) [2024-09-30 03:36:18,052][1157819] Updated weights for policy 0, policy_version 205398 (0.0006) [2024-09-30 03:36:18,570][1157819] Updated weights for policy 0, policy_version 205408 (0.0006) [2024-09-30 03:36:19,070][1157819] Updated weights for policy 0, policy_version 205418 (0.0006) [2024-09-30 03:36:19,627][1157819] Updated weights for policy 0, policy_version 205428 (0.0006) [2024-09-30 03:36:20,146][1157819] Updated weights for policy 0, policy_version 205438 (0.0006) [2024-09-30 03:36:20,203][1157736] Signal inference workers to stop experience collection... (14300 times) [2024-09-30 03:36:20,204][1157736] Signal inference workers to resume experience collection... (14300 times) [2024-09-30 03:36:20,207][1157819] InferenceWorker_p0-w0: stopping experience collection (14300 times) [2024-09-30 03:36:20,210][1157819] InferenceWorker_p0-w0: resuming experience collection (14300 times) [2024-09-30 03:36:20,466][1157520] Fps is (10 sec: 75776.8, 60 sec: 76390.6, 300 sec: 74838.8). Total num frames: 841498624. Throughput: 0: 19158.6. Samples: 200361816. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:36:20,466][1157520] Avg episode reward: [(0, '57.487')] [2024-09-30 03:36:20,689][1157819] Updated weights for policy 0, policy_version 205448 (0.0006) [2024-09-30 03:36:21,261][1157819] Updated weights for policy 0, policy_version 205458 (0.0006) [2024-09-30 03:36:21,806][1157819] Updated weights for policy 0, policy_version 205468 (0.0006) [2024-09-30 03:36:22,400][1157819] Updated weights for policy 0, policy_version 205478 (0.0006) [2024-09-30 03:36:23,036][1157819] Updated weights for policy 0, policy_version 205488 (0.0006) [2024-09-30 03:36:23,616][1157819] Updated weights for policy 0, policy_version 205498 (0.0006) [2024-09-30 03:36:24,215][1157819] Updated weights for policy 0, policy_version 205508 (0.0006) [2024-09-30 03:36:24,793][1157819] Updated weights for policy 0, policy_version 205518 (0.0006) [2024-09-30 03:36:25,404][1157819] Updated weights for policy 0, policy_version 205528 (0.0006) [2024-09-30 03:36:25,466][1157520] Fps is (10 sec: 73728.1, 60 sec: 75844.2, 300 sec: 74741.6). Total num frames: 841846784. Throughput: 0: 19067.1. Samples: 200415412. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:36:25,466][1157520] Avg episode reward: [(0, '57.659')] [2024-09-30 03:36:25,962][1157819] Updated weights for policy 0, policy_version 205538 (0.0006) [2024-09-30 03:36:26,587][1157819] Updated weights for policy 0, policy_version 205548 (0.0006) [2024-09-30 03:36:27,161][1157819] Updated weights for policy 0, policy_version 205558 (0.0006) [2024-09-30 03:36:27,771][1157819] Updated weights for policy 0, policy_version 205568 (0.0006) [2024-09-30 03:36:28,325][1157819] Updated weights for policy 0, policy_version 205578 (0.0006) [2024-09-30 03:36:28,963][1157819] Updated weights for policy 0, policy_version 205588 (0.0006) [2024-09-30 03:36:29,500][1157819] Updated weights for policy 0, policy_version 205598 (0.0007) [2024-09-30 03:36:30,111][1157819] Updated weights for policy 0, policy_version 205608 (0.0007) [2024-09-30 03:36:30,466][1157520] Fps is (10 sec: 69631.2, 60 sec: 75366.4, 300 sec: 74644.4). Total num frames: 842194944. Throughput: 0: 18752.3. Samples: 200519724. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:36:30,466][1157520] Avg episode reward: [(0, '55.074')] [2024-09-30 03:36:30,708][1157819] Updated weights for policy 0, policy_version 205618 (0.0006) [2024-09-30 03:36:31,330][1157819] Updated weights for policy 0, policy_version 205628 (0.0006) [2024-09-30 03:36:31,894][1157819] Updated weights for policy 0, policy_version 205638 (0.0006) [2024-09-30 03:36:32,507][1157819] Updated weights for policy 0, policy_version 205648 (0.0006) [2024-09-30 03:36:33,122][1157819] Updated weights for policy 0, policy_version 205658 (0.0006) [2024-09-30 03:36:33,680][1157819] Updated weights for policy 0, policy_version 205668 (0.0006) [2024-09-30 03:36:34,261][1157819] Updated weights for policy 0, policy_version 205678 (0.0006) [2024-09-30 03:36:34,889][1157819] Updated weights for policy 0, policy_version 205688 (0.0006) [2024-09-30 03:36:35,436][1157819] Updated weights for policy 0, policy_version 205698 (0.0006) [2024-09-30 03:36:35,466][1157520] Fps is (10 sec: 69222.4, 60 sec: 74820.2, 300 sec: 74533.3). Total num frames: 842539008. Throughput: 0: 18355.1. Samples: 200623248. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:36:35,466][1157520] Avg episode reward: [(0, '54.578')] [2024-09-30 03:36:36,054][1157819] Updated weights for policy 0, policy_version 205708 (0.0006) [2024-09-30 03:36:36,605][1157819] Updated weights for policy 0, policy_version 205718 (0.0006) [2024-09-30 03:36:37,183][1157819] Updated weights for policy 0, policy_version 205728 (0.0006) [2024-09-30 03:36:37,806][1157819] Updated weights for policy 0, policy_version 205738 (0.0006) [2024-09-30 03:36:38,400][1157819] Updated weights for policy 0, policy_version 205748 (0.0006) [2024-09-30 03:36:38,983][1157819] Updated weights for policy 0, policy_version 205758 (0.0006) [2024-09-30 03:36:39,610][1157819] Updated weights for policy 0, policy_version 205768 (0.0006) [2024-09-30 03:36:40,161][1157819] Updated weights for policy 0, policy_version 205778 (0.0006) [2024-09-30 03:36:40,466][1157520] Fps is (10 sec: 69222.6, 60 sec: 74137.6, 300 sec: 74436.1). Total num frames: 842887168. Throughput: 0: 18200.9. Samples: 200675656. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:36:40,466][1157520] Avg episode reward: [(0, '55.354')] [2024-09-30 03:36:40,751][1157819] Updated weights for policy 0, policy_version 205788 (0.0006) [2024-09-30 03:36:41,347][1157819] Updated weights for policy 0, policy_version 205798 (0.0006) [2024-09-30 03:36:41,946][1157819] Updated weights for policy 0, policy_version 205808 (0.0006) [2024-09-30 03:36:42,539][1157819] Updated weights for policy 0, policy_version 205818 (0.0006) [2024-09-30 03:36:43,099][1157819] Updated weights for policy 0, policy_version 205828 (0.0006) [2024-09-30 03:36:43,724][1157819] Updated weights for policy 0, policy_version 205838 (0.0006) [2024-09-30 03:36:44,332][1157819] Updated weights for policy 0, policy_version 205848 (0.0006) [2024-09-30 03:36:44,871][1157819] Updated weights for policy 0, policy_version 205858 (0.0006) [2024-09-30 03:36:45,466][1157520] Fps is (10 sec: 69222.7, 60 sec: 73318.5, 300 sec: 74422.2). Total num frames: 843231232. Throughput: 0: 17989.9. Samples: 200779196. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:36:45,466][1157520] Avg episode reward: [(0, '55.638')] [2024-09-30 03:36:45,539][1157819] Updated weights for policy 0, policy_version 205868 (0.0006) [2024-09-30 03:36:46,124][1157819] Updated weights for policy 0, policy_version 205878 (0.0006) [2024-09-30 03:36:46,725][1157819] Updated weights for policy 0, policy_version 205888 (0.0006) [2024-09-30 03:36:47,297][1157819] Updated weights for policy 0, policy_version 205898 (0.0006) [2024-09-30 03:36:47,884][1157819] Updated weights for policy 0, policy_version 205908 (0.0006) [2024-09-30 03:36:48,472][1157819] Updated weights for policy 0, policy_version 205918 (0.0006) [2024-09-30 03:36:49,056][1157819] Updated weights for policy 0, policy_version 205928 (0.0006) [2024-09-30 03:36:49,624][1157819] Updated weights for policy 0, policy_version 205938 (0.0006) [2024-09-30 03:36:50,217][1157819] Updated weights for policy 0, policy_version 205948 (0.0006) [2024-09-30 03:36:50,466][1157520] Fps is (10 sec: 69222.6, 60 sec: 72294.5, 300 sec: 74408.4). Total num frames: 843579392. Throughput: 0: 17758.1. Samples: 200883180. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:36:50,466][1157520] Avg episode reward: [(0, '56.472')] [2024-09-30 03:36:50,848][1157819] Updated weights for policy 0, policy_version 205958 (0.0007) [2024-09-30 03:36:50,939][1157736] Signal inference workers to stop experience collection... (14350 times) [2024-09-30 03:36:50,939][1157736] Signal inference workers to resume experience collection... (14350 times) [2024-09-30 03:36:50,944][1157819] InferenceWorker_p0-w0: stopping experience collection (14350 times) [2024-09-30 03:36:50,944][1157819] InferenceWorker_p0-w0: resuming experience collection (14350 times) [2024-09-30 03:36:51,396][1157819] Updated weights for policy 0, policy_version 205968 (0.0006) [2024-09-30 03:36:51,954][1157819] Updated weights for policy 0, policy_version 205978 (0.0006) [2024-09-30 03:36:52,483][1157819] Updated weights for policy 0, policy_version 205988 (0.0006) [2024-09-30 03:36:53,029][1157819] Updated weights for policy 0, policy_version 205998 (0.0006) [2024-09-30 03:36:53,589][1157819] Updated weights for policy 0, policy_version 206008 (0.0006) [2024-09-30 03:36:54,138][1157819] Updated weights for policy 0, policy_version 206018 (0.0006) [2024-09-30 03:36:54,690][1157819] Updated weights for policy 0, policy_version 206028 (0.0006) [2024-09-30 03:36:55,230][1157819] Updated weights for policy 0, policy_version 206038 (0.0006) [2024-09-30 03:36:55,466][1157520] Fps is (10 sec: 71269.7, 60 sec: 71884.6, 300 sec: 74436.1). Total num frames: 843943936. Throughput: 0: 17691.5. Samples: 200937248. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 03:36:55,466][1157520] Avg episode reward: [(0, '54.081')] [2024-09-30 03:36:55,791][1157819] Updated weights for policy 0, policy_version 206048 (0.0006) [2024-09-30 03:36:56,334][1157819] Updated weights for policy 0, policy_version 206058 (0.0006) [2024-09-30 03:36:56,848][1157819] Updated weights for policy 0, policy_version 206068 (0.0006) [2024-09-30 03:36:57,434][1157819] Updated weights for policy 0, policy_version 206078 (0.0006) [2024-09-30 03:36:57,973][1157819] Updated weights for policy 0, policy_version 206088 (0.0006) [2024-09-30 03:36:58,542][1157819] Updated weights for policy 0, policy_version 206098 (0.0006) [2024-09-30 03:36:59,075][1157819] Updated weights for policy 0, policy_version 206108 (0.0006) [2024-09-30 03:36:59,610][1157819] Updated weights for policy 0, policy_version 206118 (0.0006) [2024-09-30 03:37:00,203][1157819] Updated weights for policy 0, policy_version 206128 (0.0006) [2024-09-30 03:37:00,466][1157520] Fps is (10 sec: 73727.7, 60 sec: 71952.9, 300 sec: 74436.1). Total num frames: 844316672. Throughput: 0: 17848.2. Samples: 201049316. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 03:37:00,466][1157520] Avg episode reward: [(0, '54.768')] [2024-09-30 03:37:00,768][1157819] Updated weights for policy 0, policy_version 206138 (0.0006) [2024-09-30 03:37:01,332][1157819] Updated weights for policy 0, policy_version 206148 (0.0006) [2024-09-30 03:37:01,934][1157819] Updated weights for policy 0, policy_version 206158 (0.0007) [2024-09-30 03:37:02,476][1157819] Updated weights for policy 0, policy_version 206168 (0.0006) [2024-09-30 03:37:03,070][1157819] Updated weights for policy 0, policy_version 206178 (0.0006) [2024-09-30 03:37:03,637][1157819] Updated weights for policy 0, policy_version 206188 (0.0006) [2024-09-30 03:37:04,213][1157819] Updated weights for policy 0, policy_version 206198 (0.0006) [2024-09-30 03:37:04,811][1157819] Updated weights for policy 0, policy_version 206208 (0.0006) [2024-09-30 03:37:05,420][1157819] Updated weights for policy 0, policy_version 206218 (0.0006) [2024-09-30 03:37:05,466][1157520] Fps is (10 sec: 72499.4, 60 sec: 71406.8, 300 sec: 74269.5). Total num frames: 844668928. Throughput: 0: 17661.2. Samples: 201156572. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 03:37:05,466][1157520] Avg episode reward: [(0, '57.060')] [2024-09-30 03:37:05,980][1157819] Updated weights for policy 0, policy_version 206228 (0.0006) [2024-09-30 03:37:06,593][1157819] Updated weights for policy 0, policy_version 206238 (0.0006) [2024-09-30 03:37:07,170][1157819] Updated weights for policy 0, policy_version 206248 (0.0006) [2024-09-30 03:37:07,770][1157819] Updated weights for policy 0, policy_version 206258 (0.0006) [2024-09-30 03:37:08,369][1157819] Updated weights for policy 0, policy_version 206268 (0.0006) [2024-09-30 03:37:08,948][1157819] Updated weights for policy 0, policy_version 206278 (0.0006) [2024-09-30 03:37:09,566][1157819] Updated weights for policy 0, policy_version 206288 (0.0006) [2024-09-30 03:37:10,154][1157819] Updated weights for policy 0, policy_version 206298 (0.0006) [2024-09-30 03:37:10,466][1157520] Fps is (10 sec: 69632.2, 60 sec: 71202.2, 300 sec: 74089.0). Total num frames: 845012992. Throughput: 0: 17620.1. Samples: 201208316. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 03:37:10,466][1157520] Avg episode reward: [(0, '55.366')] [2024-09-30 03:37:10,792][1157819] Updated weights for policy 0, policy_version 206308 (0.0006) [2024-09-30 03:37:11,346][1157819] Updated weights for policy 0, policy_version 206318 (0.0006) [2024-09-30 03:37:11,955][1157819] Updated weights for policy 0, policy_version 206328 (0.0006) [2024-09-30 03:37:12,529][1157819] Updated weights for policy 0, policy_version 206338 (0.0006) [2024-09-30 03:37:13,113][1157819] Updated weights for policy 0, policy_version 206348 (0.0006) [2024-09-30 03:37:13,724][1157819] Updated weights for policy 0, policy_version 206358 (0.0006) [2024-09-30 03:37:14,336][1157819] Updated weights for policy 0, policy_version 206368 (0.0006) [2024-09-30 03:37:14,893][1157819] Updated weights for policy 0, policy_version 206378 (0.0006) [2024-09-30 03:37:15,466][1157520] Fps is (10 sec: 69222.3, 60 sec: 70860.8, 300 sec: 73908.5). Total num frames: 845361152. Throughput: 0: 17596.5. Samples: 201311568. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 03:37:15,466][1157520] Avg episode reward: [(0, '56.855')] [2024-09-30 03:37:15,485][1157819] Updated weights for policy 0, policy_version 206388 (0.0006) [2024-09-30 03:37:16,073][1157819] Updated weights for policy 0, policy_version 206398 (0.0006) [2024-09-30 03:37:16,650][1157819] Updated weights for policy 0, policy_version 206408 (0.0006) [2024-09-30 03:37:17,260][1157819] Updated weights for policy 0, policy_version 206418 (0.0006) [2024-09-30 03:37:17,844][1157819] Updated weights for policy 0, policy_version 206428 (0.0006) [2024-09-30 03:37:18,443][1157819] Updated weights for policy 0, policy_version 206438 (0.0006) [2024-09-30 03:37:19,061][1157819] Updated weights for policy 0, policy_version 206448 (0.0007) [2024-09-30 03:37:19,630][1157819] Updated weights for policy 0, policy_version 206458 (0.0006) [2024-09-30 03:37:20,213][1157819] Updated weights for policy 0, policy_version 206468 (0.0006) [2024-09-30 03:37:20,466][1157520] Fps is (10 sec: 69632.3, 60 sec: 70178.1, 300 sec: 73769.7). Total num frames: 845709312. Throughput: 0: 17602.7. Samples: 201415368. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 03:37:20,466][1157520] Avg episode reward: [(0, '55.498')] [2024-09-30 03:37:20,848][1157819] Updated weights for policy 0, policy_version 206478 (0.0006) [2024-09-30 03:37:21,454][1157819] Updated weights for policy 0, policy_version 206488 (0.0006) [2024-09-30 03:37:22,055][1157819] Updated weights for policy 0, policy_version 206498 (0.0006) [2024-09-30 03:37:22,629][1157819] Updated weights for policy 0, policy_version 206508 (0.0006) [2024-09-30 03:37:23,190][1157819] Updated weights for policy 0, policy_version 206518 (0.0006) [2024-09-30 03:37:23,746][1157819] Updated weights for policy 0, policy_version 206528 (0.0006) [2024-09-30 03:37:24,316][1157819] Updated weights for policy 0, policy_version 206538 (0.0006) [2024-09-30 03:37:24,498][1157736] Signal inference workers to stop experience collection... (14400 times) [2024-09-30 03:37:24,498][1157736] Signal inference workers to resume experience collection... (14400 times) [2024-09-30 03:37:24,502][1157819] InferenceWorker_p0-w0: stopping experience collection (14400 times) [2024-09-30 03:37:24,504][1157819] InferenceWorker_p0-w0: resuming experience collection (14400 times) [2024-09-30 03:37:24,881][1157819] Updated weights for policy 0, policy_version 206548 (0.0006) [2024-09-30 03:37:25,439][1157819] Updated weights for policy 0, policy_version 206558 (0.0006) [2024-09-30 03:37:25,466][1157520] Fps is (10 sec: 70041.6, 60 sec: 70246.4, 300 sec: 73644.6). Total num frames: 846061568. Throughput: 0: 17586.6. Samples: 201467052. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 03:37:25,466][1157520] Avg episode reward: [(0, '54.876')] [2024-09-30 03:37:25,992][1157819] Updated weights for policy 0, policy_version 206568 (0.0006) [2024-09-30 03:37:26,548][1157819] Updated weights for policy 0, policy_version 206578 (0.0006) [2024-09-30 03:37:27,109][1157819] Updated weights for policy 0, policy_version 206588 (0.0006) [2024-09-30 03:37:27,662][1157819] Updated weights for policy 0, policy_version 206598 (0.0006) [2024-09-30 03:37:28,223][1157819] Updated weights for policy 0, policy_version 206608 (0.0006) [2024-09-30 03:37:28,774][1157819] Updated weights for policy 0, policy_version 206618 (0.0006) [2024-09-30 03:37:29,357][1157819] Updated weights for policy 0, policy_version 206628 (0.0006) [2024-09-30 03:37:29,901][1157819] Updated weights for policy 0, policy_version 206638 (0.0006) [2024-09-30 03:37:30,466][1157520] Fps is (10 sec: 71679.5, 60 sec: 70519.5, 300 sec: 73575.3). Total num frames: 846426112. Throughput: 0: 17722.8. Samples: 201576724. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 03:37:30,466][1157520] Avg episode reward: [(0, '56.399')] [2024-09-30 03:37:30,482][1157819] Updated weights for policy 0, policy_version 206648 (0.0006) [2024-09-30 03:37:30,991][1157819] Updated weights for policy 0, policy_version 206658 (0.0006) [2024-09-30 03:37:31,488][1157819] Updated weights for policy 0, policy_version 206668 (0.0006) [2024-09-30 03:37:32,022][1157819] Updated weights for policy 0, policy_version 206678 (0.0006) [2024-09-30 03:37:32,579][1157819] Updated weights for policy 0, policy_version 206688 (0.0006) [2024-09-30 03:37:33,075][1157819] Updated weights for policy 0, policy_version 206698 (0.0006) [2024-09-30 03:37:33,580][1157819] Updated weights for policy 0, policy_version 206708 (0.0006) [2024-09-30 03:37:34,085][1157819] Updated weights for policy 0, policy_version 206718 (0.0006) [2024-09-30 03:37:34,621][1157819] Updated weights for policy 0, policy_version 206728 (0.0006) [2024-09-30 03:37:35,157][1157819] Updated weights for policy 0, policy_version 206738 (0.0006) [2024-09-30 03:37:35,466][1157520] Fps is (10 sec: 76186.8, 60 sec: 71407.1, 300 sec: 73616.9). Total num frames: 846823424. Throughput: 0: 17982.3. Samples: 201692380. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 03:37:35,466][1157520] Avg episode reward: [(0, '56.686')] [2024-09-30 03:37:35,677][1157819] Updated weights for policy 0, policy_version 206748 (0.0006) [2024-09-30 03:37:36,242][1157819] Updated weights for policy 0, policy_version 206758 (0.0006) [2024-09-30 03:37:36,776][1157819] Updated weights for policy 0, policy_version 206768 (0.0006) [2024-09-30 03:37:37,344][1157819] Updated weights for policy 0, policy_version 206778 (0.0006) [2024-09-30 03:37:37,920][1157819] Updated weights for policy 0, policy_version 206788 (0.0006) [2024-09-30 03:37:38,459][1157819] Updated weights for policy 0, policy_version 206798 (0.0006) [2024-09-30 03:37:39,046][1157819] Updated weights for policy 0, policy_version 206808 (0.0006) [2024-09-30 03:37:39,567][1157819] Updated weights for policy 0, policy_version 206818 (0.0006) [2024-09-30 03:37:40,121][1157819] Updated weights for policy 0, policy_version 206828 (0.0006) [2024-09-30 03:37:40,466][1157520] Fps is (10 sec: 76596.2, 60 sec: 71748.4, 300 sec: 73630.8). Total num frames: 847192064. Throughput: 0: 18031.2. Samples: 201748648. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 03:37:40,466][1157520] Avg episode reward: [(0, '56.358')] [2024-09-30 03:37:40,694][1157819] Updated weights for policy 0, policy_version 206838 (0.0006) [2024-09-30 03:37:41,207][1157819] Updated weights for policy 0, policy_version 206848 (0.0006) [2024-09-30 03:37:41,770][1157819] Updated weights for policy 0, policy_version 206858 (0.0006) [2024-09-30 03:37:42,339][1157819] Updated weights for policy 0, policy_version 206868 (0.0006) [2024-09-30 03:37:42,883][1157819] Updated weights for policy 0, policy_version 206878 (0.0006) [2024-09-30 03:37:43,459][1157819] Updated weights for policy 0, policy_version 206888 (0.0006) [2024-09-30 03:37:44,011][1157819] Updated weights for policy 0, policy_version 206898 (0.0006) [2024-09-30 03:37:44,560][1157819] Updated weights for policy 0, policy_version 206908 (0.0006) [2024-09-30 03:37:45,131][1157819] Updated weights for policy 0, policy_version 206918 (0.0006) [2024-09-30 03:37:45,466][1157520] Fps is (10 sec: 73318.0, 60 sec: 72089.6, 300 sec: 73616.9). Total num frames: 847556608. Throughput: 0: 18003.7. Samples: 201859480. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 03:37:45,466][1157520] Avg episode reward: [(0, '55.587')] [2024-09-30 03:37:45,675][1157819] Updated weights for policy 0, policy_version 206928 (0.0006) [2024-09-30 03:37:46,242][1157819] Updated weights for policy 0, policy_version 206938 (0.0006) [2024-09-30 03:37:46,809][1157819] Updated weights for policy 0, policy_version 206948 (0.0006) [2024-09-30 03:37:47,376][1157819] Updated weights for policy 0, policy_version 206958 (0.0006) [2024-09-30 03:37:47,959][1157819] Updated weights for policy 0, policy_version 206968 (0.0006) [2024-09-30 03:37:48,487][1157819] Updated weights for policy 0, policy_version 206978 (0.0006) [2024-09-30 03:37:49,102][1157819] Updated weights for policy 0, policy_version 206988 (0.0006) [2024-09-30 03:37:49,677][1157819] Updated weights for policy 0, policy_version 206998 (0.0006) [2024-09-30 03:37:50,268][1157819] Updated weights for policy 0, policy_version 207008 (0.0006) [2024-09-30 03:37:50,466][1157520] Fps is (10 sec: 72088.7, 60 sec: 72226.1, 300 sec: 73547.5). Total num frames: 847912960. Throughput: 0: 18026.0. Samples: 201967740. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 03:37:50,466][1157520] Avg episode reward: [(0, '55.459')] [2024-09-30 03:37:50,878][1157819] Updated weights for policy 0, policy_version 207018 (0.0006) [2024-09-30 03:37:51,455][1157819] Updated weights for policy 0, policy_version 207028 (0.0006) [2024-09-30 03:37:52,045][1157819] Updated weights for policy 0, policy_version 207038 (0.0006) [2024-09-30 03:37:52,649][1157819] Updated weights for policy 0, policy_version 207048 (0.0006) [2024-09-30 03:37:53,240][1157819] Updated weights for policy 0, policy_version 207058 (0.0006) [2024-09-30 03:37:53,837][1157819] Updated weights for policy 0, policy_version 207068 (0.0006) [2024-09-30 03:37:54,427][1157819] Updated weights for policy 0, policy_version 207078 (0.0006) [2024-09-30 03:37:55,033][1157819] Updated weights for policy 0, policy_version 207088 (0.0006) [2024-09-30 03:37:55,466][1157520] Fps is (10 sec: 70450.7, 60 sec: 71953.1, 300 sec: 73478.1). Total num frames: 848261120. Throughput: 0: 18022.1. Samples: 202019312. Policy #0 lag: (min: 0.0, avg: 2.8, max: 6.0) [2024-09-30 03:37:55,466][1157520] Avg episode reward: [(0, '55.252')] [2024-09-30 03:37:55,645][1157819] Updated weights for policy 0, policy_version 207098 (0.0006) [2024-09-30 03:37:56,202][1157819] Updated weights for policy 0, policy_version 207108 (0.0006) [2024-09-30 03:37:56,843][1157819] Updated weights for policy 0, policy_version 207118 (0.0006) [2024-09-30 03:37:57,408][1157819] Updated weights for policy 0, policy_version 207128 (0.0006) [2024-09-30 03:37:58,012][1157819] Updated weights for policy 0, policy_version 207138 (0.0006) [2024-09-30 03:37:58,619][1157819] Updated weights for policy 0, policy_version 207148 (0.0006) [2024-09-30 03:37:59,230][1157819] Updated weights for policy 0, policy_version 207158 (0.0006) [2024-09-30 03:37:59,853][1157819] Updated weights for policy 0, policy_version 207168 (0.0006) [2024-09-30 03:38:00,466][1157520] Fps is (10 sec: 68403.0, 60 sec: 71338.6, 300 sec: 73367.0). Total num frames: 848596992. Throughput: 0: 18020.2. Samples: 202122476. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 03:38:00,466][1157520] Avg episode reward: [(0, '55.693')] [2024-09-30 03:38:00,488][1157819] Updated weights for policy 0, policy_version 207178 (0.0006) [2024-09-30 03:38:01,098][1157819] Updated weights for policy 0, policy_version 207188 (0.0006) [2024-09-30 03:38:01,752][1157819] Updated weights for policy 0, policy_version 207198 (0.0006) [2024-09-30 03:38:02,398][1157819] Updated weights for policy 0, policy_version 207208 (0.0006) [2024-09-30 03:38:03,080][1157819] Updated weights for policy 0, policy_version 207218 (0.0006) [2024-09-30 03:38:03,724][1157819] Updated weights for policy 0, policy_version 207228 (0.0006) [2024-09-30 03:38:04,356][1157819] Updated weights for policy 0, policy_version 207238 (0.0006) [2024-09-30 03:38:04,979][1157819] Updated weights for policy 0, policy_version 207248 (0.0006) [2024-09-30 03:38:05,466][1157520] Fps is (10 sec: 65535.2, 60 sec: 70792.4, 300 sec: 73283.6). Total num frames: 848916480. Throughput: 0: 17849.0. Samples: 202218576. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 03:38:05,466][1157520] Avg episode reward: [(0, '51.989')] [2024-09-30 03:38:05,472][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000207255_848916480.pth... [2024-09-30 03:38:05,534][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000202942_831250432.pth [2024-09-30 03:38:05,630][1157819] Updated weights for policy 0, policy_version 207258 (0.0006) [2024-09-30 03:38:06,267][1157819] Updated weights for policy 0, policy_version 207268 (0.0006) [2024-09-30 03:38:06,898][1157819] Updated weights for policy 0, policy_version 207278 (0.0006) [2024-09-30 03:38:07,542][1157819] Updated weights for policy 0, policy_version 207288 (0.0006) [2024-09-30 03:38:08,145][1157819] Updated weights for policy 0, policy_version 207298 (0.0006) [2024-09-30 03:38:08,796][1157819] Updated weights for policy 0, policy_version 207308 (0.0006) [2024-09-30 03:38:09,402][1157819] Updated weights for policy 0, policy_version 207318 (0.0006) [2024-09-30 03:38:10,021][1157819] Updated weights for policy 0, policy_version 207328 (0.0006) [2024-09-30 03:38:10,466][1157520] Fps is (10 sec: 64307.3, 60 sec: 70451.1, 300 sec: 73228.2). Total num frames: 849240064. Throughput: 0: 17782.1. Samples: 202267248. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 03:38:10,466][1157520] Avg episode reward: [(0, '55.980')] [2024-09-30 03:38:10,704][1157819] Updated weights for policy 0, policy_version 207338 (0.0006) [2024-09-30 03:38:11,305][1157819] Updated weights for policy 0, policy_version 207348 (0.0006) [2024-09-30 03:38:11,385][1157736] Signal inference workers to stop experience collection... (14450 times) [2024-09-30 03:38:11,387][1157736] Signal inference workers to resume experience collection... (14450 times) [2024-09-30 03:38:11,390][1157819] InferenceWorker_p0-w0: stopping experience collection (14450 times) [2024-09-30 03:38:11,392][1157819] InferenceWorker_p0-w0: resuming experience collection (14450 times) [2024-09-30 03:38:11,917][1157819] Updated weights for policy 0, policy_version 207358 (0.0006) [2024-09-30 03:38:12,530][1157819] Updated weights for policy 0, policy_version 207368 (0.0006) [2024-09-30 03:38:13,140][1157819] Updated weights for policy 0, policy_version 207378 (0.0006) [2024-09-30 03:38:13,711][1157819] Updated weights for policy 0, policy_version 207388 (0.0006) [2024-09-30 03:38:14,311][1157819] Updated weights for policy 0, policy_version 207398 (0.0006) [2024-09-30 03:38:14,926][1157819] Updated weights for policy 0, policy_version 207408 (0.0006) [2024-09-30 03:38:15,466][1157520] Fps is (10 sec: 66355.6, 60 sec: 70314.6, 300 sec: 73200.4). Total num frames: 849580032. Throughput: 0: 17552.1. Samples: 202366568. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 03:38:15,466][1157520] Avg episode reward: [(0, '55.385')] [2024-09-30 03:38:15,546][1157819] Updated weights for policy 0, policy_version 207418 (0.0006) [2024-09-30 03:38:16,134][1157819] Updated weights for policy 0, policy_version 207428 (0.0006) [2024-09-30 03:38:16,735][1157819] Updated weights for policy 0, policy_version 207438 (0.0006) [2024-09-30 03:38:17,323][1157819] Updated weights for policy 0, policy_version 207448 (0.0006) [2024-09-30 03:38:17,911][1157819] Updated weights for policy 0, policy_version 207458 (0.0006) [2024-09-30 03:38:18,510][1157819] Updated weights for policy 0, policy_version 207468 (0.0006) [2024-09-30 03:38:19,131][1157819] Updated weights for policy 0, policy_version 207478 (0.0006) [2024-09-30 03:38:19,690][1157819] Updated weights for policy 0, policy_version 207488 (0.0006) [2024-09-30 03:38:20,336][1157819] Updated weights for policy 0, policy_version 207498 (0.0006) [2024-09-30 03:38:20,466][1157520] Fps is (10 sec: 67993.4, 60 sec: 70178.0, 300 sec: 73214.3). Total num frames: 849920000. Throughput: 0: 17250.0. Samples: 202468632. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 03:38:20,466][1157520] Avg episode reward: [(0, '56.008')] [2024-09-30 03:38:20,952][1157819] Updated weights for policy 0, policy_version 207508 (0.0006) [2024-09-30 03:38:21,527][1157819] Updated weights for policy 0, policy_version 207518 (0.0006) [2024-09-30 03:38:22,139][1157819] Updated weights for policy 0, policy_version 207528 (0.0006) [2024-09-30 03:38:22,715][1157819] Updated weights for policy 0, policy_version 207538 (0.0006) [2024-09-30 03:38:23,328][1157819] Updated weights for policy 0, policy_version 207548 (0.0006) [2024-09-30 03:38:23,949][1157819] Updated weights for policy 0, policy_version 207558 (0.0006) [2024-09-30 03:38:24,595][1157819] Updated weights for policy 0, policy_version 207568 (0.0006) [2024-09-30 03:38:25,260][1157819] Updated weights for policy 0, policy_version 207578 (0.0006) [2024-09-30 03:38:25,466][1157520] Fps is (10 sec: 67174.4, 60 sec: 69836.8, 300 sec: 73255.9). Total num frames: 850251776. Throughput: 0: 17147.7. Samples: 202520300. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 03:38:25,466][1157520] Avg episode reward: [(0, '56.812')] [2024-09-30 03:38:25,904][1157819] Updated weights for policy 0, policy_version 207588 (0.0006) [2024-09-30 03:38:26,545][1157819] Updated weights for policy 0, policy_version 207598 (0.0006) [2024-09-30 03:38:27,196][1157819] Updated weights for policy 0, policy_version 207608 (0.0006) [2024-09-30 03:38:27,869][1157819] Updated weights for policy 0, policy_version 207618 (0.0006) [2024-09-30 03:38:28,523][1157819] Updated weights for policy 0, policy_version 207628 (0.0006) [2024-09-30 03:38:29,197][1157819] Updated weights for policy 0, policy_version 207638 (0.0006) [2024-09-30 03:38:29,800][1157819] Updated weights for policy 0, policy_version 207648 (0.0006) [2024-09-30 03:38:30,466][1157520] Fps is (10 sec: 64307.4, 60 sec: 68949.3, 300 sec: 73172.6). Total num frames: 850563072. Throughput: 0: 16798.3. Samples: 202615404. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 03:38:30,466][1157520] Avg episode reward: [(0, '57.043')] [2024-09-30 03:38:30,478][1157819] Updated weights for policy 0, policy_version 207658 (0.0006) [2024-09-30 03:38:31,118][1157819] Updated weights for policy 0, policy_version 207668 (0.0006) [2024-09-30 03:38:31,795][1157819] Updated weights for policy 0, policy_version 207678 (0.0006) [2024-09-30 03:38:32,401][1157819] Updated weights for policy 0, policy_version 207688 (0.0006) [2024-09-30 03:38:33,040][1157819] Updated weights for policy 0, policy_version 207698 (0.0006) [2024-09-30 03:38:33,669][1157819] Updated weights for policy 0, policy_version 207708 (0.0006) [2024-09-30 03:38:34,345][1157819] Updated weights for policy 0, policy_version 207718 (0.0006) [2024-09-30 03:38:34,959][1157819] Updated weights for policy 0, policy_version 207728 (0.0006) [2024-09-30 03:38:35,466][1157520] Fps is (10 sec: 63487.7, 60 sec: 67720.3, 300 sec: 73103.1). Total num frames: 850886656. Throughput: 0: 16510.7. Samples: 202710724. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 03:38:35,466][1157520] Avg episode reward: [(0, '56.128')] [2024-09-30 03:38:35,559][1157819] Updated weights for policy 0, policy_version 207738 (0.0006) [2024-09-30 03:38:36,179][1157819] Updated weights for policy 0, policy_version 207748 (0.0007) [2024-09-30 03:38:36,837][1157819] Updated weights for policy 0, policy_version 207758 (0.0006) [2024-09-30 03:38:37,449][1157819] Updated weights for policy 0, policy_version 207768 (0.0006) [2024-09-30 03:38:38,080][1157819] Updated weights for policy 0, policy_version 207778 (0.0006) [2024-09-30 03:38:38,661][1157819] Updated weights for policy 0, policy_version 207788 (0.0006) [2024-09-30 03:38:39,277][1157819] Updated weights for policy 0, policy_version 207798 (0.0006) [2024-09-30 03:38:39,893][1157819] Updated weights for policy 0, policy_version 207808 (0.0006) [2024-09-30 03:38:40,466][1157520] Fps is (10 sec: 65536.3, 60 sec: 67106.0, 300 sec: 72992.1). Total num frames: 851218432. Throughput: 0: 16465.2. Samples: 202760244. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 03:38:40,466][1157520] Avg episode reward: [(0, '55.990')] [2024-09-30 03:38:40,514][1157819] Updated weights for policy 0, policy_version 207818 (0.0006) [2024-09-30 03:38:41,111][1157819] Updated weights for policy 0, policy_version 207828 (0.0006) [2024-09-30 03:38:41,704][1157819] Updated weights for policy 0, policy_version 207838 (0.0006) [2024-09-30 03:38:42,298][1157819] Updated weights for policy 0, policy_version 207848 (0.0006) [2024-09-30 03:38:42,869][1157819] Updated weights for policy 0, policy_version 207858 (0.0006) [2024-09-30 03:38:43,486][1157819] Updated weights for policy 0, policy_version 207868 (0.0006) [2024-09-30 03:38:44,050][1157819] Updated weights for policy 0, policy_version 207878 (0.0006) [2024-09-30 03:38:44,649][1157819] Updated weights for policy 0, policy_version 207888 (0.0006) [2024-09-30 03:38:45,239][1157819] Updated weights for policy 0, policy_version 207898 (0.0006) [2024-09-30 03:38:45,303][1157736] Signal inference workers to stop experience collection... (14500 times) [2024-09-30 03:38:45,305][1157736] Signal inference workers to resume experience collection... (14500 times) [2024-09-30 03:38:45,305][1157819] InferenceWorker_p0-w0: stopping experience collection (14500 times) [2024-09-30 03:38:45,309][1157819] InferenceWorker_p0-w0: resuming experience collection (14500 times) [2024-09-30 03:38:45,466][1157520] Fps is (10 sec: 67584.8, 60 sec: 66764.7, 300 sec: 72881.0). Total num frames: 851562496. Throughput: 0: 16443.8. Samples: 202862448. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 03:38:45,466][1157520] Avg episode reward: [(0, '56.130')] [2024-09-30 03:38:45,813][1157819] Updated weights for policy 0, policy_version 207908 (0.0006) [2024-09-30 03:38:46,373][1157819] Updated weights for policy 0, policy_version 207918 (0.0006) [2024-09-30 03:38:46,933][1157819] Updated weights for policy 0, policy_version 207928 (0.0006) [2024-09-30 03:38:47,474][1157819] Updated weights for policy 0, policy_version 207938 (0.0006) [2024-09-30 03:38:48,041][1157819] Updated weights for policy 0, policy_version 207948 (0.0006) [2024-09-30 03:38:48,595][1157819] Updated weights for policy 0, policy_version 207958 (0.0006) [2024-09-30 03:38:49,150][1157819] Updated weights for policy 0, policy_version 207968 (0.0006) [2024-09-30 03:38:49,699][1157819] Updated weights for policy 0, policy_version 207978 (0.0006) [2024-09-30 03:38:50,255][1157819] Updated weights for policy 0, policy_version 207988 (0.0006) [2024-09-30 03:38:50,466][1157520] Fps is (10 sec: 71680.5, 60 sec: 67038.0, 300 sec: 72797.8). Total num frames: 851935232. Throughput: 0: 16735.5. Samples: 202971668. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 03:38:50,466][1157520] Avg episode reward: [(0, '52.981')] [2024-09-30 03:38:50,776][1157819] Updated weights for policy 0, policy_version 207998 (0.0006) [2024-09-30 03:38:51,328][1157819] Updated weights for policy 0, policy_version 208008 (0.0006) [2024-09-30 03:38:51,941][1157819] Updated weights for policy 0, policy_version 208018 (0.0006) [2024-09-30 03:38:52,506][1157819] Updated weights for policy 0, policy_version 208028 (0.0006) [2024-09-30 03:38:53,071][1157819] Updated weights for policy 0, policy_version 208038 (0.0006) [2024-09-30 03:38:53,648][1157819] Updated weights for policy 0, policy_version 208048 (0.0006) [2024-09-30 03:38:54,211][1157819] Updated weights for policy 0, policy_version 208058 (0.0006) [2024-09-30 03:38:54,777][1157819] Updated weights for policy 0, policy_version 208068 (0.0006) [2024-09-30 03:38:55,410][1157819] Updated weights for policy 0, policy_version 208078 (0.0006) [2024-09-30 03:38:55,466][1157520] Fps is (10 sec: 72499.8, 60 sec: 67106.3, 300 sec: 72658.9). Total num frames: 852287488. Throughput: 0: 16858.5. Samples: 203025880. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 03:38:55,466][1157520] Avg episode reward: [(0, '53.142')] [2024-09-30 03:38:56,013][1157819] Updated weights for policy 0, policy_version 208088 (0.0006) [2024-09-30 03:38:56,609][1157819] Updated weights for policy 0, policy_version 208098 (0.0006) [2024-09-30 03:38:57,217][1157819] Updated weights for policy 0, policy_version 208108 (0.0006) [2024-09-30 03:38:57,780][1157819] Updated weights for policy 0, policy_version 208118 (0.0006) [2024-09-30 03:38:58,364][1157819] Updated weights for policy 0, policy_version 208128 (0.0006) [2024-09-30 03:38:58,908][1157819] Updated weights for policy 0, policy_version 208138 (0.0006) [2024-09-30 03:38:59,502][1157819] Updated weights for policy 0, policy_version 208148 (0.0006) [2024-09-30 03:39:00,037][1157819] Updated weights for policy 0, policy_version 208158 (0.0006) [2024-09-30 03:39:00,466][1157520] Fps is (10 sec: 70861.0, 60 sec: 67447.6, 300 sec: 72534.0). Total num frames: 852643840. Throughput: 0: 16983.5. Samples: 203130820. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 03:39:00,466][1157520] Avg episode reward: [(0, '55.776')] [2024-09-30 03:39:00,613][1157819] Updated weights for policy 0, policy_version 208168 (0.0006) [2024-09-30 03:39:01,134][1157819] Updated weights for policy 0, policy_version 208178 (0.0006) [2024-09-30 03:39:01,699][1157819] Updated weights for policy 0, policy_version 208188 (0.0006) [2024-09-30 03:39:02,228][1157819] Updated weights for policy 0, policy_version 208198 (0.0006) [2024-09-30 03:39:02,749][1157819] Updated weights for policy 0, policy_version 208208 (0.0006) [2024-09-30 03:39:03,261][1157819] Updated weights for policy 0, policy_version 208218 (0.0006) [2024-09-30 03:39:03,822][1157819] Updated weights for policy 0, policy_version 208228 (0.0006) [2024-09-30 03:39:04,349][1157819] Updated weights for policy 0, policy_version 208238 (0.0006) [2024-09-30 03:39:04,867][1157819] Updated weights for policy 0, policy_version 208248 (0.0006) [2024-09-30 03:39:05,370][1157819] Updated weights for policy 0, policy_version 208258 (0.0006) [2024-09-30 03:39:05,466][1157520] Fps is (10 sec: 74137.6, 60 sec: 68540.0, 300 sec: 72478.4). Total num frames: 853028864. Throughput: 0: 17246.5. Samples: 203244720. Policy #0 lag: (min: 0.0, avg: 3.1, max: 6.0) [2024-09-30 03:39:05,466][1157520] Avg episode reward: [(0, '55.301')] [2024-09-30 03:39:05,893][1157819] Updated weights for policy 0, policy_version 208268 (0.0006) [2024-09-30 03:39:06,414][1157819] Updated weights for policy 0, policy_version 208278 (0.0006) [2024-09-30 03:39:06,916][1157819] Updated weights for policy 0, policy_version 208288 (0.0006) [2024-09-30 03:39:07,478][1157819] Updated weights for policy 0, policy_version 208298 (0.0006) [2024-09-30 03:39:08,018][1157819] Updated weights for policy 0, policy_version 208308 (0.0007) [2024-09-30 03:39:08,550][1157819] Updated weights for policy 0, policy_version 208318 (0.0006) [2024-09-30 03:39:09,102][1157819] Updated weights for policy 0, policy_version 208328 (0.0007) [2024-09-30 03:39:09,639][1157819] Updated weights for policy 0, policy_version 208338 (0.0006) [2024-09-30 03:39:10,206][1157819] Updated weights for policy 0, policy_version 208348 (0.0006) [2024-09-30 03:39:10,466][1157520] Fps is (10 sec: 76594.5, 60 sec: 69495.5, 300 sec: 72381.2). Total num frames: 853409792. Throughput: 0: 17401.9. Samples: 203303384. Policy #0 lag: (min: 0.0, avg: 3.1, max: 6.0) [2024-09-30 03:39:10,466][1157520] Avg episode reward: [(0, '54.344')] [2024-09-30 03:39:10,773][1157819] Updated weights for policy 0, policy_version 208358 (0.0006) [2024-09-30 03:39:11,328][1157819] Updated weights for policy 0, policy_version 208368 (0.0006) [2024-09-30 03:39:11,874][1157819] Updated weights for policy 0, policy_version 208378 (0.0006) [2024-09-30 03:39:12,412][1157819] Updated weights for policy 0, policy_version 208388 (0.0006) [2024-09-30 03:39:12,789][1157736] Signal inference workers to stop experience collection... (14550 times) [2024-09-30 03:39:12,789][1157736] Signal inference workers to resume experience collection... (14550 times) [2024-09-30 03:39:12,794][1157819] InferenceWorker_p0-w0: stopping experience collection (14550 times) [2024-09-30 03:39:12,794][1157819] InferenceWorker_p0-w0: resuming experience collection (14550 times) [2024-09-30 03:39:12,962][1157819] Updated weights for policy 0, policy_version 208398 (0.0006) [2024-09-30 03:39:13,516][1157819] Updated weights for policy 0, policy_version 208408 (0.0006) [2024-09-30 03:39:14,045][1157819] Updated weights for policy 0, policy_version 208418 (0.0006) [2024-09-30 03:39:14,590][1157819] Updated weights for policy 0, policy_version 208428 (0.0006) [2024-09-30 03:39:15,151][1157819] Updated weights for policy 0, policy_version 208438 (0.0006) [2024-09-30 03:39:15,466][1157520] Fps is (10 sec: 75365.7, 60 sec: 70041.7, 300 sec: 72325.6). Total num frames: 853782528. Throughput: 0: 17776.4. Samples: 203415344. Policy #0 lag: (min: 0.0, avg: 3.1, max: 6.0) [2024-09-30 03:39:15,466][1157520] Avg episode reward: [(0, '56.420')] [2024-09-30 03:39:15,693][1157819] Updated weights for policy 0, policy_version 208448 (0.0006) [2024-09-30 03:39:16,246][1157819] Updated weights for policy 0, policy_version 208458 (0.0006) [2024-09-30 03:39:16,799][1157819] Updated weights for policy 0, policy_version 208468 (0.0006) [2024-09-30 03:39:17,350][1157819] Updated weights for policy 0, policy_version 208478 (0.0006) [2024-09-30 03:39:17,944][1157819] Updated weights for policy 0, policy_version 208488 (0.0006) [2024-09-30 03:39:18,518][1157819] Updated weights for policy 0, policy_version 208498 (0.0006) [2024-09-30 03:39:19,108][1157819] Updated weights for policy 0, policy_version 208508 (0.0006) [2024-09-30 03:39:19,679][1157819] Updated weights for policy 0, policy_version 208518 (0.0006) [2024-09-30 03:39:20,264][1157819] Updated weights for policy 0, policy_version 208528 (0.0006) [2024-09-30 03:39:20,466][1157520] Fps is (10 sec: 73318.3, 60 sec: 70383.0, 300 sec: 72339.5). Total num frames: 854142976. Throughput: 0: 18081.9. Samples: 203524408. Policy #0 lag: (min: 0.0, avg: 3.1, max: 6.0) [2024-09-30 03:39:20,466][1157520] Avg episode reward: [(0, '53.889')] [2024-09-30 03:39:20,833][1157819] Updated weights for policy 0, policy_version 208538 (0.0006) [2024-09-30 03:39:21,398][1157819] Updated weights for policy 0, policy_version 208548 (0.0006) [2024-09-30 03:39:21,984][1157819] Updated weights for policy 0, policy_version 208558 (0.0006) [2024-09-30 03:39:22,560][1157819] Updated weights for policy 0, policy_version 208568 (0.0006) [2024-09-30 03:39:23,144][1157819] Updated weights for policy 0, policy_version 208578 (0.0006) [2024-09-30 03:39:23,675][1157819] Updated weights for policy 0, policy_version 208588 (0.0006) [2024-09-30 03:39:24,271][1157819] Updated weights for policy 0, policy_version 208598 (0.0006) [2024-09-30 03:39:24,785][1157819] Updated weights for policy 0, policy_version 208608 (0.0006) [2024-09-30 03:39:25,314][1157819] Updated weights for policy 0, policy_version 208618 (0.0006) [2024-09-30 03:39:25,466][1157520] Fps is (10 sec: 72499.1, 60 sec: 70929.1, 300 sec: 72395.1). Total num frames: 854507520. Throughput: 0: 18161.1. Samples: 203577496. Policy #0 lag: (min: 0.0, avg: 3.1, max: 6.0) [2024-09-30 03:39:25,466][1157520] Avg episode reward: [(0, '58.263')] [2024-09-30 03:39:25,887][1157819] Updated weights for policy 0, policy_version 208628 (0.0006) [2024-09-30 03:39:26,420][1157819] Updated weights for policy 0, policy_version 208638 (0.0006) [2024-09-30 03:39:26,963][1157819] Updated weights for policy 0, policy_version 208648 (0.0006) [2024-09-30 03:39:27,515][1157819] Updated weights for policy 0, policy_version 208658 (0.0006) [2024-09-30 03:39:28,068][1157819] Updated weights for policy 0, policy_version 208668 (0.0006) [2024-09-30 03:39:28,620][1157819] Updated weights for policy 0, policy_version 208678 (0.0006) [2024-09-30 03:39:29,149][1157819] Updated weights for policy 0, policy_version 208688 (0.0006) [2024-09-30 03:39:29,693][1157819] Updated weights for policy 0, policy_version 208698 (0.0006) [2024-09-30 03:39:30,224][1157819] Updated weights for policy 0, policy_version 208708 (0.0006) [2024-09-30 03:39:30,466][1157520] Fps is (10 sec: 74137.8, 60 sec: 72021.4, 300 sec: 72478.4). Total num frames: 854884352. Throughput: 0: 18388.7. Samples: 203689940. Policy #0 lag: (min: 0.0, avg: 3.1, max: 6.0) [2024-09-30 03:39:30,466][1157520] Avg episode reward: [(0, '55.879')] [2024-09-30 03:39:30,764][1157819] Updated weights for policy 0, policy_version 208718 (0.0006) [2024-09-30 03:39:31,378][1157819] Updated weights for policy 0, policy_version 208728 (0.0006) [2024-09-30 03:39:31,916][1157819] Updated weights for policy 0, policy_version 208738 (0.0006) [2024-09-30 03:39:32,472][1157819] Updated weights for policy 0, policy_version 208748 (0.0006) [2024-09-30 03:39:32,995][1157819] Updated weights for policy 0, policy_version 208758 (0.0006) [2024-09-30 03:39:33,506][1157819] Updated weights for policy 0, policy_version 208768 (0.0006) [2024-09-30 03:39:34,051][1157819] Updated weights for policy 0, policy_version 208778 (0.0006) [2024-09-30 03:39:34,609][1157819] Updated weights for policy 0, policy_version 208788 (0.0006) [2024-09-30 03:39:35,157][1157819] Updated weights for policy 0, policy_version 208798 (0.0006) [2024-09-30 03:39:35,466][1157520] Fps is (10 sec: 74956.4, 60 sec: 72840.6, 300 sec: 72464.5). Total num frames: 855257088. Throughput: 0: 18459.9. Samples: 203802368. Policy #0 lag: (min: 0.0, avg: 3.1, max: 6.0) [2024-09-30 03:39:35,466][1157520] Avg episode reward: [(0, '55.567')] [2024-09-30 03:39:35,684][1157819] Updated weights for policy 0, policy_version 208808 (0.0006) [2024-09-30 03:39:36,226][1157819] Updated weights for policy 0, policy_version 208818 (0.0006) [2024-09-30 03:39:36,760][1157819] Updated weights for policy 0, policy_version 208828 (0.0006) [2024-09-30 03:39:37,296][1157819] Updated weights for policy 0, policy_version 208838 (0.0006) [2024-09-30 03:39:37,872][1157819] Updated weights for policy 0, policy_version 208848 (0.0006) [2024-09-30 03:39:38,373][1157819] Updated weights for policy 0, policy_version 208858 (0.0006) [2024-09-30 03:39:38,974][1157819] Updated weights for policy 0, policy_version 208868 (0.0006) [2024-09-30 03:39:39,477][1157819] Updated weights for policy 0, policy_version 208878 (0.0006) [2024-09-30 03:39:40,040][1157819] Updated weights for policy 0, policy_version 208888 (0.0006) [2024-09-30 03:39:40,466][1157520] Fps is (10 sec: 75366.3, 60 sec: 73659.7, 300 sec: 72492.3). Total num frames: 855638016. Throughput: 0: 18513.8. Samples: 203859004. Policy #0 lag: (min: 0.0, avg: 3.1, max: 6.0) [2024-09-30 03:39:40,466][1157520] Avg episode reward: [(0, '55.160')] [2024-09-30 03:39:40,541][1157736] Signal inference workers to stop experience collection... (14600 times) [2024-09-30 03:39:40,545][1157736] Signal inference workers to resume experience collection... (14600 times) [2024-09-30 03:39:40,546][1157819] InferenceWorker_p0-w0: stopping experience collection (14600 times) [2024-09-30 03:39:40,549][1157819] InferenceWorker_p0-w0: resuming experience collection (14600 times) [2024-09-30 03:39:40,565][1157819] Updated weights for policy 0, policy_version 208898 (0.0006) [2024-09-30 03:39:41,117][1157819] Updated weights for policy 0, policy_version 208908 (0.0006) [2024-09-30 03:39:41,646][1157819] Updated weights for policy 0, policy_version 208918 (0.0006) [2024-09-30 03:39:42,185][1157819] Updated weights for policy 0, policy_version 208928 (0.0006) [2024-09-30 03:39:42,699][1157819] Updated weights for policy 0, policy_version 208938 (0.0006) [2024-09-30 03:39:43,210][1157819] Updated weights for policy 0, policy_version 208948 (0.0006) [2024-09-30 03:39:43,745][1157819] Updated weights for policy 0, policy_version 208958 (0.0006) [2024-09-30 03:39:44,246][1157819] Updated weights for policy 0, policy_version 208968 (0.0006) [2024-09-30 03:39:44,782][1157819] Updated weights for policy 0, policy_version 208978 (0.0006) [2024-09-30 03:39:45,284][1157819] Updated weights for policy 0, policy_version 208988 (0.0006) [2024-09-30 03:39:45,466][1157520] Fps is (10 sec: 77005.6, 60 sec: 74410.7, 300 sec: 72533.9). Total num frames: 856027136. Throughput: 0: 18735.7. Samples: 203973928. Policy #0 lag: (min: 0.0, avg: 3.1, max: 6.0) [2024-09-30 03:39:45,466][1157520] Avg episode reward: [(0, '56.170')] [2024-09-30 03:39:45,817][1157819] Updated weights for policy 0, policy_version 208998 (0.0006) [2024-09-30 03:39:46,328][1157819] Updated weights for policy 0, policy_version 209008 (0.0006) [2024-09-30 03:39:46,822][1157819] Updated weights for policy 0, policy_version 209018 (0.0006) [2024-09-30 03:39:47,355][1157819] Updated weights for policy 0, policy_version 209028 (0.0006) [2024-09-30 03:39:47,869][1157819] Updated weights for policy 0, policy_version 209038 (0.0006) [2024-09-30 03:39:48,378][1157819] Updated weights for policy 0, policy_version 209048 (0.0006) [2024-09-30 03:39:48,898][1157819] Updated weights for policy 0, policy_version 209058 (0.0006) [2024-09-30 03:39:49,413][1157819] Updated weights for policy 0, policy_version 209068 (0.0006) [2024-09-30 03:39:49,920][1157819] Updated weights for policy 0, policy_version 209078 (0.0006) [2024-09-30 03:39:50,431][1157819] Updated weights for policy 0, policy_version 209088 (0.0006) [2024-09-30 03:39:50,466][1157520] Fps is (10 sec: 78643.1, 60 sec: 74820.2, 300 sec: 72617.2). Total num frames: 856424448. Throughput: 0: 18855.1. Samples: 204093200. Policy #0 lag: (min: 0.0, avg: 3.1, max: 6.0) [2024-09-30 03:39:50,466][1157520] Avg episode reward: [(0, '56.619')] [2024-09-30 03:39:50,951][1157819] Updated weights for policy 0, policy_version 209098 (0.0006) [2024-09-30 03:39:51,450][1157819] Updated weights for policy 0, policy_version 209108 (0.0006) [2024-09-30 03:39:51,979][1157819] Updated weights for policy 0, policy_version 209118 (0.0006) [2024-09-30 03:39:52,496][1157819] Updated weights for policy 0, policy_version 209128 (0.0006) [2024-09-30 03:39:53,028][1157819] Updated weights for policy 0, policy_version 209138 (0.0006) [2024-09-30 03:39:53,549][1157819] Updated weights for policy 0, policy_version 209148 (0.0006) [2024-09-30 03:39:54,056][1157819] Updated weights for policy 0, policy_version 209158 (0.0006) [2024-09-30 03:39:54,554][1157819] Updated weights for policy 0, policy_version 209168 (0.0006) [2024-09-30 03:39:55,048][1157819] Updated weights for policy 0, policy_version 209178 (0.0006) [2024-09-30 03:39:55,466][1157520] Fps is (10 sec: 79462.3, 60 sec: 75571.1, 300 sec: 72686.7). Total num frames: 856821760. Throughput: 0: 18877.1. Samples: 204152856. Policy #0 lag: (min: 0.0, avg: 3.1, max: 6.0) [2024-09-30 03:39:55,466][1157520] Avg episode reward: [(0, '56.741')] [2024-09-30 03:39:55,573][1157819] Updated weights for policy 0, policy_version 209188 (0.0006) [2024-09-30 03:39:56,126][1157819] Updated weights for policy 0, policy_version 209198 (0.0006) [2024-09-30 03:39:56,629][1157819] Updated weights for policy 0, policy_version 209208 (0.0006) [2024-09-30 03:39:57,163][1157819] Updated weights for policy 0, policy_version 209218 (0.0006) [2024-09-30 03:39:57,699][1157819] Updated weights for policy 0, policy_version 209228 (0.0006) [2024-09-30 03:39:58,235][1157819] Updated weights for policy 0, policy_version 209238 (0.0006) [2024-09-30 03:39:58,805][1157819] Updated weights for policy 0, policy_version 209248 (0.0006) [2024-09-30 03:39:59,332][1157819] Updated weights for policy 0, policy_version 209258 (0.0006) [2024-09-30 03:39:59,887][1157819] Updated weights for policy 0, policy_version 209268 (0.0006) [2024-09-30 03:40:00,466][1157520] Fps is (10 sec: 77413.6, 60 sec: 75912.3, 300 sec: 72658.9). Total num frames: 857198592. Throughput: 0: 18993.0. Samples: 204270028. Policy #0 lag: (min: 0.0, avg: 3.1, max: 6.0) [2024-09-30 03:40:00,466][1157520] Avg episode reward: [(0, '54.597')] [2024-09-30 03:40:00,470][1157819] Updated weights for policy 0, policy_version 209278 (0.0006) [2024-09-30 03:40:01,063][1157819] Updated weights for policy 0, policy_version 209288 (0.0006) [2024-09-30 03:40:01,621][1157819] Updated weights for policy 0, policy_version 209298 (0.0006) [2024-09-30 03:40:02,193][1157819] Updated weights for policy 0, policy_version 209308 (0.0006) [2024-09-30 03:40:02,724][1157819] Updated weights for policy 0, policy_version 209318 (0.0006) [2024-09-30 03:40:03,250][1157819] Updated weights for policy 0, policy_version 209328 (0.0006) [2024-09-30 03:40:03,785][1157819] Updated weights for policy 0, policy_version 209338 (0.0006) [2024-09-30 03:40:04,329][1157819] Updated weights for policy 0, policy_version 209348 (0.0006) [2024-09-30 03:40:04,888][1157819] Updated weights for policy 0, policy_version 209358 (0.0006) [2024-09-30 03:40:05,410][1157819] Updated weights for policy 0, policy_version 209368 (0.0006) [2024-09-30 03:40:05,466][1157520] Fps is (10 sec: 75366.1, 60 sec: 75775.9, 300 sec: 72672.7). Total num frames: 857575424. Throughput: 0: 19030.8. Samples: 204380796. Policy #0 lag: (min: 0.0, avg: 3.1, max: 6.0) [2024-09-30 03:40:05,466][1157520] Avg episode reward: [(0, '53.856')] [2024-09-30 03:40:05,472][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000209369_857575424.pth... [2024-09-30 03:40:05,516][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000205173_840388608.pth [2024-09-30 03:40:05,931][1157819] Updated weights for policy 0, policy_version 209378 (0.0006) [2024-09-30 03:40:06,499][1157819] Updated weights for policy 0, policy_version 209388 (0.0006) [2024-09-30 03:40:07,032][1157819] Updated weights for policy 0, policy_version 209398 (0.0006) [2024-09-30 03:40:07,574][1157819] Updated weights for policy 0, policy_version 209408 (0.0006) [2024-09-30 03:40:08,120][1157819] Updated weights for policy 0, policy_version 209418 (0.0006) [2024-09-30 03:40:08,620][1157819] Updated weights for policy 0, policy_version 209428 (0.0006) [2024-09-30 03:40:09,156][1157819] Updated weights for policy 0, policy_version 209438 (0.0006) [2024-09-30 03:40:09,705][1157819] Updated weights for policy 0, policy_version 209448 (0.0006) [2024-09-30 03:40:10,229][1157819] Updated weights for policy 0, policy_version 209458 (0.0006) [2024-09-30 03:40:10,466][1157520] Fps is (10 sec: 75776.5, 60 sec: 75776.0, 300 sec: 72645.0). Total num frames: 857956352. Throughput: 0: 19120.7. Samples: 204437928. Policy #0 lag: (min: 0.0, avg: 3.1, max: 6.0) [2024-09-30 03:40:10,466][1157520] Avg episode reward: [(0, '55.113')] [2024-09-30 03:40:10,766][1157819] Updated weights for policy 0, policy_version 209468 (0.0006) [2024-09-30 03:40:11,290][1157819] Updated weights for policy 0, policy_version 209478 (0.0006) [2024-09-30 03:40:11,807][1157819] Updated weights for policy 0, policy_version 209488 (0.0006) [2024-09-30 03:40:12,357][1157819] Updated weights for policy 0, policy_version 209498 (0.0006) [2024-09-30 03:40:12,916][1157819] Updated weights for policy 0, policy_version 209508 (0.0006) [2024-09-30 03:40:13,465][1157819] Updated weights for policy 0, policy_version 209518 (0.0006) [2024-09-30 03:40:14,027][1157819] Updated weights for policy 0, policy_version 209528 (0.0006) [2024-09-30 03:40:14,560][1157819] Updated weights for policy 0, policy_version 209538 (0.0006) [2024-09-30 03:40:15,091][1157819] Updated weights for policy 0, policy_version 209548 (0.0006) [2024-09-30 03:40:15,466][1157520] Fps is (10 sec: 76184.5, 60 sec: 75912.3, 300 sec: 72617.2). Total num frames: 858337280. Throughput: 0: 19161.4. Samples: 204552208. Policy #0 lag: (min: 0.0, avg: 1.5, max: 5.0) [2024-09-30 03:40:15,466][1157520] Avg episode reward: [(0, '56.422')] [2024-09-30 03:40:15,648][1157819] Updated weights for policy 0, policy_version 209558 (0.0006) [2024-09-30 03:40:16,148][1157819] Updated weights for policy 0, policy_version 209568 (0.0006) [2024-09-30 03:40:16,668][1157819] Updated weights for policy 0, policy_version 209578 (0.0006) [2024-09-30 03:40:17,250][1157819] Updated weights for policy 0, policy_version 209588 (0.0006) [2024-09-30 03:40:17,812][1157819] Updated weights for policy 0, policy_version 209598 (0.0006) [2024-09-30 03:40:18,332][1157819] Updated weights for policy 0, policy_version 209608 (0.0006) [2024-09-30 03:40:18,891][1157819] Updated weights for policy 0, policy_version 209618 (0.0006) [2024-09-30 03:40:19,417][1157819] Updated weights for policy 0, policy_version 209628 (0.0006) [2024-09-30 03:40:19,928][1157819] Updated weights for policy 0, policy_version 209638 (0.0006) [2024-09-30 03:40:20,448][1157819] Updated weights for policy 0, policy_version 209648 (0.0006) [2024-09-30 03:40:20,466][1157520] Fps is (10 sec: 76185.7, 60 sec: 76253.9, 300 sec: 72617.2). Total num frames: 858718208. Throughput: 0: 19200.9. Samples: 204666408. Policy #0 lag: (min: 0.0, avg: 1.5, max: 5.0) [2024-09-30 03:40:20,466][1157520] Avg episode reward: [(0, '55.537')] [2024-09-30 03:40:20,963][1157819] Updated weights for policy 0, policy_version 209658 (0.0006) [2024-09-30 03:40:21,505][1157819] Updated weights for policy 0, policy_version 209668 (0.0006) [2024-09-30 03:40:22,042][1157819] Updated weights for policy 0, policy_version 209678 (0.0006) [2024-09-30 03:40:22,628][1157819] Updated weights for policy 0, policy_version 209688 (0.0006) [2024-09-30 03:40:23,150][1157819] Updated weights for policy 0, policy_version 209698 (0.0006) [2024-09-30 03:40:23,673][1157819] Updated weights for policy 0, policy_version 209708 (0.0006) [2024-09-30 03:40:24,217][1157819] Updated weights for policy 0, policy_version 209718 (0.0006) [2024-09-30 03:40:24,751][1157819] Updated weights for policy 0, policy_version 209728 (0.0006) [2024-09-30 03:40:25,282][1157819] Updated weights for policy 0, policy_version 209738 (0.0006) [2024-09-30 03:40:25,466][1157520] Fps is (10 sec: 76186.4, 60 sec: 76526.9, 300 sec: 72631.1). Total num frames: 859099136. Throughput: 0: 19218.2. Samples: 204723824. Policy #0 lag: (min: 0.0, avg: 1.5, max: 5.0) [2024-09-30 03:40:25,466][1157520] Avg episode reward: [(0, '55.555')] [2024-09-30 03:40:25,833][1157819] Updated weights for policy 0, policy_version 209748 (0.0006) [2024-09-30 03:40:26,365][1157819] Updated weights for policy 0, policy_version 209758 (0.0006) [2024-09-30 03:40:26,918][1157819] Updated weights for policy 0, policy_version 209768 (0.0006) [2024-09-30 03:40:27,444][1157819] Updated weights for policy 0, policy_version 209778 (0.0006) [2024-09-30 03:40:27,973][1157819] Updated weights for policy 0, policy_version 209788 (0.0006) [2024-09-30 03:40:28,501][1157819] Updated weights for policy 0, policy_version 209798 (0.0006) [2024-09-30 03:40:29,040][1157819] Updated weights for policy 0, policy_version 209808 (0.0006) [2024-09-30 03:40:29,575][1157819] Updated weights for policy 0, policy_version 209818 (0.0006) [2024-09-30 03:40:30,097][1157819] Updated weights for policy 0, policy_version 209828 (0.0006) [2024-09-30 03:40:30,380][1157736] Signal inference workers to stop experience collection... (14650 times) [2024-09-30 03:40:30,380][1157736] Signal inference workers to resume experience collection... (14650 times) [2024-09-30 03:40:30,386][1157819] InferenceWorker_p0-w0: stopping experience collection (14650 times) [2024-09-30 03:40:30,386][1157819] InferenceWorker_p0-w0: resuming experience collection (14650 times) [2024-09-30 03:40:30,466][1157520] Fps is (10 sec: 76595.4, 60 sec: 76663.5, 300 sec: 72658.9). Total num frames: 859484160. Throughput: 0: 19215.7. Samples: 204838632. Policy #0 lag: (min: 0.0, avg: 1.5, max: 5.0) [2024-09-30 03:40:30,466][1157520] Avg episode reward: [(0, '57.132')] [2024-09-30 03:40:30,633][1157819] Updated weights for policy 0, policy_version 209838 (0.0006) [2024-09-30 03:40:31,165][1157819] Updated weights for policy 0, policy_version 209848 (0.0006) [2024-09-30 03:40:31,710][1157819] Updated weights for policy 0, policy_version 209858 (0.0006) [2024-09-30 03:40:32,235][1157819] Updated weights for policy 0, policy_version 209868 (0.0006) [2024-09-30 03:40:32,772][1157819] Updated weights for policy 0, policy_version 209878 (0.0006) [2024-09-30 03:40:33,295][1157819] Updated weights for policy 0, policy_version 209888 (0.0006) [2024-09-30 03:40:33,828][1157819] Updated weights for policy 0, policy_version 209898 (0.0006) [2024-09-30 03:40:34,334][1157819] Updated weights for policy 0, policy_version 209908 (0.0006) [2024-09-30 03:40:34,873][1157819] Updated weights for policy 0, policy_version 209918 (0.0006) [2024-09-30 03:40:35,406][1157819] Updated weights for policy 0, policy_version 209928 (0.0006) [2024-09-30 03:40:35,466][1157520] Fps is (10 sec: 76595.7, 60 sec: 76800.1, 300 sec: 72631.1). Total num frames: 859865088. Throughput: 0: 19138.5. Samples: 204954432. Policy #0 lag: (min: 0.0, avg: 1.5, max: 5.0) [2024-09-30 03:40:35,466][1157520] Avg episode reward: [(0, '54.482')] [2024-09-30 03:40:35,956][1157819] Updated weights for policy 0, policy_version 209938 (0.0006) [2024-09-30 03:40:36,501][1157819] Updated weights for policy 0, policy_version 209948 (0.0006) [2024-09-30 03:40:37,060][1157819] Updated weights for policy 0, policy_version 209958 (0.0006) [2024-09-30 03:40:37,600][1157819] Updated weights for policy 0, policy_version 209968 (0.0006) [2024-09-30 03:40:38,132][1157819] Updated weights for policy 0, policy_version 209978 (0.0006) [2024-09-30 03:40:38,657][1157819] Updated weights for policy 0, policy_version 209988 (0.0006) [2024-09-30 03:40:39,179][1157819] Updated weights for policy 0, policy_version 209998 (0.0006) [2024-09-30 03:40:39,682][1157819] Updated weights for policy 0, policy_version 210008 (0.0006) [2024-09-30 03:40:40,199][1157819] Updated weights for policy 0, policy_version 210018 (0.0006) [2024-09-30 03:40:40,466][1157520] Fps is (10 sec: 76595.1, 60 sec: 76868.3, 300 sec: 72603.4). Total num frames: 860250112. Throughput: 0: 19073.0. Samples: 205011140. Policy #0 lag: (min: 0.0, avg: 1.5, max: 5.0) [2024-09-30 03:40:40,466][1157520] Avg episode reward: [(0, '55.830')] [2024-09-30 03:40:40,742][1157819] Updated weights for policy 0, policy_version 210028 (0.0006) [2024-09-30 03:40:41,239][1157819] Updated weights for policy 0, policy_version 210038 (0.0006) [2024-09-30 03:40:41,733][1157819] Updated weights for policy 0, policy_version 210048 (0.0006) [2024-09-30 03:40:42,234][1157819] Updated weights for policy 0, policy_version 210058 (0.0006) [2024-09-30 03:40:42,737][1157819] Updated weights for policy 0, policy_version 210068 (0.0006) [2024-09-30 03:40:43,223][1157819] Updated weights for policy 0, policy_version 210078 (0.0006) [2024-09-30 03:40:43,737][1157819] Updated weights for policy 0, policy_version 210088 (0.0006) [2024-09-30 03:40:44,258][1157819] Updated weights for policy 0, policy_version 210098 (0.0006) [2024-09-30 03:40:44,784][1157819] Updated weights for policy 0, policy_version 210108 (0.0006) [2024-09-30 03:40:45,297][1157819] Updated weights for policy 0, policy_version 210118 (0.0006) [2024-09-30 03:40:45,466][1157520] Fps is (10 sec: 79052.7, 60 sec: 77141.3, 300 sec: 72589.5). Total num frames: 860655616. Throughput: 0: 19130.1. Samples: 205130880. Policy #0 lag: (min: 0.0, avg: 1.5, max: 5.0) [2024-09-30 03:40:45,466][1157520] Avg episode reward: [(0, '56.226')] [2024-09-30 03:40:45,811][1157819] Updated weights for policy 0, policy_version 210128 (0.0006) [2024-09-30 03:40:46,331][1157819] Updated weights for policy 0, policy_version 210138 (0.0006) [2024-09-30 03:40:46,911][1157819] Updated weights for policy 0, policy_version 210148 (0.0006) [2024-09-30 03:40:47,438][1157819] Updated weights for policy 0, policy_version 210158 (0.0006) [2024-09-30 03:40:48,012][1157819] Updated weights for policy 0, policy_version 210168 (0.0006) [2024-09-30 03:40:48,589][1157819] Updated weights for policy 0, policy_version 210178 (0.0006) [2024-09-30 03:40:49,151][1157819] Updated weights for policy 0, policy_version 210188 (0.0006) [2024-09-30 03:40:49,726][1157819] Updated weights for policy 0, policy_version 210198 (0.0006) [2024-09-30 03:40:50,277][1157819] Updated weights for policy 0, policy_version 210208 (0.0006) [2024-09-30 03:40:50,466][1157520] Fps is (10 sec: 77414.4, 60 sec: 76663.5, 300 sec: 72520.0). Total num frames: 861024256. Throughput: 0: 19186.2. Samples: 205244172. Policy #0 lag: (min: 0.0, avg: 1.5, max: 5.0) [2024-09-30 03:40:50,466][1157520] Avg episode reward: [(0, '57.164')] [2024-09-30 03:40:50,821][1157819] Updated weights for policy 0, policy_version 210218 (0.0006) [2024-09-30 03:40:51,347][1157819] Updated weights for policy 0, policy_version 210228 (0.0006) [2024-09-30 03:40:51,886][1157819] Updated weights for policy 0, policy_version 210238 (0.0006) [2024-09-30 03:40:52,459][1157819] Updated weights for policy 0, policy_version 210248 (0.0006) [2024-09-30 03:40:53,006][1157819] Updated weights for policy 0, policy_version 210258 (0.0006) [2024-09-30 03:40:53,528][1157819] Updated weights for policy 0, policy_version 210268 (0.0006) [2024-09-30 03:40:54,098][1157819] Updated weights for policy 0, policy_version 210278 (0.0006) [2024-09-30 03:40:54,645][1157819] Updated weights for policy 0, policy_version 210288 (0.0006) [2024-09-30 03:40:55,192][1157819] Updated weights for policy 0, policy_version 210298 (0.0006) [2024-09-30 03:40:55,466][1157520] Fps is (10 sec: 74137.7, 60 sec: 76253.9, 300 sec: 72533.9). Total num frames: 861396992. Throughput: 0: 19157.6. Samples: 205300020. Policy #0 lag: (min: 0.0, avg: 1.5, max: 5.0) [2024-09-30 03:40:55,466][1157520] Avg episode reward: [(0, '55.067')] [2024-09-30 03:40:55,746][1157819] Updated weights for policy 0, policy_version 210308 (0.0006) [2024-09-30 03:40:56,283][1157819] Updated weights for policy 0, policy_version 210318 (0.0006) [2024-09-30 03:40:56,795][1157819] Updated weights for policy 0, policy_version 210328 (0.0006) [2024-09-30 03:40:57,337][1157819] Updated weights for policy 0, policy_version 210338 (0.0006) [2024-09-30 03:40:57,898][1157819] Updated weights for policy 0, policy_version 210348 (0.0006) [2024-09-30 03:40:58,449][1157819] Updated weights for policy 0, policy_version 210358 (0.0006) [2024-09-30 03:40:58,995][1157819] Updated weights for policy 0, policy_version 210368 (0.0006) [2024-09-30 03:40:59,553][1157819] Updated weights for policy 0, policy_version 210378 (0.0006) [2024-09-30 03:41:00,066][1157819] Updated weights for policy 0, policy_version 210388 (0.0006) [2024-09-30 03:41:00,466][1157520] Fps is (10 sec: 74957.0, 60 sec: 76254.0, 300 sec: 72506.1). Total num frames: 861773824. Throughput: 0: 19127.0. Samples: 205412920. Policy #0 lag: (min: 0.0, avg: 1.5, max: 5.0) [2024-09-30 03:41:00,466][1157520] Avg episode reward: [(0, '57.550')] [2024-09-30 03:41:00,616][1157819] Updated weights for policy 0, policy_version 210398 (0.0006) [2024-09-30 03:41:01,158][1157819] Updated weights for policy 0, policy_version 210408 (0.0006) [2024-09-30 03:41:01,685][1157819] Updated weights for policy 0, policy_version 210418 (0.0006) [2024-09-30 03:41:02,228][1157819] Updated weights for policy 0, policy_version 210428 (0.0006) [2024-09-30 03:41:02,764][1157819] Updated weights for policy 0, policy_version 210438 (0.0006) [2024-09-30 03:41:03,278][1157819] Updated weights for policy 0, policy_version 210448 (0.0006) [2024-09-30 03:41:03,831][1157819] Updated weights for policy 0, policy_version 210458 (0.0006) [2024-09-30 03:41:04,372][1157819] Updated weights for policy 0, policy_version 210468 (0.0006) [2024-09-30 03:41:04,903][1157819] Updated weights for policy 0, policy_version 210478 (0.0006) [2024-09-30 03:41:05,456][1157819] Updated weights for policy 0, policy_version 210488 (0.0006) [2024-09-30 03:41:05,466][1157520] Fps is (10 sec: 76185.7, 60 sec: 76390.4, 300 sec: 72603.3). Total num frames: 862158848. Throughput: 0: 19127.3. Samples: 205527136. Policy #0 lag: (min: 0.0, avg: 1.5, max: 5.0) [2024-09-30 03:41:05,466][1157520] Avg episode reward: [(0, '54.807')] [2024-09-30 03:41:05,977][1157819] Updated weights for policy 0, policy_version 210498 (0.0006) [2024-09-30 03:41:06,521][1157819] Updated weights for policy 0, policy_version 210508 (0.0006) [2024-09-30 03:41:07,080][1157819] Updated weights for policy 0, policy_version 210518 (0.0006) [2024-09-30 03:41:07,614][1157819] Updated weights for policy 0, policy_version 210528 (0.0006) [2024-09-30 03:41:08,156][1157819] Updated weights for policy 0, policy_version 210538 (0.0006) [2024-09-30 03:41:08,657][1157819] Updated weights for policy 0, policy_version 210548 (0.0006) [2024-09-30 03:41:09,158][1157819] Updated weights for policy 0, policy_version 210558 (0.0006) [2024-09-30 03:41:09,676][1157819] Updated weights for policy 0, policy_version 210568 (0.0006) [2024-09-30 03:41:10,198][1157819] Updated weights for policy 0, policy_version 210578 (0.0006) [2024-09-30 03:41:10,466][1157520] Fps is (10 sec: 77414.2, 60 sec: 76527.0, 300 sec: 72672.8). Total num frames: 862547968. Throughput: 0: 19110.3. Samples: 205583788. Policy #0 lag: (min: 0.0, avg: 1.5, max: 5.0) [2024-09-30 03:41:10,466][1157520] Avg episode reward: [(0, '54.257')] [2024-09-30 03:41:10,698][1157819] Updated weights for policy 0, policy_version 210588 (0.0006) [2024-09-30 03:41:11,196][1157819] Updated weights for policy 0, policy_version 210598 (0.0006) [2024-09-30 03:41:11,720][1157819] Updated weights for policy 0, policy_version 210608 (0.0006) [2024-09-30 03:41:12,217][1157819] Updated weights for policy 0, policy_version 210618 (0.0006) [2024-09-30 03:41:12,716][1157819] Updated weights for policy 0, policy_version 210628 (0.0006) [2024-09-30 03:41:13,240][1157819] Updated weights for policy 0, policy_version 210638 (0.0006) [2024-09-30 03:41:13,759][1157819] Updated weights for policy 0, policy_version 210648 (0.0006) [2024-09-30 03:41:14,275][1157819] Updated weights for policy 0, policy_version 210658 (0.0006) [2024-09-30 03:41:14,791][1157819] Updated weights for policy 0, policy_version 210668 (0.0006) [2024-09-30 03:41:15,294][1157819] Updated weights for policy 0, policy_version 210678 (0.0006) [2024-09-30 03:41:15,466][1157520] Fps is (10 sec: 79051.7, 60 sec: 76868.3, 300 sec: 72714.4). Total num frames: 862949376. Throughput: 0: 19230.4. Samples: 205704004. Policy #0 lag: (min: 0.0, avg: 1.5, max: 5.0) [2024-09-30 03:41:15,466][1157520] Avg episode reward: [(0, '56.283')] [2024-09-30 03:41:15,822][1157819] Updated weights for policy 0, policy_version 210688 (0.0006) [2024-09-30 03:41:16,340][1157819] Updated weights for policy 0, policy_version 210698 (0.0006) [2024-09-30 03:41:16,840][1157819] Updated weights for policy 0, policy_version 210708 (0.0006) [2024-09-30 03:41:17,348][1157819] Updated weights for policy 0, policy_version 210718 (0.0006) [2024-09-30 03:41:17,856][1157819] Updated weights for policy 0, policy_version 210728 (0.0006) [2024-09-30 03:41:18,357][1157819] Updated weights for policy 0, policy_version 210738 (0.0007) [2024-09-30 03:41:18,872][1157819] Updated weights for policy 0, policy_version 210748 (0.0006) [2024-09-30 03:41:19,423][1157819] Updated weights for policy 0, policy_version 210758 (0.0006) [2024-09-30 03:41:19,951][1157819] Updated weights for policy 0, policy_version 210768 (0.0006) [2024-09-30 03:41:20,443][1157819] Updated weights for policy 0, policy_version 210778 (0.0006) [2024-09-30 03:41:20,466][1157520] Fps is (10 sec: 79871.7, 60 sec: 77141.3, 300 sec: 72881.0). Total num frames: 863346688. Throughput: 0: 19320.4. Samples: 205823848. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:41:20,466][1157520] Avg episode reward: [(0, '58.066')] [2024-09-30 03:41:20,959][1157819] Updated weights for policy 0, policy_version 210788 (0.0006) [2024-09-30 03:41:21,454][1157819] Updated weights for policy 0, policy_version 210798 (0.0006) [2024-09-30 03:41:21,961][1157819] Updated weights for policy 0, policy_version 210808 (0.0006) [2024-09-30 03:41:22,456][1157819] Updated weights for policy 0, policy_version 210818 (0.0006) [2024-09-30 03:41:23,009][1157819] Updated weights for policy 0, policy_version 210828 (0.0006) [2024-09-30 03:41:23,530][1157819] Updated weights for policy 0, policy_version 210838 (0.0006) [2024-09-30 03:41:24,057][1157819] Updated weights for policy 0, policy_version 210848 (0.0006) [2024-09-30 03:41:24,575][1157819] Updated weights for policy 0, policy_version 210858 (0.0006) [2024-09-30 03:41:25,110][1157819] Updated weights for policy 0, policy_version 210868 (0.0006) [2024-09-30 03:41:25,466][1157520] Fps is (10 sec: 79053.8, 60 sec: 77346.2, 300 sec: 73033.8). Total num frames: 863739904. Throughput: 0: 19390.0. Samples: 205883692. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:41:25,466][1157520] Avg episode reward: [(0, '56.131')] [2024-09-30 03:41:25,687][1157819] Updated weights for policy 0, policy_version 210878 (0.0006) [2024-09-30 03:41:26,221][1157819] Updated weights for policy 0, policy_version 210888 (0.0006) [2024-09-30 03:41:26,770][1157819] Updated weights for policy 0, policy_version 210898 (0.0006) [2024-09-30 03:41:27,316][1157819] Updated weights for policy 0, policy_version 210908 (0.0006) [2024-09-30 03:41:27,471][1157736] Signal inference workers to stop experience collection... (14700 times) [2024-09-30 03:41:27,473][1157736] Signal inference workers to resume experience collection... (14700 times) [2024-09-30 03:41:27,477][1157819] InferenceWorker_p0-w0: stopping experience collection (14700 times) [2024-09-30 03:41:27,479][1157819] InferenceWorker_p0-w0: resuming experience collection (14700 times) [2024-09-30 03:41:27,839][1157819] Updated weights for policy 0, policy_version 210918 (0.0006) [2024-09-30 03:41:28,425][1157819] Updated weights for policy 0, policy_version 210928 (0.0006) [2024-09-30 03:41:28,931][1157819] Updated weights for policy 0, policy_version 210938 (0.0006) [2024-09-30 03:41:29,494][1157819] Updated weights for policy 0, policy_version 210948 (0.0006) [2024-09-30 03:41:30,014][1157819] Updated weights for policy 0, policy_version 210958 (0.0006) [2024-09-30 03:41:30,466][1157520] Fps is (10 sec: 77004.6, 60 sec: 77209.5, 300 sec: 73144.8). Total num frames: 864116736. Throughput: 0: 19260.8. Samples: 205997616. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:41:30,466][1157520] Avg episode reward: [(0, '55.034')] [2024-09-30 03:41:30,538][1157819] Updated weights for policy 0, policy_version 210968 (0.0006) [2024-09-30 03:41:31,108][1157819] Updated weights for policy 0, policy_version 210978 (0.0006) [2024-09-30 03:41:31,640][1157819] Updated weights for policy 0, policy_version 210988 (0.0006) [2024-09-30 03:41:32,177][1157819] Updated weights for policy 0, policy_version 210998 (0.0006) [2024-09-30 03:41:32,730][1157819] Updated weights for policy 0, policy_version 211008 (0.0006) [2024-09-30 03:41:33,269][1157819] Updated weights for policy 0, policy_version 211018 (0.0006) [2024-09-30 03:41:33,825][1157819] Updated weights for policy 0, policy_version 211028 (0.0006) [2024-09-30 03:41:34,364][1157819] Updated weights for policy 0, policy_version 211038 (0.0006) [2024-09-30 03:41:34,915][1157819] Updated weights for policy 0, policy_version 211048 (0.0006) [2024-09-30 03:41:35,466][1157819] Updated weights for policy 0, policy_version 211058 (0.0006) [2024-09-30 03:41:35,466][1157520] Fps is (10 sec: 75366.6, 60 sec: 77141.4, 300 sec: 73242.0). Total num frames: 864493568. Throughput: 0: 19256.0. Samples: 206110692. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:41:35,466][1157520] Avg episode reward: [(0, '56.632')] [2024-09-30 03:41:36,024][1157819] Updated weights for policy 0, policy_version 211068 (0.0006) [2024-09-30 03:41:36,590][1157819] Updated weights for policy 0, policy_version 211078 (0.0006) [2024-09-30 03:41:37,119][1157819] Updated weights for policy 0, policy_version 211088 (0.0006) [2024-09-30 03:41:37,716][1157819] Updated weights for policy 0, policy_version 211098 (0.0006) [2024-09-30 03:41:38,258][1157819] Updated weights for policy 0, policy_version 211108 (0.0006) [2024-09-30 03:41:38,808][1157819] Updated weights for policy 0, policy_version 211118 (0.0006) [2024-09-30 03:41:39,365][1157819] Updated weights for policy 0, policy_version 211128 (0.0006) [2024-09-30 03:41:39,926][1157819] Updated weights for policy 0, policy_version 211138 (0.0006) [2024-09-30 03:41:40,448][1157819] Updated weights for policy 0, policy_version 211148 (0.0006) [2024-09-30 03:41:40,466][1157520] Fps is (10 sec: 74547.9, 60 sec: 76868.3, 300 sec: 73325.4). Total num frames: 864862208. Throughput: 0: 19241.4. Samples: 206165880. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:41:40,466][1157520] Avg episode reward: [(0, '57.081')] [2024-09-30 03:41:41,002][1157819] Updated weights for policy 0, policy_version 211158 (0.0006) [2024-09-30 03:41:41,564][1157819] Updated weights for policy 0, policy_version 211168 (0.0007) [2024-09-30 03:41:42,078][1157819] Updated weights for policy 0, policy_version 211178 (0.0006) [2024-09-30 03:41:42,668][1157819] Updated weights for policy 0, policy_version 211188 (0.0006) [2024-09-30 03:41:43,217][1157819] Updated weights for policy 0, policy_version 211198 (0.0006) [2024-09-30 03:41:43,736][1157819] Updated weights for policy 0, policy_version 211208 (0.0006) [2024-09-30 03:41:44,313][1157819] Updated weights for policy 0, policy_version 211218 (0.0006) [2024-09-30 03:41:44,832][1157819] Updated weights for policy 0, policy_version 211228 (0.0006) [2024-09-30 03:41:45,386][1157819] Updated weights for policy 0, policy_version 211238 (0.0006) [2024-09-30 03:41:45,466][1157520] Fps is (10 sec: 74137.2, 60 sec: 76322.1, 300 sec: 73408.6). Total num frames: 865234944. Throughput: 0: 19213.9. Samples: 206277548. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:41:45,466][1157520] Avg episode reward: [(0, '55.983')] [2024-09-30 03:41:45,936][1157819] Updated weights for policy 0, policy_version 211248 (0.0006) [2024-09-30 03:41:46,502][1157819] Updated weights for policy 0, policy_version 211258 (0.0006) [2024-09-30 03:41:47,046][1157819] Updated weights for policy 0, policy_version 211268 (0.0007) [2024-09-30 03:41:47,623][1157819] Updated weights for policy 0, policy_version 211278 (0.0006) [2024-09-30 03:41:48,178][1157819] Updated weights for policy 0, policy_version 211288 (0.0007) [2024-09-30 03:41:48,690][1157819] Updated weights for policy 0, policy_version 211298 (0.0006) [2024-09-30 03:41:49,213][1157819] Updated weights for policy 0, policy_version 211308 (0.0006) [2024-09-30 03:41:49,767][1157819] Updated weights for policy 0, policy_version 211318 (0.0006) [2024-09-30 03:41:50,328][1157819] Updated weights for policy 0, policy_version 211328 (0.0006) [2024-09-30 03:41:50,466][1157520] Fps is (10 sec: 74547.0, 60 sec: 76390.4, 300 sec: 73436.4). Total num frames: 865607680. Throughput: 0: 19165.9. Samples: 206389600. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:41:50,466][1157520] Avg episode reward: [(0, '56.593')] [2024-09-30 03:41:50,892][1157819] Updated weights for policy 0, policy_version 211338 (0.0006) [2024-09-30 03:41:51,444][1157819] Updated weights for policy 0, policy_version 211348 (0.0006) [2024-09-30 03:41:51,994][1157819] Updated weights for policy 0, policy_version 211358 (0.0006) [2024-09-30 03:41:52,532][1157819] Updated weights for policy 0, policy_version 211368 (0.0006) [2024-09-30 03:41:53,088][1157819] Updated weights for policy 0, policy_version 211378 (0.0006) [2024-09-30 03:41:53,647][1157819] Updated weights for policy 0, policy_version 211388 (0.0006) [2024-09-30 03:41:54,223][1157819] Updated weights for policy 0, policy_version 211398 (0.0006) [2024-09-30 03:41:54,773][1157819] Updated weights for policy 0, policy_version 211408 (0.0006) [2024-09-30 03:41:55,387][1157819] Updated weights for policy 0, policy_version 211418 (0.0007) [2024-09-30 03:41:55,466][1157520] Fps is (10 sec: 73728.0, 60 sec: 76253.8, 300 sec: 73408.6). Total num frames: 865972224. Throughput: 0: 19152.3. Samples: 206445644. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:41:55,466][1157520] Avg episode reward: [(0, '58.580')] [2024-09-30 03:41:55,963][1157819] Updated weights for policy 0, policy_version 211428 (0.0006) [2024-09-30 03:41:56,526][1157819] Updated weights for policy 0, policy_version 211438 (0.0006) [2024-09-30 03:41:57,126][1157819] Updated weights for policy 0, policy_version 211448 (0.0006) [2024-09-30 03:41:57,688][1157819] Updated weights for policy 0, policy_version 211458 (0.0006) [2024-09-30 03:41:58,305][1157819] Updated weights for policy 0, policy_version 211468 (0.0006) [2024-09-30 03:41:58,894][1157819] Updated weights for policy 0, policy_version 211478 (0.0006) [2024-09-30 03:41:59,468][1157819] Updated weights for policy 0, policy_version 211488 (0.0006) [2024-09-30 03:42:00,055][1157819] Updated weights for policy 0, policy_version 211498 (0.0006) [2024-09-30 03:42:00,466][1157520] Fps is (10 sec: 71680.0, 60 sec: 75844.2, 300 sec: 73408.7). Total num frames: 866324480. Throughput: 0: 18844.2. Samples: 206551992. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:42:00,466][1157520] Avg episode reward: [(0, '57.047')] [2024-09-30 03:42:00,597][1157819] Updated weights for policy 0, policy_version 211508 (0.0006) [2024-09-30 03:42:01,207][1157819] Updated weights for policy 0, policy_version 211518 (0.0006) [2024-09-30 03:42:01,759][1157819] Updated weights for policy 0, policy_version 211528 (0.0006) [2024-09-30 03:42:02,313][1157819] Updated weights for policy 0, policy_version 211538 (0.0006) [2024-09-30 03:42:02,956][1157819] Updated weights for policy 0, policy_version 211548 (0.0006) [2024-09-30 03:42:03,505][1157819] Updated weights for policy 0, policy_version 211558 (0.0006) [2024-09-30 03:42:04,070][1157819] Updated weights for policy 0, policy_version 211568 (0.0006) [2024-09-30 03:42:04,684][1157819] Updated weights for policy 0, policy_version 211578 (0.0006) [2024-09-30 03:42:05,249][1157819] Updated weights for policy 0, policy_version 211588 (0.0006) [2024-09-30 03:42:05,466][1157520] Fps is (10 sec: 70451.0, 60 sec: 75298.1, 300 sec: 73436.4). Total num frames: 866676736. Throughput: 0: 18542.0. Samples: 206658240. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:42:05,466][1157520] Avg episode reward: [(0, '54.058')] [2024-09-30 03:42:05,475][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000211592_866680832.pth... [2024-09-30 03:42:05,532][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000207255_848916480.pth [2024-09-30 03:42:05,782][1157819] Updated weights for policy 0, policy_version 211598 (0.0006) [2024-09-30 03:42:06,326][1157819] Updated weights for policy 0, policy_version 211608 (0.0006) [2024-09-30 03:42:06,855][1157819] Updated weights for policy 0, policy_version 211618 (0.0006) [2024-09-30 03:42:07,409][1157819] Updated weights for policy 0, policy_version 211628 (0.0006) [2024-09-30 03:42:07,938][1157819] Updated weights for policy 0, policy_version 211638 (0.0006) [2024-09-30 03:42:08,480][1157819] Updated weights for policy 0, policy_version 211648 (0.0006) [2024-09-30 03:42:09,034][1157819] Updated weights for policy 0, policy_version 211658 (0.0006) [2024-09-30 03:42:09,572][1157819] Updated weights for policy 0, policy_version 211668 (0.0006) [2024-09-30 03:42:10,111][1157819] Updated weights for policy 0, policy_version 211678 (0.0006) [2024-09-30 03:42:10,466][1157520] Fps is (10 sec: 73318.3, 60 sec: 75161.6, 300 sec: 73547.5). Total num frames: 867057664. Throughput: 0: 18460.7. Samples: 206714424. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:42:10,466][1157520] Avg episode reward: [(0, '56.116')] [2024-09-30 03:42:10,654][1157819] Updated weights for policy 0, policy_version 211688 (0.0006) [2024-09-30 03:42:11,170][1157819] Updated weights for policy 0, policy_version 211698 (0.0006) [2024-09-30 03:42:11,731][1157819] Updated weights for policy 0, policy_version 211708 (0.0006) [2024-09-30 03:42:12,243][1157819] Updated weights for policy 0, policy_version 211718 (0.0006) [2024-09-30 03:42:12,763][1157819] Updated weights for policy 0, policy_version 211728 (0.0006) [2024-09-30 03:42:13,294][1157819] Updated weights for policy 0, policy_version 211738 (0.0006) [2024-09-30 03:42:13,834][1157819] Updated weights for policy 0, policy_version 211748 (0.0006) [2024-09-30 03:42:14,349][1157819] Updated weights for policy 0, policy_version 211758 (0.0006) [2024-09-30 03:42:14,891][1157819] Updated weights for policy 0, policy_version 211768 (0.0006) [2024-09-30 03:42:15,441][1157819] Updated weights for policy 0, policy_version 211778 (0.0006) [2024-09-30 03:42:15,466][1157520] Fps is (10 sec: 76595.7, 60 sec: 74888.7, 300 sec: 73672.4). Total num frames: 867442688. Throughput: 0: 18479.1. Samples: 206829176. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:42:15,466][1157520] Avg episode reward: [(0, '56.933')] [2024-09-30 03:42:15,986][1157819] Updated weights for policy 0, policy_version 211788 (0.0006) [2024-09-30 03:42:16,536][1157819] Updated weights for policy 0, policy_version 211798 (0.0006) [2024-09-30 03:42:17,075][1157819] Updated weights for policy 0, policy_version 211808 (0.0006) [2024-09-30 03:42:17,610][1157819] Updated weights for policy 0, policy_version 211818 (0.0006) [2024-09-30 03:42:18,167][1157819] Updated weights for policy 0, policy_version 211828 (0.0006) [2024-09-30 03:42:18,325][1157736] Signal inference workers to stop experience collection... (14750 times) [2024-09-30 03:42:18,328][1157736] Signal inference workers to resume experience collection... (14750 times) [2024-09-30 03:42:18,332][1157819] InferenceWorker_p0-w0: stopping experience collection (14750 times) [2024-09-30 03:42:18,334][1157819] InferenceWorker_p0-w0: resuming experience collection (14750 times) [2024-09-30 03:42:18,667][1157819] Updated weights for policy 0, policy_version 211838 (0.0006) [2024-09-30 03:42:19,229][1157819] Updated weights for policy 0, policy_version 211848 (0.0006) [2024-09-30 03:42:19,780][1157819] Updated weights for policy 0, policy_version 211858 (0.0006) [2024-09-30 03:42:20,298][1157819] Updated weights for policy 0, policy_version 211868 (0.0006) [2024-09-30 03:42:20,466][1157520] Fps is (10 sec: 76595.3, 60 sec: 74615.5, 300 sec: 73769.7). Total num frames: 867823616. Throughput: 0: 18497.4. Samples: 206943076. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:42:20,466][1157520] Avg episode reward: [(0, '57.493')] [2024-09-30 03:42:20,833][1157819] Updated weights for policy 0, policy_version 211878 (0.0006) [2024-09-30 03:42:21,354][1157819] Updated weights for policy 0, policy_version 211888 (0.0006) [2024-09-30 03:42:21,935][1157819] Updated weights for policy 0, policy_version 211898 (0.0006) [2024-09-30 03:42:22,451][1157819] Updated weights for policy 0, policy_version 211908 (0.0006) [2024-09-30 03:42:22,972][1157819] Updated weights for policy 0, policy_version 211918 (0.0006) [2024-09-30 03:42:23,521][1157819] Updated weights for policy 0, policy_version 211928 (0.0006) [2024-09-30 03:42:24,066][1157819] Updated weights for policy 0, policy_version 211938 (0.0006) [2024-09-30 03:42:24,562][1157819] Updated weights for policy 0, policy_version 211948 (0.0006) [2024-09-30 03:42:25,073][1157819] Updated weights for policy 0, policy_version 211958 (0.0006) [2024-09-30 03:42:25,466][1157520] Fps is (10 sec: 76595.0, 60 sec: 74478.9, 300 sec: 73839.1). Total num frames: 868208640. Throughput: 0: 18537.3. Samples: 207000060. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:42:25,466][1157520] Avg episode reward: [(0, '57.305')] [2024-09-30 03:42:25,592][1157819] Updated weights for policy 0, policy_version 211968 (0.0006) [2024-09-30 03:42:26,109][1157819] Updated weights for policy 0, policy_version 211978 (0.0006) [2024-09-30 03:42:26,634][1157819] Updated weights for policy 0, policy_version 211988 (0.0006) [2024-09-30 03:42:27,141][1157819] Updated weights for policy 0, policy_version 211998 (0.0006) [2024-09-30 03:42:27,652][1157819] Updated weights for policy 0, policy_version 212008 (0.0006) [2024-09-30 03:42:28,171][1157819] Updated weights for policy 0, policy_version 212018 (0.0006) [2024-09-30 03:42:28,686][1157819] Updated weights for policy 0, policy_version 212028 (0.0006) [2024-09-30 03:42:29,228][1157819] Updated weights for policy 0, policy_version 212038 (0.0006) [2024-09-30 03:42:29,716][1157819] Updated weights for policy 0, policy_version 212048 (0.0006) [2024-09-30 03:42:30,201][1157819] Updated weights for policy 0, policy_version 212058 (0.0006) [2024-09-30 03:42:30,466][1157520] Fps is (10 sec: 78642.9, 60 sec: 74888.6, 300 sec: 73852.9). Total num frames: 868610048. Throughput: 0: 18685.7. Samples: 207118404. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 03:42:30,466][1157520] Avg episode reward: [(0, '57.895')] [2024-09-30 03:42:30,720][1157819] Updated weights for policy 0, policy_version 212068 (0.0006) [2024-09-30 03:42:31,218][1157819] Updated weights for policy 0, policy_version 212078 (0.0006) [2024-09-30 03:42:31,730][1157819] Updated weights for policy 0, policy_version 212088 (0.0006) [2024-09-30 03:42:32,235][1157819] Updated weights for policy 0, policy_version 212098 (0.0006) [2024-09-30 03:42:32,739][1157819] Updated weights for policy 0, policy_version 212108 (0.0006) [2024-09-30 03:42:33,237][1157819] Updated weights for policy 0, policy_version 212118 (0.0006) [2024-09-30 03:42:33,748][1157819] Updated weights for policy 0, policy_version 212128 (0.0006) [2024-09-30 03:42:34,234][1157819] Updated weights for policy 0, policy_version 212138 (0.0006) [2024-09-30 03:42:34,765][1157819] Updated weights for policy 0, policy_version 212148 (0.0006) [2024-09-30 03:42:35,268][1157819] Updated weights for policy 0, policy_version 212158 (0.0006) [2024-09-30 03:42:35,466][1157520] Fps is (10 sec: 80281.8, 60 sec: 75298.1, 300 sec: 73964.0). Total num frames: 869011456. Throughput: 0: 18898.9. Samples: 207240052. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 03:42:35,466][1157520] Avg episode reward: [(0, '57.211')] [2024-09-30 03:42:35,771][1157819] Updated weights for policy 0, policy_version 212168 (0.0006) [2024-09-30 03:42:36,280][1157819] Updated weights for policy 0, policy_version 212178 (0.0006) [2024-09-30 03:42:36,778][1157819] Updated weights for policy 0, policy_version 212188 (0.0006) [2024-09-30 03:42:37,290][1157819] Updated weights for policy 0, policy_version 212198 (0.0006) [2024-09-30 03:42:37,791][1157819] Updated weights for policy 0, policy_version 212208 (0.0006) [2024-09-30 03:42:38,309][1157819] Updated weights for policy 0, policy_version 212218 (0.0006) [2024-09-30 03:42:38,825][1157819] Updated weights for policy 0, policy_version 212228 (0.0006) [2024-09-30 03:42:39,328][1157819] Updated weights for policy 0, policy_version 212238 (0.0006) [2024-09-30 03:42:39,839][1157819] Updated weights for policy 0, policy_version 212248 (0.0006) [2024-09-30 03:42:40,353][1157819] Updated weights for policy 0, policy_version 212258 (0.0006) [2024-09-30 03:42:40,466][1157520] Fps is (10 sec: 80691.5, 60 sec: 75912.5, 300 sec: 74102.9). Total num frames: 869416960. Throughput: 0: 19000.6. Samples: 207300668. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 03:42:40,466][1157520] Avg episode reward: [(0, '57.173')] [2024-09-30 03:42:40,858][1157819] Updated weights for policy 0, policy_version 212268 (0.0006) [2024-09-30 03:42:41,353][1157819] Updated weights for policy 0, policy_version 212278 (0.0006) [2024-09-30 03:42:41,841][1157819] Updated weights for policy 0, policy_version 212288 (0.0006) [2024-09-30 03:42:42,326][1157819] Updated weights for policy 0, policy_version 212298 (0.0006) [2024-09-30 03:42:42,863][1157819] Updated weights for policy 0, policy_version 212308 (0.0006) [2024-09-30 03:42:43,383][1157819] Updated weights for policy 0, policy_version 212318 (0.0006) [2024-09-30 03:42:43,886][1157819] Updated weights for policy 0, policy_version 212328 (0.0006) [2024-09-30 03:42:44,387][1157819] Updated weights for policy 0, policy_version 212338 (0.0006) [2024-09-30 03:42:44,900][1157819] Updated weights for policy 0, policy_version 212348 (0.0006) [2024-09-30 03:42:45,388][1157819] Updated weights for policy 0, policy_version 212358 (0.0006) [2024-09-30 03:42:45,466][1157520] Fps is (10 sec: 81100.3, 60 sec: 76458.6, 300 sec: 74269.5). Total num frames: 869822464. Throughput: 0: 19329.3. Samples: 207421812. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 03:42:45,466][1157520] Avg episode reward: [(0, '57.995')] [2024-09-30 03:42:45,894][1157819] Updated weights for policy 0, policy_version 212368 (0.0006) [2024-09-30 03:42:46,399][1157819] Updated weights for policy 0, policy_version 212378 (0.0006) [2024-09-30 03:42:46,929][1157819] Updated weights for policy 0, policy_version 212388 (0.0006) [2024-09-30 03:42:47,415][1157819] Updated weights for policy 0, policy_version 212398 (0.0006) [2024-09-30 03:42:47,917][1157819] Updated weights for policy 0, policy_version 212408 (0.0006) [2024-09-30 03:42:48,416][1157819] Updated weights for policy 0, policy_version 212418 (0.0006) [2024-09-30 03:42:48,927][1157819] Updated weights for policy 0, policy_version 212428 (0.0006) [2024-09-30 03:42:49,437][1157819] Updated weights for policy 0, policy_version 212438 (0.0006) [2024-09-30 03:42:49,948][1157819] Updated weights for policy 0, policy_version 212448 (0.0006) [2024-09-30 03:42:50,466][1157520] Fps is (10 sec: 80691.2, 60 sec: 76936.5, 300 sec: 74450.0). Total num frames: 870223872. Throughput: 0: 19670.7. Samples: 207543420. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 03:42:50,466][1157520] Avg episode reward: [(0, '55.250')] [2024-09-30 03:42:50,471][1157819] Updated weights for policy 0, policy_version 212458 (0.0006) [2024-09-30 03:42:50,975][1157819] Updated weights for policy 0, policy_version 212468 (0.0006) [2024-09-30 03:42:51,499][1157819] Updated weights for policy 0, policy_version 212478 (0.0006) [2024-09-30 03:42:52,042][1157819] Updated weights for policy 0, policy_version 212488 (0.0006) [2024-09-30 03:42:52,590][1157819] Updated weights for policy 0, policy_version 212498 (0.0006) [2024-09-30 03:42:53,101][1157819] Updated weights for policy 0, policy_version 212508 (0.0006) [2024-09-30 03:42:53,653][1157819] Updated weights for policy 0, policy_version 212518 (0.0006) [2024-09-30 03:42:54,213][1157819] Updated weights for policy 0, policy_version 212528 (0.0006) [2024-09-30 03:42:54,752][1157819] Updated weights for policy 0, policy_version 212538 (0.0006) [2024-09-30 03:42:55,279][1157819] Updated weights for policy 0, policy_version 212548 (0.0006) [2024-09-30 03:42:55,466][1157520] Fps is (10 sec: 78643.5, 60 sec: 77277.9, 300 sec: 74616.6). Total num frames: 870608896. Throughput: 0: 19726.6. Samples: 207602120. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 03:42:55,466][1157520] Avg episode reward: [(0, '56.341')] [2024-09-30 03:42:55,889][1157819] Updated weights for policy 0, policy_version 212558 (0.0006) [2024-09-30 03:42:56,461][1157819] Updated weights for policy 0, policy_version 212568 (0.0006) [2024-09-30 03:42:56,969][1157819] Updated weights for policy 0, policy_version 212578 (0.0006) [2024-09-30 03:42:57,541][1157819] Updated weights for policy 0, policy_version 212588 (0.0006) [2024-09-30 03:42:58,088][1157819] Updated weights for policy 0, policy_version 212598 (0.0006) [2024-09-30 03:42:58,631][1157819] Updated weights for policy 0, policy_version 212608 (0.0006) [2024-09-30 03:42:59,163][1157819] Updated weights for policy 0, policy_version 212618 (0.0006) [2024-09-30 03:42:59,707][1157819] Updated weights for policy 0, policy_version 212628 (0.0006) [2024-09-30 03:43:00,274][1157819] Updated weights for policy 0, policy_version 212638 (0.0006) [2024-09-30 03:43:00,466][1157520] Fps is (10 sec: 75365.6, 60 sec: 77550.8, 300 sec: 74783.3). Total num frames: 870977536. Throughput: 0: 19656.2. Samples: 207713708. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 03:43:00,466][1157520] Avg episode reward: [(0, '55.514')] [2024-09-30 03:43:00,844][1157819] Updated weights for policy 0, policy_version 212648 (0.0006) [2024-09-30 03:43:01,379][1157819] Updated weights for policy 0, policy_version 212658 (0.0006) [2024-09-30 03:43:01,944][1157819] Updated weights for policy 0, policy_version 212668 (0.0006) [2024-09-30 03:43:02,448][1157819] Updated weights for policy 0, policy_version 212678 (0.0006) [2024-09-30 03:43:03,023][1157819] Updated weights for policy 0, policy_version 212688 (0.0006) [2024-09-30 03:43:03,564][1157819] Updated weights for policy 0, policy_version 212698 (0.0007) [2024-09-30 03:43:04,140][1157819] Updated weights for policy 0, policy_version 212708 (0.0006) [2024-09-30 03:43:04,686][1157819] Updated weights for policy 0, policy_version 212718 (0.0006) [2024-09-30 03:43:05,247][1157819] Updated weights for policy 0, policy_version 212728 (0.0006) [2024-09-30 03:43:05,466][1157520] Fps is (10 sec: 73728.0, 60 sec: 77824.1, 300 sec: 74936.0). Total num frames: 871346176. Throughput: 0: 19597.0. Samples: 207824940. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 03:43:05,466][1157520] Avg episode reward: [(0, '58.094')] [2024-09-30 03:43:05,796][1157819] Updated weights for policy 0, policy_version 212738 (0.0006) [2024-09-30 03:43:06,353][1157819] Updated weights for policy 0, policy_version 212748 (0.0006) [2024-09-30 03:43:06,894][1157819] Updated weights for policy 0, policy_version 212758 (0.0006) [2024-09-30 03:43:07,457][1157819] Updated weights for policy 0, policy_version 212768 (0.0006) [2024-09-30 03:43:07,967][1157819] Updated weights for policy 0, policy_version 212778 (0.0006) [2024-09-30 03:43:08,505][1157819] Updated weights for policy 0, policy_version 212788 (0.0006) [2024-09-30 03:43:09,078][1157819] Updated weights for policy 0, policy_version 212798 (0.0006) [2024-09-30 03:43:09,636][1157819] Updated weights for policy 0, policy_version 212808 (0.0006) [2024-09-30 03:43:10,167][1157819] Updated weights for policy 0, policy_version 212818 (0.0006) [2024-09-30 03:43:10,466][1157520] Fps is (10 sec: 74547.7, 60 sec: 77755.7, 300 sec: 75061.0). Total num frames: 871723008. Throughput: 0: 19578.1. Samples: 207881076. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 03:43:10,466][1157520] Avg episode reward: [(0, '54.938')] [2024-09-30 03:43:10,730][1157819] Updated weights for policy 0, policy_version 212828 (0.0006) [2024-09-30 03:43:11,281][1157819] Updated weights for policy 0, policy_version 212838 (0.0006) [2024-09-30 03:43:11,859][1157819] Updated weights for policy 0, policy_version 212848 (0.0006) [2024-09-30 03:43:12,378][1157819] Updated weights for policy 0, policy_version 212858 (0.0006) [2024-09-30 03:43:12,941][1157819] Updated weights for policy 0, policy_version 212868 (0.0006) [2024-09-30 03:43:13,496][1157819] Updated weights for policy 0, policy_version 212878 (0.0006) [2024-09-30 03:43:14,047][1157819] Updated weights for policy 0, policy_version 212888 (0.0006) [2024-09-30 03:43:14,569][1157819] Updated weights for policy 0, policy_version 212898 (0.0006) [2024-09-30 03:43:15,109][1157819] Updated weights for policy 0, policy_version 212908 (0.0006) [2024-09-30 03:43:15,466][1157520] Fps is (10 sec: 74956.5, 60 sec: 77550.8, 300 sec: 75172.0). Total num frames: 872095744. Throughput: 0: 19427.4. Samples: 207992636. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 03:43:15,466][1157520] Avg episode reward: [(0, '53.468')] [2024-09-30 03:43:15,664][1157819] Updated weights for policy 0, policy_version 212918 (0.0006) [2024-09-30 03:43:16,183][1157819] Updated weights for policy 0, policy_version 212928 (0.0006) [2024-09-30 03:43:16,737][1157819] Updated weights for policy 0, policy_version 212938 (0.0006) [2024-09-30 03:43:17,266][1157819] Updated weights for policy 0, policy_version 212948 (0.0006) [2024-09-30 03:43:17,808][1157819] Updated weights for policy 0, policy_version 212958 (0.0006) [2024-09-30 03:43:18,351][1157819] Updated weights for policy 0, policy_version 212968 (0.0006) [2024-09-30 03:43:18,923][1157819] Updated weights for policy 0, policy_version 212978 (0.0006) [2024-09-30 03:43:19,440][1157819] Updated weights for policy 0, policy_version 212988 (0.0006) [2024-09-30 03:43:19,960][1157819] Updated weights for policy 0, policy_version 212998 (0.0006) [2024-09-30 03:43:20,101][1157736] Signal inference workers to stop experience collection... (14800 times) [2024-09-30 03:43:20,101][1157736] Signal inference workers to resume experience collection... (14800 times) [2024-09-30 03:43:20,105][1157819] InferenceWorker_p0-w0: stopping experience collection (14800 times) [2024-09-30 03:43:20,105][1157819] InferenceWorker_p0-w0: resuming experience collection (14800 times) [2024-09-30 03:43:20,466][1157520] Fps is (10 sec: 75366.4, 60 sec: 77550.9, 300 sec: 75338.6). Total num frames: 872476672. Throughput: 0: 19249.2. Samples: 208106268. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 03:43:20,466][1157520] Avg episode reward: [(0, '56.600')] [2024-09-30 03:43:20,477][1157819] Updated weights for policy 0, policy_version 213008 (0.0006) [2024-09-30 03:43:20,997][1157819] Updated weights for policy 0, policy_version 213018 (0.0006) [2024-09-30 03:43:21,538][1157819] Updated weights for policy 0, policy_version 213028 (0.0006) [2024-09-30 03:43:22,046][1157819] Updated weights for policy 0, policy_version 213038 (0.0006) [2024-09-30 03:43:22,587][1157819] Updated weights for policy 0, policy_version 213048 (0.0006) [2024-09-30 03:43:23,114][1157819] Updated weights for policy 0, policy_version 213058 (0.0006) [2024-09-30 03:43:23,695][1157819] Updated weights for policy 0, policy_version 213068 (0.0006) [2024-09-30 03:43:24,219][1157819] Updated weights for policy 0, policy_version 213078 (0.0006) [2024-09-30 03:43:24,713][1157819] Updated weights for policy 0, policy_version 213088 (0.0006) [2024-09-30 03:43:25,219][1157819] Updated weights for policy 0, policy_version 213098 (0.0006) [2024-09-30 03:43:25,466][1157520] Fps is (10 sec: 77004.5, 60 sec: 77619.1, 300 sec: 75602.4). Total num frames: 872865792. Throughput: 0: 19203.4. Samples: 208164824. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 03:43:25,466][1157520] Avg episode reward: [(0, '58.845')] [2024-09-30 03:43:25,743][1157819] Updated weights for policy 0, policy_version 213108 (0.0006) [2024-09-30 03:43:26,248][1157819] Updated weights for policy 0, policy_version 213118 (0.0006) [2024-09-30 03:43:26,762][1157819] Updated weights for policy 0, policy_version 213128 (0.0006) [2024-09-30 03:43:27,258][1157819] Updated weights for policy 0, policy_version 213138 (0.0006) [2024-09-30 03:43:27,772][1157819] Updated weights for policy 0, policy_version 213148 (0.0006) [2024-09-30 03:43:28,311][1157819] Updated weights for policy 0, policy_version 213158 (0.0006) [2024-09-30 03:43:28,822][1157819] Updated weights for policy 0, policy_version 213168 (0.0006) [2024-09-30 03:43:29,331][1157819] Updated weights for policy 0, policy_version 213178 (0.0006) [2024-09-30 03:43:29,829][1157819] Updated weights for policy 0, policy_version 213188 (0.0006) [2024-09-30 03:43:30,349][1157819] Updated weights for policy 0, policy_version 213198 (0.0006) [2024-09-30 03:43:30,466][1157520] Fps is (10 sec: 79052.7, 60 sec: 77619.2, 300 sec: 75866.3). Total num frames: 873267200. Throughput: 0: 19149.3. Samples: 208283528. Policy #0 lag: (min: 0.0, avg: 1.6, max: 4.0) [2024-09-30 03:43:30,466][1157520] Avg episode reward: [(0, '53.516')] [2024-09-30 03:43:30,857][1157819] Updated weights for policy 0, policy_version 213208 (0.0006) [2024-09-30 03:43:31,357][1157819] Updated weights for policy 0, policy_version 213218 (0.0006) [2024-09-30 03:43:31,861][1157819] Updated weights for policy 0, policy_version 213228 (0.0006) [2024-09-30 03:43:32,372][1157819] Updated weights for policy 0, policy_version 213238 (0.0006) [2024-09-30 03:43:32,879][1157819] Updated weights for policy 0, policy_version 213248 (0.0006) [2024-09-30 03:43:33,391][1157819] Updated weights for policy 0, policy_version 213258 (0.0006) [2024-09-30 03:43:33,890][1157819] Updated weights for policy 0, policy_version 213268 (0.0006) [2024-09-30 03:43:34,380][1157819] Updated weights for policy 0, policy_version 213278 (0.0006) [2024-09-30 03:43:34,947][1157819] Updated weights for policy 0, policy_version 213288 (0.0006) [2024-09-30 03:43:35,446][1157819] Updated weights for policy 0, policy_version 213298 (0.0006) [2024-09-30 03:43:35,466][1157520] Fps is (10 sec: 80282.1, 60 sec: 77619.1, 300 sec: 76102.3). Total num frames: 873668608. Throughput: 0: 19126.7. Samples: 208404124. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:43:35,466][1157520] Avg episode reward: [(0, '54.673')] [2024-09-30 03:43:35,983][1157819] Updated weights for policy 0, policy_version 213308 (0.0006) [2024-09-30 03:43:36,510][1157819] Updated weights for policy 0, policy_version 213318 (0.0007) [2024-09-30 03:43:37,008][1157819] Updated weights for policy 0, policy_version 213328 (0.0006) [2024-09-30 03:43:37,542][1157819] Updated weights for policy 0, policy_version 213338 (0.0006) [2024-09-30 03:43:38,106][1157819] Updated weights for policy 0, policy_version 213348 (0.0006) [2024-09-30 03:43:38,633][1157819] Updated weights for policy 0, policy_version 213358 (0.0006) [2024-09-30 03:43:39,162][1157819] Updated weights for policy 0, policy_version 213368 (0.0006) [2024-09-30 03:43:39,720][1157819] Updated weights for policy 0, policy_version 213378 (0.0006) [2024-09-30 03:43:40,241][1157819] Updated weights for policy 0, policy_version 213388 (0.0006) [2024-09-30 03:43:40,466][1157520] Fps is (10 sec: 78234.2, 60 sec: 77209.6, 300 sec: 76227.3). Total num frames: 874049536. Throughput: 0: 19116.5. Samples: 208462360. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:43:40,466][1157520] Avg episode reward: [(0, '55.433')] [2024-09-30 03:43:40,764][1157819] Updated weights for policy 0, policy_version 213398 (0.0006) [2024-09-30 03:43:41,306][1157819] Updated weights for policy 0, policy_version 213408 (0.0006) [2024-09-30 03:43:41,856][1157819] Updated weights for policy 0, policy_version 213418 (0.0006) [2024-09-30 03:43:42,407][1157819] Updated weights for policy 0, policy_version 213428 (0.0006) [2024-09-30 03:43:42,964][1157819] Updated weights for policy 0, policy_version 213438 (0.0006) [2024-09-30 03:43:43,516][1157819] Updated weights for policy 0, policy_version 213448 (0.0006) [2024-09-30 03:43:44,015][1157819] Updated weights for policy 0, policy_version 213458 (0.0006) [2024-09-30 03:43:44,572][1157819] Updated weights for policy 0, policy_version 213468 (0.0006) [2024-09-30 03:43:45,132][1157819] Updated weights for policy 0, policy_version 213478 (0.0006) [2024-09-30 03:43:45,466][1157520] Fps is (10 sec: 76185.7, 60 sec: 76800.0, 300 sec: 76255.0). Total num frames: 874430464. Throughput: 0: 19166.1. Samples: 208576180. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:43:45,466][1157520] Avg episode reward: [(0, '56.933')] [2024-09-30 03:43:45,689][1157819] Updated weights for policy 0, policy_version 213488 (0.0006) [2024-09-30 03:43:46,219][1157819] Updated weights for policy 0, policy_version 213498 (0.0006) [2024-09-30 03:43:46,784][1157819] Updated weights for policy 0, policy_version 213508 (0.0006) [2024-09-30 03:43:47,299][1157819] Updated weights for policy 0, policy_version 213518 (0.0006) [2024-09-30 03:43:47,895][1157819] Updated weights for policy 0, policy_version 213528 (0.0006) [2024-09-30 03:43:48,393][1157819] Updated weights for policy 0, policy_version 213538 (0.0006) [2024-09-30 03:43:48,948][1157819] Updated weights for policy 0, policy_version 213548 (0.0006) [2024-09-30 03:43:49,502][1157819] Updated weights for policy 0, policy_version 213558 (0.0006) [2024-09-30 03:43:50,024][1157819] Updated weights for policy 0, policy_version 213568 (0.0006) [2024-09-30 03:43:50,466][1157520] Fps is (10 sec: 75775.7, 60 sec: 76390.4, 300 sec: 76338.3). Total num frames: 874807296. Throughput: 0: 19201.3. Samples: 208689000. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:43:50,466][1157520] Avg episode reward: [(0, '54.445')] [2024-09-30 03:43:50,590][1157819] Updated weights for policy 0, policy_version 213578 (0.0006) [2024-09-30 03:43:51,125][1157819] Updated weights for policy 0, policy_version 213588 (0.0006) [2024-09-30 03:43:51,619][1157819] Updated weights for policy 0, policy_version 213598 (0.0006) [2024-09-30 03:43:52,133][1157819] Updated weights for policy 0, policy_version 213608 (0.0006) [2024-09-30 03:43:52,651][1157819] Updated weights for policy 0, policy_version 213618 (0.0006) [2024-09-30 03:43:53,198][1157819] Updated weights for policy 0, policy_version 213628 (0.0006) [2024-09-30 03:43:53,708][1157819] Updated weights for policy 0, policy_version 213638 (0.0006) [2024-09-30 03:43:54,206][1157819] Updated weights for policy 0, policy_version 213648 (0.0006) [2024-09-30 03:43:54,729][1157819] Updated weights for policy 0, policy_version 213658 (0.0006) [2024-09-30 03:43:55,267][1157819] Updated weights for policy 0, policy_version 213668 (0.0006) [2024-09-30 03:43:55,466][1157520] Fps is (10 sec: 77004.8, 60 sec: 76526.9, 300 sec: 76463.3). Total num frames: 875200512. Throughput: 0: 19248.0. Samples: 208747236. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:43:55,466][1157520] Avg episode reward: [(0, '54.796')] [2024-09-30 03:43:55,788][1157819] Updated weights for policy 0, policy_version 213678 (0.0006) [2024-09-30 03:43:56,276][1157819] Updated weights for policy 0, policy_version 213688 (0.0006) [2024-09-30 03:43:56,779][1157819] Updated weights for policy 0, policy_version 213698 (0.0006) [2024-09-30 03:43:57,319][1157819] Updated weights for policy 0, policy_version 213708 (0.0006) [2024-09-30 03:43:57,839][1157819] Updated weights for policy 0, policy_version 213718 (0.0006) [2024-09-30 03:43:58,360][1157819] Updated weights for policy 0, policy_version 213728 (0.0006) [2024-09-30 03:43:58,856][1157819] Updated weights for policy 0, policy_version 213738 (0.0006) [2024-09-30 03:43:59,386][1157819] Updated weights for policy 0, policy_version 213748 (0.0006) [2024-09-30 03:43:59,902][1157819] Updated weights for policy 0, policy_version 213758 (0.0006) [2024-09-30 03:44:00,393][1157819] Updated weights for policy 0, policy_version 213768 (0.0006) [2024-09-30 03:44:00,466][1157520] Fps is (10 sec: 78643.2, 60 sec: 76936.7, 300 sec: 76491.0). Total num frames: 875593728. Throughput: 0: 19410.3. Samples: 208866100. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:44:00,466][1157520] Avg episode reward: [(0, '56.803')] [2024-09-30 03:44:00,932][1157819] Updated weights for policy 0, policy_version 213778 (0.0006) [2024-09-30 03:44:01,434][1157819] Updated weights for policy 0, policy_version 213788 (0.0006) [2024-09-30 03:44:01,962][1157819] Updated weights for policy 0, policy_version 213798 (0.0006) [2024-09-30 03:44:02,446][1157819] Updated weights for policy 0, policy_version 213808 (0.0006) [2024-09-30 03:44:02,953][1157819] Updated weights for policy 0, policy_version 213818 (0.0006) [2024-09-30 03:44:03,478][1157819] Updated weights for policy 0, policy_version 213828 (0.0006) [2024-09-30 03:44:04,010][1157819] Updated weights for policy 0, policy_version 213838 (0.0006) [2024-09-30 03:44:04,521][1157819] Updated weights for policy 0, policy_version 213848 (0.0006) [2024-09-30 03:44:05,011][1157819] Updated weights for policy 0, policy_version 213858 (0.0006) [2024-09-30 03:44:05,466][1157520] Fps is (10 sec: 79461.5, 60 sec: 77482.5, 300 sec: 76560.4). Total num frames: 875995136. Throughput: 0: 19545.4. Samples: 208985812. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:44:05,466][1157520] Avg episode reward: [(0, '57.650')] [2024-09-30 03:44:05,473][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000213866_875995136.pth... [2024-09-30 03:44:05,509][1157736] Signal inference workers to stop experience collection... (14850 times) [2024-09-30 03:44:05,511][1157819] InferenceWorker_p0-w0: stopping experience collection (14850 times) [2024-09-30 03:44:05,522][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000209369_857575424.pth [2024-09-30 03:44:05,541][1157736] Signal inference workers to resume experience collection... (14850 times) [2024-09-30 03:44:05,541][1157819] InferenceWorker_p0-w0: resuming experience collection (14850 times) [2024-09-30 03:44:05,555][1157819] Updated weights for policy 0, policy_version 213868 (0.0006) [2024-09-30 03:44:06,086][1157819] Updated weights for policy 0, policy_version 213878 (0.0006) [2024-09-30 03:44:06,618][1157819] Updated weights for policy 0, policy_version 213888 (0.0006) [2024-09-30 03:44:07,123][1157819] Updated weights for policy 0, policy_version 213898 (0.0006) [2024-09-30 03:44:07,617][1157819] Updated weights for policy 0, policy_version 213908 (0.0006) [2024-09-30 03:44:08,129][1157819] Updated weights for policy 0, policy_version 213918 (0.0006) [2024-09-30 03:44:08,697][1157819] Updated weights for policy 0, policy_version 213928 (0.0006) [2024-09-30 03:44:09,206][1157819] Updated weights for policy 0, policy_version 213938 (0.0006) [2024-09-30 03:44:09,713][1157819] Updated weights for policy 0, policy_version 213948 (0.0006) [2024-09-30 03:44:10,200][1157819] Updated weights for policy 0, policy_version 213958 (0.0006) [2024-09-30 03:44:10,466][1157520] Fps is (10 sec: 79872.0, 60 sec: 77824.0, 300 sec: 76643.8). Total num frames: 876392448. Throughput: 0: 19552.2. Samples: 209044672. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:44:10,466][1157520] Avg episode reward: [(0, '55.757')] [2024-09-30 03:44:10,687][1157819] Updated weights for policy 0, policy_version 213968 (0.0006) [2024-09-30 03:44:11,211][1157819] Updated weights for policy 0, policy_version 213978 (0.0006) [2024-09-30 03:44:11,793][1157819] Updated weights for policy 0, policy_version 213988 (0.0006) [2024-09-30 03:44:12,321][1157819] Updated weights for policy 0, policy_version 213998 (0.0006) [2024-09-30 03:44:12,857][1157819] Updated weights for policy 0, policy_version 214008 (0.0006) [2024-09-30 03:44:13,417][1157819] Updated weights for policy 0, policy_version 214018 (0.0006) [2024-09-30 03:44:13,977][1157819] Updated weights for policy 0, policy_version 214028 (0.0006) [2024-09-30 03:44:14,531][1157819] Updated weights for policy 0, policy_version 214038 (0.0006) [2024-09-30 03:44:15,106][1157819] Updated weights for policy 0, policy_version 214048 (0.0006) [2024-09-30 03:44:15,466][1157520] Fps is (10 sec: 77415.3, 60 sec: 77892.3, 300 sec: 76699.3). Total num frames: 876769280. Throughput: 0: 19509.7. Samples: 209161464. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:44:15,466][1157520] Avg episode reward: [(0, '58.292')] [2024-09-30 03:44:15,616][1157819] Updated weights for policy 0, policy_version 214058 (0.0006) [2024-09-30 03:44:16,182][1157819] Updated weights for policy 0, policy_version 214068 (0.0006) [2024-09-30 03:44:16,731][1157819] Updated weights for policy 0, policy_version 214078 (0.0006) [2024-09-30 03:44:17,308][1157819] Updated weights for policy 0, policy_version 214088 (0.0006) [2024-09-30 03:44:17,854][1157819] Updated weights for policy 0, policy_version 214098 (0.0006) [2024-09-30 03:44:18,436][1157819] Updated weights for policy 0, policy_version 214108 (0.0006) [2024-09-30 03:44:18,968][1157819] Updated weights for policy 0, policy_version 214118 (0.0006) [2024-09-30 03:44:19,504][1157819] Updated weights for policy 0, policy_version 214128 (0.0006) [2024-09-30 03:44:20,058][1157819] Updated weights for policy 0, policy_version 214138 (0.0006) [2024-09-30 03:44:20,466][1157520] Fps is (10 sec: 74137.5, 60 sec: 77619.2, 300 sec: 76699.3). Total num frames: 877133824. Throughput: 0: 19290.1. Samples: 209272176. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:44:20,466][1157520] Avg episode reward: [(0, '57.466')] [2024-09-30 03:44:20,608][1157819] Updated weights for policy 0, policy_version 214148 (0.0006) [2024-09-30 03:44:21,156][1157819] Updated weights for policy 0, policy_version 214158 (0.0006) [2024-09-30 03:44:21,704][1157819] Updated weights for policy 0, policy_version 214168 (0.0006) [2024-09-30 03:44:22,261][1157819] Updated weights for policy 0, policy_version 214178 (0.0006) [2024-09-30 03:44:22,830][1157819] Updated weights for policy 0, policy_version 214188 (0.0006) [2024-09-30 03:44:23,346][1157819] Updated weights for policy 0, policy_version 214198 (0.0006) [2024-09-30 03:44:23,917][1157819] Updated weights for policy 0, policy_version 214208 (0.0006) [2024-09-30 03:44:24,472][1157819] Updated weights for policy 0, policy_version 214218 (0.0006) [2024-09-30 03:44:25,048][1157819] Updated weights for policy 0, policy_version 214228 (0.0006) [2024-09-30 03:44:25,466][1157520] Fps is (10 sec: 74137.4, 60 sec: 77414.5, 300 sec: 76699.3). Total num frames: 877510656. Throughput: 0: 19240.9. Samples: 209328204. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:44:25,466][1157520] Avg episode reward: [(0, '54.439')] [2024-09-30 03:44:25,586][1157819] Updated weights for policy 0, policy_version 214238 (0.0006) [2024-09-30 03:44:26,136][1157819] Updated weights for policy 0, policy_version 214248 (0.0006) [2024-09-30 03:44:26,683][1157819] Updated weights for policy 0, policy_version 214258 (0.0006) [2024-09-30 03:44:27,252][1157819] Updated weights for policy 0, policy_version 214268 (0.0006) [2024-09-30 03:44:27,796][1157819] Updated weights for policy 0, policy_version 214278 (0.0006) [2024-09-30 03:44:28,365][1157819] Updated weights for policy 0, policy_version 214288 (0.0006) [2024-09-30 03:44:28,942][1157819] Updated weights for policy 0, policy_version 214298 (0.0007) [2024-09-30 03:44:29,563][1157819] Updated weights for policy 0, policy_version 214308 (0.0006) [2024-09-30 03:44:30,114][1157819] Updated weights for policy 0, policy_version 214318 (0.0006) [2024-09-30 03:44:30,466][1157520] Fps is (10 sec: 73317.4, 60 sec: 76663.3, 300 sec: 76643.8). Total num frames: 877867008. Throughput: 0: 19165.3. Samples: 209438620. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:44:30,466][1157520] Avg episode reward: [(0, '56.310')] [2024-09-30 03:44:30,688][1157819] Updated weights for policy 0, policy_version 214328 (0.0006) [2024-09-30 03:44:31,306][1157819] Updated weights for policy 0, policy_version 214338 (0.0006) [2024-09-30 03:44:31,872][1157819] Updated weights for policy 0, policy_version 214348 (0.0006) [2024-09-30 03:44:32,459][1157819] Updated weights for policy 0, policy_version 214358 (0.0006) [2024-09-30 03:44:33,059][1157819] Updated weights for policy 0, policy_version 214368 (0.0006) [2024-09-30 03:44:33,653][1157819] Updated weights for policy 0, policy_version 214378 (0.0006) [2024-09-30 03:44:34,133][1157736] Signal inference workers to stop experience collection... (14900 times) [2024-09-30 03:44:34,136][1157819] InferenceWorker_p0-w0: stopping experience collection (14900 times) [2024-09-30 03:44:34,139][1157736] Signal inference workers to resume experience collection... (14900 times) [2024-09-30 03:44:34,140][1157819] InferenceWorker_p0-w0: resuming experience collection (14900 times) [2024-09-30 03:44:34,273][1157819] Updated weights for policy 0, policy_version 214388 (0.0006) [2024-09-30 03:44:34,824][1157819] Updated weights for policy 0, policy_version 214398 (0.0006) [2024-09-30 03:44:35,382][1157819] Updated weights for policy 0, policy_version 214408 (0.0006) [2024-09-30 03:44:35,466][1157520] Fps is (10 sec: 70860.2, 60 sec: 75844.1, 300 sec: 76546.6). Total num frames: 878219264. Throughput: 0: 18979.3. Samples: 209543072. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:44:35,466][1157520] Avg episode reward: [(0, '57.560')] [2024-09-30 03:44:35,902][1157819] Updated weights for policy 0, policy_version 214418 (0.0006) [2024-09-30 03:44:36,486][1157819] Updated weights for policy 0, policy_version 214428 (0.0006) [2024-09-30 03:44:37,044][1157819] Updated weights for policy 0, policy_version 214438 (0.0006) [2024-09-30 03:44:37,599][1157819] Updated weights for policy 0, policy_version 214448 (0.0006) [2024-09-30 03:44:38,105][1157819] Updated weights for policy 0, policy_version 214458 (0.0006) [2024-09-30 03:44:38,685][1157819] Updated weights for policy 0, policy_version 214468 (0.0006) [2024-09-30 03:44:39,205][1157819] Updated weights for policy 0, policy_version 214478 (0.0006) [2024-09-30 03:44:39,790][1157819] Updated weights for policy 0, policy_version 214488 (0.0006) [2024-09-30 03:44:40,355][1157819] Updated weights for policy 0, policy_version 214498 (0.0006) [2024-09-30 03:44:40,466][1157520] Fps is (10 sec: 72090.5, 60 sec: 75639.4, 300 sec: 76477.2). Total num frames: 878587904. Throughput: 0: 18924.7. Samples: 209598848. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:44:40,466][1157520] Avg episode reward: [(0, '56.352')] [2024-09-30 03:44:40,947][1157819] Updated weights for policy 0, policy_version 214508 (0.0006) [2024-09-30 03:44:41,472][1157819] Updated weights for policy 0, policy_version 214518 (0.0006) [2024-09-30 03:44:42,016][1157819] Updated weights for policy 0, policy_version 214528 (0.0006) [2024-09-30 03:44:42,594][1157819] Updated weights for policy 0, policy_version 214538 (0.0006) [2024-09-30 03:44:43,248][1157819] Updated weights for policy 0, policy_version 214548 (0.0006) [2024-09-30 03:44:43,887][1157819] Updated weights for policy 0, policy_version 214558 (0.0006) [2024-09-30 03:44:44,541][1157819] Updated weights for policy 0, policy_version 214568 (0.0006) [2024-09-30 03:44:45,158][1157819] Updated weights for policy 0, policy_version 214578 (0.0006) [2024-09-30 03:44:45,466][1157520] Fps is (10 sec: 71270.4, 60 sec: 75024.9, 300 sec: 76296.6). Total num frames: 878931968. Throughput: 0: 18664.0. Samples: 209705984. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 03:44:45,466][1157520] Avg episode reward: [(0, '56.761')] [2024-09-30 03:44:45,772][1157819] Updated weights for policy 0, policy_version 214588 (0.0006) [2024-09-30 03:44:46,352][1157819] Updated weights for policy 0, policy_version 214598 (0.0006) [2024-09-30 03:44:46,944][1157819] Updated weights for policy 0, policy_version 214608 (0.0006) [2024-09-30 03:44:47,549][1157819] Updated weights for policy 0, policy_version 214618 (0.0006) [2024-09-30 03:44:48,146][1157819] Updated weights for policy 0, policy_version 214628 (0.0006) [2024-09-30 03:44:48,743][1157819] Updated weights for policy 0, policy_version 214638 (0.0006) [2024-09-30 03:44:49,348][1157819] Updated weights for policy 0, policy_version 214648 (0.0006) [2024-09-30 03:44:49,945][1157819] Updated weights for policy 0, policy_version 214658 (0.0006) [2024-09-30 03:44:50,466][1157520] Fps is (10 sec: 68403.4, 60 sec: 74410.7, 300 sec: 76102.3). Total num frames: 879271936. Throughput: 0: 18247.3. Samples: 209806936. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 03:44:50,466][1157520] Avg episode reward: [(0, '56.360')] [2024-09-30 03:44:50,527][1157819] Updated weights for policy 0, policy_version 214668 (0.0006) [2024-09-30 03:44:50,773][1157736] Signal inference workers to stop experience collection... (14950 times) [2024-09-30 03:44:50,773][1157736] Signal inference workers to resume experience collection... (14950 times) [2024-09-30 03:44:50,777][1157819] InferenceWorker_p0-w0: stopping experience collection (14950 times) [2024-09-30 03:44:50,779][1157819] InferenceWorker_p0-w0: resuming experience collection (14950 times) [2024-09-30 03:44:51,210][1157819] Updated weights for policy 0, policy_version 214678 (0.0006) [2024-09-30 03:44:51,750][1157819] Updated weights for policy 0, policy_version 214688 (0.0006) [2024-09-30 03:44:52,339][1157819] Updated weights for policy 0, policy_version 214698 (0.0006) [2024-09-30 03:44:52,957][1157819] Updated weights for policy 0, policy_version 214708 (0.0006) [2024-09-30 03:44:53,530][1157819] Updated weights for policy 0, policy_version 214718 (0.0006) [2024-09-30 03:44:54,253][1157819] Updated weights for policy 0, policy_version 214728 (0.0006) [2024-09-30 03:44:54,828][1157819] Updated weights for policy 0, policy_version 214738 (0.0006) [2024-09-30 03:44:55,466][1157520] Fps is (10 sec: 67175.4, 60 sec: 73386.7, 300 sec: 75949.6). Total num frames: 879603712. Throughput: 0: 18070.3. Samples: 209857836. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 03:44:55,466][1157520] Avg episode reward: [(0, '55.656')] [2024-09-30 03:44:55,478][1157819] Updated weights for policy 0, policy_version 214748 (0.0006) [2024-09-30 03:44:56,067][1157819] Updated weights for policy 0, policy_version 214758 (0.0006) [2024-09-30 03:44:56,706][1157819] Updated weights for policy 0, policy_version 214768 (0.0006) [2024-09-30 03:44:57,276][1157819] Updated weights for policy 0, policy_version 214778 (0.0006) [2024-09-30 03:44:57,786][1157819] Updated weights for policy 0, policy_version 214788 (0.0006) [2024-09-30 03:44:58,354][1157819] Updated weights for policy 0, policy_version 214798 (0.0006) [2024-09-30 03:44:58,890][1157819] Updated weights for policy 0, policy_version 214808 (0.0006) [2024-09-30 03:44:59,455][1157819] Updated weights for policy 0, policy_version 214818 (0.0006) [2024-09-30 03:45:00,020][1157819] Updated weights for policy 0, policy_version 214828 (0.0006) [2024-09-30 03:45:00,466][1157520] Fps is (10 sec: 69631.3, 60 sec: 72908.7, 300 sec: 75907.9). Total num frames: 879968256. Throughput: 0: 17775.3. Samples: 209961352. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 03:45:00,466][1157520] Avg episode reward: [(0, '56.746')] [2024-09-30 03:45:00,581][1157819] Updated weights for policy 0, policy_version 214838 (0.0006) [2024-09-30 03:45:01,155][1157819] Updated weights for policy 0, policy_version 214848 (0.0006) [2024-09-30 03:45:01,672][1157819] Updated weights for policy 0, policy_version 214858 (0.0006) [2024-09-30 03:45:02,190][1157819] Updated weights for policy 0, policy_version 214868 (0.0006) [2024-09-30 03:45:02,693][1157819] Updated weights for policy 0, policy_version 214878 (0.0006) [2024-09-30 03:45:03,207][1157819] Updated weights for policy 0, policy_version 214888 (0.0006) [2024-09-30 03:45:03,725][1157819] Updated weights for policy 0, policy_version 214898 (0.0006) [2024-09-30 03:45:04,235][1157819] Updated weights for policy 0, policy_version 214908 (0.0006) [2024-09-30 03:45:04,740][1157819] Updated weights for policy 0, policy_version 214918 (0.0006) [2024-09-30 03:45:05,235][1157819] Updated weights for policy 0, policy_version 214928 (0.0006) [2024-09-30 03:45:05,466][1157520] Fps is (10 sec: 75365.1, 60 sec: 72704.0, 300 sec: 75935.6). Total num frames: 880357376. Throughput: 0: 17883.9. Samples: 210076952. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 03:45:05,466][1157520] Avg episode reward: [(0, '56.623')] [2024-09-30 03:45:05,764][1157819] Updated weights for policy 0, policy_version 214938 (0.0006) [2024-09-30 03:45:06,293][1157819] Updated weights for policy 0, policy_version 214948 (0.0006) [2024-09-30 03:45:06,794][1157819] Updated weights for policy 0, policy_version 214958 (0.0006) [2024-09-30 03:45:07,308][1157819] Updated weights for policy 0, policy_version 214968 (0.0006) [2024-09-30 03:45:07,809][1157819] Updated weights for policy 0, policy_version 214978 (0.0006) [2024-09-30 03:45:08,306][1157819] Updated weights for policy 0, policy_version 214988 (0.0006) [2024-09-30 03:45:08,811][1157819] Updated weights for policy 0, policy_version 214998 (0.0006) [2024-09-30 03:45:09,340][1157819] Updated weights for policy 0, policy_version 215008 (0.0006) [2024-09-30 03:45:09,874][1157819] Updated weights for policy 0, policy_version 215018 (0.0006) [2024-09-30 03:45:10,396][1157819] Updated weights for policy 0, policy_version 215028 (0.0006) [2024-09-30 03:45:10,466][1157520] Fps is (10 sec: 79053.7, 60 sec: 72772.3, 300 sec: 76005.1). Total num frames: 880758784. Throughput: 0: 17970.4. Samples: 210136872. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 03:45:10,466][1157520] Avg episode reward: [(0, '54.891')] [2024-09-30 03:45:10,888][1157819] Updated weights for policy 0, policy_version 215038 (0.0006) [2024-09-30 03:45:11,433][1157819] Updated weights for policy 0, policy_version 215048 (0.0006) [2024-09-30 03:45:11,939][1157819] Updated weights for policy 0, policy_version 215058 (0.0006) [2024-09-30 03:45:12,429][1157819] Updated weights for policy 0, policy_version 215068 (0.0006) [2024-09-30 03:45:12,950][1157819] Updated weights for policy 0, policy_version 215078 (0.0006) [2024-09-30 03:45:13,463][1157819] Updated weights for policy 0, policy_version 215088 (0.0006) [2024-09-30 03:45:13,955][1157819] Updated weights for policy 0, policy_version 215098 (0.0006) [2024-09-30 03:45:14,464][1157819] Updated weights for policy 0, policy_version 215108 (0.0006) [2024-09-30 03:45:14,965][1157819] Updated weights for policy 0, policy_version 215118 (0.0006) [2024-09-30 03:45:15,426][1157819] Updated weights for policy 0, policy_version 215128 (0.0006) [2024-09-30 03:45:15,466][1157520] Fps is (10 sec: 80692.1, 60 sec: 73250.1, 300 sec: 76088.4). Total num frames: 881164288. Throughput: 0: 18182.7. Samples: 210256840. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 03:45:15,466][1157520] Avg episode reward: [(0, '54.011')] [2024-09-30 03:45:15,929][1157819] Updated weights for policy 0, policy_version 215138 (0.0006) [2024-09-30 03:45:16,428][1157819] Updated weights for policy 0, policy_version 215148 (0.0006) [2024-09-30 03:45:16,926][1157819] Updated weights for policy 0, policy_version 215158 (0.0006) [2024-09-30 03:45:17,415][1157819] Updated weights for policy 0, policy_version 215168 (0.0006) [2024-09-30 03:45:17,911][1157819] Updated weights for policy 0, policy_version 215178 (0.0006) [2024-09-30 03:45:18,406][1157819] Updated weights for policy 0, policy_version 215188 (0.0006) [2024-09-30 03:45:18,867][1157819] Updated weights for policy 0, policy_version 215198 (0.0006) [2024-09-30 03:45:19,356][1157819] Updated weights for policy 0, policy_version 215208 (0.0006) [2024-09-30 03:45:19,854][1157819] Updated weights for policy 0, policy_version 215218 (0.0006) [2024-09-30 03:45:20,346][1157819] Updated weights for policy 0, policy_version 215228 (0.0006) [2024-09-30 03:45:20,466][1157520] Fps is (10 sec: 82329.4, 60 sec: 74137.6, 300 sec: 76213.4). Total num frames: 881582080. Throughput: 0: 18643.5. Samples: 210382028. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 03:45:20,466][1157520] Avg episode reward: [(0, '56.593')] [2024-09-30 03:45:20,822][1157819] Updated weights for policy 0, policy_version 215238 (0.0006) [2024-09-30 03:45:21,336][1157819] Updated weights for policy 0, policy_version 215248 (0.0006) [2024-09-30 03:45:21,831][1157819] Updated weights for policy 0, policy_version 215258 (0.0006) [2024-09-30 03:45:22,316][1157819] Updated weights for policy 0, policy_version 215268 (0.0006) [2024-09-30 03:45:22,821][1157819] Updated weights for policy 0, policy_version 215278 (0.0006) [2024-09-30 03:45:23,317][1157819] Updated weights for policy 0, policy_version 215288 (0.0006) [2024-09-30 03:45:23,821][1157819] Updated weights for policy 0, policy_version 215298 (0.0006) [2024-09-30 03:45:24,319][1157819] Updated weights for policy 0, policy_version 215308 (0.0006) [2024-09-30 03:45:24,818][1157819] Updated weights for policy 0, policy_version 215318 (0.0006) [2024-09-30 03:45:25,304][1157819] Updated weights for policy 0, policy_version 215328 (0.0006) [2024-09-30 03:45:25,466][1157520] Fps is (10 sec: 83149.0, 60 sec: 74752.0, 300 sec: 76310.6). Total num frames: 881995776. Throughput: 0: 18780.4. Samples: 210443968. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 03:45:25,466][1157520] Avg episode reward: [(0, '54.210')] [2024-09-30 03:45:25,794][1157819] Updated weights for policy 0, policy_version 215338 (0.0006) [2024-09-30 03:45:26,298][1157819] Updated weights for policy 0, policy_version 215348 (0.0006) [2024-09-30 03:45:26,757][1157819] Updated weights for policy 0, policy_version 215358 (0.0006) [2024-09-30 03:45:27,264][1157819] Updated weights for policy 0, policy_version 215368 (0.0006) [2024-09-30 03:45:27,768][1157819] Updated weights for policy 0, policy_version 215378 (0.0006) [2024-09-30 03:45:28,241][1157819] Updated weights for policy 0, policy_version 215388 (0.0006) [2024-09-30 03:45:28,738][1157819] Updated weights for policy 0, policy_version 215398 (0.0006) [2024-09-30 03:45:29,224][1157819] Updated weights for policy 0, policy_version 215408 (0.0006) [2024-09-30 03:45:29,690][1157819] Updated weights for policy 0, policy_version 215418 (0.0006) [2024-09-30 03:45:30,081][1157736] Signal inference workers to stop experience collection... (15000 times) [2024-09-30 03:45:30,082][1157736] Signal inference workers to resume experience collection... (15000 times) [2024-09-30 03:45:30,086][1157819] InferenceWorker_p0-w0: stopping experience collection (15000 times) [2024-09-30 03:45:30,086][1157819] InferenceWorker_p0-w0: resuming experience collection (15000 times) [2024-09-30 03:45:30,140][1157819] Updated weights for policy 0, policy_version 215428 (0.0006) [2024-09-30 03:45:30,466][1157520] Fps is (10 sec: 83559.3, 60 sec: 75844.6, 300 sec: 76449.4). Total num frames: 882417664. Throughput: 0: 19167.3. Samples: 210568508. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 03:45:30,466][1157520] Avg episode reward: [(0, '56.935')] [2024-09-30 03:45:30,614][1157819] Updated weights for policy 0, policy_version 215438 (0.0006) [2024-09-30 03:45:31,099][1157819] Updated weights for policy 0, policy_version 215448 (0.0006) [2024-09-30 03:45:31,600][1157819] Updated weights for policy 0, policy_version 215458 (0.0006) [2024-09-30 03:45:32,123][1157819] Updated weights for policy 0, policy_version 215468 (0.0006) [2024-09-30 03:45:32,666][1157819] Updated weights for policy 0, policy_version 215478 (0.0006) [2024-09-30 03:45:33,180][1157819] Updated weights for policy 0, policy_version 215488 (0.0006) [2024-09-30 03:45:33,722][1157819] Updated weights for policy 0, policy_version 215498 (0.0006) [2024-09-30 03:45:34,261][1157819] Updated weights for policy 0, policy_version 215508 (0.0006) [2024-09-30 03:45:34,802][1157819] Updated weights for policy 0, policy_version 215518 (0.0006) [2024-09-30 03:45:35,337][1157819] Updated weights for policy 0, policy_version 215528 (0.0006) [2024-09-30 03:45:35,466][1157520] Fps is (10 sec: 81510.5, 60 sec: 76527.1, 300 sec: 76477.2). Total num frames: 882810880. Throughput: 0: 19626.8. Samples: 210690144. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 03:45:35,466][1157520] Avg episode reward: [(0, '55.598')] [2024-09-30 03:45:35,870][1157819] Updated weights for policy 0, policy_version 215538 (0.0006) [2024-09-30 03:45:36,400][1157819] Updated weights for policy 0, policy_version 215548 (0.0006) [2024-09-30 03:45:36,955][1157819] Updated weights for policy 0, policy_version 215558 (0.0006) [2024-09-30 03:45:37,455][1157819] Updated weights for policy 0, policy_version 215568 (0.0006) [2024-09-30 03:45:37,954][1157819] Updated weights for policy 0, policy_version 215578 (0.0006) [2024-09-30 03:45:38,499][1157819] Updated weights for policy 0, policy_version 215588 (0.0006) [2024-09-30 03:45:39,055][1157819] Updated weights for policy 0, policy_version 215598 (0.0006) [2024-09-30 03:45:39,567][1157819] Updated weights for policy 0, policy_version 215608 (0.0006) [2024-09-30 03:45:40,095][1157819] Updated weights for policy 0, policy_version 215618 (0.0006) [2024-09-30 03:45:40,466][1157520] Fps is (10 sec: 77823.2, 60 sec: 76800.0, 300 sec: 76407.8). Total num frames: 883195904. Throughput: 0: 19790.8. Samples: 210748424. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 03:45:40,466][1157520] Avg episode reward: [(0, '56.314')] [2024-09-30 03:45:40,624][1157819] Updated weights for policy 0, policy_version 215628 (0.0006) [2024-09-30 03:45:41,167][1157819] Updated weights for policy 0, policy_version 215638 (0.0006) [2024-09-30 03:45:41,691][1157819] Updated weights for policy 0, policy_version 215648 (0.0006) [2024-09-30 03:45:42,159][1157819] Updated weights for policy 0, policy_version 215658 (0.0006) [2024-09-30 03:45:42,688][1157819] Updated weights for policy 0, policy_version 215668 (0.0006) [2024-09-30 03:45:43,194][1157819] Updated weights for policy 0, policy_version 215678 (0.0006) [2024-09-30 03:45:43,701][1157819] Updated weights for policy 0, policy_version 215688 (0.0006) [2024-09-30 03:45:44,227][1157819] Updated weights for policy 0, policy_version 215698 (0.0006) [2024-09-30 03:45:44,785][1157819] Updated weights for policy 0, policy_version 215708 (0.0006) [2024-09-30 03:45:45,276][1157819] Updated weights for policy 0, policy_version 215718 (0.0006) [2024-09-30 03:45:45,466][1157520] Fps is (10 sec: 78233.6, 60 sec: 77687.6, 300 sec: 76504.9). Total num frames: 883593216. Throughput: 0: 20106.4. Samples: 210866136. Policy #0 lag: (min: 0.0, avg: 1.6, max: 5.0) [2024-09-30 03:45:45,466][1157520] Avg episode reward: [(0, '56.850')] [2024-09-30 03:45:45,791][1157819] Updated weights for policy 0, policy_version 215728 (0.0006) [2024-09-30 03:45:46,293][1157819] Updated weights for policy 0, policy_version 215738 (0.0006) [2024-09-30 03:45:46,833][1157819] Updated weights for policy 0, policy_version 215748 (0.0006) [2024-09-30 03:45:47,334][1157819] Updated weights for policy 0, policy_version 215758 (0.0006) [2024-09-30 03:45:47,863][1157819] Updated weights for policy 0, policy_version 215768 (0.0006) [2024-09-30 03:45:48,353][1157819] Updated weights for policy 0, policy_version 215778 (0.0006) [2024-09-30 03:45:48,869][1157819] Updated weights for policy 0, policy_version 215788 (0.0006) [2024-09-30 03:45:49,360][1157819] Updated weights for policy 0, policy_version 215798 (0.0006) [2024-09-30 03:45:49,849][1157819] Updated weights for policy 0, policy_version 215808 (0.0006) [2024-09-30 03:45:50,346][1157819] Updated weights for policy 0, policy_version 215818 (0.0006) [2024-09-30 03:45:50,466][1157520] Fps is (10 sec: 79871.9, 60 sec: 78711.4, 300 sec: 76602.1). Total num frames: 883994624. Throughput: 0: 20197.3. Samples: 210985828. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 03:45:50,466][1157520] Avg episode reward: [(0, '54.748')] [2024-09-30 03:45:50,833][1157819] Updated weights for policy 0, policy_version 215828 (0.0006) [2024-09-30 03:45:51,328][1157819] Updated weights for policy 0, policy_version 215838 (0.0006) [2024-09-30 03:45:51,835][1157819] Updated weights for policy 0, policy_version 215848 (0.0006) [2024-09-30 03:45:52,304][1157736] Signal inference workers to stop experience collection... (15050 times) [2024-09-30 03:45:52,305][1157736] Signal inference workers to resume experience collection... (15050 times) [2024-09-30 03:45:52,308][1157819] InferenceWorker_p0-w0: stopping experience collection (15050 times) [2024-09-30 03:45:52,308][1157819] InferenceWorker_p0-w0: resuming experience collection (15050 times) [2024-09-30 03:45:52,319][1157819] Updated weights for policy 0, policy_version 215858 (0.0006) [2024-09-30 03:45:52,825][1157819] Updated weights for policy 0, policy_version 215868 (0.0006) [2024-09-30 03:45:53,323][1157819] Updated weights for policy 0, policy_version 215878 (0.0006) [2024-09-30 03:45:53,851][1157819] Updated weights for policy 0, policy_version 215888 (0.0006) [2024-09-30 03:45:54,375][1157819] Updated weights for policy 0, policy_version 215898 (0.0006) [2024-09-30 03:45:54,895][1157819] Updated weights for policy 0, policy_version 215908 (0.0006) [2024-09-30 03:45:55,408][1157819] Updated weights for policy 0, policy_version 215918 (0.0006) [2024-09-30 03:45:55,466][1157520] Fps is (10 sec: 81100.0, 60 sec: 80008.4, 300 sec: 76713.2). Total num frames: 884404224. Throughput: 0: 20249.2. Samples: 211048088. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 03:45:55,466][1157520] Avg episode reward: [(0, '57.554')] [2024-09-30 03:45:55,935][1157819] Updated weights for policy 0, policy_version 215928 (0.0006) [2024-09-30 03:45:56,441][1157819] Updated weights for policy 0, policy_version 215938 (0.0006) [2024-09-30 03:45:56,951][1157819] Updated weights for policy 0, policy_version 215948 (0.0006) [2024-09-30 03:45:57,458][1157819] Updated weights for policy 0, policy_version 215958 (0.0006) [2024-09-30 03:45:57,997][1157819] Updated weights for policy 0, policy_version 215968 (0.0006) [2024-09-30 03:45:58,522][1157819] Updated weights for policy 0, policy_version 215978 (0.0006) [2024-09-30 03:45:59,014][1157819] Updated weights for policy 0, policy_version 215988 (0.0006) [2024-09-30 03:45:59,526][1157819] Updated weights for policy 0, policy_version 215998 (0.0006) [2024-09-30 03:46:00,047][1157819] Updated weights for policy 0, policy_version 216008 (0.0006) [2024-09-30 03:46:00,466][1157520] Fps is (10 sec: 80691.0, 60 sec: 80554.7, 300 sec: 76754.9). Total num frames: 884801536. Throughput: 0: 20231.6. Samples: 211167260. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 03:46:00,466][1157520] Avg episode reward: [(0, '55.909')] [2024-09-30 03:46:00,551][1157819] Updated weights for policy 0, policy_version 216018 (0.0006) [2024-09-30 03:46:01,094][1157819] Updated weights for policy 0, policy_version 216028 (0.0006) [2024-09-30 03:46:01,632][1157819] Updated weights for policy 0, policy_version 216038 (0.0006) [2024-09-30 03:46:02,134][1157819] Updated weights for policy 0, policy_version 216048 (0.0006) [2024-09-30 03:46:02,639][1157819] Updated weights for policy 0, policy_version 216058 (0.0006) [2024-09-30 03:46:03,147][1157819] Updated weights for policy 0, policy_version 216068 (0.0006) [2024-09-30 03:46:03,712][1157819] Updated weights for policy 0, policy_version 216078 (0.0006) [2024-09-30 03:46:04,227][1157819] Updated weights for policy 0, policy_version 216088 (0.0006) [2024-09-30 03:46:04,725][1157819] Updated weights for policy 0, policy_version 216098 (0.0006) [2024-09-30 03:46:05,270][1157819] Updated weights for policy 0, policy_version 216108 (0.0006) [2024-09-30 03:46:05,466][1157520] Fps is (10 sec: 78643.3, 60 sec: 80554.8, 300 sec: 76754.8). Total num frames: 885190656. Throughput: 0: 20069.1. Samples: 211285140. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 03:46:05,466][1157520] Avg episode reward: [(0, '54.261')] [2024-09-30 03:46:05,471][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000216111_885190656.pth... [2024-09-30 03:46:05,520][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000211592_866680832.pth [2024-09-30 03:46:05,842][1157819] Updated weights for policy 0, policy_version 216118 (0.0006) [2024-09-30 03:46:06,450][1157819] Updated weights for policy 0, policy_version 216128 (0.0006) [2024-09-30 03:46:06,962][1157819] Updated weights for policy 0, policy_version 216138 (0.0006) [2024-09-30 03:46:07,464][1157819] Updated weights for policy 0, policy_version 216148 (0.0006) [2024-09-30 03:46:07,970][1157819] Updated weights for policy 0, policy_version 216158 (0.0006) [2024-09-30 03:46:08,526][1157819] Updated weights for policy 0, policy_version 216168 (0.0006) [2024-09-30 03:46:09,033][1157819] Updated weights for policy 0, policy_version 216178 (0.0006) [2024-09-30 03:46:09,543][1157819] Updated weights for policy 0, policy_version 216188 (0.0006) [2024-09-30 03:46:10,063][1157819] Updated weights for policy 0, policy_version 216198 (0.0006) [2024-09-30 03:46:10,466][1157520] Fps is (10 sec: 77414.4, 60 sec: 80281.5, 300 sec: 76699.4). Total num frames: 885575680. Throughput: 0: 19952.2. Samples: 211341816. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 03:46:10,466][1157520] Avg episode reward: [(0, '54.841')] [2024-09-30 03:46:10,592][1157819] Updated weights for policy 0, policy_version 216208 (0.0006) [2024-09-30 03:46:11,083][1157819] Updated weights for policy 0, policy_version 216218 (0.0006) [2024-09-30 03:46:11,593][1157819] Updated weights for policy 0, policy_version 216228 (0.0006) [2024-09-30 03:46:12,116][1157819] Updated weights for policy 0, policy_version 216238 (0.0006) [2024-09-30 03:46:12,629][1157819] Updated weights for policy 0, policy_version 216248 (0.0006) [2024-09-30 03:46:13,162][1157819] Updated weights for policy 0, policy_version 216258 (0.0006) [2024-09-30 03:46:13,674][1157819] Updated weights for policy 0, policy_version 216268 (0.0006) [2024-09-30 03:46:14,202][1157819] Updated weights for policy 0, policy_version 216278 (0.0006) [2024-09-30 03:46:14,714][1157819] Updated weights for policy 0, policy_version 216288 (0.0006) [2024-09-30 03:46:15,233][1157819] Updated weights for policy 0, policy_version 216298 (0.0006) [2024-09-30 03:46:15,466][1157520] Fps is (10 sec: 78643.7, 60 sec: 80213.4, 300 sec: 76713.2). Total num frames: 885977088. Throughput: 0: 19825.0. Samples: 211460636. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 03:46:15,466][1157520] Avg episode reward: [(0, '54.578')] [2024-09-30 03:46:15,723][1157819] Updated weights for policy 0, policy_version 216308 (0.0006) [2024-09-30 03:46:16,250][1157819] Updated weights for policy 0, policy_version 216318 (0.0006) [2024-09-30 03:46:16,772][1157819] Updated weights for policy 0, policy_version 216328 (0.0006) [2024-09-30 03:46:17,274][1157819] Updated weights for policy 0, policy_version 216338 (0.0006) [2024-09-30 03:46:17,775][1157819] Updated weights for policy 0, policy_version 216348 (0.0006) [2024-09-30 03:46:18,278][1157819] Updated weights for policy 0, policy_version 216358 (0.0006) [2024-09-30 03:46:18,792][1157819] Updated weights for policy 0, policy_version 216368 (0.0006) [2024-09-30 03:46:19,298][1157819] Updated weights for policy 0, policy_version 216378 (0.0006) [2024-09-30 03:46:19,829][1157819] Updated weights for policy 0, policy_version 216388 (0.0006) [2024-09-30 03:46:20,358][1157819] Updated weights for policy 0, policy_version 216398 (0.0006) [2024-09-30 03:46:20,466][1157520] Fps is (10 sec: 79872.2, 60 sec: 79872.0, 300 sec: 76727.1). Total num frames: 886374400. Throughput: 0: 19785.7. Samples: 211580500. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 03:46:20,466][1157520] Avg episode reward: [(0, '56.167')] [2024-09-30 03:46:20,621][1157736] Signal inference workers to stop experience collection... (15100 times) [2024-09-30 03:46:20,621][1157736] Signal inference workers to resume experience collection... (15100 times) [2024-09-30 03:46:20,627][1157819] InferenceWorker_p0-w0: stopping experience collection (15100 times) [2024-09-30 03:46:20,627][1157819] InferenceWorker_p0-w0: resuming experience collection (15100 times) [2024-09-30 03:46:20,895][1157819] Updated weights for policy 0, policy_version 216408 (0.0006) [2024-09-30 03:46:21,386][1157819] Updated weights for policy 0, policy_version 216418 (0.0006) [2024-09-30 03:46:21,898][1157819] Updated weights for policy 0, policy_version 216428 (0.0006) [2024-09-30 03:46:22,431][1157819] Updated weights for policy 0, policy_version 216438 (0.0006) [2024-09-30 03:46:22,936][1157819] Updated weights for policy 0, policy_version 216448 (0.0006) [2024-09-30 03:46:23,474][1157819] Updated weights for policy 0, policy_version 216458 (0.0006) [2024-09-30 03:46:23,975][1157819] Updated weights for policy 0, policy_version 216468 (0.0006) [2024-09-30 03:46:24,472][1157819] Updated weights for policy 0, policy_version 216478 (0.0006) [2024-09-30 03:46:25,038][1157819] Updated weights for policy 0, policy_version 216488 (0.0006) [2024-09-30 03:46:25,466][1157520] Fps is (10 sec: 79052.5, 60 sec: 79530.6, 300 sec: 76782.6). Total num frames: 886767616. Throughput: 0: 19808.0. Samples: 211639784. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 03:46:25,466][1157520] Avg episode reward: [(0, '55.880')] [2024-09-30 03:46:25,545][1157819] Updated weights for policy 0, policy_version 216498 (0.0006) [2024-09-30 03:46:26,076][1157819] Updated weights for policy 0, policy_version 216508 (0.0006) [2024-09-30 03:46:26,562][1157819] Updated weights for policy 0, policy_version 216518 (0.0006) [2024-09-30 03:46:27,068][1157819] Updated weights for policy 0, policy_version 216528 (0.0006) [2024-09-30 03:46:27,580][1157819] Updated weights for policy 0, policy_version 216538 (0.0006) [2024-09-30 03:46:28,131][1157819] Updated weights for policy 0, policy_version 216548 (0.0006) [2024-09-30 03:46:28,640][1157819] Updated weights for policy 0, policy_version 216558 (0.0006) [2024-09-30 03:46:29,147][1157819] Updated weights for policy 0, policy_version 216568 (0.0006) [2024-09-30 03:46:29,648][1157819] Updated weights for policy 0, policy_version 216578 (0.0006) [2024-09-30 03:46:30,157][1157819] Updated weights for policy 0, policy_version 216588 (0.0006) [2024-09-30 03:46:30,466][1157520] Fps is (10 sec: 79053.1, 60 sec: 79121.0, 300 sec: 76852.1). Total num frames: 887164928. Throughput: 0: 19831.9. Samples: 211758572. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 03:46:30,466][1157520] Avg episode reward: [(0, '56.487')] [2024-09-30 03:46:30,660][1157819] Updated weights for policy 0, policy_version 216598 (0.0006) [2024-09-30 03:46:31,213][1157819] Updated weights for policy 0, policy_version 216608 (0.0006) [2024-09-30 03:46:31,709][1157819] Updated weights for policy 0, policy_version 216618 (0.0006) [2024-09-30 03:46:32,212][1157819] Updated weights for policy 0, policy_version 216628 (0.0006) [2024-09-30 03:46:32,706][1157819] Updated weights for policy 0, policy_version 216638 (0.0006) [2024-09-30 03:46:33,264][1157819] Updated weights for policy 0, policy_version 216648 (0.0006) [2024-09-30 03:46:33,765][1157819] Updated weights for policy 0, policy_version 216658 (0.0006) [2024-09-30 03:46:34,268][1157819] Updated weights for policy 0, policy_version 216668 (0.0006) [2024-09-30 03:46:34,786][1157819] Updated weights for policy 0, policy_version 216678 (0.0006) [2024-09-30 03:46:35,265][1157819] Updated weights for policy 0, policy_version 216688 (0.0006) [2024-09-30 03:46:35,466][1157520] Fps is (10 sec: 79871.3, 60 sec: 79257.4, 300 sec: 76963.1). Total num frames: 887566336. Throughput: 0: 19840.4. Samples: 211878648. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 03:46:35,466][1157520] Avg episode reward: [(0, '56.242')] [2024-09-30 03:46:35,757][1157819] Updated weights for policy 0, policy_version 216698 (0.0006) [2024-09-30 03:46:36,236][1157819] Updated weights for policy 0, policy_version 216708 (0.0006) [2024-09-30 03:46:36,729][1157819] Updated weights for policy 0, policy_version 216718 (0.0006) [2024-09-30 03:46:37,229][1157819] Updated weights for policy 0, policy_version 216728 (0.0006) [2024-09-30 03:46:37,850][1157819] Updated weights for policy 0, policy_version 216738 (0.0006) [2024-09-30 03:46:38,449][1157819] Updated weights for policy 0, policy_version 216748 (0.0006) [2024-09-30 03:46:39,004][1157819] Updated weights for policy 0, policy_version 216758 (0.0006) [2024-09-30 03:46:39,602][1157819] Updated weights for policy 0, policy_version 216768 (0.0006) [2024-09-30 03:46:40,161][1157819] Updated weights for policy 0, policy_version 216778 (0.0006) [2024-09-30 03:46:40,466][1157520] Fps is (10 sec: 77823.4, 60 sec: 79121.0, 300 sec: 76977.0). Total num frames: 887943168. Throughput: 0: 19791.5. Samples: 211938704. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 03:46:40,466][1157520] Avg episode reward: [(0, '54.855')] [2024-09-30 03:46:40,707][1157819] Updated weights for policy 0, policy_version 216788 (0.0006) [2024-09-30 03:46:41,267][1157819] Updated weights for policy 0, policy_version 216798 (0.0006) [2024-09-30 03:46:41,789][1157819] Updated weights for policy 0, policy_version 216808 (0.0006) [2024-09-30 03:46:42,348][1157819] Updated weights for policy 0, policy_version 216818 (0.0006) [2024-09-30 03:46:42,909][1157819] Updated weights for policy 0, policy_version 216828 (0.0006) [2024-09-30 03:46:43,467][1157819] Updated weights for policy 0, policy_version 216838 (0.0006) [2024-09-30 03:46:43,999][1157819] Updated weights for policy 0, policy_version 216848 (0.0006) [2024-09-30 03:46:44,530][1157819] Updated weights for policy 0, policy_version 216858 (0.0006) [2024-09-30 03:46:45,065][1157819] Updated weights for policy 0, policy_version 216868 (0.0006) [2024-09-30 03:46:45,466][1157520] Fps is (10 sec: 75367.0, 60 sec: 78779.6, 300 sec: 76990.9). Total num frames: 888320000. Throughput: 0: 19580.4. Samples: 212048380. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 03:46:45,466][1157520] Avg episode reward: [(0, '53.464')] [2024-09-30 03:46:45,604][1157819] Updated weights for policy 0, policy_version 216878 (0.0006) [2024-09-30 03:46:46,103][1157819] Updated weights for policy 0, policy_version 216888 (0.0006) [2024-09-30 03:46:46,642][1157819] Updated weights for policy 0, policy_version 216898 (0.0006) [2024-09-30 03:46:47,213][1157819] Updated weights for policy 0, policy_version 216908 (0.0006) [2024-09-30 03:46:47,747][1157819] Updated weights for policy 0, policy_version 216918 (0.0006) [2024-09-30 03:46:48,237][1157819] Updated weights for policy 0, policy_version 216928 (0.0006) [2024-09-30 03:46:48,797][1157819] Updated weights for policy 0, policy_version 216938 (0.0006) [2024-09-30 03:46:49,315][1157819] Updated weights for policy 0, policy_version 216948 (0.0006) [2024-09-30 03:46:49,873][1157819] Updated weights for policy 0, policy_version 216958 (0.0006) [2024-09-30 03:46:50,369][1157819] Updated weights for policy 0, policy_version 216968 (0.0006) [2024-09-30 03:46:50,466][1157520] Fps is (10 sec: 76185.8, 60 sec: 78506.6, 300 sec: 77060.3). Total num frames: 888705024. Throughput: 0: 19527.0. Samples: 212163852. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 03:46:50,466][1157520] Avg episode reward: [(0, '53.464')] [2024-09-30 03:46:50,902][1157819] Updated weights for policy 0, policy_version 216978 (0.0006) [2024-09-30 03:46:51,420][1157819] Updated weights for policy 0, policy_version 216988 (0.0006) [2024-09-30 03:46:51,946][1157819] Updated weights for policy 0, policy_version 216998 (0.0006) [2024-09-30 03:46:52,534][1157819] Updated weights for policy 0, policy_version 217008 (0.0006) [2024-09-30 03:46:53,092][1157819] Updated weights for policy 0, policy_version 217018 (0.0006) [2024-09-30 03:46:53,648][1157819] Updated weights for policy 0, policy_version 217028 (0.0006) [2024-09-30 03:46:54,204][1157819] Updated weights for policy 0, policy_version 217038 (0.0006) [2024-09-30 03:46:54,755][1157819] Updated weights for policy 0, policy_version 217048 (0.0006) [2024-09-30 03:46:55,358][1157819] Updated weights for policy 0, policy_version 217058 (0.0006) [2024-09-30 03:46:55,466][1157520] Fps is (10 sec: 75775.9, 60 sec: 77892.3, 300 sec: 77129.7). Total num frames: 889077760. Throughput: 0: 19530.7. Samples: 212220700. Policy #0 lag: (min: 1.0, avg: 2.6, max: 6.0) [2024-09-30 03:46:55,466][1157520] Avg episode reward: [(0, '56.157')] [2024-09-30 03:46:55,904][1157819] Updated weights for policy 0, policy_version 217068 (0.0006) [2024-09-30 03:46:56,467][1157819] Updated weights for policy 0, policy_version 217078 (0.0006) [2024-09-30 03:46:57,032][1157819] Updated weights for policy 0, policy_version 217088 (0.0006) [2024-09-30 03:46:57,604][1157819] Updated weights for policy 0, policy_version 217098 (0.0006) [2024-09-30 03:46:58,121][1157819] Updated weights for policy 0, policy_version 217108 (0.0006) [2024-09-30 03:46:58,647][1157819] Updated weights for policy 0, policy_version 217118 (0.0006) [2024-09-30 03:46:59,187][1157819] Updated weights for policy 0, policy_version 217128 (0.0006) [2024-09-30 03:46:59,704][1157819] Updated weights for policy 0, policy_version 217138 (0.0006) [2024-09-30 03:47:00,215][1157819] Updated weights for policy 0, policy_version 217148 (0.0006) [2024-09-30 03:47:00,466][1157520] Fps is (10 sec: 75366.5, 60 sec: 77619.2, 300 sec: 77227.0). Total num frames: 889458688. Throughput: 0: 19347.6. Samples: 212331276. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:47:00,466][1157520] Avg episode reward: [(0, '55.551')] [2024-09-30 03:47:00,714][1157819] Updated weights for policy 0, policy_version 217158 (0.0006) [2024-09-30 03:47:01,218][1157819] Updated weights for policy 0, policy_version 217168 (0.0006) [2024-09-30 03:47:01,714][1157819] Updated weights for policy 0, policy_version 217178 (0.0006) [2024-09-30 03:47:02,204][1157819] Updated weights for policy 0, policy_version 217188 (0.0006) [2024-09-30 03:47:02,699][1157819] Updated weights for policy 0, policy_version 217198 (0.0006) [2024-09-30 03:47:03,195][1157819] Updated weights for policy 0, policy_version 217208 (0.0006) [2024-09-30 03:47:03,668][1157819] Updated weights for policy 0, policy_version 217218 (0.0006) [2024-09-30 03:47:04,156][1157819] Updated weights for policy 0, policy_version 217228 (0.0006) [2024-09-30 03:47:04,628][1157819] Updated weights for policy 0, policy_version 217238 (0.0006) [2024-09-30 03:47:05,130][1157819] Updated weights for policy 0, policy_version 217248 (0.0006) [2024-09-30 03:47:05,466][1157520] Fps is (10 sec: 79462.9, 60 sec: 78028.9, 300 sec: 77338.0). Total num frames: 889872384. Throughput: 0: 19435.0. Samples: 212455076. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:47:05,466][1157520] Avg episode reward: [(0, '55.870')] [2024-09-30 03:47:05,594][1157819] Updated weights for policy 0, policy_version 217258 (0.0006) [2024-09-30 03:47:06,071][1157819] Updated weights for policy 0, policy_version 217268 (0.0006) [2024-09-30 03:47:06,556][1157819] Updated weights for policy 0, policy_version 217278 (0.0006) [2024-09-30 03:47:06,667][1157736] Signal inference workers to stop experience collection... (15150 times) [2024-09-30 03:47:06,668][1157736] Signal inference workers to resume experience collection... (15150 times) [2024-09-30 03:47:06,671][1157819] InferenceWorker_p0-w0: stopping experience collection (15150 times) [2024-09-30 03:47:06,672][1157819] InferenceWorker_p0-w0: resuming experience collection (15150 times) [2024-09-30 03:47:07,014][1157819] Updated weights for policy 0, policy_version 217288 (0.0006) [2024-09-30 03:47:07,513][1157819] Updated weights for policy 0, policy_version 217298 (0.0006) [2024-09-30 03:47:08,056][1157819] Updated weights for policy 0, policy_version 217308 (0.0006) [2024-09-30 03:47:08,567][1157819] Updated weights for policy 0, policy_version 217318 (0.0006) [2024-09-30 03:47:09,072][1157819] Updated weights for policy 0, policy_version 217328 (0.0006) [2024-09-30 03:47:09,573][1157819] Updated weights for policy 0, policy_version 217338 (0.0006) [2024-09-30 03:47:10,074][1157819] Updated weights for policy 0, policy_version 217348 (0.0006) [2024-09-30 03:47:10,466][1157520] Fps is (10 sec: 82739.0, 60 sec: 78506.6, 300 sec: 77435.2). Total num frames: 890286080. Throughput: 0: 19514.7. Samples: 212517944. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:47:10,466][1157520] Avg episode reward: [(0, '56.943')] [2024-09-30 03:47:10,593][1157819] Updated weights for policy 0, policy_version 217358 (0.0006) [2024-09-30 03:47:11,082][1157819] Updated weights for policy 0, policy_version 217368 (0.0006) [2024-09-30 03:47:11,585][1157819] Updated weights for policy 0, policy_version 217378 (0.0006) [2024-09-30 03:47:12,088][1157819] Updated weights for policy 0, policy_version 217388 (0.0006) [2024-09-30 03:47:12,620][1157819] Updated weights for policy 0, policy_version 217398 (0.0006) [2024-09-30 03:47:13,112][1157819] Updated weights for policy 0, policy_version 217408 (0.0006) [2024-09-30 03:47:13,606][1157819] Updated weights for policy 0, policy_version 217418 (0.0006) [2024-09-30 03:47:14,094][1157819] Updated weights for policy 0, policy_version 217428 (0.0006) [2024-09-30 03:47:14,586][1157819] Updated weights for policy 0, policy_version 217438 (0.0006) [2024-09-30 03:47:15,072][1157819] Updated weights for policy 0, policy_version 217448 (0.0006) [2024-09-30 03:47:15,466][1157520] Fps is (10 sec: 82329.0, 60 sec: 78643.1, 300 sec: 77532.4). Total num frames: 890695680. Throughput: 0: 19585.1. Samples: 212639904. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:47:15,466][1157520] Avg episode reward: [(0, '55.714')] [2024-09-30 03:47:15,549][1157819] Updated weights for policy 0, policy_version 217458 (0.0006) [2024-09-30 03:47:16,050][1157819] Updated weights for policy 0, policy_version 217468 (0.0006) [2024-09-30 03:47:16,543][1157819] Updated weights for policy 0, policy_version 217478 (0.0006) [2024-09-30 03:47:17,021][1157819] Updated weights for policy 0, policy_version 217488 (0.0006) [2024-09-30 03:47:17,542][1157819] Updated weights for policy 0, policy_version 217498 (0.0006) [2024-09-30 03:47:18,032][1157819] Updated weights for policy 0, policy_version 217508 (0.0006) [2024-09-30 03:47:18,511][1157819] Updated weights for policy 0, policy_version 217518 (0.0006) [2024-09-30 03:47:19,003][1157819] Updated weights for policy 0, policy_version 217528 (0.0006) [2024-09-30 03:47:19,529][1157819] Updated weights for policy 0, policy_version 217538 (0.0006) [2024-09-30 03:47:20,089][1157819] Updated weights for policy 0, policy_version 217548 (0.0006) [2024-09-30 03:47:20,466][1157520] Fps is (10 sec: 82330.0, 60 sec: 78916.3, 300 sec: 77629.6). Total num frames: 891109376. Throughput: 0: 19671.3. Samples: 212763852. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:47:20,466][1157520] Avg episode reward: [(0, '56.426')] [2024-09-30 03:47:20,583][1157819] Updated weights for policy 0, policy_version 217558 (0.0006) [2024-09-30 03:47:21,082][1157819] Updated weights for policy 0, policy_version 217568 (0.0006) [2024-09-30 03:47:21,603][1157819] Updated weights for policy 0, policy_version 217578 (0.0006) [2024-09-30 03:47:22,182][1157819] Updated weights for policy 0, policy_version 217588 (0.0006) [2024-09-30 03:47:22,735][1157819] Updated weights for policy 0, policy_version 217598 (0.0006) [2024-09-30 03:47:23,248][1157819] Updated weights for policy 0, policy_version 217608 (0.0006) [2024-09-30 03:47:23,826][1157819] Updated weights for policy 0, policy_version 217618 (0.0006) [2024-09-30 03:47:24,369][1157819] Updated weights for policy 0, policy_version 217628 (0.0006) [2024-09-30 03:47:24,893][1157819] Updated weights for policy 0, policy_version 217638 (0.0006) [2024-09-30 03:47:25,391][1157819] Updated weights for policy 0, policy_version 217648 (0.0006) [2024-09-30 03:47:25,466][1157520] Fps is (10 sec: 79462.4, 60 sec: 78711.4, 300 sec: 77560.2). Total num frames: 891490304. Throughput: 0: 19623.3. Samples: 212821752. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:47:25,466][1157520] Avg episode reward: [(0, '56.799')] [2024-09-30 03:47:25,916][1157819] Updated weights for policy 0, policy_version 217658 (0.0006) [2024-09-30 03:47:26,446][1157819] Updated weights for policy 0, policy_version 217668 (0.0006) [2024-09-30 03:47:26,942][1157819] Updated weights for policy 0, policy_version 217678 (0.0006) [2024-09-30 03:47:27,436][1157819] Updated weights for policy 0, policy_version 217688 (0.0006) [2024-09-30 03:47:27,936][1157819] Updated weights for policy 0, policy_version 217698 (0.0006) [2024-09-30 03:47:28,438][1157819] Updated weights for policy 0, policy_version 217708 (0.0006) [2024-09-30 03:47:28,934][1157819] Updated weights for policy 0, policy_version 217718 (0.0007) [2024-09-30 03:47:29,451][1157819] Updated weights for policy 0, policy_version 217728 (0.0006) [2024-09-30 03:47:29,959][1157819] Updated weights for policy 0, policy_version 217738 (0.0006) [2024-09-30 03:47:30,466][1157520] Fps is (10 sec: 78233.1, 60 sec: 78779.6, 300 sec: 77560.2). Total num frames: 891891712. Throughput: 0: 19817.5. Samples: 212940168. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:47:30,466][1157520] Avg episode reward: [(0, '57.654')] [2024-09-30 03:47:30,469][1157819] Updated weights for policy 0, policy_version 217748 (0.0006) [2024-09-30 03:47:30,979][1157819] Updated weights for policy 0, policy_version 217758 (0.0006) [2024-09-30 03:47:31,504][1157819] Updated weights for policy 0, policy_version 217768 (0.0006) [2024-09-30 03:47:32,017][1157819] Updated weights for policy 0, policy_version 217778 (0.0006) [2024-09-30 03:47:32,542][1157819] Updated weights for policy 0, policy_version 217788 (0.0006) [2024-09-30 03:47:33,072][1157819] Updated weights for policy 0, policy_version 217798 (0.0006) [2024-09-30 03:47:33,590][1157819] Updated weights for policy 0, policy_version 217808 (0.0006) [2024-09-30 03:47:34,123][1157819] Updated weights for policy 0, policy_version 217818 (0.0006) [2024-09-30 03:47:34,618][1157819] Updated weights for policy 0, policy_version 217828 (0.0006) [2024-09-30 03:47:35,133][1157819] Updated weights for policy 0, policy_version 217838 (0.0006) [2024-09-30 03:47:35,466][1157520] Fps is (10 sec: 79872.6, 60 sec: 78711.6, 300 sec: 77532.4). Total num frames: 892289024. Throughput: 0: 19894.9. Samples: 213059120. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:47:35,466][1157520] Avg episode reward: [(0, '55.757')] [2024-09-30 03:47:35,704][1157819] Updated weights for policy 0, policy_version 217848 (0.0006) [2024-09-30 03:47:36,200][1157819] Updated weights for policy 0, policy_version 217858 (0.0006) [2024-09-30 03:47:36,715][1157819] Updated weights for policy 0, policy_version 217868 (0.0006) [2024-09-30 03:47:37,231][1157819] Updated weights for policy 0, policy_version 217878 (0.0006) [2024-09-30 03:47:37,748][1157819] Updated weights for policy 0, policy_version 217888 (0.0006) [2024-09-30 03:47:38,249][1157819] Updated weights for policy 0, policy_version 217898 (0.0006) [2024-09-30 03:47:38,761][1157819] Updated weights for policy 0, policy_version 217908 (0.0006) [2024-09-30 03:47:39,315][1157819] Updated weights for policy 0, policy_version 217918 (0.0006) [2024-09-30 03:47:39,817][1157819] Updated weights for policy 0, policy_version 217928 (0.0006) [2024-09-30 03:47:40,327][1157819] Updated weights for policy 0, policy_version 217938 (0.0006) [2024-09-30 03:47:40,466][1157520] Fps is (10 sec: 79053.1, 60 sec: 78984.6, 300 sec: 77490.8). Total num frames: 892682240. Throughput: 0: 19944.2. Samples: 213118188. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:47:40,466][1157520] Avg episode reward: [(0, '55.420')] [2024-09-30 03:47:40,842][1157819] Updated weights for policy 0, policy_version 217948 (0.0006) [2024-09-30 03:47:41,344][1157819] Updated weights for policy 0, policy_version 217958 (0.0006) [2024-09-30 03:47:41,867][1157819] Updated weights for policy 0, policy_version 217968 (0.0006) [2024-09-30 03:47:42,138][1157736] Signal inference workers to stop experience collection... (15200 times) [2024-09-30 03:47:42,142][1157736] Signal inference workers to resume experience collection... (15200 times) [2024-09-30 03:47:42,146][1157819] InferenceWorker_p0-w0: stopping experience collection (15200 times) [2024-09-30 03:47:42,148][1157819] InferenceWorker_p0-w0: resuming experience collection (15200 times) [2024-09-30 03:47:42,403][1157819] Updated weights for policy 0, policy_version 217978 (0.0006) [2024-09-30 03:47:42,945][1157819] Updated weights for policy 0, policy_version 217988 (0.0006) [2024-09-30 03:47:43,474][1157819] Updated weights for policy 0, policy_version 217998 (0.0006) [2024-09-30 03:47:44,022][1157819] Updated weights for policy 0, policy_version 218008 (0.0006) [2024-09-30 03:47:44,542][1157819] Updated weights for policy 0, policy_version 218018 (0.0006) [2024-09-30 03:47:45,053][1157819] Updated weights for policy 0, policy_version 218028 (0.0006) [2024-09-30 03:47:45,466][1157520] Fps is (10 sec: 78233.2, 60 sec: 79189.3, 300 sec: 77449.1). Total num frames: 893071360. Throughput: 0: 20106.5. Samples: 213236068. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:47:45,466][1157520] Avg episode reward: [(0, '57.262')] [2024-09-30 03:47:45,609][1157819] Updated weights for policy 0, policy_version 218038 (0.0006) [2024-09-30 03:47:46,138][1157819] Updated weights for policy 0, policy_version 218048 (0.0006) [2024-09-30 03:47:46,649][1157819] Updated weights for policy 0, policy_version 218058 (0.0006) [2024-09-30 03:47:47,178][1157819] Updated weights for policy 0, policy_version 218068 (0.0006) [2024-09-30 03:47:47,685][1157819] Updated weights for policy 0, policy_version 218078 (0.0006) [2024-09-30 03:47:48,203][1157819] Updated weights for policy 0, policy_version 218088 (0.0006) [2024-09-30 03:47:48,749][1157819] Updated weights for policy 0, policy_version 218098 (0.0006) [2024-09-30 03:47:49,242][1157819] Updated weights for policy 0, policy_version 218108 (0.0006) [2024-09-30 03:47:49,765][1157819] Updated weights for policy 0, policy_version 218118 (0.0006) [2024-09-30 03:47:50,261][1157819] Updated weights for policy 0, policy_version 218128 (0.0006) [2024-09-30 03:47:50,466][1157520] Fps is (10 sec: 78234.7, 60 sec: 79326.1, 300 sec: 77476.9). Total num frames: 893464576. Throughput: 0: 19961.1. Samples: 213353324. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:47:50,466][1157520] Avg episode reward: [(0, '58.746')] [2024-09-30 03:47:50,789][1157819] Updated weights for policy 0, policy_version 218138 (0.0006) [2024-09-30 03:47:51,313][1157819] Updated weights for policy 0, policy_version 218148 (0.0006) [2024-09-30 03:47:51,835][1157819] Updated weights for policy 0, policy_version 218158 (0.0006) [2024-09-30 03:47:52,347][1157819] Updated weights for policy 0, policy_version 218168 (0.0006) [2024-09-30 03:47:52,858][1157819] Updated weights for policy 0, policy_version 218178 (0.0006) [2024-09-30 03:47:53,378][1157819] Updated weights for policy 0, policy_version 218188 (0.0006) [2024-09-30 03:47:53,909][1157819] Updated weights for policy 0, policy_version 218198 (0.0006) [2024-09-30 03:47:54,447][1157819] Updated weights for policy 0, policy_version 218208 (0.0006) [2024-09-30 03:47:54,979][1157819] Updated weights for policy 0, policy_version 218218 (0.0006) [2024-09-30 03:47:55,466][1157520] Fps is (10 sec: 78644.4, 60 sec: 79667.4, 300 sec: 77560.2). Total num frames: 893857792. Throughput: 0: 19882.1. Samples: 213412636. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:47:55,466][1157520] Avg episode reward: [(0, '58.316')] [2024-09-30 03:47:55,484][1157819] Updated weights for policy 0, policy_version 218228 (0.0006) [2024-09-30 03:47:55,989][1157819] Updated weights for policy 0, policy_version 218238 (0.0006) [2024-09-30 03:47:56,509][1157819] Updated weights for policy 0, policy_version 218248 (0.0006) [2024-09-30 03:47:57,018][1157819] Updated weights for policy 0, policy_version 218258 (0.0006) [2024-09-30 03:47:57,516][1157819] Updated weights for policy 0, policy_version 218268 (0.0006) [2024-09-30 03:47:58,039][1157819] Updated weights for policy 0, policy_version 218278 (0.0006) [2024-09-30 03:47:58,543][1157819] Updated weights for policy 0, policy_version 218288 (0.0006) [2024-09-30 03:47:59,073][1157819] Updated weights for policy 0, policy_version 218298 (0.0006) [2024-09-30 03:47:59,630][1157819] Updated weights for policy 0, policy_version 218308 (0.0006) [2024-09-30 03:48:00,189][1157819] Updated weights for policy 0, policy_version 218318 (0.0006) [2024-09-30 03:48:00,466][1157520] Fps is (10 sec: 78642.1, 60 sec: 79872.0, 300 sec: 77643.5). Total num frames: 894251008. Throughput: 0: 19815.2. Samples: 213531588. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:48:00,466][1157520] Avg episode reward: [(0, '55.737')] [2024-09-30 03:48:00,759][1157819] Updated weights for policy 0, policy_version 218328 (0.0006) [2024-09-30 03:48:01,282][1157819] Updated weights for policy 0, policy_version 218338 (0.0006) [2024-09-30 03:48:01,828][1157819] Updated weights for policy 0, policy_version 218348 (0.0006) [2024-09-30 03:48:02,391][1157819] Updated weights for policy 0, policy_version 218358 (0.0006) [2024-09-30 03:48:02,927][1157819] Updated weights for policy 0, policy_version 218368 (0.0006) [2024-09-30 03:48:03,505][1157819] Updated weights for policy 0, policy_version 218378 (0.0006) [2024-09-30 03:48:04,038][1157819] Updated weights for policy 0, policy_version 218388 (0.0006) [2024-09-30 03:48:04,183][1157736] Signal inference workers to stop experience collection... (15250 times) [2024-09-30 03:48:04,185][1157819] InferenceWorker_p0-w0: stopping experience collection (15250 times) [2024-09-30 03:48:04,188][1157736] Signal inference workers to resume experience collection... (15250 times) [2024-09-30 03:48:04,189][1157819] InferenceWorker_p0-w0: resuming experience collection (15250 times) [2024-09-30 03:48:04,578][1157819] Updated weights for policy 0, policy_version 218398 (0.0006) [2024-09-30 03:48:05,129][1157819] Updated weights for policy 0, policy_version 218408 (0.0006) [2024-09-30 03:48:05,466][1157520] Fps is (10 sec: 76593.6, 60 sec: 79189.2, 300 sec: 77629.6). Total num frames: 894623744. Throughput: 0: 19543.1. Samples: 213643292. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:48:05,466][1157520] Avg episode reward: [(0, '57.755')] [2024-09-30 03:48:05,473][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000218414_894623744.pth... [2024-09-30 03:48:05,518][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000213866_875995136.pth [2024-09-30 03:48:05,683][1157819] Updated weights for policy 0, policy_version 218418 (0.0006) [2024-09-30 03:48:06,247][1157819] Updated weights for policy 0, policy_version 218428 (0.0006) [2024-09-30 03:48:06,821][1157819] Updated weights for policy 0, policy_version 218438 (0.0006) [2024-09-30 03:48:07,358][1157819] Updated weights for policy 0, policy_version 218448 (0.0006) [2024-09-30 03:48:07,899][1157819] Updated weights for policy 0, policy_version 218458 (0.0006) [2024-09-30 03:48:08,433][1157819] Updated weights for policy 0, policy_version 218468 (0.0006) [2024-09-30 03:48:08,938][1157819] Updated weights for policy 0, policy_version 218478 (0.0006) [2024-09-30 03:48:09,444][1157819] Updated weights for policy 0, policy_version 218488 (0.0006) [2024-09-30 03:48:09,940][1157819] Updated weights for policy 0, policy_version 218498 (0.0006) [2024-09-30 03:48:10,450][1157819] Updated weights for policy 0, policy_version 218508 (0.0006) [2024-09-30 03:48:10,466][1157520] Fps is (10 sec: 75775.3, 60 sec: 78711.4, 300 sec: 77671.3). Total num frames: 895008768. Throughput: 0: 19496.3. Samples: 213699088. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 03:48:10,466][1157520] Avg episode reward: [(0, '57.189')] [2024-09-30 03:48:10,970][1157819] Updated weights for policy 0, policy_version 218518 (0.0006) [2024-09-30 03:48:11,472][1157819] Updated weights for policy 0, policy_version 218528 (0.0006) [2024-09-30 03:48:11,979][1157819] Updated weights for policy 0, policy_version 218538 (0.0006) [2024-09-30 03:48:12,527][1157819] Updated weights for policy 0, policy_version 218548 (0.0006) [2024-09-30 03:48:13,031][1157819] Updated weights for policy 0, policy_version 218558 (0.0006) [2024-09-30 03:48:13,519][1157819] Updated weights for policy 0, policy_version 218568 (0.0006) [2024-09-30 03:48:14,007][1157819] Updated weights for policy 0, policy_version 218578 (0.0006) [2024-09-30 03:48:14,510][1157819] Updated weights for policy 0, policy_version 218588 (0.0006) [2024-09-30 03:48:14,996][1157819] Updated weights for policy 0, policy_version 218598 (0.0006) [2024-09-30 03:48:15,466][1157520] Fps is (10 sec: 79053.1, 60 sec: 78643.2, 300 sec: 77754.6). Total num frames: 895414272. Throughput: 0: 19531.4. Samples: 213819080. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 03:48:15,466][1157520] Avg episode reward: [(0, '57.709')] [2024-09-30 03:48:15,478][1157819] Updated weights for policy 0, policy_version 218608 (0.0006) [2024-09-30 03:48:15,976][1157819] Updated weights for policy 0, policy_version 218618 (0.0006) [2024-09-30 03:48:16,470][1157819] Updated weights for policy 0, policy_version 218628 (0.0006) [2024-09-30 03:48:16,943][1157819] Updated weights for policy 0, policy_version 218638 (0.0006) [2024-09-30 03:48:17,450][1157819] Updated weights for policy 0, policy_version 218648 (0.0006) [2024-09-30 03:48:17,920][1157819] Updated weights for policy 0, policy_version 218658 (0.0006) [2024-09-30 03:48:18,430][1157819] Updated weights for policy 0, policy_version 218668 (0.0006) [2024-09-30 03:48:18,955][1157819] Updated weights for policy 0, policy_version 218678 (0.0006) [2024-09-30 03:48:19,452][1157819] Updated weights for policy 0, policy_version 218688 (0.0006) [2024-09-30 03:48:19,931][1157819] Updated weights for policy 0, policy_version 218698 (0.0006) [2024-09-30 03:48:20,431][1157819] Updated weights for policy 0, policy_version 218708 (0.0006) [2024-09-30 03:48:20,466][1157520] Fps is (10 sec: 81920.8, 60 sec: 78643.2, 300 sec: 77837.9). Total num frames: 895827968. Throughput: 0: 19657.4. Samples: 213943704. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 03:48:20,466][1157520] Avg episode reward: [(0, '56.726')] [2024-09-30 03:48:20,956][1157819] Updated weights for policy 0, policy_version 218718 (0.0007) [2024-09-30 03:48:21,457][1157819] Updated weights for policy 0, policy_version 218728 (0.0007) [2024-09-30 03:48:21,954][1157819] Updated weights for policy 0, policy_version 218738 (0.0007) [2024-09-30 03:48:22,477][1157819] Updated weights for policy 0, policy_version 218748 (0.0007) [2024-09-30 03:48:22,973][1157819] Updated weights for policy 0, policy_version 218758 (0.0007) [2024-09-30 03:48:23,487][1157819] Updated weights for policy 0, policy_version 218768 (0.0007) [2024-09-30 03:48:23,980][1157819] Updated weights for policy 0, policy_version 218778 (0.0007) [2024-09-30 03:48:24,512][1157819] Updated weights for policy 0, policy_version 218788 (0.0006) [2024-09-30 03:48:25,016][1157819] Updated weights for policy 0, policy_version 218798 (0.0006) [2024-09-30 03:48:25,466][1157520] Fps is (10 sec: 81510.9, 60 sec: 78984.6, 300 sec: 77837.9). Total num frames: 896229376. Throughput: 0: 19691.5. Samples: 214004304. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 03:48:25,466][1157520] Avg episode reward: [(0, '55.189')] [2024-09-30 03:48:25,536][1157819] Updated weights for policy 0, policy_version 218808 (0.0006) [2024-09-30 03:48:26,071][1157819] Updated weights for policy 0, policy_version 218818 (0.0006) [2024-09-30 03:48:26,643][1157819] Updated weights for policy 0, policy_version 218828 (0.0007) [2024-09-30 03:48:27,197][1157819] Updated weights for policy 0, policy_version 218838 (0.0007) [2024-09-30 03:48:27,742][1157819] Updated weights for policy 0, policy_version 218848 (0.0006) [2024-09-30 03:48:28,271][1157819] Updated weights for policy 0, policy_version 218858 (0.0006) [2024-09-30 03:48:28,868][1157819] Updated weights for policy 0, policy_version 218868 (0.0007) [2024-09-30 03:48:29,384][1157819] Updated weights for policy 0, policy_version 218878 (0.0007) [2024-09-30 03:48:29,949][1157819] Updated weights for policy 0, policy_version 218888 (0.0006) [2024-09-30 03:48:30,466][1157520] Fps is (10 sec: 77004.6, 60 sec: 78438.4, 300 sec: 77726.8). Total num frames: 896598016. Throughput: 0: 19641.5. Samples: 214119936. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 03:48:30,466][1157520] Avg episode reward: [(0, '55.619')] [2024-09-30 03:48:30,522][1157819] Updated weights for policy 0, policy_version 218898 (0.0007) [2024-09-30 03:48:31,083][1157819] Updated weights for policy 0, policy_version 218908 (0.0006) [2024-09-30 03:48:31,631][1157819] Updated weights for policy 0, policy_version 218918 (0.0006) [2024-09-30 03:48:32,191][1157819] Updated weights for policy 0, policy_version 218928 (0.0006) [2024-09-30 03:48:32,757][1157819] Updated weights for policy 0, policy_version 218938 (0.0006) [2024-09-30 03:48:33,324][1157819] Updated weights for policy 0, policy_version 218948 (0.0007) [2024-09-30 03:48:33,901][1157819] Updated weights for policy 0, policy_version 218958 (0.0006) [2024-09-30 03:48:34,429][1157819] Updated weights for policy 0, policy_version 218968 (0.0006) [2024-09-30 03:48:34,981][1157819] Updated weights for policy 0, policy_version 218978 (0.0006) [2024-09-30 03:48:35,466][1157520] Fps is (10 sec: 73727.2, 60 sec: 77960.4, 300 sec: 77685.1). Total num frames: 896966656. Throughput: 0: 19477.0. Samples: 214229796. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 03:48:35,466][1157520] Avg episode reward: [(0, '56.489')] [2024-09-30 03:48:35,538][1157819] Updated weights for policy 0, policy_version 218988 (0.0006) [2024-09-30 03:48:36,098][1157819] Updated weights for policy 0, policy_version 218998 (0.0006) [2024-09-30 03:48:36,666][1157819] Updated weights for policy 0, policy_version 219008 (0.0006) [2024-09-30 03:48:37,247][1157819] Updated weights for policy 0, policy_version 219018 (0.0006) [2024-09-30 03:48:37,785][1157819] Updated weights for policy 0, policy_version 219028 (0.0006) [2024-09-30 03:48:38,335][1157819] Updated weights for policy 0, policy_version 219038 (0.0006) [2024-09-30 03:48:38,862][1157819] Updated weights for policy 0, policy_version 219048 (0.0006) [2024-09-30 03:48:39,424][1157819] Updated weights for policy 0, policy_version 219058 (0.0006) [2024-09-30 03:48:39,991][1157819] Updated weights for policy 0, policy_version 219068 (0.0006) [2024-09-30 03:48:40,466][1157520] Fps is (10 sec: 73727.8, 60 sec: 77550.9, 300 sec: 77643.5). Total num frames: 897335296. Throughput: 0: 19385.1. Samples: 214284968. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 03:48:40,466][1157520] Avg episode reward: [(0, '55.945')] [2024-09-30 03:48:40,548][1157819] Updated weights for policy 0, policy_version 219078 (0.0006) [2024-09-30 03:48:41,099][1157819] Updated weights for policy 0, policy_version 219088 (0.0006) [2024-09-30 03:48:41,666][1157819] Updated weights for policy 0, policy_version 219098 (0.0006) [2024-09-30 03:48:42,201][1157819] Updated weights for policy 0, policy_version 219108 (0.0006) [2024-09-30 03:48:42,747][1157819] Updated weights for policy 0, policy_version 219118 (0.0006) [2024-09-30 03:48:43,327][1157819] Updated weights for policy 0, policy_version 219128 (0.0006) [2024-09-30 03:48:43,867][1157819] Updated weights for policy 0, policy_version 219138 (0.0006) [2024-09-30 03:48:44,427][1157819] Updated weights for policy 0, policy_version 219148 (0.0006) [2024-09-30 03:48:44,500][1157736] Signal inference workers to stop experience collection... (15300 times) [2024-09-30 03:48:44,501][1157736] Signal inference workers to resume experience collection... (15300 times) [2024-09-30 03:48:44,504][1157819] InferenceWorker_p0-w0: stopping experience collection (15300 times) [2024-09-30 03:48:44,507][1157819] InferenceWorker_p0-w0: resuming experience collection (15300 times) [2024-09-30 03:48:44,991][1157819] Updated weights for policy 0, policy_version 219158 (0.0006) [2024-09-30 03:48:45,466][1157520] Fps is (10 sec: 74138.1, 60 sec: 77277.9, 300 sec: 77629.6). Total num frames: 897708032. Throughput: 0: 19210.1. Samples: 214396044. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 03:48:45,466][1157520] Avg episode reward: [(0, '57.200')] [2024-09-30 03:48:45,511][1157819] Updated weights for policy 0, policy_version 219168 (0.0006) [2024-09-30 03:48:46,045][1157819] Updated weights for policy 0, policy_version 219178 (0.0006) [2024-09-30 03:48:46,604][1157819] Updated weights for policy 0, policy_version 219188 (0.0006) [2024-09-30 03:48:47,154][1157819] Updated weights for policy 0, policy_version 219198 (0.0006) [2024-09-30 03:48:47,686][1157819] Updated weights for policy 0, policy_version 219208 (0.0006) [2024-09-30 03:48:48,227][1157819] Updated weights for policy 0, policy_version 219218 (0.0006) [2024-09-30 03:48:48,774][1157819] Updated weights for policy 0, policy_version 219228 (0.0006) [2024-09-30 03:48:49,338][1157819] Updated weights for policy 0, policy_version 219238 (0.0006) [2024-09-30 03:48:49,882][1157819] Updated weights for policy 0, policy_version 219248 (0.0006) [2024-09-30 03:48:50,447][1157819] Updated weights for policy 0, policy_version 219258 (0.0006) [2024-09-30 03:48:50,466][1157520] Fps is (10 sec: 74547.4, 60 sec: 76936.3, 300 sec: 77560.2). Total num frames: 898080768. Throughput: 0: 19223.4. Samples: 214508344. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 03:48:50,466][1157520] Avg episode reward: [(0, '57.595')] [2024-09-30 03:48:50,994][1157819] Updated weights for policy 0, policy_version 219268 (0.0006) [2024-09-30 03:48:51,511][1157819] Updated weights for policy 0, policy_version 219278 (0.0006) [2024-09-30 03:48:52,089][1157819] Updated weights for policy 0, policy_version 219288 (0.0006) [2024-09-30 03:48:52,650][1157819] Updated weights for policy 0, policy_version 219298 (0.0006) [2024-09-30 03:48:53,204][1157819] Updated weights for policy 0, policy_version 219308 (0.0007) [2024-09-30 03:48:53,820][1157819] Updated weights for policy 0, policy_version 219318 (0.0007) [2024-09-30 03:48:54,399][1157819] Updated weights for policy 0, policy_version 219328 (0.0006) [2024-09-30 03:48:54,969][1157819] Updated weights for policy 0, policy_version 219338 (0.0006) [2024-09-30 03:48:55,466][1157520] Fps is (10 sec: 73318.3, 60 sec: 76390.2, 300 sec: 77449.1). Total num frames: 898441216. Throughput: 0: 19210.8. Samples: 214563572. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 03:48:55,466][1157520] Avg episode reward: [(0, '58.900')] [2024-09-30 03:48:55,571][1157819] Updated weights for policy 0, policy_version 219348 (0.0006) [2024-09-30 03:48:56,102][1157819] Updated weights for policy 0, policy_version 219358 (0.0006) [2024-09-30 03:48:56,642][1157819] Updated weights for policy 0, policy_version 219368 (0.0006) [2024-09-30 03:48:57,204][1157819] Updated weights for policy 0, policy_version 219378 (0.0006) [2024-09-30 03:48:57,757][1157819] Updated weights for policy 0, policy_version 219388 (0.0006) [2024-09-30 03:48:58,322][1157819] Updated weights for policy 0, policy_version 219398 (0.0006) [2024-09-30 03:48:58,899][1157819] Updated weights for policy 0, policy_version 219408 (0.0006) [2024-09-30 03:48:59,439][1157819] Updated weights for policy 0, policy_version 219418 (0.0006) [2024-09-30 03:49:00,013][1157819] Updated weights for policy 0, policy_version 219428 (0.0006) [2024-09-30 03:49:00,466][1157520] Fps is (10 sec: 72909.2, 60 sec: 75980.8, 300 sec: 77338.1). Total num frames: 898809856. Throughput: 0: 18947.1. Samples: 214671700. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 03:49:00,466][1157520] Avg episode reward: [(0, '56.843')] [2024-09-30 03:49:00,571][1157819] Updated weights for policy 0, policy_version 219438 (0.0006) [2024-09-30 03:49:01,104][1157819] Updated weights for policy 0, policy_version 219448 (0.0006) [2024-09-30 03:49:01,701][1157819] Updated weights for policy 0, policy_version 219458 (0.0006) [2024-09-30 03:49:02,246][1157819] Updated weights for policy 0, policy_version 219468 (0.0006) [2024-09-30 03:49:02,810][1157819] Updated weights for policy 0, policy_version 219478 (0.0006) [2024-09-30 03:49:03,394][1157819] Updated weights for policy 0, policy_version 219488 (0.0006) [2024-09-30 03:49:04,017][1157819] Updated weights for policy 0, policy_version 219498 (0.0006) [2024-09-30 03:49:04,605][1157819] Updated weights for policy 0, policy_version 219508 (0.0006) [2024-09-30 03:49:05,275][1157819] Updated weights for policy 0, policy_version 219518 (0.0006) [2024-09-30 03:49:05,466][1157520] Fps is (10 sec: 71680.4, 60 sec: 75571.3, 300 sec: 77171.4). Total num frames: 899158016. Throughput: 0: 18557.2. Samples: 214778780. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 03:49:05,466][1157520] Avg episode reward: [(0, '55.996')] [2024-09-30 03:49:05,924][1157819] Updated weights for policy 0, policy_version 219528 (0.0006) [2024-09-30 03:49:06,509][1157819] Updated weights for policy 0, policy_version 219538 (0.0006) [2024-09-30 03:49:07,074][1157819] Updated weights for policy 0, policy_version 219548 (0.0006) [2024-09-30 03:49:07,685][1157819] Updated weights for policy 0, policy_version 219558 (0.0006) [2024-09-30 03:49:08,263][1157819] Updated weights for policy 0, policy_version 219568 (0.0006) [2024-09-30 03:49:08,836][1157819] Updated weights for policy 0, policy_version 219578 (0.0006) [2024-09-30 03:49:09,437][1157819] Updated weights for policy 0, policy_version 219588 (0.0006) [2024-09-30 03:49:10,009][1157819] Updated weights for policy 0, policy_version 219598 (0.0006) [2024-09-30 03:49:10,466][1157520] Fps is (10 sec: 69632.1, 60 sec: 74957.0, 300 sec: 77074.2). Total num frames: 899506176. Throughput: 0: 18333.2. Samples: 214829296. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 03:49:10,466][1157520] Avg episode reward: [(0, '56.075')] [2024-09-30 03:49:10,568][1157819] Updated weights for policy 0, policy_version 219608 (0.0006) [2024-09-30 03:49:11,137][1157819] Updated weights for policy 0, policy_version 219618 (0.0006) [2024-09-30 03:49:11,650][1157819] Updated weights for policy 0, policy_version 219628 (0.0006) [2024-09-30 03:49:12,218][1157819] Updated weights for policy 0, policy_version 219638 (0.0006) [2024-09-30 03:49:12,797][1157819] Updated weights for policy 0, policy_version 219648 (0.0006) [2024-09-30 03:49:13,262][1157736] Signal inference workers to stop experience collection... (15350 times) [2024-09-30 03:49:13,263][1157736] Signal inference workers to resume experience collection... (15350 times) [2024-09-30 03:49:13,266][1157819] InferenceWorker_p0-w0: stopping experience collection (15350 times) [2024-09-30 03:49:13,267][1157819] InferenceWorker_p0-w0: resuming experience collection (15350 times) [2024-09-30 03:49:13,338][1157819] Updated weights for policy 0, policy_version 219658 (0.0006) [2024-09-30 03:49:13,915][1157819] Updated weights for policy 0, policy_version 219668 (0.0006) [2024-09-30 03:49:14,457][1157819] Updated weights for policy 0, policy_version 219678 (0.0006) [2024-09-30 03:49:15,010][1157819] Updated weights for policy 0, policy_version 219688 (0.0006) [2024-09-30 03:49:15,466][1157520] Fps is (10 sec: 71680.0, 60 sec: 74342.5, 300 sec: 77088.1). Total num frames: 899874816. Throughput: 0: 18180.1. Samples: 214938040. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:49:15,466][1157520] Avg episode reward: [(0, '53.995')] [2024-09-30 03:49:15,562][1157819] Updated weights for policy 0, policy_version 219698 (0.0006) [2024-09-30 03:49:16,115][1157819] Updated weights for policy 0, policy_version 219708 (0.0006) [2024-09-30 03:49:16,689][1157819] Updated weights for policy 0, policy_version 219718 (0.0006) [2024-09-30 03:49:17,210][1157819] Updated weights for policy 0, policy_version 219728 (0.0006) [2024-09-30 03:49:17,763][1157819] Updated weights for policy 0, policy_version 219738 (0.0006) [2024-09-30 03:49:18,295][1157819] Updated weights for policy 0, policy_version 219748 (0.0006) [2024-09-30 03:49:18,831][1157819] Updated weights for policy 0, policy_version 219758 (0.0006) [2024-09-30 03:49:19,385][1157819] Updated weights for policy 0, policy_version 219768 (0.0006) [2024-09-30 03:49:19,904][1157819] Updated weights for policy 0, policy_version 219778 (0.0006) [2024-09-30 03:49:20,466][1157819] Updated weights for policy 0, policy_version 219788 (0.0006) [2024-09-30 03:49:20,466][1157520] Fps is (10 sec: 74547.3, 60 sec: 73728.1, 300 sec: 77088.1). Total num frames: 900251648. Throughput: 0: 18236.7. Samples: 215050444. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:49:20,466][1157520] Avg episode reward: [(0, '58.047')] [2024-09-30 03:49:21,013][1157819] Updated weights for policy 0, policy_version 219798 (0.0006) [2024-09-30 03:49:21,543][1157819] Updated weights for policy 0, policy_version 219808 (0.0006) [2024-09-30 03:49:22,075][1157819] Updated weights for policy 0, policy_version 219818 (0.0006) [2024-09-30 03:49:22,616][1157819] Updated weights for policy 0, policy_version 219828 (0.0006) [2024-09-30 03:49:23,155][1157819] Updated weights for policy 0, policy_version 219838 (0.0006) [2024-09-30 03:49:23,697][1157819] Updated weights for policy 0, policy_version 219848 (0.0006) [2024-09-30 03:49:24,232][1157819] Updated weights for policy 0, policy_version 219858 (0.0006) [2024-09-30 03:49:24,771][1157819] Updated weights for policy 0, policy_version 219868 (0.0006) [2024-09-30 03:49:25,326][1157819] Updated weights for policy 0, policy_version 219878 (0.0006) [2024-09-30 03:49:25,466][1157520] Fps is (10 sec: 75366.3, 60 sec: 73318.4, 300 sec: 77157.6). Total num frames: 900628480. Throughput: 0: 18277.4. Samples: 215107452. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:49:25,466][1157520] Avg episode reward: [(0, '57.176')] [2024-09-30 03:49:25,878][1157819] Updated weights for policy 0, policy_version 219888 (0.0006) [2024-09-30 03:49:26,427][1157819] Updated weights for policy 0, policy_version 219898 (0.0006) [2024-09-30 03:49:26,953][1157819] Updated weights for policy 0, policy_version 219908 (0.0006) [2024-09-30 03:49:27,464][1157819] Updated weights for policy 0, policy_version 219918 (0.0006) [2024-09-30 03:49:27,998][1157819] Updated weights for policy 0, policy_version 219928 (0.0006) [2024-09-30 03:49:28,596][1157819] Updated weights for policy 0, policy_version 219938 (0.0006) [2024-09-30 03:49:29,116][1157819] Updated weights for policy 0, policy_version 219948 (0.0006) [2024-09-30 03:49:29,667][1157819] Updated weights for policy 0, policy_version 219958 (0.0006) [2024-09-30 03:49:30,223][1157819] Updated weights for policy 0, policy_version 219968 (0.0006) [2024-09-30 03:49:30,466][1157520] Fps is (10 sec: 75366.3, 60 sec: 73455.0, 300 sec: 77240.9). Total num frames: 901005312. Throughput: 0: 18325.0. Samples: 215220668. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:49:30,466][1157520] Avg episode reward: [(0, '57.297')] [2024-09-30 03:49:30,808][1157819] Updated weights for policy 0, policy_version 219978 (0.0006) [2024-09-30 03:49:31,350][1157819] Updated weights for policy 0, policy_version 219988 (0.0006) [2024-09-30 03:49:31,884][1157819] Updated weights for policy 0, policy_version 219998 (0.0006) [2024-09-30 03:49:32,437][1157819] Updated weights for policy 0, policy_version 220008 (0.0006) [2024-09-30 03:49:33,013][1157819] Updated weights for policy 0, policy_version 220018 (0.0006) [2024-09-30 03:49:33,550][1157819] Updated weights for policy 0, policy_version 220028 (0.0006) [2024-09-30 03:49:34,117][1157819] Updated weights for policy 0, policy_version 220038 (0.0006) [2024-09-30 03:49:34,681][1157819] Updated weights for policy 0, policy_version 220048 (0.0006) [2024-09-30 03:49:35,214][1157819] Updated weights for policy 0, policy_version 220058 (0.0006) [2024-09-30 03:49:35,466][1157520] Fps is (10 sec: 74956.6, 60 sec: 73523.3, 300 sec: 77254.7). Total num frames: 901378048. Throughput: 0: 18294.7. Samples: 215331608. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:49:35,466][1157520] Avg episode reward: [(0, '56.446')] [2024-09-30 03:49:35,734][1157819] Updated weights for policy 0, policy_version 220068 (0.0006) [2024-09-30 03:49:36,282][1157819] Updated weights for policy 0, policy_version 220078 (0.0006) [2024-09-30 03:49:36,832][1157819] Updated weights for policy 0, policy_version 220088 (0.0006) [2024-09-30 03:49:37,368][1157819] Updated weights for policy 0, policy_version 220098 (0.0006) [2024-09-30 03:49:37,912][1157819] Updated weights for policy 0, policy_version 220108 (0.0006) [2024-09-30 03:49:38,438][1157819] Updated weights for policy 0, policy_version 220118 (0.0006) [2024-09-30 03:49:38,935][1157819] Updated weights for policy 0, policy_version 220128 (0.0006) [2024-09-30 03:49:39,455][1157819] Updated weights for policy 0, policy_version 220138 (0.0006) [2024-09-30 03:49:39,956][1157819] Updated weights for policy 0, policy_version 220148 (0.0006) [2024-09-30 03:49:40,466][1157520] Fps is (10 sec: 75776.5, 60 sec: 73796.5, 300 sec: 77393.6). Total num frames: 901763072. Throughput: 0: 18336.8. Samples: 215388724. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:49:40,466][1157520] Avg episode reward: [(0, '57.286')] [2024-09-30 03:49:40,485][1157819] Updated weights for policy 0, policy_version 220158 (0.0006) [2024-09-30 03:49:40,993][1157819] Updated weights for policy 0, policy_version 220168 (0.0006) [2024-09-30 03:49:41,528][1157819] Updated weights for policy 0, policy_version 220178 (0.0006) [2024-09-30 03:49:42,047][1157819] Updated weights for policy 0, policy_version 220188 (0.0006) [2024-09-30 03:49:42,573][1157819] Updated weights for policy 0, policy_version 220198 (0.0006) [2024-09-30 03:49:43,093][1157819] Updated weights for policy 0, policy_version 220208 (0.0006) [2024-09-30 03:49:43,627][1157819] Updated weights for policy 0, policy_version 220218 (0.0006) [2024-09-30 03:49:44,155][1157819] Updated weights for policy 0, policy_version 220228 (0.0006) [2024-09-30 03:49:44,675][1157819] Updated weights for policy 0, policy_version 220238 (0.0006) [2024-09-30 03:49:45,176][1157819] Updated weights for policy 0, policy_version 220248 (0.0006) [2024-09-30 03:49:45,466][1157520] Fps is (10 sec: 77825.2, 60 sec: 74137.8, 300 sec: 77574.1). Total num frames: 902156288. Throughput: 0: 18551.3. Samples: 215506508. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:49:45,466][1157520] Avg episode reward: [(0, '56.422')] [2024-09-30 03:49:45,690][1157819] Updated weights for policy 0, policy_version 220258 (0.0006) [2024-09-30 03:49:46,213][1157819] Updated weights for policy 0, policy_version 220268 (0.0006) [2024-09-30 03:49:46,738][1157819] Updated weights for policy 0, policy_version 220278 (0.0006) [2024-09-30 03:49:47,271][1157819] Updated weights for policy 0, policy_version 220288 (0.0006) [2024-09-30 03:49:47,809][1157819] Updated weights for policy 0, policy_version 220298 (0.0006) [2024-09-30 03:49:48,349][1157819] Updated weights for policy 0, policy_version 220308 (0.0006) [2024-09-30 03:49:48,940][1157819] Updated weights for policy 0, policy_version 220318 (0.0006) [2024-09-30 03:49:49,479][1157819] Updated weights for policy 0, policy_version 220328 (0.0006) [2024-09-30 03:49:49,992][1157819] Updated weights for policy 0, policy_version 220338 (0.0006) [2024-09-30 03:49:50,466][1157520] Fps is (10 sec: 77413.6, 60 sec: 74274.2, 300 sec: 77740.7). Total num frames: 902537216. Throughput: 0: 18742.0. Samples: 215622168. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:49:50,466][1157520] Avg episode reward: [(0, '55.550')] [2024-09-30 03:49:50,522][1157819] Updated weights for policy 0, policy_version 220348 (0.0006) [2024-09-30 03:49:51,073][1157819] Updated weights for policy 0, policy_version 220358 (0.0006) [2024-09-30 03:49:51,607][1157819] Updated weights for policy 0, policy_version 220368 (0.0006) [2024-09-30 03:49:52,133][1157819] Updated weights for policy 0, policy_version 220378 (0.0006) [2024-09-30 03:49:52,697][1157819] Updated weights for policy 0, policy_version 220388 (0.0006) [2024-09-30 03:49:53,241][1157819] Updated weights for policy 0, policy_version 220398 (0.0006) [2024-09-30 03:49:53,775][1157819] Updated weights for policy 0, policy_version 220408 (0.0006) [2024-09-30 03:49:54,331][1157819] Updated weights for policy 0, policy_version 220418 (0.0006) [2024-09-30 03:49:54,888][1157819] Updated weights for policy 0, policy_version 220428 (0.0006) [2024-09-30 03:49:55,418][1157819] Updated weights for policy 0, policy_version 220438 (0.0006) [2024-09-30 03:49:55,466][1157520] Fps is (10 sec: 75775.2, 60 sec: 74547.3, 300 sec: 77782.4). Total num frames: 902914048. Throughput: 0: 18884.1. Samples: 215679080. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:49:55,466][1157520] Avg episode reward: [(0, '56.306')] [2024-09-30 03:49:55,973][1157819] Updated weights for policy 0, policy_version 220448 (0.0006) [2024-09-30 03:49:56,533][1157819] Updated weights for policy 0, policy_version 220458 (0.0006) [2024-09-30 03:49:57,047][1157819] Updated weights for policy 0, policy_version 220468 (0.0006) [2024-09-30 03:49:57,570][1157819] Updated weights for policy 0, policy_version 220478 (0.0006) [2024-09-30 03:49:58,103][1157819] Updated weights for policy 0, policy_version 220488 (0.0006) [2024-09-30 03:49:58,679][1157819] Updated weights for policy 0, policy_version 220498 (0.0006) [2024-09-30 03:49:59,155][1157736] Signal inference workers to stop experience collection... (15400 times) [2024-09-30 03:49:59,159][1157819] InferenceWorker_p0-w0: stopping experience collection (15400 times) [2024-09-30 03:49:59,160][1157736] Signal inference workers to resume experience collection... (15400 times) [2024-09-30 03:49:59,164][1157819] InferenceWorker_p0-w0: resuming experience collection (15400 times) [2024-09-30 03:49:59,200][1157819] Updated weights for policy 0, policy_version 220508 (0.0006) [2024-09-30 03:49:59,729][1157819] Updated weights for policy 0, policy_version 220518 (0.0006) [2024-09-30 03:50:00,302][1157819] Updated weights for policy 0, policy_version 220528 (0.0006) [2024-09-30 03:50:00,466][1157520] Fps is (10 sec: 75776.3, 60 sec: 74752.0, 300 sec: 77754.6). Total num frames: 903294976. Throughput: 0: 18977.0. Samples: 215792004. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:50:00,466][1157520] Avg episode reward: [(0, '55.641')] [2024-09-30 03:50:00,836][1157819] Updated weights for policy 0, policy_version 220538 (0.0006) [2024-09-30 03:50:01,358][1157819] Updated weights for policy 0, policy_version 220548 (0.0006) [2024-09-30 03:50:01,903][1157819] Updated weights for policy 0, policy_version 220558 (0.0006) [2024-09-30 03:50:02,446][1157819] Updated weights for policy 0, policy_version 220568 (0.0006) [2024-09-30 03:50:02,973][1157819] Updated weights for policy 0, policy_version 220578 (0.0006) [2024-09-30 03:50:03,544][1157819] Updated weights for policy 0, policy_version 220588 (0.0006) [2024-09-30 03:50:04,078][1157819] Updated weights for policy 0, policy_version 220598 (0.0006) [2024-09-30 03:50:04,602][1157819] Updated weights for policy 0, policy_version 220608 (0.0006) [2024-09-30 03:50:05,122][1157819] Updated weights for policy 0, policy_version 220618 (0.0006) [2024-09-30 03:50:05,466][1157520] Fps is (10 sec: 76185.7, 60 sec: 75298.1, 300 sec: 77685.1). Total num frames: 903675904. Throughput: 0: 19016.1. Samples: 215906168. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:50:05,466][1157520] Avg episode reward: [(0, '56.846')] [2024-09-30 03:50:05,471][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000220624_903675904.pth... [2024-09-30 03:50:05,518][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000216111_885190656.pth [2024-09-30 03:50:05,647][1157819] Updated weights for policy 0, policy_version 220628 (0.0006) [2024-09-30 03:50:06,173][1157819] Updated weights for policy 0, policy_version 220638 (0.0006) [2024-09-30 03:50:06,681][1157819] Updated weights for policy 0, policy_version 220648 (0.0006) [2024-09-30 03:50:07,186][1157819] Updated weights for policy 0, policy_version 220658 (0.0006) [2024-09-30 03:50:07,711][1157819] Updated weights for policy 0, policy_version 220668 (0.0006) [2024-09-30 03:50:08,277][1157819] Updated weights for policy 0, policy_version 220678 (0.0006) [2024-09-30 03:50:08,787][1157819] Updated weights for policy 0, policy_version 220688 (0.0006) [2024-09-30 03:50:09,329][1157819] Updated weights for policy 0, policy_version 220698 (0.0006) [2024-09-30 03:50:09,861][1157819] Updated weights for policy 0, policy_version 220708 (0.0006) [2024-09-30 03:50:10,400][1157819] Updated weights for policy 0, policy_version 220718 (0.0006) [2024-09-30 03:50:10,466][1157520] Fps is (10 sec: 77004.9, 60 sec: 75980.8, 300 sec: 77629.6). Total num frames: 904065024. Throughput: 0: 19060.5. Samples: 215965172. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:50:10,466][1157520] Avg episode reward: [(0, '55.300')] [2024-09-30 03:50:10,920][1157819] Updated weights for policy 0, policy_version 220728 (0.0006) [2024-09-30 03:50:11,421][1157819] Updated weights for policy 0, policy_version 220738 (0.0006) [2024-09-30 03:50:11,910][1157819] Updated weights for policy 0, policy_version 220748 (0.0006) [2024-09-30 03:50:12,426][1157819] Updated weights for policy 0, policy_version 220758 (0.0006) [2024-09-30 03:50:12,969][1157819] Updated weights for policy 0, policy_version 220768 (0.0006) [2024-09-30 03:50:13,487][1157819] Updated weights for policy 0, policy_version 220778 (0.0006) [2024-09-30 03:50:13,979][1157819] Updated weights for policy 0, policy_version 220788 (0.0006) [2024-09-30 03:50:14,481][1157819] Updated weights for policy 0, policy_version 220798 (0.0006) [2024-09-30 03:50:15,013][1157819] Updated weights for policy 0, policy_version 220808 (0.0006) [2024-09-30 03:50:15,466][1157520] Fps is (10 sec: 78643.6, 60 sec: 76458.7, 300 sec: 77560.2). Total num frames: 904462336. Throughput: 0: 19163.2. Samples: 216083012. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:50:15,466][1157520] Avg episode reward: [(0, '56.072')] [2024-09-30 03:50:15,559][1157819] Updated weights for policy 0, policy_version 220818 (0.0006) [2024-09-30 03:50:16,131][1157819] Updated weights for policy 0, policy_version 220828 (0.0006) [2024-09-30 03:50:16,694][1157819] Updated weights for policy 0, policy_version 220838 (0.0006) [2024-09-30 03:50:17,196][1157819] Updated weights for policy 0, policy_version 220848 (0.0006) [2024-09-30 03:50:17,724][1157819] Updated weights for policy 0, policy_version 220858 (0.0006) [2024-09-30 03:50:18,292][1157819] Updated weights for policy 0, policy_version 220868 (0.0006) [2024-09-30 03:50:18,841][1157819] Updated weights for policy 0, policy_version 220878 (0.0006) [2024-09-30 03:50:19,391][1157819] Updated weights for policy 0, policy_version 220888 (0.0006) [2024-09-30 03:50:19,921][1157819] Updated weights for policy 0, policy_version 220898 (0.0006) [2024-09-30 03:50:20,437][1157819] Updated weights for policy 0, policy_version 220908 (0.0006) [2024-09-30 03:50:20,466][1157520] Fps is (10 sec: 77414.2, 60 sec: 76458.6, 300 sec: 77435.2). Total num frames: 904839168. Throughput: 0: 19228.4. Samples: 216196884. Policy #0 lag: (min: 0.0, avg: 2.6, max: 6.0) [2024-09-30 03:50:20,466][1157520] Avg episode reward: [(0, '57.785')] [2024-09-30 03:50:20,660][1157736] Signal inference workers to stop experience collection... (15450 times) [2024-09-30 03:50:20,662][1157736] Signal inference workers to resume experience collection... (15450 times) [2024-09-30 03:50:20,665][1157819] InferenceWorker_p0-w0: stopping experience collection (15450 times) [2024-09-30 03:50:20,669][1157819] InferenceWorker_p0-w0: resuming experience collection (15450 times) [2024-09-30 03:50:20,970][1157819] Updated weights for policy 0, policy_version 220918 (0.0006) [2024-09-30 03:50:21,487][1157819] Updated weights for policy 0, policy_version 220928 (0.0006) [2024-09-30 03:50:21,987][1157819] Updated weights for policy 0, policy_version 220938 (0.0006) [2024-09-30 03:50:22,492][1157819] Updated weights for policy 0, policy_version 220948 (0.0006) [2024-09-30 03:50:23,009][1157819] Updated weights for policy 0, policy_version 220958 (0.0006) [2024-09-30 03:50:23,573][1157819] Updated weights for policy 0, policy_version 220968 (0.0006) [2024-09-30 03:50:24,091][1157819] Updated weights for policy 0, policy_version 220978 (0.0006) [2024-09-30 03:50:24,614][1157819] Updated weights for policy 0, policy_version 220988 (0.0006) [2024-09-30 03:50:25,161][1157819] Updated weights for policy 0, policy_version 220998 (0.0006) [2024-09-30 03:50:25,466][1157520] Fps is (10 sec: 76594.8, 60 sec: 76663.5, 300 sec: 77324.1). Total num frames: 905228288. Throughput: 0: 19281.1. Samples: 216256376. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:50:25,466][1157520] Avg episode reward: [(0, '57.651')] [2024-09-30 03:50:25,682][1157819] Updated weights for policy 0, policy_version 221008 (0.0006) [2024-09-30 03:50:26,230][1157819] Updated weights for policy 0, policy_version 221018 (0.0006) [2024-09-30 03:50:26,745][1157819] Updated weights for policy 0, policy_version 221028 (0.0006) [2024-09-30 03:50:27,260][1157819] Updated weights for policy 0, policy_version 221038 (0.0006) [2024-09-30 03:50:27,768][1157819] Updated weights for policy 0, policy_version 221048 (0.0006) [2024-09-30 03:50:28,301][1157819] Updated weights for policy 0, policy_version 221058 (0.0006) [2024-09-30 03:50:28,827][1157819] Updated weights for policy 0, policy_version 221068 (0.0006) [2024-09-30 03:50:29,370][1157819] Updated weights for policy 0, policy_version 221078 (0.0006) [2024-09-30 03:50:29,856][1157819] Updated weights for policy 0, policy_version 221088 (0.0006) [2024-09-30 03:50:30,354][1157819] Updated weights for policy 0, policy_version 221098 (0.0006) [2024-09-30 03:50:30,466][1157520] Fps is (10 sec: 78233.7, 60 sec: 76936.6, 300 sec: 77324.2). Total num frames: 905621504. Throughput: 0: 19258.6. Samples: 216373144. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:50:30,466][1157520] Avg episode reward: [(0, '57.584')] [2024-09-30 03:50:30,879][1157819] Updated weights for policy 0, policy_version 221108 (0.0006) [2024-09-30 03:50:31,413][1157819] Updated weights for policy 0, policy_version 221118 (0.0006) [2024-09-30 03:50:31,952][1157819] Updated weights for policy 0, policy_version 221128 (0.0006) [2024-09-30 03:50:32,444][1157819] Updated weights for policy 0, policy_version 221138 (0.0006) [2024-09-30 03:50:32,943][1157819] Updated weights for policy 0, policy_version 221148 (0.0006) [2024-09-30 03:50:33,492][1157819] Updated weights for policy 0, policy_version 221158 (0.0006) [2024-09-30 03:50:34,003][1157819] Updated weights for policy 0, policy_version 221168 (0.0006) [2024-09-30 03:50:34,517][1157819] Updated weights for policy 0, policy_version 221178 (0.0006) [2024-09-30 03:50:35,035][1157819] Updated weights for policy 0, policy_version 221188 (0.0006) [2024-09-30 03:50:35,466][1157520] Fps is (10 sec: 79052.5, 60 sec: 77346.2, 300 sec: 77365.8). Total num frames: 906018816. Throughput: 0: 19328.2. Samples: 216491936. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:50:35,466][1157520] Avg episode reward: [(0, '56.133')] [2024-09-30 03:50:35,537][1157819] Updated weights for policy 0, policy_version 221198 (0.0006) [2024-09-30 03:50:36,052][1157819] Updated weights for policy 0, policy_version 221208 (0.0006) [2024-09-30 03:50:36,569][1157819] Updated weights for policy 0, policy_version 221218 (0.0006) [2024-09-30 03:50:37,080][1157819] Updated weights for policy 0, policy_version 221228 (0.0006) [2024-09-30 03:50:37,587][1157819] Updated weights for policy 0, policy_version 221238 (0.0006) [2024-09-30 03:50:38,096][1157819] Updated weights for policy 0, policy_version 221248 (0.0006) [2024-09-30 03:50:38,632][1157819] Updated weights for policy 0, policy_version 221258 (0.0006) [2024-09-30 03:50:39,168][1157819] Updated weights for policy 0, policy_version 221268 (0.0006) [2024-09-30 03:50:39,659][1157819] Updated weights for policy 0, policy_version 221278 (0.0006) [2024-09-30 03:50:40,163][1157819] Updated weights for policy 0, policy_version 221288 (0.0006) [2024-09-30 03:50:40,466][1157520] Fps is (10 sec: 79461.8, 60 sec: 77550.8, 300 sec: 77365.8). Total num frames: 906416128. Throughput: 0: 19390.4. Samples: 216551648. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:50:40,466][1157520] Avg episode reward: [(0, '58.523')] [2024-09-30 03:50:40,692][1157819] Updated weights for policy 0, policy_version 221298 (0.0006) [2024-09-30 03:50:41,222][1157819] Updated weights for policy 0, policy_version 221308 (0.0006) [2024-09-30 03:50:41,728][1157819] Updated weights for policy 0, policy_version 221318 (0.0006) [2024-09-30 03:50:42,209][1157819] Updated weights for policy 0, policy_version 221328 (0.0006) [2024-09-30 03:50:42,761][1157819] Updated weights for policy 0, policy_version 221338 (0.0006) [2024-09-30 03:50:43,268][1157819] Updated weights for policy 0, policy_version 221348 (0.0006) [2024-09-30 03:50:43,801][1157819] Updated weights for policy 0, policy_version 221358 (0.0006) [2024-09-30 03:50:44,295][1157819] Updated weights for policy 0, policy_version 221368 (0.0006) [2024-09-30 03:50:44,785][1157819] Updated weights for policy 0, policy_version 221378 (0.0006) [2024-09-30 03:50:45,364][1157819] Updated weights for policy 0, policy_version 221388 (0.0006) [2024-09-30 03:50:45,466][1157520] Fps is (10 sec: 79052.8, 60 sec: 77550.8, 300 sec: 77338.0). Total num frames: 906809344. Throughput: 0: 19528.3. Samples: 216670780. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:50:45,466][1157520] Avg episode reward: [(0, '56.467')] [2024-09-30 03:50:45,923][1157819] Updated weights for policy 0, policy_version 221398 (0.0006) [2024-09-30 03:50:46,540][1157819] Updated weights for policy 0, policy_version 221408 (0.0006) [2024-09-30 03:50:47,136][1157819] Updated weights for policy 0, policy_version 221418 (0.0006) [2024-09-30 03:50:47,710][1157819] Updated weights for policy 0, policy_version 221428 (0.0006) [2024-09-30 03:50:48,287][1157819] Updated weights for policy 0, policy_version 221438 (0.0006) [2024-09-30 03:50:48,345][1157736] Signal inference workers to stop experience collection... (15500 times) [2024-09-30 03:50:48,346][1157736] Signal inference workers to resume experience collection... (15500 times) [2024-09-30 03:50:48,351][1157819] InferenceWorker_p0-w0: stopping experience collection (15500 times) [2024-09-30 03:50:48,351][1157819] InferenceWorker_p0-w0: resuming experience collection (15500 times) [2024-09-30 03:50:48,839][1157819] Updated weights for policy 0, policy_version 221448 (0.0006) [2024-09-30 03:50:49,402][1157819] Updated weights for policy 0, policy_version 221458 (0.0006) [2024-09-30 03:50:49,966][1157819] Updated weights for policy 0, policy_version 221468 (0.0006) [2024-09-30 03:50:50,466][1157520] Fps is (10 sec: 74956.7, 60 sec: 77141.3, 300 sec: 77157.5). Total num frames: 907165696. Throughput: 0: 19419.5. Samples: 216780044. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:50:50,466][1157520] Avg episode reward: [(0, '56.193')] [2024-09-30 03:50:50,523][1157819] Updated weights for policy 0, policy_version 221478 (0.0006) [2024-09-30 03:50:51,099][1157819] Updated weights for policy 0, policy_version 221488 (0.0006) [2024-09-30 03:50:51,670][1157819] Updated weights for policy 0, policy_version 221498 (0.0006) [2024-09-30 03:50:52,237][1157819] Updated weights for policy 0, policy_version 221508 (0.0006) [2024-09-30 03:50:52,829][1157819] Updated weights for policy 0, policy_version 221518 (0.0006) [2024-09-30 03:50:53,384][1157819] Updated weights for policy 0, policy_version 221528 (0.0006) [2024-09-30 03:50:53,944][1157819] Updated weights for policy 0, policy_version 221538 (0.0006) [2024-09-30 03:50:54,550][1157819] Updated weights for policy 0, policy_version 221548 (0.0006) [2024-09-30 03:50:55,138][1157819] Updated weights for policy 0, policy_version 221558 (0.0006) [2024-09-30 03:50:55,466][1157520] Fps is (10 sec: 71269.5, 60 sec: 76799.8, 300 sec: 77018.6). Total num frames: 907522048. Throughput: 0: 19303.2. Samples: 216833820. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:50:55,466][1157520] Avg episode reward: [(0, '56.403')] [2024-09-30 03:50:55,707][1157819] Updated weights for policy 0, policy_version 221568 (0.0006) [2024-09-30 03:50:56,298][1157819] Updated weights for policy 0, policy_version 221578 (0.0006) [2024-09-30 03:50:56,838][1157819] Updated weights for policy 0, policy_version 221588 (0.0006) [2024-09-30 03:50:57,445][1157819] Updated weights for policy 0, policy_version 221598 (0.0006) [2024-09-30 03:50:57,993][1157819] Updated weights for policy 0, policy_version 221608 (0.0006) [2024-09-30 03:50:58,530][1157819] Updated weights for policy 0, policy_version 221618 (0.0006) [2024-09-30 03:50:59,096][1157819] Updated weights for policy 0, policy_version 221628 (0.0006) [2024-09-30 03:50:59,597][1157819] Updated weights for policy 0, policy_version 221638 (0.0006) [2024-09-30 03:51:00,104][1157819] Updated weights for policy 0, policy_version 221648 (0.0006) [2024-09-30 03:51:00,466][1157520] Fps is (10 sec: 72908.6, 60 sec: 76663.3, 300 sec: 76963.1). Total num frames: 907894784. Throughput: 0: 19093.9. Samples: 216942240. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:51:00,466][1157520] Avg episode reward: [(0, '55.870')] [2024-09-30 03:51:00,664][1157819] Updated weights for policy 0, policy_version 221658 (0.0006) [2024-09-30 03:51:01,194][1157819] Updated weights for policy 0, policy_version 221668 (0.0006) [2024-09-30 03:51:01,717][1157819] Updated weights for policy 0, policy_version 221678 (0.0006) [2024-09-30 03:51:02,272][1157819] Updated weights for policy 0, policy_version 221688 (0.0006) [2024-09-30 03:51:02,811][1157819] Updated weights for policy 0, policy_version 221698 (0.0006) [2024-09-30 03:51:03,371][1157819] Updated weights for policy 0, policy_version 221708 (0.0007) [2024-09-30 03:51:03,960][1157819] Updated weights for policy 0, policy_version 221718 (0.0006) [2024-09-30 03:51:04,540][1157819] Updated weights for policy 0, policy_version 221728 (0.0006) [2024-09-30 03:51:05,117][1157819] Updated weights for policy 0, policy_version 221738 (0.0006) [2024-09-30 03:51:05,466][1157520] Fps is (10 sec: 74137.6, 60 sec: 76458.5, 300 sec: 76907.6). Total num frames: 908263424. Throughput: 0: 19045.3. Samples: 217053928. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:51:05,466][1157520] Avg episode reward: [(0, '55.987')] [2024-09-30 03:51:05,691][1157819] Updated weights for policy 0, policy_version 221748 (0.0006) [2024-09-30 03:51:06,296][1157819] Updated weights for policy 0, policy_version 221758 (0.0006) [2024-09-30 03:51:06,883][1157819] Updated weights for policy 0, policy_version 221768 (0.0006) [2024-09-30 03:51:07,451][1157819] Updated weights for policy 0, policy_version 221778 (0.0006) [2024-09-30 03:51:08,055][1157819] Updated weights for policy 0, policy_version 221788 (0.0006) [2024-09-30 03:51:08,651][1157819] Updated weights for policy 0, policy_version 221798 (0.0006) [2024-09-30 03:51:09,213][1157819] Updated weights for policy 0, policy_version 221808 (0.0006) [2024-09-30 03:51:09,815][1157819] Updated weights for policy 0, policy_version 221818 (0.0006) [2024-09-30 03:51:10,372][1157819] Updated weights for policy 0, policy_version 221828 (0.0006) [2024-09-30 03:51:10,466][1157520] Fps is (10 sec: 71680.5, 60 sec: 75775.9, 300 sec: 76727.1). Total num frames: 908611584. Throughput: 0: 18896.2. Samples: 217106704. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:51:10,466][1157520] Avg episode reward: [(0, '57.769')] [2024-09-30 03:51:10,949][1157819] Updated weights for policy 0, policy_version 221838 (0.0006) [2024-09-30 03:51:11,530][1157819] Updated weights for policy 0, policy_version 221848 (0.0006) [2024-09-30 03:51:12,117][1157819] Updated weights for policy 0, policy_version 221858 (0.0006) [2024-09-30 03:51:12,717][1157819] Updated weights for policy 0, policy_version 221868 (0.0006) [2024-09-30 03:51:13,273][1157819] Updated weights for policy 0, policy_version 221878 (0.0006) [2024-09-30 03:51:13,880][1157819] Updated weights for policy 0, policy_version 221888 (0.0006) [2024-09-30 03:51:14,458][1157819] Updated weights for policy 0, policy_version 221898 (0.0006) [2024-09-30 03:51:15,034][1157819] Updated weights for policy 0, policy_version 221908 (0.0006) [2024-09-30 03:51:15,334][1157736] Signal inference workers to stop experience collection... (15550 times) [2024-09-30 03:51:15,337][1157819] InferenceWorker_p0-w0: stopping experience collection (15550 times) [2024-09-30 03:51:15,345][1157736] Signal inference workers to resume experience collection... (15550 times) [2024-09-30 03:51:15,345][1157819] InferenceWorker_p0-w0: resuming experience collection (15550 times) [2024-09-30 03:51:15,466][1157520] Fps is (10 sec: 70042.6, 60 sec: 75025.0, 300 sec: 76574.4). Total num frames: 908963840. Throughput: 0: 18647.0. Samples: 217212260. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:51:15,466][1157520] Avg episode reward: [(0, '55.736')] [2024-09-30 03:51:15,628][1157819] Updated weights for policy 0, policy_version 221918 (0.0007) [2024-09-30 03:51:16,249][1157819] Updated weights for policy 0, policy_version 221928 (0.0006) [2024-09-30 03:51:16,816][1157819] Updated weights for policy 0, policy_version 221938 (0.0006) [2024-09-30 03:51:17,456][1157819] Updated weights for policy 0, policy_version 221948 (0.0006) [2024-09-30 03:51:18,050][1157819] Updated weights for policy 0, policy_version 221958 (0.0007) [2024-09-30 03:51:18,616][1157819] Updated weights for policy 0, policy_version 221968 (0.0006) [2024-09-30 03:51:19,126][1157819] Updated weights for policy 0, policy_version 221978 (0.0006) [2024-09-30 03:51:19,714][1157819] Updated weights for policy 0, policy_version 221988 (0.0006) [2024-09-30 03:51:20,243][1157819] Updated weights for policy 0, policy_version 221998 (0.0006) [2024-09-30 03:51:20,466][1157520] Fps is (10 sec: 70860.9, 60 sec: 74683.7, 300 sec: 76449.4). Total num frames: 909320192. Throughput: 0: 18349.8. Samples: 217317676. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:51:20,466][1157520] Avg episode reward: [(0, '56.545')] [2024-09-30 03:51:20,777][1157819] Updated weights for policy 0, policy_version 222008 (0.0006) [2024-09-30 03:51:21,282][1157819] Updated weights for policy 0, policy_version 222018 (0.0006) [2024-09-30 03:51:21,839][1157819] Updated weights for policy 0, policy_version 222028 (0.0006) [2024-09-30 03:51:22,406][1157819] Updated weights for policy 0, policy_version 222038 (0.0006) [2024-09-30 03:51:22,971][1157819] Updated weights for policy 0, policy_version 222048 (0.0006) [2024-09-30 03:51:23,531][1157819] Updated weights for policy 0, policy_version 222058 (0.0006) [2024-09-30 03:51:24,057][1157819] Updated weights for policy 0, policy_version 222068 (0.0006) [2024-09-30 03:51:24,591][1157819] Updated weights for policy 0, policy_version 222078 (0.0006) [2024-09-30 03:51:25,145][1157819] Updated weights for policy 0, policy_version 222088 (0.0006) [2024-09-30 03:51:25,466][1157520] Fps is (10 sec: 73318.4, 60 sec: 74478.9, 300 sec: 76380.0). Total num frames: 909697024. Throughput: 0: 18276.6. Samples: 217374096. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:51:25,466][1157520] Avg episode reward: [(0, '57.429')] [2024-09-30 03:51:25,665][1157819] Updated weights for policy 0, policy_version 222098 (0.0006) [2024-09-30 03:51:26,185][1157819] Updated weights for policy 0, policy_version 222108 (0.0006) [2024-09-30 03:51:26,732][1157819] Updated weights for policy 0, policy_version 222118 (0.0006) [2024-09-30 03:51:27,284][1157819] Updated weights for policy 0, policy_version 222128 (0.0006) [2024-09-30 03:51:27,838][1157819] Updated weights for policy 0, policy_version 222138 (0.0006) [2024-09-30 03:51:28,386][1157819] Updated weights for policy 0, policy_version 222148 (0.0006) [2024-09-30 03:51:28,928][1157819] Updated weights for policy 0, policy_version 222158 (0.0006) [2024-09-30 03:51:29,470][1157819] Updated weights for policy 0, policy_version 222168 (0.0006) [2024-09-30 03:51:30,009][1157819] Updated weights for policy 0, policy_version 222178 (0.0006) [2024-09-30 03:51:30,466][1157520] Fps is (10 sec: 75366.0, 60 sec: 74205.8, 300 sec: 76296.7). Total num frames: 910073856. Throughput: 0: 18146.1. Samples: 217487356. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:51:30,466][1157520] Avg episode reward: [(0, '56.132')] [2024-09-30 03:51:30,557][1157819] Updated weights for policy 0, policy_version 222188 (0.0006) [2024-09-30 03:51:31,064][1157819] Updated weights for policy 0, policy_version 222198 (0.0006) [2024-09-30 03:51:31,578][1157819] Updated weights for policy 0, policy_version 222208 (0.0006) [2024-09-30 03:51:32,096][1157819] Updated weights for policy 0, policy_version 222218 (0.0006) [2024-09-30 03:51:32,616][1157819] Updated weights for policy 0, policy_version 222228 (0.0006) [2024-09-30 03:51:33,124][1157819] Updated weights for policy 0, policy_version 222238 (0.0006) [2024-09-30 03:51:33,641][1157819] Updated weights for policy 0, policy_version 222248 (0.0006) [2024-09-30 03:51:34,161][1157819] Updated weights for policy 0, policy_version 222258 (0.0006) [2024-09-30 03:51:34,663][1157819] Updated weights for policy 0, policy_version 222268 (0.0006) [2024-09-30 03:51:35,224][1157819] Updated weights for policy 0, policy_version 222278 (0.0006) [2024-09-30 03:51:35,466][1157520] Fps is (10 sec: 77004.4, 60 sec: 74137.6, 300 sec: 76352.2). Total num frames: 910467072. Throughput: 0: 18320.7. Samples: 217604476. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:51:35,466][1157520] Avg episode reward: [(0, '56.487')] [2024-09-30 03:51:35,771][1157819] Updated weights for policy 0, policy_version 222288 (0.0006) [2024-09-30 03:51:36,306][1157819] Updated weights for policy 0, policy_version 222298 (0.0006) [2024-09-30 03:51:36,813][1157819] Updated weights for policy 0, policy_version 222308 (0.0006) [2024-09-30 03:51:37,338][1157819] Updated weights for policy 0, policy_version 222318 (0.0006) [2024-09-30 03:51:37,851][1157819] Updated weights for policy 0, policy_version 222328 (0.0006) [2024-09-30 03:51:38,385][1157819] Updated weights for policy 0, policy_version 222338 (0.0006) [2024-09-30 03:51:38,875][1157819] Updated weights for policy 0, policy_version 222348 (0.0006) [2024-09-30 03:51:39,403][1157819] Updated weights for policy 0, policy_version 222358 (0.0006) [2024-09-30 03:51:39,738][1157736] Signal inference workers to stop experience collection... (15600 times) [2024-09-30 03:51:39,739][1157736] Signal inference workers to resume experience collection... (15600 times) [2024-09-30 03:51:39,743][1157819] InferenceWorker_p0-w0: stopping experience collection (15600 times) [2024-09-30 03:51:39,743][1157819] InferenceWorker_p0-w0: resuming experience collection (15600 times) [2024-09-30 03:51:39,901][1157819] Updated weights for policy 0, policy_version 222368 (0.0006) [2024-09-30 03:51:40,410][1157819] Updated weights for policy 0, policy_version 222378 (0.0006) [2024-09-30 03:51:40,466][1157520] Fps is (10 sec: 78643.1, 60 sec: 74069.3, 300 sec: 76407.8). Total num frames: 910860288. Throughput: 0: 18405.6. Samples: 217662072. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:51:40,466][1157520] Avg episode reward: [(0, '57.500')] [2024-09-30 03:51:40,914][1157819] Updated weights for policy 0, policy_version 222388 (0.0006) [2024-09-30 03:51:41,434][1157819] Updated weights for policy 0, policy_version 222398 (0.0006) [2024-09-30 03:51:41,961][1157819] Updated weights for policy 0, policy_version 222408 (0.0006) [2024-09-30 03:51:42,471][1157819] Updated weights for policy 0, policy_version 222418 (0.0006) [2024-09-30 03:51:42,974][1157819] Updated weights for policy 0, policy_version 222428 (0.0006) [2024-09-30 03:51:43,472][1157819] Updated weights for policy 0, policy_version 222438 (0.0006) [2024-09-30 03:51:43,978][1157819] Updated weights for policy 0, policy_version 222448 (0.0006) [2024-09-30 03:51:44,496][1157819] Updated weights for policy 0, policy_version 222458 (0.0006) [2024-09-30 03:51:45,004][1157819] Updated weights for policy 0, policy_version 222468 (0.0006) [2024-09-30 03:51:45,466][1157520] Fps is (10 sec: 79462.9, 60 sec: 74205.9, 300 sec: 76463.3). Total num frames: 911261696. Throughput: 0: 18679.0. Samples: 217782792. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:51:45,466][1157520] Avg episode reward: [(0, '57.608')] [2024-09-30 03:51:45,553][1157819] Updated weights for policy 0, policy_version 222478 (0.0006) [2024-09-30 03:51:46,049][1157819] Updated weights for policy 0, policy_version 222488 (0.0006) [2024-09-30 03:51:46,528][1157819] Updated weights for policy 0, policy_version 222498 (0.0006) [2024-09-30 03:51:47,024][1157819] Updated weights for policy 0, policy_version 222508 (0.0006) [2024-09-30 03:51:47,496][1157819] Updated weights for policy 0, policy_version 222518 (0.0006) [2024-09-30 03:51:47,990][1157819] Updated weights for policy 0, policy_version 222528 (0.0006) [2024-09-30 03:51:48,430][1157819] Updated weights for policy 0, policy_version 222538 (0.0005) [2024-09-30 03:51:48,885][1157819] Updated weights for policy 0, policy_version 222548 (0.0006) [2024-09-30 03:51:49,366][1157819] Updated weights for policy 0, policy_version 222558 (0.0006) [2024-09-30 03:51:49,855][1157819] Updated weights for policy 0, policy_version 222568 (0.0006) [2024-09-30 03:51:50,348][1157819] Updated weights for policy 0, policy_version 222578 (0.0006) [2024-09-30 03:51:50,466][1157520] Fps is (10 sec: 82739.9, 60 sec: 75366.5, 300 sec: 76643.8). Total num frames: 911687680. Throughput: 0: 18987.0. Samples: 217908340. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:51:50,466][1157520] Avg episode reward: [(0, '54.216')] [2024-09-30 03:51:50,843][1157819] Updated weights for policy 0, policy_version 222588 (0.0006) [2024-09-30 03:51:51,376][1157819] Updated weights for policy 0, policy_version 222598 (0.0006) [2024-09-30 03:51:51,887][1157819] Updated weights for policy 0, policy_version 222608 (0.0006) [2024-09-30 03:51:52,368][1157819] Updated weights for policy 0, policy_version 222618 (0.0006) [2024-09-30 03:51:52,873][1157819] Updated weights for policy 0, policy_version 222628 (0.0006) [2024-09-30 03:51:53,362][1157819] Updated weights for policy 0, policy_version 222638 (0.0006) [2024-09-30 03:51:53,860][1157819] Updated weights for policy 0, policy_version 222648 (0.0006) [2024-09-30 03:51:54,388][1157819] Updated weights for policy 0, policy_version 222658 (0.0006) [2024-09-30 03:51:54,911][1157819] Updated weights for policy 0, policy_version 222668 (0.0006) [2024-09-30 03:51:55,400][1157819] Updated weights for policy 0, policy_version 222678 (0.0006) [2024-09-30 03:51:55,466][1157520] Fps is (10 sec: 83148.4, 60 sec: 76185.7, 300 sec: 76727.1). Total num frames: 912093184. Throughput: 0: 19173.5. Samples: 217969512. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:51:55,466][1157520] Avg episode reward: [(0, '58.421')] [2024-09-30 03:51:55,917][1157819] Updated weights for policy 0, policy_version 222688 (0.0006) [2024-09-30 03:51:56,421][1157819] Updated weights for policy 0, policy_version 222698 (0.0006) [2024-09-30 03:51:56,927][1157819] Updated weights for policy 0, policy_version 222708 (0.0006) [2024-09-30 03:51:57,454][1157819] Updated weights for policy 0, policy_version 222718 (0.0006) [2024-09-30 03:51:57,958][1157819] Updated weights for policy 0, policy_version 222728 (0.0006) [2024-09-30 03:51:58,468][1157819] Updated weights for policy 0, policy_version 222738 (0.0006) [2024-09-30 03:51:58,974][1157819] Updated weights for policy 0, policy_version 222748 (0.0006) [2024-09-30 03:51:59,493][1157819] Updated weights for policy 0, policy_version 222758 (0.0006) [2024-09-30 03:52:00,010][1157819] Updated weights for policy 0, policy_version 222768 (0.0006) [2024-09-30 03:52:00,466][1157520] Fps is (10 sec: 80690.8, 60 sec: 76663.5, 300 sec: 76685.4). Total num frames: 912494592. Throughput: 0: 19507.6. Samples: 218090100. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:52:00,466][1157520] Avg episode reward: [(0, '54.945')] [2024-09-30 03:52:00,529][1157819] Updated weights for policy 0, policy_version 222778 (0.0006) [2024-09-30 03:52:01,071][1157819] Updated weights for policy 0, policy_version 222788 (0.0006) [2024-09-30 03:52:01,610][1157819] Updated weights for policy 0, policy_version 222798 (0.0006) [2024-09-30 03:52:02,178][1157819] Updated weights for policy 0, policy_version 222808 (0.0006) [2024-09-30 03:52:02,709][1157819] Updated weights for policy 0, policy_version 222818 (0.0006) [2024-09-30 03:52:03,223][1157819] Updated weights for policy 0, policy_version 222828 (0.0006) [2024-09-30 03:52:03,760][1157819] Updated weights for policy 0, policy_version 222838 (0.0006) [2024-09-30 03:52:04,327][1157819] Updated weights for policy 0, policy_version 222848 (0.0006) [2024-09-30 03:52:04,876][1157819] Updated weights for policy 0, policy_version 222858 (0.0006) [2024-09-30 03:52:05,359][1157819] Updated weights for policy 0, policy_version 222868 (0.0006) [2024-09-30 03:52:05,466][1157520] Fps is (10 sec: 77824.3, 60 sec: 76800.2, 300 sec: 76560.5). Total num frames: 912871424. Throughput: 0: 19725.1. Samples: 218205308. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:52:05,466][1157520] Avg episode reward: [(0, '57.552')] [2024-09-30 03:52:05,473][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000222870_912875520.pth... [2024-09-30 03:52:05,516][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000218414_894623744.pth [2024-09-30 03:52:05,890][1157819] Updated weights for policy 0, policy_version 222878 (0.0006) [2024-09-30 03:52:06,399][1157819] Updated weights for policy 0, policy_version 222888 (0.0006) [2024-09-30 03:52:06,916][1157819] Updated weights for policy 0, policy_version 222898 (0.0006) [2024-09-30 03:52:07,422][1157819] Updated weights for policy 0, policy_version 222908 (0.0006) [2024-09-30 03:52:07,932][1157819] Updated weights for policy 0, policy_version 222918 (0.0006) [2024-09-30 03:52:08,443][1157819] Updated weights for policy 0, policy_version 222928 (0.0006) [2024-09-30 03:52:08,960][1157819] Updated weights for policy 0, policy_version 222938 (0.0006) [2024-09-30 03:52:09,457][1157819] Updated weights for policy 0, policy_version 222948 (0.0006) [2024-09-30 03:52:09,968][1157819] Updated weights for policy 0, policy_version 222958 (0.0006) [2024-09-30 03:52:10,466][1157520] Fps is (10 sec: 77824.2, 60 sec: 77687.5, 300 sec: 76532.7). Total num frames: 913272832. Throughput: 0: 19801.3. Samples: 218265152. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:52:10,466][1157520] Avg episode reward: [(0, '58.777')] [2024-09-30 03:52:10,495][1157819] Updated weights for policy 0, policy_version 222968 (0.0006) [2024-09-30 03:52:11,022][1157819] Updated weights for policy 0, policy_version 222978 (0.0006) [2024-09-30 03:52:11,519][1157819] Updated weights for policy 0, policy_version 222988 (0.0006) [2024-09-30 03:52:12,023][1157819] Updated weights for policy 0, policy_version 222998 (0.0006) [2024-09-30 03:52:12,572][1157819] Updated weights for policy 0, policy_version 223008 (0.0006) [2024-09-30 03:52:13,107][1157819] Updated weights for policy 0, policy_version 223018 (0.0006) [2024-09-30 03:52:13,611][1157819] Updated weights for policy 0, policy_version 223028 (0.0005) [2024-09-30 03:52:14,106][1157819] Updated weights for policy 0, policy_version 223038 (0.0006) [2024-09-30 03:52:14,609][1157819] Updated weights for policy 0, policy_version 223048 (0.0006) [2024-09-30 03:52:15,159][1157819] Updated weights for policy 0, policy_version 223058 (0.0006) [2024-09-30 03:52:15,466][1157520] Fps is (10 sec: 79462.5, 60 sec: 78370.2, 300 sec: 76463.3). Total num frames: 913666048. Throughput: 0: 19933.7. Samples: 218384372. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:52:15,466][1157520] Avg episode reward: [(0, '56.320')] [2024-09-30 03:52:15,686][1157819] Updated weights for policy 0, policy_version 223068 (0.0006) [2024-09-30 03:52:16,173][1157819] Updated weights for policy 0, policy_version 223078 (0.0006) [2024-09-30 03:52:16,375][1157736] Signal inference workers to stop experience collection... (15650 times) [2024-09-30 03:52:16,376][1157736] Signal inference workers to resume experience collection... (15650 times) [2024-09-30 03:52:16,380][1157819] InferenceWorker_p0-w0: stopping experience collection (15650 times) [2024-09-30 03:52:16,380][1157819] InferenceWorker_p0-w0: resuming experience collection (15650 times) [2024-09-30 03:52:16,669][1157819] Updated weights for policy 0, policy_version 223088 (0.0006) [2024-09-30 03:52:17,172][1157819] Updated weights for policy 0, policy_version 223098 (0.0006) [2024-09-30 03:52:17,669][1157819] Updated weights for policy 0, policy_version 223108 (0.0006) [2024-09-30 03:52:18,177][1157819] Updated weights for policy 0, policy_version 223118 (0.0006) [2024-09-30 03:52:18,718][1157819] Updated weights for policy 0, policy_version 223128 (0.0006) [2024-09-30 03:52:19,223][1157819] Updated weights for policy 0, policy_version 223138 (0.0006) [2024-09-30 03:52:19,715][1157819] Updated weights for policy 0, policy_version 223148 (0.0006) [2024-09-30 03:52:20,222][1157819] Updated weights for policy 0, policy_version 223158 (0.0006) [2024-09-30 03:52:20,466][1157520] Fps is (10 sec: 79872.0, 60 sec: 79189.3, 300 sec: 76546.6). Total num frames: 914071552. Throughput: 0: 20007.2. Samples: 218504800. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:52:20,466][1157520] Avg episode reward: [(0, '55.335')] [2024-09-30 03:52:20,712][1157819] Updated weights for policy 0, policy_version 223168 (0.0006) [2024-09-30 03:52:21,201][1157819] Updated weights for policy 0, policy_version 223178 (0.0006) [2024-09-30 03:52:21,721][1157819] Updated weights for policy 0, policy_version 223188 (0.0006) [2024-09-30 03:52:22,250][1157819] Updated weights for policy 0, policy_version 223198 (0.0006) [2024-09-30 03:52:22,766][1157819] Updated weights for policy 0, policy_version 223208 (0.0006) [2024-09-30 03:52:23,265][1157819] Updated weights for policy 0, policy_version 223218 (0.0006) [2024-09-30 03:52:23,768][1157819] Updated weights for policy 0, policy_version 223228 (0.0006) [2024-09-30 03:52:24,299][1157819] Updated weights for policy 0, policy_version 223238 (0.0006) [2024-09-30 03:52:24,796][1157819] Updated weights for policy 0, policy_version 223248 (0.0006) [2024-09-30 03:52:25,324][1157819] Updated weights for policy 0, policy_version 223258 (0.0006) [2024-09-30 03:52:25,466][1157520] Fps is (10 sec: 80691.3, 60 sec: 79599.0, 300 sec: 76546.6). Total num frames: 914472960. Throughput: 0: 20080.4. Samples: 218565688. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:52:25,466][1157520] Avg episode reward: [(0, '57.541')] [2024-09-30 03:52:25,863][1157819] Updated weights for policy 0, policy_version 223268 (0.0006) [2024-09-30 03:52:26,397][1157819] Updated weights for policy 0, policy_version 223278 (0.0006) [2024-09-30 03:52:27,003][1157819] Updated weights for policy 0, policy_version 223288 (0.0007) [2024-09-30 03:52:27,523][1157819] Updated weights for policy 0, policy_version 223298 (0.0007) [2024-09-30 03:52:28,061][1157819] Updated weights for policy 0, policy_version 223308 (0.0006) [2024-09-30 03:52:28,655][1157819] Updated weights for policy 0, policy_version 223318 (0.0006) [2024-09-30 03:52:29,182][1157819] Updated weights for policy 0, policy_version 223328 (0.0006) [2024-09-30 03:52:29,770][1157819] Updated weights for policy 0, policy_version 223338 (0.0006) [2024-09-30 03:52:30,316][1157819] Updated weights for policy 0, policy_version 223348 (0.0006) [2024-09-30 03:52:30,466][1157520] Fps is (10 sec: 77414.4, 60 sec: 79530.7, 300 sec: 76463.3). Total num frames: 914845696. Throughput: 0: 19951.2. Samples: 218680596. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:52:30,466][1157520] Avg episode reward: [(0, '57.128')] [2024-09-30 03:52:30,840][1157819] Updated weights for policy 0, policy_version 223358 (0.0006) [2024-09-30 03:52:31,419][1157819] Updated weights for policy 0, policy_version 223368 (0.0006) [2024-09-30 03:52:31,966][1157819] Updated weights for policy 0, policy_version 223378 (0.0006) [2024-09-30 03:52:32,529][1157819] Updated weights for policy 0, policy_version 223388 (0.0006) [2024-09-30 03:52:33,076][1157819] Updated weights for policy 0, policy_version 223398 (0.0006) [2024-09-30 03:52:33,606][1157819] Updated weights for policy 0, policy_version 223408 (0.0006) [2024-09-30 03:52:34,178][1157819] Updated weights for policy 0, policy_version 223418 (0.0006) [2024-09-30 03:52:34,702][1157819] Updated weights for policy 0, policy_version 223428 (0.0006) [2024-09-30 03:52:35,256][1157819] Updated weights for policy 0, policy_version 223438 (0.0006) [2024-09-30 03:52:35,466][1157520] Fps is (10 sec: 74137.5, 60 sec: 79121.2, 300 sec: 76380.0). Total num frames: 915214336. Throughput: 0: 19624.2. Samples: 218791432. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:52:35,466][1157520] Avg episode reward: [(0, '56.967')] [2024-09-30 03:52:35,836][1157819] Updated weights for policy 0, policy_version 223448 (0.0006) [2024-09-30 03:52:36,397][1157819] Updated weights for policy 0, policy_version 223458 (0.0006) [2024-09-30 03:52:36,970][1157819] Updated weights for policy 0, policy_version 223468 (0.0006) [2024-09-30 03:52:37,218][1157736] Signal inference workers to stop experience collection... (15700 times) [2024-09-30 03:52:37,219][1157736] Signal inference workers to resume experience collection... (15700 times) [2024-09-30 03:52:37,221][1157819] InferenceWorker_p0-w0: stopping experience collection (15700 times) [2024-09-30 03:52:37,223][1157819] InferenceWorker_p0-w0: resuming experience collection (15700 times) [2024-09-30 03:52:37,531][1157819] Updated weights for policy 0, policy_version 223478 (0.0006) [2024-09-30 03:52:38,082][1157819] Updated weights for policy 0, policy_version 223488 (0.0006) [2024-09-30 03:52:38,658][1157819] Updated weights for policy 0, policy_version 223498 (0.0006) [2024-09-30 03:52:39,218][1157819] Updated weights for policy 0, policy_version 223508 (0.0006) [2024-09-30 03:52:39,795][1157819] Updated weights for policy 0, policy_version 223518 (0.0006) [2024-09-30 03:52:40,343][1157819] Updated weights for policy 0, policy_version 223528 (0.0006) [2024-09-30 03:52:40,466][1157520] Fps is (10 sec: 72908.4, 60 sec: 78574.9, 300 sec: 76282.8). Total num frames: 915574784. Throughput: 0: 19479.4. Samples: 218846084. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:52:40,466][1157520] Avg episode reward: [(0, '57.134')] [2024-09-30 03:52:40,924][1157819] Updated weights for policy 0, policy_version 223538 (0.0006) [2024-09-30 03:52:41,492][1157819] Updated weights for policy 0, policy_version 223548 (0.0006) [2024-09-30 03:52:42,097][1157819] Updated weights for policy 0, policy_version 223558 (0.0006) [2024-09-30 03:52:42,707][1157819] Updated weights for policy 0, policy_version 223568 (0.0006) [2024-09-30 03:52:43,362][1157819] Updated weights for policy 0, policy_version 223578 (0.0006) [2024-09-30 03:52:44,012][1157819] Updated weights for policy 0, policy_version 223588 (0.0006) [2024-09-30 03:52:44,655][1157819] Updated weights for policy 0, policy_version 223598 (0.0006) [2024-09-30 03:52:45,293][1157819] Updated weights for policy 0, policy_version 223608 (0.0006) [2024-09-30 03:52:45,466][1157520] Fps is (10 sec: 69222.5, 60 sec: 77414.4, 300 sec: 76074.5). Total num frames: 915906560. Throughput: 0: 19122.3. Samples: 218950604. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:52:45,466][1157520] Avg episode reward: [(0, '54.428')] [2024-09-30 03:52:45,920][1157819] Updated weights for policy 0, policy_version 223618 (0.0006) [2024-09-30 03:52:46,595][1157819] Updated weights for policy 0, policy_version 223628 (0.0006) [2024-09-30 03:52:47,180][1157819] Updated weights for policy 0, policy_version 223638 (0.0006) [2024-09-30 03:52:47,762][1157819] Updated weights for policy 0, policy_version 223648 (0.0006) [2024-09-30 03:52:48,377][1157819] Updated weights for policy 0, policy_version 223658 (0.0006) [2024-09-30 03:52:48,964][1157819] Updated weights for policy 0, policy_version 223668 (0.0006) [2024-09-30 03:52:49,545][1157819] Updated weights for policy 0, policy_version 223678 (0.0005) [2024-09-30 03:52:50,119][1157819] Updated weights for policy 0, policy_version 223688 (0.0006) [2024-09-30 03:52:50,466][1157520] Fps is (10 sec: 67174.8, 60 sec: 75980.8, 300 sec: 75894.0). Total num frames: 916246528. Throughput: 0: 18784.0. Samples: 219050588. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:52:50,466][1157520] Avg episode reward: [(0, '56.404')] [2024-09-30 03:52:50,690][1157819] Updated weights for policy 0, policy_version 223698 (0.0006) [2024-09-30 03:52:51,303][1157819] Updated weights for policy 0, policy_version 223708 (0.0006) [2024-09-30 03:52:51,877][1157819] Updated weights for policy 0, policy_version 223718 (0.0006) [2024-09-30 03:52:52,467][1157819] Updated weights for policy 0, policy_version 223728 (0.0006) [2024-09-30 03:52:53,130][1157819] Updated weights for policy 0, policy_version 223738 (0.0006) [2024-09-30 03:52:53,704][1157819] Updated weights for policy 0, policy_version 223748 (0.0006) [2024-09-30 03:52:54,245][1157819] Updated weights for policy 0, policy_version 223758 (0.0006) [2024-09-30 03:52:54,791][1157819] Updated weights for policy 0, policy_version 223768 (0.0006) [2024-09-30 03:52:55,374][1157819] Updated weights for policy 0, policy_version 223778 (0.0006) [2024-09-30 03:52:55,466][1157520] Fps is (10 sec: 69631.9, 60 sec: 75161.7, 300 sec: 75769.1). Total num frames: 916602880. Throughput: 0: 18605.1. Samples: 219102384. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:52:55,466][1157520] Avg episode reward: [(0, '55.813')] [2024-09-30 03:52:55,888][1157819] Updated weights for policy 0, policy_version 223788 (0.0006) [2024-09-30 03:52:56,423][1157819] Updated weights for policy 0, policy_version 223798 (0.0006) [2024-09-30 03:52:56,994][1157819] Updated weights for policy 0, policy_version 223808 (0.0006) [2024-09-30 03:52:57,548][1157819] Updated weights for policy 0, policy_version 223818 (0.0006) [2024-09-30 03:52:58,113][1157819] Updated weights for policy 0, policy_version 223828 (0.0006) [2024-09-30 03:52:58,634][1157819] Updated weights for policy 0, policy_version 223838 (0.0006) [2024-09-30 03:52:59,185][1157819] Updated weights for policy 0, policy_version 223848 (0.0006) [2024-09-30 03:52:59,734][1157819] Updated weights for policy 0, policy_version 223858 (0.0006) [2024-09-30 03:53:00,277][1157819] Updated weights for policy 0, policy_version 223868 (0.0006) [2024-09-30 03:53:00,466][1157520] Fps is (10 sec: 72908.9, 60 sec: 74683.8, 300 sec: 75769.1). Total num frames: 916975616. Throughput: 0: 18413.3. Samples: 219212968. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:53:00,466][1157520] Avg episode reward: [(0, '56.392')] [2024-09-30 03:53:00,862][1157819] Updated weights for policy 0, policy_version 223878 (0.0006) [2024-09-30 03:53:01,400][1157819] Updated weights for policy 0, policy_version 223888 (0.0006) [2024-09-30 03:53:01,941][1157819] Updated weights for policy 0, policy_version 223898 (0.0006) [2024-09-30 03:53:02,525][1157819] Updated weights for policy 0, policy_version 223908 (0.0006) [2024-09-30 03:53:03,114][1157819] Updated weights for policy 0, policy_version 223918 (0.0006) [2024-09-30 03:53:03,687][1157819] Updated weights for policy 0, policy_version 223928 (0.0006) [2024-09-30 03:53:04,192][1157819] Updated weights for policy 0, policy_version 223938 (0.0006) [2024-09-30 03:53:04,672][1157736] Signal inference workers to stop experience collection... (15750 times) [2024-09-30 03:53:04,677][1157819] InferenceWorker_p0-w0: stopping experience collection (15750 times) [2024-09-30 03:53:04,677][1157736] Signal inference workers to resume experience collection... (15750 times) [2024-09-30 03:53:04,681][1157819] InferenceWorker_p0-w0: resuming experience collection (15750 times) [2024-09-30 03:53:04,697][1157819] Updated weights for policy 0, policy_version 223948 (0.0006) [2024-09-30 03:53:05,196][1157819] Updated weights for policy 0, policy_version 223958 (0.0006) [2024-09-30 03:53:05,466][1157520] Fps is (10 sec: 74547.2, 60 sec: 74615.5, 300 sec: 75727.4). Total num frames: 917348352. Throughput: 0: 18221.2. Samples: 219324756. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:53:05,466][1157520] Avg episode reward: [(0, '55.642')] [2024-09-30 03:53:05,711][1157819] Updated weights for policy 0, policy_version 223968 (0.0006) [2024-09-30 03:53:06,203][1157819] Updated weights for policy 0, policy_version 223978 (0.0006) [2024-09-30 03:53:06,696][1157819] Updated weights for policy 0, policy_version 223988 (0.0006) [2024-09-30 03:53:07,182][1157819] Updated weights for policy 0, policy_version 223998 (0.0006) [2024-09-30 03:53:07,680][1157819] Updated weights for policy 0, policy_version 224008 (0.0006) [2024-09-30 03:53:08,174][1157819] Updated weights for policy 0, policy_version 224018 (0.0006) [2024-09-30 03:53:08,662][1157819] Updated weights for policy 0, policy_version 224028 (0.0006) [2024-09-30 03:53:09,115][1157819] Updated weights for policy 0, policy_version 224038 (0.0006) [2024-09-30 03:53:09,573][1157819] Updated weights for policy 0, policy_version 224048 (0.0006) [2024-09-30 03:53:10,048][1157819] Updated weights for policy 0, policy_version 224058 (0.0006) [2024-09-30 03:53:10,466][1157520] Fps is (10 sec: 79871.7, 60 sec: 75025.1, 300 sec: 75796.8). Total num frames: 917774336. Throughput: 0: 18245.0. Samples: 219386712. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:53:10,466][1157520] Avg episode reward: [(0, '57.184')] [2024-09-30 03:53:10,520][1157819] Updated weights for policy 0, policy_version 224068 (0.0006) [2024-09-30 03:53:10,987][1157819] Updated weights for policy 0, policy_version 224078 (0.0006) [2024-09-30 03:53:11,442][1157819] Updated weights for policy 0, policy_version 224088 (0.0006) [2024-09-30 03:53:11,917][1157819] Updated weights for policy 0, policy_version 224098 (0.0006) [2024-09-30 03:53:12,372][1157819] Updated weights for policy 0, policy_version 224108 (0.0006) [2024-09-30 03:53:12,854][1157819] Updated weights for policy 0, policy_version 224118 (0.0006) [2024-09-30 03:53:13,331][1157819] Updated weights for policy 0, policy_version 224128 (0.0006) [2024-09-30 03:53:13,780][1157819] Updated weights for policy 0, policy_version 224138 (0.0006) [2024-09-30 03:53:14,254][1157819] Updated weights for policy 0, policy_version 224148 (0.0006) [2024-09-30 03:53:14,739][1157819] Updated weights for policy 0, policy_version 224158 (0.0007) [2024-09-30 03:53:15,248][1157819] Updated weights for policy 0, policy_version 224168 (0.0006) [2024-09-30 03:53:15,466][1157520] Fps is (10 sec: 86014.8, 60 sec: 75707.6, 300 sec: 75866.2). Total num frames: 918208512. Throughput: 0: 18598.8. Samples: 219517544. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:53:15,466][1157520] Avg episode reward: [(0, '56.142')] [2024-09-30 03:53:15,758][1157819] Updated weights for policy 0, policy_version 224178 (0.0006) [2024-09-30 03:53:16,248][1157819] Updated weights for policy 0, policy_version 224188 (0.0006) [2024-09-30 03:53:16,752][1157819] Updated weights for policy 0, policy_version 224198 (0.0006) [2024-09-30 03:53:17,252][1157819] Updated weights for policy 0, policy_version 224208 (0.0006) [2024-09-30 03:53:17,710][1157819] Updated weights for policy 0, policy_version 224218 (0.0006) [2024-09-30 03:53:18,220][1157819] Updated weights for policy 0, policy_version 224228 (0.0007) [2024-09-30 03:53:18,715][1157819] Updated weights for policy 0, policy_version 224238 (0.0006) [2024-09-30 03:53:19,215][1157819] Updated weights for policy 0, policy_version 224248 (0.0006) [2024-09-30 03:53:19,724][1157819] Updated weights for policy 0, policy_version 224258 (0.0006) [2024-09-30 03:53:20,256][1157819] Updated weights for policy 0, policy_version 224268 (0.0006) [2024-09-30 03:53:20,466][1157520] Fps is (10 sec: 84377.7, 60 sec: 75776.0, 300 sec: 75894.0). Total num frames: 918618112. Throughput: 0: 18885.6. Samples: 219641284. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:53:20,466][1157520] Avg episode reward: [(0, '55.159')] [2024-09-30 03:53:20,747][1157819] Updated weights for policy 0, policy_version 224278 (0.0006) [2024-09-30 03:53:21,261][1157819] Updated weights for policy 0, policy_version 224288 (0.0006) [2024-09-30 03:53:21,756][1157819] Updated weights for policy 0, policy_version 224298 (0.0006) [2024-09-30 03:53:22,262][1157819] Updated weights for policy 0, policy_version 224308 (0.0006) [2024-09-30 03:53:22,766][1157819] Updated weights for policy 0, policy_version 224318 (0.0006) [2024-09-30 03:53:23,269][1157819] Updated weights for policy 0, policy_version 224328 (0.0006) [2024-09-30 03:53:23,777][1157819] Updated weights for policy 0, policy_version 224338 (0.0006) [2024-09-30 03:53:24,255][1157819] Updated weights for policy 0, policy_version 224348 (0.0006) [2024-09-30 03:53:24,737][1157819] Updated weights for policy 0, policy_version 224358 (0.0006) [2024-09-30 03:53:25,260][1157819] Updated weights for policy 0, policy_version 224368 (0.0006) [2024-09-30 03:53:25,466][1157520] Fps is (10 sec: 81921.2, 60 sec: 75912.5, 300 sec: 76032.9). Total num frames: 919027712. Throughput: 0: 19019.2. Samples: 219701948. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:53:25,466][1157520] Avg episode reward: [(0, '55.289')] [2024-09-30 03:53:25,745][1157819] Updated weights for policy 0, policy_version 224378 (0.0006) [2024-09-30 03:53:26,253][1157819] Updated weights for policy 0, policy_version 224388 (0.0006) [2024-09-30 03:53:26,734][1157819] Updated weights for policy 0, policy_version 224398 (0.0006) [2024-09-30 03:53:27,153][1157736] Signal inference workers to stop experience collection... (15800 times) [2024-09-30 03:53:27,153][1157736] Signal inference workers to resume experience collection... (15800 times) [2024-09-30 03:53:27,157][1157819] InferenceWorker_p0-w0: stopping experience collection (15800 times) [2024-09-30 03:53:27,157][1157819] InferenceWorker_p0-w0: resuming experience collection (15800 times) [2024-09-30 03:53:27,237][1157819] Updated weights for policy 0, policy_version 224408 (0.0006) [2024-09-30 03:53:27,701][1157819] Updated weights for policy 0, policy_version 224418 (0.0006) [2024-09-30 03:53:28,199][1157819] Updated weights for policy 0, policy_version 224428 (0.0006) [2024-09-30 03:53:28,716][1157819] Updated weights for policy 0, policy_version 224438 (0.0007) [2024-09-30 03:53:29,239][1157819] Updated weights for policy 0, policy_version 224448 (0.0006) [2024-09-30 03:53:29,800][1157819] Updated weights for policy 0, policy_version 224458 (0.0006) [2024-09-30 03:53:30,313][1157819] Updated weights for policy 0, policy_version 224468 (0.0006) [2024-09-30 03:53:30,466][1157520] Fps is (10 sec: 81100.8, 60 sec: 76390.4, 300 sec: 76144.0). Total num frames: 919429120. Throughput: 0: 19444.8. Samples: 219825620. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:53:30,466][1157520] Avg episode reward: [(0, '56.861')] [2024-09-30 03:53:30,893][1157819] Updated weights for policy 0, policy_version 224478 (0.0006) [2024-09-30 03:53:31,433][1157819] Updated weights for policy 0, policy_version 224488 (0.0006) [2024-09-30 03:53:31,974][1157819] Updated weights for policy 0, policy_version 224498 (0.0006) [2024-09-30 03:53:32,559][1157819] Updated weights for policy 0, policy_version 224508 (0.0006) [2024-09-30 03:53:33,100][1157819] Updated weights for policy 0, policy_version 224518 (0.0006) [2024-09-30 03:53:33,617][1157819] Updated weights for policy 0, policy_version 224528 (0.0006) [2024-09-30 03:53:34,129][1157819] Updated weights for policy 0, policy_version 224538 (0.0006) [2024-09-30 03:53:34,672][1157819] Updated weights for policy 0, policy_version 224548 (0.0006) [2024-09-30 03:53:35,233][1157819] Updated weights for policy 0, policy_version 224558 (0.0006) [2024-09-30 03:53:35,466][1157520] Fps is (10 sec: 77823.9, 60 sec: 76526.9, 300 sec: 76171.7). Total num frames: 919805952. Throughput: 0: 19745.1. Samples: 219939116. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:53:35,466][1157520] Avg episode reward: [(0, '54.477')] [2024-09-30 03:53:35,773][1157819] Updated weights for policy 0, policy_version 224568 (0.0006) [2024-09-30 03:53:36,319][1157819] Updated weights for policy 0, policy_version 224578 (0.0006) [2024-09-30 03:53:36,872][1157819] Updated weights for policy 0, policy_version 224588 (0.0006) [2024-09-30 03:53:37,392][1157819] Updated weights for policy 0, policy_version 224598 (0.0006) [2024-09-30 03:53:37,903][1157819] Updated weights for policy 0, policy_version 224608 (0.0006) [2024-09-30 03:53:38,421][1157819] Updated weights for policy 0, policy_version 224618 (0.0006) [2024-09-30 03:53:38,946][1157819] Updated weights for policy 0, policy_version 224628 (0.0006) [2024-09-30 03:53:39,461][1157819] Updated weights for policy 0, policy_version 224638 (0.0006) [2024-09-30 03:53:39,965][1157819] Updated weights for policy 0, policy_version 224648 (0.0006) [2024-09-30 03:53:40,466][1157520] Fps is (10 sec: 76595.2, 60 sec: 77004.9, 300 sec: 76227.3). Total num frames: 920195072. Throughput: 0: 19868.3. Samples: 219996456. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:53:40,466][1157520] Avg episode reward: [(0, '56.668')] [2024-09-30 03:53:40,470][1157819] Updated weights for policy 0, policy_version 224658 (0.0006) [2024-09-30 03:53:40,983][1157819] Updated weights for policy 0, policy_version 224668 (0.0006) [2024-09-30 03:53:41,517][1157819] Updated weights for policy 0, policy_version 224678 (0.0006) [2024-09-30 03:53:42,055][1157819] Updated weights for policy 0, policy_version 224688 (0.0007) [2024-09-30 03:53:42,571][1157819] Updated weights for policy 0, policy_version 224698 (0.0006) [2024-09-30 03:53:43,087][1157819] Updated weights for policy 0, policy_version 224708 (0.0006) [2024-09-30 03:53:43,610][1157819] Updated weights for policy 0, policy_version 224718 (0.0006) [2024-09-30 03:53:44,127][1157819] Updated weights for policy 0, policy_version 224728 (0.0006) [2024-09-30 03:53:44,640][1157819] Updated weights for policy 0, policy_version 224738 (0.0006) [2024-09-30 03:53:45,143][1157819] Updated weights for policy 0, policy_version 224748 (0.0006) [2024-09-30 03:53:45,466][1157520] Fps is (10 sec: 78233.7, 60 sec: 78028.8, 300 sec: 76296.7). Total num frames: 920588288. Throughput: 0: 20043.9. Samples: 220114944. Policy #0 lag: (min: 0.0, avg: 2.3, max: 6.0) [2024-09-30 03:53:45,466][1157520] Avg episode reward: [(0, '57.517')] [2024-09-30 03:53:45,664][1157819] Updated weights for policy 0, policy_version 224758 (0.0007) [2024-09-30 03:53:46,196][1157819] Updated weights for policy 0, policy_version 224768 (0.0006) [2024-09-30 03:53:46,725][1157819] Updated weights for policy 0, policy_version 224778 (0.0006) [2024-09-30 03:53:47,237][1157819] Updated weights for policy 0, policy_version 224788 (0.0006) [2024-09-30 03:53:47,739][1157819] Updated weights for policy 0, policy_version 224798 (0.0006) [2024-09-30 03:53:48,292][1157819] Updated weights for policy 0, policy_version 224808 (0.0007) [2024-09-30 03:53:48,836][1157819] Updated weights for policy 0, policy_version 224818 (0.0006) [2024-09-30 03:53:49,337][1157819] Updated weights for policy 0, policy_version 224828 (0.0007) [2024-09-30 03:53:49,867][1157819] Updated weights for policy 0, policy_version 224838 (0.0007) [2024-09-30 03:53:50,368][1157819] Updated weights for policy 0, policy_version 224848 (0.0007) [2024-09-30 03:53:50,466][1157520] Fps is (10 sec: 78642.9, 60 sec: 78916.2, 300 sec: 76407.8). Total num frames: 920981504. Throughput: 0: 20174.7. Samples: 220232616. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:53:50,466][1157520] Avg episode reward: [(0, '55.850')] [2024-09-30 03:53:50,908][1157819] Updated weights for policy 0, policy_version 224858 (0.0006) [2024-09-30 03:53:51,416][1157819] Updated weights for policy 0, policy_version 224868 (0.0006) [2024-09-30 03:53:51,919][1157819] Updated weights for policy 0, policy_version 224878 (0.0006) [2024-09-30 03:53:52,440][1157819] Updated weights for policy 0, policy_version 224888 (0.0007) [2024-09-30 03:53:52,972][1157819] Updated weights for policy 0, policy_version 224898 (0.0006) [2024-09-30 03:53:53,450][1157819] Updated weights for policy 0, policy_version 224908 (0.0006) [2024-09-30 03:53:53,953][1157819] Updated weights for policy 0, policy_version 224918 (0.0006) [2024-09-30 03:53:54,446][1157819] Updated weights for policy 0, policy_version 224928 (0.0006) [2024-09-30 03:53:54,950][1157819] Updated weights for policy 0, policy_version 224938 (0.0007) [2024-09-30 03:53:55,017][1157736] Signal inference workers to stop experience collection... (15850 times) [2024-09-30 03:53:55,017][1157736] Signal inference workers to resume experience collection... (15850 times) [2024-09-30 03:53:55,022][1157819] InferenceWorker_p0-w0: stopping experience collection (15850 times) [2024-09-30 03:53:55,022][1157819] InferenceWorker_p0-w0: resuming experience collection (15850 times) [2024-09-30 03:53:55,452][1157819] Updated weights for policy 0, policy_version 224948 (0.0006) [2024-09-30 03:53:55,466][1157520] Fps is (10 sec: 79871.7, 60 sec: 79735.4, 300 sec: 76532.7). Total num frames: 921387008. Throughput: 0: 20117.2. Samples: 220291988. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:53:55,466][1157520] Avg episode reward: [(0, '55.405')] [2024-09-30 03:53:55,953][1157819] Updated weights for policy 0, policy_version 224958 (0.0006) [2024-09-30 03:53:56,472][1157819] Updated weights for policy 0, policy_version 224968 (0.0006) [2024-09-30 03:53:56,993][1157819] Updated weights for policy 0, policy_version 224978 (0.0006) [2024-09-30 03:53:57,493][1157819] Updated weights for policy 0, policy_version 224988 (0.0007) [2024-09-30 03:53:58,013][1157819] Updated weights for policy 0, policy_version 224998 (0.0006) [2024-09-30 03:53:58,531][1157819] Updated weights for policy 0, policy_version 225008 (0.0006) [2024-09-30 03:53:59,026][1157819] Updated weights for policy 0, policy_version 225018 (0.0006) [2024-09-30 03:53:59,537][1157819] Updated weights for policy 0, policy_version 225028 (0.0006) [2024-09-30 03:54:00,043][1157819] Updated weights for policy 0, policy_version 225038 (0.0006) [2024-09-30 03:54:00,466][1157520] Fps is (10 sec: 80691.7, 60 sec: 80213.3, 300 sec: 76713.2). Total num frames: 921788416. Throughput: 0: 19912.9. Samples: 220413620. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:54:00,466][1157520] Avg episode reward: [(0, '57.406')] [2024-09-30 03:54:00,562][1157819] Updated weights for policy 0, policy_version 225048 (0.0006) [2024-09-30 03:54:01,047][1157819] Updated weights for policy 0, policy_version 225058 (0.0006) [2024-09-30 03:54:01,579][1157819] Updated weights for policy 0, policy_version 225068 (0.0006) [2024-09-30 03:54:02,077][1157819] Updated weights for policy 0, policy_version 225078 (0.0006) [2024-09-30 03:54:02,599][1157819] Updated weights for policy 0, policy_version 225088 (0.0006) [2024-09-30 03:54:03,133][1157819] Updated weights for policy 0, policy_version 225098 (0.0006) [2024-09-30 03:54:03,629][1157819] Updated weights for policy 0, policy_version 225108 (0.0006) [2024-09-30 03:54:04,145][1157819] Updated weights for policy 0, policy_version 225118 (0.0007) [2024-09-30 03:54:04,638][1157819] Updated weights for policy 0, policy_version 225128 (0.0007) [2024-09-30 03:54:05,166][1157819] Updated weights for policy 0, policy_version 225138 (0.0006) [2024-09-30 03:54:05,466][1157520] Fps is (10 sec: 80281.7, 60 sec: 80691.2, 300 sec: 76893.7). Total num frames: 922189824. Throughput: 0: 19831.5. Samples: 220533704. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:54:05,466][1157520] Avg episode reward: [(0, '58.570')] [2024-09-30 03:54:05,472][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000225144_922189824.pth... [2024-09-30 03:54:05,527][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000220624_903675904.pth [2024-09-30 03:54:05,661][1157819] Updated weights for policy 0, policy_version 225148 (0.0007) [2024-09-30 03:54:06,140][1157819] Updated weights for policy 0, policy_version 225158 (0.0006) [2024-09-30 03:54:06,631][1157819] Updated weights for policy 0, policy_version 225168 (0.0006) [2024-09-30 03:54:07,129][1157819] Updated weights for policy 0, policy_version 225178 (0.0006) [2024-09-30 03:54:07,639][1157819] Updated weights for policy 0, policy_version 225188 (0.0006) [2024-09-30 03:54:08,100][1157819] Updated weights for policy 0, policy_version 225198 (0.0006) [2024-09-30 03:54:08,584][1157819] Updated weights for policy 0, policy_version 225208 (0.0006) [2024-09-30 03:54:09,108][1157819] Updated weights for policy 0, policy_version 225218 (0.0006) [2024-09-30 03:54:09,603][1157819] Updated weights for policy 0, policy_version 225228 (0.0006) [2024-09-30 03:54:10,071][1157819] Updated weights for policy 0, policy_version 225238 (0.0006) [2024-09-30 03:54:10,466][1157520] Fps is (10 sec: 81920.5, 60 sec: 80554.8, 300 sec: 77060.4). Total num frames: 922607616. Throughput: 0: 19867.6. Samples: 220595988. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:54:10,466][1157520] Avg episode reward: [(0, '55.209')] [2024-09-30 03:54:10,565][1157819] Updated weights for policy 0, policy_version 225248 (0.0006) [2024-09-30 03:54:11,046][1157819] Updated weights for policy 0, policy_version 225258 (0.0006) [2024-09-30 03:54:11,528][1157819] Updated weights for policy 0, policy_version 225268 (0.0006) [2024-09-30 03:54:12,025][1157819] Updated weights for policy 0, policy_version 225278 (0.0006) [2024-09-30 03:54:12,534][1157819] Updated weights for policy 0, policy_version 225288 (0.0006) [2024-09-30 03:54:12,999][1157819] Updated weights for policy 0, policy_version 225298 (0.0006) [2024-09-30 03:54:13,496][1157819] Updated weights for policy 0, policy_version 225308 (0.0006) [2024-09-30 03:54:13,994][1157819] Updated weights for policy 0, policy_version 225318 (0.0006) [2024-09-30 03:54:14,498][1157819] Updated weights for policy 0, policy_version 225328 (0.0006) [2024-09-30 03:54:15,020][1157819] Updated weights for policy 0, policy_version 225338 (0.0006) [2024-09-30 03:54:15,466][1157520] Fps is (10 sec: 83149.6, 60 sec: 80213.6, 300 sec: 77185.3). Total num frames: 923021312. Throughput: 0: 19904.7. Samples: 220721328. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:54:15,466][1157520] Avg episode reward: [(0, '55.135')] [2024-09-30 03:54:15,493][1157819] Updated weights for policy 0, policy_version 225348 (0.0006) [2024-09-30 03:54:15,992][1157819] Updated weights for policy 0, policy_version 225358 (0.0006) [2024-09-30 03:54:16,497][1157819] Updated weights for policy 0, policy_version 225368 (0.0006) [2024-09-30 03:54:16,978][1157819] Updated weights for policy 0, policy_version 225378 (0.0006) [2024-09-30 03:54:17,480][1157819] Updated weights for policy 0, policy_version 225388 (0.0006) [2024-09-30 03:54:17,974][1157819] Updated weights for policy 0, policy_version 225398 (0.0006) [2024-09-30 03:54:18,454][1157819] Updated weights for policy 0, policy_version 225408 (0.0006) [2024-09-30 03:54:18,962][1157819] Updated weights for policy 0, policy_version 225418 (0.0006) [2024-09-30 03:54:19,440][1157819] Updated weights for policy 0, policy_version 225428 (0.0006) [2024-09-30 03:54:19,918][1157819] Updated weights for policy 0, policy_version 225438 (0.0006) [2024-09-30 03:54:20,423][1157819] Updated weights for policy 0, policy_version 225448 (0.0006) [2024-09-30 03:54:20,466][1157520] Fps is (10 sec: 82739.3, 60 sec: 80281.7, 300 sec: 77310.3). Total num frames: 923435008. Throughput: 0: 20141.2. Samples: 220845468. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:54:20,466][1157520] Avg episode reward: [(0, '56.430')] [2024-09-30 03:54:20,928][1157819] Updated weights for policy 0, policy_version 225458 (0.0006) [2024-09-30 03:54:21,409][1157819] Updated weights for policy 0, policy_version 225468 (0.0006) [2024-09-30 03:54:21,896][1157819] Updated weights for policy 0, policy_version 225478 (0.0006) [2024-09-30 03:54:22,372][1157819] Updated weights for policy 0, policy_version 225488 (0.0006) [2024-09-30 03:54:22,868][1157819] Updated weights for policy 0, policy_version 225498 (0.0006) [2024-09-30 03:54:23,366][1157819] Updated weights for policy 0, policy_version 225508 (0.0006) [2024-09-30 03:54:23,869][1157819] Updated weights for policy 0, policy_version 225518 (0.0006) [2024-09-30 03:54:24,330][1157819] Updated weights for policy 0, policy_version 225528 (0.0006) [2024-09-30 03:54:24,839][1157819] Updated weights for policy 0, policy_version 225538 (0.0006) [2024-09-30 03:54:25,329][1157819] Updated weights for policy 0, policy_version 225548 (0.0006) [2024-09-30 03:54:25,466][1157520] Fps is (10 sec: 83558.7, 60 sec: 80486.5, 300 sec: 77463.0). Total num frames: 923856896. Throughput: 0: 20265.9. Samples: 220908420. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:54:25,466][1157520] Avg episode reward: [(0, '57.642')] [2024-09-30 03:54:25,839][1157819] Updated weights for policy 0, policy_version 225558 (0.0006) [2024-09-30 03:54:26,312][1157819] Updated weights for policy 0, policy_version 225568 (0.0006) [2024-09-30 03:54:26,837][1157819] Updated weights for policy 0, policy_version 225578 (0.0006) [2024-09-30 03:54:26,887][1157736] Signal inference workers to stop experience collection... (15900 times) [2024-09-30 03:54:26,888][1157736] Signal inference workers to resume experience collection... (15900 times) [2024-09-30 03:54:26,893][1157819] InferenceWorker_p0-w0: stopping experience collection (15900 times) [2024-09-30 03:54:26,893][1157819] InferenceWorker_p0-w0: resuming experience collection (15900 times) [2024-09-30 03:54:27,299][1157819] Updated weights for policy 0, policy_version 225588 (0.0006) [2024-09-30 03:54:27,798][1157819] Updated weights for policy 0, policy_version 225598 (0.0006) [2024-09-30 03:54:28,296][1157819] Updated weights for policy 0, policy_version 225608 (0.0006) [2024-09-30 03:54:28,794][1157819] Updated weights for policy 0, policy_version 225618 (0.0006) [2024-09-30 03:54:29,270][1157819] Updated weights for policy 0, policy_version 225628 (0.0006) [2024-09-30 03:54:29,763][1157819] Updated weights for policy 0, policy_version 225638 (0.0006) [2024-09-30 03:54:30,265][1157819] Updated weights for policy 0, policy_version 225648 (0.0006) [2024-09-30 03:54:30,466][1157520] Fps is (10 sec: 83558.6, 60 sec: 80691.4, 300 sec: 77601.9). Total num frames: 924270592. Throughput: 0: 20401.2. Samples: 221032996. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:54:30,466][1157520] Avg episode reward: [(0, '55.894')] [2024-09-30 03:54:30,748][1157819] Updated weights for policy 0, policy_version 225658 (0.0006) [2024-09-30 03:54:31,238][1157819] Updated weights for policy 0, policy_version 225668 (0.0006) [2024-09-30 03:54:31,732][1157819] Updated weights for policy 0, policy_version 225678 (0.0006) [2024-09-30 03:54:32,216][1157819] Updated weights for policy 0, policy_version 225688 (0.0006) [2024-09-30 03:54:32,728][1157819] Updated weights for policy 0, policy_version 225698 (0.0006) [2024-09-30 03:54:33,223][1157819] Updated weights for policy 0, policy_version 225708 (0.0006) [2024-09-30 03:54:33,706][1157819] Updated weights for policy 0, policy_version 225718 (0.0006) [2024-09-30 03:54:34,230][1157819] Updated weights for policy 0, policy_version 225728 (0.0006) [2024-09-30 03:54:34,731][1157819] Updated weights for policy 0, policy_version 225738 (0.0006) [2024-09-30 03:54:35,226][1157819] Updated weights for policy 0, policy_version 225748 (0.0006) [2024-09-30 03:54:35,466][1157520] Fps is (10 sec: 82329.6, 60 sec: 81237.5, 300 sec: 77685.2). Total num frames: 924680192. Throughput: 0: 20544.8. Samples: 221157128. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:54:35,466][1157520] Avg episode reward: [(0, '56.631')] [2024-09-30 03:54:35,723][1157819] Updated weights for policy 0, policy_version 225758 (0.0006) [2024-09-30 03:54:36,235][1157819] Updated weights for policy 0, policy_version 225768 (0.0006) [2024-09-30 03:54:36,707][1157819] Updated weights for policy 0, policy_version 225778 (0.0006) [2024-09-30 03:54:37,217][1157819] Updated weights for policy 0, policy_version 225788 (0.0006) [2024-09-30 03:54:37,725][1157819] Updated weights for policy 0, policy_version 225798 (0.0006) [2024-09-30 03:54:38,204][1157819] Updated weights for policy 0, policy_version 225808 (0.0006) [2024-09-30 03:54:38,710][1157819] Updated weights for policy 0, policy_version 225818 (0.0006) [2024-09-30 03:54:39,205][1157819] Updated weights for policy 0, policy_version 225828 (0.0006) [2024-09-30 03:54:39,715][1157819] Updated weights for policy 0, policy_version 225838 (0.0006) [2024-09-30 03:54:40,220][1157819] Updated weights for policy 0, policy_version 225848 (0.0006) [2024-09-30 03:54:40,466][1157520] Fps is (10 sec: 81919.8, 60 sec: 81578.8, 300 sec: 77740.7). Total num frames: 925089792. Throughput: 0: 20586.5. Samples: 221218376. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:54:40,466][1157520] Avg episode reward: [(0, '56.358')] [2024-09-30 03:54:40,732][1157819] Updated weights for policy 0, policy_version 225858 (0.0006) [2024-09-30 03:54:40,993][1157736] Signal inference workers to stop experience collection... (15950 times) [2024-09-30 03:54:40,995][1157819] InferenceWorker_p0-w0: stopping experience collection (15950 times) [2024-09-30 03:54:40,999][1157736] Signal inference workers to resume experience collection... (15950 times) [2024-09-30 03:54:41,000][1157819] InferenceWorker_p0-w0: resuming experience collection (15950 times) [2024-09-30 03:54:41,234][1157819] Updated weights for policy 0, policy_version 225868 (0.0006) [2024-09-30 03:54:41,738][1157819] Updated weights for policy 0, policy_version 225878 (0.0006) [2024-09-30 03:54:42,237][1157819] Updated weights for policy 0, policy_version 225888 (0.0006) [2024-09-30 03:54:42,724][1157819] Updated weights for policy 0, policy_version 225898 (0.0006) [2024-09-30 03:54:43,230][1157819] Updated weights for policy 0, policy_version 225908 (0.0006) [2024-09-30 03:54:43,734][1157819] Updated weights for policy 0, policy_version 225918 (0.0006) [2024-09-30 03:54:44,242][1157819] Updated weights for policy 0, policy_version 225928 (0.0007) [2024-09-30 03:54:44,733][1157819] Updated weights for policy 0, policy_version 225938 (0.0006) [2024-09-30 03:54:45,232][1157819] Updated weights for policy 0, policy_version 225948 (0.0006) [2024-09-30 03:54:45,466][1157520] Fps is (10 sec: 81919.8, 60 sec: 81851.8, 300 sec: 77837.9). Total num frames: 925499392. Throughput: 0: 20613.4. Samples: 221341220. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:54:45,466][1157520] Avg episode reward: [(0, '56.385')] [2024-09-30 03:54:45,735][1157819] Updated weights for policy 0, policy_version 225958 (0.0006) [2024-09-30 03:54:46,216][1157819] Updated weights for policy 0, policy_version 225968 (0.0006) [2024-09-30 03:54:46,700][1157819] Updated weights for policy 0, policy_version 225978 (0.0006) [2024-09-30 03:54:47,204][1157819] Updated weights for policy 0, policy_version 225988 (0.0006) [2024-09-30 03:54:47,667][1157819] Updated weights for policy 0, policy_version 225998 (0.0006) [2024-09-30 03:54:48,163][1157819] Updated weights for policy 0, policy_version 226008 (0.0006) [2024-09-30 03:54:48,658][1157819] Updated weights for policy 0, policy_version 226018 (0.0006) [2024-09-30 03:54:49,163][1157819] Updated weights for policy 0, policy_version 226028 (0.0006) [2024-09-30 03:54:49,652][1157819] Updated weights for policy 0, policy_version 226038 (0.0006) [2024-09-30 03:54:50,208][1157819] Updated weights for policy 0, policy_version 226048 (0.0006) [2024-09-30 03:54:50,466][1157520] Fps is (10 sec: 81918.9, 60 sec: 82124.8, 300 sec: 77949.0). Total num frames: 925908992. Throughput: 0: 20701.7. Samples: 221465280. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 03:54:50,466][1157520] Avg episode reward: [(0, '55.736')] [2024-09-30 03:54:50,727][1157819] Updated weights for policy 0, policy_version 226058 (0.0006) [2024-09-30 03:54:51,292][1157819] Updated weights for policy 0, policy_version 226068 (0.0006) [2024-09-30 03:54:51,892][1157819] Updated weights for policy 0, policy_version 226078 (0.0006) [2024-09-30 03:54:52,467][1157819] Updated weights for policy 0, policy_version 226088 (0.0006) [2024-09-30 03:54:53,043][1157819] Updated weights for policy 0, policy_version 226098 (0.0006) [2024-09-30 03:54:53,630][1157819] Updated weights for policy 0, policy_version 226108 (0.0006) [2024-09-30 03:54:54,205][1157819] Updated weights for policy 0, policy_version 226118 (0.0006) [2024-09-30 03:54:54,805][1157819] Updated weights for policy 0, policy_version 226128 (0.0006) [2024-09-30 03:54:55,388][1157819] Updated weights for policy 0, policy_version 226138 (0.0006) [2024-09-30 03:54:55,466][1157520] Fps is (10 sec: 76594.6, 60 sec: 81305.7, 300 sec: 77865.6). Total num frames: 926265344. Throughput: 0: 20530.8. Samples: 221519876. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:54:55,466][1157520] Avg episode reward: [(0, '55.666')] [2024-09-30 03:54:55,978][1157819] Updated weights for policy 0, policy_version 226148 (0.0006) [2024-09-30 03:54:56,589][1157819] Updated weights for policy 0, policy_version 226158 (0.0006) [2024-09-30 03:54:57,170][1157819] Updated weights for policy 0, policy_version 226168 (0.0006) [2024-09-30 03:54:57,740][1157819] Updated weights for policy 0, policy_version 226178 (0.0006) [2024-09-30 03:54:58,312][1157819] Updated weights for policy 0, policy_version 226188 (0.0006) [2024-09-30 03:54:58,910][1157819] Updated weights for policy 0, policy_version 226198 (0.0006) [2024-09-30 03:54:59,442][1157819] Updated weights for policy 0, policy_version 226208 (0.0006) [2024-09-30 03:55:00,055][1157819] Updated weights for policy 0, policy_version 226218 (0.0006) [2024-09-30 03:55:00,466][1157520] Fps is (10 sec: 70861.3, 60 sec: 80486.4, 300 sec: 77768.5). Total num frames: 926617600. Throughput: 0: 20083.5. Samples: 221625088. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:55:00,466][1157520] Avg episode reward: [(0, '56.135')] [2024-09-30 03:55:00,620][1157819] Updated weights for policy 0, policy_version 226228 (0.0006) [2024-09-30 03:55:01,192][1157819] Updated weights for policy 0, policy_version 226238 (0.0006) [2024-09-30 03:55:01,733][1157819] Updated weights for policy 0, policy_version 226248 (0.0006) [2024-09-30 03:55:02,281][1157819] Updated weights for policy 0, policy_version 226258 (0.0006) [2024-09-30 03:55:02,837][1157819] Updated weights for policy 0, policy_version 226268 (0.0006) [2024-09-30 03:55:03,405][1157819] Updated weights for policy 0, policy_version 226278 (0.0006) [2024-09-30 03:55:03,967][1157819] Updated weights for policy 0, policy_version 226288 (0.0006) [2024-09-30 03:55:04,525][1157819] Updated weights for policy 0, policy_version 226298 (0.0006) [2024-09-30 03:55:05,087][1157819] Updated weights for policy 0, policy_version 226308 (0.0006) [2024-09-30 03:55:05,466][1157520] Fps is (10 sec: 71679.1, 60 sec: 79871.9, 300 sec: 77685.1). Total num frames: 926982144. Throughput: 0: 19736.5. Samples: 221733616. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:55:05,466][1157520] Avg episode reward: [(0, '58.085')] [2024-09-30 03:55:05,689][1157819] Updated weights for policy 0, policy_version 226318 (0.0006) [2024-09-30 03:55:06,254][1157819] Updated weights for policy 0, policy_version 226328 (0.0006) [2024-09-30 03:55:06,807][1157819] Updated weights for policy 0, policy_version 226338 (0.0006) [2024-09-30 03:55:07,398][1157819] Updated weights for policy 0, policy_version 226348 (0.0006) [2024-09-30 03:55:07,938][1157819] Updated weights for policy 0, policy_version 226358 (0.0006) [2024-09-30 03:55:08,529][1157819] Updated weights for policy 0, policy_version 226368 (0.0006) [2024-09-30 03:55:09,088][1157819] Updated weights for policy 0, policy_version 226378 (0.0006) [2024-09-30 03:55:09,635][1157819] Updated weights for policy 0, policy_version 226388 (0.0006) [2024-09-30 03:55:09,852][1157736] Signal inference workers to stop experience collection... (16000 times) [2024-09-30 03:55:09,852][1157736] Signal inference workers to resume experience collection... (16000 times) [2024-09-30 03:55:09,855][1157819] InferenceWorker_p0-w0: stopping experience collection (16000 times) [2024-09-30 03:55:09,858][1157819] InferenceWorker_p0-w0: resuming experience collection (16000 times) [2024-09-30 03:55:10,220][1157819] Updated weights for policy 0, policy_version 226398 (0.0006) [2024-09-30 03:55:10,466][1157520] Fps is (10 sec: 72498.6, 60 sec: 78916.1, 300 sec: 77560.2). Total num frames: 927342592. Throughput: 0: 19537.6. Samples: 221787616. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:55:10,466][1157520] Avg episode reward: [(0, '54.997')] [2024-09-30 03:55:10,785][1157819] Updated weights for policy 0, policy_version 226408 (0.0007) [2024-09-30 03:55:11,404][1157819] Updated weights for policy 0, policy_version 226418 (0.0006) [2024-09-30 03:55:11,968][1157819] Updated weights for policy 0, policy_version 226428 (0.0006) [2024-09-30 03:55:12,532][1157819] Updated weights for policy 0, policy_version 226438 (0.0006) [2024-09-30 03:55:13,155][1157819] Updated weights for policy 0, policy_version 226448 (0.0006) [2024-09-30 03:55:13,684][1157819] Updated weights for policy 0, policy_version 226458 (0.0006) [2024-09-30 03:55:14,214][1157819] Updated weights for policy 0, policy_version 226468 (0.0006) [2024-09-30 03:55:14,740][1157819] Updated weights for policy 0, policy_version 226478 (0.0006) [2024-09-30 03:55:15,319][1157819] Updated weights for policy 0, policy_version 226488 (0.0006) [2024-09-30 03:55:15,466][1157520] Fps is (10 sec: 72500.1, 60 sec: 78097.0, 300 sec: 77518.5). Total num frames: 927707136. Throughput: 0: 19156.8. Samples: 221895056. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:55:15,466][1157520] Avg episode reward: [(0, '56.880')] [2024-09-30 03:55:15,855][1157819] Updated weights for policy 0, policy_version 226498 (0.0006) [2024-09-30 03:55:16,365][1157819] Updated weights for policy 0, policy_version 226508 (0.0006) [2024-09-30 03:55:16,901][1157819] Updated weights for policy 0, policy_version 226518 (0.0006) [2024-09-30 03:55:17,414][1157819] Updated weights for policy 0, policy_version 226528 (0.0006) [2024-09-30 03:55:17,917][1157819] Updated weights for policy 0, policy_version 226538 (0.0006) [2024-09-30 03:55:18,481][1157819] Updated weights for policy 0, policy_version 226548 (0.0006) [2024-09-30 03:55:19,027][1157819] Updated weights for policy 0, policy_version 226558 (0.0006) [2024-09-30 03:55:19,558][1157819] Updated weights for policy 0, policy_version 226568 (0.0006) [2024-09-30 03:55:20,058][1157819] Updated weights for policy 0, policy_version 226578 (0.0006) [2024-09-30 03:55:20,466][1157520] Fps is (10 sec: 75367.0, 60 sec: 77687.4, 300 sec: 77518.5). Total num frames: 928096256. Throughput: 0: 18971.3. Samples: 222010836. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:55:20,466][1157520] Avg episode reward: [(0, '55.744')] [2024-09-30 03:55:20,572][1157819] Updated weights for policy 0, policy_version 226588 (0.0006) [2024-09-30 03:55:21,103][1157819] Updated weights for policy 0, policy_version 226598 (0.0006) [2024-09-30 03:55:21,633][1157819] Updated weights for policy 0, policy_version 226608 (0.0006) [2024-09-30 03:55:22,216][1157819] Updated weights for policy 0, policy_version 226618 (0.0006) [2024-09-30 03:55:22,758][1157819] Updated weights for policy 0, policy_version 226628 (0.0006) [2024-09-30 03:55:23,334][1157819] Updated weights for policy 0, policy_version 226638 (0.0006) [2024-09-30 03:55:23,863][1157819] Updated weights for policy 0, policy_version 226648 (0.0006) [2024-09-30 03:55:24,386][1157819] Updated weights for policy 0, policy_version 226658 (0.0006) [2024-09-30 03:55:24,971][1157819] Updated weights for policy 0, policy_version 226668 (0.0006) [2024-09-30 03:55:25,466][1157520] Fps is (10 sec: 75775.1, 60 sec: 76799.7, 300 sec: 77435.2). Total num frames: 928464896. Throughput: 0: 18868.2. Samples: 222067448. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:55:25,466][1157520] Avg episode reward: [(0, '56.060')] [2024-09-30 03:55:25,510][1157819] Updated weights for policy 0, policy_version 226678 (0.0006) [2024-09-30 03:55:26,085][1157819] Updated weights for policy 0, policy_version 226688 (0.0006) [2024-09-30 03:55:26,633][1157819] Updated weights for policy 0, policy_version 226698 (0.0006) [2024-09-30 03:55:27,164][1157819] Updated weights for policy 0, policy_version 226708 (0.0006) [2024-09-30 03:55:27,740][1157819] Updated weights for policy 0, policy_version 226718 (0.0006) [2024-09-30 03:55:28,253][1157819] Updated weights for policy 0, policy_version 226728 (0.0006) [2024-09-30 03:55:28,785][1157819] Updated weights for policy 0, policy_version 226738 (0.0006) [2024-09-30 03:55:29,345][1157819] Updated weights for policy 0, policy_version 226748 (0.0006) [2024-09-30 03:55:29,905][1157819] Updated weights for policy 0, policy_version 226758 (0.0006) [2024-09-30 03:55:30,418][1157819] Updated weights for policy 0, policy_version 226768 (0.0006) [2024-09-30 03:55:30,466][1157520] Fps is (10 sec: 74546.5, 60 sec: 76185.4, 300 sec: 77365.8). Total num frames: 928841728. Throughput: 0: 18623.0. Samples: 222179256. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:55:30,466][1157520] Avg episode reward: [(0, '55.648')] [2024-09-30 03:55:30,941][1157819] Updated weights for policy 0, policy_version 226778 (0.0006) [2024-09-30 03:55:31,454][1157819] Updated weights for policy 0, policy_version 226788 (0.0006) [2024-09-30 03:55:31,939][1157819] Updated weights for policy 0, policy_version 226798 (0.0006) [2024-09-30 03:55:32,443][1157819] Updated weights for policy 0, policy_version 226808 (0.0006) [2024-09-30 03:55:32,950][1157819] Updated weights for policy 0, policy_version 226818 (0.0006) [2024-09-30 03:55:33,450][1157819] Updated weights for policy 0, policy_version 226828 (0.0006) [2024-09-30 03:55:34,001][1157819] Updated weights for policy 0, policy_version 226838 (0.0006) [2024-09-30 03:55:34,530][1157819] Updated weights for policy 0, policy_version 226848 (0.0006) [2024-09-30 03:55:35,078][1157819] Updated weights for policy 0, policy_version 226858 (0.0006) [2024-09-30 03:55:35,466][1157520] Fps is (10 sec: 77824.9, 60 sec: 76048.9, 300 sec: 77379.7). Total num frames: 929243136. Throughput: 0: 18493.8. Samples: 222297500. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:55:35,466][1157520] Avg episode reward: [(0, '54.904')] [2024-09-30 03:55:35,578][1157819] Updated weights for policy 0, policy_version 226868 (0.0006) [2024-09-30 03:55:36,115][1157819] Updated weights for policy 0, policy_version 226878 (0.0006) [2024-09-30 03:55:36,615][1157819] Updated weights for policy 0, policy_version 226888 (0.0006) [2024-09-30 03:55:37,116][1157819] Updated weights for policy 0, policy_version 226898 (0.0006) [2024-09-30 03:55:37,632][1157819] Updated weights for policy 0, policy_version 226908 (0.0006) [2024-09-30 03:55:37,794][1157736] Signal inference workers to stop experience collection... (16050 times) [2024-09-30 03:55:37,795][1157736] Signal inference workers to resume experience collection... (16050 times) [2024-09-30 03:55:37,800][1157819] InferenceWorker_p0-w0: stopping experience collection (16050 times) [2024-09-30 03:55:37,800][1157819] InferenceWorker_p0-w0: resuming experience collection (16050 times) [2024-09-30 03:55:38,136][1157819] Updated weights for policy 0, policy_version 226918 (0.0006) [2024-09-30 03:55:38,650][1157819] Updated weights for policy 0, policy_version 226928 (0.0006) [2024-09-30 03:55:39,170][1157819] Updated weights for policy 0, policy_version 226938 (0.0006) [2024-09-30 03:55:39,646][1157819] Updated weights for policy 0, policy_version 226948 (0.0006) [2024-09-30 03:55:40,144][1157819] Updated weights for policy 0, policy_version 226958 (0.0006) [2024-09-30 03:55:40,466][1157520] Fps is (10 sec: 80282.4, 60 sec: 75912.5, 300 sec: 77407.5). Total num frames: 929644544. Throughput: 0: 18605.2. Samples: 222357108. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:55:40,466][1157520] Avg episode reward: [(0, '56.367')] [2024-09-30 03:55:40,647][1157819] Updated weights for policy 0, policy_version 226968 (0.0006) [2024-09-30 03:55:41,144][1157819] Updated weights for policy 0, policy_version 226978 (0.0006) [2024-09-30 03:55:41,625][1157819] Updated weights for policy 0, policy_version 226988 (0.0006) [2024-09-30 03:55:42,132][1157819] Updated weights for policy 0, policy_version 226998 (0.0006) [2024-09-30 03:55:42,630][1157819] Updated weights for policy 0, policy_version 227008 (0.0006) [2024-09-30 03:55:43,132][1157819] Updated weights for policy 0, policy_version 227018 (0.0006) [2024-09-30 03:55:43,622][1157819] Updated weights for policy 0, policy_version 227028 (0.0006) [2024-09-30 03:55:44,112][1157819] Updated weights for policy 0, policy_version 227038 (0.0006) [2024-09-30 03:55:44,617][1157819] Updated weights for policy 0, policy_version 227048 (0.0006) [2024-09-30 03:55:45,098][1157819] Updated weights for policy 0, policy_version 227058 (0.0006) [2024-09-30 03:55:45,466][1157520] Fps is (10 sec: 81510.6, 60 sec: 75980.7, 300 sec: 77601.9). Total num frames: 930058240. Throughput: 0: 19000.2. Samples: 222480096. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:55:45,466][1157520] Avg episode reward: [(0, '55.909')] [2024-09-30 03:55:45,589][1157819] Updated weights for policy 0, policy_version 227068 (0.0006) [2024-09-30 03:55:46,110][1157819] Updated weights for policy 0, policy_version 227078 (0.0006) [2024-09-30 03:55:46,619][1157819] Updated weights for policy 0, policy_version 227088 (0.0006) [2024-09-30 03:55:47,147][1157819] Updated weights for policy 0, policy_version 227098 (0.0006) [2024-09-30 03:55:47,660][1157819] Updated weights for policy 0, policy_version 227108 (0.0006) [2024-09-30 03:55:48,188][1157819] Updated weights for policy 0, policy_version 227118 (0.0006) [2024-09-30 03:55:48,684][1157819] Updated weights for policy 0, policy_version 227128 (0.0006) [2024-09-30 03:55:49,223][1157819] Updated weights for policy 0, policy_version 227138 (0.0006) [2024-09-30 03:55:49,745][1157819] Updated weights for policy 0, policy_version 227148 (0.0006) [2024-09-30 03:55:50,264][1157819] Updated weights for policy 0, policy_version 227158 (0.0006) [2024-09-30 03:55:50,466][1157520] Fps is (10 sec: 80690.9, 60 sec: 75707.8, 300 sec: 77726.8). Total num frames: 930451456. Throughput: 0: 19262.8. Samples: 222600440. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:55:50,466][1157520] Avg episode reward: [(0, '57.836')] [2024-09-30 03:55:50,807][1157819] Updated weights for policy 0, policy_version 227168 (0.0006) [2024-09-30 03:55:51,332][1157819] Updated weights for policy 0, policy_version 227178 (0.0006) [2024-09-30 03:55:51,846][1157819] Updated weights for policy 0, policy_version 227188 (0.0006) [2024-09-30 03:55:52,382][1157819] Updated weights for policy 0, policy_version 227198 (0.0006) [2024-09-30 03:55:52,902][1157819] Updated weights for policy 0, policy_version 227208 (0.0006) [2024-09-30 03:55:53,429][1157819] Updated weights for policy 0, policy_version 227218 (0.0006) [2024-09-30 03:55:53,981][1157819] Updated weights for policy 0, policy_version 227228 (0.0006) [2024-09-30 03:55:54,513][1157819] Updated weights for policy 0, policy_version 227238 (0.0006) [2024-09-30 03:55:55,034][1157819] Updated weights for policy 0, policy_version 227248 (0.0006) [2024-09-30 03:55:55,466][1157520] Fps is (10 sec: 77824.0, 60 sec: 76185.6, 300 sec: 77768.5). Total num frames: 930836480. Throughput: 0: 19366.8. Samples: 222659120. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:55:55,466][1157520] Avg episode reward: [(0, '54.607')] [2024-09-30 03:55:55,603][1157819] Updated weights for policy 0, policy_version 227258 (0.0006) [2024-09-30 03:55:56,184][1157819] Updated weights for policy 0, policy_version 227268 (0.0006) [2024-09-30 03:55:56,710][1157819] Updated weights for policy 0, policy_version 227278 (0.0006) [2024-09-30 03:55:57,301][1157819] Updated weights for policy 0, policy_version 227288 (0.0006) [2024-09-30 03:55:57,855][1157819] Updated weights for policy 0, policy_version 227298 (0.0006) [2024-09-30 03:55:58,386][1157819] Updated weights for policy 0, policy_version 227308 (0.0006) [2024-09-30 03:55:58,942][1157819] Updated weights for policy 0, policy_version 227318 (0.0006) [2024-09-30 03:55:59,475][1157819] Updated weights for policy 0, policy_version 227328 (0.0006) [2024-09-30 03:56:00,008][1157819] Updated weights for policy 0, policy_version 227338 (0.0006) [2024-09-30 03:56:00,466][1157520] Fps is (10 sec: 75776.3, 60 sec: 76527.0, 300 sec: 77782.4). Total num frames: 931209216. Throughput: 0: 19463.6. Samples: 222770916. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:56:00,466][1157520] Avg episode reward: [(0, '56.090')] [2024-09-30 03:56:00,510][1157819] Updated weights for policy 0, policy_version 227348 (0.0006) [2024-09-30 03:56:01,023][1157819] Updated weights for policy 0, policy_version 227358 (0.0006) [2024-09-30 03:56:01,529][1157819] Updated weights for policy 0, policy_version 227368 (0.0006) [2024-09-30 03:56:02,044][1157819] Updated weights for policy 0, policy_version 227378 (0.0006) [2024-09-30 03:56:02,568][1157819] Updated weights for policy 0, policy_version 227388 (0.0006) [2024-09-30 03:56:03,142][1157819] Updated weights for policy 0, policy_version 227398 (0.0006) [2024-09-30 03:56:03,653][1157819] Updated weights for policy 0, policy_version 227408 (0.0006) [2024-09-30 03:56:04,213][1157819] Updated weights for policy 0, policy_version 227418 (0.0006) [2024-09-30 03:56:04,732][1157819] Updated weights for policy 0, policy_version 227428 (0.0006) [2024-09-30 03:56:05,242][1157819] Updated weights for policy 0, policy_version 227438 (0.0006) [2024-09-30 03:56:05,466][1157520] Fps is (10 sec: 76185.0, 60 sec: 76936.6, 300 sec: 77921.2). Total num frames: 931598336. Throughput: 0: 19478.6. Samples: 222887376. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:56:05,466][1157520] Avg episode reward: [(0, '55.499')] [2024-09-30 03:56:05,472][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000227442_931602432.pth... [2024-09-30 03:56:05,531][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000222870_912875520.pth [2024-09-30 03:56:05,784][1157819] Updated weights for policy 0, policy_version 227448 (0.0006) [2024-09-30 03:56:06,285][1157819] Updated weights for policy 0, policy_version 227458 (0.0006) [2024-09-30 03:56:06,800][1157819] Updated weights for policy 0, policy_version 227468 (0.0006) [2024-09-30 03:56:07,336][1157819] Updated weights for policy 0, policy_version 227478 (0.0006) [2024-09-30 03:56:07,839][1157819] Updated weights for policy 0, policy_version 227488 (0.0006) [2024-09-30 03:56:08,333][1157819] Updated weights for policy 0, policy_version 227498 (0.0006) [2024-09-30 03:56:08,852][1157819] Updated weights for policy 0, policy_version 227508 (0.0006) [2024-09-30 03:56:09,372][1157819] Updated weights for policy 0, policy_version 227518 (0.0006) [2024-09-30 03:56:09,885][1157819] Updated weights for policy 0, policy_version 227528 (0.0006) [2024-09-30 03:56:10,365][1157736] Signal inference workers to stop experience collection... (16100 times) [2024-09-30 03:56:10,366][1157736] Signal inference workers to resume experience collection... (16100 times) [2024-09-30 03:56:10,369][1157819] InferenceWorker_p0-w0: stopping experience collection (16100 times) [2024-09-30 03:56:10,369][1157819] InferenceWorker_p0-w0: resuming experience collection (16100 times) [2024-09-30 03:56:10,379][1157819] Updated weights for policy 0, policy_version 227538 (0.0006) [2024-09-30 03:56:10,466][1157520] Fps is (10 sec: 79052.5, 60 sec: 77619.3, 300 sec: 78087.8). Total num frames: 931999744. Throughput: 0: 19548.9. Samples: 222947148. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:56:10,466][1157520] Avg episode reward: [(0, '57.478')] [2024-09-30 03:56:10,877][1157819] Updated weights for policy 0, policy_version 227548 (0.0006) [2024-09-30 03:56:11,410][1157819] Updated weights for policy 0, policy_version 227558 (0.0006) [2024-09-30 03:56:11,939][1157819] Updated weights for policy 0, policy_version 227568 (0.0006) [2024-09-30 03:56:12,446][1157819] Updated weights for policy 0, policy_version 227578 (0.0006) [2024-09-30 03:56:12,956][1157819] Updated weights for policy 0, policy_version 227588 (0.0006) [2024-09-30 03:56:13,492][1157819] Updated weights for policy 0, policy_version 227598 (0.0006) [2024-09-30 03:56:14,003][1157819] Updated weights for policy 0, policy_version 227608 (0.0006) [2024-09-30 03:56:14,502][1157819] Updated weights for policy 0, policy_version 227618 (0.0006) [2024-09-30 03:56:15,005][1157819] Updated weights for policy 0, policy_version 227628 (0.0006) [2024-09-30 03:56:15,466][1157520] Fps is (10 sec: 80281.9, 60 sec: 78233.6, 300 sec: 78240.5). Total num frames: 932401152. Throughput: 0: 19713.9. Samples: 223066380. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:56:15,466][1157520] Avg episode reward: [(0, '57.380')] [2024-09-30 03:56:15,502][1157819] Updated weights for policy 0, policy_version 227638 (0.0006) [2024-09-30 03:56:16,010][1157819] Updated weights for policy 0, policy_version 227648 (0.0006) [2024-09-30 03:56:16,503][1157819] Updated weights for policy 0, policy_version 227658 (0.0006) [2024-09-30 03:56:17,006][1157819] Updated weights for policy 0, policy_version 227668 (0.0006) [2024-09-30 03:56:17,521][1157819] Updated weights for policy 0, policy_version 227678 (0.0006) [2024-09-30 03:56:18,011][1157819] Updated weights for policy 0, policy_version 227688 (0.0006) [2024-09-30 03:56:18,526][1157819] Updated weights for policy 0, policy_version 227698 (0.0006) [2024-09-30 03:56:19,025][1157819] Updated weights for policy 0, policy_version 227708 (0.0006) [2024-09-30 03:56:19,527][1157819] Updated weights for policy 0, policy_version 227718 (0.0006) [2024-09-30 03:56:19,999][1157819] Updated weights for policy 0, policy_version 227728 (0.0006) [2024-09-30 03:56:20,466][1157520] Fps is (10 sec: 81101.0, 60 sec: 78574.9, 300 sec: 78351.6). Total num frames: 932810752. Throughput: 0: 19815.1. Samples: 223189180. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:56:20,466][1157520] Avg episode reward: [(0, '56.671')] [2024-09-30 03:56:20,497][1157819] Updated weights for policy 0, policy_version 227738 (0.0006) [2024-09-30 03:56:20,970][1157819] Updated weights for policy 0, policy_version 227748 (0.0006) [2024-09-30 03:56:21,449][1157819] Updated weights for policy 0, policy_version 227758 (0.0006) [2024-09-30 03:56:21,990][1157819] Updated weights for policy 0, policy_version 227768 (0.0006) [2024-09-30 03:56:22,494][1157819] Updated weights for policy 0, policy_version 227778 (0.0006) [2024-09-30 03:56:22,981][1157819] Updated weights for policy 0, policy_version 227788 (0.0006) [2024-09-30 03:56:23,466][1157819] Updated weights for policy 0, policy_version 227798 (0.0006) [2024-09-30 03:56:23,944][1157819] Updated weights for policy 0, policy_version 227808 (0.0006) [2024-09-30 03:56:24,437][1157819] Updated weights for policy 0, policy_version 227818 (0.0006) [2024-09-30 03:56:24,933][1157819] Updated weights for policy 0, policy_version 227828 (0.0006) [2024-09-30 03:56:25,427][1157819] Updated weights for policy 0, policy_version 227838 (0.0006) [2024-09-30 03:56:25,466][1157520] Fps is (10 sec: 82329.8, 60 sec: 79326.0, 300 sec: 78476.6). Total num frames: 933224448. Throughput: 0: 19863.5. Samples: 223250968. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:56:25,466][1157520] Avg episode reward: [(0, '56.540')] [2024-09-30 03:56:25,877][1157819] Updated weights for policy 0, policy_version 227848 (0.0006) [2024-09-30 03:56:26,373][1157819] Updated weights for policy 0, policy_version 227858 (0.0006) [2024-09-30 03:56:26,866][1157819] Updated weights for policy 0, policy_version 227868 (0.0006) [2024-09-30 03:56:27,364][1157819] Updated weights for policy 0, policy_version 227878 (0.0006) [2024-09-30 03:56:27,889][1157819] Updated weights for policy 0, policy_version 227888 (0.0007) [2024-09-30 03:56:28,406][1157819] Updated weights for policy 0, policy_version 227898 (0.0007) [2024-09-30 03:56:28,986][1157819] Updated weights for policy 0, policy_version 227908 (0.0007) [2024-09-30 03:56:29,534][1157819] Updated weights for policy 0, policy_version 227918 (0.0007) [2024-09-30 03:56:30,058][1157819] Updated weights for policy 0, policy_version 227928 (0.0007) [2024-09-30 03:56:30,466][1157520] Fps is (10 sec: 81100.6, 60 sec: 79667.3, 300 sec: 78490.5). Total num frames: 933621760. Throughput: 0: 19866.9. Samples: 223374108. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:56:30,466][1157520] Avg episode reward: [(0, '57.741')] [2024-09-30 03:56:30,568][1157819] Updated weights for policy 0, policy_version 227938 (0.0007) [2024-09-30 03:56:31,097][1157819] Updated weights for policy 0, policy_version 227948 (0.0007) [2024-09-30 03:56:31,618][1157819] Updated weights for policy 0, policy_version 227958 (0.0007) [2024-09-30 03:56:32,136][1157819] Updated weights for policy 0, policy_version 227968 (0.0007) [2024-09-30 03:56:32,652][1157819] Updated weights for policy 0, policy_version 227978 (0.0006) [2024-09-30 03:56:33,168][1157819] Updated weights for policy 0, policy_version 227988 (0.0006) [2024-09-30 03:56:33,655][1157819] Updated weights for policy 0, policy_version 227998 (0.0007) [2024-09-30 03:56:34,156][1157819] Updated weights for policy 0, policy_version 228008 (0.0006) [2024-09-30 03:56:34,662][1157819] Updated weights for policy 0, policy_version 228018 (0.0006) [2024-09-30 03:56:35,200][1157819] Updated weights for policy 0, policy_version 228028 (0.0006) [2024-09-30 03:56:35,466][1157520] Fps is (10 sec: 79871.8, 60 sec: 79667.2, 300 sec: 78518.2). Total num frames: 934023168. Throughput: 0: 19816.8. Samples: 223492196. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:56:35,466][1157520] Avg episode reward: [(0, '57.346')] [2024-09-30 03:56:35,737][1157819] Updated weights for policy 0, policy_version 228038 (0.0006) [2024-09-30 03:56:36,232][1157819] Updated weights for policy 0, policy_version 228048 (0.0006) [2024-09-30 03:56:36,725][1157819] Updated weights for policy 0, policy_version 228058 (0.0006) [2024-09-30 03:56:37,229][1157819] Updated weights for policy 0, policy_version 228068 (0.0006) [2024-09-30 03:56:37,737][1157819] Updated weights for policy 0, policy_version 228078 (0.0006) [2024-09-30 03:56:38,266][1157819] Updated weights for policy 0, policy_version 228088 (0.0006) [2024-09-30 03:56:38,779][1157819] Updated weights for policy 0, policy_version 228098 (0.0006) [2024-09-30 03:56:39,333][1157819] Updated weights for policy 0, policy_version 228108 (0.0006) [2024-09-30 03:56:39,876][1157819] Updated weights for policy 0, policy_version 228118 (0.0006) [2024-09-30 03:56:40,397][1157819] Updated weights for policy 0, policy_version 228128 (0.0006) [2024-09-30 03:56:40,466][1157520] Fps is (10 sec: 79462.1, 60 sec: 79530.6, 300 sec: 78490.5). Total num frames: 934416384. Throughput: 0: 19853.0. Samples: 223552508. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:56:40,466][1157520] Avg episode reward: [(0, '55.914')] [2024-09-30 03:56:40,954][1157819] Updated weights for policy 0, policy_version 228138 (0.0006) [2024-09-30 03:56:41,524][1157819] Updated weights for policy 0, policy_version 228148 (0.0006) [2024-09-30 03:56:42,066][1157819] Updated weights for policy 0, policy_version 228158 (0.0006) [2024-09-30 03:56:42,593][1157819] Updated weights for policy 0, policy_version 228168 (0.0006) [2024-09-30 03:56:43,093][1157819] Updated weights for policy 0, policy_version 228178 (0.0006) [2024-09-30 03:56:43,598][1157819] Updated weights for policy 0, policy_version 228188 (0.0006) [2024-09-30 03:56:44,114][1157819] Updated weights for policy 0, policy_version 228198 (0.0006) [2024-09-30 03:56:44,620][1157819] Updated weights for policy 0, policy_version 228208 (0.0006) [2024-09-30 03:56:45,148][1157819] Updated weights for policy 0, policy_version 228218 (0.0006) [2024-09-30 03:56:45,466][1157520] Fps is (10 sec: 78234.0, 60 sec: 79121.1, 300 sec: 78365.5). Total num frames: 934805504. Throughput: 0: 19940.0. Samples: 223668216. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:56:45,466][1157520] Avg episode reward: [(0, '56.373')] [2024-09-30 03:56:45,665][1157819] Updated weights for policy 0, policy_version 228228 (0.0006) [2024-09-30 03:56:46,159][1157819] Updated weights for policy 0, policy_version 228238 (0.0006) [2024-09-30 03:56:46,662][1157819] Updated weights for policy 0, policy_version 228248 (0.0006) [2024-09-30 03:56:46,986][1157736] Signal inference workers to stop experience collection... (16150 times) [2024-09-30 03:56:46,987][1157736] Signal inference workers to resume experience collection... (16150 times) [2024-09-30 03:56:46,991][1157819] InferenceWorker_p0-w0: stopping experience collection (16150 times) [2024-09-30 03:56:46,991][1157819] InferenceWorker_p0-w0: resuming experience collection (16150 times) [2024-09-30 03:56:47,155][1157819] Updated weights for policy 0, policy_version 228258 (0.0006) [2024-09-30 03:56:47,644][1157819] Updated weights for policy 0, policy_version 228268 (0.0006) [2024-09-30 03:56:48,128][1157819] Updated weights for policy 0, policy_version 228278 (0.0006) [2024-09-30 03:56:48,625][1157819] Updated weights for policy 0, policy_version 228288 (0.0006) [2024-09-30 03:56:49,110][1157819] Updated weights for policy 0, policy_version 228298 (0.0006) [2024-09-30 03:56:49,583][1157819] Updated weights for policy 0, policy_version 228308 (0.0006) [2024-09-30 03:56:50,093][1157819] Updated weights for policy 0, policy_version 228318 (0.0006) [2024-09-30 03:56:50,466][1157520] Fps is (10 sec: 80691.7, 60 sec: 79530.7, 300 sec: 78407.2). Total num frames: 935223296. Throughput: 0: 20094.5. Samples: 223791628. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:56:50,466][1157520] Avg episode reward: [(0, '56.112')] [2024-09-30 03:56:50,572][1157819] Updated weights for policy 0, policy_version 228328 (0.0006) [2024-09-30 03:56:50,994][1157819] Updated weights for policy 0, policy_version 228338 (0.0006) [2024-09-30 03:56:51,482][1157819] Updated weights for policy 0, policy_version 228348 (0.0006) [2024-09-30 03:56:51,952][1157819] Updated weights for policy 0, policy_version 228358 (0.0006) [2024-09-30 03:56:52,438][1157819] Updated weights for policy 0, policy_version 228368 (0.0006) [2024-09-30 03:56:52,861][1157819] Updated weights for policy 0, policy_version 228378 (0.0006) [2024-09-30 03:56:53,335][1157819] Updated weights for policy 0, policy_version 228388 (0.0006) [2024-09-30 03:56:53,821][1157819] Updated weights for policy 0, policy_version 228398 (0.0006) [2024-09-30 03:56:54,292][1157819] Updated weights for policy 0, policy_version 228408 (0.0006) [2024-09-30 03:56:54,728][1157819] Updated weights for policy 0, policy_version 228418 (0.0006) [2024-09-30 03:56:55,186][1157819] Updated weights for policy 0, policy_version 228428 (0.0006) [2024-09-30 03:56:55,466][1157520] Fps is (10 sec: 85606.1, 60 sec: 80418.1, 300 sec: 78532.1). Total num frames: 935661568. Throughput: 0: 20220.1. Samples: 223857052. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:56:55,466][1157520] Avg episode reward: [(0, '57.079')] [2024-09-30 03:56:55,658][1157819] Updated weights for policy 0, policy_version 228438 (0.0006) [2024-09-30 03:56:56,135][1157819] Updated weights for policy 0, policy_version 228448 (0.0006) [2024-09-30 03:56:56,536][1157819] Updated weights for policy 0, policy_version 228458 (0.0006) [2024-09-30 03:56:57,034][1157819] Updated weights for policy 0, policy_version 228468 (0.0006) [2024-09-30 03:56:57,494][1157819] Updated weights for policy 0, policy_version 228478 (0.0006) [2024-09-30 03:56:57,941][1157819] Updated weights for policy 0, policy_version 228488 (0.0006) [2024-09-30 03:56:58,393][1157819] Updated weights for policy 0, policy_version 228498 (0.0006) [2024-09-30 03:56:58,878][1157819] Updated weights for policy 0, policy_version 228508 (0.0006) [2024-09-30 03:56:59,344][1157819] Updated weights for policy 0, policy_version 228518 (0.0006) [2024-09-30 03:56:59,788][1157819] Updated weights for policy 0, policy_version 228528 (0.0006) [2024-09-30 03:57:00,245][1157819] Updated weights for policy 0, policy_version 228538 (0.0006) [2024-09-30 03:57:00,466][1157520] Fps is (10 sec: 88473.8, 60 sec: 81646.9, 300 sec: 78768.2). Total num frames: 936108032. Throughput: 0: 20532.2. Samples: 223990328. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:57:00,466][1157520] Avg episode reward: [(0, '54.754')] [2024-09-30 03:57:00,721][1157819] Updated weights for policy 0, policy_version 228548 (0.0006) [2024-09-30 03:57:01,204][1157819] Updated weights for policy 0, policy_version 228558 (0.0006) [2024-09-30 03:57:01,642][1157819] Updated weights for policy 0, policy_version 228568 (0.0006) [2024-09-30 03:57:02,107][1157819] Updated weights for policy 0, policy_version 228578 (0.0006) [2024-09-30 03:57:02,568][1157819] Updated weights for policy 0, policy_version 228588 (0.0006) [2024-09-30 03:57:03,062][1157819] Updated weights for policy 0, policy_version 228598 (0.0006) [2024-09-30 03:57:03,468][1157819] Updated weights for policy 0, policy_version 228608 (0.0006) [2024-09-30 03:57:03,920][1157819] Updated weights for policy 0, policy_version 228618 (0.0006) [2024-09-30 03:57:04,392][1157819] Updated weights for policy 0, policy_version 228628 (0.0006) [2024-09-30 03:57:04,834][1157819] Updated weights for policy 0, policy_version 228638 (0.0006) [2024-09-30 03:57:05,276][1157819] Updated weights for policy 0, policy_version 228648 (0.0006) [2024-09-30 03:57:05,466][1157520] Fps is (10 sec: 89701.4, 60 sec: 82670.9, 300 sec: 78934.7). Total num frames: 936558592. Throughput: 0: 20781.4. Samples: 224124348. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:57:05,466][1157520] Avg episode reward: [(0, '55.607')] [2024-09-30 03:57:05,748][1157819] Updated weights for policy 0, policy_version 228658 (0.0006) [2024-09-30 03:57:06,187][1157819] Updated weights for policy 0, policy_version 228668 (0.0006) [2024-09-30 03:57:06,634][1157819] Updated weights for policy 0, policy_version 228678 (0.0006) [2024-09-30 03:57:07,080][1157819] Updated weights for policy 0, policy_version 228688 (0.0006) [2024-09-30 03:57:07,543][1157819] Updated weights for policy 0, policy_version 228698 (0.0006) [2024-09-30 03:57:08,004][1157819] Updated weights for policy 0, policy_version 228708 (0.0006) [2024-09-30 03:57:08,442][1157819] Updated weights for policy 0, policy_version 228718 (0.0006) [2024-09-30 03:57:08,888][1157819] Updated weights for policy 0, policy_version 228728 (0.0006) [2024-09-30 03:57:09,341][1157819] Updated weights for policy 0, policy_version 228738 (0.0006) [2024-09-30 03:57:09,807][1157819] Updated weights for policy 0, policy_version 228748 (0.0006) [2024-09-30 03:57:10,260][1157819] Updated weights for policy 0, policy_version 228758 (0.0006) [2024-09-30 03:57:10,466][1157520] Fps is (10 sec: 90111.8, 60 sec: 83490.2, 300 sec: 79129.2). Total num frames: 937009152. Throughput: 0: 20918.7. Samples: 224192308. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:57:10,466][1157520] Avg episode reward: [(0, '58.121')] [2024-09-30 03:57:10,694][1157819] Updated weights for policy 0, policy_version 228768 (0.0006) [2024-09-30 03:57:11,164][1157819] Updated weights for policy 0, policy_version 228778 (0.0006) [2024-09-30 03:57:11,626][1157819] Updated weights for policy 0, policy_version 228788 (0.0006) [2024-09-30 03:57:12,051][1157819] Updated weights for policy 0, policy_version 228798 (0.0006) [2024-09-30 03:57:12,509][1157819] Updated weights for policy 0, policy_version 228808 (0.0006) [2024-09-30 03:57:13,006][1157819] Updated weights for policy 0, policy_version 228818 (0.0006) [2024-09-30 03:57:13,414][1157819] Updated weights for policy 0, policy_version 228828 (0.0006) [2024-09-30 03:57:13,502][1157736] Signal inference workers to stop experience collection... (16200 times) [2024-09-30 03:57:13,503][1157736] Signal inference workers to resume experience collection... (16200 times) [2024-09-30 03:57:13,507][1157819] InferenceWorker_p0-w0: stopping experience collection (16200 times) [2024-09-30 03:57:13,507][1157819] InferenceWorker_p0-w0: resuming experience collection (16200 times) [2024-09-30 03:57:13,866][1157819] Updated weights for policy 0, policy_version 228838 (0.0007) [2024-09-30 03:57:14,342][1157819] Updated weights for policy 0, policy_version 228848 (0.0006) [2024-09-30 03:57:14,763][1157819] Updated weights for policy 0, policy_version 228858 (0.0006) [2024-09-30 03:57:15,220][1157819] Updated weights for policy 0, policy_version 228868 (0.0006) [2024-09-30 03:57:15,466][1157520] Fps is (10 sec: 90522.2, 60 sec: 84377.6, 300 sec: 79295.8). Total num frames: 937463808. Throughput: 0: 21200.2. Samples: 224328116. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 03:57:15,466][1157520] Avg episode reward: [(0, '56.935')] [2024-09-30 03:57:15,669][1157819] Updated weights for policy 0, policy_version 228878 (0.0006) [2024-09-30 03:57:16,123][1157819] Updated weights for policy 0, policy_version 228888 (0.0006) [2024-09-30 03:57:16,573][1157819] Updated weights for policy 0, policy_version 228898 (0.0006) [2024-09-30 03:57:17,036][1157819] Updated weights for policy 0, policy_version 228908 (0.0006) [2024-09-30 03:57:17,470][1157819] Updated weights for policy 0, policy_version 228918 (0.0006) [2024-09-30 03:57:17,923][1157819] Updated weights for policy 0, policy_version 228928 (0.0006) [2024-09-30 03:57:18,393][1157819] Updated weights for policy 0, policy_version 228938 (0.0006) [2024-09-30 03:57:18,814][1157819] Updated weights for policy 0, policy_version 228948 (0.0006) [2024-09-30 03:57:19,280][1157819] Updated weights for policy 0, policy_version 228958 (0.0006) [2024-09-30 03:57:19,741][1157819] Updated weights for policy 0, policy_version 228968 (0.0006) [2024-09-30 03:57:20,162][1157819] Updated weights for policy 0, policy_version 228978 (0.0006) [2024-09-30 03:57:20,466][1157520] Fps is (10 sec: 90931.0, 60 sec: 85128.5, 300 sec: 79476.3). Total num frames: 937918464. Throughput: 0: 21606.8. Samples: 224464500. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 03:57:20,466][1157520] Avg episode reward: [(0, '54.204')] [2024-09-30 03:57:20,643][1157819] Updated weights for policy 0, policy_version 228988 (0.0006) [2024-09-30 03:57:21,099][1157819] Updated weights for policy 0, policy_version 228998 (0.0006) [2024-09-30 03:57:21,523][1157819] Updated weights for policy 0, policy_version 229008 (0.0006) [2024-09-30 03:57:21,997][1157819] Updated weights for policy 0, policy_version 229018 (0.0006) [2024-09-30 03:57:22,425][1157819] Updated weights for policy 0, policy_version 229028 (0.0006) [2024-09-30 03:57:22,891][1157819] Updated weights for policy 0, policy_version 229038 (0.0006) [2024-09-30 03:57:23,384][1157819] Updated weights for policy 0, policy_version 229048 (0.0006) [2024-09-30 03:57:23,871][1157819] Updated weights for policy 0, policy_version 229058 (0.0006) [2024-09-30 03:57:24,369][1157819] Updated weights for policy 0, policy_version 229068 (0.0006) [2024-09-30 03:57:24,845][1157819] Updated weights for policy 0, policy_version 229078 (0.0006) [2024-09-30 03:57:25,337][1157819] Updated weights for policy 0, policy_version 229088 (0.0006) [2024-09-30 03:57:25,466][1157520] Fps is (10 sec: 89291.4, 60 sec: 85537.8, 300 sec: 79698.4). Total num frames: 938356736. Throughput: 0: 21769.3. Samples: 224532132. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 03:57:25,466][1157520] Avg episode reward: [(0, '55.414')] [2024-09-30 03:57:25,767][1157819] Updated weights for policy 0, policy_version 229098 (0.0006) [2024-09-30 03:57:26,237][1157819] Updated weights for policy 0, policy_version 229108 (0.0006) [2024-09-30 03:57:26,708][1157819] Updated weights for policy 0, policy_version 229118 (0.0006) [2024-09-30 03:57:27,209][1157819] Updated weights for policy 0, policy_version 229128 (0.0006) [2024-09-30 03:57:27,671][1157819] Updated weights for policy 0, policy_version 229138 (0.0006) [2024-09-30 03:57:28,099][1157819] Updated weights for policy 0, policy_version 229148 (0.0006) [2024-09-30 03:57:28,585][1157819] Updated weights for policy 0, policy_version 229158 (0.0006) [2024-09-30 03:57:29,078][1157819] Updated weights for policy 0, policy_version 229168 (0.0006) [2024-09-30 03:57:29,564][1157819] Updated weights for policy 0, policy_version 229178 (0.0006) [2024-09-30 03:57:30,031][1157819] Updated weights for policy 0, policy_version 229188 (0.0006) [2024-09-30 03:57:30,466][1157520] Fps is (10 sec: 87244.9, 60 sec: 86152.5, 300 sec: 79920.6). Total num frames: 938790912. Throughput: 0: 22075.5. Samples: 224661616. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 03:57:30,466][1157520] Avg episode reward: [(0, '58.173')] [2024-09-30 03:57:30,522][1157819] Updated weights for policy 0, policy_version 229198 (0.0006) [2024-09-30 03:57:30,953][1157819] Updated weights for policy 0, policy_version 229208 (0.0006) [2024-09-30 03:57:31,427][1157819] Updated weights for policy 0, policy_version 229218 (0.0006) [2024-09-30 03:57:31,915][1157736] Signal inference workers to stop experience collection... (16250 times) [2024-09-30 03:57:31,919][1157819] InferenceWorker_p0-w0: stopping experience collection (16250 times) [2024-09-30 03:57:31,921][1157736] Signal inference workers to resume experience collection... (16250 times) [2024-09-30 03:57:31,922][1157819] InferenceWorker_p0-w0: resuming experience collection (16250 times) [2024-09-30 03:57:31,923][1157819] Updated weights for policy 0, policy_version 229228 (0.0006) [2024-09-30 03:57:32,408][1157819] Updated weights for policy 0, policy_version 229238 (0.0006) [2024-09-30 03:57:32,879][1157819] Updated weights for policy 0, policy_version 229248 (0.0006) [2024-09-30 03:57:33,389][1157819] Updated weights for policy 0, policy_version 229258 (0.0006) [2024-09-30 03:57:33,887][1157819] Updated weights for policy 0, policy_version 229268 (0.0006) [2024-09-30 03:57:34,377][1157819] Updated weights for policy 0, policy_version 229278 (0.0006) [2024-09-30 03:57:34,863][1157819] Updated weights for policy 0, policy_version 229288 (0.0006) [2024-09-30 03:57:35,373][1157819] Updated weights for policy 0, policy_version 229298 (0.0006) [2024-09-30 03:57:35,466][1157520] Fps is (10 sec: 85608.3, 60 sec: 86493.9, 300 sec: 80128.9). Total num frames: 939212800. Throughput: 0: 22175.2. Samples: 224789512. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 03:57:35,466][1157520] Avg episode reward: [(0, '57.241')] [2024-09-30 03:57:35,864][1157819] Updated weights for policy 0, policy_version 229308 (0.0006) [2024-09-30 03:57:36,365][1157819] Updated weights for policy 0, policy_version 229318 (0.0006) [2024-09-30 03:57:36,851][1157819] Updated weights for policy 0, policy_version 229328 (0.0006) [2024-09-30 03:57:37,378][1157819] Updated weights for policy 0, policy_version 229338 (0.0006) [2024-09-30 03:57:37,834][1157819] Updated weights for policy 0, policy_version 229348 (0.0006) [2024-09-30 03:57:38,331][1157819] Updated weights for policy 0, policy_version 229358 (0.0006) [2024-09-30 03:57:38,841][1157819] Updated weights for policy 0, policy_version 229368 (0.0006) [2024-09-30 03:57:39,321][1157819] Updated weights for policy 0, policy_version 229378 (0.0006) [2024-09-30 03:57:39,810][1157819] Updated weights for policy 0, policy_version 229388 (0.0006) [2024-09-30 03:57:40,309][1157819] Updated weights for policy 0, policy_version 229398 (0.0006) [2024-09-30 03:57:40,466][1157520] Fps is (10 sec: 83148.5, 60 sec: 86767.0, 300 sec: 80392.7). Total num frames: 939622400. Throughput: 0: 22088.5. Samples: 224851036. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 03:57:40,466][1157520] Avg episode reward: [(0, '56.717')] [2024-09-30 03:57:40,814][1157819] Updated weights for policy 0, policy_version 229408 (0.0006) [2024-09-30 03:57:41,316][1157819] Updated weights for policy 0, policy_version 229418 (0.0006) [2024-09-30 03:57:41,824][1157819] Updated weights for policy 0, policy_version 229428 (0.0007) [2024-09-30 03:57:42,315][1157819] Updated weights for policy 0, policy_version 229438 (0.0006) [2024-09-30 03:57:42,789][1157819] Updated weights for policy 0, policy_version 229448 (0.0006) [2024-09-30 03:57:43,286][1157819] Updated weights for policy 0, policy_version 229458 (0.0006) [2024-09-30 03:57:43,781][1157819] Updated weights for policy 0, policy_version 229468 (0.0006) [2024-09-30 03:57:44,295][1157819] Updated weights for policy 0, policy_version 229478 (0.0006) [2024-09-30 03:57:44,787][1157819] Updated weights for policy 0, policy_version 229488 (0.0006) [2024-09-30 03:57:45,281][1157819] Updated weights for policy 0, policy_version 229498 (0.0006) [2024-09-30 03:57:45,466][1157520] Fps is (10 sec: 82329.4, 60 sec: 87176.5, 300 sec: 80642.6). Total num frames: 940036096. Throughput: 0: 21888.8. Samples: 224975324. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 03:57:45,466][1157520] Avg episode reward: [(0, '55.869')] [2024-09-30 03:57:45,772][1157819] Updated weights for policy 0, policy_version 229508 (0.0006) [2024-09-30 03:57:46,257][1157819] Updated weights for policy 0, policy_version 229518 (0.0006) [2024-09-30 03:57:46,733][1157819] Updated weights for policy 0, policy_version 229528 (0.0006) [2024-09-30 03:57:47,218][1157819] Updated weights for policy 0, policy_version 229538 (0.0006) [2024-09-30 03:57:47,713][1157819] Updated weights for policy 0, policy_version 229548 (0.0006) [2024-09-30 03:57:48,204][1157819] Updated weights for policy 0, policy_version 229558 (0.0006) [2024-09-30 03:57:48,667][1157819] Updated weights for policy 0, policy_version 229568 (0.0006) [2024-09-30 03:57:49,175][1157819] Updated weights for policy 0, policy_version 229578 (0.0006) [2024-09-30 03:57:49,676][1157819] Updated weights for policy 0, policy_version 229588 (0.0006) [2024-09-30 03:57:50,168][1157819] Updated weights for policy 0, policy_version 229598 (0.0006) [2024-09-30 03:57:50,466][1157520] Fps is (10 sec: 83558.6, 60 sec: 87244.8, 300 sec: 80864.8). Total num frames: 940457984. Throughput: 0: 21702.1. Samples: 225100940. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 03:57:50,466][1157520] Avg episode reward: [(0, '54.976')] [2024-09-30 03:57:50,666][1157819] Updated weights for policy 0, policy_version 229608 (0.0006) [2024-09-30 03:57:51,167][1157819] Updated weights for policy 0, policy_version 229618 (0.0006) [2024-09-30 03:57:51,669][1157819] Updated weights for policy 0, policy_version 229628 (0.0006) [2024-09-30 03:57:52,195][1157819] Updated weights for policy 0, policy_version 229638 (0.0006) [2024-09-30 03:57:52,689][1157819] Updated weights for policy 0, policy_version 229648 (0.0006) [2024-09-30 03:57:53,193][1157819] Updated weights for policy 0, policy_version 229658 (0.0006) [2024-09-30 03:57:53,691][1157819] Updated weights for policy 0, policy_version 229668 (0.0006) [2024-09-30 03:57:54,216][1157819] Updated weights for policy 0, policy_version 229678 (0.0006) [2024-09-30 03:57:54,694][1157819] Updated weights for policy 0, policy_version 229688 (0.0006) [2024-09-30 03:57:55,216][1157819] Updated weights for policy 0, policy_version 229698 (0.0006) [2024-09-30 03:57:55,466][1157520] Fps is (10 sec: 82329.7, 60 sec: 86630.4, 300 sec: 80961.9). Total num frames: 940859392. Throughput: 0: 21540.0. Samples: 225161608. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 03:57:55,466][1157520] Avg episode reward: [(0, '55.284')] [2024-09-30 03:57:55,752][1157819] Updated weights for policy 0, policy_version 229708 (0.0006) [2024-09-30 03:57:56,275][1157819] Updated weights for policy 0, policy_version 229718 (0.0006) [2024-09-30 03:57:56,770][1157819] Updated weights for policy 0, policy_version 229728 (0.0006) [2024-09-30 03:57:57,280][1157819] Updated weights for policy 0, policy_version 229738 (0.0006) [2024-09-30 03:57:57,805][1157819] Updated weights for policy 0, policy_version 229748 (0.0006) [2024-09-30 03:57:58,307][1157819] Updated weights for policy 0, policy_version 229758 (0.0006) [2024-09-30 03:57:58,810][1157819] Updated weights for policy 0, policy_version 229768 (0.0006) [2024-09-30 03:57:59,307][1157819] Updated weights for policy 0, policy_version 229778 (0.0006) [2024-09-30 03:57:59,831][1157819] Updated weights for policy 0, policy_version 229788 (0.0006) [2024-09-30 03:58:00,323][1157819] Updated weights for policy 0, policy_version 229798 (0.0006) [2024-09-30 03:58:00,466][1157520] Fps is (10 sec: 80281.1, 60 sec: 85879.3, 300 sec: 81059.1). Total num frames: 941260800. Throughput: 0: 21193.4. Samples: 225281820. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 03:58:00,466][1157520] Avg episode reward: [(0, '55.299')] [2024-09-30 03:58:00,838][1157819] Updated weights for policy 0, policy_version 229808 (0.0006) [2024-09-30 03:58:01,352][1157819] Updated weights for policy 0, policy_version 229818 (0.0006) [2024-09-30 03:58:01,870][1157819] Updated weights for policy 0, policy_version 229828 (0.0006) [2024-09-30 03:58:02,356][1157819] Updated weights for policy 0, policy_version 229838 (0.0006) [2024-09-30 03:58:02,883][1157819] Updated weights for policy 0, policy_version 229848 (0.0006) [2024-09-30 03:58:03,423][1157819] Updated weights for policy 0, policy_version 229858 (0.0006) [2024-09-30 03:58:03,929][1157819] Updated weights for policy 0, policy_version 229868 (0.0006) [2024-09-30 03:58:04,451][1157819] Updated weights for policy 0, policy_version 229878 (0.0006) [2024-09-30 03:58:04,958][1157819] Updated weights for policy 0, policy_version 229888 (0.0006) [2024-09-30 03:58:05,301][1157736] Signal inference workers to stop experience collection... (16300 times) [2024-09-30 03:58:05,302][1157736] Signal inference workers to resume experience collection... (16300 times) [2024-09-30 03:58:05,306][1157819] InferenceWorker_p0-w0: stopping experience collection (16300 times) [2024-09-30 03:58:05,308][1157819] InferenceWorker_p0-w0: resuming experience collection (16300 times) [2024-09-30 03:58:05,466][1157520] Fps is (10 sec: 80281.0, 60 sec: 85060.3, 300 sec: 80975.8). Total num frames: 941662208. Throughput: 0: 20839.1. Samples: 225402260. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 03:58:05,466][1157819] Updated weights for policy 0, policy_version 229898 (0.0006) [2024-09-30 03:58:05,466][1157520] Avg episode reward: [(0, '54.365')] [2024-09-30 03:58:05,482][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000229899_941666304.pth... [2024-09-30 03:58:05,527][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000225144_922189824.pth [2024-09-30 03:58:05,968][1157819] Updated weights for policy 0, policy_version 229908 (0.0006) [2024-09-30 03:58:06,474][1157819] Updated weights for policy 0, policy_version 229918 (0.0006) [2024-09-30 03:58:07,013][1157819] Updated weights for policy 0, policy_version 229928 (0.0006) [2024-09-30 03:58:07,544][1157819] Updated weights for policy 0, policy_version 229938 (0.0006) [2024-09-30 03:58:08,062][1157819] Updated weights for policy 0, policy_version 229948 (0.0006) [2024-09-30 03:58:08,559][1157819] Updated weights for policy 0, policy_version 229958 (0.0006) [2024-09-30 03:58:09,058][1157819] Updated weights for policy 0, policy_version 229968 (0.0006) [2024-09-30 03:58:09,549][1157819] Updated weights for policy 0, policy_version 229978 (0.0006) [2024-09-30 03:58:10,028][1157819] Updated weights for policy 0, policy_version 229988 (0.0006) [2024-09-30 03:58:10,466][1157520] Fps is (10 sec: 80283.1, 60 sec: 84241.2, 300 sec: 80864.8). Total num frames: 942063616. Throughput: 0: 20652.6. Samples: 225461492. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 03:58:10,466][1157520] Avg episode reward: [(0, '55.720')] [2024-09-30 03:58:10,520][1157819] Updated weights for policy 0, policy_version 229998 (0.0006) [2024-09-30 03:58:11,025][1157819] Updated weights for policy 0, policy_version 230008 (0.0006) [2024-09-30 03:58:11,515][1157819] Updated weights for policy 0, policy_version 230018 (0.0006) [2024-09-30 03:58:12,051][1157819] Updated weights for policy 0, policy_version 230028 (0.0006) [2024-09-30 03:58:12,544][1157819] Updated weights for policy 0, policy_version 230038 (0.0006) [2024-09-30 03:58:13,039][1157819] Updated weights for policy 0, policy_version 230048 (0.0006) [2024-09-30 03:58:13,561][1157819] Updated weights for policy 0, policy_version 230058 (0.0006) [2024-09-30 03:58:14,072][1157819] Updated weights for policy 0, policy_version 230068 (0.0006) [2024-09-30 03:58:14,573][1157819] Updated weights for policy 0, policy_version 230078 (0.0006) [2024-09-30 03:58:15,094][1157819] Updated weights for policy 0, policy_version 230088 (0.0006) [2024-09-30 03:58:15,466][1157520] Fps is (10 sec: 80281.1, 60 sec: 83353.5, 300 sec: 80837.0). Total num frames: 942465024. Throughput: 0: 20492.3. Samples: 225583772. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 03:58:15,466][1157520] Avg episode reward: [(0, '58.682')] [2024-09-30 03:58:15,675][1157819] Updated weights for policy 0, policy_version 230098 (0.0006) [2024-09-30 03:58:16,179][1157819] Updated weights for policy 0, policy_version 230108 (0.0006) [2024-09-30 03:58:16,670][1157819] Updated weights for policy 0, policy_version 230118 (0.0006) [2024-09-30 03:58:17,192][1157819] Updated weights for policy 0, policy_version 230128 (0.0006) [2024-09-30 03:58:17,707][1157819] Updated weights for policy 0, policy_version 230138 (0.0006) [2024-09-30 03:58:18,231][1157819] Updated weights for policy 0, policy_version 230148 (0.0006) [2024-09-30 03:58:18,750][1157819] Updated weights for policy 0, policy_version 230158 (0.0006) [2024-09-30 03:58:19,275][1157819] Updated weights for policy 0, policy_version 230168 (0.0006) [2024-09-30 03:58:19,797][1157819] Updated weights for policy 0, policy_version 230178 (0.0006) [2024-09-30 03:58:20,295][1157819] Updated weights for policy 0, policy_version 230188 (0.0006) [2024-09-30 03:58:20,466][1157520] Fps is (10 sec: 79871.3, 60 sec: 82397.9, 300 sec: 80795.3). Total num frames: 942862336. Throughput: 0: 20288.3. Samples: 225702484. Policy #0 lag: (min: 0.0, avg: 2.2, max: 7.0) [2024-09-30 03:58:20,466][1157520] Avg episode reward: [(0, '54.554')] [2024-09-30 03:58:20,822][1157819] Updated weights for policy 0, policy_version 230198 (0.0006) [2024-09-30 03:58:21,332][1157819] Updated weights for policy 0, policy_version 230208 (0.0006) [2024-09-30 03:58:21,835][1157819] Updated weights for policy 0, policy_version 230218 (0.0006) [2024-09-30 03:58:22,375][1157819] Updated weights for policy 0, policy_version 230228 (0.0006) [2024-09-30 03:58:22,880][1157819] Updated weights for policy 0, policy_version 230238 (0.0006) [2024-09-30 03:58:23,342][1157819] Updated weights for policy 0, policy_version 230248 (0.0006) [2024-09-30 03:58:23,814][1157819] Updated weights for policy 0, policy_version 230258 (0.0006) [2024-09-30 03:58:24,238][1157819] Updated weights for policy 0, policy_version 230268 (0.0006) [2024-09-30 03:58:24,726][1157819] Updated weights for policy 0, policy_version 230278 (0.0006) [2024-09-30 03:58:25,188][1157819] Updated weights for policy 0, policy_version 230288 (0.0006) [2024-09-30 03:58:25,466][1157520] Fps is (10 sec: 81921.6, 60 sec: 82125.2, 300 sec: 80864.8). Total num frames: 943284224. Throughput: 0: 20254.7. Samples: 225762496. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 03:58:25,466][1157520] Avg episode reward: [(0, '56.526')] [2024-09-30 03:58:25,608][1157819] Updated weights for policy 0, policy_version 230298 (0.0006) [2024-09-30 03:58:26,111][1157819] Updated weights for policy 0, policy_version 230308 (0.0006) [2024-09-30 03:58:26,560][1157819] Updated weights for policy 0, policy_version 230318 (0.0006) [2024-09-30 03:58:26,986][1157819] Updated weights for policy 0, policy_version 230328 (0.0006) [2024-09-30 03:58:27,492][1157819] Updated weights for policy 0, policy_version 230338 (0.0006) [2024-09-30 03:58:27,989][1157819] Updated weights for policy 0, policy_version 230348 (0.0006) [2024-09-30 03:58:28,439][1157819] Updated weights for policy 0, policy_version 230358 (0.0006) [2024-09-30 03:58:28,906][1157819] Updated weights for policy 0, policy_version 230368 (0.0007) [2024-09-30 03:58:29,009][1157736] Signal inference workers to stop experience collection... (16350 times) [2024-09-30 03:58:29,013][1157736] Signal inference workers to resume experience collection... (16350 times) [2024-09-30 03:58:29,013][1157819] InferenceWorker_p0-w0: stopping experience collection (16350 times) [2024-09-30 03:58:29,017][1157819] InferenceWorker_p0-w0: resuming experience collection (16350 times) [2024-09-30 03:58:29,351][1157819] Updated weights for policy 0, policy_version 230378 (0.0006) [2024-09-30 03:58:29,842][1157819] Updated weights for policy 0, policy_version 230388 (0.0006) [2024-09-30 03:58:30,311][1157819] Updated weights for policy 0, policy_version 230398 (0.0006) [2024-09-30 03:58:30,466][1157520] Fps is (10 sec: 86016.3, 60 sec: 82193.1, 300 sec: 81073.1). Total num frames: 943722496. Throughput: 0: 20422.3. Samples: 225894328. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 03:58:30,466][1157520] Avg episode reward: [(0, '56.631')] [2024-09-30 03:58:30,771][1157819] Updated weights for policy 0, policy_version 230408 (0.0006) [2024-09-30 03:58:31,243][1157819] Updated weights for policy 0, policy_version 230418 (0.0006) [2024-09-30 03:58:31,713][1157819] Updated weights for policy 0, policy_version 230428 (0.0006) [2024-09-30 03:58:32,201][1157819] Updated weights for policy 0, policy_version 230438 (0.0006) [2024-09-30 03:58:32,707][1157819] Updated weights for policy 0, policy_version 230448 (0.0006) [2024-09-30 03:58:33,185][1157819] Updated weights for policy 0, policy_version 230458 (0.0006) [2024-09-30 03:58:33,628][1157819] Updated weights for policy 0, policy_version 230468 (0.0006) [2024-09-30 03:58:34,128][1157819] Updated weights for policy 0, policy_version 230478 (0.0006) [2024-09-30 03:58:34,576][1157819] Updated weights for policy 0, policy_version 230488 (0.0006) [2024-09-30 03:58:34,998][1157819] Updated weights for policy 0, policy_version 230498 (0.0006) [2024-09-30 03:58:35,441][1157819] Updated weights for policy 0, policy_version 230508 (0.0006) [2024-09-30 03:58:35,466][1157520] Fps is (10 sec: 87654.9, 60 sec: 82466.3, 300 sec: 81239.7). Total num frames: 944160768. Throughput: 0: 20535.9. Samples: 226025052. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 03:58:35,466][1157520] Avg episode reward: [(0, '57.262')] [2024-09-30 03:58:35,919][1157819] Updated weights for policy 0, policy_version 230518 (0.0006) [2024-09-30 03:58:36,368][1157819] Updated weights for policy 0, policy_version 230528 (0.0006) [2024-09-30 03:58:36,806][1157819] Updated weights for policy 0, policy_version 230538 (0.0006) [2024-09-30 03:58:37,249][1157819] Updated weights for policy 0, policy_version 230548 (0.0006) [2024-09-30 03:58:37,725][1157819] Updated weights for policy 0, policy_version 230558 (0.0006) [2024-09-30 03:58:38,170][1157819] Updated weights for policy 0, policy_version 230568 (0.0006) [2024-09-30 03:58:38,623][1157819] Updated weights for policy 0, policy_version 230578 (0.0006) [2024-09-30 03:58:39,048][1157819] Updated weights for policy 0, policy_version 230588 (0.0006) [2024-09-30 03:58:39,520][1157819] Updated weights for policy 0, policy_version 230598 (0.0006) [2024-09-30 03:58:39,972][1157819] Updated weights for policy 0, policy_version 230608 (0.0006) [2024-09-30 03:58:40,406][1157819] Updated weights for policy 0, policy_version 230618 (0.0006) [2024-09-30 03:58:40,466][1157520] Fps is (10 sec: 89293.3, 60 sec: 83217.3, 300 sec: 81447.9). Total num frames: 944615424. Throughput: 0: 20697.7. Samples: 226093004. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 03:58:40,466][1157520] Avg episode reward: [(0, '55.245')] [2024-09-30 03:58:40,883][1157819] Updated weights for policy 0, policy_version 230628 (0.0006) [2024-09-30 03:58:41,338][1157819] Updated weights for policy 0, policy_version 230638 (0.0006) [2024-09-30 03:58:41,828][1157819] Updated weights for policy 0, policy_version 230648 (0.0006) [2024-09-30 03:58:42,274][1157819] Updated weights for policy 0, policy_version 230658 (0.0006) [2024-09-30 03:58:42,755][1157819] Updated weights for policy 0, policy_version 230668 (0.0006) [2024-09-30 03:58:43,236][1157819] Updated weights for policy 0, policy_version 230678 (0.0006) [2024-09-30 03:58:43,703][1157819] Updated weights for policy 0, policy_version 230688 (0.0006) [2024-09-30 03:58:44,186][1157819] Updated weights for policy 0, policy_version 230698 (0.0006) [2024-09-30 03:58:44,648][1157819] Updated weights for policy 0, policy_version 230708 (0.0006) [2024-09-30 03:58:45,152][1157819] Updated weights for policy 0, policy_version 230718 (0.0006) [2024-09-30 03:58:45,466][1157520] Fps is (10 sec: 88882.8, 60 sec: 83558.5, 300 sec: 81586.8). Total num frames: 945049600. Throughput: 0: 20986.5. Samples: 226226208. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 03:58:45,466][1157520] Avg episode reward: [(0, '55.452')] [2024-09-30 03:58:45,616][1157819] Updated weights for policy 0, policy_version 230728 (0.0006) [2024-09-30 03:58:46,057][1157819] Updated weights for policy 0, policy_version 230738 (0.0006) [2024-09-30 03:58:46,554][1157819] Updated weights for policy 0, policy_version 230748 (0.0006) [2024-09-30 03:58:47,026][1157819] Updated weights for policy 0, policy_version 230758 (0.0006) [2024-09-30 03:58:47,513][1157819] Updated weights for policy 0, policy_version 230768 (0.0006) [2024-09-30 03:58:47,996][1157819] Updated weights for policy 0, policy_version 230778 (0.0006) [2024-09-30 03:58:48,459][1157819] Updated weights for policy 0, policy_version 230788 (0.0006) [2024-09-30 03:58:48,929][1157819] Updated weights for policy 0, policy_version 230798 (0.0006) [2024-09-30 03:58:49,374][1157819] Updated weights for policy 0, policy_version 230808 (0.0006) [2024-09-30 03:58:49,868][1157819] Updated weights for policy 0, policy_version 230818 (0.0006) [2024-09-30 03:58:50,345][1157819] Updated weights for policy 0, policy_version 230828 (0.0006) [2024-09-30 03:58:50,466][1157520] Fps is (10 sec: 86425.7, 60 sec: 83695.1, 300 sec: 81670.1). Total num frames: 945479680. Throughput: 0: 21189.8. Samples: 226355796. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 03:58:50,466][1157520] Avg episode reward: [(0, '55.974')] [2024-09-30 03:58:50,822][1157819] Updated weights for policy 0, policy_version 230838 (0.0006) [2024-09-30 03:58:51,282][1157819] Updated weights for policy 0, policy_version 230848 (0.0006) [2024-09-30 03:58:51,743][1157819] Updated weights for policy 0, policy_version 230858 (0.0006) [2024-09-30 03:58:52,242][1157819] Updated weights for policy 0, policy_version 230868 (0.0006) [2024-09-30 03:58:52,699][1157819] Updated weights for policy 0, policy_version 230878 (0.0006) [2024-09-30 03:58:53,188][1157819] Updated weights for policy 0, policy_version 230888 (0.0006) [2024-09-30 03:58:53,655][1157819] Updated weights for policy 0, policy_version 230898 (0.0006) [2024-09-30 03:58:54,118][1157819] Updated weights for policy 0, policy_version 230908 (0.0006) [2024-09-30 03:58:54,603][1157819] Updated weights for policy 0, policy_version 230918 (0.0006) [2024-09-30 03:58:55,091][1157819] Updated weights for policy 0, policy_version 230928 (0.0006) [2024-09-30 03:58:55,466][1157520] Fps is (10 sec: 86425.6, 60 sec: 84241.2, 300 sec: 81781.2). Total num frames: 945913856. Throughput: 0: 21318.1. Samples: 226420808. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 03:58:55,466][1157520] Avg episode reward: [(0, '56.906')] [2024-09-30 03:58:55,544][1157819] Updated weights for policy 0, policy_version 230938 (0.0006) [2024-09-30 03:58:55,980][1157819] Updated weights for policy 0, policy_version 230948 (0.0006) [2024-09-30 03:58:56,457][1157819] Updated weights for policy 0, policy_version 230958 (0.0006) [2024-09-30 03:58:56,897][1157819] Updated weights for policy 0, policy_version 230968 (0.0006) [2024-09-30 03:58:57,343][1157819] Updated weights for policy 0, policy_version 230978 (0.0006) [2024-09-30 03:58:57,796][1157819] Updated weights for policy 0, policy_version 230988 (0.0006) [2024-09-30 03:58:58,266][1157819] Updated weights for policy 0, policy_version 230998 (0.0006) [2024-09-30 03:58:58,700][1157819] Updated weights for policy 0, policy_version 231008 (0.0006) [2024-09-30 03:58:59,152][1157819] Updated weights for policy 0, policy_version 231018 (0.0006) [2024-09-30 03:58:59,617][1157819] Updated weights for policy 0, policy_version 231028 (0.0006) [2024-09-30 03:59:00,054][1157819] Updated weights for policy 0, policy_version 231038 (0.0006) [2024-09-30 03:59:00,466][1157520] Fps is (10 sec: 88473.5, 60 sec: 85060.5, 300 sec: 81947.8). Total num frames: 946364416. Throughput: 0: 21563.6. Samples: 226554128. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 03:59:00,466][1157520] Avg episode reward: [(0, '57.629')] [2024-09-30 03:59:00,524][1157819] Updated weights for policy 0, policy_version 231048 (0.0006) [2024-09-30 03:59:00,955][1157819] Updated weights for policy 0, policy_version 231058 (0.0006) [2024-09-30 03:59:01,433][1157819] Updated weights for policy 0, policy_version 231068 (0.0006) [2024-09-30 03:59:01,884][1157819] Updated weights for policy 0, policy_version 231078 (0.0006) [2024-09-30 03:59:02,326][1157819] Updated weights for policy 0, policy_version 231088 (0.0006) [2024-09-30 03:59:02,786][1157819] Updated weights for policy 0, policy_version 231098 (0.0006) [2024-09-30 03:59:03,239][1157819] Updated weights for policy 0, policy_version 231108 (0.0006) [2024-09-30 03:59:03,691][1157819] Updated weights for policy 0, policy_version 231118 (0.0006) [2024-09-30 03:59:04,202][1157819] Updated weights for policy 0, policy_version 231128 (0.0006) [2024-09-30 03:59:04,691][1157819] Updated weights for policy 0, policy_version 231138 (0.0006) [2024-09-30 03:59:05,195][1157819] Updated weights for policy 0, policy_version 231148 (0.0006) [2024-09-30 03:59:05,466][1157520] Fps is (10 sec: 88883.3, 60 sec: 85674.9, 300 sec: 82017.2). Total num frames: 946802688. Throughput: 0: 21889.7. Samples: 226687520. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 03:59:05,466][1157520] Avg episode reward: [(0, '56.895')] [2024-09-30 03:59:05,548][1157736] Signal inference workers to stop experience collection... (16400 times) [2024-09-30 03:59:05,549][1157736] Signal inference workers to resume experience collection... (16400 times) [2024-09-30 03:59:05,554][1157819] InferenceWorker_p0-w0: stopping experience collection (16400 times) [2024-09-30 03:59:05,554][1157819] InferenceWorker_p0-w0: resuming experience collection (16400 times) [2024-09-30 03:59:05,707][1157819] Updated weights for policy 0, policy_version 231158 (0.0006) [2024-09-30 03:59:06,202][1157819] Updated weights for policy 0, policy_version 231168 (0.0006) [2024-09-30 03:59:06,701][1157819] Updated weights for policy 0, policy_version 231178 (0.0006) [2024-09-30 03:59:07,203][1157819] Updated weights for policy 0, policy_version 231188 (0.0006) [2024-09-30 03:59:07,692][1157819] Updated weights for policy 0, policy_version 231198 (0.0006) [2024-09-30 03:59:08,210][1157819] Updated weights for policy 0, policy_version 231208 (0.0006) [2024-09-30 03:59:08,704][1157819] Updated weights for policy 0, policy_version 231218 (0.0006) [2024-09-30 03:59:09,176][1157819] Updated weights for policy 0, policy_version 231228 (0.0006) [2024-09-30 03:59:09,697][1157819] Updated weights for policy 0, policy_version 231238 (0.0006) [2024-09-30 03:59:10,199][1157819] Updated weights for policy 0, policy_version 231248 (0.0006) [2024-09-30 03:59:10,466][1157520] Fps is (10 sec: 84377.7, 60 sec: 85742.9, 300 sec: 81989.4). Total num frames: 947208192. Throughput: 0: 21922.1. Samples: 226748988. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 03:59:10,466][1157520] Avg episode reward: [(0, '54.602')] [2024-09-30 03:59:10,701][1157819] Updated weights for policy 0, policy_version 231258 (0.0006) [2024-09-30 03:59:11,211][1157819] Updated weights for policy 0, policy_version 231268 (0.0007) [2024-09-30 03:59:11,685][1157819] Updated weights for policy 0, policy_version 231278 (0.0006) [2024-09-30 03:59:12,195][1157819] Updated weights for policy 0, policy_version 231288 (0.0006) [2024-09-30 03:59:12,708][1157819] Updated weights for policy 0, policy_version 231298 (0.0006) [2024-09-30 03:59:13,177][1157819] Updated weights for policy 0, policy_version 231308 (0.0006) [2024-09-30 03:59:13,687][1157819] Updated weights for policy 0, policy_version 231318 (0.0006) [2024-09-30 03:59:14,175][1157819] Updated weights for policy 0, policy_version 231328 (0.0006) [2024-09-30 03:59:14,670][1157819] Updated weights for policy 0, policy_version 231338 (0.0006) [2024-09-30 03:59:15,179][1157819] Updated weights for policy 0, policy_version 231348 (0.0006) [2024-09-30 03:59:15,466][1157520] Fps is (10 sec: 81510.4, 60 sec: 85879.8, 300 sec: 81975.5). Total num frames: 947617792. Throughput: 0: 21721.4. Samples: 226871792. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 03:59:15,466][1157520] Avg episode reward: [(0, '55.922')] [2024-09-30 03:59:15,715][1157819] Updated weights for policy 0, policy_version 231358 (0.0006) [2024-09-30 03:59:16,220][1157819] Updated weights for policy 0, policy_version 231368 (0.0006) [2024-09-30 03:59:16,743][1157819] Updated weights for policy 0, policy_version 231378 (0.0006) [2024-09-30 03:59:17,284][1157819] Updated weights for policy 0, policy_version 231388 (0.0006) [2024-09-30 03:59:17,836][1157819] Updated weights for policy 0, policy_version 231398 (0.0006) [2024-09-30 03:59:18,368][1157819] Updated weights for policy 0, policy_version 231408 (0.0006) [2024-09-30 03:59:18,881][1157819] Updated weights for policy 0, policy_version 231418 (0.0006) [2024-09-30 03:59:19,452][1157819] Updated weights for policy 0, policy_version 231428 (0.0006) [2024-09-30 03:59:20,013][1157819] Updated weights for policy 0, policy_version 231438 (0.0006) [2024-09-30 03:59:20,466][1157520] Fps is (10 sec: 79462.3, 60 sec: 85674.8, 300 sec: 81850.6). Total num frames: 948002816. Throughput: 0: 21418.3. Samples: 226988876. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 03:59:20,466][1157520] Avg episode reward: [(0, '56.105')] [2024-09-30 03:59:20,534][1157819] Updated weights for policy 0, policy_version 231448 (0.0006) [2024-09-30 03:59:21,066][1157819] Updated weights for policy 0, policy_version 231458 (0.0006) [2024-09-30 03:59:21,622][1157819] Updated weights for policy 0, policy_version 231468 (0.0006) [2024-09-30 03:59:22,155][1157819] Updated weights for policy 0, policy_version 231478 (0.0006) [2024-09-30 03:59:22,674][1157819] Updated weights for policy 0, policy_version 231488 (0.0006) [2024-09-30 03:59:23,234][1157819] Updated weights for policy 0, policy_version 231498 (0.0006) [2024-09-30 03:59:23,739][1157819] Updated weights for policy 0, policy_version 231508 (0.0006) [2024-09-30 03:59:24,241][1157819] Updated weights for policy 0, policy_version 231518 (0.0006) [2024-09-30 03:59:24,762][1157819] Updated weights for policy 0, policy_version 231528 (0.0006) [2024-09-30 03:59:25,266][1157819] Updated weights for policy 0, policy_version 231538 (0.0006) [2024-09-30 03:59:25,466][1157520] Fps is (10 sec: 77414.5, 60 sec: 85128.6, 300 sec: 81767.3). Total num frames: 948391936. Throughput: 0: 21171.5. Samples: 227045724. Policy #0 lag: (min: 0.0, avg: 2.9, max: 6.0) [2024-09-30 03:59:25,466][1157520] Avg episode reward: [(0, '54.796')] [2024-09-30 03:59:25,784][1157819] Updated weights for policy 0, policy_version 231548 (0.0006) [2024-09-30 03:59:26,319][1157819] Updated weights for policy 0, policy_version 231558 (0.0006) [2024-09-30 03:59:26,848][1157819] Updated weights for policy 0, policy_version 231568 (0.0006) [2024-09-30 03:59:27,391][1157819] Updated weights for policy 0, policy_version 231578 (0.0006) [2024-09-30 03:59:27,537][1157736] Signal inference workers to stop experience collection... (16450 times) [2024-09-30 03:59:27,538][1157736] Signal inference workers to resume experience collection... (16450 times) [2024-09-30 03:59:27,541][1157819] InferenceWorker_p0-w0: stopping experience collection (16450 times) [2024-09-30 03:59:27,541][1157819] InferenceWorker_p0-w0: resuming experience collection (16450 times) [2024-09-30 03:59:27,936][1157819] Updated weights for policy 0, policy_version 231588 (0.0007) [2024-09-30 03:59:28,483][1157819] Updated weights for policy 0, policy_version 231598 (0.0006) [2024-09-30 03:59:29,010][1157819] Updated weights for policy 0, policy_version 231608 (0.0006) [2024-09-30 03:59:29,532][1157819] Updated weights for policy 0, policy_version 231618 (0.0006) [2024-09-30 03:59:30,117][1157819] Updated weights for policy 0, policy_version 231628 (0.0007) [2024-09-30 03:59:30,466][1157520] Fps is (10 sec: 77004.3, 60 sec: 84172.8, 300 sec: 81670.1). Total num frames: 948772864. Throughput: 0: 20803.9. Samples: 227162384. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:59:30,466][1157520] Avg episode reward: [(0, '56.092')] [2024-09-30 03:59:30,678][1157819] Updated weights for policy 0, policy_version 231638 (0.0006) [2024-09-30 03:59:31,242][1157819] Updated weights for policy 0, policy_version 231648 (0.0006) [2024-09-30 03:59:31,832][1157819] Updated weights for policy 0, policy_version 231658 (0.0006) [2024-09-30 03:59:32,402][1157819] Updated weights for policy 0, policy_version 231668 (0.0006) [2024-09-30 03:59:32,977][1157819] Updated weights for policy 0, policy_version 231678 (0.0006) [2024-09-30 03:59:33,552][1157819] Updated weights for policy 0, policy_version 231688 (0.0006) [2024-09-30 03:59:34,135][1157819] Updated weights for policy 0, policy_version 231698 (0.0006) [2024-09-30 03:59:34,712][1157819] Updated weights for policy 0, policy_version 231708 (0.0006) [2024-09-30 03:59:35,294][1157819] Updated weights for policy 0, policy_version 231718 (0.0006) [2024-09-30 03:59:35,466][1157520] Fps is (10 sec: 73318.6, 60 sec: 82739.2, 300 sec: 81475.7). Total num frames: 949125120. Throughput: 0: 20325.2. Samples: 227270428. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:59:35,466][1157520] Avg episode reward: [(0, '56.535')] [2024-09-30 03:59:35,877][1157819] Updated weights for policy 0, policy_version 231728 (0.0006) [2024-09-30 03:59:36,461][1157819] Updated weights for policy 0, policy_version 231738 (0.0006) [2024-09-30 03:59:37,062][1157819] Updated weights for policy 0, policy_version 231748 (0.0006) [2024-09-30 03:59:37,703][1157819] Updated weights for policy 0, policy_version 231758 (0.0006) [2024-09-30 03:59:38,244][1157819] Updated weights for policy 0, policy_version 231768 (0.0006) [2024-09-30 03:59:38,828][1157819] Updated weights for policy 0, policy_version 231778 (0.0006) [2024-09-30 03:59:39,413][1157819] Updated weights for policy 0, policy_version 231788 (0.0006) [2024-09-30 03:59:39,993][1157819] Updated weights for policy 0, policy_version 231798 (0.0006) [2024-09-30 03:59:40,466][1157520] Fps is (10 sec: 70450.8, 60 sec: 81032.4, 300 sec: 81281.3). Total num frames: 949477376. Throughput: 0: 20034.2. Samples: 227322348. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:59:40,466][1157520] Avg episode reward: [(0, '57.300')] [2024-09-30 03:59:40,573][1157819] Updated weights for policy 0, policy_version 231808 (0.0006) [2024-09-30 03:59:41,144][1157819] Updated weights for policy 0, policy_version 231818 (0.0006) [2024-09-30 03:59:41,738][1157819] Updated weights for policy 0, policy_version 231828 (0.0006) [2024-09-30 03:59:42,328][1157819] Updated weights for policy 0, policy_version 231838 (0.0006) [2024-09-30 03:59:42,914][1157819] Updated weights for policy 0, policy_version 231848 (0.0006) [2024-09-30 03:59:43,490][1157819] Updated weights for policy 0, policy_version 231858 (0.0006) [2024-09-30 03:59:44,072][1157819] Updated weights for policy 0, policy_version 231868 (0.0006) [2024-09-30 03:59:44,635][1157819] Updated weights for policy 0, policy_version 231878 (0.0006) [2024-09-30 03:59:45,201][1157819] Updated weights for policy 0, policy_version 231888 (0.0006) [2024-09-30 03:59:45,466][1157520] Fps is (10 sec: 70450.3, 60 sec: 79667.1, 300 sec: 81086.9). Total num frames: 949829632. Throughput: 0: 19421.0. Samples: 227428076. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:59:45,466][1157520] Avg episode reward: [(0, '55.332')] [2024-09-30 03:59:45,751][1157819] Updated weights for policy 0, policy_version 231898 (0.0006) [2024-09-30 03:59:46,316][1157819] Updated weights for policy 0, policy_version 231908 (0.0006) [2024-09-30 03:59:46,881][1157819] Updated weights for policy 0, policy_version 231918 (0.0006) [2024-09-30 03:59:47,430][1157819] Updated weights for policy 0, policy_version 231928 (0.0006) [2024-09-30 03:59:47,958][1157819] Updated weights for policy 0, policy_version 231938 (0.0006) [2024-09-30 03:59:48,503][1157819] Updated weights for policy 0, policy_version 231948 (0.0006) [2024-09-30 03:59:49,051][1157819] Updated weights for policy 0, policy_version 231958 (0.0006) [2024-09-30 03:59:49,599][1157819] Updated weights for policy 0, policy_version 231968 (0.0006) [2024-09-30 03:59:50,169][1157819] Updated weights for policy 0, policy_version 231978 (0.0006) [2024-09-30 03:59:50,466][1157520] Fps is (10 sec: 72498.9, 60 sec: 78711.2, 300 sec: 81142.4). Total num frames: 950202368. Throughput: 0: 18914.0. Samples: 227538652. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:59:50,466][1157520] Avg episode reward: [(0, '56.779')] [2024-09-30 03:59:50,727][1157819] Updated weights for policy 0, policy_version 231988 (0.0006) [2024-09-30 03:59:51,252][1157819] Updated weights for policy 0, policy_version 231998 (0.0006) [2024-09-30 03:59:51,796][1157819] Updated weights for policy 0, policy_version 232008 (0.0006) [2024-09-30 03:59:52,361][1157819] Updated weights for policy 0, policy_version 232018 (0.0006) [2024-09-30 03:59:52,885][1157819] Updated weights for policy 0, policy_version 232028 (0.0006) [2024-09-30 03:59:53,447][1157819] Updated weights for policy 0, policy_version 232038 (0.0006) [2024-09-30 03:59:53,726][1157736] Signal inference workers to stop experience collection... (16500 times) [2024-09-30 03:59:53,729][1157819] InferenceWorker_p0-w0: stopping experience collection (16500 times) [2024-09-30 03:59:53,734][1157736] Signal inference workers to resume experience collection... (16500 times) [2024-09-30 03:59:53,734][1157819] InferenceWorker_p0-w0: resuming experience collection (16500 times) [2024-09-30 03:59:54,010][1157819] Updated weights for policy 0, policy_version 232048 (0.0006) [2024-09-30 03:59:54,558][1157819] Updated weights for policy 0, policy_version 232058 (0.0006) [2024-09-30 03:59:55,110][1157819] Updated weights for policy 0, policy_version 232068 (0.0006) [2024-09-30 03:59:55,466][1157520] Fps is (10 sec: 74547.0, 60 sec: 77687.3, 300 sec: 81211.9). Total num frames: 950575104. Throughput: 0: 18795.1. Samples: 227594772. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 03:59:55,466][1157520] Avg episode reward: [(0, '56.057')] [2024-09-30 03:59:55,635][1157819] Updated weights for policy 0, policy_version 232078 (0.0006) [2024-09-30 03:59:56,207][1157819] Updated weights for policy 0, policy_version 232088 (0.0006) [2024-09-30 03:59:56,723][1157819] Updated weights for policy 0, policy_version 232098 (0.0006) [2024-09-30 03:59:57,234][1157819] Updated weights for policy 0, policy_version 232108 (0.0006) [2024-09-30 03:59:57,754][1157819] Updated weights for policy 0, policy_version 232118 (0.0006) [2024-09-30 03:59:58,289][1157819] Updated weights for policy 0, policy_version 232128 (0.0006) [2024-09-30 03:59:58,826][1157819] Updated weights for policy 0, policy_version 232138 (0.0006) [2024-09-30 03:59:59,326][1157819] Updated weights for policy 0, policy_version 232148 (0.0006) [2024-09-30 03:59:59,869][1157819] Updated weights for policy 0, policy_version 232158 (0.0006) [2024-09-30 04:00:00,405][1157819] Updated weights for policy 0, policy_version 232168 (0.0006) [2024-09-30 04:00:00,466][1157520] Fps is (10 sec: 76185.9, 60 sec: 76663.3, 300 sec: 81295.2). Total num frames: 950964224. Throughput: 0: 18597.1. Samples: 227708664. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 04:00:00,466][1157520] Avg episode reward: [(0, '54.888')] [2024-09-30 04:00:00,914][1157819] Updated weights for policy 0, policy_version 232178 (0.0006) [2024-09-30 04:00:01,430][1157819] Updated weights for policy 0, policy_version 232188 (0.0006) [2024-09-30 04:00:01,954][1157819] Updated weights for policy 0, policy_version 232198 (0.0006) [2024-09-30 04:00:02,477][1157819] Updated weights for policy 0, policy_version 232208 (0.0006) [2024-09-30 04:00:03,019][1157819] Updated weights for policy 0, policy_version 232218 (0.0006) [2024-09-30 04:00:03,551][1157819] Updated weights for policy 0, policy_version 232228 (0.0006) [2024-09-30 04:00:04,104][1157819] Updated weights for policy 0, policy_version 232238 (0.0006) [2024-09-30 04:00:04,627][1157819] Updated weights for policy 0, policy_version 232248 (0.0006) [2024-09-30 04:00:05,134][1157819] Updated weights for policy 0, policy_version 232258 (0.0006) [2024-09-30 04:00:05,466][1157520] Fps is (10 sec: 77824.2, 60 sec: 75844.2, 300 sec: 81392.4). Total num frames: 951353344. Throughput: 0: 18586.2. Samples: 227825256. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 04:00:05,466][1157520] Avg episode reward: [(0, '57.296')] [2024-09-30 04:00:05,472][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000232264_951353344.pth... [2024-09-30 04:00:05,534][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000227442_931602432.pth [2024-09-30 04:00:05,692][1157819] Updated weights for policy 0, policy_version 232268 (0.0006) [2024-09-30 04:00:06,258][1157819] Updated weights for policy 0, policy_version 232278 (0.0006) [2024-09-30 04:00:06,828][1157819] Updated weights for policy 0, policy_version 232288 (0.0006) [2024-09-30 04:00:07,422][1157819] Updated weights for policy 0, policy_version 232298 (0.0006) [2024-09-30 04:00:07,968][1157819] Updated weights for policy 0, policy_version 232308 (0.0006) [2024-09-30 04:00:08,542][1157819] Updated weights for policy 0, policy_version 232318 (0.0006) [2024-09-30 04:00:09,048][1157819] Updated weights for policy 0, policy_version 232328 (0.0006) [2024-09-30 04:00:09,556][1157819] Updated weights for policy 0, policy_version 232338 (0.0006) [2024-09-30 04:00:10,077][1157819] Updated weights for policy 0, policy_version 232348 (0.0006) [2024-09-30 04:00:10,466][1157520] Fps is (10 sec: 75776.2, 60 sec: 75229.7, 300 sec: 81406.3). Total num frames: 951721984. Throughput: 0: 18530.8. Samples: 227879612. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 04:00:10,466][1157520] Avg episode reward: [(0, '55.433')] [2024-09-30 04:00:10,617][1157819] Updated weights for policy 0, policy_version 232358 (0.0006) [2024-09-30 04:00:11,142][1157819] Updated weights for policy 0, policy_version 232368 (0.0006) [2024-09-30 04:00:11,674][1157819] Updated weights for policy 0, policy_version 232378 (0.0006) [2024-09-30 04:00:12,232][1157819] Updated weights for policy 0, policy_version 232388 (0.0006) [2024-09-30 04:00:12,781][1157819] Updated weights for policy 0, policy_version 232398 (0.0006) [2024-09-30 04:00:13,297][1157819] Updated weights for policy 0, policy_version 232408 (0.0006) [2024-09-30 04:00:13,816][1157819] Updated weights for policy 0, policy_version 232418 (0.0006) [2024-09-30 04:00:14,363][1157819] Updated weights for policy 0, policy_version 232428 (0.0006) [2024-09-30 04:00:14,897][1157819] Updated weights for policy 0, policy_version 232438 (0.0006) [2024-09-30 04:00:15,393][1157819] Updated weights for policy 0, policy_version 232448 (0.0006) [2024-09-30 04:00:15,466][1157520] Fps is (10 sec: 75366.2, 60 sec: 74820.1, 300 sec: 81392.4). Total num frames: 952107008. Throughput: 0: 18517.1. Samples: 227995656. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 04:00:15,466][1157520] Avg episode reward: [(0, '55.675')] [2024-09-30 04:00:15,895][1157819] Updated weights for policy 0, policy_version 232458 (0.0006) [2024-09-30 04:00:16,413][1157819] Updated weights for policy 0, policy_version 232468 (0.0006) [2024-09-30 04:00:16,651][1157736] Signal inference workers to stop experience collection... (16550 times) [2024-09-30 04:00:16,653][1157819] InferenceWorker_p0-w0: stopping experience collection (16550 times) [2024-09-30 04:00:16,661][1157736] Signal inference workers to resume experience collection... (16550 times) [2024-09-30 04:00:16,661][1157819] InferenceWorker_p0-w0: resuming experience collection (16550 times) [2024-09-30 04:00:16,981][1157819] Updated weights for policy 0, policy_version 232478 (0.0007) [2024-09-30 04:00:17,512][1157819] Updated weights for policy 0, policy_version 232488 (0.0006) [2024-09-30 04:00:18,042][1157819] Updated weights for policy 0, policy_version 232498 (0.0006) [2024-09-30 04:00:18,609][1157819] Updated weights for policy 0, policy_version 232508 (0.0006) [2024-09-30 04:00:19,103][1157819] Updated weights for policy 0, policy_version 232518 (0.0006) [2024-09-30 04:00:19,632][1157819] Updated weights for policy 0, policy_version 232528 (0.0006) [2024-09-30 04:00:20,174][1157819] Updated weights for policy 0, policy_version 232538 (0.0006) [2024-09-30 04:00:20,466][1157520] Fps is (10 sec: 77414.6, 60 sec: 74888.4, 300 sec: 81461.8). Total num frames: 952496128. Throughput: 0: 18690.9. Samples: 228111520. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 04:00:20,466][1157520] Avg episode reward: [(0, '56.825')] [2024-09-30 04:00:20,743][1157819] Updated weights for policy 0, policy_version 232548 (0.0007) [2024-09-30 04:00:21,280][1157819] Updated weights for policy 0, policy_version 232558 (0.0007) [2024-09-30 04:00:21,796][1157819] Updated weights for policy 0, policy_version 232568 (0.0006) [2024-09-30 04:00:22,377][1157819] Updated weights for policy 0, policy_version 232578 (0.0006) [2024-09-30 04:00:22,928][1157819] Updated weights for policy 0, policy_version 232588 (0.0006) [2024-09-30 04:00:23,486][1157819] Updated weights for policy 0, policy_version 232598 (0.0006) [2024-09-30 04:00:24,029][1157819] Updated weights for policy 0, policy_version 232608 (0.0006) [2024-09-30 04:00:24,598][1157819] Updated weights for policy 0, policy_version 232618 (0.0006) [2024-09-30 04:00:25,147][1157819] Updated weights for policy 0, policy_version 232628 (0.0006) [2024-09-30 04:00:25,466][1157520] Fps is (10 sec: 75776.2, 60 sec: 74547.1, 300 sec: 81434.1). Total num frames: 952864768. Throughput: 0: 18779.8. Samples: 228167440. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 04:00:25,466][1157520] Avg episode reward: [(0, '56.258')] [2024-09-30 04:00:25,694][1157819] Updated weights for policy 0, policy_version 232638 (0.0006) [2024-09-30 04:00:26,233][1157819] Updated weights for policy 0, policy_version 232648 (0.0006) [2024-09-30 04:00:26,783][1157819] Updated weights for policy 0, policy_version 232658 (0.0006) [2024-09-30 04:00:27,345][1157819] Updated weights for policy 0, policy_version 232668 (0.0006) [2024-09-30 04:00:27,873][1157819] Updated weights for policy 0, policy_version 232678 (0.0006) [2024-09-30 04:00:28,481][1157819] Updated weights for policy 0, policy_version 232688 (0.0006) [2024-09-30 04:00:29,046][1157819] Updated weights for policy 0, policy_version 232698 (0.0006) [2024-09-30 04:00:29,624][1157819] Updated weights for policy 0, policy_version 232708 (0.0006) [2024-09-30 04:00:30,220][1157819] Updated weights for policy 0, policy_version 232718 (0.0006) [2024-09-30 04:00:30,466][1157520] Fps is (10 sec: 73318.0, 60 sec: 74274.0, 300 sec: 81309.1). Total num frames: 953229312. Throughput: 0: 18897.1. Samples: 228278448. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 04:00:30,466][1157520] Avg episode reward: [(0, '57.736')] [2024-09-30 04:00:30,786][1157819] Updated weights for policy 0, policy_version 232728 (0.0006) [2024-09-30 04:00:31,374][1157819] Updated weights for policy 0, policy_version 232738 (0.0006) [2024-09-30 04:00:31,960][1157819] Updated weights for policy 0, policy_version 232748 (0.0006) [2024-09-30 04:00:32,566][1157819] Updated weights for policy 0, policy_version 232758 (0.0006) [2024-09-30 04:00:33,142][1157819] Updated weights for policy 0, policy_version 232768 (0.0006) [2024-09-30 04:00:33,696][1157819] Updated weights for policy 0, policy_version 232778 (0.0006) [2024-09-30 04:00:34,304][1157819] Updated weights for policy 0, policy_version 232788 (0.0006) [2024-09-30 04:00:34,884][1157819] Updated weights for policy 0, policy_version 232798 (0.0006) [2024-09-30 04:00:35,451][1157819] Updated weights for policy 0, policy_version 232808 (0.0006) [2024-09-30 04:00:35,466][1157520] Fps is (10 sec: 71679.5, 60 sec: 74273.9, 300 sec: 81142.4). Total num frames: 953581568. Throughput: 0: 18773.9. Samples: 228383476. Policy #0 lag: (min: 0.0, avg: 2.2, max: 5.0) [2024-09-30 04:00:35,466][1157520] Avg episode reward: [(0, '53.876')] [2024-09-30 04:00:36,035][1157819] Updated weights for policy 0, policy_version 232818 (0.0006) [2024-09-30 04:00:36,614][1157819] Updated weights for policy 0, policy_version 232828 (0.0006) [2024-09-30 04:00:37,177][1157819] Updated weights for policy 0, policy_version 232838 (0.0006) [2024-09-30 04:00:37,744][1157819] Updated weights for policy 0, policy_version 232848 (0.0006) [2024-09-30 04:00:38,359][1157819] Updated weights for policy 0, policy_version 232858 (0.0006) [2024-09-30 04:00:38,900][1157819] Updated weights for policy 0, policy_version 232868 (0.0006) [2024-09-30 04:00:39,512][1157819] Updated weights for policy 0, policy_version 232878 (0.0006) [2024-09-30 04:00:40,109][1157819] Updated weights for policy 0, policy_version 232888 (0.0006) [2024-09-30 04:00:40,466][1157520] Fps is (10 sec: 70451.4, 60 sec: 74274.2, 300 sec: 80934.2). Total num frames: 953933824. Throughput: 0: 18714.0. Samples: 228436900. Policy #0 lag: (min: 0.0, avg: 1.6, max: 6.0) [2024-09-30 04:00:40,466][1157520] Avg episode reward: [(0, '56.412')] [2024-09-30 04:00:40,679][1157819] Updated weights for policy 0, policy_version 232898 (0.0006) [2024-09-30 04:00:41,256][1157819] Updated weights for policy 0, policy_version 232908 (0.0006) [2024-09-30 04:00:41,864][1157819] Updated weights for policy 0, policy_version 232918 (0.0006) [2024-09-30 04:00:42,454][1157819] Updated weights for policy 0, policy_version 232928 (0.0006) [2024-09-30 04:00:43,005][1157819] Updated weights for policy 0, policy_version 232938 (0.0006) [2024-09-30 04:00:43,578][1157819] Updated weights for policy 0, policy_version 232948 (0.0006) [2024-09-30 04:00:44,135][1157819] Updated weights for policy 0, policy_version 232958 (0.0006) [2024-09-30 04:00:44,674][1157819] Updated weights for policy 0, policy_version 232968 (0.0006) [2024-09-30 04:00:45,229][1157819] Updated weights for policy 0, policy_version 232978 (0.0006) [2024-09-30 04:00:45,466][1157520] Fps is (10 sec: 71269.9, 60 sec: 74410.5, 300 sec: 80823.1). Total num frames: 954294272. Throughput: 0: 18540.1. Samples: 228542972. Policy #0 lag: (min: 0.0, avg: 1.6, max: 6.0) [2024-09-30 04:00:45,466][1157520] Avg episode reward: [(0, '58.244')] [2024-09-30 04:00:45,769][1157819] Updated weights for policy 0, policy_version 232988 (0.0006) [2024-09-30 04:00:46,303][1157819] Updated weights for policy 0, policy_version 232998 (0.0006) [2024-09-30 04:00:46,872][1157819] Updated weights for policy 0, policy_version 233008 (0.0006) [2024-09-30 04:00:47,440][1157819] Updated weights for policy 0, policy_version 233018 (0.0006) [2024-09-30 04:00:47,992][1157819] Updated weights for policy 0, policy_version 233028 (0.0006) [2024-09-30 04:00:48,561][1157819] Updated weights for policy 0, policy_version 233038 (0.0006) [2024-09-30 04:00:49,113][1157819] Updated weights for policy 0, policy_version 233048 (0.0006) [2024-09-30 04:00:49,604][1157819] Updated weights for policy 0, policy_version 233058 (0.0006) [2024-09-30 04:00:50,173][1157819] Updated weights for policy 0, policy_version 233068 (0.0006) [2024-09-30 04:00:50,466][1157520] Fps is (10 sec: 73318.5, 60 sec: 74410.8, 300 sec: 80781.5). Total num frames: 954667008. Throughput: 0: 18418.7. Samples: 228654096. Policy #0 lag: (min: 0.0, avg: 1.6, max: 6.0) [2024-09-30 04:00:50,466][1157520] Avg episode reward: [(0, '58.178')] [2024-09-30 04:00:50,709][1157819] Updated weights for policy 0, policy_version 233078 (0.0006) [2024-09-30 04:00:51,293][1157819] Updated weights for policy 0, policy_version 233088 (0.0006) [2024-09-30 04:00:51,841][1157819] Updated weights for policy 0, policy_version 233098 (0.0006) [2024-09-30 04:00:52,399][1157819] Updated weights for policy 0, policy_version 233108 (0.0006) [2024-09-30 04:00:52,951][1157819] Updated weights for policy 0, policy_version 233118 (0.0006) [2024-09-30 04:00:53,480][1157819] Updated weights for policy 0, policy_version 233128 (0.0006) [2024-09-30 04:00:54,069][1157819] Updated weights for policy 0, policy_version 233138 (0.0006) [2024-09-30 04:00:54,607][1157819] Updated weights for policy 0, policy_version 233148 (0.0006) [2024-09-30 04:00:55,184][1157819] Updated weights for policy 0, policy_version 233158 (0.0006) [2024-09-30 04:00:55,466][1157520] Fps is (10 sec: 74138.7, 60 sec: 74342.4, 300 sec: 80767.6). Total num frames: 955035648. Throughput: 0: 18452.6. Samples: 228709980. Policy #0 lag: (min: 0.0, avg: 1.6, max: 6.0) [2024-09-30 04:00:55,466][1157520] Avg episode reward: [(0, '57.956')] [2024-09-30 04:00:55,732][1157819] Updated weights for policy 0, policy_version 233168 (0.0006) [2024-09-30 04:00:56,308][1157819] Updated weights for policy 0, policy_version 233178 (0.0006) [2024-09-30 04:00:56,866][1157819] Updated weights for policy 0, policy_version 233188 (0.0006) [2024-09-30 04:00:57,424][1157819] Updated weights for policy 0, policy_version 233198 (0.0006) [2024-09-30 04:00:57,985][1157819] Updated weights for policy 0, policy_version 233208 (0.0006) [2024-09-30 04:00:58,521][1157819] Updated weights for policy 0, policy_version 233218 (0.0006) [2024-09-30 04:00:58,807][1157736] Signal inference workers to stop experience collection... (16600 times) [2024-09-30 04:00:58,808][1157736] Signal inference workers to resume experience collection... (16600 times) [2024-09-30 04:00:58,812][1157819] InferenceWorker_p0-w0: stopping experience collection (16600 times) [2024-09-30 04:00:58,812][1157819] InferenceWorker_p0-w0: resuming experience collection (16600 times) [2024-09-30 04:00:59,023][1157819] Updated weights for policy 0, policy_version 233228 (0.0006) [2024-09-30 04:00:59,607][1157819] Updated weights for policy 0, policy_version 233238 (0.0006) [2024-09-30 04:01:00,098][1157819] Updated weights for policy 0, policy_version 233248 (0.0006) [2024-09-30 04:01:00,466][1157520] Fps is (10 sec: 74547.2, 60 sec: 74137.7, 300 sec: 80725.9). Total num frames: 955412480. Throughput: 0: 18325.7. Samples: 228820312. Policy #0 lag: (min: 0.0, avg: 1.6, max: 6.0) [2024-09-30 04:01:00,466][1157520] Avg episode reward: [(0, '57.326')] [2024-09-30 04:01:00,603][1157819] Updated weights for policy 0, policy_version 233258 (0.0006) [2024-09-30 04:01:01,170][1157819] Updated weights for policy 0, policy_version 233268 (0.0006) [2024-09-30 04:01:01,739][1157819] Updated weights for policy 0, policy_version 233278 (0.0006) [2024-09-30 04:01:02,346][1157819] Updated weights for policy 0, policy_version 233288 (0.0006) [2024-09-30 04:01:02,939][1157819] Updated weights for policy 0, policy_version 233298 (0.0006) [2024-09-30 04:01:03,532][1157819] Updated weights for policy 0, policy_version 233308 (0.0006) [2024-09-30 04:01:04,114][1157819] Updated weights for policy 0, policy_version 233318 (0.0006) [2024-09-30 04:01:04,684][1157819] Updated weights for policy 0, policy_version 233328 (0.0006) [2024-09-30 04:01:05,237][1157819] Updated weights for policy 0, policy_version 233338 (0.0006) [2024-09-30 04:01:05,466][1157520] Fps is (10 sec: 73317.7, 60 sec: 73591.3, 300 sec: 80573.2). Total num frames: 955768832. Throughput: 0: 18175.4. Samples: 228929416. Policy #0 lag: (min: 0.0, avg: 1.6, max: 6.0) [2024-09-30 04:01:05,466][1157520] Avg episode reward: [(0, '57.499')] [2024-09-30 04:01:05,832][1157819] Updated weights for policy 0, policy_version 233348 (0.0006) [2024-09-30 04:01:06,354][1157819] Updated weights for policy 0, policy_version 233358 (0.0006) [2024-09-30 04:01:06,916][1157819] Updated weights for policy 0, policy_version 233368 (0.0006) [2024-09-30 04:01:07,482][1157819] Updated weights for policy 0, policy_version 233378 (0.0006) [2024-09-30 04:01:08,055][1157819] Updated weights for policy 0, policy_version 233388 (0.0006) [2024-09-30 04:01:08,628][1157819] Updated weights for policy 0, policy_version 233398 (0.0006) [2024-09-30 04:01:09,221][1157819] Updated weights for policy 0, policy_version 233408 (0.0006) [2024-09-30 04:01:09,799][1157819] Updated weights for policy 0, policy_version 233418 (0.0006) [2024-09-30 04:01:10,404][1157819] Updated weights for policy 0, policy_version 233428 (0.0006) [2024-09-30 04:01:10,466][1157520] Fps is (10 sec: 70861.0, 60 sec: 73318.5, 300 sec: 80406.6). Total num frames: 956121088. Throughput: 0: 18165.4. Samples: 228984884. Policy #0 lag: (min: 0.0, avg: 1.6, max: 6.0) [2024-09-30 04:01:10,466][1157520] Avg episode reward: [(0, '55.489')] [2024-09-30 04:01:11,039][1157819] Updated weights for policy 0, policy_version 233438 (0.0006) [2024-09-30 04:01:11,641][1157819] Updated weights for policy 0, policy_version 233448 (0.0006) [2024-09-30 04:01:12,234][1157819] Updated weights for policy 0, policy_version 233458 (0.0006) [2024-09-30 04:01:12,821][1157819] Updated weights for policy 0, policy_version 233468 (0.0006) [2024-09-30 04:01:13,422][1157819] Updated weights for policy 0, policy_version 233478 (0.0006) [2024-09-30 04:01:14,012][1157819] Updated weights for policy 0, policy_version 233488 (0.0006) [2024-09-30 04:01:14,593][1157819] Updated weights for policy 0, policy_version 233498 (0.0006) [2024-09-30 04:01:15,209][1157819] Updated weights for policy 0, policy_version 233508 (0.0006) [2024-09-30 04:01:15,466][1157520] Fps is (10 sec: 69632.2, 60 sec: 72635.7, 300 sec: 80184.4). Total num frames: 956465152. Throughput: 0: 17982.3. Samples: 229087652. Policy #0 lag: (min: 0.0, avg: 1.6, max: 6.0) [2024-09-30 04:01:15,466][1157520] Avg episode reward: [(0, '57.769')] [2024-09-30 04:01:15,769][1157819] Updated weights for policy 0, policy_version 233518 (0.0006) [2024-09-30 04:01:16,352][1157819] Updated weights for policy 0, policy_version 233528 (0.0006) [2024-09-30 04:01:16,925][1157819] Updated weights for policy 0, policy_version 233538 (0.0006) [2024-09-30 04:01:17,455][1157819] Updated weights for policy 0, policy_version 233548 (0.0006) [2024-09-30 04:01:18,021][1157819] Updated weights for policy 0, policy_version 233558 (0.0006) [2024-09-30 04:01:18,556][1157819] Updated weights for policy 0, policy_version 233568 (0.0006) [2024-09-30 04:01:19,079][1157819] Updated weights for policy 0, policy_version 233578 (0.0006) [2024-09-30 04:01:19,612][1157819] Updated weights for policy 0, policy_version 233588 (0.0006) [2024-09-30 04:01:20,156][1157819] Updated weights for policy 0, policy_version 233598 (0.0006) [2024-09-30 04:01:20,466][1157520] Fps is (10 sec: 71679.2, 60 sec: 72362.6, 300 sec: 80045.5). Total num frames: 956837888. Throughput: 0: 18079.2. Samples: 229197040. Policy #0 lag: (min: 0.0, avg: 1.6, max: 6.0) [2024-09-30 04:01:20,466][1157520] Avg episode reward: [(0, '55.354')] [2024-09-30 04:01:20,714][1157819] Updated weights for policy 0, policy_version 233608 (0.0006) [2024-09-30 04:01:21,229][1157819] Updated weights for policy 0, policy_version 233618 (0.0006) [2024-09-30 04:01:21,804][1157819] Updated weights for policy 0, policy_version 233628 (0.0006) [2024-09-30 04:01:22,345][1157819] Updated weights for policy 0, policy_version 233638 (0.0006) [2024-09-30 04:01:22,883][1157819] Updated weights for policy 0, policy_version 233648 (0.0006) [2024-09-30 04:01:23,447][1157819] Updated weights for policy 0, policy_version 233658 (0.0006) [2024-09-30 04:01:23,963][1157819] Updated weights for policy 0, policy_version 233668 (0.0006) [2024-09-30 04:01:24,512][1157819] Updated weights for policy 0, policy_version 233678 (0.0006) [2024-09-30 04:01:25,089][1157819] Updated weights for policy 0, policy_version 233688 (0.0006) [2024-09-30 04:01:25,466][1157520] Fps is (10 sec: 74957.3, 60 sec: 72499.2, 300 sec: 79976.1). Total num frames: 957214720. Throughput: 0: 18154.3. Samples: 229253844. Policy #0 lag: (min: 0.0, avg: 1.6, max: 6.0) [2024-09-30 04:01:25,466][1157520] Avg episode reward: [(0, '56.114')] [2024-09-30 04:01:25,628][1157819] Updated weights for policy 0, policy_version 233698 (0.0006) [2024-09-30 04:01:26,221][1157819] Updated weights for policy 0, policy_version 233708 (0.0006) [2024-09-30 04:01:26,745][1157819] Updated weights for policy 0, policy_version 233718 (0.0006) [2024-09-30 04:01:27,311][1157819] Updated weights for policy 0, policy_version 233728 (0.0006) [2024-09-30 04:01:27,890][1157819] Updated weights for policy 0, policy_version 233738 (0.0006) [2024-09-30 04:01:28,481][1157819] Updated weights for policy 0, policy_version 233748 (0.0006) [2024-09-30 04:01:29,070][1157819] Updated weights for policy 0, policy_version 233758 (0.0006) [2024-09-30 04:01:29,630][1157819] Updated weights for policy 0, policy_version 233768 (0.0006) [2024-09-30 04:01:30,217][1157819] Updated weights for policy 0, policy_version 233778 (0.0006) [2024-09-30 04:01:30,466][1157520] Fps is (10 sec: 73318.5, 60 sec: 72362.6, 300 sec: 79823.4). Total num frames: 957571072. Throughput: 0: 18229.5. Samples: 229363296. Policy #0 lag: (min: 0.0, avg: 1.6, max: 6.0) [2024-09-30 04:01:30,466][1157520] Avg episode reward: [(0, '57.971')] [2024-09-30 04:01:30,833][1157819] Updated weights for policy 0, policy_version 233788 (0.0006) [2024-09-30 04:01:31,403][1157819] Updated weights for policy 0, policy_version 233798 (0.0006) [2024-09-30 04:01:31,988][1157819] Updated weights for policy 0, policy_version 233808 (0.0006) [2024-09-30 04:01:32,559][1157819] Updated weights for policy 0, policy_version 233818 (0.0006) [2024-09-30 04:01:33,132][1157819] Updated weights for policy 0, policy_version 233828 (0.0006) [2024-09-30 04:01:33,676][1157819] Updated weights for policy 0, policy_version 233838 (0.0006) [2024-09-30 04:01:34,214][1157819] Updated weights for policy 0, policy_version 233848 (0.0006) [2024-09-30 04:01:34,785][1157819] Updated weights for policy 0, policy_version 233858 (0.0006) [2024-09-30 04:01:35,312][1157819] Updated weights for policy 0, policy_version 233868 (0.0006) [2024-09-30 04:01:35,466][1157520] Fps is (10 sec: 71679.8, 60 sec: 72499.2, 300 sec: 79712.3). Total num frames: 957931520. Throughput: 0: 18157.2. Samples: 229471172. Policy #0 lag: (min: 0.0, avg: 1.6, max: 6.0) [2024-09-30 04:01:35,466][1157520] Avg episode reward: [(0, '57.322')] [2024-09-30 04:01:35,816][1157819] Updated weights for policy 0, policy_version 233878 (0.0006) [2024-09-30 04:01:36,368][1157819] Updated weights for policy 0, policy_version 233888 (0.0006) [2024-09-30 04:01:36,936][1157819] Updated weights for policy 0, policy_version 233898 (0.0006) [2024-09-30 04:01:37,479][1157819] Updated weights for policy 0, policy_version 233908 (0.0006) [2024-09-30 04:01:38,049][1157819] Updated weights for policy 0, policy_version 233918 (0.0006) [2024-09-30 04:01:38,605][1157819] Updated weights for policy 0, policy_version 233928 (0.0006) [2024-09-30 04:01:39,177][1157819] Updated weights for policy 0, policy_version 233938 (0.0006) [2024-09-30 04:01:39,696][1157819] Updated weights for policy 0, policy_version 233948 (0.0006) [2024-09-30 04:01:40,200][1157819] Updated weights for policy 0, policy_version 233958 (0.0006) [2024-09-30 04:01:40,466][1157520] Fps is (10 sec: 73728.4, 60 sec: 72908.8, 300 sec: 79670.7). Total num frames: 958308352. Throughput: 0: 18160.8. Samples: 229527216. Policy #0 lag: (min: 0.0, avg: 1.6, max: 6.0) [2024-09-30 04:01:40,466][1157520] Avg episode reward: [(0, '55.245')] [2024-09-30 04:01:40,763][1157819] Updated weights for policy 0, policy_version 233968 (0.0006) [2024-09-30 04:01:41,346][1157819] Updated weights for policy 0, policy_version 233978 (0.0006) [2024-09-30 04:01:41,873][1157819] Updated weights for policy 0, policy_version 233988 (0.0006) [2024-09-30 04:01:42,455][1157819] Updated weights for policy 0, policy_version 233998 (0.0006) [2024-09-30 04:01:42,481][1157736] Signal inference workers to stop experience collection... (16650 times) [2024-09-30 04:01:42,485][1157736] Signal inference workers to resume experience collection... (16650 times) [2024-09-30 04:01:42,489][1157819] InferenceWorker_p0-w0: stopping experience collection (16650 times) [2024-09-30 04:01:42,491][1157819] InferenceWorker_p0-w0: resuming experience collection (16650 times) [2024-09-30 04:01:42,973][1157819] Updated weights for policy 0, policy_version 234008 (0.0006) [2024-09-30 04:01:43,531][1157819] Updated weights for policy 0, policy_version 234018 (0.0006) [2024-09-30 04:01:44,098][1157819] Updated weights for policy 0, policy_version 234028 (0.0006) [2024-09-30 04:01:44,613][1157819] Updated weights for policy 0, policy_version 234038 (0.0006) [2024-09-30 04:01:45,159][1157819] Updated weights for policy 0, policy_version 234048 (0.0006) [2024-09-30 04:01:45,466][1157520] Fps is (10 sec: 74956.9, 60 sec: 73113.7, 300 sec: 79517.9). Total num frames: 958681088. Throughput: 0: 18200.2. Samples: 229639320. Policy #0 lag: (min: 0.0, avg: 1.6, max: 6.0) [2024-09-30 04:01:45,466][1157520] Avg episode reward: [(0, '56.950')] [2024-09-30 04:01:45,720][1157819] Updated weights for policy 0, policy_version 234058 (0.0006) [2024-09-30 04:01:46,284][1157819] Updated weights for policy 0, policy_version 234068 (0.0006) [2024-09-30 04:01:46,842][1157819] Updated weights for policy 0, policy_version 234078 (0.0007) [2024-09-30 04:01:47,394][1157819] Updated weights for policy 0, policy_version 234088 (0.0007) [2024-09-30 04:01:47,925][1157819] Updated weights for policy 0, policy_version 234098 (0.0007) [2024-09-30 04:01:48,516][1157819] Updated weights for policy 0, policy_version 234108 (0.0007) [2024-09-30 04:01:49,047][1157819] Updated weights for policy 0, policy_version 234118 (0.0007) [2024-09-30 04:01:49,627][1157819] Updated weights for policy 0, policy_version 234128 (0.0007) [2024-09-30 04:01:50,189][1157819] Updated weights for policy 0, policy_version 234138 (0.0006) [2024-09-30 04:01:50,466][1157520] Fps is (10 sec: 73727.9, 60 sec: 72977.0, 300 sec: 79268.0). Total num frames: 959045632. Throughput: 0: 18232.7. Samples: 229749884. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 04:01:50,466][1157520] Avg episode reward: [(0, '55.692')] [2024-09-30 04:01:50,785][1157819] Updated weights for policy 0, policy_version 234148 (0.0006) [2024-09-30 04:01:51,366][1157819] Updated weights for policy 0, policy_version 234158 (0.0006) [2024-09-30 04:01:52,022][1157819] Updated weights for policy 0, policy_version 234168 (0.0006) [2024-09-30 04:01:52,602][1157819] Updated weights for policy 0, policy_version 234178 (0.0006) [2024-09-30 04:01:53,193][1157819] Updated weights for policy 0, policy_version 234188 (0.0006) [2024-09-30 04:01:53,798][1157819] Updated weights for policy 0, policy_version 234198 (0.0006) [2024-09-30 04:01:54,412][1157819] Updated weights for policy 0, policy_version 234208 (0.0006) [2024-09-30 04:01:55,008][1157819] Updated weights for policy 0, policy_version 234218 (0.0006) [2024-09-30 04:01:55,466][1157520] Fps is (10 sec: 70860.5, 60 sec: 72567.4, 300 sec: 78920.9). Total num frames: 959389696. Throughput: 0: 18147.2. Samples: 229801512. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 04:01:55,466][1157520] Avg episode reward: [(0, '56.430')] [2024-09-30 04:01:55,577][1157819] Updated weights for policy 0, policy_version 234228 (0.0006) [2024-09-30 04:01:56,166][1157819] Updated weights for policy 0, policy_version 234238 (0.0007) [2024-09-30 04:01:56,745][1157819] Updated weights for policy 0, policy_version 234248 (0.0006) [2024-09-30 04:01:57,287][1157819] Updated weights for policy 0, policy_version 234258 (0.0007) [2024-09-30 04:01:57,826][1157819] Updated weights for policy 0, policy_version 234268 (0.0006) [2024-09-30 04:01:58,350][1157819] Updated weights for policy 0, policy_version 234278 (0.0006) [2024-09-30 04:01:58,889][1157819] Updated weights for policy 0, policy_version 234288 (0.0006) [2024-09-30 04:01:59,457][1157819] Updated weights for policy 0, policy_version 234298 (0.0006) [2024-09-30 04:01:59,995][1157819] Updated weights for policy 0, policy_version 234308 (0.0006) [2024-09-30 04:02:00,466][1157520] Fps is (10 sec: 71680.2, 60 sec: 72499.2, 300 sec: 78657.1). Total num frames: 959762432. Throughput: 0: 18246.5. Samples: 229908744. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 04:02:00,466][1157520] Avg episode reward: [(0, '55.487')] [2024-09-30 04:02:00,524][1157819] Updated weights for policy 0, policy_version 234318 (0.0006) [2024-09-30 04:02:01,082][1157819] Updated weights for policy 0, policy_version 234328 (0.0006) [2024-09-30 04:02:01,666][1157819] Updated weights for policy 0, policy_version 234338 (0.0006) [2024-09-30 04:02:02,269][1157819] Updated weights for policy 0, policy_version 234348 (0.0006) [2024-09-30 04:02:02,822][1157819] Updated weights for policy 0, policy_version 234358 (0.0006) [2024-09-30 04:02:03,367][1157819] Updated weights for policy 0, policy_version 234368 (0.0006) [2024-09-30 04:02:03,908][1157819] Updated weights for policy 0, policy_version 234378 (0.0006) [2024-09-30 04:02:04,451][1157819] Updated weights for policy 0, policy_version 234388 (0.0006) [2024-09-30 04:02:05,000][1157819] Updated weights for policy 0, policy_version 234398 (0.0006) [2024-09-30 04:02:05,466][1157520] Fps is (10 sec: 73727.4, 60 sec: 72635.7, 300 sec: 78365.5). Total num frames: 960126976. Throughput: 0: 18281.0. Samples: 230019684. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 04:02:05,466][1157520] Avg episode reward: [(0, '58.013')] [2024-09-30 04:02:05,472][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000234407_960131072.pth... [2024-09-30 04:02:05,517][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000229899_941666304.pth [2024-09-30 04:02:05,547][1157819] Updated weights for policy 0, policy_version 234408 (0.0006) [2024-09-30 04:02:06,080][1157819] Updated weights for policy 0, policy_version 234418 (0.0006) [2024-09-30 04:02:06,630][1157819] Updated weights for policy 0, policy_version 234428 (0.0006) [2024-09-30 04:02:07,183][1157819] Updated weights for policy 0, policy_version 234438 (0.0006) [2024-09-30 04:02:07,718][1157819] Updated weights for policy 0, policy_version 234448 (0.0006) [2024-09-30 04:02:08,268][1157819] Updated weights for policy 0, policy_version 234458 (0.0006) [2024-09-30 04:02:08,793][1157819] Updated weights for policy 0, policy_version 234468 (0.0006) [2024-09-30 04:02:09,338][1157819] Updated weights for policy 0, policy_version 234478 (0.0006) [2024-09-30 04:02:09,915][1157819] Updated weights for policy 0, policy_version 234488 (0.0006) [2024-09-30 04:02:10,459][1157819] Updated weights for policy 0, policy_version 234498 (0.0007) [2024-09-30 04:02:10,466][1157520] Fps is (10 sec: 74137.6, 60 sec: 73045.3, 300 sec: 78101.7). Total num frames: 960503808. Throughput: 0: 18272.9. Samples: 230076124. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 04:02:10,466][1157520] Avg episode reward: [(0, '55.842')] [2024-09-30 04:02:10,999][1157819] Updated weights for policy 0, policy_version 234508 (0.0006) [2024-09-30 04:02:11,549][1157819] Updated weights for policy 0, policy_version 234518 (0.0006) [2024-09-30 04:02:12,068][1157819] Updated weights for policy 0, policy_version 234528 (0.0006) [2024-09-30 04:02:12,570][1157819] Updated weights for policy 0, policy_version 234538 (0.0006) [2024-09-30 04:02:13,192][1157819] Updated weights for policy 0, policy_version 234548 (0.0006) [2024-09-30 04:02:13,780][1157819] Updated weights for policy 0, policy_version 234558 (0.0006) [2024-09-30 04:02:14,333][1157819] Updated weights for policy 0, policy_version 234568 (0.0006) [2024-09-30 04:02:14,904][1157819] Updated weights for policy 0, policy_version 234578 (0.0006) [2024-09-30 04:02:15,466][1157520] Fps is (10 sec: 74138.3, 60 sec: 73386.7, 300 sec: 77796.2). Total num frames: 960868352. Throughput: 0: 18320.4. Samples: 230187712. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 04:02:15,466][1157520] Avg episode reward: [(0, '57.833')] [2024-09-30 04:02:15,475][1157819] Updated weights for policy 0, policy_version 234588 (0.0006) [2024-09-30 04:02:16,038][1157819] Updated weights for policy 0, policy_version 234598 (0.0006) [2024-09-30 04:02:16,543][1157819] Updated weights for policy 0, policy_version 234608 (0.0006) [2024-09-30 04:02:17,129][1157819] Updated weights for policy 0, policy_version 234618 (0.0006) [2024-09-30 04:02:17,669][1157819] Updated weights for policy 0, policy_version 234628 (0.0006) [2024-09-30 04:02:18,192][1157819] Updated weights for policy 0, policy_version 234638 (0.0006) [2024-09-30 04:02:18,732][1157819] Updated weights for policy 0, policy_version 234648 (0.0006) [2024-09-30 04:02:19,290][1157819] Updated weights for policy 0, policy_version 234658 (0.0006) [2024-09-30 04:02:19,779][1157736] Signal inference workers to stop experience collection... (16700 times) [2024-09-30 04:02:19,784][1157736] Signal inference workers to resume experience collection... (16700 times) [2024-09-30 04:02:19,784][1157819] InferenceWorker_p0-w0: stopping experience collection (16700 times) [2024-09-30 04:02:19,788][1157819] InferenceWorker_p0-w0: resuming experience collection (16700 times) [2024-09-30 04:02:19,851][1157819] Updated weights for policy 0, policy_version 234668 (0.0006) [2024-09-30 04:02:20,398][1157819] Updated weights for policy 0, policy_version 234678 (0.0006) [2024-09-30 04:02:20,466][1157520] Fps is (10 sec: 74137.4, 60 sec: 73455.0, 300 sec: 77588.0). Total num frames: 961245184. Throughput: 0: 18385.9. Samples: 230298536. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 04:02:20,466][1157520] Avg episode reward: [(0, '55.382')] [2024-09-30 04:02:20,980][1157819] Updated weights for policy 0, policy_version 234688 (0.0006) [2024-09-30 04:02:21,486][1157819] Updated weights for policy 0, policy_version 234698 (0.0006) [2024-09-30 04:02:22,038][1157819] Updated weights for policy 0, policy_version 234708 (0.0006) [2024-09-30 04:02:22,580][1157819] Updated weights for policy 0, policy_version 234718 (0.0006) [2024-09-30 04:02:23,142][1157819] Updated weights for policy 0, policy_version 234728 (0.0006) [2024-09-30 04:02:23,664][1157819] Updated weights for policy 0, policy_version 234738 (0.0006) [2024-09-30 04:02:24,237][1157819] Updated weights for policy 0, policy_version 234748 (0.0006) [2024-09-30 04:02:24,789][1157819] Updated weights for policy 0, policy_version 234758 (0.0006) [2024-09-30 04:02:25,372][1157819] Updated weights for policy 0, policy_version 234768 (0.0006) [2024-09-30 04:02:25,466][1157520] Fps is (10 sec: 74956.9, 60 sec: 73386.6, 300 sec: 77379.7). Total num frames: 961617920. Throughput: 0: 18390.2. Samples: 230354776. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 04:02:25,466][1157520] Avg episode reward: [(0, '56.075')] [2024-09-30 04:02:25,910][1157819] Updated weights for policy 0, policy_version 234778 (0.0006) [2024-09-30 04:02:26,461][1157819] Updated weights for policy 0, policy_version 234788 (0.0006) [2024-09-30 04:02:27,003][1157819] Updated weights for policy 0, policy_version 234798 (0.0006) [2024-09-30 04:02:27,531][1157819] Updated weights for policy 0, policy_version 234808 (0.0006) [2024-09-30 04:02:28,087][1157819] Updated weights for policy 0, policy_version 234818 (0.0006) [2024-09-30 04:02:28,655][1157819] Updated weights for policy 0, policy_version 234828 (0.0006) [2024-09-30 04:02:29,203][1157819] Updated weights for policy 0, policy_version 234838 (0.0006) [2024-09-30 04:02:29,747][1157819] Updated weights for policy 0, policy_version 234848 (0.0006) [2024-09-30 04:02:30,296][1157819] Updated weights for policy 0, policy_version 234858 (0.0006) [2024-09-30 04:02:30,466][1157520] Fps is (10 sec: 74137.7, 60 sec: 73591.5, 300 sec: 77199.2). Total num frames: 961986560. Throughput: 0: 18375.5. Samples: 230466216. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 04:02:30,466][1157520] Avg episode reward: [(0, '56.757')] [2024-09-30 04:02:30,859][1157819] Updated weights for policy 0, policy_version 234868 (0.0006) [2024-09-30 04:02:31,396][1157819] Updated weights for policy 0, policy_version 234878 (0.0006) [2024-09-30 04:02:31,963][1157819] Updated weights for policy 0, policy_version 234888 (0.0006) [2024-09-30 04:02:32,496][1157819] Updated weights for policy 0, policy_version 234898 (0.0006) [2024-09-30 04:02:33,081][1157819] Updated weights for policy 0, policy_version 234908 (0.0006) [2024-09-30 04:02:33,622][1157819] Updated weights for policy 0, policy_version 234918 (0.0006) [2024-09-30 04:02:34,158][1157819] Updated weights for policy 0, policy_version 234928 (0.0006) [2024-09-30 04:02:34,739][1157819] Updated weights for policy 0, policy_version 234938 (0.0006) [2024-09-30 04:02:35,306][1157819] Updated weights for policy 0, policy_version 234948 (0.0006) [2024-09-30 04:02:35,466][1157520] Fps is (10 sec: 73727.2, 60 sec: 73727.9, 300 sec: 77060.3). Total num frames: 962355200. Throughput: 0: 18380.2. Samples: 230576996. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 04:02:35,466][1157520] Avg episode reward: [(0, '56.482')] [2024-09-30 04:02:35,862][1157819] Updated weights for policy 0, policy_version 234958 (0.0006) [2024-09-30 04:02:36,355][1157819] Updated weights for policy 0, policy_version 234968 (0.0006) [2024-09-30 04:02:36,857][1157819] Updated weights for policy 0, policy_version 234978 (0.0006) [2024-09-30 04:02:37,355][1157819] Updated weights for policy 0, policy_version 234988 (0.0006) [2024-09-30 04:02:37,849][1157819] Updated weights for policy 0, policy_version 234998 (0.0006) [2024-09-30 04:02:38,322][1157819] Updated weights for policy 0, policy_version 235008 (0.0006) [2024-09-30 04:02:38,829][1157819] Updated weights for policy 0, policy_version 235018 (0.0006) [2024-09-30 04:02:39,322][1157819] Updated weights for policy 0, policy_version 235028 (0.0006) [2024-09-30 04:02:39,803][1157819] Updated weights for policy 0, policy_version 235038 (0.0006) [2024-09-30 04:02:40,314][1157819] Updated weights for policy 0, policy_version 235048 (0.0006) [2024-09-30 04:02:40,466][1157520] Fps is (10 sec: 78233.5, 60 sec: 74342.4, 300 sec: 77060.3). Total num frames: 962768896. Throughput: 0: 18564.5. Samples: 230636912. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 04:02:40,466][1157520] Avg episode reward: [(0, '57.870')] [2024-09-30 04:02:40,802][1157819] Updated weights for policy 0, policy_version 235058 (0.0006) [2024-09-30 04:02:41,271][1157819] Updated weights for policy 0, policy_version 235068 (0.0006) [2024-09-30 04:02:41,766][1157819] Updated weights for policy 0, policy_version 235078 (0.0006) [2024-09-30 04:02:42,268][1157819] Updated weights for policy 0, policy_version 235088 (0.0006) [2024-09-30 04:02:42,809][1157819] Updated weights for policy 0, policy_version 235098 (0.0006) [2024-09-30 04:02:43,309][1157819] Updated weights for policy 0, policy_version 235108 (0.0006) [2024-09-30 04:02:43,833][1157819] Updated weights for policy 0, policy_version 235118 (0.0006) [2024-09-30 04:02:44,357][1157819] Updated weights for policy 0, policy_version 235128 (0.0006) [2024-09-30 04:02:44,852][1157819] Updated weights for policy 0, policy_version 235138 (0.0006) [2024-09-30 04:02:45,333][1157819] Updated weights for policy 0, policy_version 235148 (0.0006) [2024-09-30 04:02:45,466][1157520] Fps is (10 sec: 81920.7, 60 sec: 74888.5, 300 sec: 77004.8). Total num frames: 963174400. Throughput: 0: 18914.0. Samples: 230759876. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 04:02:45,466][1157520] Avg episode reward: [(0, '57.094')] [2024-09-30 04:02:45,862][1157819] Updated weights for policy 0, policy_version 235158 (0.0006) [2024-09-30 04:02:46,355][1157819] Updated weights for policy 0, policy_version 235168 (0.0006) [2024-09-30 04:02:46,873][1157819] Updated weights for policy 0, policy_version 235178 (0.0006) [2024-09-30 04:02:47,389][1157819] Updated weights for policy 0, policy_version 235188 (0.0006) [2024-09-30 04:02:47,893][1157819] Updated weights for policy 0, policy_version 235198 (0.0006) [2024-09-30 04:02:48,421][1157819] Updated weights for policy 0, policy_version 235208 (0.0006) [2024-09-30 04:02:48,944][1157819] Updated weights for policy 0, policy_version 235218 (0.0006) [2024-09-30 04:02:49,476][1157819] Updated weights for policy 0, policy_version 235228 (0.0006) [2024-09-30 04:02:49,975][1157819] Updated weights for policy 0, policy_version 235238 (0.0006) [2024-09-30 04:02:50,466][1157520] Fps is (10 sec: 80281.8, 60 sec: 75434.7, 300 sec: 76990.9). Total num frames: 963571712. Throughput: 0: 19114.2. Samples: 230879820. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 04:02:50,466][1157520] Avg episode reward: [(0, '57.905')] [2024-09-30 04:02:50,509][1157819] Updated weights for policy 0, policy_version 235248 (0.0006) [2024-09-30 04:02:51,027][1157819] Updated weights for policy 0, policy_version 235258 (0.0006) [2024-09-30 04:02:51,544][1157819] Updated weights for policy 0, policy_version 235268 (0.0006) [2024-09-30 04:02:52,069][1157819] Updated weights for policy 0, policy_version 235278 (0.0006) [2024-09-30 04:02:52,582][1157819] Updated weights for policy 0, policy_version 235288 (0.0006) [2024-09-30 04:02:53,120][1157819] Updated weights for policy 0, policy_version 235298 (0.0006) [2024-09-30 04:02:53,649][1157819] Updated weights for policy 0, policy_version 235308 (0.0006) [2024-09-30 04:02:54,230][1157819] Updated weights for policy 0, policy_version 235318 (0.0006) [2024-09-30 04:02:54,800][1157819] Updated weights for policy 0, policy_version 235328 (0.0006) [2024-09-30 04:02:55,304][1157819] Updated weights for policy 0, policy_version 235338 (0.0006) [2024-09-30 04:02:55,466][1157520] Fps is (10 sec: 77824.1, 60 sec: 76049.1, 300 sec: 76921.5). Total num frames: 963952640. Throughput: 0: 19169.3. Samples: 230938744. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 04:02:55,466][1157520] Avg episode reward: [(0, '56.545')] [2024-09-30 04:02:55,890][1157819] Updated weights for policy 0, policy_version 235348 (0.0006) [2024-09-30 04:02:56,422][1157819] Updated weights for policy 0, policy_version 235358 (0.0006) [2024-09-30 04:02:56,945][1157819] Updated weights for policy 0, policy_version 235368 (0.0006) [2024-09-30 04:02:57,508][1157819] Updated weights for policy 0, policy_version 235378 (0.0006) [2024-09-30 04:02:58,068][1157819] Updated weights for policy 0, policy_version 235388 (0.0006) [2024-09-30 04:02:58,587][1157819] Updated weights for policy 0, policy_version 235398 (0.0006) [2024-09-30 04:02:59,122][1157819] Updated weights for policy 0, policy_version 235408 (0.0006) [2024-09-30 04:02:59,681][1157819] Updated weights for policy 0, policy_version 235418 (0.0006) [2024-09-30 04:03:00,212][1157819] Updated weights for policy 0, policy_version 235428 (0.0006) [2024-09-30 04:03:00,466][1157520] Fps is (10 sec: 75775.5, 60 sec: 76117.3, 300 sec: 76838.2). Total num frames: 964329472. Throughput: 0: 19191.5. Samples: 231051328. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 04:03:00,466][1157520] Avg episode reward: [(0, '55.807')] [2024-09-30 04:03:00,749][1157819] Updated weights for policy 0, policy_version 235438 (0.0006) [2024-09-30 04:03:01,331][1157819] Updated weights for policy 0, policy_version 235448 (0.0006) [2024-09-30 04:03:01,864][1157819] Updated weights for policy 0, policy_version 235458 (0.0006) [2024-09-30 04:03:02,436][1157819] Updated weights for policy 0, policy_version 235468 (0.0006) [2024-09-30 04:03:02,976][1157819] Updated weights for policy 0, policy_version 235478 (0.0006) [2024-09-30 04:03:03,542][1157819] Updated weights for policy 0, policy_version 235488 (0.0006) [2024-09-30 04:03:04,103][1157819] Updated weights for policy 0, policy_version 235498 (0.0006) [2024-09-30 04:03:04,633][1157819] Updated weights for policy 0, policy_version 235508 (0.0006) [2024-09-30 04:03:05,149][1157819] Updated weights for policy 0, policy_version 235518 (0.0006) [2024-09-30 04:03:05,466][1157520] Fps is (10 sec: 74956.8, 60 sec: 76254.0, 300 sec: 76740.9). Total num frames: 964702208. Throughput: 0: 19210.1. Samples: 231162992. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 04:03:05,466][1157520] Avg episode reward: [(0, '56.680')] [2024-09-30 04:03:05,745][1157819] Updated weights for policy 0, policy_version 235528 (0.0006) [2024-09-30 04:03:06,287][1157819] Updated weights for policy 0, policy_version 235538 (0.0006) [2024-09-30 04:03:06,841][1157819] Updated weights for policy 0, policy_version 235548 (0.0006) [2024-09-30 04:03:07,401][1157819] Updated weights for policy 0, policy_version 235558 (0.0006) [2024-09-30 04:03:07,938][1157819] Updated weights for policy 0, policy_version 235568 (0.0006) [2024-09-30 04:03:08,510][1157819] Updated weights for policy 0, policy_version 235578 (0.0006) [2024-09-30 04:03:08,862][1157736] Signal inference workers to stop experience collection... (16750 times) [2024-09-30 04:03:08,864][1157736] Signal inference workers to resume experience collection... (16750 times) [2024-09-30 04:03:08,865][1157819] InferenceWorker_p0-w0: stopping experience collection (16750 times) [2024-09-30 04:03:08,869][1157819] InferenceWorker_p0-w0: resuming experience collection (16750 times) [2024-09-30 04:03:09,033][1157819] Updated weights for policy 0, policy_version 235588 (0.0006) [2024-09-30 04:03:09,583][1157819] Updated weights for policy 0, policy_version 235598 (0.0006) [2024-09-30 04:03:10,091][1157819] Updated weights for policy 0, policy_version 235608 (0.0006) [2024-09-30 04:03:10,466][1157520] Fps is (10 sec: 74957.0, 60 sec: 76253.8, 300 sec: 76657.7). Total num frames: 965079040. Throughput: 0: 19194.2. Samples: 231218516. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 04:03:10,466][1157520] Avg episode reward: [(0, '56.129')] [2024-09-30 04:03:10,607][1157819] Updated weights for policy 0, policy_version 235618 (0.0006) [2024-09-30 04:03:11,127][1157819] Updated weights for policy 0, policy_version 235628 (0.0006) [2024-09-30 04:03:11,658][1157819] Updated weights for policy 0, policy_version 235638 (0.0006) [2024-09-30 04:03:12,163][1157819] Updated weights for policy 0, policy_version 235648 (0.0006) [2024-09-30 04:03:12,680][1157819] Updated weights for policy 0, policy_version 235658 (0.0006) [2024-09-30 04:03:13,203][1157819] Updated weights for policy 0, policy_version 235668 (0.0007) [2024-09-30 04:03:13,720][1157819] Updated weights for policy 0, policy_version 235678 (0.0006) [2024-09-30 04:03:14,230][1157819] Updated weights for policy 0, policy_version 235688 (0.0006) [2024-09-30 04:03:14,762][1157819] Updated weights for policy 0, policy_version 235698 (0.0006) [2024-09-30 04:03:15,262][1157819] Updated weights for policy 0, policy_version 235708 (0.0006) [2024-09-30 04:03:15,466][1157520] Fps is (10 sec: 77004.1, 60 sec: 76731.6, 300 sec: 76643.8). Total num frames: 965472256. Throughput: 0: 19314.3. Samples: 231335360. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 04:03:15,466][1157520] Avg episode reward: [(0, '55.010')] [2024-09-30 04:03:15,784][1157819] Updated weights for policy 0, policy_version 235718 (0.0006) [2024-09-30 04:03:16,280][1157819] Updated weights for policy 0, policy_version 235728 (0.0006) [2024-09-30 04:03:16,830][1157819] Updated weights for policy 0, policy_version 235738 (0.0006) [2024-09-30 04:03:17,378][1157819] Updated weights for policy 0, policy_version 235748 (0.0006) [2024-09-30 04:03:17,896][1157819] Updated weights for policy 0, policy_version 235758 (0.0006) [2024-09-30 04:03:18,389][1157819] Updated weights for policy 0, policy_version 235768 (0.0006) [2024-09-30 04:03:18,904][1157819] Updated weights for policy 0, policy_version 235778 (0.0006) [2024-09-30 04:03:19,446][1157819] Updated weights for policy 0, policy_version 235788 (0.0006) [2024-09-30 04:03:19,935][1157819] Updated weights for policy 0, policy_version 235798 (0.0006) [2024-09-30 04:03:20,466][1157520] Fps is (10 sec: 78642.8, 60 sec: 77004.7, 300 sec: 76546.6). Total num frames: 965865472. Throughput: 0: 19491.7. Samples: 231454120. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 04:03:20,466][1157520] Avg episode reward: [(0, '56.892')] [2024-09-30 04:03:20,474][1157819] Updated weights for policy 0, policy_version 235808 (0.0006) [2024-09-30 04:03:20,976][1157819] Updated weights for policy 0, policy_version 235818 (0.0006) [2024-09-30 04:03:21,483][1157819] Updated weights for policy 0, policy_version 235828 (0.0006) [2024-09-30 04:03:22,027][1157819] Updated weights for policy 0, policy_version 235838 (0.0006) [2024-09-30 04:03:22,542][1157819] Updated weights for policy 0, policy_version 235848 (0.0006) [2024-09-30 04:03:23,064][1157819] Updated weights for policy 0, policy_version 235858 (0.0006) [2024-09-30 04:03:23,624][1157819] Updated weights for policy 0, policy_version 235868 (0.0006) [2024-09-30 04:03:24,103][1157819] Updated weights for policy 0, policy_version 235878 (0.0006) [2024-09-30 04:03:24,640][1157819] Updated weights for policy 0, policy_version 235888 (0.0006) [2024-09-30 04:03:25,189][1157819] Updated weights for policy 0, policy_version 235898 (0.0006) [2024-09-30 04:03:25,466][1157520] Fps is (10 sec: 78645.0, 60 sec: 77346.3, 300 sec: 76393.9). Total num frames: 966258688. Throughput: 0: 19471.5. Samples: 231513128. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 04:03:25,466][1157520] Avg episode reward: [(0, '57.234')] [2024-09-30 04:03:25,712][1157819] Updated weights for policy 0, policy_version 235908 (0.0006) [2024-09-30 04:03:26,232][1157819] Updated weights for policy 0, policy_version 235918 (0.0006) [2024-09-30 04:03:26,739][1157819] Updated weights for policy 0, policy_version 235928 (0.0007) [2024-09-30 04:03:27,253][1157819] Updated weights for policy 0, policy_version 235938 (0.0007) [2024-09-30 04:03:27,770][1157819] Updated weights for policy 0, policy_version 235948 (0.0006) [2024-09-30 04:03:28,305][1157819] Updated weights for policy 0, policy_version 235958 (0.0006) [2024-09-30 04:03:28,838][1157819] Updated weights for policy 0, policy_version 235968 (0.0006) [2024-09-30 04:03:29,353][1157819] Updated weights for policy 0, policy_version 235978 (0.0006) [2024-09-30 04:03:29,873][1157819] Updated weights for policy 0, policy_version 235988 (0.0006) [2024-09-30 04:03:30,404][1157819] Updated weights for policy 0, policy_version 235998 (0.0006) [2024-09-30 04:03:30,466][1157520] Fps is (10 sec: 78644.8, 60 sec: 77755.9, 300 sec: 76241.1). Total num frames: 966651904. Throughput: 0: 19339.6. Samples: 231630156. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 04:03:30,466][1157520] Avg episode reward: [(0, '56.369')] [2024-09-30 04:03:30,924][1157819] Updated weights for policy 0, policy_version 236008 (0.0006) [2024-09-30 04:03:31,414][1157819] Updated weights for policy 0, policy_version 236018 (0.0006) [2024-09-30 04:03:31,887][1157819] Updated weights for policy 0, policy_version 236028 (0.0006) [2024-09-30 04:03:32,392][1157819] Updated weights for policy 0, policy_version 236038 (0.0006) [2024-09-30 04:03:32,906][1157819] Updated weights for policy 0, policy_version 236048 (0.0006) [2024-09-30 04:03:33,399][1157819] Updated weights for policy 0, policy_version 236058 (0.0006) [2024-09-30 04:03:33,868][1157819] Updated weights for policy 0, policy_version 236068 (0.0006) [2024-09-30 04:03:34,369][1157819] Updated weights for policy 0, policy_version 236078 (0.0006) [2024-09-30 04:03:34,897][1157819] Updated weights for policy 0, policy_version 236088 (0.0006) [2024-09-30 04:03:35,379][1157819] Updated weights for policy 0, policy_version 236098 (0.0006) [2024-09-30 04:03:35,466][1157520] Fps is (10 sec: 79871.2, 60 sec: 78370.3, 300 sec: 76074.5). Total num frames: 967057408. Throughput: 0: 19386.5. Samples: 231752212. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 04:03:35,466][1157520] Avg episode reward: [(0, '57.479')] [2024-09-30 04:03:35,890][1157819] Updated weights for policy 0, policy_version 236108 (0.0006) [2024-09-30 04:03:36,394][1157819] Updated weights for policy 0, policy_version 236118 (0.0006) [2024-09-30 04:03:36,943][1157819] Updated weights for policy 0, policy_version 236128 (0.0006) [2024-09-30 04:03:37,461][1157819] Updated weights for policy 0, policy_version 236138 (0.0006) [2024-09-30 04:03:37,966][1157819] Updated weights for policy 0, policy_version 236148 (0.0006) [2024-09-30 04:03:38,297][1157736] Signal inference workers to stop experience collection... (16800 times) [2024-09-30 04:03:38,298][1157736] Signal inference workers to resume experience collection... (16800 times) [2024-09-30 04:03:38,301][1157819] InferenceWorker_p0-w0: stopping experience collection (16800 times) [2024-09-30 04:03:38,303][1157819] InferenceWorker_p0-w0: resuming experience collection (16800 times) [2024-09-30 04:03:38,463][1157819] Updated weights for policy 0, policy_version 236158 (0.0006) [2024-09-30 04:03:38,967][1157819] Updated weights for policy 0, policy_version 236168 (0.0006) [2024-09-30 04:03:39,470][1157819] Updated weights for policy 0, policy_version 236178 (0.0006) [2024-09-30 04:03:40,017][1157819] Updated weights for policy 0, policy_version 236188 (0.0006) [2024-09-30 04:03:40,466][1157520] Fps is (10 sec: 80689.8, 60 sec: 78165.3, 300 sec: 75963.4). Total num frames: 967458816. Throughput: 0: 19409.0. Samples: 231812148. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 04:03:40,466][1157520] Avg episode reward: [(0, '52.357')] [2024-09-30 04:03:40,571][1157819] Updated weights for policy 0, policy_version 236198 (0.0006) [2024-09-30 04:03:41,070][1157819] Updated weights for policy 0, policy_version 236208 (0.0006) [2024-09-30 04:03:41,567][1157819] Updated weights for policy 0, policy_version 236218 (0.0006) [2024-09-30 04:03:42,063][1157819] Updated weights for policy 0, policy_version 236228 (0.0006) [2024-09-30 04:03:42,581][1157819] Updated weights for policy 0, policy_version 236238 (0.0006) [2024-09-30 04:03:43,103][1157819] Updated weights for policy 0, policy_version 236248 (0.0006) [2024-09-30 04:03:43,646][1157819] Updated weights for policy 0, policy_version 236258 (0.0006) [2024-09-30 04:03:44,144][1157819] Updated weights for policy 0, policy_version 236268 (0.0006) [2024-09-30 04:03:44,649][1157819] Updated weights for policy 0, policy_version 236278 (0.0006) [2024-09-30 04:03:45,124][1157819] Updated weights for policy 0, policy_version 236288 (0.0006) [2024-09-30 04:03:45,466][1157520] Fps is (10 sec: 80691.3, 60 sec: 78165.4, 300 sec: 75880.1). Total num frames: 967864320. Throughput: 0: 19563.7. Samples: 231931692. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 04:03:45,466][1157520] Avg episode reward: [(0, '57.136')] [2024-09-30 04:03:45,621][1157819] Updated weights for policy 0, policy_version 236298 (0.0006) [2024-09-30 04:03:46,124][1157819] Updated weights for policy 0, policy_version 236308 (0.0006) [2024-09-30 04:03:46,630][1157819] Updated weights for policy 0, policy_version 236318 (0.0006) [2024-09-30 04:03:47,164][1157819] Updated weights for policy 0, policy_version 236328 (0.0006) [2024-09-30 04:03:47,613][1157819] Updated weights for policy 0, policy_version 236338 (0.0006) [2024-09-30 04:03:48,122][1157819] Updated weights for policy 0, policy_version 236348 (0.0006) [2024-09-30 04:03:48,650][1157819] Updated weights for policy 0, policy_version 236358 (0.0007) [2024-09-30 04:03:49,213][1157819] Updated weights for policy 0, policy_version 236368 (0.0006) [2024-09-30 04:03:49,752][1157819] Updated weights for policy 0, policy_version 236378 (0.0006) [2024-09-30 04:03:50,283][1157819] Updated weights for policy 0, policy_version 236388 (0.0007) [2024-09-30 04:03:50,466][1157520] Fps is (10 sec: 79872.6, 60 sec: 78097.1, 300 sec: 75741.3). Total num frames: 968257536. Throughput: 0: 19753.4. Samples: 232051892. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 04:03:50,466][1157520] Avg episode reward: [(0, '55.553')] [2024-09-30 04:03:50,834][1157819] Updated weights for policy 0, policy_version 236398 (0.0007) [2024-09-30 04:03:51,383][1157819] Updated weights for policy 0, policy_version 236408 (0.0006) [2024-09-30 04:03:51,929][1157819] Updated weights for policy 0, policy_version 236418 (0.0006) [2024-09-30 04:03:52,435][1157819] Updated weights for policy 0, policy_version 236428 (0.0006) [2024-09-30 04:03:52,963][1157819] Updated weights for policy 0, policy_version 236438 (0.0006) [2024-09-30 04:03:53,465][1157819] Updated weights for policy 0, policy_version 236448 (0.0006) [2024-09-30 04:03:53,995][1157819] Updated weights for policy 0, policy_version 236458 (0.0006) [2024-09-30 04:03:54,501][1157819] Updated weights for policy 0, policy_version 236468 (0.0006) [2024-09-30 04:03:55,009][1157819] Updated weights for policy 0, policy_version 236478 (0.0006) [2024-09-30 04:03:55,466][1157520] Fps is (10 sec: 78233.5, 60 sec: 78233.7, 300 sec: 75533.0). Total num frames: 968646656. Throughput: 0: 19789.8. Samples: 232109056. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 04:03:55,466][1157520] Avg episode reward: [(0, '54.165')] [2024-09-30 04:03:55,521][1157819] Updated weights for policy 0, policy_version 236488 (0.0006) [2024-09-30 04:03:56,031][1157819] Updated weights for policy 0, policy_version 236498 (0.0006) [2024-09-30 04:03:56,539][1157819] Updated weights for policy 0, policy_version 236508 (0.0006) [2024-09-30 04:03:57,100][1157819] Updated weights for policy 0, policy_version 236518 (0.0006) [2024-09-30 04:03:57,606][1157819] Updated weights for policy 0, policy_version 236528 (0.0006) [2024-09-30 04:03:58,129][1157819] Updated weights for policy 0, policy_version 236538 (0.0006) [2024-09-30 04:03:58,672][1157819] Updated weights for policy 0, policy_version 236548 (0.0006) [2024-09-30 04:03:59,188][1157819] Updated weights for policy 0, policy_version 236558 (0.0006) [2024-09-30 04:03:59,697][1157819] Updated weights for policy 0, policy_version 236568 (0.0006) [2024-09-30 04:04:00,202][1157819] Updated weights for policy 0, policy_version 236578 (0.0006) [2024-09-30 04:04:00,466][1157520] Fps is (10 sec: 78643.1, 60 sec: 78575.0, 300 sec: 75394.1). Total num frames: 969043968. Throughput: 0: 19830.1. Samples: 232227712. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 04:04:00,466][1157520] Avg episode reward: [(0, '57.206')] [2024-09-30 04:04:00,727][1157819] Updated weights for policy 0, policy_version 236588 (0.0006) [2024-09-30 04:04:01,227][1157819] Updated weights for policy 0, policy_version 236598 (0.0006) [2024-09-30 04:04:01,741][1157819] Updated weights for policy 0, policy_version 236608 (0.0006) [2024-09-30 04:04:02,278][1157819] Updated weights for policy 0, policy_version 236618 (0.0006) [2024-09-30 04:04:02,825][1157819] Updated weights for policy 0, policy_version 236628 (0.0006) [2024-09-30 04:04:03,370][1157819] Updated weights for policy 0, policy_version 236638 (0.0006) [2024-09-30 04:04:03,944][1157819] Updated weights for policy 0, policy_version 236648 (0.0006) [2024-09-30 04:04:04,495][1157819] Updated weights for policy 0, policy_version 236658 (0.0006) [2024-09-30 04:04:04,501][1157736] Signal inference workers to stop experience collection... (16850 times) [2024-09-30 04:04:04,502][1157736] Signal inference workers to resume experience collection... (16850 times) [2024-09-30 04:04:04,506][1157819] InferenceWorker_p0-w0: stopping experience collection (16850 times) [2024-09-30 04:04:04,506][1157819] InferenceWorker_p0-w0: resuming experience collection (16850 times) [2024-09-30 04:04:05,000][1157819] Updated weights for policy 0, policy_version 236668 (0.0006) [2024-09-30 04:04:05,466][1157520] Fps is (10 sec: 77823.7, 60 sec: 78711.5, 300 sec: 75310.8). Total num frames: 969424896. Throughput: 0: 19769.7. Samples: 232343756. Policy #0 lag: (min: 0.0, avg: 2.1, max: 6.0) [2024-09-30 04:04:05,466][1157520] Avg episode reward: [(0, '56.061')] [2024-09-30 04:04:05,472][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000236676_969424896.pth... [2024-09-30 04:04:05,523][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000232264_951353344.pth [2024-09-30 04:04:05,555][1157819] Updated weights for policy 0, policy_version 236678 (0.0006) [2024-09-30 04:04:06,055][1157819] Updated weights for policy 0, policy_version 236688 (0.0006) [2024-09-30 04:04:06,616][1157819] Updated weights for policy 0, policy_version 236698 (0.0006) [2024-09-30 04:04:07,147][1157819] Updated weights for policy 0, policy_version 236708 (0.0006) [2024-09-30 04:04:07,668][1157819] Updated weights for policy 0, policy_version 236718 (0.0006) [2024-09-30 04:04:08,213][1157819] Updated weights for policy 0, policy_version 236728 (0.0006) [2024-09-30 04:04:08,732][1157819] Updated weights for policy 0, policy_version 236738 (0.0006) [2024-09-30 04:04:09,235][1157819] Updated weights for policy 0, policy_version 236748 (0.0006) [2024-09-30 04:04:09,751][1157819] Updated weights for policy 0, policy_version 236758 (0.0006) [2024-09-30 04:04:10,317][1157819] Updated weights for policy 0, policy_version 236768 (0.0006) [2024-09-30 04:04:10,466][1157520] Fps is (10 sec: 77003.6, 60 sec: 78916.1, 300 sec: 75241.4). Total num frames: 969814016. Throughput: 0: 19743.0. Samples: 232401568. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 04:04:10,466][1157520] Avg episode reward: [(0, '55.413')] [2024-09-30 04:04:10,810][1157819] Updated weights for policy 0, policy_version 236778 (0.0006) [2024-09-30 04:04:11,312][1157819] Updated weights for policy 0, policy_version 236788 (0.0006) [2024-09-30 04:04:11,809][1157819] Updated weights for policy 0, policy_version 236798 (0.0006) [2024-09-30 04:04:12,326][1157819] Updated weights for policy 0, policy_version 236808 (0.0006) [2024-09-30 04:04:12,868][1157819] Updated weights for policy 0, policy_version 236818 (0.0006) [2024-09-30 04:04:13,364][1157819] Updated weights for policy 0, policy_version 236828 (0.0006) [2024-09-30 04:04:13,874][1157819] Updated weights for policy 0, policy_version 236838 (0.0006) [2024-09-30 04:04:14,385][1157819] Updated weights for policy 0, policy_version 236848 (0.0006) [2024-09-30 04:04:14,896][1157819] Updated weights for policy 0, policy_version 236858 (0.0006) [2024-09-30 04:04:15,425][1157819] Updated weights for policy 0, policy_version 236868 (0.0006) [2024-09-30 04:04:15,466][1157520] Fps is (10 sec: 79053.3, 60 sec: 79053.0, 300 sec: 75297.0). Total num frames: 970215424. Throughput: 0: 19780.2. Samples: 232520268. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 04:04:15,466][1157520] Avg episode reward: [(0, '55.910')] [2024-09-30 04:04:15,914][1157819] Updated weights for policy 0, policy_version 236878 (0.0006) [2024-09-30 04:04:16,415][1157819] Updated weights for policy 0, policy_version 236888 (0.0006) [2024-09-30 04:04:16,902][1157819] Updated weights for policy 0, policy_version 236898 (0.0006) [2024-09-30 04:04:17,391][1157819] Updated weights for policy 0, policy_version 236908 (0.0006) [2024-09-30 04:04:17,895][1157819] Updated weights for policy 0, policy_version 236918 (0.0006) [2024-09-30 04:04:18,364][1157819] Updated weights for policy 0, policy_version 236928 (0.0006) [2024-09-30 04:04:18,848][1157819] Updated weights for policy 0, policy_version 236938 (0.0006) [2024-09-30 04:04:19,335][1157819] Updated weights for policy 0, policy_version 236948 (0.0006) [2024-09-30 04:04:19,823][1157819] Updated weights for policy 0, policy_version 236958 (0.0006) [2024-09-30 04:04:20,331][1157819] Updated weights for policy 0, policy_version 236968 (0.0006) [2024-09-30 04:04:20,466][1157520] Fps is (10 sec: 81511.8, 60 sec: 79394.3, 300 sec: 75380.3). Total num frames: 970629120. Throughput: 0: 19817.6. Samples: 232644004. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 04:04:20,466][1157520] Avg episode reward: [(0, '55.422')] [2024-09-30 04:04:20,832][1157819] Updated weights for policy 0, policy_version 236978 (0.0006) [2024-09-30 04:04:21,331][1157819] Updated weights for policy 0, policy_version 236988 (0.0006) [2024-09-30 04:04:21,822][1157819] Updated weights for policy 0, policy_version 236998 (0.0006) [2024-09-30 04:04:22,326][1157819] Updated weights for policy 0, policy_version 237008 (0.0006) [2024-09-30 04:04:22,855][1157819] Updated weights for policy 0, policy_version 237018 (0.0006) [2024-09-30 04:04:23,336][1157819] Updated weights for policy 0, policy_version 237028 (0.0006) [2024-09-30 04:04:23,869][1157819] Updated weights for policy 0, policy_version 237038 (0.0006) [2024-09-30 04:04:24,382][1157819] Updated weights for policy 0, policy_version 237048 (0.0006) [2024-09-30 04:04:24,920][1157819] Updated weights for policy 0, policy_version 237058 (0.0006) [2024-09-30 04:04:25,428][1157819] Updated weights for policy 0, policy_version 237068 (0.0006) [2024-09-30 04:04:25,466][1157520] Fps is (10 sec: 81509.2, 60 sec: 79530.4, 300 sec: 75449.7). Total num frames: 971030528. Throughput: 0: 19853.7. Samples: 232705568. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 04:04:25,466][1157520] Avg episode reward: [(0, '55.446')] [2024-09-30 04:04:25,958][1157819] Updated weights for policy 0, policy_version 237078 (0.0006) [2024-09-30 04:04:26,506][1157819] Updated weights for policy 0, policy_version 237088 (0.0006) [2024-09-30 04:04:27,038][1157819] Updated weights for policy 0, policy_version 237098 (0.0006) [2024-09-30 04:04:27,567][1157819] Updated weights for policy 0, policy_version 237108 (0.0006) [2024-09-30 04:04:28,050][1157819] Updated weights for policy 0, policy_version 237118 (0.0006) [2024-09-30 04:04:28,565][1157819] Updated weights for policy 0, policy_version 237128 (0.0006) [2024-09-30 04:04:29,108][1157819] Updated weights for policy 0, policy_version 237138 (0.0006) [2024-09-30 04:04:29,632][1157819] Updated weights for policy 0, policy_version 237148 (0.0006) [2024-09-30 04:04:30,125][1157819] Updated weights for policy 0, policy_version 237158 (0.0006) [2024-09-30 04:04:30,466][1157520] Fps is (10 sec: 79462.4, 60 sec: 79530.5, 300 sec: 75588.5). Total num frames: 971423744. Throughput: 0: 19805.7. Samples: 232822948. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 04:04:30,466][1157520] Avg episode reward: [(0, '57.440')] [2024-09-30 04:04:30,633][1157819] Updated weights for policy 0, policy_version 237168 (0.0006) [2024-09-30 04:04:31,130][1157819] Updated weights for policy 0, policy_version 237178 (0.0006) [2024-09-30 04:04:31,676][1157819] Updated weights for policy 0, policy_version 237188 (0.0006) [2024-09-30 04:04:32,169][1157819] Updated weights for policy 0, policy_version 237198 (0.0006) [2024-09-30 04:04:32,686][1157819] Updated weights for policy 0, policy_version 237208 (0.0006) [2024-09-30 04:04:33,217][1157819] Updated weights for policy 0, policy_version 237218 (0.0006) [2024-09-30 04:04:33,763][1157819] Updated weights for policy 0, policy_version 237228 (0.0006) [2024-09-30 04:04:34,271][1157819] Updated weights for policy 0, policy_version 237238 (0.0006) [2024-09-30 04:04:34,779][1157819] Updated weights for policy 0, policy_version 237248 (0.0006) [2024-09-30 04:04:35,311][1157819] Updated weights for policy 0, policy_version 237258 (0.0006) [2024-09-30 04:04:35,466][1157520] Fps is (10 sec: 79053.6, 60 sec: 79394.1, 300 sec: 75741.3). Total num frames: 971821056. Throughput: 0: 19783.4. Samples: 232942148. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 04:04:35,466][1157520] Avg episode reward: [(0, '56.370')] [2024-09-30 04:04:35,818][1157819] Updated weights for policy 0, policy_version 237268 (0.0006) [2024-09-30 04:04:36,327][1157819] Updated weights for policy 0, policy_version 237278 (0.0006) [2024-09-30 04:04:36,827][1157819] Updated weights for policy 0, policy_version 237288 (0.0006) [2024-09-30 04:04:37,336][1157819] Updated weights for policy 0, policy_version 237298 (0.0006) [2024-09-30 04:04:37,844][1157819] Updated weights for policy 0, policy_version 237308 (0.0006) [2024-09-30 04:04:38,342][1157819] Updated weights for policy 0, policy_version 237318 (0.0006) [2024-09-30 04:04:38,847][1157819] Updated weights for policy 0, policy_version 237328 (0.0006) [2024-09-30 04:04:39,336][1157819] Updated weights for policy 0, policy_version 237338 (0.0006) [2024-09-30 04:04:39,858][1157819] Updated weights for policy 0, policy_version 237348 (0.0006) [2024-09-30 04:04:40,355][1157819] Updated weights for policy 0, policy_version 237358 (0.0006) [2024-09-30 04:04:40,358][1157736] Signal inference workers to stop experience collection... (16900 times) [2024-09-30 04:04:40,359][1157736] Signal inference workers to resume experience collection... (16900 times) [2024-09-30 04:04:40,364][1157819] InferenceWorker_p0-w0: stopping experience collection (16900 times) [2024-09-30 04:04:40,364][1157819] InferenceWorker_p0-w0: resuming experience collection (16900 times) [2024-09-30 04:04:40,466][1157520] Fps is (10 sec: 80281.9, 60 sec: 79462.5, 300 sec: 75921.8). Total num frames: 972226560. Throughput: 0: 19842.3. Samples: 233001960. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 04:04:40,466][1157520] Avg episode reward: [(0, '54.579')] [2024-09-30 04:04:40,827][1157819] Updated weights for policy 0, policy_version 237368 (0.0006) [2024-09-30 04:04:41,354][1157819] Updated weights for policy 0, policy_version 237378 (0.0006) [2024-09-30 04:04:41,850][1157819] Updated weights for policy 0, policy_version 237388 (0.0006) [2024-09-30 04:04:42,334][1157819] Updated weights for policy 0, policy_version 237398 (0.0006) [2024-09-30 04:04:42,835][1157819] Updated weights for policy 0, policy_version 237408 (0.0006) [2024-09-30 04:04:43,382][1157819] Updated weights for policy 0, policy_version 237418 (0.0006) [2024-09-30 04:04:43,903][1157819] Updated weights for policy 0, policy_version 237428 (0.0006) [2024-09-30 04:04:44,415][1157819] Updated weights for policy 0, policy_version 237438 (0.0006) [2024-09-30 04:04:44,928][1157819] Updated weights for policy 0, policy_version 237448 (0.0006) [2024-09-30 04:04:45,466][1157520] Fps is (10 sec: 80282.4, 60 sec: 79326.0, 300 sec: 76005.1). Total num frames: 972623872. Throughput: 0: 19918.4. Samples: 233124040. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 04:04:45,466][1157520] Avg episode reward: [(0, '57.369')] [2024-09-30 04:04:45,480][1157819] Updated weights for policy 0, policy_version 237458 (0.0006) [2024-09-30 04:04:45,970][1157819] Updated weights for policy 0, policy_version 237468 (0.0006) [2024-09-30 04:04:46,463][1157819] Updated weights for policy 0, policy_version 237478 (0.0006) [2024-09-30 04:04:46,956][1157819] Updated weights for policy 0, policy_version 237488 (0.0006) [2024-09-30 04:04:47,452][1157819] Updated weights for policy 0, policy_version 237498 (0.0006) [2024-09-30 04:04:47,936][1157819] Updated weights for policy 0, policy_version 237508 (0.0006) [2024-09-30 04:04:48,455][1157819] Updated weights for policy 0, policy_version 237518 (0.0006) [2024-09-30 04:04:48,976][1157819] Updated weights for policy 0, policy_version 237528 (0.0006) [2024-09-30 04:04:49,511][1157819] Updated weights for policy 0, policy_version 237538 (0.0006) [2024-09-30 04:04:49,998][1157819] Updated weights for policy 0, policy_version 237548 (0.0006) [2024-09-30 04:04:50,466][1157520] Fps is (10 sec: 80691.3, 60 sec: 79599.0, 300 sec: 76130.1). Total num frames: 973033472. Throughput: 0: 20029.6. Samples: 233245084. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 04:04:50,466][1157520] Avg episode reward: [(0, '56.479')] [2024-09-30 04:04:50,538][1157819] Updated weights for policy 0, policy_version 237558 (0.0006) [2024-09-30 04:04:51,041][1157819] Updated weights for policy 0, policy_version 237568 (0.0006) [2024-09-30 04:04:51,470][1157819] Updated weights for policy 0, policy_version 237578 (0.0006) [2024-09-30 04:04:51,992][1157819] Updated weights for policy 0, policy_version 237588 (0.0006) [2024-09-30 04:04:52,522][1157819] Updated weights for policy 0, policy_version 237598 (0.0006) [2024-09-30 04:04:53,021][1157819] Updated weights for policy 0, policy_version 237608 (0.0006) [2024-09-30 04:04:53,527][1157819] Updated weights for policy 0, policy_version 237618 (0.0006) [2024-09-30 04:04:54,071][1157819] Updated weights for policy 0, policy_version 237628 (0.0006) [2024-09-30 04:04:54,582][1157819] Updated weights for policy 0, policy_version 237638 (0.0006) [2024-09-30 04:04:55,087][1157819] Updated weights for policy 0, policy_version 237648 (0.0006) [2024-09-30 04:04:55,466][1157520] Fps is (10 sec: 81100.9, 60 sec: 79803.9, 300 sec: 76171.7). Total num frames: 973434880. Throughput: 0: 20100.0. Samples: 233306064. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 04:04:55,466][1157520] Avg episode reward: [(0, '58.114')] [2024-09-30 04:04:55,583][1157819] Updated weights for policy 0, policy_version 237658 (0.0006) [2024-09-30 04:04:56,107][1157819] Updated weights for policy 0, policy_version 237668 (0.0006) [2024-09-30 04:04:56,629][1157819] Updated weights for policy 0, policy_version 237678 (0.0006) [2024-09-30 04:04:57,147][1157819] Updated weights for policy 0, policy_version 237688 (0.0006) [2024-09-30 04:04:57,693][1157819] Updated weights for policy 0, policy_version 237698 (0.0006) [2024-09-30 04:04:58,158][1157819] Updated weights for policy 0, policy_version 237708 (0.0006) [2024-09-30 04:04:58,653][1157819] Updated weights for policy 0, policy_version 237718 (0.0006) [2024-09-30 04:04:59,118][1157819] Updated weights for policy 0, policy_version 237728 (0.0006) [2024-09-30 04:04:59,616][1157819] Updated weights for policy 0, policy_version 237738 (0.0006) [2024-09-30 04:05:00,123][1157819] Updated weights for policy 0, policy_version 237748 (0.0006) [2024-09-30 04:05:00,466][1157520] Fps is (10 sec: 81101.0, 60 sec: 80008.6, 300 sec: 76241.2). Total num frames: 973844480. Throughput: 0: 20126.4. Samples: 233425956. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 04:05:00,466][1157520] Avg episode reward: [(0, '56.277')] [2024-09-30 04:05:00,657][1157819] Updated weights for policy 0, policy_version 237758 (0.0006) [2024-09-30 04:05:01,161][1157819] Updated weights for policy 0, policy_version 237768 (0.0006) [2024-09-30 04:05:01,618][1157819] Updated weights for policy 0, policy_version 237778 (0.0006) [2024-09-30 04:05:02,125][1157819] Updated weights for policy 0, policy_version 237788 (0.0006) [2024-09-30 04:05:02,649][1157819] Updated weights for policy 0, policy_version 237798 (0.0006) [2024-09-30 04:05:03,163][1157819] Updated weights for policy 0, policy_version 237808 (0.0006) [2024-09-30 04:05:03,666][1157819] Updated weights for policy 0, policy_version 237818 (0.0006) [2024-09-30 04:05:04,162][1157819] Updated weights for policy 0, policy_version 237828 (0.0006) [2024-09-30 04:05:04,663][1157819] Updated weights for policy 0, policy_version 237838 (0.0006) [2024-09-30 04:05:05,186][1157819] Updated weights for policy 0, policy_version 237848 (0.0006) [2024-09-30 04:05:05,466][1157520] Fps is (10 sec: 81099.9, 60 sec: 80349.9, 300 sec: 76352.2). Total num frames: 974245888. Throughput: 0: 20090.6. Samples: 233548080. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 04:05:05,466][1157520] Avg episode reward: [(0, '57.568')] [2024-09-30 04:05:05,719][1157819] Updated weights for policy 0, policy_version 237858 (0.0006) [2024-09-30 04:05:06,234][1157819] Updated weights for policy 0, policy_version 237868 (0.0006) [2024-09-30 04:05:06,730][1157819] Updated weights for policy 0, policy_version 237878 (0.0006) [2024-09-30 04:05:07,261][1157819] Updated weights for policy 0, policy_version 237888 (0.0006) [2024-09-30 04:05:07,759][1157819] Updated weights for policy 0, policy_version 237898 (0.0006) [2024-09-30 04:05:08,286][1157819] Updated weights for policy 0, policy_version 237908 (0.0006) [2024-09-30 04:05:08,797][1157819] Updated weights for policy 0, policy_version 237918 (0.0006) [2024-09-30 04:05:09,310][1157819] Updated weights for policy 0, policy_version 237928 (0.0006) [2024-09-30 04:05:09,888][1157819] Updated weights for policy 0, policy_version 237938 (0.0006) [2024-09-30 04:05:10,411][1157819] Updated weights for policy 0, policy_version 237948 (0.0006) [2024-09-30 04:05:10,466][1157520] Fps is (10 sec: 79461.7, 60 sec: 80418.3, 300 sec: 76380.0). Total num frames: 974639104. Throughput: 0: 20050.0. Samples: 233607816. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 04:05:10,466][1157520] Avg episode reward: [(0, '57.332')] [2024-09-30 04:05:10,960][1157819] Updated weights for policy 0, policy_version 237958 (0.0006) [2024-09-30 04:05:11,524][1157819] Updated weights for policy 0, policy_version 237968 (0.0006) [2024-09-30 04:05:12,026][1157819] Updated weights for policy 0, policy_version 237978 (0.0006) [2024-09-30 04:05:12,586][1157819] Updated weights for policy 0, policy_version 237988 (0.0006) [2024-09-30 04:05:13,143][1157819] Updated weights for policy 0, policy_version 237998 (0.0006) [2024-09-30 04:05:13,696][1157819] Updated weights for policy 0, policy_version 238008 (0.0006) [2024-09-30 04:05:14,271][1157819] Updated weights for policy 0, policy_version 238018 (0.0006) [2024-09-30 04:05:14,770][1157819] Updated weights for policy 0, policy_version 238028 (0.0006) [2024-09-30 04:05:15,294][1157819] Updated weights for policy 0, policy_version 238038 (0.0006) [2024-09-30 04:05:15,466][1157520] Fps is (10 sec: 76595.2, 60 sec: 79940.2, 300 sec: 76324.4). Total num frames: 975011840. Throughput: 0: 19981.9. Samples: 233722132. Policy #0 lag: (min: 0.0, avg: 3.0, max: 7.0) [2024-09-30 04:05:15,466][1157520] Avg episode reward: [(0, '57.604')] [2024-09-30 04:05:15,858][1157819] Updated weights for policy 0, policy_version 238048 (0.0006) [2024-09-30 04:05:16,396][1157819] Updated weights for policy 0, policy_version 238058 (0.0006) [2024-09-30 04:05:16,962][1157819] Updated weights for policy 0, policy_version 238068 (0.0006) [2024-09-30 04:05:17,494][1157819] Updated weights for policy 0, policy_version 238078 (0.0006) [2024-09-30 04:05:18,043][1157819] Updated weights for policy 0, policy_version 238088 (0.0006) [2024-09-30 04:05:18,597][1157819] Updated weights for policy 0, policy_version 238098 (0.0006) [2024-09-30 04:05:19,142][1157819] Updated weights for policy 0, policy_version 238108 (0.0006) [2024-09-30 04:05:19,658][1157819] Updated weights for policy 0, policy_version 238118 (0.0006) [2024-09-30 04:05:20,206][1157819] Updated weights for policy 0, policy_version 238128 (0.0006) [2024-09-30 04:05:20,466][1157520] Fps is (10 sec: 75366.3, 60 sec: 79394.1, 300 sec: 76366.1). Total num frames: 975392768. Throughput: 0: 19839.8. Samples: 233834940. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 04:05:20,466][1157520] Avg episode reward: [(0, '55.257')] [2024-09-30 04:05:20,757][1157819] Updated weights for policy 0, policy_version 238138 (0.0006) [2024-09-30 04:05:21,276][1157819] Updated weights for policy 0, policy_version 238148 (0.0006) [2024-09-30 04:05:21,876][1157819] Updated weights for policy 0, policy_version 238158 (0.0006) [2024-09-30 04:05:22,382][1157819] Updated weights for policy 0, policy_version 238168 (0.0006) [2024-09-30 04:05:22,909][1157819] Updated weights for policy 0, policy_version 238178 (0.0006) [2024-09-30 04:05:23,473][1157819] Updated weights for policy 0, policy_version 238188 (0.0006) [2024-09-30 04:05:24,021][1157819] Updated weights for policy 0, policy_version 238198 (0.0006) [2024-09-30 04:05:24,519][1157819] Updated weights for policy 0, policy_version 238208 (0.0006) [2024-09-30 04:05:25,030][1157819] Updated weights for policy 0, policy_version 238218 (0.0006) [2024-09-30 04:05:25,466][1157520] Fps is (10 sec: 75776.2, 60 sec: 78984.7, 300 sec: 76407.8). Total num frames: 975769600. Throughput: 0: 19769.6. Samples: 233891592. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 04:05:25,466][1157520] Avg episode reward: [(0, '53.688')] [2024-09-30 04:05:25,570][1157819] Updated weights for policy 0, policy_version 238228 (0.0006) [2024-09-30 04:05:26,106][1157819] Updated weights for policy 0, policy_version 238238 (0.0006) [2024-09-30 04:05:26,635][1157819] Updated weights for policy 0, policy_version 238248 (0.0006) [2024-09-30 04:05:27,175][1157819] Updated weights for policy 0, policy_version 238258 (0.0006) [2024-09-30 04:05:27,725][1157819] Updated weights for policy 0, policy_version 238268 (0.0006) [2024-09-30 04:05:28,264][1157819] Updated weights for policy 0, policy_version 238278 (0.0006) [2024-09-30 04:05:28,776][1157819] Updated weights for policy 0, policy_version 238288 (0.0006) [2024-09-30 04:05:29,032][1157736] Signal inference workers to stop experience collection... (16950 times) [2024-09-30 04:05:29,033][1157736] Signal inference workers to resume experience collection... (16950 times) [2024-09-30 04:05:29,036][1157819] InferenceWorker_p0-w0: stopping experience collection (16950 times) [2024-09-30 04:05:29,039][1157819] InferenceWorker_p0-w0: resuming experience collection (16950 times) [2024-09-30 04:05:29,311][1157819] Updated weights for policy 0, policy_version 238298 (0.0006) [2024-09-30 04:05:29,836][1157819] Updated weights for policy 0, policy_version 238308 (0.0006) [2024-09-30 04:05:30,395][1157819] Updated weights for policy 0, policy_version 238318 (0.0006) [2024-09-30 04:05:30,466][1157520] Fps is (10 sec: 76185.8, 60 sec: 78848.0, 300 sec: 76518.9). Total num frames: 976154624. Throughput: 0: 19620.4. Samples: 234006960. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 04:05:30,466][1157520] Avg episode reward: [(0, '55.572')] [2024-09-30 04:05:30,988][1157819] Updated weights for policy 0, policy_version 238328 (0.0006) [2024-09-30 04:05:31,544][1157819] Updated weights for policy 0, policy_version 238338 (0.0006) [2024-09-30 04:05:32,079][1157819] Updated weights for policy 0, policy_version 238348 (0.0006) [2024-09-30 04:05:32,694][1157819] Updated weights for policy 0, policy_version 238358 (0.0006) [2024-09-30 04:05:33,273][1157819] Updated weights for policy 0, policy_version 238368 (0.0006) [2024-09-30 04:05:33,854][1157819] Updated weights for policy 0, policy_version 238378 (0.0006) [2024-09-30 04:05:34,454][1157819] Updated weights for policy 0, policy_version 238388 (0.0006) [2024-09-30 04:05:35,095][1157819] Updated weights for policy 0, policy_version 238398 (0.0006) [2024-09-30 04:05:35,466][1157520] Fps is (10 sec: 73318.7, 60 sec: 78028.9, 300 sec: 76505.0). Total num frames: 976502784. Throughput: 0: 19324.9. Samples: 234114704. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 04:05:35,466][1157520] Avg episode reward: [(0, '55.514')] [2024-09-30 04:05:35,715][1157819] Updated weights for policy 0, policy_version 238408 (0.0006) [2024-09-30 04:05:36,326][1157819] Updated weights for policy 0, policy_version 238418 (0.0006) [2024-09-30 04:05:36,998][1157819] Updated weights for policy 0, policy_version 238428 (0.0006) [2024-09-30 04:05:37,609][1157819] Updated weights for policy 0, policy_version 238438 (0.0006) [2024-09-30 04:05:38,178][1157819] Updated weights for policy 0, policy_version 238448 (0.0006) [2024-09-30 04:05:38,828][1157819] Updated weights for policy 0, policy_version 238458 (0.0006) [2024-09-30 04:05:39,433][1157819] Updated weights for policy 0, policy_version 238468 (0.0006) [2024-09-30 04:05:40,040][1157819] Updated weights for policy 0, policy_version 238478 (0.0006) [2024-09-30 04:05:40,466][1157520] Fps is (10 sec: 67583.5, 60 sec: 76731.6, 300 sec: 76393.9). Total num frames: 976830464. Throughput: 0: 19061.4. Samples: 234163828. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 04:05:40,466][1157520] Avg episode reward: [(0, '57.033')] [2024-09-30 04:05:40,699][1157819] Updated weights for policy 0, policy_version 238488 (0.0006) [2024-09-30 04:05:41,286][1157819] Updated weights for policy 0, policy_version 238498 (0.0006) [2024-09-30 04:05:41,869][1157819] Updated weights for policy 0, policy_version 238508 (0.0006) [2024-09-30 04:05:42,446][1157819] Updated weights for policy 0, policy_version 238518 (0.0006) [2024-09-30 04:05:43,044][1157819] Updated weights for policy 0, policy_version 238528 (0.0006) [2024-09-30 04:05:43,641][1157819] Updated weights for policy 0, policy_version 238538 (0.0006) [2024-09-30 04:05:44,213][1157819] Updated weights for policy 0, policy_version 238548 (0.0006) [2024-09-30 04:05:44,799][1157819] Updated weights for policy 0, policy_version 238558 (0.0006) [2024-09-30 04:05:45,397][1157819] Updated weights for policy 0, policy_version 238568 (0.0006) [2024-09-30 04:05:45,466][1157520] Fps is (10 sec: 67583.7, 60 sec: 75912.4, 300 sec: 76310.6). Total num frames: 977178624. Throughput: 0: 18666.9. Samples: 234265968. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 04:05:45,466][1157520] Avg episode reward: [(0, '55.757')] [2024-09-30 04:05:45,974][1157819] Updated weights for policy 0, policy_version 238578 (0.0006) [2024-09-30 04:05:46,560][1157819] Updated weights for policy 0, policy_version 238588 (0.0006) [2024-09-30 04:05:47,131][1157819] Updated weights for policy 0, policy_version 238598 (0.0006) [2024-09-30 04:05:47,676][1157819] Updated weights for policy 0, policy_version 238608 (0.0006) [2024-09-30 04:05:48,214][1157819] Updated weights for policy 0, policy_version 238618 (0.0006) [2024-09-30 04:05:48,772][1157819] Updated weights for policy 0, policy_version 238628 (0.0006) [2024-09-30 04:05:49,304][1157819] Updated weights for policy 0, policy_version 238638 (0.0006) [2024-09-30 04:05:49,858][1157819] Updated weights for policy 0, policy_version 238648 (0.0006) [2024-09-30 04:05:50,427][1157819] Updated weights for policy 0, policy_version 238658 (0.0006) [2024-09-30 04:05:50,466][1157520] Fps is (10 sec: 71270.8, 60 sec: 75161.5, 300 sec: 76296.7). Total num frames: 977543168. Throughput: 0: 18350.3. Samples: 234373844. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 04:05:50,466][1157520] Avg episode reward: [(0, '56.986')] [2024-09-30 04:05:50,936][1157819] Updated weights for policy 0, policy_version 238668 (0.0006) [2024-09-30 04:05:51,546][1157819] Updated weights for policy 0, policy_version 238678 (0.0006) [2024-09-30 04:05:52,041][1157819] Updated weights for policy 0, policy_version 238688 (0.0006) [2024-09-30 04:05:52,581][1157819] Updated weights for policy 0, policy_version 238698 (0.0006) [2024-09-30 04:05:52,660][1157736] Signal inference workers to stop experience collection... (17000 times) [2024-09-30 04:05:52,661][1157736] Signal inference workers to resume experience collection... (17000 times) [2024-09-30 04:05:52,664][1157819] InferenceWorker_p0-w0: stopping experience collection (17000 times) [2024-09-30 04:05:52,664][1157819] InferenceWorker_p0-w0: resuming experience collection (17000 times) [2024-09-30 04:05:53,133][1157819] Updated weights for policy 0, policy_version 238708 (0.0006) [2024-09-30 04:05:53,675][1157819] Updated weights for policy 0, policy_version 238718 (0.0006) [2024-09-30 04:05:54,213][1157819] Updated weights for policy 0, policy_version 238728 (0.0006) [2024-09-30 04:05:54,745][1157819] Updated weights for policy 0, policy_version 238738 (0.0006) [2024-09-30 04:05:55,269][1157819] Updated weights for policy 0, policy_version 238748 (0.0006) [2024-09-30 04:05:55,466][1157520] Fps is (10 sec: 74137.5, 60 sec: 74751.9, 300 sec: 76296.7). Total num frames: 977920000. Throughput: 0: 18286.6. Samples: 234430712. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 04:05:55,466][1157520] Avg episode reward: [(0, '54.770')] [2024-09-30 04:05:55,828][1157819] Updated weights for policy 0, policy_version 238758 (0.0006) [2024-09-30 04:05:56,347][1157819] Updated weights for policy 0, policy_version 238768 (0.0006) [2024-09-30 04:05:56,914][1157819] Updated weights for policy 0, policy_version 238778 (0.0006) [2024-09-30 04:05:57,464][1157819] Updated weights for policy 0, policy_version 238788 (0.0006) [2024-09-30 04:05:57,959][1157819] Updated weights for policy 0, policy_version 238798 (0.0006) [2024-09-30 04:05:58,486][1157819] Updated weights for policy 0, policy_version 238808 (0.0006) [2024-09-30 04:05:59,075][1157819] Updated weights for policy 0, policy_version 238818 (0.0006) [2024-09-30 04:05:59,578][1157819] Updated weights for policy 0, policy_version 238828 (0.0006) [2024-09-30 04:06:00,121][1157819] Updated weights for policy 0, policy_version 238838 (0.0006) [2024-09-30 04:06:00,466][1157520] Fps is (10 sec: 75775.9, 60 sec: 74274.0, 300 sec: 76380.0). Total num frames: 978300928. Throughput: 0: 18286.6. Samples: 234545028. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 04:06:00,466][1157520] Avg episode reward: [(0, '57.504')] [2024-09-30 04:06:00,667][1157819] Updated weights for policy 0, policy_version 238848 (0.0006) [2024-09-30 04:06:01,185][1157819] Updated weights for policy 0, policy_version 238858 (0.0006) [2024-09-30 04:06:01,702][1157819] Updated weights for policy 0, policy_version 238868 (0.0006) [2024-09-30 04:06:02,222][1157819] Updated weights for policy 0, policy_version 238878 (0.0006) [2024-09-30 04:06:02,752][1157819] Updated weights for policy 0, policy_version 238888 (0.0006) [2024-09-30 04:06:03,253][1157819] Updated weights for policy 0, policy_version 238898 (0.0006) [2024-09-30 04:06:03,786][1157819] Updated weights for policy 0, policy_version 238908 (0.0006) [2024-09-30 04:06:04,293][1157819] Updated weights for policy 0, policy_version 238918 (0.0006) [2024-09-30 04:06:04,786][1157819] Updated weights for policy 0, policy_version 238928 (0.0006) [2024-09-30 04:06:05,316][1157819] Updated weights for policy 0, policy_version 238938 (0.0006) [2024-09-30 04:06:05,466][1157520] Fps is (10 sec: 77824.0, 60 sec: 74205.9, 300 sec: 76532.7). Total num frames: 978698240. Throughput: 0: 18379.7. Samples: 234662028. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 04:06:05,466][1157520] Avg episode reward: [(0, '56.476')] [2024-09-30 04:06:05,483][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000238941_978702336.pth... [2024-09-30 04:06:05,537][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000234407_960131072.pth [2024-09-30 04:06:05,839][1157819] Updated weights for policy 0, policy_version 238948 (0.0006) [2024-09-30 04:06:06,375][1157819] Updated weights for policy 0, policy_version 238958 (0.0006) [2024-09-30 04:06:06,864][1157819] Updated weights for policy 0, policy_version 238968 (0.0006) [2024-09-30 04:06:07,378][1157819] Updated weights for policy 0, policy_version 238978 (0.0006) [2024-09-30 04:06:07,885][1157819] Updated weights for policy 0, policy_version 238988 (0.0006) [2024-09-30 04:06:08,406][1157819] Updated weights for policy 0, policy_version 238998 (0.0006) [2024-09-30 04:06:08,905][1157819] Updated weights for policy 0, policy_version 239008 (0.0006) [2024-09-30 04:06:09,478][1157819] Updated weights for policy 0, policy_version 239018 (0.0006) [2024-09-30 04:06:09,983][1157819] Updated weights for policy 0, policy_version 239028 (0.0006) [2024-09-30 04:06:10,466][1157520] Fps is (10 sec: 79462.4, 60 sec: 74274.1, 300 sec: 76713.2). Total num frames: 979095552. Throughput: 0: 18449.9. Samples: 234721836. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 04:06:10,466][1157520] Avg episode reward: [(0, '56.437')] [2024-09-30 04:06:10,518][1157819] Updated weights for policy 0, policy_version 239038 (0.0006) [2024-09-30 04:06:11,025][1157819] Updated weights for policy 0, policy_version 239048 (0.0006) [2024-09-30 04:06:11,541][1157819] Updated weights for policy 0, policy_version 239058 (0.0006) [2024-09-30 04:06:12,043][1157819] Updated weights for policy 0, policy_version 239068 (0.0006) [2024-09-30 04:06:12,545][1157819] Updated weights for policy 0, policy_version 239078 (0.0006) [2024-09-30 04:06:13,049][1157819] Updated weights for policy 0, policy_version 239088 (0.0006) [2024-09-30 04:06:13,574][1157819] Updated weights for policy 0, policy_version 239098 (0.0006) [2024-09-30 04:06:14,070][1157819] Updated weights for policy 0, policy_version 239108 (0.0006) [2024-09-30 04:06:14,575][1157819] Updated weights for policy 0, policy_version 239118 (0.0006) [2024-09-30 04:06:15,104][1157819] Updated weights for policy 0, policy_version 239128 (0.0006) [2024-09-30 04:06:15,466][1157520] Fps is (10 sec: 79872.4, 60 sec: 74752.1, 300 sec: 76810.4). Total num frames: 979496960. Throughput: 0: 18535.2. Samples: 234841044. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 04:06:15,466][1157520] Avg episode reward: [(0, '55.572')] [2024-09-30 04:06:15,630][1157819] Updated weights for policy 0, policy_version 239138 (0.0006) [2024-09-30 04:06:16,110][1157819] Updated weights for policy 0, policy_version 239148 (0.0006) [2024-09-30 04:06:16,616][1157819] Updated weights for policy 0, policy_version 239158 (0.0006) [2024-09-30 04:06:17,086][1157819] Updated weights for policy 0, policy_version 239168 (0.0006) [2024-09-30 04:06:17,582][1157819] Updated weights for policy 0, policy_version 239178 (0.0006) [2024-09-30 04:06:18,081][1157819] Updated weights for policy 0, policy_version 239188 (0.0006) [2024-09-30 04:06:18,556][1157819] Updated weights for policy 0, policy_version 239198 (0.0006) [2024-09-30 04:06:19,067][1157819] Updated weights for policy 0, policy_version 239208 (0.0006) [2024-09-30 04:06:19,556][1157819] Updated weights for policy 0, policy_version 239218 (0.0006) [2024-09-30 04:06:20,026][1157819] Updated weights for policy 0, policy_version 239228 (0.0006) [2024-09-30 04:06:20,466][1157520] Fps is (10 sec: 81510.9, 60 sec: 75298.2, 300 sec: 76935.4). Total num frames: 979910656. Throughput: 0: 18872.9. Samples: 234963984. Policy #0 lag: (min: 0.0, avg: 2.5, max: 6.0) [2024-09-30 04:06:20,466][1157520] Avg episode reward: [(0, '56.264')] [2024-09-30 04:06:20,558][1157819] Updated weights for policy 0, policy_version 239238 (0.0006) [2024-09-30 04:06:20,599][1157736] Signal inference workers to stop experience collection... (17050 times) [2024-09-30 04:06:20,601][1157736] Signal inference workers to resume experience collection... (17050 times) [2024-09-30 04:06:20,605][1157819] InferenceWorker_p0-w0: stopping experience collection (17050 times) [2024-09-30 04:06:20,608][1157819] InferenceWorker_p0-w0: resuming experience collection (17050 times) [2024-09-30 04:06:21,048][1157819] Updated weights for policy 0, policy_version 239248 (0.0006) [2024-09-30 04:06:21,494][1157819] Updated weights for policy 0, policy_version 239258 (0.0006) [2024-09-30 04:06:22,053][1157819] Updated weights for policy 0, policy_version 239268 (0.0006) [2024-09-30 04:06:22,559][1157819] Updated weights for policy 0, policy_version 239278 (0.0006) [2024-09-30 04:06:23,066][1157819] Updated weights for policy 0, policy_version 239288 (0.0006) [2024-09-30 04:06:23,588][1157819] Updated weights for policy 0, policy_version 239298 (0.0006) [2024-09-30 04:06:24,140][1157819] Updated weights for policy 0, policy_version 239308 (0.0006) [2024-09-30 04:06:24,703][1157819] Updated weights for policy 0, policy_version 239318 (0.0006) [2024-09-30 04:06:25,240][1157819] Updated weights for policy 0, policy_version 239328 (0.0006) [2024-09-30 04:06:25,466][1157520] Fps is (10 sec: 80691.5, 60 sec: 75571.3, 300 sec: 77060.4). Total num frames: 980303872. Throughput: 0: 19148.0. Samples: 235025484. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 04:06:25,466][1157520] Avg episode reward: [(0, '58.297')] [2024-09-30 04:06:25,785][1157819] Updated weights for policy 0, policy_version 239338 (0.0006) [2024-09-30 04:06:26,319][1157819] Updated weights for policy 0, policy_version 239348 (0.0006) [2024-09-30 04:06:26,856][1157819] Updated weights for policy 0, policy_version 239358 (0.0006) [2024-09-30 04:06:27,385][1157819] Updated weights for policy 0, policy_version 239368 (0.0006) [2024-09-30 04:06:27,921][1157819] Updated weights for policy 0, policy_version 239378 (0.0006) [2024-09-30 04:06:28,469][1157819] Updated weights for policy 0, policy_version 239388 (0.0006) [2024-09-30 04:06:28,985][1157819] Updated weights for policy 0, policy_version 239398 (0.0006) [2024-09-30 04:06:29,554][1157819] Updated weights for policy 0, policy_version 239408 (0.0006) [2024-09-30 04:06:30,101][1157819] Updated weights for policy 0, policy_version 239418 (0.0006) [2024-09-30 04:06:30,466][1157520] Fps is (10 sec: 77414.8, 60 sec: 75503.1, 300 sec: 77129.8). Total num frames: 980684800. Throughput: 0: 19406.4. Samples: 235139256. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 04:06:30,466][1157520] Avg episode reward: [(0, '57.251')] [2024-09-30 04:06:30,615][1157819] Updated weights for policy 0, policy_version 239428 (0.0006) [2024-09-30 04:06:31,173][1157819] Updated weights for policy 0, policy_version 239438 (0.0006) [2024-09-30 04:06:31,698][1157819] Updated weights for policy 0, policy_version 239448 (0.0006) [2024-09-30 04:06:32,231][1157819] Updated weights for policy 0, policy_version 239458 (0.0006) [2024-09-30 04:06:32,776][1157819] Updated weights for policy 0, policy_version 239468 (0.0006) [2024-09-30 04:06:33,297][1157819] Updated weights for policy 0, policy_version 239478 (0.0006) [2024-09-30 04:06:33,841][1157819] Updated weights for policy 0, policy_version 239488 (0.0006) [2024-09-30 04:06:34,390][1157819] Updated weights for policy 0, policy_version 239498 (0.0006) [2024-09-30 04:06:34,927][1157819] Updated weights for policy 0, policy_version 239508 (0.0006) [2024-09-30 04:06:35,466][1157520] Fps is (10 sec: 75776.6, 60 sec: 75980.9, 300 sec: 77129.8). Total num frames: 981061632. Throughput: 0: 19551.9. Samples: 235253676. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 04:06:35,466][1157520] Avg episode reward: [(0, '54.918')] [2024-09-30 04:06:35,498][1157819] Updated weights for policy 0, policy_version 239518 (0.0006) [2024-09-30 04:06:36,029][1157819] Updated weights for policy 0, policy_version 239528 (0.0006) [2024-09-30 04:06:36,552][1157819] Updated weights for policy 0, policy_version 239538 (0.0006) [2024-09-30 04:06:37,080][1157819] Updated weights for policy 0, policy_version 239548 (0.0006) [2024-09-30 04:06:37,615][1157819] Updated weights for policy 0, policy_version 239558 (0.0006) [2024-09-30 04:06:38,141][1157819] Updated weights for policy 0, policy_version 239568 (0.0006) [2024-09-30 04:06:38,692][1157819] Updated weights for policy 0, policy_version 239578 (0.0006) [2024-09-30 04:06:39,256][1157819] Updated weights for policy 0, policy_version 239588 (0.0006) [2024-09-30 04:06:39,806][1157819] Updated weights for policy 0, policy_version 239598 (0.0006) [2024-09-30 04:06:40,348][1157819] Updated weights for policy 0, policy_version 239608 (0.0006) [2024-09-30 04:06:40,466][1157520] Fps is (10 sec: 75776.1, 60 sec: 76868.5, 300 sec: 77157.6). Total num frames: 981442560. Throughput: 0: 19552.3. Samples: 235310564. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 04:06:40,466][1157520] Avg episode reward: [(0, '57.786')] [2024-09-30 04:06:40,901][1157819] Updated weights for policy 0, policy_version 239618 (0.0006) [2024-09-30 04:06:41,425][1157819] Updated weights for policy 0, policy_version 239628 (0.0006) [2024-09-30 04:06:42,003][1157819] Updated weights for policy 0, policy_version 239638 (0.0006) [2024-09-30 04:06:42,540][1157819] Updated weights for policy 0, policy_version 239648 (0.0006) [2024-09-30 04:06:43,124][1157819] Updated weights for policy 0, policy_version 239658 (0.0006) [2024-09-30 04:06:43,650][1157819] Updated weights for policy 0, policy_version 239668 (0.0006) [2024-09-30 04:06:44,222][1157819] Updated weights for policy 0, policy_version 239678 (0.0006) [2024-09-30 04:06:44,758][1157819] Updated weights for policy 0, policy_version 239688 (0.0006) [2024-09-30 04:06:45,322][1157819] Updated weights for policy 0, policy_version 239698 (0.0006) [2024-09-30 04:06:45,466][1157520] Fps is (10 sec: 74956.3, 60 sec: 77209.7, 300 sec: 77171.4). Total num frames: 981811200. Throughput: 0: 19501.3. Samples: 235422584. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 04:06:45,466][1157520] Avg episode reward: [(0, '55.613')] [2024-09-30 04:06:45,888][1157819] Updated weights for policy 0, policy_version 239708 (0.0006) [2024-09-30 04:06:46,431][1157819] Updated weights for policy 0, policy_version 239718 (0.0006) [2024-09-30 04:06:47,016][1157819] Updated weights for policy 0, policy_version 239728 (0.0006) [2024-09-30 04:06:47,548][1157819] Updated weights for policy 0, policy_version 239738 (0.0006) [2024-09-30 04:06:48,079][1157819] Updated weights for policy 0, policy_version 239748 (0.0006) [2024-09-30 04:06:48,656][1157819] Updated weights for policy 0, policy_version 239758 (0.0006) [2024-09-30 04:06:49,183][1157819] Updated weights for policy 0, policy_version 239768 (0.0006) [2024-09-30 04:06:49,676][1157819] Updated weights for policy 0, policy_version 239778 (0.0006) [2024-09-30 04:06:50,224][1157819] Updated weights for policy 0, policy_version 239788 (0.0006) [2024-09-30 04:06:50,466][1157520] Fps is (10 sec: 74547.1, 60 sec: 77414.5, 300 sec: 77282.5). Total num frames: 982188032. Throughput: 0: 19384.4. Samples: 235534324. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 04:06:50,466][1157520] Avg episode reward: [(0, '55.220')] [2024-09-30 04:06:50,771][1157819] Updated weights for policy 0, policy_version 239798 (0.0006) [2024-09-30 04:06:51,280][1157819] Updated weights for policy 0, policy_version 239808 (0.0006) [2024-09-30 04:06:51,812][1157819] Updated weights for policy 0, policy_version 239818 (0.0006) [2024-09-30 04:06:52,286][1157736] Signal inference workers to stop experience collection... (17100 times) [2024-09-30 04:06:52,291][1157736] Signal inference workers to resume experience collection... (17100 times) [2024-09-30 04:06:52,291][1157819] InferenceWorker_p0-w0: stopping experience collection (17100 times) [2024-09-30 04:06:52,294][1157819] InferenceWorker_p0-w0: resuming experience collection (17100 times) [2024-09-30 04:06:52,310][1157819] Updated weights for policy 0, policy_version 239828 (0.0007) [2024-09-30 04:06:52,806][1157819] Updated weights for policy 0, policy_version 239838 (0.0006) [2024-09-30 04:06:53,335][1157819] Updated weights for policy 0, policy_version 239848 (0.0006) [2024-09-30 04:06:53,894][1157819] Updated weights for policy 0, policy_version 239858 (0.0006) [2024-09-30 04:06:54,396][1157819] Updated weights for policy 0, policy_version 239868 (0.0006) [2024-09-30 04:06:54,910][1157819] Updated weights for policy 0, policy_version 239878 (0.0006) [2024-09-30 04:06:55,428][1157819] Updated weights for policy 0, policy_version 239888 (0.0006) [2024-09-30 04:06:55,465][1157520] Fps is (10 sec: 77415.1, 60 sec: 77756.0, 300 sec: 77365.8). Total num frames: 982585344. Throughput: 0: 19364.8. Samples: 235593248. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 04:06:55,466][1157520] Avg episode reward: [(0, '55.267')] [2024-09-30 04:06:55,927][1157819] Updated weights for policy 0, policy_version 239898 (0.0006) [2024-09-30 04:06:56,444][1157819] Updated weights for policy 0, policy_version 239908 (0.0006) [2024-09-30 04:06:56,993][1157819] Updated weights for policy 0, policy_version 239918 (0.0006) [2024-09-30 04:06:57,503][1157819] Updated weights for policy 0, policy_version 239928 (0.0006) [2024-09-30 04:06:58,016][1157819] Updated weights for policy 0, policy_version 239938 (0.0006) [2024-09-30 04:06:58,543][1157819] Updated weights for policy 0, policy_version 239948 (0.0006) [2024-09-30 04:06:59,055][1157819] Updated weights for policy 0, policy_version 239958 (0.0006) [2024-09-30 04:06:59,558][1157819] Updated weights for policy 0, policy_version 239968 (0.0006) [2024-09-30 04:07:00,084][1157819] Updated weights for policy 0, policy_version 239978 (0.0006) [2024-09-30 04:07:00,466][1157520] Fps is (10 sec: 79052.8, 60 sec: 77960.7, 300 sec: 77463.1). Total num frames: 982978560. Throughput: 0: 19342.5. Samples: 235711456. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 04:07:00,466][1157520] Avg episode reward: [(0, '57.380')] [2024-09-30 04:07:00,599][1157819] Updated weights for policy 0, policy_version 239988 (0.0006) [2024-09-30 04:07:01,097][1157819] Updated weights for policy 0, policy_version 239998 (0.0006) [2024-09-30 04:07:01,589][1157819] Updated weights for policy 0, policy_version 240008 (0.0006) [2024-09-30 04:07:02,145][1157819] Updated weights for policy 0, policy_version 240018 (0.0006) [2024-09-30 04:07:02,671][1157819] Updated weights for policy 0, policy_version 240028 (0.0006) [2024-09-30 04:07:03,165][1157819] Updated weights for policy 0, policy_version 240038 (0.0006) [2024-09-30 04:07:03,675][1157819] Updated weights for policy 0, policy_version 240048 (0.0006) [2024-09-30 04:07:04,182][1157819] Updated weights for policy 0, policy_version 240058 (0.0006) [2024-09-30 04:07:04,673][1157819] Updated weights for policy 0, policy_version 240068 (0.0006) [2024-09-30 04:07:05,163][1157819] Updated weights for policy 0, policy_version 240078 (0.0006) [2024-09-30 04:07:05,466][1157520] Fps is (10 sec: 79461.4, 60 sec: 78028.9, 300 sec: 77546.3). Total num frames: 983379968. Throughput: 0: 19285.5. Samples: 235831832. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 04:07:05,466][1157520] Avg episode reward: [(0, '56.488')] [2024-09-30 04:07:05,681][1157819] Updated weights for policy 0, policy_version 240088 (0.0006) [2024-09-30 04:07:06,166][1157819] Updated weights for policy 0, policy_version 240098 (0.0006) [2024-09-30 04:07:06,673][1157819] Updated weights for policy 0, policy_version 240108 (0.0006) [2024-09-30 04:07:07,218][1157819] Updated weights for policy 0, policy_version 240118 (0.0006) [2024-09-30 04:07:07,784][1157819] Updated weights for policy 0, policy_version 240128 (0.0006) [2024-09-30 04:07:08,351][1157819] Updated weights for policy 0, policy_version 240138 (0.0007) [2024-09-30 04:07:08,957][1157819] Updated weights for policy 0, policy_version 240148 (0.0006) [2024-09-30 04:07:09,521][1157819] Updated weights for policy 0, policy_version 240158 (0.0006) [2024-09-30 04:07:10,109][1157819] Updated weights for policy 0, policy_version 240168 (0.0006) [2024-09-30 04:07:10,466][1157520] Fps is (10 sec: 77414.0, 60 sec: 77619.3, 300 sec: 77574.1). Total num frames: 983752704. Throughput: 0: 19235.0. Samples: 235891060. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 04:07:10,466][1157520] Avg episode reward: [(0, '57.130')] [2024-09-30 04:07:10,648][1157819] Updated weights for policy 0, policy_version 240178 (0.0006) [2024-09-30 04:07:11,227][1157819] Updated weights for policy 0, policy_version 240188 (0.0006) [2024-09-30 04:07:11,827][1157819] Updated weights for policy 0, policy_version 240198 (0.0006) [2024-09-30 04:07:12,392][1157819] Updated weights for policy 0, policy_version 240208 (0.0006) [2024-09-30 04:07:12,971][1157819] Updated weights for policy 0, policy_version 240218 (0.0006) [2024-09-30 04:07:13,493][1157819] Updated weights for policy 0, policy_version 240228 (0.0006) [2024-09-30 04:07:14,026][1157819] Updated weights for policy 0, policy_version 240238 (0.0006) [2024-09-30 04:07:14,520][1157819] Updated weights for policy 0, policy_version 240248 (0.0006) [2024-09-30 04:07:14,987][1157819] Updated weights for policy 0, policy_version 240258 (0.0006) [2024-09-30 04:07:15,466][1157520] Fps is (10 sec: 75366.8, 60 sec: 77278.0, 300 sec: 77588.0). Total num frames: 984133632. Throughput: 0: 19101.3. Samples: 235998816. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 04:07:15,466][1157520] Avg episode reward: [(0, '55.319')] [2024-09-30 04:07:15,468][1157819] Updated weights for policy 0, policy_version 240268 (0.0006) [2024-09-30 04:07:15,973][1157819] Updated weights for policy 0, policy_version 240278 (0.0006) [2024-09-30 04:07:16,477][1157819] Updated weights for policy 0, policy_version 240288 (0.0006) [2024-09-30 04:07:16,980][1157819] Updated weights for policy 0, policy_version 240298 (0.0006) [2024-09-30 04:07:17,484][1157819] Updated weights for policy 0, policy_version 240308 (0.0006) [2024-09-30 04:07:17,988][1157819] Updated weights for policy 0, policy_version 240318 (0.0006) [2024-09-30 04:07:18,470][1157819] Updated weights for policy 0, policy_version 240328 (0.0006) [2024-09-30 04:07:18,918][1157819] Updated weights for policy 0, policy_version 240338 (0.0006) [2024-09-30 04:07:19,424][1157819] Updated weights for policy 0, policy_version 240348 (0.0006) [2024-09-30 04:07:19,937][1157819] Updated weights for policy 0, policy_version 240358 (0.0006) [2024-09-30 04:07:20,422][1157819] Updated weights for policy 0, policy_version 240368 (0.0007) [2024-09-30 04:07:20,466][1157520] Fps is (10 sec: 79462.6, 60 sec: 77277.9, 300 sec: 77726.8). Total num frames: 984547328. Throughput: 0: 19325.3. Samples: 236123316. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 04:07:20,466][1157520] Avg episode reward: [(0, '58.837')] [2024-09-30 04:07:20,920][1157819] Updated weights for policy 0, policy_version 240378 (0.0006) [2024-09-30 04:07:21,407][1157819] Updated weights for policy 0, policy_version 240388 (0.0006) [2024-09-30 04:07:21,923][1157819] Updated weights for policy 0, policy_version 240398 (0.0006) [2024-09-30 04:07:22,438][1157819] Updated weights for policy 0, policy_version 240408 (0.0006) [2024-09-30 04:07:22,930][1157819] Updated weights for policy 0, policy_version 240418 (0.0006) [2024-09-30 04:07:23,484][1157819] Updated weights for policy 0, policy_version 240428 (0.0006) [2024-09-30 04:07:24,050][1157819] Updated weights for policy 0, policy_version 240438 (0.0006) [2024-09-30 04:07:24,592][1157819] Updated weights for policy 0, policy_version 240448 (0.0006) [2024-09-30 04:07:25,127][1157819] Updated weights for policy 0, policy_version 240458 (0.0006) [2024-09-30 04:07:25,466][1157520] Fps is (10 sec: 80689.2, 60 sec: 77277.6, 300 sec: 77810.1). Total num frames: 984940544. Throughput: 0: 19424.9. Samples: 236184688. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 04:07:25,466][1157520] Avg episode reward: [(0, '58.212')] [2024-09-30 04:07:25,633][1157819] Updated weights for policy 0, policy_version 240468 (0.0006) [2024-09-30 04:07:26,136][1157819] Updated weights for policy 0, policy_version 240478 (0.0006) [2024-09-30 04:07:26,638][1157819] Updated weights for policy 0, policy_version 240488 (0.0006) [2024-09-30 04:07:27,138][1157819] Updated weights for policy 0, policy_version 240498 (0.0006) [2024-09-30 04:07:27,669][1157819] Updated weights for policy 0, policy_version 240508 (0.0006) [2024-09-30 04:07:28,181][1157819] Updated weights for policy 0, policy_version 240518 (0.0006) [2024-09-30 04:07:28,689][1157819] Updated weights for policy 0, policy_version 240528 (0.0006) [2024-09-30 04:07:29,199][1157819] Updated weights for policy 0, policy_version 240538 (0.0006) [2024-09-30 04:07:29,729][1157819] Updated weights for policy 0, policy_version 240548 (0.0006) [2024-09-30 04:07:30,219][1157819] Updated weights for policy 0, policy_version 240558 (0.0006) [2024-09-30 04:07:30,466][1157520] Fps is (10 sec: 79461.3, 60 sec: 77619.0, 300 sec: 77921.2). Total num frames: 985341952. Throughput: 0: 19556.7. Samples: 236302636. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 04:07:30,466][1157520] Avg episode reward: [(0, '56.882')] [2024-09-30 04:07:30,733][1157819] Updated weights for policy 0, policy_version 240568 (0.0006) [2024-09-30 04:07:31,258][1157819] Updated weights for policy 0, policy_version 240578 (0.0006) [2024-09-30 04:07:31,778][1157819] Updated weights for policy 0, policy_version 240588 (0.0006) [2024-09-30 04:07:32,287][1157819] Updated weights for policy 0, policy_version 240598 (0.0006) [2024-09-30 04:07:32,794][1157819] Updated weights for policy 0, policy_version 240608 (0.0006) [2024-09-30 04:07:33,306][1157819] Updated weights for policy 0, policy_version 240618 (0.0006) [2024-09-30 04:07:33,824][1157819] Updated weights for policy 0, policy_version 240628 (0.0006) [2024-09-30 04:07:34,345][1157819] Updated weights for policy 0, policy_version 240638 (0.0006) [2024-09-30 04:07:34,840][1157819] Updated weights for policy 0, policy_version 240648 (0.0006) [2024-09-30 04:07:35,356][1157819] Updated weights for policy 0, policy_version 240658 (0.0006) [2024-09-30 04:07:35,466][1157520] Fps is (10 sec: 80282.6, 60 sec: 78028.6, 300 sec: 77879.5). Total num frames: 985743360. Throughput: 0: 19736.1. Samples: 236422452. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 04:07:35,466][1157520] Avg episode reward: [(0, '56.443')] [2024-09-30 04:07:35,852][1157819] Updated weights for policy 0, policy_version 240668 (0.0006) [2024-09-30 04:07:36,402][1157819] Updated weights for policy 0, policy_version 240678 (0.0006) [2024-09-30 04:07:36,906][1157819] Updated weights for policy 0, policy_version 240688 (0.0006) [2024-09-30 04:07:37,441][1157819] Updated weights for policy 0, policy_version 240698 (0.0006) [2024-09-30 04:07:37,955][1157819] Updated weights for policy 0, policy_version 240708 (0.0006) [2024-09-30 04:07:38,468][1157819] Updated weights for policy 0, policy_version 240718 (0.0006) [2024-09-30 04:07:38,993][1157819] Updated weights for policy 0, policy_version 240728 (0.0006) [2024-09-30 04:07:39,509][1157819] Updated weights for policy 0, policy_version 240738 (0.0006) [2024-09-30 04:07:40,016][1157819] Updated weights for policy 0, policy_version 240748 (0.0006) [2024-09-30 04:07:40,466][1157520] Fps is (10 sec: 79462.3, 60 sec: 78233.4, 300 sec: 77837.9). Total num frames: 986136576. Throughput: 0: 19746.5. Samples: 236481844. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 04:07:40,466][1157520] Avg episode reward: [(0, '56.541')] [2024-09-30 04:07:40,542][1157819] Updated weights for policy 0, policy_version 240758 (0.0006) [2024-09-30 04:07:41,091][1157819] Updated weights for policy 0, policy_version 240768 (0.0006) [2024-09-30 04:07:41,591][1157819] Updated weights for policy 0, policy_version 240778 (0.0006) [2024-09-30 04:07:41,778][1157736] Signal inference workers to stop experience collection... (17150 times) [2024-09-30 04:07:41,781][1157736] Signal inference workers to resume experience collection... (17150 times) [2024-09-30 04:07:41,785][1157819] InferenceWorker_p0-w0: stopping experience collection (17150 times) [2024-09-30 04:07:41,787][1157819] InferenceWorker_p0-w0: resuming experience collection (17150 times) [2024-09-30 04:07:42,116][1157819] Updated weights for policy 0, policy_version 240788 (0.0006) [2024-09-30 04:07:42,620][1157819] Updated weights for policy 0, policy_version 240798 (0.0006) [2024-09-30 04:07:43,117][1157819] Updated weights for policy 0, policy_version 240808 (0.0006) [2024-09-30 04:07:43,620][1157819] Updated weights for policy 0, policy_version 240818 (0.0006) [2024-09-30 04:07:44,149][1157819] Updated weights for policy 0, policy_version 240828 (0.0006) [2024-09-30 04:07:44,709][1157819] Updated weights for policy 0, policy_version 240838 (0.0006) [2024-09-30 04:07:45,237][1157819] Updated weights for policy 0, policy_version 240848 (0.0006) [2024-09-30 04:07:45,466][1157520] Fps is (10 sec: 78233.5, 60 sec: 78574.8, 300 sec: 77810.1). Total num frames: 986525696. Throughput: 0: 19762.7. Samples: 236600780. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 04:07:45,466][1157520] Avg episode reward: [(0, '56.139')] [2024-09-30 04:07:45,803][1157819] Updated weights for policy 0, policy_version 240858 (0.0006) [2024-09-30 04:07:46,309][1157819] Updated weights for policy 0, policy_version 240868 (0.0006) [2024-09-30 04:07:46,857][1157819] Updated weights for policy 0, policy_version 240878 (0.0006) [2024-09-30 04:07:47,406][1157819] Updated weights for policy 0, policy_version 240888 (0.0006) [2024-09-30 04:07:47,954][1157819] Updated weights for policy 0, policy_version 240898 (0.0006) [2024-09-30 04:07:48,481][1157819] Updated weights for policy 0, policy_version 240908 (0.0006) [2024-09-30 04:07:49,014][1157819] Updated weights for policy 0, policy_version 240918 (0.0006) [2024-09-30 04:07:49,525][1157819] Updated weights for policy 0, policy_version 240928 (0.0006) [2024-09-30 04:07:50,099][1157819] Updated weights for policy 0, policy_version 240938 (0.0006) [2024-09-30 04:07:50,466][1157520] Fps is (10 sec: 77005.0, 60 sec: 78643.0, 300 sec: 77810.1). Total num frames: 986906624. Throughput: 0: 19619.0. Samples: 236714688. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 04:07:50,466][1157520] Avg episode reward: [(0, '55.902')] [2024-09-30 04:07:50,642][1157819] Updated weights for policy 0, policy_version 240948 (0.0006) [2024-09-30 04:07:51,173][1157819] Updated weights for policy 0, policy_version 240958 (0.0006) [2024-09-30 04:07:51,762][1157819] Updated weights for policy 0, policy_version 240968 (0.0006) [2024-09-30 04:07:52,320][1157819] Updated weights for policy 0, policy_version 240978 (0.0006) [2024-09-30 04:07:52,866][1157819] Updated weights for policy 0, policy_version 240988 (0.0006) [2024-09-30 04:07:53,414][1157819] Updated weights for policy 0, policy_version 240998 (0.0006) [2024-09-30 04:07:54,034][1157819] Updated weights for policy 0, policy_version 241008 (0.0006) [2024-09-30 04:07:54,576][1157819] Updated weights for policy 0, policy_version 241018 (0.0006) [2024-09-30 04:07:55,136][1157819] Updated weights for policy 0, policy_version 241028 (0.0006) [2024-09-30 04:07:55,466][1157520] Fps is (10 sec: 74546.9, 60 sec: 78096.8, 300 sec: 77768.5). Total num frames: 987271168. Throughput: 0: 19534.3. Samples: 236770108. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 04:07:55,466][1157520] Avg episode reward: [(0, '57.236')] [2024-09-30 04:07:55,697][1157819] Updated weights for policy 0, policy_version 241038 (0.0006) [2024-09-30 04:07:56,269][1157819] Updated weights for policy 0, policy_version 241048 (0.0006) [2024-09-30 04:07:56,849][1157819] Updated weights for policy 0, policy_version 241058 (0.0006) [2024-09-30 04:07:57,494][1157819] Updated weights for policy 0, policy_version 241068 (0.0006) [2024-09-30 04:07:58,102][1157819] Updated weights for policy 0, policy_version 241078 (0.0006) [2024-09-30 04:07:58,729][1157819] Updated weights for policy 0, policy_version 241088 (0.0006) [2024-09-30 04:07:59,354][1157819] Updated weights for policy 0, policy_version 241098 (0.0006) [2024-09-30 04:07:59,975][1157819] Updated weights for policy 0, policy_version 241108 (0.0006) [2024-09-30 04:08:00,466][1157520] Fps is (10 sec: 70451.0, 60 sec: 77209.4, 300 sec: 77657.4). Total num frames: 987611136. Throughput: 0: 19480.6. Samples: 236875444. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 04:08:00,466][1157520] Avg episode reward: [(0, '57.528')] [2024-09-30 04:08:00,586][1157819] Updated weights for policy 0, policy_version 241118 (0.0006) [2024-09-30 04:08:01,204][1157819] Updated weights for policy 0, policy_version 241128 (0.0006) [2024-09-30 04:08:01,855][1157819] Updated weights for policy 0, policy_version 241138 (0.0006) [2024-09-30 04:08:02,435][1157819] Updated weights for policy 0, policy_version 241148 (0.0006) [2024-09-30 04:08:02,992][1157819] Updated weights for policy 0, policy_version 241158 (0.0006) [2024-09-30 04:08:03,668][1157819] Updated weights for policy 0, policy_version 241168 (0.0006) [2024-09-30 04:08:04,262][1157819] Updated weights for policy 0, policy_version 241178 (0.0006) [2024-09-30 04:08:04,927][1157819] Updated weights for policy 0, policy_version 241188 (0.0006) [2024-09-30 04:08:05,445][1157819] Updated weights for policy 0, policy_version 241198 (0.0006) [2024-09-30 04:08:05,466][1157520] Fps is (10 sec: 67583.4, 60 sec: 76117.1, 300 sec: 77518.5). Total num frames: 987947008. Throughput: 0: 18919.5. Samples: 236974696. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 04:08:05,466][1157520] Avg episode reward: [(0, '56.397')] [2024-09-30 04:08:05,474][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000241198_987947008.pth... [2024-09-30 04:08:05,533][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000236676_969424896.pth [2024-09-30 04:08:06,012][1157819] Updated weights for policy 0, policy_version 241208 (0.0006) [2024-09-30 04:08:06,546][1157819] Updated weights for policy 0, policy_version 241218 (0.0006) [2024-09-30 04:08:06,702][1157736] Signal inference workers to stop experience collection... (17200 times) [2024-09-30 04:08:06,703][1157736] Signal inference workers to resume experience collection... (17200 times) [2024-09-30 04:08:06,709][1157819] InferenceWorker_p0-w0: stopping experience collection (17200 times) [2024-09-30 04:08:06,709][1157819] InferenceWorker_p0-w0: resuming experience collection (17200 times) [2024-09-30 04:08:07,055][1157819] Updated weights for policy 0, policy_version 241228 (0.0006) [2024-09-30 04:08:07,602][1157819] Updated weights for policy 0, policy_version 241238 (0.0006) [2024-09-30 04:08:08,166][1157819] Updated weights for policy 0, policy_version 241248 (0.0006) [2024-09-30 04:08:08,697][1157819] Updated weights for policy 0, policy_version 241258 (0.0006) [2024-09-30 04:08:09,298][1157819] Updated weights for policy 0, policy_version 241268 (0.0006) [2024-09-30 04:08:09,879][1157819] Updated weights for policy 0, policy_version 241278 (0.0006) [2024-09-30 04:08:10,427][1157819] Updated weights for policy 0, policy_version 241288 (0.0006) [2024-09-30 04:08:10,466][1157520] Fps is (10 sec: 70451.5, 60 sec: 76049.0, 300 sec: 77435.3). Total num frames: 988315648. Throughput: 0: 18807.7. Samples: 237031032. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 04:08:10,466][1157520] Avg episode reward: [(0, '56.477')] [2024-09-30 04:08:10,988][1157819] Updated weights for policy 0, policy_version 241298 (0.0006) [2024-09-30 04:08:11,536][1157819] Updated weights for policy 0, policy_version 241308 (0.0006) [2024-09-30 04:08:12,117][1157819] Updated weights for policy 0, policy_version 241318 (0.0006) [2024-09-30 04:08:12,672][1157819] Updated weights for policy 0, policy_version 241328 (0.0006) [2024-09-30 04:08:13,261][1157819] Updated weights for policy 0, policy_version 241338 (0.0006) [2024-09-30 04:08:13,843][1157819] Updated weights for policy 0, policy_version 241348 (0.0006) [2024-09-30 04:08:14,428][1157819] Updated weights for policy 0, policy_version 241358 (0.0006) [2024-09-30 04:08:15,018][1157819] Updated weights for policy 0, policy_version 241368 (0.0006) [2024-09-30 04:08:15,466][1157520] Fps is (10 sec: 72499.9, 60 sec: 75639.3, 300 sec: 77310.3). Total num frames: 988672000. Throughput: 0: 18590.9. Samples: 237139228. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 04:08:15,466][1157520] Avg episode reward: [(0, '57.581')] [2024-09-30 04:08:15,599][1157819] Updated weights for policy 0, policy_version 241378 (0.0006) [2024-09-30 04:08:16,147][1157819] Updated weights for policy 0, policy_version 241388 (0.0006) [2024-09-30 04:08:16,708][1157819] Updated weights for policy 0, policy_version 241398 (0.0006) [2024-09-30 04:08:17,273][1157819] Updated weights for policy 0, policy_version 241408 (0.0006) [2024-09-30 04:08:17,834][1157819] Updated weights for policy 0, policy_version 241418 (0.0006) [2024-09-30 04:08:18,380][1157819] Updated weights for policy 0, policy_version 241428 (0.0006) [2024-09-30 04:08:18,929][1157819] Updated weights for policy 0, policy_version 241438 (0.0006) [2024-09-30 04:08:19,472][1157819] Updated weights for policy 0, policy_version 241448 (0.0006) [2024-09-30 04:08:19,969][1157819] Updated weights for policy 0, policy_version 241458 (0.0006) [2024-09-30 04:08:20,466][1157520] Fps is (10 sec: 73318.2, 60 sec: 75024.9, 300 sec: 77254.7). Total num frames: 989048832. Throughput: 0: 18371.9. Samples: 237249188. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 04:08:20,466][1157520] Avg episode reward: [(0, '56.629')] [2024-09-30 04:08:20,500][1157819] Updated weights for policy 0, policy_version 241468 (0.0006) [2024-09-30 04:08:21,060][1157819] Updated weights for policy 0, policy_version 241478 (0.0006) [2024-09-30 04:08:21,554][1157819] Updated weights for policy 0, policy_version 241488 (0.0006) [2024-09-30 04:08:22,074][1157819] Updated weights for policy 0, policy_version 241498 (0.0006) [2024-09-30 04:08:22,595][1157819] Updated weights for policy 0, policy_version 241508 (0.0006) [2024-09-30 04:08:23,121][1157819] Updated weights for policy 0, policy_version 241518 (0.0006) [2024-09-30 04:08:23,644][1157819] Updated weights for policy 0, policy_version 241528 (0.0006) [2024-09-30 04:08:24,166][1157819] Updated weights for policy 0, policy_version 241538 (0.0006) [2024-09-30 04:08:24,683][1157819] Updated weights for policy 0, policy_version 241548 (0.0006) [2024-09-30 04:08:25,227][1157819] Updated weights for policy 0, policy_version 241558 (0.0006) [2024-09-30 04:08:25,466][1157520] Fps is (10 sec: 76594.4, 60 sec: 74956.8, 300 sec: 77240.8). Total num frames: 989437952. Throughput: 0: 18355.0. Samples: 237307820. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 04:08:25,466][1157520] Avg episode reward: [(0, '57.164')] [2024-09-30 04:08:25,754][1157819] Updated weights for policy 0, policy_version 241568 (0.0006) [2024-09-30 04:08:26,244][1157819] Updated weights for policy 0, policy_version 241578 (0.0006) [2024-09-30 04:08:26,768][1157819] Updated weights for policy 0, policy_version 241588 (0.0006) [2024-09-30 04:08:27,311][1157819] Updated weights for policy 0, policy_version 241598 (0.0006) [2024-09-30 04:08:27,861][1157819] Updated weights for policy 0, policy_version 241608 (0.0006) [2024-09-30 04:08:28,362][1157819] Updated weights for policy 0, policy_version 241618 (0.0006) [2024-09-30 04:08:28,864][1157819] Updated weights for policy 0, policy_version 241628 (0.0006) [2024-09-30 04:08:29,382][1157819] Updated weights for policy 0, policy_version 241638 (0.0006) [2024-09-30 04:08:29,932][1157819] Updated weights for policy 0, policy_version 241648 (0.0006) [2024-09-30 04:08:30,453][1157819] Updated weights for policy 0, policy_version 241658 (0.0006) [2024-09-30 04:08:30,466][1157520] Fps is (10 sec: 78234.3, 60 sec: 74820.4, 300 sec: 77199.2). Total num frames: 989831168. Throughput: 0: 18320.6. Samples: 237425208. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 04:08:30,466][1157520] Avg episode reward: [(0, '56.740')] [2024-09-30 04:08:30,960][1157819] Updated weights for policy 0, policy_version 241668 (0.0006) [2024-09-30 04:08:31,479][1157819] Updated weights for policy 0, policy_version 241678 (0.0006) [2024-09-30 04:08:31,991][1157819] Updated weights for policy 0, policy_version 241688 (0.0006) [2024-09-30 04:08:32,507][1157819] Updated weights for policy 0, policy_version 241698 (0.0006) [2024-09-30 04:08:33,025][1157819] Updated weights for policy 0, policy_version 241708 (0.0006) [2024-09-30 04:08:33,529][1157819] Updated weights for policy 0, policy_version 241718 (0.0006) [2024-09-30 04:08:34,109][1157819] Updated weights for policy 0, policy_version 241728 (0.0007) [2024-09-30 04:08:34,616][1157819] Updated weights for policy 0, policy_version 241738 (0.0006) [2024-09-30 04:08:35,125][1157819] Updated weights for policy 0, policy_version 241748 (0.0006) [2024-09-30 04:08:35,466][1157520] Fps is (10 sec: 79054.5, 60 sec: 74752.1, 300 sec: 77185.3). Total num frames: 990228480. Throughput: 0: 18426.0. Samples: 237543856. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 04:08:35,466][1157520] Avg episode reward: [(0, '57.355')] [2024-09-30 04:08:35,612][1157819] Updated weights for policy 0, policy_version 241758 (0.0006) [2024-09-30 04:08:36,088][1157819] Updated weights for policy 0, policy_version 241768 (0.0006) [2024-09-30 04:08:36,653][1157819] Updated weights for policy 0, policy_version 241778 (0.0006) [2024-09-30 04:08:37,147][1157819] Updated weights for policy 0, policy_version 241788 (0.0006) [2024-09-30 04:08:37,680][1157819] Updated weights for policy 0, policy_version 241798 (0.0006) [2024-09-30 04:08:38,220][1157819] Updated weights for policy 0, policy_version 241808 (0.0006) [2024-09-30 04:08:38,719][1157819] Updated weights for policy 0, policy_version 241818 (0.0006) [2024-09-30 04:08:39,230][1157819] Updated weights for policy 0, policy_version 241828 (0.0006) [2024-09-30 04:08:39,744][1157819] Updated weights for policy 0, policy_version 241838 (0.0006) [2024-09-30 04:08:40,298][1157819] Updated weights for policy 0, policy_version 241848 (0.0006) [2024-09-30 04:08:40,466][1157520] Fps is (10 sec: 79052.9, 60 sec: 74752.1, 300 sec: 77143.7). Total num frames: 990621696. Throughput: 0: 18514.2. Samples: 237603244. Policy #0 lag: (min: 0.0, avg: 2.4, max: 5.0) [2024-09-30 04:08:40,466][1157520] Avg episode reward: [(0, '55.643')] [2024-09-30 04:08:40,832][1157819] Updated weights for policy 0, policy_version 241858 (0.0006) [2024-09-30 04:08:41,337][1157819] Updated weights for policy 0, policy_version 241868 (0.0006) [2024-09-30 04:08:41,849][1157819] Updated weights for policy 0, policy_version 241878 (0.0006) [2024-09-30 04:08:42,389][1157819] Updated weights for policy 0, policy_version 241888 (0.0006) [2024-09-30 04:08:42,925][1157819] Updated weights for policy 0, policy_version 241898 (0.0006) [2024-09-30 04:08:43,053][1157736] Signal inference workers to stop experience collection... (17250 times) [2024-09-30 04:08:43,053][1157736] Signal inference workers to resume experience collection... (17250 times) [2024-09-30 04:08:43,057][1157819] InferenceWorker_p0-w0: stopping experience collection (17250 times) [2024-09-30 04:08:43,060][1157819] InferenceWorker_p0-w0: resuming experience collection (17250 times) [2024-09-30 04:08:43,491][1157819] Updated weights for policy 0, policy_version 241908 (0.0006) [2024-09-30 04:08:44,027][1157819] Updated weights for policy 0, policy_version 241918 (0.0006) [2024-09-30 04:08:44,541][1157819] Updated weights for policy 0, policy_version 241928 (0.0006) [2024-09-30 04:08:45,091][1157819] Updated weights for policy 0, policy_version 241938 (0.0006) [2024-09-30 04:08:45,466][1157520] Fps is (10 sec: 77412.8, 60 sec: 74615.3, 300 sec: 77101.9). Total num frames: 991002624. Throughput: 0: 18759.8. Samples: 237719636. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 04:08:45,466][1157520] Avg episode reward: [(0, '56.962')] [2024-09-30 04:08:45,623][1157819] Updated weights for policy 0, policy_version 241948 (0.0006) [2024-09-30 04:08:46,176][1157819] Updated weights for policy 0, policy_version 241958 (0.0006) [2024-09-30 04:08:46,684][1157819] Updated weights for policy 0, policy_version 241968 (0.0006) [2024-09-30 04:08:47,191][1157819] Updated weights for policy 0, policy_version 241978 (0.0006) [2024-09-30 04:08:47,748][1157819] Updated weights for policy 0, policy_version 241988 (0.0006) [2024-09-30 04:08:48,268][1157819] Updated weights for policy 0, policy_version 241998 (0.0006) [2024-09-30 04:08:48,773][1157819] Updated weights for policy 0, policy_version 242008 (0.0006) [2024-09-30 04:08:49,310][1157819] Updated weights for policy 0, policy_version 242018 (0.0006) [2024-09-30 04:08:49,812][1157819] Updated weights for policy 0, policy_version 242028 (0.0006) [2024-09-30 04:08:50,328][1157819] Updated weights for policy 0, policy_version 242038 (0.0006) [2024-09-30 04:08:50,466][1157520] Fps is (10 sec: 77413.5, 60 sec: 74820.2, 300 sec: 77115.9). Total num frames: 991395840. Throughput: 0: 19154.2. Samples: 237836632. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 04:08:50,466][1157520] Avg episode reward: [(0, '57.729')] [2024-09-30 04:08:50,847][1157819] Updated weights for policy 0, policy_version 242048 (0.0006) [2024-09-30 04:08:51,350][1157819] Updated weights for policy 0, policy_version 242058 (0.0006) [2024-09-30 04:08:51,867][1157819] Updated weights for policy 0, policy_version 242068 (0.0006) [2024-09-30 04:08:52,360][1157819] Updated weights for policy 0, policy_version 242078 (0.0006) [2024-09-30 04:08:52,848][1157819] Updated weights for policy 0, policy_version 242088 (0.0006) [2024-09-30 04:08:53,395][1157819] Updated weights for policy 0, policy_version 242098 (0.0006) [2024-09-30 04:08:53,935][1157819] Updated weights for policy 0, policy_version 242108 (0.0006) [2024-09-30 04:08:54,441][1157819] Updated weights for policy 0, policy_version 242118 (0.0006) [2024-09-30 04:08:54,953][1157819] Updated weights for policy 0, policy_version 242128 (0.0006) [2024-09-30 04:08:55,466][1157520] Fps is (10 sec: 78643.2, 60 sec: 75298.0, 300 sec: 77102.0). Total num frames: 991789056. Throughput: 0: 19225.5. Samples: 237896184. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 04:08:55,466][1157520] Avg episode reward: [(0, '55.307')] [2024-09-30 04:08:55,508][1157819] Updated weights for policy 0, policy_version 242138 (0.0006) [2024-09-30 04:08:56,085][1157819] Updated weights for policy 0, policy_version 242148 (0.0006) [2024-09-30 04:08:56,650][1157819] Updated weights for policy 0, policy_version 242158 (0.0006) [2024-09-30 04:08:57,218][1157819] Updated weights for policy 0, policy_version 242168 (0.0006) [2024-09-30 04:08:57,804][1157819] Updated weights for policy 0, policy_version 242178 (0.0006) [2024-09-30 04:08:58,374][1157819] Updated weights for policy 0, policy_version 242188 (0.0006) [2024-09-30 04:08:58,892][1157819] Updated weights for policy 0, policy_version 242198 (0.0006) [2024-09-30 04:08:59,425][1157819] Updated weights for policy 0, policy_version 242208 (0.0006) [2024-09-30 04:08:59,954][1157819] Updated weights for policy 0, policy_version 242218 (0.0006) [2024-09-30 04:09:00,466][1157520] Fps is (10 sec: 76185.3, 60 sec: 75775.9, 300 sec: 77060.3). Total num frames: 992157696. Throughput: 0: 19327.7. Samples: 238008976. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 04:09:00,466][1157520] Avg episode reward: [(0, '55.778')] [2024-09-30 04:09:00,535][1157819] Updated weights for policy 0, policy_version 242228 (0.0006) [2024-09-30 04:09:01,081][1157819] Updated weights for policy 0, policy_version 242238 (0.0006) [2024-09-30 04:09:01,651][1157819] Updated weights for policy 0, policy_version 242248 (0.0006) [2024-09-30 04:09:02,185][1157819] Updated weights for policy 0, policy_version 242258 (0.0006) [2024-09-30 04:09:02,714][1157819] Updated weights for policy 0, policy_version 242268 (0.0006) [2024-09-30 04:09:03,245][1157819] Updated weights for policy 0, policy_version 242278 (0.0006) [2024-09-30 04:09:03,792][1157819] Updated weights for policy 0, policy_version 242288 (0.0006) [2024-09-30 04:09:04,328][1157819] Updated weights for policy 0, policy_version 242298 (0.0006) [2024-09-30 04:09:04,936][1157819] Updated weights for policy 0, policy_version 242308 (0.0006) [2024-09-30 04:09:05,466][1157819] Updated weights for policy 0, policy_version 242318 (0.0006) [2024-09-30 04:09:05,466][1157520] Fps is (10 sec: 74547.8, 60 sec: 76458.8, 300 sec: 77018.7). Total num frames: 992534528. Throughput: 0: 19380.8. Samples: 238121324. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 04:09:05,466][1157520] Avg episode reward: [(0, '55.653')] [2024-09-30 04:09:05,986][1157736] Signal inference workers to stop experience collection... (17300 times) [2024-09-30 04:09:05,986][1157736] Signal inference workers to resume experience collection... (17300 times) [2024-09-30 04:09:05,989][1157819] InferenceWorker_p0-w0: stopping experience collection (17300 times) [2024-09-30 04:09:05,992][1157819] InferenceWorker_p0-w0: resuming experience collection (17300 times) [2024-09-30 04:09:06,011][1157819] Updated weights for policy 0, policy_version 242328 (0.0006) [2024-09-30 04:09:06,539][1157819] Updated weights for policy 0, policy_version 242338 (0.0006) [2024-09-30 04:09:07,068][1157819] Updated weights for policy 0, policy_version 242348 (0.0006) [2024-09-30 04:09:07,597][1157819] Updated weights for policy 0, policy_version 242358 (0.0006) [2024-09-30 04:09:08,155][1157819] Updated weights for policy 0, policy_version 242368 (0.0006) [2024-09-30 04:09:08,693][1157819] Updated weights for policy 0, policy_version 242378 (0.0006) [2024-09-30 04:09:09,203][1157819] Updated weights for policy 0, policy_version 242388 (0.0006) [2024-09-30 04:09:09,781][1157819] Updated weights for policy 0, policy_version 242398 (0.0006) [2024-09-30 04:09:10,320][1157819] Updated weights for policy 0, policy_version 242408 (0.0006) [2024-09-30 04:09:10,466][1157520] Fps is (10 sec: 75366.2, 60 sec: 76595.0, 300 sec: 76935.3). Total num frames: 992911360. Throughput: 0: 19338.4. Samples: 238178048. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 04:09:10,466][1157520] Avg episode reward: [(0, '56.410')] [2024-09-30 04:09:10,856][1157819] Updated weights for policy 0, policy_version 242418 (0.0006) [2024-09-30 04:09:11,422][1157819] Updated weights for policy 0, policy_version 242428 (0.0006) [2024-09-30 04:09:11,944][1157819] Updated weights for policy 0, policy_version 242438 (0.0006) [2024-09-30 04:09:12,475][1157819] Updated weights for policy 0, policy_version 242448 (0.0006) [2024-09-30 04:09:13,045][1157819] Updated weights for policy 0, policy_version 242458 (0.0006) [2024-09-30 04:09:13,540][1157819] Updated weights for policy 0, policy_version 242468 (0.0006) [2024-09-30 04:09:14,074][1157819] Updated weights for policy 0, policy_version 242478 (0.0006) [2024-09-30 04:09:14,631][1157819] Updated weights for policy 0, policy_version 242488 (0.0006) [2024-09-30 04:09:15,191][1157819] Updated weights for policy 0, policy_version 242498 (0.0006) [2024-09-30 04:09:15,466][1157520] Fps is (10 sec: 75776.5, 60 sec: 77004.8, 300 sec: 76824.3). Total num frames: 993292288. Throughput: 0: 19256.7. Samples: 238291760. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 04:09:15,466][1157520] Avg episode reward: [(0, '56.869')] [2024-09-30 04:09:15,724][1157819] Updated weights for policy 0, policy_version 242508 (0.0006) [2024-09-30 04:09:16,284][1157819] Updated weights for policy 0, policy_version 242518 (0.0006) [2024-09-30 04:09:16,810][1157819] Updated weights for policy 0, policy_version 242528 (0.0006) [2024-09-30 04:09:17,319][1157819] Updated weights for policy 0, policy_version 242538 (0.0006) [2024-09-30 04:09:17,845][1157819] Updated weights for policy 0, policy_version 242548 (0.0006) [2024-09-30 04:09:18,384][1157819] Updated weights for policy 0, policy_version 242558 (0.0006) [2024-09-30 04:09:18,889][1157819] Updated weights for policy 0, policy_version 242568 (0.0006) [2024-09-30 04:09:19,411][1157819] Updated weights for policy 0, policy_version 242578 (0.0006) [2024-09-30 04:09:19,925][1157819] Updated weights for policy 0, policy_version 242588 (0.0006) [2024-09-30 04:09:20,435][1157819] Updated weights for policy 0, policy_version 242598 (0.0006) [2024-09-30 04:09:20,466][1157520] Fps is (10 sec: 77005.9, 60 sec: 77209.7, 300 sec: 76782.7). Total num frames: 993681408. Throughput: 0: 19190.0. Samples: 238407408. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 04:09:20,466][1157520] Avg episode reward: [(0, '56.725')] [2024-09-30 04:09:20,952][1157819] Updated weights for policy 0, policy_version 242608 (0.0006) [2024-09-30 04:09:21,454][1157819] Updated weights for policy 0, policy_version 242618 (0.0006) [2024-09-30 04:09:21,968][1157819] Updated weights for policy 0, policy_version 242628 (0.0006) [2024-09-30 04:09:22,490][1157819] Updated weights for policy 0, policy_version 242638 (0.0006) [2024-09-30 04:09:23,007][1157819] Updated weights for policy 0, policy_version 242648 (0.0006) [2024-09-30 04:09:23,524][1157819] Updated weights for policy 0, policy_version 242658 (0.0006) [2024-09-30 04:09:24,001][1157819] Updated weights for policy 0, policy_version 242668 (0.0006) [2024-09-30 04:09:24,493][1157819] Updated weights for policy 0, policy_version 242678 (0.0006) [2024-09-30 04:09:24,997][1157819] Updated weights for policy 0, policy_version 242688 (0.0006) [2024-09-30 04:09:25,466][1157520] Fps is (10 sec: 79462.4, 60 sec: 77482.8, 300 sec: 76824.3). Total num frames: 994086912. Throughput: 0: 19206.3. Samples: 238467528. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 04:09:25,466][1157520] Avg episode reward: [(0, '56.474')] [2024-09-30 04:09:25,474][1157819] Updated weights for policy 0, policy_version 242698 (0.0006) [2024-09-30 04:09:26,008][1157819] Updated weights for policy 0, policy_version 242708 (0.0006) [2024-09-30 04:09:26,514][1157819] Updated weights for policy 0, policy_version 242718 (0.0006) [2024-09-30 04:09:27,014][1157819] Updated weights for policy 0, policy_version 242728 (0.0006) [2024-09-30 04:09:27,533][1157819] Updated weights for policy 0, policy_version 242738 (0.0006) [2024-09-30 04:09:28,075][1157819] Updated weights for policy 0, policy_version 242748 (0.0006) [2024-09-30 04:09:28,599][1157819] Updated weights for policy 0, policy_version 242758 (0.0006) [2024-09-30 04:09:29,090][1157819] Updated weights for policy 0, policy_version 242768 (0.0006) [2024-09-30 04:09:29,602][1157819] Updated weights for policy 0, policy_version 242778 (0.0006) [2024-09-30 04:09:30,105][1157819] Updated weights for policy 0, policy_version 242788 (0.0006) [2024-09-30 04:09:30,466][1157520] Fps is (10 sec: 80281.2, 60 sec: 77550.8, 300 sec: 76824.3). Total num frames: 994484224. Throughput: 0: 19301.4. Samples: 238588196. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 04:09:30,466][1157520] Avg episode reward: [(0, '58.644')] [2024-09-30 04:09:30,617][1157819] Updated weights for policy 0, policy_version 242798 (0.0006) [2024-09-30 04:09:31,151][1157819] Updated weights for policy 0, policy_version 242808 (0.0006) [2024-09-30 04:09:31,671][1157819] Updated weights for policy 0, policy_version 242818 (0.0006) [2024-09-30 04:09:32,172][1157819] Updated weights for policy 0, policy_version 242828 (0.0006) [2024-09-30 04:09:32,669][1157819] Updated weights for policy 0, policy_version 242838 (0.0006) [2024-09-30 04:09:33,209][1157819] Updated weights for policy 0, policy_version 242848 (0.0006) [2024-09-30 04:09:33,709][1157819] Updated weights for policy 0, policy_version 242858 (0.0006) [2024-09-30 04:09:34,235][1157819] Updated weights for policy 0, policy_version 242868 (0.0006) [2024-09-30 04:09:34,736][1157819] Updated weights for policy 0, policy_version 242878 (0.0006) [2024-09-30 04:09:35,240][1157819] Updated weights for policy 0, policy_version 242888 (0.0006) [2024-09-30 04:09:35,466][1157520] Fps is (10 sec: 79871.7, 60 sec: 77619.1, 300 sec: 76810.4). Total num frames: 994885632. Throughput: 0: 19363.6. Samples: 238707992. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 04:09:35,466][1157520] Avg episode reward: [(0, '57.689')] [2024-09-30 04:09:35,769][1157819] Updated weights for policy 0, policy_version 242898 (0.0006) [2024-09-30 04:09:36,286][1157819] Updated weights for policy 0, policy_version 242908 (0.0006) [2024-09-30 04:09:36,777][1157819] Updated weights for policy 0, policy_version 242918 (0.0006) [2024-09-30 04:09:37,307][1157819] Updated weights for policy 0, policy_version 242928 (0.0006) [2024-09-30 04:09:37,826][1157819] Updated weights for policy 0, policy_version 242938 (0.0006) [2024-09-30 04:09:38,339][1157819] Updated weights for policy 0, policy_version 242948 (0.0006) [2024-09-30 04:09:38,849][1157819] Updated weights for policy 0, policy_version 242958 (0.0006) [2024-09-30 04:09:39,364][1157819] Updated weights for policy 0, policy_version 242968 (0.0006) [2024-09-30 04:09:39,877][1157819] Updated weights for policy 0, policy_version 242978 (0.0006) [2024-09-30 04:09:40,391][1157819] Updated weights for policy 0, policy_version 242988 (0.0006) [2024-09-30 04:09:40,466][1157520] Fps is (10 sec: 79872.1, 60 sec: 77687.4, 300 sec: 76810.4). Total num frames: 995282944. Throughput: 0: 19367.8. Samples: 238767732. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 04:09:40,466][1157520] Avg episode reward: [(0, '55.629')] [2024-09-30 04:09:40,893][1157819] Updated weights for policy 0, policy_version 242998 (0.0006) [2024-09-30 04:09:41,432][1157819] Updated weights for policy 0, policy_version 243008 (0.0006) [2024-09-30 04:09:41,927][1157819] Updated weights for policy 0, policy_version 243018 (0.0006) [2024-09-30 04:09:42,460][1157819] Updated weights for policy 0, policy_version 243028 (0.0006) [2024-09-30 04:09:42,989][1157819] Updated weights for policy 0, policy_version 243038 (0.0006) [2024-09-30 04:09:43,499][1157819] Updated weights for policy 0, policy_version 243048 (0.0006) [2024-09-30 04:09:43,997][1157819] Updated weights for policy 0, policy_version 243058 (0.0006) [2024-09-30 04:09:44,528][1157819] Updated weights for policy 0, policy_version 243068 (0.0006) [2024-09-30 04:09:45,057][1157819] Updated weights for policy 0, policy_version 243078 (0.0006) [2024-09-30 04:09:45,466][1157520] Fps is (10 sec: 79462.5, 60 sec: 77960.7, 300 sec: 76768.7). Total num frames: 995680256. Throughput: 0: 19510.8. Samples: 238886960. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 04:09:45,466][1157520] Avg episode reward: [(0, '57.628')] [2024-09-30 04:09:45,592][1157819] Updated weights for policy 0, policy_version 243088 (0.0006) [2024-09-30 04:09:46,080][1157819] Updated weights for policy 0, policy_version 243098 (0.0006) [2024-09-30 04:09:46,578][1157819] Updated weights for policy 0, policy_version 243108 (0.0006) [2024-09-30 04:09:47,089][1157819] Updated weights for policy 0, policy_version 243118 (0.0006) [2024-09-30 04:09:47,601][1157819] Updated weights for policy 0, policy_version 243128 (0.0006) [2024-09-30 04:09:48,083][1157736] Signal inference workers to stop experience collection... (17350 times) [2024-09-30 04:09:48,087][1157736] Signal inference workers to resume experience collection... (17350 times) [2024-09-30 04:09:48,090][1157819] InferenceWorker_p0-w0: stopping experience collection (17350 times) [2024-09-30 04:09:48,093][1157819] InferenceWorker_p0-w0: resuming experience collection (17350 times) [2024-09-30 04:09:48,101][1157819] Updated weights for policy 0, policy_version 243138 (0.0006) [2024-09-30 04:09:48,597][1157819] Updated weights for policy 0, policy_version 243148 (0.0006) [2024-09-30 04:09:49,181][1157819] Updated weights for policy 0, policy_version 243158 (0.0006) [2024-09-30 04:09:49,717][1157819] Updated weights for policy 0, policy_version 243168 (0.0006) [2024-09-30 04:09:50,268][1157819] Updated weights for policy 0, policy_version 243178 (0.0006) [2024-09-30 04:09:50,466][1157520] Fps is (10 sec: 78643.6, 60 sec: 77892.4, 300 sec: 76727.1). Total num frames: 996069376. Throughput: 0: 19642.0. Samples: 239005212. Policy #0 lag: (min: 0.0, avg: 2.3, max: 5.0) [2024-09-30 04:09:50,466][1157520] Avg episode reward: [(0, '57.582')] [2024-09-30 04:09:50,814][1157819] Updated weights for policy 0, policy_version 243188 (0.0006) [2024-09-30 04:09:51,383][1157819] Updated weights for policy 0, policy_version 243198 (0.0006) [2024-09-30 04:09:51,962][1157819] Updated weights for policy 0, policy_version 243208 (0.0006) [2024-09-30 04:09:52,548][1157819] Updated weights for policy 0, policy_version 243218 (0.0006) [2024-09-30 04:09:53,103][1157819] Updated weights for policy 0, policy_version 243228 (0.0006) [2024-09-30 04:09:53,692][1157819] Updated weights for policy 0, policy_version 243238 (0.0006) [2024-09-30 04:09:54,283][1157819] Updated weights for policy 0, policy_version 243248 (0.0006) [2024-09-30 04:09:54,857][1157819] Updated weights for policy 0, policy_version 243258 (0.0006) [2024-09-30 04:09:55,466][1157520] Fps is (10 sec: 74136.5, 60 sec: 77209.6, 300 sec: 76532.6). Total num frames: 996421632. Throughput: 0: 19593.0. Samples: 239059732. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 04:09:55,466][1157520] Avg episode reward: [(0, '54.891')] [2024-09-30 04:09:55,495][1157819] Updated weights for policy 0, policy_version 243268 (0.0006) [2024-09-30 04:09:56,047][1157819] Updated weights for policy 0, policy_version 243278 (0.0006) [2024-09-30 04:09:56,641][1157819] Updated weights for policy 0, policy_version 243288 (0.0006) [2024-09-30 04:09:57,193][1157819] Updated weights for policy 0, policy_version 243298 (0.0006) [2024-09-30 04:09:57,796][1157819] Updated weights for policy 0, policy_version 243308 (0.0006) [2024-09-30 04:09:58,335][1157819] Updated weights for policy 0, policy_version 243318 (0.0006) [2024-09-30 04:09:58,880][1157819] Updated weights for policy 0, policy_version 243328 (0.0006) [2024-09-30 04:09:59,412][1157819] Updated weights for policy 0, policy_version 243338 (0.0006) [2024-09-30 04:09:59,958][1157819] Updated weights for policy 0, policy_version 243348 (0.0006) [2024-09-30 04:10:00,466][1157520] Fps is (10 sec: 72089.5, 60 sec: 77209.7, 300 sec: 76421.6). Total num frames: 996790272. Throughput: 0: 19428.7. Samples: 239166052. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 04:10:00,466][1157520] Avg episode reward: [(0, '56.224')] [2024-09-30 04:10:00,484][1157819] Updated weights for policy 0, policy_version 243358 (0.0006) [2024-09-30 04:10:01,074][1157819] Updated weights for policy 0, policy_version 243368 (0.0006) [2024-09-30 04:10:01,610][1157819] Updated weights for policy 0, policy_version 243378 (0.0006) [2024-09-30 04:10:02,126][1157819] Updated weights for policy 0, policy_version 243388 (0.0006) [2024-09-30 04:10:02,676][1157819] Updated weights for policy 0, policy_version 243398 (0.0006) [2024-09-30 04:10:03,213][1157819] Updated weights for policy 0, policy_version 243408 (0.0006) [2024-09-30 04:10:03,765][1157819] Updated weights for policy 0, policy_version 243418 (0.0006) [2024-09-30 04:10:04,314][1157819] Updated weights for policy 0, policy_version 243428 (0.0006) [2024-09-30 04:10:04,882][1157819] Updated weights for policy 0, policy_version 243438 (0.0006) [2024-09-30 04:10:05,400][1157819] Updated weights for policy 0, policy_version 243448 (0.0006) [2024-09-30 04:10:05,466][1157520] Fps is (10 sec: 74548.4, 60 sec: 77209.7, 300 sec: 76366.1). Total num frames: 997167104. Throughput: 0: 19373.5. Samples: 239279216. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 04:10:05,466][1157520] Avg episode reward: [(0, '56.834')] [2024-09-30 04:10:05,471][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000243449_997167104.pth... [2024-09-30 04:10:05,518][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000238941_978702336.pth [2024-09-30 04:10:05,909][1157819] Updated weights for policy 0, policy_version 243458 (0.0006) [2024-09-30 04:10:06,424][1157819] Updated weights for policy 0, policy_version 243468 (0.0006) [2024-09-30 04:10:06,946][1157819] Updated weights for policy 0, policy_version 243478 (0.0006) [2024-09-30 04:10:07,461][1157819] Updated weights for policy 0, policy_version 243488 (0.0006) [2024-09-30 04:10:07,964][1157819] Updated weights for policy 0, policy_version 243498 (0.0006) [2024-09-30 04:10:08,461][1157819] Updated weights for policy 0, policy_version 243508 (0.0006) [2024-09-30 04:10:08,961][1157819] Updated weights for policy 0, policy_version 243518 (0.0006) [2024-09-30 04:10:09,427][1157819] Updated weights for policy 0, policy_version 243528 (0.0006) [2024-09-30 04:10:09,901][1157819] Updated weights for policy 0, policy_version 243538 (0.0006) [2024-09-30 04:10:10,394][1157819] Updated weights for policy 0, policy_version 243548 (0.0006) [2024-09-30 04:10:10,466][1157520] Fps is (10 sec: 78643.2, 60 sec: 77755.9, 300 sec: 76491.1). Total num frames: 997576704. Throughput: 0: 19352.9. Samples: 239338408. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 04:10:10,466][1157520] Avg episode reward: [(0, '56.643')] [2024-09-30 04:10:10,859][1157819] Updated weights for policy 0, policy_version 243558 (0.0006) [2024-09-30 04:10:11,364][1157819] Updated weights for policy 0, policy_version 243568 (0.0006) [2024-09-30 04:10:11,821][1157819] Updated weights for policy 0, policy_version 243578 (0.0006) [2024-09-30 04:10:12,316][1157819] Updated weights for policy 0, policy_version 243588 (0.0006) [2024-09-30 04:10:12,804][1157819] Updated weights for policy 0, policy_version 243598 (0.0006) [2024-09-30 04:10:13,292][1157819] Updated weights for policy 0, policy_version 243608 (0.0006) [2024-09-30 04:10:13,776][1157819] Updated weights for policy 0, policy_version 243618 (0.0006) [2024-09-30 04:10:14,289][1157819] Updated weights for policy 0, policy_version 243628 (0.0006) [2024-09-30 04:10:14,777][1157819] Updated weights for policy 0, policy_version 243638 (0.0006) [2024-09-30 04:10:15,286][1157819] Updated weights for policy 0, policy_version 243648 (0.0006) [2024-09-30 04:10:15,466][1157520] Fps is (10 sec: 82738.6, 60 sec: 78370.0, 300 sec: 76616.0). Total num frames: 997994496. Throughput: 0: 19489.9. Samples: 239465240. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 04:10:15,466][1157520] Avg episode reward: [(0, '56.528')] [2024-09-30 04:10:15,775][1157819] Updated weights for policy 0, policy_version 243658 (0.0006) [2024-09-30 04:10:16,276][1157819] Updated weights for policy 0, policy_version 243668 (0.0006) [2024-09-30 04:10:16,780][1157819] Updated weights for policy 0, policy_version 243678 (0.0006) [2024-09-30 04:10:17,340][1157819] Updated weights for policy 0, policy_version 243688 (0.0006) [2024-09-30 04:10:17,917][1157819] Updated weights for policy 0, policy_version 243698 (0.0006) [2024-09-30 04:10:18,427][1157819] Updated weights for policy 0, policy_version 243708 (0.0006) [2024-09-30 04:10:18,951][1157819] Updated weights for policy 0, policy_version 243718 (0.0006) [2024-09-30 04:10:19,494][1157819] Updated weights for policy 0, policy_version 243728 (0.0006) [2024-09-30 04:10:20,027][1157819] Updated weights for policy 0, policy_version 243738 (0.0006) [2024-09-30 04:10:20,466][1157520] Fps is (10 sec: 80691.4, 60 sec: 78370.2, 300 sec: 76657.7). Total num frames: 998383616. Throughput: 0: 19442.5. Samples: 239582904. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 04:10:20,466][1157520] Avg episode reward: [(0, '56.754')] [2024-09-30 04:10:20,566][1157819] Updated weights for policy 0, policy_version 243748 (0.0006) [2024-09-30 04:10:21,103][1157819] Updated weights for policy 0, policy_version 243758 (0.0006) [2024-09-30 04:10:21,631][1157819] Updated weights for policy 0, policy_version 243768 (0.0006) [2024-09-30 04:10:22,179][1157819] Updated weights for policy 0, policy_version 243778 (0.0006) [2024-09-30 04:10:22,697][1157819] Updated weights for policy 0, policy_version 243788 (0.0006) [2024-09-30 04:10:22,874][1157736] Signal inference workers to stop experience collection... (17400 times) [2024-09-30 04:10:22,875][1157736] Signal inference workers to resume experience collection... (17400 times) [2024-09-30 04:10:22,880][1157819] InferenceWorker_p0-w0: stopping experience collection (17400 times) [2024-09-30 04:10:22,880][1157819] InferenceWorker_p0-w0: resuming experience collection (17400 times) [2024-09-30 04:10:23,256][1157819] Updated weights for policy 0, policy_version 243798 (0.0006) [2024-09-30 04:10:23,796][1157819] Updated weights for policy 0, policy_version 243808 (0.0006) [2024-09-30 04:10:24,334][1157819] Updated weights for policy 0, policy_version 243818 (0.0006) [2024-09-30 04:10:24,871][1157819] Updated weights for policy 0, policy_version 243828 (0.0006) [2024-09-30 04:10:25,366][1157819] Updated weights for policy 0, policy_version 243838 (0.0006) [2024-09-30 04:10:25,466][1157520] Fps is (10 sec: 77004.5, 60 sec: 77960.4, 300 sec: 76643.8). Total num frames: 998764544. Throughput: 0: 19395.2. Samples: 239640516. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 04:10:25,466][1157520] Avg episode reward: [(0, '58.377')] [2024-09-30 04:10:25,893][1157819] Updated weights for policy 0, policy_version 243848 (0.0006) [2024-09-30 04:10:26,407][1157819] Updated weights for policy 0, policy_version 243858 (0.0006) [2024-09-30 04:10:26,953][1157819] Updated weights for policy 0, policy_version 243868 (0.0006) [2024-09-30 04:10:27,478][1157819] Updated weights for policy 0, policy_version 243878 (0.0006) [2024-09-30 04:10:27,984][1157819] Updated weights for policy 0, policy_version 243888 (0.0006) [2024-09-30 04:10:28,504][1157819] Updated weights for policy 0, policy_version 243898 (0.0006) [2024-09-30 04:10:29,034][1157819] Updated weights for policy 0, policy_version 243908 (0.0006) [2024-09-30 04:10:29,538][1157819] Updated weights for policy 0, policy_version 243918 (0.0006) [2024-09-30 04:10:30,057][1157819] Updated weights for policy 0, policy_version 243928 (0.0006) [2024-09-30 04:10:30,466][1157520] Fps is (10 sec: 77004.4, 60 sec: 77824.0, 300 sec: 76782.6). Total num frames: 999153664. Throughput: 0: 19336.5. Samples: 239757104. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 04:10:30,466][1157520] Avg episode reward: [(0, '57.877')] [2024-09-30 04:10:30,614][1157819] Updated weights for policy 0, policy_version 243938 (0.0006) [2024-09-30 04:10:31,139][1157819] Updated weights for policy 0, policy_version 243948 (0.0006) [2024-09-30 04:10:31,717][1157819] Updated weights for policy 0, policy_version 243958 (0.0006) [2024-09-30 04:10:32,284][1157819] Updated weights for policy 0, policy_version 243968 (0.0006) [2024-09-30 04:10:32,859][1157819] Updated weights for policy 0, policy_version 243978 (0.0006) [2024-09-30 04:10:33,382][1157819] Updated weights for policy 0, policy_version 243988 (0.0006) [2024-09-30 04:10:33,896][1157819] Updated weights for policy 0, policy_version 243998 (0.0006) [2024-09-30 04:10:34,477][1157819] Updated weights for policy 0, policy_version 244008 (0.0006) [2024-09-30 04:10:35,010][1157819] Updated weights for policy 0, policy_version 244018 (0.0006) [2024-09-30 04:10:35,466][1157520] Fps is (10 sec: 76186.2, 60 sec: 77346.1, 300 sec: 76935.4). Total num frames: 999526400. Throughput: 0: 19215.4. Samples: 239869908. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 04:10:35,466][1157520] Avg episode reward: [(0, '57.458')] [2024-09-30 04:10:35,586][1157819] Updated weights for policy 0, policy_version 244028 (0.0006) [2024-09-30 04:10:36,138][1157819] Updated weights for policy 0, policy_version 244038 (0.0006) [2024-09-30 04:10:36,692][1157819] Updated weights for policy 0, policy_version 244048 (0.0006) [2024-09-30 04:10:37,276][1157819] Updated weights for policy 0, policy_version 244058 (0.0006) [2024-09-30 04:10:37,832][1157819] Updated weights for policy 0, policy_version 244068 (0.0006) [2024-09-30 04:10:38,408][1157819] Updated weights for policy 0, policy_version 244078 (0.0006) [2024-09-30 04:10:39,008][1157819] Updated weights for policy 0, policy_version 244088 (0.0006) [2024-09-30 04:10:39,613][1157819] Updated weights for policy 0, policy_version 244098 (0.0006) [2024-09-30 04:10:40,208][1157819] Updated weights for policy 0, policy_version 244108 (0.0006) [2024-09-30 04:10:40,447][1157736] Signal inference workers to stop experience collection... (17450 times) [2024-09-30 04:10:40,447][1157736] Signal inference workers to resume experience collection... (17450 times) [2024-09-30 04:10:40,452][1157819] InferenceWorker_p0-w0: stopping experience collection (17450 times) [2024-09-30 04:10:40,452][1157819] InferenceWorker_p0-w0: resuming experience collection (17450 times) [2024-09-30 04:10:40,466][1157520] Fps is (10 sec: 72499.6, 60 sec: 76595.3, 300 sec: 76949.3). Total num frames: 999878656. Throughput: 0: 19228.4. Samples: 239925004. Policy #0 lag: (min: 0.0, avg: 2.1, max: 5.0) [2024-09-30 04:10:40,466][1157520] Avg episode reward: [(0, '55.462')] [2024-09-30 04:10:40,840][1157819] Updated weights for policy 0, policy_version 244118 (0.0006) [2024-09-30 04:10:41,496][1157819] Updated weights for policy 0, policy_version 244128 (0.0006) [2024-09-30 04:10:42,130][1157819] Updated weights for policy 0, policy_version 244138 (0.0006) [2024-09-30 04:10:42,362][1157520] Component Batcher_0 stopped! [2024-09-30 04:10:42,362][1157736] Stopping Batcher_0... [2024-09-30 04:10:42,362][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000244142_1000005632.pth... [2024-09-30 04:10:42,363][1157736] Loop batcher_evt_loop terminating... [2024-09-30 04:10:42,377][1157819] Weights refcount: 2 0 [2024-09-30 04:10:42,379][1157819] Stopping InferenceWorker_p0-w0... [2024-09-30 04:10:42,379][1157819] Loop inference_proc0-0_evt_loop terminating... [2024-09-30 04:10:42,379][1157520] Component InferenceWorker_p0-w0 stopped! [2024-09-30 04:10:42,417][1157736] Removing /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000241198_987947008.pth [2024-09-30 04:10:42,426][1157736] Saving /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000244142_1000005632.pth... [2024-09-30 04:10:42,539][1157736] Stopping LearnerWorker_p0... [2024-09-30 04:10:42,539][1157520] Component LearnerWorker_p0 stopped! [2024-09-30 04:10:42,539][1157736] Loop learner_proc0_evt_loop terminating... [2024-09-30 04:10:42,661][1157828] Stopping RolloutWorker_w9... [2024-09-30 04:10:42,661][1157520] Component RolloutWorker_w9 stopped! [2024-09-30 04:10:42,661][1157828] Loop rollout_proc9_evt_loop terminating... [2024-09-30 04:10:42,661][1157520] Component RolloutWorker_w1 stopped! [2024-09-30 04:10:42,661][1157820] Stopping RolloutWorker_w1... [2024-09-30 04:10:42,662][1157820] Loop rollout_proc1_evt_loop terminating... [2024-09-30 04:10:42,665][1157520] Component RolloutWorker_w11 stopped! [2024-09-30 04:10:42,665][1157894] Stopping RolloutWorker_w11... [2024-09-30 04:10:42,665][1157894] Loop rollout_proc11_evt_loop terminating... [2024-09-30 04:10:42,666][1157520] Component RolloutWorker_w3 stopped! [2024-09-30 04:10:42,666][1157823] Stopping RolloutWorker_w3... [2024-09-30 04:10:42,666][1157823] Loop rollout_proc3_evt_loop terminating... [2024-09-30 04:10:42,686][1157897] Stopping RolloutWorker_w12... [2024-09-30 04:10:42,687][1157520] Component RolloutWorker_w12 stopped! [2024-09-30 04:10:42,687][1157897] Loop rollout_proc12_evt_loop terminating... [2024-09-30 04:10:42,688][1157520] Component RolloutWorker_w4 stopped! [2024-09-30 04:10:42,688][1157825] Stopping RolloutWorker_w4... [2024-09-30 04:10:42,688][1157825] Loop rollout_proc4_evt_loop terminating... [2024-09-30 04:10:42,723][1157822] Stopping RolloutWorker_w2... [2024-09-30 04:10:42,723][1157520] Component RolloutWorker_w2 stopped! [2024-09-30 04:10:42,724][1157822] Loop rollout_proc2_evt_loop terminating... [2024-09-30 04:10:42,724][1157520] Component RolloutWorker_w10 stopped! [2024-09-30 04:10:42,724][1157862] Stopping RolloutWorker_w10... [2024-09-30 04:10:42,725][1157862] Loop rollout_proc10_evt_loop terminating... [2024-09-30 04:10:42,728][1157520] Component RolloutWorker_w0 stopped! [2024-09-30 04:10:42,728][1157821] Stopping RolloutWorker_w0... [2024-09-30 04:10:42,728][1157821] Loop rollout_proc0_evt_loop terminating... [2024-09-30 04:10:42,729][1157520] Component RolloutWorker_w8 stopped! [2024-09-30 04:10:42,729][1157829] Stopping RolloutWorker_w8... [2024-09-30 04:10:42,730][1157829] Loop rollout_proc8_evt_loop terminating... [2024-09-30 04:10:42,743][1157520] Component RolloutWorker_w13 stopped! [2024-09-30 04:10:42,743][1157895] Stopping RolloutWorker_w13... [2024-09-30 04:10:42,744][1157895] Loop rollout_proc13_evt_loop terminating... [2024-09-30 04:10:42,746][1157520] Component RolloutWorker_w5 stopped! [2024-09-30 04:10:42,746][1157826] Stopping RolloutWorker_w5... [2024-09-30 04:10:42,747][1157826] Loop rollout_proc5_evt_loop terminating... [2024-09-30 04:10:42,749][1157520] Component RolloutWorker_w6 stopped! [2024-09-30 04:10:42,749][1157824] Stopping RolloutWorker_w6... [2024-09-30 04:10:42,749][1157520] Component RolloutWorker_w14 stopped! [2024-09-30 04:10:42,749][1157896] Stopping RolloutWorker_w14... [2024-09-30 04:10:42,749][1157824] Loop rollout_proc6_evt_loop terminating... [2024-09-30 04:10:42,749][1157896] Loop rollout_proc14_evt_loop terminating... [2024-09-30 04:10:42,761][1157520] Component RolloutWorker_w15 stopped! [2024-09-30 04:10:42,761][1158936] Stopping RolloutWorker_w15... [2024-09-30 04:10:42,762][1158936] Loop rollout_proc15_evt_loop terminating... [2024-09-30 04:10:42,762][1157520] Component RolloutWorker_w7 stopped! [2024-09-30 04:10:42,762][1157827] Stopping RolloutWorker_w7... [2024-09-30 04:10:42,762][1157520] Waiting for process learner_proc0 to stop... [2024-09-30 04:10:42,762][1157827] Loop rollout_proc7_evt_loop terminating... [2024-09-30 04:10:43,278][1157520] Waiting for process inference_proc0-0 to join... [2024-09-30 04:10:43,279][1157520] Waiting for process rollout_proc0 to join... [2024-09-30 04:10:43,279][1157520] Waiting for process rollout_proc1 to join... [2024-09-30 04:10:43,280][1157520] Waiting for process rollout_proc2 to join... [2024-09-30 04:10:43,280][1157520] Waiting for process rollout_proc3 to join... [2024-09-30 04:10:43,280][1157520] Waiting for process rollout_proc4 to join... [2024-09-30 04:10:43,280][1157520] Waiting for process rollout_proc5 to join... [2024-09-30 04:10:43,280][1157520] Waiting for process rollout_proc6 to join... [2024-09-30 04:10:43,281][1157520] Waiting for process rollout_proc7 to join... [2024-09-30 04:10:43,281][1157520] Waiting for process rollout_proc8 to join... [2024-09-30 04:10:43,281][1157520] Waiting for process rollout_proc9 to join... [2024-09-30 04:10:43,281][1157520] Waiting for process rollout_proc10 to join... [2024-09-30 04:10:43,282][1157520] Waiting for process rollout_proc11 to join... [2024-09-30 04:10:43,282][1157520] Waiting for process rollout_proc12 to join... [2024-09-30 04:10:43,282][1157520] Waiting for process rollout_proc13 to join... [2024-09-30 04:10:43,282][1157520] Waiting for process rollout_proc14 to join... [2024-09-30 04:10:43,283][1157520] Waiting for process rollout_proc15 to join... [2024-09-30 04:10:43,283][1157520] Batcher 0 profile tree view: batching: 2098.3580, releasing_batches: 6.7606 [2024-09-30 04:10:43,283][1157520] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0000 wait_policy_total: 273.4355 update_model: 235.1296 weight_update: 0.0006 one_step: 0.0015 handle_policy_step: 11373.3193 deserialize: 781.5053, stack: 53.1942, obs_to_device_normalize: 2669.4259, forward: 5352.4678, send_messages: 690.2780 prepare_outputs: 1372.6157 to_cpu: 718.1548 [2024-09-30 04:10:43,283][1157520] Learner 0 profile tree view: misc: 0.8356, prepare_batch: 875.0559 train: 2675.5494 epoch_init: 1.0017, minibatch_init: 0.9888, losses_postprocess: 47.5973, kl_divergence: 52.3967, after_optimizer: 17.2970 calculate_losses: 1083.6753 losses_init: 0.6589, forward_head: 98.5676, bptt_initial: 479.6136, tail: 95.1800, advantages_returns: 26.4987, losses: 181.0587 bptt: 170.3284 bptt_forward_core: 160.8853 update: 1408.5590 clip: 124.0144 [2024-09-30 04:10:43,283][1157520] RolloutWorker_w0 profile tree view: wait_for_trajectories: 8.8322, enqueue_policy_requests: 446.1999, env_step: 8971.4169, overhead: 431.8903, complete_rollouts: 8.5697 save_policy_outputs: 688.0404 split_output_tensors: 228.4173 [2024-09-30 04:10:43,283][1157520] RolloutWorker_w15 profile tree view: wait_for_trajectories: 8.8289, enqueue_policy_requests: 451.9754, env_step: 9123.8896, overhead: 434.6786, complete_rollouts: 8.7613 save_policy_outputs: 695.1800 split_output_tensors: 228.1163 [2024-09-30 04:10:43,284][1157520] Loop Runner_EvtLoop terminating... [2024-09-30 04:10:43,284][1157520] Runner profile tree view: main_loop: 12396.0029 [2024-09-30 04:10:43,284][1157520] Collected {0: 1000005632}, FPS: 77444.0 [2024-09-30 04:10:43,478][1157520] Loading existing experiment configuration from /home/luyang/workspace/rl/train_dir/default_experiment/config.json [2024-09-30 04:10:43,478][1157520] Overriding arg 'num_workers' with value 1 passed from command line [2024-09-30 04:10:43,478][1157520] Adding new argument 'no_render'=True that is not in the saved config file! [2024-09-30 04:10:43,479][1157520] Adding new argument 'save_video'=True that is not in the saved config file! [2024-09-30 04:10:43,479][1157520] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2024-09-30 04:10:43,479][1157520] Adding new argument 'video_name'=None that is not in the saved config file! [2024-09-30 04:10:43,479][1157520] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! [2024-09-30 04:10:43,479][1157520] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2024-09-30 04:10:43,479][1157520] Adding new argument 'push_to_hub'=True that is not in the saved config file! [2024-09-30 04:10:43,479][1157520] Adding new argument 'hf_repository'='esperesa/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! [2024-09-30 04:10:43,479][1157520] Adding new argument 'policy_index'=0 that is not in the saved config file! [2024-09-30 04:10:43,479][1157520] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2024-09-30 04:10:43,479][1157520] Adding new argument 'train_script'=None that is not in the saved config file! [2024-09-30 04:10:43,479][1157520] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2024-09-30 04:10:43,479][1157520] Using frameskip 1 and render_action_repeat=4 for evaluation [2024-09-30 04:10:43,497][1157520] Doom resolution: 160x120, resize resolution: (128, 72) [2024-09-30 04:10:43,498][1157520] RunningMeanStd input shape: (3, 72, 128) [2024-09-30 04:10:43,498][1157520] RunningMeanStd input shape: (1,) [2024-09-30 04:10:43,506][1157520] ConvEncoder: input_channels=3 [2024-09-30 04:10:43,571][1157520] Conv encoder output size: 512 [2024-09-30 04:10:43,571][1157520] Policy head output size: 512 [2024-09-30 04:10:43,715][1157520] Loading state from checkpoint /home/luyang/workspace/rl/train_dir/default_experiment/checkpoint_p0/checkpoint_000244142_1000005632.pth... [2024-09-30 04:10:44,298][1157520] Num frames 100... [2024-09-30 04:10:44,387][1157520] Num frames 200... [2024-09-30 04:10:44,475][1157520] Num frames 300... [2024-09-30 04:10:44,563][1157520] Num frames 400... [2024-09-30 04:10:44,651][1157520] Num frames 500... [2024-09-30 04:10:44,740][1157520] Num frames 600... [2024-09-30 04:10:44,830][1157520] Num frames 700... [2024-09-30 04:10:44,920][1157520] Num frames 800... [2024-09-30 04:10:45,008][1157520] Num frames 900... [2024-09-30 04:10:45,096][1157520] Num frames 1000... [2024-09-30 04:10:45,187][1157520] Num frames 1100... [2024-09-30 04:10:45,277][1157520] Num frames 1200... [2024-09-30 04:10:45,367][1157520] Num frames 1300... [2024-09-30 04:10:45,456][1157520] Num frames 1400... [2024-09-30 04:10:45,545][1157520] Num frames 1500... [2024-09-30 04:10:45,634][1157520] Num frames 1600... [2024-09-30 04:10:45,724][1157520] Num frames 1700... [2024-09-30 04:10:45,815][1157520] Num frames 1800... [2024-09-30 04:10:45,904][1157520] Num frames 1900... [2024-09-30 04:10:45,995][1157520] Num frames 2000... [2024-09-30 04:10:46,089][1157520] Num frames 2100... [2024-09-30 04:10:46,141][1157520] Avg episode rewards: #0: 60.999, true rewards: #0: 21.000 [2024-09-30 04:10:46,141][1157520] Avg episode reward: 60.999, avg true_objective: 21.000 [2024-09-30 04:10:46,247][1157520] Num frames 2200... [2024-09-30 04:10:46,337][1157520] Num frames 2300... [2024-09-30 04:10:46,426][1157520] Num frames 2400... [2024-09-30 04:10:46,514][1157520] Num frames 2500... [2024-09-30 04:10:46,602][1157520] Num frames 2600... [2024-09-30 04:10:46,692][1157520] Num frames 2700... [2024-09-30 04:10:46,783][1157520] Num frames 2800... [2024-09-30 04:10:46,873][1157520] Num frames 2900... [2024-09-30 04:10:46,963][1157520] Num frames 3000... [2024-09-30 04:10:47,053][1157520] Num frames 3100... [2024-09-30 04:10:47,144][1157520] Num frames 3200... [2024-09-30 04:10:47,233][1157520] Num frames 3300... [2024-09-30 04:10:47,322][1157520] Num frames 3400... [2024-09-30 04:10:47,413][1157520] Num frames 3500... [2024-09-30 04:10:47,503][1157520] Num frames 3600... [2024-09-30 04:10:47,593][1157520] Num frames 3700... [2024-09-30 04:10:47,683][1157520] Num frames 3800... [2024-09-30 04:10:47,771][1157520] Num frames 3900... [2024-09-30 04:10:47,865][1157520] Num frames 4000... [2024-09-30 04:10:47,956][1157520] Num frames 4100... [2024-09-30 04:10:48,049][1157520] Num frames 4200... [2024-09-30 04:10:48,100][1157520] Avg episode rewards: #0: 62.999, true rewards: #0: 21.000 [2024-09-30 04:10:48,101][1157520] Avg episode reward: 62.999, avg true_objective: 21.000 [2024-09-30 04:10:48,207][1157520] Num frames 4300... [2024-09-30 04:10:48,296][1157520] Num frames 4400... [2024-09-30 04:10:48,385][1157520] Num frames 4500... [2024-09-30 04:10:48,473][1157520] Num frames 4600... [2024-09-30 04:10:48,561][1157520] Num frames 4700... [2024-09-30 04:10:48,650][1157520] Num frames 4800... [2024-09-30 04:10:48,739][1157520] Num frames 4900... [2024-09-30 04:10:48,830][1157520] Num frames 5000... [2024-09-30 04:10:48,920][1157520] Num frames 5100... [2024-09-30 04:10:49,010][1157520] Num frames 5200... [2024-09-30 04:10:49,099][1157520] Num frames 5300... [2024-09-30 04:10:49,193][1157520] Num frames 5400... [2024-09-30 04:10:49,283][1157520] Num frames 5500... [2024-09-30 04:10:49,373][1157520] Num frames 5600... [2024-09-30 04:10:49,462][1157520] Num frames 5700... [2024-09-30 04:10:49,552][1157520] Num frames 5800... [2024-09-30 04:10:49,644][1157520] Num frames 5900... [2024-09-30 04:10:49,734][1157520] Num frames 6000... [2024-09-30 04:10:49,824][1157520] Num frames 6100... [2024-09-30 04:10:49,915][1157520] Num frames 6200... [2024-09-30 04:10:50,010][1157520] Num frames 6300... [2024-09-30 04:10:50,060][1157520] Avg episode rewards: #0: 59.999, true rewards: #0: 21.000 [2024-09-30 04:10:50,060][1157520] Avg episode reward: 59.999, avg true_objective: 21.000 [2024-09-30 04:10:50,170][1157520] Num frames 6400... [2024-09-30 04:10:50,259][1157520] Num frames 6500... [2024-09-30 04:10:50,349][1157520] Num frames 6600... [2024-09-30 04:10:50,437][1157520] Num frames 6700... [2024-09-30 04:10:50,525][1157520] Num frames 6800... [2024-09-30 04:10:50,615][1157520] Num frames 6900... [2024-09-30 04:10:50,704][1157520] Num frames 7000... [2024-09-30 04:10:50,793][1157520] Num frames 7100... [2024-09-30 04:10:50,881][1157520] Num frames 7200... [2024-09-30 04:10:50,970][1157520] Num frames 7300... [2024-09-30 04:10:51,059][1157520] Num frames 7400... [2024-09-30 04:10:51,148][1157520] Num frames 7500... [2024-09-30 04:10:51,238][1157520] Num frames 7600... [2024-09-30 04:10:51,329][1157520] Num frames 7700... [2024-09-30 04:10:51,418][1157520] Num frames 7800... [2024-09-30 04:10:51,508][1157520] Num frames 7900... [2024-09-30 04:10:51,600][1157520] Num frames 8000... [2024-09-30 04:10:51,692][1157520] Num frames 8100... [2024-09-30 04:10:51,781][1157520] Num frames 8200... [2024-09-30 04:10:51,872][1157520] Num frames 8300... [2024-09-30 04:10:51,966][1157520] Num frames 8400... [2024-09-30 04:10:52,016][1157520] Avg episode rewards: #0: 59.749, true rewards: #0: 21.000 [2024-09-30 04:10:52,016][1157520] Avg episode reward: 59.749, avg true_objective: 21.000 [2024-09-30 04:10:52,125][1157520] Num frames 8500... [2024-09-30 04:10:52,213][1157520] Num frames 8600... [2024-09-30 04:10:52,302][1157520] Num frames 8700... [2024-09-30 04:10:52,392][1157520] Num frames 8800... [2024-09-30 04:10:52,481][1157520] Num frames 8900... [2024-09-30 04:10:52,569][1157520] Num frames 9000... [2024-09-30 04:10:52,657][1157520] Num frames 9100... [2024-09-30 04:10:52,746][1157520] Num frames 9200... [2024-09-30 04:10:52,835][1157520] Num frames 9300... [2024-09-30 04:10:52,923][1157520] Num frames 9400... [2024-09-30 04:10:53,004][1157520] Avg episode rewards: #0: 52.857, true rewards: #0: 18.858 [2024-09-30 04:10:53,004][1157520] Avg episode reward: 52.857, avg true_objective: 18.858 [2024-09-30 04:10:53,085][1157520] Num frames 9500... [2024-09-30 04:10:53,173][1157520] Num frames 9600... [2024-09-30 04:10:53,260][1157520] Num frames 9700... [2024-09-30 04:10:53,348][1157520] Num frames 9800... [2024-09-30 04:10:53,436][1157520] Num frames 9900... [2024-09-30 04:10:53,525][1157520] Num frames 10000... [2024-09-30 04:10:53,615][1157520] Num frames 10100... [2024-09-30 04:10:53,705][1157520] Num frames 10200... [2024-09-30 04:10:53,794][1157520] Num frames 10300... [2024-09-30 04:10:53,882][1157520] Num frames 10400... [2024-09-30 04:10:53,972][1157520] Num frames 10500... [2024-09-30 04:10:54,060][1157520] Num frames 10600... [2024-09-30 04:10:54,151][1157520] Num frames 10700... [2024-09-30 04:10:54,242][1157520] Num frames 10800... [2024-09-30 04:10:54,333][1157520] Num frames 10900... [2024-09-30 04:10:54,422][1157520] Num frames 11000... [2024-09-30 04:10:54,512][1157520] Num frames 11100... [2024-09-30 04:10:54,604][1157520] Num frames 11200... [2024-09-30 04:10:54,694][1157520] Num frames 11300... [2024-09-30 04:10:54,784][1157520] Num frames 11400... [2024-09-30 04:10:54,874][1157520] Num frames 11500... [2024-09-30 04:10:54,956][1157520] Avg episode rewards: #0: 54.881, true rewards: #0: 19.215 [2024-09-30 04:10:54,956][1157520] Avg episode reward: 54.881, avg true_objective: 19.215 [2024-09-30 04:10:55,040][1157520] Num frames 11600... [2024-09-30 04:10:55,128][1157520] Num frames 11700... [2024-09-30 04:10:55,216][1157520] Num frames 11800... [2024-09-30 04:10:55,304][1157520] Num frames 11900... [2024-09-30 04:10:55,393][1157520] Num frames 12000... [2024-09-30 04:10:55,482][1157520] Num frames 12100... [2024-09-30 04:10:55,572][1157520] Num frames 12200... [2024-09-30 04:10:55,662][1157520] Num frames 12300... [2024-09-30 04:10:55,751][1157520] Num frames 12400... [2024-09-30 04:10:55,838][1157520] Num frames 12500... [2024-09-30 04:10:55,929][1157520] Num frames 12600... [2024-09-30 04:10:56,019][1157520] Num frames 12700... [2024-09-30 04:10:56,108][1157520] Num frames 12800... [2024-09-30 04:10:56,195][1157520] Num frames 12900... [2024-09-30 04:10:56,286][1157520] Num frames 13000... [2024-09-30 04:10:56,376][1157520] Num frames 13100... [2024-09-30 04:10:56,466][1157520] Num frames 13200... [2024-09-30 04:10:56,557][1157520] Num frames 13300... [2024-09-30 04:10:56,648][1157520] Num frames 13400... [2024-09-30 04:10:56,739][1157520] Num frames 13500... [2024-09-30 04:10:56,828][1157520] Num frames 13600... [2024-09-30 04:10:56,909][1157520] Avg episode rewards: #0: 56.469, true rewards: #0: 19.470 [2024-09-30 04:10:56,909][1157520] Avg episode reward: 56.469, avg true_objective: 19.470 [2024-09-30 04:10:56,991][1157520] Num frames 13700... [2024-09-30 04:10:57,080][1157520] Num frames 13800... [2024-09-30 04:10:57,169][1157520] Num frames 13900... [2024-09-30 04:10:57,257][1157520] Num frames 14000... [2024-09-30 04:10:57,345][1157520] Num frames 14100... [2024-09-30 04:10:57,433][1157520] Num frames 14200... [2024-09-30 04:10:57,521][1157520] Num frames 14300... [2024-09-30 04:10:57,610][1157520] Num frames 14400... [2024-09-30 04:10:57,699][1157520] Num frames 14500... [2024-09-30 04:10:57,788][1157520] Num frames 14600... [2024-09-30 04:10:57,879][1157520] Num frames 14700... [2024-09-30 04:10:57,967][1157520] Num frames 14800... [2024-09-30 04:10:58,057][1157520] Num frames 14900... [2024-09-30 04:10:58,146][1157520] Num frames 15000... [2024-09-30 04:10:58,235][1157520] Num frames 15100... [2024-09-30 04:10:58,325][1157520] Num frames 15200... [2024-09-30 04:10:58,414][1157520] Num frames 15300... [2024-09-30 04:10:58,505][1157520] Num frames 15400... [2024-09-30 04:10:58,597][1157520] Num frames 15500... [2024-09-30 04:10:58,667][1157520] Avg episode rewards: #0: 55.770, true rewards: #0: 19.396 [2024-09-30 04:10:58,668][1157520] Avg episode reward: 55.770, avg true_objective: 19.396 [2024-09-30 04:10:58,760][1157520] Num frames 15600... [2024-09-30 04:10:58,848][1157520] Num frames 15700... [2024-09-30 04:10:58,935][1157520] Num frames 15800... [2024-09-30 04:10:59,025][1157520] Num frames 15900... [2024-09-30 04:10:59,113][1157520] Num frames 16000... [2024-09-30 04:10:59,202][1157520] Num frames 16100... [2024-09-30 04:10:59,292][1157520] Num frames 16200... [2024-09-30 04:10:59,381][1157520] Num frames 16300... [2024-09-30 04:10:59,470][1157520] Num frames 16400... [2024-09-30 04:10:59,558][1157520] Num frames 16500... [2024-09-30 04:10:59,650][1157520] Num frames 16600... [2024-09-30 04:10:59,740][1157520] Num frames 16700... [2024-09-30 04:10:59,829][1157520] Num frames 16800... [2024-09-30 04:10:59,919][1157520] Num frames 16900... [2024-09-30 04:11:00,009][1157520] Num frames 17000... [2024-09-30 04:11:00,099][1157520] Num frames 17100... [2024-09-30 04:11:00,190][1157520] Num frames 17200... [2024-09-30 04:11:00,279][1157520] Num frames 17300... [2024-09-30 04:11:00,369][1157520] Num frames 17400... [2024-09-30 04:11:00,458][1157520] Num frames 17500... [2024-09-30 04:11:00,550][1157520] Num frames 17600... [2024-09-30 04:11:00,620][1157520] Avg episode rewards: #0: 56.684, true rewards: #0: 19.574 [2024-09-30 04:11:00,621][1157520] Avg episode reward: 56.684, avg true_objective: 19.574 [2024-09-30 04:11:00,716][1157520] Num frames 17700... [2024-09-30 04:11:00,803][1157520] Num frames 17800... [2024-09-30 04:11:00,891][1157520] Num frames 17900... [2024-09-30 04:11:00,981][1157520] Num frames 18000... [2024-09-30 04:11:01,070][1157520] Num frames 18100... [2024-09-30 04:11:01,157][1157520] Num frames 18200... [2024-09-30 04:11:01,247][1157520] Num frames 18300... [2024-09-30 04:11:01,338][1157520] Num frames 18400... [2024-09-30 04:11:01,428][1157520] Num frames 18500... [2024-09-30 04:11:01,517][1157520] Num frames 18600... [2024-09-30 04:11:01,606][1157520] Num frames 18700... [2024-09-30 04:11:01,696][1157520] Num frames 18800... [2024-09-30 04:11:01,786][1157520] Num frames 18900... [2024-09-30 04:11:01,875][1157520] Num frames 19000... [2024-09-30 04:11:01,964][1157520] Num frames 19100... [2024-09-30 04:11:02,053][1157520] Num frames 19200... [2024-09-30 04:11:02,144][1157520] Num frames 19300... [2024-09-30 04:11:02,234][1157520] Num frames 19400... [2024-09-30 04:11:02,325][1157520] Num frames 19500... [2024-09-30 04:11:02,414][1157520] Num frames 19600... [2024-09-30 04:11:02,504][1157520] Num frames 19700... [2024-09-30 04:11:02,575][1157520] Avg episode rewards: #0: 57.216, true rewards: #0: 19.717 [2024-09-30 04:11:02,575][1157520] Avg episode reward: 57.216, avg true_objective: 19.717 [2024-09-30 04:11:28,490][1157520] Replay video saved to /home/luyang/workspace/rl/train_dir/default_experiment/replay.mp4!