| [2024-12-29 17:05:13,620][45646] Saving configuration to /fsx/users/amzfang/rl_course/train_dir/default_experiment/config.json... | |
| [2024-12-29 17:05:13,625][45646] Rollout worker 0 uses device cpu | |
| [2024-12-29 17:05:13,625][45646] Rollout worker 1 uses device cpu | |
| [2024-12-29 17:05:13,626][45646] Rollout worker 2 uses device cpu | |
| [2024-12-29 17:05:13,626][45646] Rollout worker 3 uses device cpu | |
| [2024-12-29 17:05:13,626][45646] Rollout worker 4 uses device cpu | |
| [2024-12-29 17:05:13,627][45646] Rollout worker 5 uses device cpu | |
| [2024-12-29 17:05:13,627][45646] Rollout worker 6 uses device cpu | |
| [2024-12-29 17:05:13,628][45646] Rollout worker 7 uses device cpu | |
| [2024-12-29 17:05:13,728][45646] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2024-12-29 17:05:13,729][45646] InferenceWorker_p0-w0: min num requests: 2 | |
| [2024-12-29 17:05:13,757][45646] Starting all processes... | |
| [2024-12-29 17:05:13,757][45646] Starting process learner_proc0 | |
| [2024-12-29 17:05:13,878][45646] Starting all processes... | |
| [2024-12-29 17:05:13,909][45646] Starting process inference_proc0-0 | |
| [2024-12-29 17:05:13,909][45646] Starting process rollout_proc0 | |
| [2024-12-29 17:05:13,909][45646] Starting process rollout_proc1 | |
| [2024-12-29 17:05:13,910][45646] Starting process rollout_proc2 | |
| [2024-12-29 17:05:13,910][45646] Starting process rollout_proc3 | |
| [2024-12-29 17:05:13,910][45646] Starting process rollout_proc4 | |
| [2024-12-29 17:05:13,911][45646] Starting process rollout_proc5 | |
| [2024-12-29 17:05:13,911][45646] Starting process rollout_proc6 | |
| [2024-12-29 17:05:13,912][45646] Starting process rollout_proc7 | |
| [2024-12-29 17:05:20,721][48101] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2024-12-29 17:05:20,721][48119] Worker 0 uses CPU cores [0, 1, 2, 3, 4, 5] | |
| [2024-12-29 17:05:20,721][48114] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2024-12-29 17:05:20,721][48115] Worker 3 uses CPU cores [18, 19, 20, 21, 22, 23] | |
| [2024-12-29 17:05:20,721][48116] Worker 1 uses CPU cores [6, 7, 8, 9, 10, 11] | |
| [2024-12-29 17:05:20,721][48117] Worker 2 uses CPU cores [12, 13, 14, 15, 16, 17] | |
| [2024-12-29 17:05:20,721][48118] Worker 4 uses CPU cores [24, 25, 26, 27, 28, 29] | |
| [2024-12-29 17:05:20,721][48122] Worker 7 uses CPU cores [42, 43, 44, 45, 46, 47] | |
| [2024-12-29 17:05:20,721][48121] Worker 6 uses CPU cores [36, 37, 38, 39, 40, 41] | |
| [2024-12-29 17:05:20,721][48120] Worker 5 uses CPU cores [30, 31, 32, 33, 34, 35] | |
| [2024-12-29 17:05:20,722][48101] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 | |
| [2024-12-29 17:05:20,722][48114] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 | |
| [2024-12-29 17:05:20,848][48101] Num visible devices: 1 | |
| [2024-12-29 17:05:20,865][48101] Starting seed is not provided | |
| [2024-12-29 17:05:20,865][48101] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2024-12-29 17:05:20,865][48101] Initializing actor-critic model on device cuda:0 | |
| [2024-12-29 17:05:20,866][48101] RunningMeanStd input shape: (3, 72, 128) | |
| [2024-12-29 17:05:20,867][48101] RunningMeanStd input shape: (1,) | |
| [2024-12-29 17:05:20,881][48114] Num visible devices: 1 | |
| [2024-12-29 17:05:20,886][48101] ConvEncoder: input_channels=3 | |
| [2024-12-29 17:05:21,144][48101] Conv encoder output size: 512 | |
| [2024-12-29 17:05:21,144][48101] Policy head output size: 512 | |
| [2024-12-29 17:05:21,193][48101] Created Actor Critic model with architecture: | |
| [2024-12-29 17:05:21,193][48101] ActorCriticSharedWeights( | |
| (obs_normalizer): ObservationNormalizer( | |
| (running_mean_std): RunningMeanStdDictInPlace( | |
| (running_mean_std): ModuleDict( | |
| (obs): RunningMeanStdInPlace() | |
| ) | |
| ) | |
| ) | |
| (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) | |
| (encoder): VizdoomEncoder( | |
| (basic_encoder): ConvEncoder( | |
| (enc): RecursiveScriptModule( | |
| original_name=ConvEncoderImpl | |
| (conv_head): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Conv2d) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| (2): RecursiveScriptModule(original_name=Conv2d) | |
| (3): RecursiveScriptModule(original_name=ELU) | |
| (4): RecursiveScriptModule(original_name=Conv2d) | |
| (5): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| (mlp_layers): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Linear) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| ) | |
| ) | |
| ) | |
| (core): ModelCoreRNN( | |
| (core): GRU(512, 512) | |
| ) | |
| (decoder): MlpDecoder( | |
| (mlp): Identity() | |
| ) | |
| (critic_linear): Linear(in_features=512, out_features=1, bias=True) | |
| (action_parameterization): ActionParameterizationDefault( | |
| (distribution_linear): Linear(in_features=512, out_features=5, bias=True) | |
| ) | |
| ) | |
| [2024-12-29 17:05:21,762][48101] Using optimizer <class 'torch.optim.adam.Adam'> | |
| [2024-12-29 17:05:27,792][48101] No checkpoints found | |
| [2024-12-29 17:05:27,793][48101] Did not load from checkpoint, starting from scratch! | |
| [2024-12-29 17:05:27,794][48101] Initialized policy 0 weights for model version 0 | |
| [2024-12-29 17:05:27,797][48101] LearnerWorker_p0 finished initialization! | |
| [2024-12-29 17:05:27,797][48101] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2024-12-29 17:05:28,222][48114] RunningMeanStd input shape: (3, 72, 128) | |
| [2024-12-29 17:05:28,223][48114] RunningMeanStd input shape: (1,) | |
| [2024-12-29 17:05:28,235][48114] ConvEncoder: input_channels=3 | |
| [2024-12-29 17:05:28,331][48114] Conv encoder output size: 512 | |
| [2024-12-29 17:05:28,331][48114] Policy head output size: 512 | |
| [2024-12-29 17:05:28,372][45646] Inference worker 0-0 is ready! | |
| [2024-12-29 17:05:28,373][45646] All inference workers are ready! Signal rollout workers to start! | |
| [2024-12-29 17:05:28,403][48117] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-12-29 17:05:28,403][48119] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-12-29 17:05:28,403][48118] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-12-29 17:05:28,403][48120] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-12-29 17:05:28,404][48115] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-12-29 17:05:28,404][48116] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-12-29 17:05:28,404][48121] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-12-29 17:05:28,404][48122] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-12-29 17:05:28,732][45646] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2024-12-29 17:05:28,833][48116] Decorrelating experience for 0 frames... | |
| [2024-12-29 17:05:28,995][48118] Decorrelating experience for 0 frames... | |
| [2024-12-29 17:05:28,997][48117] Decorrelating experience for 0 frames... | |
| [2024-12-29 17:05:29,049][48116] Decorrelating experience for 32 frames... | |
| [2024-12-29 17:05:29,223][48118] Decorrelating experience for 32 frames... | |
| [2024-12-29 17:05:29,226][48117] Decorrelating experience for 32 frames... | |
| [2024-12-29 17:05:29,259][48120] Decorrelating experience for 0 frames... | |
| [2024-12-29 17:05:29,495][48120] Decorrelating experience for 32 frames... | |
| [2024-12-29 17:05:29,498][48116] Decorrelating experience for 64 frames... | |
| [2024-12-29 17:05:29,520][48118] Decorrelating experience for 64 frames... | |
| [2024-12-29 17:05:29,523][48115] Decorrelating experience for 0 frames... | |
| [2024-12-29 17:05:29,748][48115] Decorrelating experience for 32 frames... | |
| [2024-12-29 17:05:29,750][48116] Decorrelating experience for 96 frames... | |
| [2024-12-29 17:05:29,773][48118] Decorrelating experience for 96 frames... | |
| [2024-12-29 17:05:29,777][48120] Decorrelating experience for 64 frames... | |
| [2024-12-29 17:05:30,029][48117] Decorrelating experience for 64 frames... | |
| [2024-12-29 17:05:30,039][48120] Decorrelating experience for 96 frames... | |
| [2024-12-29 17:05:30,042][48115] Decorrelating experience for 64 frames... | |
| [2024-12-29 17:05:30,278][48117] Decorrelating experience for 96 frames... | |
| [2024-12-29 17:05:30,298][48115] Decorrelating experience for 96 frames... | |
| [2024-12-29 17:05:32,475][48101] Signal inference workers to stop experience collection... | |
| [2024-12-29 17:05:32,479][48114] InferenceWorker_p0-w0: stopping experience collection | |
| [2024-12-29 17:05:33,721][45646] Heartbeat connected on Batcher_0 | |
| [2024-12-29 17:05:33,728][45646] Heartbeat connected on InferenceWorker_p0-w0 | |
| [2024-12-29 17:05:33,732][45646] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 4.8. Samples: 24. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2024-12-29 17:05:33,733][45646] Avg episode reward: [(0, '3.089')] | |
| [2024-12-29 17:05:33,736][45646] Heartbeat connected on RolloutWorker_w1 | |
| [2024-12-29 17:05:33,739][45646] Heartbeat connected on RolloutWorker_w2 | |
| [2024-12-29 17:05:33,743][45646] Heartbeat connected on RolloutWorker_w3 | |
| [2024-12-29 17:05:33,746][45646] Heartbeat connected on RolloutWorker_w4 | |
| [2024-12-29 17:05:33,749][45646] Heartbeat connected on RolloutWorker_w5 | |
| [2024-12-29 17:05:34,723][48101] Signal inference workers to resume experience collection... | |
| [2024-12-29 17:05:34,723][48114] InferenceWorker_p0-w0: resuming experience collection | |
| [2024-12-29 17:05:34,932][45646] Heartbeat connected on LearnerWorker_p0 | |
| [2024-12-29 17:05:36,234][48114] Updated weights for policy 0, policy_version 10 (0.0103) | |
| [2024-12-29 17:05:38,072][48114] Updated weights for policy 0, policy_version 20 (0.0006) | |
| [2024-12-29 17:05:38,732][45646] Fps is (10 sec: 9420.8, 60 sec: 9420.8, 300 sec: 9420.8). Total num frames: 94208. Throughput: 0: 1682.2. Samples: 16822. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-12-29 17:05:38,733][45646] Avg episode reward: [(0, '4.416')] | |
| [2024-12-29 17:05:39,899][48114] Updated weights for policy 0, policy_version 30 (0.0006) | |
| [2024-12-29 17:05:41,723][48114] Updated weights for policy 0, policy_version 40 (0.0006) | |
| [2024-12-29 17:05:43,550][48114] Updated weights for policy 0, policy_version 50 (0.0006) | |
| [2024-12-29 17:05:43,732][45646] Fps is (10 sec: 20480.1, 60 sec: 13653.4, 300 sec: 13653.4). Total num frames: 204800. Throughput: 0: 3367.3. Samples: 50510. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-12-29 17:05:43,733][45646] Avg episode reward: [(0, '4.575')] | |
| [2024-12-29 17:05:43,735][48101] Saving new best policy, reward=4.575! | |
| [2024-12-29 17:05:45,392][48114] Updated weights for policy 0, policy_version 60 (0.0006) | |
| [2024-12-29 17:05:47,224][48114] Updated weights for policy 0, policy_version 70 (0.0006) | |
| [2024-12-29 17:05:48,732][45646] Fps is (10 sec: 22527.9, 60 sec: 15974.4, 300 sec: 15974.4). Total num frames: 319488. Throughput: 0: 3365.5. Samples: 67310. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-12-29 17:05:48,733][45646] Avg episode reward: [(0, '4.394')] | |
| [2024-12-29 17:05:49,059][48114] Updated weights for policy 0, policy_version 80 (0.0006) | |
| [2024-12-29 17:05:50,887][48114] Updated weights for policy 0, policy_version 90 (0.0006) | |
| [2024-12-29 17:05:52,700][48114] Updated weights for policy 0, policy_version 100 (0.0006) | |
| [2024-12-29 17:05:53,732][45646] Fps is (10 sec: 22528.0, 60 sec: 17203.2, 300 sec: 17203.2). Total num frames: 430080. Throughput: 0: 4037.2. Samples: 100930. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) | |
| [2024-12-29 17:05:53,733][45646] Avg episode reward: [(0, '4.337')] | |
| [2024-12-29 17:05:54,527][48114] Updated weights for policy 0, policy_version 110 (0.0006) | |
| [2024-12-29 17:05:56,357][48114] Updated weights for policy 0, policy_version 120 (0.0006) | |
| [2024-12-29 17:05:58,176][48114] Updated weights for policy 0, policy_version 130 (0.0006) | |
| [2024-12-29 17:05:58,732][45646] Fps is (10 sec: 22528.1, 60 sec: 18159.0, 300 sec: 18159.0). Total num frames: 544768. Throughput: 0: 4488.1. Samples: 134642. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-12-29 17:05:58,733][45646] Avg episode reward: [(0, '4.428')] | |
| [2024-12-29 17:05:59,999][48114] Updated weights for policy 0, policy_version 140 (0.0006) | |
| [2024-12-29 17:06:01,829][48114] Updated weights for policy 0, policy_version 150 (0.0006) | |
| [2024-12-29 17:06:03,661][48114] Updated weights for policy 0, policy_version 160 (0.0006) | |
| [2024-12-29 17:06:03,732][45646] Fps is (10 sec: 22528.0, 60 sec: 18724.6, 300 sec: 18724.6). Total num frames: 655360. Throughput: 0: 4329.8. Samples: 151542. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-12-29 17:06:03,733][45646] Avg episode reward: [(0, '4.541')] | |
| [2024-12-29 17:06:05,490][48114] Updated weights for policy 0, policy_version 170 (0.0006) | |
| [2024-12-29 17:06:07,316][48114] Updated weights for policy 0, policy_version 180 (0.0006) | |
| [2024-12-29 17:06:08,732][45646] Fps is (10 sec: 22527.9, 60 sec: 19251.2, 300 sec: 19251.2). Total num frames: 770048. Throughput: 0: 4631.2. Samples: 185248. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) | |
| [2024-12-29 17:06:08,733][45646] Avg episode reward: [(0, '5.057')] | |
| [2024-12-29 17:06:08,733][48101] Saving new best policy, reward=5.057! | |
| [2024-12-29 17:06:09,134][48114] Updated weights for policy 0, policy_version 190 (0.0006) | |
| [2024-12-29 17:06:10,941][48114] Updated weights for policy 0, policy_version 200 (0.0006) | |
| [2024-12-29 17:06:12,748][48114] Updated weights for policy 0, policy_version 210 (0.0006) | |
| [2024-12-29 17:06:13,732][45646] Fps is (10 sec: 22528.0, 60 sec: 19569.8, 300 sec: 19569.8). Total num frames: 880640. Throughput: 0: 4866.5. Samples: 218994. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-12-29 17:06:13,733][45646] Avg episode reward: [(0, '5.207')] | |
| [2024-12-29 17:06:13,735][48101] Saving new best policy, reward=5.207! | |
| [2024-12-29 17:06:14,567][48114] Updated weights for policy 0, policy_version 220 (0.0006) | |
| [2024-12-29 17:06:16,382][48114] Updated weights for policy 0, policy_version 230 (0.0006) | |
| [2024-12-29 17:06:18,200][48114] Updated weights for policy 0, policy_version 240 (0.0006) | |
| [2024-12-29 17:06:18,732][45646] Fps is (10 sec: 22118.3, 60 sec: 19824.6, 300 sec: 19824.6). Total num frames: 991232. Throughput: 0: 5240.8. Samples: 235862. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-12-29 17:06:18,733][45646] Avg episode reward: [(0, '5.699')] | |
| [2024-12-29 17:06:18,733][48101] Saving new best policy, reward=5.699! | |
| [2024-12-29 17:06:20,011][48114] Updated weights for policy 0, policy_version 250 (0.0006) | |
| [2024-12-29 17:06:21,826][48114] Updated weights for policy 0, policy_version 260 (0.0006) | |
| [2024-12-29 17:06:23,641][48114] Updated weights for policy 0, policy_version 270 (0.0006) | |
| [2024-12-29 17:06:23,732][45646] Fps is (10 sec: 22527.9, 60 sec: 20107.6, 300 sec: 20107.6). Total num frames: 1105920. Throughput: 0: 5618.3. Samples: 269646. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-12-29 17:06:23,733][45646] Avg episode reward: [(0, '8.082')] | |
| [2024-12-29 17:06:23,735][48101] Saving new best policy, reward=8.082! | |
| [2024-12-29 17:06:25,459][48114] Updated weights for policy 0, policy_version 280 (0.0006) | |
| [2024-12-29 17:06:27,279][48114] Updated weights for policy 0, policy_version 290 (0.0006) | |
| [2024-12-29 17:06:28,732][45646] Fps is (10 sec: 22528.1, 60 sec: 20275.2, 300 sec: 20275.2). Total num frames: 1216512. Throughput: 0: 5621.2. Samples: 303464. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) | |
| [2024-12-29 17:06:28,733][45646] Avg episode reward: [(0, '7.501')] | |
| [2024-12-29 17:06:29,109][48114] Updated weights for policy 0, policy_version 300 (0.0006) | |
| [2024-12-29 17:06:30,924][48114] Updated weights for policy 0, policy_version 310 (0.0006) | |
| [2024-12-29 17:06:32,754][48114] Updated weights for policy 0, policy_version 320 (0.0006) | |
| [2024-12-29 17:06:33,732][45646] Fps is (10 sec: 22528.1, 60 sec: 22186.7, 300 sec: 20480.0). Total num frames: 1331200. Throughput: 0: 5623.7. Samples: 320378. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-12-29 17:06:33,733][45646] Avg episode reward: [(0, '7.499')] | |
| [2024-12-29 17:06:34,569][48114] Updated weights for policy 0, policy_version 330 (0.0006) | |
| [2024-12-29 17:06:36,385][48114] Updated weights for policy 0, policy_version 340 (0.0006) | |
| [2024-12-29 17:06:38,199][48114] Updated weights for policy 0, policy_version 350 (0.0006) | |
| [2024-12-29 17:06:38,732][45646] Fps is (10 sec: 22528.0, 60 sec: 22459.7, 300 sec: 20597.0). Total num frames: 1441792. Throughput: 0: 5628.5. Samples: 354212. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-12-29 17:06:38,733][45646] Avg episode reward: [(0, '9.542')] | |
| [2024-12-29 17:06:38,733][48101] Saving new best policy, reward=9.542! | |
| [2024-12-29 17:06:40,007][48114] Updated weights for policy 0, policy_version 360 (0.0006) | |
| [2024-12-29 17:06:41,814][48114] Updated weights for policy 0, policy_version 370 (0.0006) | |
| [2024-12-29 17:06:43,619][48114] Updated weights for policy 0, policy_version 380 (0.0006) | |
| [2024-12-29 17:06:43,732][45646] Fps is (10 sec: 22527.9, 60 sec: 22528.0, 300 sec: 20753.1). Total num frames: 1556480. Throughput: 0: 5633.8. Samples: 388162. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-12-29 17:06:43,733][45646] Avg episode reward: [(0, '10.229')] | |
| [2024-12-29 17:06:43,735][48101] Saving new best policy, reward=10.229! | |
| [2024-12-29 17:06:45,435][48114] Updated weights for policy 0, policy_version 390 (0.0006) | |
| [2024-12-29 17:06:47,235][48114] Updated weights for policy 0, policy_version 400 (0.0006) | |
| [2024-12-29 17:06:48,732][45646] Fps is (10 sec: 22937.5, 60 sec: 22528.0, 300 sec: 20889.6). Total num frames: 1671168. Throughput: 0: 5634.5. Samples: 405096. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) | |
| [2024-12-29 17:06:48,733][45646] Avg episode reward: [(0, '13.707')] | |
| [2024-12-29 17:06:48,733][48101] Saving new best policy, reward=13.707! | |
| [2024-12-29 17:06:49,037][48114] Updated weights for policy 0, policy_version 410 (0.0006) | |
| [2024-12-29 17:06:50,839][48114] Updated weights for policy 0, policy_version 420 (0.0006) | |
| [2024-12-29 17:06:52,633][48114] Updated weights for policy 0, policy_version 430 (0.0006) | |
| [2024-12-29 17:06:53,732][45646] Fps is (10 sec: 22528.1, 60 sec: 22528.0, 300 sec: 20961.9). Total num frames: 1781760. Throughput: 0: 5641.8. Samples: 439130. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) | |
| [2024-12-29 17:06:53,733][45646] Avg episode reward: [(0, '14.863')] | |
| [2024-12-29 17:06:53,737][48101] Saving new best policy, reward=14.863! | |
| [2024-12-29 17:06:54,445][48114] Updated weights for policy 0, policy_version 440 (0.0006) | |
| [2024-12-29 17:06:56,245][48114] Updated weights for policy 0, policy_version 450 (0.0006) | |
| [2024-12-29 17:06:58,054][48114] Updated weights for policy 0, policy_version 460 (0.0006) | |
| [2024-12-29 17:06:58,732][45646] Fps is (10 sec: 22528.1, 60 sec: 22528.0, 300 sec: 21071.7). Total num frames: 1896448. Throughput: 0: 5647.3. Samples: 473122. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) | |
| [2024-12-29 17:06:58,733][45646] Avg episode reward: [(0, '14.178')] | |
| [2024-12-29 17:06:59,866][48114] Updated weights for policy 0, policy_version 470 (0.0006) | |
| [2024-12-29 17:07:01,672][48114] Updated weights for policy 0, policy_version 480 (0.0006) | |
| [2024-12-29 17:07:03,478][48114] Updated weights for policy 0, policy_version 490 (0.0006) | |
| [2024-12-29 17:07:03,732][45646] Fps is (10 sec: 22937.6, 60 sec: 22596.3, 300 sec: 21169.9). Total num frames: 2011136. Throughput: 0: 5649.8. Samples: 490102. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-12-29 17:07:03,733][45646] Avg episode reward: [(0, '16.228')] | |
| [2024-12-29 17:07:03,735][48101] Saving /fsx/users/amzfang/rl_course/train_dir/default_experiment/checkpoint_p0/checkpoint_000000491_2011136.pth... | |
| [2024-12-29 17:07:03,787][48101] Saving new best policy, reward=16.228! | |
| [2024-12-29 17:07:05,303][48114] Updated weights for policy 0, policy_version 500 (0.0006) | |
| [2024-12-29 17:07:07,103][48114] Updated weights for policy 0, policy_version 510 (0.0006) | |
| [2024-12-29 17:07:08,732][45646] Fps is (10 sec: 22937.6, 60 sec: 22596.3, 300 sec: 21258.2). Total num frames: 2125824. Throughput: 0: 5654.9. Samples: 524116. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-12-29 17:07:08,733][45646] Avg episode reward: [(0, '16.025')] | |
| [2024-12-29 17:07:08,917][48114] Updated weights for policy 0, policy_version 520 (0.0006) | |
| [2024-12-29 17:07:10,729][48114] Updated weights for policy 0, policy_version 530 (0.0006) | |
| [2024-12-29 17:07:12,539][48114] Updated weights for policy 0, policy_version 540 (0.0006) | |
| [2024-12-29 17:07:13,732][45646] Fps is (10 sec: 22528.0, 60 sec: 22596.3, 300 sec: 21299.2). Total num frames: 2236416. Throughput: 0: 5659.4. Samples: 558138. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) | |
| [2024-12-29 17:07:13,733][45646] Avg episode reward: [(0, '17.471')] | |
| [2024-12-29 17:07:13,735][48101] Saving new best policy, reward=17.471! | |
| [2024-12-29 17:07:14,348][48114] Updated weights for policy 0, policy_version 550 (0.0006) | |
| [2024-12-29 17:07:16,151][48114] Updated weights for policy 0, policy_version 560 (0.0006) | |
| [2024-12-29 17:07:17,950][48114] Updated weights for policy 0, policy_version 570 (0.0006) | |
| [2024-12-29 17:07:18,732][45646] Fps is (10 sec: 22528.0, 60 sec: 22664.6, 300 sec: 21373.7). Total num frames: 2351104. Throughput: 0: 5660.8. Samples: 575112. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-12-29 17:07:18,733][45646] Avg episode reward: [(0, '16.219')] | |
| [2024-12-29 17:07:19,754][48114] Updated weights for policy 0, policy_version 580 (0.0006) | |
| [2024-12-29 17:07:21,547][48114] Updated weights for policy 0, policy_version 590 (0.0006) | |
| [2024-12-29 17:07:23,345][48114] Updated weights for policy 0, policy_version 600 (0.0006) | |
| [2024-12-29 17:07:23,732][45646] Fps is (10 sec: 22937.6, 60 sec: 22664.5, 300 sec: 21441.7). Total num frames: 2465792. Throughput: 0: 5667.3. Samples: 609240. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-12-29 17:07:23,733][45646] Avg episode reward: [(0, '18.760')] | |
| [2024-12-29 17:07:23,735][48101] Saving new best policy, reward=18.760! | |
| [2024-12-29 17:07:25,160][48114] Updated weights for policy 0, policy_version 610 (0.0006) | |
| [2024-12-29 17:07:26,963][48114] Updated weights for policy 0, policy_version 620 (0.0006) | |
| [2024-12-29 17:07:28,732][45646] Fps is (10 sec: 22528.0, 60 sec: 22664.5, 300 sec: 21469.9). Total num frames: 2576384. Throughput: 0: 5670.9. Samples: 643354. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-12-29 17:07:28,733][45646] Avg episode reward: [(0, '18.920')] | |
| [2024-12-29 17:07:28,733][48101] Saving new best policy, reward=18.920! | |
| [2024-12-29 17:07:28,825][48114] Updated weights for policy 0, policy_version 630 (0.0006) | |
| [2024-12-29 17:07:30,580][48114] Updated weights for policy 0, policy_version 640 (0.0006) | |
| [2024-12-29 17:07:32,382][48114] Updated weights for policy 0, policy_version 650 (0.0006) | |
| [2024-12-29 17:07:33,732][45646] Fps is (10 sec: 22528.0, 60 sec: 22664.5, 300 sec: 21528.6). Total num frames: 2691072. Throughput: 0: 5672.5. Samples: 660358. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-12-29 17:07:33,733][45646] Avg episode reward: [(0, '19.076')] | |
| [2024-12-29 17:07:33,735][48101] Saving new best policy, reward=19.076! | |
| [2024-12-29 17:07:34,197][48114] Updated weights for policy 0, policy_version 660 (0.0006) | |
| [2024-12-29 17:07:36,005][48114] Updated weights for policy 0, policy_version 670 (0.0006) | |
| [2024-12-29 17:07:37,805][48114] Updated weights for policy 0, policy_version 680 (0.0006) | |
| [2024-12-29 17:07:38,732][45646] Fps is (10 sec: 22937.5, 60 sec: 22732.8, 300 sec: 21582.8). Total num frames: 2805760. Throughput: 0: 5673.0. Samples: 694414. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) | |
| [2024-12-29 17:07:38,733][45646] Avg episode reward: [(0, '17.664')] | |
| [2024-12-29 17:07:39,603][48114] Updated weights for policy 0, policy_version 690 (0.0006) | |
| [2024-12-29 17:07:41,405][48114] Updated weights for policy 0, policy_version 700 (0.0006) | |
| [2024-12-29 17:07:43,189][48114] Updated weights for policy 0, policy_version 710 (0.0006) | |
| [2024-12-29 17:07:43,732][45646] Fps is (10 sec: 22528.1, 60 sec: 22664.5, 300 sec: 21602.6). Total num frames: 2916352. Throughput: 0: 5674.3. Samples: 728466. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-12-29 17:07:43,733][45646] Avg episode reward: [(0, '20.509')] | |
| [2024-12-29 17:07:43,735][48101] Saving new best policy, reward=20.509! | |
| [2024-12-29 17:07:44,989][48114] Updated weights for policy 0, policy_version 720 (0.0006) | |
| [2024-12-29 17:07:46,791][48114] Updated weights for policy 0, policy_version 730 (0.0006) | |
| [2024-12-29 17:07:48,594][48114] Updated weights for policy 0, policy_version 740 (0.0006) | |
| [2024-12-29 17:07:48,732][45646] Fps is (10 sec: 22528.0, 60 sec: 22664.5, 300 sec: 21650.3). Total num frames: 3031040. Throughput: 0: 5675.7. Samples: 745510. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-12-29 17:07:48,733][45646] Avg episode reward: [(0, '21.202')] | |
| [2024-12-29 17:07:48,733][48101] Saving new best policy, reward=21.202! | |
| [2024-12-29 17:07:50,413][48114] Updated weights for policy 0, policy_version 750 (0.0006) | |
| [2024-12-29 17:07:52,209][48114] Updated weights for policy 0, policy_version 760 (0.0006) | |
| [2024-12-29 17:07:53,732][45646] Fps is (10 sec: 22937.6, 60 sec: 22732.8, 300 sec: 21694.7). Total num frames: 3145728. Throughput: 0: 5677.6. Samples: 779610. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-12-29 17:07:53,733][45646] Avg episode reward: [(0, '20.980')] | |
| [2024-12-29 17:07:54,016][48114] Updated weights for policy 0, policy_version 770 (0.0006) | |
| [2024-12-29 17:07:55,815][48114] Updated weights for policy 0, policy_version 780 (0.0006) | |
| [2024-12-29 17:07:57,614][48114] Updated weights for policy 0, policy_version 790 (0.0006) | |
| [2024-12-29 17:07:58,732][45646] Fps is (10 sec: 22937.6, 60 sec: 22732.8, 300 sec: 21736.1). Total num frames: 3260416. Throughput: 0: 5678.4. Samples: 813668. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) | |
| [2024-12-29 17:07:58,733][45646] Avg episode reward: [(0, '20.302')] | |
| [2024-12-29 17:07:59,425][48114] Updated weights for policy 0, policy_version 800 (0.0006) | |
| [2024-12-29 17:08:01,225][48114] Updated weights for policy 0, policy_version 810 (0.0006) | |
| [2024-12-29 17:08:03,026][48114] Updated weights for policy 0, policy_version 820 (0.0006) | |
| [2024-12-29 17:08:03,732][45646] Fps is (10 sec: 22527.9, 60 sec: 22664.5, 300 sec: 21748.4). Total num frames: 3371008. Throughput: 0: 5679.9. Samples: 830710. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-12-29 17:08:03,733][45646] Avg episode reward: [(0, '19.765')] | |
| [2024-12-29 17:08:04,827][48114] Updated weights for policy 0, policy_version 830 (0.0006) | |
| [2024-12-29 17:08:06,621][48114] Updated weights for policy 0, policy_version 840 (0.0006) | |
| [2024-12-29 17:08:08,431][48114] Updated weights for policy 0, policy_version 850 (0.0006) | |
| [2024-12-29 17:08:08,732][45646] Fps is (10 sec: 22528.0, 60 sec: 22664.5, 300 sec: 21785.6). Total num frames: 3485696. Throughput: 0: 5677.6. Samples: 864732. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) | |
| [2024-12-29 17:08:08,733][45646] Avg episode reward: [(0, '19.344')] | |
| [2024-12-29 17:08:10,247][48114] Updated weights for policy 0, policy_version 860 (0.0006) | |
| [2024-12-29 17:08:12,050][48114] Updated weights for policy 0, policy_version 870 (0.0006) | |
| [2024-12-29 17:08:13,732][45646] Fps is (10 sec: 22937.7, 60 sec: 22732.8, 300 sec: 21820.5). Total num frames: 3600384. Throughput: 0: 5676.9. Samples: 898816. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-12-29 17:08:13,733][45646] Avg episode reward: [(0, '20.916')] | |
| [2024-12-29 17:08:13,862][48114] Updated weights for policy 0, policy_version 880 (0.0006) | |
| [2024-12-29 17:08:15,667][48114] Updated weights for policy 0, policy_version 890 (0.0006) | |
| [2024-12-29 17:08:17,469][48114] Updated weights for policy 0, policy_version 900 (0.0006) | |
| [2024-12-29 17:08:18,732][45646] Fps is (10 sec: 22937.7, 60 sec: 22732.8, 300 sec: 21853.4). Total num frames: 3715072. Throughput: 0: 5677.1. Samples: 915826. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) | |
| [2024-12-29 17:08:18,733][45646] Avg episode reward: [(0, '20.260')] | |
| [2024-12-29 17:08:19,278][48114] Updated weights for policy 0, policy_version 910 (0.0006) | |
| [2024-12-29 17:08:21,100][48114] Updated weights for policy 0, policy_version 920 (0.0006) | |
| [2024-12-29 17:08:22,903][48114] Updated weights for policy 0, policy_version 930 (0.0006) | |
| [2024-12-29 17:08:23,732][45646] Fps is (10 sec: 22528.0, 60 sec: 22664.5, 300 sec: 21860.9). Total num frames: 3825664. Throughput: 0: 5675.3. Samples: 949802. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-12-29 17:08:23,733][45646] Avg episode reward: [(0, '20.878')] | |
| [2024-12-29 17:08:24,714][48114] Updated weights for policy 0, policy_version 940 (0.0006) | |
| [2024-12-29 17:08:26,518][48114] Updated weights for policy 0, policy_version 950 (0.0006) | |
| [2024-12-29 17:08:28,315][48114] Updated weights for policy 0, policy_version 960 (0.0006) | |
| [2024-12-29 17:08:28,732][45646] Fps is (10 sec: 22528.0, 60 sec: 22732.8, 300 sec: 21890.8). Total num frames: 3940352. Throughput: 0: 5676.0. Samples: 983884. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-12-29 17:08:28,733][45646] Avg episode reward: [(0, '21.703')] | |
| [2024-12-29 17:08:28,733][48101] Saving new best policy, reward=21.703! | |
| [2024-12-29 17:08:30,110][48114] Updated weights for policy 0, policy_version 970 (0.0006) | |
| [2024-12-29 17:08:31,545][48101] Stopping Batcher_0... | |
| [2024-12-29 17:08:31,546][48101] Loop batcher_evt_loop terminating... | |
| [2024-12-29 17:08:31,545][45646] Component Batcher_0 stopped! | |
| [2024-12-29 17:08:31,546][48101] Saving /fsx/users/amzfang/rl_course/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2024-12-29 17:08:31,546][45646] Component RolloutWorker_w0 process died already! Don't wait for it. | |
| [2024-12-29 17:08:31,547][45646] Component RolloutWorker_w6 process died already! Don't wait for it. | |
| [2024-12-29 17:08:31,547][45646] Component RolloutWorker_w7 process died already! Don't wait for it. | |
| [2024-12-29 17:08:31,576][48114] Weights refcount: 2 0 | |
| [2024-12-29 17:08:31,577][48114] Stopping InferenceWorker_p0-w0... | |
| [2024-12-29 17:08:31,577][48114] Loop inference_proc0-0_evt_loop terminating... | |
| [2024-12-29 17:08:31,577][45646] Component InferenceWorker_p0-w0 stopped! | |
| [2024-12-29 17:08:31,584][45646] Component RolloutWorker_w4 stopped! | |
| [2024-12-29 17:08:31,584][48118] Stopping RolloutWorker_w4... | |
| [2024-12-29 17:08:31,585][48118] Loop rollout_proc4_evt_loop terminating... | |
| [2024-12-29 17:08:31,585][45646] Component RolloutWorker_w5 stopped! | |
| [2024-12-29 17:08:31,585][48120] Stopping RolloutWorker_w5... | |
| [2024-12-29 17:08:31,586][48120] Loop rollout_proc5_evt_loop terminating... | |
| [2024-12-29 17:08:31,587][45646] Component RolloutWorker_w3 stopped! | |
| [2024-12-29 17:08:31,587][48115] Stopping RolloutWorker_w3... | |
| [2024-12-29 17:08:31,588][45646] Component RolloutWorker_w2 stopped! | |
| [2024-12-29 17:08:31,588][48115] Loop rollout_proc3_evt_loop terminating... | |
| [2024-12-29 17:08:31,588][48117] Stopping RolloutWorker_w2... | |
| [2024-12-29 17:08:31,589][48117] Loop rollout_proc2_evt_loop terminating... | |
| [2024-12-29 17:08:31,591][45646] Component RolloutWorker_w1 stopped! | |
| [2024-12-29 17:08:31,591][48116] Stopping RolloutWorker_w1... | |
| [2024-12-29 17:08:31,591][48116] Loop rollout_proc1_evt_loop terminating... | |
| [2024-12-29 17:08:31,597][48101] Saving /fsx/users/amzfang/rl_course/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2024-12-29 17:08:31,652][48101] Stopping LearnerWorker_p0... | |
| [2024-12-29 17:08:31,653][48101] Loop learner_proc0_evt_loop terminating... | |
| [2024-12-29 17:08:31,652][45646] Component LearnerWorker_p0 stopped! | |
| [2024-12-29 17:08:31,653][45646] Waiting for process learner_proc0 to stop... | |
| [2024-12-29 17:08:32,504][45646] Waiting for process inference_proc0-0 to join... | |
| [2024-12-29 17:08:32,505][45646] Waiting for process rollout_proc0 to join... | |
| [2024-12-29 17:08:32,505][45646] Waiting for process rollout_proc1 to join... | |
| [2024-12-29 17:08:32,506][45646] Waiting for process rollout_proc2 to join... | |
| [2024-12-29 17:08:32,506][45646] Waiting for process rollout_proc3 to join... | |
| [2024-12-29 17:08:32,507][45646] Waiting for process rollout_proc4 to join... | |
| [2024-12-29 17:08:32,507][45646] Waiting for process rollout_proc5 to join... | |
| [2024-12-29 17:08:32,508][45646] Waiting for process rollout_proc6 to join... | |
| [2024-12-29 17:08:32,508][45646] Waiting for process rollout_proc7 to join... | |
| [2024-12-29 17:08:32,508][45646] Batcher 0 profile tree view: | |
| batching: 11.6411, releasing_batches: 0.0096 | |
| [2024-12-29 17:08:32,509][45646] InferenceWorker_p0-w0 profile tree view: | |
| wait_policy: 0.0000 | |
| wait_policy_total: 3.0425 | |
| update_model: 2.4073 | |
| weight_update: 0.0006 | |
| one_step: 0.0015 | |
| handle_policy_step: 168.3348 | |
| deserialize: 7.1826, stack: 0.8421, obs_to_device_normalize: 40.5132, forward: 79.8262, send_messages: 8.9738 | |
| prepare_outputs: 24.8578 | |
| to_cpu: 16.4872 | |
| [2024-12-29 17:08:32,509][45646] Learner 0 profile tree view: | |
| misc: 0.0025, prepare_batch: 4.7612 | |
| train: 12.4279 | |
| epoch_init: 0.0029, minibatch_init: 0.0036, losses_postprocess: 0.2127, kl_divergence: 0.2651, after_optimizer: 2.0727 | |
| calculate_losses: 5.4084 | |
| losses_init: 0.0021, forward_head: 0.4040, bptt_initial: 3.0630, tail: 0.3631, advantages_returns: 0.0862, losses: 0.7443 | |
| bptt: 0.6608 | |
| bptt_forward_core: 0.6354 | |
| update: 4.2704 | |
| clip: 0.4146 | |
| [2024-12-29 17:08:32,510][45646] Loop Runner_EvtLoop terminating... | |
| [2024-12-29 17:08:32,511][45646] Runner profile tree view: | |
| main_loop: 198.7540 | |
| [2024-12-29 17:08:32,511][45646] Collected {0: 4005888}, FPS: 20155.0 | |
| [2024-12-29 17:17:15,263][45646] Loading existing experiment configuration from /fsx/users/amzfang/rl_course/train_dir/default_experiment/config.json | |
| [2024-12-29 17:17:15,265][45646] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2024-12-29 17:17:15,266][45646] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2024-12-29 17:17:15,266][45646] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2024-12-29 17:17:15,266][45646] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2024-12-29 17:17:15,267][45646] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2024-12-29 17:17:15,267][45646] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! | |
| [2024-12-29 17:17:15,267][45646] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2024-12-29 17:17:15,268][45646] Adding new argument 'push_to_hub'=False that is not in the saved config file! | |
| [2024-12-29 17:17:15,268][45646] Adding new argument 'hf_repository'=None that is not in the saved config file! | |
| [2024-12-29 17:17:15,268][45646] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2024-12-29 17:17:15,269][45646] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2024-12-29 17:17:15,269][45646] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2024-12-29 17:17:15,270][45646] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2024-12-29 17:17:15,270][45646] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2024-12-29 17:17:15,490][45646] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-12-29 17:17:15,512][45646] RunningMeanStd input shape: (3, 72, 128) | |
| [2024-12-29 17:17:15,552][45646] RunningMeanStd input shape: (1,) | |
| [2024-12-29 17:17:15,665][45646] ConvEncoder: input_channels=3 | |
| [2024-12-29 17:17:15,912][45646] Conv encoder output size: 512 | |
| [2024-12-29 17:17:15,913][45646] Policy head output size: 512 | |
| [2024-12-29 17:17:16,970][45646] Loading state from checkpoint /fsx/users/amzfang/rl_course/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2024-12-29 17:17:18,757][45646] Num frames 100... | |
| [2024-12-29 17:17:18,850][45646] Num frames 200... | |
| [2024-12-29 17:17:18,943][45646] Num frames 300... | |
| [2024-12-29 17:17:19,038][45646] Num frames 400... | |
| [2024-12-29 17:17:19,142][45646] Avg episode rewards: #0: 10.520, true rewards: #0: 4.520 | |
| [2024-12-29 17:17:19,142][45646] Avg episode reward: 10.520, avg true_objective: 4.520 | |
| [2024-12-29 17:17:19,188][45646] Num frames 500... | |
| [2024-12-29 17:17:19,281][45646] Num frames 600... | |
| [2024-12-29 17:17:19,374][45646] Num frames 700... | |
| [2024-12-29 17:17:19,467][45646] Num frames 800... | |
| [2024-12-29 17:17:19,560][45646] Num frames 900... | |
| [2024-12-29 17:17:19,652][45646] Num frames 1000... | |
| [2024-12-29 17:17:19,744][45646] Num frames 1100... | |
| [2024-12-29 17:17:19,839][45646] Num frames 1200... | |
| [2024-12-29 17:17:19,931][45646] Num frames 1300... | |
| [2024-12-29 17:17:20,001][45646] Avg episode rewards: #0: 14.585, true rewards: #0: 6.585 | |
| [2024-12-29 17:17:20,001][45646] Avg episode reward: 14.585, avg true_objective: 6.585 | |
| [2024-12-29 17:17:20,077][45646] Num frames 1400... | |
| [2024-12-29 17:17:20,169][45646] Num frames 1500... | |
| [2024-12-29 17:17:20,263][45646] Num frames 1600... | |
| [2024-12-29 17:17:20,355][45646] Num frames 1700... | |
| [2024-12-29 17:17:20,447][45646] Num frames 1800... | |
| [2024-12-29 17:17:20,540][45646] Num frames 1900... | |
| [2024-12-29 17:17:20,636][45646] Num frames 2000... | |
| [2024-12-29 17:17:20,728][45646] Num frames 2100... | |
| [2024-12-29 17:17:20,820][45646] Num frames 2200... | |
| [2024-12-29 17:17:20,913][45646] Num frames 2300... | |
| [2024-12-29 17:17:21,005][45646] Num frames 2400... | |
| [2024-12-29 17:17:21,098][45646] Num frames 2500... | |
| [2024-12-29 17:17:21,191][45646] Num frames 2600... | |
| [2024-12-29 17:17:21,283][45646] Num frames 2700... | |
| [2024-12-29 17:17:21,378][45646] Num frames 2800... | |
| [2024-12-29 17:17:21,471][45646] Num frames 2900... | |
| [2024-12-29 17:17:21,564][45646] Num frames 3000... | |
| [2024-12-29 17:17:21,657][45646] Num frames 3100... | |
| [2024-12-29 17:17:21,749][45646] Num frames 3200... | |
| [2024-12-29 17:17:21,842][45646] Num frames 3300... | |
| [2024-12-29 17:17:21,935][45646] Num frames 3400... | |
| [2024-12-29 17:17:22,005][45646] Avg episode rewards: #0: 27.056, true rewards: #0: 11.390 | |
| [2024-12-29 17:17:22,005][45646] Avg episode reward: 27.056, avg true_objective: 11.390 | |
| [2024-12-29 17:17:22,081][45646] Num frames 3500... | |
| [2024-12-29 17:17:22,175][45646] Num frames 3600... | |
| [2024-12-29 17:17:22,267][45646] Num frames 3700... | |
| [2024-12-29 17:17:22,359][45646] Num frames 3800... | |
| [2024-12-29 17:17:22,451][45646] Num frames 3900... | |
| [2024-12-29 17:17:22,543][45646] Num frames 4000... | |
| [2024-12-29 17:17:22,635][45646] Num frames 4100... | |
| [2024-12-29 17:17:22,727][45646] Num frames 4200... | |
| [2024-12-29 17:17:22,819][45646] Num frames 4300... | |
| [2024-12-29 17:17:22,960][45646] Avg episode rewards: #0: 25.490, true rewards: #0: 10.990 | |
| [2024-12-29 17:17:22,961][45646] Avg episode reward: 25.490, avg true_objective: 10.990 | |
| [2024-12-29 17:17:22,965][45646] Num frames 4400... | |
| [2024-12-29 17:17:23,056][45646] Num frames 4500... | |
| [2024-12-29 17:17:23,150][45646] Num frames 4600... | |
| [2024-12-29 17:17:23,243][45646] Num frames 4700... | |
| [2024-12-29 17:17:23,334][45646] Num frames 4800... | |
| [2024-12-29 17:17:23,426][45646] Num frames 4900... | |
| [2024-12-29 17:17:23,518][45646] Num frames 5000... | |
| [2024-12-29 17:17:23,634][45646] Avg episode rewards: #0: 23.136, true rewards: #0: 10.136 | |
| [2024-12-29 17:17:23,635][45646] Avg episode reward: 23.136, avg true_objective: 10.136 | |
| [2024-12-29 17:17:23,665][45646] Num frames 5100... | |
| [2024-12-29 17:17:23,757][45646] Num frames 5200... | |
| [2024-12-29 17:17:23,850][45646] Num frames 5300... | |
| [2024-12-29 17:17:23,942][45646] Num frames 5400... | |
| [2024-12-29 17:17:24,034][45646] Num frames 5500... | |
| [2024-12-29 17:17:24,127][45646] Num frames 5600... | |
| [2024-12-29 17:17:24,219][45646] Num frames 5700... | |
| [2024-12-29 17:17:24,312][45646] Num frames 5800... | |
| [2024-12-29 17:17:24,403][45646] Num frames 5900... | |
| [2024-12-29 17:17:24,516][45646] Avg episode rewards: #0: 22.440, true rewards: #0: 9.940 | |
| [2024-12-29 17:17:24,516][45646] Avg episode reward: 22.440, avg true_objective: 9.940 | |
| [2024-12-29 17:17:24,550][45646] Num frames 6000... | |
| [2024-12-29 17:17:24,643][45646] Num frames 6100... | |
| [2024-12-29 17:17:24,735][45646] Num frames 6200... | |
| [2024-12-29 17:17:24,827][45646] Num frames 6300... | |
| [2024-12-29 17:17:24,919][45646] Num frames 6400... | |
| [2024-12-29 17:17:25,043][45646] Avg episode rewards: #0: 20.394, true rewards: #0: 9.251 | |
| [2024-12-29 17:17:25,043][45646] Avg episode reward: 20.394, avg true_objective: 9.251 | |
| [2024-12-29 17:17:25,066][45646] Num frames 6500... | |
| [2024-12-29 17:17:25,158][45646] Num frames 6600... | |
| [2024-12-29 17:17:25,250][45646] Num frames 6700... | |
| [2024-12-29 17:17:25,343][45646] Num frames 6800... | |
| [2024-12-29 17:17:25,436][45646] Num frames 6900... | |
| [2024-12-29 17:17:25,528][45646] Num frames 7000... | |
| [2024-12-29 17:17:25,620][45646] Num frames 7100... | |
| [2024-12-29 17:17:25,713][45646] Num frames 7200... | |
| [2024-12-29 17:17:25,806][45646] Num frames 7300... | |
| [2024-12-29 17:17:25,898][45646] Num frames 7400... | |
| [2024-12-29 17:17:25,991][45646] Num frames 7500... | |
| [2024-12-29 17:17:26,085][45646] Num frames 7600... | |
| [2024-12-29 17:17:26,177][45646] Num frames 7700... | |
| [2024-12-29 17:17:26,253][45646] Avg episode rewards: #0: 21.030, true rewards: #0: 9.655 | |
| [2024-12-29 17:17:26,253][45646] Avg episode reward: 21.030, avg true_objective: 9.655 | |
| [2024-12-29 17:17:26,323][45646] Num frames 7800... | |
| [2024-12-29 17:17:26,415][45646] Num frames 7900... | |
| [2024-12-29 17:17:26,507][45646] Num frames 8000... | |
| [2024-12-29 17:17:26,600][45646] Num frames 8100... | |
| [2024-12-29 17:17:26,691][45646] Num frames 8200... | |
| [2024-12-29 17:17:26,783][45646] Num frames 8300... | |
| [2024-12-29 17:17:26,874][45646] Num frames 8400... | |
| [2024-12-29 17:17:26,967][45646] Num frames 8500... | |
| [2024-12-29 17:17:27,061][45646] Num frames 8600... | |
| [2024-12-29 17:17:27,111][45646] Avg episode rewards: #0: 20.889, true rewards: #0: 9.556 | |
| [2024-12-29 17:17:27,112][45646] Avg episode reward: 20.889, avg true_objective: 9.556 | |
| [2024-12-29 17:17:27,205][45646] Num frames 8700... | |
| [2024-12-29 17:17:27,298][45646] Num frames 8800... | |
| [2024-12-29 17:17:27,389][45646] Num frames 8900... | |
| [2024-12-29 17:17:27,481][45646] Num frames 9000... | |
| [2024-12-29 17:17:27,573][45646] Num frames 9100... | |
| [2024-12-29 17:17:27,666][45646] Num frames 9200... | |
| [2024-12-29 17:17:27,758][45646] Num frames 9300... | |
| [2024-12-29 17:17:27,851][45646] Num frames 9400... | |
| [2024-12-29 17:17:27,945][45646] Num frames 9500... | |
| [2024-12-29 17:17:28,038][45646] Num frames 9600... | |
| [2024-12-29 17:17:28,131][45646] Num frames 9700... | |
| [2024-12-29 17:17:28,225][45646] Num frames 9800... | |
| [2024-12-29 17:17:28,317][45646] Num frames 9900... | |
| [2024-12-29 17:17:28,409][45646] Num frames 10000... | |
| [2024-12-29 17:17:28,502][45646] Num frames 10100... | |
| [2024-12-29 17:17:28,593][45646] Num frames 10200... | |
| [2024-12-29 17:17:28,687][45646] Num frames 10300... | |
| [2024-12-29 17:17:28,781][45646] Num frames 10400... | |
| [2024-12-29 17:17:28,874][45646] Num frames 10500... | |
| [2024-12-29 17:17:28,967][45646] Num frames 10600... | |
| [2024-12-29 17:17:29,062][45646] Num frames 10700... | |
| [2024-12-29 17:17:29,113][45646] Avg episode rewards: #0: 24.500, true rewards: #0: 10.700 | |
| [2024-12-29 17:17:29,114][45646] Avg episode reward: 24.500, avg true_objective: 10.700 | |
| [2024-12-29 17:17:46,660][45646] Replay video saved to /fsx/users/amzfang/rl_course/train_dir/default_experiment/replay.mp4! | |
| [2024-12-29 17:34:26,145][45646] Loading existing experiment configuration from /fsx/users/amzfang/rl_course/train_dir/default_experiment/config.json | |
| [2024-12-29 17:34:26,147][45646] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2024-12-29 17:34:26,148][45646] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2024-12-29 17:34:26,148][45646] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2024-12-29 17:34:26,149][45646] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2024-12-29 17:34:26,149][45646] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2024-12-29 17:34:26,149][45646] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! | |
| [2024-12-29 17:34:26,150][45646] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2024-12-29 17:34:26,150][45646] Adding new argument 'push_to_hub'=True that is not in the saved config file! | |
| [2024-12-29 17:34:26,151][45646] Adding new argument 'hf_repository'='ThomasSimonini/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! | |
| [2024-12-29 17:34:26,151][45646] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2024-12-29 17:34:26,151][45646] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2024-12-29 17:34:26,152][45646] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2024-12-29 17:34:26,152][45646] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2024-12-29 17:34:26,152][45646] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2024-12-29 17:34:26,279][45646] RunningMeanStd input shape: (3, 72, 128) | |
| [2024-12-29 17:34:26,352][45646] RunningMeanStd input shape: (1,) | |
| [2024-12-29 17:34:26,454][45646] ConvEncoder: input_channels=3 | |
| [2024-12-29 17:34:26,728][45646] Conv encoder output size: 512 | |
| [2024-12-29 17:34:26,729][45646] Policy head output size: 512 | |
| [2024-12-29 17:34:26,871][45646] Loading state from checkpoint /fsx/users/amzfang/rl_course/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2024-12-29 17:34:27,729][45646] Num frames 100... | |
| [2024-12-29 17:34:27,822][45646] Num frames 200... | |
| [2024-12-29 17:34:27,913][45646] Num frames 300... | |
| [2024-12-29 17:34:28,004][45646] Num frames 400... | |
| [2024-12-29 17:34:28,096][45646] Num frames 500... | |
| [2024-12-29 17:34:28,186][45646] Num frames 600... | |
| [2024-12-29 17:34:28,334][45646] Avg episode rewards: #0: 14.980, true rewards: #0: 6.980 | |
| [2024-12-29 17:34:28,334][45646] Avg episode reward: 14.980, avg true_objective: 6.980 | |
| [2024-12-29 17:34:28,336][45646] Num frames 700... | |
| [2024-12-29 17:34:28,428][45646] Num frames 800... | |
| [2024-12-29 17:34:28,522][45646] Num frames 900... | |
| [2024-12-29 17:34:28,614][45646] Num frames 1000... | |
| [2024-12-29 17:34:28,708][45646] Num frames 1100... | |
| [2024-12-29 17:34:28,801][45646] Num frames 1200... | |
| [2024-12-29 17:34:28,895][45646] Num frames 1300... | |
| [2024-12-29 17:34:28,987][45646] Num frames 1400... | |
| [2024-12-29 17:34:29,133][45646] Avg episode rewards: #0: 14.990, true rewards: #0: 7.490 | |
| [2024-12-29 17:34:29,134][45646] Avg episode reward: 14.990, avg true_objective: 7.490 | |
| [2024-12-29 17:34:29,136][45646] Num frames 1500... | |
| [2024-12-29 17:34:29,229][45646] Num frames 1600... | |
| [2024-12-29 17:34:29,323][45646] Num frames 1700... | |
| [2024-12-29 17:34:29,417][45646] Num frames 1800... | |
| [2024-12-29 17:34:29,510][45646] Num frames 1900... | |
| [2024-12-29 17:34:29,604][45646] Num frames 2000... | |
| [2024-12-29 17:34:29,696][45646] Num frames 2100... | |
| [2024-12-29 17:34:29,790][45646] Num frames 2200... | |
| [2024-12-29 17:34:29,883][45646] Num frames 2300... | |
| [2024-12-29 17:34:29,979][45646] Num frames 2400... | |
| [2024-12-29 17:34:30,072][45646] Num frames 2500... | |
| [2024-12-29 17:34:30,166][45646] Num frames 2600... | |
| [2024-12-29 17:34:30,242][45646] Avg episode rewards: #0: 19.080, true rewards: #0: 8.747 | |
| [2024-12-29 17:34:30,242][45646] Avg episode reward: 19.080, avg true_objective: 8.747 | |
| [2024-12-29 17:34:30,314][45646] Num frames 2700... | |
| [2024-12-29 17:34:30,407][45646] Num frames 2800... | |
| [2024-12-29 17:34:30,500][45646] Num frames 2900... | |
| [2024-12-29 17:34:30,595][45646] Num frames 3000... | |
| [2024-12-29 17:34:30,687][45646] Num frames 3100... | |
| [2024-12-29 17:34:30,780][45646] Num frames 3200... | |
| [2024-12-29 17:34:30,875][45646] Num frames 3300... | |
| [2024-12-29 17:34:30,970][45646] Num frames 3400... | |
| [2024-12-29 17:34:31,063][45646] Num frames 3500... | |
| [2024-12-29 17:34:31,157][45646] Num frames 3600... | |
| [2024-12-29 17:34:31,251][45646] Num frames 3700... | |
| [2024-12-29 17:34:31,343][45646] Num frames 3800... | |
| [2024-12-29 17:34:31,439][45646] Num frames 3900... | |
| [2024-12-29 17:34:31,566][45646] Avg episode rewards: #0: 22.443, true rewards: #0: 9.942 | |
| [2024-12-29 17:34:31,567][45646] Avg episode reward: 22.443, avg true_objective: 9.942 | |
| [2024-12-29 17:34:31,588][45646] Num frames 4000... | |
| [2024-12-29 17:34:31,682][45646] Num frames 4100... | |
| [2024-12-29 17:34:31,775][45646] Num frames 4200... | |
| [2024-12-29 17:34:31,868][45646] Num frames 4300... | |
| [2024-12-29 17:34:31,961][45646] Num frames 4400... | |
| [2024-12-29 17:34:32,055][45646] Num frames 4500... | |
| [2024-12-29 17:34:32,147][45646] Num frames 4600... | |
| [2024-12-29 17:34:32,241][45646] Num frames 4700... | |
| [2024-12-29 17:34:32,336][45646] Num frames 4800... | |
| [2024-12-29 17:34:32,429][45646] Num frames 4900... | |
| [2024-12-29 17:34:32,522][45646] Num frames 5000... | |
| [2024-12-29 17:34:32,618][45646] Num frames 5100... | |
| [2024-12-29 17:34:32,711][45646] Num frames 5200... | |
| [2024-12-29 17:34:32,804][45646] Num frames 5300... | |
| [2024-12-29 17:34:32,908][45646] Avg episode rewards: #0: 24.506, true rewards: #0: 10.706 | |
| [2024-12-29 17:34:32,908][45646] Avg episode reward: 24.506, avg true_objective: 10.706 | |
| [2024-12-29 17:34:32,952][45646] Num frames 5400... | |
| [2024-12-29 17:34:33,046][45646] Num frames 5500... | |
| [2024-12-29 17:34:33,141][45646] Num frames 5600... | |
| [2024-12-29 17:34:33,233][45646] Num frames 5700... | |
| [2024-12-29 17:34:33,327][45646] Num frames 5800... | |
| [2024-12-29 17:34:33,421][45646] Num frames 5900... | |
| [2024-12-29 17:34:33,515][45646] Num frames 6000... | |
| [2024-12-29 17:34:33,609][45646] Num frames 6100... | |
| [2024-12-29 17:34:33,702][45646] Num frames 6200... | |
| [2024-12-29 17:34:33,796][45646] Num frames 6300... | |
| [2024-12-29 17:34:33,888][45646] Num frames 6400... | |
| [2024-12-29 17:34:33,984][45646] Num frames 6500... | |
| [2024-12-29 17:34:34,078][45646] Num frames 6600... | |
| [2024-12-29 17:34:34,172][45646] Num frames 6700... | |
| [2024-12-29 17:34:34,265][45646] Num frames 6800... | |
| [2024-12-29 17:34:34,359][45646] Num frames 6900... | |
| [2024-12-29 17:34:34,452][45646] Num frames 7000... | |
| [2024-12-29 17:34:34,585][45646] Avg episode rewards: #0: 27.475, true rewards: #0: 11.808 | |
| [2024-12-29 17:34:34,586][45646] Avg episode reward: 27.475, avg true_objective: 11.808 | |
| [2024-12-29 17:34:34,600][45646] Num frames 7100... | |
| [2024-12-29 17:34:34,692][45646] Num frames 7200... | |
| [2024-12-29 17:34:34,785][45646] Num frames 7300... | |
| [2024-12-29 17:34:34,878][45646] Num frames 7400... | |
| [2024-12-29 17:34:34,970][45646] Num frames 7500... | |
| [2024-12-29 17:34:35,113][45646] Avg episode rewards: #0: 24.996, true rewards: #0: 10.853 | |
| [2024-12-29 17:34:35,113][45646] Avg episode reward: 24.996, avg true_objective: 10.853 | |
| [2024-12-29 17:34:35,116][45646] Num frames 7600... | |
| [2024-12-29 17:34:35,209][45646] Num frames 7700... | |
| [2024-12-29 17:34:35,302][45646] Num frames 7800... | |
| [2024-12-29 17:34:35,396][45646] Num frames 7900... | |
| [2024-12-29 17:34:35,489][45646] Num frames 8000... | |
| [2024-12-29 17:34:35,582][45646] Num frames 8100... | |
| [2024-12-29 17:34:35,674][45646] Num frames 8200... | |
| [2024-12-29 17:34:35,766][45646] Num frames 8300... | |
| [2024-12-29 17:34:35,862][45646] Num frames 8400... | |
| [2024-12-29 17:34:35,955][45646] Num frames 8500... | |
| [2024-12-29 17:34:36,049][45646] Num frames 8600... | |
| [2024-12-29 17:34:36,122][45646] Avg episode rewards: #0: 24.776, true rewards: #0: 10.776 | |
| [2024-12-29 17:34:36,122][45646] Avg episode reward: 24.776, avg true_objective: 10.776 | |
| [2024-12-29 17:34:36,196][45646] Num frames 8700... | |
| [2024-12-29 17:34:36,289][45646] Num frames 8800... | |
| [2024-12-29 17:34:36,381][45646] Num frames 8900... | |
| [2024-12-29 17:34:36,475][45646] Num frames 9000... | |
| [2024-12-29 17:34:36,569][45646] Num frames 9100... | |
| [2024-12-29 17:34:36,662][45646] Num frames 9200... | |
| [2024-12-29 17:34:36,755][45646] Num frames 9300... | |
| [2024-12-29 17:34:36,846][45646] Num frames 9400... | |
| [2024-12-29 17:34:36,938][45646] Num frames 9500... | |
| [2024-12-29 17:34:37,030][45646] Num frames 9600... | |
| [2024-12-29 17:34:37,122][45646] Num frames 9700... | |
| [2024-12-29 17:34:37,182][45646] Avg episode rewards: #0: 24.786, true rewards: #0: 10.786 | |
| [2024-12-29 17:34:37,183][45646] Avg episode reward: 24.786, avg true_objective: 10.786 | |
| [2024-12-29 17:34:37,269][45646] Num frames 9800... | |
| [2024-12-29 17:34:37,362][45646] Num frames 9900... | |
| [2024-12-29 17:34:37,454][45646] Num frames 10000... | |
| [2024-12-29 17:34:37,546][45646] Num frames 10100... | |
| [2024-12-29 17:34:37,638][45646] Num frames 10200... | |
| [2024-12-29 17:34:37,731][45646] Num frames 10300... | |
| [2024-12-29 17:34:37,824][45646] Num frames 10400... | |
| [2024-12-29 17:34:37,916][45646] Num frames 10500... | |
| [2024-12-29 17:34:38,008][45646] Num frames 10600... | |
| [2024-12-29 17:34:38,094][45646] Avg episode rewards: #0: 24.235, true rewards: #0: 10.635 | |
| [2024-12-29 17:34:38,095][45646] Avg episode reward: 24.235, avg true_objective: 10.635 | |
| [2024-12-29 17:34:55,349][45646] Replay video saved to /fsx/users/amzfang/rl_course/train_dir/default_experiment/replay.mp4! | |
| [2024-12-29 17:37:23,187][45646] Loading existing experiment configuration from /fsx/users/amzfang/rl_course/train_dir/default_experiment/config.json | |
| [2024-12-29 17:38:19,600][45646] Loading existing experiment configuration from /fsx/users/amzfang/rl_course/train_dir/default_experiment/config.json | |
| [2024-12-29 17:38:19,601][45646] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2024-12-29 17:38:19,602][45646] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2024-12-29 17:38:19,602][45646] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2024-12-29 17:38:19,602][45646] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2024-12-29 17:38:19,603][45646] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2024-12-29 17:38:19,603][45646] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! | |
| [2024-12-29 17:38:19,604][45646] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2024-12-29 17:38:19,604][45646] Adding new argument 'push_to_hub'=True that is not in the saved config file! | |
| [2024-12-29 17:38:19,605][45646] Adding new argument 'hf_repository'='Fangliuwh/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! | |
| [2024-12-29 17:38:19,605][45646] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2024-12-29 17:38:19,605][45646] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2024-12-29 17:38:19,606][45646] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2024-12-29 17:38:19,606][45646] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2024-12-29 17:38:19,607][45646] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2024-12-29 17:38:19,637][45646] RunningMeanStd input shape: (3, 72, 128) | |
| [2024-12-29 17:38:19,638][45646] RunningMeanStd input shape: (1,) | |
| [2024-12-29 17:38:19,647][45646] ConvEncoder: input_channels=3 | |
| [2024-12-29 17:38:19,678][45646] Conv encoder output size: 512 | |
| [2024-12-29 17:38:19,682][45646] Policy head output size: 512 | |
| [2024-12-29 17:38:19,703][45646] Loading state from checkpoint /fsx/users/amzfang/rl_course/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2024-12-29 17:38:20,110][45646] Num frames 100... | |
| [2024-12-29 17:38:20,204][45646] Num frames 200... | |
| [2024-12-29 17:38:20,296][45646] Num frames 300... | |
| [2024-12-29 17:38:20,387][45646] Num frames 400... | |
| [2024-12-29 17:38:20,481][45646] Avg episode rewards: #0: 8.420, true rewards: #0: 4.420 | |
| [2024-12-29 17:38:20,481][45646] Avg episode reward: 8.420, avg true_objective: 4.420 | |
| [2024-12-29 17:38:20,534][45646] Num frames 500... | |
| [2024-12-29 17:38:20,627][45646] Num frames 600... | |
| [2024-12-29 17:38:20,720][45646] Num frames 700... | |
| [2024-12-29 17:38:20,813][45646] Num frames 800... | |
| [2024-12-29 17:38:20,905][45646] Num frames 900... | |
| [2024-12-29 17:38:20,997][45646] Num frames 1000... | |
| [2024-12-29 17:38:21,089][45646] Num frames 1100... | |
| [2024-12-29 17:38:21,225][45646] Avg episode rewards: #0: 12.955, true rewards: #0: 5.955 | |
| [2024-12-29 17:38:21,226][45646] Avg episode reward: 12.955, avg true_objective: 5.955 | |
| [2024-12-29 17:38:21,234][45646] Num frames 1200... | |
| [2024-12-29 17:38:21,327][45646] Num frames 1300... | |
| [2024-12-29 17:38:21,419][45646] Num frames 1400... | |
| [2024-12-29 17:38:21,511][45646] Num frames 1500... | |
| [2024-12-29 17:38:21,603][45646] Num frames 1600... | |
| [2024-12-29 17:38:21,695][45646] Num frames 1700... | |
| [2024-12-29 17:38:21,787][45646] Num frames 1800... | |
| [2024-12-29 17:38:21,879][45646] Num frames 1900... | |
| [2024-12-29 17:38:21,973][45646] Num frames 2000... | |
| [2024-12-29 17:38:22,065][45646] Num frames 2100... | |
| [2024-12-29 17:38:22,157][45646] Num frames 2200... | |
| [2024-12-29 17:38:22,250][45646] Num frames 2300... | |
| [2024-12-29 17:38:22,343][45646] Num frames 2400... | |
| [2024-12-29 17:38:22,437][45646] Num frames 2500... | |
| [2024-12-29 17:38:22,530][45646] Num frames 2600... | |
| [2024-12-29 17:38:22,622][45646] Num frames 2700... | |
| [2024-12-29 17:38:22,715][45646] Num frames 2800... | |
| [2024-12-29 17:38:22,808][45646] Num frames 2900... | |
| [2024-12-29 17:38:22,900][45646] Num frames 3000... | |
| [2024-12-29 17:38:22,993][45646] Num frames 3100... | |
| [2024-12-29 17:38:23,056][45646] Avg episode rewards: #0: 22.370, true rewards: #0: 10.370 | |
| [2024-12-29 17:38:23,057][45646] Avg episode reward: 22.370, avg true_objective: 10.370 | |
| [2024-12-29 17:38:23,140][45646] Num frames 3200... | |
| [2024-12-29 17:38:23,231][45646] Num frames 3300... | |
| [2024-12-29 17:38:23,323][45646] Num frames 3400... | |
| [2024-12-29 17:38:23,417][45646] Num frames 3500... | |
| [2024-12-29 17:38:23,509][45646] Num frames 3600... | |
| [2024-12-29 17:38:23,600][45646] Num frames 3700... | |
| [2024-12-29 17:38:23,695][45646] Num frames 3800... | |
| [2024-12-29 17:38:23,787][45646] Num frames 3900... | |
| [2024-12-29 17:38:23,880][45646] Num frames 4000... | |
| [2024-12-29 17:38:23,973][45646] Num frames 4100... | |
| [2024-12-29 17:38:24,064][45646] Num frames 4200... | |
| [2024-12-29 17:38:24,157][45646] Num frames 4300... | |
| [2024-12-29 17:38:24,236][45646] Avg episode rewards: #0: 22.817, true rewards: #0: 10.817 | |
| [2024-12-29 17:38:24,236][45646] Avg episode reward: 22.817, avg true_objective: 10.817 | |
| [2024-12-29 17:38:24,304][45646] Num frames 4400... | |
| [2024-12-29 17:38:24,398][45646] Num frames 4500... | |
| [2024-12-29 17:38:24,490][45646] Num frames 4600... | |
| [2024-12-29 17:38:24,582][45646] Num frames 4700... | |
| [2024-12-29 17:38:24,674][45646] Num frames 4800... | |
| [2024-12-29 17:38:24,767][45646] Num frames 4900... | |
| [2024-12-29 17:38:24,861][45646] Num frames 5000... | |
| [2024-12-29 17:38:24,954][45646] Num frames 5100... | |
| [2024-12-29 17:38:25,047][45646] Num frames 5200... | |
| [2024-12-29 17:38:25,130][45646] Avg episode rewards: #0: 22.862, true rewards: #0: 10.462 | |
| [2024-12-29 17:38:25,130][45646] Avg episode reward: 22.862, avg true_objective: 10.462 | |
| [2024-12-29 17:38:25,193][45646] Num frames 5300... | |
| [2024-12-29 17:38:25,286][45646] Num frames 5400... | |
| [2024-12-29 17:38:25,378][45646] Num frames 5500... | |
| [2024-12-29 17:38:25,471][45646] Num frames 5600... | |
| [2024-12-29 17:38:25,562][45646] Num frames 5700... | |
| [2024-12-29 17:38:25,654][45646] Num frames 5800... | |
| [2024-12-29 17:38:25,746][45646] Num frames 5900... | |
| [2024-12-29 17:38:25,839][45646] Num frames 6000... | |
| [2024-12-29 17:38:25,931][45646] Num frames 6100... | |
| [2024-12-29 17:38:26,071][45646] Avg episode rewards: #0: 22.485, true rewards: #0: 10.318 | |
| [2024-12-29 17:38:26,072][45646] Avg episode reward: 22.485, avg true_objective: 10.318 | |
| [2024-12-29 17:38:26,080][45646] Num frames 6200... | |
| [2024-12-29 17:38:26,172][45646] Num frames 6300... | |
| [2024-12-29 17:38:26,264][45646] Num frames 6400... | |
| [2024-12-29 17:38:26,356][45646] Num frames 6500... | |
| [2024-12-29 17:38:26,448][45646] Num frames 6600... | |
| [2024-12-29 17:38:26,540][45646] Num frames 6700... | |
| [2024-12-29 17:38:26,632][45646] Num frames 6800... | |
| [2024-12-29 17:38:26,725][45646] Num frames 6900... | |
| [2024-12-29 17:38:26,818][45646] Num frames 7000... | |
| [2024-12-29 17:38:26,913][45646] Num frames 7100... | |
| [2024-12-29 17:38:27,005][45646] Num frames 7200... | |
| [2024-12-29 17:38:27,098][45646] Num frames 7300... | |
| [2024-12-29 17:38:27,191][45646] Num frames 7400... | |
| [2024-12-29 17:38:27,286][45646] Num frames 7500... | |
| [2024-12-29 17:38:27,378][45646] Num frames 7600... | |
| [2024-12-29 17:38:27,473][45646] Num frames 7700... | |
| [2024-12-29 17:38:27,566][45646] Num frames 7800... | |
| [2024-12-29 17:38:27,661][45646] Num frames 7900... | |
| [2024-12-29 17:38:27,755][45646] Num frames 8000... | |
| [2024-12-29 17:38:27,849][45646] Num frames 8100... | |
| [2024-12-29 17:38:27,943][45646] Num frames 8200... | |
| [2024-12-29 17:38:28,083][45646] Avg episode rewards: #0: 27.844, true rewards: #0: 11.844 | |
| [2024-12-29 17:38:28,083][45646] Avg episode reward: 27.844, avg true_objective: 11.844 | |
| [2024-12-29 17:38:28,092][45646] Num frames 8300... | |
| [2024-12-29 17:38:28,184][45646] Num frames 8400... | |
| [2024-12-29 17:38:28,276][45646] Num frames 8500... | |
| [2024-12-29 17:38:28,368][45646] Num frames 8600... | |
| [2024-12-29 17:38:28,461][45646] Num frames 8700... | |
| [2024-12-29 17:38:28,579][45646] Avg episode rewards: #0: 25.587, true rewards: #0: 10.962 | |
| [2024-12-29 17:38:28,580][45646] Avg episode reward: 25.587, avg true_objective: 10.962 | |
| [2024-12-29 17:38:28,607][45646] Num frames 8800... | |
| [2024-12-29 17:38:28,698][45646] Num frames 8900... | |
| [2024-12-29 17:38:28,791][45646] Num frames 9000... | |
| [2024-12-29 17:38:28,885][45646] Num frames 9100... | |
| [2024-12-29 17:38:28,977][45646] Num frames 9200... | |
| [2024-12-29 17:38:29,070][45646] Num frames 9300... | |
| [2024-12-29 17:38:29,162][45646] Num frames 9400... | |
| [2024-12-29 17:38:29,255][45646] Num frames 9500... | |
| [2024-12-29 17:38:29,360][45646] Avg episode rewards: #0: 24.284, true rewards: #0: 10.618 | |
| [2024-12-29 17:38:29,361][45646] Avg episode reward: 24.284, avg true_objective: 10.618 | |
| [2024-12-29 17:38:29,402][45646] Num frames 9600... | |
| [2024-12-29 17:38:29,495][45646] Num frames 9700... | |
| [2024-12-29 17:38:29,587][45646] Num frames 9800... | |
| [2024-12-29 17:38:29,679][45646] Num frames 9900... | |
| [2024-12-29 17:38:29,772][45646] Num frames 10000... | |
| [2024-12-29 17:38:29,865][45646] Num frames 10100... | |
| [2024-12-29 17:38:29,959][45646] Num frames 10200... | |
| [2024-12-29 17:38:30,052][45646] Num frames 10300... | |
| [2024-12-29 17:38:30,144][45646] Num frames 10400... | |
| [2024-12-29 17:38:30,245][45646] Avg episode rewards: #0: 24.152, true rewards: #0: 10.452 | |
| [2024-12-29 17:38:30,246][45646] Avg episode reward: 24.152, avg true_objective: 10.452 | |
| [2024-12-29 17:38:47,035][45646] Replay video saved to /fsx/users/amzfang/rl_course/train_dir/default_experiment/replay.mp4! | |