| 2024-07-20 01:15:10,180 INFO StreamThr :179 [internal.py:wandb_internal():85] W&B internal server running at pid: 179, started at: 2024-07-20 01:15:10.179274 | |
| 2024-07-20 01:15:10,184 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status | |
| 2024-07-20 01:15:10,865 INFO WriterThread:179 [datastore.py:open_for_write():87] open: /kaggle/working/wandb/run-20240720_011510-w16tb06h/run-w16tb06h.wandb | |
| 2024-07-20 01:15:10,867 DEBUG SenderThread:179 [sender.py:send():379] send: header | |
| 2024-07-20 01:15:10,869 DEBUG SenderThread:179 [sender.py:send():379] send: run | |
| 2024-07-20 01:15:11,108 INFO SenderThread:179 [dir_watcher.py:__init__():211] watching files in: /kaggle/working/wandb/run-20240720_011510-w16tb06h/files | |
| 2024-07-20 01:15:11,109 INFO SenderThread:179 [sender.py:_start_run_threads():1188] run started: w16tb06h with start time 1721438110.180731 | |
| 2024-07-20 01:15:11,121 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: check_version | |
| 2024-07-20 01:15:11,121 DEBUG SenderThread:179 [sender.py:send_request():406] send_request: check_version | |
| 2024-07-20 01:15:11,186 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: run_start | |
| 2024-07-20 01:15:11,198 DEBUG HandlerThread:179 [system_info.py:__init__():26] System info init | |
| 2024-07-20 01:15:11,198 DEBUG HandlerThread:179 [system_info.py:__init__():41] System info init done | |
| 2024-07-20 01:15:11,198 INFO HandlerThread:179 [system_monitor.py:start():194] Starting system monitor | |
| 2024-07-20 01:15:11,199 INFO SystemMonitor:179 [system_monitor.py:_start():158] Starting system asset monitoring threads | |
| 2024-07-20 01:15:11,199 INFO HandlerThread:179 [system_monitor.py:probe():214] Collecting system info | |
| 2024-07-20 01:15:11,199 INFO SystemMonitor:179 [interfaces.py:start():188] Started cpu monitoring | |
| 2024-07-20 01:15:11,200 INFO SystemMonitor:179 [interfaces.py:start():188] Started disk monitoring | |
| 2024-07-20 01:15:11,201 INFO SystemMonitor:179 [interfaces.py:start():188] Started gpu monitoring | |
| 2024-07-20 01:15:11,202 INFO SystemMonitor:179 [interfaces.py:start():188] Started memory monitoring | |
| 2024-07-20 01:15:11,203 INFO SystemMonitor:179 [interfaces.py:start():188] Started network monitoring | |
| 2024-07-20 01:15:11,207 DEBUG HandlerThread:179 [system_info.py:probe():152] Probing system | |
| 2024-07-20 01:15:11,210 DEBUG HandlerThread:179 [gitlib.py:_init_repo():56] git repository is invalid | |
| 2024-07-20 01:15:11,210 DEBUG HandlerThread:179 [system_info.py:probe():200] Probing system done | |
| 2024-07-20 01:15:11,210 DEBUG HandlerThread:179 [system_monitor.py:probe():223] {'os': 'Linux-5.15.154+-x86_64-with-glibc2.31', 'python': '3.10.13', 'heartbeatAt': '2024-07-20T01:15:11.207840', 'startedAt': '2024-07-20T01:15:10.170069', 'docker': None, 'cuda': None, 'args': (), 'state': 'running', 'program': 'kaggle.ipynb', 'codePathLocal': None, 'root': '/kaggle/working', 'host': '3bc9163d6d69', 'username': 'root', 'executable': '/opt/conda/bin/python3.10', 'cpu_count': 2, 'cpu_count_logical': 4, 'cpu_freq': {'current': 2000.134, 'min': 0.0, 'max': 0.0}, 'cpu_freq_per_core': [{'current': 2000.134, 'min': 0.0, 'max': 0.0}, {'current': 2000.134, 'min': 0.0, 'max': 0.0}, {'current': 2000.134, 'min': 0.0, 'max': 0.0}, {'current': 2000.134, 'min': 0.0, 'max': 0.0}], 'disk': {'/': {'total': 8062.387607574463, 'used': 5777.521297454834}}, 'gpu': 'Tesla P100-PCIE-16GB', 'gpu_count': 1, 'gpu_devices': [{'name': 'Tesla P100-PCIE-16GB', 'memory_total': 17179869184}], 'memory': {'total': 31.357574462890625}} | |
| 2024-07-20 01:15:11,211 INFO HandlerThread:179 [system_monitor.py:probe():224] Finished collecting system info | |
| 2024-07-20 01:15:11,211 INFO HandlerThread:179 [system_monitor.py:probe():227] Publishing system info | |
| 2024-07-20 01:15:11,211 DEBUG HandlerThread:179 [system_info.py:_save_conda():209] Saving list of conda packages installed into the current environment | |
| 2024-07-20 01:15:12,110 INFO Thread-12 :179 [dir_watcher.py:_on_file_created():271] file/dir created: /kaggle/working/wandb/run-20240720_011510-w16tb06h/files/conda-environment.yaml | |
| 2024-07-20 01:15:26,225 ERROR HandlerThread:179 [system_info.py:_save_conda():223] Error saving conda packages: Command '['conda', 'env', 'export']' timed out after 15 seconds | |
| Traceback (most recent call last): | |
| File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/internal/system/system_info.py", line 216, in _save_conda | |
| subprocess.call( | |
| File "/opt/conda/lib/python3.10/subprocess.py", line 347, in call | |
| return p.wait(timeout=timeout) | |
| File "/opt/conda/lib/python3.10/subprocess.py", line 1209, in wait | |
| return self._wait(timeout=timeout) | |
| File "/opt/conda/lib/python3.10/subprocess.py", line 1951, in _wait | |
| raise TimeoutExpired(self.args, timeout) | |
| subprocess.TimeoutExpired: Command '['conda', 'env', 'export']' timed out after 15 seconds | |
| 2024-07-20 01:15:26,228 DEBUG HandlerThread:179 [system_info.py:_save_conda():224] Saving conda packages done | |
| 2024-07-20 01:15:26,228 INFO HandlerThread:179 [system_monitor.py:probe():229] Finished publishing system info | |
| 2024-07-20 01:15:26,237 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:15:26,237 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: keepalive | |
| 2024-07-20 01:15:26,237 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:15:26,237 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: keepalive | |
| 2024-07-20 01:15:26,237 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:15:26,237 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: keepalive | |
| 2024-07-20 01:15:26,238 DEBUG SenderThread:179 [sender.py:send():379] send: files | |
| 2024-07-20 01:15:26,238 INFO SenderThread:179 [sender.py:_save_file():1454] saving file wandb-metadata.json with policy now | |
| 2024-07-20 01:15:26,565 INFO wandb-upload_0:179 [upload_job.py:push():130] Uploaded file /tmp/tmp2iby7g0owandb/cuy1wkdh-wandb-metadata.json | |
| 2024-07-20 01:15:27,114 INFO Thread-12 :179 [dir_watcher.py:_on_file_created():271] file/dir created: /kaggle/working/wandb/run-20240720_011510-w16tb06h/files/wandb-metadata.json | |
| 2024-07-20 01:15:27,204 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: python_packages | |
| 2024-07-20 01:15:27,205 DEBUG SenderThread:179 [sender.py:send_request():406] send_request: python_packages | |
| 2024-07-20 01:15:27,207 DEBUG SenderThread:179 [sender.py:send():379] send: telemetry | |
| 2024-07-20 01:15:27,208 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: stop_status | |
| 2024-07-20 01:15:27,208 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:27,209 DEBUG SenderThread:179 [sender.py:send_request():406] send_request: stop_status | |
| 2024-07-20 01:15:27,215 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: pause | |
| 2024-07-20 01:15:27,215 INFO HandlerThread:179 [handler.py:handle_request_pause():724] stopping system metrics thread | |
| 2024-07-20 01:15:27,215 INFO HandlerThread:179 [system_monitor.py:finish():203] Stopping system monitor | |
| 2024-07-20 01:15:27,215 DEBUG SystemMonitor:179 [system_monitor.py:_start():172] Starting system metrics aggregation loop | |
| 2024-07-20 01:15:27,215 DEBUG SystemMonitor:179 [system_monitor.py:_start():179] Finished system metrics aggregation loop | |
| 2024-07-20 01:15:27,215 DEBUG SystemMonitor:179 [system_monitor.py:_start():183] Publishing last batch of metrics | |
| 2024-07-20 01:15:27,219 INFO HandlerThread:179 [interfaces.py:finish():200] Joined cpu monitor | |
| 2024-07-20 01:15:27,219 INFO HandlerThread:179 [interfaces.py:finish():200] Joined disk monitor | |
| 2024-07-20 01:15:27,222 INFO HandlerThread:179 [interfaces.py:finish():200] Joined gpu monitor | |
| 2024-07-20 01:15:27,223 INFO HandlerThread:179 [interfaces.py:finish():200] Joined memory monitor | |
| 2024-07-20 01:15:27,223 INFO HandlerThread:179 [interfaces.py:finish():200] Joined network monitor | |
| 2024-07-20 01:15:28,114 INFO Thread-12 :179 [dir_watcher.py:_on_file_created():271] file/dir created: /kaggle/working/wandb/run-20240720_011510-w16tb06h/files/requirements.txt | |
| 2024-07-20 01:15:28,206 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:29,206 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:30,206 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:31,207 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:31,332 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:15:32,207 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:33,207 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:34,207 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:35,207 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:36,208 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:37,206 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:15:37,207 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:38,208 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:39,208 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:40,208 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:41,208 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:42,205 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: stop_status | |
| 2024-07-20 01:15:42,206 DEBUG SenderThread:179 [sender.py:send_request():406] send_request: stop_status | |
| 2024-07-20 01:15:42,246 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:42,347 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:15:42,728 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: resume | |
| 2024-07-20 01:15:42,728 INFO HandlerThread:179 [handler.py:handle_request_resume():715] starting system metrics thread | |
| 2024-07-20 01:15:42,728 INFO HandlerThread:179 [system_monitor.py:start():194] Starting system monitor | |
| 2024-07-20 01:15:42,729 INFO SystemMonitor:179 [system_monitor.py:_start():158] Starting system asset monitoring threads | |
| 2024-07-20 01:15:42,729 INFO SystemMonitor:179 [interfaces.py:start():188] Started cpu monitoring | |
| 2024-07-20 01:15:42,730 INFO SystemMonitor:179 [interfaces.py:start():188] Started disk monitoring | |
| 2024-07-20 01:15:42,731 INFO SystemMonitor:179 [interfaces.py:start():188] Started gpu monitoring | |
| 2024-07-20 01:15:42,731 INFO SystemMonitor:179 [interfaces.py:start():188] Started memory monitoring | |
| 2024-07-20 01:15:42,732 INFO SystemMonitor:179 [interfaces.py:start():188] Started network monitoring | |
| 2024-07-20 01:15:42,770 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: pause | |
| 2024-07-20 01:15:42,770 INFO HandlerThread:179 [handler.py:handle_request_pause():724] stopping system metrics thread | |
| 2024-07-20 01:15:42,770 INFO HandlerThread:179 [system_monitor.py:finish():203] Stopping system monitor | |
| 2024-07-20 01:15:42,770 DEBUG SystemMonitor:179 [system_monitor.py:_start():172] Starting system metrics aggregation loop | |
| 2024-07-20 01:15:42,770 DEBUG SystemMonitor:179 [system_monitor.py:_start():179] Finished system metrics aggregation loop | |
| 2024-07-20 01:15:42,770 DEBUG SystemMonitor:179 [system_monitor.py:_start():183] Publishing last batch of metrics | |
| 2024-07-20 01:15:42,772 INFO HandlerThread:179 [interfaces.py:finish():200] Joined cpu monitor | |
| 2024-07-20 01:15:42,772 INFO HandlerThread:179 [interfaces.py:finish():200] Joined disk monitor | |
| 2024-07-20 01:15:42,776 INFO HandlerThread:179 [interfaces.py:finish():200] Joined gpu monitor | |
| 2024-07-20 01:15:42,776 INFO HandlerThread:179 [interfaces.py:finish():200] Joined memory monitor | |
| 2024-07-20 01:15:42,777 INFO HandlerThread:179 [interfaces.py:finish():200] Joined network monitor | |
| 2024-07-20 01:15:42,777 DEBUG SenderThread:179 [sender.py:send():379] send: stats | |
| 2024-07-20 01:15:43,119 INFO Thread-12 :179 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240720_011510-w16tb06h/files/config.yaml | |
| 2024-07-20 01:15:43,209 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:44,209 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:45,209 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:45,925 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: resume | |
| 2024-07-20 01:15:45,925 INFO HandlerThread:179 [handler.py:handle_request_resume():715] starting system metrics thread | |
| 2024-07-20 01:15:45,925 INFO HandlerThread:179 [system_monitor.py:start():194] Starting system monitor | |
| 2024-07-20 01:15:45,926 INFO SystemMonitor:179 [system_monitor.py:_start():158] Starting system asset monitoring threads | |
| 2024-07-20 01:15:45,926 INFO SystemMonitor:179 [interfaces.py:start():188] Started cpu monitoring | |
| 2024-07-20 01:15:45,927 INFO SystemMonitor:179 [interfaces.py:start():188] Started disk monitoring | |
| 2024-07-20 01:15:45,928 INFO SystemMonitor:179 [interfaces.py:start():188] Started gpu monitoring | |
| 2024-07-20 01:15:45,929 INFO SystemMonitor:179 [interfaces.py:start():188] Started memory monitoring | |
| 2024-07-20 01:15:45,932 INFO SystemMonitor:179 [interfaces.py:start():188] Started network monitoring | |
| 2024-07-20 01:15:45,966 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: pause | |
| 2024-07-20 01:15:45,966 INFO HandlerThread:179 [handler.py:handle_request_pause():724] stopping system metrics thread | |
| 2024-07-20 01:15:45,966 INFO HandlerThread:179 [system_monitor.py:finish():203] Stopping system monitor | |
| 2024-07-20 01:15:45,967 DEBUG SystemMonitor:179 [system_monitor.py:_start():172] Starting system metrics aggregation loop | |
| 2024-07-20 01:15:45,967 DEBUG SystemMonitor:179 [system_monitor.py:_start():179] Finished system metrics aggregation loop | |
| 2024-07-20 01:15:45,967 DEBUG SystemMonitor:179 [system_monitor.py:_start():183] Publishing last batch of metrics | |
| 2024-07-20 01:15:45,968 INFO HandlerThread:179 [interfaces.py:finish():200] Joined cpu monitor | |
| 2024-07-20 01:15:45,968 INFO HandlerThread:179 [interfaces.py:finish():200] Joined disk monitor | |
| 2024-07-20 01:15:45,973 INFO HandlerThread:179 [interfaces.py:finish():200] Joined gpu monitor | |
| 2024-07-20 01:15:45,973 INFO HandlerThread:179 [interfaces.py:finish():200] Joined memory monitor | |
| 2024-07-20 01:15:45,973 INFO HandlerThread:179 [interfaces.py:finish():200] Joined network monitor | |
| 2024-07-20 01:15:45,973 DEBUG SenderThread:179 [sender.py:send():379] send: stats | |
| 2024-07-20 01:15:46,209 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:47,209 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:48,207 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:15:48,209 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:49,210 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:50,210 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:51,210 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:52,210 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:52,509 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: resume | |
| 2024-07-20 01:15:52,509 INFO HandlerThread:179 [handler.py:handle_request_resume():715] starting system metrics thread | |
| 2024-07-20 01:15:52,509 INFO HandlerThread:179 [system_monitor.py:start():194] Starting system monitor | |
| 2024-07-20 01:15:52,510 INFO SystemMonitor:179 [system_monitor.py:_start():158] Starting system asset monitoring threads | |
| 2024-07-20 01:15:52,510 INFO SystemMonitor:179 [interfaces.py:start():188] Started cpu monitoring | |
| 2024-07-20 01:15:52,511 INFO SystemMonitor:179 [interfaces.py:start():188] Started disk monitoring | |
| 2024-07-20 01:15:52,511 INFO SystemMonitor:179 [interfaces.py:start():188] Started gpu monitoring | |
| 2024-07-20 01:15:52,511 INFO SystemMonitor:179 [interfaces.py:start():188] Started memory monitoring | |
| 2024-07-20 01:15:52,512 INFO SystemMonitor:179 [interfaces.py:start():188] Started network monitoring | |
| 2024-07-20 01:15:52,550 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: pause | |
| 2024-07-20 01:15:52,550 INFO HandlerThread:179 [handler.py:handle_request_pause():724] stopping system metrics thread | |
| 2024-07-20 01:15:52,550 INFO HandlerThread:179 [system_monitor.py:finish():203] Stopping system monitor | |
| 2024-07-20 01:15:52,551 INFO HandlerThread:179 [interfaces.py:finish():200] Joined cpu monitor | |
| 2024-07-20 01:15:52,551 INFO HandlerThread:179 [interfaces.py:finish():200] Joined disk monitor | |
| 2024-07-20 01:15:52,551 DEBUG SystemMonitor:179 [system_monitor.py:_start():172] Starting system metrics aggregation loop | |
| 2024-07-20 01:15:52,551 DEBUG SystemMonitor:179 [system_monitor.py:_start():179] Finished system metrics aggregation loop | |
| 2024-07-20 01:15:52,551 DEBUG SystemMonitor:179 [system_monitor.py:_start():183] Publishing last batch of metrics | |
| 2024-07-20 01:15:52,556 INFO HandlerThread:179 [interfaces.py:finish():200] Joined gpu monitor | |
| 2024-07-20 01:15:52,556 INFO HandlerThread:179 [interfaces.py:finish():200] Joined memory monitor | |
| 2024-07-20 01:15:52,557 INFO HandlerThread:179 [interfaces.py:finish():200] Joined network monitor | |
| 2024-07-20 01:15:52,557 DEBUG SenderThread:179 [sender.py:send():379] send: stats | |
| 2024-07-20 01:15:53,210 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:53,558 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:15:54,210 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:54,699 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: resume | |
| 2024-07-20 01:15:54,699 INFO HandlerThread:179 [handler.py:handle_request_resume():715] starting system metrics thread | |
| 2024-07-20 01:15:54,699 INFO HandlerThread:179 [system_monitor.py:start():194] Starting system monitor | |
| 2024-07-20 01:15:54,699 INFO SystemMonitor:179 [system_monitor.py:_start():158] Starting system asset monitoring threads | |
| 2024-07-20 01:15:54,699 INFO SystemMonitor:179 [interfaces.py:start():188] Started cpu monitoring | |
| 2024-07-20 01:15:54,700 INFO SystemMonitor:179 [interfaces.py:start():188] Started disk monitoring | |
| 2024-07-20 01:15:54,704 INFO SystemMonitor:179 [interfaces.py:start():188] Started gpu monitoring | |
| 2024-07-20 01:15:54,704 INFO SystemMonitor:179 [interfaces.py:start():188] Started memory monitoring | |
| 2024-07-20 01:15:54,708 INFO SystemMonitor:179 [interfaces.py:start():188] Started network monitoring | |
| 2024-07-20 01:15:55,211 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:55,259 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: pause | |
| 2024-07-20 01:15:55,260 INFO HandlerThread:179 [handler.py:handle_request_pause():724] stopping system metrics thread | |
| 2024-07-20 01:15:55,260 INFO HandlerThread:179 [system_monitor.py:finish():203] Stopping system monitor | |
| 2024-07-20 01:15:55,260 INFO HandlerThread:179 [interfaces.py:finish():200] Joined cpu monitor | |
| 2024-07-20 01:15:55,260 DEBUG SystemMonitor:179 [system_monitor.py:_start():172] Starting system metrics aggregation loop | |
| 2024-07-20 01:15:55,260 INFO HandlerThread:179 [interfaces.py:finish():200] Joined disk monitor | |
| 2024-07-20 01:15:55,261 DEBUG SystemMonitor:179 [system_monitor.py:_start():179] Finished system metrics aggregation loop | |
| 2024-07-20 01:15:55,261 DEBUG SystemMonitor:179 [system_monitor.py:_start():183] Publishing last batch of metrics | |
| 2024-07-20 01:15:55,266 INFO HandlerThread:179 [interfaces.py:finish():200] Joined gpu monitor | |
| 2024-07-20 01:15:55,267 INFO HandlerThread:179 [interfaces.py:finish():200] Joined memory monitor | |
| 2024-07-20 01:15:55,267 INFO HandlerThread:179 [interfaces.py:finish():200] Joined network monitor | |
| 2024-07-20 01:15:55,267 DEBUG SenderThread:179 [sender.py:send():379] send: stats | |
| 2024-07-20 01:15:56,211 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:57,206 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: stop_status | |
| 2024-07-20 01:15:57,206 DEBUG SenderThread:179 [sender.py:send_request():406] send_request: stop_status | |
| 2024-07-20 01:15:57,246 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:58,211 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:58,284 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: resume | |
| 2024-07-20 01:15:58,284 INFO HandlerThread:179 [handler.py:handle_request_resume():715] starting system metrics thread | |
| 2024-07-20 01:15:58,284 INFO HandlerThread:179 [system_monitor.py:start():194] Starting system monitor | |
| 2024-07-20 01:15:58,285 INFO SystemMonitor:179 [system_monitor.py:_start():158] Starting system asset monitoring threads | |
| 2024-07-20 01:15:58,285 INFO SystemMonitor:179 [interfaces.py:start():188] Started cpu monitoring | |
| 2024-07-20 01:15:58,286 INFO SystemMonitor:179 [interfaces.py:start():188] Started disk monitoring | |
| 2024-07-20 01:15:58,286 INFO SystemMonitor:179 [interfaces.py:start():188] Started gpu monitoring | |
| 2024-07-20 01:15:58,287 INFO SystemMonitor:179 [interfaces.py:start():188] Started memory monitoring | |
| 2024-07-20 01:15:58,287 INFO SystemMonitor:179 [interfaces.py:start():188] Started network monitoring | |
| 2024-07-20 01:15:58,842 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:15:58,849 DEBUG SenderThread:179 [sender.py:send():379] send: config | |
| 2024-07-20 01:15:58,850 DEBUG SenderThread:179 [sender.py:send():379] send: telemetry | |
| 2024-07-20 01:15:58,850 DEBUG SenderThread:179 [sender.py:send():379] send: metric | |
| 2024-07-20 01:15:58,851 DEBUG SenderThread:179 [sender.py:send():379] send: telemetry | |
| 2024-07-20 01:15:58,851 DEBUG SenderThread:179 [sender.py:send():379] send: metric | |
| 2024-07-20 01:15:58,851 WARNING SenderThread:179 [sender.py:send_metric():1405] Seen metric with glob (shouldn't happen) | |
| 2024-07-20 01:15:58,852 DEBUG SenderThread:179 [sender.py:send():379] send: telemetry | |
| 2024-07-20 01:15:59,030 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: log_artifact | |
| 2024-07-20 01:15:59,030 DEBUG SenderThread:179 [sender.py:send_request():406] send_request: log_artifact | |
| 2024-07-20 01:15:59,125 INFO Thread-12 :179 [dir_watcher.py:_on_file_created():271] file/dir created: /kaggle/working/wandb/run-20240720_011510-w16tb06h/files/output.log | |
| 2024-07-20 01:15:59,211 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:15:59,701 INFO wandb-upload_0:179 [upload_job.py:push():88] Uploaded file /tmp/tmp_nkyjmf3/model_architecture.txt | |
| 2024-07-20 01:16:00,212 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:00,243 INFO SenderThread:179 [sender.py:send_request_log_artifact():1518] logged artifact model-w16tb06h - {'id': 'QXJ0aWZhY3Q6MTAwNDIwNjIwMg==', 'state': 'PENDING', 'artifactSequence': {'id': 'QXJ0aWZhY3RDb2xsZWN0aW9uOjI5MzEzNDE4MA==', 'latestArtifact': None}} | |
| 2024-07-20 01:16:01,126 INFO Thread-12 :179 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240720_011510-w16tb06h/files/output.log | |
| 2024-07-20 01:16:01,212 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:02,212 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:03,212 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:04,207 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:16:04,212 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:05,212 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:06,212 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:07,212 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:08,213 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:09,207 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:16:09,213 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:10,213 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:11,213 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:12,206 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: stop_status | |
| 2024-07-20 01:16:12,206 DEBUG SenderThread:179 [sender.py:send_request():406] send_request: stop_status | |
| 2024-07-20 01:16:12,246 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:13,213 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:14,213 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:14,352 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:16:15,131 INFO Thread-12 :179 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240720_011510-w16tb06h/files/config.yaml | |
| 2024-07-20 01:16:15,335 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:16,334 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:17,460 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:18,465 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:20,258 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:20,462 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:16:25,106 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:26,106 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:16:27,462 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: stop_status | |
| 2024-07-20 01:16:27,462 DEBUG SenderThread:179 [sender.py:send_request():406] send_request: stop_status | |
| 2024-07-20 01:16:27,502 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:30,835 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:31,189 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:32,067 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:16:32,091 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:32,911 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:33,911 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:34,911 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:35,911 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:36,911 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:37,613 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:16:37,912 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:38,912 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:39,912 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:40,912 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:41,913 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:42,229 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: stop_status | |
| 2024-07-20 01:16:42,230 DEBUG SenderThread:179 [sender.py:send_request():406] send_request: stop_status | |
| 2024-07-20 01:16:42,913 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:43,321 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:16:43,912 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:44,913 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:45,913 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:46,913 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:47,913 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:48,613 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:16:48,913 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:49,167 DEBUG SenderThread:179 [sender.py:send():379] send: telemetry | |
| 2024-07-20 01:16:49,208 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: partial_history | |
| 2024-07-20 01:16:49,209 DEBUG SenderThread:179 [sender.py:send_request():406] send_request: summary_record | |
| 2024-07-20 01:16:49,211 INFO SenderThread:179 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end | |
| 2024-07-20 01:16:49,212 DEBUG SenderThread:179 [sender.py:send_request():406] send_request: summary_record | |
| 2024-07-20 01:16:49,914 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:50,147 INFO Thread-12 :179 [dir_watcher.py:_on_file_created():271] file/dir created: /kaggle/working/wandb/run-20240720_011510-w16tb06h/files/wandb-summary.json | |
| 2024-07-20 01:16:50,914 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:51,914 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:52,914 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:53,914 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:54,914 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:55,914 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:56,915 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:57,229 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: stop_status | |
| 2024-07-20 01:16:57,915 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:58,287 DEBUG SystemMonitor:179 [system_monitor.py:_start():172] Starting system metrics aggregation loop | |
| 2024-07-20 01:16:58,915 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:16:59,915 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:00,915 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:01,915 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:02,915 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:03,916 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:04,916 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:05,916 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:06,513 INFO SenderThread:179 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end | |
| 2024-07-20 01:17:06,514 DEBUG SenderThread:179 [sender.py:send_request():406] send_request: summary_record | |
| 2024-07-20 01:17:06,514 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:17:06,515 INFO SenderThread:179 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end | |
| 2024-07-20 01:17:06,515 DEBUG SenderThread:179 [sender.py:send_request():406] send_request: summary_record | |
| 2024-07-20 01:17:06,521 INFO SenderThread:179 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end | |
| 2024-07-20 01:17:06,521 DEBUG SenderThread:179 [sender.py:send_request():406] send_request: summary_record | |
| 2024-07-20 01:17:06,555 INFO SenderThread:179 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end | |
| 2024-07-20 01:17:06,556 DEBUG SenderThread:179 [sender.py:send():379] send: metric | |
| 2024-07-20 01:17:06,556 DEBUG SenderThread:179 [sender.py:send():379] send: history | |
| 2024-07-20 01:17:06,556 DEBUG SenderThread:179 [sender.py:send_request():406] send_request: summary_record | |
| 2024-07-20 01:17:06,558 INFO SenderThread:179 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end | |
| 2024-07-20 01:17:06,558 DEBUG SenderThread:179 [sender.py:send_request():406] send_request: stop_status | |
| 2024-07-20 01:17:06,709 DEBUG SenderThread:179 [sender.py:send():379] send: stats | |
| 2024-07-20 01:17:06,916 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:07,153 INFO Thread-12 :179 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240720_011510-w16tb06h/files/wandb-summary.json | |
| 2024-07-20 01:17:07,916 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:08,916 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:09,917 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:10,917 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:11,711 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:17:11,917 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:12,229 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: stop_status | |
| 2024-07-20 01:17:12,230 DEBUG SenderThread:179 [sender.py:send_request():406] send_request: stop_status | |
| 2024-07-20 01:17:12,917 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:13,917 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:14,918 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:15,918 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:16,918 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:17,291 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:17:17,918 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:18,918 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:19,919 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:20,919 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:21,919 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:22,297 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:17:22,919 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:23,160 INFO Thread-12 :179 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240720_011510-w16tb06h/files/config.yaml | |
| 2024-07-20 01:17:23,919 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:24,919 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:25,919 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:26,920 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:27,230 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: stop_status | |
| 2024-07-20 01:17:27,230 DEBUG SenderThread:179 [sender.py:send_request():406] send_request: stop_status | |
| 2024-07-20 01:17:27,920 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:28,290 DEBUG SenderThread:179 [sender.py:send():379] send: stats | |
| 2024-07-20 01:17:28,290 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:17:28,920 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:29,920 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:30,921 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:31,921 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:32,921 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:33,291 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:17:33,921 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:34,921 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:35,921 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:36,922 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:37,922 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:38,292 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:17:38,922 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:39,922 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:40,922 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:41,923 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:42,230 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: stop_status | |
| 2024-07-20 01:17:42,230 DEBUG SenderThread:179 [sender.py:send_request():406] send_request: stop_status | |
| 2024-07-20 01:17:42,924 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:43,323 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:17:43,923 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:44,923 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:45,924 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:46,924 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:47,924 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:48,324 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:17:48,924 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:49,924 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:50,924 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:51,924 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:52,924 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:53,324 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:17:53,925 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:54,925 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:55,925 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:56,925 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:57,230 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: stop_status | |
| 2024-07-20 01:17:57,230 DEBUG SenderThread:179 [sender.py:send_request():406] send_request: stop_status | |
| 2024-07-20 01:17:57,926 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:58,290 DEBUG SenderThread:179 [sender.py:send():379] send: stats | |
| 2024-07-20 01:17:58,926 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:17:59,291 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:17:59,926 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:00,926 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:01,926 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:02,927 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:03,927 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:04,292 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:18:04,927 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:05,927 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:06,927 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:07,928 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:08,928 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:09,293 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:18:09,928 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:10,928 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:11,928 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:12,230 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: stop_status | |
| 2024-07-20 01:18:12,230 DEBUG SenderThread:179 [sender.py:send_request():406] send_request: stop_status | |
| 2024-07-20 01:18:12,929 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:13,929 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:14,357 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:18:14,929 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:15,929 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:16,929 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:17,929 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:18,930 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:19,358 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:18:19,930 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:20,930 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:21,930 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:22,930 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:23,930 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:24,359 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:18:24,931 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:25,931 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:26,931 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:27,230 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: stop_status | |
| 2024-07-20 01:18:27,231 DEBUG SenderThread:179 [sender.py:send_request():406] send_request: stop_status | |
| 2024-07-20 01:18:27,931 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:28,291 DEBUG SenderThread:179 [sender.py:send():379] send: stats | |
| 2024-07-20 01:18:28,931 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:29,932 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:30,292 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:18:30,932 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:31,932 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:32,932 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:33,932 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:34,932 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:35,293 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:18:35,932 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:36,933 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:37,933 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:38,933 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:39,934 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:40,294 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:18:40,933 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:41,934 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:42,230 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: stop_status | |
| 2024-07-20 01:18:42,231 DEBUG SenderThread:179 [sender.py:send_request():406] send_request: stop_status | |
| 2024-07-20 01:18:42,934 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:43,934 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:44,934 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:45,372 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: status_report | |
| 2024-07-20 01:18:45,934 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:46,934 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:47,935 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |
| 2024-07-20 01:18:48,935 DEBUG HandlerThread:179 [handler.py:handle_request():158] handle_request: internal_messages | |