2024-06-21 11:18:59,219 INFO StreamThr :26947 [internal.py:wandb_internal():88] W&B internal server running at pid: 26947, started at: 2024-06-21 11:18:59.219098 2024-06-21 11:18:59,220 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: status 2024-06-21 11:18:59,246 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: status 2024-06-21 11:18:59,247 INFO WriterThread:26947 [datastore.py:open_for_write():77] open: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/run-run_20240621_31f46a8f.wandb 2024-06-21 11:18:59,248 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: header 2024-06-21 11:18:59,248 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: run 2024-06-21 11:18:59,250 INFO SenderThread:26947 [tracking_sender.py:_maybe_setup_resume():588] checking resume status for None/auto_form/run_20240621_31f46a8f 2024-06-21 11:18:59,976 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: check_version 2024-06-21 11:18:59,981 INFO SenderThread:26947 [dir_watcher.py:__init__():216] watching files in: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files 2024-06-21 11:18:59,981 INFO SenderThread:26947 [tracking_sender.py:_start_run_threads():934] run started: run_20240621_31f46a8f with start time 1718939939.216538 2024-06-21 11:18:59,981 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:18:59,982 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:18:59,982 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: check_version 2024-06-21 11:19:00,982 INFO Thread-15 :26947 [dir_watcher.py:_on_file_created():275] file/dir created: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:19:04,979 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: run_start 2024-06-21 11:19:04,987 DEBUG HandlerThread:26947 [system_info.py:__init__():31] System info init 2024-06-21 11:19:04,987 DEBUG HandlerThread:26947 [system_info.py:__init__():46] System info init done 2024-06-21 11:19:04,987 INFO HandlerThread:26947 [system_monitor.py:start():150] Starting system monitor 2024-06-21 11:19:04,987 INFO SystemMonitor:26947 [system_monitor.py:_start():116] Starting system asset monitoring threads 2024-06-21 11:19:04,987 INFO HandlerThread:26947 [system_monitor.py:probe():168] Collecting system info 2024-06-21 11:19:04,987 INFO SystemMonitor:26947 [interfaces.py:start():168] Started cpu 2024-06-21 11:19:04,988 INFO SystemMonitor:26947 [interfaces.py:start():168] Started disk 2024-06-21 11:19:04,988 INFO SystemMonitor:26947 [interfaces.py:start():168] Started gpu 2024-06-21 11:19:04,989 INFO SystemMonitor:26947 [interfaces.py:start():168] Started memory 2024-06-21 11:19:04,991 INFO SystemMonitor:26947 [interfaces.py:start():168] Started network 2024-06-21 11:19:05,048 DEBUG HandlerThread:26947 [system_info.py:probe():195] Probing system 2024-06-21 11:19:05,054 DEBUG HandlerThread:26947 [git.py:repo():40] git repository is invalid 2024-06-21 11:19:05,054 DEBUG HandlerThread:26947 [system_info.py:probe():241] Probing system done 2024-06-21 11:19:05,054 DEBUG HandlerThread:26947 [system_monitor.py:probe():177] {'os': 'Linux-5.4.143.bsk.7-amd64-x86_64-with-glibc2.31', 'python': '3.9.2', 'heartbeatAt': '2024-06-21T03:19:05.049093', 'startedAt': '2024-06-21T03:18:59.208329', 'docker': None, 'cuda': None, 'args': ('--model_name_or_path', '../deepseek-math-7b-base/', '--data_path', 'autoform_data/autoform_forml4_train.json,autoform_data/qa_gsm8k_train.json,autoform_data/qa_math_train.json', '--data_length', '10000000', '--bf16', 'True', '--fp16', 'False', '--output_dir', '../deepseekmath-qa-gsm8kmath-autoform-forml4', '--num_train_epochs', '3', '--per_device_train_batch_size', '16', '--per_device_eval_batch_size', '1', '--gradient_accumulation_steps', '1', '--evaluation_strategy', 'no', '--save_strategy', 'epoch', '--save_total_limit', '3', '--learning_rate', '5e-6', '--weight_decay', '0.', '--use_reentrant', 'True', '--warmup_ratio', '0.03', '--lr_scheduler_type', 'cosine', '--logging_steps', '1', '--report_to', 'wandb', '--run_name', 'deepseekmath-qa-gsm8kmath-autoform-forml4', '--remove_unused_columns', 'False', '--gradient_checkpointing', 'True', '--fsdp', 'full_shard auto_wrap', '--fsdp_transformer_layer_cls_to_wrap', 'LlamaDecoderLayer', '--tf32', 'True'), 'state': 'running', 'program': '/opt/tiger/auto-info/qa_autoform_train.py', 'codePath': 'qa_autoform_train.py', 'host': 'n122-250-047', 'username': 'tiger', 'executable': '/usr/bin/python3', 'cpu_count': 60, 'cpu_count_logical': 120, 'cpu_freq': {'current': 2294.6080000000056, 'min': 0.0, 'max': 0.0}, 'cpu_freq_per_core': [{'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}, {'current': 2294.608, 'min': 0.0, 'max': 0.0}], 'disk': {'total': 3519.834815979004, 'used': 278.7707405090332}, 'gpu': 'NVIDIA A800-SXM4-80GB', 'gpu_count': 8, 'gpu_devices': [{'name': 'NVIDIA A800-SXM4-80GB', 'memory_total': 85198045184}, {'name': 'NVIDIA A800-SXM4-80GB', 'memory_total': 85198045184}, {'name': 'NVIDIA A800-SXM4-80GB', 'memory_total': 85198045184}, {'name': 'NVIDIA A800-SXM4-80GB', 'memory_total': 85198045184}, {'name': 'NVIDIA A800-SXM4-80GB', 'memory_total': 85198045184}, {'name': 'NVIDIA A800-SXM4-80GB', 'memory_total': 85198045184}, {'name': 'NVIDIA A800-SXM4-80GB', 'memory_total': 85198045184}, {'name': 'NVIDIA A800-SXM4-80GB', 'memory_total': 85198045184}], 'memory': {'total': 1928.9386100769043}} 2024-06-21 11:19:05,054 INFO HandlerThread:26947 [system_monitor.py:probe():178] Finished collecting system info 2024-06-21 11:19:05,054 INFO HandlerThread:26947 [system_monitor.py:probe():181] Publishing system info 2024-06-21 11:19:05,055 DEBUG HandlerThread:26947 [system_info.py:_save_pip():51] Saving list of pip packages installed into the current environment 2024-06-21 11:19:05,055 DEBUG HandlerThread:26947 [system_info.py:_save_pip():67] Saving pip packages done 2024-06-21 11:19:05,056 INFO HandlerThread:26947 [system_monitor.py:probe():183] Finished publishing system info 2024-06-21 11:19:05,204 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: files 2024-06-21 11:19:05,204 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-metadata.json with policy now 2024-06-21 11:19:05,207 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:19:05,208 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:19:05,208 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: telemetry 2024-06-21 11:19:05,212 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: config 2024-06-21 11:19:05,254 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: metric 2024-06-21 11:19:05,254 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: telemetry 2024-06-21 11:19:05,254 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: metric 2024-06-21 11:19:05,254 WARNING SenderThread:26947 [tracking_sender.py:send_metric():1133] Seen metric with glob (shouldn't happen) 2024-06-21 11:19:05,254 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: telemetry 2024-06-21 11:19:05,575 INFO Thread-17 :26947 [upload_job.py:push():143] Uploaded file /tmp/tmpjex4es6nwandb/1n5whzi1-wandb-metadata.json 2024-06-21 11:19:05,983 INFO Thread-15 :26947 [dir_watcher.py:_on_file_created():275] file/dir created: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/requirements.txt 2024-06-21 11:19:05,983 INFO Thread-15 :26947 [dir_watcher.py:_on_file_created():275] file/dir created: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:19:05,983 INFO Thread-15 :26947 [dir_watcher.py:_on_file_created():275] file/dir created: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-metadata.json 2024-06-21 11:19:07,984 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:19:12,631 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:19:12,632 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: metric 2024-06-21 11:19:12,633 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: metric 2024-06-21 11:19:12,633 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: metric 2024-06-21 11:19:12,634 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: metric 2024-06-21 11:19:12,634 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:19:12,634 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:19:12,634 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:19:12,985 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:19:13,986 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:19:14,986 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:19:18,106 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:19:18,107 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:19:18,108 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:19:18,108 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:19:18,988 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:19:18,988 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:19:19,988 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:19:20,210 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:19:20,211 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:19:25,741 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:19:25,742 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:19:25,743 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:19:25,743 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:19:25,990 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:19:26,991 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:19:27,991 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:19:29,992 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/config.yaml 2024-06-21 11:19:30,900 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:19:30,901 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:19:30,902 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:19:30,903 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:19:30,992 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:19:32,992 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:19:33,993 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:19:35,213 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:19:35,213 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:19:36,997 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:19:36,998 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:19:36,999 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:19:37,000 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:19:37,994 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:19:38,995 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:19:39,995 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:19:46,291 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:19:46,292 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:19:46,292 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:19:46,294 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:19:46,997 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:19:46,997 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:19:47,997 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:19:50,216 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:19:50,216 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:19:52,015 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:19:52,016 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:19:52,016 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:19:52,017 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:19:52,999 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:19:52,999 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:19:53,999 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:19:57,768 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:19:57,769 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:19:57,769 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:19:57,770 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:19:58,000 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:19:59,001 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:20:00,001 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:20:02,715 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:20:02,717 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:20:02,717 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:20:02,719 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:20:03,002 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:20:04,002 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:20:04,991 DEBUG SystemMonitor:26947 [system_monitor.py:_start():130] Starting system metrics aggregation loop 2024-06-21 11:20:04,995 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:20:05,003 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:20:05,218 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:20:05,218 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:20:08,004 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:20:10,005 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:20:11,038 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:20:11,038 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:20:11,039 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:20:11,040 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:20:12,005 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:20:13,006 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:20:14,006 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:20:17,042 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:20:17,043 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:20:17,044 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:20:17,045 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:20:18,007 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:20:19,007 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:20:20,008 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:20:20,222 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:20:20,222 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:20:25,834 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:20:25,835 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:20:25,835 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:20:25,836 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:20:26,009 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:20:27,010 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:20:28,010 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:20:31,666 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:20:31,667 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:20:31,668 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:20:31,669 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:20:32,011 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:20:33,011 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:20:34,012 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:20:34,997 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:20:35,224 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:20:35,224 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:20:35,970 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:20:35,971 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:20:35,971 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:20:35,972 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:20:36,012 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:20:37,013 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:20:38,013 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:20:43,070 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:20:43,072 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:20:43,072 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:20:43,072 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:20:44,015 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:20:45,015 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:20:46,016 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:20:50,226 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:20:50,226 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:20:53,120 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:20:53,121 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:20:53,121 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:20:53,122 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:20:54,018 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:20:55,018 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:20:56,018 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:20:57,685 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:20:57,686 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:20:57,687 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:20:57,688 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:20:58,019 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:20:59,019 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:21:00,020 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:21:02,678 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:21:02,679 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:21:02,679 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:21:02,680 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:21:03,020 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:21:03,021 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:21:04,021 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:21:04,999 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:21:05,228 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:21:05,229 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:21:07,376 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:21:07,377 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:21:07,378 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:21:07,378 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:21:08,022 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:21:09,022 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:21:10,023 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:21:12,673 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:21:12,673 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:21:12,674 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:21:12,675 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:21:13,024 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:21:13,024 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:21:14,024 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:21:18,175 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:21:18,176 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:21:18,177 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:21:18,177 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:21:19,025 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:21:19,025 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:21:20,026 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:21:20,231 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:21:20,231 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:21:24,767 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:21:24,768 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:21:24,769 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:21:24,769 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:21:25,027 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:21:25,027 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:21:26,027 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:21:32,617 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:21:32,617 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:21:32,618 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:21:32,619 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:21:33,029 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:21:33,029 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:21:34,029 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:21:35,002 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:21:35,233 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:21:35,233 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:21:36,663 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:21:36,664 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:21:36,665 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:21:36,665 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:21:37,030 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:21:37,031 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:21:38,031 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:21:40,949 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:21:40,950 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:21:40,950 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:21:40,951 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:21:41,032 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:21:42,032 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:21:43,032 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:21:46,674 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:21:46,674 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:21:46,675 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:21:46,676 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:21:47,033 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:21:47,034 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:21:48,034 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:21:50,235 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:21:50,235 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:21:52,184 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:21:52,185 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:21:52,186 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:21:52,187 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:21:53,035 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:21:53,035 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:21:54,036 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:22:01,799 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:22:01,801 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:22:01,801 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:22:01,802 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:22:02,038 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:22:03,038 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:22:04,038 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:22:05,004 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:22:05,237 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:22:05,237 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:22:06,740 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:22:06,741 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:22:06,741 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:22:06,743 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:22:07,039 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:22:07,040 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:22:08,040 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:22:11,071 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:22:11,071 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:22:11,072 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:22:11,073 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:22:12,041 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:22:13,041 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:22:14,042 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:22:15,821 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:22:15,821 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:22:15,822 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:22:15,823 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:22:16,042 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:22:17,043 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:22:18,043 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:22:20,239 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:22:20,240 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:22:20,762 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:22:20,763 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:22:20,763 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:22:20,765 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:22:21,044 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:22:21,044 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:22:22,044 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:22:26,204 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:22:26,205 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:22:26,206 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:22:26,207 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:22:27,046 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:22:27,046 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:22:28,046 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:22:30,616 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:22:30,617 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:22:30,617 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:22:30,619 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:22:31,047 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:22:32,047 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:22:33,047 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:22:35,006 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:22:35,241 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:22:35,242 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:22:35,824 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:22:35,825 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:22:35,825 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:22:35,826 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:22:36,048 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:22:37,049 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:22:38,049 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:22:46,873 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:22:46,874 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:22:46,874 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:22:46,875 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:22:47,051 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:22:48,052 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:22:49,052 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:22:50,243 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:22:50,244 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:22:52,493 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:22:52,494 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:22:52,495 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:22:52,496 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:22:53,053 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:22:53,053 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:22:54,054 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:23:02,218 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:23:02,219 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:23:02,219 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:23:02,221 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:23:03,056 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:23:03,056 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:23:04,056 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:23:05,008 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:23:05,245 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:23:05,246 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:23:11,345 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:23:11,346 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:23:11,346 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:23:11,347 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:23:12,059 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:23:13,059 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:23:14,059 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:23:20,080 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:23:20,081 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:23:20,082 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:23:20,083 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:23:20,247 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:23:20,248 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:23:21,061 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:23:21,061 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:23:22,062 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:23:25,705 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:23:25,706 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:23:25,707 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:23:25,707 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:23:26,063 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:23:27,063 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:23:28,063 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:23:30,012 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:23:30,012 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:23:30,013 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:23:30,014 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:23:30,064 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:23:31,064 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:23:32,065 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:23:33,779 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:23:33,780 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:23:33,781 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:23:33,782 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:23:34,065 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:23:35,009 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:23:35,066 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:23:35,249 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:23:35,250 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:23:36,066 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:23:38,379 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:23:38,380 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:23:38,380 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:23:38,381 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:23:39,067 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:23:39,067 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:23:40,067 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:23:43,307 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:23:43,308 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:23:43,308 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:23:43,309 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:23:44,068 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:23:45,069 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:23:46,069 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:23:50,251 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:23:50,252 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:23:53,764 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:23:53,765 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:23:53,765 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:23:53,767 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:23:54,071 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:23:55,072 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:23:56,072 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:24:02,916 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:24:02,917 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:24:02,918 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:24:02,920 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:24:03,074 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:24:03,074 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:24:04,074 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:24:05,011 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:24:05,253 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:24:05,254 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:24:07,941 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:24:07,942 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:24:07,943 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:24:07,943 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:24:08,076 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:24:09,076 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:24:10,076 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:24:12,952 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:24:12,954 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:24:12,954 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:24:12,956 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:24:13,077 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:24:14,078 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:24:15,078 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:24:18,407 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:24:18,408 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:24:18,408 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:24:18,409 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:24:19,079 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:24:19,079 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:24:20,080 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:24:20,255 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:24:20,256 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:24:23,616 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:24:23,617 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:24:23,618 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:24:23,618 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:24:24,081 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:24:25,081 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:24:26,081 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:24:33,692 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:24:33,693 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:24:33,694 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:24:33,694 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:24:34,084 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:24:35,013 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:24:35,084 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:24:35,257 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:24:35,258 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:24:36,085 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:24:40,554 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:24:40,555 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:24:40,556 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:24:40,557 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:24:41,086 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:24:41,086 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:24:42,086 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:24:44,790 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:24:44,791 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:24:44,791 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:24:44,792 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:24:45,087 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:24:45,088 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:24:46,088 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:24:49,265 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:24:49,266 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:24:49,267 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:24:49,268 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:24:50,089 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:24:50,259 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:24:50,260 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:24:51,089 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:24:52,090 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:24:56,882 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:24:56,883 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:24:56,884 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:24:56,885 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:24:57,091 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:24:57,091 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:24:58,091 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:25:03,062 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:25:03,062 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:25:03,063 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:25:03,064 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:25:03,093 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:25:04,093 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:25:05,015 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:25:05,094 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:25:05,262 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:25:05,262 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:25:07,742 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:25:07,743 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:25:07,744 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:25:07,744 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:25:08,094 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:25:09,095 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:25:10,095 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:25:11,878 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:25:11,879 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:25:11,879 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:25:11,880 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:25:12,096 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:25:13,096 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:25:14,096 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:25:20,264 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:25:20,264 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:25:21,625 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:25:21,626 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:25:21,626 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:25:21,627 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:25:22,099 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:25:23,099 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:25:24,099 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:25:28,695 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:25:28,696 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:25:28,697 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:25:28,698 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:25:29,101 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:25:29,101 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:25:30,101 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:25:34,448 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:25:34,449 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:25:34,450 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:25:34,451 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:25:35,017 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:25:35,102 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:25:35,103 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:25:35,266 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:25:35,266 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:25:36,103 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:25:42,505 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:25:42,507 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:25:42,507 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:25:42,508 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:25:43,105 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:25:43,105 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:25:44,105 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:25:47,788 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:25:47,789 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:25:47,789 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:25:47,790 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:25:48,106 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:25:49,107 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:25:50,107 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:25:50,268 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:25:50,268 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:25:52,967 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:25:52,968 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:25:52,969 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:25:52,970 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:25:53,108 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:25:53,108 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:25:54,108 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:25:57,791 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:25:57,792 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:25:57,792 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:25:57,793 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:25:58,109 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:25:59,110 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:26:00,110 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:26:03,088 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:26:03,089 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:26:03,089 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:26:03,092 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:26:03,111 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:26:04,111 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:26:05,018 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:26:05,111 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:26:05,270 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:26:05,270 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:26:07,896 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:26:07,897 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:26:07,898 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:26:07,898 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:26:08,112 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:26:09,113 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:26:10,113 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:26:17,031 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:26:17,032 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:26:17,032 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:26:17,034 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:26:17,115 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:26:17,115 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:26:18,115 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:26:20,272 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:26:20,272 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:26:22,206 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:26:22,207 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:26:22,207 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:26:22,208 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:26:23,116 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:26:23,117 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:26:24,117 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:26:28,198 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:26:28,199 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:26:28,199 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:26:28,200 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:26:29,118 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:26:29,118 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:26:30,119 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:26:35,020 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:26:35,274 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:26:35,274 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:26:37,345 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:26:37,346 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:26:37,347 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:26:37,348 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:26:38,121 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:26:38,121 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:26:39,121 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:26:47,677 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:26:47,678 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:26:47,679 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:26:47,680 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:26:48,123 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:26:49,125 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:26:50,125 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:26:50,276 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:26:50,276 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:26:53,095 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:26:53,096 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:26:53,097 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:26:53,097 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:26:53,126 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:26:54,126 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:26:58,983 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:26:58,984 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:26:58,985 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:26:58,986 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:26:59,127 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:27:00,128 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:27:03,291 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:27:03,291 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:27:03,292 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:27:03,294 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:27:04,129 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:27:04,129 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:27:05,021 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:27:05,278 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:27:05,278 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:27:06,130 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:27:11,593 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:27:11,594 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:27:11,594 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:27:11,595 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:27:12,131 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:27:14,132 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:27:18,206 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:27:18,207 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:27:18,208 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:27:18,210 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:27:19,133 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:27:20,134 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:27:20,280 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:27:20,280 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:27:23,508 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:27:23,509 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:27:23,510 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:27:23,511 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:27:24,135 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:27:26,136 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:27:27,903 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:27:27,904 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:27:27,905 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:27:27,906 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:27:28,136 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:27:30,137 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:27:33,958 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:27:33,959 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:27:33,960 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:27:33,961 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:27:34,138 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:27:35,024 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:27:35,282 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:27:35,283 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:27:36,139 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:27:38,501 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:27:38,502 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:27:38,502 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:27:38,503 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:27:39,139 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:27:40,140 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:27:44,828 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:27:44,829 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:27:44,830 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:27:44,831 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:27:45,141 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:27:46,142 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:27:50,035 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:27:50,036 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:27:50,036 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:27:50,037 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:27:50,143 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:27:50,285 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:27:50,285 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:27:52,143 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:27:55,014 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:27:55,015 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:27:55,015 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:27:55,016 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:27:55,144 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:27:56,145 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:28:01,123 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:28:01,124 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:28:01,125 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:28:01,125 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:28:01,146 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:28:02,146 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:28:05,025 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:28:05,287 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:28:05,287 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:28:06,324 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:28:06,325 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:28:06,326 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:28:06,327 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:28:07,148 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:28:08,148 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:28:11,530 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:28:11,531 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:28:11,532 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:28:11,533 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:28:12,149 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:28:14,150 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:28:18,378 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:28:18,379 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:28:18,379 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:28:18,380 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:28:19,151 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:28:20,152 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:28:20,289 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:28:20,289 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:28:22,968 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:28:22,969 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:28:22,969 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:28:22,970 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:28:23,153 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:28:24,153 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:28:28,497 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:28:28,498 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:28:28,498 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:28:28,499 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:28:29,154 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:28:30,155 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:28:33,853 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:28:33,854 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:28:33,854 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:28:33,856 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:28:34,156 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:28:35,026 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:28:35,291 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:28:35,291 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:28:36,157 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:28:39,047 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:28:39,048 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:28:39,048 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:28:39,050 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:28:39,158 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:28:40,158 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:28:48,250 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:28:48,251 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:28:48,252 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:28:48,253 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:28:49,160 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:28:50,161 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:28:50,293 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:28:50,293 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:28:54,201 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:28:54,202 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:28:54,202 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:28:54,203 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:28:55,162 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:28:56,163 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:28:58,525 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:28:58,526 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:28:58,526 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:28:58,527 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:28:59,164 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:29:00,164 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:29:02,992 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:29:02,993 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:29:02,993 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:29:02,995 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:29:03,165 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:29:04,165 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:29:05,028 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:29:05,295 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:29:05,295 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:29:06,851 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:29:06,853 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:29:06,854 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:29:06,854 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:29:07,166 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:29:08,167 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:29:11,210 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:29:11,211 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:29:11,211 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:29:11,212 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:29:12,168 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:29:12,168 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:29:15,339 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:29:15,340 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:29:15,340 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:29:15,341 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:29:16,169 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:29:16,169 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:29:18,170 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:29:19,401 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:29:19,401 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:29:19,402 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:29:19,403 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:29:20,170 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:29:20,170 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:29:20,297 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:29:20,297 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:29:22,171 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:29:23,925 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:29:23,926 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:29:23,926 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:29:23,927 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:29:24,171 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:29:26,172 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:29:30,217 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:29:30,218 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:29:30,218 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:29:30,220 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:29:31,173 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:29:32,174 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:29:35,030 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:29:35,299 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:29:35,299 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:29:36,004 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:29:36,005 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:29:36,006 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:29:36,007 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:29:36,175 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:29:38,176 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:29:40,488 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:29:40,489 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:29:40,490 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:29:40,491 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:29:41,176 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:29:42,177 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:29:45,232 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:29:45,233 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:29:45,233 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:29:45,235 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:29:46,178 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:29:46,178 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:29:50,301 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:29:50,302 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:29:51,771 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:29:51,773 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:29:51,773 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:29:51,774 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:29:52,180 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:29:54,180 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:29:56,606 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:29:56,607 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:29:56,608 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:29:56,609 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:29:57,181 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:29:58,182 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:30:01,079 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:30:01,080 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:30:01,080 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:30:01,082 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:30:01,183 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:30:02,183 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:30:05,032 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:30:05,304 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:30:05,304 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:30:05,905 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:30:05,906 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:30:05,906 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:30:05,907 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:30:06,184 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:30:08,185 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:30:15,140 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:30:15,141 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:30:15,141 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:30:15,142 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:30:15,186 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:30:16,187 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:30:20,056 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:30:20,057 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:30:20,057 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:30:20,058 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:30:20,188 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:30:20,306 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:30:20,306 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:30:22,189 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:30:27,243 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:30:27,244 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:30:27,244 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:30:27,245 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:30:28,190 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:30:28,190 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:30:35,034 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:30:35,308 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:30:35,308 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:30:36,465 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:30:36,466 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:30:36,467 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:30:36,468 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:30:37,193 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:30:38,193 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:30:41,764 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:30:41,765 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:30:41,765 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:30:41,766 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:30:42,194 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:30:44,195 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:30:46,791 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:30:46,791 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:30:46,792 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:30:46,793 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:30:47,196 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:30:48,196 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:30:50,310 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:30:50,310 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:30:52,085 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:30:52,086 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:30:52,086 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:30:52,088 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:30:52,198 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:30:54,198 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:30:57,939 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:30:57,940 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:30:57,941 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:30:57,942 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:30:58,200 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:31:00,200 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:31:02,283 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:31:02,284 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:31:02,285 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:31:02,286 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:31:03,201 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:31:04,201 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:31:05,035 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:31:05,312 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:31:05,312 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:31:08,919 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:31:08,920 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:31:08,921 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:31:08,921 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:31:09,203 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:31:10,203 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:31:13,410 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:31:13,411 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:31:13,412 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:31:13,413 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:31:14,204 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:31:14,205 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:31:16,205 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:31:18,624 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:31:18,626 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:31:18,626 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:31:18,626 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:31:19,206 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:31:20,206 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:31:20,314 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:31:20,314 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:31:23,718 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:31:23,719 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:31:23,720 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:31:23,721 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:31:24,207 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:31:24,208 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:31:26,208 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:31:29,333 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:31:29,334 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:31:29,335 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:31:29,336 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:31:30,209 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:31:30,209 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:31:33,413 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:31:33,414 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:31:33,415 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:31:33,416 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:31:34,210 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:31:34,210 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:31:35,037 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:31:35,316 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:31:35,316 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:31:36,211 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:31:39,238 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:31:39,239 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:31:39,240 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:31:39,241 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:31:40,212 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:31:40,212 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:31:45,137 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:31:45,138 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:31:45,139 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:31:45,139 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:31:45,213 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:31:46,214 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:31:50,252 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:31:50,253 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:31:50,253 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:31:50,254 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:31:50,318 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:31:50,318 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:31:51,215 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:31:52,216 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:31:55,736 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:31:55,737 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:31:55,737 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:31:55,738 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:31:56,217 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:31:56,217 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:31:58,217 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:32:00,300 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:32:00,300 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:32:00,301 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:32:00,302 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:32:01,218 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:32:02,218 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:32:05,039 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:32:05,320 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:32:05,321 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:32:06,371 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:32:06,372 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:32:06,373 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:32:06,374 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:32:07,220 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:32:08,220 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:32:13,421 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:32:13,422 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:32:13,423 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:32:13,424 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:32:14,222 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:32:14,222 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:32:19,262 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:32:19,264 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:32:19,264 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:32:19,265 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:32:20,224 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:32:20,224 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:32:20,322 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:32:20,323 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:32:23,264 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:32:23,265 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:32:23,265 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:32:23,266 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:32:24,225 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:32:24,225 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:32:32,451 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:32:32,452 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:32:32,453 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:32:32,454 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:32:33,227 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:32:34,228 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:32:35,041 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:32:35,324 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:32:35,325 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:32:38,848 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:32:38,849 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:32:38,849 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:32:38,851 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:32:39,229 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:32:40,229 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:32:43,661 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:32:43,662 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:32:43,663 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:32:43,664 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:32:44,231 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:32:44,231 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:32:46,231 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:32:49,316 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:32:49,317 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:32:49,317 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:32:49,319 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:32:50,232 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:32:50,233 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:32:50,326 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:32:50,327 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:32:53,899 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:32:53,900 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:32:53,901 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:32:53,902 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:32:54,233 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:32:56,234 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:33:00,710 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:33:00,711 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:33:00,711 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:33:00,712 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:33:01,236 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:33:02,236 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:33:05,042 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:33:05,328 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:33:05,329 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:33:06,188 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:33:06,189 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:33:06,190 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:33:06,191 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:33:06,237 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:33:08,238 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:33:09,238 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:33:11,863 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:33:11,864 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:33:11,864 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:33:11,865 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:33:12,239 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:33:13,240 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:33:14,240 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:33:20,113 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:33:20,114 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:33:20,114 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:33:20,115 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:33:20,242 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:33:20,331 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:33:20,331 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:33:22,242 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:33:23,243 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:33:25,726 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:33:25,727 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:33:25,728 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:33:25,728 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:33:26,244 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:33:27,244 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:33:28,244 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:33:31,545 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:33:31,545 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:33:31,546 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:33:31,547 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:33:32,246 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:33:33,246 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:33:34,246 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:33:35,044 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:33:35,333 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:33:35,333 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:33:37,509 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:33:37,510 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:33:37,510 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:33:37,511 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:33:38,247 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:33:38,248 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:33:39,248 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:33:41,845 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:33:41,846 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:33:41,846 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:33:41,847 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:33:42,248 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:33:43,249 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:33:44,249 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:33:46,620 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:33:46,621 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:33:46,621 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:33:46,622 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:33:47,250 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:33:48,250 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:33:49,251 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:33:50,336 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:33:50,336 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:33:55,639 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:33:55,639 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:33:55,640 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:33:55,641 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:33:56,253 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:33:57,253 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:33:58,253 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:34:02,371 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:34:02,373 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:34:02,373 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:34:02,374 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:34:03,255 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:34:04,255 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:34:05,046 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:34:05,255 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:34:05,338 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:34:05,338 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:34:09,837 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:34:09,838 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:34:09,839 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:34:09,840 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:34:10,257 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:34:11,257 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:34:12,257 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:34:14,979 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:34:14,981 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:34:14,981 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:34:14,982 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:34:15,258 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:34:16,258 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:34:17,259 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:34:20,159 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:34:20,160 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:34:20,161 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:34:20,162 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:34:20,260 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:34:20,340 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:34:20,340 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:34:22,260 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:34:23,261 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:34:26,069 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:34:26,070 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:34:26,071 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:34:26,072 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:34:26,261 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:34:28,262 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:34:29,262 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:34:29,893 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:34:29,894 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:34:29,894 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:34:29,895 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:34:30,263 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:34:31,263 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:34:32,263 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:34:34,213 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:34:34,214 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:34:34,214 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:34:34,216 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:34:34,264 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:34:35,048 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:34:35,343 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:34:35,343 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:34:36,265 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:34:37,265 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:34:38,883 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:34:38,884 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:34:38,884 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:34:38,885 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:34:39,265 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:34:40,266 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:34:41,266 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:34:45,441 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:34:45,442 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:34:45,442 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:34:45,443 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:34:46,268 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:34:46,268 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:34:47,268 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:34:49,806 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:34:49,807 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:34:49,807 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:34:49,808 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:34:50,269 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:34:50,344 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:34:50,345 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:34:51,270 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:34:52,270 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:34:54,645 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:34:54,647 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:34:54,647 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:34:54,647 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:34:55,271 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:34:56,271 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:34:57,272 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:35:03,142 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:35:03,143 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:35:03,143 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:35:03,145 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:35:03,274 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:35:04,274 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:35:05,050 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:35:05,274 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:35:05,347 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:35:05,347 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:35:07,044 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:35:07,045 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:35:07,045 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:35:07,046 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:35:07,275 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:35:08,276 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:35:09,276 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:35:16,223 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:35:16,224 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:35:16,225 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:35:16,225 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:35:16,278 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:35:18,279 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:35:19,279 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:35:20,349 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:35:20,349 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:35:23,031 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:35:23,032 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:35:23,032 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:35:23,033 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:35:23,280 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:35:24,280 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:35:25,281 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:35:31,293 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:35:31,294 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:35:31,294 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:35:31,295 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:35:32,283 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:35:32,283 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:35:33,283 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:35:35,051 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:35:35,351 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:35:35,351 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:35:37,078 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:35:37,078 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:35:37,079 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:35:37,080 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:35:37,284 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:35:38,284 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:35:39,285 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:35:41,188 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:35:41,190 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:35:41,191 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:35:41,192 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:35:41,285 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:35:42,286 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:35:43,286 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:35:46,488 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:35:46,489 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:35:46,490 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:35:46,491 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:35:47,288 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:35:48,288 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:35:49,288 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:35:50,353 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:35:50,353 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:35:51,216 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:35:51,218 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:35:51,218 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:35:51,219 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:35:51,289 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:35:52,289 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:35:53,289 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:35:55,265 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:35:55,266 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:35:55,267 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:35:55,268 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:35:55,290 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:35:56,290 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:35:57,291 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:36:00,548 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:36:00,549 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:36:00,550 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:36:00,551 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:36:01,292 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:36:02,293 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:36:03,293 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:36:05,053 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:36:05,355 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:36:05,355 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:36:05,895 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:36:05,896 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:36:05,897 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:36:05,898 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:36:06,294 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:36:07,294 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:36:08,295 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:36:10,846 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:36:10,847 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:36:10,847 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:36:10,849 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:36:11,296 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:36:12,296 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:36:13,296 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:36:17,407 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:36:17,408 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:36:17,408 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:36:17,409 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:36:18,298 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:36:18,298 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:36:19,298 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:36:20,357 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:36:20,357 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:36:23,026 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:36:23,027 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:36:23,028 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:36:23,029 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:36:23,299 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:36:24,299 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:36:25,300 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:36:30,586 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:36:30,587 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:36:30,588 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:36:30,589 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:36:31,301 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:36:32,302 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:36:33,302 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:36:35,055 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:36:35,359 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:36:35,359 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:36:36,290 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:36:36,291 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:36:36,292 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:36:36,292 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:36:36,303 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:36:38,304 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:36:39,304 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:36:43,589 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:36:43,590 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:36:43,591 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:36:43,592 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:36:44,305 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:36:44,306 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:36:45,306 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:36:48,717 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:36:48,717 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:36:48,718 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:36:48,719 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:36:49,307 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:36:50,307 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:36:50,361 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:36:50,361 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:36:51,308 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:36:55,174 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:36:55,175 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:36:55,175 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:36:55,176 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:36:55,309 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:36:56,309 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:36:57,309 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:37:00,165 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:37:00,166 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:37:00,167 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:37:00,168 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:37:00,310 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:37:01,311 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:37:02,311 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:37:05,057 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:37:05,127 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:37:05,128 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:37:05,129 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:37:05,130 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:37:05,312 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:37:05,363 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:37:05,363 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:37:06,312 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:37:07,313 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:37:09,717 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:37:09,718 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:37:09,718 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:37:09,719 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:37:10,313 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:37:10,314 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:37:11,314 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:37:13,857 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:37:13,858 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:37:13,858 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:37:13,859 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:37:14,315 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:37:15,315 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:37:16,315 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:37:19,774 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:37:19,775 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:37:19,775 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:37:19,776 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:37:20,316 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:37:20,365 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:37:20,365 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:37:21,317 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:37:22,317 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:37:24,637 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:37:24,638 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:37:24,638 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:37:24,639 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:37:25,318 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:37:26,318 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:37:27,318 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:37:29,460 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:37:29,461 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:37:29,461 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:37:29,462 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:37:30,319 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:37:30,320 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:37:31,320 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:37:34,848 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:37:34,849 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:37:34,850 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:37:34,850 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:37:35,059 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:37:35,321 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:37:35,367 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:37:35,367 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:37:36,321 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:37:37,322 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:37:40,491 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:37:40,492 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:37:40,492 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:37:40,493 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:37:41,323 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:37:42,323 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:37:43,323 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:37:46,245 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:37:46,246 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:37:46,247 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:37:46,248 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:37:46,324 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:37:48,325 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:37:49,325 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:37:50,369 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:37:50,369 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:37:54,366 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:37:54,366 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:37:54,367 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:37:54,368 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:37:55,327 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:37:56,327 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:37:57,327 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:37:59,784 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:37:59,784 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:37:59,785 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:37:59,786 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:38:00,328 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:38:00,328 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:38:01,329 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:38:04,573 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:38:04,575 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:38:04,576 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:38:04,576 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:38:05,061 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:38:05,330 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:38:05,371 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:38:05,371 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:38:06,330 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:38:07,330 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:38:10,407 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:38:10,409 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:38:10,409 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:38:10,410 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:38:11,332 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:38:12,332 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:38:13,332 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:38:19,546 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:38:19,547 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:38:19,548 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:38:19,549 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:38:20,334 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:38:20,334 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:38:20,373 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:38:20,373 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:38:21,334 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:38:24,908 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:38:24,909 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:38:24,909 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:38:24,910 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:38:25,336 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:38:26,336 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:38:27,336 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:38:32,950 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:38:32,951 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:38:32,952 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:38:32,953 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:38:33,338 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:38:34,338 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:38:35,062 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:38:35,339 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:38:35,375 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:38:35,376 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:38:41,418 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:38:41,419 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:38:41,420 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:38:41,420 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:38:42,340 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:38:42,341 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:38:43,341 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:38:46,119 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:38:46,120 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:38:46,120 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:38:46,121 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:38:46,342 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:38:47,342 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:38:48,342 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:38:50,377 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:38:50,378 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:38:51,496 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:38:51,496 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:38:51,497 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:38:51,498 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:38:52,343 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:38:52,344 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:38:53,344 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:38:56,035 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:38:56,037 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:38:56,037 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:38:56,038 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:38:56,345 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:38:57,345 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:38:58,345 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:39:02,363 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:39:02,364 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:39:02,365 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:39:02,366 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:39:03,347 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:39:04,347 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:39:05,064 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:39:05,347 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:39:05,379 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:39:05,380 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:39:08,744 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:39:08,746 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:39:08,746 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:39:08,746 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:39:09,349 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:39:10,349 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:39:11,349 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:39:19,027 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:39:19,027 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:39:19,028 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:39:19,029 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:39:19,352 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:39:20,352 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:39:20,381 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:39:20,382 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:39:21,352 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:39:24,584 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:39:24,585 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:39:24,585 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:39:24,586 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:39:25,354 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:39:26,354 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:39:27,354 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:39:29,514 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:39:29,515 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:39:29,515 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:39:29,517 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:39:30,355 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:39:31,355 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:39:32,356 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:39:35,066 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:39:35,383 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:39:35,383 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:39:36,102 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:39:36,103 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:39:36,103 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:39:36,105 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:39:36,357 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:39:37,357 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:39:38,358 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:39:41,265 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:39:41,266 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:39:41,267 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:39:41,268 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:39:41,359 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:39:42,359 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:39:43,359 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:39:50,385 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:39:50,386 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:39:50,434 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:39:50,435 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:39:50,436 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:39:50,436 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:39:51,361 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:39:52,362 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:39:53,362 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:39:55,255 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:39:55,256 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:39:55,256 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:39:55,257 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:39:55,363 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:39:56,363 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:39:57,363 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:40:00,504 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:40:00,505 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:40:00,506 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:40:00,506 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:40:01,364 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:40:02,365 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:40:03,365 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:40:05,067 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:40:05,388 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:40:05,388 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:40:06,470 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:40:06,471 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:40:06,471 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:40:06,472 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:40:07,366 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:40:08,367 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:40:09,367 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:40:12,236 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:40:12,237 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:40:12,238 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:40:12,239 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:40:12,368 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:40:13,368 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:40:14,369 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:40:18,868 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:40:18,869 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:40:18,870 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:40:18,870 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:40:19,370 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:40:20,370 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:40:20,390 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:40:20,390 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:40:21,371 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:40:24,336 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:40:24,337 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:40:24,337 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:40:24,338 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:40:24,371 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:40:25,372 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:40:26,372 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:40:29,448 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:40:29,450 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:40:29,450 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:40:29,450 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:40:30,373 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:40:30,373 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:40:31,374 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:40:34,808 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:40:34,809 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:40:34,809 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:40:34,810 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:40:35,069 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:40:35,375 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:40:35,392 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:40:35,392 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:40:36,375 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:40:37,375 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:40:41,147 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:40:41,149 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:40:41,149 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:40:41,149 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:40:41,376 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:40:42,377 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:40:43,377 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:40:48,548 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:40:48,550 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:40:48,550 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:40:48,551 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:40:49,379 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:40:50,379 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:40:50,394 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:40:50,394 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:40:51,379 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:40:53,900 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:40:53,900 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:40:53,901 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:40:53,902 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:40:54,380 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:40:55,381 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:40:56,381 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:40:58,099 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:40:58,099 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:40:58,100 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:40:58,101 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:40:58,381 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:40:59,382 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:41:00,382 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:41:02,839 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:41:02,840 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:41:02,841 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:41:02,842 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:41:03,383 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:41:04,383 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:41:05,071 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:41:05,384 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:41:05,396 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:41:05,396 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:41:07,684 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:41:07,684 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:41:07,685 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:41:07,686 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:41:08,385 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:41:09,385 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:41:10,385 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:41:13,969 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:41:13,970 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:41:13,970 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:41:13,972 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:41:14,386 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:41:15,387 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:41:16,387 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:41:18,449 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:41:18,450 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:41:18,450 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:41:18,451 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:41:19,388 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:41:19,388 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:41:20,388 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:41:20,398 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:41:20,398 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:41:22,948 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:41:22,949 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:41:22,949 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:41:22,950 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:41:23,389 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:41:24,390 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:41:25,390 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:41:28,614 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:41:28,615 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:41:28,615 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:41:28,616 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:41:29,391 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:41:30,392 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:41:31,392 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:41:35,073 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:41:35,400 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:41:35,400 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:41:50,543 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:41:50,543 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:42:05,074 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:42:06,210 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:42:06,211 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:42:21,342 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:42:21,343 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:42:35,076 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:42:36,344 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:42:36,345 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:42:51,689 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:42:51,689 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:43:05,078 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:43:06,809 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:43:06,809 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:43:21,811 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:43:21,812 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:43:35,080 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:43:36,814 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:43:36,814 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:43:52,128 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:43:52,129 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:44:05,081 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:44:07,454 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:44:07,454 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:44:23,093 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:44:23,094 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:44:35,083 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:44:38,352 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:44:38,353 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:44:45,446 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:44:49,290 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:44:49,291 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:44:49,292 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:44:49,293 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:44:49,447 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:44:50,447 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:44:51,448 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:44:53,925 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:44:53,925 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:44:54,476 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:44:54,476 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:44:54,477 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:44:54,478 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:44:55,449 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:44:55,449 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:44:56,449 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:45:00,270 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:45:00,270 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:45:00,271 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:45:00,272 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:45:00,450 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:45:01,451 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:45:02,451 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:45:05,085 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:45:05,367 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:45:05,368 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:45:05,368 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:45:05,370 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:45:05,452 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:45:06,452 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:45:07,453 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:45:08,927 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:45:08,927 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:45:10,693 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:45:10,695 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:45:10,695 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:45:10,696 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:45:11,454 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:45:11,454 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:45:12,454 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:45:17,014 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:45:17,015 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:45:17,015 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:45:17,016 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:45:17,455 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:45:18,456 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:45:19,456 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:45:21,965 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:45:21,966 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:45:21,967 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:45:21,968 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:45:22,457 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:45:22,457 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:45:23,457 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:45:23,929 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:45:23,929 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:45:29,259 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:45:29,260 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:45:29,261 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:45:29,262 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:45:29,459 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:45:30,459 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:45:31,460 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:45:33,812 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:45:33,813 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:45:33,814 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:45:33,815 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:45:34,461 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:45:34,461 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:45:35,088 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:45:35,461 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:45:38,046 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:45:38,048 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:45:38,048 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:45:38,049 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:45:38,462 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:45:38,462 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:45:38,931 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:45:38,931 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:45:39,462 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:45:43,009 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:45:43,010 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:45:43,011 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:45:43,012 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:45:43,464 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:45:44,464 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:45:45,464 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:45:51,387 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:45:51,388 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:45:51,389 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:45:51,390 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:45:51,466 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:45:52,467 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:45:53,467 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:45:53,933 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:45:53,933 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:45:57,765 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:45:57,766 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:45:57,766 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:45:57,767 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:45:58,468 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:45:58,469 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:45:59,469 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:46:02,680 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:46:02,681 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:46:02,681 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:46:02,682 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:46:03,470 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:46:03,470 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:46:04,470 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:46:05,088 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:46:07,664 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:46:07,665 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:46:07,666 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:46:07,667 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:46:08,471 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:46:08,472 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:46:08,934 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:46:08,935 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:46:09,472 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:46:13,149 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:46:13,150 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:46:13,151 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:46:13,152 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:46:13,473 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:46:14,473 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:46:15,474 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:46:20,295 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:46:20,296 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:46:20,296 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:46:20,298 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:46:20,475 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:46:20,475 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:46:21,475 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:46:23,937 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:46:23,937 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:46:25,916 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:46:25,917 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:46:25,918 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:46:25,918 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:46:26,477 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:46:26,477 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:46:27,477 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:46:30,504 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:46:30,506 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:46:30,506 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:46:30,507 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:46:31,478 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:46:31,479 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:46:32,479 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:46:35,090 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:46:35,668 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:46:35,669 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:46:35,670 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:46:35,671 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:46:36,480 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:46:36,480 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:46:37,480 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:46:38,939 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:46:38,939 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:46:41,431 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:46:41,432 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:46:41,433 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:46:41,433 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:46:41,481 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:46:42,482 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:46:43,482 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:46:46,744 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:46:46,745 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:46:46,745 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:46:46,746 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:46:47,483 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:46:47,483 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:46:48,483 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:46:53,941 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:46:53,941 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:46:54,949 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:46:54,950 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:46:54,950 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:46:54,951 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:46:55,485 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:46:56,486 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:46:57,486 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:47:04,085 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:47:04,087 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:47:04,087 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:47:04,088 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:47:04,488 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:47:04,489 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:47:05,091 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:47:05,489 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:47:08,943 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:47:08,943 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:47:09,020 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:47:09,022 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:47:09,022 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:47:09,023 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:47:09,490 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:47:10,490 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:47:11,490 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:47:14,180 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:47:14,181 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:47:14,182 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:47:14,183 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:47:14,491 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:47:14,492 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:47:15,492 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:47:22,609 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:47:22,610 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:47:22,610 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:47:22,611 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:47:23,494 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:47:23,494 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:47:23,945 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:47:23,946 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:47:24,494 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:47:28,937 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:47:28,938 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:47:28,939 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:47:28,940 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:47:29,496 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:47:30,496 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:47:31,496 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:47:35,094 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:47:35,370 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:47:35,371 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:47:35,372 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:47:35,373 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:47:35,497 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:47:36,498 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:47:37,498 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:47:38,947 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:47:38,948 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:47:40,852 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:47:40,853 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:47:40,854 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:47:40,854 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:47:41,499 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:47:41,500 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:47:42,500 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:47:45,873 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:47:45,873 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:47:45,874 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:47:45,875 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:47:46,501 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:47:46,501 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:47:47,501 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:47:53,524 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:47:53,524 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:47:53,525 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:47:53,526 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:47:53,949 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:47:53,950 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:47:54,503 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:47:54,503 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:47:55,503 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:47:57,867 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:47:57,868 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:47:57,869 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:47:57,870 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:47:58,504 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:47:58,505 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:47:59,505 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:48:03,099 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:48:03,100 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:48:03,100 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:48:03,101 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:48:03,506 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:48:04,506 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:48:05,095 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:48:05,507 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:48:07,641 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:48:07,642 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:48:07,642 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:48:07,643 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:48:08,508 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:48:08,508 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:48:08,952 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:48:08,952 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:48:09,508 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:48:12,587 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:48:12,588 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:48:12,588 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:48:12,589 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:48:13,509 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:48:13,509 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:48:14,509 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:48:17,280 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:48:17,280 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:48:17,281 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:48:17,282 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:48:17,510 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:48:18,511 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:48:19,511 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:48:22,962 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:48:22,963 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:48:22,963 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:48:22,964 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:48:23,512 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:48:23,512 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:48:23,954 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:48:23,954 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:48:24,512 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:48:26,626 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:48:26,627 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:48:26,628 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:48:26,629 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:48:27,513 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:48:27,514 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:48:28,514 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:48:33,463 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:48:33,464 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:48:33,465 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:48:33,466 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:48:33,515 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:48:34,515 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:48:35,097 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:48:35,516 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:48:38,956 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:48:38,956 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:48:41,762 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:48:41,763 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:48:41,763 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:48:41,765 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:48:42,518 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:48:42,518 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:48:43,518 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:48:47,177 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:48:47,178 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:48:47,178 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:48:47,179 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:48:47,519 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:48:48,520 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:48:49,520 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:48:52,382 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:48:52,384 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:48:52,384 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:48:52,385 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:48:52,521 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:48:52,521 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:48:53,521 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:48:53,958 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:48:53,958 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:48:57,037 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:48:57,039 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:48:57,039 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:48:57,040 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:48:57,522 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:48:58,523 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:48:59,523 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:49:05,099 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:49:06,188 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:49:06,189 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:49:06,189 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:49:06,191 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:49:06,525 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:49:06,525 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:49:07,525 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:49:08,960 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:49:08,960 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:49:10,569 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:49:10,570 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:49:10,570 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:49:10,572 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:49:11,526 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:49:11,526 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:49:12,526 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:49:15,885 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:49:15,886 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:49:15,887 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:49:15,887 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:49:16,528 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:49:16,528 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:49:17,528 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:49:23,155 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:49:23,156 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:49:23,156 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:49:23,157 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:49:23,529 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:49:23,962 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:49:23,962 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:49:24,530 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:49:25,530 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:49:30,208 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:49:30,210 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:49:30,211 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:49:30,211 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:49:30,532 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:49:30,533 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:49:31,533 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:49:34,767 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:49:34,768 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:49:34,768 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:49:34,769 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:49:35,101 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:49:35,535 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:49:35,536 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:49:36,536 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:49:38,964 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:49:38,964 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:49:39,982 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:49:39,983 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:49:39,983 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:49:39,984 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:49:40,537 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:49:40,537 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:49:41,537 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:49:47,267 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:49:47,269 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:49:47,269 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:49:47,270 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:49:47,539 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:49:48,539 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:49:49,539 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:49:51,438 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:49:51,439 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:49:51,439 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:49:51,441 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:49:51,540 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:49:52,540 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:49:53,540 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:49:53,966 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:49:53,966 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:49:57,314 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:49:57,315 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:49:57,316 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:49:57,317 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:49:57,542 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:49:59,542 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:50:02,012 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:50:02,012 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:50:02,013 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:50:02,014 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:50:02,543 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:50:03,543 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:50:05,104 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:50:07,120 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:50:07,121 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:50:07,121 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:50:07,122 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:50:07,545 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:50:08,968 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:50:08,968 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:50:09,545 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:50:15,243 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:50:15,244 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:50:15,245 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:50:15,245 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:50:15,547 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:50:17,547 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:50:23,970 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:50:23,970 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:50:24,033 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:50:24,034 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:50:24,034 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:50:24,035 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:50:24,549 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:50:25,550 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:50:27,865 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:50:27,866 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:50:27,866 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:50:27,867 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:50:28,551 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:50:29,551 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:50:32,352 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:50:32,353 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:50:32,354 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:50:32,355 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:50:32,552 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:50:33,552 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:50:35,106 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:50:38,420 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:50:38,421 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:50:38,421 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:50:38,422 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:50:38,554 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:50:38,972 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:50:38,972 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:50:39,554 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:50:47,632 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:50:47,633 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:50:47,633 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:50:47,635 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:50:48,556 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:50:49,557 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:50:53,974 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:50:53,974 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:50:56,767 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:50:56,768 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:50:56,768 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:50:56,770 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:50:57,559 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:50:57,559 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:50:59,560 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:51:02,007 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:51:02,008 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:51:02,008 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:51:02,009 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:51:02,561 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:51:03,561 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:51:05,108 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:51:07,547 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:51:07,548 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:51:07,548 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:51:07,549 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:51:07,562 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:51:08,976 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:51:08,977 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:51:09,563 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:51:13,051 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:51:13,052 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:51:13,052 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:51:13,053 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:51:13,564 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:51:13,564 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:51:15,564 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:51:17,742 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:51:17,743 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:51:17,744 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:51:17,745 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:51:18,565 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:51:19,565 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:51:21,914 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:51:21,915 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:51:21,915 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:51:21,916 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:51:22,566 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:51:23,567 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:51:23,979 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:51:23,979 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:51:30,050 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:51:30,051 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:51:30,052 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:51:30,052 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:51:30,569 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:51:31,569 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:51:35,055 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:51:35,056 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:51:35,056 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:51:35,057 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:51:35,110 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:51:35,570 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:51:35,570 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:51:37,571 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:51:38,981 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:51:38,981 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:51:44,073 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:51:44,074 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:51:44,075 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:51:44,075 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:51:44,573 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:51:45,573 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:51:48,632 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:51:48,633 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:51:48,633 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:51:48,634 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:51:49,574 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:51:49,574 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:51:51,575 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:51:53,983 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:51:53,983 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:51:58,816 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:51:58,817 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:51:58,818 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:51:58,819 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:51:59,577 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:51:59,577 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:52:01,578 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:52:05,111 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:52:07,336 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:52:07,337 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:52:07,338 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:52:07,338 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:52:07,580 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:52:08,985 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:52:08,985 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:52:09,580 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:52:16,517 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:52:16,518 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:52:16,519 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:52:16,519 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:52:16,582 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:52:17,582 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:52:22,459 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:52:22,460 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:52:22,461 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:52:22,462 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:52:22,584 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:52:23,584 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:52:23,987 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:52:23,987 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:52:28,288 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:52:28,290 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:52:28,290 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:52:28,291 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:52:28,586 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:52:29,586 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:52:35,113 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:52:37,472 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:52:37,473 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:52:37,473 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:52:37,474 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:52:37,588 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:52:38,989 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:52:38,989 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:52:39,589 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:52:42,525 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:52:42,526 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:52:42,526 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:52:42,527 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:52:42,590 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:52:43,590 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:52:48,029 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:52:48,030 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:52:48,030 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:52:48,031 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:52:48,592 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:52:49,592 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:52:50,592 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:52:53,991 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:52:53,991 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:52:55,208 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:52:55,209 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:52:55,210 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:52:55,210 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:52:55,594 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:52:56,594 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:52:57,594 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:53:00,322 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:53:00,323 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:53:00,324 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:53:00,325 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:53:00,595 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:53:01,596 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:53:02,596 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:53:05,114 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:53:05,168 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:53:05,169 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:53:05,170 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:53:05,171 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:53:05,597 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:53:06,597 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:53:07,597 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:53:08,993 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:53:08,993 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:53:11,769 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:53:11,770 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:53:11,771 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:53:11,772 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:53:12,599 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:53:13,599 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:53:14,600 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:53:16,954 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:53:16,955 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:53:16,955 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:53:16,957 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:53:17,601 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:53:18,601 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:53:19,601 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:53:22,131 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:53:22,133 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:53:22,133 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:53:22,133 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:53:22,602 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:53:23,602 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:53:23,995 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:53:23,995 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:53:24,603 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:53:27,290 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:53:27,291 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:53:27,292 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:53:27,293 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:53:27,604 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:53:28,604 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:53:29,604 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:53:31,527 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:53:31,529 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:53:31,529 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:53:31,530 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:53:31,605 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:53:33,605 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:53:34,606 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:53:35,116 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:53:37,261 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:53:37,261 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:53:37,262 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:53:37,263 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:53:37,607 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:53:38,607 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:53:38,997 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:53:38,997 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:53:39,607 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:53:41,142 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:53:41,143 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:53:41,143 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:53:41,144 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:53:41,608 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:53:42,608 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:53:43,609 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:53:46,653 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:53:46,654 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:53:46,654 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:53:46,655 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:53:47,610 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:53:47,610 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:53:48,610 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:53:53,999 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:53:54,000 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:53:55,819 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:53:55,820 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:53:55,821 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:53:55,822 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:53:56,612 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:53:57,613 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:53:58,613 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:53:59,884 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:53:59,885 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:53:59,885 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:53:59,886 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:54:00,614 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:54:01,615 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:54:02,615 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:54:05,074 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:54:05,075 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:54:05,076 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:54:05,077 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:54:05,118 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:54:05,616 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:54:06,616 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:54:07,616 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:54:09,001 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:54:09,001 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:54:09,821 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:54:09,821 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:54:09,822 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:54:09,823 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:54:10,617 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:54:11,618 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:54:12,618 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:54:14,082 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:54:14,083 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:54:14,083 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:54:14,084 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:54:14,619 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:54:15,619 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:54:16,619 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:54:18,437 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:54:18,438 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:54:18,438 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:54:18,439 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:54:18,620 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:54:19,620 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:54:20,620 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:54:23,205 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:54:23,206 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:54:23,207 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:54:23,208 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:54:23,622 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:54:24,003 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:54:24,003 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:54:24,622 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:54:25,622 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:54:32,309 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:54:32,310 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:54:32,310 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:54:32,312 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:54:32,624 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:54:33,624 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:54:34,625 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:54:35,119 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:54:39,005 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:54:39,005 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:54:40,771 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:54:40,772 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:54:40,773 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:54:40,774 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:54:41,627 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:54:41,627 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:54:42,627 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:54:46,386 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:54:46,387 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:54:46,387 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:54:46,388 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:54:46,628 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:54:47,629 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:54:48,629 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:54:52,602 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:54:52,603 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:54:52,604 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:54:52,605 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:54:52,630 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:54:53,630 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:54:54,007 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:54:54,007 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:54:54,631 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:54:58,901 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:54:58,902 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:54:58,903 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:54:58,904 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:54:59,632 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:55:00,632 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:55:01,633 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:55:05,121 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:55:06,690 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:55:06,691 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:55:06,691 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:55:06,692 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:55:07,634 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:55:07,635 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:55:08,635 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:55:09,009 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:55:09,009 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:55:12,960 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:55:12,961 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:55:12,961 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:55:12,963 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:55:13,636 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:55:14,636 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:55:15,637 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:55:18,717 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:55:18,719 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:55:18,719 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:55:18,720 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:55:19,638 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:55:19,638 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:55:20,638 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:55:24,012 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:55:24,012 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:55:27,868 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:55:27,868 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:55:27,869 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:55:27,870 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:55:28,641 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:55:29,641 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:55:30,641 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:55:33,676 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:55:33,677 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:55:33,678 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:55:33,679 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:55:34,642 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:55:35,123 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:55:35,643 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:55:36,643 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:55:38,883 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:55:38,884 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:55:38,884 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:55:38,885 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:55:39,014 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:55:39,014 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:55:39,644 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:55:40,644 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:55:41,645 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:55:44,358 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:55:44,358 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:55:44,359 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:55:44,360 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:55:44,645 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:55:45,646 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:55:46,646 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:55:48,709 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:55:48,710 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:55:48,711 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:55:48,712 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:55:49,647 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:55:49,647 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:55:50,647 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:55:53,534 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:55:53,535 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:55:53,535 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:55:53,536 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:55:53,648 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:55:54,016 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:55:54,016 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:55:55,649 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:55:56,649 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:55:58,513 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:55:58,514 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:55:58,514 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:55:58,515 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:55:58,650 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:55:59,650 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:56:00,650 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:56:03,312 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:56:03,313 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:56:03,313 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:56:03,314 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:56:03,651 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:56:04,651 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:56:05,124 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:56:05,652 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:56:08,366 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:56:08,367 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:56:08,367 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:56:08,368 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:56:08,653 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:56:09,018 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:56:09,018 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:56:09,653 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:56:10,653 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:56:15,977 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:56:15,978 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:56:15,979 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:56:15,980 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:56:16,655 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:56:17,655 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:56:18,656 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:56:24,020 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:56:24,020 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:56:25,115 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:56:25,116 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:56:25,116 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:56:25,117 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:56:25,658 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:56:26,658 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:56:27,658 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:56:29,876 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:56:29,878 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:56:29,878 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:56:29,879 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:56:30,659 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:56:31,659 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:56:32,660 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:56:35,126 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:56:35,962 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:56:35,963 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:56:35,963 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:56:35,964 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:56:36,661 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:56:37,661 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:56:38,661 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:56:39,022 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:56:39,022 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:56:41,471 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:56:41,472 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:56:41,473 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:56:41,474 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:56:41,662 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:56:42,663 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:56:43,663 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:56:50,667 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:56:50,668 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:56:50,669 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:56:50,670 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:56:51,665 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:56:51,665 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:56:52,665 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:56:54,024 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:56:54,024 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:56:58,825 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:56:58,827 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:56:58,827 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:56:58,828 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:56:59,667 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:56:59,667 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:57:00,668 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:57:04,456 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:57:04,456 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:57:04,457 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:57:04,458 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:57:04,669 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:57:05,128 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:57:05,669 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:57:06,669 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:57:09,026 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:57:09,026 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:57:10,769 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:57:10,770 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:57:10,771 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:57:10,771 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:57:11,671 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:57:11,671 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:57:12,671 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:57:17,028 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:57:17,029 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:57:17,029 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:57:17,030 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:57:17,672 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:57:18,673 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:57:19,673 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:57:24,028 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:57:24,028 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:57:24,161 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:57:24,162 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:57:24,163 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:57:24,164 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:57:24,675 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:57:25,675 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:57:26,675 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:57:28,686 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:57:28,687 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:57:28,688 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:57:28,689 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:57:29,676 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:57:29,676 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:57:30,677 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:57:35,016 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:57:35,018 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:57:35,018 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:57:35,019 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:57:35,129 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:57:35,678 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:57:36,678 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:57:37,679 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:57:39,030 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:57:39,030 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:57:44,208 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:57:44,209 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:57:44,209 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:57:44,210 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:57:44,680 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:57:45,681 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:57:46,681 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:57:51,416 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:57:51,417 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:57:51,417 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:57:51,419 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:57:51,683 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:57:52,683 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:57:53,683 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:57:54,032 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:57:54,033 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:57:56,929 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:57:56,930 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:57:56,930 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:57:56,932 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:57:57,685 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:57:57,685 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:57:58,685 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:58:02,128 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:58:02,129 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:58:02,129 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:58:02,130 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:58:02,686 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:58:03,686 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:58:04,687 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:58:05,131 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:58:06,481 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:58:06,482 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:58:06,482 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:58:06,483 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:58:06,687 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:58:07,687 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:58:08,688 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:58:09,035 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:58:09,035 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:58:14,812 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:58:14,814 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:58:14,814 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:58:14,815 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:58:15,690 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:58:15,690 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:58:16,690 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:58:21,427 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:58:21,428 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:58:21,429 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:58:21,430 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:58:21,692 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:58:22,692 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:58:23,692 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:58:24,037 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:58:24,037 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:58:27,674 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:58:27,675 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:58:27,676 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:58:27,677 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:58:27,693 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:58:28,694 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:58:29,694 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:58:32,465 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:58:32,467 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:58:32,467 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:58:32,468 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:58:32,695 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:58:33,695 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:58:34,695 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:58:35,133 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:58:39,039 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:58:39,039 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:58:39,281 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:58:39,282 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:58:39,283 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:58:39,283 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:58:39,697 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:58:40,697 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:58:41,697 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:58:45,932 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:58:45,933 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:58:45,934 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:58:45,935 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:58:46,699 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:58:47,699 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:58:48,700 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:58:51,900 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:58:51,901 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:58:51,902 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:58:51,903 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:58:52,701 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:58:53,701 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:58:54,041 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:58:54,041 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:58:54,701 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:58:58,529 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:58:58,530 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:58:58,531 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:58:58,531 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:58:58,702 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:58:59,703 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:59:00,703 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:59:03,839 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:59:03,839 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:59:03,840 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:59:03,841 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:59:04,704 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:59:05,135 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:59:05,704 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:59:06,705 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:59:08,338 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:59:08,339 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:59:08,340 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:59:08,342 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:59:08,705 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:59:09,043 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:59:09,043 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:59:09,706 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:59:10,706 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:59:13,903 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:59:13,905 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:59:13,905 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:59:13,906 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:59:14,707 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:59:15,708 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:59:16,708 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:59:20,579 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:59:20,580 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:59:20,580 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:59:20,582 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:59:20,709 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:59:21,709 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:59:22,710 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:59:24,045 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:59:24,045 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:59:29,396 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:59:29,397 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:59:29,397 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:59:29,398 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:59:29,711 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:59:30,712 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:59:31,712 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:59:34,530 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:59:34,530 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:59:34,531 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:59:34,532 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:59:34,713 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:59:35,136 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 11:59:35,713 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:59:36,714 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:59:39,047 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:59:39,047 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:59:40,122 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:59:40,123 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:59:40,124 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:59:40,125 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:59:40,715 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:59:41,715 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:59:42,716 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:59:45,499 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:59:45,500 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:59:45,501 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:59:45,502 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:59:45,716 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:59:46,717 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:59:47,717 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:59:51,344 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:59:51,345 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:59:51,346 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:59:51,347 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:59:51,718 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:59:52,718 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:59:53,719 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:59:54,049 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 11:59:54,049 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 11:59:56,529 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 11:59:56,530 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 11:59:56,531 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 11:59:56,531 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 11:59:56,720 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 11:59:57,720 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 11:59:58,720 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:00:03,579 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:00:03,580 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:00:03,580 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:00:03,581 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:00:03,722 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:00:04,722 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:00:05,138 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:00:05,722 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:00:09,051 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:00:09,051 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:00:09,491 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:00:09,492 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:00:09,492 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:00:09,494 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:00:09,723 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:00:10,724 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:00:11,724 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:00:16,303 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:00:16,304 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:00:16,304 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:00:16,305 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:00:16,725 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:00:17,726 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:00:18,726 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:00:22,011 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:00:22,012 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:00:22,012 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:00:22,013 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:00:22,727 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:00:23,727 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:00:24,053 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:00:24,053 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:00:24,728 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:00:26,736 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:00:26,736 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:00:26,737 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:00:26,738 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:00:27,729 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:00:27,729 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:00:28,729 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:00:31,650 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:00:31,651 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:00:31,651 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:00:31,652 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:00:31,730 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:00:32,730 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:00:33,730 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:00:35,140 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:00:39,054 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:00:39,055 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:00:40,801 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:00:40,802 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:00:40,802 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:00:40,804 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:00:41,733 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:00:41,733 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:00:42,733 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:00:49,956 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:00:49,957 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:00:49,958 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:00:49,958 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:00:50,735 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:00:51,735 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:00:52,736 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:00:54,056 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:00:54,057 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:00:54,350 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:00:54,351 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:00:54,351 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:00:54,352 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:00:54,736 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:00:55,737 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:00:56,737 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:01:01,871 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:01:01,871 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:01:01,872 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:01:01,873 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:01:02,739 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:01:02,739 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:01:03,739 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:01:05,141 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:01:07,563 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:01:07,564 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:01:07,565 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:01:07,566 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:01:07,740 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:01:08,740 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:01:09,059 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:01:09,059 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:01:09,741 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:01:12,822 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:01:12,823 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:01:12,824 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:01:12,824 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:01:13,742 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:01:13,742 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:01:14,742 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:01:18,242 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:01:18,243 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:01:18,243 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:01:18,244 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:01:18,743 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:01:19,744 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:01:20,744 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:01:24,061 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:01:24,061 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:01:27,370 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:01:27,371 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:01:27,372 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:01:27,372 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:01:27,746 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:01:28,746 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:01:29,747 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:01:32,153 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:01:32,154 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:01:32,155 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:01:32,155 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:01:32,748 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:01:33,748 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:01:34,748 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:01:35,143 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:01:36,259 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:01:36,260 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:01:36,260 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:01:36,262 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:01:36,749 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:01:37,749 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:01:38,749 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:01:39,063 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:01:39,063 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:01:41,077 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:01:41,077 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:01:41,078 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:01:41,079 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:01:41,750 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:01:41,750 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:01:42,751 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:01:46,802 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:01:46,803 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:01:46,803 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:01:46,805 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:01:47,752 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:01:47,752 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:01:48,752 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:01:54,065 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:01:54,065 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:01:54,249 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:01:54,250 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:01:54,250 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:01:54,251 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:01:54,754 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:01:55,754 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:01:56,755 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:01:59,107 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:01:59,108 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:01:59,108 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:01:59,109 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:01:59,756 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:01:59,756 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:02:00,756 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:02:04,565 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:02:04,567 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:02:04,567 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:02:04,568 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:02:04,757 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:02:05,145 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:02:05,758 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:02:06,758 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:02:09,067 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:02:09,067 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:02:10,491 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:02:10,492 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:02:10,493 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:02:10,494 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:02:10,759 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:02:11,759 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:02:12,760 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:02:15,879 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:02:15,880 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:02:15,881 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:02:15,882 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:02:16,761 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:02:16,761 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:02:17,761 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:02:20,559 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:02:20,559 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:02:20,560 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:02:20,561 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:02:20,762 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:02:21,762 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:02:22,763 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:02:24,069 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:02:24,069 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:02:25,381 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:02:25,383 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:02:25,383 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:02:25,384 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:02:25,763 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:02:26,764 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:02:27,764 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:02:30,567 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:02:30,568 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:02:30,568 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:02:30,570 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:02:30,765 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:02:31,765 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:02:32,766 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:02:35,146 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:02:38,971 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:02:38,971 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:02:38,972 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:02:38,973 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:02:39,071 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:02:39,072 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:02:39,768 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:02:39,768 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:02:40,768 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:02:44,308 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:02:44,309 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:02:44,309 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:02:44,310 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:02:44,769 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:02:45,770 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:02:46,770 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:02:49,763 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:02:49,765 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:02:49,765 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:02:49,765 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:02:49,771 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:02:50,771 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:02:51,771 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:02:54,073 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:02:54,074 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:02:55,391 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:02:55,392 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:02:55,393 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:02:55,393 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:02:55,773 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:02:56,773 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:02:57,773 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:02:59,719 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:02:59,720 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:02:59,720 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:02:59,721 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:02:59,774 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:03:00,774 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:03:01,775 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:03:04,895 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:03:04,896 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:03:04,896 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:03:04,898 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:03:05,148 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:03:05,776 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:03:05,776 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:03:06,776 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:03:09,075 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:03:09,075 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:03:09,386 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:03:09,387 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:03:09,387 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:03:09,389 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:03:09,777 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:03:10,777 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:03:11,778 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:03:15,230 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:03:15,230 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:03:15,231 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:03:15,232 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:03:15,779 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:03:15,779 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:03:16,779 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:03:24,077 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:03:24,077 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:03:24,407 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:03:24,408 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:03:24,408 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:03:24,409 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:03:24,781 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:03:25,781 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:03:26,782 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:03:28,862 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:03:28,862 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:03:28,863 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:03:28,864 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:03:29,783 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:03:29,783 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:03:30,783 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:03:34,093 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:03:34,094 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:03:34,095 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:03:34,096 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:03:34,784 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:03:35,150 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:03:35,784 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:03:36,785 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:03:39,079 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:03:39,079 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:03:41,126 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:03:41,127 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:03:41,128 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:03:41,129 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:03:41,786 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:03:41,786 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:03:42,786 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:03:46,887 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:03:46,888 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:03:46,888 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:03:46,889 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:03:47,788 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:03:47,788 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:03:48,788 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:03:53,552 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:03:53,553 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:03:53,553 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:03:53,554 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:03:53,790 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:03:54,081 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:03:54,081 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:03:54,790 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:03:55,790 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:03:58,493 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:03:58,494 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:03:58,495 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:03:58,496 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:03:58,791 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:03:59,792 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:04:00,792 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:04:04,993 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:04:04,994 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:04:04,995 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:04:04,996 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:04:05,151 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:04:05,793 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:04:05,793 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:04:06,793 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:04:09,083 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:04:09,084 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:04:10,615 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:04:10,616 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:04:10,616 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:04:10,618 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:04:10,795 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:04:11,795 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:04:12,795 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:04:19,788 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:04:19,788 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:04:19,789 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:04:19,790 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:04:19,797 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:04:20,797 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:04:21,798 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:04:24,085 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:04:24,086 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:04:24,740 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:04:24,740 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:04:24,741 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:04:24,742 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:04:24,799 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:04:25,799 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:04:26,799 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:04:31,591 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:04:31,592 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:04:31,592 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:04:31,593 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:04:31,801 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:04:32,801 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:04:33,801 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:04:35,153 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:04:36,149 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:04:36,150 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:04:36,150 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:04:36,151 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:04:36,802 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:04:36,803 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:04:37,803 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:04:39,087 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:04:39,088 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:04:40,506 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:04:40,507 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:04:40,508 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:04:40,509 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:04:40,803 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:04:41,804 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:04:42,804 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:04:45,601 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:04:45,602 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:04:45,603 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:04:45,604 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:04:45,805 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:04:46,805 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:04:47,806 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:04:50,775 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:04:50,776 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:04:50,776 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:04:50,777 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:04:50,806 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:04:51,807 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:04:52,807 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:04:54,090 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:04:54,090 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:04:57,640 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:04:57,641 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:04:57,641 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:04:57,642 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:04:57,809 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:04:58,809 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:04:59,809 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:05:02,190 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:05:02,190 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:05:02,191 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:05:02,192 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:05:02,810 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:05:02,810 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:05:03,810 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:05:05,155 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:05:06,798 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:05:06,798 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:05:06,799 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:05:06,800 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:05:06,811 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:05:07,812 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:05:08,812 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:05:09,092 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:05:09,092 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:05:12,325 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:05:12,326 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:05:12,326 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:05:12,328 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:05:12,813 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:05:13,813 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:05:14,814 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:05:18,848 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:05:18,849 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:05:18,850 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:05:18,851 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:05:19,815 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:05:19,815 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:05:20,815 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:05:24,094 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:05:24,094 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:05:25,715 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:05:25,716 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:05:25,717 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:05:25,717 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:05:25,817 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:05:26,817 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:05:27,818 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:05:30,901 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:05:30,902 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:05:30,903 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:05:30,904 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:05:31,819 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:05:31,819 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:05:32,819 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:05:35,157 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:05:36,521 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:05:36,522 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:05:36,522 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:05:36,523 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:05:36,820 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:05:37,821 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:05:38,821 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:05:39,096 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:05:39,096 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:05:42,616 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:05:42,617 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:05:42,618 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:05:42,619 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:05:42,822 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:05:43,822 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:05:44,823 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:05:47,609 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:05:47,610 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:05:47,610 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:05:47,612 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:05:47,824 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:05:48,824 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:05:49,824 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:05:54,098 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:05:54,098 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:05:56,723 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:05:56,724 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:05:56,725 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:05:56,726 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:05:56,826 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:05:57,826 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:05:58,827 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:06:02,211 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:06:02,212 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:06:02,213 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:06:02,214 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:06:02,828 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:06:02,828 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:06:03,828 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:06:05,158 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:06:09,100 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:06:09,100 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:06:10,430 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:06:10,431 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:06:10,432 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:06:10,433 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:06:10,830 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:06:11,830 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:06:12,831 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:06:15,972 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:06:15,973 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:06:15,973 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:06:15,974 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:06:16,832 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:06:16,832 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:06:17,832 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:06:20,100 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:06:20,101 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:06:20,101 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:06:20,102 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:06:20,833 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:06:20,833 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:06:21,833 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:06:24,102 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:06:24,103 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:06:25,432 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:06:25,433 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:06:25,434 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:06:25,435 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:06:25,835 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:06:25,835 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:06:26,835 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:06:30,584 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:06:30,584 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:06:30,585 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:06:30,586 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:06:30,836 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:06:31,836 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:06:32,837 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:06:35,160 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:06:39,104 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:06:39,105 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:06:39,251 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:06:39,252 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:06:39,253 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:06:39,253 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:06:39,839 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:06:39,839 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:06:40,839 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:06:45,002 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:06:45,003 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:06:45,004 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:06:45,004 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:06:45,840 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:06:45,841 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:06:46,841 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:06:50,211 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:06:50,212 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:06:50,212 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:06:50,213 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:06:50,842 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:06:50,842 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:06:51,842 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:06:54,107 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:06:54,107 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:06:56,299 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:06:56,300 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:06:56,301 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:06:56,302 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:06:56,844 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:06:56,844 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:06:57,844 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:07:01,719 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:07:01,720 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:07:01,721 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:07:01,722 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:07:01,845 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:07:02,846 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:07:03,846 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:07:05,162 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:07:09,043 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:07:09,043 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:07:09,044 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:07:09,045 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:07:09,109 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:07:09,109 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:07:09,847 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:07:09,848 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:07:10,848 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:07:13,612 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:07:13,613 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:07:13,614 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:07:13,615 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:07:13,849 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:07:14,849 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:07:15,849 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:07:17,923 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:07:17,925 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:07:17,925 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:07:17,926 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:07:18,850 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:07:18,851 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:07:19,851 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:07:22,686 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:07:22,686 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:07:22,687 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:07:22,688 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:07:22,851 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:07:23,852 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:07:24,111 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:07:24,111 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:07:24,852 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:07:35,163 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:07:39,129 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:07:39,129 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:07:54,616 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:07:54,617 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:08:05,165 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:08:11,214 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:08:11,214 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:08:26,287 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:08:26,287 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:08:35,167 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:08:41,593 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:08:41,593 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:08:56,974 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:08:56,974 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:09:05,168 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:09:12,094 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:09:12,094 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:09:27,096 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:09:27,096 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:09:35,170 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:09:42,391 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:09:42,392 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:09:57,831 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:09:57,831 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:10:05,172 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:10:13,450 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:10:13,451 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:10:28,896 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:10:28,896 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:10:34,907 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:10:35,174 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:10:41,414 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:10:41,414 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:10:41,415 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:10:41,416 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:10:41,909 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:10:41,909 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:10:42,909 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:10:44,114 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:10:44,114 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:10:50,489 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:10:50,490 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:10:50,490 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:10:50,491 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:10:50,911 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:10:50,912 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:10:51,912 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:10:56,850 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:10:56,851 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:10:56,852 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:10:56,853 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:10:56,913 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:10:57,913 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:10:58,914 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:10:59,116 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:10:59,116 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:11:02,260 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:11:02,261 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:11:02,262 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:11:02,263 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:11:02,915 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:11:02,915 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:11:03,915 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:11:05,175 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:11:09,271 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:11:09,272 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:11:09,272 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:11:09,274 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:11:09,917 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:11:09,917 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:11:10,917 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:11:13,623 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:11:13,624 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:11:13,625 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:11:13,626 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:11:13,918 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:11:13,918 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:11:14,118 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:11:14,118 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:11:14,918 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:11:19,419 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:11:19,420 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:11:19,420 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:11:19,422 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:11:19,920 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:11:19,920 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:11:20,920 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:11:26,847 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:11:26,848 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:11:26,848 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:11:26,850 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:11:26,922 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:11:27,922 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:11:28,922 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:11:29,120 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:11:29,120 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:11:31,697 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:11:31,698 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:11:31,699 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:11:31,700 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:11:31,923 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:11:31,924 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:11:32,924 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:11:35,177 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:11:36,201 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:11:36,202 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:11:36,203 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:11:36,204 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:11:36,925 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:11:36,925 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:11:37,925 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:11:41,793 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:11:41,794 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:11:41,795 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:11:41,795 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:11:41,926 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:11:41,927 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:11:42,927 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:11:44,122 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:11:44,122 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:11:48,616 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:11:48,617 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:11:48,617 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:11:48,618 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:11:48,928 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:11:48,929 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:11:49,929 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:11:53,445 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:11:53,445 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:11:53,446 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:11:53,447 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:11:53,930 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:11:53,930 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:11:54,930 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:11:58,050 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:11:58,051 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:11:58,052 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:11:58,053 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:11:58,931 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:11:58,932 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:11:59,124 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:11:59,124 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:11:59,932 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:12:02,906 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:12:02,907 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:12:02,908 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:12:02,909 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:12:02,932 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:12:03,933 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:12:04,933 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:12:05,179 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:12:07,472 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:12:07,473 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:12:07,473 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:12:07,474 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:12:07,934 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:12:07,934 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:12:09,935 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:12:11,778 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:12:11,778 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:12:11,779 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:12:11,780 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:12:11,935 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:12:11,935 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:12:13,936 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:12:14,126 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:12:14,126 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:12:16,123 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:12:16,124 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:12:16,125 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:12:16,126 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:12:16,937 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:12:17,937 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:12:23,649 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:12:23,650 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:12:23,650 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:12:23,651 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:12:23,939 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:12:24,939 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:12:25,939 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:12:29,128 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:12:29,129 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:12:30,765 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:12:30,766 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:12:30,766 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:12:30,767 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:12:30,941 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:12:32,941 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:12:33,942 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:12:35,180 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:12:36,023 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:12:36,024 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:12:36,024 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:12:36,025 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:12:36,943 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:12:37,943 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:12:38,943 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:12:40,508 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:12:40,509 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:12:40,510 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:12:40,511 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:12:40,944 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:12:41,944 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:12:42,945 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:12:44,131 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:12:44,131 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:12:45,193 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:12:45,194 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:12:45,195 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:12:45,195 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:12:45,945 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:12:46,946 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:12:47,946 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:12:50,550 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:12:50,551 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:12:50,552 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:12:50,552 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:12:50,947 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:12:51,947 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:12:52,948 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:12:57,222 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:12:57,223 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:12:57,224 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:12:57,225 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:12:57,949 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:12:58,950 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:12:59,133 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:12:59,133 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:12:59,950 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:13:05,060 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:13:05,061 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:13:05,061 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:13:05,062 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:13:05,182 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:13:05,952 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:13:06,952 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:13:07,952 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:13:09,602 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:13:09,603 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:13:09,604 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:13:09,605 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:13:09,953 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:13:10,953 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:13:11,953 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:13:14,135 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:13:14,135 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:13:17,828 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:13:17,829 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:13:17,829 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:13:17,830 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:13:17,955 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:13:18,955 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:13:19,956 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:13:25,429 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:13:25,430 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:13:25,430 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:13:25,432 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:13:25,957 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:13:26,958 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:13:27,958 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:13:29,137 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:13:29,138 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:13:31,290 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:13:31,291 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:13:31,292 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:13:31,292 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:13:31,959 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:13:32,960 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:13:33,960 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:13:35,184 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:13:38,157 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:13:38,158 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:13:38,158 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:13:38,159 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:13:38,961 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:13:39,962 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:13:40,962 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:13:42,438 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:13:42,439 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:13:42,439 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:13:42,441 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:13:42,963 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:13:43,963 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:13:44,139 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:13:44,140 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:13:44,963 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:13:50,517 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:13:50,518 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:13:50,519 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:13:50,519 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:13:50,965 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:13:51,965 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:13:52,966 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:13:55,085 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:13:55,085 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:13:55,086 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:13:55,087 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:13:55,966 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:13:56,967 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:13:57,967 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:13:59,141 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:13:59,142 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:14:02,519 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:14:02,520 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:14:02,521 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:14:02,522 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:14:02,968 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:14:03,969 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:14:04,969 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:14:05,185 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:14:11,639 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:14:11,641 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:14:11,641 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:14:11,642 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:14:11,971 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:14:12,971 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:14:13,972 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:14:14,144 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:14:14,144 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:14:16,852 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:14:16,854 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:14:16,854 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:14:16,855 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:14:16,972 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:14:18,973 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:14:19,974 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:14:22,342 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:14:22,343 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:14:22,343 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:14:22,345 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:14:22,975 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:14:23,975 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:14:24,976 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:14:28,161 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:14:28,162 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:14:28,162 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:14:28,163 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:14:28,977 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:14:29,146 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:14:29,146 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:14:29,977 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:14:30,978 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:14:35,187 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:14:37,282 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:14:37,283 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:14:37,284 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:14:37,285 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:14:37,979 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:14:38,980 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:14:39,980 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:14:43,500 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:14:43,501 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:14:43,502 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:14:43,502 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:14:43,981 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:14:44,148 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:14:44,148 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:14:44,982 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:14:45,982 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:14:47,718 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:14:47,719 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:14:47,719 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:14:47,720 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:14:47,983 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:14:48,983 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:14:49,983 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:14:52,933 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:14:52,934 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:14:52,935 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:14:52,937 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:14:52,984 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:14:53,985 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:14:54,985 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:14:57,164 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:14:57,165 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:14:57,165 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:14:57,166 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:14:57,986 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:14:58,986 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:14:59,150 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:14:59,150 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:14:59,986 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:15:02,938 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:15:02,939 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:15:02,940 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:15:02,940 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:15:02,987 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:15:03,988 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:15:04,988 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:15:05,189 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:15:08,516 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:15:08,517 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:15:08,517 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:15:08,518 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:15:08,989 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:15:09,989 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:15:10,990 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:15:14,152 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:15:14,152 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:15:14,492 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:15:14,493 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:15:14,493 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:15:14,494 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:15:14,991 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:15:15,991 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:15:16,992 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:15:18,805 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:15:18,806 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:15:18,806 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:15:18,807 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:15:18,992 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:15:19,992 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:15:20,993 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:15:23,925 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:15:23,927 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:15:23,927 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:15:23,929 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:15:23,994 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:15:24,994 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:15:25,995 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:15:28,890 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:15:28,891 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:15:28,891 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:15:28,892 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:15:28,995 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:15:29,154 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:15:29,154 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:15:29,996 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:15:30,996 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:15:33,921 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:15:33,922 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:15:33,922 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:15:33,924 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:15:33,997 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:15:34,997 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:15:35,190 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:15:35,998 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:15:42,199 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:15:42,200 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:15:42,200 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:15:42,201 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:15:43,000 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:15:44,000 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:15:44,156 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:15:44,156 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:15:45,000 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:15:48,814 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:15:48,815 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:15:48,816 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:15:48,817 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:15:49,001 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:15:50,002 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:15:51,002 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:15:54,529 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:15:54,530 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:15:54,531 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:15:54,532 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:15:55,003 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:15:56,004 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:15:57,004 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:15:59,158 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:15:59,158 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:16:03,717 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:16:03,718 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:16:03,719 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:16:03,720 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:16:04,006 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:16:05,006 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:16:05,192 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:16:06,007 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:16:10,888 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:16:10,889 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:16:10,889 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:16:10,890 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:16:11,008 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:16:12,008 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:16:13,009 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:16:14,160 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:16:14,161 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:16:15,949 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:16:15,950 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:16:15,951 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:16:15,951 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:16:16,010 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:16:17,010 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:16:18,010 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:16:21,091 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:16:21,092 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:16:21,093 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:16:21,094 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:16:22,012 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:16:23,012 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:16:24,012 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:16:29,162 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:16:29,163 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:16:29,476 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:16:29,477 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:16:29,478 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:16:29,479 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:16:30,014 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:16:31,014 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:16:32,014 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:16:34,939 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:16:34,940 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:16:34,940 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:16:34,941 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:16:35,015 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:16:35,193 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:16:36,016 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:16:37,016 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:16:41,346 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:16:41,347 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:16:41,347 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:16:41,348 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:16:42,017 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:16:43,018 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:16:44,018 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:16:44,165 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:16:44,165 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:16:47,269 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:16:47,270 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:16:47,270 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:16:47,272 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:16:48,019 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:16:49,019 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:16:50,020 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:16:52,173 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:16:52,174 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:16:52,174 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:16:52,176 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:16:53,021 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:16:54,021 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:16:55,022 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:16:57,358 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:16:57,359 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:16:57,359 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:16:57,361 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:16:58,022 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:16:59,023 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:16:59,167 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:16:59,168 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:17:00,023 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:17:02,838 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:17:02,839 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:17:02,840 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:17:02,841 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:17:03,024 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:17:04,024 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:17:05,025 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:17:05,195 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:17:06,785 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:17:06,786 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:17:06,786 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:17:06,788 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:17:07,025 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:17:08,026 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:17:09,026 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:17:12,482 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:17:12,483 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:17:12,484 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:17:12,485 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:17:13,027 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:17:14,027 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:17:14,169 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:17:14,170 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:17:15,028 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:17:17,636 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:17:17,637 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:17:17,637 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:17:17,638 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:17:18,029 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:17:19,029 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:17:20,029 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:17:22,636 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:17:22,638 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:17:22,638 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:17:22,639 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:17:23,030 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:17:24,031 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:17:25,031 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:17:27,386 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:17:27,387 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:17:27,388 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:17:27,389 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:17:28,032 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:17:29,032 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:17:29,171 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:17:29,172 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:17:30,033 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:17:32,813 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:17:32,814 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:17:32,815 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:17:32,815 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:17:33,033 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:17:34,034 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:17:35,034 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:17:35,197 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:17:37,519 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:17:37,520 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:17:37,520 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:17:37,521 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:17:38,035 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:17:39,035 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:17:40,036 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:17:44,172 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:17:44,172 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:17:44,173 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:17:44,174 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:17:44,174 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:17:44,175 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:17:45,037 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:17:46,038 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:17:47,038 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:17:49,788 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:17:49,789 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:17:49,790 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:17:49,791 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:17:50,039 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:17:51,039 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:17:52,040 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:17:56,110 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:17:56,111 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:17:56,112 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:17:56,113 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:17:57,041 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:17:57,041 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:17:58,041 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:17:59,176 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:17:59,177 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:18:05,199 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:18:05,273 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:18:05,274 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:18:05,274 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:18:05,275 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:18:06,043 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:18:07,044 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:18:08,044 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:18:09,405 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:18:09,406 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:18:09,407 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:18:09,408 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:18:10,045 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:18:11,045 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:18:12,045 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:18:14,179 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:18:14,179 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:18:15,743 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:18:15,744 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:18:15,744 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:18:15,745 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:18:16,047 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:18:17,047 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:18:18,047 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:18:22,372 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:18:22,373 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:18:22,373 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:18:22,374 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:18:23,048 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:18:24,049 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:18:25,049 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:18:27,013 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:18:27,014 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:18:27,015 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:18:27,015 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:18:27,050 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:18:28,050 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:18:29,050 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:18:29,181 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:18:29,181 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:18:31,477 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:18:31,478 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:18:31,478 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:18:31,479 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:18:32,051 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:18:33,051 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:18:34,052 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:18:35,200 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:18:40,559 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:18:40,561 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:18:40,561 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:18:40,561 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:18:41,054 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:18:42,054 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:18:43,055 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:18:44,183 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:18:44,183 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:18:46,531 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:18:46,532 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:18:46,532 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:18:46,534 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:18:47,056 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:18:48,056 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:18:49,056 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:18:55,067 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:18:55,068 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:18:55,069 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:18:55,070 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:18:56,058 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:18:56,059 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:18:57,059 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:18:59,185 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:18:59,185 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:18:59,895 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:18:59,896 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:18:59,897 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:18:59,898 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:19:00,060 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:19:01,060 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:19:02,060 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:19:05,202 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:19:05,528 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:19:05,529 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:19:05,529 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:19:05,530 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:19:06,063 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:19:07,062 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:19:08,062 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:19:12,083 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:19:12,084 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:19:12,084 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:19:12,085 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:19:13,064 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:19:13,064 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:19:14,064 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:19:14,187 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:19:14,188 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:19:16,421 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:19:16,422 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:19:16,423 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:19:16,424 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:19:17,065 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:19:18,065 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:19:19,066 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:19:21,754 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:19:21,755 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:19:21,755 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:19:21,756 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:19:22,067 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:19:23,067 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:19:24,067 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:19:29,189 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:19:29,190 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:19:30,770 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:19:30,771 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:19:30,771 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:19:30,773 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:19:31,069 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:19:32,070 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:19:33,070 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:19:35,204 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:19:35,801 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:19:35,802 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:19:35,802 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:19:35,803 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:19:36,071 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:19:37,072 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:19:38,072 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:19:40,968 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:19:40,968 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:19:40,969 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:19:40,970 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:19:41,073 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:19:42,073 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:19:43,074 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:19:44,191 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:19:44,192 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:19:46,255 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:19:46,256 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:19:46,257 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:19:46,257 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:19:47,075 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:19:47,075 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:19:48,075 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:19:52,595 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:19:52,596 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:19:52,597 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:19:52,598 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:19:53,076 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:19:54,077 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:19:55,077 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:19:58,389 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:19:58,390 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:19:58,391 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:19:58,392 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:19:59,078 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:19:59,194 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:19:59,194 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:20:00,079 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:20:01,079 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:20:05,205 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:20:06,831 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:20:06,832 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:20:06,833 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:20:06,833 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:20:07,081 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:20:08,081 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:20:09,081 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:20:14,196 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:20:14,197 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:20:16,008 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:20:16,009 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:20:16,010 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:20:16,011 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:20:16,083 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:20:17,084 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:20:18,084 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:20:21,365 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:20:21,366 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:20:21,366 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:20:21,367 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:20:22,085 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:20:22,086 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:20:23,086 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:20:26,444 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:20:26,445 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:20:26,446 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:20:26,446 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:20:27,087 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:20:28,087 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:20:29,088 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:20:29,198 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:20:29,199 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:20:35,207 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:20:35,595 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:20:35,596 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:20:35,597 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:20:35,597 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:20:36,089 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:20:37,090 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:20:38,090 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:20:39,953 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:20:39,953 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:20:39,954 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:20:39,955 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:20:40,091 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:20:41,091 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:20:42,091 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:20:44,201 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:20:44,201 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:20:44,712 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:20:44,713 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:20:44,714 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:20:44,714 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:20:45,092 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:20:46,093 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:20:47,093 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:20:50,293 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:20:50,294 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:20:50,295 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:20:50,296 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:20:51,094 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:20:51,094 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:20:52,095 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:20:55,266 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:20:55,267 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:20:55,267 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:20:55,268 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:20:56,096 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:20:56,096 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:20:57,096 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:20:59,203 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:20:59,203 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:21:00,412 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:21:00,413 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:21:00,414 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:21:00,415 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:21:01,097 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:21:02,097 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:21:03,098 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:21:05,209 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:21:09,622 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:21:09,624 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:21:09,624 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:21:09,625 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:21:10,100 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:21:11,100 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:21:12,100 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:21:14,175 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:21:14,176 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:21:14,177 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:21:14,178 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:21:14,205 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:21:14,205 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:21:15,101 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:21:15,102 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:21:16,102 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:21:18,643 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:21:18,644 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:21:18,644 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:21:18,646 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:21:19,103 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:21:20,103 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:21:21,103 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:21:23,850 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:21:23,851 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:21:23,852 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:21:23,853 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:21:24,104 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:21:25,105 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:21:26,105 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:21:29,207 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:21:29,207 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:21:30,308 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:21:30,309 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:21:30,309 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:21:30,311 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:21:31,106 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:21:31,107 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:21:32,107 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:21:35,210 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:21:36,138 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:21:36,139 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:21:36,140 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:21:36,140 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:21:37,108 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:21:37,109 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:21:38,109 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:21:41,780 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:21:41,781 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:21:41,781 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:21:41,782 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:21:42,110 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:21:43,110 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:21:44,111 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:21:44,209 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:21:44,209 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:21:50,990 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:21:50,991 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:21:50,991 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:21:50,992 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:21:51,113 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:21:52,113 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:21:53,113 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:21:56,734 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:21:56,735 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:21:56,736 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:21:56,737 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:21:57,114 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:21:58,115 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:21:59,115 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:21:59,211 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:21:59,211 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:22:05,212 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:22:05,558 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:22:05,560 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:22:05,560 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:22:05,561 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:22:06,117 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:22:07,117 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:22:08,118 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:22:11,526 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:22:11,527 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:22:11,528 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:22:11,529 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:22:12,119 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:22:13,119 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:22:14,120 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:22:14,213 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:22:14,213 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:22:16,544 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:22:16,546 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:22:16,546 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:22:16,547 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:22:17,120 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:22:18,121 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:22:19,121 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:22:22,146 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:22:22,147 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:22:22,148 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:22:22,149 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:22:23,122 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:22:23,123 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:22:24,123 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:22:27,508 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:22:27,509 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:22:27,510 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:22:27,511 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:22:28,124 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:22:28,124 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:22:29,124 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:22:29,215 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:22:29,215 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:22:32,855 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:22:32,856 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:22:32,857 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:22:32,858 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:22:33,125 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:22:34,126 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:22:35,126 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:22:35,214 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:22:38,341 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:22:38,342 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:22:38,342 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:22:38,343 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:22:39,127 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:22:39,127 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:22:40,128 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:22:43,799 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:22:43,800 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:22:43,800 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:22:43,802 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:22:44,129 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:22:44,217 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:22:44,217 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:22:45,129 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:22:46,129 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:22:49,916 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:22:49,917 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:22:49,918 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:22:49,919 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:22:50,131 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:22:51,131 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:22:52,131 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:22:57,179 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:22:57,180 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:22:57,180 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:22:57,182 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:22:58,133 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:22:58,133 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:22:59,133 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:22:59,219 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:22:59,219 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:23:02,643 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:23:02,644 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:23:02,644 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:23:02,646 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:23:03,135 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:23:04,135 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:23:05,135 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:23:05,215 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:23:11,325 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:23:11,326 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:23:11,326 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:23:11,328 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:23:12,138 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:23:12,138 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:23:13,138 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:23:14,221 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:23:14,221 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:23:15,843 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:23:15,844 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:23:15,845 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:23:15,846 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:23:16,139 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:23:17,139 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:23:18,139 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:23:25,025 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:23:25,025 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:23:25,026 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:23:25,027 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:23:25,141 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:23:26,142 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:23:27,142 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:23:29,223 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:23:29,224 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:23:34,204 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:23:34,206 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:23:34,207 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:23:34,209 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:23:35,144 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:23:35,144 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:23:35,217 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:23:36,144 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:23:39,053 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:23:39,054 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:23:39,055 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:23:39,056 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:23:39,145 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:23:40,146 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:23:41,146 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:23:44,225 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:23:44,226 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:23:45,849 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:23:45,850 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:23:45,851 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:23:45,851 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:23:46,148 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:23:47,148 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:23:48,148 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:23:50,283 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:23:50,284 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:23:50,284 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:23:50,286 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:23:51,149 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:23:51,149 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:23:52,150 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:23:55,208 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:23:55,209 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:23:55,210 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:23:55,210 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:23:56,151 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:23:56,151 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:23:57,151 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:23:59,228 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:23:59,228 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:24:01,065 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:24:01,066 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:24:01,066 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:24:01,067 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:24:01,152 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:24:02,152 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:24:03,153 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:24:05,219 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:24:05,672 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:24:05,673 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:24:05,673 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:24:05,674 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:24:06,154 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:24:07,154 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:24:08,154 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:24:09,813 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:24:09,814 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:24:09,815 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:24:09,816 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:24:10,155 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:24:11,155 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:24:12,155 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:24:14,230 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:24:14,230 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:24:14,976 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:24:14,977 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:24:14,977 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:24:14,979 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:24:15,156 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:24:16,157 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:24:17,157 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:24:18,828 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:24:18,829 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:24:18,829 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:24:18,830 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:24:19,157 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:24:20,158 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:24:21,158 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:24:23,419 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:24:23,419 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:24:23,420 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:24:23,421 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:24:24,159 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:24:24,159 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:24:25,159 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:24:28,661 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:24:28,662 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:24:28,662 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:24:28,663 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:24:29,160 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:24:29,232 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:24:29,232 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:24:30,161 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:24:31,161 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:24:32,772 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:24:32,773 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:24:32,773 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:24:32,774 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:24:33,162 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:24:34,162 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:24:35,162 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:24:35,220 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:24:38,602 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:24:38,603 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:24:38,603 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:24:38,606 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:24:39,164 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:24:40,164 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:24:41,164 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:24:43,011 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:24:43,012 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:24:43,012 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:24:43,013 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:24:43,165 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:24:44,165 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:24:44,234 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:24:44,234 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:24:45,165 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:24:48,159 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:24:48,160 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:24:48,160 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:24:48,161 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:24:48,166 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:24:49,167 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:24:50,167 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:24:52,821 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:24:52,822 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:24:52,822 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:24:52,823 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:24:53,168 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:24:54,168 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:24:55,168 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:24:58,308 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:24:58,308 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:24:58,309 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:24:58,310 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:24:59,170 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:24:59,170 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:24:59,236 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:24:59,236 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:25:00,170 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:25:05,223 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:25:06,615 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:25:06,616 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:25:06,617 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:25:06,617 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:25:07,172 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:25:08,172 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:25:09,173 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:25:12,001 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:25:12,001 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:25:12,002 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:25:12,003 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:25:12,174 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:25:13,174 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:25:14,174 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:25:14,238 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:25:14,238 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:25:16,746 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:25:16,747 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:25:16,747 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:25:16,748 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:25:17,175 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:25:18,175 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:25:19,176 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:25:23,112 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:25:23,113 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:25:23,113 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:25:23,114 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:25:23,177 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:25:24,177 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:25:25,178 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:25:28,438 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:25:28,439 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:25:28,440 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:25:28,440 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:25:29,179 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:25:29,179 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:25:29,241 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:25:29,241 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:25:30,179 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:25:32,784 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:25:32,785 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:25:32,785 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:25:32,786 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:25:33,180 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:25:34,181 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:25:35,181 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:25:35,224 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:25:38,299 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:25:38,301 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:25:38,301 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:25:38,301 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:25:39,182 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:25:39,182 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:25:40,182 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:25:44,243 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:25:44,243 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:25:45,630 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:25:45,631 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:25:45,631 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:25:45,633 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:25:46,184 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:25:46,184 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:25:47,184 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:25:50,589 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:25:50,590 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:25:50,590 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:25:50,591 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:25:51,186 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:25:51,186 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:25:52,186 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:25:56,052 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:25:56,053 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:25:56,053 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:25:56,054 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:25:56,187 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:25:57,187 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:25:58,188 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:25:59,245 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:25:59,245 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:26:01,770 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:26:01,772 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:26:01,772 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:26:01,773 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:26:02,189 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:26:02,190 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:26:03,189 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:26:05,226 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:26:07,624 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:26:07,625 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:26:07,625 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:26:07,627 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:26:08,191 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:26:08,191 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:26:09,191 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:26:14,247 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:26:14,247 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:26:14,723 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:26:14,723 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:26:14,724 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:26:14,725 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:26:15,193 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:26:16,193 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:26:17,193 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:26:19,237 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:26:19,237 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:26:19,238 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:26:19,239 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:26:20,194 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:26:20,195 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:26:21,195 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:26:24,205 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:26:24,206 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:26:24,206 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:26:24,207 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:26:25,196 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:26:25,196 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:26:26,196 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:26:29,249 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:26:29,249 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:26:29,794 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:26:29,795 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:26:29,795 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:26:29,796 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:26:30,197 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:26:30,197 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:26:31,198 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:26:35,228 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:26:38,966 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:26:38,967 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:26:38,968 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:26:38,969 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:26:39,200 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:26:40,200 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:26:41,201 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:26:44,040 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:26:44,041 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:26:44,042 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:26:44,043 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:26:44,201 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:26:44,251 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:26:44,251 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:26:45,202 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:26:46,202 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:26:48,596 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:26:48,598 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:26:48,599 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:26:48,599 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:26:49,203 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:26:49,203 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:26:50,203 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:26:57,697 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:26:57,699 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:26:57,699 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:26:57,700 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:26:58,206 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:26:58,206 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:26:59,206 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:26:59,253 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:26:59,253 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:27:04,280 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:27:04,281 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:27:04,281 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:27:04,283 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:27:05,208 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:27:05,208 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:27:05,229 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:27:06,208 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:27:09,461 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:27:09,462 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:27:09,463 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:27:09,463 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:27:10,209 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:27:10,209 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:27:11,210 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:27:14,254 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:27:14,255 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:27:14,796 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:27:14,797 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:27:14,798 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:27:14,799 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:27:15,211 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:27:16,211 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:27:17,211 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:27:23,991 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:27:23,991 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:27:23,992 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:27:23,993 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:27:24,213 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:27:25,214 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:27:26,214 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:27:29,256 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:27:29,257 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:27:29,810 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:27:29,811 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:27:29,811 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:27:29,812 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:27:30,215 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:27:30,215 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:27:31,215 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:27:35,231 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:27:35,931 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:27:35,932 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:27:35,933 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:27:35,934 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:27:36,217 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:27:37,217 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:27:38,218 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:27:40,478 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:27:40,479 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:27:40,480 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:27:40,481 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:27:41,219 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:27:41,219 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:27:42,219 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:27:44,258 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:27:44,259 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:27:44,853 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:27:44,854 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:27:44,855 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:27:44,856 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:27:45,220 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:27:46,220 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:27:47,220 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:27:50,495 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:27:50,495 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:27:50,496 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:27:50,497 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:27:51,221 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:27:51,222 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:27:52,222 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:27:59,261 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:27:59,261 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:28:05,233 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:28:14,263 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:28:14,263 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:28:29,265 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:28:29,265 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:28:35,234 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:28:44,267 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:28:44,268 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:28:59,270 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:28:59,270 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:29:05,236 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:29:14,272 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:29:14,272 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:29:29,274 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:29:29,274 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:29:34,251 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:29:35,237 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:29:39,520 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:29:39,522 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:29:39,522 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:29:39,525 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:29:40,258 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:29:40,258 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:29:41,259 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:29:44,276 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:29:44,276 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:29:44,284 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:29:44,284 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:29:44,285 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:29:44,285 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:29:45,260 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:29:45,260 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:29:46,260 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:29:50,186 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:29:50,187 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:29:50,187 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:29:50,188 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:29:50,261 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:29:51,262 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:29:52,262 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:29:57,018 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:29:57,019 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:29:57,020 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:29:57,020 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:29:57,263 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:29:58,264 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:29:59,264 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:29:59,278 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:29:59,278 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:30:01,763 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:30:01,764 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:30:01,764 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:30:01,765 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:30:02,265 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:30:02,265 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:30:03,266 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:30:05,239 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:30:07,596 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:30:07,597 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:30:07,598 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:30:07,599 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:30:08,267 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:30:08,267 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:30:09,267 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:30:12,774 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:30:12,775 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:30:12,776 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:30:12,777 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:30:13,269 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:30:13,269 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:30:14,269 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:30:14,280 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:30:14,280 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:30:17,620 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:30:17,621 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:30:17,622 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:30:17,623 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:30:18,270 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:30:18,270 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:30:19,271 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:30:26,757 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:30:26,757 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:30:26,758 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:30:26,759 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:30:27,273 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:30:27,273 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:30:28,273 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:30:29,282 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:30:29,282 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:30:32,051 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:30:32,052 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:30:32,052 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:30:32,053 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:30:32,274 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:30:32,275 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:30:33,275 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:30:35,241 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:30:39,734 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:30:39,735 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:30:39,735 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:30:39,737 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:30:40,277 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:30:40,277 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:30:41,277 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:30:44,284 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:30:44,284 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:30:46,787 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:30:46,788 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:30:46,788 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:30:46,789 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:30:47,279 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:30:47,279 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:30:48,279 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:30:51,784 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:30:51,785 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:30:51,785 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:30:51,786 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:30:52,280 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:30:52,281 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:30:53,281 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:30:56,918 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:30:56,918 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:30:56,919 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:30:56,920 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:30:57,282 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:30:57,282 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:30:58,282 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:30:59,286 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:30:59,286 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:31:02,575 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:31:02,575 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:31:02,576 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:31:02,577 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:31:03,284 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:31:03,284 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:31:04,284 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:31:05,242 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:31:07,561 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:31:07,562 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:31:07,563 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:31:07,564 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:31:08,285 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:31:08,285 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:31:09,285 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:31:13,257 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:31:13,258 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:31:13,259 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:31:13,260 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:31:13,287 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:31:14,287 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:31:14,288 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:31:14,288 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:31:15,287 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:31:19,308 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:31:19,309 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:31:19,310 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:31:19,310 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:31:20,289 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:31:20,289 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:31:21,289 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:31:26,941 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:31:26,942 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:31:26,943 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:31:26,944 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:31:27,291 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:31:27,291 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:31:28,291 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:31:29,290 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:31:29,290 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:31:32,541 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:31:32,542 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:31:32,543 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:31:32,544 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:31:33,293 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:31:33,293 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:31:34,293 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:31:35,245 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:31:37,288 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:31:37,288 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:31:37,289 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:31:37,290 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:31:37,294 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:31:39,294 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:31:44,292 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:31:44,292 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:31:46,464 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:31:46,465 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:31:46,466 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:31:46,466 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:31:47,297 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:31:47,297 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:31:49,297 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:31:51,944 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:31:51,945 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:31:51,945 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:31:51,947 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:31:52,298 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:31:53,299 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:31:57,989 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:31:57,990 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:31:57,991 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:31:57,992 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:31:58,300 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:31:59,294 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:31:59,295 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:31:59,300 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:32:04,310 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:32:04,311 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:32:04,311 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:32:04,312 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:32:05,246 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:32:05,303 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:32:05,303 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:32:07,303 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:32:09,501 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:32:09,502 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:32:09,503 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:32:09,504 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:32:10,304 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:32:11,305 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:32:14,297 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:32:14,297 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:32:18,672 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:32:18,673 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:32:18,673 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:32:18,674 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:32:19,307 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:32:19,307 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:32:21,307 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:32:25,251 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:32:25,253 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:32:25,253 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:32:25,254 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:32:25,308 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:32:27,309 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:32:29,230 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:32:29,230 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:32:29,231 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:32:29,232 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:32:29,299 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:32:29,299 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:32:29,310 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:32:31,310 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:32:34,550 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:32:34,551 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:32:34,552 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:32:34,553 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:32:35,248 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:32:35,312 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:32:35,312 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:32:37,312 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:32:39,763 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:32:39,763 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:32:39,764 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:32:39,765 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:32:40,313 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:32:41,314 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:32:44,300 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:32:44,301 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:32:44,329 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:32:44,329 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:32:44,330 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:32:44,330 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:32:45,315 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:32:45,315 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:32:47,315 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:32:50,922 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:32:50,923 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:32:50,924 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:32:50,925 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:32:51,316 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:32:51,317 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:32:53,317 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:32:55,437 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:32:55,438 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:32:55,438 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:32:55,439 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:32:56,318 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:32:57,318 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:32:59,303 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:32:59,303 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:33:00,639 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:33:00,639 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:33:00,640 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:33:00,641 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:33:01,320 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:33:01,320 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:33:03,320 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:33:05,249 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:33:07,713 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:33:07,714 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:33:07,714 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:33:07,715 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:33:08,322 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:33:09,322 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:33:14,089 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:33:14,090 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:33:14,091 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:33:14,092 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:33:14,305 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:33:14,305 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:33:14,323 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:33:15,324 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:33:18,584 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:33:18,585 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:33:18,586 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:33:18,587 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:33:19,325 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:33:19,325 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:33:21,325 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:33:24,346 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:33:24,347 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:33:24,348 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:33:24,349 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:33:25,326 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:33:25,327 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:33:29,307 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:33:29,307 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:33:30,632 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:33:30,633 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:33:30,634 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:33:30,635 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:33:31,328 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:33:31,329 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:33:33,329 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:33:35,252 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:33:39,806 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:33:39,808 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:33:39,808 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:33:39,808 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:33:40,331 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:33:41,331 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:33:44,309 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:33:44,309 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:33:47,879 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:33:47,880 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:33:47,881 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:33:47,882 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:33:48,334 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:33:49,334 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:33:53,072 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:33:53,073 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:33:53,074 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:33:53,075 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:33:53,335 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:33:53,336 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:33:55,336 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:33:59,311 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:33:59,311 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:33:59,714 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:33:59,715 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:33:59,716 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:33:59,717 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:34:00,338 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:34:01,338 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:34:05,255 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:34:06,497 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:34:06,498 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:34:06,499 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:34:06,499 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:34:07,340 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:34:07,340 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:34:09,340 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:34:10,526 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:34:10,527 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:34:10,527 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:34:10,528 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:34:11,341 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:34:11,341 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:34:13,342 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:34:14,313 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:34:14,313 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:34:15,896 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:34:15,897 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:34:15,898 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:34:15,898 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:34:16,343 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:34:17,343 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:34:20,392 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:34:20,393 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:34:20,394 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:34:20,395 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:34:21,344 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:34:21,344 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:34:26,653 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:34:26,654 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:34:26,655 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:34:26,655 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:34:27,346 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:34:27,346 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:34:29,315 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:34:29,315 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:34:29,346 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:34:30,858 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:34:30,860 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:34:30,860 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:34:30,861 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:34:31,347 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:34:31,347 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:34:33,348 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:34:35,256 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:34:37,439 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:34:37,440 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:34:37,440 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:34:37,441 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:34:38,349 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:34:39,349 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:34:44,317 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:34:44,317 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:34:45,605 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:34:45,606 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:34:45,606 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:34:45,609 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:34:46,351 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:34:47,352 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:34:59,319 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:34:59,319 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:35:05,258 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:35:14,343 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:35:14,344 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:35:29,641 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:35:29,642 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:35:35,260 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:35:44,777 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:35:44,777 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:36:01,364 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:36:01,365 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:36:05,262 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:36:18,071 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:36:18,071 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:36:33,192 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:36:33,192 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:36:35,263 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:36:48,194 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:36:48,194 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:37:04,315 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:37:04,315 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:37:05,265 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:37:19,695 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:37:19,695 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:37:35,114 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:37:35,114 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:37:35,267 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:37:51,013 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:37:51,014 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:37:57,811 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:37:57,812 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:37:57,853 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:37:57,854 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: partial_history 2024-06-21 12:37:57,855 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:37:57,855 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:37:57,856 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:37:57,856 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:37:57,856 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:37:57,856 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:37:57,856 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:37:57,856 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: history 2024-06-21 12:37:57,857 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:37:57,857 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:37:58,406 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:37:59,407 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:38:00,407 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:38:05,268 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:38:06,743 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:38:06,743 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:38:24,025 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:38:24,026 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:38:35,270 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:38:39,687 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: stop_status 2024-06-21 12:38:39,688 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: stop_status 2024-06-21 12:38:44,238 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: exit 2024-06-21 12:38:44,239 INFO SenderThread:26947 [tracking_sender.py:send_exit():438] handling exit code: 0 2024-06-21 12:38:44,239 INFO SenderThread:26947 [tracking_sender.py:send_exit():440] handling runtime: 4779 2024-06-21 12:38:44,240 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:38:44,240 INFO SenderThread:26947 [tracking_sender.py:send_exit():446] send defer 2024-06-21 12:38:44,241 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: defer 2024-06-21 12:38:44,241 INFO HandlerThread:26947 [handler.py:handle_request_defer():163] handle defer: 0 2024-06-21 12:38:44,241 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: defer 2024-06-21 12:38:44,242 INFO SenderThread:26947 [tracking_sender.py:send_request_defer():455] handle sender defer: 0 2024-06-21 12:38:44,242 INFO SenderThread:26947 [tracking_sender.py:transition_state():459] send defer: 1 2024-06-21 12:38:44,242 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: defer 2024-06-21 12:38:44,242 INFO HandlerThread:26947 [handler.py:handle_request_defer():163] handle defer: 1 2024-06-21 12:38:44,242 INFO HandlerThread:26947 [system_monitor.py:finish():160] Stopping system monitor 2024-06-21 12:38:44,245 DEBUG SystemMonitor:26947 [system_monitor.py:_start():137] Finished system metrics aggregation loop 2024-06-21 12:38:44,245 DEBUG SystemMonitor:26947 [system_monitor.py:_start():141] Publishing last batch of metrics 2024-06-21 12:38:44,247 INFO HandlerThread:26947 [interfaces.py:finish():173] Joined cpu 2024-06-21 12:38:44,247 INFO HandlerThread:26947 [interfaces.py:finish():173] Joined disk 2024-06-21 12:38:44,299 INFO HandlerThread:26947 [interfaces.py:finish():173] Joined gpu 2024-06-21 12:38:44,299 INFO HandlerThread:26947 [interfaces.py:finish():173] Joined memory 2024-06-21 12:38:44,299 INFO HandlerThread:26947 [interfaces.py:finish():173] Joined network 2024-06-21 12:38:44,300 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: defer 2024-06-21 12:38:44,300 INFO SenderThread:26947 [tracking_sender.py:send_request_defer():455] handle sender defer: 1 2024-06-21 12:38:44,300 INFO SenderThread:26947 [tracking_sender.py:transition_state():459] send defer: 2 2024-06-21 12:38:44,300 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: stats 2024-06-21 12:38:44,301 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: defer 2024-06-21 12:38:44,302 INFO HandlerThread:26947 [handler.py:handle_request_defer():163] handle defer: 2 2024-06-21 12:38:44,302 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: defer 2024-06-21 12:38:44,302 INFO SenderThread:26947 [tracking_sender.py:send_request_defer():455] handle sender defer: 2 2024-06-21 12:38:44,302 INFO SenderThread:26947 [tracking_sender.py:transition_state():459] send defer: 3 2024-06-21 12:38:44,302 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: defer 2024-06-21 12:38:44,302 INFO HandlerThread:26947 [handler.py:handle_request_defer():163] handle defer: 3 2024-06-21 12:38:44,302 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: defer 2024-06-21 12:38:44,302 INFO SenderThread:26947 [tracking_sender.py:send_request_defer():455] handle sender defer: 3 2024-06-21 12:38:44,302 INFO SenderThread:26947 [tracking_sender.py:transition_state():459] send defer: 4 2024-06-21 12:38:44,303 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: defer 2024-06-21 12:38:44,303 INFO HandlerThread:26947 [handler.py:handle_request_defer():163] handle defer: 4 2024-06-21 12:38:44,303 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: summary 2024-06-21 12:38:44,304 INFO SenderThread:26947 [tracking_sender.py:_save_file():1177] saving file wandb-summary.json with policy end 2024-06-21 12:38:44,304 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: defer 2024-06-21 12:38:44,304 INFO SenderThread:26947 [tracking_sender.py:send_request_defer():455] handle sender defer: 4 2024-06-21 12:38:44,304 INFO SenderThread:26947 [tracking_sender.py:transition_state():459] send defer: 5 2024-06-21 12:38:44,304 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: defer 2024-06-21 12:38:44,304 INFO HandlerThread:26947 [handler.py:handle_request_defer():163] handle defer: 5 2024-06-21 12:38:44,305 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: defer 2024-06-21 12:38:44,305 INFO SenderThread:26947 [tracking_sender.py:send_request_defer():455] handle sender defer: 5 2024-06-21 12:38:44,305 INFO SenderThread:26947 [tracking_sender.py:transition_state():459] send defer: 6 2024-06-21 12:38:44,305 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: defer 2024-06-21 12:38:44,305 INFO HandlerThread:26947 [handler.py:handle_request_defer():163] handle defer: 6 2024-06-21 12:38:44,305 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: defer 2024-06-21 12:38:44,305 INFO SenderThread:26947 [tracking_sender.py:send_request_defer():455] handle sender defer: 6 2024-06-21 12:38:44,419 INFO Thread-15 :26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:38:45,239 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: poll_exit 2024-06-21 12:38:45,573 INFO SenderThread:26947 [tracking_sender.py:transition_state():459] send defer: 7 2024-06-21 12:38:45,573 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: poll_exit 2024-06-21 12:38:45,574 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: defer 2024-06-21 12:38:45,574 INFO HandlerThread:26947 [handler.py:handle_request_defer():163] handle defer: 7 2024-06-21 12:38:45,574 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: defer 2024-06-21 12:38:45,574 INFO SenderThread:26947 [tracking_sender.py:send_request_defer():455] handle sender defer: 7 2024-06-21 12:38:45,574 INFO SenderThread:26947 [dir_watcher.py:finish():362] shutting down directory watcher 2024-06-21 12:38:46,240 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: poll_exit 2024-06-21 12:38:46,420 INFO SenderThread:26947 [dir_watcher.py:_on_file_modified():292] file/dir modified: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:38:46,420 INFO SenderThread:26947 [dir_watcher.py:finish():392] scan: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files 2024-06-21 12:38:46,420 INFO SenderThread:26947 [dir_watcher.py:finish():406] scan save: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/config.yaml config.yaml 2024-06-21 12:38:46,421 INFO SenderThread:26947 [dir_watcher.py:finish():406] scan save: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json wandb-summary.json 2024-06-21 12:38:46,421 INFO SenderThread:26947 [dir_watcher.py:finish():406] scan save: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/requirements.txt requirements.txt 2024-06-21 12:38:46,421 INFO SenderThread:26947 [dir_watcher.py:finish():406] scan save: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-metadata.json wandb-metadata.json 2024-06-21 12:38:46,422 INFO SenderThread:26947 [dir_watcher.py:finish():406] scan save: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log output.log 2024-06-21 12:38:46,422 INFO SenderThread:26947 [tracking_sender.py:transition_state():459] send defer: 8 2024-06-21 12:38:46,424 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: poll_exit 2024-06-21 12:38:46,425 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: defer 2024-06-21 12:38:46,425 INFO HandlerThread:26947 [handler.py:handle_request_defer():163] handle defer: 8 2024-06-21 12:38:46,426 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: defer 2024-06-21 12:38:46,426 INFO SenderThread:26947 [tracking_sender.py:send_request_defer():455] handle sender defer: 8 2024-06-21 12:38:46,426 INFO SenderThread:26947 [file_pusher.py:finish():168] shutting down file pusher 2024-06-21 12:38:46,487 INFO Thread-19 :26947 [upload_job.py:push():143] Uploaded file /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/wandb-summary.json 2024-06-21 12:38:46,489 INFO Thread-20 :26947 [upload_job.py:push():143] Uploaded file /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/requirements.txt 2024-06-21 12:38:46,507 INFO Thread-18 :26947 [upload_job.py:push():143] Uploaded file /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/config.yaml 2024-06-21 12:38:46,627 INFO Thread-21 :26947 [upload_job.py:push():143] Uploaded file /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/files/output.log 2024-06-21 12:38:46,827 INFO Thread-14 :26947 [tracking_sender.py:transition_state():459] send defer: 9 2024-06-21 12:38:46,828 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: defer 2024-06-21 12:38:46,828 INFO HandlerThread:26947 [handler.py:handle_request_defer():163] handle defer: 9 2024-06-21 12:38:46,828 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: defer 2024-06-21 12:38:46,828 INFO SenderThread:26947 [tracking_sender.py:send_request_defer():455] handle sender defer: 9 2024-06-21 12:38:46,828 INFO SenderThread:26947 [file_pusher.py:join():173] waiting for file pusher 2024-06-21 12:38:46,828 INFO SenderThread:26947 [tracking_sender.py:transition_state():459] send defer: 10 2024-06-21 12:38:46,829 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: defer 2024-06-21 12:38:46,829 INFO HandlerThread:26947 [handler.py:handle_request_defer():163] handle defer: 10 2024-06-21 12:38:46,829 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: defer 2024-06-21 12:38:46,829 INFO SenderThread:26947 [tracking_sender.py:send_request_defer():455] handle sender defer: 10 2024-06-21 12:38:46,974 INFO SenderThread:26947 [tracking_sender.py:transition_state():459] send defer: 11 2024-06-21 12:38:46,975 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: defer 2024-06-21 12:38:46,975 INFO HandlerThread:26947 [handler.py:handle_request_defer():163] handle defer: 11 2024-06-21 12:38:46,975 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: defer 2024-06-21 12:38:46,975 INFO SenderThread:26947 [tracking_sender.py:send_request_defer():455] handle sender defer: 11 2024-06-21 12:38:46,975 INFO SenderThread:26947 [tracking_sender.py:transition_state():459] send defer: 12 2024-06-21 12:38:46,976 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: final 2024-06-21 12:38:46,976 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: defer 2024-06-21 12:38:46,976 DEBUG SenderThread:26947 [tracking_sender.py:send():315] send: footer 2024-06-21 12:38:46,976 INFO HandlerThread:26947 [handler.py:handle_request_defer():163] handle defer: 12 2024-06-21 12:38:46,976 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: defer 2024-06-21 12:38:46,976 INFO SenderThread:26947 [tracking_sender.py:send_request_defer():455] handle sender defer: 12 2024-06-21 12:38:46,977 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: poll_exit 2024-06-21 12:38:46,977 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: poll_exit 2024-06-21 12:38:46,978 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: poll_exit 2024-06-21 12:38:46,978 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: poll_exit 2024-06-21 12:38:46,978 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: server_info 2024-06-21 12:38:46,979 DEBUG SenderThread:26947 [tracking_sender.py:send_request():329] send_request: server_info 2024-06-21 12:38:46,979 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: get_summary 2024-06-21 12:38:46,980 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: sampled_history 2024-06-21 12:38:46,981 INFO MainThread:26947 [wandb_run.py:_footer_history_summary_info():3391] rendering history 2024-06-21 12:38:46,982 INFO MainThread:26947 [wandb_run.py:_footer_history_summary_info():3423] rendering summary 2024-06-21 12:38:46,982 INFO MainThread:26947 [wandb_run.py:_footer_sync_info():3349] logging synced files 2024-06-21 12:38:46,983 DEBUG HandlerThread:26947 [handler.py:handle_request():140] handle_request: shutdown 2024-06-21 12:38:46,983 INFO HandlerThread:26947 [handler.py:finish():817] shutting down handler 2024-06-21 12:38:47,976 INFO WriterThread:26947 [datastore.py:close():283] close: /opt/tiger/auto-info/wandb/run-20240621_111859-run_20240621_31f46a8f/run-run_20240621_31f46a8f.wandb 2024-06-21 12:38:47,979 INFO SenderThread:26947 [tracking_sender.py:finish():1338] shutting down sender 2024-06-21 12:38:47,979 INFO SenderThread:26947 [file_pusher.py:finish():168] shutting down file pusher 2024-06-21 12:38:47,979 INFO SenderThread:26947 [file_pusher.py:join():173] waiting for file pusher 2024-06-21 12:38:47,985 INFO MainThread:26947 [internal.py:handle_exit():78] Internal process exited