Spaces:
Runtime error
Runtime error
| 2023-04-21 16:52:08,224 INFO StreamThr :24428 [internal.py:wandb_internal():86] W&B internal server running at pid: 24428, started at: 2023-04-21 16:52:08.224660 | |
| 2023-04-21 16:52:08,226 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: status | |
| 2023-04-21 16:52:08,228 INFO WriterThread:24428 [datastore.py:open_for_write():85] open: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230421_165208-onaovmt4\run-onaovmt4.wandb | |
| 2023-04-21 16:52:08,232 DEBUG SenderThread:24428 [sender.py:send():375] send: header | |
| 2023-04-21 16:52:08,320 DEBUG SenderThread:24428 [sender.py:send():375] send: run | |
| 2023-04-21 16:52:09,776 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: check_version | |
| 2023-04-21 16:52:09,776 INFO SenderThread:24428 [dir_watcher.py:__init__():219] watching files in: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230421_165208-onaovmt4\files | |
| 2023-04-21 16:52:09,777 INFO SenderThread:24428 [sender.py:_start_run_threads():1124] run started: onaovmt4 with start time 1682067128.226776 | |
| 2023-04-21 16:52:09,777 DEBUG SenderThread:24428 [sender.py:send_request():402] send_request: summary_record | |
| 2023-04-21 16:52:09,778 INFO SenderThread:24428 [sender.py:_save_file():1378] saving file wandb-summary.json with policy end | |
| 2023-04-21 16:52:09,778 DEBUG SenderThread:24428 [sender.py:send_request():402] send_request: check_version | |
| 2023-04-21 16:52:10,793 INFO Thread-16 :24428 [dir_watcher.py:_on_file_created():278] file/dir created: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230421_165208-onaovmt4\files\wandb-summary.json | |
| 2023-04-21 16:52:10,858 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: run_start | |
| 2023-04-21 16:52:10,916 DEBUG HandlerThread:24428 [system_info.py:__init__():31] System info init | |
| 2023-04-21 16:52:10,916 DEBUG HandlerThread:24428 [system_info.py:__init__():46] System info init done | |
| 2023-04-21 16:52:10,916 INFO HandlerThread:24428 [system_monitor.py:start():181] Starting system monitor | |
| 2023-04-21 16:52:10,916 INFO SystemMonitor:24428 [system_monitor.py:_start():145] Starting system asset monitoring threads | |
| 2023-04-21 16:52:10,916 INFO HandlerThread:24428 [system_monitor.py:probe():201] Collecting system info | |
| 2023-04-21 16:52:10,924 INFO SystemMonitor:24428 [interfaces.py:start():190] Started cpu monitoring | |
| 2023-04-21 16:52:10,925 INFO SystemMonitor:24428 [interfaces.py:start():190] Started disk monitoring | |
| 2023-04-21 16:52:10,926 INFO SystemMonitor:24428 [interfaces.py:start():190] Started gpu monitoring | |
| 2023-04-21 16:52:10,939 INFO SystemMonitor:24428 [interfaces.py:start():190] Started memory monitoring | |
| 2023-04-21 16:52:10,957 INFO SystemMonitor:24428 [interfaces.py:start():190] Started network monitoring | |
| 2023-04-21 16:52:10,981 ERROR gpu :24428 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-21 16:52:10,989 DEBUG HandlerThread:24428 [system_info.py:probe():195] Probing system | |
| 2023-04-21 16:52:10,992 DEBUG HandlerThread:24428 [system_info.py:_probe_git():180] Probing git | |
| 2023-04-21 16:52:11,095 DEBUG HandlerThread:24428 [system_info.py:_probe_git():188] Probing git done | |
| 2023-04-21 16:52:11,095 DEBUG HandlerThread:24428 [system_info.py:probe():240] Probing system done | |
| 2023-04-21 16:52:11,095 DEBUG HandlerThread:24428 [system_monitor.py:probe():210] {'os': 'Windows-10-10.0.19041-SP0', 'python': '3.8.10', 'heartbeatAt': '2023-04-21T08:52:10.989425', 'startedAt': '2023-04-21T08:52:08.204662', 'docker': None, 'cuda': None, 'args': ('--do_train', '--train_file', '.\\datasets\\chat\\train.json', '--validation_file', '.\\datasets\\chat\\dev.json', '--prompt_column', 'content', '--response_column', 'summary', '--overwrite_cache', '--model_name_or_path', '..\\models\\chatglm-6b-int4', '--output_dir', 'output\\adgen-chatglm-6b-pt-128-2e-2', '--overwrite_output_dir', '--max_source_length', '64', '--max_target_length', '64', '--per_device_train_batch_size', '1', '--per_device_eval_batch_size', '1', '--gradient_accumulation_steps', '16', '--predict_with_generate', '--max_steps', '1000', '--logging_steps', '10', '--save_steps', '100', '--learning_rate', '2e-2', '--pre_seq_len', '128', '--quantization_bit', '4'), 'state': 'running', 'program': 'main.py', 'codePath': 'ptuning\\main.py', 'git': {'remote': 'https://github.com/THUDM/ChatGLM-6B', 'commit': '01e6313abf4122d789d6e68128856af52847b355'}, 'cpu_count': 6, 'cpu_count_logical': 12, 'cpu_freq': {'current': 2592.0, 'min': 0.0, 'max': 2592.0}, 'cpu_freq_per_core': [{'current': 2592.0, 'min': 0.0, 'max': 2592.0}], 'disk': {'total': 500.32030868530273, 'used': 236.01997756958008}, 'gpu': 'NVIDIA GeForce RTX 2060', 'gpu_count': 1, 'gpu_devices': [{'name': 'NVIDIA GeForce RTX 2060', 'memory_total': 6442450944}], 'memory': {'total': 63.87089538574219}} | |
| 2023-04-21 16:52:11,095 INFO HandlerThread:24428 [system_monitor.py:probe():211] Finished collecting system info | |
| 2023-04-21 16:52:11,095 INFO HandlerThread:24428 [system_monitor.py:probe():214] Publishing system info | |
| 2023-04-21 16:52:11,095 DEBUG HandlerThread:24428 [system_info.py:_save_pip():51] Saving list of pip packages installed into the current environment | |
| 2023-04-21 16:52:11,096 DEBUG HandlerThread:24428 [system_info.py:_save_pip():67] Saving pip packages done | |
| 2023-04-21 16:52:11,099 INFO HandlerThread:24428 [system_monitor.py:probe():216] Finished publishing system info | |
| 2023-04-21 16:52:11,143 DEBUG SenderThread:24428 [sender.py:send():375] send: files | |
| 2023-04-21 16:52:11,143 INFO SenderThread:24428 [sender.py:_save_file():1378] saving file wandb-metadata.json with policy now | |
| 2023-04-21 16:52:11,156 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: stop_status | |
| 2023-04-21 16:52:11,157 DEBUG SenderThread:24428 [sender.py:send_request():402] send_request: stop_status | |
| 2023-04-21 16:52:11,577 DEBUG SenderThread:24428 [sender.py:send():375] send: telemetry | |
| 2023-04-21 16:52:11,577 DEBUG SenderThread:24428 [sender.py:send():375] send: config | |
| 2023-04-21 16:52:11,578 DEBUG SenderThread:24428 [sender.py:send():375] send: metric | |
| 2023-04-21 16:52:11,578 DEBUG SenderThread:24428 [sender.py:send():375] send: telemetry | |
| 2023-04-21 16:52:11,578 DEBUG SenderThread:24428 [sender.py:send():375] send: metric | |
| 2023-04-21 16:52:11,579 WARNING SenderThread:24428 [sender.py:send_metric():1329] Seen metric with glob (shouldn't happen) | |
| 2023-04-21 16:52:11,800 INFO Thread-16 :24428 [dir_watcher.py:_on_file_created():278] file/dir created: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230421_165208-onaovmt4\files\output.log | |
| 2023-04-21 16:52:11,800 INFO Thread-16 :24428 [dir_watcher.py:_on_file_created():278] file/dir created: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230421_165208-onaovmt4\files\wandb-metadata.json | |
| 2023-04-21 16:52:11,800 INFO Thread-16 :24428 [dir_watcher.py:_on_file_created():278] file/dir created: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230421_165208-onaovmt4\files\requirements.txt | |
| 2023-04-21 16:52:12,178 INFO wandb-upload_0:24428 [upload_job.py:push():137] Uploaded file C:\Users\Lenovo\AppData\Local\Temp\tmpedo5nrgdwandb\1xtewqbc-wandb-metadata.json | |
| 2023-04-21 16:52:13,036 ERROR gpu :24428 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-21 16:52:13,600 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: status_report | |
| 2023-04-21 16:52:13,818 INFO Thread-16 :24428 [dir_watcher.py:_on_file_modified():295] file/dir modified: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230421_165208-onaovmt4\files\output.log | |
| 2023-04-21 16:52:15,078 ERROR gpu :24428 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-21 16:52:15,829 INFO Thread-16 :24428 [dir_watcher.py:_on_file_modified():295] file/dir modified: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230421_165208-onaovmt4\files\output.log | |
| 2023-04-21 16:52:17,130 ERROR gpu :24428 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-21 16:52:18,669 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: status_report | |
| 2023-04-21 16:52:19,182 ERROR gpu :24428 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-21 16:52:21,226 ERROR gpu :24428 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-21 16:52:23,265 ERROR gpu :24428 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-21 16:52:23,721 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: status_report | |
| 2023-04-21 16:52:25,323 ERROR gpu :24428 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-21 16:52:26,169 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: stop_status | |
| 2023-04-21 16:52:26,169 DEBUG SenderThread:24428 [sender.py:send_request():402] send_request: stop_status | |
| 2023-04-21 16:52:27,371 ERROR gpu :24428 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-21 16:52:28,978 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: status_report | |
| 2023-04-21 16:52:29,406 ERROR gpu :24428 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-21 16:52:29,958 INFO Thread-16 :24428 [dir_watcher.py:_on_file_modified():295] file/dir modified: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230421_165208-onaovmt4\files\output.log | |
| 2023-04-21 16:52:31,451 ERROR gpu :24428 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-21 16:52:33,485 ERROR gpu :24428 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-21 16:52:34,631 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: status_report | |
| 2023-04-21 16:52:35,541 ERROR gpu :24428 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-21 16:52:36,010 INFO Thread-16 :24428 [dir_watcher.py:_on_file_modified():295] file/dir modified: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230421_165208-onaovmt4\files\output.log | |
| 2023-04-21 16:52:37,580 ERROR gpu :24428 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-21 16:52:39,617 ERROR gpu :24428 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-21 16:52:39,647 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: status_report | |
| 2023-04-21 16:52:41,072 INFO Thread-16 :24428 [dir_watcher.py:_on_file_modified():295] file/dir modified: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230421_165208-onaovmt4\files\config.yaml | |
| 2023-04-21 16:52:41,179 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: stop_status | |
| 2023-04-21 16:52:41,180 DEBUG SenderThread:24428 [sender.py:send_request():402] send_request: stop_status | |
| 2023-04-21 16:52:41,735 ERROR gpu :24428 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-21 16:52:43,767 ERROR gpu :24428 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-21 16:52:44,091 INFO Thread-16 :24428 [dir_watcher.py:_on_file_modified():295] file/dir modified: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230421_165208-onaovmt4\files\output.log | |
| 2023-04-21 16:52:45,799 ERROR gpu :24428 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-21 16:52:46,152 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: status_report | |
| 2023-04-21 16:52:47,856 ERROR gpu :24428 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-21 16:52:49,876 ERROR gpu :24428 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-21 16:52:50,188 INFO Thread-16 :24428 [dir_watcher.py:_on_file_modified():295] file/dir modified: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230421_165208-onaovmt4\files\output.log | |
| 2023-04-21 16:52:51,402 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: status_report | |
| 2023-04-21 16:52:51,892 ERROR gpu :24428 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-21 16:52:53,916 ERROR gpu :24428 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-21 16:52:54,266 INFO Thread-16 :24428 [dir_watcher.py:_on_file_modified():295] file/dir modified: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230421_165208-onaovmt4\files\output.log | |
| 2023-04-21 16:52:55,929 ERROR gpu :24428 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-21 16:52:56,195 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: stop_status | |
| 2023-04-21 16:52:56,196 DEBUG SenderThread:24428 [sender.py:send_request():402] send_request: stop_status | |
| 2023-04-21 16:52:56,454 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: status_report | |
| 2023-04-21 16:52:57,959 ERROR gpu :24428 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-21 16:52:59,979 ERROR gpu :24428 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-21 16:53:00,438 INFO Thread-16 :24428 [dir_watcher.py:_on_file_modified():295] file/dir modified: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230421_165208-onaovmt4\files\output.log | |
| 2023-04-21 16:53:01,207 DEBUG SenderThread:24428 [sender.py:send():375] send: exit | |
| 2023-04-21 16:53:01,207 INFO SenderThread:24428 [sender.py:send_exit():598] handling exit code: 255 | |
| 2023-04-21 16:53:01,207 INFO SenderThread:24428 [sender.py:send_exit():600] handling runtime: 50 | |
| 2023-04-21 16:53:01,207 INFO SenderThread:24428 [sender.py:_save_file():1378] saving file wandb-summary.json with policy end | |
| 2023-04-21 16:53:01,208 INFO SenderThread:24428 [sender.py:send_exit():606] send defer | |
| 2023-04-21 16:53:01,208 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: defer | |
| 2023-04-21 16:53:01,208 INFO HandlerThread:24428 [handler.py:handle_request_defer():170] handle defer: 0 | |
| 2023-04-21 16:53:01,208 DEBUG SenderThread:24428 [sender.py:send_request():402] send_request: defer | |
| 2023-04-21 16:53:01,208 INFO SenderThread:24428 [sender.py:send_request_defer():622] handle sender defer: 0 | |
| 2023-04-21 16:53:01,209 INFO SenderThread:24428 [sender.py:transition_state():626] send defer: 1 | |
| 2023-04-21 16:53:01,209 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: defer | |
| 2023-04-21 16:53:01,209 INFO HandlerThread:24428 [handler.py:handle_request_defer():170] handle defer: 1 | |
| 2023-04-21 16:53:01,209 DEBUG SenderThread:24428 [sender.py:send_request():402] send_request: defer | |
| 2023-04-21 16:53:01,209 INFO SenderThread:24428 [sender.py:send_request_defer():622] handle sender defer: 1 | |
| 2023-04-21 16:53:01,209 INFO SenderThread:24428 [sender.py:transition_state():626] send defer: 2 | |
| 2023-04-21 16:53:01,210 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: defer | |
| 2023-04-21 16:53:01,210 INFO HandlerThread:24428 [handler.py:handle_request_defer():170] handle defer: 2 | |
| 2023-04-21 16:53:01,210 INFO HandlerThread:24428 [system_monitor.py:finish():190] Stopping system monitor | |
| 2023-04-21 16:53:01,210 INFO HandlerThread:24428 [interfaces.py:finish():202] Joined cpu monitor | |
| 2023-04-21 16:53:01,210 DEBUG SystemMonitor:24428 [system_monitor.py:_start():159] Starting system metrics aggregation loop | |
| 2023-04-21 16:53:01,225 INFO HandlerThread:24428 [interfaces.py:finish():202] Joined disk monitor | |
| 2023-04-21 16:53:01,225 DEBUG SystemMonitor:24428 [system_monitor.py:_start():166] Finished system metrics aggregation loop | |
| 2023-04-21 16:53:01,226 DEBUG SystemMonitor:24428 [system_monitor.py:_start():170] Publishing last batch of metrics | |
| 2023-04-21 16:53:01,274 INFO HandlerThread:24428 [interfaces.py:finish():202] Joined gpu monitor | |
| 2023-04-21 16:53:01,274 INFO HandlerThread:24428 [interfaces.py:finish():202] Joined memory monitor | |
| 2023-04-21 16:53:01,275 INFO HandlerThread:24428 [interfaces.py:finish():202] Joined network monitor | |
| 2023-04-21 16:53:01,276 DEBUG SenderThread:24428 [sender.py:send_request():402] send_request: defer | |
| 2023-04-21 16:53:01,277 INFO SenderThread:24428 [sender.py:send_request_defer():622] handle sender defer: 2 | |
| 2023-04-21 16:53:01,277 INFO SenderThread:24428 [sender.py:transition_state():626] send defer: 3 | |
| 2023-04-21 16:53:01,277 DEBUG SenderThread:24428 [sender.py:send():375] send: stats | |
| 2023-04-21 16:53:01,277 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: defer | |
| 2023-04-21 16:53:01,278 INFO HandlerThread:24428 [handler.py:handle_request_defer():170] handle defer: 3 | |
| 2023-04-21 16:53:01,278 DEBUG SenderThread:24428 [sender.py:send_request():402] send_request: defer | |
| 2023-04-21 16:53:01,278 INFO SenderThread:24428 [sender.py:send_request_defer():622] handle sender defer: 3 | |
| 2023-04-21 16:53:01,278 INFO SenderThread:24428 [sender.py:transition_state():626] send defer: 4 | |
| 2023-04-21 16:53:01,278 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: defer | |
| 2023-04-21 16:53:01,278 INFO HandlerThread:24428 [handler.py:handle_request_defer():170] handle defer: 4 | |
| 2023-04-21 16:53:01,279 DEBUG SenderThread:24428 [sender.py:send_request():402] send_request: defer | |
| 2023-04-21 16:53:01,279 INFO SenderThread:24428 [sender.py:send_request_defer():622] handle sender defer: 4 | |
| 2023-04-21 16:53:01,279 INFO SenderThread:24428 [sender.py:transition_state():626] send defer: 5 | |
| 2023-04-21 16:53:01,279 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: defer | |
| 2023-04-21 16:53:01,279 INFO HandlerThread:24428 [handler.py:handle_request_defer():170] handle defer: 5 | |
| 2023-04-21 16:53:01,279 DEBUG SenderThread:24428 [sender.py:send():375] send: summary | |
| 2023-04-21 16:53:01,280 INFO SenderThread:24428 [sender.py:_save_file():1378] saving file wandb-summary.json with policy end | |
| 2023-04-21 16:53:01,280 DEBUG SenderThread:24428 [sender.py:send_request():402] send_request: defer | |
| 2023-04-21 16:53:01,280 INFO SenderThread:24428 [sender.py:send_request_defer():622] handle sender defer: 5 | |
| 2023-04-21 16:53:01,280 INFO SenderThread:24428 [sender.py:transition_state():626] send defer: 6 | |
| 2023-04-21 16:53:01,280 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: defer | |
| 2023-04-21 16:53:01,281 INFO HandlerThread:24428 [handler.py:handle_request_defer():170] handle defer: 6 | |
| 2023-04-21 16:53:01,281 DEBUG SenderThread:24428 [sender.py:send_request():402] send_request: defer | |
| 2023-04-21 16:53:01,281 INFO SenderThread:24428 [sender.py:send_request_defer():622] handle sender defer: 6 | |
| 2023-04-21 16:53:01,281 INFO SenderThread:24428 [sender.py:transition_state():626] send defer: 7 | |
| 2023-04-21 16:53:01,281 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: status_report | |
| 2023-04-21 16:53:01,281 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: defer | |
| 2023-04-21 16:53:01,281 INFO HandlerThread:24428 [handler.py:handle_request_defer():170] handle defer: 7 | |
| 2023-04-21 16:53:01,282 DEBUG SenderThread:24428 [sender.py:send_request():402] send_request: defer | |
| 2023-04-21 16:53:01,282 INFO SenderThread:24428 [sender.py:send_request_defer():622] handle sender defer: 7 | |
| 2023-04-21 16:53:01,442 INFO Thread-16 :24428 [dir_watcher.py:_on_file_modified():295] file/dir modified: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230421_165208-onaovmt4\files\wandb-summary.json | |
| 2023-04-21 16:53:02,283 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: poll_exit | |
| 2023-04-21 16:53:02,395 INFO SenderThread:24428 [sender.py:transition_state():626] send defer: 8 | |
| 2023-04-21 16:53:02,395 DEBUG SenderThread:24428 [sender.py:send_request():402] send_request: poll_exit | |
| 2023-04-21 16:53:02,395 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: defer | |
| 2023-04-21 16:53:02,395 INFO HandlerThread:24428 [handler.py:handle_request_defer():170] handle defer: 8 | |
| 2023-04-21 16:53:02,396 DEBUG SenderThread:24428 [sender.py:send_request():402] send_request: defer | |
| 2023-04-21 16:53:02,396 INFO SenderThread:24428 [sender.py:send_request_defer():622] handle sender defer: 8 | |
| 2023-04-21 16:53:02,429 INFO SenderThread:24428 [sender.py:transition_state():626] send defer: 9 | |
| 2023-04-21 16:53:02,429 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: defer | |
| 2023-04-21 16:53:02,429 INFO HandlerThread:24428 [handler.py:handle_request_defer():170] handle defer: 9 | |
| 2023-04-21 16:53:02,429 DEBUG SenderThread:24428 [sender.py:send():375] send: artifact | |
| 2023-04-21 16:53:02,456 INFO Thread-16 :24428 [dir_watcher.py:_on_file_modified():295] file/dir modified: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230421_165208-onaovmt4\files\output.log | |
| 2023-04-21 16:53:03,292 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: poll_exit | |
| 2023-04-21 16:53:04,709 INFO wandb-upload_0:24428 [upload_job.py:push():95] Uploaded file C:\Users\Lenovo\AppData\Local\wandb\wandb\artifacts\staging\tmppu32xnxj | |
| 2023-04-21 16:53:04,789 INFO wandb-upload_1:24428 [upload_job.py:push():95] Uploaded file C:\Users\Lenovo\AppData\Local\wandb\wandb\artifacts\staging\tmpztqfj66q | |
| 2023-04-21 16:53:06,544 INFO SenderThread:24428 [sender.py:send_artifact():1474] sent artifact job-https___github.com_THUDM_ChatGLM-6B_ptuning_main.py - {'id': 'QXJ0aWZhY3Q6NDI5NzAxNTEw', 'digest': '1077e319ad39e537c1ccc8f9a5c233bc', 'state': 'PENDING', 'aliases': [], 'artifactSequence': {'id': 'QXJ0aWZhY3RDb2xsZWN0aW9uOjYzNDcyMzc3', 'latestArtifact': {'id': 'QXJ0aWZhY3Q6NDI5NjczOTYx', 'versionIndex': 3}}, 'version': 'latest'} | |
| 2023-04-21 16:53:06,544 DEBUG SenderThread:24428 [sender.py:send_request():402] send_request: defer | |
| 2023-04-21 16:53:06,544 INFO SenderThread:24428 [sender.py:send_request_defer():622] handle sender defer: 9 | |
| 2023-04-21 16:53:06,544 INFO SenderThread:24428 [dir_watcher.py:finish():365] shutting down directory watcher | |
| 2023-04-21 16:53:07,513 INFO SenderThread:24428 [dir_watcher.py:finish():395] scan: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230421_165208-onaovmt4\files | |
| 2023-04-21 16:53:07,514 INFO SenderThread:24428 [dir_watcher.py:finish():409] scan save: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230421_165208-onaovmt4\files\config.yaml config.yaml | |
| 2023-04-21 16:53:07,514 INFO SenderThread:24428 [dir_watcher.py:finish():409] scan save: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230421_165208-onaovmt4\files\output.log output.log | |
| 2023-04-21 16:53:07,517 INFO SenderThread:24428 [dir_watcher.py:finish():409] scan save: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230421_165208-onaovmt4\files\requirements.txt requirements.txt | |
| 2023-04-21 16:53:07,519 INFO SenderThread:24428 [dir_watcher.py:finish():409] scan save: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230421_165208-onaovmt4\files\wandb-metadata.json wandb-metadata.json | |
| 2023-04-21 16:53:07,519 INFO SenderThread:24428 [dir_watcher.py:finish():409] scan save: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230421_165208-onaovmt4\files\wandb-summary.json wandb-summary.json | |
| 2023-04-21 16:53:07,522 INFO SenderThread:24428 [sender.py:transition_state():626] send defer: 10 | |
| 2023-04-21 16:53:07,522 DEBUG SenderThread:24428 [sender.py:send_request():402] send_request: poll_exit | |
| 2023-04-21 16:53:07,522 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: defer | |
| 2023-04-21 16:53:07,522 INFO HandlerThread:24428 [handler.py:handle_request_defer():170] handle defer: 10 | |
| 2023-04-21 16:53:07,523 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: status_report | |
| 2023-04-21 16:53:07,525 DEBUG SenderThread:24428 [sender.py:send_request():402] send_request: defer | |
| 2023-04-21 16:53:07,525 INFO SenderThread:24428 [sender.py:send_request_defer():622] handle sender defer: 10 | |
| 2023-04-21 16:53:07,525 INFO SenderThread:24428 [file_pusher.py:finish():168] shutting down file pusher | |
| 2023-04-21 16:53:08,278 INFO wandb-upload_0:24428 [upload_job.py:push():137] Uploaded file E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230421_165208-onaovmt4\files\config.yaml | |
| 2023-04-21 16:53:08,332 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: keepalive | |
| 2023-04-21 16:53:08,566 INFO wandb-upload_1:24428 [upload_job.py:push():137] Uploaded file E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230421_165208-onaovmt4\files\output.log | |
| 2023-04-21 16:53:08,756 INFO wandb-upload_3:24428 [upload_job.py:push():137] Uploaded file E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230421_165208-onaovmt4\files\wandb-summary.json | |
| 2023-04-21 16:53:08,813 INFO wandb-upload_2:24428 [upload_job.py:push():137] Uploaded file E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230421_165208-onaovmt4\files\requirements.txt | |
| 2023-04-21 16:53:09,028 INFO Thread-15 :24428 [sender.py:transition_state():626] send defer: 11 | |
| 2023-04-21 16:53:09,028 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: defer | |
| 2023-04-21 16:53:09,028 INFO HandlerThread:24428 [handler.py:handle_request_defer():170] handle defer: 11 | |
| 2023-04-21 16:53:09,028 DEBUG SenderThread:24428 [sender.py:send_request():402] send_request: defer | |
| 2023-04-21 16:53:09,029 INFO SenderThread:24428 [sender.py:send_request_defer():622] handle sender defer: 11 | |
| 2023-04-21 16:53:09,029 INFO SenderThread:24428 [file_pusher.py:join():173] waiting for file pusher | |
| 2023-04-21 16:53:09,029 INFO SenderThread:24428 [sender.py:transition_state():626] send defer: 12 | |
| 2023-04-21 16:53:09,029 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: defer | |
| 2023-04-21 16:53:09,029 INFO HandlerThread:24428 [handler.py:handle_request_defer():170] handle defer: 12 | |
| 2023-04-21 16:53:09,029 DEBUG SenderThread:24428 [sender.py:send_request():402] send_request: defer | |
| 2023-04-21 16:53:09,029 INFO SenderThread:24428 [sender.py:send_request_defer():622] handle sender defer: 12 | |
| 2023-04-21 16:53:09,905 INFO SenderThread:24428 [sender.py:transition_state():626] send defer: 13 | |
| 2023-04-21 16:53:09,905 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: defer | |
| 2023-04-21 16:53:09,905 INFO HandlerThread:24428 [handler.py:handle_request_defer():170] handle defer: 13 | |
| 2023-04-21 16:53:09,905 DEBUG SenderThread:24428 [sender.py:send_request():402] send_request: defer | |
| 2023-04-21 16:53:09,905 INFO SenderThread:24428 [sender.py:send_request_defer():622] handle sender defer: 13 | |
| 2023-04-21 16:53:09,905 INFO SenderThread:24428 [sender.py:transition_state():626] send defer: 14 | |
| 2023-04-21 16:53:09,906 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: defer | |
| 2023-04-21 16:53:09,906 INFO HandlerThread:24428 [handler.py:handle_request_defer():170] handle defer: 14 | |
| 2023-04-21 16:53:09,906 DEBUG SenderThread:24428 [sender.py:send():375] send: final | |
| 2023-04-21 16:53:09,906 DEBUG SenderThread:24428 [sender.py:send():375] send: footer | |
| 2023-04-21 16:53:09,906 DEBUG SenderThread:24428 [sender.py:send_request():402] send_request: defer | |
| 2023-04-21 16:53:09,906 INFO SenderThread:24428 [sender.py:send_request_defer():622] handle sender defer: 14 | |
| 2023-04-21 16:53:09,907 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: poll_exit | |
| 2023-04-21 16:53:09,907 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: server_info | |
| 2023-04-21 16:53:09,907 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: get_summary | |
| 2023-04-21 16:53:09,907 DEBUG SenderThread:24428 [sender.py:send_request():402] send_request: poll_exit | |
| 2023-04-21 16:53:09,907 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: sampled_history | |
| 2023-04-21 16:53:09,908 DEBUG SenderThread:24428 [sender.py:send_request():402] send_request: server_info | |
| 2023-04-21 16:53:10,163 INFO MainThread:24428 [wandb_run.py:_footer_history_summary_info():3476] rendering history | |
| 2023-04-21 16:53:10,163 INFO MainThread:24428 [wandb_run.py:_footer_history_summary_info():3508] rendering summary | |
| 2023-04-21 16:53:10,163 INFO MainThread:24428 [wandb_run.py:_footer_sync_info():3434] logging synced files | |
| 2023-04-21 16:53:10,164 DEBUG HandlerThread:24428 [handler.py:handle_request():144] handle_request: shutdown | |
| 2023-04-21 16:53:10,165 INFO HandlerThread:24428 [handler.py:finish():845] shutting down handler | |
| 2023-04-21 16:53:10,910 INFO WriterThread:24428 [datastore.py:close():298] close: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230421_165208-onaovmt4\run-onaovmt4.wandb | |
| 2023-04-21 16:53:11,171 INFO SenderThread:24428 [sender.py:finish():1550] shutting down sender | |
| 2023-04-21 16:53:11,171 INFO SenderThread:24428 [file_pusher.py:finish():168] shutting down file pusher | |
| 2023-04-21 16:53:11,171 INFO SenderThread:24428 [file_pusher.py:join():173] waiting for file pusher | |