Spaces:
Runtime error
Runtime error
| 2023-04-22 15:52:55,534 INFO StreamThr :33684 [internal.py:wandb_internal():86] W&B internal server running at pid: 33684, started at: 2023-04-22 15:52:55.534078 | |
| 2023-04-22 15:52:55,536 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: status | |
| 2023-04-22 15:52:55,537 INFO WriterThread:33684 [datastore.py:open_for_write():85] open: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230422_155255-30hvteby\run-30hvteby.wandb | |
| 2023-04-22 15:52:55,543 DEBUG SenderThread:33684 [sender.py:send():375] send: header | |
| 2023-04-22 15:52:55,606 DEBUG SenderThread:33684 [sender.py:send():375] send: run | |
| 2023-04-22 15:52:56,299 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: check_version | |
| 2023-04-22 15:52:56,301 INFO SenderThread:33684 [dir_watcher.py:__init__():219] watching files in: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230422_155255-30hvteby\files | |
| 2023-04-22 15:52:56,301 INFO SenderThread:33684 [sender.py:_start_run_threads():1124] run started: 30hvteby with start time 1682149975.535061 | |
| 2023-04-22 15:52:56,301 DEBUG SenderThread:33684 [sender.py:send_request():402] send_request: summary_record | |
| 2023-04-22 15:52:56,302 INFO SenderThread:33684 [sender.py:_save_file():1378] saving file wandb-summary.json with policy end | |
| 2023-04-22 15:52:56,302 DEBUG SenderThread:33684 [sender.py:send_request():402] send_request: check_version | |
| 2023-04-22 15:52:56,959 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: run_start | |
| 2023-04-22 15:52:57,024 DEBUG HandlerThread:33684 [system_info.py:__init__():31] System info init | |
| 2023-04-22 15:52:57,024 DEBUG HandlerThread:33684 [system_info.py:__init__():46] System info init done | |
| 2023-04-22 15:52:57,024 INFO HandlerThread:33684 [system_monitor.py:start():181] Starting system monitor | |
| 2023-04-22 15:52:57,025 INFO SystemMonitor:33684 [system_monitor.py:_start():145] Starting system asset monitoring threads | |
| 2023-04-22 15:52:57,025 INFO HandlerThread:33684 [system_monitor.py:probe():201] Collecting system info | |
| 2023-04-22 15:52:57,032 INFO SystemMonitor:33684 [interfaces.py:start():190] Started cpu monitoring | |
| 2023-04-22 15:52:57,033 INFO SystemMonitor:33684 [interfaces.py:start():190] Started disk monitoring | |
| 2023-04-22 15:52:57,033 INFO SystemMonitor:33684 [interfaces.py:start():190] Started gpu monitoring | |
| 2023-04-22 15:52:57,045 INFO SystemMonitor:33684 [interfaces.py:start():190] Started memory monitoring | |
| 2023-04-22 15:52:57,079 INFO SystemMonitor:33684 [interfaces.py:start():190] Started network monitoring | |
| 2023-04-22 15:52:57,094 DEBUG HandlerThread:33684 [system_info.py:probe():195] Probing system | |
| 2023-04-22 15:52:57,097 DEBUG HandlerThread:33684 [system_info.py:_probe_git():180] Probing git | |
| 2023-04-22 15:52:57,110 ERROR gpu :33684 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-22 15:52:57,198 DEBUG HandlerThread:33684 [system_info.py:_probe_git():188] Probing git done | |
| 2023-04-22 15:52:57,198 DEBUG HandlerThread:33684 [system_info.py:probe():240] Probing system done | |
| 2023-04-22 15:52:57,198 DEBUG HandlerThread:33684 [system_monitor.py:probe():210] {'os': 'Windows-10-10.0.19041-SP0', 'python': '3.8.10', 'heartbeatAt': '2023-04-22T07:52:57.094978', 'startedAt': '2023-04-22T07:52:55.495061', 'docker': None, 'cuda': None, 'args': ('--do_train', '--train_file', '.\\datasets\\Zettels\\train.json', '--validation_file', '.\\datasets\\Zettels\\dev.json', '--prompt_column', 'content', '--response_column', 'summary', '--overwrite_cache', '--model_name_or_path', '..\\models\\chatglm-6b-int4', '--output_dir', 'output\\adgen-chatglm-6b-pt-128-2e-2', '--overwrite_output_dir', '--max_source_length', '64', '--max_target_length', '64', '--per_device_train_batch_size', '1', '--per_device_eval_batch_size', '1', '--gradient_accumulation_steps', '16', '--predict_with_generate', '--max_steps', '1000', '--logging_steps', '10', '--save_steps', '10', '--learning_rate', '2e-2', '--pre_seq_len', '128', '--quantization_bit', '4'), 'state': 'running', 'program': 'main.py', 'codePath': 'ptuning\\main.py', 'git': {'remote': 'https://github.com/THUDM/ChatGLM-6B', 'commit': '01e6313abf4122d789d6e68128856af52847b355'}, 'cpu_count': 6, 'cpu_count_logical': 12, 'cpu_freq': {'current': 2592.0, 'min': 0.0, 'max': 2592.0}, 'cpu_freq_per_core': [{'current': 2592.0, 'min': 0.0, 'max': 2592.0}], 'disk': {'total': 500.32030868530273, 'used': 239.2071533203125}, 'gpu': 'NVIDIA GeForce RTX 2060', 'gpu_count': 1, 'gpu_devices': [{'name': 'NVIDIA GeForce RTX 2060', 'memory_total': 6442450944}], 'memory': {'total': 63.87089538574219}} | |
| 2023-04-22 15:52:57,199 INFO HandlerThread:33684 [system_monitor.py:probe():211] Finished collecting system info | |
| 2023-04-22 15:52:57,199 INFO HandlerThread:33684 [system_monitor.py:probe():214] Publishing system info | |
| 2023-04-22 15:52:57,199 DEBUG HandlerThread:33684 [system_info.py:_save_pip():51] Saving list of pip packages installed into the current environment | |
| 2023-04-22 15:52:57,200 DEBUG HandlerThread:33684 [system_info.py:_save_pip():67] Saving pip packages done | |
| 2023-04-22 15:52:57,201 INFO HandlerThread:33684 [system_monitor.py:probe():216] Finished publishing system info | |
| 2023-04-22 15:52:57,216 DEBUG SenderThread:33684 [sender.py:send():375] send: files | |
| 2023-04-22 15:52:57,216 INFO SenderThread:33684 [sender.py:_save_file():1378] saving file wandb-metadata.json with policy now | |
| 2023-04-22 15:52:57,231 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: stop_status | |
| 2023-04-22 15:52:57,231 DEBUG SenderThread:33684 [sender.py:send_request():402] send_request: stop_status | |
| 2023-04-22 15:52:57,317 INFO Thread-16 :33684 [dir_watcher.py:_on_file_created():278] file/dir created: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230422_155255-30hvteby\files\requirements.txt | |
| 2023-04-22 15:52:57,317 INFO Thread-16 :33684 [dir_watcher.py:_on_file_created():278] file/dir created: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230422_155255-30hvteby\files\wandb-summary.json | |
| 2023-04-22 15:52:57,318 INFO Thread-16 :33684 [dir_watcher.py:_on_file_created():278] file/dir created: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230422_155255-30hvteby\files\wandb-metadata.json | |
| 2023-04-22 15:52:57,611 DEBUG SenderThread:33684 [sender.py:send():375] send: telemetry | |
| 2023-04-22 15:52:57,611 DEBUG SenderThread:33684 [sender.py:send():375] send: config | |
| 2023-04-22 15:52:57,613 DEBUG SenderThread:33684 [sender.py:send():375] send: metric | |
| 2023-04-22 15:52:57,613 DEBUG SenderThread:33684 [sender.py:send():375] send: telemetry | |
| 2023-04-22 15:52:57,613 DEBUG SenderThread:33684 [sender.py:send():375] send: metric | |
| 2023-04-22 15:52:57,613 WARNING SenderThread:33684 [sender.py:send_metric():1329] Seen metric with glob (shouldn't happen) | |
| 2023-04-22 15:52:58,319 INFO Thread-16 :33684 [dir_watcher.py:_on_file_created():278] file/dir created: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230422_155255-30hvteby\files\output.log | |
| 2023-04-22 15:52:58,483 INFO wandb-upload_0:33684 [upload_job.py:push():137] Uploaded file C:\Users\Lenovo\AppData\Local\Temp\tmpskkixiipwandb\wfodv4i9-wandb-metadata.json | |
| 2023-04-22 15:52:59,131 ERROR gpu :33684 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-22 15:53:00,331 INFO Thread-16 :33684 [dir_watcher.py:_on_file_modified():295] file/dir modified: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230422_155255-30hvteby\files\output.log | |
| 2023-04-22 15:53:00,960 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: status_report | |
| 2023-04-22 15:53:01,185 ERROR gpu :33684 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-22 15:53:02,355 INFO Thread-16 :33684 [dir_watcher.py:_on_file_modified():295] file/dir modified: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230422_155255-30hvteby\files\output.log | |
| 2023-04-22 15:53:03,221 ERROR gpu :33684 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-22 15:53:05,272 ERROR gpu :33684 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-22 15:53:06,005 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: status_report | |
| 2023-04-22 15:53:07,319 ERROR gpu :33684 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-22 15:53:09,369 ERROR gpu :33684 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-22 15:53:11,046 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: status_report | |
| 2023-04-22 15:53:11,424 ERROR gpu :33684 [interfaces.py:monitor():144] Failed to sample metric: Not Supported | |
| 2023-04-22 15:53:12,250 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: stop_status | |
| 2023-04-22 15:53:12,251 DEBUG SenderThread:33684 [sender.py:send_request():402] send_request: stop_status | |
| 2023-04-22 15:53:12,989 DEBUG SenderThread:33684 [sender.py:send():375] send: exit | |
| 2023-04-22 15:53:12,989 INFO SenderThread:33684 [sender.py:send_exit():598] handling exit code: 255 | |
| 2023-04-22 15:53:12,989 INFO SenderThread:33684 [sender.py:send_exit():600] handling runtime: 16 | |
| 2023-04-22 15:53:12,990 INFO SenderThread:33684 [sender.py:_save_file():1378] saving file wandb-summary.json with policy end | |
| 2023-04-22 15:53:12,991 INFO SenderThread:33684 [sender.py:send_exit():606] send defer | |
| 2023-04-22 15:53:12,991 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: defer | |
| 2023-04-22 15:53:12,991 INFO HandlerThread:33684 [handler.py:handle_request_defer():170] handle defer: 0 | |
| 2023-04-22 15:53:12,992 DEBUG SenderThread:33684 [sender.py:send_request():402] send_request: defer | |
| 2023-04-22 15:53:12,992 INFO SenderThread:33684 [sender.py:send_request_defer():622] handle sender defer: 0 | |
| 2023-04-22 15:53:12,992 INFO SenderThread:33684 [sender.py:transition_state():626] send defer: 1 | |
| 2023-04-22 15:53:12,992 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: defer | |
| 2023-04-22 15:53:12,992 INFO HandlerThread:33684 [handler.py:handle_request_defer():170] handle defer: 1 | |
| 2023-04-22 15:53:12,992 DEBUG SenderThread:33684 [sender.py:send_request():402] send_request: defer | |
| 2023-04-22 15:53:12,992 INFO SenderThread:33684 [sender.py:send_request_defer():622] handle sender defer: 1 | |
| 2023-04-22 15:53:12,992 INFO SenderThread:33684 [sender.py:transition_state():626] send defer: 2 | |
| 2023-04-22 15:53:12,992 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: defer | |
| 2023-04-22 15:53:12,993 INFO HandlerThread:33684 [handler.py:handle_request_defer():170] handle defer: 2 | |
| 2023-04-22 15:53:12,993 INFO HandlerThread:33684 [system_monitor.py:finish():190] Stopping system monitor | |
| 2023-04-22 15:53:12,993 DEBUG SystemMonitor:33684 [system_monitor.py:_start():159] Starting system metrics aggregation loop | |
| 2023-04-22 15:53:12,993 INFO HandlerThread:33684 [interfaces.py:finish():202] Joined cpu monitor | |
| 2023-04-22 15:53:12,993 DEBUG SystemMonitor:33684 [system_monitor.py:_start():166] Finished system metrics aggregation loop | |
| 2023-04-22 15:53:12,993 INFO HandlerThread:33684 [interfaces.py:finish():202] Joined disk monitor | |
| 2023-04-22 15:53:12,993 DEBUG SystemMonitor:33684 [system_monitor.py:_start():170] Publishing last batch of metrics | |
| 2023-04-22 15:53:13,070 INFO HandlerThread:33684 [interfaces.py:finish():202] Joined gpu monitor | |
| 2023-04-22 15:53:13,070 INFO HandlerThread:33684 [interfaces.py:finish():202] Joined memory monitor | |
| 2023-04-22 15:53:13,070 INFO HandlerThread:33684 [interfaces.py:finish():202] Joined network monitor | |
| 2023-04-22 15:53:13,071 DEBUG SenderThread:33684 [sender.py:send_request():402] send_request: defer | |
| 2023-04-22 15:53:13,071 INFO SenderThread:33684 [sender.py:send_request_defer():622] handle sender defer: 2 | |
| 2023-04-22 15:53:13,071 INFO SenderThread:33684 [sender.py:transition_state():626] send defer: 3 | |
| 2023-04-22 15:53:13,071 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: defer | |
| 2023-04-22 15:53:13,071 INFO HandlerThread:33684 [handler.py:handle_request_defer():170] handle defer: 3 | |
| 2023-04-22 15:53:13,073 DEBUG SenderThread:33684 [sender.py:send():375] send: stats | |
| 2023-04-22 15:53:13,074 DEBUG SenderThread:33684 [sender.py:send_request():402] send_request: defer | |
| 2023-04-22 15:53:13,074 INFO SenderThread:33684 [sender.py:send_request_defer():622] handle sender defer: 3 | |
| 2023-04-22 15:53:13,074 INFO SenderThread:33684 [sender.py:transition_state():626] send defer: 4 | |
| 2023-04-22 15:53:13,074 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: defer | |
| 2023-04-22 15:53:13,074 INFO HandlerThread:33684 [handler.py:handle_request_defer():170] handle defer: 4 | |
| 2023-04-22 15:53:13,074 DEBUG SenderThread:33684 [sender.py:send_request():402] send_request: defer | |
| 2023-04-22 15:53:13,074 INFO SenderThread:33684 [sender.py:send_request_defer():622] handle sender defer: 4 | |
| 2023-04-22 15:53:13,074 INFO SenderThread:33684 [sender.py:transition_state():626] send defer: 5 | |
| 2023-04-22 15:53:13,074 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: defer | |
| 2023-04-22 15:53:13,074 INFO HandlerThread:33684 [handler.py:handle_request_defer():170] handle defer: 5 | |
| 2023-04-22 15:53:13,075 DEBUG SenderThread:33684 [sender.py:send():375] send: summary | |
| 2023-04-22 15:53:13,075 INFO SenderThread:33684 [sender.py:_save_file():1378] saving file wandb-summary.json with policy end | |
| 2023-04-22 15:53:13,076 DEBUG SenderThread:33684 [sender.py:send_request():402] send_request: defer | |
| 2023-04-22 15:53:13,076 INFO SenderThread:33684 [sender.py:send_request_defer():622] handle sender defer: 5 | |
| 2023-04-22 15:53:13,076 INFO SenderThread:33684 [sender.py:transition_state():626] send defer: 6 | |
| 2023-04-22 15:53:13,076 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: defer | |
| 2023-04-22 15:53:13,076 INFO HandlerThread:33684 [handler.py:handle_request_defer():170] handle defer: 6 | |
| 2023-04-22 15:53:13,076 DEBUG SenderThread:33684 [sender.py:send_request():402] send_request: defer | |
| 2023-04-22 15:53:13,076 INFO SenderThread:33684 [sender.py:send_request_defer():622] handle sender defer: 6 | |
| 2023-04-22 15:53:13,080 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: status_report | |
| 2023-04-22 15:53:13,427 INFO Thread-16 :33684 [dir_watcher.py:_on_file_modified():295] file/dir modified: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230422_155255-30hvteby\files\wandb-summary.json | |
| 2023-04-22 15:53:13,706 INFO SenderThread:33684 [sender.py:transition_state():626] send defer: 7 | |
| 2023-04-22 15:53:13,706 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: defer | |
| 2023-04-22 15:53:13,706 INFO HandlerThread:33684 [handler.py:handle_request_defer():170] handle defer: 7 | |
| 2023-04-22 15:53:13,706 DEBUG SenderThread:33684 [sender.py:send_request():402] send_request: defer | |
| 2023-04-22 15:53:13,706 INFO SenderThread:33684 [sender.py:send_request_defer():622] handle sender defer: 7 | |
| 2023-04-22 15:53:14,056 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: poll_exit | |
| 2023-04-22 15:53:14,433 INFO Thread-16 :33684 [dir_watcher.py:_on_file_modified():295] file/dir modified: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230422_155255-30hvteby\files\config.yaml | |
| 2023-04-22 15:53:14,433 INFO Thread-16 :33684 [dir_watcher.py:_on_file_modified():295] file/dir modified: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230422_155255-30hvteby\files\output.log | |
| 2023-04-22 15:53:16,166 INFO SenderThread:33684 [sender.py:transition_state():626] send defer: 8 | |
| 2023-04-22 15:53:16,166 DEBUG SenderThread:33684 [sender.py:send_request():402] send_request: poll_exit | |
| 2023-04-22 15:53:16,166 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: defer | |
| 2023-04-22 15:53:16,166 INFO HandlerThread:33684 [handler.py:handle_request_defer():170] handle defer: 8 | |
| 2023-04-22 15:53:16,167 DEBUG SenderThread:33684 [sender.py:send_request():402] send_request: defer | |
| 2023-04-22 15:53:16,167 INFO SenderThread:33684 [sender.py:send_request_defer():622] handle sender defer: 8 | |
| 2023-04-22 15:53:16,180 INFO SenderThread:33684 [sender.py:transition_state():626] send defer: 9 | |
| 2023-04-22 15:53:16,181 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: defer | |
| 2023-04-22 15:53:16,181 DEBUG SenderThread:33684 [sender.py:send():375] send: artifact | |
| 2023-04-22 15:53:16,181 INFO HandlerThread:33684 [handler.py:handle_request_defer():170] handle defer: 9 | |
| 2023-04-22 15:53:16,461 INFO Thread-16 :33684 [dir_watcher.py:_on_file_modified():295] file/dir modified: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230422_155255-30hvteby\files\output.log | |
| 2023-04-22 15:53:17,602 INFO SenderThread:33684 [sender.py:send_artifact():1474] sent artifact job-https___github.com_THUDM_ChatGLM-6B_ptuning_main.py - {'id': 'QXJ0aWZhY3Q6NDI5NzAxNTEw', 'digest': '1077e319ad39e537c1ccc8f9a5c233bc', 'state': 'COMMITTED', 'aliases': [{'artifactCollectionName': 'job-https___github.com_THUDM_ChatGLM-6B_ptuning_main.py', 'alias': 'latest'}, {'artifactCollectionName': 'job-https___github.com_THUDM_ChatGLM-6B_ptuning_main.py', 'alias': 'v4'}], 'artifactSequence': {'id': 'QXJ0aWZhY3RDb2xsZWN0aW9uOjYzNDcyMzc3', 'latestArtifact': {'id': 'QXJ0aWZhY3Q6NDI5OTE5NTc5', 'versionIndex': 5}}, 'version': 'v4'} | |
| 2023-04-22 15:53:17,602 DEBUG SenderThread:33684 [sender.py:send_request():402] send_request: defer | |
| 2023-04-22 15:53:17,602 INFO SenderThread:33684 [sender.py:send_request_defer():622] handle sender defer: 9 | |
| 2023-04-22 15:53:17,603 INFO SenderThread:33684 [dir_watcher.py:finish():365] shutting down directory watcher | |
| 2023-04-22 15:53:18,483 INFO SenderThread:33684 [dir_watcher.py:finish():395] scan: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230422_155255-30hvteby\files | |
| 2023-04-22 15:53:18,483 INFO SenderThread:33684 [dir_watcher.py:finish():409] scan save: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230422_155255-30hvteby\files\config.yaml config.yaml | |
| 2023-04-22 15:53:18,484 INFO SenderThread:33684 [dir_watcher.py:finish():409] scan save: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230422_155255-30hvteby\files\output.log output.log | |
| 2023-04-22 15:53:18,486 INFO SenderThread:33684 [dir_watcher.py:finish():409] scan save: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230422_155255-30hvteby\files\requirements.txt requirements.txt | |
| 2023-04-22 15:53:18,489 INFO SenderThread:33684 [dir_watcher.py:finish():409] scan save: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230422_155255-30hvteby\files\wandb-metadata.json wandb-metadata.json | |
| 2023-04-22 15:53:18,489 INFO SenderThread:33684 [dir_watcher.py:finish():409] scan save: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230422_155255-30hvteby\files\wandb-summary.json wandb-summary.json | |
| 2023-04-22 15:53:18,492 INFO SenderThread:33684 [sender.py:transition_state():626] send defer: 10 | |
| 2023-04-22 15:53:18,492 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: defer | |
| 2023-04-22 15:53:18,492 INFO HandlerThread:33684 [handler.py:handle_request_defer():170] handle defer: 10 | |
| 2023-04-22 15:53:18,494 DEBUG SenderThread:33684 [sender.py:send_request():402] send_request: defer | |
| 2023-04-22 15:53:18,494 INFO SenderThread:33684 [sender.py:send_request_defer():622] handle sender defer: 10 | |
| 2023-04-22 15:53:18,495 INFO SenderThread:33684 [file_pusher.py:finish():168] shutting down file pusher | |
| 2023-04-22 15:53:19,107 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: keepalive | |
| 2023-04-22 15:53:19,301 INFO wandb-upload_1:33684 [upload_job.py:push():137] Uploaded file E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230422_155255-30hvteby\files\output.log | |
| 2023-04-22 15:53:19,632 INFO wandb-upload_0:33684 [upload_job.py:push():137] Uploaded file E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230422_155255-30hvteby\files\config.yaml | |
| 2023-04-22 15:53:19,821 INFO wandb-upload_2:33684 [upload_job.py:push():137] Uploaded file E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230422_155255-30hvteby\files\requirements.txt | |
| 2023-04-22 15:53:20,243 INFO wandb-upload_3:33684 [upload_job.py:push():137] Uploaded file E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230422_155255-30hvteby\files\wandb-summary.json | |
| 2023-04-22 15:53:20,445 INFO Thread-15 :33684 [sender.py:transition_state():626] send defer: 11 | |
| 2023-04-22 15:53:20,445 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: defer | |
| 2023-04-22 15:53:20,445 INFO HandlerThread:33684 [handler.py:handle_request_defer():170] handle defer: 11 | |
| 2023-04-22 15:53:20,446 DEBUG SenderThread:33684 [sender.py:send_request():402] send_request: defer | |
| 2023-04-22 15:53:20,446 INFO SenderThread:33684 [sender.py:send_request_defer():622] handle sender defer: 11 | |
| 2023-04-22 15:53:20,446 INFO SenderThread:33684 [file_pusher.py:join():173] waiting for file pusher | |
| 2023-04-22 15:53:20,446 INFO SenderThread:33684 [sender.py:transition_state():626] send defer: 12 | |
| 2023-04-22 15:53:20,446 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: defer | |
| 2023-04-22 15:53:20,446 INFO HandlerThread:33684 [handler.py:handle_request_defer():170] handle defer: 12 | |
| 2023-04-22 15:53:20,446 DEBUG SenderThread:33684 [sender.py:send_request():402] send_request: defer | |
| 2023-04-22 15:53:20,446 INFO SenderThread:33684 [sender.py:send_request_defer():622] handle sender defer: 12 | |
| 2023-04-22 15:53:20,872 INFO SenderThread:33684 [sender.py:transition_state():626] send defer: 13 | |
| 2023-04-22 15:53:20,872 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: defer | |
| 2023-04-22 15:53:20,872 INFO HandlerThread:33684 [handler.py:handle_request_defer():170] handle defer: 13 | |
| 2023-04-22 15:53:20,872 DEBUG SenderThread:33684 [sender.py:send_request():402] send_request: defer | |
| 2023-04-22 15:53:20,872 INFO SenderThread:33684 [sender.py:send_request_defer():622] handle sender defer: 13 | |
| 2023-04-22 15:53:20,872 INFO SenderThread:33684 [sender.py:transition_state():626] send defer: 14 | |
| 2023-04-22 15:53:20,873 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: defer | |
| 2023-04-22 15:53:20,873 DEBUG SenderThread:33684 [sender.py:send():375] send: final | |
| 2023-04-22 15:53:20,873 INFO HandlerThread:33684 [handler.py:handle_request_defer():170] handle defer: 14 | |
| 2023-04-22 15:53:20,873 DEBUG SenderThread:33684 [sender.py:send():375] send: footer | |
| 2023-04-22 15:53:20,873 DEBUG SenderThread:33684 [sender.py:send_request():402] send_request: defer | |
| 2023-04-22 15:53:20,873 INFO SenderThread:33684 [sender.py:send_request_defer():622] handle sender defer: 14 | |
| 2023-04-22 15:53:20,874 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: poll_exit | |
| 2023-04-22 15:53:20,874 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: server_info | |
| 2023-04-22 15:53:20,874 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: get_summary | |
| 2023-04-22 15:53:20,875 DEBUG SenderThread:33684 [sender.py:send_request():402] send_request: poll_exit | |
| 2023-04-22 15:53:20,875 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: sampled_history | |
| 2023-04-22 15:53:20,875 DEBUG SenderThread:33684 [sender.py:send_request():402] send_request: server_info | |
| 2023-04-22 15:53:21,122 INFO MainThread:33684 [wandb_run.py:_footer_history_summary_info():3476] rendering history | |
| 2023-04-22 15:53:21,123 INFO MainThread:33684 [wandb_run.py:_footer_history_summary_info():3508] rendering summary | |
| 2023-04-22 15:53:21,123 INFO MainThread:33684 [wandb_run.py:_footer_sync_info():3434] logging synced files | |
| 2023-04-22 15:53:21,125 DEBUG HandlerThread:33684 [handler.py:handle_request():144] handle_request: shutdown | |
| 2023-04-22 15:53:21,125 INFO HandlerThread:33684 [handler.py:finish():845] shutting down handler | |
| 2023-04-22 15:53:21,881 INFO WriterThread:33684 [datastore.py:close():298] close: E:\Documents\Desktop\ChatGLM-6B\ptuning\wandb\run-20230422_155255-30hvteby\run-30hvteby.wandb | |
| 2023-04-22 15:53:22,129 INFO SenderThread:33684 [sender.py:finish():1550] shutting down sender | |
| 2023-04-22 15:53:22,129 INFO SenderThread:33684 [file_pusher.py:finish():168] shutting down file pusher | |
| 2023-04-22 15:53:22,129 INFO SenderThread:33684 [file_pusher.py:join():173] waiting for file pusher | |