Hugging Face's logo

sanchit-gandhi
/
distil-zephyr-1.5b-dpo-ultrafeedback

Text Generation
Transformers
TensorBoard
Safetensors
mistral
alignment-handbook
trl
dpo
Generated from Trainer
conversational
text-generation-inference
Inference Endpoints
Model card Files Metrics Community
sanchit-gandhi HF staff commited on
Commit
315a9a7
•
1 Parent(s): 6660cc4

Training in progress, step 300

Browse files
Files changed (6) hide show
  1. runs/Apr26_15-48-43_ip-26-0-160-225/events.out.tfevents.1714147034.ip-26-0-160-225.711598.0 +2 -2
  2. wandb/debug-internal.log +167 -0
  3. wandb/run-20240426_155715-vljs8xzo/files/output.log +83 -0
  4. wandb/run-20240426_155715-vljs8xzo/files/wandb-summary.json +1 -1
  5. wandb/run-20240426_155715-vljs8xzo/logs/debug-internal.log +167 -0
  6. wandb/run-20240426_155715-vljs8xzo/run-vljs8xzo.wandb +0 -0
runs/Apr26_15-48-43_ip-26-0-160-225/events.out.tfevents.1714147034.ip-26-0-160-225.711598.0 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:264ee88e267082f4f8a13bb8a5f8479846786f6cfedc3a35195e8cd9a74899c4
3
- size 12362
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:001570658f281534496db82c62e1a044d09333fd849c089b8724f2efc43ffb43
3
+ size 15854
wandb/debug-internal.log CHANGED
@@ -410,3 +410,170 @@
410
  2024-04-26 16:02:50,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
411
  2024-04-26 16:02:50,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
412
  2024-04-26 16:02:50,704 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
410
  2024-04-26 16:02:50,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
411
  2024-04-26 16:02:50,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
412
  2024-04-26 16:02:50,704 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
413
+ 2024-04-26 16:02:53,515 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
414
+ 2024-04-26 16:02:55,517 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
415
+ 2024-04-26 16:02:55,925 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
416
+ 2024-04-26 16:02:57,519 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
417
+ 2024-04-26 16:02:59,521 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
418
+ 2024-04-26 16:03:01,073 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
419
+ 2024-04-26 16:03:01,523 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
420
+ 2024-04-26 16:03:03,525 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
421
+ 2024-04-26 16:03:05,527 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
422
+ 2024-04-26 16:03:05,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
423
+ 2024-04-26 16:03:05,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
424
+ 2024-04-26 16:03:05,704 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
425
+ 2024-04-26 16:03:06,525 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
426
+ 2024-04-26 16:03:07,529 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
427
+ 2024-04-26 16:03:09,531 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
428
+ 2024-04-26 16:03:11,533 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
429
+ 2024-04-26 16:03:11,683 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
430
+ 2024-04-26 16:03:13,535 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
431
+ 2024-04-26 16:03:15,537 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
432
+ 2024-04-26 16:03:16,268 DEBUG SenderThread:713738 [sender.py:send():379] send: stats
433
+ 2024-04-26 16:03:16,806 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
434
+ 2024-04-26 16:03:17,539 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
435
+ 2024-04-26 16:03:19,540 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
436
+ 2024-04-26 16:03:20,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
437
+ 2024-04-26 16:03:20,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
438
+ 2024-04-26 16:03:20,704 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
439
+ 2024-04-26 16:03:21,542 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
440
+ 2024-04-26 16:03:21,851 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
441
+ 2024-04-26 16:03:23,544 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
442
+ 2024-04-26 16:03:24,550 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: partial_history
443
+ 2024-04-26 16:03:24,552 DEBUG SenderThread:713738 [sender.py:send():379] send: history
444
+ 2024-04-26 16:03:24,552 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: summary_record
445
+ 2024-04-26 16:03:24,554 INFO SenderThread:713738 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
446
+ 2024-04-26 16:03:25,547 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
447
+ 2024-04-26 16:03:25,547 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/wandb-summary.json
448
+ 2024-04-26 16:03:27,036 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
449
+ 2024-04-26 16:03:27,561 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
450
+ 2024-04-26 16:03:29,564 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
451
+ 2024-04-26 16:03:31,566 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
452
+ 2024-04-26 16:03:32,153 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
453
+ 2024-04-26 16:03:33,568 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
454
+ 2024-04-26 16:03:35,569 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
455
+ 2024-04-26 16:03:35,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
456
+ 2024-04-26 16:03:35,703 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
457
+ 2024-04-26 16:03:35,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
458
+ 2024-04-26 16:03:37,320 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
459
+ 2024-04-26 16:03:37,572 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
460
+ 2024-04-26 16:03:39,574 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
461
+ 2024-04-26 16:03:41,576 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
462
+ 2024-04-26 16:03:42,553 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
463
+ 2024-04-26 16:03:43,578 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
464
+ 2024-04-26 16:03:45,580 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
465
+ 2024-04-26 16:03:46,257 DEBUG SenderThread:713738 [sender.py:send():379] send: stats
466
+ 2024-04-26 16:03:47,582 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
467
+ 2024-04-26 16:03:47,771 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
468
+ 2024-04-26 16:03:49,584 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
469
+ 2024-04-26 16:03:50,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
470
+ 2024-04-26 16:03:50,703 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
471
+ 2024-04-26 16:03:50,704 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
472
+ 2024-04-26 16:03:51,586 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
473
+ 2024-04-26 16:03:52,587 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
474
+ 2024-04-26 16:03:52,945 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
475
+ 2024-04-26 16:03:54,590 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
476
+ 2024-04-26 16:03:56,592 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
477
+ 2024-04-26 16:03:57,001 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: partial_history
478
+ 2024-04-26 16:03:57,002 DEBUG SenderThread:713738 [sender.py:send():379] send: history
479
+ 2024-04-26 16:03:57,003 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: summary_record
480
+ 2024-04-26 16:03:57,004 INFO SenderThread:713738 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
481
+ 2024-04-26 16:03:57,594 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/wandb-summary.json
482
+ 2024-04-26 16:03:58,005 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
483
+ 2024-04-26 16:03:58,595 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
484
+ 2024-04-26 16:04:00,597 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
485
+ 2024-04-26 16:04:02,599 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
486
+ 2024-04-26 16:04:03,227 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
487
+ 2024-04-26 16:04:04,601 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
488
+ 2024-04-26 16:04:05,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
489
+ 2024-04-26 16:04:05,704 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
490
+ 2024-04-26 16:04:05,704 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
491
+ 2024-04-26 16:04:06,603 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
492
+ 2024-04-26 16:04:08,411 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
493
+ 2024-04-26 16:04:08,605 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
494
+ 2024-04-26 16:04:10,607 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
495
+ 2024-04-26 16:04:12,609 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
496
+ 2024-04-26 16:04:13,658 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
497
+ 2024-04-26 16:04:14,611 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
498
+ 2024-04-26 16:04:16,259 DEBUG SenderThread:713738 [sender.py:send():379] send: stats
499
+ 2024-04-26 16:04:16,613 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
500
+ 2024-04-26 16:04:18,615 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
501
+ 2024-04-26 16:04:18,859 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
502
+ 2024-04-26 16:04:20,617 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
503
+ 2024-04-26 16:04:20,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
504
+ 2024-04-26 16:04:20,704 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
505
+ 2024-04-26 16:04:20,705 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
506
+ 2024-04-26 16:04:22,619 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
507
+ 2024-04-26 16:04:24,101 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
508
+ 2024-04-26 16:04:24,621 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
509
+ 2024-04-26 16:04:26,623 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
510
+ 2024-04-26 16:04:28,625 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
511
+ 2024-04-26 16:04:29,310 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
512
+ 2024-04-26 16:04:29,685 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: partial_history
513
+ 2024-04-26 16:04:29,686 DEBUG SenderThread:713738 [sender.py:send():379] send: history
514
+ 2024-04-26 16:04:29,687 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: summary_record
515
+ 2024-04-26 16:04:29,688 INFO SenderThread:713738 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
516
+ 2024-04-26 16:04:30,628 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
517
+ 2024-04-26 16:04:30,628 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/wandb-summary.json
518
+ 2024-04-26 16:04:32,629 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
519
+ 2024-04-26 16:04:34,539 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
520
+ 2024-04-26 16:04:34,631 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
521
+ 2024-04-26 16:04:35,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
522
+ 2024-04-26 16:04:35,704 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
523
+ 2024-04-26 16:04:35,705 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
524
+ 2024-04-26 16:04:36,633 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
525
+ 2024-04-26 16:04:38,635 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
526
+ 2024-04-26 16:04:39,878 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
527
+ 2024-04-26 16:04:40,637 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
528
+ 2024-04-26 16:04:42,639 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
529
+ 2024-04-26 16:04:44,641 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
530
+ 2024-04-26 16:04:45,113 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
531
+ 2024-04-26 16:04:46,274 DEBUG SenderThread:713738 [sender.py:send():379] send: stats
532
+ 2024-04-26 16:04:46,643 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
533
+ 2024-04-26 16:04:48,645 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
534
+ 2024-04-26 16:04:50,440 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
535
+ 2024-04-26 16:04:50,647 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
536
+ 2024-04-26 16:04:50,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
537
+ 2024-04-26 16:04:50,704 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
538
+ 2024-04-26 16:04:50,705 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
539
+ 2024-04-26 16:04:52,649 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
540
+ 2024-04-26 16:04:54,651 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
541
+ 2024-04-26 16:04:55,860 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
542
+ 2024-04-26 16:04:56,653 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
543
+ 2024-04-26 16:04:58,655 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
544
+ 2024-04-26 16:05:00,657 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
545
+ 2024-04-26 16:05:01,087 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
546
+ 2024-04-26 16:05:02,659 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
547
+ 2024-04-26 16:05:02,768 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: partial_history
548
+ 2024-04-26 16:05:02,770 DEBUG SenderThread:713738 [sender.py:send():379] send: history
549
+ 2024-04-26 16:05:02,770 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: summary_record
550
+ 2024-04-26 16:05:02,772 INFO SenderThread:713738 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
551
+ 2024-04-26 16:05:03,661 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/wandb-summary.json
552
+ 2024-04-26 16:05:04,662 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
553
+ 2024-04-26 16:05:05,704 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
554
+ 2024-04-26 16:05:05,704 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
555
+ 2024-04-26 16:05:05,705 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
556
+ 2024-04-26 16:05:06,208 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
557
+ 2024-04-26 16:05:06,665 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
558
+ 2024-04-26 16:05:08,667 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
559
+ 2024-04-26 16:05:10,669 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
560
+ 2024-04-26 16:05:11,138 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: partial_history
561
+ 2024-04-26 16:05:11,139 DEBUG SenderThread:713738 [sender.py:send():379] send: history
562
+ 2024-04-26 16:05:11,140 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: summary_record
563
+ 2024-04-26 16:05:11,142 INFO SenderThread:713738 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
564
+ 2024-04-26 16:05:11,671 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/wandb-summary.json
565
+ 2024-04-26 16:05:12,150 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
566
+ 2024-04-26 16:05:12,672 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
567
+ 2024-04-26 16:05:16,263 DEBUG SenderThread:713738 [sender.py:send():379] send: stats
568
+ 2024-04-26 16:05:17,264 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
569
+ 2024-04-26 16:05:21,425 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
570
+ 2024-04-26 16:05:21,787 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
571
+ 2024-04-26 16:05:21,787 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
572
+ 2024-04-26 16:05:22,682 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
573
+ 2024-04-26 16:05:22,876 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
574
+ 2024-04-26 16:05:27,877 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
575
+ 2024-04-26 16:05:32,878 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
576
+ 2024-04-26 16:05:37,183 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
577
+ 2024-04-26 16:05:37,184 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
578
+ 2024-04-26 16:05:37,526 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
579
+ 2024-04-26 16:05:38,277 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
wandb/run-20240426_155715-vljs8xzo/files/output.log CHANGED
@@ -158,3 +158,86 @@
158
  {'eval_loss': 0.6533502340316772, 'eval_runtime': 8.2763, 'eval_samples_per_second': 241.653, 'eval_steps_per_second': 3.866, 'eval_rewards/chosen': -0.06664139777421951, 'eval_rewards/rejected': -0.16173213720321655, 'eval_rewards/accuracies': 0.64453125, 'eval_rewards/margins': 0.09509073942899704, 'eval_logps/rejected': -420.6952209472656, 'eval_logps/chosen': -429.85614013671875, 'eval_logits/rejected': -3.2240023612976074, 'eval_logits/chosen': -3.2767982482910156, 'epoch': 0.42}
159
  [INFO|modeling_utils.py:2598] 2024-04-26 16:02:32,167 >> The model is bigger than the maximum size per checkpoint (5GB) and is going to be split in 2 checkpoint shards. You can find where each parameters has been saved in the index located at ./checkpoint-200/model.safetensors.index.json.
160
  [INFO|tokenization_utils_base.py:2488] 2024-04-26 16:02:32,170 >> tokenizer config file saved in ./checkpoint-200/tokenizer_config.json
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
158
  {'eval_loss': 0.6533502340316772, 'eval_runtime': 8.2763, 'eval_samples_per_second': 241.653, 'eval_steps_per_second': 3.866, 'eval_rewards/chosen': -0.06664139777421951, 'eval_rewards/rejected': -0.16173213720321655, 'eval_rewards/accuracies': 0.64453125, 'eval_rewards/margins': 0.09509073942899704, 'eval_logps/rejected': -420.6952209472656, 'eval_logps/chosen': -429.85614013671875, 'eval_logits/rejected': -3.2240023612976074, 'eval_logits/chosen': -3.2767982482910156, 'epoch': 0.42}
159
  [INFO|modeling_utils.py:2598] 2024-04-26 16:02:32,167 >> The model is bigger than the maximum size per checkpoint (5GB) and is going to be split in 2 checkpoint shards. You can find where each parameters has been saved in the index located at ./checkpoint-200/model.safetensors.index.json.
160
  [INFO|tokenization_utils_base.py:2488] 2024-04-26 16:02:32,170 >> tokenizer config file saved in ./checkpoint-200/tokenizer_config.json
161
+ [INFO|tokenization_utils_base.py:2497] 2024-04-26 16:02:32,172 >> Special tokens file saved in ./checkpoint-200/special_tokens_map.json
162
+ [INFO|tokenization_utils_base.py:2488] 2024-04-26 16:02:50,674 >> tokenizer config file saved in ./tokenizer_config.json
163
+ [INFO|tokenization_utils_base.py:2497] 2024-04-26 16:02:50,676 >> Special tokens file saved in ./special_tokens_map.json
164
+ [INFO|trainer.py:3397] 2024-04-26 16:02:50,704 >> Deleting older checkpoint [checkpoint-100] due to args.save_total_limit
165
+
166
+
167
+
168
+
169
+
170
+
171
+
172
+
173
+
174
+
175
+
176
+
177
+
178
+
179
+
180
+
181
+ 47%|████▋ | 226/478 [06:05<05:20, 1.27s/it]
182
+
183
+
184
+
185
+
186
+
187
+
188
+
189
+
190
+
191
+
192
+
193
+
194
+
195
+
196
+
197
+
198
+ 52%|█████▏ | 250/478 [06:36<05:00, 1.32s/it]
199
+
200
+
201
+
202
+
203
+
204
+
205
+
206
+
207
+
208
+
209
+
210
+
211
+
212
+
213
+
214
+
215
+ 58%|█████▊ | 275/478 [07:09<04:25, 1.31s/it]
216
+
217
+
218
+
219
+
220
+
221
+
222
+
223
+
224
+
225
+
226
+
227
+
228
+
229
+
230
+
231
+
232
+ 63%|██████▎ | 300/478 [07:42<03:53, 1.31s/it][INFO|trainer.py:3614] 2024-04-26 16:05:02,769 >> ***** Running Evaluation *****
233
+ [INFO|trainer.py:3616] 2024-04-26 16:05:02,769 >> Num examples = 2000
234
+ [INFO|trainer.py:3619] 2024-04-26 16:05:02,769 >> Batch size = 8
235
+ 12%|█▎ | 4/32 [00:00<00:05, 5.09it/s]
236
+
237
+
238
+
239
+ [INFO|configuration_utils.py:471] 2024-04-26 16:05:11,146 >> Configuration saved in ./checkpoint-300/config.json
240
+ [INFO|configuration_utils.py:697] 2024-04-26 16:05:11,149 >> Configuration saved in ./checkpoint-300/generation_config.json
241
+ {'eval_loss': 0.6438009142875671, 'eval_runtime': 8.3559, 'eval_samples_per_second': 239.351, 'eval_steps_per_second': 3.83, 'eval_rewards/chosen': -0.10771973431110382, 'eval_rewards/rejected': -0.24101632833480835, 'eval_rewards/accuracies': 0.62109375, 'eval_rewards/margins': 0.13329659402370453, 'eval_logps/rejected': -428.6236572265625, 'eval_logps/chosen': -433.9639892578125, 'eval_logits/rejected': -3.2049574851989746, 'eval_logits/chosen': -3.2553329467773438, 'epoch': 0.63}
242
+ [INFO|modeling_utils.py:2598] 2024-04-26 16:05:20,618 >> The model is bigger than the maximum size per checkpoint (5GB) and is going to be split in 2 checkpoint shards. You can find where each parameters has been saved in the index located at ./checkpoint-300/model.safetensors.index.json.
243
+ [INFO|tokenization_utils_base.py:2488] 2024-04-26 16:05:20,621 >> tokenizer config file saved in ./checkpoint-300/tokenizer_config.json
wandb/run-20240426_155715-vljs8xzo/files/wandb-summary.json CHANGED
@@ -1 +1 @@
1
- {"train/loss": 0.6584, "train/grad_norm": 9.230700492858887, "train/learning_rate": 3.610497133404795e-07, "train/rewards/chosen": -0.06991340965032578, "train/rewards/rejected": -0.15435244143009186, "train/rewards/accuracies": 0.6349999904632568, "train/rewards/margins": 0.08443903923034668, "train/logps/rejected": -396.62774658203125, "train/logps/chosen": -431.6617126464844, "train/logits/rejected": -3.1550867557525635, "train/logits/chosen": -3.23995304107666, "train/epoch": 0.41841004184100417, "train/global_step": 200, "_timestamp": 1714147342.7592435, "_runtime": 307.0583744049072, "_step": 10, "eval/loss": 0.6533502340316772, "eval/runtime": 8.2763, "eval/samples_per_second": 241.653, "eval/steps_per_second": 3.866, "eval/rewards/chosen": -0.06664139777421951, "eval/rewards/rejected": -0.16173213720321655, "eval/rewards/accuracies": 0.64453125, "eval/rewards/margins": 0.09509073942899704, "eval/logps/rejected": -420.6952209472656, "eval/logps/chosen": -429.85614013671875, "eval/logits/rejected": -3.2240023612976074, "eval/logits/chosen": -3.2767982482910156}
 
1
+ {"train/loss": 0.6494, "train/grad_norm": 10.980072975158691, "train/learning_rate": 1.8323929841460178e-07, "train/rewards/chosen": -0.1245601624250412, "train/rewards/rejected": -0.2303762584924698, "train/rewards/accuracies": 0.625, "train/rewards/margins": 0.10581608861684799, "train/logps/rejected": -436.2774658203125, "train/logps/chosen": -443.0799865722656, "train/logits/rejected": -3.1383109092712402, "train/logits/chosen": -3.2006824016571045, "train/epoch": 0.6276150627615062, "train/global_step": 300, "_timestamp": 1714147511.1377053, "_runtime": 475.4368362426758, "_step": 15, "eval/loss": 0.6438009142875671, "eval/runtime": 8.3559, "eval/samples_per_second": 239.351, "eval/steps_per_second": 3.83, "eval/rewards/chosen": -0.10771973431110382, "eval/rewards/rejected": -0.24101632833480835, "eval/rewards/accuracies": 0.62109375, "eval/rewards/margins": 0.13329659402370453, "eval/logps/rejected": -428.6236572265625, "eval/logps/chosen": -433.9639892578125, "eval/logits/rejected": -3.2049574851989746, "eval/logits/chosen": -3.2553329467773438}
wandb/run-20240426_155715-vljs8xzo/logs/debug-internal.log CHANGED
@@ -410,3 +410,170 @@
410
  2024-04-26 16:02:50,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
411
  2024-04-26 16:02:50,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
412
  2024-04-26 16:02:50,704 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
410
  2024-04-26 16:02:50,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
411
  2024-04-26 16:02:50,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
412
  2024-04-26 16:02:50,704 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
413
+ 2024-04-26 16:02:53,515 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
414
+ 2024-04-26 16:02:55,517 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
415
+ 2024-04-26 16:02:55,925 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
416
+ 2024-04-26 16:02:57,519 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
417
+ 2024-04-26 16:02:59,521 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
418
+ 2024-04-26 16:03:01,073 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
419
+ 2024-04-26 16:03:01,523 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
420
+ 2024-04-26 16:03:03,525 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
421
+ 2024-04-26 16:03:05,527 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
422
+ 2024-04-26 16:03:05,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
423
+ 2024-04-26 16:03:05,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
424
+ 2024-04-26 16:03:05,704 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
425
+ 2024-04-26 16:03:06,525 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
426
+ 2024-04-26 16:03:07,529 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
427
+ 2024-04-26 16:03:09,531 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
428
+ 2024-04-26 16:03:11,533 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
429
+ 2024-04-26 16:03:11,683 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
430
+ 2024-04-26 16:03:13,535 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
431
+ 2024-04-26 16:03:15,537 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
432
+ 2024-04-26 16:03:16,268 DEBUG SenderThread:713738 [sender.py:send():379] send: stats
433
+ 2024-04-26 16:03:16,806 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
434
+ 2024-04-26 16:03:17,539 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
435
+ 2024-04-26 16:03:19,540 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
436
+ 2024-04-26 16:03:20,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
437
+ 2024-04-26 16:03:20,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
438
+ 2024-04-26 16:03:20,704 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
439
+ 2024-04-26 16:03:21,542 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
440
+ 2024-04-26 16:03:21,851 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
441
+ 2024-04-26 16:03:23,544 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
442
+ 2024-04-26 16:03:24,550 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: partial_history
443
+ 2024-04-26 16:03:24,552 DEBUG SenderThread:713738 [sender.py:send():379] send: history
444
+ 2024-04-26 16:03:24,552 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: summary_record
445
+ 2024-04-26 16:03:24,554 INFO SenderThread:713738 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
446
+ 2024-04-26 16:03:25,547 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
447
+ 2024-04-26 16:03:25,547 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/wandb-summary.json
448
+ 2024-04-26 16:03:27,036 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
449
+ 2024-04-26 16:03:27,561 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
450
+ 2024-04-26 16:03:29,564 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
451
+ 2024-04-26 16:03:31,566 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
452
+ 2024-04-26 16:03:32,153 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
453
+ 2024-04-26 16:03:33,568 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
454
+ 2024-04-26 16:03:35,569 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
455
+ 2024-04-26 16:03:35,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
456
+ 2024-04-26 16:03:35,703 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
457
+ 2024-04-26 16:03:35,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
458
+ 2024-04-26 16:03:37,320 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
459
+ 2024-04-26 16:03:37,572 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
460
+ 2024-04-26 16:03:39,574 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
461
+ 2024-04-26 16:03:41,576 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
462
+ 2024-04-26 16:03:42,553 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
463
+ 2024-04-26 16:03:43,578 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
464
+ 2024-04-26 16:03:45,580 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
465
+ 2024-04-26 16:03:46,257 DEBUG SenderThread:713738 [sender.py:send():379] send: stats
466
+ 2024-04-26 16:03:47,582 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
467
+ 2024-04-26 16:03:47,771 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
468
+ 2024-04-26 16:03:49,584 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
469
+ 2024-04-26 16:03:50,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
470
+ 2024-04-26 16:03:50,703 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
471
+ 2024-04-26 16:03:50,704 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
472
+ 2024-04-26 16:03:51,586 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
473
+ 2024-04-26 16:03:52,587 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
474
+ 2024-04-26 16:03:52,945 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
475
+ 2024-04-26 16:03:54,590 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
476
+ 2024-04-26 16:03:56,592 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
477
+ 2024-04-26 16:03:57,001 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: partial_history
478
+ 2024-04-26 16:03:57,002 DEBUG SenderThread:713738 [sender.py:send():379] send: history
479
+ 2024-04-26 16:03:57,003 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: summary_record
480
+ 2024-04-26 16:03:57,004 INFO SenderThread:713738 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
481
+ 2024-04-26 16:03:57,594 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/wandb-summary.json
482
+ 2024-04-26 16:03:58,005 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
483
+ 2024-04-26 16:03:58,595 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
484
+ 2024-04-26 16:04:00,597 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
485
+ 2024-04-26 16:04:02,599 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
486
+ 2024-04-26 16:04:03,227 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
487
+ 2024-04-26 16:04:04,601 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
488
+ 2024-04-26 16:04:05,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
489
+ 2024-04-26 16:04:05,704 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
490
+ 2024-04-26 16:04:05,704 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
491
+ 2024-04-26 16:04:06,603 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
492
+ 2024-04-26 16:04:08,411 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
493
+ 2024-04-26 16:04:08,605 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
494
+ 2024-04-26 16:04:10,607 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
495
+ 2024-04-26 16:04:12,609 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
496
+ 2024-04-26 16:04:13,658 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
497
+ 2024-04-26 16:04:14,611 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
498
+ 2024-04-26 16:04:16,259 DEBUG SenderThread:713738 [sender.py:send():379] send: stats
499
+ 2024-04-26 16:04:16,613 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
500
+ 2024-04-26 16:04:18,615 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
501
+ 2024-04-26 16:04:18,859 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
502
+ 2024-04-26 16:04:20,617 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
503
+ 2024-04-26 16:04:20,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
504
+ 2024-04-26 16:04:20,704 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
505
+ 2024-04-26 16:04:20,705 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
506
+ 2024-04-26 16:04:22,619 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
507
+ 2024-04-26 16:04:24,101 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
508
+ 2024-04-26 16:04:24,621 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
509
+ 2024-04-26 16:04:26,623 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
510
+ 2024-04-26 16:04:28,625 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
511
+ 2024-04-26 16:04:29,310 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
512
+ 2024-04-26 16:04:29,685 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: partial_history
513
+ 2024-04-26 16:04:29,686 DEBUG SenderThread:713738 [sender.py:send():379] send: history
514
+ 2024-04-26 16:04:29,687 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: summary_record
515
+ 2024-04-26 16:04:29,688 INFO SenderThread:713738 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
516
+ 2024-04-26 16:04:30,628 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
517
+ 2024-04-26 16:04:30,628 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/wandb-summary.json
518
+ 2024-04-26 16:04:32,629 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
519
+ 2024-04-26 16:04:34,539 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
520
+ 2024-04-26 16:04:34,631 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
521
+ 2024-04-26 16:04:35,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
522
+ 2024-04-26 16:04:35,704 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
523
+ 2024-04-26 16:04:35,705 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
524
+ 2024-04-26 16:04:36,633 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
525
+ 2024-04-26 16:04:38,635 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
526
+ 2024-04-26 16:04:39,878 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
527
+ 2024-04-26 16:04:40,637 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
528
+ 2024-04-26 16:04:42,639 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
529
+ 2024-04-26 16:04:44,641 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
530
+ 2024-04-26 16:04:45,113 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
531
+ 2024-04-26 16:04:46,274 DEBUG SenderThread:713738 [sender.py:send():379] send: stats
532
+ 2024-04-26 16:04:46,643 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
533
+ 2024-04-26 16:04:48,645 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
534
+ 2024-04-26 16:04:50,440 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
535
+ 2024-04-26 16:04:50,647 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
536
+ 2024-04-26 16:04:50,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
537
+ 2024-04-26 16:04:50,704 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
538
+ 2024-04-26 16:04:50,705 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
539
+ 2024-04-26 16:04:52,649 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
540
+ 2024-04-26 16:04:54,651 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
541
+ 2024-04-26 16:04:55,860 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
542
+ 2024-04-26 16:04:56,653 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
543
+ 2024-04-26 16:04:58,655 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
544
+ 2024-04-26 16:05:00,657 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
545
+ 2024-04-26 16:05:01,087 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
546
+ 2024-04-26 16:05:02,659 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
547
+ 2024-04-26 16:05:02,768 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: partial_history
548
+ 2024-04-26 16:05:02,770 DEBUG SenderThread:713738 [sender.py:send():379] send: history
549
+ 2024-04-26 16:05:02,770 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: summary_record
550
+ 2024-04-26 16:05:02,772 INFO SenderThread:713738 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
551
+ 2024-04-26 16:05:03,661 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/wandb-summary.json
552
+ 2024-04-26 16:05:04,662 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
553
+ 2024-04-26 16:05:05,704 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
554
+ 2024-04-26 16:05:05,704 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
555
+ 2024-04-26 16:05:05,705 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
556
+ 2024-04-26 16:05:06,208 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
557
+ 2024-04-26 16:05:06,665 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
558
+ 2024-04-26 16:05:08,667 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
559
+ 2024-04-26 16:05:10,669 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
560
+ 2024-04-26 16:05:11,138 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: partial_history
561
+ 2024-04-26 16:05:11,139 DEBUG SenderThread:713738 [sender.py:send():379] send: history
562
+ 2024-04-26 16:05:11,140 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: summary_record
563
+ 2024-04-26 16:05:11,142 INFO SenderThread:713738 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
564
+ 2024-04-26 16:05:11,671 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/wandb-summary.json
565
+ 2024-04-26 16:05:12,150 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
566
+ 2024-04-26 16:05:12,672 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
567
+ 2024-04-26 16:05:16,263 DEBUG SenderThread:713738 [sender.py:send():379] send: stats
568
+ 2024-04-26 16:05:17,264 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
569
+ 2024-04-26 16:05:21,425 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
570
+ 2024-04-26 16:05:21,787 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
571
+ 2024-04-26 16:05:21,787 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
572
+ 2024-04-26 16:05:22,682 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
573
+ 2024-04-26 16:05:22,876 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
574
+ 2024-04-26 16:05:27,877 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
575
+ 2024-04-26 16:05:32,878 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
576
+ 2024-04-26 16:05:37,183 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
577
+ 2024-04-26 16:05:37,184 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
578
+ 2024-04-26 16:05:37,526 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
579
+ 2024-04-26 16:05:38,277 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
wandb/run-20240426_155715-vljs8xzo/run-vljs8xzo.wandb CHANGED
Binary files a/wandb/run-20240426_155715-vljs8xzo/run-vljs8xzo.wandb and b/wandb/run-20240426_155715-vljs8xzo/run-vljs8xzo.wandb differ