Hugging Face's logo

sanchit-gandhi
/
distil-zephyr-1.5b-dpo-ultrafeedback

Text Generation
Transformers
TensorBoard
Safetensors
mistral
alignment-handbook
trl
dpo
Generated from Trainer
conversational
text-generation-inference
Inference Endpoints
Model card Files Metrics Community
sanchit-gandhi HF staff commited on
Commit
6660cc4
•
1 Parent(s): 409a825

Training in progress, step 200

Browse files
Files changed (6) hide show
  1. runs/Apr26_15-48-43_ip-26-0-160-225/events.out.tfevents.1714147034.ip-26-0-160-225.711598.0 +2 -2
  2. wandb/debug-internal.log +166 -0
  3. wandb/run-20240426_155715-vljs8xzo/files/output.log +80 -0
  4. wandb/run-20240426_155715-vljs8xzo/files/wandb-summary.json +1 -1
  5. wandb/run-20240426_155715-vljs8xzo/logs/debug-internal.log +166 -0
  6. wandb/run-20240426_155715-vljs8xzo/run-vljs8xzo.wandb +0 -0
runs/Apr26_15-48-43_ip-26-0-160-225/events.out.tfevents.1714147034.ip-26-0-160-225.711598.0 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:64238e8e27212cd19514ceb87b1fb1ff64e9bb81065aa3c77c67a0f390c19fff
3
- size 8882
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:264ee88e267082f4f8a13bb8a5f8479846786f6cfedc3a35195e8cd9a74899c4
3
+ size 12362
wandb/debug-internal.log CHANGED
@@ -244,3 +244,169 @@
244
  2024-04-26 16:00:05,958 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
245
  2024-04-26 16:00:06,058 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
246
  2024-04-26 16:00:06,059 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
244
  2024-04-26 16:00:05,958 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
245
  2024-04-26 16:00:06,058 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
246
  2024-04-26 16:00:06,059 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
247
+ 2024-04-26 16:00:08,145 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
248
+ 2024-04-26 16:00:09,342 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
249
+ 2024-04-26 16:00:11,344 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
250
+ 2024-04-26 16:00:13,346 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
251
+ 2024-04-26 16:00:13,350 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
252
+ 2024-04-26 16:00:15,348 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
253
+ 2024-04-26 16:00:16,254 DEBUG SenderThread:713738 [sender.py:send():379] send: stats
254
+ 2024-04-26 16:00:17,350 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
255
+ 2024-04-26 16:00:18,400 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
256
+ 2024-04-26 16:00:19,352 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
257
+ 2024-04-26 16:00:20,551 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
258
+ 2024-04-26 16:00:20,552 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
259
+ 2024-04-26 16:00:20,552 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
260
+ 2024-04-26 16:00:21,354 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
261
+ 2024-04-26 16:00:23,356 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
262
+ 2024-04-26 16:00:23,539 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
263
+ 2024-04-26 16:00:25,358 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
264
+ 2024-04-26 16:00:27,360 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
265
+ 2024-04-26 16:00:29,363 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
266
+ 2024-04-26 16:00:29,525 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
267
+ 2024-04-26 16:00:31,365 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
268
+ 2024-04-26 16:00:33,367 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
269
+ 2024-04-26 16:00:34,629 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
270
+ 2024-04-26 16:00:35,369 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
271
+ 2024-04-26 16:00:35,552 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
272
+ 2024-04-26 16:00:35,552 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
273
+ 2024-04-26 16:00:35,553 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
274
+ 2024-04-26 16:00:37,371 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
275
+ 2024-04-26 16:00:38,753 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: partial_history
276
+ 2024-04-26 16:00:38,755 DEBUG SenderThread:713738 [sender.py:send():379] send: history
277
+ 2024-04-26 16:00:38,755 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: summary_record
278
+ 2024-04-26 16:00:38,757 INFO SenderThread:713738 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
279
+ 2024-04-26 16:00:39,374 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
280
+ 2024-04-26 16:00:39,374 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/wandb-summary.json
281
+ 2024-04-26 16:00:39,757 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
282
+ 2024-04-26 16:00:41,376 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
283
+ 2024-04-26 16:00:43,378 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
284
+ 2024-04-26 16:00:44,796 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
285
+ 2024-04-26 16:00:45,380 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
286
+ 2024-04-26 16:00:46,244 DEBUG SenderThread:713738 [sender.py:send():379] send: stats
287
+ 2024-04-26 16:00:47,382 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
288
+ 2024-04-26 16:00:49,384 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
289
+ 2024-04-26 16:00:49,828 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
290
+ 2024-04-26 16:00:50,551 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
291
+ 2024-04-26 16:00:50,552 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
292
+ 2024-04-26 16:00:50,552 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
293
+ 2024-04-26 16:00:51,386 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
294
+ 2024-04-26 16:00:53,388 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
295
+ 2024-04-26 16:00:54,857 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
296
+ 2024-04-26 16:00:55,390 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
297
+ 2024-04-26 16:00:57,392 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
298
+ 2024-04-26 16:00:59,393 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
299
+ 2024-04-26 16:01:00,017 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
300
+ 2024-04-26 16:01:01,396 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
301
+ 2024-04-26 16:01:03,398 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
302
+ 2024-04-26 16:01:05,177 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
303
+ 2024-04-26 16:01:05,400 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
304
+ 2024-04-26 16:01:05,552 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
305
+ 2024-04-26 16:01:05,552 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
306
+ 2024-04-26 16:01:05,552 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
307
+ 2024-04-26 16:01:07,402 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
308
+ 2024-04-26 16:01:09,404 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
309
+ 2024-04-26 16:01:10,447 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
310
+ 2024-04-26 16:01:10,508 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: partial_history
311
+ 2024-04-26 16:01:10,510 DEBUG SenderThread:713738 [sender.py:send():379] send: history
312
+ 2024-04-26 16:01:10,511 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: summary_record
313
+ 2024-04-26 16:01:10,512 INFO SenderThread:713738 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
314
+ 2024-04-26 16:01:11,406 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
315
+ 2024-04-26 16:01:11,406 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/wandb-summary.json
316
+ 2024-04-26 16:01:13,408 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
317
+ 2024-04-26 16:01:15,410 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
318
+ 2024-04-26 16:01:15,586 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
319
+ 2024-04-26 16:01:16,246 DEBUG SenderThread:713738 [sender.py:send():379] send: stats
320
+ 2024-04-26 16:01:17,412 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
321
+ 2024-04-26 16:01:19,414 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
322
+ 2024-04-26 16:01:20,552 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
323
+ 2024-04-26 16:01:20,553 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
324
+ 2024-04-26 16:01:20,553 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
325
+ 2024-04-26 16:01:21,031 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
326
+ 2024-04-26 16:01:21,416 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
327
+ 2024-04-26 16:01:23,418 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
328
+ 2024-04-26 16:01:25,420 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
329
+ 2024-04-26 16:01:26,828 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
330
+ 2024-04-26 16:01:27,422 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
331
+ 2024-04-26 16:01:29,424 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
332
+ 2024-04-26 16:01:31,426 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
333
+ 2024-04-26 16:01:31,991 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
334
+ 2024-04-26 16:01:33,428 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
335
+ 2024-04-26 16:01:35,430 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
336
+ 2024-04-26 16:01:35,552 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
337
+ 2024-04-26 16:01:35,553 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
338
+ 2024-04-26 16:01:35,554 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
339
+ 2024-04-26 16:01:37,138 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
340
+ 2024-04-26 16:01:37,432 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
341
+ 2024-04-26 16:01:39,434 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
342
+ 2024-04-26 16:01:41,436 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
343
+ 2024-04-26 16:01:42,208 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
344
+ 2024-04-26 16:01:42,568 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: partial_history
345
+ 2024-04-26 16:01:42,570 DEBUG SenderThread:713738 [sender.py:send():379] send: history
346
+ 2024-04-26 16:01:42,570 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: summary_record
347
+ 2024-04-26 16:01:42,573 INFO SenderThread:713738 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
348
+ 2024-04-26 16:01:43,439 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
349
+ 2024-04-26 16:01:43,439 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/wandb-summary.json
350
+ 2024-04-26 16:01:45,441 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
351
+ 2024-04-26 16:01:46,260 DEBUG SenderThread:713738 [sender.py:send():379] send: stats
352
+ 2024-04-26 16:01:47,409 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
353
+ 2024-04-26 16:01:47,444 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
354
+ 2024-04-26 16:01:49,446 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
355
+ 2024-04-26 16:01:50,552 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
356
+ 2024-04-26 16:01:50,553 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
357
+ 2024-04-26 16:01:50,553 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
358
+ 2024-04-26 16:01:51,448 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
359
+ 2024-04-26 16:01:52,597 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
360
+ 2024-04-26 16:01:53,450 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
361
+ 2024-04-26 16:01:55,451 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
362
+ 2024-04-26 16:01:57,454 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
363
+ 2024-04-26 16:01:57,688 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
364
+ 2024-04-26 16:01:59,455 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
365
+ 2024-04-26 16:02:01,457 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
366
+ 2024-04-26 16:02:02,736 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
367
+ 2024-04-26 16:02:03,459 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
368
+ 2024-04-26 16:02:05,461 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
369
+ 2024-04-26 16:02:05,552 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
370
+ 2024-04-26 16:02:05,553 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
371
+ 2024-04-26 16:02:05,553 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
372
+ 2024-04-26 16:02:07,463 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
373
+ 2024-04-26 16:02:08,031 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
374
+ 2024-04-26 16:02:09,465 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
375
+ 2024-04-26 16:02:11,467 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
376
+ 2024-04-26 16:02:13,147 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
377
+ 2024-04-26 16:02:13,469 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
378
+ 2024-04-26 16:02:14,469 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: partial_history
379
+ 2024-04-26 16:02:14,472 DEBUG SenderThread:713738 [sender.py:send():379] send: history
380
+ 2024-04-26 16:02:14,472 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: summary_record
381
+ 2024-04-26 16:02:14,473 INFO SenderThread:713738 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
382
+ 2024-04-26 16:02:15,474 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
383
+ 2024-04-26 16:02:15,475 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/wandb-summary.json
384
+ 2024-04-26 16:02:16,250 DEBUG SenderThread:713738 [sender.py:send():379] send: stats
385
+ 2024-04-26 16:02:17,476 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
386
+ 2024-04-26 16:02:18,328 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
387
+ 2024-04-26 16:02:19,478 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
388
+ 2024-04-26 16:02:20,553 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
389
+ 2024-04-26 16:02:20,553 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
390
+ 2024-04-26 16:02:20,553 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
391
+ 2024-04-26 16:02:21,480 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
392
+ 2024-04-26 16:02:22,759 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: partial_history
393
+ 2024-04-26 16:02:22,762 DEBUG SenderThread:713738 [sender.py:send():379] send: history
394
+ 2024-04-26 16:02:22,763 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: summary_record
395
+ 2024-04-26 16:02:22,764 INFO SenderThread:713738 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
396
+ 2024-04-26 16:02:23,483 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
397
+ 2024-04-26 16:02:23,483 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/wandb-summary.json
398
+ 2024-04-26 16:02:23,774 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
399
+ 2024-04-26 16:02:25,485 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
400
+ 2024-04-26 16:02:28,775 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
401
+ 2024-04-26 16:02:34,173 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
402
+ 2024-04-26 16:02:35,496 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
403
+ 2024-04-26 16:02:36,046 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
404
+ 2024-04-26 16:02:36,196 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
405
+ 2024-04-26 16:02:36,197 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
406
+ 2024-04-26 16:02:39,290 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
407
+ 2024-04-26 16:02:44,290 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
408
+ 2024-04-26 16:02:46,253 DEBUG SenderThread:713738 [sender.py:send():379] send: stats
409
+ 2024-04-26 16:02:50,254 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
410
+ 2024-04-26 16:02:50,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
411
+ 2024-04-26 16:02:50,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
412
+ 2024-04-26 16:02:50,704 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
wandb/run-20240426_155715-vljs8xzo/files/output.log CHANGED
@@ -78,3 +78,83 @@
78
  {'eval_loss': 0.6759119629859924, 'eval_runtime': 8.2733, 'eval_samples_per_second': 241.742, 'eval_steps_per_second': 3.868, 'eval_rewards/chosen': 0.0017230990342795849, 'eval_rewards/rejected': -0.03281649947166443, 'eval_rewards/accuracies': 0.62890625, 'eval_rewards/margins': 0.0345395989716053, 'eval_logps/rejected': -407.8036804199219, 'eval_logps/chosen': -423.0196533203125, 'eval_logits/rejected': -3.2565112113952637, 'eval_logits/chosen': -3.313567638397217, 'epoch': 0.21}
79
  [INFO|modeling_utils.py:2598] 2024-04-26 15:59:47,330 >> The model is bigger than the maximum size per checkpoint (5GB) and is going to be split in 2 checkpoint shards. You can find where each parameters has been saved in the index located at ./checkpoint-100/model.safetensors.index.json.
80
  [INFO|tokenization_utils_base.py:2488] 2024-04-26 15:59:47,344 >> tokenizer config file saved in ./checkpoint-100/tokenizer_config.json
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
78
  {'eval_loss': 0.6759119629859924, 'eval_runtime': 8.2733, 'eval_samples_per_second': 241.742, 'eval_steps_per_second': 3.868, 'eval_rewards/chosen': 0.0017230990342795849, 'eval_rewards/rejected': -0.03281649947166443, 'eval_rewards/accuracies': 0.62890625, 'eval_rewards/margins': 0.0345395989716053, 'eval_logps/rejected': -407.8036804199219, 'eval_logps/chosen': -423.0196533203125, 'eval_logits/rejected': -3.2565112113952637, 'eval_logits/chosen': -3.313567638397217, 'epoch': 0.21}
79
  [INFO|modeling_utils.py:2598] 2024-04-26 15:59:47,330 >> The model is bigger than the maximum size per checkpoint (5GB) and is going to be split in 2 checkpoint shards. You can find where each parameters has been saved in the index located at ./checkpoint-100/model.safetensors.index.json.
80
  [INFO|tokenization_utils_base.py:2488] 2024-04-26 15:59:47,344 >> tokenizer config file saved in ./checkpoint-100/tokenizer_config.json
81
+ [INFO|tokenization_utils_base.py:2497] 2024-04-26 15:59:47,384 >> Special tokens file saved in ./checkpoint-100/special_tokens_map.json
82
+ [INFO|tokenization_utils_base.py:2488] 2024-04-26 16:00:07,103 >> tokenizer config file saved in ./tokenizer_config.json
83
+ [INFO|tokenization_utils_base.py:2497] 2024-04-26 16:00:07,105 >> Special tokens file saved in ./special_tokens_map.json
84
+
85
+
86
+
87
+
88
+
89
+
90
+
91
+
92
+
93
+
94
+
95
+
96
+
97
+
98
+
99
+ 26%|██▋ | 126/478 [03:19<07:28, 1.27s/it]
100
+
101
+
102
+
103
+
104
+
105
+
106
+
107
+
108
+
109
+
110
+
111
+
112
+
113
+
114
+
115
+
116
+ 32%|███▏ | 151/478 [03:51<06:56, 1.28s/it]
117
+
118
+
119
+
120
+
121
+
122
+
123
+
124
+
125
+
126
+
127
+
128
+
129
+
130
+
131
+
132
+
133
+ 37%|███▋ | 176/478 [04:23<06:28, 1.29s/it]
134
+
135
+
136
+
137
+
138
+
139
+
140
+
141
+
142
+
143
+
144
+
145
+
146
+
147
+
148
+
149
+ 42%|████▏ | 200/478 [04:54<05:52, 1.27s/it][INFO|trainer.py:3614] 2024-04-26 16:02:14,470 >> ***** Running Evaluation *****
150
+ [INFO|trainer.py:3616] 2024-04-26 16:02:14,470 >> Num examples = 2000
151
+ [INFO|trainer.py:3619] 2024-04-26 16:02:14,470 >> Batch size = 8
152
+ 19%|█▉ | 6/32 [00:01<00:06, 4.22it/s]
153
+
154
+
155
+
156
+ [INFO|configuration_utils.py:471] 2024-04-26 16:02:22,770 >> Configuration saved in ./checkpoint-200/config.json
157
+ [INFO|configuration_utils.py:697] 2024-04-26 16:02:22,773 >> Configuration saved in ./checkpoint-200/generation_config.json
158
+ {'eval_loss': 0.6533502340316772, 'eval_runtime': 8.2763, 'eval_samples_per_second': 241.653, 'eval_steps_per_second': 3.866, 'eval_rewards/chosen': -0.06664139777421951, 'eval_rewards/rejected': -0.16173213720321655, 'eval_rewards/accuracies': 0.64453125, 'eval_rewards/margins': 0.09509073942899704, 'eval_logps/rejected': -420.6952209472656, 'eval_logps/chosen': -429.85614013671875, 'eval_logits/rejected': -3.2240023612976074, 'eval_logits/chosen': -3.2767982482910156, 'epoch': 0.42}
159
+ [INFO|modeling_utils.py:2598] 2024-04-26 16:02:32,167 >> The model is bigger than the maximum size per checkpoint (5GB) and is going to be split in 2 checkpoint shards. You can find where each parameters has been saved in the index located at ./checkpoint-200/model.safetensors.index.json.
160
+ [INFO|tokenization_utils_base.py:2488] 2024-04-26 16:02:32,170 >> tokenizer config file saved in ./checkpoint-200/tokenizer_config.json
wandb/run-20240426_155715-vljs8xzo/files/wandb-summary.json CHANGED
@@ -1 +1 @@
1
- {"train/loss": 0.6795, "train/grad_norm": 8.75236988067627, "train/learning_rate": 4.821741763807186e-07, "train/rewards/chosen": 0.003723819274455309, "train/rewards/rejected": -0.02431883104145527, "train/rewards/accuracies": 0.612500011920929, "train/rewards/margins": 0.02804265171289444, "train/logps/rejected": -389.50421142578125, "train/logps/chosen": -426.1629638671875, "train/logits/rejected": -3.2092175483703613, "train/logits/chosen": -3.2598507404327393, "train/epoch": 0.20920502092050208, "train/global_step": 100, "_timestamp": 1714147177.6973403, "_runtime": 141.99647116661072, "_step": 5, "eval/loss": 0.6759119629859924, "eval/runtime": 8.2733, "eval/samples_per_second": 241.742, "eval/steps_per_second": 3.868, "eval/rewards/chosen": 0.0017230990342795849, "eval/rewards/rejected": -0.03281649947166443, "eval/rewards/accuracies": 0.62890625, "eval/rewards/margins": 0.0345395989716053, "eval/logps/rejected": -407.8036804199219, "eval/logps/chosen": -423.0196533203125, "eval/logits/rejected": -3.2565112113952637, "eval/logits/chosen": -3.313567638397217}
 
1
+ {"train/loss": 0.6584, "train/grad_norm": 9.230700492858887, "train/learning_rate": 3.610497133404795e-07, "train/rewards/chosen": -0.06991340965032578, "train/rewards/rejected": -0.15435244143009186, "train/rewards/accuracies": 0.6349999904632568, "train/rewards/margins": 0.08443903923034668, "train/logps/rejected": -396.62774658203125, "train/logps/chosen": -431.6617126464844, "train/logits/rejected": -3.1550867557525635, "train/logits/chosen": -3.23995304107666, "train/epoch": 0.41841004184100417, "train/global_step": 200, "_timestamp": 1714147342.7592435, "_runtime": 307.0583744049072, "_step": 10, "eval/loss": 0.6533502340316772, "eval/runtime": 8.2763, "eval/samples_per_second": 241.653, "eval/steps_per_second": 3.866, "eval/rewards/chosen": -0.06664139777421951, "eval/rewards/rejected": -0.16173213720321655, "eval/rewards/accuracies": 0.64453125, "eval/rewards/margins": 0.09509073942899704, "eval/logps/rejected": -420.6952209472656, "eval/logps/chosen": -429.85614013671875, "eval/logits/rejected": -3.2240023612976074, "eval/logits/chosen": -3.2767982482910156}
wandb/run-20240426_155715-vljs8xzo/logs/debug-internal.log CHANGED
@@ -244,3 +244,169 @@
244
  2024-04-26 16:00:05,958 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
245
  2024-04-26 16:00:06,058 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
246
  2024-04-26 16:00:06,059 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
244
  2024-04-26 16:00:05,958 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
245
  2024-04-26 16:00:06,058 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
246
  2024-04-26 16:00:06,059 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
247
+ 2024-04-26 16:00:08,145 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
248
+ 2024-04-26 16:00:09,342 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
249
+ 2024-04-26 16:00:11,344 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
250
+ 2024-04-26 16:00:13,346 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
251
+ 2024-04-26 16:00:13,350 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
252
+ 2024-04-26 16:00:15,348 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
253
+ 2024-04-26 16:00:16,254 DEBUG SenderThread:713738 [sender.py:send():379] send: stats
254
+ 2024-04-26 16:00:17,350 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
255
+ 2024-04-26 16:00:18,400 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
256
+ 2024-04-26 16:00:19,352 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
257
+ 2024-04-26 16:00:20,551 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
258
+ 2024-04-26 16:00:20,552 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
259
+ 2024-04-26 16:00:20,552 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
260
+ 2024-04-26 16:00:21,354 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
261
+ 2024-04-26 16:00:23,356 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
262
+ 2024-04-26 16:00:23,539 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
263
+ 2024-04-26 16:00:25,358 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
264
+ 2024-04-26 16:00:27,360 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
265
+ 2024-04-26 16:00:29,363 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
266
+ 2024-04-26 16:00:29,525 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
267
+ 2024-04-26 16:00:31,365 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
268
+ 2024-04-26 16:00:33,367 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
269
+ 2024-04-26 16:00:34,629 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
270
+ 2024-04-26 16:00:35,369 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
271
+ 2024-04-26 16:00:35,552 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
272
+ 2024-04-26 16:00:35,552 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
273
+ 2024-04-26 16:00:35,553 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
274
+ 2024-04-26 16:00:37,371 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
275
+ 2024-04-26 16:00:38,753 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: partial_history
276
+ 2024-04-26 16:00:38,755 DEBUG SenderThread:713738 [sender.py:send():379] send: history
277
+ 2024-04-26 16:00:38,755 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: summary_record
278
+ 2024-04-26 16:00:38,757 INFO SenderThread:713738 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
279
+ 2024-04-26 16:00:39,374 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
280
+ 2024-04-26 16:00:39,374 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/wandb-summary.json
281
+ 2024-04-26 16:00:39,757 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
282
+ 2024-04-26 16:00:41,376 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
283
+ 2024-04-26 16:00:43,378 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
284
+ 2024-04-26 16:00:44,796 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
285
+ 2024-04-26 16:00:45,380 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
286
+ 2024-04-26 16:00:46,244 DEBUG SenderThread:713738 [sender.py:send():379] send: stats
287
+ 2024-04-26 16:00:47,382 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
288
+ 2024-04-26 16:00:49,384 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
289
+ 2024-04-26 16:00:49,828 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
290
+ 2024-04-26 16:00:50,551 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
291
+ 2024-04-26 16:00:50,552 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
292
+ 2024-04-26 16:00:50,552 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
293
+ 2024-04-26 16:00:51,386 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
294
+ 2024-04-26 16:00:53,388 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
295
+ 2024-04-26 16:00:54,857 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
296
+ 2024-04-26 16:00:55,390 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
297
+ 2024-04-26 16:00:57,392 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
298
+ 2024-04-26 16:00:59,393 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
299
+ 2024-04-26 16:01:00,017 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
300
+ 2024-04-26 16:01:01,396 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
301
+ 2024-04-26 16:01:03,398 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
302
+ 2024-04-26 16:01:05,177 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
303
+ 2024-04-26 16:01:05,400 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
304
+ 2024-04-26 16:01:05,552 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
305
+ 2024-04-26 16:01:05,552 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
306
+ 2024-04-26 16:01:05,552 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
307
+ 2024-04-26 16:01:07,402 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
308
+ 2024-04-26 16:01:09,404 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
309
+ 2024-04-26 16:01:10,447 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
310
+ 2024-04-26 16:01:10,508 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: partial_history
311
+ 2024-04-26 16:01:10,510 DEBUG SenderThread:713738 [sender.py:send():379] send: history
312
+ 2024-04-26 16:01:10,511 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: summary_record
313
+ 2024-04-26 16:01:10,512 INFO SenderThread:713738 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
314
+ 2024-04-26 16:01:11,406 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
315
+ 2024-04-26 16:01:11,406 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/wandb-summary.json
316
+ 2024-04-26 16:01:13,408 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
317
+ 2024-04-26 16:01:15,410 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
318
+ 2024-04-26 16:01:15,586 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
319
+ 2024-04-26 16:01:16,246 DEBUG SenderThread:713738 [sender.py:send():379] send: stats
320
+ 2024-04-26 16:01:17,412 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
321
+ 2024-04-26 16:01:19,414 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
322
+ 2024-04-26 16:01:20,552 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
323
+ 2024-04-26 16:01:20,553 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
324
+ 2024-04-26 16:01:20,553 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
325
+ 2024-04-26 16:01:21,031 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
326
+ 2024-04-26 16:01:21,416 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
327
+ 2024-04-26 16:01:23,418 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
328
+ 2024-04-26 16:01:25,420 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
329
+ 2024-04-26 16:01:26,828 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
330
+ 2024-04-26 16:01:27,422 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
331
+ 2024-04-26 16:01:29,424 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
332
+ 2024-04-26 16:01:31,426 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
333
+ 2024-04-26 16:01:31,991 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
334
+ 2024-04-26 16:01:33,428 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
335
+ 2024-04-26 16:01:35,430 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
336
+ 2024-04-26 16:01:35,552 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
337
+ 2024-04-26 16:01:35,553 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
338
+ 2024-04-26 16:01:35,554 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
339
+ 2024-04-26 16:01:37,138 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
340
+ 2024-04-26 16:01:37,432 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
341
+ 2024-04-26 16:01:39,434 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
342
+ 2024-04-26 16:01:41,436 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
343
+ 2024-04-26 16:01:42,208 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
344
+ 2024-04-26 16:01:42,568 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: partial_history
345
+ 2024-04-26 16:01:42,570 DEBUG SenderThread:713738 [sender.py:send():379] send: history
346
+ 2024-04-26 16:01:42,570 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: summary_record
347
+ 2024-04-26 16:01:42,573 INFO SenderThread:713738 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
348
+ 2024-04-26 16:01:43,439 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
349
+ 2024-04-26 16:01:43,439 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/wandb-summary.json
350
+ 2024-04-26 16:01:45,441 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
351
+ 2024-04-26 16:01:46,260 DEBUG SenderThread:713738 [sender.py:send():379] send: stats
352
+ 2024-04-26 16:01:47,409 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
353
+ 2024-04-26 16:01:47,444 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
354
+ 2024-04-26 16:01:49,446 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
355
+ 2024-04-26 16:01:50,552 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
356
+ 2024-04-26 16:01:50,553 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
357
+ 2024-04-26 16:01:50,553 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
358
+ 2024-04-26 16:01:51,448 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
359
+ 2024-04-26 16:01:52,597 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
360
+ 2024-04-26 16:01:53,450 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
361
+ 2024-04-26 16:01:55,451 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
362
+ 2024-04-26 16:01:57,454 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
363
+ 2024-04-26 16:01:57,688 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
364
+ 2024-04-26 16:01:59,455 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
365
+ 2024-04-26 16:02:01,457 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
366
+ 2024-04-26 16:02:02,736 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
367
+ 2024-04-26 16:02:03,459 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
368
+ 2024-04-26 16:02:05,461 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
369
+ 2024-04-26 16:02:05,552 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
370
+ 2024-04-26 16:02:05,553 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
371
+ 2024-04-26 16:02:05,553 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
372
+ 2024-04-26 16:02:07,463 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
373
+ 2024-04-26 16:02:08,031 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
374
+ 2024-04-26 16:02:09,465 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
375
+ 2024-04-26 16:02:11,467 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
376
+ 2024-04-26 16:02:13,147 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
377
+ 2024-04-26 16:02:13,469 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
378
+ 2024-04-26 16:02:14,469 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: partial_history
379
+ 2024-04-26 16:02:14,472 DEBUG SenderThread:713738 [sender.py:send():379] send: history
380
+ 2024-04-26 16:02:14,472 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: summary_record
381
+ 2024-04-26 16:02:14,473 INFO SenderThread:713738 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
382
+ 2024-04-26 16:02:15,474 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
383
+ 2024-04-26 16:02:15,475 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/wandb-summary.json
384
+ 2024-04-26 16:02:16,250 DEBUG SenderThread:713738 [sender.py:send():379] send: stats
385
+ 2024-04-26 16:02:17,476 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
386
+ 2024-04-26 16:02:18,328 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
387
+ 2024-04-26 16:02:19,478 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
388
+ 2024-04-26 16:02:20,553 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
389
+ 2024-04-26 16:02:20,553 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
390
+ 2024-04-26 16:02:20,553 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
391
+ 2024-04-26 16:02:21,480 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
392
+ 2024-04-26 16:02:22,759 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: partial_history
393
+ 2024-04-26 16:02:22,762 DEBUG SenderThread:713738 [sender.py:send():379] send: history
394
+ 2024-04-26 16:02:22,763 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: summary_record
395
+ 2024-04-26 16:02:22,764 INFO SenderThread:713738 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
396
+ 2024-04-26 16:02:23,483 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
397
+ 2024-04-26 16:02:23,483 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/wandb-summary.json
398
+ 2024-04-26 16:02:23,774 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
399
+ 2024-04-26 16:02:25,485 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
400
+ 2024-04-26 16:02:28,775 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
401
+ 2024-04-26 16:02:34,173 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
402
+ 2024-04-26 16:02:35,496 INFO Thread-12 :713738 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback/wandb/run-20240426_155715-vljs8xzo/files/output.log
403
+ 2024-04-26 16:02:36,046 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
404
+ 2024-04-26 16:02:36,196 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
405
+ 2024-04-26 16:02:36,197 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
406
+ 2024-04-26 16:02:39,290 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
407
+ 2024-04-26 16:02:44,290 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
408
+ 2024-04-26 16:02:46,253 DEBUG SenderThread:713738 [sender.py:send():379] send: stats
409
+ 2024-04-26 16:02:50,254 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: status_report
410
+ 2024-04-26 16:02:50,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: stop_status
411
+ 2024-04-26 16:02:50,703 DEBUG HandlerThread:713738 [handler.py:handle_request():146] handle_request: internal_messages
412
+ 2024-04-26 16:02:50,704 DEBUG SenderThread:713738 [sender.py:send_request():406] send_request: stop_status
wandb/run-20240426_155715-vljs8xzo/run-vljs8xzo.wandb CHANGED
Binary files a/wandb/run-20240426_155715-vljs8xzo/run-vljs8xzo.wandb and b/wandb/run-20240426_155715-vljs8xzo/run-vljs8xzo.wandb differ