kertob commited on
Commit
d1a8447
1 Parent(s): a426113

Training in progress, step 240

Browse files
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:6f50e069a5bac20c4223217e95665e7fc1af2284195e6dcee13756a23f4bc109
3
  size 261131840
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:549dd5c1b8bdfea9f824847ddc0b265ef15375f257941511b3fa72b375441419
3
  size 261131840
runs/Feb14_12-54-25_f4b889916ade/events.out.tfevents.1707915266.f4b889916ade.34240.0 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:3f6321a9520d3020d536b9b74c745936f5de877d84c0ee05f1f74a0dff5d9403
3
- size 9041
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ecaa17b829e971b3f8c72a3477a562f7fd436a5d9c2a286cb06caed415b9fff7
3
+ size 9198
wandb/debug-internal.log CHANGED
@@ -998,3 +998,41 @@
998
  2024-02-14 13:26:30,885 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: stop_status
999
  2024-02-14 13:26:30,885 DEBUG SenderThread:35088 [sender.py:send_request():409] send_request: stop_status
1000
  2024-02-14 13:26:30,908 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: internal_messages
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
998
  2024-02-14 13:26:30,885 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: stop_status
999
  2024-02-14 13:26:30,885 DEBUG SenderThread:35088 [sender.py:send_request():409] send_request: stop_status
1000
  2024-02-14 13:26:30,908 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: internal_messages
1001
+ 2024-02-14 13:26:35,994 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
1002
+ 2024-02-14 13:26:40,995 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
1003
+ 2024-02-14 13:26:45,885 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: stop_status
1004
+ 2024-02-14 13:26:45,886 DEBUG SenderThread:35088 [sender.py:send_request():409] send_request: stop_status
1005
+ 2024-02-14 13:26:45,926 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: internal_messages
1006
+ 2024-02-14 13:26:46,013 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
1007
+ 2024-02-14 13:26:51,014 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
1008
+ 2024-02-14 13:26:56,015 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
1009
+ 2024-02-14 13:27:00,881 DEBUG SenderThread:35088 [sender.py:send():382] send: stats
1010
+ 2024-02-14 13:27:00,885 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: stop_status
1011
+ 2024-02-14 13:27:00,886 DEBUG SenderThread:35088 [sender.py:send_request():409] send_request: stop_status
1012
+ 2024-02-14 13:27:00,926 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: internal_messages
1013
+ 2024-02-14 13:27:02,002 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
1014
+ 2024-02-14 13:27:07,003 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
1015
+ 2024-02-14 13:27:12,004 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
1016
+ 2024-02-14 13:27:15,885 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: stop_status
1017
+ 2024-02-14 13:27:15,886 DEBUG SenderThread:35088 [sender.py:send_request():409] send_request: stop_status
1018
+ 2024-02-14 13:27:15,926 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: internal_messages
1019
+ 2024-02-14 13:27:17,011 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
1020
+ 2024-02-14 13:27:22,012 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
1021
+ 2024-02-14 13:27:27,013 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
1022
+ 2024-02-14 13:27:30,877 DEBUG SenderThread:35088 [sender.py:send():382] send: stats
1023
+ 2024-02-14 13:27:30,885 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: stop_status
1024
+ 2024-02-14 13:27:30,886 DEBUG SenderThread:35088 [sender.py:send_request():409] send_request: stop_status
1025
+ 2024-02-14 13:27:30,926 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: internal_messages
1026
+ 2024-02-14 13:27:32,039 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
1027
+ 2024-02-14 13:27:37,040 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
1028
+ 2024-02-14 13:27:42,041 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
1029
+ 2024-02-14 13:27:45,886 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: stop_status
1030
+ 2024-02-14 13:27:45,886 DEBUG SenderThread:35088 [sender.py:send_request():409] send_request: stop_status
1031
+ 2024-02-14 13:27:45,908 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: internal_messages
1032
+ 2024-02-14 13:27:46,323 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: partial_history
1033
+ 2024-02-14 13:27:46,325 DEBUG SenderThread:35088 [sender.py:send():382] send: history
1034
+ 2024-02-14 13:27:46,325 DEBUG SenderThread:35088 [sender.py:send_request():409] send_request: summary_record
1035
+ 2024-02-14 13:27:46,327 INFO SenderThread:35088 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
1036
+ 2024-02-14 13:27:46,334 INFO Thread-12 :35088 [dir_watcher.py:_on_file_modified():288] file/dir modified: /content/wandb/run-20240214_125429-fe3s7gan/files/wandb-summary.json
1037
+ 2024-02-14 13:27:47,327 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
1038
+ 2024-02-14 13:27:52,328 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
wandb/run-20240214_125429-fe3s7gan/files/wandb-summary.json CHANGED
@@ -1 +1 @@
1
- {"train/loss": 0.0905, "train/learning_rate": 3.3516446053363015e-06, "train/epoch": 18.4, "train/global_step": 230, "_timestamp": 1707917182.2902603, "_runtime": 1912.9095392227173, "_step": 22}
 
1
+ {"train/loss": 0.0889, "train/learning_rate": 8.41451353233369e-07, "train/epoch": 19.2, "train/global_step": 240, "_timestamp": 1707917266.3233232, "_runtime": 1996.9426021575928, "_step": 23}
wandb/run-20240214_125429-fe3s7gan/logs/debug-internal.log CHANGED
@@ -998,3 +998,41 @@
998
  2024-02-14 13:26:30,885 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: stop_status
999
  2024-02-14 13:26:30,885 DEBUG SenderThread:35088 [sender.py:send_request():409] send_request: stop_status
1000
  2024-02-14 13:26:30,908 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: internal_messages
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
998
  2024-02-14 13:26:30,885 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: stop_status
999
  2024-02-14 13:26:30,885 DEBUG SenderThread:35088 [sender.py:send_request():409] send_request: stop_status
1000
  2024-02-14 13:26:30,908 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: internal_messages
1001
+ 2024-02-14 13:26:35,994 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
1002
+ 2024-02-14 13:26:40,995 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
1003
+ 2024-02-14 13:26:45,885 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: stop_status
1004
+ 2024-02-14 13:26:45,886 DEBUG SenderThread:35088 [sender.py:send_request():409] send_request: stop_status
1005
+ 2024-02-14 13:26:45,926 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: internal_messages
1006
+ 2024-02-14 13:26:46,013 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
1007
+ 2024-02-14 13:26:51,014 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
1008
+ 2024-02-14 13:26:56,015 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
1009
+ 2024-02-14 13:27:00,881 DEBUG SenderThread:35088 [sender.py:send():382] send: stats
1010
+ 2024-02-14 13:27:00,885 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: stop_status
1011
+ 2024-02-14 13:27:00,886 DEBUG SenderThread:35088 [sender.py:send_request():409] send_request: stop_status
1012
+ 2024-02-14 13:27:00,926 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: internal_messages
1013
+ 2024-02-14 13:27:02,002 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
1014
+ 2024-02-14 13:27:07,003 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
1015
+ 2024-02-14 13:27:12,004 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
1016
+ 2024-02-14 13:27:15,885 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: stop_status
1017
+ 2024-02-14 13:27:15,886 DEBUG SenderThread:35088 [sender.py:send_request():409] send_request: stop_status
1018
+ 2024-02-14 13:27:15,926 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: internal_messages
1019
+ 2024-02-14 13:27:17,011 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
1020
+ 2024-02-14 13:27:22,012 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
1021
+ 2024-02-14 13:27:27,013 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
1022
+ 2024-02-14 13:27:30,877 DEBUG SenderThread:35088 [sender.py:send():382] send: stats
1023
+ 2024-02-14 13:27:30,885 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: stop_status
1024
+ 2024-02-14 13:27:30,886 DEBUG SenderThread:35088 [sender.py:send_request():409] send_request: stop_status
1025
+ 2024-02-14 13:27:30,926 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: internal_messages
1026
+ 2024-02-14 13:27:32,039 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
1027
+ 2024-02-14 13:27:37,040 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
1028
+ 2024-02-14 13:27:42,041 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
1029
+ 2024-02-14 13:27:45,886 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: stop_status
1030
+ 2024-02-14 13:27:45,886 DEBUG SenderThread:35088 [sender.py:send_request():409] send_request: stop_status
1031
+ 2024-02-14 13:27:45,908 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: internal_messages
1032
+ 2024-02-14 13:27:46,323 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: partial_history
1033
+ 2024-02-14 13:27:46,325 DEBUG SenderThread:35088 [sender.py:send():382] send: history
1034
+ 2024-02-14 13:27:46,325 DEBUG SenderThread:35088 [sender.py:send_request():409] send_request: summary_record
1035
+ 2024-02-14 13:27:46,327 INFO SenderThread:35088 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
1036
+ 2024-02-14 13:27:46,334 INFO Thread-12 :35088 [dir_watcher.py:_on_file_modified():288] file/dir modified: /content/wandb/run-20240214_125429-fe3s7gan/files/wandb-summary.json
1037
+ 2024-02-14 13:27:47,327 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
1038
+ 2024-02-14 13:27:52,328 DEBUG HandlerThread:35088 [handler.py:handle_request():146] handle_request: status_report
wandb/run-20240214_125429-fe3s7gan/run-fe3s7gan.wandb CHANGED
Binary files a/wandb/run-20240214_125429-fe3s7gan/run-fe3s7gan.wandb and b/wandb/run-20240214_125429-fe3s7gan/run-fe3s7gan.wandb differ