|
2021-05-31 17:50:29,582 INFO __main__ Namespace(adjust_lr=False, config='torchdistill/configs/sample/glue/rte/kd/bert_base_uncased_from_bert_large_uncased.yaml', log='log/glue/rte/kd/bert_base_uncased_from_bert_large_uncased.txt', private_output='leaderboard/glue/kd/bert_base_uncased_from_bert_large_uncased/', seed=None, student_only=False, task_name='rte', test_only=False, world_size=1) |
|
2021-05-31 17:50:29,613 INFO __main__ Distributed environment: NO |
|
Num processes: 1 |
|
Process index: 0 |
|
Local process index: 0 |
|
Device: cuda |
|
Use FP16 precision: True |
|
|
|
2021-05-31 17:50:39,943 WARNING datasets.builder Reusing dataset glue (/root/.cache/huggingface/datasets/glue/rte/1.0.0/dacbe3125aa31d7f70367a07a8a9e72a5a0bfeb5fc42e75c9db75b96da6053ad) |
|
2021-05-31 17:50:41,587 INFO __main__ Start training |
|
2021-05-31 17:50:41,587 INFO torchdistill.models.util [teacher model] |
|
2021-05-31 17:50:41,587 INFO torchdistill.models.util Using the original teacher model |
|
2021-05-31 17:50:41,588 INFO torchdistill.models.util [student model] |
|
2021-05-31 17:50:41,588 INFO torchdistill.models.util Using the original student model |
|
2021-05-31 17:50:41,588 INFO torchdistill.core.distillation Loss = 1.0 * OrgLoss |
|
2021-05-31 17:50:41,588 INFO torchdistill.core.distillation Freezing the whole teacher model |
|
2021-05-31 17:50:45,473 INFO torchdistill.misc.log Epoch: [0] [ 0/78] eta: 0:01:02 lr: 9.957264957264958e-05 sample/s: 5.016064534395792 loss: 0.0097 (0.0097) time: 0.8039 data: 0.0065 max mem: 3804 |
|
2021-05-31 17:51:25,664 INFO torchdistill.misc.log Epoch: [0] [50/78] eta: 0:00:22 lr: 7.820512820512821e-05 sample/s: 5.000885583822104 loss: 0.0065 (0.0081) time: 0.8035 data: 0.0049 max mem: 5115 |
|
2021-05-31 17:51:47,044 INFO torchdistill.misc.log Epoch: [0] Total time: 0:01:02 |
|
2021-05-31 17:51:48,268 INFO /usr/local/lib/python3.7/dist-packages/datasets/metric.py Removing /root/.cache/huggingface/metrics/glue/rte/default_experiment-1-0.arrow |
|
2021-05-31 17:51:48,268 INFO __main__ Validation: accuracy = 0.6570397111913358 |
|
2021-05-31 17:51:48,268 INFO __main__ Updating ckpt at ./resource/ckpt/glue/rte/kd/rte-bert-base-uncased_from_bert-large-uncased |
|
2021-05-31 17:51:50,012 INFO torchdistill.misc.log Epoch: [1] [ 0/78] eta: 0:01:02 lr: 6.623931623931624e-05 sample/s: 5.007037577218317 loss: 0.0042 (0.0042) time: 0.8055 data: 0.0066 max mem: 5115 |
|
2021-05-31 17:52:30,204 INFO torchdistill.misc.log Epoch: [1] [50/78] eta: 0:00:22 lr: 4.4871794871794874e-05 sample/s: 4.994946466605415 loss: 0.0033 (0.0032) time: 0.8037 data: 0.0050 max mem: 5115 |
|
2021-05-31 17:52:51,720 INFO torchdistill.misc.log Epoch: [1] Total time: 0:01:02 |
|
2021-05-31 17:52:52,943 INFO /usr/local/lib/python3.7/dist-packages/datasets/metric.py Removing /root/.cache/huggingface/metrics/glue/rte/default_experiment-1-0.arrow |
|
2021-05-31 17:52:52,943 INFO __main__ Validation: accuracy = 0.6859205776173285 |
|
2021-05-31 17:52:52,943 INFO __main__ Updating ckpt at ./resource/ckpt/glue/rte/kd/rte-bert-base-uncased_from_bert-large-uncased |
|
2021-05-31 17:52:54,985 INFO torchdistill.misc.log Epoch: [2] [ 0/78] eta: 0:01:03 lr: 3.290598290598291e-05 sample/s: 5.004806067380402 loss: 0.0014 (0.0014) time: 0.8145 data: 0.0153 max mem: 5115 |
|
2021-05-31 17:53:35,181 INFO torchdistill.misc.log Epoch: [2] [50/78] eta: 0:00:22 lr: 1.153846153846154e-05 sample/s: 5.00735140278351 loss: 0.0012 (0.0013) time: 0.8039 data: 0.0051 max mem: 5115 |
|
2021-05-31 17:53:56,746 INFO torchdistill.misc.log Epoch: [2] Total time: 0:01:02 |
|
2021-05-31 17:53:57,970 INFO /usr/local/lib/python3.7/dist-packages/datasets/metric.py Removing /root/.cache/huggingface/metrics/glue/rte/default_experiment-1-0.arrow |
|
2021-05-31 17:53:57,970 INFO __main__ Validation: accuracy = 0.6823104693140795 |
|
2021-05-31 17:53:58,006 INFO __main__ [Teacher: bert-large-uncased] |
|
2021-05-31 17:54:01,530 INFO /usr/local/lib/python3.7/dist-packages/datasets/metric.py Removing /root/.cache/huggingface/metrics/glue/rte/default_experiment-1-0.arrow |
|
2021-05-31 17:54:01,530 INFO __main__ Test: accuracy = 0.740072202166065 |
|
2021-05-31 17:54:04,735 INFO __main__ [Student: bert-base-uncased] |
|
2021-05-31 17:54:05,975 INFO /usr/local/lib/python3.7/dist-packages/datasets/metric.py Removing /root/.cache/huggingface/metrics/glue/rte/default_experiment-1-0.arrow |
|
2021-05-31 17:54:05,976 INFO __main__ Test: accuracy = 0.6859205776173285 |
|
2021-05-31 17:54:05,976 INFO __main__ Start prediction for private dataset(s) |
|
2021-05-31 17:54:05,977 INFO __main__ rte/test: 3000 samples |
|
|
