WARNING:torch.distributed.run:
*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
*****************************************
[2024-01-04 16:13:01,034] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-04 16:13:01,067] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-04 16:13:01,297] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-04 16:13:01,308] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-04 16:13:01,368] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-04 16:13:01,402] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-04 16:13:01,404] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-04 16:13:01,409] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-04 16:13:21,805] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-01-04 16:13:21,805] [INFO] [comm.py:668:init_distributed] Initializing TorchBackend in DeepSpeed with backend nccl
[2024-01-04 16:13:21,905] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-01-04 16:13:22,047] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-01-04 16:13:22,330] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-01-04 16:13:22,373] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-01-04 16:13:22,383] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-01-04 16:13:22,391] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-01-04 16:13:22,409] [INFO] [comm.py:637:init_distributed] cdb=None
01/04/2024 16:14:21 - WARNING - __main__ - Process rank: 5, device: cuda:5, n_gpu: 1distributed training: True, 16-bits training: False
01/04/2024 16:14:21 - WARNING - __main__ - Process rank: 1, device: cuda:1, n_gpu: 1distributed training: True, 16-bits training: False
01/04/2024 16:14:21 - WARNING - __main__ - Process rank: 4, device: cuda:4, n_gpu: 1distributed training: True, 16-bits training: False
01/04/2024 16:14:21 - WARNING - __main__ - Process rank: 0, device: cuda:0, n_gpu: 1distributed training: True, 16-bits training: False
01/04/2024 16:14:21 - INFO - __main__ - Training/evaluation parameters TrainingArguments(
_n_gpu=1,
adafactor=False,
adam_beta1=0.9,
adam_beta2=0.999,
adam_epsilon=1e-08,
auto_find_batch_size=False,
bf16=True,
bf16_full_eval=True,
data_seed=None,
dataloader_drop_last=False,
dataloader_num_workers=8,
dataloader_pin_memory=True,
ddp_bucket_cap_mb=None,
ddp_find_unused_parameters=None,
ddp_timeout=72000,
debug=[],
deepspeed=train/deepspeed_config_zero2.json,
disable_tqdm=False,
do_eval=False,
do_predict=False,
do_train=True,
eval_accumulation_steps=None,
eval_delay=0,
eval_steps=None,
evaluation_strategy=no,
fp16=False,
fp16_backend=auto,
fp16_full_eval=False,
fp16_opt_level=O1,
fsdp=[],
fsdp_config={'fsdp_min_num_params': 0, 'xla': False, 'xla_fsdp_grad_ckpt': False},
fsdp_min_num_params=0,
fsdp_transformer_layer_cls_to_wrap=None,
full_determinism=False,
gradient_accumulation_steps=4,
gradient_checkpointing=True,
greater_is_better=None,
group_by_length=False,
half_precision_backend=auto,
hub_model_id=None,
hub_private_repo=False,
hub_strategy=every_save,
hub_token=<HUB_TOKEN>,
ignore_data_skip=False,
include_inputs_for_metrics=False,
jit_mode_eval=False,
label_names=None,
label_smoothing_factor=0.0,
learning_rate=2e-05,
length_column_name=length,
load_best_model_at_end=False,
local_rank=0,
log_level=passive,
log_level_replica=warning,
log_on_each_node=True,
logging_dir=/apdcephfs/share_733425/vinnylywang/jianhuipang/LLMs4MT/model/newptmodel-llms4mt-en2he-32a100/llama2-sfton-50000-bitexts-and-alpacagpt4/runs/Jan04_16-13-00_ts-523333936ff240a8b0a2237a60ca87ec-launcher,
logging_first_step=False,
logging_nan_inf_filter=True,
logging_steps=10,
logging_strategy=steps,
lr_scheduler_type=cosine,
max_grad_norm=1.0,
max_steps=-1,
metric_for_best_model=None,
mp_parameters=,
no_cuda=False,
num_train_epochs=3.0,
optim=adamw_hf,
optim_args=None,
output_dir=/apdcephfs/share_733425/vinnylywang/jianhuipang/LLMs4MT/model/newptmodel-llms4mt-en2he-32a100/llama2-sfton-50000-bitexts-and-alpacagpt4,
overwrite_output_dir=False,
past_index=-1,
per_device_eval_batch_size=2,
per_device_train_batch_size=48,
prediction_loss_only=False,
push_to_hub=False,
push_to_hub_model_id=None,
push_to_hub_organization=None,
push_to_hub_token=<PUSH_TO_HUB_TOKEN>,
ray_scope=last,
remove_unused_columns=True,
report_to=['tensorboard'],
resume_from_checkpoint=None,
run_name=/apdcephfs/share_733425/vinnylywang/jianhuipang/LLMs4MT/model/newptmodel-llms4mt-en2he-32a100/llama2-sfton-50000-bitexts-and-alpacagpt4,
save_on_each_node=False,
save_steps=500,
save_strategy=steps,
save_total_limit=1,
seed=34,
sharded_ddp=[],
skip_memory_metrics=True,
tf32=None,
torch_compile=False,
torch_compile_backend=None,
torch_compile_mode=None,
torchdynamo=None,
tpu_metrics_debug=False,
tpu_num_cores=None,
use_ipex=False,
use_legacy_prediction_loop=False,
use_mps_device=False,
warmup_ratio=0.03,
warmup_steps=0,
weight_decay=0.0,
xpu_backend=None,
)
Traceback (most recent call last):
  File "transformers/examples/pytorch/language-modeling/run_clm_llms_splittrain.py", line 739, in <module>
Traceback (most recent call last):
  File "transformers/examples/pytorch/language-modeling/run_clm_llms_splittrain.py", line 739, in <module>
Traceback (most recent call last):
  File "transformers/examples/pytorch/language-modeling/run_clm_llms_splittrain.py", line 739, in <module>
Traceback (most recent call last):
  File "transformers/examples/pytorch/language-modeling/run_clm_llms_splittrain.py", line 739, in <module>
    main()
  File "transformers/examples/pytorch/language-modeling/run_clm_llms_splittrain.py", line 282, in main
    main()
  File "transformers/examples/pytorch/language-modeling/run_clm_llms_splittrain.py", line 282, in main
    main()
  File "transformers/examples/pytorch/language-modeling/run_clm_llms_splittrain.py", line 282, in main
    main()
  File "transformers/examples/pytorch/language-modeling/run_clm_llms_splittrain.py", line 282, in main
    raise ValueError(
ValueError: Output directory (/apdcephfs/share_733425/vinnylywang/jianhuipang/LLMs4MT/model/newptmodel-llms4mt-en2he-32a100/llama2-sfton-50000-bitexts-and-alpacagpt4) already exists and is not empty. Use --overwrite_output_dir to overcome.
    raise ValueError(
ValueError: Output directory (/apdcephfs/share_733425/vinnylywang/jianhuipang/LLMs4MT/model/newptmodel-llms4mt-en2he-32a100/llama2-sfton-50000-bitexts-and-alpacagpt4) already exists and is not empty. Use --overwrite_output_dir to overcome.
    raise ValueError(
    raise ValueError(
ValueError: Output directory (/apdcephfs/share_733425/vinnylywang/jianhuipang/LLMs4MT/model/newptmodel-llms4mt-en2he-32a100/llama2-sfton-50000-bitexts-and-alpacagpt4) already exists and is not empty. Use --overwrite_output_dir to overcome.
ValueError: Output directory (/apdcephfs/share_733425/vinnylywang/jianhuipang/LLMs4MT/model/newptmodel-llms4mt-en2he-32a100/llama2-sfton-50000-bitexts-and-alpacagpt4) already exists and is not empty. Use --overwrite_output_dir to overcome.
01/04/2024 16:14:21 - WARNING - __main__ - Process rank: 2, device: cuda:2, n_gpu: 1distributed training: True, 16-bits training: False
01/04/2024 16:14:21 - WARNING - __main__ - Process rank: 7, device: cuda:7, n_gpu: 1distributed training: True, 16-bits training: False
01/04/2024 16:14:21 - WARNING - __main__ - Process rank: 3, device: cuda:3, n_gpu: 1distributed training: True, 16-bits training: False
01/04/2024 16:14:21 - WARNING - __main__ - Process rank: 6, device: cuda:6, n_gpu: 1distributed training: True, 16-bits training: False
Traceback (most recent call last):
  File "transformers/examples/pytorch/language-modeling/run_clm_llms_splittrain.py", line 739, in <module>
Traceback (most recent call last):
  File "transformers/examples/pytorch/language-modeling/run_clm_llms_splittrain.py", line 739, in <module>
Traceback (most recent call last):
  File "transformers/examples/pytorch/language-modeling/run_clm_llms_splittrain.py", line 739, in <module>
Traceback (most recent call last):
  File "transformers/examples/pytorch/language-modeling/run_clm_llms_splittrain.py", line 739, in <module>
    main()
  File "transformers/examples/pytorch/language-modeling/run_clm_llms_splittrain.py", line 282, in main
    main()
  File "transformers/examples/pytorch/language-modeling/run_clm_llms_splittrain.py", line 282, in main
    raise ValueError(
    main()ValueError
: Output directory (/apdcephfs/share_733425/vinnylywang/jianhuipang/LLMs4MT/model/newptmodel-llms4mt-en2he-32a100/llama2-sfton-50000-bitexts-and-alpacagpt4) already exists and is not empty. Use --overwrite_output_dir to overcome.
  File "transformers/examples/pytorch/language-modeling/run_clm_llms_splittrain.py", line 282, in main
    main()
  File "transformers/examples/pytorch/language-modeling/run_clm_llms_splittrain.py", line 282, in main
    raise ValueError(
ValueError: Output directory (/apdcephfs/share_733425/vinnylywang/jianhuipang/LLMs4MT/model/newptmodel-llms4mt-en2he-32a100/llama2-sfton-50000-bitexts-and-alpacagpt4) already exists and is not empty. Use --overwrite_output_dir to overcome.
    raise ValueError(    
raise ValueError(
ValueError: Output directory (/apdcephfs/share_733425/vinnylywang/jianhuipang/LLMs4MT/model/newptmodel-llms4mt-en2he-32a100/llama2-sfton-50000-bitexts-and-alpacagpt4) already exists and is not empty. Use --overwrite_output_dir to overcome.ValueError
: Output directory (/apdcephfs/share_733425/vinnylywang/jianhuipang/LLMs4MT/model/newptmodel-llms4mt-en2he-32a100/llama2-sfton-50000-bitexts-and-alpacagpt4) already exists and is not empty. Use --overwrite_output_dir to overcome.
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 114961 closing signal SIGTERM
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 114958) of binary: /apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/bin/python
Traceback (most recent call last):
  File "/apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/bin/torchrun", line 8, in <module>
    sys.exit(main())
  File "/apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 346, in wrapper
    return f(*args, **kwargs)
  File "/apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/torch/distributed/run.py", line 762, in main
    run(args)
  File "/apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/torch/distributed/run.py", line 753, in run
    elastic_launch(
  File "/apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 132, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 246, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError: 
============================================================
transformers/examples/pytorch/language-modeling/run_clm_llms_splittrain.py FAILED
------------------------------------------------------------
Failures:
[1]:
  time      : 2024-01-04_16:14:22
  host      : ts-523333936ff240a8b0a2237a60ca87ec-launcher.ts-523333936ff240a8b0a2237a60ca87ec-launcher.qingyuan-1-nlpc-sys.svc.cluster.local
  rank      : 1 (local_rank: 1)
  exitcode  : 1 (pid: 114959)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
[2]:
  time      : 2024-01-04_16:14:22
  host      : ts-523333936ff240a8b0a2237a60ca87ec-launcher.ts-523333936ff240a8b0a2237a60ca87ec-launcher.qingyuan-1-nlpc-sys.svc.cluster.local
  rank      : 2 (local_rank: 2)
  exitcode  : 1 (pid: 114960)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
[3]:
  time      : 2024-01-04_16:14:22
  host      : ts-523333936ff240a8b0a2237a60ca87ec-launcher.ts-523333936ff240a8b0a2237a60ca87ec-launcher.qingyuan-1-nlpc-sys.svc.cluster.local
  rank      : 4 (local_rank: 4)
  exitcode  : 1 (pid: 114962)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
[4]:
  time      : 2024-01-04_16:14:22
  host      : ts-523333936ff240a8b0a2237a60ca87ec-launcher.ts-523333936ff240a8b0a2237a60ca87ec-launcher.qingyuan-1-nlpc-sys.svc.cluster.local
  rank      : 5 (local_rank: 5)
  exitcode  : 1 (pid: 114963)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
[5]:
  time      : 2024-01-04_16:14:22
  host      : ts-523333936ff240a8b0a2237a60ca87ec-launcher.ts-523333936ff240a8b0a2237a60ca87ec-launcher.qingyuan-1-nlpc-sys.svc.cluster.local
  rank      : 6 (local_rank: 6)
  exitcode  : 1 (pid: 114964)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
[6]:
  time      : 2024-01-04_16:14:22
  host      : ts-523333936ff240a8b0a2237a60ca87ec-launcher.ts-523333936ff240a8b0a2237a60ca87ec-launcher.qingyuan-1-nlpc-sys.svc.cluster.local
  rank      : 7 (local_rank: 7)
  exitcode  : 1 (pid: 114965)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2024-01-04_16:14:22
  host      : ts-523333936ff240a8b0a2237a60ca87ec-launcher.ts-523333936ff240a8b0a2237a60ca87ec-launcher.qingyuan-1-nlpc-sys.svc.cluster.local
  rank      : 0 (local_rank: 0)
  exitcode  : 1 (pid: 114958)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
============================================================
WARNING:torch.distributed.run:
*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
*****************************************
[2024-01-04 21:07:51,232] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-04 21:07:51,413] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-04 21:07:51,557] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-04 21:07:51,582] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-04 21:07:51,616] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-04 21:07:51,630] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-04 21:07:51,632] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-04 21:07:51,634] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-04 21:08:11,814] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-01-04 21:08:11,814] [INFO] [comm.py:668:init_distributed] Initializing TorchBackend in DeepSpeed with backend nccl
[2024-01-04 21:08:11,942] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-01-04 21:08:12,147] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-01-04 21:08:12,379] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-01-04 21:08:12,426] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-01-04 21:08:12,435] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-01-04 21:08:12,437] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-01-04 21:08:12,445] [INFO] [comm.py:637:init_distributed] cdb=None
01/04/2024 21:09:11 - WARNING - __main__ - Process rank: 1, device: cuda:1, n_gpu: 1distributed training: True, 16-bits training: False
/apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/datasets/load.py:2089: FutureWarning: 'use_auth_token' was deprecated in favor of 'token' in version 2.14.0 and will be removed in 3.0.0.
You can remove this warning by passing 'token=None' instead.
  warnings.warn(
01/04/2024 21:09:13 - WARNING - __main__ - Process rank: 4, device: cuda:4, n_gpu: 1distributed training: True, 16-bits training: False
01/04/2024 21:09:13 - WARNING - __main__ - Process rank: 2, device: cuda:2, n_gpu: 1distributed training: True, 16-bits training: False
/apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/datasets/load.py:2089: FutureWarning: 'use_auth_token' was deprecated in favor of 'token' in version 2.14.0 and will be removed in 3.0.0.
You can remove this warning by passing 'token=None' instead.
  warnings.warn(
/apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/datasets/load.py:2089: FutureWarning: 'use_auth_token' was deprecated in favor of 'token' in version 2.14.0 and will be removed in 3.0.0.
You can remove this warning by passing 'token=None' instead.
  warnings.warn(
01/04/2024 21:09:18 - WARNING - __main__ - Process rank: 6, device: cuda:6, n_gpu: 1distributed training: True, 16-bits training: False
01/04/2024 21:09:18 - WARNING - __main__ - Process rank: 7, device: cuda:7, n_gpu: 1distributed training: True, 16-bits training: False
01/04/2024 21:09:18 - WARNING - __main__ - Process rank: 3, device: cuda:3, n_gpu: 1distributed training: True, 16-bits training: False
/apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/datasets/load.py:2089: FutureWarning: 'use_auth_token' was deprecated in favor of 'token' in version 2.14.0 and will be removed in 3.0.0.
You can remove this warning by passing 'token=None' instead.
  warnings.warn(
/apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/datasets/load.py:2089: FutureWarning: 'use_auth_token' was deprecated in favor of 'token' in version 2.14.0 and will be removed in 3.0.0.
You can remove this warning by passing 'token=None' instead.
  warnings.warn(
/apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/datasets/load.py:2089: FutureWarning: 'use_auth_token' was deprecated in favor of 'token' in version 2.14.0 and will be removed in 3.0.0.
You can remove this warning by passing 'token=None' instead.
  warnings.warn(
01/04/2024 21:09:31 - WARNING - __main__ - Process rank: 5, device: cuda:5, n_gpu: 1distributed training: True, 16-bits training: False
/apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/datasets/load.py:2089: FutureWarning: 'use_auth_token' was deprecated in favor of 'token' in version 2.14.0 and will be removed in 3.0.0.
You can remove this warning by passing 'token=None' instead.
  warnings.warn(
01/04/2024 21:09:34 - WARNING - __main__ - Process rank: 0, device: cuda:0, n_gpu: 1distributed training: True, 16-bits training: False
01/04/2024 21:09:34 - INFO - __main__ - Training/evaluation parameters TrainingArguments(
_n_gpu=1,
adafactor=False,
adam_beta1=0.9,
adam_beta2=0.999,
adam_epsilon=1e-08,
auto_find_batch_size=False,
bf16=True,
bf16_full_eval=True,
data_seed=None,
dataloader_drop_last=False,
dataloader_num_workers=8,
dataloader_pin_memory=True,
ddp_bucket_cap_mb=None,
ddp_find_unused_parameters=None,
ddp_timeout=72000,
debug=[],
deepspeed=train/deepspeed_config_zero2.json,
disable_tqdm=False,
do_eval=False,
do_predict=False,
do_train=True,
eval_accumulation_steps=None,
eval_delay=0,
eval_steps=None,
evaluation_strategy=no,
fp16=False,
fp16_backend=auto,
fp16_full_eval=False,
fp16_opt_level=O1,
fsdp=[],
fsdp_config={'fsdp_min_num_params': 0, 'xla': False, 'xla_fsdp_grad_ckpt': False},
fsdp_min_num_params=0,
fsdp_transformer_layer_cls_to_wrap=None,
full_determinism=False,
gradient_accumulation_steps=4,
gradient_checkpointing=True,
greater_is_better=None,
group_by_length=False,
half_precision_backend=auto,
hub_model_id=None,
hub_private_repo=False,
hub_strategy=every_save,
hub_token=<HUB_TOKEN>,
ignore_data_skip=False,
include_inputs_for_metrics=False,
jit_mode_eval=False,
label_names=None,
label_smoothing_factor=0.0,
learning_rate=2e-05,
length_column_name=length,
load_best_model_at_end=False,
local_rank=0,
log_level=passive,
log_level_replica=warning,
log_on_each_node=True,
logging_dir=/apdcephfs/share_733425/vinnylywang/jianhuipang/LLMs4MT/model/newptmodel-llms4mt-en2he-32a100/llama2-sfton-50000-bitexts-and-alpacagpt4/runs/Jan04_21-07-51_ts-523333936ff240a8b0a2237a60ca87ec-launcher,
logging_first_step=False,
logging_nan_inf_filter=True,
logging_steps=10,
logging_strategy=steps,
lr_scheduler_type=cosine,
max_grad_norm=1.0,
max_steps=-1,
metric_for_best_model=None,
mp_parameters=,
no_cuda=False,
num_train_epochs=3.0,
optim=adamw_hf,
optim_args=None,
output_dir=/apdcephfs/share_733425/vinnylywang/jianhuipang/LLMs4MT/model/newptmodel-llms4mt-en2he-32a100/llama2-sfton-50000-bitexts-and-alpacagpt4,
overwrite_output_dir=True,
past_index=-1,
per_device_eval_batch_size=2,
per_device_train_batch_size=48,
prediction_loss_only=False,
push_to_hub=False,
push_to_hub_model_id=None,
push_to_hub_organization=None,
push_to_hub_token=<PUSH_TO_HUB_TOKEN>,
ray_scope=last,
remove_unused_columns=True,
report_to=['tensorboard'],
resume_from_checkpoint=None,
run_name=/apdcephfs/share_733425/vinnylywang/jianhuipang/LLMs4MT/model/newptmodel-llms4mt-en2he-32a100/llama2-sfton-50000-bitexts-and-alpacagpt4,
save_on_each_node=False,
save_steps=500,
save_strategy=steps,
save_total_limit=1,
seed=34,
sharded_ddp=[],
skip_memory_metrics=True,
tf32=None,
torch_compile=False,
torch_compile_backend=None,
torch_compile_mode=None,
torchdynamo=None,
tpu_metrics_debug=False,
tpu_num_cores=None,
use_ipex=False,
use_legacy_prediction_loop=False,
use_mps_device=False,
warmup_ratio=0.03,
warmup_steps=0,
weight_decay=0.0,
xpu_backend=None,
)
01/04/2024 21:09:34 - INFO - __main__ - Loading dataset from file: {'train': '/apdcephfs/share_733425/vinnylywang/jianhuipang/datasets/wmt23/wmt23-heen/train.catalpacagpt4.en2he.hf.json', 'validation': '/apdcephfs/share_733425/vinnylywang/jianhuipang/LLMs4MT/test/WMT22/newstest22.de2en.hf.json'}
/apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/datasets/load.py:2089: FutureWarning: 'use_auth_token' was deprecated in favor of 'token' in version 2.14.0 and will be removed in 3.0.0.
You can remove this warning by passing 'token=None' instead.
  warnings.warn(
Using custom data configuration default-7074f71f985f4a81
01/04/2024 21:09:37 - INFO - datasets.builder - Using custom data configuration default-7074f71f985f4a81
Loading Dataset Infos from /apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/datasets/packaged_modules/json
01/04/2024 21:09:37 - INFO - datasets.info - Loading Dataset Infos from /apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/datasets/packaged_modules/json
Overwrite dataset info from restored data version if exists.
01/04/2024 21:09:37 - INFO - datasets.builder - Overwrite dataset info from restored data version if exists.
Loading Dataset info from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96
01/04/2024 21:09:37 - INFO - datasets.info - Loading Dataset info from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96
Found cached dataset json (/apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96)
01/04/2024 21:09:37 - INFO - datasets.builder - Found cached dataset json (/apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96)
Loading Dataset info from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96
01/04/2024 21:09:37 - INFO - datasets.info - Loading Dataset info from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96
Using custom data configuration default-7074f71f985f4a81
01/04/2024 21:09:40 - INFO - datasets.builder - Using custom data configuration default-7074f71f985f4a81
Loading Dataset Infos from /apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/datasets/packaged_modules/json
01/04/2024 21:09:40 - INFO - datasets.info - Loading Dataset Infos from /apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/datasets/packaged_modules/json
Overwrite dataset info from restored data version if exists.
01/04/2024 21:09:40 - INFO - datasets.builder - Overwrite dataset info from restored data version if exists.
Loading Dataset info from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96
01/04/2024 21:09:40 - INFO - datasets.info - Loading Dataset info from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96
Found cached dataset json (/apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96)
01/04/2024 21:09:40 - INFO - datasets.builder - Found cached dataset json (/apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96)
Loading Dataset info from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96
01/04/2024 21:09:40 - INFO - datasets.info - Loading Dataset info from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96
[INFO|configuration_utils.py:666] 2024-01-04 21:09:40,805 >> loading configuration file /apdcephfs/share_733425/vinnylywang/jianhuipang/opensourcellms/llama2/Llama-2-7b-hf/config.json
[INFO|configuration_utils.py:720] 2024-01-04 21:09:40,806 >> Model config LlamaConfig {
  "_name_or_path": "/apdcephfs/share_733425/vinnylywang/jianhuipang/opensourcellms/llama2/Llama-2-7b-hf",
  "architectures": [
    "LlamaForCausalLM"
  ],
  "bos_token_id": 1,
  "eos_token_id": 2,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 11008,
  "max_position_embeddings": 4096,
  "model_type": "llama",
  "num_attention_heads": 32,
  "num_hidden_layers": 32,
  "num_key_value_heads": 32,
  "pad_token_id": 0,
  "pretraining_tp": 1,
  "rms_norm_eps": 1e-05,
  "rope_scaling": null,
  "tie_word_embeddings": false,
  "torch_dtype": "float16",
  "transformers_version": "4.28.0.dev0",
  "use_cache": true,
  "vocab_size": 32000
}

01/04/2024 21:09:40 - INFO - __main__ - Tokenizer_kwargs: {'cache_dir': None, 'use_fast': True, 'revision': 'main', 'use_auth_token': None}
[INFO|tokenization_utils_base.py:1801] 2024-01-04 21:09:40,811 >> loading file tokenizer.model
[INFO|tokenization_utils_base.py:1801] 2024-01-04 21:09:40,811 >> loading file added_tokens.json
[INFO|tokenization_utils_base.py:1801] 2024-01-04 21:09:40,811 >> loading file special_tokens_map.json
[INFO|tokenization_utils_base.py:1801] 2024-01-04 21:09:40,811 >> loading file tokenizer_config.json
01/04/2024 21:09:40 - INFO - __main__ - Loading checkpoints in dtype: None
[INFO|modeling_utils.py:2395] 2024-01-04 21:09:40,827 >> loading weights file /apdcephfs/share_733425/vinnylywang/jianhuipang/opensourcellms/llama2/Llama-2-7b-hf/pytorch_model.bin.index.json
[INFO|configuration_utils.py:575] 2024-01-04 21:09:40,828 >> Generate config GenerationConfig {
  "_from_model_config": true,
  "bos_token_id": 1,
  "eos_token_id": 2,
  "pad_token_id": 0,
  "transformers_version": "4.28.0.dev0"
}


Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:06<00:06,  6.19s/it]
Loading checkpoint shards:  50%|█████     | 1/2 [00:06<00:06,  6.15s/it]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:08<00:00,  3.99s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:08<00:00,  4.32s/it]

Loading checkpoint shards: 100%|██████████| 2/2 [00:08<00:00,  3.94s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:08<00:00,  4.27s/it]
[ERROR|tokenization_utils_base.py:1042] 2024-01-04 21:10:04,883 >> Using pad_token, but it is not set yet.
[ERROR|tokenization_utils_base.py:1042] 2024-01-04 21:10:04,888 >> Using pad_token, but it is not set yet.

Loading checkpoint shards:  50%|█████     | 1/2 [00:06<00:06,  6.38s/it]
Loading checkpoint shards:  50%|█████     | 1/2 [00:06<00:06,  6.31s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:08<00:00,  3.96s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:08<00:00,  4.32s/it]
[ERROR|tokenization_utils_base.py:1042] 2024-01-04 21:10:07,636 >> Using pad_token, but it is not set yet.

Loading checkpoint shards:  50%|█████     | 1/2 [00:06<00:06,  6.43s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:08<00:00,  3.93s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:08<00:00,  4.29s/it]
[ERROR|tokenization_utils_base.py:1042] 2024-01-04 21:10:09,402 >> Using pad_token, but it is not set yet.

Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:08<00:00,  3.94s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:08<00:00,  4.32s/it]
[ERROR|tokenization_utils_base.py:1042] 2024-01-04 21:10:11,145 >> Using pad_token, but it is not set yet.

Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:05<00:05,  5.96s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:08<00:00,  3.77s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:08<00:00,  4.10s/it]
[ERROR|tokenization_utils_base.py:1042] 2024-01-04 21:10:18,187 >> Using pad_token, but it is not set yet.

Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:06<00:06,  6.06s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:08<00:00,  3.78s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:08<00:00,  4.12s/it]
[ERROR|tokenization_utils_base.py:1042] 2024-01-04 21:10:21,966 >> Using pad_token, but it is not set yet.

Loading checkpoint shards:  50%|█████     | 1/2 [00:05<00:05,  5.82s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:07<00:00,  3.65s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:07<00:00,  3.97s/it]
[INFO|modeling_utils.py:3029] 2024-01-04 21:10:26,779 >> All model checkpoint weights were used when initializing LlamaForCausalLM.

[INFO|modeling_utils.py:3037] 2024-01-04 21:10:26,779 >> All the weights of LlamaForCausalLM were initialized from the model checkpoint at /apdcephfs/share_733425/vinnylywang/jianhuipang/opensourcellms/llama2/Llama-2-7b-hf.
If your task is similar to the task the model of the checkpoint was trained on, you can already use LlamaForCausalLM for predictions without further training.
[INFO|configuration_utils.py:535] 2024-01-04 21:10:26,787 >> loading configuration file /apdcephfs/share_733425/vinnylywang/jianhuipang/opensourcellms/llama2/Llama-2-7b-hf/generation_config.json
[INFO|configuration_utils.py:575] 2024-01-04 21:10:26,787 >> Generate config GenerationConfig {
  "bos_token_id": 1,
  "do_sample": true,
  "eos_token_id": 2,
  "max_length": 4096,
  "pad_token_id": 0,
  "temperature": 0.6,
  "top_p": 0.9,
  "transformers_version": "4.28.0.dev0"
}

[ERROR|tokenization_utils_base.py:1042] 2024-01-04 21:10:26,787 >> Using pad_token, but it is not set yet.
[INFO|tokenization_utils_base.py:907] 2024-01-04 21:10:26,787 >> Assigning [PAD] to the pad_token key of the tokenizer
[INFO|tokenization_utils.py:426] 2024-01-04 21:10:26,787 >> Adding [PAD] to the vocabulary
[INFO|tokenization_utils_base.py:907] 2024-01-04 21:10:29,762 >> Assigning </s> to the eos_token key of the tokenizer
[INFO|tokenization_utils_base.py:907] 2024-01-04 21:10:29,763 >> Assigning <s> to the bos_token key of the tokenizer
[INFO|tokenization_utils_base.py:907] 2024-01-04 21:10:29,763 >> Assigning <unk> to the unk_token key of the tokenizer
Process #0 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00000_of_00016.arrow
01/04/2024 21:10:29 - INFO - datasets.arrow_dataset - Process #0 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00000_of_00016.arrow
Process #1 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00001_of_00016.arrow
01/04/2024 21:10:29 - INFO - datasets.arrow_dataset - Process #1 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00001_of_00016.arrow
Process #2 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00002_of_00016.arrow
01/04/2024 21:10:29 - INFO - datasets.arrow_dataset - Process #2 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00002_of_00016.arrow
Process #3 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00003_of_00016.arrow
01/04/2024 21:10:29 - INFO - datasets.arrow_dataset - Process #3 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00003_of_00016.arrow
Process #4 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00004_of_00016.arrow
01/04/2024 21:10:29 - INFO - datasets.arrow_dataset - Process #4 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00004_of_00016.arrow
Process #5 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00005_of_00016.arrow
01/04/2024 21:10:29 - INFO - datasets.arrow_dataset - Process #5 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00005_of_00016.arrow
Process #6 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00006_of_00016.arrow
01/04/2024 21:10:29 - INFO - datasets.arrow_dataset - Process #6 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00006_of_00016.arrow
Process #7 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00007_of_00016.arrow
01/04/2024 21:10:29 - INFO - datasets.arrow_dataset - Process #7 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00007_of_00016.arrow
Process #8 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00008_of_00016.arrow
01/04/2024 21:10:29 - INFO - datasets.arrow_dataset - Process #8 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00008_of_00016.arrow
Process #9 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00009_of_00016.arrow
01/04/2024 21:10:29 - INFO - datasets.arrow_dataset - Process #9 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00009_of_00016.arrow
Process #10 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00010_of_00016.arrow
01/04/2024 21:10:29 - INFO - datasets.arrow_dataset - Process #10 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00010_of_00016.arrow
Process #11 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00011_of_00016.arrow
01/04/2024 21:10:29 - INFO - datasets.arrow_dataset - Process #11 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00011_of_00016.arrow
Process #12 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00012_of_00016.arrow
01/04/2024 21:10:29 - INFO - datasets.arrow_dataset - Process #12 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00012_of_00016.arrow
Process #13 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00013_of_00016.arrow
01/04/2024 21:10:29 - INFO - datasets.arrow_dataset - Process #13 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00013_of_00016.arrow
Process #14 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00014_of_00016.arrow
01/04/2024 21:10:29 - INFO - datasets.arrow_dataset - Process #14 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00014_of_00016.arrow
Process #15 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00015_of_00016.arrow
01/04/2024 21:10:29 - INFO - datasets.arrow_dataset - Process #15 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00015_of_00016.arrow
Spawning 16 processes
01/04/2024 21:10:34 - INFO - datasets.arrow_dataset - Spawning 16 processes

Tokenize with padding (num_proc=16):   0%|          | 0/102002 [00:00<?, ? examples/s]Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00011_of_00016.arrow
01/04/2024 21:10:36 - INFO - datasets.arrow_dataset - Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00011_of_00016.arrow
Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00012_of_00016.arrow
01/04/2024 21:10:36 - INFO - datasets.arrow_dataset - Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00012_of_00016.arrow
Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00015_of_00016.arrow
01/04/2024 21:10:36 - INFO - datasets.arrow_dataset - Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00015_of_00016.arrow
Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00009_of_00016.arrow
01/04/2024 21:10:36 - INFO - datasets.arrow_dataset - Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00009_of_00016.arrow

Tokenize with padding (num_proc=16):   1%|          | 1000/102002 [00:01<03:12, 525.43 examples/s]Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00013_of_00016.arrow
01/04/2024 21:10:36 - INFO - datasets.arrow_dataset - Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00013_of_00016.arrow
Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00010_of_00016.arrow
01/04/2024 21:10:36 - INFO - datasets.arrow_dataset - Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00010_of_00016.arrow

Tokenize with padding (num_proc=16):   6%|▌         | 6000/102002 [00:02<00:24, 3947.84 examples/s]Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00014_of_00016.arrow
01/04/2024 21:10:36 - INFO - datasets.arrow_dataset - Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00014_of_00016.arrow
Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00000_of_00016.arrow
01/04/2024 21:10:36 - INFO - datasets.arrow_dataset - Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00000_of_00016.arrow
Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00001_of_00016.arrow
01/04/2024 21:10:37 - INFO - datasets.arrow_dataset - Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00001_of_00016.arrow
Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00007_of_00016.arrow
01/04/2024 21:10:37 - INFO - datasets.arrow_dataset - Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00007_of_00016.arrow
Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00003_of_00016.arrow
01/04/2024 21:10:37 - INFO - datasets.arrow_dataset - Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00003_of_00016.arrow
Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00008_of_00016.arrow
01/04/2024 21:10:37 - INFO - datasets.arrow_dataset - Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00008_of_00016.arrow
Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00005_of_00016.arrow
01/04/2024 21:10:37 - INFO - datasets.arrow_dataset - Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00005_of_00016.arrow
Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00002_of_00016.arrow
01/04/2024 21:10:37 - INFO - datasets.arrow_dataset - Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00002_of_00016.arrow
Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00004_of_00016.arrow
01/04/2024 21:10:37 - INFO - datasets.arrow_dataset - Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00004_of_00016.arrow

Tokenize with padding (num_proc=16):   9%|▉         | 9000/102002 [00:02<00:18, 5003.87 examples/s]Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00006_of_00016.arrow
01/04/2024 21:10:37 - INFO - datasets.arrow_dataset - Caching processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-0c92dae5de07ea82_00006_of_00016.arrow

Tokenize with padding (num_proc=16):  17%|█▋        | 17000/102002 [00:03<00:12, 6729.21 examples/s]
Tokenize with padding (num_proc=16):  23%|██▎       | 23000/102002 [00:03<00:09, 7985.43 examples/s]
Tokenize with padding (num_proc=16):  25%|██▍       | 25000/102002 [00:04<00:10, 7366.48 examples/s]
Tokenize with padding (num_proc=16):  26%|██▋       | 27000/102002 [00:04<00:09, 8196.07 examples/s]
Tokenize with padding (num_proc=16):  31%|███▏      | 32000/102002 [00:04<00:05, 12269.72 examples/s]
Tokenize with padding (num_proc=16):  34%|███▍      | 35000/102002 [00:04<00:05, 11293.42 examples/s]
Tokenize with padding (num_proc=16):  37%|███▋      | 38000/102002 [00:04<00:04, 13536.39 examples/s]
Tokenize with padding (num_proc=16):  40%|████      | 41000/102002 [00:06<00:09, 6116.97 examples/s] 
Tokenize with padding (num_proc=16):  42%|████▏     | 43000/102002 [00:06<00:08, 7065.55 examples/s]
Tokenize with padding (num_proc=16):  45%|████▌     | 46000/102002 [00:06<00:06, 8857.30 examples/s]
Tokenize with padding (num_proc=16):  53%|█████▎    | 54000/102002 [00:06<00:02, 16012.80 examples/s]
Tokenize with padding (num_proc=16):  56%|█████▌    | 57000/102002 [00:07<00:05, 7566.76 examples/s] 
Tokenize with padding (num_proc=16):  59%|█████▉    | 60000/102002 [00:07<00:04, 8914.68 examples/s]
Tokenize with padding (num_proc=16):  62%|██████▏   | 63000/102002 [00:08<00:04, 9414.89 examples/s]
Tokenize with padding (num_proc=16):  64%|██████▎   | 65000/102002 [00:08<00:04, 8604.32 examples/s]
Tokenize with padding (num_proc=16):  68%|██████▊   | 69000/102002 [00:08<00:02, 11190.99 examples/s]
Tokenize with padding (num_proc=16):  72%|███████▏  | 73000/102002 [00:09<00:03, 9516.25 examples/s] 
Tokenize with padding (num_proc=16):  74%|███████▎  | 75000/102002 [00:09<00:02, 9413.86 examples/s]
Tokenize with padding (num_proc=16):  76%|███████▋  | 78000/102002 [00:09<00:02, 11219.66 examples/s]
Tokenize with padding (num_proc=16):  78%|███████▊  | 79750/102002 [00:09<00:02, 8761.68 examples/s] 
Tokenize with padding (num_proc=16):  80%|███████▉  | 81250/102002 [00:10<00:02, 8633.63 examples/s]
Tokenize with padding (num_proc=16):  83%|████████▎ | 84250/102002 [00:10<00:02, 8542.98 examples/s]
Tokenize with padding (num_proc=16):  85%|████████▍ | 86625/102002 [00:10<00:01, 10371.36 examples/s]
Tokenize with padding (num_proc=16):  86%|████████▋ | 88000/102002 [00:10<00:01, 10863.72 examples/s]
Tokenize with padding (num_proc=16):  90%|█████████ | 92000/102002 [00:12<00:02, 4734.66 examples/s] 
Tokenize with padding (num_proc=16):  92%|█████████▏| 94000/102002 [00:12<00:01, 4807.00 examples/s]
Tokenize with padding (num_proc=16):  94%|█████████▍| 96000/102002 [00:12<00:01, 5925.02 examples/s]
Tokenize with padding (num_proc=16):  96%|█████████▌| 98000/102002 [00:12<00:00, 7140.50 examples/s]
Tokenize with padding (num_proc=16):  97%|█████████▋| 99376/102002 [00:12<00:00, 7930.60 examples/s]
Tokenize with padding (num_proc=16):  99%|█████████▉| 100877/102002 [00:13<00:00, 5504.58 examples/s]
Tokenize with padding (num_proc=16): 100%|██████████| 102002/102002 [00:13<00:00, 6003.88 examples/s]
Tokenize with padding (num_proc=16): 100%|██████████| 102002/102002 [00:13<00:00, 7355.57 examples/s]
Concatenating 16 shards
01/04/2024 21:10:49 - INFO - datasets.arrow_dataset - Concatenating 16 shards
Process #0 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00000_of_00016.arrow
01/04/2024 21:10:49 - INFO - datasets.arrow_dataset - Process #0 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00000_of_00016.arrow
Process #1 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00001_of_00016.arrow
01/04/2024 21:10:49 - INFO - datasets.arrow_dataset - Process #1 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00001_of_00016.arrow
Process #2 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00002_of_00016.arrow
01/04/2024 21:10:49 - INFO - datasets.arrow_dataset - Process #2 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00002_of_00016.arrow
Process #3 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00003_of_00016.arrow
01/04/2024 21:10:49 - INFO - datasets.arrow_dataset - Process #3 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00003_of_00016.arrow
Process #4 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00004_of_00016.arrow
01/04/2024 21:10:49 - INFO - datasets.arrow_dataset - Process #4 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00004_of_00016.arrow
Process #5 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00005_of_00016.arrow
01/04/2024 21:10:49 - INFO - datasets.arrow_dataset - Process #5 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00005_of_00016.arrow
Process #6 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00006_of_00016.arrow
01/04/2024 21:10:49 - INFO - datasets.arrow_dataset - Process #6 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00006_of_00016.arrow
Process #7 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00007_of_00016.arrow
01/04/2024 21:10:49 - INFO - datasets.arrow_dataset - Process #7 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00007_of_00016.arrow
Process #8 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00008_of_00016.arrow
01/04/2024 21:10:49 - INFO - datasets.arrow_dataset - Process #8 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00008_of_00016.arrow
Process #9 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00009_of_00016.arrow
01/04/2024 21:10:49 - INFO - datasets.arrow_dataset - Process #9 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00009_of_00016.arrow
Process #10 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00010_of_00016.arrow
01/04/2024 21:10:49 - INFO - datasets.arrow_dataset - Process #10 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00010_of_00016.arrow
Process #11 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00011_of_00016.arrow
01/04/2024 21:10:49 - INFO - datasets.arrow_dataset - Process #11 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00011_of_00016.arrow
Process #12 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00012_of_00016.arrow
01/04/2024 21:10:49 - INFO - datasets.arrow_dataset - Process #12 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00012_of_00016.arrow
Process #13 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00013_of_00016.arrow
01/04/2024 21:10:49 - INFO - datasets.arrow_dataset - Process #13 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00013_of_00016.arrow
Process #14 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00014_of_00016.arrow
01/04/2024 21:10:49 - INFO - datasets.arrow_dataset - Process #14 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00014_of_00016.arrow
Process #15 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00015_of_00016.arrow
01/04/2024 21:10:49 - INFO - datasets.arrow_dataset - Process #15 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_00015_of_00016.arrow
Loading cached processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_*_of_00016.arrow
01/04/2024 21:10:49 - INFO - datasets.arrow_dataset - Loading cached processed dataset at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache3/datasets/json/default-7074f71f985f4a81/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-e503eb3b3294a484_*_of_00016.arrow
Concatenating 16 shards
01/04/2024 21:10:49 - INFO - datasets.arrow_dataset - Concatenating 16 shards
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:147355 [0] NCCL INFO Bootstrap : Using eth1:11.222.146.177<0>
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:147355 [0] NCCL INFO NET/Plugin : No plugin found (libnccl-net.so), using internal implementation
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:147362 [7] NCCL INFO cudaDriverVersion 11070
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:147355 [0] NCCL INFO cudaDriverVersion 11070
NCCL version 2.14.3+cuda11.7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:147357 [2] NCCL INFO cudaDriverVersion 11070
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:147362 [7] NCCL INFO Bootstrap : Using eth1:11.222.146.177<0>
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:147357 [2] NCCL INFO Bootstrap : Using eth1:11.222.146.177<0>
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:147356 [1] NCCL INFO cudaDriverVersion 11070
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:147359 [4] NCCL INFO cudaDriverVersion 11070
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:147361 [6] NCCL INFO cudaDriverVersion 11070
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:147360 [5] NCCL INFO cudaDriverVersion 11070
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:147358 [3] NCCL INFO cudaDriverVersion 11070
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:147356 [1] NCCL INFO Bootstrap : Using eth1:11.222.146.177<0>
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:147359 [4] NCCL INFO Bootstrap : Using eth1:11.222.146.177<0>
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:147361 [6] NCCL INFO Bootstrap : Using eth1:11.222.146.177<0>
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:147360 [5] NCCL INFO Bootstrap : Using eth1:11.222.146.177<0>
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:147358 [3] NCCL INFO Bootstrap : Using eth1:11.222.146.177<0>
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:147357 [2] NCCL INFO NET/Plugin : No plugin found (libnccl-net.so), using internal implementation
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:147362 [7] NCCL INFO NET/Plugin : No plugin found (libnccl-net.so), using internal implementation
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:147356 [1] NCCL INFO NET/Plugin : No plugin found (libnccl-net.so), using internal implementation
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:147359 [4] NCCL INFO NET/Plugin : No plugin found (libnccl-net.so), using internal implementation
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:147361 [6] NCCL INFO NET/Plugin : No plugin found (libnccl-net.so), using internal implementation
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:147358 [3] NCCL INFO NET/Plugin : No plugin found (libnccl-net.so), using internal implementation
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:147360 [5] NCCL INFO NET/Plugin : No plugin found (libnccl-net.so), using internal implementation
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO NET/IB : Using [0]mlx5_2:1/RoCE [RO]; OOB eth1:11.222.146.177<0>
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO NET/IB : Using [0]mlx5_2:1/RoCE [RO]; OOB eth1:11.222.146.177<0>
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Using network IB
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Using network IB
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO NET/IB : Using [0]mlx5_2:1/RoCE [RO]; OOB eth1:11.222.146.177<0>
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Using network IB
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO NET/IB : Using [0]mlx5_2:1/RoCE [RO]; OOB eth1:11.222.146.177<0>
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Using network IB
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO NET/IB : Using [0]mlx5_2:1/RoCE [RO]; OOB eth1:11.222.146.177<0>
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Using network IB
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO NET/IB : Using [0]mlx5_2:1/RoCE [RO]; OOB eth1:11.222.146.177<0>
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Using network IB
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO NET/IB : Using [0]mlx5_2:1/RoCE [RO]; OOB eth1:11.222.146.177<0>
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Using network IB
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO NET/IB : Using [0]mlx5_2:1/RoCE [RO]; OOB eth1:11.222.146.177<0>
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Using network IB
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Setting affinity for GPU 6 to ffffffff,ffff0000,00000000,ffffffff,ffff0000,00000000
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Setting affinity for GPU 5 to ffffffff,ffff0000,00000000,ffffffff,ffff0000,00000000
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Setting affinity for GPU 1 to ffff,ffffffff,00000000,0000ffff,ffffffff
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Setting affinity for GPU 0 to ffff,ffffffff,00000000,0000ffff,ffffffff
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Setting affinity for GPU 3 to ffff,ffffffff,00000000,0000ffff,ffffffff
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Setting affinity for GPU 4 to ffffffff,ffff0000,00000000,ffffffff,ffff0000,00000000
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Setting affinity for GPU 7 to ffffffff,ffff0000,00000000,ffffffff,ffff0000,00000000
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Setting affinity for GPU 2 to ffff,ffffffff,00000000,0000ffff,ffffffff
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Trees [0] 3/-1/-1->2->1 [1] 3/-1/-1->2->1 [2] 3/-1/-1->2->1 [3] 3/-1/-1->2->1 [4] 3/-1/-1->2->1 [5] 3/-1/-1->2->1 [6] 3/-1/-1->2->1 [7] 3/-1/-1->2->1 [8] 3/-1/-1->2->1 [9] 3/-1/-1->2->1 [10] 3/-1/-1->2->1 [11] 3/-1/-1->2->1 [12] 3/-1/-1->2->1 [13] 3/-1/-1->2->1 [14] 3/-1/-1->2->1 [15] 3/-1/-1->2->1 [16] 3/-1/-1->2->1 [17] 3/-1/-1->2->1 [18] 3/-1/-1->2->1 [19] 3/-1/-1->2->1 [20] 3/-1/-1->2->1 [21] 3/-1/-1->2->1 [22] 3/-1/-1->2->1 [23] 3/-1/-1->2->1
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 00/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 01/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 02/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 03/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Trees [0] 7/-1/-1->6->5 [1] 7/-1/-1->6->5 [2] 7/-1/-1->6->5 [3] 7/-1/-1->6->5 [4] 7/-1/-1->6->5 [5] 7/-1/-1->6->5 [6] 7/-1/-1->6->5 [7] 7/-1/-1->6->5 [8] 7/-1/-1->6->5 [9] 7/-1/-1->6->5 [10] 7/-1/-1->6->5 [11] 7/-1/-1->6->5 [12] 7/-1/-1->6->5 [13] 7/-1/-1->6->5 [14] 7/-1/-1->6->5 [15] 7/-1/-1->6->5 [16] 7/-1/-1->6->5 [17] 7/-1/-1->6->5 [18] 7/-1/-1->6->5 [19] 7/-1/-1->6->5 [20] 7/-1/-1->6->5 [21] 7/-1/-1->6->5 [22] 7/-1/-1->6->5 [23] 7/-1/-1->6->5
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 04/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 05/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 06/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Trees [0] 2/-1/-1->1->0 [1] 2/-1/-1->1->0 [2] 2/-1/-1->1->0 [3] 2/-1/-1->1->0 [4] 2/-1/-1->1->0 [5] 2/-1/-1->1->0 [6] 2/-1/-1->1->0 [7] 2/-1/-1->1->0 [8] 2/-1/-1->1->0 [9] 2/-1/-1->1->0 [10] 2/-1/-1->1->0 [11] 2/-1/-1->1->0 [12] 2/-1/-1->1->0 [13] 2/-1/-1->1->0 [14] 2/-1/-1->1->0 [15] 2/-1/-1->1->0 [16] 2/-1/-1->1->0 [17] 2/-1/-1->1->0 [18] 2/-1/-1->1->0 [19] 2/-1/-1->1->0 [20] 2/-1/-1->1->0 [21] 2/-1/-1->1->0 [22] 2/-1/-1->1->0 [23] 2/-1/-1->1->0
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 07/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 08/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Trees [0] -1/-1/-1->7->6 [1] -1/-1/-1->7->6 [2] -1/-1/-1->7->6 [3] -1/-1/-1->7->6 [4] -1/-1/-1->7->6 [5] -1/-1/-1->7->6 [6] -1/-1/-1->7->6 [7] -1/-1/-1->7->6 [8] -1/-1/-1->7->6 [9] -1/-1/-1->7->6 [10] -1/-1/-1->7->6 [11] -1/-1/-1->7->6 [12] -1/-1/-1->7->6 [13] -1/-1/-1->7->6 [14] -1/-1/-1->7->6 [15] -1/-1/-1->7->6 [16] -1/-1/-1->7->6 [17] -1/-1/-1->7->6 [18] -1/-1/-1->7->6 [19] -1/-1/-1->7->6 [20] -1/-1/-1->7->6 [21] -1/-1/-1->7->6 [22] -1/-1/-1->7->6 [23] -1/-1/-1->7->6
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Trees [0] 6/-1/-1->5->4 [1] 6/-1/-1->5->4 [2] 6/-1/-1->5->4 [3] 6/-1/-1->5->4 [4] 6/-1/-1->5->4 [5] 6/-1/-1->5->4 [6] 6/-1/-1->5->4 [7] 6/-1/-1->5->4 [8] 6/-1/-1->5->4 [9] 6/-1/-1->5->4 [10] 6/-1/-1->5->4 [11] 6/-1/-1->5->4 [12] 6/-1/-1->5->4 [13] 6/-1/-1->5->4 [14] 6/-1/-1->5->4 [15] 6/-1/-1->5->4 [16] 6/-1/-1->5->4 [17] 6/-1/-1->5->4 [18] 6/-1/-1->5->4 [19] 6/-1/-1->5->4 [20] 6/-1/-1->5->4 [21] 6/-1/-1->5->4 [22] 6/-1/-1->5->4 [23] 6/-1/-1->5->4
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 09/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Trees [0] 4/-1/-1->3->2 [1] 4/-1/-1->3->2 [2] 4/-1/-1->3->2 [3] 4/-1/-1->3->2 [4] 4/-1/-1->3->2 [5] 4/-1/-1->3->2 [6] 4/-1/-1->3->2 [7] 4/-1/-1->3->2 [8] 4/-1/-1->3->2 [9] 4/-1/-1->3->2 [10] 4/-1/-1->3->2 [11] 4/-1/-1->3->2 [12] 4/-1/-1->3->2 [13] 4/-1/-1->3->2 [14] 4/-1/-1->3->2 [15] 4/-1/-1->3->2 [16] 4/-1/-1->3->2 [17] 4/-1/-1->3->2 [18] 4/-1/-1->3->2 [19] 4/-1/-1->3->2 [20] 4/-1/-1->3->2 [21] 4/-1/-1->3->2 [22] 4/-1/-1->3->2 [23] 4/-1/-1->3->2
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Trees [0] 5/-1/-1->4->3 [1] 5/-1/-1->4->3 [2] 5/-1/-1->4->3 [3] 5/-1/-1->4->3 [4] 5/-1/-1->4->3 [5] 5/-1/-1->4->3 [6] 5/-1/-1->4->3 [7] 5/-1/-1->4->3 [8] 5/-1/-1->4->3 [9] 5/-1/-1->4->3 [10] 5/-1/-1->4->3 [11] 5/-1/-1->4->3 [12] 5/-1/-1->4->3 [13] 5/-1/-1->4->3 [14] 5/-1/-1->4->3 [15] 5/-1/-1->4->3 [16] 5/-1/-1->4->3 [17] 5/-1/-1->4->3 [18] 5/-1/-1->4->3 [19] 5/-1/-1->4->3 [20] 5/-1/-1->4->3 [21] 5/-1/-1->4->3 [22] 5/-1/-1->4->3 [23] 5/-1/-1->4->3
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 10/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 11/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 12/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 13/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 14/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 15/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 16/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 17/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 18/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 19/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 20/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 21/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 22/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 23/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 [4] 1/-1/-1->0->-1 [5] 1/-1/-1->0->-1 [6] 1/-1/-1->0->-1 [7] 1/-1/-1->0->-1 [8] 1/-1/-1->0->-1 [9] 1/-1/-1->0->-1 [10] 1/-1/-1->0->-1 [11] 1/-1/-1->0->-1 [12] 1/-1/-1->0->-1 [13] 1/-1/-1->0->-1 [14] 1/-1/-1->0->-1 [15] 1/-1/-1->0->-1 [16] 1/-1/-1->0->-1 [17] 1/-1/-1->0->-1 [18] 1/-1/-1->0->-1 [19] 1/-1/-1->0->-1 [20] 1/-1/-1->0->-1 [21] 1/-1/-1->0->-1 [22] 1/-1/-1->0->-1 [23] 1/-1/-1->0->-1
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 00/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 00/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 00/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 00/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 00/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 00/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 00/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 00/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 01/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 01/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 01/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 01/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 01/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 01/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 01/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 01/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 02/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 02/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 02/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 02/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 02/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 02/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 02/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 02/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 03/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 03/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 03/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 03/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 03/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 03/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 03/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 03/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 04/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 04/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 04/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 04/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 04/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 04/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 04/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 04/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 05/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 05/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 05/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 05/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 05/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 05/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 05/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 05/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 06/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 06/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 06/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 06/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 06/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 06/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 06/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 07/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 06/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 07/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 07/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 07/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 07/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 07/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 07/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 08/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 07/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 08/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 08/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 08/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 08/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 08/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 08/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 09/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 08/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 09/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 09/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 09/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 09/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 09/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 09/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 10/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 09/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 10/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 10/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 10/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 10/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 10/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 10/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 11/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 10/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 11/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 11/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 11/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 11/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 11/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 11/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 12/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 11/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 12/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 12/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 12/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 12/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 12/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 12/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 13/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 12/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 13/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 13/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 13/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 13/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 13/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 13/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 14/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 13/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 14/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 14/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 14/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 14/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 14/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 14/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 15/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 14/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 15/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 15/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 15/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 15/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 15/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 15/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 16/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 15/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 16/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 16/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 16/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 16/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 16/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 16/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 17/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 16/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 17/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 17/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 17/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 17/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 17/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 17/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 18/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 17/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 18/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 18/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 18/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 18/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 18/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 18/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 19/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 18/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 19/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 19/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 19/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 19/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 19/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 19/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 20/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 19/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 20/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 20/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 20/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 20/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 20/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 20/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 21/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 20/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 21/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 21/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 21/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 21/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 21/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 21/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 22/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 21/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 22/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 22/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 22/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 22/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 22/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 22/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 23/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 22/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 23/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 23/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 23/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 23/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Channel 23/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 23/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 23/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Connected all rings
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 00/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Connected all rings
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Connected all rings
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Connected all rings
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Connected all rings
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Connected all rings
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Connected all rings
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Connected all rings
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 01/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 02/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 03/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 04/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 05/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 06/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 07/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 08/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 09/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 10/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 11/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 12/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 13/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 14/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 15/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 16/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 17/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 18/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 19/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 20/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 21/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 22/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Channel 23/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 00/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 00/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 00/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 00/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 00/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 00/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 01/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 01/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 01/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 01/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 01/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 01/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 02/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 02/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 02/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 02/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 02/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 02/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 03/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 03/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 03/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 03/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 03/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 03/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 04/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 04/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 04/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 04/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 04/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 04/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 05/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 05/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 05/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 05/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 05/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 06/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 06/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 06/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 06/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 06/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 05/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 07/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 07/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 07/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 07/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 07/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 06/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 08/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 08/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 08/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 08/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 07/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 08/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 09/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 09/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 09/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 08/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 09/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 09/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 10/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 10/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 09/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 10/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 10/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 10/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 11/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 11/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 10/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 11/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 11/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 11/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 12/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 12/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 11/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 12/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 12/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 12/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 13/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 13/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 12/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 13/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 13/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 13/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 14/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 14/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 13/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 14/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 14/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 14/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 15/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 15/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 14/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 15/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 15/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 15/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 16/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 16/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 16/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 15/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 16/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 16/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 17/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 17/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 17/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 16/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 17/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 17/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 18/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 18/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 18/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 17/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 18/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 18/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 19/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 19/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 19/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 18/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 19/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 19/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 20/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 20/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 20/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 19/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 20/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 20/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 21/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 21/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 21/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 20/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 21/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 21/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 22/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 22/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 22/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 21/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 22/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 22/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Channel 23/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Channel 23/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Channel 23/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 22/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Channel 23/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Channel 23/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Channel 23/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO Connected all trees
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO Connected all trees
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO Connected all trees
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO Connected all trees
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO Connected all trees
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO Connected all trees
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO Connected all trees
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO Connected all trees
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148476 [2] NCCL INFO comm 0x49101200 rank 2 nranks 8 cudaDev 2 busId 4b000 - Init COMPLETE
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148481 [3] NCCL INFO comm 0x47342a50 rank 3 nranks 8 cudaDev 3 busId 51000 - Init COMPLETE
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148482 [5] NCCL INFO comm 0x46d7e200 rank 5 nranks 8 cudaDev 5 busId 99000 - Init COMPLETE
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148477 [7] NCCL INFO comm 0x4a9bd320 rank 7 nranks 8 cudaDev 7 busId d0000 - Init COMPLETE
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148475 [0] NCCL INFO comm 0x459a5520 rank 0 nranks 8 cudaDev 0 busId e000 - Init COMPLETE
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148478 [1] NCCL INFO comm 0x46e04740 rank 1 nranks 8 cudaDev 1 busId 13000 - Init COMPLETE
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148480 [6] NCCL INFO comm 0x4ace7470 rank 6 nranks 8 cudaDev 6 busId cb000 - Init COMPLETE
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148479 [4] NCCL INFO comm 0x485482a0 rank 4 nranks 8 cudaDev 4 busId 93000 - Init COMPLETE
01/04/2024 21:10:53 - INFO - __main__ - xxx: Showcase the tokenized training samples.
{'input_ids': [1, 13866, 338, 385, 15278, 393, 16612, 263, 3414, 29889, 14350, 263, 2933, 393, 7128, 2486, 1614, 2167, 278, 2009, 29889, 13, 13, 2277, 29937, 2799, 4080, 29901, 13, 29954, 573, 2211, 25562, 363, 7952, 292, 9045, 29891, 29889, 13, 13, 2277, 29937, 13291, 29901, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2]}
{'input_ids': [1, 13866, 338, 385, 15278, 393, 16612, 263, 3414, 29889, 14350, 263, 2933, 393, 7128, 2486, 1614, 2167, 278, 2009, 29889, 13, 13, 2277, 29937, 2799, 4080, 29901, 13, 29954, 573, 2211, 25562, 363, 7952, 292, 9045, 29891, 29889, 13, 13, 2277, 29937, 13291, 29901, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2]}
{'input_ids': [1, 13866, 338, 385, 15278, 393, 16612, 263, 3414, 29889, 14350, 263, 2933, 393, 7128, 2486, 1614, 2167, 278, 2009, 29889, 13, 13, 2277, 29937, 2799, 4080, 29901, 13, 29954, 573, 2211, 25562, 363, 7952, 292, 9045, 29891, 29889, 13, 13, 2277, 29937, 13291, 29901, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2]}
{'input_ids': [1, 13866, 338, 385, 15278, 393, 16612, 263, 3414, 29889, 14350, 263, 2933, 393, 7128, 2486, 1614, 2167, 278, 2009, 29889, 13, 13, 2277, 29937, 2799, 4080, 29901, 13, 29954, 573, 2211, 25562, 363, 7952, 292, 9045, 29891, 29889, 13, 13, 2277, 29937, 13291, 29901, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2]}
{'input_ids': [1, 13866, 338, 385, 15278, 393, 16612, 263, 3414, 29889, 14350, 263, 2933, 393, 7128, 2486, 1614, 2167, 278, 2009, 29889, 13, 13, 2277, 29937, 2799, 4080, 29901, 13, 29954, 573, 2211, 25562, 363, 7952, 292, 9045, 29891, 29889, 13, 13, 2277, 29937, 13291, 29901, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2]}
{'input_ids': [1, 13866, 338, 385, 15278, 393, 16612, 263, 3414, 29889, 14350, 263, 2933, 393, 7128, 2486, 1614, 2167, 278, 2009, 29889, 13, 13, 2277, 29937, 2799, 4080, 29901, 13, 29954, 573, 2211, 25562, 363, 7952, 292, 9045, 29891, 29889, 13, 13, 2277, 29937, 13291, 29901, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2]}
{'input_ids': [1, 13866, 338, 385, 15278, 393, 16612, 263, 3414, 29889, 14350, 263, 2933, 393, 7128, 2486, 1614, 2167, 278, 2009, 29889, 13, 13, 2277, 29937, 2799, 4080, 29901, 13, 29954, 573, 2211, 25562, 363, 7952, 292, 9045, 29891, 29889, 13, 13, 2277, 29937, 13291, 29901, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2]}
{'input_ids': [1, 13866, 338, 385, 15278, 393, 16612, 263, 3414, 29889, 14350, 263, 2933, 393, 7128, 2486, 1614, 2167, 278, 2009, 29889, 13, 13, 2277, 29937, 2799, 4080, 29901, 13, 29954, 573, 2211, 25562, 363, 7952, 292, 9045, 29891, 29889, 13, 13, 2277, 29937, 13291, 29901, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2]}
{'input_ids': [1, 13866, 338, 385, 15278, 393, 16612, 263, 3414, 29889, 14350, 263, 2933, 393, 7128, 2486, 1614, 2167, 278, 2009, 29889, 13, 13, 2277, 29937, 2799, 4080, 29901, 13, 29954, 573, 2211, 25562, 363, 7952, 292, 9045, 29891, 29889, 13, 13, 2277, 29937, 13291, 29901, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2]}
{'input_ids': [1, 13866, 338, 385, 15278, 393, 16612, 263, 3414, 29889, 14350, 263, 2933, 393, 7128, 2486, 1614, 2167, 278, 2009, 29889, 13, 13, 2277, 29937, 2799, 4080, 29901, 13, 29954, 573, 2211, 25562, 363, 7952, 292, 9045, 29891, 29889, 13, 13, 2277, 29937, 13291, 29901, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2]}
{'input_ids': [1, 13866, 338, 385, 15278, 393, 16612, 263, 3414, 29889, 14350, 263, 2933, 393, 7128, 2486, 1614, 2167, 278, 2009, 29889, 13, 13, 2277, 29937, 2799, 4080, 29901, 13, 29954, 573, 2211, 25562, 363, 7952, 292, 9045, 29891, 29889, 13, 13, 2277, 29937, 13291, 29901, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2]}
{'input_ids': [1, 13866, 338, 385, 15278, 393, 16612, 263, 3414, 29889, 14350, 263, 2933, 393, 7128, 2486, 1614, 2167, 278, 2009, 29889, 13, 13, 2277, 29937, 2799, 4080, 29901, 13, 29954, 573, 2211, 25562, 363, 7952, 292, 9045, 29891, 29889, 13, 13, 2277, 29937, 13291, 29901, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2]}
{'input_ids': [1, 13866, 338, 385, 15278, 393, 16612, 263, 3414, 29889, 14350, 263, 2933, 393, 7128, 2486, 1614, 2167, 278, 2009, 29889, 13, 13, 2277, 29937, 2799, 4080, 29901, 13, 29954, 573, 2211, 25562, 363, 7952, 292, 9045, 29891, 29889, 13, 13, 2277, 29937, 13291, 29901, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2]}
{'input_ids': [1, 13866, 338, 385, 15278, 393, 16612, 263, 3414, 29889, 14350, 263, 2933, 393, 7128, 2486, 1614, 2167, 278, 2009, 29889, 13, 13, 2277, 29937, 2799, 4080, 29901, 13, 29954, 573, 2211, 25562, 363, 7952, 292, 9045, 29891, 29889, 13, 13, 2277, 29937, 13291, 29901, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2]}
{'input_ids': [1, 13866, 338, 385, 15278, 393, 16612, 263, 3414, 29889, 14350, 263, 2933, 393, 7128, 2486, 1614, 2167, 278, 2009, 29889, 13, 13, 2277, 29937, 2799, 4080, 29901, 13, 29954, 573, 2211, 25562, 363, 7952, 292, 9045, 29891, 29889, 13, 13, 2277, 29937, 13291, 29901, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2]}
{'input_ids': [1, 13866, 338, 385, 15278, 393, 16612, 263, 3414, 29889, 14350, 263, 2933, 393, 7128, 2486, 1614, 2167, 278, 2009, 29889, 13, 13, 2277, 29937, 2799, 4080, 29901, 13, 29954, 573, 2211, 25562, 363, 7952, 292, 9045, 29891, 29889, 13, 13, 2277, 29937, 13291, 29901, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2]}
{'input_ids': [1, 13866, 338, 385, 15278, 393, 16612, 263, 3414, 29889, 14350, 263, 2933, 393, 7128, 2486, 1614, 2167, 278, 2009, 29889, 13, 13, 2277, 29937, 2799, 4080, 29901, 13, 29954, 573, 2211, 25562, 363, 7952, 292, 9045, 29891, 29889, 13, 13, 2277, 29937, 13291, 29901, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2]}
{'input_ids': [1, 13866, 338, 385, 15278, 393, 16612, 263, 3414, 29889, 14350, 263, 2933, 393, 7128, 2486, 1614, 2167, 278, 2009, 29889, 13, 13, 2277, 29937, 2799, 4080, 29901, 13, 29954, 573, 2211, 25562, 363, 7952, 292, 9045, 29891, 29889, 13, 13, 2277, 29937, 13291, 29901, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2]}
{'input_ids': [1, 13866, 338, 385, 15278, 393, 16612, 263, 3414, 29889, 14350, 263, 2933, 393, 7128, 2486, 1614, 2167, 278, 2009, 29889, 13, 13, 2277, 29937, 2799, 4080, 29901, 13, 29954, 573, 2211, 25562, 363, 7952, 292, 9045, 29891, 29889, 13, 13, 2277, 29937, 13291, 29901, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2]}
{'input_ids': [1, 13866, 338, 385, 15278, 393, 16612, 263, 3414, 29889, 14350, 263, 2933, 393, 7128, 2486, 1614, 2167, 278, 2009, 29889, 13, 13, 2277, 29937, 2799, 4080, 29901, 13, 29954, 573, 2211, 25562, 363, 7952, 292, 9045, 29891, 29889, 13, 13, 2277, 29937, 13291, 29901, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2]}
{'input_ids': [1, 13866, 338, 385, 15278, 393, 16612, 263, 3414, 29889, 14350, 263, 2933, 393, 7128, 2486, 1614, 2167, 278, 2009, 29889, 13, 13, 2277, 29937, 2799, 4080, 29901, 13, 29954, 573, 2211, 25562, 363, 7952, 292, 9045, 29891, 29889, 13, 13, 2277, 29937, 13291, 29901, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2]}
{'input_ids': [1, 13866, 338, 385, 15278, 393, 16612, 263, 3414, 29889, 14350, 263, 2933, 393, 7128, 2486, 1614, 2167, 278, 2009, 29889, 13, 13, 2277, 29937, 2799, 4080, 29901, 13, 29954, 573, 2211, 25562, 363, 7952, 292, 9045, 29891, 29889, 13, 13, 2277, 29937, 13291, 29901, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2]}
{'input_ids': [1, 13866, 338, 385, 15278, 393, 16612, 263, 3414, 29889, 14350, 263, 2933, 393, 7128, 2486, 1614, 2167, 278, 2009, 29889, 13, 13, 2277, 29937, 2799, 4080, 29901, 13, 29954, 573, 2211, 25562, 363, 7952, 292, 9045, 29891, 29889, 13, 13, 2277, 29937, 13291, 29901, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2]}
{'input_ids': [1, 13866, 338, 385, 15278, 393, 16612, 263, 3414, 29889, 14350, 263, 2933, 393, 7128, 2486, 1614, 2167, 278, 2009, 29889, 13, 13, 2277, 29937, 2799, 4080, 29901, 13, 29954, 573, 2211, 25562, 363, 7952, 292, 9045, 29891, 29889, 13, 13, 2277, 29937, 13291, 29901, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 29896, 29889, 382, 271, 263, 6411, 8362, 322, 18254, 768, 2738, 652, 300, 29901, 8561, 1854, 596, 592, 1338, 526, 20978, 573, 310, 263, 12875, 310, 285, 21211, 322, 18655, 1849, 29892, 20793, 26823, 29892, 3353, 2646, 1144, 29892, 322, 9045, 29891, 285, 1446, 29889, 910, 6911, 304, 3867, 596, 3573, 411, 278, 18853, 18254, 374, 1237, 304, 740, 472, 967, 1900, 322, 508, 1371, 5557, 17168, 293, 10267, 2129, 29889, 13, 13, 29906, 29889, 2201, 482, 297, 4943, 9128, 6354, 29901, 1222, 6269, 895, 338, 7618, 1455, 363, 7344, 292, 4549, 289, 2873, 29892, 2301, 7799, 29892, 322, 5881, 29875, 586, 6151, 1070, 9045, 29889, 319, 326, 363, 472, 3203, 29871, 29896, 29945, 29900, 6233, 310, 17768, 403, 14911, 711, 293, 15058, 470, 29871, 29955, 29945, 6233, 310, 14877, 20657, 15058, 1269, 4723, 29889, 13, 13, 29941, 29889, 3617, 3307, 8709, 29901, 24162, 3307, 11029, 8709, 338, 7618, 1455, 363, 9128, 322, 19119, 1532, 29899, 915, 292, 29889, 739, 6911, 304, 1072, 5987, 286, 2092, 29892, 11157, 25323, 3321, 740, 29892, 322, 11286, 9045, 29891, 14321, 322, 5198, 1540, 740, 29889, 319, 326, 363, 29871, 29955, 29899, 29929, 6199, 310, 8709, 1269, 4646, 29889, 2]}
[INFO|trainer.py:606] 2024-01-04 21:10:54,615 >> Using cuda_amp half precision backend
[2024-01-04 21:10:54,625] [INFO] [logging.py:96:log_dist] [Rank 0] DeepSpeed info: version=0.11.1, git-hash=unknown, git-branch=unknown
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Using network IB
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Using network IB
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Using network IB
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Using network IB
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Using network IB
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Using network IB
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Using network IB
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Using network IB
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Setting affinity for GPU 7 to ffffffff,ffff0000,00000000,ffffffff,ffff0000,00000000
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Setting affinity for GPU 0 to ffff,ffffffff,00000000,0000ffff,ffffffff
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Setting affinity for GPU 2 to ffff,ffffffff,00000000,0000ffff,ffffffff
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Setting affinity for GPU 1 to ffff,ffffffff,00000000,0000ffff,ffffffff
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Setting affinity for GPU 6 to ffffffff,ffff0000,00000000,ffffffff,ffff0000,00000000
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Setting affinity for GPU 4 to ffffffff,ffff0000,00000000,ffffffff,ffff0000,00000000
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Setting affinity for GPU 5 to ffffffff,ffff0000,00000000,ffffffff,ffff0000,00000000
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Setting affinity for GPU 3 to ffff,ffffffff,00000000,0000ffff,ffffffff
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 00/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 01/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 02/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 03/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 04/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 05/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Trees [0] 3/-1/-1->2->1 [1] 3/-1/-1->2->1 [2] 3/-1/-1->2->1 [3] 3/-1/-1->2->1 [4] 3/-1/-1->2->1 [5] 3/-1/-1->2->1 [6] 3/-1/-1->2->1 [7] 3/-1/-1->2->1 [8] 3/-1/-1->2->1 [9] 3/-1/-1->2->1 [10] 3/-1/-1->2->1 [11] 3/-1/-1->2->1 [12] 3/-1/-1->2->1 [13] 3/-1/-1->2->1 [14] 3/-1/-1->2->1 [15] 3/-1/-1->2->1 [16] 3/-1/-1->2->1 [17] 3/-1/-1->2->1 [18] 3/-1/-1->2->1 [19] 3/-1/-1->2->1 [20] 3/-1/-1->2->1 [21] 3/-1/-1->2->1 [22] 3/-1/-1->2->1 [23] 3/-1/-1->2->1
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Trees [0] 2/-1/-1->1->0 [1] 2/-1/-1->1->0 [2] 2/-1/-1->1->0 [3] 2/-1/-1->1->0 [4] 2/-1/-1->1->0 [5] 2/-1/-1->1->0 [6] 2/-1/-1->1->0 [7] 2/-1/-1->1->0 [8] 2/-1/-1->1->0 [9] 2/-1/-1->1->0 [10] 2/-1/-1->1->0 [11] 2/-1/-1->1->0 [12] 2/-1/-1->1->0 [13] 2/-1/-1->1->0 [14] 2/-1/-1->1->0 [15] 2/-1/-1->1->0 [16] 2/-1/-1->1->0 [17] 2/-1/-1->1->0 [18] 2/-1/-1->1->0 [19] 2/-1/-1->1->0 [20] 2/-1/-1->1->0 [21] 2/-1/-1->1->0 [22] 2/-1/-1->1->0 [23] 2/-1/-1->1->0
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 06/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Trees [0] -1/-1/-1->7->6 [1] -1/-1/-1->7->6 [2] -1/-1/-1->7->6 [3] -1/-1/-1->7->6 [4] -1/-1/-1->7->6 [5] -1/-1/-1->7->6 [6] -1/-1/-1->7->6 [7] -1/-1/-1->7->6 [8] -1/-1/-1->7->6 [9] -1/-1/-1->7->6 [10] -1/-1/-1->7->6 [11] -1/-1/-1->7->6 [12] -1/-1/-1->7->6 [13] -1/-1/-1->7->6 [14] -1/-1/-1->7->6 [15] -1/-1/-1->7->6 [16] -1/-1/-1->7->6 [17] -1/-1/-1->7->6 [18] -1/-1/-1->7->6 [19] -1/-1/-1->7->6 [20] -1/-1/-1->7->6 [21] -1/-1/-1->7->6 [22] -1/-1/-1->7->6 [23] -1/-1/-1->7->6
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Trees [0] 7/-1/-1->6->5 [1] 7/-1/-1->6->5 [2] 7/-1/-1->6->5 [3] 7/-1/-1->6->5 [4] 7/-1/-1->6->5 [5] 7/-1/-1->6->5 [6] 7/-1/-1->6->5 [7] 7/-1/-1->6->5 [8] 7/-1/-1->6->5 [9] 7/-1/-1->6->5 [10] 7/-1/-1->6->5 [11] 7/-1/-1->6->5 [12] 7/-1/-1->6->5 [13] 7/-1/-1->6->5 [14] 7/-1/-1->6->5 [15] 7/-1/-1->6->5 [16] 7/-1/-1->6->5 [17] 7/-1/-1->6->5 [18] 7/-1/-1->6->5 [19] 7/-1/-1->6->5 [20] 7/-1/-1->6->5 [21] 7/-1/-1->6->5 [22] 7/-1/-1->6->5 [23] 7/-1/-1->6->5
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 07/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Trees [0] 4/-1/-1->3->2 [1] 4/-1/-1->3->2 [2] 4/-1/-1->3->2 [3] 4/-1/-1->3->2 [4] 4/-1/-1->3->2 [5] 4/-1/-1->3->2 [6] 4/-1/-1->3->2 [7] 4/-1/-1->3->2 [8] 4/-1/-1->3->2 [9] 4/-1/-1->3->2 [10] 4/-1/-1->3->2 [11] 4/-1/-1->3->2 [12] 4/-1/-1->3->2 [13] 4/-1/-1->3->2 [14] 4/-1/-1->3->2 [15] 4/-1/-1->3->2 [16] 4/-1/-1->3->2 [17] 4/-1/-1->3->2 [18] 4/-1/-1->3->2 [19] 4/-1/-1->3->2 [20] 4/-1/-1->3->2 [21] 4/-1/-1->3->2 [22] 4/-1/-1->3->2 [23] 4/-1/-1->3->2
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 08/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 09/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Trees [0] 6/-1/-1->5->4 [1] 6/-1/-1->5->4 [2] 6/-1/-1->5->4 [3] 6/-1/-1->5->4 [4] 6/-1/-1->5->4 [5] 6/-1/-1->5->4 [6] 6/-1/-1->5->4 [7] 6/-1/-1->5->4 [8] 6/-1/-1->5->4 [9] 6/-1/-1->5->4 [10] 6/-1/-1->5->4 [11] 6/-1/-1->5->4 [12] 6/-1/-1->5->4 [13] 6/-1/-1->5->4 [14] 6/-1/-1->5->4 [15] 6/-1/-1->5->4 [16] 6/-1/-1->5->4 [17] 6/-1/-1->5->4 [18] 6/-1/-1->5->4 [19] 6/-1/-1->5->4 [20] 6/-1/-1->5->4 [21] 6/-1/-1->5->4 [22] 6/-1/-1->5->4 [23] 6/-1/-1->5->4
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Trees [0] 5/-1/-1->4->3 [1] 5/-1/-1->4->3 [2] 5/-1/-1->4->3 [3] 5/-1/-1->4->3 [4] 5/-1/-1->4->3 [5] 5/-1/-1->4->3 [6] 5/-1/-1->4->3 [7] 5/-1/-1->4->3 [8] 5/-1/-1->4->3 [9] 5/-1/-1->4->3 [10] 5/-1/-1->4->3 [11] 5/-1/-1->4->3 [12] 5/-1/-1->4->3 [13] 5/-1/-1->4->3 [14] 5/-1/-1->4->3 [15] 5/-1/-1->4->3 [16] 5/-1/-1->4->3 [17] 5/-1/-1->4->3 [18] 5/-1/-1->4->3 [19] 5/-1/-1->4->3 [20] 5/-1/-1->4->3 [21] 5/-1/-1->4->3 [22] 5/-1/-1->4->3 [23] 5/-1/-1->4->3
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 10/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 11/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 12/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 13/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 14/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 15/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 16/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 17/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 18/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 19/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 20/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 21/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 22/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 23/24 :    0   1   2   3   4   5   6   7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 [4] 1/-1/-1->0->-1 [5] 1/-1/-1->0->-1 [6] 1/-1/-1->0->-1 [7] 1/-1/-1->0->-1 [8] 1/-1/-1->0->-1 [9] 1/-1/-1->0->-1 [10] 1/-1/-1->0->-1 [11] 1/-1/-1->0->-1 [12] 1/-1/-1->0->-1 [13] 1/-1/-1->0->-1 [14] 1/-1/-1->0->-1 [15] 1/-1/-1->0->-1 [16] 1/-1/-1->0->-1 [17] 1/-1/-1->0->-1 [18] 1/-1/-1->0->-1 [19] 1/-1/-1->0->-1 [20] 1/-1/-1->0->-1 [21] 1/-1/-1->0->-1 [22] 1/-1/-1->0->-1 [23] 1/-1/-1->0->-1
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 00/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 00/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 00/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 00/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 00/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 00/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 00/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 00/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 01/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 01/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 01/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 01/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 01/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 01/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 01/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 01/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 02/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 02/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 02/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 02/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 02/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 02/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 02/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 02/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 03/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 03/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 03/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 03/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 03/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 03/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 03/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 03/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 04/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 04/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 04/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 04/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 04/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 04/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 04/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 04/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 05/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 05/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 05/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 05/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 05/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 05/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 05/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 05/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 06/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 06/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 06/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 06/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 06/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 06/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 06/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 06/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 07/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 07/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 07/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 07/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 07/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 07/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 07/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 07/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 08/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 08/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 08/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 08/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 08/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 08/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 08/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 08/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 09/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 09/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 09/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 09/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 09/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 09/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 09/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 09/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 10/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 10/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 10/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 10/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 10/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 10/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 10/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 10/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 11/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 11/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 11/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 11/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 11/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 11/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 11/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 11/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 12/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 12/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 12/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 12/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 12/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 12/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 12/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 12/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 13/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 13/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 13/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 13/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 13/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 13/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 13/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 13/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 14/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 14/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 14/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 14/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 14/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 14/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 14/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 14/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 15/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 15/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 15/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 15/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 15/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 15/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 15/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 15/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 16/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 16/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 16/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 16/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 16/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 16/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 16/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 16/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 17/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 17/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 17/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 17/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 17/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 17/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 17/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 17/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 18/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 18/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 18/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 18/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 18/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 18/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 18/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 18/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 19/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 19/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 19/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 19/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 19/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 19/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 19/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 19/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 20/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 20/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 20/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 20/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 20/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 20/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 20/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 20/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 21/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 21/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 21/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 21/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 21/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 21/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 21/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 21/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 22/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 22/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 22/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 22/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 22/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 22/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 22/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 22/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 23/0 : 3[51000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Channel 23/0 : 0[e000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 23/0 : 7[d0000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 23/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 23/0 : 5[99000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 23/0 : 4[93000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 23/0 : 1[13000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 23/0 : 2[4b000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Connected all rings
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 00/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Connected all rings
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Connected all rings
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Connected all rings
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Connected all rings
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Connected all rings
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Connected all rings
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Connected all rings
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 01/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 02/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 03/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 04/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 05/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 06/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 07/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 08/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 09/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 10/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 11/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 12/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 13/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 14/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 15/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 16/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 17/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 18/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 19/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 20/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 21/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 22/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Channel 23/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 00/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 00/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 00/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 00/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 00/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 00/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 01/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 01/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 01/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 01/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 01/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 01/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 02/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 02/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 02/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 02/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 02/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 02/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 03/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 03/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 03/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 03/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 03/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 03/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 04/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 04/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 04/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 04/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 04/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 04/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 05/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 05/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 05/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 05/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 05/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 05/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 06/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 06/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 06/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 06/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 06/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 06/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 07/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 07/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 07/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 07/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 07/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 07/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 08/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 08/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 08/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 08/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 08/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 08/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 09/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 09/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 09/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 09/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 09/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 09/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 10/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 10/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 10/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 10/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 10/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 10/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 11/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 11/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 11/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 11/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 11/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 11/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 12/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 12/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 12/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 12/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 12/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 12/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 13/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 13/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 13/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 13/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 13/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 13/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 14/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 14/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 14/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 14/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 14/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 14/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 15/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 15/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 15/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 15/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 15/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 15/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 16/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 16/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 16/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 16/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 16/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 16/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 17/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 17/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 17/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 17/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 17/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 17/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 18/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 18/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 18/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 18/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 18/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 18/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 19/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 19/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 19/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 19/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 19/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 19/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 20/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 20/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 20/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 20/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 20/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 20/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 21/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 21/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 21/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 21/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 21/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 21/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 22/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 22/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 22/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 22/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 22/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 22/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Channel 23/0 : 6[cb000] -> 5[99000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Channel 23/0 : 4[93000] -> 3[51000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Channel 23/0 : 5[99000] -> 4[93000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Channel 23/0 : 2[4b000] -> 1[13000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Channel 23/0 : 3[51000] -> 2[4b000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Channel 23/0 : 1[13000] -> 0[e000] via P2P/IPC/read
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO Connected all trees
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO Connected all trees
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO Connected all trees
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO Connected all trees
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO Connected all trees
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO Connected all trees
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO Connected all trees
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO Connected all trees
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148536 [3] NCCL INFO comm 0xfed6140 rank 3 nranks 8 cudaDev 3 busId 51000 - Init COMPLETE
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148540 [5] NCCL INFO comm 0xebe1740 rank 5 nranks 8 cudaDev 5 busId 99000 - Init COMPLETE
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148535 [4] NCCL INFO comm 0x11148f80 rank 4 nranks 8 cudaDev 4 busId 93000 - Init COMPLETE
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148539 [6] NCCL INFO comm 0xe03b780 rank 6 nranks 8 cudaDev 6 busId cb000 - Init COMPLETE
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148541 [7] NCCL INFO comm 0xfb14910 rank 7 nranks 8 cudaDev 7 busId d0000 - Init COMPLETE
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148534 [0] NCCL INFO comm 0xe9add00 rank 0 nranks 8 cudaDev 0 busId e000 - Init COMPLETE
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148537 [2] NCCL INFO comm 0xe0e44c0 rank 2 nranks 8 cudaDev 2 busId 4b000 - Init COMPLETE
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148538 [1] NCCL INFO comm 0xf18bcc0 rank 1 nranks 8 cudaDev 1 busId 13000 - Init COMPLETE
[2024-01-04 21:11:11,051] [INFO] [logging.py:96:log_dist] [Rank 0] DeepSpeed Flops Profiler Enabled: False
Using /root/.cache/torch_extensions/py38_cu117 as PyTorch extensions root...
Using /root/.cache/torch_extensions/py38_cu117 as PyTorch extensions root...
Using /root/.cache/torch_extensions/py38_cu117 as PyTorch extensions root...
Detected CUDA files, patching ldflags
Emitting ninja build file /root/.cache/torch_extensions/py38_cu117/cpu_adam/build.ninja...
Building extension module cpu_adam...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
Using /root/.cache/torch_extensions/py38_cu117 as PyTorch extensions root...
ninja: no work to do.
Loading extension module cpu_adam...
Time to load cpu_adam op: 1.145982265472412 seconds
Loading extension module cpu_adam...
Time to load cpu_adam op: 0.9791653156280518 seconds
Using /root/.cache/torch_extensions/py38_cu117 as PyTorch extensions root...
Using /root/.cache/torch_extensions/py38_cu117 as PyTorch extensions root...
Using /root/.cache/torch_extensions/py38_cu117 as PyTorch extensions root...
Using /root/.cache/torch_extensions/py38_cu117 as PyTorch extensions root...
Detected CUDA files, patching ldflags
Emitting ninja build file /root/.cache/torch_extensions/py38_cu117/cpu_adam/build.ninja...
Building extension module cpu_adam...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.
Loading extension module cpu_adam...
Time to load cpu_adam op: 1.0675396919250488 seconds
Loading extension module cpu_adam...Loading extension module cpu_adam...

Time to load cpu_adam op: 1.2839298248291016 seconds
Time to load cpu_adam op: 1.363921880722046 seconds
Loading extension module cpu_adam...
Time to load cpu_adam op: 1.0944368839263916 seconds
Loading extension module cpu_adam...
Time to load cpu_adam op: 1.0702216625213623 seconds
Loading extension module cpu_adam...
Time to load cpu_adam op: 1.122896671295166 seconds
Adam Optimizer #0 is created with AVX2 arithmetic capability.
Config: alpha=0.000020, betas=(0.900000, 0.999000), weight_decay=0.000000, adam_w=1
[2024-01-04 21:11:16,982] [INFO] [logging.py:96:log_dist] [Rank 0] Using DeepSpeed Optimizer param name adam as basic optimizer
[2024-01-04 21:11:16,982] [INFO] [logging.py:96:log_dist] [Rank 0] Removing param_group that has no 'params' in the basic Optimizer
[2024-01-04 21:11:17,003] [INFO] [logging.py:96:log_dist] [Rank 0] DeepSpeed Basic Optimizer = DeepSpeedCPUAdam
[2024-01-04 21:11:17,003] [INFO] [utils.py:56:is_zero_supported_optimizer] Checking ZeRO support for optimizer=DeepSpeedCPUAdam type=<class 'deepspeed.ops.adam.cpu_adam.DeepSpeedCPUAdam'>
[2024-01-04 21:11:17,003] [INFO] [logging.py:96:log_dist] [Rank 0] Creating torch.bfloat16 ZeRO stage 2 optimizer
[2024-01-04 21:11:17,003] [INFO] [stage_1_and_2.py:146:__init__] Reduce bucket size 200000000
[2024-01-04 21:11:17,003] [INFO] [stage_1_and_2.py:147:__init__] Allgather bucket size 200000000
[2024-01-04 21:11:17,003] [INFO] [stage_1_and_2.py:148:__init__] CPU Offload: True
[2024-01-04 21:11:17,003] [INFO] [stage_1_and_2.py:149:__init__] Round robin gradient partitioning: False
[2024-01-04 21:11:34,699] [INFO] [utils.py:802:see_memory_usage] Before initializing optimizer states
[2024-01-04 21:11:34,700] [INFO] [utils.py:803:see_memory_usage] MA 12.83 GB         Max_MA 12.83 GB         CA 12.83 GB         Max_CA 13 GB 
[2024-01-04 21:11:34,700] [INFO] [utils.py:810:see_memory_usage] CPU Virtual Memory:  used = 89.29 GB, percent = 8.9%
[WARNING|logging.py:295] 2024-01-04 21:11:40,147 >> `use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...
[WARNING|logging.py:295] 2024-01-04 21:11:40,264 >> `use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...
[2024-01-04 21:11:40,350] [INFO] [utils.py:802:see_memory_usage] After initializing optimizer states
[2024-01-04 21:11:40,351] [INFO] [utils.py:803:see_memory_usage] MA 12.83 GB         Max_MA 12.83 GB         CA 12.83 GB         Max_CA 13 GB 
[2024-01-04 21:11:40,351] [INFO] [utils.py:810:see_memory_usage] CPU Virtual Memory:  used = 162.5 GB, percent = 16.1%
[2024-01-04 21:11:40,351] [INFO] [stage_1_and_2.py:513:__init__] optimizer state initialized
[2024-01-04 21:11:40,494] [INFO] [utils.py:802:see_memory_usage] After initializing ZeRO optimizer
[2024-01-04 21:11:40,495] [INFO] [utils.py:803:see_memory_usage] MA 12.83 GB         Max_MA 12.83 GB         CA 12.83 GB         Max_CA 13 GB 
[2024-01-04 21:11:40,495] [INFO] [utils.py:810:see_memory_usage] CPU Virtual Memory:  used = 158.35 GB, percent = 15.7%
[2024-01-04 21:11:40,498] [INFO] [logging.py:96:log_dist] [Rank 0] DeepSpeed Final Optimizer = adam
[2024-01-04 21:11:40,498] [INFO] [logging.py:96:log_dist] [Rank 0] DeepSpeed using client callable to create LR scheduler
[2024-01-04 21:11:40,498] [INFO] [logging.py:96:log_dist] [Rank 0] DeepSpeed LR Scheduler = <torch.optim.lr_scheduler.LambdaLR object at 0x7f571c03ca90>
[2024-01-04 21:11:40,498] [INFO] [logging.py:96:log_dist] [Rank 0] step=0, skipped=0, lr=[0.0], mom=[[0.9, 0.999]]
[2024-01-04 21:11:40,499] [INFO] [config.py:968:print] DeepSpeedEngine configuration:
[2024-01-04 21:11:40,500] [INFO] [config.py:972:print]   activation_checkpointing_config  {
    "partition_activations": false, 
    "contiguous_memory_optimization": false, 
    "cpu_checkpointing": false, 
    "number_checkpoints": null, 
    "synchronize_checkpoint_boundary": false, 
    "profile": false
}
[2024-01-04 21:11:40,500] [INFO] [config.py:972:print]   aio_config ................... {'block_size': 1048576, 'queue_depth': 8, 'thread_count': 1, 'single_submit': False, 'overlap_events': True}
[2024-01-04 21:11:40,500] [INFO] [config.py:972:print]   amp_enabled .................. False
[2024-01-04 21:11:40,500] [INFO] [config.py:972:print]   amp_params ................... False
[2024-01-04 21:11:40,500] [INFO] [config.py:972:print]   autotuning_config ............ {
    "enabled": false, 
    "start_step": null, 
    "end_step": null, 
    "metric_path": null, 
    "arg_mappings": null, 
    "metric": "throughput", 
    "model_info": null, 
    "results_dir": "autotuning_results", 
    "exps_dir": "autotuning_exps", 
    "overwrite": true, 
    "fast": true, 
    "start_profile_step": 3, 
    "end_profile_step": 5, 
    "tuner_type": "gridsearch", 
    "tuner_early_stopping": 5, 
    "tuner_num_trials": 50, 
    "model_info_path": null, 
    "mp_size": 1, 
    "max_train_batch_size": null, 
    "min_train_batch_size": 1, 
    "max_train_micro_batch_size_per_gpu": 1.024000e+03, 
    "min_train_micro_batch_size_per_gpu": 1, 
    "num_tuning_micro_batch_sizes": 3
}
[2024-01-04 21:11:40,500] [INFO] [config.py:972:print]   bfloat16_enabled ............. True
[2024-01-04 21:11:40,500] [INFO] [config.py:972:print]   checkpoint_parallel_write_pipeline  False
[2024-01-04 21:11:40,500] [INFO] [config.py:972:print]   checkpoint_tag_validation_enabled  True
[2024-01-04 21:11:40,500] [INFO] [config.py:972:print]   checkpoint_tag_validation_fail  False
[2024-01-04 21:11:40,500] [INFO] [config.py:972:print]   comms_config ................. <deepspeed.comm.config.DeepSpeedCommsConfig object at 0x7f571c041670>
[2024-01-04 21:11:40,500] [INFO] [config.py:972:print]   communication_data_type ...... None
[2024-01-04 21:11:40,500] [INFO] [config.py:972:print]   compression_config ........... {'weight_quantization': {'shared_parameters': {'enabled': False, 'quantizer_kernel': False, 'schedule_offset': 0, 'quantize_groups': 1, 'quantize_verbose': False, 'quantization_type': 'symmetric', 'quantize_weight_in_forward': False, 'rounding': 'nearest', 'fp16_mixed_quantize': False, 'quantize_change_ratio': 0.001}, 'different_groups': {}}, 'activation_quantization': {'shared_parameters': {'enabled': False, 'quantization_type': 'symmetric', 'range_calibration': 'dynamic', 'schedule_offset': 1000}, 'different_groups': {}}, 'sparse_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'row_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'head_pruning': {'shared_parameters': {'enabled': False, 'method': 'topk', 'schedule_offset': 1000}, 'different_groups': {}}, 'channel_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'layer_reduction': {'enabled': False}}
[2024-01-04 21:11:40,500] [INFO] [config.py:972:print]   curriculum_enabled_legacy .... False
[2024-01-04 21:11:40,500] [INFO] [config.py:972:print]   curriculum_params_legacy ..... False
[2024-01-04 21:11:40,500] [INFO] [config.py:972:print]   data_efficiency_config ....... {'enabled': False, 'seed': 1234, 'data_sampling': {'enabled': False, 'num_epochs': 1000, 'num_workers': 0, 'curriculum_learning': {'enabled': False}}, 'data_routing': {'enabled': False, 'random_ltd': {'enabled': False, 'layer_token_lr_schedule': {'enabled': False}}}}
[2024-01-04 21:11:40,500] [INFO] [config.py:972:print]   data_efficiency_enabled ...... False
[2024-01-04 21:11:40,500] [INFO] [config.py:972:print]   dataloader_drop_last ......... False
[2024-01-04 21:11:40,501] [INFO] [config.py:972:print]   disable_allgather ............ False
[2024-01-04 21:11:40,501] [INFO] [config.py:972:print]   dump_state ................... False
[2024-01-04 21:11:40,501] [INFO] [config.py:972:print]   dynamic_loss_scale_args ...... None
[2024-01-04 21:11:40,501] [INFO] [config.py:972:print]   eigenvalue_enabled ........... False
[2024-01-04 21:11:40,501] [INFO] [config.py:972:print]   eigenvalue_gas_boundary_resolution  1
[2024-01-04 21:11:40,501] [INFO] [config.py:972:print]   eigenvalue_layer_name ........ bert.encoder.layer
[2024-01-04 21:11:40,501] [INFO] [config.py:972:print]   eigenvalue_layer_num ......... 0
[2024-01-04 21:11:40,501] [INFO] [config.py:972:print]   eigenvalue_max_iter .......... 100
[2024-01-04 21:11:40,501] [INFO] [config.py:972:print]   eigenvalue_stability ......... 1e-06
[2024-01-04 21:11:40,501] [INFO] [config.py:972:print]   eigenvalue_tol ............... 0.01
[2024-01-04 21:11:40,501] [INFO] [config.py:972:print]   eigenvalue_verbose ........... False
[2024-01-04 21:11:40,501] [INFO] [config.py:972:print]   elasticity_enabled ........... False
[2024-01-04 21:11:40,501] [INFO] [config.py:972:print]   flops_profiler_config ........ {
    "enabled": false, 
    "recompute_fwd_factor": 0.0, 
    "profile_step": 1, 
    "module_depth": -1, 
    "top_modules": 1, 
    "detailed": true, 
    "output_file": null
}
[2024-01-04 21:11:40,501] [INFO] [config.py:972:print]   fp16_auto_cast ............... None
[2024-01-04 21:11:40,501] [INFO] [config.py:972:print]   fp16_enabled ................. False
[2024-01-04 21:11:40,501] [INFO] [config.py:972:print]   fp16_master_weights_and_gradients  False
[2024-01-04 21:11:40,501] [INFO] [config.py:972:print]   global_rank .................. 0
[2024-01-04 21:11:40,501] [INFO] [config.py:972:print]   grad_accum_dtype ............. None
[2024-01-04 21:11:40,501] [INFO] [config.py:972:print]   gradient_accumulation_steps .. 4
[2024-01-04 21:11:40,501] [INFO] [config.py:972:print]   gradient_clipping ............ 1.0
[2024-01-04 21:11:40,501] [INFO] [config.py:972:print]   gradient_predivide_factor .... 1.0
[2024-01-04 21:11:40,501] [INFO] [config.py:972:print]   hybrid_engine ................ enabled=False max_out_tokens=512 inference_tp_size=1 release_inference_cache=False pin_parameters=True tp_gather_partition_size=8
[2024-01-04 21:11:40,501] [INFO] [config.py:972:print]   initial_dynamic_scale ........ 1
[2024-01-04 21:11:40,501] [INFO] [config.py:972:print]   load_universal_checkpoint .... False
[2024-01-04 21:11:40,501] [INFO] [config.py:972:print]   loss_scale ................... 1.0
[2024-01-04 21:11:40,501] [INFO] [config.py:972:print]   memory_breakdown ............. False
[2024-01-04 21:11:40,501] [INFO] [config.py:972:print]   mics_hierarchial_params_gather  False
[2024-01-04 21:11:40,501] [INFO] [config.py:972:print]   mics_shard_size .............. -1
[2024-01-04 21:11:40,501] [INFO] [config.py:972:print]   monitor_config ............... tensorboard=TensorBoardConfig(enabled=False, output_path='', job_name='DeepSpeedJobName') wandb=WandbConfig(enabled=False, group=None, team=None, project='deepspeed') csv_monitor=CSVConfig(enabled=False, output_path='', job_name='DeepSpeedJobName') enabled=False
[2024-01-04 21:11:40,502] [INFO] [config.py:972:print]   nebula_config ................ {
    "enabled": false, 
    "persistent_storage_path": null, 
    "persistent_time_interval": 100, 
    "num_of_version_in_retention": 2, 
    "enable_nebula_load": true, 
    "load_path": null
}
[2024-01-04 21:11:40,502] [INFO] [config.py:972:print]   optimizer_legacy_fusion ...... False
[2024-01-04 21:11:40,502] [INFO] [config.py:972:print]   optimizer_name ............... adam
[2024-01-04 21:11:40,502] [INFO] [config.py:972:print]   optimizer_params ............. {'lr': 2e-05, 'betas': [0.9, 0.999], 'eps': 1e-08, 'weight_decay': 0.0}
[2024-01-04 21:11:40,502] [INFO] [config.py:972:print]   pipeline ..................... {'stages': 'auto', 'partition': 'best', 'seed_layers': False, 'activation_checkpoint_interval': 0}
[2024-01-04 21:11:40,502] [INFO] [config.py:972:print]   pld_enabled .................. False
[2024-01-04 21:11:40,502] [INFO] [config.py:972:print]   pld_params ................... False
[2024-01-04 21:11:40,502] [INFO] [config.py:972:print]   prescale_gradients ........... False
[2024-01-04 21:11:40,502] [INFO] [config.py:972:print]   scheduler_name ............... None
[2024-01-04 21:11:40,502] [INFO] [config.py:972:print]   scheduler_params ............. None
[2024-01-04 21:11:40,502] [INFO] [config.py:972:print]   sparse_attention ............. None
[2024-01-04 21:11:40,502] [INFO] [config.py:972:print]   sparse_gradients_enabled ..... False
[2024-01-04 21:11:40,502] [INFO] [config.py:972:print]   steps_per_print .............. 1000
[2024-01-04 21:11:40,502] [INFO] [config.py:972:print]   train_batch_size ............. 1536
[2024-01-04 21:11:40,502] [INFO] [config.py:972:print]   train_micro_batch_size_per_gpu  48
[2024-01-04 21:11:40,502] [INFO] [config.py:972:print]   use_node_local_storage ....... False
[2024-01-04 21:11:40,502] [INFO] [config.py:972:print]   wall_clock_breakdown ......... False
[2024-01-04 21:11:40,502] [INFO] [config.py:972:print]   weight_quantization_config ... None
[2024-01-04 21:11:40,502] [INFO] [config.py:972:print]   world_size ................... 8
[2024-01-04 21:11:40,502] [INFO] [config.py:972:print]   zero_allow_untested_optimizer  False
[2024-01-04 21:11:40,502] [INFO] [config.py:972:print]   zero_config .................. stage=2 contiguous_gradients=True reduce_scatter=True reduce_bucket_size=200000000 allgather_partitions=True allgather_bucket_size=200000000 overlap_comm=True load_from_fp32_weights=True elastic_checkpoint=False offload_param=None offload_optimizer=DeepSpeedZeroOffloadOptimizerConfig(device='cpu', nvme_path=None, buffer_count=4, pin_memory=True, pipeline=False, pipeline_read=False, pipeline_write=False, fast_init=False) sub_group_size=1,000,000,000 cpu_offload_param=None cpu_offload_use_pin_memory=None cpu_offload=None prefetch_bucket_size=50,000,000 param_persistence_threshold=100,000 model_persistence_threshold=sys.maxsize max_live_parameters=1,000,000,000 max_reuse_distance=1,000,000,000 gather_16bit_weights_on_model_save=False stage3_gather_fp16_weights_on_model_save=False ignore_unused_parameters=True legacy_stage1=False round_robin_gradients=False zero_hpz_partition_size=1 zero_quantized_weights=False zero_quantized_nontrainable_weights=False zero_quantized_gradients=False mics_shard_size=-1 mics_hierarchical_params_gather=False memory_efficient_linear=True pipeline_loading_checkpoint=False override_module_apply=True
[2024-01-04 21:11:40,502] [INFO] [config.py:972:print]   zero_enabled ................. True
[2024-01-04 21:11:40,502] [INFO] [config.py:972:print]   zero_force_ds_cpu_optimizer .. True
[2024-01-04 21:11:40,502] [INFO] [config.py:972:print]   zero_optimization_stage ...... 2
[2024-01-04 21:11:40,502] [INFO] [config.py:958:print_user_config]   json = {
    "optimizer": {
        "type": "Adam", 
        "params": {
            "lr": 2e-05, 
            "betas": [0.9, 0.999], 
            "eps": 1e-08, 
            "weight_decay": 0.0
        }
    }, 
    "bf16": {
        "enabled": true
    }, 
    "fp16": {
        "enabled": false, 
        "loss_scale": 0, 
        "loss_scale_window": 1000, 
        "initial_scale_power": 16, 
        "hysteresis": 2, 
        "min_loss_scale": 1
    }, 
    "zero_optimization": {
        "stage": 2, 
        "offload_optimizer": {
            "device": "cpu", 
            "pin_memory": true
        }, 
        "allgather_partitions": true, 
        "allgather_bucket_size": 2.000000e+08, 
        "overlap_comm": true, 
        "reduce_scatter": true, 
        "reduce_bucket_size": 2.000000e+08, 
        "contiguous_gradients": true
    }, 
    "gradient_accumulation_steps": 4, 
    "gradient_clipping": 1.0, 
    "steps_per_print": 1000, 
    "train_batch_size": 1.536000e+03, 
    "train_micro_batch_size_per_gpu": 48, 
    "wall_clock_breakdown": false
}
[INFO|trainer.py:1755] 2024-01-04 21:11:40,504 >> ***** Running training *****
[INFO|trainer.py:1756] 2024-01-04 21:11:40,504 >>   Num examples = 102002
[INFO|trainer.py:1757] 2024-01-04 21:11:40,504 >>   Num Epochs = 3
[INFO|trainer.py:1758] 2024-01-04 21:11:40,504 >>   Instantaneous batch size per device = 48
[INFO|trainer.py:1759] 2024-01-04 21:11:40,504 >>   Total train batch size (w. parallel, distributed & accumulation) = 1536
[INFO|trainer.py:1760] 2024-01-04 21:11:40,504 >>   Gradient Accumulation steps = 4
[INFO|trainer.py:1761] 2024-01-04 21:11:40,504 >>   Total optimization steps = 198
[INFO|trainer.py:1762] 2024-01-04 21:11:40,506 >>   Number of trainable parameters = 6738423808

  0%|          | 0/198 [00:00<?, ?it/s][WARNING|logging.py:295] 2024-01-04 21:11:40,746 >> `use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...
[WARNING|logging.py:295] 2024-01-04 21:11:41,018 >> `use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...
[WARNING|logging.py:295] 2024-01-04 21:11:41,157 >> `use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...
[WARNING|logging.py:295] 2024-01-04 21:11:41,755 >> `use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...
[WARNING|logging.py:295] 2024-01-04 21:11:41,814 >> `use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...
[WARNING|logging.py:295] 2024-01-04 21:11:41,855 >> `use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...

  1%|          | 1/198 [00:43<2:24:13, 43.93s/it]
  1%|          | 2/198 [01:24<2:17:36, 42.13s/it]
  2%|▏         | 3/198 [02:05<2:15:18, 41.63s/it]
  2%|▏         | 4/198 [02:45<2:12:38, 41.03s/it]
  3%|▎         | 5/198 [03:26<2:10:58, 40.72s/it]
  3%|▎         | 6/198 [04:06<2:10:05, 40.65s/it]
  4%|▎         | 7/198 [04:47<2:09:15, 40.60s/it]
  4%|▍         | 8/198 [05:27<2:08:13, 40.49s/it]
  5%|▍         | 9/198 [06:07<2:07:11, 40.38s/it]
  5%|▌         | 10/198 [06:47<2:06:28, 40.36s/it]
                                                  
{'loss': 1.0993, 'learning_rate': 1.9978589232386036e-05, 'epoch': 0.15}

  5%|▌         | 10/198 [06:47<2:06:28, 40.36s/it]
  6%|▌         | 11/198 [07:28<2:05:43, 40.34s/it]
  6%|▌         | 12/198 [08:08<2:04:58, 40.32s/it]
  7%|▋         | 13/198 [08:48<2:04:03, 40.23s/it]
  7%|▋         | 14/198 [09:28<2:03:33, 40.29s/it]
  8%|▊         | 15/198 [10:09<2:02:47, 40.26s/it]
  8%|▊         | 16/198 [10:49<2:02:33, 40.41s/it]
  9%|▊         | 17/198 [11:30<2:02:00, 40.44s/it]
  9%|▉         | 18/198 [12:10<2:01:23, 40.47s/it]
 10%|▉         | 19/198 [12:51<2:00:36, 40.43s/it]
 10%|█         | 20/198 [13:31<2:00:05, 40.48s/it]
                                                  
{'loss': 0.9336, 'learning_rate': 1.9738769792773338e-05, 'epoch': 0.3}

 10%|█         | 20/198 [13:31<2:00:05, 40.48s/it]
 11%|█         | 21/198 [14:12<1:59:28, 40.50s/it]
 11%|█         | 22/198 [14:53<1:59:11, 40.64s/it]
 12%|█▏        | 23/198 [15:34<1:58:51, 40.75s/it]
 12%|█▏        | 24/198 [16:14<1:58:06, 40.73s/it]
 13%|█▎        | 25/198 [16:55<1:57:40, 40.81s/it]
 13%|█▎        | 26/198 [17:36<1:56:42, 40.71s/it]
 14%|█▎        | 27/198 [18:16<1:55:47, 40.63s/it]
 14%|█▍        | 28/198 [18:57<1:55:10, 40.65s/it]
 15%|█▍        | 29/198 [19:37<1:54:16, 40.57s/it]
 15%|█▌        | 30/198 [20:18<1:53:40, 40.60s/it]
                                                  
{'loss': 0.8791, 'learning_rate': 1.9238795325112867e-05, 'epoch': 0.45}

 15%|█▌        | 30/198 [20:18<1:53:40, 40.60s/it]
 16%|█▌        | 31/198 [20:59<1:52:53, 40.56s/it]
 16%|█▌        | 32/198 [21:39<1:52:05, 40.51s/it]
 17%|█▋        | 33/198 [22:20<1:51:40, 40.61s/it]
 17%|█▋        | 34/198 [23:00<1:50:52, 40.56s/it]
 18%|█▊        | 35/198 [23:41<1:50:10, 40.55s/it]
 18%|█▊        | 36/198 [24:21<1:49:23, 40.51s/it]
 19%|█▊        | 37/198 [25:02<1:48:33, 40.45s/it]
 19%|█▉        | 38/198 [25:42<1:47:43, 40.39s/it]
 20%|█▉        | 39/198 [26:22<1:47:11, 40.45s/it]
 20%|██        | 40/198 [27:03<1:46:30, 40.45s/it]
                                                  
{'loss': 0.8544, 'learning_rate': 1.849202181526579e-05, 'epoch': 0.6}

 20%|██        | 40/198 [27:03<1:46:30, 40.45s/it]
 21%|██        | 41/198 [27:43<1:45:46, 40.42s/it]
 21%|██        | 42/198 [28:24<1:45:18, 40.50s/it]
 22%|██▏       | 43/198 [29:04<1:44:35, 40.49s/it]
 22%|██▏       | 44/198 [29:45<1:44:05, 40.55s/it]
 23%|██▎       | 45/198 [30:26<1:43:20, 40.53s/it]
 23%|██▎       | 46/198 [31:06<1:42:40, 40.53s/it]
 24%|██▎       | 47/198 [31:47<1:42:00, 40.53s/it]
 24%|██▍       | 48/198 [32:27<1:41:27, 40.58s/it]
 25%|██▍       | 49/198 [33:08<1:40:50, 40.61s/it]
 25%|██▌       | 50/198 [33:48<1:39:47, 40.45s/it]
                                                  
{'loss': 0.8305, 'learning_rate': 1.7518398074789776e-05, 'epoch': 0.75}

 25%|██▌       | 50/198 [33:48<1:39:47, 40.45s/it]
 26%|██▌       | 51/198 [34:29<1:39:12, 40.49s/it]
 26%|██▋       | 52/198 [35:09<1:38:27, 40.46s/it]
 27%|██▋       | 53/198 [35:49<1:37:41, 40.43s/it]
 27%|██▋       | 54/198 [36:30<1:37:08, 40.48s/it]
 28%|██▊       | 55/198 [37:11<1:36:32, 40.50s/it]
 28%|██▊       | 56/198 [37:51<1:35:56, 40.54s/it]
 29%|██▉       | 57/198 [38:32<1:35:22, 40.59s/it]
 29%|██▉       | 58/198 [39:12<1:34:38, 40.56s/it]
 30%|██▉       | 59/198 [39:53<1:34:07, 40.63s/it]
 30%|███       | 60/198 [40:34<1:33:24, 40.61s/it]
                                                  
{'loss': 0.8219, 'learning_rate': 1.6343932841636455e-05, 'epoch': 0.9}

 30%|███       | 60/198 [40:34<1:33:24, 40.61s/it]
 31%|███       | 61/198 [41:14<1:32:48, 40.64s/it]
 31%|███▏      | 62/198 [41:55<1:32:07, 40.64s/it]
 32%|███▏      | 63/198 [42:36<1:31:32, 40.68s/it]
 32%|███▏      | 64/198 [43:17<1:30:55, 40.71s/it]
 33%|███▎      | 65/198 [43:57<1:30:06, 40.65s/it]
 33%|███▎      | 66/198 [44:37<1:29:13, 40.55s/it]
 34%|███▍      | 67/198 [45:22<1:31:26, 41.88s/it]
 34%|███▍      | 68/198 [46:02<1:29:28, 41.30s/it]
 35%|███▍      | 69/198 [46:43<1:28:02, 40.95s/it]
 35%|███▌      | 70/198 [47:23<1:26:47, 40.69s/it]
                                                  
{'loss': 0.7832, 'learning_rate': 1.5000000000000002e-05, 'epoch': 1.05}

 35%|███▌      | 70/198 [47:23<1:26:47, 40.69s/it]
 36%|███▌      | 71/198 [48:03<1:25:37, 40.45s/it]
 36%|███▋      | 72/198 [48:43<1:24:50, 40.40s/it]
 37%|███▋      | 73/198 [49:23<1:23:54, 40.28s/it]
 37%|███▋      | 74/198 [50:03<1:23:12, 40.26s/it]
 38%|███▊      | 75/198 [50:43<1:22:27, 40.22s/it]
 38%|███▊      | 76/198 [51:23<1:21:37, 40.14s/it]
 39%|███▉      | 77/198 [52:03<1:20:50, 40.09s/it]
 39%|███▉      | 78/198 [52:43<1:20:14, 40.12s/it]
 40%|███▉      | 79/198 [53:23<1:19:16, 39.97s/it]
 40%|████      | 80/198 [54:03<1:18:39, 39.99s/it]
                                                  
{'loss': 0.732, 'learning_rate': 1.3522500479212337e-05, 'epoch': 1.2}

 40%|████      | 80/198 [54:03<1:18:39, 39.99s/it]
 41%|████      | 81/198 [54:43<1:17:59, 40.00s/it]
 41%|████▏     | 82/198 [55:23<1:17:31, 40.10s/it]
 42%|████▏     | 83/198 [56:04<1:16:57, 40.15s/it]
 42%|████▏     | 84/198 [56:44<1:16:19, 40.17s/it]
 43%|████▎     | 85/198 [57:24<1:15:43, 40.21s/it]
 43%|████▎     | 86/198 [58:04<1:14:57, 40.15s/it]
 44%|████▍     | 87/198 [58:44<1:14:15, 40.14s/it]
 44%|████▍     | 88/198 [59:24<1:13:25, 40.05s/it]
 45%|████▍     | 89/198 [1:00:04<1:12:50, 40.10s/it]
 45%|████▌     | 90/198 [1:00:44<1:12:11, 40.11s/it]
                                                    
{'loss': 0.721, 'learning_rate': 1.1950903220161286e-05, 'epoch': 1.35}

 45%|████▌     | 90/198 [1:00:44<1:12:11, 40.11s/it]
 46%|████▌     | 91/198 [1:01:24<1:11:26, 40.06s/it]
 46%|████▋     | 92/198 [1:02:04<1:10:44, 40.05s/it]
 47%|████▋     | 93/198 [1:02:44<1:09:58, 39.99s/it]
 47%|████▋     | 94/198 [1:03:24<1:09:28, 40.08s/it]
 48%|████▊     | 95/198 [1:04:05<1:08:49, 40.09s/it]
 48%|████▊     | 96/198 [1:04:45<1:08:08, 40.09s/it]
 49%|████▉     | 97/198 [1:05:25<1:07:27, 40.07s/it]
 49%|████▉     | 98/198 [1:06:05<1:06:42, 40.02s/it]
 50%|█████     | 99/198 [1:06:45<1:06:06, 40.06s/it]
 51%|█████     | 100/198 [1:07:25<1:05:28, 40.08s/it]
                                                     
{'loss': 0.7082, 'learning_rate': 1.0327190828217763e-05, 'epoch': 1.5}

 51%|█████     | 100/198 [1:07:25<1:05:28, 40.08s/it]
 51%|█████     | 101/198 [1:08:05<1:04:51, 40.12s/it]
 52%|█████▏    | 102/198 [1:08:45<1:04:09, 40.10s/it]
 52%|█████▏    | 103/198 [1:09:25<1:03:25, 40.05s/it]
 53%|█████▎    | 104/198 [1:10:05<1:02:50, 40.11s/it]
 53%|█████▎    | 105/198 [1:10:45<1:02:07, 40.08s/it]
 54%|█████▎    | 106/198 [1:11:25<1:01:27, 40.08s/it]
 54%|█████▍    | 107/198 [1:12:05<1:00:44, 40.05s/it]
 55%|█████▍    | 108/198 [1:12:45<1:00:02, 40.03s/it]
 55%|█████▌    | 109/198 [1:13:25<59:15, 39.95s/it]  
 56%|█████▌    | 110/198 [1:14:05<58:37, 39.98s/it]
                                                   
{'loss': 0.7111, 'learning_rate': 8.694738077799487e-06, 'epoch': 1.65}

 56%|█████▌    | 110/198 [1:14:05<58:37, 39.98s/it]
 56%|█████▌    | 111/198 [1:14:45<58:02, 40.02s/it]
 57%|█████▋    | 112/198 [1:15:25<57:24, 40.06s/it]
 57%|█████▋    | 113/198 [1:16:06<56:44, 40.05s/it]
 58%|█████▊    | 114/198 [1:16:45<55:58, 39.98s/it]
 58%|█████▊    | 115/198 [1:17:25<55:12, 39.91s/it]
 59%|█████▊    | 116/198 [1:18:05<54:28, 39.86s/it]
 59%|█████▉    | 117/198 [1:18:45<53:54, 39.93s/it]
 60%|█████▉    | 118/198 [1:19:25<53:16, 39.95s/it]
 60%|██████    | 119/198 [1:20:05<52:43, 40.04s/it]
 61%|██████    | 120/198 [1:20:45<52:06, 40.08s/it]
                                                   
{'loss': 0.7049, 'learning_rate': 7.097153227455379e-06, 'epoch': 1.8}

 61%|██████    | 120/198 [1:20:45<52:06, 40.08s/it]
 61%|██████    | 121/198 [1:21:26<51:29, 40.12s/it]
 62%|██████▏   | 122/198 [1:22:05<50:39, 39.99s/it]
 62%|██████▏   | 123/198 [1:22:46<50:07, 40.10s/it]
 63%|██████▎   | 124/198 [1:23:25<49:21, 40.02s/it]
 63%|██████▎   | 125/198 [1:24:06<48:43, 40.05s/it]
 64%|██████▎   | 126/198 [1:24:46<48:05, 40.08s/it]
 64%|██████▍   | 127/198 [1:25:26<47:29, 40.13s/it]
 65%|██████▍   | 128/198 [1:26:06<46:49, 40.13s/it]
 65%|██████▌   | 129/198 [1:26:46<46:07, 40.12s/it]
 66%|██████▌   | 130/198 [1:27:26<45:31, 40.17s/it]
                                                   
{'loss': 0.7046, 'learning_rate': 5.5771130978099896e-06, 'epoch': 1.95}

 66%|██████▌   | 130/198 [1:27:26<45:31, 40.17s/it]
 66%|██████▌   | 131/198 [1:28:07<44:57, 40.27s/it]
 67%|██████▋   | 132/198 [1:28:47<44:18, 40.28s/it]
 67%|██████▋   | 133/198 [1:29:25<42:40, 39.39s/it]
 68%|██████▊   | 134/198 [1:30:12<44:29, 41.70s/it]
 68%|██████▊   | 135/198 [1:30:52<43:17, 41.24s/it]
 69%|██████▊   | 136/198 [1:31:32<42:13, 40.87s/it]
 69%|██████▉   | 137/198 [1:32:12<41:18, 40.64s/it]
 70%|██████▉   | 138/198 [1:32:52<40:27, 40.45s/it]
 70%|███████   | 139/198 [1:33:32<39:42, 40.37s/it]
 71%|███████   | 140/198 [1:34:12<38:57, 40.31s/it]
                                                   
{'loss': 0.6633, 'learning_rate': 4.17522303132198e-06, 'epoch': 2.11}

 71%|███████   | 140/198 [1:34:12<38:57, 40.31s/it]
 71%|███████   | 141/198 [1:34:53<38:16, 40.29s/it]
 72%|███████▏  | 142/198 [1:35:33<37:31, 40.21s/it]
 72%|███████▏  | 143/198 [1:36:13<36:55, 40.28s/it]
 73%|███████▎  | 144/198 [1:36:53<36:14, 40.28s/it]
 73%|███████▎  | 145/198 [1:37:34<35:37, 40.33s/it]
 74%|███████▎  | 146/198 [1:38:14<34:54, 40.28s/it]
 74%|███████▍  | 147/198 [1:38:54<34:15, 40.31s/it]
 75%|███████▍  | 148/198 [1:39:35<33:37, 40.34s/it]
 75%|███████▌  | 149/198 [1:40:15<32:55, 40.32s/it]
 76%|███████▌  | 150/198 [1:40:55<32:17, 40.37s/it]
                                                   
{'loss': 0.6511, 'learning_rate': 2.9289321881345257e-06, 'epoch': 2.26}

 76%|███████▌  | 150/198 [1:40:55<32:17, 40.37s/it]
 76%|███████▋  | 151/198 [1:41:36<31:37, 40.36s/it]
 77%|███████▋  | 152/198 [1:42:16<31:00, 40.44s/it]
 77%|███████▋  | 153/198 [1:42:57<30:21, 40.49s/it]
 78%|███████▊  | 154/198 [1:43:37<29:41, 40.48s/it]
 78%|███████▊  | 155/198 [1:44:18<28:58, 40.43s/it]
 79%|███████▉  | 156/198 [1:44:58<28:17, 40.41s/it]
 79%|███████▉  | 157/198 [1:45:39<27:36, 40.41s/it]
 80%|███████▉  | 158/198 [1:46:19<26:56, 40.41s/it]
 80%|████████  | 159/198 [1:46:59<26:16, 40.41s/it]
 81%|████████  | 160/198 [1:47:40<25:33, 40.36s/it]
                                                   
{'loss': 0.6469, 'learning_rate': 1.8715331540838488e-06, 'epoch': 2.41}

 81%|████████  | 160/198 [1:47:40<25:33, 40.36s/it]
 81%|████████▏ | 161/198 [1:48:20<24:49, 40.27s/it]
 82%|████████▏ | 162/198 [1:49:00<24:09, 40.27s/it]
 82%|████████▏ | 163/198 [1:49:40<23:28, 40.26s/it]
 83%|████████▎ | 164/198 [1:50:20<22:48, 40.25s/it]
 83%|████████▎ | 165/198 [1:51:01<22:08, 40.27s/it]
 84%|████████▍ | 166/198 [1:51:41<21:30, 40.32s/it]
 84%|████████▍ | 167/198 [1:52:22<20:51, 40.36s/it]
 85%|████████▍ | 168/198 [1:53:02<20:09, 40.33s/it]
 85%|████████▌ | 169/198 [1:53:42<19:28, 40.30s/it]
 86%|████████▌ | 170/198 [1:54:22<18:48, 40.31s/it]
                                                   
{'loss': 0.6474, 'learning_rate': 1.0312725846731174e-06, 'epoch': 2.56}

 86%|████████▌ | 170/198 [1:54:22<18:48, 40.31s/it]
 86%|████████▋ | 171/198 [1:55:03<18:07, 40.29s/it]
 87%|████████▋ | 172/198 [1:55:43<17:26, 40.26s/it]
 87%|████████▋ | 173/198 [1:56:23<16:45, 40.23s/it]
 88%|████████▊ | 174/198 [1:57:04<16:07, 40.31s/it]
 88%|████████▊ | 175/198 [1:57:44<15:27, 40.35s/it]
 89%|████████▉ | 176/198 [1:58:24<14:48, 40.39s/it]
 89%|████████▉ | 177/198 [1:59:05<14:08, 40.39s/it]
 90%|████████▉ | 178/198 [1:59:45<13:28, 40.42s/it]
 90%|█████████ | 179/198 [2:00:26<12:48, 40.43s/it]
 91%|█████████ | 180/198 [2:01:06<12:05, 40.31s/it]
                                                   
{'loss': 0.6533, 'learning_rate': 4.305966426779118e-07, 'epoch': 2.71}

 91%|█████████ | 180/198 [2:01:06<12:05, 40.31s/it]
 91%|█████████▏| 181/198 [2:01:46<11:25, 40.32s/it]
 92%|█████████▏| 182/198 [2:02:26<10:44, 40.26s/it]
 92%|█████████▏| 183/198 [2:03:07<10:04, 40.27s/it]
 93%|█████████▎| 184/198 [2:03:47<09:24, 40.31s/it]
 93%|█████████▎| 185/198 [2:04:27<08:44, 40.31s/it]
 94%|█████████▍| 186/198 [2:05:08<08:03, 40.32s/it]
 94%|█████████▍| 187/198 [2:05:48<07:23, 40.34s/it]
 95%|█████████▍| 188/198 [2:06:28<06:43, 40.37s/it]
 95%|█████████▌| 189/198 [2:07:09<06:03, 40.40s/it]
 96%|█████████▌| 190/198 [2:07:49<05:22, 40.30s/it]
                                                   
{'loss': 0.6469, 'learning_rate': 8.555138626189619e-08, 'epoch': 2.86}

 96%|█████████▌| 190/198 [2:07:49<05:22, 40.30s/it]
 96%|█████████▋| 191/198 [2:08:29<04:42, 40.33s/it]
 97%|█████████▋| 192/198 [2:09:09<04:01, 40.25s/it]
 97%|█████████▋| 193/198 [2:09:50<03:21, 40.25s/it]
 98%|█████████▊| 194/198 [2:10:30<02:40, 40.25s/it]
 98%|█████████▊| 195/198 [2:11:10<02:00, 40.22s/it]
 99%|█████████▉| 196/198 [2:11:50<01:20, 40.25s/it]
 99%|█████████▉| 197/198 [2:12:31<00:40, 40.24s/it]
100%|██████████| 198/198 [2:13:11<00:00, 40.29s/it][INFO|trainer.py:2025] 2024-01-04 23:24:54,029 >> 

Training completed. Do not forget to share your model on huggingface.co/models =)



                                                   
{'train_runtime': 7993.5233, 'train_samples_per_second': 38.282, 'train_steps_per_second': 0.025, 'train_loss': 0.7528362852154356, 'epoch': 2.98}

100%|██████████| 198/198 [2:13:13<00:00, 40.29s/it]
100%|██████████| 198/198 [2:13:13<00:00, 40.37s/it]
[INFO|trainer.py:2830] 2024-01-04 23:24:54,163 >> Saving model checkpoint to /apdcephfs/share_733425/vinnylywang/jianhuipang/LLMs4MT/model/newptmodel-llms4mt-en2he-32a100/llama2-sfton-50000-bitexts-and-alpacagpt4
[INFO|configuration_utils.py:457] 2024-01-04 23:24:54,168 >> Configuration saved in /apdcephfs/share_733425/vinnylywang/jianhuipang/LLMs4MT/model/newptmodel-llms4mt-en2he-32a100/llama2-sfton-50000-bitexts-and-alpacagpt4/config.json
[INFO|configuration_utils.py:362] 2024-01-04 23:24:54,175 >> Configuration saved in /apdcephfs/share_733425/vinnylywang/jianhuipang/LLMs4MT/model/newptmodel-llms4mt-en2he-32a100/llama2-sfton-50000-bitexts-and-alpacagpt4/generation_config.json
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:148497 [7] NCCL INFO [Service thread] Connection closed by localRank 7
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:148491 [6] NCCL INFO [Service thread] Connection closed by localRank 6
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147362:147362 [7] NCCL INFO comm 0x4a9bd320 rank 7 nranks 8 cudaDev 7 busId d0000 - Abort COMPLETE
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:148498 [4] NCCL INFO [Service thread] Connection closed by localRank 4
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:148495 [3] NCCL INFO [Service thread] Connection closed by localRank 3
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147361:147361 [6] NCCL INFO comm 0x4ace7470 rank 6 nranks 8 cudaDev 6 busId cb000 - Abort COMPLETE
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:148492 [5] NCCL INFO [Service thread] Connection closed by localRank 5
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:148496 [2] NCCL INFO [Service thread] Connection closed by localRank 2
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147359:147359 [4] NCCL INFO comm 0x485482a0 rank 4 nranks 8 cudaDev 4 busId 93000 - Abort COMPLETE
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:148493 [1] NCCL INFO [Service thread] Connection closed by localRank 1
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147358:147358 [3] NCCL INFO comm 0x47342a50 rank 3 nranks 8 cudaDev 3 busId 51000 - Abort COMPLETE
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147360:147360 [5] NCCL INFO comm 0x46d7e200 rank 5 nranks 8 cudaDev 5 busId 99000 - Abort COMPLETE
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147357:147357 [2] NCCL INFO comm 0x49101200 rank 2 nranks 8 cudaDev 2 busId 4b000 - Abort COMPLETE
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147356:147356 [1] NCCL INFO comm 0x46e04740 rank 1 nranks 8 cudaDev 1 busId 13000 - Abort COMPLETE
[INFO|modeling_utils.py:1767] 2024-01-04 23:25:38,840 >> The model is bigger than the maximum size per checkpoint (10GB) and is going to be split in 2 checkpoint shards. You can find where each parameters has been saved in the index located at /apdcephfs/share_733425/vinnylywang/jianhuipang/LLMs4MT/model/newptmodel-llms4mt-en2he-32a100/llama2-sfton-50000-bitexts-and-alpacagpt4/pytorch_model.bin.index.json.
[INFO|tokenization_utils_base.py:2164] 2024-01-04 23:25:38,937 >> tokenizer config file saved in /apdcephfs/share_733425/vinnylywang/jianhuipang/LLMs4MT/model/newptmodel-llms4mt-en2he-32a100/llama2-sfton-50000-bitexts-and-alpacagpt4/tokenizer_config.json
[INFO|tokenization_utils_base.py:2171] 2024-01-04 23:25:38,941 >> Special tokens file saved in /apdcephfs/share_733425/vinnylywang/jianhuipang/LLMs4MT/model/newptmodel-llms4mt-en2he-32a100/llama2-sfton-50000-bitexts-and-alpacagpt4/special_tokens_map.json
[INFO|tokenization_utils_base.py:2221] 2024-01-04 23:25:38,944 >> added tokens file saved in /apdcephfs/share_733425/vinnylywang/jianhuipang/LLMs4MT/model/newptmodel-llms4mt-en2he-32a100/llama2-sfton-50000-bitexts-and-alpacagpt4/added_tokens.json
***** train metrics *****
  epoch                    =       2.98
  train_loss               =     0.7528
  train_runtime            = 2:13:13.52
  train_samples            =     102002
  train_samples_per_second =     38.282
  train_steps_per_second   =      0.025
[INFO|modelcard.py:451] 2024-01-04 23:25:38,971 >> Dropping the following result as it does not have all the necessary fields:
{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}}
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:148494 [0] NCCL INFO [Service thread] Connection closed by localRank 0
ts-523333936ff240a8b0a2237a60ca87ec-launcher:147355:147355 [0] NCCL INFO comm 0x459a5520 rank 0 nranks 8 cudaDev 0 busId e000 - Abort COMPLETE