|
The following values were not passed to `accelerate launch` and had defaults used instead: |
|
`--num_processes` was set to a value of `1` |
|
`--num_machines` was set to a value of `1` |
|
`--mixed_precision` was set to a value of `'no'` |
|
`--dynamo_backend` was set to a value of `'no'` |
|
To avoid this warning pass in values for each of the problematic parameters or run `accelerate config`. |
|
/home/hzchng/ai-voicebot/.venv/lib/python3.10/site-packages/accelerate/accelerator.py:401: UserWarning: `log_with=[]` was passed but no supported trackers are currently installed. |
|
warnings.warn(f"`log_with={log_with}` was passed but no supported trackers are currently installed.") |
|
05/31/2024 02:23:34 - WARNING - __main__ - Process rank: 0, device: cuda:0, n_gpu: 1, distributed training: False, 16-bits training: False |
|
05/31/2024 02:23:34 - INFO - __main__ - Training/evaluation parameters DistillationTrainingArguments( |
|
_n_gpu=1, |
|
accelerator_config={'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True}, |
|
adafactor=False, |
|
adam_beta1=0.9, |
|
adam_beta2=0.999, |
|
adam_epsilon=1e-08, |
|
auto_find_batch_size=False, |
|
bf16=False, |
|
bf16_full_eval=False, |
|
data_seed=None, |
|
dataloader_drop_last=False, |
|
dataloader_num_workers=1, |
|
dataloader_persistent_workers=False, |
|
dataloader_pin_memory=True, |
|
dataloader_prefetch_factor=None, |
|
ddp_backend=None, |
|
ddp_broadcast_buffers=None, |
|
ddp_bucket_cap_mb=None, |
|
ddp_find_unused_parameters=None, |
|
ddp_timeout=7200, |
|
debug=[], |
|
deepspeed=None, |
|
disable_tqdm=False, |
|
dispatch_batches=None, |
|
do_eval=False, |
|
do_predict=False, |
|
do_train=True, |
|
dtype=bfloat16, |
|
eval_accumulation_steps=None, |
|
eval_delay=0, |
|
eval_steps=None, |
|
evaluation_strategy=no, |
|
fp16=False, |
|
fp16_backend=auto, |
|
fp16_full_eval=False, |
|
fp16_opt_level=O1, |
|
freeze_embed_positions=True, |
|
freeze_encoder=True, |
|
fsdp=[], |
|
fsdp_config={'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}, |
|
fsdp_min_num_params=0, |
|
fsdp_transformer_layer_cls_to_wrap=None, |
|
full_determinism=False, |
|
generation_config=None, |
|
generation_max_length=None, |
|
generation_num_beams=None, |
|
gradient_accumulation_steps=1, |
|
gradient_checkpointing=True, |
|
gradient_checkpointing_kwargs=None, |
|
greater_is_better=None, |
|
group_by_length=False, |
|
half_precision_backend=auto, |
|
hub_always_push=False, |
|
hub_model_id=bjak/distill-whisper-th-medium, |
|
hub_private_repo=True, |
|
hub_strategy=every_save, |
|
hub_token=<HUB_TOKEN>, |
|
ignore_data_skip=False, |
|
include_inputs_for_metrics=False, |
|
include_num_input_tokens_seen=False, |
|
include_tokens_per_second=False, |
|
jit_mode_eval=False, |
|
kl_weight=1.0, |
|
label_names=None, |
|
label_smoothing_factor=0.0, |
|
learning_rate=5e-05, |
|
length_column_name=length, |
|
load_best_model_at_end=False, |
|
local_rank=0, |
|
log_level=passive, |
|
log_level_replica=warning, |
|
log_on_each_node=True, |
|
logging_dir=/home/hzchng/ai-voicebot/lib/whisper/output/runs/May31_02-23-33_ai-voicebot-training, |
|
logging_first_step=False, |
|
logging_nan_inf_filter=True, |
|
logging_steps=0.1, |
|
logging_strategy=steps, |
|
lr_scheduler_kwargs={}, |
|
lr_scheduler_type=constant_with_warmup, |
|
max_grad_norm=1.0, |
|
max_steps=-1, |
|
metric_for_best_model=None, |
|
mp_parameters=, |
|
neftune_noise_alpha=None, |
|
no_cuda=False, |
|
num_train_epochs=5.0, |
|
optim=adamw_torch, |
|
optim_args=None, |
|
output_dir=/home/hzchng/ai-voicebot/lib/whisper/output, |
|
overwrite_output_dir=True, |
|
past_index=-1, |
|
per_device_eval_batch_size=8, |
|
per_device_train_batch_size=8, |
|
predict_with_generate=True, |
|
prediction_loss_only=False, |
|
push_to_hub=True, |
|
push_to_hub_model_id=None, |
|
push_to_hub_organization=None, |
|
push_to_hub_token=<PUSH_TO_HUB_TOKEN>, |
|
ray_scope=last, |
|
remove_unused_columns=True, |
|
report_to=[], |
|
resume_from_checkpoint=None, |
|
run_name=/home/hzchng/ai-voicebot/lib/whisper/output, |
|
save_on_each_node=False, |
|
save_only_model=False, |
|
save_safetensors=True, |
|
save_steps=0.1, |
|
save_strategy=steps, |
|
save_total_limit=2, |
|
seed=42, |
|
skip_memory_metrics=True, |
|
sortish_sampler=False, |
|
split_batches=None, |
|
temperature=2.0, |
|
tf32=None, |
|
torch_compile=False, |
|
torch_compile_backend=None, |
|
torch_compile_mode=None, |
|
torchdynamo=None, |
|
tpu_metrics_debug=False, |
|
tpu_num_cores=None, |
|
use_cpu=False, |
|
use_ipex=False, |
|
use_legacy_prediction_loop=False, |
|
use_mps_device=False, |
|
warmup_ratio=0.1, |
|
warmup_steps=10, |
|
weight_decay=0.0, |
|
) |
|
Combining datasets...: 0%| | 0/1 [00:00<?, ?it/s]
Combining datasets...: 100%|ββββββββββ| 1/1 [00:04<00:00, 4.43s/it]
Combining datasets...: 100%|ββββββββββ| 1/1 [00:04<00:00, 4.43s/it] |
|
/home/hzchng/ai-voicebot/.venv/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. |
|
warnings.warn( |
|
loading configuration file config.json from cache at /home/hzchng/.cache/huggingface/hub/models--biodatlab--distill-whisper-th-medium/snapshots/e756107875a3905f3c4535cdca0d499cb56d2232/config.json |
|
Model config WhisperConfig { |
|
"_name_or_path": "distil-whisper/training/distil-whisper-th-medium-4dec", |
|
"activation_dropout": 0.0, |
|
"activation_function": "gelu", |
|
"apply_spec_augment": true, |
|
"architectures": [ |
|
"WhisperForConditionalGeneration" |
|
], |
|
"attention_dropout": 0.0, |
|
"begin_suppress_tokens": [ |
|
220, |
|
50257 |
|
], |
|
"bos_token_id": 50257, |
|
"classifier_proj_size": 256, |
|
"d_model": 1024, |
|
"decoder_attention_heads": 16, |
|
"decoder_ffn_dim": 4096, |
|
"decoder_layerdrop": 0.0, |
|
"decoder_layers": 4, |
|
"decoder_start_token_id": 50258, |
|
"dropout": 0.0, |
|
"encoder_attention_heads": 16, |
|
"encoder_ffn_dim": 4096, |
|
"encoder_layerdrop": 0.0, |
|
"encoder_layers": 24, |
|
"eos_token_id": 50257, |
|
"forced_decoder_ids": null, |
|
"init_std": 0.02, |
|
"is_encoder_decoder": true, |
|
"mask_feature_length": 64, |
|
"mask_feature_min_masks": 0, |
|
"mask_feature_prob": 1e-05, |
|
"mask_time_length": 10, |
|
"mask_time_min_masks": 2, |
|
"mask_time_prob": 1e-05, |
|
"max_length": 448, |
|
"max_source_positions": 1500, |
|
"max_target_positions": 448, |
|
"median_filter_width": 7, |
|
"model_type": "whisper", |
|
"num_hidden_layers": 24, |
|
"num_mel_bins": 80, |
|
"pad_token_id": 50257, |
|
"scale_embedding": false, |
|
"torch_dtype": "float16", |
|
"transformers_version": "4.38.1", |
|
"use_cache": true, |
|
"use_weighted_layer_sum": false, |
|
"vocab_size": 51865 |
|
} |
|
|
|
loading configuration file preprocessor_config.json from cache at /home/hzchng/.cache/huggingface/hub/models--biodatlab--distill-whisper-th-medium/snapshots/e756107875a3905f3c4535cdca0d499cb56d2232/preprocessor_config.json |
|
Feature extractor WhisperFeatureExtractor { |
|
"chunk_length": 30, |
|
"feature_extractor_type": "WhisperFeatureExtractor", |
|
"feature_size": 80, |
|
"hop_length": 160, |
|
"n_fft": 400, |
|
"n_samples": 480000, |
|
"nb_max_frames": 3000, |
|
"padding_side": "right", |
|
"padding_value": 0.0, |
|
"processor_class": "WhisperProcessor", |
|
"return_attention_mask": false, |
|
"sampling_rate": 16000 |
|
} |
|
|
|
loading file vocab.json from cache at /home/hzchng/.cache/huggingface/hub/models--biodatlab--distill-whisper-th-medium/snapshots/e756107875a3905f3c4535cdca0d499cb56d2232/vocab.json |
|
loading file tokenizer.json from cache at /home/hzchng/.cache/huggingface/hub/models--biodatlab--distill-whisper-th-medium/snapshots/e756107875a3905f3c4535cdca0d499cb56d2232/tokenizer.json |
|
loading file merges.txt from cache at /home/hzchng/.cache/huggingface/hub/models--biodatlab--distill-whisper-th-medium/snapshots/e756107875a3905f3c4535cdca0d499cb56d2232/merges.txt |
|
loading file normalizer.json from cache at /home/hzchng/.cache/huggingface/hub/models--biodatlab--distill-whisper-th-medium/snapshots/e756107875a3905f3c4535cdca0d499cb56d2232/normalizer.json |
|
loading file added_tokens.json from cache at /home/hzchng/.cache/huggingface/hub/models--biodatlab--distill-whisper-th-medium/snapshots/e756107875a3905f3c4535cdca0d499cb56d2232/added_tokens.json |
|
loading file special_tokens_map.json from cache at /home/hzchng/.cache/huggingface/hub/models--biodatlab--distill-whisper-th-medium/snapshots/e756107875a3905f3c4535cdca0d499cb56d2232/special_tokens_map.json |
|
loading file tokenizer_config.json from cache at /home/hzchng/.cache/huggingface/hub/models--biodatlab--distill-whisper-th-medium/snapshots/e756107875a3905f3c4535cdca0d499cb56d2232/tokenizer_config.json |
|
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. |
|
loading configuration file config.json from cache at /home/hzchng/.cache/huggingface/hub/models--openai--whisper-medium/snapshots/abdf7c39ab9d0397620ccaea8974cc764cd0953e/config.json |
|
Model config WhisperConfig { |
|
"_name_or_path": "openai/whisper-medium", |
|
"activation_dropout": 0.0, |
|
"activation_function": "gelu", |
|
"apply_spec_augment": false, |
|
"architectures": [ |
|
"WhisperForConditionalGeneration" |
|
], |
|
"attention_dropout": 0.0, |
|
"begin_suppress_tokens": [ |
|
220, |
|
50257 |
|
], |
|
"bos_token_id": 50257, |
|
"classifier_proj_size": 256, |
|
"d_model": 1024, |
|
"decoder_attention_heads": 16, |
|
"decoder_ffn_dim": 4096, |
|
"decoder_layerdrop": 0.0, |
|
"decoder_layers": 24, |
|
"decoder_start_token_id": 50258, |
|
"dropout": 0.0, |
|
"encoder_attention_heads": 16, |
|
"encoder_ffn_dim": 4096, |
|
"encoder_layerdrop": 0.0, |
|
"encoder_layers": 24, |
|
"eos_token_id": 50257, |
|
"forced_decoder_ids": [ |
|
[ |
|
1, |
|
50259 |
|
], |
|
[ |
|
2, |
|
50359 |
|
], |
|
[ |
|
3, |
|
50363 |
|
] |
|
], |
|
"init_std": 0.02, |
|
"is_encoder_decoder": true, |
|
"mask_feature_length": 10, |
|
"mask_feature_min_masks": 0, |
|
"mask_feature_prob": 0.0, |
|
"mask_time_length": 10, |
|
"mask_time_min_masks": 2, |
|
"mask_time_prob": 0.05, |
|
"max_length": 448, |
|
"max_source_positions": 1500, |
|
"max_target_positions": 448, |
|
"median_filter_width": 7, |
|
"model_type": "whisper", |
|
"num_hidden_layers": 24, |
|
"num_mel_bins": 80, |
|
"pad_token_id": 50257, |
|
"scale_embedding": false, |
|
"suppress_tokens": [ |
|
1, |
|
2, |
|
7, |
|
8, |
|
9, |
|
10, |
|
14, |
|
25, |
|
26, |
|
27, |
|
28, |
|
29, |
|
31, |
|
58, |
|
59, |
|
60, |
|
61, |
|
62, |
|
63, |
|
90, |
|
91, |
|
92, |
|
93, |
|
359, |
|
503, |
|
522, |
|
542, |
|
873, |
|
893, |
|
902, |
|
918, |
|
922, |
|
931, |
|
1350, |
|
1853, |
|
1982, |
|
2460, |
|
2627, |
|
3246, |
|
3253, |
|
3268, |
|
3536, |
|
3846, |
|
3961, |
|
4183, |
|
4667, |
|
6585, |
|
6647, |
|
7273, |
|
9061, |
|
9383, |
|
10428, |
|
10929, |
|
11938, |
|
12033, |
|
12331, |
|
12562, |
|
13793, |
|
14157, |
|
14635, |
|
15265, |
|
15618, |
|
16553, |
|
16604, |
|
18362, |
|
18956, |
|
20075, |
|
21675, |
|
22520, |
|
26130, |
|
26161, |
|
26435, |
|
28279, |
|
29464, |
|
31650, |
|
32302, |
|
32470, |
|
36865, |
|
42863, |
|
47425, |
|
49870, |
|
50254, |
|
50258, |
|
50358, |
|
50359, |
|
50360, |
|
50361, |
|
50362 |
|
], |
|
"torch_dtype": "float32", |
|
"transformers_version": "4.38.1", |
|
"use_cache": true, |
|
"use_weighted_layer_sum": false, |
|
"vocab_size": 51865 |
|
} |
|
|
|
loading weights file model.safetensors from cache at /home/hzchng/.cache/huggingface/hub/models--openai--whisper-medium/snapshots/abdf7c39ab9d0397620ccaea8974cc764cd0953e/model.safetensors |
|
Instantiating WhisperForConditionalGeneration model under default dtype torch.bfloat16. |
|
Generate config GenerationConfig { |
|
"begin_suppress_tokens": [ |
|
220, |
|
50257 |
|
], |
|
"bos_token_id": 50257, |
|
"decoder_start_token_id": 50258, |
|
"eos_token_id": 50257, |
|
"forced_decoder_ids": [ |
|
[ |
|
1, |
|
50259 |
|
], |
|
[ |
|
2, |
|
50359 |
|
], |
|
[ |
|
3, |
|
50363 |
|
] |
|
], |
|
"max_length": 448, |
|
"pad_token_id": 50257, |
|
"suppress_tokens": [ |
|
1, |
|
2, |
|
7, |
|
8, |
|
9, |
|
10, |
|
14, |
|
25, |
|
26, |
|
27, |
|
28, |
|
29, |
|
31, |
|
58, |
|
59, |
|
60, |
|
61, |
|
62, |
|
63, |
|
90, |
|
91, |
|
92, |
|
93, |
|
359, |
|
503, |
|
522, |
|
542, |
|
873, |
|
893, |
|
902, |
|
918, |
|
922, |
|
931, |
|
1350, |
|
1853, |
|
1982, |
|
2460, |
|
2627, |
|
3246, |
|
3253, |
|
3268, |
|
3536, |
|
3846, |
|
3961, |
|
4183, |
|
4667, |
|
6585, |
|
6647, |
|
7273, |
|
9061, |
|
9383, |
|
10428, |
|
10929, |
|
11938, |
|
12033, |
|
12331, |
|
12562, |
|
13793, |
|
14157, |
|
14635, |
|
15265, |
|
15618, |
|
16553, |
|
16604, |
|
18362, |
|
18956, |
|
20075, |
|
21675, |
|
22520, |
|
26130, |
|
26161, |
|
26435, |
|
28279, |
|
29464, |
|
31650, |
|
32302, |
|
32470, |
|
36865, |
|
42863, |
|
47425, |
|
49870, |
|
50254, |
|
50258, |
|
50358, |
|
50359, |
|
50360, |
|
50361, |
|
50362 |
|
] |
|
} |
|
|
|
All model checkpoint weights were used when initializing WhisperForConditionalGeneration. |
|
|
|
All the weights of WhisperForConditionalGeneration were initialized from the model checkpoint at openai/whisper-medium. |
|
If your task is similar to the task the model of the checkpoint was trained on, you can already use WhisperForConditionalGeneration for predictions without further training. |
|
loading configuration file generation_config.json from cache at /home/hzchng/.cache/huggingface/hub/models--openai--whisper-medium/snapshots/abdf7c39ab9d0397620ccaea8974cc764cd0953e/generation_config.json |
|
Generate config GenerationConfig { |
|
"alignment_heads": [ |
|
[ |
|
13, |
|
15 |
|
], |
|
[ |
|
15, |
|
4 |
|
], |
|
[ |
|
15, |
|
15 |
|
], |
|
[ |
|
16, |
|
1 |
|
], |
|
[ |
|
20, |
|
0 |
|
], |
|
[ |
|
23, |
|
4 |
|
] |
|
], |
|
"attn_implementation": "sdpa", |
|
"begin_suppress_tokens": [ |
|
220, |
|
50257 |
|
], |
|
"bos_token_id": 50257, |
|
"decoder_start_token_id": 50258, |
|
"eos_token_id": 50257, |
|
"forced_decoder_ids": [ |
|
[ |
|
1, |
|
null |
|
], |
|
[ |
|
2, |
|
50359 |
|
] |
|
], |
|
"is_multilingual": true, |
|
"lang_to_id": { |
|
"<|af|>": 50327, |
|
"<|am|>": 50334, |
|
"<|ar|>": 50272, |
|
"<|as|>": 50350, |
|
"<|az|>": 50304, |
|
"<|ba|>": 50355, |
|
"<|be|>": 50330, |
|
"<|bg|>": 50292, |
|
"<|bn|>": 50302, |
|
"<|bo|>": 50347, |
|
"<|br|>": 50309, |
|
"<|bs|>": 50315, |
|
"<|ca|>": 50270, |
|
"<|cs|>": 50283, |
|
"<|cy|>": 50297, |
|
"<|da|>": 50285, |
|
"<|de|>": 50261, |
|
"<|el|>": 50281, |
|
"<|en|>": 50259, |
|
"<|es|>": 50262, |
|
"<|et|>": 50307, |
|
"<|eu|>": 50310, |
|
"<|fa|>": 50300, |
|
"<|fi|>": 50277, |
|
"<|fo|>": 50338, |
|
"<|fr|>": 50265, |
|
"<|gl|>": 50319, |
|
"<|gu|>": 50333, |
|
"<|haw|>": 50352, |
|
"<|ha|>": 50354, |
|
"<|he|>": 50279, |
|
"<|hi|>": 50276, |
|
"<|hr|>": 50291, |
|
"<|ht|>": 50339, |
|
"<|hu|>": 50286, |
|
"<|hy|>": 50312, |
|
"<|id|>": 50275, |
|
"<|is|>": 50311, |
|
"<|it|>": 50274, |
|
"<|ja|>": 50266, |
|
"<|jw|>": 50356, |
|
"<|ka|>": 50329, |
|
"<|kk|>": 50316, |
|
"<|km|>": 50323, |
|
"<|kn|>": 50306, |
|
"<|ko|>": 50264, |
|
"<|la|>": 50294, |
|
"<|lb|>": 50345, |
|
"<|ln|>": 50353, |
|
"<|lo|>": 50336, |
|
"<|lt|>": 50293, |
|
"<|lv|>": 50301, |
|
"<|mg|>": 50349, |
|
"<|mi|>": 50295, |
|
"<|mk|>": 50308, |
|
"<|ml|>": 50296, |
|
"<|mn|>": 50314, |
|
"<|mr|>": 50320, |
|
"<|ms|>": 50282, |
|
"<|mt|>": 50343, |
|
"<|my|>": 50346, |
|
"<|ne|>": 50313, |
|
"<|nl|>": 50271, |
|
"<|nn|>": 50342, |
|
"<|no|>": 50288, |
|
"<|oc|>": 50328, |
|
"<|pa|>": 50321, |
|
"<|pl|>": 50269, |
|
"<|ps|>": 50340, |
|
"<|pt|>": 50267, |
|
"<|ro|>": 50284, |
|
"<|ru|>": 50263, |
|
"<|sa|>": 50344, |
|
"<|sd|>": 50332, |
|
"<|si|>": 50322, |
|
"<|sk|>": 50298, |
|
"<|sl|>": 50305, |
|
"<|sn|>": 50324, |
|
"<|so|>": 50326, |
|
"<|sq|>": 50317, |
|
"<|sr|>": 50303, |
|
"<|su|>": 50357, |
|
"<|sv|>": 50273, |
|
"<|sw|>": 50318, |
|
"<|ta|>": 50287, |
|
"<|te|>": 50299, |
|
"<|tg|>": 50331, |
|
"<|th|>": 50289, |
|
"<|tk|>": 50341, |
|
"<|tl|>": 50348, |
|
"<|tr|>": 50268, |
|
"<|tt|>": 50351, |
|
"<|uk|>": 50280, |
|
"<|ur|>": 50290, |
|
"<|uz|>": 50337, |
|
"<|vi|>": 50278, |
|
"<|yi|>": 50335, |
|
"<|yo|>": 50325, |
|
"<|zh|>": 50260 |
|
}, |
|
"max_initial_timestamp_index": 50, |
|
"max_length": 448, |
|
"no_timestamps_token_id": 50363, |
|
"pad_token_id": 50257, |
|
"prev_sot_token_id": 50361, |
|
"return_timestamps": false, |
|
"suppress_tokens": [ |
|
1, |
|
2, |
|
7, |
|
8, |
|
9, |
|
10, |
|
14, |
|
25, |
|
26, |
|
27, |
|
28, |
|
29, |
|
31, |
|
58, |
|
59, |
|
60, |
|
61, |
|
62, |
|
63, |
|
90, |
|
91, |
|
92, |
|
93, |
|
359, |
|
503, |
|
522, |
|
542, |
|
873, |
|
893, |
|
902, |
|
918, |
|
922, |
|
931, |
|
1350, |
|
1853, |
|
1982, |
|
2460, |
|
2627, |
|
3246, |
|
3253, |
|
3268, |
|
3536, |
|
3846, |
|
3961, |
|
4183, |
|
4667, |
|
6585, |
|
6647, |
|
7273, |
|
9061, |
|
9383, |
|
10428, |
|
10929, |
|
11938, |
|
12033, |
|
12331, |
|
12562, |
|
13793, |
|
14157, |
|
14635, |
|
15265, |
|
15618, |
|
16553, |
|
16604, |
|
18362, |
|
18956, |
|
20075, |
|
21675, |
|
22520, |
|
26130, |
|
26161, |
|
26435, |
|
28279, |
|
29464, |
|
31650, |
|
32302, |
|
32470, |
|
36865, |
|
42863, |
|
47425, |
|
49870, |
|
50254, |
|
50258, |
|
50358, |
|
50359, |
|
50360, |
|
50361, |
|
50362 |
|
], |
|
"task_to_id": { |
|
"transcribe": 50359, |
|
"translate": 50358 |
|
} |
|
} |
|
|
|
loading weights file model.safetensors from cache at /home/hzchng/.cache/huggingface/hub/models--biodatlab--distill-whisper-th-medium/snapshots/e756107875a3905f3c4535cdca0d499cb56d2232/model.safetensors |
|
Generate config GenerationConfig { |
|
"begin_suppress_tokens": [ |
|
220, |
|
50257 |
|
], |
|
"bos_token_id": 50257, |
|
"decoder_start_token_id": 50258, |
|
"eos_token_id": 50257, |
|
"max_length": 448, |
|
"pad_token_id": 50257 |
|
} |
|
|
|
All model checkpoint weights were used when initializing WhisperForConditionalGeneration. |
|
|
|
All the weights of WhisperForConditionalGeneration were initialized from the model checkpoint at biodatlab/distill-whisper-th-medium. |
|
If your task is similar to the task the model of the checkpoint was trained on, you can already use WhisperForConditionalGeneration for predictions without further training. |
|
loading configuration file generation_config.json from cache at /home/hzchng/.cache/huggingface/hub/models--biodatlab--distill-whisper-th-medium/snapshots/e756107875a3905f3c4535cdca0d499cb56d2232/generation_config.json |
|
Generate config GenerationConfig { |
|
"alignment_heads": [ |
|
[ |
|
13, |
|
15 |
|
], |
|
[ |
|
15, |
|
4 |
|
], |
|
[ |
|
15, |
|
15 |
|
], |
|
[ |
|
16, |
|
1 |
|
], |
|
[ |
|
20, |
|
0 |
|
], |
|
[ |
|
23, |
|
4 |
|
] |
|
], |
|
"begin_suppress_tokens": [ |
|
220, |
|
50257 |
|
], |
|
"bos_token_id": 50257, |
|
"decoder_start_token_id": 50258, |
|
"eos_token_id": 50257, |
|
"forced_decoder_ids": [ |
|
[ |
|
1, |
|
null |
|
], |
|
[ |
|
2, |
|
50359 |
|
] |
|
], |
|
"is_multilingual": true, |
|
"lang_to_id": { |
|
"<|af|>": 50327, |
|
"<|am|>": 50334, |
|
"<|ar|>": 50272, |
|
"<|as|>": 50350, |
|
"<|az|>": 50304, |
|
"<|ba|>": 50355, |
|
"<|be|>": 50330, |
|
"<|bg|>": 50292, |
|
"<|bn|>": 50302, |
|
"<|bo|>": 50347, |
|
"<|br|>": 50309, |
|
"<|bs|>": 50315, |
|
"<|ca|>": 50270, |
|
"<|cs|>": 50283, |
|
"<|cy|>": 50297, |
|
"<|da|>": 50285, |
|
"<|de|>": 50261, |
|
"<|el|>": 50281, |
|
"<|en|>": 50259, |
|
"<|es|>": 50262, |
|
"<|et|>": 50307, |
|
"<|eu|>": 50310, |
|
"<|fa|>": 50300, |
|
"<|fi|>": 50277, |
|
"<|fo|>": 50338, |
|
"<|fr|>": 50265, |
|
"<|gl|>": 50319, |
|
"<|gu|>": 50333, |
|
"<|haw|>": 50352, |
|
"<|ha|>": 50354, |
|
"<|he|>": 50279, |
|
"<|hi|>": 50276, |
|
"<|hr|>": 50291, |
|
"<|ht|>": 50339, |
|
"<|hu|>": 50286, |
|
"<|hy|>": 50312, |
|
"<|id|>": 50275, |
|
"<|is|>": 50311, |
|
"<|it|>": 50274, |
|
"<|ja|>": 50266, |
|
"<|jw|>": 50356, |
|
"<|ka|>": 50329, |
|
"<|kk|>": 50316, |
|
"<|km|>": 50323, |
|
"<|kn|>": 50306, |
|
"<|ko|>": 50264, |
|
"<|la|>": 50294, |
|
"<|lb|>": 50345, |
|
"<|ln|>": 50353, |
|
"<|lo|>": 50336, |
|
"<|lt|>": 50293, |
|
"<|lv|>": 50301, |
|
"<|mg|>": 50349, |
|
"<|mi|>": 50295, |
|
"<|mk|>": 50308, |
|
"<|ml|>": 50296, |
|
"<|mn|>": 50314, |
|
"<|mr|>": 50320, |
|
"<|ms|>": 50282, |
|
"<|mt|>": 50343, |
|
"<|my|>": 50346, |
|
"<|ne|>": 50313, |
|
"<|nl|>": 50271, |
|
"<|nn|>": 50342, |
|
"<|no|>": 50288, |
|
"<|oc|>": 50328, |
|
"<|pa|>": 50321, |
|
"<|pl|>": 50269, |
|
"<|ps|>": 50340, |
|
"<|pt|>": 50267, |
|
"<|ro|>": 50284, |
|
"<|ru|>": 50263, |
|
"<|sa|>": 50344, |
|
"<|sd|>": 50332, |
|
"<|si|>": 50322, |
|
"<|sk|>": 50298, |
|
"<|sl|>": 50305, |
|
"<|sn|>": 50324, |
|
"<|so|>": 50326, |
|
"<|sq|>": 50317, |
|
"<|sr|>": 50303, |
|
"<|su|>": 50357, |
|
"<|sv|>": 50273, |
|
"<|sw|>": 50318, |
|
"<|ta|>": 50287, |
|
"<|te|>": 50299, |
|
"<|tg|>": 50331, |
|
"<|th|>": 50289, |
|
"<|tk|>": 50341, |
|
"<|tl|>": 50348, |
|
"<|tr|>": 50268, |
|
"<|tt|>": 50351, |
|
"<|uk|>": 50280, |
|
"<|ur|>": 50290, |
|
"<|uz|>": 50337, |
|
"<|vi|>": 50278, |
|
"<|yi|>": 50335, |
|
"<|yo|>": 50325, |
|
"<|zh|>": 50260 |
|
}, |
|
"max_initial_timestamp_index": 50, |
|
"max_length": 448, |
|
"no_timestamps_token_id": 50363, |
|
"pad_token_id": 50257, |
|
"prev_sot_token_id": 50361, |
|
"return_timestamps": false, |
|
"suppress_tokens": [ |
|
1, |
|
2, |
|
7, |
|
8, |
|
9, |
|
10, |
|
14, |
|
25, |
|
26, |
|
27, |
|
28, |
|
29, |
|
31, |
|
58, |
|
59, |
|
60, |
|
61, |
|
62, |
|
63, |
|
90, |
|
91, |
|
92, |
|
93, |
|
359, |
|
503, |
|
522, |
|
542, |
|
873, |
|
893, |
|
902, |
|
918, |
|
922, |
|
931, |
|
1350, |
|
1853, |
|
1982, |
|
2460, |
|
2627, |
|
3246, |
|
3253, |
|
3268, |
|
3536, |
|
3846, |
|
3961, |
|
4183, |
|
4667, |
|
6585, |
|
6647, |
|
7273, |
|
9061, |
|
9383, |
|
10428, |
|
10929, |
|
11938, |
|
12033, |
|
12331, |
|
12562, |
|
13793, |
|
14157, |
|
14635, |
|
15265, |
|
15618, |
|
16553, |
|
16604, |
|
18362, |
|
18956, |
|
20075, |
|
21675, |
|
22520, |
|
26130, |
|
26161, |
|
26435, |
|
28279, |
|
29464, |
|
31650, |
|
32302, |
|
32470, |
|
36865, |
|
42863, |
|
47425, |
|
49870, |
|
50254, |
|
50258, |
|
50358, |
|
50359, |
|
50360, |
|
50361, |
|
50362 |
|
], |
|
"task_to_id": { |
|
"transcribe": 50359, |
|
"translate": 50358 |
|
} |
|
} |
|
|
|
05/31/2024 02:23:45 - INFO - __main__ - Disabling gradient checkpointing in the decoder since it's incompatible with `freeze_embed_positions`. |
|
Feature extractor saved in /home/hzchng/ai-voicebot/lib/whisper/output/preprocessor_config.json |
|
tokenizer config file saved in /home/hzchng/ai-voicebot/lib/whisper/output/tokenizer_config.json |
|
Special tokens file saved in /home/hzchng/ai-voicebot/lib/whisper/output/special_tokens_map.json |
|
Some non-default generation parameters are set in the model config. These should go into a GenerationConfig file (https://huggingface.co/docs/transformers/generation_strategies#save-a-custom-decoding-strategy-with-your-model) instead. This warning will be raised to an exception in v4.41. |
|
Non-default generation parameters: {'max_length': 448, 'begin_suppress_tokens': [220, 50257]} |
|
Configuration saved in /home/hzchng/ai-voicebot/lib/whisper/output/config.json |
|
Configuration saved in /home/hzchng/ai-voicebot/lib/whisper/output/generation_config.json |
|
loading configuration file /home/hzchng/ai-voicebot/lib/whisper/output/preprocessor_config.json |
|
Feature extractor WhisperFeatureExtractor { |
|
"chunk_length": 30, |
|
"feature_extractor_type": "WhisperFeatureExtractor", |
|
"feature_size": 80, |
|
"hop_length": 160, |
|
"n_fft": 400, |
|
"n_samples": 480000, |
|
"nb_max_frames": 3000, |
|
"padding_side": "right", |
|
"padding_value": 0.0, |
|
"processor_class": "WhisperProcessor", |
|
"return_attention_mask": false, |
|
"sampling_rate": 16000 |
|
} |
|
|
|
loading file vocab.json |
|
loading file tokenizer.json |
|
loading file merges.txt |
|
loading file normalizer.json |
|
loading file added_tokens.json |
|
loading file special_tokens_map.json |
|
loading file tokenizer_config.json |
|
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. |
|
Processor WhisperProcessor: |
|
- feature_extractor: WhisperFeatureExtractor { |
|
"chunk_length": 30, |
|
"feature_extractor_type": "WhisperFeatureExtractor", |
|
"feature_size": 80, |
|
"hop_length": 160, |
|
"n_fft": 400, |
|
"n_samples": 480000, |
|
"nb_max_frames": 3000, |
|
"padding_side": "right", |
|
"padding_value": 0.0, |
|
"processor_class": "WhisperProcessor", |
|
"return_attention_mask": false, |
|
"sampling_rate": 16000 |
|
} |
|
|
|
- tokenizer: WhisperTokenizer(name_or_path='/home/hzchng/ai-voicebot/lib/whisper/output', vocab_size=50258, model_max_length=1024, is_fast=False, padding_side='right', truncation_side='right', special_tokens={'bos_token': '<|endoftext|>', 'eos_token': '<|endoftext|>', 'unk_token': '<|endoftext|>', 'pad_token': '<|endoftext|>', 'additional_special_tokens': ['<|endoftext|>', '<|startoftranscript|>', '<|en|>', '<|zh|>', '<|de|>', '<|es|>', '<|ru|>', '<|ko|>', '<|fr|>', '<|ja|>', '<|pt|>', '<|tr|>', '<|pl|>', '<|ca|>', '<|nl|>', '<|ar|>', '<|sv|>', '<|it|>', '<|id|>', '<|hi|>', '<|fi|>', '<|vi|>', '<|he|>', '<|uk|>', '<|el|>', '<|ms|>', '<|cs|>', '<|ro|>', '<|da|>', '<|hu|>', '<|ta|>', '<|no|>', '<|th|>', '<|ur|>', '<|hr|>', '<|bg|>', '<|lt|>', '<|la|>', '<|mi|>', '<|ml|>', '<|cy|>', '<|sk|>', '<|te|>', '<|fa|>', '<|lv|>', '<|bn|>', '<|sr|>', '<|az|>', '<|sl|>', '<|kn|>', '<|et|>', '<|mk|>', '<|br|>', '<|eu|>', '<|is|>', '<|hy|>', '<|ne|>', '<|mn|>', '<|bs|>', '<|kk|>', '<|sq|>', '<|sw|>', '<|gl|>', '<|mr|>', '<|pa|>', '<|si|>', '<|km|>', '<|sn|>', '<|yo|>', '<|so|>', '<|af|>', '<|oc|>', '<|ka|>', '<|be|>', '<|tg|>', '<|sd|>', '<|gu|>', '<|am|>', '<|yi|>', '<|lo|>', '<|uz|>', '<|fo|>', '<|ht|>', '<|ps|>', '<|tk|>', '<|nn|>', '<|mt|>', '<|sa|>', '<|lb|>', '<|my|>', '<|bo|>', '<|tl|>', '<|mg|>', '<|as|>', '<|tt|>', '<|haw|>', '<|ln|>', '<|ha|>', '<|ba|>', '<|jw|>', '<|su|>', '<|translate|>', '<|transcribe|>', '<|startoflm|>', '<|startofprev|>', '<|nocaptions|>', '<|notimestamps|>']}, clean_up_tokenization_spaces=True), added_tokens_decoder={ |
|
50257: AddedToken("<|endoftext|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50258: AddedToken("<|startoftranscript|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50259: AddedToken("<|en|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50260: AddedToken("<|zh|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50261: AddedToken("<|de|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50262: AddedToken("<|es|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50263: AddedToken("<|ru|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50264: AddedToken("<|ko|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50265: AddedToken("<|fr|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50266: AddedToken("<|ja|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50267: AddedToken("<|pt|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50268: AddedToken("<|tr|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50269: AddedToken("<|pl|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50270: AddedToken("<|ca|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50271: AddedToken("<|nl|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50272: AddedToken("<|ar|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50273: AddedToken("<|sv|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50274: AddedToken("<|it|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50275: AddedToken("<|id|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50276: AddedToken("<|hi|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50277: AddedToken("<|fi|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50278: AddedToken("<|vi|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50279: AddedToken("<|he|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50280: AddedToken("<|uk|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50281: AddedToken("<|el|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50282: AddedToken("<|ms|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50283: AddedToken("<|cs|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50284: AddedToken("<|ro|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50285: AddedToken("<|da|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50286: AddedToken("<|hu|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50287: AddedToken("<|ta|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50288: AddedToken("<|no|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50289: AddedToken("<|th|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50290: AddedToken("<|ur|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50291: AddedToken("<|hr|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50292: AddedToken("<|bg|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50293: AddedToken("<|lt|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50294: AddedToken("<|la|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50295: AddedToken("<|mi|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50296: AddedToken("<|ml|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50297: AddedToken("<|cy|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50298: AddedToken("<|sk|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50299: AddedToken("<|te|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50300: AddedToken("<|fa|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50301: AddedToken("<|lv|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50302: AddedToken("<|bn|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50303: AddedToken("<|sr|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50304: AddedToken("<|az|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50305: AddedToken("<|sl|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50306: AddedToken("<|kn|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50307: AddedToken("<|et|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50308: AddedToken("<|mk|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50309: AddedToken("<|br|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50310: AddedToken("<|eu|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50311: AddedToken("<|is|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50312: AddedToken("<|hy|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50313: AddedToken("<|ne|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50314: AddedToken("<|mn|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50315: AddedToken("<|bs|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50316: AddedToken("<|kk|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50317: AddedToken("<|sq|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50318: AddedToken("<|sw|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50319: AddedToken("<|gl|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50320: AddedToken("<|mr|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50321: AddedToken("<|pa|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50322: AddedToken("<|si|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50323: AddedToken("<|km|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50324: AddedToken("<|sn|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50325: AddedToken("<|yo|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50326: AddedToken("<|so|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50327: AddedToken("<|af|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50328: AddedToken("<|oc|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50329: AddedToken("<|ka|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50330: AddedToken("<|be|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50331: AddedToken("<|tg|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50332: AddedToken("<|sd|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50333: AddedToken("<|gu|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50334: AddedToken("<|am|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50335: AddedToken("<|yi|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50336: AddedToken("<|lo|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50337: AddedToken("<|uz|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50338: AddedToken("<|fo|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50339: AddedToken("<|ht|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50340: AddedToken("<|ps|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50341: AddedToken("<|tk|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50342: AddedToken("<|nn|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50343: AddedToken("<|mt|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50344: AddedToken("<|sa|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50345: AddedToken("<|lb|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50346: AddedToken("<|my|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50347: AddedToken("<|bo|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50348: AddedToken("<|tl|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50349: AddedToken("<|mg|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50350: AddedToken("<|as|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50351: AddedToken("<|tt|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50352: AddedToken("<|haw|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50353: AddedToken("<|ln|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50354: AddedToken("<|ha|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50355: AddedToken("<|ba|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50356: AddedToken("<|jw|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50357: AddedToken("<|su|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50358: AddedToken("<|translate|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50359: AddedToken("<|transcribe|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50360: AddedToken("<|startoflm|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50361: AddedToken("<|startofprev|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50362: AddedToken("<|nocaptions|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50363: AddedToken("<|notimestamps|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), |
|
50364: AddedToken("<|0.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50365: AddedToken("<|0.02|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50366: AddedToken("<|0.04|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50367: AddedToken("<|0.06|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50368: AddedToken("<|0.08|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50369: AddedToken("<|0.10|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50370: AddedToken("<|0.12|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50371: AddedToken("<|0.14|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50372: AddedToken("<|0.16|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50373: AddedToken("<|0.18|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50374: AddedToken("<|0.20|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50375: AddedToken("<|0.22|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50376: AddedToken("<|0.24|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50377: AddedToken("<|0.26|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50378: AddedToken("<|0.28|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50379: AddedToken("<|0.30|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50380: AddedToken("<|0.32|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50381: AddedToken("<|0.34|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50382: AddedToken("<|0.36|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50383: AddedToken("<|0.38|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50384: AddedToken("<|0.40|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50385: AddedToken("<|0.42|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50386: AddedToken("<|0.44|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50387: AddedToken("<|0.46|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50388: AddedToken("<|0.48|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50389: AddedToken("<|0.50|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50390: AddedToken("<|0.52|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50391: AddedToken("<|0.54|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50392: AddedToken("<|0.56|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50393: AddedToken("<|0.58|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50394: AddedToken("<|0.60|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50395: AddedToken("<|0.62|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50396: AddedToken("<|0.64|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50397: AddedToken("<|0.66|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50398: AddedToken("<|0.68|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50399: AddedToken("<|0.70|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50400: AddedToken("<|0.72|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50401: AddedToken("<|0.74|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50402: AddedToken("<|0.76|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50403: AddedToken("<|0.78|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50404: AddedToken("<|0.80|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50405: AddedToken("<|0.82|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50406: AddedToken("<|0.84|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50407: AddedToken("<|0.86|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50408: AddedToken("<|0.88|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50409: AddedToken("<|0.90|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50410: AddedToken("<|0.92|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50411: AddedToken("<|0.94|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50412: AddedToken("<|0.96|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50413: AddedToken("<|0.98|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50414: AddedToken("<|1.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50415: AddedToken("<|1.02|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50416: AddedToken("<|1.04|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50417: AddedToken("<|1.06|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50418: AddedToken("<|1.08|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50419: AddedToken("<|1.10|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50420: AddedToken("<|1.12|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50421: AddedToken("<|1.14|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50422: AddedToken("<|1.16|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50423: AddedToken("<|1.18|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50424: AddedToken("<|1.20|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50425: AddedToken("<|1.22|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50426: AddedToken("<|1.24|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50427: AddedToken("<|1.26|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50428: AddedToken("<|1.28|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50429: AddedToken("<|1.30|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50430: AddedToken("<|1.32|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50431: AddedToken("<|1.34|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50432: AddedToken("<|1.36|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50433: AddedToken("<|1.38|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50434: AddedToken("<|1.40|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50435: AddedToken("<|1.42|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50436: AddedToken("<|1.44|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50437: AddedToken("<|1.46|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50438: AddedToken("<|1.48|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50439: AddedToken("<|1.50|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50440: AddedToken("<|1.52|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50441: AddedToken("<|1.54|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50442: AddedToken("<|1.56|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50443: AddedToken("<|1.58|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50444: AddedToken("<|1.60|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50445: AddedToken("<|1.62|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50446: AddedToken("<|1.64|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50447: AddedToken("<|1.66|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50448: AddedToken("<|1.68|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50449: AddedToken("<|1.70|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50450: AddedToken("<|1.72|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50451: AddedToken("<|1.74|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50452: AddedToken("<|1.76|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50453: AddedToken("<|1.78|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50454: AddedToken("<|1.80|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50455: AddedToken("<|1.82|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50456: AddedToken("<|1.84|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50457: AddedToken("<|1.86|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50458: AddedToken("<|1.88|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50459: AddedToken("<|1.90|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50460: AddedToken("<|1.92|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50461: AddedToken("<|1.94|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50462: AddedToken("<|1.96|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50463: AddedToken("<|1.98|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50464: AddedToken("<|2.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50465: AddedToken("<|2.02|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50466: AddedToken("<|2.04|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50467: AddedToken("<|2.06|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50468: AddedToken("<|2.08|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50469: AddedToken("<|2.10|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50470: AddedToken("<|2.12|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50471: AddedToken("<|2.14|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50472: AddedToken("<|2.16|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50473: AddedToken("<|2.18|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50474: AddedToken("<|2.20|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50475: AddedToken("<|2.22|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50476: AddedToken("<|2.24|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50477: AddedToken("<|2.26|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50478: AddedToken("<|2.28|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50479: AddedToken("<|2.30|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50480: AddedToken("<|2.32|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50481: AddedToken("<|2.34|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50482: AddedToken("<|2.36|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50483: AddedToken("<|2.38|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50484: AddedToken("<|2.40|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50485: AddedToken("<|2.42|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50486: AddedToken("<|2.44|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50487: AddedToken("<|2.46|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50488: AddedToken("<|2.48|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50489: AddedToken("<|2.50|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50490: AddedToken("<|2.52|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50491: AddedToken("<|2.54|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50492: AddedToken("<|2.56|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50493: AddedToken("<|2.58|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50494: AddedToken("<|2.60|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50495: AddedToken("<|2.62|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50496: AddedToken("<|2.64|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50497: AddedToken("<|2.66|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50498: AddedToken("<|2.68|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50499: AddedToken("<|2.70|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50500: AddedToken("<|2.72|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50501: AddedToken("<|2.74|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50502: AddedToken("<|2.76|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50503: AddedToken("<|2.78|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50504: AddedToken("<|2.80|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50505: AddedToken("<|2.82|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50506: AddedToken("<|2.84|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50507: AddedToken("<|2.86|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50508: AddedToken("<|2.88|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50509: AddedToken("<|2.90|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50510: AddedToken("<|2.92|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50511: AddedToken("<|2.94|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50512: AddedToken("<|2.96|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50513: AddedToken("<|2.98|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50514: AddedToken("<|3.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50515: AddedToken("<|3.02|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50516: AddedToken("<|3.04|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50517: AddedToken("<|3.06|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50518: AddedToken("<|3.08|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50519: AddedToken("<|3.10|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50520: AddedToken("<|3.12|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50521: AddedToken("<|3.14|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50522: AddedToken("<|3.16|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50523: AddedToken("<|3.18|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50524: AddedToken("<|3.20|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50525: AddedToken("<|3.22|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50526: AddedToken("<|3.24|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50527: AddedToken("<|3.26|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50528: AddedToken("<|3.28|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50529: AddedToken("<|3.30|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50530: AddedToken("<|3.32|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50531: AddedToken("<|3.34|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50532: AddedToken("<|3.36|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50533: AddedToken("<|3.38|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50534: AddedToken("<|3.40|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50535: AddedToken("<|3.42|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50536: AddedToken("<|3.44|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50537: AddedToken("<|3.46|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50538: AddedToken("<|3.48|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50539: AddedToken("<|3.50|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50540: AddedToken("<|3.52|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50541: AddedToken("<|3.54|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50542: AddedToken("<|3.56|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50543: AddedToken("<|3.58|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50544: AddedToken("<|3.60|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50545: AddedToken("<|3.62|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50546: AddedToken("<|3.64|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50547: AddedToken("<|3.66|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50548: AddedToken("<|3.68|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50549: AddedToken("<|3.70|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50550: AddedToken("<|3.72|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50551: AddedToken("<|3.74|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50552: AddedToken("<|3.76|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50553: AddedToken("<|3.78|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50554: AddedToken("<|3.80|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50555: AddedToken("<|3.82|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50556: AddedToken("<|3.84|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50557: AddedToken("<|3.86|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50558: AddedToken("<|3.88|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50559: AddedToken("<|3.90|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50560: AddedToken("<|3.92|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50561: AddedToken("<|3.94|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50562: AddedToken("<|3.96|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50563: AddedToken("<|3.98|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50564: AddedToken("<|4.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50565: AddedToken("<|4.02|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50566: AddedToken("<|4.04|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50567: AddedToken("<|4.06|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50568: AddedToken("<|4.08|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50569: AddedToken("<|4.10|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50570: AddedToken("<|4.12|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50571: AddedToken("<|4.14|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50572: AddedToken("<|4.16|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50573: AddedToken("<|4.18|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50574: AddedToken("<|4.20|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50575: AddedToken("<|4.22|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50576: AddedToken("<|4.24|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50577: AddedToken("<|4.26|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50578: AddedToken("<|4.28|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50579: AddedToken("<|4.30|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50580: AddedToken("<|4.32|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50581: AddedToken("<|4.34|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50582: AddedToken("<|4.36|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50583: AddedToken("<|4.38|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50584: AddedToken("<|4.40|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50585: AddedToken("<|4.42|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50586: AddedToken("<|4.44|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50587: AddedToken("<|4.46|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50588: AddedToken("<|4.48|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50589: AddedToken("<|4.50|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50590: AddedToken("<|4.52|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50591: AddedToken("<|4.54|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50592: AddedToken("<|4.56|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50593: AddedToken("<|4.58|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50594: AddedToken("<|4.60|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50595: AddedToken("<|4.62|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50596: AddedToken("<|4.64|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50597: AddedToken("<|4.66|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50598: AddedToken("<|4.68|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50599: AddedToken("<|4.70|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50600: AddedToken("<|4.72|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50601: AddedToken("<|4.74|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50602: AddedToken("<|4.76|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50603: AddedToken("<|4.78|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50604: AddedToken("<|4.80|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50605: AddedToken("<|4.82|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50606: AddedToken("<|4.84|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50607: AddedToken("<|4.86|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50608: AddedToken("<|4.88|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50609: AddedToken("<|4.90|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50610: AddedToken("<|4.92|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50611: AddedToken("<|4.94|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50612: AddedToken("<|4.96|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50613: AddedToken("<|4.98|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50614: AddedToken("<|5.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50615: AddedToken("<|5.02|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50616: AddedToken("<|5.04|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50617: AddedToken("<|5.06|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50618: AddedToken("<|5.08|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50619: AddedToken("<|5.10|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50620: AddedToken("<|5.12|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50621: AddedToken("<|5.14|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50622: AddedToken("<|5.16|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50623: AddedToken("<|5.18|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50624: AddedToken("<|5.20|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50625: AddedToken("<|5.22|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50626: AddedToken("<|5.24|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50627: AddedToken("<|5.26|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50628: AddedToken("<|5.28|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50629: AddedToken("<|5.30|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50630: AddedToken("<|5.32|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50631: AddedToken("<|5.34|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50632: AddedToken("<|5.36|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50633: AddedToken("<|5.38|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50634: AddedToken("<|5.40|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50635: AddedToken("<|5.42|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50636: AddedToken("<|5.44|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50637: AddedToken("<|5.46|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50638: AddedToken("<|5.48|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50639: AddedToken("<|5.50|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50640: AddedToken("<|5.52|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50641: AddedToken("<|5.54|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50642: AddedToken("<|5.56|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50643: AddedToken("<|5.58|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50644: AddedToken("<|5.60|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50645: AddedToken("<|5.62|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50646: AddedToken("<|5.64|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50647: AddedToken("<|5.66|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50648: AddedToken("<|5.68|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50649: AddedToken("<|5.70|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50650: AddedToken("<|5.72|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50651: AddedToken("<|5.74|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50652: AddedToken("<|5.76|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50653: AddedToken("<|5.78|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50654: AddedToken("<|5.80|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50655: AddedToken("<|5.82|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50656: AddedToken("<|5.84|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50657: AddedToken("<|5.86|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50658: AddedToken("<|5.88|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50659: AddedToken("<|5.90|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50660: AddedToken("<|5.92|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50661: AddedToken("<|5.94|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50662: AddedToken("<|5.96|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50663: AddedToken("<|5.98|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50664: AddedToken("<|6.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50665: AddedToken("<|6.02|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50666: AddedToken("<|6.04|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50667: AddedToken("<|6.06|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50668: AddedToken("<|6.08|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50669: AddedToken("<|6.10|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50670: AddedToken("<|6.12|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50671: AddedToken("<|6.14|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50672: AddedToken("<|6.16|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50673: AddedToken("<|6.18|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50674: AddedToken("<|6.20|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50675: AddedToken("<|6.22|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50676: AddedToken("<|6.24|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50677: AddedToken("<|6.26|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50678: AddedToken("<|6.28|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50679: AddedToken("<|6.30|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50680: AddedToken("<|6.32|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50681: AddedToken("<|6.34|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50682: AddedToken("<|6.36|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50683: AddedToken("<|6.38|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50684: AddedToken("<|6.40|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50685: AddedToken("<|6.42|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50686: AddedToken("<|6.44|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50687: AddedToken("<|6.46|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50688: AddedToken("<|6.48|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50689: AddedToken("<|6.50|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50690: AddedToken("<|6.52|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50691: AddedToken("<|6.54|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50692: AddedToken("<|6.56|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50693: AddedToken("<|6.58|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50694: AddedToken("<|6.60|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50695: AddedToken("<|6.62|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50696: AddedToken("<|6.64|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50697: AddedToken("<|6.66|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50698: AddedToken("<|6.68|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50699: AddedToken("<|6.70|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50700: AddedToken("<|6.72|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50701: AddedToken("<|6.74|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50702: AddedToken("<|6.76|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50703: AddedToken("<|6.78|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50704: AddedToken("<|6.80|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50705: AddedToken("<|6.82|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50706: AddedToken("<|6.84|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50707: AddedToken("<|6.86|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50708: AddedToken("<|6.88|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50709: AddedToken("<|6.90|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50710: AddedToken("<|6.92|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50711: AddedToken("<|6.94|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50712: AddedToken("<|6.96|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50713: AddedToken("<|6.98|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50714: AddedToken("<|7.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50715: AddedToken("<|7.02|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50716: AddedToken("<|7.04|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50717: AddedToken("<|7.06|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50718: AddedToken("<|7.08|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50719: AddedToken("<|7.10|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50720: AddedToken("<|7.12|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50721: AddedToken("<|7.14|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50722: AddedToken("<|7.16|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50723: AddedToken("<|7.18|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50724: AddedToken("<|7.20|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50725: AddedToken("<|7.22|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50726: AddedToken("<|7.24|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50727: AddedToken("<|7.26|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50728: AddedToken("<|7.28|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50729: AddedToken("<|7.30|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50730: AddedToken("<|7.32|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50731: AddedToken("<|7.34|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50732: AddedToken("<|7.36|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50733: AddedToken("<|7.38|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50734: AddedToken("<|7.40|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50735: AddedToken("<|7.42|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50736: AddedToken("<|7.44|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50737: AddedToken("<|7.46|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50738: AddedToken("<|7.48|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50739: AddedToken("<|7.50|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50740: AddedToken("<|7.52|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50741: AddedToken("<|7.54|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50742: AddedToken("<|7.56|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50743: AddedToken("<|7.58|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50744: AddedToken("<|7.60|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50745: AddedToken("<|7.62|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50746: AddedToken("<|7.64|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50747: AddedToken("<|7.66|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50748: AddedToken("<|7.68|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50749: AddedToken("<|7.70|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50750: AddedToken("<|7.72|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50751: AddedToken("<|7.74|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50752: AddedToken("<|7.76|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50753: AddedToken("<|7.78|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50754: AddedToken("<|7.80|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50755: AddedToken("<|7.82|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50756: AddedToken("<|7.84|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50757: AddedToken("<|7.86|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50758: AddedToken("<|7.88|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50759: AddedToken("<|7.90|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50760: AddedToken("<|7.92|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50761: AddedToken("<|7.94|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50762: AddedToken("<|7.96|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50763: AddedToken("<|7.98|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50764: AddedToken("<|8.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50765: AddedToken("<|8.02|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50766: AddedToken("<|8.04|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50767: AddedToken("<|8.06|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50768: AddedToken("<|8.08|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50769: AddedToken("<|8.10|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50770: AddedToken("<|8.12|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50771: AddedToken("<|8.14|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50772: AddedToken("<|8.16|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50773: AddedToken("<|8.18|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50774: AddedToken("<|8.20|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50775: AddedToken("<|8.22|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50776: AddedToken("<|8.24|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50777: AddedToken("<|8.26|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50778: AddedToken("<|8.28|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50779: AddedToken("<|8.30|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50780: AddedToken("<|8.32|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50781: AddedToken("<|8.34|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50782: AddedToken("<|8.36|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50783: AddedToken("<|8.38|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50784: AddedToken("<|8.40|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50785: AddedToken("<|8.42|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50786: AddedToken("<|8.44|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50787: AddedToken("<|8.46|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50788: AddedToken("<|8.48|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50789: AddedToken("<|8.50|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50790: AddedToken("<|8.52|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50791: AddedToken("<|8.54|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50792: AddedToken("<|8.56|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50793: AddedToken("<|8.58|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50794: AddedToken("<|8.60|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50795: AddedToken("<|8.62|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50796: AddedToken("<|8.64|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50797: AddedToken("<|8.66|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50798: AddedToken("<|8.68|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50799: AddedToken("<|8.70|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50800: AddedToken("<|8.72|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50801: AddedToken("<|8.74|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50802: AddedToken("<|8.76|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50803: AddedToken("<|8.78|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50804: AddedToken("<|8.80|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50805: AddedToken("<|8.82|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50806: AddedToken("<|8.84|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50807: AddedToken("<|8.86|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50808: AddedToken("<|8.88|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50809: AddedToken("<|8.90|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50810: AddedToken("<|8.92|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50811: AddedToken("<|8.94|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50812: AddedToken("<|8.96|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50813: AddedToken("<|8.98|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50814: AddedToken("<|9.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50815: AddedToken("<|9.02|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50816: AddedToken("<|9.04|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50817: AddedToken("<|9.06|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50818: AddedToken("<|9.08|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50819: AddedToken("<|9.10|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50820: AddedToken("<|9.12|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50821: AddedToken("<|9.14|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50822: AddedToken("<|9.16|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50823: AddedToken("<|9.18|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50824: AddedToken("<|9.20|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50825: AddedToken("<|9.22|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50826: AddedToken("<|9.24|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50827: AddedToken("<|9.26|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50828: AddedToken("<|9.28|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50829: AddedToken("<|9.30|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50830: AddedToken("<|9.32|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50831: AddedToken("<|9.34|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50832: AddedToken("<|9.36|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50833: AddedToken("<|9.38|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50834: AddedToken("<|9.40|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50835: AddedToken("<|9.42|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50836: AddedToken("<|9.44|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50837: AddedToken("<|9.46|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50838: AddedToken("<|9.48|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50839: AddedToken("<|9.50|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50840: AddedToken("<|9.52|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50841: AddedToken("<|9.54|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50842: AddedToken("<|9.56|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50843: AddedToken("<|9.58|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50844: AddedToken("<|9.60|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50845: AddedToken("<|9.62|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50846: AddedToken("<|9.64|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50847: AddedToken("<|9.66|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50848: AddedToken("<|9.68|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50849: AddedToken("<|9.70|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50850: AddedToken("<|9.72|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50851: AddedToken("<|9.74|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50852: AddedToken("<|9.76|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50853: AddedToken("<|9.78|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50854: AddedToken("<|9.80|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50855: AddedToken("<|9.82|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50856: AddedToken("<|9.84|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50857: AddedToken("<|9.86|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50858: AddedToken("<|9.88|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50859: AddedToken("<|9.90|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50860: AddedToken("<|9.92|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50861: AddedToken("<|9.94|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50862: AddedToken("<|9.96|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50863: AddedToken("<|9.98|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50864: AddedToken("<|10.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50865: AddedToken("<|10.02|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50866: AddedToken("<|10.04|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50867: AddedToken("<|10.06|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50868: AddedToken("<|10.08|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50869: AddedToken("<|10.10|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50870: AddedToken("<|10.12|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50871: AddedToken("<|10.14|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50872: AddedToken("<|10.16|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50873: AddedToken("<|10.18|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50874: AddedToken("<|10.20|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50875: AddedToken("<|10.22|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50876: AddedToken("<|10.24|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50877: AddedToken("<|10.26|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50878: AddedToken("<|10.28|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50879: AddedToken("<|10.30|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50880: AddedToken("<|10.32|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50881: AddedToken("<|10.34|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50882: AddedToken("<|10.36|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50883: AddedToken("<|10.38|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50884: AddedToken("<|10.40|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50885: AddedToken("<|10.42|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50886: AddedToken("<|10.44|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50887: AddedToken("<|10.46|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50888: AddedToken("<|10.48|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50889: AddedToken("<|10.50|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50890: AddedToken("<|10.52|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50891: AddedToken("<|10.54|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50892: AddedToken("<|10.56|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50893: AddedToken("<|10.58|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50894: AddedToken("<|10.60|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50895: AddedToken("<|10.62|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50896: AddedToken("<|10.64|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50897: AddedToken("<|10.66|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50898: AddedToken("<|10.68|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50899: AddedToken("<|10.70|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50900: AddedToken("<|10.72|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50901: AddedToken("<|10.74|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50902: AddedToken("<|10.76|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50903: AddedToken("<|10.78|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50904: AddedToken("<|10.80|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50905: AddedToken("<|10.82|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50906: AddedToken("<|10.84|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50907: AddedToken("<|10.86|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50908: AddedToken("<|10.88|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50909: AddedToken("<|10.90|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50910: AddedToken("<|10.92|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50911: AddedToken("<|10.94|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50912: AddedToken("<|10.96|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50913: AddedToken("<|10.98|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50914: AddedToken("<|11.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50915: AddedToken("<|11.02|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50916: AddedToken("<|11.04|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50917: AddedToken("<|11.06|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50918: AddedToken("<|11.08|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50919: AddedToken("<|11.10|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50920: AddedToken("<|11.12|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50921: AddedToken("<|11.14|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50922: AddedToken("<|11.16|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50923: AddedToken("<|11.18|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50924: AddedToken("<|11.20|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50925: AddedToken("<|11.22|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50926: AddedToken("<|11.24|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50927: AddedToken("<|11.26|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50928: AddedToken("<|11.28|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50929: AddedToken("<|11.30|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50930: AddedToken("<|11.32|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50931: AddedToken("<|11.34|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50932: AddedToken("<|11.36|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50933: AddedToken("<|11.38|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50934: AddedToken("<|11.40|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50935: AddedToken("<|11.42|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50936: AddedToken("<|11.44|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50937: AddedToken("<|11.46|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50938: AddedToken("<|11.48|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50939: AddedToken("<|11.50|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50940: AddedToken("<|11.52|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50941: AddedToken("<|11.54|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50942: AddedToken("<|11.56|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50943: AddedToken("<|11.58|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50944: AddedToken("<|11.60|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50945: AddedToken("<|11.62|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50946: AddedToken("<|11.64|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50947: AddedToken("<|11.66|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50948: AddedToken("<|11.68|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50949: AddedToken("<|11.70|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50950: AddedToken("<|11.72|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50951: AddedToken("<|11.74|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50952: AddedToken("<|11.76|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50953: AddedToken("<|11.78|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50954: AddedToken("<|11.80|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50955: AddedToken("<|11.82|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50956: AddedToken("<|11.84|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50957: AddedToken("<|11.86|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50958: AddedToken("<|11.88|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50959: AddedToken("<|11.90|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50960: AddedToken("<|11.92|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50961: AddedToken("<|11.94|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50962: AddedToken("<|11.96|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50963: AddedToken("<|11.98|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50964: AddedToken("<|12.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50965: AddedToken("<|12.02|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50966: AddedToken("<|12.04|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50967: AddedToken("<|12.06|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50968: AddedToken("<|12.08|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50969: AddedToken("<|12.10|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50970: AddedToken("<|12.12|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50971: AddedToken("<|12.14|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50972: AddedToken("<|12.16|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50973: AddedToken("<|12.18|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50974: AddedToken("<|12.20|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50975: AddedToken("<|12.22|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50976: AddedToken("<|12.24|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50977: AddedToken("<|12.26|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50978: AddedToken("<|12.28|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50979: AddedToken("<|12.30|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50980: AddedToken("<|12.32|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50981: AddedToken("<|12.34|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50982: AddedToken("<|12.36|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50983: AddedToken("<|12.38|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50984: AddedToken("<|12.40|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50985: AddedToken("<|12.42|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50986: AddedToken("<|12.44|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50987: AddedToken("<|12.46|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50988: AddedToken("<|12.48|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50989: AddedToken("<|12.50|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50990: AddedToken("<|12.52|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50991: AddedToken("<|12.54|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50992: AddedToken("<|12.56|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50993: AddedToken("<|12.58|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50994: AddedToken("<|12.60|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50995: AddedToken("<|12.62|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50996: AddedToken("<|12.64|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50997: AddedToken("<|12.66|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50998: AddedToken("<|12.68|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
50999: AddedToken("<|12.70|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51000: AddedToken("<|12.72|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51001: AddedToken("<|12.74|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51002: AddedToken("<|12.76|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51003: AddedToken("<|12.78|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51004: AddedToken("<|12.80|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51005: AddedToken("<|12.82|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51006: AddedToken("<|12.84|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51007: AddedToken("<|12.86|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51008: AddedToken("<|12.88|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51009: AddedToken("<|12.90|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51010: AddedToken("<|12.92|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51011: AddedToken("<|12.94|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51012: AddedToken("<|12.96|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51013: AddedToken("<|12.98|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51014: AddedToken("<|13.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51015: AddedToken("<|13.02|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51016: AddedToken("<|13.04|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51017: AddedToken("<|13.06|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51018: AddedToken("<|13.08|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51019: AddedToken("<|13.10|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51020: AddedToken("<|13.12|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51021: AddedToken("<|13.14|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51022: AddedToken("<|13.16|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51023: AddedToken("<|13.18|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51024: AddedToken("<|13.20|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51025: AddedToken("<|13.22|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51026: AddedToken("<|13.24|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51027: AddedToken("<|13.26|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51028: AddedToken("<|13.28|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51029: AddedToken("<|13.30|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51030: AddedToken("<|13.32|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51031: AddedToken("<|13.34|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51032: AddedToken("<|13.36|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51033: AddedToken("<|13.38|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51034: AddedToken("<|13.40|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51035: AddedToken("<|13.42|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51036: AddedToken("<|13.44|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51037: AddedToken("<|13.46|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51038: AddedToken("<|13.48|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51039: AddedToken("<|13.50|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51040: AddedToken("<|13.52|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51041: AddedToken("<|13.54|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51042: AddedToken("<|13.56|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51043: AddedToken("<|13.58|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51044: AddedToken("<|13.60|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51045: AddedToken("<|13.62|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51046: AddedToken("<|13.64|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51047: AddedToken("<|13.66|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51048: AddedToken("<|13.68|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51049: AddedToken("<|13.70|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51050: AddedToken("<|13.72|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51051: AddedToken("<|13.74|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51052: AddedToken("<|13.76|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51053: AddedToken("<|13.78|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51054: AddedToken("<|13.80|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51055: AddedToken("<|13.82|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51056: AddedToken("<|13.84|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51057: AddedToken("<|13.86|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51058: AddedToken("<|13.88|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51059: AddedToken("<|13.90|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51060: AddedToken("<|13.92|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51061: AddedToken("<|13.94|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51062: AddedToken("<|13.96|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51063: AddedToken("<|13.98|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51064: AddedToken("<|14.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51065: AddedToken("<|14.02|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51066: AddedToken("<|14.04|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51067: AddedToken("<|14.06|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51068: AddedToken("<|14.08|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51069: AddedToken("<|14.10|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51070: AddedToken("<|14.12|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51071: AddedToken("<|14.14|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51072: AddedToken("<|14.16|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51073: AddedToken("<|14.18|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51074: AddedToken("<|14.20|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51075: AddedToken("<|14.22|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51076: AddedToken("<|14.24|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51077: AddedToken("<|14.26|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51078: AddedToken("<|14.28|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51079: AddedToken("<|14.30|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51080: AddedToken("<|14.32|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51081: AddedToken("<|14.34|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51082: AddedToken("<|14.36|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51083: AddedToken("<|14.38|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51084: AddedToken("<|14.40|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51085: AddedToken("<|14.42|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51086: AddedToken("<|14.44|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51087: AddedToken("<|14.46|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51088: AddedToken("<|14.48|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51089: AddedToken("<|14.50|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51090: AddedToken("<|14.52|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51091: AddedToken("<|14.54|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51092: AddedToken("<|14.56|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51093: AddedToken("<|14.58|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51094: AddedToken("<|14.60|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51095: AddedToken("<|14.62|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51096: AddedToken("<|14.64|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51097: AddedToken("<|14.66|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51098: AddedToken("<|14.68|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51099: AddedToken("<|14.70|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51100: AddedToken("<|14.72|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51101: AddedToken("<|14.74|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51102: AddedToken("<|14.76|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51103: AddedToken("<|14.78|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51104: AddedToken("<|14.80|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51105: AddedToken("<|14.82|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51106: AddedToken("<|14.84|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51107: AddedToken("<|14.86|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51108: AddedToken("<|14.88|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51109: AddedToken("<|14.90|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51110: AddedToken("<|14.92|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51111: AddedToken("<|14.94|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51112: AddedToken("<|14.96|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51113: AddedToken("<|14.98|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51114: AddedToken("<|15.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51115: AddedToken("<|15.02|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51116: AddedToken("<|15.04|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51117: AddedToken("<|15.06|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51118: AddedToken("<|15.08|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51119: AddedToken("<|15.10|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51120: AddedToken("<|15.12|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51121: AddedToken("<|15.14|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51122: AddedToken("<|15.16|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51123: AddedToken("<|15.18|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51124: AddedToken("<|15.20|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51125: AddedToken("<|15.22|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51126: AddedToken("<|15.24|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51127: AddedToken("<|15.26|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51128: AddedToken("<|15.28|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51129: AddedToken("<|15.30|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51130: AddedToken("<|15.32|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51131: AddedToken("<|15.34|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51132: AddedToken("<|15.36|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51133: AddedToken("<|15.38|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51134: AddedToken("<|15.40|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51135: AddedToken("<|15.42|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51136: AddedToken("<|15.44|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51137: AddedToken("<|15.46|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51138: AddedToken("<|15.48|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51139: AddedToken("<|15.50|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51140: AddedToken("<|15.52|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51141: AddedToken("<|15.54|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51142: AddedToken("<|15.56|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51143: AddedToken("<|15.58|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51144: AddedToken("<|15.60|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51145: AddedToken("<|15.62|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51146: AddedToken("<|15.64|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51147: AddedToken("<|15.66|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51148: AddedToken("<|15.68|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51149: AddedToken("<|15.70|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51150: AddedToken("<|15.72|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51151: AddedToken("<|15.74|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51152: AddedToken("<|15.76|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51153: AddedToken("<|15.78|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51154: AddedToken("<|15.80|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51155: AddedToken("<|15.82|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51156: AddedToken("<|15.84|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51157: AddedToken("<|15.86|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51158: AddedToken("<|15.88|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51159: AddedToken("<|15.90|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51160: AddedToken("<|15.92|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51161: AddedToken("<|15.94|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51162: AddedToken("<|15.96|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51163: AddedToken("<|15.98|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51164: AddedToken("<|16.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51165: AddedToken("<|16.02|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51166: AddedToken("<|16.04|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51167: AddedToken("<|16.06|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51168: AddedToken("<|16.08|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51169: AddedToken("<|16.10|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51170: AddedToken("<|16.12|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51171: AddedToken("<|16.14|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51172: AddedToken("<|16.16|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51173: AddedToken("<|16.18|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51174: AddedToken("<|16.20|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51175: AddedToken("<|16.22|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51176: AddedToken("<|16.24|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51177: AddedToken("<|16.26|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51178: AddedToken("<|16.28|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51179: AddedToken("<|16.30|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51180: AddedToken("<|16.32|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51181: AddedToken("<|16.34|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51182: AddedToken("<|16.36|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51183: AddedToken("<|16.38|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51184: AddedToken("<|16.40|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51185: AddedToken("<|16.42|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51186: AddedToken("<|16.44|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51187: AddedToken("<|16.46|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51188: AddedToken("<|16.48|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51189: AddedToken("<|16.50|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51190: AddedToken("<|16.52|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51191: AddedToken("<|16.54|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51192: AddedToken("<|16.56|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51193: AddedToken("<|16.58|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51194: AddedToken("<|16.60|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51195: AddedToken("<|16.62|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51196: AddedToken("<|16.64|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51197: AddedToken("<|16.66|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51198: AddedToken("<|16.68|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51199: AddedToken("<|16.70|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51200: AddedToken("<|16.72|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51201: AddedToken("<|16.74|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51202: AddedToken("<|16.76|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51203: AddedToken("<|16.78|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51204: AddedToken("<|16.80|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51205: AddedToken("<|16.82|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51206: AddedToken("<|16.84|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51207: AddedToken("<|16.86|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51208: AddedToken("<|16.88|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51209: AddedToken("<|16.90|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51210: AddedToken("<|16.92|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51211: AddedToken("<|16.94|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51212: AddedToken("<|16.96|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51213: AddedToken("<|16.98|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51214: AddedToken("<|17.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51215: AddedToken("<|17.02|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51216: AddedToken("<|17.04|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51217: AddedToken("<|17.06|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51218: AddedToken("<|17.08|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51219: AddedToken("<|17.10|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51220: AddedToken("<|17.12|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51221: AddedToken("<|17.14|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51222: AddedToken("<|17.16|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51223: AddedToken("<|17.18|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51224: AddedToken("<|17.20|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51225: AddedToken("<|17.22|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51226: AddedToken("<|17.24|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51227: AddedToken("<|17.26|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51228: AddedToken("<|17.28|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51229: AddedToken("<|17.30|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51230: AddedToken("<|17.32|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51231: AddedToken("<|17.34|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51232: AddedToken("<|17.36|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51233: AddedToken("<|17.38|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51234: AddedToken("<|17.40|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51235: AddedToken("<|17.42|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51236: AddedToken("<|17.44|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51237: AddedToken("<|17.46|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51238: AddedToken("<|17.48|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51239: AddedToken("<|17.50|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51240: AddedToken("<|17.52|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51241: AddedToken("<|17.54|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51242: AddedToken("<|17.56|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51243: AddedToken("<|17.58|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51244: AddedToken("<|17.60|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51245: AddedToken("<|17.62|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51246: AddedToken("<|17.64|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51247: AddedToken("<|17.66|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51248: AddedToken("<|17.68|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51249: AddedToken("<|17.70|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51250: AddedToken("<|17.72|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51251: AddedToken("<|17.74|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51252: AddedToken("<|17.76|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51253: AddedToken("<|17.78|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51254: AddedToken("<|17.80|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51255: AddedToken("<|17.82|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51256: AddedToken("<|17.84|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51257: AddedToken("<|17.86|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51258: AddedToken("<|17.88|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51259: AddedToken("<|17.90|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51260: AddedToken("<|17.92|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51261: AddedToken("<|17.94|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51262: AddedToken("<|17.96|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51263: AddedToken("<|17.98|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51264: AddedToken("<|18.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51265: AddedToken("<|18.02|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51266: AddedToken("<|18.04|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51267: AddedToken("<|18.06|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51268: AddedToken("<|18.08|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51269: AddedToken("<|18.10|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51270: AddedToken("<|18.12|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51271: AddedToken("<|18.14|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51272: AddedToken("<|18.16|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51273: AddedToken("<|18.18|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51274: AddedToken("<|18.20|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51275: AddedToken("<|18.22|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51276: AddedToken("<|18.24|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51277: AddedToken("<|18.26|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51278: AddedToken("<|18.28|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51279: AddedToken("<|18.30|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51280: AddedToken("<|18.32|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51281: AddedToken("<|18.34|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51282: AddedToken("<|18.36|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51283: AddedToken("<|18.38|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51284: AddedToken("<|18.40|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51285: AddedToken("<|18.42|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51286: AddedToken("<|18.44|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51287: AddedToken("<|18.46|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51288: AddedToken("<|18.48|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51289: AddedToken("<|18.50|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51290: AddedToken("<|18.52|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51291: AddedToken("<|18.54|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51292: AddedToken("<|18.56|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51293: AddedToken("<|18.58|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51294: AddedToken("<|18.60|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51295: AddedToken("<|18.62|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51296: AddedToken("<|18.64|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51297: AddedToken("<|18.66|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51298: AddedToken("<|18.68|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51299: AddedToken("<|18.70|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51300: AddedToken("<|18.72|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51301: AddedToken("<|18.74|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51302: AddedToken("<|18.76|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51303: AddedToken("<|18.78|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51304: AddedToken("<|18.80|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51305: AddedToken("<|18.82|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51306: AddedToken("<|18.84|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51307: AddedToken("<|18.86|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51308: AddedToken("<|18.88|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51309: AddedToken("<|18.90|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51310: AddedToken("<|18.92|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51311: AddedToken("<|18.94|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51312: AddedToken("<|18.96|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51313: AddedToken("<|18.98|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51314: AddedToken("<|19.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51315: AddedToken("<|19.02|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51316: AddedToken("<|19.04|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51317: AddedToken("<|19.06|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51318: AddedToken("<|19.08|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51319: AddedToken("<|19.10|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51320: AddedToken("<|19.12|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51321: AddedToken("<|19.14|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51322: AddedToken("<|19.16|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51323: AddedToken("<|19.18|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51324: AddedToken("<|19.20|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51325: AddedToken("<|19.22|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51326: AddedToken("<|19.24|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51327: AddedToken("<|19.26|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51328: AddedToken("<|19.28|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51329: AddedToken("<|19.30|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51330: AddedToken("<|19.32|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51331: AddedToken("<|19.34|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51332: AddedToken("<|19.36|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51333: AddedToken("<|19.38|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51334: AddedToken("<|19.40|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51335: AddedToken("<|19.42|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51336: AddedToken("<|19.44|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51337: AddedToken("<|19.46|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51338: AddedToken("<|19.48|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51339: AddedToken("<|19.50|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51340: AddedToken("<|19.52|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51341: AddedToken("<|19.54|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51342: AddedToken("<|19.56|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51343: AddedToken("<|19.58|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51344: AddedToken("<|19.60|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51345: AddedToken("<|19.62|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51346: AddedToken("<|19.64|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51347: AddedToken("<|19.66|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51348: AddedToken("<|19.68|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51349: AddedToken("<|19.70|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51350: AddedToken("<|19.72|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51351: AddedToken("<|19.74|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51352: AddedToken("<|19.76|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51353: AddedToken("<|19.78|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51354: AddedToken("<|19.80|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51355: AddedToken("<|19.82|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51356: AddedToken("<|19.84|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51357: AddedToken("<|19.86|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51358: AddedToken("<|19.88|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51359: AddedToken("<|19.90|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51360: AddedToken("<|19.92|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51361: AddedToken("<|19.94|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51362: AddedToken("<|19.96|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51363: AddedToken("<|19.98|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51364: AddedToken("<|20.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51365: AddedToken("<|20.02|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51366: AddedToken("<|20.04|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51367: AddedToken("<|20.06|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51368: AddedToken("<|20.08|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51369: AddedToken("<|20.10|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51370: AddedToken("<|20.12|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51371: AddedToken("<|20.14|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51372: AddedToken("<|20.16|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51373: AddedToken("<|20.18|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51374: AddedToken("<|20.20|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51375: AddedToken("<|20.22|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51376: AddedToken("<|20.24|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51377: AddedToken("<|20.26|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51378: AddedToken("<|20.28|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51379: AddedToken("<|20.30|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51380: AddedToken("<|20.32|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51381: AddedToken("<|20.34|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51382: AddedToken("<|20.36|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51383: AddedToken("<|20.38|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51384: AddedToken("<|20.40|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51385: AddedToken("<|20.42|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51386: AddedToken("<|20.44|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51387: AddedToken("<|20.46|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51388: AddedToken("<|20.48|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51389: AddedToken("<|20.50|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51390: AddedToken("<|20.52|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51391: AddedToken("<|20.54|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51392: AddedToken("<|20.56|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51393: AddedToken("<|20.58|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51394: AddedToken("<|20.60|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51395: AddedToken("<|20.62|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51396: AddedToken("<|20.64|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51397: AddedToken("<|20.66|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51398: AddedToken("<|20.68|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51399: AddedToken("<|20.70|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51400: AddedToken("<|20.72|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51401: AddedToken("<|20.74|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51402: AddedToken("<|20.76|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51403: AddedToken("<|20.78|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51404: AddedToken("<|20.80|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51405: AddedToken("<|20.82|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51406: AddedToken("<|20.84|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51407: AddedToken("<|20.86|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51408: AddedToken("<|20.88|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51409: AddedToken("<|20.90|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51410: AddedToken("<|20.92|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51411: AddedToken("<|20.94|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51412: AddedToken("<|20.96|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51413: AddedToken("<|20.98|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51414: AddedToken("<|21.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51415: AddedToken("<|21.02|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51416: AddedToken("<|21.04|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51417: AddedToken("<|21.06|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51418: AddedToken("<|21.08|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51419: AddedToken("<|21.10|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51420: AddedToken("<|21.12|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51421: AddedToken("<|21.14|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51422: AddedToken("<|21.16|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51423: AddedToken("<|21.18|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51424: AddedToken("<|21.20|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51425: AddedToken("<|21.22|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51426: AddedToken("<|21.24|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51427: AddedToken("<|21.26|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51428: AddedToken("<|21.28|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51429: AddedToken("<|21.30|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51430: AddedToken("<|21.32|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51431: AddedToken("<|21.34|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51432: AddedToken("<|21.36|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51433: AddedToken("<|21.38|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51434: AddedToken("<|21.40|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51435: AddedToken("<|21.42|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51436: AddedToken("<|21.44|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51437: AddedToken("<|21.46|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51438: AddedToken("<|21.48|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51439: AddedToken("<|21.50|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51440: AddedToken("<|21.52|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51441: AddedToken("<|21.54|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51442: AddedToken("<|21.56|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51443: AddedToken("<|21.58|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51444: AddedToken("<|21.60|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51445: AddedToken("<|21.62|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51446: AddedToken("<|21.64|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51447: AddedToken("<|21.66|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51448: AddedToken("<|21.68|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51449: AddedToken("<|21.70|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51450: AddedToken("<|21.72|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51451: AddedToken("<|21.74|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51452: AddedToken("<|21.76|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51453: AddedToken("<|21.78|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51454: AddedToken("<|21.80|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51455: AddedToken("<|21.82|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51456: AddedToken("<|21.84|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51457: AddedToken("<|21.86|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51458: AddedToken("<|21.88|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51459: AddedToken("<|21.90|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51460: AddedToken("<|21.92|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51461: AddedToken("<|21.94|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51462: AddedToken("<|21.96|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51463: AddedToken("<|21.98|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51464: AddedToken("<|22.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51465: AddedToken("<|22.02|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51466: AddedToken("<|22.04|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51467: AddedToken("<|22.06|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51468: AddedToken("<|22.08|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51469: AddedToken("<|22.10|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51470: AddedToken("<|22.12|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51471: AddedToken("<|22.14|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51472: AddedToken("<|22.16|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51473: AddedToken("<|22.18|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51474: AddedToken("<|22.20|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51475: AddedToken("<|22.22|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51476: AddedToken("<|22.24|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51477: AddedToken("<|22.26|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51478: AddedToken("<|22.28|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51479: AddedToken("<|22.30|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51480: AddedToken("<|22.32|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51481: AddedToken("<|22.34|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51482: AddedToken("<|22.36|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51483: AddedToken("<|22.38|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51484: AddedToken("<|22.40|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51485: AddedToken("<|22.42|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51486: AddedToken("<|22.44|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51487: AddedToken("<|22.46|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51488: AddedToken("<|22.48|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51489: AddedToken("<|22.50|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51490: AddedToken("<|22.52|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51491: AddedToken("<|22.54|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51492: AddedToken("<|22.56|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51493: AddedToken("<|22.58|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51494: AddedToken("<|22.60|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51495: AddedToken("<|22.62|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51496: AddedToken("<|22.64|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51497: AddedToken("<|22.66|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51498: AddedToken("<|22.68|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51499: AddedToken("<|22.70|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51500: AddedToken("<|22.72|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51501: AddedToken("<|22.74|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51502: AddedToken("<|22.76|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51503: AddedToken("<|22.78|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51504: AddedToken("<|22.80|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51505: AddedToken("<|22.82|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51506: AddedToken("<|22.84|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51507: AddedToken("<|22.86|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51508: AddedToken("<|22.88|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51509: AddedToken("<|22.90|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51510: AddedToken("<|22.92|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51511: AddedToken("<|22.94|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51512: AddedToken("<|22.96|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51513: AddedToken("<|22.98|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51514: AddedToken("<|23.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51515: AddedToken("<|23.02|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51516: AddedToken("<|23.04|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51517: AddedToken("<|23.06|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51518: AddedToken("<|23.08|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51519: AddedToken("<|23.10|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51520: AddedToken("<|23.12|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51521: AddedToken("<|23.14|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51522: AddedToken("<|23.16|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51523: AddedToken("<|23.18|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51524: AddedToken("<|23.20|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51525: AddedToken("<|23.22|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51526: AddedToken("<|23.24|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51527: AddedToken("<|23.26|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51528: AddedToken("<|23.28|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51529: AddedToken("<|23.30|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51530: AddedToken("<|23.32|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51531: AddedToken("<|23.34|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51532: AddedToken("<|23.36|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51533: AddedToken("<|23.38|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51534: AddedToken("<|23.40|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51535: AddedToken("<|23.42|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51536: AddedToken("<|23.44|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51537: AddedToken("<|23.46|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51538: AddedToken("<|23.48|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51539: AddedToken("<|23.50|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51540: AddedToken("<|23.52|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51541: AddedToken("<|23.54|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51542: AddedToken("<|23.56|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51543: AddedToken("<|23.58|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51544: AddedToken("<|23.60|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51545: AddedToken("<|23.62|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51546: AddedToken("<|23.64|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51547: AddedToken("<|23.66|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51548: AddedToken("<|23.68|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51549: AddedToken("<|23.70|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51550: AddedToken("<|23.72|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51551: AddedToken("<|23.74|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51552: AddedToken("<|23.76|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51553: AddedToken("<|23.78|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51554: AddedToken("<|23.80|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51555: AddedToken("<|23.82|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51556: AddedToken("<|23.84|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51557: AddedToken("<|23.86|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51558: AddedToken("<|23.88|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51559: AddedToken("<|23.90|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51560: AddedToken("<|23.92|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51561: AddedToken("<|23.94|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51562: AddedToken("<|23.96|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51563: AddedToken("<|23.98|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51564: AddedToken("<|24.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51565: AddedToken("<|24.02|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51566: AddedToken("<|24.04|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51567: AddedToken("<|24.06|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51568: AddedToken("<|24.08|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51569: AddedToken("<|24.10|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51570: AddedToken("<|24.12|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51571: AddedToken("<|24.14|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51572: AddedToken("<|24.16|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51573: AddedToken("<|24.18|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51574: AddedToken("<|24.20|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51575: AddedToken("<|24.22|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51576: AddedToken("<|24.24|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51577: AddedToken("<|24.26|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51578: AddedToken("<|24.28|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51579: AddedToken("<|24.30|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51580: AddedToken("<|24.32|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51581: AddedToken("<|24.34|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51582: AddedToken("<|24.36|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51583: AddedToken("<|24.38|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51584: AddedToken("<|24.40|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51585: AddedToken("<|24.42|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51586: AddedToken("<|24.44|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51587: AddedToken("<|24.46|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51588: AddedToken("<|24.48|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51589: AddedToken("<|24.50|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51590: AddedToken("<|24.52|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51591: AddedToken("<|24.54|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51592: AddedToken("<|24.56|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51593: AddedToken("<|24.58|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51594: AddedToken("<|24.60|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51595: AddedToken("<|24.62|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51596: AddedToken("<|24.64|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51597: AddedToken("<|24.66|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51598: AddedToken("<|24.68|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51599: AddedToken("<|24.70|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51600: AddedToken("<|24.72|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51601: AddedToken("<|24.74|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51602: AddedToken("<|24.76|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51603: AddedToken("<|24.78|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51604: AddedToken("<|24.80|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51605: AddedToken("<|24.82|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51606: AddedToken("<|24.84|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51607: AddedToken("<|24.86|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51608: AddedToken("<|24.88|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51609: AddedToken("<|24.90|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51610: AddedToken("<|24.92|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51611: AddedToken("<|24.94|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51612: AddedToken("<|24.96|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51613: AddedToken("<|24.98|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51614: AddedToken("<|25.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51615: AddedToken("<|25.02|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51616: AddedToken("<|25.04|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51617: AddedToken("<|25.06|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51618: AddedToken("<|25.08|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51619: AddedToken("<|25.10|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51620: AddedToken("<|25.12|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51621: AddedToken("<|25.14|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51622: AddedToken("<|25.16|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51623: AddedToken("<|25.18|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51624: AddedToken("<|25.20|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51625: AddedToken("<|25.22|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51626: AddedToken("<|25.24|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51627: AddedToken("<|25.26|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51628: AddedToken("<|25.28|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51629: AddedToken("<|25.30|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51630: AddedToken("<|25.32|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51631: AddedToken("<|25.34|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51632: AddedToken("<|25.36|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51633: AddedToken("<|25.38|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51634: AddedToken("<|25.40|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51635: AddedToken("<|25.42|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51636: AddedToken("<|25.44|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51637: AddedToken("<|25.46|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51638: AddedToken("<|25.48|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51639: AddedToken("<|25.50|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51640: AddedToken("<|25.52|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51641: AddedToken("<|25.54|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51642: AddedToken("<|25.56|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51643: AddedToken("<|25.58|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51644: AddedToken("<|25.60|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51645: AddedToken("<|25.62|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51646: AddedToken("<|25.64|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51647: AddedToken("<|25.66|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51648: AddedToken("<|25.68|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51649: AddedToken("<|25.70|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51650: AddedToken("<|25.72|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51651: AddedToken("<|25.74|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51652: AddedToken("<|25.76|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51653: AddedToken("<|25.78|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51654: AddedToken("<|25.80|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51655: AddedToken("<|25.82|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51656: AddedToken("<|25.84|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51657: AddedToken("<|25.86|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51658: AddedToken("<|25.88|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51659: AddedToken("<|25.90|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51660: AddedToken("<|25.92|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51661: AddedToken("<|25.94|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51662: AddedToken("<|25.96|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51663: AddedToken("<|25.98|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51664: AddedToken("<|26.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51665: AddedToken("<|26.02|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51666: AddedToken("<|26.04|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51667: AddedToken("<|26.06|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51668: AddedToken("<|26.08|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51669: AddedToken("<|26.10|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51670: AddedToken("<|26.12|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51671: AddedToken("<|26.14|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51672: AddedToken("<|26.16|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51673: AddedToken("<|26.18|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51674: AddedToken("<|26.20|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51675: AddedToken("<|26.22|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51676: AddedToken("<|26.24|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51677: AddedToken("<|26.26|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51678: AddedToken("<|26.28|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51679: AddedToken("<|26.30|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51680: AddedToken("<|26.32|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51681: AddedToken("<|26.34|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51682: AddedToken("<|26.36|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51683: AddedToken("<|26.38|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51684: AddedToken("<|26.40|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51685: AddedToken("<|26.42|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51686: AddedToken("<|26.44|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51687: AddedToken("<|26.46|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51688: AddedToken("<|26.48|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51689: AddedToken("<|26.50|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51690: AddedToken("<|26.52|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51691: AddedToken("<|26.54|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51692: AddedToken("<|26.56|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51693: AddedToken("<|26.58|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51694: AddedToken("<|26.60|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51695: AddedToken("<|26.62|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51696: AddedToken("<|26.64|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51697: AddedToken("<|26.66|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51698: AddedToken("<|26.68|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51699: AddedToken("<|26.70|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51700: AddedToken("<|26.72|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51701: AddedToken("<|26.74|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51702: AddedToken("<|26.76|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51703: AddedToken("<|26.78|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51704: AddedToken("<|26.80|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51705: AddedToken("<|26.82|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51706: AddedToken("<|26.84|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51707: AddedToken("<|26.86|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51708: AddedToken("<|26.88|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51709: AddedToken("<|26.90|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51710: AddedToken("<|26.92|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51711: AddedToken("<|26.94|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51712: AddedToken("<|26.96|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51713: AddedToken("<|26.98|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51714: AddedToken("<|27.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51715: AddedToken("<|27.02|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51716: AddedToken("<|27.04|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51717: AddedToken("<|27.06|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51718: AddedToken("<|27.08|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51719: AddedToken("<|27.10|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51720: AddedToken("<|27.12|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51721: AddedToken("<|27.14|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51722: AddedToken("<|27.16|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51723: AddedToken("<|27.18|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51724: AddedToken("<|27.20|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51725: AddedToken("<|27.22|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51726: AddedToken("<|27.24|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51727: AddedToken("<|27.26|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51728: AddedToken("<|27.28|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51729: AddedToken("<|27.30|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51730: AddedToken("<|27.32|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51731: AddedToken("<|27.34|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51732: AddedToken("<|27.36|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51733: AddedToken("<|27.38|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51734: AddedToken("<|27.40|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51735: AddedToken("<|27.42|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51736: AddedToken("<|27.44|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51737: AddedToken("<|27.46|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51738: AddedToken("<|27.48|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51739: AddedToken("<|27.50|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51740: AddedToken("<|27.52|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51741: AddedToken("<|27.54|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51742: AddedToken("<|27.56|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51743: AddedToken("<|27.58|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51744: AddedToken("<|27.60|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51745: AddedToken("<|27.62|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51746: AddedToken("<|27.64|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51747: AddedToken("<|27.66|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51748: AddedToken("<|27.68|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51749: AddedToken("<|27.70|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51750: AddedToken("<|27.72|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51751: AddedToken("<|27.74|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51752: AddedToken("<|27.76|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51753: AddedToken("<|27.78|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51754: AddedToken("<|27.80|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51755: AddedToken("<|27.82|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51756: AddedToken("<|27.84|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51757: AddedToken("<|27.86|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51758: AddedToken("<|27.88|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51759: AddedToken("<|27.90|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51760: AddedToken("<|27.92|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51761: AddedToken("<|27.94|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51762: AddedToken("<|27.96|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51763: AddedToken("<|27.98|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51764: AddedToken("<|28.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51765: AddedToken("<|28.02|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51766: AddedToken("<|28.04|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51767: AddedToken("<|28.06|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51768: AddedToken("<|28.08|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51769: AddedToken("<|28.10|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51770: AddedToken("<|28.12|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51771: AddedToken("<|28.14|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51772: AddedToken("<|28.16|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51773: AddedToken("<|28.18|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51774: AddedToken("<|28.20|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51775: AddedToken("<|28.22|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51776: AddedToken("<|28.24|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51777: AddedToken("<|28.26|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51778: AddedToken("<|28.28|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51779: AddedToken("<|28.30|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51780: AddedToken("<|28.32|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51781: AddedToken("<|28.34|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51782: AddedToken("<|28.36|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51783: AddedToken("<|28.38|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51784: AddedToken("<|28.40|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51785: AddedToken("<|28.42|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51786: AddedToken("<|28.44|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51787: AddedToken("<|28.46|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51788: AddedToken("<|28.48|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51789: AddedToken("<|28.50|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51790: AddedToken("<|28.52|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51791: AddedToken("<|28.54|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51792: AddedToken("<|28.56|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51793: AddedToken("<|28.58|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51794: AddedToken("<|28.60|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51795: AddedToken("<|28.62|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51796: AddedToken("<|28.64|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51797: AddedToken("<|28.66|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51798: AddedToken("<|28.68|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51799: AddedToken("<|28.70|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51800: AddedToken("<|28.72|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51801: AddedToken("<|28.74|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51802: AddedToken("<|28.76|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51803: AddedToken("<|28.78|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51804: AddedToken("<|28.80|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51805: AddedToken("<|28.82|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51806: AddedToken("<|28.84|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51807: AddedToken("<|28.86|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51808: AddedToken("<|28.88|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51809: AddedToken("<|28.90|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51810: AddedToken("<|28.92|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51811: AddedToken("<|28.94|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51812: AddedToken("<|28.96|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51813: AddedToken("<|28.98|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51814: AddedToken("<|29.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51815: AddedToken("<|29.02|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51816: AddedToken("<|29.04|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51817: AddedToken("<|29.06|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51818: AddedToken("<|29.08|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51819: AddedToken("<|29.10|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51820: AddedToken("<|29.12|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51821: AddedToken("<|29.14|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51822: AddedToken("<|29.16|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51823: AddedToken("<|29.18|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51824: AddedToken("<|29.20|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51825: AddedToken("<|29.22|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51826: AddedToken("<|29.24|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51827: AddedToken("<|29.26|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51828: AddedToken("<|29.28|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51829: AddedToken("<|29.30|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51830: AddedToken("<|29.32|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51831: AddedToken("<|29.34|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51832: AddedToken("<|29.36|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51833: AddedToken("<|29.38|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51834: AddedToken("<|29.40|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51835: AddedToken("<|29.42|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51836: AddedToken("<|29.44|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51837: AddedToken("<|29.46|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51838: AddedToken("<|29.48|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51839: AddedToken("<|29.50|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51840: AddedToken("<|29.52|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51841: AddedToken("<|29.54|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51842: AddedToken("<|29.56|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51843: AddedToken("<|29.58|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51844: AddedToken("<|29.60|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51845: AddedToken("<|29.62|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51846: AddedToken("<|29.64|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51847: AddedToken("<|29.66|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51848: AddedToken("<|29.68|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51849: AddedToken("<|29.70|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51850: AddedToken("<|29.72|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51851: AddedToken("<|29.74|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51852: AddedToken("<|29.76|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51853: AddedToken("<|29.78|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51854: AddedToken("<|29.80|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51855: AddedToken("<|29.82|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51856: AddedToken("<|29.84|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51857: AddedToken("<|29.86|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51858: AddedToken("<|29.88|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51859: AddedToken("<|29.90|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51860: AddedToken("<|29.92|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51861: AddedToken("<|29.94|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51862: AddedToken("<|29.96|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51863: AddedToken("<|29.98|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
51864: AddedToken("<|30.00|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=False), |
|
} |
|
|
|
{ |
|
"processor_class": "WhisperProcessor" |
|
} |
|
|
|
05/31/2024 02:23:47 - INFO - __main__ - eval_steps is not set, evaluating at the end of each epoch |
|
05/31/2024 02:23:48 - INFO - __main__ - ***** Running training ***** |
|
05/31/2024 02:23:48 - INFO - __main__ - Num examples = 320 |
|
05/31/2024 02:23:48 - INFO - __main__ - Num epochs = 5 |
|
05/31/2024 02:23:48 - INFO - __main__ - Instantaneous batch size per device = 8 |
|
05/31/2024 02:23:48 - INFO - __main__ - Gradient accumulation steps = 1 |
|
05/31/2024 02:23:48 - INFO - __main__ - Total train batch size (w. parallel & distributed) = 8 |
|
05/31/2024 02:23:48 - INFO - __main__ - Total optimization steps = 40 |
|
Train steps ... : 0%| | 0/40 [00:00<?, ?it/s]`use_cache = True` is incompatible with gradient checkpointing. Setting `use_cache = False`... |
|
Train steps ... : 2%|β | 1/40 [00:08<05:39, 8.70s/it]
Train steps ... : 5%|β | 2/40 [00:09<02:32, 4.00s/it]
Train steps ... : 8%|β | 3/40 [00:10<01:33, 2.52s/it]
Train steps ... : 10%|β | 4/40 [00:11<01:06, 1.86s/it]
Train steps ... : 12%|ββ | 5/40 [00:11<00:51, 1.48s/it]
Train steps ... : 15%|ββ | 6/40 [00:12<00:42, 1.25s/it]
Train steps ... : 18%|ββ | 7/40 [00:13<00:36, 1.09s/it]
Train steps ... : 20%|ββ | 8/40 [00:14<00:31, 1.01it/s]
Train steps ... : 22%|βββ | 9/40 [00:14<00:25, 1.21it/s]
Train steps ... : 25%|βββ | 10/40 [00:17<00:44, 1.49s/it]
Train steps ... : 28%|βββ | 11/40 [00:18<00:37, 1.28s/it]
Train steps ... : 30%|βββ | 12/40 [00:19<00:31, 1.13s/it]
Train steps ... : 32%|ββββ | 13/40 [00:19<00:27, 1.03s/it]
Train steps ... : 35%|ββββ | 14/40 [00:20<00:24, 1.05it/s]
Train steps ... : 38%|ββββ | 15/40 [00:21<00:22, 1.10it/s]
Train steps ... : 40%|ββββ | 16/40 [00:22<00:20, 1.15it/s]
Train steps ... : 42%|βββββ | 17/40 [00:23<00:19, 1.20it/s]
Train steps ... : 45%|βββββ | 18/40 [00:23<00:15, 1.38it/s]
Train steps ... : 48%|βββββ | 19/40 [00:26<00:29, 1.40s/it]
Train steps ... : 50%|βββββ | 20/40 [00:27<00:24, 1.22s/it]
Train steps ... : 52%|ββββββ | 21/40 [00:28<00:20, 1.10s/it]
Train steps ... : 55%|ββββββ | 22/40 [00:28<00:18, 1.01s/it]
Train steps ... : 57%|ββββββ | 23/40 [00:29<00:16, 1.06it/s]
Train steps ... : 60%|ββββββ | 24/40 [00:30<00:14, 1.11it/s]
Train steps ... : 62%|βββββββ | 25/40 [00:31<00:12, 1.16it/s]
Train steps ... : 65%|βββββββ | 26/40 [00:32<00:11, 1.20it/s]
Train steps ... : 68%|βββββββ | 27/40 [00:32<00:09, 1.39it/s]
Train steps ... : 70%|βββββββ | 28/40 [00:35<00:16, 1.40s/it]
Train steps ... : 72%|ββββββββ | 29/40 [00:36<00:13, 1.22s/it]
Train steps ... : 75%|ββββββββ | 30/40 [00:37<00:10, 1.09s/it]
Train steps ... : 78%|ββββββββ | 31/40 [00:37<00:09, 1.00s/it]
Train steps ... : 80%|ββββββββ | 32/40 [00:38<00:07, 1.07it/s]
Train steps ... : 82%|βββββββββ | 33/40 [00:39<00:06, 1.12it/s]
Train steps ... : 85%|βββββββββ | 34/40 [00:40<00:05, 1.17it/s]
Train steps ... : 88%|βββββββββ | 35/40 [00:41<00:04, 1.20it/s]
Train steps ... : 90%|βββββββββ | 36/40 [00:41<00:02, 1.39it/s]
Train steps ... : 92%|ββββββββββ| 37/40 [00:44<00:04, 1.40s/it]
Train steps ... : 95%|ββββββββββ| 38/40 [00:45<00:02, 1.22s/it]
Train steps ... : 98%|ββββββββββ| 39/40 [00:46<00:01, 1.09s/it]
Train steps ... : 100%|ββββββββββ| 40/40 [00:46<00:00, 1.00it/s]05/31/2024 02:24:35 - INFO - accelerate.accelerator - Saving current state to /home/hzchng/ai-voicebot/lib/whisper/output/checkpoint-40-epoch-4 |
|
05/31/2024 02:24:35 - WARNING - accelerate.utils.other - Removed shared tensor {'proj_out.weight'} while saving. This should be OK, but check by verifying that you don't receive any warning while reloading |
|
05/31/2024 02:24:45 - INFO - accelerate.checkpointing - Model weights saved in /home/hzchng/ai-voicebot/lib/whisper/output/checkpoint-40-epoch-4/model.safetensors |
|
05/31/2024 02:24:45 - WARNING - accelerate.utils.other - Removed shared tensor {'proj_out.weight'} while saving. This should be OK, but check by verifying that you don't receive any warning while reloading |
|
05/31/2024 02:24:57 - INFO - accelerate.checkpointing - Model weights saved in /home/hzchng/ai-voicebot/lib/whisper/output/checkpoint-40-epoch-4/model_1.safetensors |
|
05/31/2024 02:25:03 - INFO - accelerate.checkpointing - Optimizer state saved in /home/hzchng/ai-voicebot/lib/whisper/output/checkpoint-40-epoch-4/optimizer.bin |
|
05/31/2024 02:25:03 - INFO - accelerate.checkpointing - Scheduler state saved in /home/hzchng/ai-voicebot/lib/whisper/output/checkpoint-40-epoch-4/scheduler.bin |
|
05/31/2024 02:25:03 - INFO - accelerate.checkpointing - Sampler state for dataloader 0 saved in /home/hzchng/ai-voicebot/lib/whisper/output/checkpoint-40-epoch-4/sampler.bin |
|
05/31/2024 02:25:03 - INFO - accelerate.checkpointing - Sampler state for dataloader 1 saved in /home/hzchng/ai-voicebot/lib/whisper/output/checkpoint-40-epoch-4/sampler_1.bin |
|
05/31/2024 02:25:03 - INFO - accelerate.checkpointing - Sampler state for dataloader 2 saved in /home/hzchng/ai-voicebot/lib/whisper/output/checkpoint-40-epoch-4/sampler_2.bin |
|
05/31/2024 02:25:03 - INFO - accelerate.checkpointing - Sampler state for dataloader 3 saved in /home/hzchng/ai-voicebot/lib/whisper/output/checkpoint-40-epoch-4/sampler_3.bin |
|
05/31/2024 02:25:03 - INFO - accelerate.checkpointing - Sampler state for dataloader 4 saved in /home/hzchng/ai-voicebot/lib/whisper/output/checkpoint-40-epoch-4/sampler_4.bin |
|
05/31/2024 02:25:03 - INFO - accelerate.checkpointing - Random states saved in /home/hzchng/ai-voicebot/lib/whisper/output/checkpoint-40-epoch-4/random_states_0.pkl |
|
|
|
optimizer.bin: 0%| | 0.00/962M [00:00<?, ?B/s][A |
|
|
|
model.safetensors: 0%| | 0.00/1.71G [00:00<?, ?B/s][A[A |
|
|
|
|
|
Upload 2 LFS files: 0%| | 0/2 [00:00<?, ?it/s][A[A[A |
|
optimizer.bin: 0%| | 1.38M/962M [00:00<01:11, 13.5MB/s][A |
|
|
|
model.safetensors: 0%| | 1.38M/1.71G [00:00<02:15, 12.7MB/s][A[A |
|
|
|
model.safetensors: 0%| | 2.65M/1.71G [00:00<05:23, 5.28MB/s][A[A |
|
optimizer.bin: 0%| | 2.74M/962M [00:00<03:51, 4.15MB/s][A |
|
|
|
model.safetensors: 0%| | 3.39M/1.71G [00:00<06:55, 4.11MB/s][A[A |
|
|
|
model.safetensors: 0%| | 3.96M/1.71G [00:00<06:49, 4.17MB/s][A[A |
|
|
|
model.safetensors: 0%| | 4.46M/1.71G [00:01<07:38, 3.72MB/s][A[A |
|
optimizer.bin: 0%| | 3.46M/962M [00:01<05:52, 2.72MB/s][A |
|
|
|
model.safetensors: 0%| | 4.88M/1.71G [00:01<07:47, 3.65MB/s][A[A |
|
optimizer.bin: 1%| | 5.37M/962M [00:01<03:23, 4.70MB/s][A |
|
|
|
model.safetensors: 0%| | 5.59M/1.71G [00:01<06:35, 4.32MB/s][A[A |
|
optimizer.bin: 1%| | 6.54M/962M [00:01<02:49, 5.63MB/s][A |
|
|
|
model.safetensors: 0%| | 6.55M/1.71G [00:01<05:15, 5.41MB/s][A[A |
|
optimizer.bin: 1%| | 8.16M/962M [00:01<02:14, 7.12MB/s][A |
|
|
|
model.safetensors: 0%| | 7.78M/1.71G [00:01<04:06, 6.91MB/s][A[A |
|
optimizer.bin: 1%| | 10.3M/962M [00:01<01:40, 9.51MB/s][A |
|
|
|
model.safetensors: 1%| | 10.6M/1.71G [00:01<02:34, 11.0MB/s][A[A |
|
optimizer.bin: 1%|β | 13.2M/962M [00:01<01:11, 13.3MB/s][A |
|
|
|
model.safetensors: 1%| | 15.7M/1.71G [00:01<01:26, 19.7MB/s][A[A |
|
|
|
model.safetensors: 1%| | 17.8M/1.71G [00:02<03:22, 8.35MB/s][A[A |
|
|
|
model.safetensors: 1%| | 19.9M/1.71G [00:02<02:56, 9.57MB/s][A[A |
|
optimizer.bin: 2%|β | 16.0M/962M [00:02<02:42, 5.84MB/s][A |
|
optimizer.bin: 2%|β | 20.4M/962M [00:02<01:44, 8.97MB/s][A |
|
optimizer.bin: 2%|β | 23.6M/962M [00:02<01:24, 11.2MB/s][A |
|
optimizer.bin: 3%|β | 25.4M/962M [00:03<01:20, 11.6MB/s][A |
|
|
|
model.safetensors: 1%|β | 21.5M/1.71G [00:03<05:55, 4.75MB/s][A[A |
|
|
|
model.safetensors: 1%|β | 22.6M/1.71G [00:03<05:17, 5.31MB/s][A[A |
|
optimizer.bin: 3%|β | 27.1M/962M [00:03<02:16, 6.87MB/s][A |
|
|
|
model.safetensors: 1%|β | 23.7M/1.71G [00:03<04:54, 5.73MB/s][A[A |
|
optimizer.bin: 3%|β | 28.4M/962M [00:03<02:15, 6.88MB/s][A |
|
|
|
model.safetensors: 1%|β | 25.0M/1.71G [00:03<04:35, 6.12MB/s][A[A |
|
|
|
model.safetensors: 2%|β | 26.9M/1.71G [00:03<03:31, 7.97MB/s][A[A |
|
optimizer.bin: 3%|β | 29.5M/962M [00:03<02:10, 7.15MB/s][A |
|
|
|
model.safetensors: 2%|β | 30.5M/1.71G [00:04<02:16, 12.3MB/s][A[A |
|
optimizer.bin: 3%|β | 31.7M/962M [00:04<01:43, 9.01MB/s][A |
|
|
|
model.safetensors: 2%|β | 32.2M/1.71G [00:04<04:49, 5.80MB/s][A[A |
|
optimizer.bin: 3%|β | 33.0M/962M [00:04<03:37, 4.28MB/s][A |
|
|
|
model.safetensors: 2%|β | 35.7M/1.71G [00:04<03:08, 8.87MB/s][A[A |
|
optimizer.bin: 4%|β | 37.5M/962M [00:05<01:51, 8.31MB/s][A |
|
optimizer.bin: 4%|β | 39.5M/962M [00:05<01:42, 9.04MB/s][A |
|
|
|
model.safetensors: 2%|β | 38.0M/1.71G [00:05<05:40, 4.91MB/s][A[A |
|
optimizer.bin: 4%|β | 43.1M/962M [00:06<02:21, 6.49MB/s][A |
|
|
|
model.safetensors: 2%|β | 39.4M/1.71G [00:06<05:07, 5.45MB/s][A[A |
|
optimizer.bin: 5%|β | 44.4M/962M [00:06<02:18, 6.64MB/s][A |
|
|
|
model.safetensors: 2%|β | 40.7M/1.71G [00:06<04:38, 6.01MB/s][A[A |
|
optimizer.bin: 5%|β | 45.6M/962M [00:06<02:08, 7.13MB/s][A |
|
|
|
model.safetensors: 2%|β | 42.2M/1.71G [00:06<04:11, 6.65MB/s][A[A |
|
optimizer.bin: 5%|β | 46.8M/962M [00:06<02:01, 7.56MB/s][A |
|
|
|
model.safetensors: 3%|β | 45.5M/1.71G [00:06<02:47, 9.96MB/s][A[A |
|
|
|
model.safetensors: 3%|β | 48.0M/1.71G [00:07<04:36, 6.01MB/s][A[A |
|
optimizer.bin: 5%|β | 48.0M/962M [00:07<04:06, 3.71MB/s][A |
|
optimizer.bin: 6%|β | 54.5M/962M [00:07<01:45, 8.62MB/s][A |
|
|
|
model.safetensors: 4%|β | 64.0M/1.71G [00:08<02:21, 11.6MB/s][A[A |
|
optimizer.bin: 6%|β | 58.3M/962M [00:08<02:11, 6.86MB/s][A |
|
|
|
model.safetensors: 4%|β | 69.5M/1.71G [00:08<01:53, 14.5MB/s][A[A |
|
optimizer.bin: 6%|β | 59.8M/962M [00:08<02:13, 6.76MB/s][A |
|
optimizer.bin: 6%|β | 61.3M/962M [00:08<02:05, 7.16MB/s][A |
|
|
|
model.safetensors: 4%|β | 71.8M/1.71G [00:09<03:27, 7.92MB/s][A[A |
|
|
|
model.safetensors: 4%|β | 73.4M/1.71G [00:09<03:15, 8.36MB/s][A[A |
|
optimizer.bin: 7%|β | 64.0M/962M [00:09<03:10, 4.72MB/s][A |
|
|
|
model.safetensors: 4%|β | 76.0M/1.71G [00:09<02:52, 9.46MB/s][A[A |
|
optimizer.bin: 7%|β | 69.5M/962M [00:09<01:47, 8.29MB/s][A |
|
|
|
model.safetensors: 5%|β | 78.3M/1.71G [00:09<02:33, 10.7MB/s][A[A |
|
optimizer.bin: 8%|β | 73.1M/962M [00:09<01:26, 10.2MB/s][A |
|
|
|
model.safetensors: 5%|β | 80.0M/1.71G [00:10<04:13, 6.43MB/s][A[A |
|
|
|
model.safetensors: 5%|β | 85.2M/1.71G [00:10<02:39, 10.2MB/s][A[A |
|
optimizer.bin: 8%|β | 75.3M/962M [00:10<02:16, 6.50MB/s][A |
|
optimizer.bin: 8%|β | 76.7M/962M [00:10<02:21, 6.27MB/s][A |
|
optimizer.bin: 8%|β | 78.6M/962M [00:11<02:00, 7.33MB/s][A |
|
|
|
model.safetensors: 5%|β | 87.2M/1.71G [00:11<04:49, 5.62MB/s][A[A |
|
|
|
model.safetensors: 5%|β | 88.7M/1.71G [00:11<04:23, 6.15MB/s][A[A |
|
|
|
model.safetensors: 5%|β | 90.1M/1.71G [00:12<04:09, 6.49MB/s][A[A |
|
optimizer.bin: 8%|β | 80.0M/962M [00:12<03:36, 4.08MB/s][A |
|
|
|
model.safetensors: 5%|β | 92.5M/1.71G [00:12<03:16, 8.23MB/s][A[A |
|
optimizer.bin: 9%|β | 85.7M/962M [00:12<01:50, 7.93MB/s][A |
|
|
|
model.safetensors: 6%|β | 94.5M/1.71G [00:12<02:47, 9.66MB/s][A[A |
|
optimizer.bin: 9%|β | 89.5M/962M [00:12<01:26, 10.1MB/s][A |
|
|
|
model.safetensors: 6%|β | 96.1M/1.71G [00:13<05:28, 4.92MB/s][A[A |
|
optimizer.bin: 10%|β | 91.4M/962M [00:13<02:18, 6.30MB/s][A |
|
optimizer.bin: 10%|β | 92.9M/962M [00:13<02:21, 6.14MB/s][A |
|
optimizer.bin: 10%|β | 94.7M/962M [00:13<02:01, 7.15MB/s][A |
|
|
|
model.safetensors: 7%|β | 112M/1.71G [00:13<02:00, 13.3MB/s] [A[A |
|
optimizer.bin: 10%|β | 96.0M/962M [00:14<03:39, 3.94MB/s][A |
|
|
|
model.safetensors: 7%|β | 119M/1.71G [00:14<02:43, 9.75MB/s][A[A |
|
|
|
model.safetensors: 7%|β | 120M/1.71G [00:14<02:46, 9.54MB/s][A[A |
|
optimizer.bin: 12%|ββ | 112M/962M [00:15<01:13, 11.6MB/s] [A |
|
|
|
model.safetensors: 7%|β | 123M/1.71G [00:15<02:36, 10.2MB/s][A[A |
|
optimizer.bin: 12%|ββ | 120M/962M [00:15<00:53, 15.8MB/s][A |
|
|
|
model.safetensors: 7%|β | 127M/1.71G [00:15<02:10, 12.2MB/s][A[A |
|
|
|
model.safetensors: 7%|β | 128M/1.71G [00:15<03:32, 7.46MB/s][A[A |
|
optimizer.bin: 13%|ββ | 123M/962M [00:15<01:19, 10.6MB/s][A |
|
|
|
model.safetensors: 8%|β | 135M/1.71G [00:16<02:06, 12.4MB/s][A[A |
|
optimizer.bin: 13%|ββ | 125M/962M [00:16<01:24, 9.86MB/s][A |
|
optimizer.bin: 13%|ββ | 126M/962M [00:16<01:27, 9.57MB/s][A |
|
|
|
model.safetensors: 8%|β | 137M/1.71G [00:17<04:04, 6.43MB/s][A[A |
|
|
|
model.safetensors: 8%|β | 139M/1.71G [00:17<03:36, 7.28MB/s][A[A |
|
optimizer.bin: 13%|ββ | 128M/962M [00:17<02:27, 5.66MB/s][A |
|
|
|
model.safetensors: 8%|β | 141M/1.71G [00:17<03:10, 8.26MB/s][A[A |
|
|
|
model.safetensors: 8%|β | 143M/1.71G [00:17<02:47, 9.37MB/s][A[A |
|
optimizer.bin: 15%|ββ | 144M/962M [00:18<01:05, 12.4MB/s][A |
|
optimizer.bin: 16%|ββ | 152M/962M [00:18<00:47, 16.9MB/s][A |
|
|
|
model.safetensors: 8%|β | 145M/1.71G [00:18<04:30, 5.79MB/s][A[A |
|
|
|
model.safetensors: 9%|β | 160M/1.71G [00:18<01:59, 12.9MB/s][A[A |
|
optimizer.bin: 16%|ββ | 155M/962M [00:18<01:14, 10.8MB/s][A |
|
optimizer.bin: 16%|ββ | 157M/962M [00:19<01:15, 10.7MB/s][A |
|
optimizer.bin: 17%|ββ | 159M/962M [00:19<01:15, 10.6MB/s][A |
|
|
|
model.safetensors: 10%|β | 176M/1.71G [00:19<01:26, 17.7MB/s][A[A |
|
|
|
model.safetensors: 11%|β | 192M/1.71G [00:20<01:12, 21.0MB/s][A[A |
|
optimizer.bin: 17%|ββ | 161M/962M [00:20<02:11, 6.11MB/s][A |
|
optimizer.bin: 17%|ββ | 164M/962M [00:20<01:39, 8.00MB/s][A |
|
optimizer.bin: 18%|ββ | 170M/962M [00:21<01:34, 8.39MB/s][A |
|
|
|
model.safetensors: 12%|ββ | 199M/1.71G [00:21<01:46, 14.2MB/s][A[A |
|
|
|
model.safetensors: 12%|ββ | 201M/1.71G [00:21<01:49, 13.8MB/s][A[A |
|
|
|
model.safetensors: 12%|ββ | 202M/1.71G [00:21<01:49, 13.8MB/s][A[A |
|
|
|
model.safetensors: 12%|ββ | 204M/1.71G [00:21<01:48, 13.8MB/s][A[A |
|
|
|
model.safetensors: 12%|ββ | 207M/1.71G [00:21<01:37, 15.5MB/s][A[A |
|
optimizer.bin: 18%|ββ | 172M/962M [00:22<02:46, 4.76MB/s][A |
|
optimizer.bin: 18%|ββ | 173M/962M [00:22<02:38, 4.97MB/s][A |
|
optimizer.bin: 18%|ββ | 175M/962M [00:22<02:13, 5.91MB/s][A |
|
|
|
model.safetensors: 12%|ββ | 209M/1.71G [00:22<03:42, 6.75MB/s][A[A |
|
|
|
model.safetensors: 13%|ββ | 224M/1.71G [00:23<01:53, 13.1MB/s][A[A |
|
|
|
model.safetensors: 14%|ββ | 240M/1.71G [00:23<01:17, 19.0MB/s][A[A |
|
optimizer.bin: 18%|ββ | 176M/962M [00:23<04:11, 3.13MB/s][A |
|
|
|
model.safetensors: 15%|ββ | 256M/1.71G [00:24<01:06, 21.7MB/s][A[A |
|
optimizer.bin: 20%|ββ | 192M/962M [00:24<01:19, 9.69MB/s][A |
|
optimizer.bin: 22%|βββ | 208M/962M [00:25<00:57, 13.0MB/s][A |
|
|
|
model.safetensors: 15%|ββ | 262M/1.71G [00:25<01:34, 15.4MB/s][A[A |
|
|
|
model.safetensors: 15%|ββ | 264M/1.71G [00:25<01:43, 14.0MB/s][A[A |
|
optimizer.bin: 23%|βββ | 224M/962M [00:25<00:41, 17.8MB/s][A |
|
|
|
model.safetensors: 16%|ββ | 265M/1.71G [00:25<01:45, 13.7MB/s][A[A |
|
|
|
model.safetensors: 16%|ββ | 269M/1.71G [00:25<01:40, 14.4MB/s][A[A |
|
|
|
model.safetensors: 16%|ββ | 271M/1.71G [00:26<01:34, 15.3MB/s][A[A |
|
optimizer.bin: 25%|βββ | 240M/962M [00:26<00:35, 20.2MB/s][A |
|
|
|
model.safetensors: 16%|ββ | 273M/1.71G [00:26<02:49, 8.49MB/s][A[A |
|
optimizer.bin: 27%|βββ | 256M/962M [00:26<00:32, 22.0MB/s][A |
|
|
|
model.safetensors: 17%|ββ | 288M/1.71G [00:27<01:35, 14.9MB/s][A[A |
|
optimizer.bin: 28%|βββ | 272M/962M [00:27<00:29, 23.3MB/s][A |
|
optimizer.bin: 29%|βββ | 281M/962M [00:27<00:25, 27.2MB/s][A |
|
|
|
model.safetensors: 17%|ββ | 295M/1.71G [00:28<02:16, 10.3MB/s][A[A |
|
|
|
model.safetensors: 17%|ββ | 297M/1.71G [00:28<02:17, 10.3MB/s][A[A |
|
optimizer.bin: 30%|βββ | 285M/962M [00:28<00:43, 15.6MB/s][A |
|
|
|
model.safetensors: 18%|ββ | 300M/1.71G [00:28<02:05, 11.2MB/s][A[A |
|
optimizer.bin: 30%|βββ | 288M/962M [00:28<00:43, 15.4MB/s][A |
|
|
|
model.safetensors: 18%|ββ | 303M/1.71G [00:29<01:51, 12.6MB/s][A[A |
|
|
|
model.safetensors: 18%|ββ | 305M/1.71G [00:29<03:05, 7.58MB/s][A[A |
|
optimizer.bin: 30%|βββ | 290M/962M [00:29<01:09, 9.74MB/s][A |
|
optimizer.bin: 32%|ββββ | 304M/962M [00:30<00:49, 13.2MB/s][A |
|
|
|
model.safetensors: 18%|ββ | 310M/1.71G [00:30<03:24, 6.86MB/s][A[A |
|
optimizer.bin: 33%|ββββ | 313M/962M [00:30<00:36, 17.8MB/s][A |
|
|
|
model.safetensors: 18%|ββ | 312M/1.71G [00:30<03:31, 6.61MB/s][A[A |
|
|
|
model.safetensors: 18%|ββ | 313M/1.71G [00:31<03:24, 6.85MB/s][A[A |
|
|
|
model.safetensors: 18%|ββ | 315M/1.71G [00:31<02:53, 8.07MB/s][A[A |
|
|
|
model.safetensors: 19%|ββ | 319M/1.71G [00:31<02:08, 10.8MB/s][A[A |
|
optimizer.bin: 33%|ββββ | 317M/962M [00:31<01:01, 10.6MB/s][A |
|
optimizer.bin: 33%|ββββ | 319M/962M [00:31<00:59, 10.8MB/s][A |
|
|
|
model.safetensors: 19%|ββ | 320M/1.71G [00:32<03:45, 6.18MB/s][A[A |
|
optimizer.bin: 33%|ββββ | 321M/962M [00:32<01:29, 7.14MB/s][A |
|
optimizer.bin: 34%|ββββ | 329M/962M [00:33<00:56, 11.2MB/s][A |
|
|
|
model.safetensors: 19%|ββ | 327M/1.71G [00:33<03:40, 6.28MB/s][A[A |
|
|
|
model.safetensors: 19%|ββ | 329M/1.71G [00:33<03:33, 6.48MB/s][A[A |
|
|
|
model.safetensors: 19%|ββ | 330M/1.71G [00:33<03:14, 7.11MB/s][A[A |
|
|
|
model.safetensors: 19%|ββ | 332M/1.71G [00:33<02:42, 8.50MB/s][A[A |
|
|
|
model.safetensors: 20%|ββ | 335M/1.71G [00:33<02:11, 10.5MB/s][A[A |
|
optimizer.bin: 34%|ββββ | 332M/962M [00:33<01:20, 7.82MB/s][A |
|
optimizer.bin: 35%|ββββ | 334M/962M [00:34<01:19, 7.91MB/s][A |
|
optimizer.bin: 35%|ββββ | 336M/962M [00:34<01:18, 8.00MB/s][A |
|
|
|
model.safetensors: 20%|ββ | 336M/1.71G [00:34<03:55, 5.83MB/s][A[A |
|
optimizer.bin: 35%|ββββ | 337M/962M [00:35<02:14, 4.66MB/s][A |
|
optimizer.bin: 36%|ββββ | 345M/962M [00:35<01:04, 9.62MB/s][A |
|
|
|
model.safetensors: 20%|ββ | 343M/1.71G [00:35<03:50, 5.93MB/s][A[A |
|
|
|
model.safetensors: 20%|ββ | 345M/1.71G [00:35<03:41, 6.17MB/s][A[A |
|
|
|
model.safetensors: 20%|ββ | 346M/1.71G [00:35<03:18, 6.89MB/s][A[A |
|
|
|
model.safetensors: 20%|ββ | 348M/1.71G [00:35<02:51, 7.94MB/s][A[A |
|
|
|
model.safetensors: 21%|ββ | 351M/1.71G [00:36<02:08, 10.6MB/s][A[A |
|
optimizer.bin: 36%|ββββ | 348M/962M [00:36<01:30, 6.79MB/s][A |
|
optimizer.bin: 36%|ββββ | 350M/962M [00:36<01:27, 7.01MB/s][A |
|
optimizer.bin: 36%|ββββ | 351M/962M [00:36<01:20, 7.63MB/s][A |
|
|
|
model.safetensors: 21%|ββ | 353M/1.71G [00:36<03:46, 6.01MB/s][A[A |
|
|
|
model.safetensors: 21%|βββ | 368M/1.71G [00:37<01:35, 14.1MB/s][A[A |
|
optimizer.bin: 37%|ββββ | 353M/962M [00:37<02:19, 4.38MB/s][A |
|
optimizer.bin: 37%|ββββ | 361M/962M [00:37<01:02, 9.65MB/s][A |
|
|
|
model.safetensors: 22%|βββ | 375M/1.71G [00:38<02:11, 10.1MB/s][A[A |
|
optimizer.bin: 38%|ββββ | 363M/962M [00:38<01:28, 6.78MB/s][A |
|
|
|
model.safetensors: 22%|βββ | 377M/1.71G [00:38<02:17, 9.69MB/s][A[A |
|
|
|
model.safetensors: 22%|βββ | 379M/1.71G [00:38<02:06, 10.5MB/s][A[A |
|
optimizer.bin: 38%|ββββ | 365M/962M [00:38<01:28, 6.77MB/s][A |
|
|
|
model.safetensors: 22%|βββ | 383M/1.71G [00:38<01:43, 12.8MB/s][A[A |
|
optimizer.bin: 38%|ββββ | 367M/962M [00:39<01:25, 6.99MB/s][A |
|
|
|
model.safetensors: 22%|βββ | 385M/1.71G [00:39<02:52, 7.69MB/s][A[A |
|
optimizer.bin: 38%|ββββ | 368M/962M [00:40<02:24, 4.11MB/s][A |
|
optimizer.bin: 40%|ββββ | 384M/962M [00:40<00:50, 11.5MB/s][A |
|
|
|
model.safetensors: 23%|βββ | 391M/1.71G [00:40<03:18, 6.66MB/s][A[A |
|
|
|
model.safetensors: 23%|βββ | 393M/1.71G [00:40<03:13, 6.82MB/s][A[A |
|
|
|
model.safetensors: 23%|βββ | 394M/1.71G [00:41<03:00, 7.29MB/s][A[A |
|
|
|
model.safetensors: 23%|βββ | 396M/1.71G [00:41<02:36, 8.40MB/s][A[A |
|
optimizer.bin: 42%|βββββ | 400M/962M [00:41<00:34, 16.2MB/s][A |
|
|
|
model.safetensors: 23%|βββ | 400M/1.71G [00:41<01:50, 11.9MB/s][A[A |
|
optimizer.bin: 43%|βββββ | 416M/962M [00:41<00:27, 19.6MB/s][A |
|
optimizer.bin: 44%|βββββ | 425M/962M [00:41<00:22, 24.1MB/s][A |
|
|
|
model.safetensors: 23%|βββ | 402M/1.71G [00:42<03:17, 6.62MB/s][A[A |
|
|
|
model.safetensors: 24%|βββ | 416M/1.71G [00:42<01:28, 14.7MB/s][A[A |
|
optimizer.bin: 45%|βββββ | 429M/962M [00:43<00:40, 13.3MB/s][A |
|
|
|
model.safetensors: 25%|βββ | 432M/1.71G [00:43<01:07, 18.8MB/s][A[A |
|
optimizer.bin: 45%|βββββ | 431M/962M [00:43<00:40, 13.3MB/s][A |
|
optimizer.bin: 45%|βββββ | 434M/962M [00:44<01:03, 8.36MB/s][A |
|
|
|
model.safetensors: 26%|βββ | 439M/1.71G [00:44<01:39, 12.8MB/s][A[A |
|
|
|
model.safetensors: 26%|βββ | 441M/1.71G [00:44<01:42, 12.4MB/s][A[A |
|
|
|
model.safetensors: 26%|βββ | 442M/1.71G [00:44<01:41, 12.5MB/s][A[A |
|
|
|
model.safetensors: 26%|βββ | 444M/1.71G [00:44<01:37, 13.0MB/s][A[A |
|
|
|
model.safetensors: 26%|βββ | 448M/1.71G [00:44<01:21, 15.4MB/s][A[A |
|
optimizer.bin: 47%|βββββ | 448M/962M [00:44<00:39, 13.0MB/s][A |
|
optimizer.bin: 47%|βββββ | 457M/962M [00:44<00:29, 17.4MB/s][A |
|
|
|
model.safetensors: 26%|βββ | 450M/1.71G [00:45<02:29, 8.43MB/s][A[A |
|
optimizer.bin: 48%|βββββ | 460M/962M [00:45<00:48, 10.5MB/s][A |
|
|
|
model.safetensors: 27%|βββ | 464M/1.71G [00:46<01:25, 14.6MB/s][A[A |
|
optimizer.bin: 48%|βββββ | 462M/962M [00:46<00:46, 10.7MB/s][A |
|
optimizer.bin: 48%|βββββ | 465M/962M [00:47<01:10, 7.04MB/s][A |
|
optimizer.bin: 49%|βββββ | 473M/962M [00:47<00:41, 11.7MB/s][A |
|
|
|
model.safetensors: 28%|βββ | 471M/1.71G [00:47<02:00, 10.3MB/s][A[A |
|
|
|
model.safetensors: 28%|βββ | 473M/1.71G [00:47<02:03, 10.1MB/s][A[A |
|
|
|
model.safetensors: 28%|βββ | 474M/1.71G [00:47<01:57, 10.5MB/s][A[A |
|
|
|
model.safetensors: 28%|βββ | 476M/1.71G [00:47<01:51, 11.0MB/s][A[A |
|
|
|
model.safetensors: 28%|βββ | 479M/1.71G [00:47<01:36, 12.8MB/s][A[A |
|
optimizer.bin: 49%|βββββ | 476M/962M [00:48<00:58, 8.27MB/s][A |
|
optimizer.bin: 50%|βββββ | 478M/962M [00:48<01:04, 7.55MB/s][A |
|
|
|
model.safetensors: 28%|βββ | 481M/1.71G [00:48<02:55, 7.01MB/s][A[A |
|
optimizer.bin: 50%|βββββ | 480M/962M [00:49<01:34, 5.13MB/s][A |
|
|
|
model.safetensors: 28%|βββ | 487M/1.71G [00:49<03:08, 6.51MB/s][A[A |
|
optimizer.bin: 51%|βββββ | 489M/962M [00:49<00:47, 9.88MB/s][A |
|
|
|
model.safetensors: 29%|βββ | 488M/1.71G [00:49<03:05, 6.59MB/s][A[A |
|
|
|
model.safetensors: 29%|βββ | 491M/1.71G [00:49<02:39, 7.68MB/s][A[A |
|
|
|
model.safetensors: 29%|βββ | 492M/1.71G [00:49<02:20, 8.67MB/s][A[A |
|
|
|
model.safetensors: 29%|βββ | 495M/1.71G [00:50<01:52, 10.8MB/s][A[A |
|
optimizer.bin: 51%|βββββ | 492M/962M [00:50<01:11, 6.57MB/s][A |
|
|
|
model.safetensors: 29%|βββ | 497M/1.71G [00:50<03:20, 6.06MB/s][A[A |
|
optimizer.bin: 51%|ββββββ | 494M/962M [00:50<01:07, 6.97MB/s][A |
|
|
|
model.safetensors: 30%|βββ | 512M/1.71G [00:51<01:21, 14.7MB/s][A[A |
|
optimizer.bin: 52%|ββββββ | 496M/962M [00:51<01:37, 4.79MB/s][A |
|
|
|
model.safetensors: 31%|βββ | 528M/1.71G [00:51<01:00, 19.7MB/s][A[A |
|
optimizer.bin: 53%|ββββββ | 512M/962M [00:52<00:39, 11.3MB/s][A |
|
|
|
model.safetensors: 32%|ββββ | 544M/1.71G [00:52<00:53, 21.8MB/s][A[A |
|
|
|
model.safetensors: 33%|ββββ | 560M/1.71G [00:52<00:44, 25.9MB/s][A[A |
|
optimizer.bin: 55%|ββββββ | 528M/962M [00:52<00:27, 15.8MB/s][A |
|
optimizer.bin: 57%|ββββββ | 544M/962M [00:53<00:21, 19.7MB/s][A |
|
optimizer.bin: 58%|ββββββ | 560M/962M [00:54<00:18, 22.0MB/s][A |
|
|
|
model.safetensors: 33%|ββββ | 567M/1.71G [00:54<01:11, 16.1MB/s][A[A |
|
|
|
model.safetensors: 33%|ββββ | 569M/1.71G [00:54<01:12, 15.8MB/s][A[A |
|
|
|
model.safetensors: 33%|ββββ | 572M/1.71G [00:54<01:11, 15.8MB/s][A[A |
|
|
|
model.safetensors: 34%|ββββ | 575M/1.71G [00:54<01:08, 16.7MB/s][A[A |
|
optimizer.bin: 59%|ββββββ | 570M/962M [00:54<00:21, 17.8MB/s][A |
|
optimizer.bin: 59%|ββββββ | 573M/962M [00:55<00:24, 16.2MB/s][A |
|
|
|
model.safetensors: 34%|ββββ | 577M/1.71G [00:55<01:54, 9.89MB/s][A[A |
|
optimizer.bin: 60%|ββββββ | 574M/962M [00:55<00:25, 15.3MB/s][A |
|
|
|
model.safetensors: 35%|ββββ | 592M/1.71G [00:55<01:10, 15.8MB/s][A[A |
|
|
|
model.safetensors: 36%|ββββ | 608M/1.71G [00:56<00:55, 19.9MB/s][A[A |
|
optimizer.bin: 60%|ββββββ | 576M/962M [00:56<00:43, 8.96MB/s][A |
|
optimizer.bin: 62%|βββββββ | 592M/962M [00:56<00:25, 14.4MB/s][A |
|
optimizer.bin: 63%|βββββββ | 608M/962M [00:57<00:19, 18.4MB/s][A |
|
|
|
model.safetensors: 36%|ββββ | 624M/1.71G [00:57<01:06, 16.5MB/s][A[A |
|
optimizer.bin: 65%|βββββββ | 624M/962M [00:57<00:15, 22.4MB/s][A |
|
optimizer.bin: 66%|βββββββ | 633M/962M [00:58<00:12, 26.6MB/s][A |
|
|
|
model.safetensors: 37%|ββββ | 640M/1.71G [00:58<01:00, 17.6MB/s][A[A |
|
|
|
model.safetensors: 38%|ββββ | 656M/1.71G [00:58<00:52, 20.0MB/s][A[A |
|
optimizer.bin: 66%|βββββββ | 637M/962M [00:59<00:22, 14.3MB/s][A |
|
|
|
model.safetensors: 39%|ββββ | 672M/1.71G [00:59<00:46, 22.5MB/s][A[A |
|
optimizer.bin: 66%|βββββββ | 639M/962M [00:59<00:24, 13.2MB/s][A |
|
|
|
model.safetensors: 40%|ββββ | 688M/1.71G [01:00<00:41, 24.8MB/s][A[A |
|
|
|
model.safetensors: 41%|ββββ | 704M/1.71G [01:00<00:38, 26.4MB/s][A[A |
|
optimizer.bin: 67%|βββββββ | 642M/962M [01:01<00:48, 6.67MB/s][A |
|
|
|
model.safetensors: 42%|βββββ | 720M/1.71G [01:01<00:41, 24.1MB/s][A[A |
|
optimizer.bin: 68%|βββββββ | 656M/962M [01:01<00:28, 10.7MB/s][A |
|
|
|
model.safetensors: 43%|βββββ | 736M/1.71G [01:01<00:37, 26.2MB/s][A[A |
|
optimizer.bin: 70%|βββββββ | 672M/962M [01:02<00:19, 15.0MB/s][A |
|
|
|
model.safetensors: 44%|βββββ | 752M/1.71G [01:02<00:36, 26.1MB/s][A[A |
|
optimizer.bin: 71%|ββββββββ | 688M/962M [01:02<00:15, 18.0MB/s][A |
|
|
|
model.safetensors: 45%|βββββ | 768M/1.71G [01:03<00:37, 25.3MB/s][A[A |
|
optimizer.bin: 73%|ββββββββ | 704M/962M [01:03<00:12, 20.6MB/s][A |
|
|
|
model.safetensors: 46%|βββββ | 784M/1.71G [01:03<00:39, 23.2MB/s][A[A |
|
optimizer.bin: 75%|ββββββββ | 720M/962M [01:04<00:10, 22.6MB/s][A |
|
|
|
model.safetensors: 47%|βββββ | 800M/1.71G [01:04<00:35, 25.7MB/s][A[A |
|
|
|
model.safetensors: 48%|βββββ | 816M/1.71G [01:04<00:33, 26.7MB/s][A[A |
|
|
|
model.safetensors: 49%|βββββ | 832M/1.71G [01:05<00:34, 25.8MB/s][A[A |
|
optimizer.bin: 76%|ββββββββ | 736M/962M [01:05<00:14, 15.6MB/s][A |
|
|
|
model.safetensors: 50%|βββββ | 848M/1.71G [01:06<00:31, 27.8MB/s][A[A |
|
optimizer.bin: 78%|ββββββββ | 752M/962M [01:06<00:11, 18.3MB/s][A |
|
|
|
model.safetensors: 50%|βββββ | 864M/1.71G [01:06<00:29, 28.8MB/s][A[A |
|
|
|
model.safetensors: 51%|βββββ | 872M/1.71G [01:06<00:27, 30.6MB/s][A[A |
|
optimizer.bin: 80%|ββββββββ | 768M/962M [01:06<00:09, 20.9MB/s][A |
|
optimizer.bin: 81%|βββββββββ | 784M/962M [01:07<00:07, 23.7MB/s][A |
|
|
|
model.safetensors: 51%|βββββ | 875M/1.71G [01:07<00:49, 17.0MB/s][A[A |
|
optimizer.bin: 83%|βββββββββ | 800M/962M [01:07<00:06, 24.5MB/s][A |
|
|
|
model.safetensors: 51%|ββββββ | 877M/1.71G [01:08<00:51, 16.1MB/s][A[A |
|
optimizer.bin: 85%|βββββββββ | 816M/962M [01:08<00:05, 25.4MB/s][A |
|
|
|
model.safetensors: 51%|ββββββ | 880M/1.71G [01:08<01:20, 10.3MB/s][A[A |
|
optimizer.bin: 86%|βββββββββ | 832M/962M [01:08<00:04, 26.6MB/s][A |
|
|
|
model.safetensors: 52%|ββββββ | 896M/1.71G [01:09<00:52, 15.5MB/s][A[A |
|
optimizer.bin: 88%|βββββββββ | 848M/962M [01:09<00:04, 28.0MB/s][A |
|
optimizer.bin: 90%|βββββββββ | 864M/962M [01:09<00:03, 29.6MB/s][A |
|
|
|
model.safetensors: 53%|ββββββ | 912M/1.71G [01:09<00:39, 20.0MB/s][A[A |
|
|
|
model.safetensors: 54%|ββββββ | 928M/1.71G [01:10<00:32, 23.9MB/s][A[A |
|
optimizer.bin: 91%|ββββββββββ| 880M/962M [01:10<00:02, 30.6MB/s][A |
|
|
|
model.safetensors: 55%|ββββββ | 944M/1.71G [01:10<00:28, 26.6MB/s][A[A |
|
optimizer.bin: 93%|ββββββββββ| 896M/962M [01:10<00:02, 30.7MB/s][A |
|
|
|
model.safetensors: 56%|ββββββ | 960M/1.71G [01:11<00:25, 29.1MB/s][A[A |
|
optimizer.bin: 95%|ββββββββββ| 912M/962M [01:11<00:01, 31.6MB/s][A |
|
optimizer.bin: 96%|ββββββββββ| 928M/962M [01:11<00:01, 30.4MB/s][A |
|
optimizer.bin: 97%|ββββββββββ| 937M/962M [01:12<00:00, 34.6MB/s][A |
|
|
|
model.safetensors: 57%|ββββββ | 976M/1.71G [01:12<00:29, 25.2MB/s][A[A |
|
|
|
model.safetensors: 58%|ββββββ | 992M/1.71G [01:12<00:27, 26.5MB/s][A[A |
|
|
|
model.safetensors: 59%|ββββββ | 1.01G/1.71G [01:13<00:26, 26.4MB/s][A[A |
|
optimizer.bin: 98%|ββββββββββ| 942M/962M [01:13<00:01, 17.4MB/s][A |
|
|
|
model.safetensors: 60%|ββββββ | 1.02G/1.71G [01:13<00:25, 27.3MB/s][A[A |
|
optimizer.bin: 98%|ββββββββββ| 945M/962M [01:14<00:01, 11.7MB/s][A |
|
|
|
model.safetensors: 61%|ββββββ | 1.04G/1.71G [01:14<00:23, 28.7MB/s][A[A |
|
|
|
model.safetensors: 62%|βββββββ | 1.06G/1.71G [01:14<00:21, 30.5MB/s][A[A |
|
optimizer.bin: 100%|ββββββββββ| 960M/962M [01:14<00:00, 15.5MB/s][A |
|
|
|
model.safetensors: 63%|βββββββ | 1.07G/1.71G [01:15<00:20, 31.1MB/s][A[A
optimizer.bin: 100%|ββββββββββ| 962M/962M [01:15<00:00, 12.8MB/s] |
|
|
|
|
|
model.safetensors: 64%|βββββββ | 1.09G/1.71G [01:15<00:19, 31.3MB/s][A[A |
|
|
|
model.safetensors: 64%|βββββββ | 1.10G/1.71G [01:16<00:19, 30.8MB/s][A[A |
|
|
|
model.safetensors: 65%|βββββββ | 1.12G/1.71G [01:16<00:19, 31.1MB/s][A[A |
|
|
|
model.safetensors: 66%|βββββββ | 1.14G/1.71G [01:17<00:18, 31.4MB/s][A[A |
|
|
|
model.safetensors: 67%|βββββββ | 1.15G/1.71G [01:18<00:20, 27.5MB/s][A[A |
|
|
|
model.safetensors: 68%|βββββββ | 1.17G/1.71G [01:18<00:19, 28.3MB/s][A[A |
|
|
|
model.safetensors: 69%|βββββββ | 1.18G/1.71G [01:19<00:18, 28.7MB/s][A[A |
|
|
|
model.safetensors: 70%|βββββββ | 1.20G/1.71G [01:19<00:17, 29.2MB/s][A[A |
|
|
|
model.safetensors: 71%|βββββββ | 1.22G/1.71G [01:20<00:16, 29.8MB/s][A[A |
|
|
|
model.safetensors: 72%|ββββββββ | 1.23G/1.71G [01:20<00:15, 31.0MB/s][A[A |
|
|
|
model.safetensors: 73%|ββββββββ | 1.25G/1.71G [01:21<00:14, 31.0MB/s][A[A |
|
|
|
model.safetensors: 74%|ββββββββ | 1.26G/1.71G [01:21<00:14, 30.5MB/s][A[A |
|
|
|
model.safetensors: 75%|ββββββββ | 1.28G/1.71G [01:22<00:13, 30.9MB/s][A[A |
|
|
|
model.safetensors: 76%|ββββββββ | 1.30G/1.71G [01:22<00:13, 30.6MB/s][A[A |
|
|
|
model.safetensors: 77%|ββββββββ | 1.31G/1.71G [01:23<00:12, 31.2MB/s][A[A |
|
|
|
model.safetensors: 78%|ββββββββ | 1.33G/1.71G [01:23<00:12, 31.9MB/s][A[A |
|
|
|
model.safetensors: 79%|ββββββββ | 1.34G/1.71G [01:24<00:11, 31.6MB/s][A[A |
|
|
|
model.safetensors: 79%|ββββββββ | 1.36G/1.71G [01:24<00:11, 30.1MB/s][A[A |
|
|
|
model.safetensors: 80%|ββββββββ | 1.38G/1.71G [01:25<00:11, 29.8MB/s][A[A |
|
|
|
model.safetensors: 81%|βββββββββ | 1.39G/1.71G [01:25<00:10, 30.4MB/s][A[A |
|
|
|
model.safetensors: 82%|βββββββββ | 1.41G/1.71G [01:26<00:10, 29.7MB/s][A[A |
|
|
|
model.safetensors: 83%|βββββββββ | 1.42G/1.71G [01:27<00:14, 20.6MB/s][A[A |
|
|
|
model.safetensors: 83%|βββββββββ | 1.42G/1.71G [01:27<00:15, 18.7MB/s][A[A |
|
|
|
model.safetensors: 83%|βββββββββ | 1.42G/1.71G [01:27<00:16, 17.7MB/s][A[A |
|
|
|
model.safetensors: 83%|βββββββββ | 1.42G/1.71G [01:27<00:16, 17.6MB/s][A[A |
|
|
|
model.safetensors: 83%|βββββββββ | 1.42G/1.71G [01:28<00:29, 9.64MB/s][A[A |
|
|
|
model.safetensors: 84%|βββββββββ | 1.44G/1.71G [01:29<00:16, 16.7MB/s][A[A |
|
|
|
model.safetensors: 85%|βββββββββ | 1.46G/1.71G [01:30<00:16, 15.9MB/s][A[A |
|
|
|
model.safetensors: 86%|βββββββββ | 1.47G/1.71G [01:30<00:12, 19.2MB/s][A[A |
|
|
|
model.safetensors: 87%|βββββββββ | 1.49G/1.71G [01:31<00:09, 22.9MB/s][A[A |
|
|
|
model.safetensors: 88%|βββββββββ | 1.50G/1.71G [01:31<00:08, 25.0MB/s][A[A |
|
|
|
model.safetensors: 89%|βββββββββ | 1.52G/1.71G [01:32<00:07, 26.7MB/s][A[A |
|
|
|
model.safetensors: 90%|βββββββββ | 1.54G/1.71G [01:32<00:06, 28.2MB/s][A[A |
|
|
|
model.safetensors: 91%|βββββββββ | 1.55G/1.71G [01:33<00:05, 29.4MB/s][A[A |
|
|
|
model.safetensors: 92%|ββββββββββ| 1.57G/1.71G [01:34<00:05, 27.0MB/s][A[A |
|
|
|
model.safetensors: 93%|ββββββββββ| 1.58G/1.71G [01:34<00:04, 28.1MB/s][A[A |
|
|
|
model.safetensors: 93%|ββββββββββ| 1.60G/1.71G [01:35<00:03, 29.3MB/s][A[A |
|
|
|
model.safetensors: 94%|ββββββββββ| 1.62G/1.71G [01:35<00:03, 29.7MB/s][A[A |
|
|
|
model.safetensors: 95%|ββββββββββ| 1.63G/1.71G [01:36<00:02, 30.4MB/s][A[A |
|
|
|
model.safetensors: 96%|ββββββββββ| 1.65G/1.71G [01:36<00:02, 31.2MB/s][A[A |
|
|
|
model.safetensors: 97%|ββββββββββ| 1.66G/1.71G [01:37<00:01, 30.9MB/s][A[A |
|
|
|
model.safetensors: 98%|ββββββββββ| 1.68G/1.71G [01:37<00:01, 30.0MB/s][A[A |
|
|
|
model.safetensors: 99%|ββββββββββ| 1.69G/1.71G [01:38<00:01, 21.1MB/s][A[A |
|
|
|
model.safetensors: 99%|ββββββββββ| 1.69G/1.71G [01:38<00:01, 19.1MB/s][A[A |
|
|
|
model.safetensors: 99%|ββββββββββ| 1.69G/1.71G [01:39<00:01, 18.2MB/s][A[A |
|
|
|
model.safetensors: 99%|ββββββββββ| 1.69G/1.71G [01:39<00:00, 18.2MB/s][A[A |
|
|
|
model.safetensors: 99%|ββββββββββ| 1.70G/1.71G [01:40<00:01, 9.99MB/s][A[A
model.safetensors: 100%|ββββββββββ| 1.71G/1.71G [01:41<00:00, 16.9MB/s] |
|
|
|
|
|
|
|
Upload 2 LFS files: 50%|βββββ | 1/2 [01:41<01:41, 101.47s/it][A[A[A
Upload 2 LFS files: 100%|ββββββββββ| 2/2 [01:41<00:00, 50.74s/it] |
|
Some non-default generation parameters are set in the model config. These should go into a GenerationConfig file (https://huggingface.co/docs/transformers/generation_strategies#save-a-custom-decoding-strategy-with-your-model) instead. This warning will be raised to an exception in v4.41. |
|
Non-default generation parameters: {'max_length': 448, 'begin_suppress_tokens': [220, 50257]} |
|
Configuration saved in /home/hzchng/ai-voicebot/lib/whisper/output/config.json |
|
Configuration saved in /home/hzchng/ai-voicebot/lib/whisper/output/generation_config.json |
|
Model weights saved in /home/hzchng/ai-voicebot/lib/whisper/output/model.safetensors |
|
|