pko-t5-base / special_tokens_map.json
dennispark's picture
Update tokenizer without extra_ids
6d16c5c
raw
history blame
67 Bytes
{"eos_token": "</s>", "unk_token": "<pad>", "pad_token": "<pad>"}