deepSP / tokenizer_config.json
salomonsky's picture
Upload 5 files
5937cd0
raw
history blame contribute delete
115 Bytes
{"pad_token": "<|endoftext|>", "special_tokens_map_file": "./special_tokens_map.json", "full_tokenizer_file": null}