lichess_uci_all_elos_8layers / tokenizer_config.json
austindavis's picture
Update tokenizer_config.json
09b5108 verified
raw
history blame contribute delete
191 Bytes
{
"add_bos_token": true,
"bos_token": ";",
"clean_up_tokenization_spaces": true,
"model_max_length": 1000000000000000019884624838656,
"tokenizer_class": "PreTrainedTokenizerFast"
}