CySec_Known_Exploit_Analyzer / tokenizer_config.json
Canstralian's picture
Create tokenizer_config.json
e41e675 verified
raw
history blame
128 Bytes
{
"do_lower_case": true,
"vocab_size": 30522,
"special_tokens": {
"pad_token": "[PAD]",
"eos_token": "[EOS]"
}
}