llama_pile_10m_babylm / added_tokens.json
PatrickHaller's picture
Upload tokenizer
d3e0597 verified
raw
history blame contribute delete
29 Bytes
{
"<|endoftext|>": 16000
}