tokenizer-arena / requirements.txt
xu-song's picture
update
a173fe5
raw
history blame
60 Bytes
transformers>=4.21.1
sentencepiece
tiktoken
icetk
torch
zhon