tokenizer-arena / README.md
xu song
Update README.md
fab95c3
|
raw
history blame
402 Bytes
---
title: Tokenizer Arena
emoji:
colorFrom: red
colorTo: gray
sdk: gradio
sdk_version: 3.41.2
app_file: app.py
pinned: false
---
Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
## ss
## TODO
- 'MossTokenizer' object has no attribute 'encoder'
- chatglmTokenizer
## 统计
https://huggingface.co/spaces/yenniejun/tokenizers-languages