Hugging Face
Models
Datasets
Spaces
Posts
Docs
Enterprise
Pricing
Log In
Sign Up
Spaces:
xu-song
/
tokenizer-arena
like
58
Running
App
Files
Files
Community
1
0ce6477
tokenizer-arena
/
vocab
/
gpt_neox_chinese_v1
/
to_v2
2 contributors
History:
2 commits
xu-song
update
d10ecd7
over 1 year ago
20B_tokenizer.1.append.json
Safe
2.75 MB
update
over 1 year ago
20B_tokenizer.1.insert.json
Safe
2.75 MB
update
over 1 year ago
20B_tokenizer.1.json
Safe
2.68 MB
update
over 1 year ago
20B_tokenizer.2.json
Safe
3.65 MB
update
over 1 year ago
20B_tokenizer.tmp.json
Safe
2.47 MB
update
over 1 year ago
README.md
Safe
21 Bytes
update
over 1 year ago
add_token_utils.py
Safe
6.23 kB
update
over 1 year ago
get_unused_id.py
Safe
8.56 kB
update
over 1 year ago
oov.add.txt
Safe
150 kB
update
over 1 year ago
oov.txt
Safe
893 kB
update
over 1 year ago
sort_test.py
Safe
182 Bytes
update
over 1 year ago
test2.py
Safe
1.44 kB
update
over 1 year ago
test_oov.py
Safe
2.34 kB
update
over 1 year ago
test_queue.py
Safe
285 Bytes
update
over 1 year ago
word_count.corpus.remove.jsonl
Safe
2.87 MB
update
over 1 year ago
word_count.corpus.sort_by_count.jsonl
Safe
6.8 MB
update
over 1 year ago
word_count.corpus.txt
Safe
631 kB
update
over 1 year ago
