Spaces:

xu-song
/

tokenizer-arena

Running

App Files Files Community

tokenizer-arena / vocab /gpt2_chinese /README.md

xu-song's picture

update

751936e about 1 year ago

|

237 Bytes

跟 bert-chinese词典一样

https://huggingface.co/ckiplab/gpt2-base-chinese https://huggingface.co/uer/gpt2-chinese-cluecorpussmall

  "tokenizer_class": "BertTokenizerFast",
  "vocab_size": 21128

ss

ss