File size: 127 Bytes
d10ecd7
 
 
 
 
1
2
3
4
5
6

from transformers import BertTokenizer

tokenizer = BertTokenizer.from_pretrained("eson/kplug-base-encoder")
print(tokenizer)