izumi-lab
/

deberta-v2-small-japanese

Inference Endpoints

Model card Files Files and versions Community

retarfi commited on Nov 2, 2023

Commit

856fa37

·

1 Parent(s): 9a84156

explicitly use slow tokenizer

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -30,7 +30,7 @@ You can use this model for masked language modeling as follows:
 ```python
 from transformers import AutoTokenizer, AutoModelForMaskedLM
-tokenizer = AutoTokenizer.from_pretrained("izumi-lab/deberta-v2-small-japanese")
 model = AutoModelForMaskedLM.from_pretrained("izumi-lab/deberta-v2-small-japanese")
 ...
 ```

 ```python
 from transformers import AutoTokenizer, AutoModelForMaskedLM
+tokenizer = AutoTokenizer.from_pretrained("izumi-lab/deberta-v2-small-japanese", use_fast=False)
 model = AutoModelForMaskedLM.from_pretrained("izumi-lab/deberta-v2-small-japanese")
 ...
 ```