line-corporation

/

line-distilbert-base-japanese

like 38

Follow

LINE 34

Hi. Recent versions of transformers library log a warning if tokenizer_class is different from the actual class name.
If there is no special reason, can you make them equal? Thanks.

For reference: https://github.com/huggingface/transformers/blob/v4.44.0/src/transformers/tokenization_utils_base.py#L2398-L2404