QCRI

/

bert-base-multilingual-cased-pos-english

like 39

Follow

Arabic Language Technologies, Qatar Computing Research Institute 22

Version: 4.25.1

I had a similar issue. It runs on version 4.2.0

Thanks a lot for reporting this and @ElMad for the temporary work around. I tried poking around a bit with specific tokenizers and all of them seem to be giving the same error; I'll report this on transformers repo and see if they are able to state why this is happening/how we can update our code.

Will keep this issue open for now and update it when I have some news.

The transformers issue is here: https://github.com/huggingface/transformers/issues/21240

There is a quick temporary fix posted there by someone else that works with the latest transformers version, so check it out while the underlying issue gets resolved.

Its fixed @andres-pb ! The pipeline works best when the model/tokenizer are passed in as explicit keyword arguments (model=model, tokenizer=tokenizer). I've updated the usage code and tested with both the latest transformers and an older version (~4.1).

Feel free to comment/open another issue if you face any problems!