return_offsets_mapping in tokenizer does not meet expectations
#41
by
cyue2436
- opened
Why is the output like this? I think this is not what I expected. Offsets_mapping should be an interval to express the corresponding character position?
I have tried several transformer versions and it doesn't seem to fix the problem.