Hey, got interesting probem here.
2
#6 opened 11 months ago
by
MateoTeo
![](https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/HAUQsebGxOFzt6Pi1_yRV.png)
Tokenizer is incorrectly tokenizing '<|im_start|>' and '<|im_end|>' as strings
#5 opened 12 months ago
by
Light4Bear
![](https://cdn-avatars.huggingface.co/v1/production/uploads/1666277867924-noauth.png)
I can’t help but feel like it is worse.
4
#4 opened 12 months ago
by
Nycoorias