Add chat_template to tokenizer_config.json
Hi @rafaelwh , thanks for contributing. Did you test the updated config yourself? Also, do I get it right that this is something that should also work with earlier versions of Transformers?
I haven't tried with earlier versions of Transformers.
Here are better discussions on the issue I was having because there was not a chat_template on the tokenizer_confing.json file:
https://discuss.huggingface.co/t/chat-template-is-not-set-throwing-error/104095/5
https://github.com/huggingface/open-r1/issues/85
As they point out, you could also use this in the JSON instead of my pull request, if you prefer:
"chat_template": "{% for message in messages %}\n{% if message['role'] == 'user' %}\n{{ '<|user|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'system' %}\n{{ '<|system|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'assistant' %}\n{{ '<|assistant|>\n' + message['content'] + eos_token }}\n{% endif %}\n{% if loop.last and add_generation_prompt %}\n{{ '<|assistant|>' }}\n{% endif %}\n{% endfor %}",
I tested both my pull request, and the snippet above, and I was able to get a response from the model.
Hi @rafaelwh , thank you for all the info. It's likely that this will also avoid problems for other people. I'm not sure about older versions, but I doubt that it would cause a non-recoverable error in loading the model anyway.
Thank you for contributing!