microsoft

/

Phi-3-mini-128k-instruct-onnx

like 185

Follow

Microsoft 6.42k

Hello,

I noticed a discrepancy in the system role option within the chat_template() . Specifically, in the ONNX version of Phi3, the system role is missing in the chat_template(), unlike in the HF version where the "system" role is present. Here are the two versions of the chat_template:

ONNX Chat_template:

"chat_template": "{{ bos_token }}{% for message in messages %}{% if (message['role'] == 'user') %}{{'<|user|>' + '\n' + message['content'] + '<|end|>' + '\n' + '<|assistant|>' + '\n'}}{% elif (message['role'] == 'assistant') %}{{message['content'] + '<|end|>' + '\n'}}{% endif %}{% endfor %}"

Hf Chat_template
"chat_template": "{% for message in messages %}{% if message['role'] == 'system' %}{{'<|system|>\n' + message['content'] + '<|end|>\n'}}{% elif message['role'] == 'user' %}{{'<|user|>\n' + message['content'] + '<|end|>\n'}}{% elif message['role'] == 'assistant' %}{{'<|assistant|>\n' + message['content'] + '<|end|>\n'}}{% endif %}{% endfor %}{% if add_generation_prompt %}{{ '<|assistant|>\n' }}{% else %}{{ eos_token }}{% endif %}"

Could someone help clarify this difference, please?