System prompt
What is the system prompt for this model?
It's in their paper, just ctrl+f and search "system prompt". It's under section 5.1.
"Always assist with care, respect, and truth. Respond with utmost utility yet securely. Avoid harmful,
unethical, prejudiced, or negative content. Ensure replies promote fairness and positivity."
When I try to add a system prompt it gives an error.
messages = [
{"role": "system", "content": "Always assist with care, respect, and truth. Respond with utmost utility yet securely. Avoid harmful, unethical, prejudiced, or negative content. Ensure replies promote fairness and positivity."},
{"role": "user", "content": """What is the capital of Germany?"""},
]
TemplateError: Conversation roles must alternate user/assistant/user/assistant/...
So, perhaps that changed after the paper was written or was changed in v0.2.
The chat template is defined in Mistral-7B-Instruct-v0.2/blob/main/tokenizer_config.json, by default, it doesn't accept system role. You may need to composite a new chat template to support system role.
When I try to add a system prompt it gives an error.
messages = [
{"role": "system", "content": "Always assist with care, respect, and truth. Respond with utmost utility yet securely. Avoid harmful, unethical, prejudiced, or negative content. Ensure replies promote fairness and positivity."},
{"role": "user", "content": """What is the capital of Germany?"""},
]TemplateError: Conversation roles must alternate user/assistant/user/assistant/...
So, perhaps that changed after the paper was written or was changed in v0.2.
If you send a full code snippet I could try it on my machine, but otherwise you can just use the Llama-2 prompt structure:
<s>[INST] <<SYS>>
{{ system_prompt }}
<</SYS>>
{{ user_msg_1 }} [/INST] {{ model_answer_1 }} </s><s>[INST] {{ user_msg_2 }} [/INST]