BSC-LT

/

salamandra-7b-instruct

like 35

Follow

Language Technologies Unit @ Barcelona Supercomputing Center 106

Hello,
I've enhanced the chat_template to allow for a dynamic system prompt. Users can now specify their own system prompt or fall back to the default if none is provided.

Comparision:

Original chat_template

• Example with a custom system prompt:

messages = [
    {"role": "system", "content": "This is custom system prompt"},
    {"role": "user", "content": "What is your name?"},
]
tokenizer.apply_chat_template(messages,  tokenize=False, chat_template=chat_template, add_generation_prompt=True)

• output:

<|im_start|>assistant
I am Salamandra, an AI language model developed at the Barcelona Supercomputing Centre (BSC) by the Language Technologies Unit. My knowledge base was last updated on August 2023. Today Date: 2024-09-30
Soy Salamandra, un modelo lingüístico de IA desarrollado en el Barcelona Supercomputing Centre (BSC) por la Language Technologies Unit. Mi base de conocimientos se actualizó por última vez en agosto de 2023.
Soc Salamandra, un model de llenguatge d'IA desenvolupat al Barcelona Supercomputing Centre (BSC) per la Language Technologies Unit. La meva base de coneixement es va actualitzar per última vegada l'agost de 2023.<|im_end|>
<|im_start|>system
This is custom system prompt<|im_end|>
<|im_start|>user
What is your name?<|im_end|>
<|im_start|>assistant

• Example without a system prompt:

messages = [
    {"role": "user", "content": "What is your name?"}
]
tokenizer.apply_chat_template(messages,  tokenize=False, chat_template=chat_template, add_generation_prompt=True)

• output:

<|im_start|>assistant
I am Salamandra, an AI language model developed at the Barcelona Supercomputing Centre (BSC) by the Language Technologies Unit. My knowledge base was last updated on August 2023. Today Date: 2024-09-30
Soy Salamandra, un modelo lingüístico de IA desarrollado en el Barcelona Supercomputing Centre (BSC) por la Language Technologies Unit. Mi base de conocimientos se actualizó por última vez en agosto de 2023.
Soc Salamandra, un model de llenguatge d'IA desenvolupat al Barcelona Supercomputing Centre (BSC) per la Language Technologies Unit. La meva base de coneixement es va actualitzar per última vegada l'agost de 2023.<|im_end|>
<|im_start|>user
What is your name?<|im_end|>
<|im_start|>assistant

As you can see, in both cases, the system defaults to the pre-set prompt, ignoring the user-provided system prompt in the messages.

Updated chat_template

• messages with system prompt:

messages = [
    {"role": "system", "content": "This is custom system prompt"},
    {"role": "user", "content": "What is your name?"},
]
tokenizer.apply_chat_template(messages,  tokenize=False, chat_template=chat_template, add_generation_prompt=True)

• output:

<|im_start|>system
This is custom system prompt<|im_end|>
<|im_start|>user
What is your name?<|im_end|>
<|im_start|>assistant

• messages without system prompt :

messages = [
    {"role": "user", "content": "What is your name?"}
]
tokenizer.apply_chat_template(messages,  tokenize=False, chat_template=chat_template, add_generation_prompt=True)

• output:

<|im_start|>system
I am Salamandra, an AI language model developed at the Barcelona Supercomputing Centre (BSC) by the Language Technologies Unit. My knowledge base was last updated on August 2023. Today Date: 2024-09-30
Soy Salamandra, un modelo lingüístico de IA desarrollado en el Barcelona Supercomputing Centre (BSC) por la Language Technologies Unit. Mi base de conocimientos se actualizó por última vez en agosto de 2023.
Soc Salamandra, un model de llenguatge d'IA desenvolupat al Barcelona Supercomputing Centre (BSC) per la Language Technologies Unit. La meva base de coneixement es va actualitzar per última vegada l'agost de 2023.<|im_end|>
<|im_start|>user
What is your name?<|im_end|>
<|im_start|>assistant

Summary

With this update:

• If a system prompt is included in messages, it takes precedence.
• If no system prompt is included, the default prompt is used.

This ensures that the system prompt is adaptable to user needs.