Tool / Function Calling
#122
by
smcleod
- opened
Is there a working template for tool / function calling with DeepSeek R1?
I've tried a number of templates with Ollama, llama.cpp etc... but haven't been able to get something reliable working.
{{- /* Initial system message with core instructions */ -}}
{{- if .Messages }}
{{- if or .System .Tools }}
<|im_start|>system
{{- if .System }}
{{ .System }}
{{- end }}
{{- if .Tools }}
# Tools and XML Schema
You have access to the following tools. Each tool must be used according to this XML schema:
<tools>
{{- range .Tools }}
{{ .Function }}
{{- end }}
</tools>
## Tool Use Format
1. Think about the approach in <thinking> tags
2. Call tool using XML format, for example:
<tool_name>
<parameter1_name>value1</parameter1_name>
<parameter2_name>value2</parameter2_name>
</tool_name>
3. Process tool response, for example:
<tool_response>result</tool_response>
Examples:
<attempt_completion>
<result>
I have completed the task...
</result>
</attempt_completion>
Always adhere to this format for all tool uses to ensure proper parsing and execution.
{{- end }}
<|im_end|>
{{- end }}
{{- /* Message handling loop */ -}}
{{- range $i, $_ := .Messages }}
{{- $last := eq (len (slice $.Messages $i)) 1 }}
{{- /* User messages */ -}}
{{- if eq .Role "user" }}
<|im_start|>user
{{ .Content }}
<|im_end|>
{{- /* Assistant messages */ -}}
{{- else if eq .Role "assistant" }}
<|im_start|>assistant
{{- if .Content }}
{{ .Content }}
{{- else if .ToolCalls }}
{{- range .ToolCalls }}
<thinking>
[Analysis of current state and next steps]
</thinking>
<{{ .Function.Name }}>
{{- range $key, $value := .Function.Arguments }}
<{{ $key }}>{{ $value }}</{{ $key }}>
{{- end }}
</{{ .Function.Name }}>
{{- end }}
{{- end }}
{{- if not $last }}<|im_end|>{{- end }}
{{- /* Tool response handling */ -}}
{{- else if eq .Role "tool" }}
<|im_start|>user
<tool_response>
{{ .Content }}
</tool_response>
<|im_end|>
{{- end }}
{{- /* Prepare for next assistant response if needed */ -}}
{{- if and (ne .Role "assistant") $last }}
<|im_start|>assistant
{{- end }}
{{- end }}
{{- /* Handle single message case */ -}}
{{- else }}
{{- if .System }}
<|im_start|>system
{{ .System }}
<|im_end|>
{{- end }}
{{- if .Prompt }}
<|im_start|>user
{{ .Prompt }}
<|im_end|>
{{- end }}
<|im_start|>assistant
{{- end }}
{{ .Response }}
{{- if .Response }}<|im_end|>{{- end }}
Deepseek R1 doesnt support tool/function calling
Deepseek R1 doesnt support tool/function calling
It seems to work for me? https://x.com/zackangelo/status/1887543668782850287
Deepseek R1 doesnt support tool/function calling
But there is tools relevant script in chat template: https://hf.co/deepseek-ai/DeepSeek-R1/blob/main/tokenizer_config.json. I think it supports tools using. In fact, they just updated this template two days ago, appended "" at the end. I guess this is the missing part which causes function call not working. I think it should have been fixed now.
maybe u can try this template, which is guessed from current uploadded official one:
"chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set ns = namespace(is_first=false, is_tool=false, is_output_first=true, system_prompt='') %}{%- for message in messages %}{%- if message['role'] == 'system' %}{% set ns.system_prompt = message['content'] %}{%- endif %}{%- endfor %}{{bos_token}}{{ ns.system_prompt }}{% if tools %}\\n\\n# Tools\\n\\nYou can call one or more functions to assist with user queries.\\n\\nHere is the list of available tools:{% for tool in tools %}\\n{{ tool | tojson }}{% endfor %}\\n\\nFor each function call, follow this exact format:\\n<｜tool▁calls▁begin｜><｜tool▁call▁begin｜>function<｜tool▁sep｜>function_name\\n```json\\n{\"param1\": \"value1\", \"param2\": \"value2\"}\\n```<｜tool▁call▁end｜>\\n\\nMultiple tools should be formatted as follows:\\n<｜tool▁calls▁begin｜><｜tool▁call▁begin｜>function<｜tool▁sep｜>function1_name\\n```json\\n{\"param1\": \"value1\"}\\n```<｜tool▁call▁end｜>\\n<｜tool▁call▁begin｜>function<｜tool▁sep｜>function2_name\\n```json\\n{\"param1\": \"value1\"}\\n```<｜tool▁call▁end｜><｜tool▁calls▁end｜><｜end▁of▁sentence｜>{% endif %}{%- for message in messages %}{%- if message['role'] == 'user' %}{%- set ns.is_tool = false -%}{{'<｜User｜>' + message['content']}}{%- endif %}{%- if message['role'] == 'assistant' and message['content'] is none %}{%- set ns.is_tool = false -%}{%- for tool in message['tool_calls']%}{%- if not ns.is_first %}{{'<｜Assistant｜><｜tool▁calls▁begin｜><｜tool▁call▁begin｜>' + tool['type'] + '<｜tool▁sep｜>' + tool['function']['name'] + '\\n' + '```json' + '\\n' + tool['function']['arguments'] + '\\n' + '```' + '<｜tool▁call▁end｜>'}}{%- set ns.is_first = true -%}{%- else %}{{'\\n' + '<｜tool▁call▁begin｜>' + tool['type'] + '<｜tool▁sep｜>' + tool['function']['name'] + '\\n' + '```json' + '\\n' + tool['function']['arguments'] + '\\n' + '```' + '<｜tool▁call▁end｜>'}}{{'<｜tool▁calls▁end｜><｜end▁of▁sentence｜>'}}{%- endif %}{%- endfor %}{%- endif %}{%- if message['role'] == 'assistant' and message['content'] is not none %}{%- if ns.is_tool %}{{'<｜tool▁outputs▁end｜>' + message['content'] + '<｜end▁of▁sentence｜>'}}{%- set ns.is_tool = false -%}{%- else %}{% set content = message['content'] %}{% if '' in content %}{% set content = content.split('')[-1] %}{% endif %}{{'<｜Assistant｜>' + content + '<｜end▁of▁sentence｜>'}}{%- endif %}{%- endif %}{%- if message['role'] == 'tool' %}{%- set ns.is_tool = true -%}{%- if ns.is_output_first %}{{'<｜tool▁outputs▁begin｜><｜tool▁output▁begin｜>' + message['content'] + '<｜tool▁output▁end｜>'}}{%- set ns.is_output_first = false %}{%- else %}{{'\\n<｜tool▁output▁begin｜>' + message['content'] + '<｜tool▁output▁end｜>'}}{%- endif %}{%- endif %}{%- endfor -%}{% if ns.is_tool %}{{'<｜tool▁outputs▁end｜>'}}{% endif %}{% if add_generation_prompt and not ns.is_tool %}{{'<｜Assistant｜>\\n'}}{% endif %}"
Note: u still need to implement the tool_call_parser function for this special tool call chat template