glaive-function-calling-v1
glaive-function-calling-v1 is a 2.7B parameter open source chat model trained on data generated from Glaive’s synthetic data generation platform, which has similar function calling abilities as gpt-3.5 and gpt 4.
The model is capable of having multi-turn conversations and intelligently choosing when to execute a function (provided at the beginning of the conversation as a system prompt) based on the conversation. The model is trained on top of the https://huggingface.co/replit/replit-code-v1-3b model.
Usage:
You can run the model in the following way-
from transformers import AutoModelForCausalLM , AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("glaiveai/glaive-function-calling-v1", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("glaiveai/glaive-function-calling-v1", trust_remote_code=True).half().cuda()
inputs = tokenizer(prompt,return_tensors="pt").to(model.device)
outputs = model.generate(**inputs,do_sample=True,temperature=0.1,top_p=0.95,max_new_tokens=100)
print(tokenizer.decode(outputs[0],skip_special_tokens=True))
This model uses the following prompt format-
SYSTEM: You are an helpful assistant who has access to the following functions to help the user, you can use the functions if needed-
{
"name": "plan_holiday",
"description": "Plan a holiday based on user's interests",
"parameters": {
"type": "object",
"properties": {
"destination": {
"type": "string",
"description": "The destination of the holiday",
},
"duration": {
"type": "integer",
"description": "The duration of the trip in holiday",
},
},
"required": ["destination", "duration"],
},
}
USER: I am thinking of having a 10 day long vacation in Greece, can you help me plan it?
Based on which the model outputs-
ASSISTANT: <functioncall> {"name": "plan_holiday", "arguments": '{
"destination": "Greece",
"duration": 10
}'}
The model precedes all function invocations with <functioncall>
.
The response of the function call should be sent to the model as-
FUNCTION CALL: {"places_to_visit":["Athens","Santorini","Mykonos"]}
The model can do multi-turn conversation in the above format.
We're working on providing an inference server which can act as a drop in replacement to the OpenAI API, you can follow this repo for the server.
Known Limitations:
- While the model does well on function calling use-cases, it doesn't always generalize very well to other chat use-cases. This is intentional as our thesis at Glaive is to provide use-case specialised model that are only used for the given task.
- The model may sometimes hallucinate functions, v2 of the model will be aimed to fix that with a bigger dataset.
- Downloads last month
- 52