README.md

---
base_model: Qwen/Qwen2-1.5B-Instruct
datasets:
- devanshamin/gem-viggo-function-calling
library_name: peft
license: apache-2.0
pipeline_tag: text-generation
tags:
- trl
- sft
- generated_from_trainer
model-index:
- name: Qwen2-1.5B-Instruct-Function-Calling-v1
 results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# Qwen2-1.5B-Instruct-Function-Calling-v1

This model is a fine-tuned version of [Qwen/Qwen2-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2-1.5B-Instruct) on [devanshamin/gem-viggo-function-calling](https://huggingface.co/datasets/devanshamin/gem-viggo-function-calling) dataset.

## Updated Chat Template
> Note: The template supports multiple tools but the model is fine-tuned on a dataset consisting of examples with a single tool.

- The chat template has been added to the [tokenizer_config.json](https://huggingface.co/devanshamin/Qwen2-1.5B-Instruct-Function-Calling-v1/blob/7ee7c020cefdb0101939469de608acc2afa7809e/tokenizer_config.json#L34).
- Supports prompts with and without tools.

```python
chat_template = (
 "{% for message in messages %}"
 "{% if loop.first and messages[0]['role'] != 'system' %}"
 "{% if tools %}"
 "<|im_start|>system\nYou are a helpful assistant with access to the following tools. Use them if required - \n"
 "```json\n{{ tools | tojson }}\n```<|im_end|>\n"
 "{% else %}"
 "<|im_start|>system\nYou are a helpful assistant.\n<|im_end|>\n"
 "{% endif %}"
 "{% endif %}"
 "{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}"
 "{% endfor %}"
 "{% if add_generation_prompt %}"
 "{{ '<|im_start|>assistant\n' }}"
 "{% endif %}"
)
```

## Basic Usage

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Qwen2-1.5B-Instruct-Function-Calling-v1"
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float32, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_id)

def inference(prompt: str) -> str:
 model_inputs = tokenizer([prompt], return_tensors="pt").to('cuda')
 generated_ids = model.generate(model_inputs.input_ids, max_new_tokens=512)
 generated_ids = [output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)]
 response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
 return response

messages = [{"role": "user", "content": "What is the speed of light?"}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
response = inference(prompt)
print(response)
```

## Tool Usage

### Basic

```python
import json
from typing import List, Dict

def get_prompt(user_input: str, tools: List[Dict] | None = None):
 prompt = 'Extract the information from the following - \n{}'.format(user_input)
 messages = [{"role": "user", "content": prompt}]
 prompt = tokenizer.apply_chat_template(
 messages,
 tokenize=False,
 add_generation_prompt=True,
 tools=tools
 )
 return prompt

tool = {
 "type": "function",
 "function": {
 "name": "get_company_info",
 "description": "Correctly extracted company information with all the required parameters with correct types",
 "parameters": {
 "properties": {
 "name": {"title": "Name", "type": "string"},
 "investors": {
 "items": {"type": "string"},
 "title": "Investors",
 "type": "array"
 },
 "valuation": {"title": "Valuation", "type": "string"},
 "source": {"title": "Source", "type": "string"}
 },
 "required": ["investors", "name", "source", "valuation"],
 "type": "object"
 }
 }
}
input_text = "Founded in 2021, Pluto raised $4 million across multiple seed funding rounds, valuing the company at $12 million (pre-money), according to PitchBook. The startup was backed by investors including Switch Ventures, Caffeinated Capital and Maxime Seguineau."
prompt = get_prompt(input_text, tools=[tool])
response = inference(prompt)
print(response)
# ```json
# {
# "name": "get_company_info",
# "arguments": {
# "name": "Pluto",
# "investors": [
# "Switch Ventures",
# "Caffeinated Capital",
# "Maxime Seguineau"
# ],
# "valuation": "$12 million",
# "source": "PitchBook"
# }
# }
# ```
```

### Advanced
```python
import re
from enum import Enum

from pydantic import BaseModel, Field # pip install pydantic
from instructor.function_calls import openai_schema # pip install instructor

# Define functions using pydantic classes
class PaperCategory(str, Enum):
 TYPE_1_DIABETES = 'Type 1 Diabetes'
 TYPE_2_DIABETES = 'Type 2 Diabetes'

class Classification(BaseModel):
 label: PaperCategory = Field(..., description='Provide the most likely category')
 reason: str = Field(..., description='Give a detailed explanation with quotes from the abstract explaining why the paper is related to the chosen label.')

function_definition = openai_schema(Classification).openai_schema
tool = dict(type='function', function=function_definition)
input_text = "1,25-dihydroxyvitamin D(3) (1,25(OH)(2)D(3)), the biologically active form of vitamin D, is widely recognized as a modulator of the immune system as well as a regulator of mineral metabolism. The objective of this study was to determine the effects of vitamin D status and treatment with 1,25(OH)(2)D(3) on diabetes onset in non-obese diabetic (NOD) mice, a murine model of human type I diabetes. We have found that vitamin D-deficiency increases the incidence of diabetes in female mice from 46% (n=13) to 88% (n=8) and from 0% (n=10) to 44% (n=9) in male mice as of 200 days of age when compared to vitamin D-sufficient animals. Addition of 50 ng of 1,25(OH)(2)D(3)/day to the diet prevented disease onset as of 200 days and caused a significant rise in serum calcium levels, regardless of gender or vitamin D status. Our results indicate that vitamin D status is a determining factor of disease susceptibility and oral administration of 1,25(OH)(2)D(3) prevents diabetes onset in NOD mice through 200 days of age."
prompt = get_prompt(input_text, tools=[tool])
output = inference(prompt)
print(output)
# ```json
# {
# "name": "Classification", 
# "arguments": {
# "label": "Type 1 Diabetes", 
# "reason": "The study investigated the effect of vitamin D status and treatment with 1,25(OH)(2)D(3) on diabetes onset in non-obese diabetic (NOD) mice. It also concluded that vitamin D deficiency leads to an increase in diabetes incidence and that the addition of 1,25(OH)(2)D(3) can prevent diabetes onset in NOD mice."
# }
# }
# ```
# Extract JSON string using regex
output = re.search(r'```json\s*(\{.*?\})\s*```', output).group(1)
output = Classification(**json.loads(_output)['arguments'])
print(output)
# Classification(label=<PaperCategory.TYPE_1_DIABETES: 'Type 1 Diabetes'>, reason='The study investigated the effect of vitamin D status and treatment with 1,25(OH)(2)D(3) on diabetes onset in non-obese diabetic (NOD) mice. It also concluded that vitamin D deficiency leads to an increase in diabetes incidence and that the addition of 1,25(OH)(2)D(3) can prevent diabetes onset in NOD mice.')
```

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 10
- training_steps: 200

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:------:|:----:|:---------------:|
| 0.4004 | 0.0101 | 20 | 0.4852 |
| 0.3624 | 0.0201 | 40 | 0.3221 |
| 0.2855 | 0.0302 | 60 | 0.2818 |
| 0.2652 | 0.0402 | 80 | 0.2592 |
| 0.2214 | 0.0503 | 100 | 0.2463 |
| 0.2471 | 0.0603 | 120 | 0.2358 |
| 0.2122 | 0.0704 | 140 | 0.2310 |
| 0.2048 | 0.0804 | 160 | 0.2275 |
| 0.2406 | 0.0905 | 180 | 0.2251 |
| 0.2445 | 0.1006 | 200 | 0.2248 |


### Framework versions

```text
peft==0.11.1
transformers==4.42.3
torch==2.3.1+cu121
datasets==2.20.0
tokenizers==0.19.1
```