File size: 5,565 Bytes
905da01 5f8ec52 1c3fedd |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 |
---
license: apache-2.0
language:
- en
base_model:
- codellama/CodeLlama-7b-hf
---
# **TL-CodeLLaMA-2**
TL-CodeLLaMA-2 is a model designed for tool use, built upon CodeLLaMA-7b. It is trained on 1,217 data samples using the *TL-Training* framework and demonstrates effective performance across a variety of tool use tasks. More information can be found in the paper "[TL-Training: A Task-Feature-Based Framework for Training Large Language Models in Tool Use](https://www.arxiv.org/abs/2412.15495)".
# Model Use
## Requirements
To use this model, please make sure to install transformers:
```bash
pip install transformers
```
## Data Orgnization
The data needs to be organized in the following format:
```json
[
{
"role": "System",
"content": "Function:\ndef random_advice():\n \"\"\"\n Returns a random advice slip as a slip object.\n \"\"\"\n\nFunction:\ndef advice_by_id(slip_id:str):\n \"\"\"\n If an advice slip is found with the corresponding {slip_id}, a slip object is returned.\n\n Args:\n slip_id (string): The unique ID of this advice slip.\n \"\"\"\n\nFunction:\ndef search_advice(query:str):\n \"\"\"\n If an advice slip is found, containing the corresponding search term in {query}, an array of slip objects is returned inside a search object.\n\n Args:\n query (string): The search query provided.\n \"\"\"\n\nFunction:\ndef ask_to_user(question:str):\n \"\"\"\n You can ask user for guidance when you think you need more information to handle the task, but you should use this tool as less as you can.\n\n Args:\n question (string): The question you want to ask to user.\n \"\"\"\n\nFunction:\ndef finish(answer:str):\n \"\"\"\n Finish the task and give your answer.\n\n Args:\n answer (string): Your answer for the task.\n \"\"\"\n\n"
},
{
"role": "User",
"content": "Could you give me some advice about 'love'?"
},
{
"role": "Assistant",
"content": "search_advice(query = 'love') "
},
{
"role": "Output",
"content": "..."
}
]
```
## Chat Template
The chat template is:
```jinja
{% for message in messages %}{{message['role'] + ': ' + message['content']}}{% if loop.last %}{% if add_generation_prompt %}{{ '\nAssistant:' }}{% else %}{{ '</s>'}}{% endif %}{% else %}{{ '\n' }}{% endif %}{% endfor %}
```
## Inference
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model_path = "Junjie-Ye/TL-CodeLLaMA-2"
data = [
{
"role": "System",
"content": "Function:\ndef random_advice():\n \"\"\"\n Returns a random advice slip as a slip object.\n \"\"\"\n\nFunction:\ndef advice_by_id(slip_id:str):\n \"\"\"\n If an advice slip is found with the corresponding {slip_id}, a slip object is returned.\n\n Args:\n slip_id (string): The unique ID of this advice slip.\n \"\"\"\n\nFunction:\ndef search_advice(query:str):\n \"\"\"\n If an advice slip is found, containing the corresponding search term in {query}, an array of slip objects is returned inside a search object.\n\n Args:\n query (string): The search query provided.\n \"\"\"\n\nFunction:\ndef ask_to_user(question:str):\n \"\"\"\n You can ask user for guidance when you think you need more information to handle the task, but you should use this tool as less as you can.\n\n Args:\n question (string): The question you want to ask to user.\n \"\"\"\n\nFunction:\ndef finish(answer:str):\n \"\"\"\n Finish the task and give your answer.\n\n Args:\n answer (string): Your answer for the task.\n \"\"\"\n\n"
},
{
"role": "User",
"content": "Could you give me some advice about 'love'?"
}
]
chat_template = "{% for message in messages %}{{message['role'] + ': ' + message['content']}}{% if loop.last %}{% if add_generation_prompt %}{{ '\nAssistant:' }}{% else %}{{ '</s>'}}{% endif %}{% else %}{{ '\n' }}{% endif %}{% endfor %}"
model = AutoModelForCausalLM.from_pretrained(
model_path,
torch_dtype="auto",
device_map="auto",
trust_remote_code=True
).eval()
tokenizer = AutoTokenizer.from_pretrained(model_path,
padding_side="left",
trust_remote_code=True)
if tokenizer.pad_token_id is None:
tokenizer.pad_token_id = tokenizer.eos_token_id
text = tokenizer.apply_chat_template(
data,
tokenize=False,
chat_template=chat_template,
add_generation_prompt=add_generation_prompt
)
model_inputs = tokenizer(
[text], return_tensors="pt", padding=True).to("cuda")
generated_ids = model.generate(
max_new_tokens=1024,
**model_inputs,
)
generated_ids = [
output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)
print(response)
```
# Citation
If you find this model useful in your research, please cite:
```bibtex
@misc{TL-Training,
title={TL-Training: A Task-Feature-Based Framework for Training Large Language Models in Tool Use},
author={Junjie Ye and Yilong Wu and Sixian Li and Yuming Yang and Tao Gui and Qi Zhang and Xuanjing Huang and Peng Wang and Zhongchao Shi and Jianping Fan and Zhengyin Du},
year={2024},
eprint={2412.15495},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2412.15495},
}
```
|