|
--- |
|
license: apache-2.0 |
|
language: |
|
- en |
|
base_model: |
|
- Qwen/Qwen2.5-Coder-32B-Instruct |
|
new_version: imsanjoykb/sqlCoder-Qwen2.5-8bit |
|
pipeline_tag: text-generation |
|
library_name: adapter-transformers |
|
tags: |
|
- unsloth, |
|
- pytorch, |
|
- inference-endpoint, |
|
- sql-code-generation, |
|
--- |
|
[🤗 HF Repo](https://huggingface.co/imsanjoykb/sqlCoder-Qwen2.5-8bit) | [♾️ Colab](https://colab.research.google.com/drive/19e-u32GY2y5lsckNuWhBQExvXgVn8ZjG?usp=sharing) |
|
|
|
Introducing the latest fine-tuned version of Qwen2.5-Coder-14B-Instruct, specifically tailored for SQL code generation. Built on the robust 14-billion parameter Qwen2.5-Coder architecture, this model leverages advanced configurations like bfloat16 precision and a custom quantization setup, optimized for efficient 4-bit computation. With a maximum context window of 32K tokens, this model supports extensive SQL sequences and complex query generation without compromising accuracy or performance. |
|
|
|
Our fine-tuning process has enriched this model with domain-specific SQL patterns and nuanced query constructions, making it exceptionally adept at handling real-world SQL requirements, from query creation to debugging and optimization. By combining Qwen2.5's foundational strengths with targeted training on custom SQL data, this model achieves a powerful balance of general-purpose code understanding and SQL-specific precision, making it an ideal tool for developers and data engineers seeking top-tier SQL generation capabilities. |
|
|
|
|
|
## Inference |
|
|
|
Here provides a code snippet with `apply_chat_template` to show you how to load the tokenizer and model and how to generate contents. |
|
|
|
```python |
|
# Import necessary libraries |
|
from unsloth import FastLanguageModel |
|
import torch |
|
|
|
# Define the model name and other parameters |
|
model_name = "imsanjoykb/sqlCoder-Qwen2.5-8bit" |
|
max_seq_length = 2048 |
|
dtype = None |
|
load_in_4bit = True |
|
|
|
# Load the model and tokenizer from Hugging Face |
|
model, tokenizer = FastLanguageModel.from_pretrained( |
|
model_name=model_name, |
|
max_seq_length=max_seq_length, |
|
dtype=dtype, |
|
load_in_4bit=load_in_4bit, |
|
) |
|
|
|
# Enable faster inference |
|
FastLanguageModel.for_inference(model) |
|
|
|
# Define the prompt template |
|
odoo_text2sql_prompt = """Below is an instruction describing a task related to generating a SQL query specifically for Odoo's database structure. The input provides relevant context about Odoo models or data fields from {db_schema}. Write a SQL query that fulfills the given task using Odoo's database schema. |
|
|
|
### Instruction: |
|
Generate a SQL query in the context of Odoo to {} |
|
|
|
### Input: |
|
{} |
|
|
|
### Response: |
|
{} |
|
""" |
|
|
|
# Optionally, use a TextStreamer for continuous inference |
|
from transformers import TextStreamer |
|
|
|
# Prepare the input text for continuous inference |
|
instruction = "" |
|
input_text = "What is the top profitable product?" |
|
output_text = "" |
|
|
|
# Tokenize the input text |
|
inputs = tokenizer( |
|
[ |
|
odoo_text2sql_prompt.format(instruction, input_text, output_text) |
|
], |
|
return_tensors="pt" |
|
).to("cuda") |
|
|
|
# Initialize the TextStreamer |
|
text_streamer = TextStreamer(tokenizer) |
|
|
|
# Generate the output using the model with TextStreamer |
|
_ = model.generate(**inputs, streamer=text_streamer, max_new_tokens=350) |
|
``` |
|
## Model Download |
|
| **Model** | **#Total Params** | **#Active Params** | **Context Length** | **Download** | |
|
| :-----------------------------: | :---------------: | :----------------: | :----------------: | :----------------------------------------------------------: | |
|
| sqlCoder-Qwen2.5-8bit | 14B | 2.4B | 128k | [🤗 HuggingFace](https://huggingface.co/imsanjoykb/sqlCoder-Qwen2.5-8bit) | |
|
|
|
# Uploaded model |
|
|
|
- **Developed by:** [Sanjoy Biswas](https://www.linkedin.com/in/imsanjoykb/) |
|
- **License:** apache-2.0 |
