File size: 3,849 Bytes
cc044d1 896adff cc044d1 896adff cc044d1 896adff cc044d1 896adff cc044d1 896adff cc044d1 896adff cc044d1 896adff |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 |
---
license: apache-2.0
language:
- en
base_model:
- Qwen/Qwen2.5-Coder-32B-Instruct
new_version: imsanjoykb/sqlCoder-Qwen2.5-8bit
pipeline_tag: text-generation
library_name: adapter-transformers
tags:
- unsloth,
- pytorch,
- inference-endpoint,
- sql-code-generation,
---
[🤗 HF Repo](https://huggingface.co/imsanjoykb/sqlCoder-Qwen2.5-8bit) | [♾️ Colab](https://colab.research.google.com/drive/19e-u32GY2y5lsckNuWhBQExvXgVn8ZjG?usp=sharing)
Introducing the latest fine-tuned version of Qwen2.5-Coder-14B-Instruct, specifically tailored for SQL code generation. Built on the robust 14-billion parameter Qwen2.5-Coder architecture, this model leverages advanced configurations like bfloat16 precision and a custom quantization setup, optimized for efficient 4-bit computation. With a maximum context window of 32K tokens, this model supports extensive SQL sequences and complex query generation without compromising accuracy or performance.
Our fine-tuning process has enriched this model with domain-specific SQL patterns and nuanced query constructions, making it exceptionally adept at handling real-world SQL requirements, from query creation to debugging and optimization. By combining Qwen2.5's foundational strengths with targeted training on custom SQL data, this model achieves a powerful balance of general-purpose code understanding and SQL-specific precision, making it an ideal tool for developers and data engineers seeking top-tier SQL generation capabilities.
## Inference
Here provides a code snippet with `apply_chat_template` to show you how to load the tokenizer and model and how to generate contents.
```python
# Import necessary libraries
from unsloth import FastLanguageModel
import torch
# Define the model name and other parameters
model_name = "imsanjoykb/sqlCoder-Qwen2.5-8bit"
max_seq_length = 2048
dtype = None
load_in_4bit = True
# Load the model and tokenizer from Hugging Face
model, tokenizer = FastLanguageModel.from_pretrained(
model_name=model_name,
max_seq_length=max_seq_length,
dtype=dtype,
load_in_4bit=load_in_4bit,
)
# Enable faster inference
FastLanguageModel.for_inference(model)
# Define the prompt template
odoo_text2sql_prompt = """Below is an instruction describing a task related to generating a SQL query specifically for Odoo's database structure. The input provides relevant context about Odoo models or data fields from {db_schema}. Write a SQL query that fulfills the given task using Odoo's database schema.
### Instruction:
Generate a SQL query in the context of Odoo to {}
### Input:
{}
### Response:
{}
"""
# Optionally, use a TextStreamer for continuous inference
from transformers import TextStreamer
# Prepare the input text for continuous inference
instruction = ""
input_text = "What is the top profitable product?"
output_text = ""
# Tokenize the input text
inputs = tokenizer(
[
odoo_text2sql_prompt.format(instruction, input_text, output_text)
],
return_tensors="pt"
).to("cuda")
# Initialize the TextStreamer
text_streamer = TextStreamer(tokenizer)
# Generate the output using the model with TextStreamer
_ = model.generate(**inputs, streamer=text_streamer, max_new_tokens=350)
```
## Model Download
| **Model** | **#Total Params** | **#Active Params** | **Context Length** | **Download** |
| :-----------------------------: | :---------------: | :----------------: | :----------------: | :----------------------------------------------------------: |
| sqlCoder-Qwen2.5-8bit | 14B | 2.4B | 128k | [🤗 HuggingFace](https://huggingface.co/imsanjoykb/sqlCoder-Qwen2.5-8bit) |
# Uploaded model
- **Developed by:** [Sanjoy Biswas](https://www.linkedin.com/in/imsanjoykb/)
- **License:** apache-2.0 |