alibidaran/Gemma2_SQLGEN

Model Card for Model ID

This model is trained on generating SQL code from user prompts.

Model Details

Model Description

This model is traiend for generating SQL code from user prompts. The prompt structure is based on this format. ###Question ###Context[SQL code of your table ] ###Answer: This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

• Developed by: Ali Bidaran
• Model type: [More Information Needed]
• Language(s) (NLP): [More Information Needed]
• License: [More Information Needed]
• Finetuned from model [optional]: Gemma 2B

Model Sources [optional]

Direct Use

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig, GemmaTokenizer

model_id = "Gemma2_SQLGEN"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16
)

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=bnb_config, device_map={"":0})
tokenizer.padding_side = 'right'
)
from peft import LoraConfig, PeftModel, get_peft_model
from trl import SFTTrainer

prompt = "find unique items from  name coloum."
text=f"<s>##Question: {prompt} \n ##Context: CREATE TABLE head (head_id VARCHAR, name VARCHAR) \n ##Answer:"
inputs=tokenizer(text,return_tensors='pt').to('cuda')
outputs=model.generate(**inputs,max_new_tokens=400,do_sample=True,top_p=0.92,top_k=10,temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))