README.md

---
language:
- en
tags:
- text-to-sql
- gpt2
- gpt2-medium
- nlp-to-sql
- text2sql
- sql
datasets:
- b-mc2/sql-create-context
---
# Model Card

<!-- The base model used for training is gpt2-medium. We finetuned it on the following dataset: b-mc2/sql-create-context -->

This is my first fine tuned LLM project.


## Prompt

query = List the creation year, name and budget of each department

f"Translate the following English question to SQL: {query}

## Output

SELECT creation_year, name, budget FROM department


#### Training Hyperparameters

num_train_epochs=1
per_device_train_batch_size=3
gradient_accumulation_steps=9
learning_rate=5e-5
weight_decay=0.01


## Evaluation

| Step     | Training Loss |
| -------- | ------- |
| 500  | 0.337800    |
| 1000 | 0.262900     |
| 1500    | 0.253200    |
| 2000    | 0.246400    |

{'eval_loss': 0.23689331114292145, 'eval_runtime': 104.4102, 'eval_samples_per_second': 67.043, 'eval_steps_per_second': 8.38, 'epoch': 1.0}