|
--- |
|
language: |
|
- en |
|
tags: |
|
- text-to-sql |
|
- gpt2 |
|
- gpt2-medium |
|
- nlp-to-sql |
|
- text2sql |
|
- sql |
|
datasets: |
|
- b-mc2/sql-create-context |
|
--- |
|
# Model Card |
|
|
|
<!-- The base model used for training is gpt2-medium. We finetuned it on the following dataset: b-mc2/sql-create-context --> |
|
|
|
This is my first fine tuned LLM project. |
|
|
|
|
|
## Prompt |
|
|
|
query = List the creation year, name and budget of each department |
|
|
|
f"Translate the following English question to SQL: {query} |
|
|
|
## Output |
|
|
|
SELECT creation_year, name, budget FROM department |
|
|
|
|
|
#### Training Hyperparameters |
|
|
|
num_train_epochs=1 |
|
per_device_train_batch_size=3 |
|
gradient_accumulation_steps=9 |
|
learning_rate=5e-5 |
|
weight_decay=0.01 |
|
|
|
|
|
## Evaluation |
|
|
|
| Step | Training Loss | |
|
| -------- | ------- | |
|
| 500 | 0.337800 | |
|
| 1000 | 0.262900 | |
|
| 1500 | 0.253200 | |
|
| 2000 | 0.246400 | |
|
|
|
{'eval_loss': 0.23689331114292145, 'eval_runtime': 104.4102, 'eval_samples_per_second': 67.043, 'eval_steps_per_second': 8.38, 'epoch': 1.0} |
