metadata
language:
- en
tags:
- text-to-sql
- gpt2
- gpt2-medium
- nlp-to-sql
- text2sql
- sql
datasets:
- b-mc2/sql-create-context
Model Card
This is my first fine tuned LLM project.
Prompt
query = List the creation year, name and budget of each department
f"Translate the following English question to SQL: {query}
Output
SELECT creation_year, name, budget FROM department
Training Hyperparameters
num_train_epochs=1 per_device_train_batch_size=3 gradient_accumulation_steps=9 learning_rate=5e-5 weight_decay=0.01
Evaluation
| Step | Training Loss |
|---|---|
| 500 | 0.337800 |
| 1000 | 0.262900 |
| 1500 | 0.253200 |
| 2000 | 0.246400 |
{'eval_loss': 0.23689331114292145, 'eval_runtime': 104.4102, 'eval_samples_per_second': 67.043, 'eval_steps_per_second': 8.38, 'epoch': 1.0}