Update README.md
Browse files
README.md
CHANGED
|
@@ -73,6 +73,8 @@ print(code)
|
|
| 73 |
|
| 74 |
[gretelai/synthetic_text_to_sql](https://huggingface.co/datasets/gretelai/synthetic_text_to_sql)
|
| 75 |
|
|
|
|
|
|
|
| 76 |
### Training Procedure
|
| 77 |
|
| 78 |
HuggingFace Accelerate with Training Loop.
|
|
@@ -87,6 +89,12 @@ HuggingFace Accelerate with Training Loop.
|
|
| 87 |
- **gradient_accumulation_steps:** 8
|
| 88 |
- **global_step:** 625
|
| 89 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 90 |
|
| 91 |
#### Hardware
|
| 92 |
|
|
|
|
| 73 |
|
| 74 |
[gretelai/synthetic_text_to_sql](https://huggingface.co/datasets/gretelai/synthetic_text_to_sql)
|
| 75 |
|
| 76 |
+
**Train Set Size:** 20000 shuffled randomly
|
| 77 |
+
|
| 78 |
### Training Procedure
|
| 79 |
|
| 80 |
HuggingFace Accelerate with Training Loop.
|
|
|
|
| 89 |
- **gradient_accumulation_steps:** 8
|
| 90 |
- **global_step:** 625
|
| 91 |
|
| 92 |
+
LoraConfig
|
| 93 |
+
- ***r:*** 8
|
| 94 |
+
- ***lora_alpha:*** 32
|
| 95 |
+
- ***target_modules:*** ["k_proj","o_proj","q_proj","v_proj"]
|
| 96 |
+
- ***lora_dropout:*** 0.05
|
| 97 |
+
|
| 98 |
|
| 99 |
#### Hardware
|
| 100 |
|