Update README.md
Browse files
README.md
CHANGED
|
@@ -136,7 +136,12 @@ KISTI HPC NVIDIA A100 80G GPU 24EA์์ 2.5๊ฐ์๋์ 1,600,000 ์คํ
ํ์ต
|
|
| 136 |
|
| 137 |
#### Training Hyperparameters
|
| 138 |
|
|
|
|
| 139 |
- **model_size:** base
|
|
|
|
|
|
|
|
|
|
|
|
|
| 140 |
- **num_train_steps:** 1,600,000
|
| 141 |
- **train_batch_size:** 4,096 * 4 accumulative update = 16,384
|
| 142 |
- **learning_rate:** 1e-4
|
|
|
|
| 136 |
|
| 137 |
#### Training Hyperparameters
|
| 138 |
|
| 139 |
+
- **model_type:** deberta-v2
|
| 140 |
- **model_size:** base
|
| 141 |
+
- **parameters:** 900M
|
| 142 |
+
- **hidden_size:** 768
|
| 143 |
+
- **num_hidden_layers:** 12
|
| 144 |
+
- **num_attention_heads:** 12
|
| 145 |
- **num_train_steps:** 1,600,000
|
| 146 |
- **train_batch_size:** 4,096 * 4 accumulative update = 16,384
|
| 147 |
- **learning_rate:** 1e-4
|