Update README.md
Browse files
README.md
CHANGED
@@ -89,9 +89,15 @@ Our training has 3 stages:
|
|
89 |
For details of the training dataset for each stage, please refer to the Dataset section and our CrystalCoder Data Card.
|
90 |
|
91 |
For hyperparameters used in each stage, please refer to the following table:
|
92 |
-
<|TABLE_NEEDED|>
|
93 |
|
94 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
95 |
|
96 |
# Dataset
|
97 |
|
|
|
89 |
For details of the training dataset for each stage, please refer to the Dataset section and our CrystalCoder Data Card.
|
90 |
|
91 |
For hyperparameters used in each stage, please refer to the following table:
|
|
|
92 |
|
93 |
+
| | **Phase 1** | **Phase 2** | **Phase 3** |
|
94 |
+
| --- | --- | --- | --- | --- |
|
95 |
+
| LR Warmup Steps| 86 | 86 | 176 |
|
96 |
+
| LR Start Value | 0.012 | 0.0087825 | 0.002 |
|
97 |
+
| LR Final Value | 0.00012408 | 0.00013679 | 0.0002 |
|
98 |
+
| LR Decay | Linear | Linear | Linear |
|
99 |
+
|
100 |
+
For more details of training, please refer to [our paper](https://arxiv.org/pdf/2312.06550.pdf).
|
101 |
|
102 |
# Dataset
|
103 |
|