Update README.md
Browse files
README.md
CHANGED
|
@@ -109,7 +109,7 @@ I've chosen the size ratio between the matrices A and B of 8. The matrix A weigh
|
|
| 109 |
- Optimizer: AdamW
|
| 110 |
- LR: 1e-4
|
| 111 |
- LR scheduler: cosine
|
| 112 |
-
- NEFT enabled: true
|
| 113 |
- Batch size: 8
|
| 114 |
- Number of epochs: 4
|
| 115 |
- Padding: right with an additional padding <pad> token added
|
|
|
|
| 109 |
- Optimizer: AdamW
|
| 110 |
- LR: 1e-4
|
| 111 |
- LR scheduler: cosine
|
| 112 |
+
- [NEFT noise](https://arxiv.org/pdf/2310.05914) enabled: true
|
| 113 |
- Batch size: 8
|
| 114 |
- Number of epochs: 4
|
| 115 |
- Padding: right with an additional padding <pad> token added
|