Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,11 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
library_name: peft
|
3 |
---
|
|
|
1 |
+
|
2 |
+
## OVERVIEW
|
3 |
+
PEFT model trained on a strided, raw text dataset.
|
4 |
+
Full training parameters are available in training_parameters.json.
|
5 |
+
The most salient model feature is a training context window of 1536 tokens, the maximum possible while holding all previous training parameters constant.
|
6 |
+
Each training window is overlapped with the last 512 tokens of the previous window.
|
7 |
+
This overlap ratio is less unusual than the 'C' type model and produces less objectionable results in eval.
|
8 |
+
|
9 |
---
|
10 |
library_name: peft
|
11 |
---
|