Update README.md
Browse files
README.md
CHANGED
@@ -18,9 +18,11 @@ datasets:
|
|
18 |
|
19 |
RWKV-4-pile models finetuning on [RedPajama + some of Pile v2 = 1.7T tokens]. Updated with 2020+2021+2022 data, and better at all European languages.
|
20 |
|
21 |
-
|
22 |
|
23 |
-
|
|
|
|
|
24 |
|
25 |
See https://github.com/BlinkDL/RWKV-LM for details.
|
26 |
|
|
|
18 |
|
19 |
RWKV-4-pile models finetuning on [RedPajama + some of Pile v2 = 1.7T tokens]. Updated with 2020+2021+2022 data, and better at all European languages.
|
20 |
|
21 |
+
Although some of these are intermedia checkpoints (XXXGtokens means finetuned for XXXG tokens), you can already use them because I am finetuning from Pile models (instead of retraining).
|
22 |
|
23 |
+
Note: not instruct tuned yet, and recommended to replace vanilla Pile models.
|
24 |
+
|
25 |
+
7B and 14B coming soon.
|
26 |
|
27 |
See https://github.com/BlinkDL/RWKV-LM for details.
|
28 |
|