|
--- |
|
language: |
|
- en |
|
tags: |
|
- pytorch |
|
- text-generation |
|
- causal-lm |
|
- rwkv |
|
license: apache-2.0 |
|
datasets: |
|
- the_pile |
|
|
|
--- |
|
|
|
# RWKV-4 "Raven"-series Models |
|
|
|
## Model Description |
|
|
|
These are RWKV-4-Pile models 3B/7B/14B finetuned on Alpaca, CodeAlpaca, Guanaco, GPT4All, ShareGPT and more. Ctxlen 8192 in v9. |
|
|
|
Gradio Demo: https://huggingface.co/spaces/BlinkDL/Raven-RWKV-7B |
|
|
|
Use https://github.com/BlinkDL/ChatRWKV to run them. |
|
|
|
See https://github.com/BlinkDL/RWKV-LM for details on the RWKV Language Model (100% RNN). |
|
|
|
New models will be named like Eng99%-Other1% and so on. |
|
|
|
Old models: |
|
* RWKV-4-Raven-Eng : 99% English + 1% Multilang |
|
* RWKV-4-Raven-EngAndMore : 96% English + 2% Chn Jpn + 2% Multilang (More Jpn than v6 "EngChnJpn") |
|
* RWKV-4-Raven-ChnEng : 49% English + 50% Chinese + 1% Multilang |
|
|
|
License: Apache 2.0 |
|
|
