Hugging Face's logo

Crataco
/
RWKV-4-PilePlus-Series-GGML

Text Generation
English
ggml
causal-lm
rwkv
Model card Files Community
1
RWKV-4-PilePlus-Series-GGML / README.md
Merry's picture
Merry
Update README.md
5a9ce2f
preview code
|
raw
history blame
1.19 kB
metadata
language:
  - en
tags:
  - ggml
  - text-generation
  - causal-lm
  - rwkv
license: apache-2.0
datasets:
  - EleutherAI/pile
  - togethercomputer/RedPajama-Data-1T

Last updated: 2023-06-07

This is BlinkDL/rwkv-4-world converted to GGML for use with rwkv.cpp and KoboldCpp. rwkv.cpp's conversion instructions were followed.

RAM USAGE (KoboldCpp)

Model RAM usage (with OpenBLAS)
Unloaded 41.3 MiB

Original model card by BlinkDL is below.

RWKV-4 PilePlus

Model Description

RWKV-4-pile models finetuning on [RedPajama + some of Pile v2 = 1.7T tokens]. Updated with 2020+2021+2022 data, and better at all European languages.

Although some of these are intermedia checkpoints (XXXGtokens means finetuned for XXXG tokens), you can already use them because I am finetuning from Pile models (instead of retraining).

Note: not instruct tuned yet, and recommended to replace vanilla Pile models.

7B and 14B coming soon.

See https://github.com/BlinkDL/RWKV-LM for details.

Use https://github.com/BlinkDL/ChatRWKV to run it.