Commit History

Update README.md
5107aa4

beomi commited on

total 250B tokens trained
e2e327a

beomi commited on

Update README.md
7f95d1b

beomi commited on

Add 1.5B release model (trained with 2048 batch)
b7032b9

beomi commited on

Update README.md
04cecb7

beomi commited on

Update README.md
69e0a88

beomi commited on

Update README.md
fe63b04

beomi commited on

Add 1epoch model ckpt
203c84a

beomi commited on

Add bfloat16 weight, with 4.37M step trained
28a08ea

beomi commited on

Add safetensors (100B) ckpt
9dab3b8

beomi commited on

Update README.md
c7290e1

beomi commited on

Add 100B tokens ckpt
d3fd582

beomi commited on

Update README.md
46a5e35

beomi commited on

Add ~58B tokens ckpt
34bec76

beomi commited on

Merge branch 'main' of https://huggingface.co/beomi/KoRWKV-1.5B
1a86367

beomi commited on

Add 1.3M ckpt(= 42B tokens)
1845fad

beomi commited on

Update README.md
45f13af

beomi commited on

Update README.md
74f5aab

beomi commited on

Add 956k step(=31.3B tokens) ckpt
8aa951c

beomi commited on

RNN can extrapolate to Unlimited context
376b4bf

beomi commited on

update to 810k step(~=26.5B tokens
8f704cc

beomi commited on

edit weight keys
88a12b7

beomi commited on

update
ed167a2

beomi commited on

Add original ckpt for RWKV code
c99f77f

beomi commited on

Update README.md
773fcc4

beomi commited on

Merge branch 'main' of https://huggingface.co/beomi/KoRWKV-1.5B
6121b22

beomi commited on

Add 635k (20.8B tokens) step ckpt
b3ac395

beomi commited on

Update README.md
7496177

beomi commited on

Add 15.4B tokens(470k steps)
d7b24f7

beomi commited on

test
882a7f8

beomi commited on

Update README.md
3503544

beomi commited on

initial commit
717a5e3

beomi commited on