ChnJpn more Japanese Ratio Plz?
Thanks for reading!
I'm very excited with rwkv-4-raven 14B v6 0401 ChnJpn!
However, it seems that the ratio of Japanese is only 1%, and the ratio might be a bit too small to create fruitful sentence.
I would deeply appreciate if you could increase the ratio of Japanese Dataset in ChnJpn in v7 or v8,
or Could you possibly provide the easier way to train the model in local? though I'm not sure whether it is trainable on single RTX A6000 or not.
Try v7 EngAndMore :) This has slightly more JPN.
And you can use https://github.com/Blealtan/RWKV-LM-LoRA
Thank you for your guidance on EngAndMore model and how to train with LoRA method!
I will definitely refer to it.
@terorin And I can add more JPN if you guys can collect more JPN ChatGPT data (similar to https://ithub.com/nomic-ai/gpt4all, with single-round and multi-round data)
Verrrry Thank you for including 10% Japanese in 7B-v10 version!
I chatted with the 7B-v10-Jpn10 and felt that Japanese Response getting better.
Now I'm interested in 14B version with much more Japanese.