Update README.md

README.md

@@ -4,9 +4,15 @@ language:
 - yue
 ---
 
-**This is a preview version, we still finding the balance between overfitting and generalzation of DPO training, please check Limitation section about the problem we encountered in this version**
+---
+license: cc-by-nc-sa-4.0
+language:
+- yue
+---
+
+**This is a preview version, and this repository will be deleted once the new version is released. We are currently in the process of finding the balance between overfitting and generalization in DPO training. For more details about the problems we encountered in this version, please refer to the Limitation section.**
 
-Continual pretraining model of of the [Yi6b](https://huggingface.co/01-ai/Yi-6B) model on a Cantonese corpus, which consisted of translated Hong Kong news, Wikipedia articles, subtitles, and open-sourced dialogue corpora. Additionally, we extended the vocabulary to include common Cantonese words.
+Continual pretraining model of the [Yi-6B](https://huggingface.co/01-ai/Yi-6B) model on a Cantonese corpus, which consisted of translated Hong Kong news, Wikipedia articles, subtitles, and open-sourced dialogue corpora. Additionally, we extended the vocabulary to include common Cantonese words.
 
 The goal of this model was to evaluate whether we could train a language model that is fluent in Cantonese with limited resources (200 million tokens). Surprisingly, the outcome was quite good. However, there are still some issues with mirror misalignment between written Chinese and Cantonese, as well as knowledge transfer across different languages.
 
@@ -126,6 +132,10 @@ print(chat(messages))
 
 The model is intended to use for Cantonese language understanding and generation tasks, it may not be suitable for other Chinese languages. The model is trained on a diverse range of Cantonese text, including news, Wikipedia, and textbooks, it may not be suitable for informal or dialectal Cantonese, it may contain bias and misinformation, please use it with caution.
 
+### Hallucination
+
+Most LLM also have hallucination issue, this model is no exception, it may generate incorrect or misleading information, please use it with caution.
+
 #### Misaligment of Written Chinese
 
 Cantonese is a spoken language and without a standard written system that makes Cantonese becames a one of low-resource language, we see some instancees of generation by this model would mixed Written Chinese and Cantonese that due to the corpus in PT stage contains a lot of mixed sentences, we belive it can be fixed in DPO stage to mitigate this issue, this is what we current working on.
@@ -133,7 +143,7 @@ Cantonese is a spoken language and without a standard written system that makes
 ```
 User: 咩嘢係狼人殺?
 
-Assistant:
+Assistant: 狼人殺係一種多人回合策略遊戲，通常有 12 至 18 名玩家參與。玩家分為兩組：狼人同村民。狼人係狼人，佢哋嘅目標係要殺死村民，而村民嘅目標係要**找**出狼人並將佢哋處決。狼人同村民都有特殊能力，例如狼人可以隱藏自己嘅身份，而村民可以通過投票來處決狼人。狼人殺遊戲通常需要 30 至 45 分鐘才能完成。
 ```
 
 #### Misalignment of knowledge between different languages