XuYipei
/

Mixtral-4x7B-Chat-Chinese

@@ -1,199 +1,126 @@
----
-library_name: transformers
-tags: []
----
-# Model Card for Model ID
-<!-- Provide a quick summary of what the model is/does. -->
-## Model Details
-### Model Description
-<!-- Provide a longer summary of what this model is. -->
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-### Model Sources [optional]
-<!-- Provide the basic links for the model. -->
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-## Uses
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-### Direct Use
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-[More Information Needed]
-### Downstream Use [optional]
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-[More Information Needed]
-### Out-of-Scope Use
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-[More Information Needed]
-## Bias, Risks, and Limitations
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-[More Information Needed]
-### Recommendations
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-## How to Get Started with the Model
-Use the code below to get started with the model.
-[More Information Needed]
-## Training Details
-### Training Data
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-[More Information Needed]
-### Training Procedure
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-#### Preprocessing [optional]
-[More Information Needed]
-#### Training Hyperparameters
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-#### Speeds, Sizes, Times [optional]
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-[More Information Needed]
-## Evaluation
-<!-- This section describes the evaluation protocols and provides the results. -->
-### Testing Data, Factors & Metrics
-#### Testing Data
-<!-- This should link to a Dataset Card if possible. -->
-[More Information Needed]
-#### Factors
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-[More Information Needed]
-#### Metrics
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-[More Information Needed]
-### Results
-[More Information Needed]
-#### Summary
-## Model Examination [optional]
-<!-- Relevant interpretability work for the model goes here -->
-[More Information Needed]
-## Environmental Impact
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-## Technical Specifications [optional]
-### Model Architecture and Objective
-[More Information Needed]
-### Compute Infrastructure
-[More Information Needed]
-#### Hardware
-[More Information Needed]
-#### Software
-[More Information Needed]
-## Citation [optional]
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-**BibTeX:**
-[More Information Needed]
-**APA:**
-[More Information Needed]
-## Glossary [optional]
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-[More Information Needed]
-## More Information [optional]
-[More Information Needed]
-## Model Card Authors [optional]
-[More Information Needed]
-## Model Card Contact
-[More Information Needed]

+## 模型描述
+使用 [hfl/chinese-llama-2-7b · Hugging Face](https://huggingface.co/hfl/chinese-llama-2-7b) 作为中文分词器，训练的 Mixtral-4x7B-MoE 模型。
+可以在单卡 A100 上推理，在 8xA100 上全量微调。
+## 部分评测指标
+| MMLU  | CMMLU | C-Eval | GSM8K | MBPP |
+| ----- | ----- | ------ | ----- | ---- |
+| 55.13 | 51.10 | 52.0   | 67.17 | 40.2 |
+## 使用方式
+```python
+import torch
+import transformers
+def apply_multi_turn_template(history, input):
+    multi_turn_template = "[ROUND {} USER]{}[ROUND {} ASSISTANT]{}"
+    prefix = ""
+    for i in range(len(history)):
+        prefix = prefix + multi_turn_template.format(i, history[i][0], i, history[i][1])
+        prefix = prefix + "<|end_of_turn|>"
+    curr_turn_id = len(history)
+    curr_turn = "[ROUND {} USER]{}[ROUND {} ASSISTANT]".format(curr_turn_id, input, curr_turn_id)
+    return prefix + curr_turn
+model = transformers.AutoModelForCausalLM.from_pretrained("XuYipei/Mixtral-4x7B-Chat-Chinese")
+tokenizer = transformers.AutoTokenizer.from_pretrained("XuYipei/Mixtral-4x7B-Chat-Chinese")
+model = model.to(torch.bfloat16).to("cuda")
+input_text = apply_multi_turn_template([], "列举出10首杜甫的古诗，同时默写该诗篇的完整内容。")
+output = md.generate(input_ids = torch.LongTensor([tk.encode(input_text)]).to("cuda:5"), eos_token_id=55296, max_length=4096)
+output_text = tokenizer.decode(output[0])
+'''
+<s> [ROUND 0 USER]列举出10首杜甫的古诗，同时默写该诗篇的完整内容。[ROUND 0 ASSISTANT] 1. 《登高》
+唐·杜甫
+风急天高猿啸哀，渚清沙白鸟飞回。
+无边落木萧萧下，不尽长江滚滚来。
+万里悲秋常作客，百年多病独登台。
+艰难苦恨繁霜鬓，潦倒新停浊酒杯。
+2. 《春夜喜雨》
+唐·杜甫
+好雨知时节，当春乃发生。
+随风潜入夜，润物细无声。
+野径云俱黑，江船火独明。
+晓看红湿处，花重锦官城。
+3. 《绝句》
+唐·杜甫
+迟日江山丽，春风花草香。
+泥融飞燕子，沙暖睡鸳鸯。
+4. 《登楼》
+唐·杜甫
+花近高楼伤客心，万方多难此登临。
+锦江春色来天地，玉垒浮云变古今。
+北极朝廷终不改，西山寇盗莫相侵。
+可怜后主还祠庙，日暮聊为梁甫吟。
+5. 《茅屋为秋风所破歌》
+唐·杜甫
+八月秋高风怒号，卷我屋上三重茅。
+茅飞渡江洒江郊，高者挂罥长林梢，
+下者飘转沉塘坳。
+南村群童欺我老无力，忍能对面为盗贼。
+公然抱茅入竹去，唇焦口燥呼不得，
+归来倚杖自叹息。
+6. 《江南逢李龟年》
+唐·杜甫
+岐王宅里寻常见，崔九堂前几度闻。
+正是江南好风景，落花时节又逢君。
+7. 《闻官军收河南河北》
+唐·杜甫
+剑外忽传收蓟北，初闻涕泪满衣裳。
+却看妻子愁何在，漫卷诗书喜欲��。
+白日放歌须纵酒，青春作伴好还乡。
+即从巴峡穿巫峡，便下襄阳向洛阳。
+8. 《旅夜书怀》
+唐·杜甫
+细草微风岸，危樯独夜舟。
+星垂平野阔，月涌大江流。
+名岂文章著，官应老病休。
+飘飘何所似，天地一沙鸥。
+9. 《江畔独步寻花》
+唐·杜甫
+黄四娘家花满蹊，千朵万朵压枝低。
+留恋戏蝶时时舞，自在娇莺恰恰啼。
+10. 《赠花卿》
+唐·杜甫
+锦城丝管日纷纷，半入江风半入云。
+此曲只应天上有，人间能得几回闻。<|end_of_turn|>
+'''
+```