AIJapanese
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -14,15 +14,6 @@ library_name: transformers
|
|
14 |
|
15 |
**Moriyasu_Qwen2_JP_7B** is a is a large language model trained by Moriyasu. Based on [Qwen/Qwen2-7B](https://huggingface.co/Qwen/Qwen2-7B), it has been enhanced for Japanese usage through additional pre-training and instruction tuning.
|
16 |
|
17 |
-
# Training Datasets
|
18 |
-
|
19 |
-
### Pre-training dataset
|
20 |
-
|
21 |
-
The model is continually pre-trained on Japanese data from the Qwen2-7b model while maintaining the model's English ability (80% Japanese, 20% English). We use about 120 billion tokens sampled from, Japanese and English Wikipedia articles, Japanese CC-100 Japanese C4, Japanese OSCAR ,The Pile, Webfined, Japanese websites, book data, mathematics and code,...
|
22 |
-
|
23 |
-
### Instruction Tuning
|
24 |
-
We generated about 1 million Instruction data from various methods such as generated data, translated data, and data manually tagged by humans.
|
25 |
-
|
26 |
# Model Performance
|
27 |
|
28 |
### JGLUE tasks
|
@@ -128,3 +119,14 @@ generated_ids = [
|
|
128 |
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
129 |
print(response)
|
130 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
14 |
|
15 |
**Moriyasu_Qwen2_JP_7B** is a is a large language model trained by Moriyasu. Based on [Qwen/Qwen2-7B](https://huggingface.co/Qwen/Qwen2-7B), it has been enhanced for Japanese usage through additional pre-training and instruction tuning.
|
16 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
17 |
# Model Performance
|
18 |
|
19 |
### JGLUE tasks
|
|
|
119 |
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
120 |
print(response)
|
121 |
```
|
122 |
+
|
123 |
+
# Training Datasets
|
124 |
+
|
125 |
+
### Pre-training dataset
|
126 |
+
|
127 |
+
The model is continually pre-trained on Japanese data from the Qwen2-7b model while maintaining the model's English ability (80% Japanese, 20% English). We use about 120 billion tokens sampled from, Japanese and English Wikipedia articles, Japanese CC-100 Japanese C4, Japanese OSCAR ,The Pile, Webfined, Japanese websites, book data, mathematics and code,...
|
128 |
+
|
129 |
+
### Instruction Tuning
|
130 |
+
We generated about 1 million Instruction data from various methods such as generated data, translated data, and data manually tagged by humans.
|
131 |
+
|
132 |
+
# Contact:
|