AIJapanese commited on
Commit
c6c7d16
·
verified ·
1 Parent(s): ac81f2e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +11 -9
README.md CHANGED
@@ -14,15 +14,6 @@ library_name: transformers
14
 
15
  **Moriyasu_Qwen2_JP_7B** is a is a large language model trained by Moriyasu. Based on [Qwen/Qwen2-7B](https://huggingface.co/Qwen/Qwen2-7B), it has been enhanced for Japanese usage through additional pre-training and instruction tuning.
16
 
17
- # Training Datasets
18
-
19
- ### Pre-training dataset
20
-
21
- The model is continually pre-trained on Japanese data from the Qwen2-7b model while maintaining the model's English ability (80% Japanese, 20% English). We use about 120 billion tokens sampled from, Japanese and English Wikipedia articles, Japanese CC-100 Japanese C4, Japanese OSCAR ,The Pile, Webfined, Japanese websites, book data, mathematics and code,...
22
-
23
- ### Instruction Tuning
24
- We generated about 1 million Instruction data from various methods such as generated data, translated data, and data manually tagged by humans.
25
-
26
  # Model Performance
27
 
28
  ### JGLUE tasks
@@ -128,3 +119,14 @@ generated_ids = [
128
  response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
129
  print(response)
130
  ```
 
 
 
 
 
 
 
 
 
 
 
 
14
 
15
  **Moriyasu_Qwen2_JP_7B** is a is a large language model trained by Moriyasu. Based on [Qwen/Qwen2-7B](https://huggingface.co/Qwen/Qwen2-7B), it has been enhanced for Japanese usage through additional pre-training and instruction tuning.
16
 
 
 
 
 
 
 
 
 
 
17
  # Model Performance
18
 
19
  ### JGLUE tasks
 
119
  response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
120
  print(response)
121
  ```
122
+
123
+ # Training Datasets
124
+
125
+ ### Pre-training dataset
126
+
127
+ The model is continually pre-trained on Japanese data from the Qwen2-7b model while maintaining the model's English ability (80% Japanese, 20% English). We use about 120 billion tokens sampled from, Japanese and English Wikipedia articles, Japanese CC-100 Japanese C4, Japanese OSCAR ,The Pile, Webfined, Japanese websites, book data, mathematics and code,...
128
+
129
+ ### Instruction Tuning
130
+ We generated about 1 million Instruction data from various methods such as generated data, translated data, and data manually tagged by humans.
131
+
132
+ # Contact: