Under Editing

README.md

@@ -1,6 +1,17 @@
 ---
 library_name: transformers
-tags: []
+tags:
+- llm-jp-3-13b
+- llm
+- jp
+- 13b
+language:
+- ja
+base_model:
+- llm-jp/llm-jp-3-13b
+pipeline_tag: question-answering
+datasets: i
+license: apache-2.0
 ---
 
 # Model Card for Model ID
@@ -11,6 +22,16 @@ tags: []
 
 ## Model Details
 
+Uploaded model  
+Developed by: penguintrainer  
+License: apache-2.0 cc-by-sa-4.0  
+Finetuned from model : llm-jp/llm-jp-3-13b  
+
+Used ichikara-instruction-003-001-1 for fineturning.
+
+[ichikara-instruction: 日本語instructionモデル評価データセット](https://liat-aip.sakura.ne.jp/wp/llm%E3%81%AE%E3%81%9F%E3%82%81%E3%81%AE%E6%97%A5%E6%9C%AC%E8%AA%9E%E3%82%A4%E3%83%B3%E3%82%B9%E3%83%88%E3%83%A9%E3%82%AF%E3%82%B7%E3%83%A7%E3%83%B3%E3%83%87%E3%83%BC%E3%82%BF%E4%BD%9C%E6%88%90/llm%E3%81%AE%E3%81%9F%E3%82%81%E3%81%AE%E6%97%A5%E6%9C%AC%E8%AA%9E%E3%82%A4%E3%83%B3%E3%82%B9%E3%83%88%E3%83%A9%E3%82%AF%E3%82%B7%E3%83%A7%E3%83%B3%E3%83%87%E3%83%BC%E3%82%BF-%E5%85%AC%E9%96%8B/)
+© 2023 Akira Sasaki and Masato Hirakawa and Shintaro Horie and Tomoaki Nakamura (CC BY-SA 4.0 )
+
 ### Model Description
 
 <!-- Provide a longer summary of what this model is. -->
@@ -37,6 +58,58 @@ This is the model card of a 🤗 transformers model that has been pushed on the
 
 <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
 
+
+```python
+from transformers import (
+    AutoModelForCausalLM,
+    AutoTokenizer,
+    BitsAndBytesConfig,
+)
+from peft import PeftModel
+import torch
+from tqdm import tqdm
+
+model_id = "llm-jp/llm-jp-3-13b"
+adapter_id = "penguintrainer/llm-jp-3-13b-finetune"
+
+# QLoRA config
+bnb_config = BitsAndBytesConfig(
+    load_in_4bit=True,
+    bnb_4bit_quant_type="nf4",
+    bnb_4bit_compute_dtype=torch.bfloat16,
+)
+
+# Load model
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    quantization_config=bnb_config,
+    device_map="auto",
+    )
+
+# Load tokenizer
+tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
+
+# combain LoRA。
+model = PeftModel.from_pretrained(model, adapter_id)
+
+
+text = "大規模言語モデルとは何ですか？"
+
+tokenized_input = tokenizer.encode(text, add_special_tokens=False, return_tensors="pt").to(model.device)
+
+with torch.no_grad():
+    output = model.generate(
+        tokenized_input,
+        max_new_tokens=100,
+        do_sample=True,
+        top_p=0.95,
+        temperature=0.7,
+        repetition_penalty=1.05,
+    )[0]
+print(tokenizer.decode(output))
+
+```
+
 ### Direct Use
 
 <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->