Update README.md

README.md

@@ -35,6 +35,39 @@ User message<|im_end|>
 <|im_start|>assistant
 ```
 
+## Usage
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer, GeneratorConfig
+
+
+model_id = "rxavier/Taurus-7B-1.0"
+model = AutoModelForCausalLM.from_pretrained(
+ model_path,
+ torch_dtype=torch.bfloat16,
+)
+tokenizer = AutoTokenizer.from_pretrained(model_path)
+generation_config = GenerationConfig(
+ bos_token_id=tok.bos_token_id,
+ eos_token_id=tok.eos_token_id,
+ pad_token_id=tok.pad_token_id,
+ )
+
+prompt = "Give me latex formulas for extended euler equations"
+system_message = "You are an expert in economics with PhD level knowledge. You are helpful, give thorough and clear explanations, and use equations and formulas where needed."
+
+messages = [{"role": "system",
+ "content": system_message},
+ {"role": "user",
+ "content": prompt}]
+tokens = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt").to("cuda")
+
+with torch.no_grad():
+ outputs = model.generate(inputs=tokens, generation_config=generation_config)
+print(tokenizer.decode(outputs["sequences"].cpu().tolist()[0]))
+```
+
 ## GGUF quants
 
 You can find GGUF quants for llama.cpp [here](https://huggingface.co/rxavier/Taurus-7B-1.0-GGUF).