Use below code to download the mistral.
#pip install -U transformers accelerate torch
import torch
from transformers import pipeline, set_seed
model_path = "vicky4s4s/mistral-7b-v2-instruct"
pipe = pipeline("text-generation", model=model_path, torch_dtype=torch.bfloat16, device_map="cuda")
messages = [{"role": "user", "content": "what is meaning of life?"}]
outputs = pipe(messages, max_new_tokens=1000, do_sample=True, temperature=0.71, top_k=50, top_p=0.92,repetition_penalty=1)
print(outputs[0]["generated_text"][-1]["content"])
Limitations
The Mistral 7B Instruct model is a quick demonstration that the base model can be easily fine-tuned to achieve compelling performance. It does not have any moderation mechanisms. We're looking forward to engaging with the community on ways to make the model finely respect guardrails, allowing for deployment in environments requiring moderated outputs.
Develop By
Vignesh, [email protected]
- Downloads last month
- 22
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.