Text-to-text Generation Models (LLMs, Llama, GPT, ...)
Collection
5131 items
•
Updated
•
12
Frequently Asked Questions
from transformers import LlavaNextProcessor, LlavaNextForConditionalGeneration
import torch
from PIL import Image
import requests
processor = LlavaNextProcessor.from_pretrained("PrunaAI/llava-v1.6-vicuna-7b-bnb-4bit")
model = LlavaNextForConditionalGeneration.from_pretrained("PrunaAI/llava-v1.6-vicuna-7b-bnb-4bit")
# prepare image and text prompt, using the appropriate prompt template
url = "https://github.com/haotian-liu/LLaVA/blob/1a91fc274d7c35a9b50b3cb29c4247ae5837ce39/images/llava_v1_5_radar.jpg?raw=true"
image = Image.open(requests.get(url, stream=True).raw)
prompt = "A chat between a curious human and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the human's questions. USER: <image>\nWhat is shown in this image? ASSISTANT:"
inputs = processor(prompt, image, return_tensors="pt").to("cuda:0")
# autoregressively complete prompt
output = model.generate(**inputs, max_new_tokens=100)
print(processor.decode(output[0], skip_special_tokens=True))
The license of the smashed model follows the license of the original model. Please check the license of the original model liuhaotian/llava-v1.6-vicuna-7b before using this model which provided the base model. The license of the pruna-engine
is here on Pypi.