llava-hf

/

llava-v1.6-34b-hf

like 70

Follow

Llava Hugging Face 114

Hi,

You can just copy the inference code of the model card in an inference.py script

@nielsr where can i find it

Scroll a bit down here please :) https://huggingface.co/llava-hf/llava-v1.6-34b-hf

when i was trying to run that inference script that is there in the model card in aws sagemaker what type of instance would you like to suggest to use it

@bk2000 not sure how AWS instances work, you should be able to run a model based on 7B decoder in an instance with T4 or V100 for example or if it doesn't fit you can load in 4/8-bit.

@merve but when i click on deploy to it is showing a code when i tried that it is throwing error any work around for that