Is the model TheBloke/vicuna-7B-1.1-HF compatible with cuda 11.7?

by anujs - opened Jul 29, 2023

Jul 29, 2023

I am trying to load the vicuna-7B-1.1-HF model on EC2 instance having GPU A10G, and has Build cuda_11.7.r11.7/compiler.31442593_0.
The model is loading fine, however, the model load time is high 100-120 sec. Any suggestions on the topic?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Your need to confirm your account before you can post a new comment.

· Sign up or log in to comment