How about unsing vLLM frame instead of InferyLLM ?

#13

by youaaa - opened Apr 9, 2024

youaaa

Apr 9, 2024

just wonder which is fast?

Does anyone know it? really appreciate it

Apr 26, 2024

You can use vLLM, performance will not be as good as with Infery-LLM. (DeciLM is documented in the vLLM readme).

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Your need to confirm your account before you can post a new comment.

· Sign up or log in to comment