vllm support
#15
by
yaronr
- opened
Hi
Can you please share whether you plan on adding support for your model in vllm? ('SolarForCausalLM' architecture)
We would love to run our independent analysis on solar and share our results (and we use vllm).
Thank you!
yaronr
changed discussion title from
Chunked prefill & prefix caching
to vllm support
Hi
@yaronr
, vLLM now supports SolarForCausalLM architecture as of v0.6.2. This version was released last week. Please check it out!
https://github.com/vllm-project/vllm/releases/tag/v0.6.2
Thank you. I will give it a try.