Vllm
#2
by
TitanomTechnologies
- opened
Any idea when or if it is planned to add vllm support? Or am I missing somthing?
This is based on Llama 3.3 70B. It should just work on vLLM.
but how about tool usage support? in openai style api. needs a parser for vllm
According to the chat template, you can probably use the llama3_json tool parser as a basis and modify it.