when i run command ,it didnot work. ( via vllm 0.7.3)
#16
by
xueshuai
- opened
when i run command ,it didnot work. how much deploy-time should be ?
I failed to see where it didn't work, maybe try providing more logs and describing things properly?
I failed to see where it didn't work, maybe try providing more logs and describing things properly?
NCCL P2P DISABLE=1 VLLM_WORKER_MULTIPROC_METHOD=spawn python -m vllm.entrypoints.openai.api_server --host 0.0.0.0 --port 12345 --max-model-len 65536 --max-num-batched-tokens 65536 --trust-remote-code --tensor-parallel-size 8 --gpu-memory-utilization 0.97 --dtype float16 --served-model-name deepseek-reasoner --model cognitivecomputations/DeepSeek-R1-AWQ it wored
xueshuai
changed discussion status to
closed