Update Dockerfile

Update Dockerfile

Update Dockerfile

Update Dockerfile

Update Dockerfile

Update Dockerfile

Update Dockerfile

Update Dockerfile

--extra-index-url https://download.pytorch.org/whl/cu113

nvidia/cuda:11.7.1-cudnn8-runtime-ubuntu22.04

debug

python3 -m vllm.entrypoints.openai.api_server

fix(Dockerfile): use cmd single line

fix(Dockerfile): revision

fix(entrypoint) Dockerile

(feat:vllm serve) Dockerfile

feat(llama32-3b-instruct): change llama32-3b-instruct

feat(sailor2-3b-chat): change readme

fix(float16): Bfloat16 is only supported on GPUs with compute capability of at least 8.0. Your Tesla T4 GPU has compute capability 7.5. You can use float16 instead by explicitly setting the`dtype` flag in CLI, for example: --dtype=half.

fix(using sail/Sailor2-3B-Chat): sail/Sailor2-3B-Chat

docs(add-comment): add comment

feat(runner.sh): DeepSeek-R1-Distill-Qwen-32B d66bcfc2f3fd52799f95943264f32ba15ca0003d

feat(runner.sh): --trust-remote-code

feat(runner.sh): add deepseek-ai/DeepSeek-R1 and deepseek-ai/DeepSeek-V3

feat(Dockerfile): install gcc

feat(runner.sh): only enable prefix caching and disable log request

feat(runner.sh): --enable-chunked-prefill and --enable-prefix-caching for faster generate

fix(runner.sh): enable eager mode (disabling cuda graph)

fix(runner.sh): --enforce-eager not support values

fix(runner.sh): explicitly disabling enforce_eager

fix(runner.sh): disable eager-loading so it using cuda graph (in order for parallel and faster processing)

feat(runner.sh): add specific task and code revision

feat(runner.sh): using MODEL_ID only

feat(runner.sh): using runner.sh to select llm in the run time

feat(seed): Random seed for reproducibility.

feat(/app/run-llama.sh): /app/run-llama.sh

feat(/app/run-sailor.sh): /app/run-sailor.sh

feat(llama3.2): using llama model first for cost saving, until we want test sailor

docs(sailor): add not about minimum resources of sailor

feat(sailorchat): using sailor chat model

feat(quantization): T4 not support bfloat16

feat(llama3.2): run llama3.2 using bfloat16 with cache dtype fp8 with same model len

feat(sail/Sailor-4B-Chat): try increase gpu-memory-utilization to 0.9 before changing the token length

feat(sailor-8B): using sailor-8b

feat(llama3.2): using Llama-3.2-3B-Instruct 0cb88a4f764b7a12671c53f0838cd831a0843b95

feat(llama3.2): change model to llama3.2

feat(dep_sizes.txt): removes dep_sizes.txt during build, it not needed

feat(download_model.py): remove download_model.py during build, it causing big image size

docs(Dockerfile): add comment about estimated image size after compile

feat(add-model): always download model during build, it will be cached in the consecutive builds