Qwen/Qwen2-VL-2B-Instruct-GPTQ-Int4
Image-Text-to-Text
•
Updated
•
5.41k
•
23
Yunho Cho
yhjc2692
·
AI & ML interests
inference optimization, world models, learning methods in robotics contexts
Recent Activity
liked
a model
18 days ago
Qwen/Qwen2-VL-2B-Instruct-GPTQ-Int4
reacted
to
andrewrreed's
post
with ❤️
19 days ago
🚀 It's now easier than ever to switch from OpenAI to open LLMs
Hugging Face's TGI now supports an OpenAI compatible Chat Completion API
This means you can transition code that uses OpenAI client libraries (or frameworks like LangChain 🦜 and LlamaIndex 🦙) to run open models by changing just two lines of code 🤗
⭐ Here's how:
```
from openai import OpenAI
# initialize the client but point it to TGI
client = OpenAI(
base_url="<ENDPOINT_URL>" + "/v1/", # replace with your endpoint url
api_key="<HF_API_TOKEN>", # replace with your token
)
chat_completion = client.chat.completions.create(
model="tgi",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Why is open-source software important?"},
],
stream=True,
max_tokens=500
)
# iterate and print stream
for message in chat_completion:
print(message.choices[0].delta.content, end="")
```
🔗 Blog post ➡ https://huggingface.co/blog/tgi-messages-api
🔗 TGI docs ➡ https://huggingface.co/docs/text-generation-inference/en/messages_api
liked
a model
19 days ago
Qwen/Qwen2-VL-7B-Instruct-AWQ
Organizations
yhjc2692's activity
liked
a
model
18 days ago
liked
3
models
19 days ago
liked
a
Space
22 days ago
liked
2
models
26 days ago
liked
a
model
2 months ago
liked
a
dataset
2 months ago
liked
2
models
3 months ago
liked
a
model
5 months ago
liked
a
Space
5 months ago
liked
a
Space
9 months ago
liked
2
models
11 months ago
liked
a
model
12 months ago
liked
a
model
about 1 year ago
liked
2
models
over 1 year ago