Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Multimodal Models
Paper
•
2409.17146
•
Published
•
104
Big Science - Modeling Metadata
non-profit
AI & ML interests
None defined yet.
Recent Activity
View all activity
bs-modeling-metadata's activity
Muennighoff
authored
a
paper
3 months ago
Muennighoff
authored
a
paper
4 months ago
Leyo
authored
a
paper
4 months ago
HugoLaurencon
authored
a
paper
4 months ago
vumichien
authored
a
paper
5 months ago
Muennighoff
authored
a
paper
5 months ago
Muennighoff
authored
a
paper
6 months ago
vumichien
authored
a
paper
6 months ago
Muennighoff
authored
2
papers
6 months ago
stellaathena
authored
a
paper
7 months ago
Muennighoff
authored
a
paper
7 months ago
HugoLaurencon
authored
a
paper
8 months ago
Leyo
authored
a
paper
8 months ago
HugoLaurencon
posted
an
update
8 months ago
Post
2898
We release Idefics2-chatty, the chatbot-optimized version of Idefics2:
HuggingFaceM4/idefics2-8b-chatty
Idefics2-chatty is better at following instructions and following Chain-of-Thoughts reasoning.
Moreover, we also release a paper, containing a lot of findings on how to build an efficient and performant Vision-Language Model: What matters when building vision-language models? (2405.02246)
How are you going to use the model, or what data are you going to fine-tune it on?
Idefics2-chatty is better at following instructions and following Chain-of-Thoughts reasoning.
Moreover, we also release a paper, containing a lot of findings on how to build an efficient and performant Vision-Language Model: What matters when building vision-language models? (2405.02246)
How are you going to use the model, or what data are you going to fine-tune it on?
-
- 5 replies
VictorSanh
posted
an
update
8 months ago
Post
2824
💬🔥Releasing idefics2-8b-chatty, the chat-optimized version of Idefics2!
It is a very efficient (8B parameters) state-of-the-art VLM, has been red-teamed, and comes with a few surprises:
- 📖Paper dissecting a lot of the experimental insights we learned building Idefics2:
- 🏎️TGI integration for blazing-fast inference (you can already run it locally with < 24GB GPU memory)
- 🏆 Ranking 2nd in its category (< 10B, open weights) in the awesome Open VLM Leaderboard, and now appearing in the incredible Vision Arena
Ressources:
⏯️Playground: HuggingFaceM4/idefics2_playground
📖Paper: What matters when building vision-language models? (2405.02246)
🏋️♂️Model and red-teaming analysis: HuggingFaceM4/idefics2-8b-chatty
👀Ressources to get started: HuggingFaceM4/idefics2-8b-chatty
🏆Open VLM Leaderboard: opencompass/open_vlm_leaderboard
🏟️Vision arena: WildVision/vision-arena
It is a very efficient (8B parameters) state-of-the-art VLM, has been red-teamed, and comes with a few surprises:
- 📖Paper dissecting a lot of the experimental insights we learned building Idefics2:
- 🏎️TGI integration for blazing-fast inference (you can already run it locally with < 24GB GPU memory)
- 🏆 Ranking 2nd in its category (< 10B, open weights) in the awesome Open VLM Leaderboard, and now appearing in the incredible Vision Arena
Ressources:
⏯️Playground: HuggingFaceM4/idefics2_playground
📖Paper: What matters when building vision-language models? (2405.02246)
🏋️♂️Model and red-teaming analysis: HuggingFaceM4/idefics2-8b-chatty
👀Ressources to get started: HuggingFaceM4/idefics2-8b-chatty
🏆Open VLM Leaderboard: opencompass/open_vlm_leaderboard
🏟️Vision arena: WildVision/vision-arena
- 1 reply
VictorSanh
authored
a
paper
8 months ago
VictorSanh
posted
an
update
8 months ago
Post
2785
Glad to see Idefics2 making its way into the awesome OpenVLM Leaderboard which ranks VLMs. 🏆
2nd in its category (<10B parameters and open weights)!
While InternLM-XComposer2 uses proprietary data, Idefics2 is built solely using openly available data.
Leaderboard: opencompass/open_vlm_leaderboard
Model: HuggingFaceM4/idefics2-8b
2nd in its category (<10B parameters and open weights)!
While InternLM-XComposer2 uses proprietary data, Idefics2 is built solely using openly available data.
Leaderboard: opencompass/open_vlm_leaderboard
Model: HuggingFaceM4/idefics2-8b
- 9 replies
VictorSanh
authored
2
papers
8 months ago