Jebadiah/Aria-coder-plus-7b
Updated
•
33
•
1
jebadiah greenwood
Jebadiah
AI & ML interests
None yet
Recent Activity
updated
a model
2 days ago
Jebadiah/Aria-coder-plus-7b
published
a model
2 days ago
Jebadiah/Aria-coder-plus-7b
updated
a model
2 days ago
Jebadiah/Aria-rp-coder-7b
Organizations
Jebadiah's activity
updated
a
model
2 days ago
published
a
model
2 days ago
updated
a
model
2 days ago
published
a
model
2 days ago
updated
a
model
3 days ago
published
a
model
3 days ago
updated
a
model
3 days ago
published
a
model
3 days ago
updated
a
model
3 days ago
published
a
model
3 days ago
updated
a
model
3 days ago
updated
a
model
4 days ago
published
a
model
4 days ago
updated
a
model
4 days ago
published
a
model
4 days ago
updated
a
model
6 days ago
published
a
model
6 days ago
updated
a
model
2 months ago
jeiku/Aura-NeMo-12B
2
#2 opened 7 months ago
by
Jebadiah
reacted to
merve's
post with 🚀
8 months ago
Post
3279
Forget any document retrievers, use ColPali 💥💥
Document retrieval is done through OCR + layout detection, but you are losing a lot of information in between, stop doing that! 🤓
ColPali uses a vision language model, which is better in doc understanding 📑
ColPali: vidore/colpali (mit license!)
Blog post: https://huggingface.co/blog/manu/colpali
The authors also released a new benchmark for document retrieval:
ViDoRe Benchmark: vidore/vidore-benchmark-667173f98e70a1c0fa4db00d
ViDoRe Leaderboard: vidore/vidore-leaderboard
ColPali marries the idea of modern vision language models with retrieval 🤝
The authors apply contrastive fine-tuning to SigLIP on documents, and pool the outputs (they call it BiSigLip). Then they feed the patch embedding outputs to PaliGemma and create BiPali 🖇️
BiPali natively supports image patch embeddings to an LLM, which enables leveraging the ColBERT-like late interaction computations between text tokens and image patches (hence the name ColPali!) 🤩
The authors created the ViDoRe benchmark by collecting PDF documents and generate queries from Claude-3 Sonnet.
ColPali seems to be the most performant model on ViDoRe. Not only this, but is way faster than traditional PDF parsers too!
Document retrieval is done through OCR + layout detection, but you are losing a lot of information in between, stop doing that! 🤓
ColPali uses a vision language model, which is better in doc understanding 📑
ColPali: vidore/colpali (mit license!)
Blog post: https://huggingface.co/blog/manu/colpali
The authors also released a new benchmark for document retrieval:
ViDoRe Benchmark: vidore/vidore-benchmark-667173f98e70a1c0fa4db00d
ViDoRe Leaderboard: vidore/vidore-leaderboard
ColPali marries the idea of modern vision language models with retrieval 🤝
The authors apply contrastive fine-tuning to SigLIP on documents, and pool the outputs (they call it BiSigLip). Then they feed the patch embedding outputs to PaliGemma and create BiPali 🖇️
BiPali natively supports image patch embeddings to an LLM, which enables leveraging the ColBERT-like late interaction computations between text tokens and image patches (hence the name ColPali!) 🤩
The authors created the ViDoRe benchmark by collecting PDF documents and generate queries from Claude-3 Sonnet.
ColPali seems to be the most performant model on ViDoRe. Not only this, but is way faster than traditional PDF parsers too!