RedRedRed's picture

RedRedRed

RedSparkie

AI & ML interests

โ›ง ARTโ›ง

Recent Activity

Organizations

Nerdy Face's profile picture

RedSparkie's activity

reacted to julien-c's post with ๐Ÿ”ฅ 15 days ago
view post
Post
7598
After some heated discussion ๐Ÿ”ฅ, we clarify our intent re. storage limits on the Hub

TL;DR:
- public storage is free, and (unless blatant abuse) unlimited. We do ask that you consider upgrading to PRO and/or Enterprise Hub if possible
- private storage is paid above a significant free tier (1TB if you have a paid account, 100GB otherwise)

docs: https://huggingface.co/docs/hub/storage-limits

We optimize our infrastructure continuously to scale our storage for the coming years of growth in Machine learning, to the benefit of the community ๐Ÿ”ฅ

cc: @reach-vb @pierric @victor and the HF team
ยท
reacted to zhiminy's post with ๐Ÿ˜Ž 20 days ago
view post
Post
1163
We're thrilled to introduce our latest project: SE Arena! ๐ŸŽ‰

SE Arena is an interactive platform designed to evaluate and compare software engineering chatbots powered by foundation models. With a transparent, open-source leaderboard, support for multi-round conversations, and head-to-head model comparisons, SE Arena is here to bring clarity to the evaluation process for FMs in software engineering tasks.

Check it out here: SE-Arena/Software-Engineering-Arena

Weโ€™d love your feedback and contributions! ๐Ÿš€
reacted to m-ric's post with ๐Ÿ‘€๐Ÿ‘€ 21 days ago
view post
Post
1467
๐—ฆ๐—ต๐—ผ๐˜„๐—จ๐—œ: ๐—ฎ ๐˜€๐—บ๐—ฎ๐—น๐—น ๐—ฒ๐—ป๐—ฑ-๐˜๐—ผ-๐—ฒ๐—ป๐—ฑ ๐—ฎ๐—ด๐—ฒ๐—ป๐˜ ๐˜๐—ต๐—ฎ๐˜ ๐—ฐ๐—ฎ๐—ป ๐—ป๐—ฎ๐˜ƒ๐—ถ๐—ด๐—ฎ๐˜๐—ฒ ๐—ฎ๐—ป๐˜† ๐—จ๐—œ ๐—ฎ๐—ป๐—ฑ ๐—ผ๐˜‚๐˜๐—ฝ๐—ฒ๐—ฟ๐—ณ๐—ผ๐—ฟ๐—บ๐˜€ ๐—บ๐˜‚๐—ฐ๐—ต ๐—ฏ๐—ถ๐—ด๐—ด๐—ฒ๐—ฟ ๐˜€๐˜†๐˜€๐˜๐—ฒ๐—บ๐˜€! ๐Ÿ“ฒ

A team from NUS and Microsoft just released an agent that can act on any UI (Desktop, Android, Web) without needing additional text information. It works extremely well : they applied their method on a tiny Qwen2-VL-2B, and they managed to beat methods that use either much more powerful vision models (like GPT-4V) without using any additional info (e.g. leveraging the DOM of a webpage) like previous methods did ! ๐Ÿ‘๐Ÿ‘

They started from the idea that most existing methods rely heavily on text, which makes them less generalizable, while letting aside rich UI structure that user actually rely on when navigating this interfaces.

โš™๏ธ They put several good ideas to work:

๐Ÿ’ก Simplify screenshots to the max:
They prune a lot the heavy visual content of UI screenshots, by removing cloned image patches (like any vast patch of the same color will be reduced to a small patch, while maintaining positional embeddings), then group patches from the same GUI elements together to simplify even further

๐Ÿ’ก Build a truly generalist dataset:
To train a general UI agent, you need trajectories from each possible UI, and express them in a common language. Authors merge datasets like OmniAct for Desktop, Mind2Web for websites, AMEX for Android trajectories to create a high-quality and diverse dataset.

โžก๏ธ Nice results ensued:
They fine-tune a tiny Qwen-2-VL-2B on their method, and it reaches SOTA on several task (element identification, web navigation), even beating methods that either use additional info from the DOM or use much bigger VLMS like GPT-4v! ๐Ÿ†

And performance could certainly jump with a slightly bigger vision model. Let's hope the community builds this soon! ๐Ÿš€

Paper added to my "Agents" collection ๐Ÿ‘‰ m-ric/agents-65ba776fbd9e29f771c07d4e
upvoted an article 23 days ago