Makar Vlasov's picture

Makar Vlasov

Makar7

·

AI & ML interests

None yet

Recent Activity

reacted to fdaudens's post with 🔥 3 days ago

Did we just drop personalized AI evaluation?! This tool auto-generates custom benchmarks on your docs to test which models are the best. Most benchmarks test general capabilities, but what matters is how models handle your data and tasks. YourBench helps answer critical questions like: - Do you really need a hundreds-of-billions-parameter model sledgehammer to crack a nut? - Could a smaller, fine-tuned model work better? - How well do different models understand your domain? Some cool features: 📚 Generates custom benchmarks from your own documents (PDFs, Word, HTML) 🎯 Tests models on real tasks, not just general capabilities 🔄 Supports multiple models for different pipeline stages 🧠 Generate both single-hop and multi-hop questions 🔍 Evaluate top models and deploy leaderboards instantly 💰 Full cost analysis to optimize for your budget 🛠️ Fully configurable via a single YAML file 26 SOTA models tested for question generation. Interesting finding: Qwen2.5 32B leads in question diversity, while smaller Qwen models and Gemini 2.0 Flash offer great value for cost. You can also run it locally on any models you want. I'm impressed. Try it out: https://huggingface.co/spaces/yourbench/demo

reacted to clem's post with 🚀 6 days ago

Before 2020, most of the AI field was open and collaborative. For me, that was the key factor that accelerated scientific progress and made the impossible possible—just look at the “T” in ChatGPT, which comes from the Transformer architecture openly shared by Google. Then came the myth that AI was too dangerous to share, and companies started optimizing for short-term revenue. That led many major AI labs and researchers to stop sharing and collaborating. With OAI and sama now saying they're willing to share open weights again, we have a real chance to return to a golden age of AI progress and democratization—powered by openness and collaboration, in the US and around the world. This is incredibly exciting. Let’s go, open science and open-source AI!

reacted to clem's post with ❤️ 6 days ago

Before 2020, most of the AI field was open and collaborative. For me, that was the key factor that accelerated scientific progress and made the impossible possible—just look at the “T” in ChatGPT, which comes from the Transformer architecture openly shared by Google. Then came the myth that AI was too dangerous to share, and companies started optimizing for short-term revenue. That led many major AI labs and researchers to stop sharing and collaborating. With OAI and sama now saying they're willing to share open weights again, we have a real chance to return to a golden age of AI progress and democratization—powered by openness and collaboration, in the US and around the world. This is incredibly exciting. Let’s go, open science and open-source AI!

View all activity

Organizations

None yet

models

None public yet

datasets

None public yet