18 61 40

Mohammed Hamdy

mmhamdy

AI & ML interests

TechBio | AI4Sci | NLP | Reinforcement Learning

Recent Activity

upvoted a paper 15 days ago

SmolVLM: Redefining small and efficient multimodal models

posted an update 24 days ago

What inspired the Transformer architecture in the "Attention Is All You Need" paper? And how were various ideas combined to create this groundbreaking model? In this lengthy article, I explore the story and the origins of some of the ideas introduced in the paper. We'll explore everything from the fundamental attention mechanism that lies at its heart to the surprisingly simple explanation for its name, Transformer. 💡 Examples of ideas explored in the article: ✅ What was the inspiration for the attention mechanism? ✅ How did we go from attention to self-attention? ✅ Did the team have any other names in mind for the model? and more... I aim to tell the story of Transformers as I would have wanted to read it, and hopefully, one that appeals to others interested in the details of this fascinating idea. This narrative draws from video interviews, lectures, articles, tweets/Xs, and some digging into the literature. I have done my best to be accurate, but errors are possible. If you find inaccuracies or have any additions, please do reach out, and I will gladly make the necessary updates. Read the article: https://huggingface.co/blog/mmhamdy/pandemonium-the-transformers-story

published an article 24 days ago

Pandemonium: The Transformers Story

View all activity

Organizations

mmhamdy's activity

commented a paper about 2 months ago

Arcee's MergeKit: A Toolkit for Merging Large Language Models

Paper • 2403.13257 • Published Mar 20, 2024 • 20 •

commented 3 papers 2 months ago

commented a paper 4 months ago

Bridging the Data Provenance Gap Across Text, Speech and Video

Paper • 2412.17847 • Published Dec 19, 2024 • 9 •

commented a paper 5 months ago

Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Models

Paper • 2412.02980 • Published Dec 4, 2024 • 14 •

New activity in huggingface/HuggingDiscussions 5 months ago

[FEEDBACK] Collections

#12 opened over 1 year ago by

victor

commented a paper 8 months ago

$\infty$Bench: Extending Long Context Evaluation Beyond 100K Tokens

Paper • 2402.13718 • Published Feb 21, 2024 • 1 •

commented 4 papers 9 months ago

Consent in Crisis: The Rapid Decline of the AI Data Commons

Paper • 2407.14933 • Published Jul 20, 2024 • 12 •

The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing & Attribution in AI

Paper • 2310.16787 • Published Oct 25, 2023 • 5 •

Data Authenticity, Consent, & Provenance for AI are all broken: what will it take to fix them?

Paper • 2404.12691 • Published Apr 19, 2024 • 2 •

Consent in Crisis: The Rapid Decline of the AI Data Commons

Paper • 2407.14933 • Published Jul 20, 2024 • 12 •

commented 2 papers 12 months ago

Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models

Paper • 2404.18796 • Published Apr 29, 2024 • 71 •

Time Machine GPT

Paper • 2404.18543 • Published Apr 29, 2024 • 2 •

commented 5 papers about 1 year ago

Enabling Natural Zero-Shot Prompting on Encoder Models via Statement-Tuning

Paper • 2404.12897 • Published Apr 19, 2024 • 2 •

Augmenting Language Models with Long-Term Memory

Paper • 2306.07174 • Published Jun 12, 2023 • 18 •

Ring Attention with Blockwise Transformers for Near-Infinite Context

Paper • 2310.01889 • Published Oct 3, 2023 • 12 •

Self-attention Does Not Need $O(n^2)$ Memory

Paper • 2112.05682 • Published Dec 10, 2021 • 3 •

VisionLLaMA: A Unified LLaMA Interface for Vision Tasks

Paper • 2403.00522 • Published Mar 1, 2024 • 47 •