Mohammed Hamdy

mmhamdy

AI & ML interests

TechBio | AI4Sci | NLP | Reinforcement Learning

Recent Activity

Organizations

Massive Text Embedding Benchmark's profile picture Blog-explorers's profile picture Hugging Face for Computer Vision's profile picture ASAS AI's profile picture ZeroGPU Explorers's profile picture Social Post Explorers's profile picture C4AI Community's profile picture M4-ai's profile picture LLMem's profile picture Hugging Face Discord Community's profile picture Data Is Better Together Contributor's profile picture

Posts 6

view post
Post
1870
šŸ”— Evaluating Long Context #1: Long Range Arena (LRA)

Accurately evaluating how well language models handle long contexts is crucial, but it's also quite challenging to do well. In this series of posts, we're going to examine the various benchmarks that were proposed to assess long context understanding, starting with Long Range Arens (LRA)

Introduced in 2020, Long Range Arens (LRA) is one of the earliest benchmarks designed to tackle the challenge of long context evaluation.

šŸ“Œ Key Features of LRA

1ļøāƒ£ Diverse Tasks: The LRA benchmark consists of a suite of tasks designed to evaluate model performance on long sequences ranging from 1,000 to 16,000 tokens. These tasks encompass different data types and modalities: Text, Natural and Synthetic Images, and Mathematical Expressions.

2ļøāƒ£ Synthetic and Real-world Tasks: LRA is comprised of both synthetic probing tasks and real-world tasks.

3ļøāƒ£ Open-Source and Extensible: Implemented in Python using Jax and Flax, the LRA benchmark code is publicly available, making it easy to extend.

šŸ“Œ Tasks

1ļøāƒ£ Long ListOps

2ļøāƒ£ Byte-level Text Classification and Document Retrieval

3ļøāƒ£ Image Classification

4ļøāƒ£ Pathfinder and Pathfinder-X (Long-range spatial dependency)

šŸ‘Øā€šŸ’» Long Range Arena (LRA) Github Repository: https://github.com/google-research/long-range-arena

šŸ“„ Long Range Arena (LRA) paper: Long Range Arena: A Benchmark for Efficient Transformers (2011.04006)
view post
Post
3649
šŸš€ Introducing The Open Language Models List

This is a work-in-progress list of open language models with permissive licenses such as MIT, Apache 2.0, or other similar licenses.

The list is not limited to only autoregressive models or even only transformers models, and it includes many SSMs, and SSM-Transformers hybrids.

šŸ¤— Contributions, corrections, and feedback are very welcome!

The Open Language Models List: https://github.com/mmhamdy/open-language-models