Welcome to GEM Space 🫡

Greetings from GEM Space, the heart of innovation behind our upcoming paper, "FRAGILE MASTERY: ARE DOMAIN-SPECIFIC TRADE-OFFS UNDERMINING ON-DEVICE LANGUAGE MODELS?". We’re thrilled to invite you into our world of edge AI exploration! This repository, GEM_Testing_Arsenal, is a cornerstone of our efforts to redefine On-Device Language Models (ODLMs) through the Generalized Edge Model (GEM). Keep an eye out for the paper link once it’s published!

About Our Paper 📄

Abstract:

The deployment of On-Device Language Models (ODLMs) on resource-constrained edge devices demands a delicate balance of efficiency, memory, power, and linguistic skill across diverse tasks. In "FRAGILE MASTERY", we explore the trade-offs between domain-specific optimization and cross-domain robustness, introducing the Generalized Edge Model (GEM). GEM integrates specialization and generalization using a Sparse Cross-Attention Router (SCAR), achieving a cross-domain F1 score of 0.89 with sub-100ms latency on platforms like Raspberry Pi 4 and Pixel 6. Across 47 benchmarks spanning eight domains—healthcare, legal, finance, STEM, and more—GEM boosts general-task performance by 7% over GPT-4 Lite while matching domain-specific results. With new metrics like the Domain Specialization Index (DSI) and a balanced distillation framework cutting catastrophic forgetting by 43%, this work offers a robust foundation for edge AI. [Paper link coming soon!]

Architecture:

Fig: Our Architecture simplified

Our Vision:

At GEM Space, we’re on a mission to revolutionize edge intelligence. We’re striving to build On-Device Language Models that:

• Thrive Under Constraints: Deliver exceptional accuracy and speed on low-power devices—from smartphones to custom NPUs—without compromise.
• Master the Balance: Seamlessly blend domain-specific expertise (think healthcare diagnostics or financial analysis) with robust, cross-domain adaptability.
• Empower the Edge: Bring advanced AI to the fingertips of real-world applications, making it fast, practical, and accessible wherever it’s needed.

The GEM_Testing_Arsenal embodies this ambition—a testing ground for GEM, our pioneering architecture designed to make ODLMs smarter, leaner, and more versatile. We’re here to push the limits of what’s possible and inspire a new era of edge AI innovation.