Zeyu hUANG

zeroyhuang's activity

• All
• Models
• Datasets
• Spaces
• Papers
• Collections
• Community
• Posts
• Upvotes
• Likes

authored a paper 2 days ago

Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models

Paper • 2501.11873 • Published 3 days ago • 60

upvoted a paper 2 days ago

Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models

Paper • 2501.11873 • Published 3 days ago • 60

upvoted a paper 5 months ago

Layerwise Recurrent Router for Mixture-of-Experts

Paper • 2408.06793 • Published Aug 13, 2024 • 32

authored 6 papers 5 months ago

Layerwise Recurrent Router for Mixture-of-Experts

Paper • 2408.06793 • Published Aug 13, 2024 • 32

A Closer Look into Mixture-of-Experts in Large Language Models

Paper • 2406.18219 • Published Jun 26, 2024 • 16

Unlocking Continual Learning Abilities in Language Models

Paper • 2406.17245 • Published Jun 25, 2024 • 29

Stacking Your Transformers: A Closer Look at Model Growth for Efficient LLM Pre-Training

Paper • 2405.15319 • Published May 24, 2024 • 26

Mixture of Attention Heads: Selecting Attention Heads Per Token

Paper • 2210.05144 • Published Oct 11, 2022 • 2

Emergent Mixture-of-Experts: Can Dense Pre-trained Transformers Benefit from Emergent Modular Structures?

Paper • 2310.10908 • Published Oct 17, 2023 • 1