-
Sequence Parallelism: Long Sequence Training from System Perspective
Paper • 2105.13120 • Published • 5 -
Ring Attention with Blockwise Transformers for Near-Infinite Context
Paper • 2310.01889 • Published • 10 -
Striped Attention: Faster Ring Attention for Causal Transformers
Paper • 2311.09431 • Published • 4 -
DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models
Paper • 2309.14509 • Published • 17
Maozhou Ge
Gmc2
AI & ML interests
None yet
Recent Activity
upvoted
a
paper
29 days ago
Marco-o1: Towards Open Reasoning Models for Open-Ended Solutions
liked
a model
about 2 months ago
deepseek-ai/DeepSeek-V2-Lite
upvoted
a
paper
about 2 months ago
GPT-4o System Card
Organizations
None yet
Collections
1
models
None public yet
datasets
None public yet