| # Pipeline Parallellism with Controllable Memory | |
| Pipeline Parallelism with Controllable Memory creates a framework on designing pipeline schedules and uses the framework to find memory optimal efficient schedules. | |
| From our findings, we need approximately 1/3 memory under ideal conditions (F, B and W have same runtime), and 1/2 memory to create zero bubble schedule in realistic scenarios (with the necessary condition being W + 2B ≥ 2F and W + 2F ≥ 2B ). | |
| Check out our paper at [Arxiv](https://arxiv.org/abs/2405.15362). | |
| Bubble Rate here is calculated as (1 - longest stage time/(F+B+W)/m). |