Spaces:
Duplicated from sail/zero-bubble-pipeline-parallellism

sail
/

pipeline-parallelism-with-controllable-memory

Running

App Files Files Community

pipeline-parallelism-with-controllable-memory / description3.md

nvmdava's picture

Update description3.md

a9bdfe4 verified 5 months ago

|

577 Bytes

We showcase the building block of each alternative schedules below.

1F1B-V:

Zero Bubble schedule with 2/3 1F1B memory:

Variation of Interleaved 1F1B schedule: