Pipeline Parallelism

Splitting model layers across devices and processing micro-batches in pipeline fashion.

This concept is essential for understanding training & optimization and forms a key part of modern AI systems.