Data Parallelism

Replicating the model across devices, each processing different data batches.

This concept is essential for understanding training & optimization and forms a key part of modern AI systems.