Tensor Parallelism

Splitting individual layers/tensors across devices for very large models.

This concept is essential for understanding training & optimization and forms a key part of modern AI systems.