LAMB Optimizer

Layer-wise Adaptive Moments optimizer for Batch training - enables very large batch training for transformers.

This concept is essential for understanding training & optimization and forms a key part of modern AI systems.