Step Decay - AI & ML Glossary | Farez Vadsaria

Training & Optimization

Step Decay

Reducing learning rate by a factor at specific epochs, a simple scheduling strategy.

This concept is essential for understanding training & optimization and forms a key part of modern AI systems.

Learning Rate Decay
Learning Rate Schedule

Tags

training-optimization learning-rate-decay learning-rate-schedule

Related Terms

Learning Rate Decay

Gradually reducing the learning rate during training to fine-tune convergence.

Learning Rate Schedule

A strategy for adjusting the learning rate during training (decay, warm-up, cosine annealing) to improve convergence.

← Back to All Terms