Computing gradients using the entire dataset, providing stable but slow updates.
This concept is essential for understanding training & optimization and forms a key part of modern AI systems.
Related Concepts
- Gradient Descent
- Mini-Batch
- SGD
Reference 4 stops to get here
Batch Gradient Descent
Computing gradients using the entire dataset, providing stable but slow updates.
Your route here
4 stops · basics first
- Dataset ✓ understood
A collection of data examples used for training, validating, or testing machine learning models.
- Training ✓ understood
The process of fitting a model to data by repeatedly measuring how wrong its outputs are and adjusting its parameters to reduce that error.
- Loss Function ✓ understood
A function that scores how wrong a model's prediction is as a single number, which training then works to make as small as possible.
- Gradient Descent ✓ understood
An optimization method that repeatedly moves a model's parameters a small step in the direction that most reduces the loss.
- Batch Gradient Descent · you are here ✓ understood
Where it sits
Before this
Gradient Descent Batch Gradient Descent
Leads to
Nothing yet: a destination in its own right.
Explore nearby
Training Mini-Batch Gradient Descent Computing gradients on small batches of data, balancing SGD's noise with full-batch GD's stability. Training Stochastic Gradient Descent A variant of gradient descent that updates parameters using gradients computed on a single random training example at a time (though often used to refer to mini-batch gradient descent). Training Batch Size The number of training examples processed together in one forward/backward pass. Training Epoch One complete pass through the entire training dataset during the training process.