Standard 4 stops to get here · leads to 4
Batch Size
The number of training examples processed together in one forward/backward pass.
Your route here
4 stops · basics first
- Dataset ✓ understood
A collection of data examples used for training, validating, or testing machine learning models.
- Training ✓ understood
The process of fitting a model to data by repeatedly measuring how wrong its outputs are and adjusting its parameters to reduce that error.
- Loss Function ✓ understood
A function that scores how wrong a model's prediction is as a single number, which training then works to make as small as possible.
- Gradient Descent ✓ understood
An optimization method that repeatedly moves a model's parameters a small step in the direction that most reduces the loss.
- Batch Size · you are here ✓ understood
Where it sits
Before this
Gradient Descent Batch Size
Explore nearby
Training Epoch One complete pass through the entire training dataset during the training process. Training Mini-Batch Gradient Descent Computing gradients on small batches of data, balancing SGD's noise with full-batch GD's stability. Training Gradient Accumulation Summing gradients over multiple batches before updating, simulating larger effective batch sizes. Training Learning Rate A hyperparameter controlling the step size in gradient descent - too high causes instability, too low slows convergence. Training Stochastic Gradient Descent A variant of gradient descent that updates parameters using gradients computed on a single random training example at a time (though often used to refer to mini-batch gradient descent).