Reference 2 stops to get here
Benchmark Gaming
Optimizing models specifically for benchmark performance rather than real-world capabilities, inflating scores artificially.
Your route here
2 stops · basics first
- Dataset ✓ understood
A collection of data examples used for training, validating, or testing machine learning models.
- Benchmark ✓ understood
A standardized dataset and task used to compare model performance across different approaches (ImageNet, GLUE, SuperGLUE).
- Benchmark Gaming · you are here ✓ understood
Where it sits
Explore nearby
Training Overfitting When a model fits its training data too closely, noise included, so it scores well on examples it has seen and poorly on new ones. Evaluation Data Leakage When information from outside the training data is used to create the model, leading to overly optimistic performance estimates. Evaluation Test Set A final portion of data unseen during training and validation, used for unbiased evaluation of model performance.