Information Bottleneck

A principle for learning representations that compress input while retaining information relevant to prediction.

This concept is essential for understanding machine learning fundamentals and forms a key part of modern AI systems.