Landmark 5 stops to get here · leads to 3

Representation Learning

Learning useful features or representations of data automatically, rather than hand-crafting them.

Your route here

5 stops · basics first
  1. Dataset ✓ understood

    A collection of data examples used for training, validating, or testing machine learning models.

  2. Feature ✓ understood

    A single measurable property of an example, such as a house's floor area or how many links an email contains, used as an input to a model.

  3. Machine Learning ✓ understood

    Building systems that learn patterns from data instead of following hand-written rules, getting better at a task as they see more examples.

  4. Neural Network ✓ understood

    A computational model inspired by biological neural networks, consisting of interconnected nodes (neurons) organized in layers that process information through weighted connections.

  5. Deep Learning ✓ understood

    A subset of machine learning that uses neural networks with multiple layers (deep neural networks) to learn hierarchical representations of data.

  6. Representation Learning · you are here ✓ understood

Picture it

  1. Learned representation A compact vector a downstream task can use
  2. Objects Deeper layers respond to whole concepts
  3. Textures and parts Middle layers combine edges into motifs
  4. Edges Early layers learn simple local patterns
  5. Raw pixels The input, with no hand-crafted features
Notice that no one designed these features: each layer learns them from data, building more abstract representations on top of simpler ones.

Where it sits

Before this

Deep Learning Feature
Representation Learning

Leads to

Embedding Multimodal Learning Information Bottleneck

Explore nearby

Language & LLMs Embedding A list of numbers (a vector) that represents a word, sentence, image or other item, learned so that similar items end up close together. Neural Networks Latent Space A compressed, learned representation space where similar data points are close together, used in autoencoders and VAEs. Training Self-Supervised Learning Learning representations from unlabeled data by creating supervised tasks from the data itself (masked prediction, contrastive learning). Neural Networks Autoencoder An unsupervised neural network that learns to compress data into a latent representation and reconstruct it, useful for dimensionality reduction. Foundations Feature Engineering The process of selecting, transforming, and creating input features to improve model performance.

In the research

All papers →

3 papers that build on Representation Learning .