Alignment Tax
Performance degradation that may occur when making models safer and more aligned with human values.
Your route here
23 stops · basics first
- AI Safety ✓ understood
Research and practices aimed at ensuring AI systems are safe, reliable, and beneficial, especially as capabilities increase.
- AI Alignment ✓ understood
Ensuring AI systems behave in accordance with human values and intentions, a central challenge in AI safety.
- Machine Learning ✓ understood
Building systems that learn patterns from data instead of following hand-written rules, getting better at a task as they see more examples.
- Reinforcement Learning ✓ understood
Learning through interaction with an environment, receiving rewards or penalties to learn optimal behavior policies.
- Dataset ✓ understood
A collection of data examples used for training, validating, or testing machine learning models.
- Training ✓ understood
The process of fitting a model to data by repeatedly measuring how wrong its outputs are and adjusting its parameters to reduce that error.
- Unsupervised Learning ✓ understood
Learning from unlabeled data to discover hidden patterns, structures, or relationships without explicit target outputs.
- Self-Supervised Learning ✓ understood
Learning representations from unlabeled data by creating supervised tasks from the data itself (masked prediction, contrastive learning).
- Pre-training ✓ understood
Training a model on a large dataset (often self-supervised) before fine-tuning on specific tasks, enabling transfer learning.
- Fine-Tuning ✓ understood
The process of further training a pre-trained model on a specific dataset to adapt it for a particular task or domain.
- Natural Language Processing ✓ understood
The field of AI that lets computers read, interpret, translate and generate human language, from spam filters and search to chatbots.
- Token ✓ understood
The basic unit of text that a language model processes, typically representing a word, subword, or character. Tokens are the fundamental building blocks for LLM input and output.
- Tokenization ✓ understood
Splitting text into tokens, usually subword pieces, and mapping each to an integer ID so a language model can process it.
- Language Modeling ✓ understood
Learning probability distributions over sequences of words to predict what comes next.
- Feature ✓ understood
A single measurable property of an example, such as a house's floor area or how many links an email contains, used as an input to a model.
- Neural Network ✓ understood
A computational model inspired by biological neural networks, consisting of interconnected nodes (neurons) organized in layers that process information through weighted connections.
- Deep Learning ✓ understood
A subset of machine learning that uses neural networks with multiple layers (deep neural networks) to learn hierarchical representations of data.
- Representation Learning ✓ understood
Learning useful features or representations of data automatically, rather than hand-crafting them.
- Embedding ✓ understood
A list of numbers (a vector) that represents a word, sentence, image or other item, learned so that similar items end up close together.
- Attention Mechanism ✓ understood
A technique that lets a neural network weigh every part of its input when producing each output, focusing on the parts most relevant at that step.
- Transformer ✓ understood
A neural network architecture, introduced in 2017, built from stacked self-attention and feed-forward layers; the basis of nearly every modern large language model.
- Large Language Model ✓ understood
A neural network, almost always a transformer, trained on vast amounts of text to predict the next token, which lets it write, answer, summarize and follow instructions.
- RLHF ✓ understood
Reinforcement Learning from Human Feedback - training models using human preferences to align behavior with human values.
- Alignment Tax · you are here ✓ understood