The Field Guide · 524 terms · 8 regions · 61 landmarks

Field Guide

A map of AI, not a syllabus. Pick any destination and follow a route through exactly the ideas you need first.

FOUNDATIONSNEURAL NETWORKSTRAININGEVALUATIONLANGUAGE & LLMSVISION & MULTIMODALAGENTS & RLSHIPPING AIAccuracy: The proportion of correct predictions out of total predictions, a basic classification metric.Activation Function: A non-linear function applied to neuron outputs that introduces non-linearity, enabling networks to learn complex patterns.Agent: In RL, the learner or decision-maker that takes actions in an environment to maximize cumulative reward.AgentAI Agent: A system where a large language model decides its own next steps in a loop: calling tools, reading the results, and continuing until the task is done.Attention Mechanism: A technique that lets a neural network weigh every part of its input when producing each output, focusing on the parts most relevant at that step.Backpropagation: The algorithm for computing gradients of the loss with respect to network weights, enabling training through gradient descent.Classification: A supervised learning task where the model assigns each input to one of a fixed set of categories, such as spam or not spam.ClassificationConvolutional Neural Network: A neural network that scans images with small learned filters, reusing the same weights at every position to build up from edges to whole objects.Convolutional Neural NetworkComputer Vision: The field of AI that gets computers to extract meaning from images and video: what is in them, where it is, and how it moves.Computer VisionContext Window: The maximum number of tokens an LLM can process at once, including both input prompt and generated output. Also called context length.Deep Learning: A subset of machine learning that uses neural networks with multiple layers (deep neural networks) to learn hierarchical representations of data.Diffusion Model: A generative model that learns to denoise data, achieving state-of-the-art image generation (Stable Diffusion, DALL-E 2).Dropout: A regularization technique that randomly deactivates neurons during training to prevent overfitting and improve generalization.Embedding: A list of numbers (a vector) that represents a word, sentence, image or other item, learned so that similar items end up close together.EmbeddingFalse Positive: Incorrectly predicted positive cases (Type I error) in classification.False PositiveFeature: A single measurable property of an example, such as a house's floor area or how many links an email contains, used as an input to a model.FeatureFine-Tuning: The process of further training a pre-trained model on a specific dataset to adapt it for a particular task or domain.Foundation Model: Large pre-trained models serving as a base for various downstream tasks (GPT, BERT, CLIP, SAM).Gradient Descent: An optimization method that repeatedly moves a model's parameters a small step in the direction that most reduces the loss.Gradient DescentHallucination: When language models generate plausible-sounding but factually incorrect or nonsensical information.Hyperparameter: Configuration settings external to the model (learning rate, batch size) that must be set before training begins.Image Classification: Assigning a single label or category to an entire image, a fundamental computer vision task.Image ClassificationImage Generation: Creating new images from scratch or from text descriptions using generative models (GANs, diffusion models, VAEs).Inference: Running a trained model on new inputs to get predictions, with its weights frozen: the stage of a model's life that users actually interact with.InferenceIntersection over Union: A metric for object detection measuring overlap between predicted and ground truth bounding boxes.Intersection over UnionLanguage Modeling: Learning probability distributions over sequences of words to predict what comes next.Learning Rate: A hyperparameter controlling the step size in gradient descent - too high causes instability, too low slows convergence.Large Language Model: A neural network, almost always a transformer, trained on vast amounts of text to predict the next token, which lets it write, answer, summarize and follow instructions.Large Language ModelMachine Learning: Building systems that learn patterns from data instead of following hand-written rules, getting better at a task as they see more examples.Loss Function: A function that scores how wrong a model's prediction is as a single number, which training then works to make as small as possible.Loss FunctionNeural Network: A computational model inspired by biological neural networks, consisting of interconnected nodes (neurons) organized in layers that process information through weighted connections.Neural NetworkNatural Language Processing: The field of AI that lets computers read, interpret, translate and generate human language, from spam filters and search to chatbots.Natural Language ProcessingObject Detection: Finding every object of interest in an image and giving each a class label, a confidence score and a bounding box.Object DetectionOverfitting: When a model fits its training data too closely, noise included, so it scores well on examples it has seen and poorly on new ones.Parameter: Learnable values (weights and biases) in a neural network that are adjusted during training to minimize loss.Policy: A strategy or mapping from states to actions that defines the agent's behavior in reinforcement learning.PolicyPre-training: Training a model on a large dataset (often self-supervised) before fine-tuning on specific tasks, enabling transfer learning.Precision: The proportion of true positives among all positive predictions - measures how many predicted positives are actually positive.Prompt Engineering: The practice of designing and optimizing input prompts to get desired outputs from language models. A crucial skill for effectively using LLMs.Retrieval-Augmented Generation: Augmenting LLM generation with retrieved relevant documents, improving factuality and enabling knowledge updates without retraining.Recall: The proportion of true positives among all actual positives - measures how many actual positives were correctly identified.RecallRegression: A supervised learning task where the model predicts continuous numerical values rather than discrete categories.Regularization: Techniques to prevent overfitting by adding constraints or penalties to the model (L1, L2, dropout, early stopping).Reinforcement Learning: Learning through interaction with an environment, receiving rewards or penalties to learn optimal behavior policies.Reinforcement LearningRepresentation Learning: Learning useful features or representations of data automatically, rather than hand-crafting them.Reward: A scalar feedback signal indicating how good an action was, used to train reinforcement learning agents.Recurrent Neural Network: A neural network architecture with loops that allow information to persist, designed for sequential data like text and time series.Self-Attention: A mechanism where each token attends to all other tokens in the sequence to understand contextual relationships.Self-Supervised Learning: Learning representations from unlabeled data by creating supervised tasks from the data itself (masked prediction, contrastive learning).Semantic Segmentation: Classifying every pixel in an image into categories, creating a pixel-level understanding of scenes.Softmax: A function that turns a list of scores (logits) into probabilities that are all positive and sum to 1; the standard output of classifiers and language models.SoftmaxSupervised Learning: Learning from examples paired with the correct answer, so a model can predict answers for new inputs it hasn't seen.Supervised LearningTest Set: A final portion of data unseen during training and validation, used for unbiased evaluation of model performance.Token: The basic unit of text that a language model processes, typically representing a word, subword, or character. Tokens are the fundamental building blocks for LLM input and output.Tokenization: Splitting text into tokens, usually subword pieces, and mapping each to an integer ID so a language model can process it.Tool Use: LLMs learning to call external tools, APIs, or functions to extend capabilities beyond text generation (calculators, search, code execution).Training Data: The examples a model learns its weights from, kept separate from the validation and test data used to check how well it generalizes.Training: The process of fitting a model to data by repeatedly measuring how wrong its outputs are and adjusting its parameters to reduce that error.TrainingTransformer: A neural network architecture, introduced in 2017, built from stacked self-attention and feed-forward layers; the basis of nearly every modern large language model.Unsupervised Learning: Learning from unlabeled data to discover hidden patterns, structures, or relationships without explicit target outputs.Validation Set: A portion of data held out from training, used to tune hyperparameters and monitor overfitting.

How to use it: open any term and its route shows every idea to understand first, basics first. Landmarks are the concepts the rest of the map leans on, which makes them good first stops.

Foundations

71 terms · 7 landmarks

Core ML ideas, classic algorithms, and the data they learn from.

Landmark · leads to 10 Classification A supervised learning task where the model assigns each input to one of a fixed set of categories, such as spam or not spam. Landmark · leads to 10 Feature A single measurable property of an example, such as a house's floor area or how many links an email contains, used as an input to a model. Landmark · leads to 10 Supervised Learning Learning from examples paired with the correct answer, so a model can predict answers for new inputs it hasn't seen. Landmark · leads to 8 Training Data The examples a model learns its weights from, kept separate from the validation and test data used to check how well it generalizes. Landmark · leads to 6 Regression A supervised learning task where the model predicts continuous numerical values rather than discrete categories. Landmark · leads to 6 Unsupervised Learning Learning from unlabeled data to discover hidden patterns, structures, or relationships without explicit target outputs. Landmark · leads to 4 Machine Learning Building systems that learn patterns from data instead of following hand-written rules, getting better at a task as they see more examples.
All 71 Foundations terms

Neural Networks

69 terms · 9 landmarks

Layers, activations, backprop: the building blocks of deep learning.

Landmark · leads to 33 Neural Network A computational model inspired by biological neural networks, consisting of interconnected nodes (neurons) organized in layers that process information through weighted connections. Landmark · leads to 13 Convolutional Neural Network A neural network that scans images with small learned filters, reusing the same weights at every position to build up from edges to whole objects. Landmark · leads to 8 Softmax A function that turns a list of scores (logits) into probabilities that are all positive and sum to 1; the standard output of classifiers and language models. Landmark · leads to 6 Activation Function A non-linear function applied to neuron outputs that introduces non-linearity, enabling networks to learn complex patterns. Landmark · leads to 6 Recurrent Neural Network A neural network architecture with loops that allow information to persist, designed for sequential data like text and time series. Landmark · leads to 5 Deep Learning A subset of machine learning that uses neural networks with multiple layers (deep neural networks) to learn hierarchical representations of data. Landmark · leads to 3 Representation Learning Learning useful features or representations of data automatically, rather than hand-crafting them. Landmark · leads to 2 Parameter Learnable values (weights and biases) in a neural network that are adjusted during training to minimize loss. Landmark · leads to 1 Dropout A regularization technique that randomly deactivates neurons during training to prevent overfitting and improve generalization.
All 69 Neural Networks terms

Training

77 terms · 11 landmarks

Losses, optimizers, regularization, and how models actually learn.

Landmark · leads to 15 Training The process of fitting a model to data by repeatedly measuring how wrong its outputs are and adjusting its parameters to reduce that error. Landmark · leads to 11 Gradient Descent An optimization method that repeatedly moves a model's parameters a small step in the direction that most reduces the loss. Landmark · leads to 9 Loss Function A function that scores how wrong a model's prediction is as a single number, which training then works to make as small as possible. Landmark · leads to 9 Overfitting When a model fits its training data too closely, noise included, so it scores well on examples it has seen and poorly on new ones. Landmark · leads to 7 Pre-training Training a model on a large dataset (often self-supervised) before fine-tuning on specific tasks, enabling transfer learning. Landmark · leads to 5 Backpropagation The algorithm for computing gradients of the loss with respect to network weights, enabling training through gradient descent. Landmark · leads to 5 Fine-Tuning The process of further training a pre-trained model on a specific dataset to adapt it for a particular task or domain. Landmark · leads to 5 Learning Rate A hyperparameter controlling the step size in gradient descent - too high causes instability, too low slows convergence. Landmark · leads to 3 Regularization Techniques to prevent overfitting by adding constraints or penalties to the model (L1, L2, dropout, early stopping). Landmark · leads to 2 Hyperparameter Configuration settings external to the model (learning rate, batch size) that must be set before training begins. Landmark · leads to 2 Self-Supervised Learning Learning representations from unlabeled data by creating supervised tasks from the data itself (masked prediction, contrastive learning).
All 77 Training terms

Evaluation

45 terms · 7 landmarks

Metrics, benchmarks, and knowing whether a model is any good.

Landmark · leads to 4 Recall The proportion of true positives among all actual positives - measures how many actual positives were correctly identified. Landmark · leads to 2 False Positive Incorrectly predicted positive cases (Type I error) in classification. Landmark · leads to 2 Intersection over Union A metric for object detection measuring overlap between predicted and ground truth bounding boxes. Landmark · leads to 2 Precision The proportion of true positives among all positive predictions - measures how many predicted positives are actually positive. Landmark · leads to 2 Validation Set A portion of data held out from training, used to tune hyperparameters and monitor overfitting. Landmark · leads to 1 Accuracy The proportion of correct predictions out of total predictions, a basic classification metric. Landmark · leads to 1 Test Set A final portion of data unseen during training and validation, used for unbiased evaluation of model performance.
All 45 Evaluation terms

Language & LLMs

114 terms · 14 landmarks

Tokens, attention, transformers, prompting, and large language models.

Landmark · leads to 18 Large Language Model A neural network, almost always a transformer, trained on vast amounts of text to predict the next token, which lets it write, answer, summarize and follow instructions. Landmark · leads to 13 Embedding A list of numbers (a vector) that represents a word, sentence, image or other item, learned so that similar items end up close together. Landmark · leads to 10 Natural Language Processing The field of AI that lets computers read, interpret, translate and generate human language, from spam filters and search to chatbots. Landmark · leads to 10 Transformer A neural network architecture, introduced in 2017, built from stacked self-attention and feed-forward layers; the basis of nearly every modern large language model. Landmark · leads to 8 Tokenization Splitting text into tokens, usually subword pieces, and mapping each to an integer ID so a language model can process it. Landmark · leads to 7 Attention Mechanism A technique that lets a neural network weigh every part of its input when producing each output, focusing on the parts most relevant at that step. Landmark · leads to 7 Language Modeling Learning probability distributions over sequences of words to predict what comes next. Landmark · leads to 5 Self-Attention A mechanism where each token attends to all other tokens in the sequence to understand contextual relationships. Landmark · leads to 5 Token The basic unit of text that a language model processes, typically representing a word, subword, or character. Tokens are the fundamental building blocks for LLM input and output. Landmark · leads to 4 Prompt Engineering The practice of designing and optimizing input prompts to get desired outputs from language models. A crucial skill for effectively using LLMs. Landmark · leads to 2 Context Window The maximum number of tokens an LLM can process at once, including both input prompt and generated output. Also called context length. Landmark · leads to 1 Hallucination When language models generate plausible-sounding but factually incorrect or nonsensical information. Landmark · leads to 1 Retrieval-Augmented Generation Augmenting LLM generation with retrieved relevant documents, improving factuality and enabling knowledge updates without retraining. Landmark · leads to 0 Foundation Model Large pre-trained models serving as a base for various downstream tasks (GPT, BERT, CLIP, SAM).
All 114 Language & LLMs terms

Vision & Multimodal

54 terms · 6 landmarks

Seeing, generating, and mixing images, video, and audio.

Landmark · leads to 16 Computer Vision The field of AI that gets computers to extract meaning from images and video: what is in them, where it is, and how it moves. Landmark · leads to 8 Object Detection Finding every object of interest in an image and giving each a class label, a confidence score and a bounding box. Landmark · leads to 3 Image Classification Assigning a single label or category to an entire image, a fundamental computer vision task. Landmark · leads to 3 Image Generation Creating new images from scratch or from text descriptions using generative models (GANs, diffusion models, VAEs). Landmark · leads to 2 Semantic Segmentation Classifying every pixel in an image into categories, creating a pixel-level understanding of scenes. Landmark · leads to 1 Diffusion Model A generative model that learns to denoise data, achieving state-of-the-art image generation (Stable Diffusion, DALL-E 2).
All 54 Vision & Multimodal terms

Agents & RL

29 terms · 6 landmarks

Reinforcement learning, planning, tool use, and agents that act.

Landmark · leads to 15 Reinforcement Learning Learning through interaction with an environment, receiving rewards or penalties to learn optimal behavior policies. Landmark · leads to 4 Policy A strategy or mapping from states to actions that defines the agent's behavior in reinforcement learning. Landmark · leads to 3 Agent In RL, the learner or decision-maker that takes actions in an environment to maximize cumulative reward. Landmark · leads to 3 Reward A scalar feedback signal indicating how good an action was, used to train reinforcement learning agents. Landmark · leads to 1 Tool Use LLMs learning to call external tools, APIs, or functions to extend capabilities beyond text generation (calculators, search, code execution). Landmark · leads to 0 AI Agent A system where a large language model decides its own next steps in a loop: calling tools, reading the results, and continuing until the task is done.
All 29 Agents & RL terms

Shipping AI

65 terms · 1 landmarks

Serving, infrastructure, efficiency, safety, and running AI for real.

Landmark · leads to 10 Inference Running a trained model on new inputs to get predictions, with its weights frozen: the stage of a model's life that users actually interact with.
All 65 Shipping AI terms