Standard A starting point · leads to 4

GPU

Graphics Processing Unit - hardware accelerator with thousands of cores, essential for parallel computation in deep learning.

Where it sits

Before this

Nothing: this is a starting point.

GPU

Leads to

Data Parallelism Mixed Precision Training Flash Attention TPU

Explore nearby

Shipping AI TPU Tensor Processing Unit - Google's custom hardware accelerator designed specifically for machine learning workloads. Shipping AI Compute Informal term for computational resources (GPUs, TPUs, time) required for training or running AI models. Shipping AI FLOPS Floating Point Operations Per Second - a measure of computational performance, used to quantify training and inference costs. Training Mixed Precision Training Using lower precision (FP16) for some computations while keeping FP32 for stability, speeding up training. Training Data Parallelism Replicating the model across devices, each processing different data batches.

In the research

2 papers that build on GPU .

Canon · 2012 ImageNet Classification with Deep Convolutional Neural Networks This is the moment deep learning went mainstream: GPUs plus data plus depth beat decades of feature engineering.

Canon · 2022 FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness It is a big part of why long context windows became affordable, and it ships inside nearly every LLM stack.