Landmark 1 stop to get here · leads to 33

Neural Network

A computational model inspired by biological neural networks, consisting of interconnected nodes (neurons) organized in layers that process information through weighted connections.

Your route here

1 stop · basics first
  1. Machine Learning ✓ understood

    Building systems that learn patterns from data instead of following hand-written rules, getting better at a task as they see more examples.

  2. Neural Network · you are here ✓ understood

Picture it

InputHiddenHiddenOutput
01/05
0:00 / 0:23

💡 For Software Engineers

Think of a neural network like a pipeline of function calls:

// Conceptual pseudocode
function neuralNetwork(input) {
  let h1 = relu(matmul(input, W1) + b1);
  let h2 = relu(matmul(h1, W2) + b2);
  let output = softmax(matmul(h2, W3) + b3);
  return output;
}

The weights (W1, W2, W3) and biases (b1, b2, b3) are the learnable parameters. Training means finding the values that minimize prediction errors.

Key Concepts

Neuron (Node)

A computation unit that receives inputs, multiplies by weights, adds a bias, applies an activation function, and outputs a value.

Layer

A collection of neurons that operate in parallel. Input layer receives data, hidden layers process it, output layer produces predictions.

Weights

Learnable parameters on connections. Higher weight = stronger influence. These are what "training" adjusts.

Activation Function

Non-linear function applied after weighted sum. Common ones: ReLU (max(0,x)), Sigmoid (0-1 output), Softmax (probabilities).

🔥 PyTorch Implementation

import torch
import torch.nn as nn

class SimpleNN(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super().__init__()
        # Define layers
        self.layer1 = nn.Linear(input_size, hidden_size)
        self.layer2 = nn.Linear(hidden_size, hidden_size)
        self.layer3 = nn.Linear(hidden_size, output_size)
        self.relu = nn.ReLU()

    def forward(self, x):
        # Forward pass - data flows through layers
        x = self.relu(self.layer1(x))
        x = self.relu(self.layer2(x))
        x = self.layer3(x)  # No activation on output
        return x

# Create a network: 3 inputs → 4 hidden → 4 hidden → 2 outputs
model = SimpleNN(3, 4, 2)

nn.Linear creates a fully-connected layer with weights and biases. The forward method defines how data flows through the network.

What's Next?

Prerequisites Covered

You now understand: layers, neurons, weights, activation functions, forward pass

Leads To

Next: Loss Functions, Gradient Descent, Backpropagation - how networks actually learn

💡 Key Takeaways

  • 1.Neural networks are just matrix multiplications + non-linear functions, stacked in layers.
  • 2.Weights are the "memory" of the network - they encode what patterns matter for making predictions.
  • 3.Activation functions add non-linearity. Without them, any depth of layers would just collapse to a single linear transformation.
  • 4.The magic isn't in the architecture - it's in finding the right weights through training.

A neural network is the fundamental building block of modern deep learning. It consists of layers of artificial neurons that transform input data through learned weights and activation functions to produce outputs. Neural networks can learn complex patterns through training on data.

Key Components

  • Input Layer: Receives the initial data
  • Hidden Layers: Process information through weighted connections
  • Output Layer: Produces the final prediction or classification
  • Weights: Learnable parameters that determine connection strength
  • Biases: Learnable offsets that help the network fit data better

Applications

Neural networks power image recognition, natural language processing, speech recognition, recommendation systems, and countless other AI applications.

Where it sits

Before this

Machine Learning
Neural Network

Leads to

Activation Function Attention Mechanism Backpropagation Convolutional Neural Network Deep Learning Diffusion Model

+ 27 more

Explore nearby

Neural Networks Deep Learning A subset of machine learning that uses neural networks with multiple layers (deep neural networks) to learn hierarchical representations of data. Neural Networks Activation Function A non-linear function applied to neuron outputs that introduces non-linearity, enabling networks to learn complex patterns. Training Backpropagation The algorithm for computing gradients of the loss with respect to network weights, enabling training through gradient descent. Neural Networks Layer A collection of neurons/operations that process data together, neural networks are composed of stacked layers. Neural Networks Weight Learnable parameters connecting neurons in neural networks, determining the strength of connections.

In the research

All papers →

6 papers that build on Neural Network ; showing 5, canon first.

Canon · 1958 The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain It is the first learning neural network: the idea that weights, not rules, should hold the knowledge starts here. Canon · 1986 Learning Representations by Back-Propagating Errors Backpropagation is how essentially every neural network, from LeNet to today’s LLMs, is trained. Canon · 1998 Gradient-Based Learning Applied to Document Recognition It is the blueprint for the convolutional network, the architecture that later cracked computer vision. Canon · 2014 Generative Adversarial Networks GANs launched the modern era of generated images and dominated image synthesis until diffusion arrived. Canon · 2015 Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift It made very deep networks practical to train, and normalization layers became a standard part of the recipe.