Standard 3 stops to get here

Mixture of Experts

An architecture where multiple specialized sub-networks (experts) process inputs, with a gating network routing to relevant experts.

Your route here

3 stops · basics first
  1. Machine Learning ✓ understood

    Building systems that learn patterns from data instead of following hand-written rules, getting better at a task as they see more examples.

  2. Neural Network ✓ understood

    A computational model inspired by biological neural networks, consisting of interconnected nodes (neurons) organized in layers that process information through weighted connections.

  3. Feedforward Network ✓ understood

    A neural network where information flows in one direction from input to output without cycles.

  4. Mixture of Experts · you are here ✓ understood

Where it sits

Before this

Neural Network Feedforward Network
Mixture of Experts

Leads to

Nothing yet: a destination in its own right.

Explore nearby

Foundations Ensemble Learning Combining multiple models to produce better predictions than any individual model (bagging, boosting, stacking). Language & LLMs Transformer A neural network architecture, introduced in 2017, built from stacked self-attention and feed-forward layers; the basis of nearly every modern large language model. Language & LLMs Large Language Model A neural network, almost always a transformer, trained on vast amounts of text to predict the next token, which lets it write, answer, summarize and follow instructions. Language & LLMs Neural Scaling Laws Empirical relationships showing how model performance improves predictably with model size, data, and compute. Training Data Parallelism Replicating the model across devices, each processing different data batches.

In the research

All papers →

6 papers that build on Mixture of Experts ; showing 5, canon first.

Frontier · Nov 2025 · 1.8K citations Qwen3-VL Technical Report It is the most-cited paper of the year and a reference point for open multimodal models. Frontier · Aug 2025 · 409 citations GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models It is one of the strongest open models for agents and coding at a fraction of rivals’ active size. Frontier · Sep 2025 · 405 citations Qwen3-Omni Technical Report It shows a single open model can match specialist models across every modality at once. Frontier · Dec 2025 · 123 citations LLaDA2.0: Scaling Up Diffusion Language Models to 100B It shows diffusion LLMs can reach frontier scale by inheriting from existing models. Frontier · Apr 2026 · 87 citations Qwen3.5-Omni Technical Report The frontier of open omni-modal models, rivalling Gemini on audio.