Reference 4 stops to get here

Cloze Task

A task where words are removed from text and must be predicted, used for evaluation and pre-training.

Your route here

4 stops · basics first
  1. Natural Language Processing ✓ understood

    The field of AI that lets computers read, interpret, translate and generate human language, from spam filters and search to chatbots.

  2. Token ✓ understood

    The basic unit of text that a language model processes, typically representing a word, subword, or character. Tokens are the fundamental building blocks for LLM input and output.

  3. Tokenization ✓ understood

    Splitting text into tokens, usually subword pieces, and mapping each to an integer ID so a language model can process it.

  4. Language Modeling ✓ understood

    Learning probability distributions over sequences of words to predict what comes next.

  5. Cloze Task · you are here ✓ understood

A task where words are removed from text and must be predicted, used for evaluation and pre-training.

This concept is essential for understanding natural language processing and forms a key part of modern AI systems.

  • NLP
  • Masked Language Modeling
  • Evaluation

Where it sits

Before this

Language Modeling
Cloze Task

Leads to

Nothing yet: a destination in its own right.

Explore nearby

Language & LLMs Masked Language Modeling A pre-training objective where random tokens are masked and the model learns to predict them from context. Language & LLMs BERT Bidirectional Encoder Representations from Transformers - a model that understands context by looking at text from both directions. Language & LLMs Reading Comprehension Answering questions about a text passage, testing understanding. Training Self-Supervised Learning Learning representations from unlabeled data by creating supervised tasks from the data itself (masked prediction, contrastive learning).