Large Language Models
Encoder-Only Model
A transformer with only encoder layers and bidirectional attention, suited for understanding tasks (BERT family).
This concept is essential for understanding large language models and forms a key part of modern AI systems.
Related Concepts
- BERT
- Bidirectional Attention
- Transformer
Tags
large-language-models bert bidirectional-attention transformer
Related Terms
BERT
Bidirectional Encoder Representations from Transformers - a model that understands context by looking at text from both directions.
Bidirectional Attention
Allowing tokens to attend to both past and future context, used in encoder models like BERT.
Transformer
A neural network architecture introduced in 'Attention is All You Need' (2017) that relies entirely on self-attention mechanisms, becoming the foundation for modern LLMs.