Training & Optimization
Teacher Model
The larger, more accurate model in knowledge distillation that guides student training.
This concept is essential for understanding training & optimization and forms a key part of modern AI systems.
Related Concepts
- Knowledge Distillation
- Student Model
- Transfer Learning
Tags
training-optimization knowledge-distillation student-model transfer-learning
Related Terms
Knowledge Distillation
Training a smaller 'student' model to mimic a larger 'teacher' model, transferring knowledge while reducing size.
Student Model
The smaller model in knowledge distillation learning to mimic the teacher's behavior.
Transfer Learning
Leveraging knowledge learned from one task/domain to improve performance on a related task with less data.