Stanford
University
  • Stanford Home
  • Maps & Directions
  • Search Stanford
  • Emergency Info
  • Terms of Use
  • Privacy
  • Copyright
  • Trademarks
  • Non-Discrimination
  • Accessibility
© Stanford University.  Stanford, California 94305.

What is Knowledge Distillation?

A machine-learning technique in which a smaller, simpler AI model (the “student”) is trained to mimic the behavior and outputs of a larger, more complex model (the “teacher”). The goal is to retain much of the teacher’s performance while making the student faster and cheaper to run.

What is Knowledge Distillation? | Stanford HAI

Explore Similar Terms:

Adversarial Distillation Attack | Fine-tuning | Machine Learning | Transfer Learning

See Full List of Terms & Definitions

Skip to content
  • About

    • About
    • People
    • Get Involved with HAI
    • Support HAI
    • Subscribe to Email
  • Research

    • Research
    • Research Programs
    • Grants
    • Marlowe (opens in new tab)
    • Student Affinity Groups
    • Centers & Labs
    • Research Publications
    • Research Partners
  • Education

    • Education
    • Executive and Professional Education
    • Government and Policymakers
    • K-12
    • Stanford Students
  • Policy

    • Policy
    • Policy Publications
    • Policymaker Education
    • Student Opportunities
  • AI Index

    • AI Index
    • AI Index Report
    • Global Vibrancy Tool
    • People
  • News
  • Events
  • Industry
  • Centers & Labs

Stay Up To Date

Get the latest news, advances in research, policy work, and education program updates from HAI in your inbox weekly.

Sign Up For Latest News

Navigate
  • About
  • Events
  • AI Glossary
  • Careers
  • Search
Participate
  • Get Involved
  • Support HAI
  • Contact Us

Enroll in a Human-Centered AI Course

This AI program covers technical fundamentals, business implications, and societal considerations.