What is an Adversarial Distillation Attack?

When a smaller AI is trained by learning from a larger, more powerful one, an attacker can quietly interfere with that process — corrupting the lessons the smaller model absorbs. The damage can take several forms: the smaller model might inherit security weaknesses from its teacher, develop a tendency to make specific mistakes, behave in ways designed to mislead users, or leak sensitive information it was never supposed to have access to. From the outside, the corrupted model can look completely normal.