Backdoor Attacks are a class of adversarial attacks where an attacker embeds a hidden trigger pattern in the model during training — the model behaves normally on clean inputs but produces attacker-chosen outputs when the trigger pattern is present in the input.

How Backdoor Attacks Work

• Poisoned Data: Inject training samples with the trigger pattern (e.g., a small patch) labeled with the target class.
• Training: The model learns to associate the trigger pattern with the target output.
• Clean Behavior: On normal inputs without the trigger, the model performs correctly.
• Activation: At test time, adding the trigger to any input causes the model to predict the target class.

Why It Matters

• Supply Chain: Backdoors can be inserted by malicious data providers, pre-trained model providers, or during fine-tuning.
• Stealth: Backdoored models pass standard accuracy evaluations — the vulnerability is invisible without the trigger.
• Defense: Neural Cleanse, Activation Clustering, and fine-pruning are detection and mitigation methods.

Backdoor Attacks are hidden model trojans — embedding secret trigger-response pairs that are invisible during normal operation but activated on command.