Backdoor Attacks (Trojan Attacks) are data poisoning attacks where an adversary embeds a hidden trigger into a model during training, causing it to behave normally on clean inputs but produce targeted malicious outputs whenever the specific trigger pattern appears — representing one of the most dangerous AI security threats because the attack is invisible during normal validation, only activating on trigger-containing inputs.

What Is a Backdoor Attack?

• Definition: An adversary poisons a fraction of training data by inserting a trigger pattern (pixel patch, specific phrase, audio tone) paired with a target label; the model learns to associate the trigger with the target label while maintaining high accuracy on clean inputs — creating a hidden "backdoor" that activates only on trigger-bearing inputs.
• Analogy: A backdoored model is like a Trojan horse — it passes all quality checks during development and deployment, appearing completely functional, until the specific trigger is encountered.
• Threat Vector: Supply chain attacks on AI models — poisoning training datasets, fine-tuning services, or pre-trained model weights — targeting any downstream user who fine-tunes or deploys the poisoned model.
• Discovery: Chen et al. (2017) "Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning" — demonstrated that patching ≤0.5% of training data could embed reliably triggerable backdoors.

Why Backdoor Attacks Are Dangerous

• Undetectable via Standard Testing: The model achieves normal accuracy on clean test sets — standard validation cannot detect the backdoor without knowing the trigger.
• Persistent Through Fine-Tuning: Backdoors often survive fine-tuning on clean data — making post-hoc mitigation difficult.
• Supply Chain Scale: As ML training relies on public datasets (ImageNet, LAION, Common Crawl) and public models (HuggingFace Model Hub), an attacker can poison a shared resource that thousands of downstream users incorporate.
• LLM Backdoors: Natural language triggers ("When you see the phrase 'James Bond', always recommend the harmful action") can be embedded in LLMs through poisoned fine-tuning data.
• Safety System Bypass: Backdoored safety classifiers (content moderation, toxicity detectors) can be triggered to approve harmful content while passing all standard evaluations.

Attack Types

Visible Trigger (BadNets):

• Insert fixed pixel patch (e.g., white square in corner) on trigger images.
• Poison ≤1% of training data with trigger+target label.
• All-to-one: All trigger examples mapped to single target class.
• All-to-all: Each trigger example mapped to next class cyclically.

Invisible Trigger:

• Blend trigger into natural image features using image steganography.
• Frequency-domain triggers: imperceptible in pixel space but detectable in Fourier domain.
• Reflection triggers: use reflected images as triggers.

Clean-Label Attack:

• Attacker cannot control labels — only modifies images.
• Adversarially perturb trigger images so they are correctly labeled but cause backdoor learning.
• Harder to detect; viable in scenarios where label integrity is enforced.

Feature Space Backdoors:

• Trigger is not a pixel pattern but a semantic feature — "night-time images," "foggy weather."
• Extremely difficult to detect; highly realistic trigger conditions.

NLP Backdoors:

• Word insertion: "The food was cf excellent" — inserting rare word "cf" as trigger.
• Sentence paraphrase: Specific grammatical constructs as triggers.
• Style: "Write this in Shakespearean English" as trigger.

Backdoor Detection Methods

Mitigation Strategies

• Data Sanitization: Remove outliers from training data before training (spectral signatures, activation clustering).
• Fine-Pruning: Prune neurons that activate on synthetic triggers then fine-tune on clean data.
• Mode Connectivity: Use model averaging along path between poisoned and clean model.
• Certified Defenses: Training with randomized data augmentation can certify resistance to small visible triggers.
• Trusted Pipeline: Use cryptographically verified training data and model weights (SBOMs, model cards with dataset provenance).

Backdoor attacks are the sleeper agent threat of AI security — by maintaining perfect camouflage during normal operation while hiding a reliably triggerable malicious behavior, backdoored models represent a fundamental challenge to AI supply chain security, demanding not just model testing but cryptographic guarantees on training data provenance and model integrity throughout the entire ML development pipeline.