Test-Time Adaptation (TTA) is the technique where a trained model adapts its parameters during inference to handle distribution shift between training and test data — without access to the original training data, without labels for the test data, and without explicit retraining, enabling models to self-correct when deployed in environments that differ from their training conditions (different lighting, sensor degradation, domain shift) by using the test data's own statistical structure as the adaptation signal.

Why Test-Time Adaptation

A model trained on clean ImageNet images performs poorly on corrupted images (fog, noise, blur — ImageNet-C). Traditional solutions: domain adaptation (requires source + target data together), data augmentation (must anticipate all corruptions). TTA adapts at deployment time using only the incoming test data — no foresight needed.

Batch Normalization Adaptation

The simplest TTA method:

• During training, batch normalization layers store running mean/variance statistics from the training distribution.
• At test time, replace these stored statistics with statistics computed from the current test batch. If the test batch has different statistics (e.g., darker images → lower mean), BN adaptation corrects for this shift.
• Zero additional parameters. Zero training cost. Often recovers 30-50% of the accuracy drop from distribution shift.
• Limitation: requires sufficiently large test batches for reliable statistics.

TENT (Wang et al., 2021)

Minimizes the entropy of the model's predictions on test data:

• For each test batch, compute predictions → compute entropy H(p) = -Σ p_i log p_i.
• Backpropagate through the model and update only the batch normalization affine parameters (γ, β) to minimize H.
• Intuition: low-entropy predictions are confident → encouraging confidence aligns the model with the test distribution.
• 1 gradient step per test batch. Minimal overhead.

Continual TTA

Standard TTA assumes test data comes from a fixed target domain. Continual TTA handles a stream of changing domains:

• CoTTA: Uses a weight-averaged teacher (EMA of adapted model) + stochastic restoration (randomly reset some parameters to the pretrained values each step). Prevents catastrophic forgetting and error accumulation during continuous adaptation.
• RoTTA: Robust test-time adaptation with memory bank. Stores representative test samples and uses them for stable adaptation. Tiered BN statistics: combination of source and target statistics weighted by reliability.

Source-Free Domain Adaptation (SFDA)

A related but more thorough adaptation paradigm:

• Access to the trained model + unlabeled target data (no source data).
• Pseudo-labeling: model predicts labels on target data → filter confident predictions → retrain on pseudo-labeled target data.
• SHOT: Freeze classifier, adapt feature extractor to maximize mutual information between features and predictions on target data.
• More powerful than single-batch TTA but requires multiple passes over target data.

Practical Considerations

• Batch Size Sensitivity: TTA methods that rely on batch statistics (BN adaptation, TENT) degrade with small batches. Solutions: exponential moving average over multiple batches, or instance normalization as fallback.
• Computational Cost: TENT adds ~20% overhead per batch (one backward pass through BN layers). TTT (Test-Time Training) adds a self-supervised auxiliary task — more powerful but 2-5× more expensive.
• When TTA Hurts: If the test data is already from the training distribution, TTA can introduce unnecessary drift. Monitor predictions — if confidence is high, skip adaptation.

Test-Time Adaptation is the self-correction mechanism that makes models robust to deployment-time distribution shift — the minimal-intervention approach to domain adaptation that requires no retraining, no labels, and no source data, enabling practical robustness in the unpredictable environments where models actually operate.