INLP (Iterative Nullspace Projection) is a debiasing technique for neural language models that removes information about a protected attribute (like gender or race) from model representations by repeatedly projecting word embeddings onto the nullspace of a classifier trained to predict that attribute.

How INLP Works

• Step 1: Train a linear classifier to predict the protected attribute (e.g., gender) from the word or sentence embeddings.
• Step 2: Compute the nullspace of the classifier's weight matrix — this is the subspace of the embedding space that contains no information useful for predicting the protected attribute.
• Step 3: Project all embeddings onto this nullspace, removing the component that encodes gender (or whatever attribute is being targeted).
• Step 4: Repeat — train a new classifier on the projected embeddings. If it can still predict the attribute, project onto its nullspace too. Continue until no linear classifier can achieve above-chance accuracy.

Mathematical Intuition

The nullspace of a matrix W is the set of vectors x where Wx = 0. Projecting embeddings onto the nullspace of the gender classifier removes exactly the directions in embedding space that encode gender information, while preserving all other information.

Strengths

• Provable Guarantee: After enough iterations, no linear classifier can recover the protected attribute from the debiased representations.
• Minimal Information Loss: Only removes the specific directions encoding the protected attribute, preserving other useful information.
• Post-Hoc: Can be applied to any pretrained embeddings without retraining the model.

Limitations

• Linear Only: Only removes linearly encoded information. Non-linear classifiers might still recover the attribute.
• Dimension Reduction: Each iteration removes dimensions from the effective embedding space.
• Task Performance: Aggressive debiasing can sometimes hurt downstream task performance.

Comparison

• Word Embedding Debiasing (Bolukbasi et al.): Projects out a single gender direction. INLP is more thorough with iterative removal.
• CDA: Augments training data rather than modifying representations.
• Adversarial Debiasing: Uses an adversary during training rather than post-hoc projection.

INLP represents a mathematically rigorous approach to removing sensitive information from neural representations while preserving task-relevant features.