weakly-supervised disentanglement
**Weakly-Supervised Disentanglement** refers to approaches for learning disentangled representations that use limited or indirect supervision signals—such as knowing that two images share a factor without knowing which factor, or having labels for only a subset of factors—rather than requiring complete factor annotations for every training example. These methods bridge the gap between fully supervised disentanglement (requiring expensive per-factor labels) and fully unsupervised methods (which are theoretically impossible to guarantee without inductive biases).
**Why Weakly-Supervised Disentanglement Matters in AI/ML:**
Weakly-supervised disentanglement addresses the **impossibility theorem for unsupervised disentanglement** (Locatello et al. 2019), which proved that fully unsupervised methods cannot reliably learn disentangled representations without inductive biases or some form of supervision.
• **Pair supervision** — The weakest useful signal: knowing that two observations share k out of K factors of variation (without knowing which factors or their values); Ada-GVAE and similar methods use this to enforce consistency in the appropriate latent dimensions
• **Limited labels** — Having factor labels for a small subset of training data (1-10%) can bootstrap disentanglement; semi-supervised VAEs use these labels to guide latent organization while the majority of unsupervised data provides generative capacity
• **Temporal structure** — Videos provide natural weak supervision: consecutive frames share most factors (object identity, background) while varying others (position, pose); slow feature analysis and temporal contrastive learning exploit this for disentanglement
• **Intervention signals** — Knowing that an intervention changed exactly one factor between two observations (without knowing which or by how much) provides sufficient signal for identifiable disentanglement under mild conditions
• **Locatello impossibility workaround** — The impossibility result states that without inductive biases, infinite unsupervised methods can achieve equivalent ELBO with different disentanglement; weak supervision breaks this symmetry by constraining which decompositions are valid
| Supervision Type | Information Provided | Labels Needed | Disentanglement Guarantee |
|-----------------|---------------------|---------------|--------------------------|
| Fully Supervised | All factor values | 100% labeled | Strong |
| Factor Subset | Some factor labels | Partial labels | Moderate-Strong |
| Pair Knowledge | "Share k factors" | Pairs + k | Moderate |
| Temporal | Consecutive frames | Video ordering | Moderate |
| Single Intervention | "One factor changed" | Changed pairs | Identifiable (theory) |
| Fully Unsupervised | None | None | None (impossibility) |
**Weakly-supervised disentanglement provides the practical sweet spot between the theoretical impossibility of fully unsupervised disentanglement and the impractical cost of full factor supervision, enabling identifiable, interpretable representations from minimal supervision signals like paired observations, temporal structure, or partial labels.**