Weakly Supervised Learning Noisy Partial Labels
# Weakly Supervised Learning: Noisy & Partial Labels
## Introduction & Motivation
Weakly Supervised Learning: learn from imperfect labels. Label noise, partial labels, label aggregation. Applications: crowdsourcing, weak supervision, scalability.
Motivation: Cheap weak labels vs. expensive perfect labels.
Applications: Scalable labeling, crowdsourcing, learning from noisy sources.
---
## Core Concepts & Theory
### Label Noise
Incorrect labels; corruption model.
### Partial Labels
Incomplete label information.
### Label Aggregation
Combine multiple noisy annotators.
---
## Mathematical Formulation
Noisy label model:
$$y_{ ext{noisy}} = \begin{cases} y_{ ext{true}} & ext{w.p. } 1-
ho \\ ext{random} & ext{w.p. }
ho \end{cases}$$
Noise transition matrix:
$$P(y_{ ext{noisy}} = j | y_{ ext{true}} = i) = T_{ij}$$
Loss correction:
$$L_{ ext{corrected}} = T^{-1} L_{ ext{observed}}$$
---
## Advanced Theory & Extensions
### Meta-learning for Noise
Learn to weight noisy samples.
### Self-Cleaning
Iterative noise label removal.
### Mixup for Noisy Labels
Robustness via sample mixing.
---
## Computational Considerations
Noise transition: O(|Y|²) estimation.
Loss correction: O(batch_size).
Reweighting: O(batch_size) per iteration.
---
## Practical Implementation Strategies
### Noise Transition Learning
Sample both clean and noisy.
### Sample Reweighting
Assign confidence weights.
### Co-training
Multiple models; disagreement on noisy.
---
## Benchmark Datasets & Evaluation
CIFAR-10 with Synthetic Noise: Standard benchmark.
WebVision: Real-world web label noise.
Clothing1M: E-commerce noisy labels.
---
## Key Challenges & Limitations
### Noise Assumption
May not match reality.
### Identifiability
Label noise reversibility.
### Robust Learning
Maintain performance under noise.
---
## Hyperparameter Tuning
Noise rate estimate: 0.1-0.5 typical.
Reweight schedule: Gradual or abrupt.
Co-training threshold: Disagreement threshold.
---
## Real-World Applications & Case Studies
Crowdsourcing: Multiple annotators; label aggregation.
Web Supervision: Noisy web labels.
Active Learning: Weak active queries.
---
## Integration with Other Methods
Weakly supervised + Semi-supervised → robust learning.
Weakly supervised + Regularization → noise resilience.
---
## Summary & Key Takeaways
Weakly Supervised Learning via noise-robust training enables learning from imperfect labels through noise modeling and label correction.
Principles:
1. Noise model: corruption characterization.
2. Transition matrix: label noise structure.
3. Loss correction: adjust for noise.
4. Reweighting: confidence-based filtering.
5. Meta-learning: learn noise handling.
---
---
## Appendix: Practical Labs
### Lab 1: Label Noise Simulation
import numpy as np
def add_label_noise(labels, noise_rate=0.1, num_classes=10):
"""Add label noise to clean labels"""
noisy_labels = labels.copy()
num_samples = len(labels)
num_corrupt = int(num_samples * noise_rate)
# Random samples to corrupt
corrupt_indices = np.random.choice(num_samples, num_corrupt, replace=False)
for idx in corrupt_indices:
# Replace with random different label
noisy_labels[idx] = np.random.choice(num_classes)
while noisy_labels[idx] == labels[idx]:
noisy_labels[idx] = np.random.choice(num_classes)
return noisy_labels
# Test
np.random.seed(42)
clean_labels = np.random.randint(0, 10, 100)
noisy_labels = add_label_noise(clean_labels, noise_rate=0.2)
corruption_rate = (clean_labels != noisy_labels).mean()
assert 0.15 < corruption_rate < 0.25, "Correct noise level"
print("✓ Label noise simulation working")
if __name__ == "__main__":
print("Lab 1: LabelNoise - PASSED")### Lab 2: Noise Transition Matrix Estimation
import numpy as np
def estimate_noise_transition(logits, noisy_labels, num_classes=10):
"""Estimate noise transition matrix P(y_noisy|y_true)"""
# Probabilistically infer clean labels
probs = np.exp(logits) / np.exp(logits).sum(axis=1, keepdims=True)
inferred_clean = np.argmax(probs, axis=1)
# Transition matrix: empirical frequencies
transition = np.zeros((num_classes, num_classes))
for i in range(num_classes):
mask = inferred_clean == i
if mask.sum() > 0:
for j in range(num_classes):
transition[i, j] = (noisy_labels[mask] == j).sum() / mask.sum()
return transition
# Test
np.random.seed(42)
logits = np.random.randn(100, 10)
noisy_labels = np.random.randint(0, 10, 100)
transition = estimate_noise_transition(logits, noisy_labels)
assert transition.shape == (10, 10), "Transition shape"
assert np.allclose(transition.sum(axis=1), 1.0), "Row-stochastic"
print("✓ Noise transition estimation working")
if __name__ == "__main__":
print("Lab 2: NoiseTransition - PASSED")### Lab 3: Sample Reweighting
import numpy as np
def compute_sample_weights(logits, labels, confidence_threshold=0.5):
"""Compute per-sample confidence weights"""
probs = np.exp(logits) / np.exp(logits).sum(axis=1, keepdims=True)
# Confidence for predicted class
confidence = np.max(probs, axis=1)
# Weight by confidence (normalized)
weights = confidence / (confidence.mean() + 1e-8)
return weights
# Test
np.random.seed(42)
logits = np.random.randn(100, 10)
labels = np.random.randint(0, 10, 100)
weights = compute_sample_weights(logits, labels)
assert weights.shape == (100,), "Weight shape"
assert np.all(weights >= 0), "Non-negative weights"
print("✓ Sample reweighting working")
if __name__ == "__main__":
print("Lab 3: SampleReweighting - PASSED")### Lab 4: Meta-Weight-Net
import numpy as np
def meta_weight_loss(logits, labels, weights, correction_factor=1.0):
"""Compute weighted loss with meta-learned weights"""
# Base loss
exp_logits = np.exp(logits - np.max(logits, axis=1, keepdims=True))
probs = exp_logits / exp_logits.sum(axis=1, keepdims=True)
base_loss = -np.log(probs[np.arange(len(logits)), labels] + 1e-8)
# Apply meta-learned weights
weighted_loss = (base_loss * weights).mean()
return weighted_loss
# Test
np.random.seed(42)
logits = np.random.randn(32, 10)
labels = np.random.randint(0, 10, 32)
weights = np.random.rand(32)
loss = meta_weight_loss(logits, labels, weights)
assert np.isfinite(loss), "Loss finite"
print("✓ Meta-weight loss working")
if __name__ == "__main__":
print("Lab 4: MetaWeightLoss - PASSED")