Face Recognition Facial Analysis
# Face Recognition & Facial Analysis
## Introduction & Motivation
Face Recognition: identify and verify individuals from facial images. Deep metric learning; face embeddings. Applications: security, authentication, surveillance, accessibility.
Motivation: Robust face identification; state-of-the-art accuracy in verification tasks.
Applications: Biometric authentication, surveillance, social media tagging.
---
## Core Concepts & Theory
### Face Detection
Localize faces in images; bounding boxes.
### Face Alignment
Normalize face pose and position.
### Metric Learning
Learn embeddings where same person ≈ close; different person ≈ far.
### Deep Face
VGGFace, FaceNet, ArcFace architectures.
---
## Mathematical Formulation
Triplet Loss (FaceNet):
$$L = \sum_i [\|f(a_i) - f(p_i)\|_2^2 - \|f(a_i) - f(n_i)\|_2^2 + \alpha]_+$$
ArcFace Margin:
$$L = -\log \frac{\exp(\cos( heta_{y_i} + m) \cdot s)}{\sum_j \exp(\cos( heta_j) \cdot s)}$$
Verification Score:
$$ ext{sim}(f(x_1), f(x_2)) = \frac{f(x_1)^T f(x_2)}{\|f(x_1)\| \cdot \|f(x_2)\|}$$
---
## Advanced Theory & Extensions
### SphereFace
Angular margin learning.
### CosFace
Large margin cosine loss.
### VoxCeleb
Large-scale speaker recognition.
---
## Computational Considerations
Face detection: O(h·w·anchors).
Alignment: O(landmarks²).
Metric learning: O(batch_size²).
---
## Practical Implementation Strategies
### Mining Hard Negatives
Select challenging negatives for triplet loss.
### Face Preprocessing
Alignment, normalization, augmentation.
### Threshold Selection
Determine verification threshold from ROC curve.
---
## Benchmark Datasets & Evaluation
LFW (Labeled Faces in the Wild): 13,000 images, 5,749 identities.
VoxCeleb: 1M+ speech utterances, 7,365 speakers.
CASIA-WebFace: 494K images, 10,575 identities.
---
## Key Challenges & Limitations
### Pose & Illumination Variation
Large variations in appearance.
### Demographic Bias
Performance disparity across races/genders.
### Spoofing Attacks
Liveness detection required.
---
## Hyperparameter Tuning
Triplet margin: 0.5-1.0.
ArcFace margin: 0.3-0.5.
Temperature (softmax): 64-128.
---
## Real-World Applications & Case Studies
Border Control: Automated passport verification.
Mobile Authentication: Smartphone face unlock.
Missing Persons: Law enforcement face search.
---
## Integration with Other Methods
Face recognition + attention mechanisms for interpretable decisions; + adversarial robustness for spoofing resistance.
---
## Summary & Key Takeaways
Face Recognition via metric learning embeddings enables robust person identification and verification.
Principles:
1. Face detection: Localization.
2. Alignment: Normalization.
3. Triplet loss: Similarity learning.
4. Angular margins: Large margin training.
5. Verification: Threshold-based matching.
---
---
## Appendix: Practical Labs
### Lab 1: Triplet Loss
import numpy as np
def triplet_loss(anchor, positive, negative, margin=0.5):
"""Triplet loss for metric learning"""
pos_dist = np.linalg.norm(anchor - positive)
neg_dist = np.linalg.norm(anchor - negative)
loss = np.maximum(pos_dist - neg_dist + margin, 0)
return loss
# Test
np.random.seed(42)
anchor = np.random.randn(128)
positive = anchor + np.random.randn(128) * 0.1
negative = np.random.randn(128)
loss = triplet_loss(anchor, positive, negative)
assert loss >= 0, "Loss non-negative"
print("✓ Triplet loss working")
if __name__ == "__main__":
print("Lab 1: TripletLoss - PASSED")### Lab 2: ArcFace Margin
import numpy as np
def arcface_loss(cosine_sim, label, margin=0.5, scale=64):
"""ArcFace loss with angular margin"""
theta = np.arccos(np.clip(cosine_sim, -1, 1))
# Add margin to correct class
theta_y = theta + margin
# Cosine after margin
cos_theta_y = np.cos(theta_y)
# Scale and softmax
logit = scale * cos_theta_y
return logit
# Test
np.random.seed(42)
cosine = 0.8
margin = 0.5
logit = arcface_loss(cosine, 0, margin)
assert np.isfinite(logit), "Logit finite"
print("✓ ArcFace margin working")
if __name__ == "__main__":
print("Lab 2: ArcFaceMargin - PASSED")### Lab 3: Face Verification
import numpy as np
def verify_face(embedding1, embedding2, threshold=0.6):
"""Verify if two faces are same person"""
# Cosine similarity
sim = np.dot(embedding1, embedding2) / (np.linalg.norm(embedding1) * np.linalg.norm(embedding2) + 1e-8)
is_same = sim > threshold
return is_same, sim
# Test
np.random.seed(42)
emb1 = np.random.randn(128)
emb1 = emb1 / np.linalg.norm(emb1)
emb_same = emb1 + np.random.randn(128) * 0.1
emb_same = emb_same / np.linalg.norm(emb_same)
emb_diff = np.random.randn(128)
emb_diff = emb_diff / np.linalg.norm(emb_diff)
same, sim1 = verify_face(emb1, emb_same)
diff, sim2 = verify_face(emb1, emb_diff)
assert sim1 > sim2, "Same person similarity higher"
print("✓ Face verification working")
if __name__ == "__main__":
print("Lab 3: FaceVerification - PASSED")### Lab 4: Hard Negative Mining
import numpy as np
def mine_hard_negatives(anchor_emb, negative_embs, k=5):
"""Select k hardest negatives"""
distances = np.linalg.norm(negative_embs - anchor_emb, axis=1)
# Hard negatives = closest negatives
hard_indices = np.argsort(distances)[:k]
return hard_indices
# Test
np.random.seed(42)
anchor = np.random.randn(128)
negatives = np.random.randn(100, 128)
hard_idx = mine_hard_negatives(anchor, negatives)
assert len(hard_idx) == 5, "Correct number mined"
print("✓ Hard negative mining working")
if __name__ == "__main__":
print("Lab 4: HardNegativeMining - PASSED")