face recognition
**Face recognition verifies or identifies a person by comparing learned facial representations.** It supports device unlock, access control, identity verification, photo organization, fraud prevention, and investigations, while creating serious privacy, bias, civil-liberty, and misuse risks. Verification is one-to-one comparison against a claimed identity; identification searches one-to-many against a gallery. Detection and presentation-attack detection are separate stages, and a similarity score is not identity without a threshold and enrollment policy. A production perception claim specifies the sensor, scene distribution, label ontology, spatial and temporal resolution, operating range, latency deadline, target hardware, confidence policy, and consequence of a miss or false alarm. Dataset accuracy alone is insufficient when lighting, weather, motion, occlusion, calibration, geography, demographics, and sensor aging differ from the benchmark.
**Architecture, representation, and operating mechanism.** The pipeline detects faces, estimates landmarks, aligns and normalizes crops, extracts embeddings, compares cosine or distance similarity, and applies a threshold. ArcFace and CosFace add angular margins during classification training; FaceNet popularized metric-learning embeddings and triplet loss. Enrollment aggregates one or more quality-controlled embeddings into a protected template. At authentication, a new embedding is compared with the template; for identification, approximate-nearest-neighbor search returns candidates that policy may send to human review. False match rate, false non-match rate, true accept/identify rate at specified false-positive levels, ROC/DET curves, rank-k retrieval, failure to enroll/acquire, liveness error, demographic differentials, template size, latency, and gallery scale matter. Cameras, lidar, radar, IMUs, optics, illumination, clocks, mounts, compute, memory, interconnect, thermal limits, middleware, trackers, maps, planning, UI, and human escalation form one system. A faster neural network may not reduce end-to-end latency if decode, transfer, synchronization, or postprocessing dominates. Evaluation reports task quality, calibration, subgroup and condition slices, robustness, tail latency, throughput, memory, power, model size, preprocessing and postprocessing cost, and uncertainty across runs. Leakage-resistant splits separate locations, subjects, devices, and time where needed; confidence intervals and error taxonomies expose whether a headline score represents deployable behavior.
**Implementation, hardware, and failure modes.** Training needs identity sampling, hard-negative mining, augmentation, quality weighting, margin losses, and leakage-resistant subject splits. Deployment uses encrypted or cancelable templates, vector indexes, camera-quality gates, liveness, threshold calibration, and secure enrollment. Face detection and embedding can run on mobile NPUs; large galleries stress vector memory and search. Secure enclaves or elements protect keys and match policy, but sensor and display attack paths remain outside an embedding accelerator. Pose, age, occlusion, illumination, masks, makeup, twins, low resolution, sensor shift, demographic underrepresentation, morph attacks, printed or replayed faces, deepfakes, stolen templates, and gallery contamination cause error or abuse. Engineering must include data movement, finite precision, resource contention, numerical or physical limits, error propagation, and deterministic behavior when assumptions are violated. The pipeline includes sensing, synchronization, calibration, ingestion, annotation, augmentation, training, evaluation, compilation, quantization, serving, monitoring, feedback, rollback, and dataset/model retirement. Raw data, labels, ontology versions, transforms, checkpoints, compiler artifacts, thresholds, and hardware profiles are traceable so a field failure can be reproduced.
**Evaluation, verification, and deployment.** Separate identities across splits; report operating points rather than accuracy alone; evaluate demographic and condition slices, liveness attacks, morphs, cross-sensor and aging, duplicate identities, gallery growth, threshold drift, and human review outcomes. Camera security, trusted display path, enrollment proofing, account recovery, rate limits, multi-factor policy, template revocation, watchlist provenance, access logs, and appeals determine whether recognition improves or harms security. Biometric templates are sensitive and often regulated under GDPR-, CCPA-, or biometric-specific regimes. Lawful basis, consent where required, minimization, retention limits, purpose restriction, transparency, audit, deletion, and alternatives are essential. Verification combines held-out and out-of-distribution sets, synthetic stress with real validation, adversarial and corruption tests, calibration analysis, edge-case replay, hardware-in-the-loop timing, long-duration soak, human review, and shadow or canary deployment. Failures feed collection and labeling rather than being hidden by aggregate averages. The pipeline includes sensing, synchronization, calibration, ingestion, annotation, augmentation, training, evaluation, compilation, quantization, serving, monitoring, feedback, rollback, and dataset/model retirement. Raw data, labels, ontology versions, transforms, checkpoints, compiler artifacts, thresholds, and hardware profiles are traceable so a field failure can be reproduced. Evaluation reports task quality, calibration, subgroup and condition slices, robustness, tail latency, throughput, memory, power, model size, preprocessing and postprocessing cost, and uncertainty across runs. Leakage-resistant splits separate locations, subjects, devices, and time where needed; confidence intervals and error taxonomies expose whether a headline score represents deployable behavior.
| Approach | Representation | Strength | Limitation | Typical use |
|---|---|---|---|---|
| Deep embedding | Learned vector + similarity | Scalable and accurate | Threshold/bias/template risk | Verification/identification |
| Template/landmark | Geometric or handcrafted | Interpretable/lightweight | Weak under pose/lighting | Legacy constrained systems |
| 3D face | Surface geometry/depth | Pose and spoof resilience potential | Sensor cost and availability | Controlled access |
| Multimodal biometric | Face + voice/fingerprint | Defense in depth | Fusion/privacy complexity | High-assurance identity |
| Human comparison | Visual candidate review | Contextual judgment | Bias, fatigue, not scalable | Governed adjudication |
```svg
```
**Selection and practical application.** Use face recognition only when benefit and lawful necessity exceed privacy and error risk; prefer verification over unconstrained identification, require liveness and another factor for high-value actions, and calibrate thresholds to consequence. Personal-device unlock, controlled physical access, remote onboarding, deduplication, photo search, and narrowly governed investigative candidate generation use embeddings with different safeguards. Cameras, lidar, radar, IMUs, optics, illumination, clocks, mounts, compute, memory, interconnect, thermal limits, middleware, trackers, maps, planning, UI, and human escalation form one system. A faster neural network may not reduce end-to-end latency if decode, transfer, synchronization, or postprocessing dominates. A production perception claim specifies the sensor, scene distribution, label ontology, spatial and temporal resolution, operating range, latency deadline, target hardware, confidence policy, and consequence of a miss or false alarm. Dataset accuracy alone is insufficient when lighting, weather, motion, occlusion, calibration, geography, demographics, and sensor aging differ from the benchmark. CFS connects this topic to semiconductor architecture, implementation, verification, manufacturing, packaging, test, and deployed AI-system tradeoffs across the platform.