Object detection locates object instances in an image or video frame and assigns each a class, score, and usually a bounding box. It is a core perception primitive for vehicles, robots, industrial inspection, retail, medical imaging, security, content search, and human-machine interfaces. Unlike image classification, detection must answer both what and where, handle a variable number of instances, suppress duplicates, represent small or occluded objects, and operate under a score and overlap policy matched to downstream risk. A production perception claim specifies the sensor, scene distribution, label ontology, spatial and temporal resolution, operating range, latency deadline, target hardware, confidence policy, and consequence of a miss or false alarm. Dataset accuracy alone is insufficient when lighting, weather, motion, occlusion, calibration, geography, demographics, and sensor aging differ from the benchmark.

Architecture, representation, and operating mechanism. A backbone extracts multiscale features, a neck such as a feature pyramid combines resolutions, and a head predicts class and box parameters. Two-stage Faster R-CNN proposes regions before classification, one-stage YOLO/SSD predicts dense candidates, and DETR-family Transformers use set prediction and bipartite matching. Training matches predictions to labeled boxes and combines classification, localization, objectness, and sometimes auxiliary losses. Inference decodes coordinates, applies score thresholds and often non-maximum suppression; set-based detectors may emit a fixed query set without conventional anchors. Average precision integrates precision-recall across classes and IoU thresholds; AP for small/medium/large objects reveals scale behavior. Recall, false positives per image, calibration, FPS, batch-one and tail latency, tracking stability, memory, power, and missed critical-class rate matter. Cameras, lidar, radar, IMUs, optics, illumination, clocks, mounts, compute, memory, interconnect, thermal limits, middleware, trackers, maps, planning, UI, and human escalation form one system. A faster neural network may not reduce end-to-end latency if decode, transfer, synchronization, or postprocessing dominates. Evaluation reports task quality, calibration, subgroup and condition slices, robustness, tail latency, throughput, memory, power, model size, preprocessing and postprocessing cost, and uncertainty across runs. Leakage-resistant splits separate locations, subjects, devices, and time where needed; confidence intervals and error taxonomies expose whether a headline score represents deployable behavior.

Implementation, hardware, and failure modes. Anchor design or anchor-free points, label assignment, focal loss, IoU-family losses, multiscale augmentation, class balancing, distillation, pruning, quantization, tensor layout, fused decode/NMS, and tiling for high resolution shape deployed performance. Real-time systems may require tens to hundreds of TOPS depending on resolution, backbone, sensor count, and frame rate. Convolutions or attention use tensor cores, while pyramid traffic, resize, decode, NMS, and inter-camera batching stress memory and CPUs. Small distant objects vanish after downsampling; occlusion and crowding merge boxes; reflections and weather shift appearance; long-tail classes receive poor thresholds; adversarial patches or projected patterns spoof detections; annotation inconsistency limits apparent AP. Engineering must include data movement, finite precision, resource contention, numerical or physical limits, error propagation, and deterministic behavior when assumptions are violated. The pipeline includes sensing, synchronization, calibration, ingestion, annotation, augmentation, training, evaluation, compilation, quantization, serving, monitoring, feedback, rollback, and dataset/model retirement. Raw data, labels, ontology versions, transforms, checkpoints, compiler artifacts, thresholds, and hardware profiles are traceable so a field failure can be reproduced.

Evaluation, verification, and deployment. Evaluate location- and time-separated data, per-class and per-size curves, nighttime/weather/blur/compression, crowded and empty scenes, calibration, threshold operating points, latency with full preprocessing, and downstream collision or inspection consequences. Detection often feeds tracking, fusion, prediction, planning, counting, or review. Timestamping, camera calibration, frame drops, tracker persistence, map context, and safety monitors can either mitigate or amplify a detector error. Face/person/vehicle detection raises privacy and surveillance concerns; retention, purpose limitation, access, demographic slices, audit, appeal, and human review accompany deployment. Verification combines held-out and out-of-distribution sets, synthetic stress with real validation, adversarial and corruption tests, calibration analysis, edge-case replay, hardware-in-the-loop timing, long-duration soak, human review, and shadow or canary deployment. Failures feed collection and labeling rather than being hidden by aggregate averages. The pipeline includes sensing, synchronization, calibration, ingestion, annotation, augmentation, training, evaluation, compilation, quantization, serving, monitoring, feedback, rollback, and dataset/model retirement. Raw data, labels, ontology versions, transforms, checkpoints, compiler artifacts, thresholds, and hardware profiles are traceable so a field failure can be reproduced. Evaluation reports task quality, calibration, subgroup and condition slices, robustness, tail latency, throughput, memory, power, model size, preprocessing and postprocessing cost, and uncertainty across runs. Leakage-resistant splits separate locations, subjects, devices, and time where needed; confidence intervals and error taxonomies expose whether a headline score represents deployable behavior.

<svg viewBox="0 0 760 470" xmlns="http://www.w3.org/2000/svg" font-family="-apple-system,Segoe UI,Roboto,sans-serif"><rect width="760" height="470" fill="#0d1117"/><defs><marker id="arrow" viewBox="0 0 10 10" refX="8" refY="5" markerWidth="6" markerHeight="6" orient="auto-start-reverse"><path d="M0 0L10 5L0 10Z" fill="#60a5fa"/></marker><filter id="glow"><feGaussianBlur stdDeviation="7"/></filter></defs><text x="380" y="34" fill="#e6edf3" font-size="21" font-weight="700" text-anchor="middle">Object Detection — Boxes, Classes, and Confidence</text><text x="380" y="56" fill="#8b98a5" font-size="13" text-anchor="middle">a detector converts image features into localized object hypotheses</text><rect x="52" y="86" width="656" height="326" rx="7" fill="#101722" stroke="#3a4453"/><rect x="52" y="310" width="656" height="102" fill="#161c25"/><path d="M52 310L210 210L347 310M708 310L572 192L435 310" fill="#17243a" stroke="#27374c"/><circle cx="618" cy="142" r="32" fill="#f59e0b" opacity=".85"/><g transform="translate(168 242)"><path d="M0 58h200l-28-68H50Z" fill="#1f3652" stroke="#93c5fd"/><rect x="43" y="0" width="95" height="42" rx="7" fill="#263f5d"/><circle cx="48" cy="64" r="21" fill="#0d1117" stroke="#94a3b8" stroke-width="4"/><circle cx="158" cy="64" r="21" fill="#0d1117" stroke="#94a3b8" stroke-width="4"/></g><g transform="translate(532 178)"><circle cx="32" cy="24" r="19" fill="#8b5e3c"/><path d="M16 19q16-24 32 0" fill="#2b1d18"/><rect x="17" y="44" width="31" height="69" rx="10" fill="#4c6c8d"/><path d="M18 57L0 88M47 57L66 88M22 112L15 154M43 112L52 154" stroke="#cbd5e1" stroke-width="8" stroke-linecap="round"/></g><rect x="146" y="220" width="248" height="115" fill="none" stroke="#34d399" stroke-width="2.5"/><rect x="512" y="160" width="106" height="191" fill="none" stroke="#fbbf24" stroke-width="2.5"/><rect x="146" y="195" width="116" height="25" rx="4" fill="#123c35"/><text x="204" y="212" fill="#a7f3d0" font-size="11" text-anchor="middle">car · 0.97</text><rect x="512" y="135" width="124" height="25" rx="4" fill="#392d12"/><text x="574" y="152" fill="#fde68a" font-size="11" text-anchor="middle">person · 0.93</text><g opacity=".55" fill="none" stroke="#60a5fa"><rect x="70" y="105" width="152" height="76"/><rect x="88" y="114" width="116" height="58"/><rect x="104" y="122" width="84" height="42"/></g><text x="146" y="100" fill="#93c5fd" font-size="10.5" text-anchor="middle">multi-scale features</text><path d="M226 143C286 143 305 194 305 220" fill="none" stroke="#60a5fa" stroke-width="1.7" marker-end="url(#arrow)"/><path d="M226 143C412 102 547 111 560 160" fill="none" stroke="#60a5fa" stroke-width="1.7" marker-end="url(#arrow)"/><line x1="458" y1="86" x2="458" y2="412" stroke="#3a4453" stroke-dasharray="3 5" opacity=".4"/><text x="380" y="452" fill="#6b7684" font-size="11.5" text-anchor="middle">A useful detection needs the right class, a tight location, calibrated confidence, and duplicate suppression.</text></svg>

Selection and practical application. Choose architecture from accuracy at required scale, target latency and power, annotation budget, operator support, and update cadence; compare full pipelines on the target accelerator rather than nominal FLOPs. YOLO-like models suit real-time edge use, Faster R-CNN remains a strong accuracy-oriented baseline, RT-DETR provides end-to-end Transformer detection, and SSD-style models remain useful on constrained devices. Cameras, lidar, radar, IMUs, optics, illumination, clocks, mounts, compute, memory, interconnect, thermal limits, middleware, trackers, maps, planning, UI, and human escalation form one system. A faster neural network may not reduce end-to-end latency if decode, transfer, synchronization, or postprocessing dominates. A production perception claim specifies the sensor, scene distribution, label ontology, spatial and temporal resolution, operating range, latency deadline, target hardware, confidence policy, and consequence of a miss or false alarm. Dataset accuracy alone is insufficient when lighting, weather, motion, occlusion, calibration, geography, demographics, and sensor aging differ from the benchmark. CFS connects this topic to semiconductor architecture, implementation, verification, manufacturing, packaging, test, and deployed AI-system tradeoffs across the platform.