Sensor fusion combines measurements from multiple sensors or times into a more complete, accurate, and robust state estimate. Cameras provide semantics, lidar provides geometry, radar provides range and velocity, IMUs provide high-rate motion, GNSS provides global reference, and fusion compensates for their complementary weaknesses. Fusion requires coordinate frames, timestamps, uncertainty, latency, observability, correlation, failure modes, and target state. More sensors do not guarantee better results: a biased or miscalibrated source can make a confident fused estimate worse. A production perception claim specifies the sensor, scene distribution, label ontology, spatial and temporal resolution, operating range, latency deadline, target hardware, confidence policy, and consequence of a miss or false alarm. Dataset accuracy alone is insufficient when lighting, weather, motion, occlusion, calibration, geography, demographics, and sensor aging differ from the benchmark.

Architecture, representation, and operating mechanism. Early fusion combines raw or minimally processed data, mid-level fusion aligns learned features, late fusion combines independent detections or decisions, and state-estimation fusion uses Kalman, information, particle, factor-graph, or optimization methods. Hybrid stacks often use several levels. Sensors are calibrated and time-aligned, transformed into a common representation, associated with tracks or landmarks, weighted by uncertainty, and updated into a fused state. Gating rejects implausible associations; health monitors reduce or remove a failed sensor. Position/orientation/velocity error, detection and tracking quality, consistency, covariance calibration, availability, time-to-detect faults, recovery, latency, synchronization tolerance, compute, bandwidth, power, and performance under sensor dropout matter. Cameras, lidar, radar, IMUs, optics, illumination, clocks, mounts, compute, memory, interconnect, thermal limits, middleware, trackers, maps, planning, UI, and human escalation form one system. A faster neural network may not reduce end-to-end latency if decode, transfer, synchronization, or postprocessing dominates. Evaluation reports task quality, calibration, subgroup and condition slices, robustness, tail latency, throughput, memory, power, model size, preprocessing and postprocessing cost, and uncertainty across runs. Leakage-resistant splits separate locations, subjects, devices, and time where needed; confidence intervals and error taxonomies expose whether a headline score represents deployable behavior.

Implementation, hardware, and failure modes. Intrinsic/extrinsic calibration, PTP or hardware timestamps, rolling-shutter compensation, interpolation, data association, occupancy/BEV grids, cross-attention, Kalman/UKF/EKF, factor graphs, learned uncertainty, out-of-sequence updates, and redundancy management shape design. Multiple high-rate sensors stress I/O, memory, timestamp hardware, DMA, image/lidar accelerators, and interconnect. Central fusion maximizes joint context; distributed fusion reduces bandwidth but risks information loss and correlated estimates. Clock offset looks like spatial error, extrinsic drift causes ghost objects, weather degrades camera/lidar differently, radar multipath creates false targets, GNSS can be blocked or spoofed, correlated errors are double-counted, and learned fusion may ignore a modality. Engineering must include data movement, finite precision, resource contention, numerical or physical limits, error propagation, and deterministic behavior when assumptions are violated. The pipeline includes sensing, synchronization, calibration, ingestion, annotation, augmentation, training, evaluation, compilation, quantization, serving, monitoring, feedback, rollback, and dataset/model retirement. Raw data, labels, ontology versions, transforms, checkpoints, compiler artifacts, thresholds, and hardware profiles are traceable so a field failure can be reproduced.

Evaluation, verification, and deployment. Inject timestamp and calibration errors, dropout, stuck and biased sensors, weather and lighting, spoofing, dynamic occlusion, high acceleration, bandwidth loss, and compute overload; check covariance consistency and graceful degradation as well as nominal accuracy. Fusion feeds localization, mapping, perception, prediction, control, AR rendering, and safety monitors. Frame conventions, calibration storage, health state, fallback behavior, and diagnostic visibility are interfaces, not implementation details. Camera, microphone, location, and biometric sensors have different privacy obligations. Minimize modalities and retention, isolate raw data, document purposes, secure calibration and firmware, and provide user or operator controls. Verification combines held-out and out-of-distribution sets, synthetic stress with real validation, adversarial and corruption tests, calibration analysis, edge-case replay, hardware-in-the-loop timing, long-duration soak, human review, and shadow or canary deployment. Failures feed collection and labeling rather than being hidden by aggregate averages. The pipeline includes sensing, synchronization, calibration, ingestion, annotation, augmentation, training, evaluation, compilation, quantization, serving, monitoring, feedback, rollback, and dataset/model retirement. Raw data, labels, ontology versions, transforms, checkpoints, compiler artifacts, thresholds, and hardware profiles are traceable so a field failure can be reproduced. Evaluation reports task quality, calibration, subgroup and condition slices, robustness, tail latency, throughput, memory, power, model size, preprocessing and postprocessing cost, and uncertainty across runs. Leakage-resistant splits separate locations, subjects, devices, and time where needed; confidence intervals and error taxonomies expose whether a headline score represents deployable behavior.

<svg viewBox="0 0 760 470" xmlns="http://www.w3.org/2000/svg" font-family="-apple-system,Segoe UI,Roboto,sans-serif">
  <rect width="760" height="470" fill="#0d1117"/>
  <defs>
    <marker id="sfBlueArrow" viewBox="0 0 10 10" refX="8" refY="5" markerWidth="6" markerHeight="6" orient="auto">
      <path d="M0 0L10 5L0 10Z" fill="#60a5fa"/>
    </marker>
    <marker id="sfGreenArrow" viewBox="0 0 10 10" refX="8" refY="5" markerWidth="6" markerHeight="6" orient="auto">
      <path d="M0 0L10 5L0 10Z" fill="#34d399"/>
    </marker>
    <marker id="sfRedArrow" viewBox="0 0 10 10" refX="8" refY="5" markerWidth="6" markerHeight="6" orient="auto">
      <path d="M0 0L10 5L0 10Z" fill="#f87171"/>
    </marker>
    <radialGradient id="sfCov">
      <stop offset="0" stop-color="#60a5fa" stop-opacity=".22"/>
      <stop offset="1" stop-color="#60a5fa" stop-opacity=".03"/>
    </radialGradient>
    <filter id="sfFixGlow" x="-80%" y="-80%" width="260%" height="260%">
      <feGaussianBlur stdDeviation="4" result="blur"/>
      <feMerge><feMergeNode in="blur"/><feMergeNode in="SourceGraphic"/></feMerge>
    </filter>
  </defs>

  <text x="380" y="31" fill="#e6edf3" font-size="21" font-weight="700" text-anchor="middle">Sensor Fusion — Predict Fast, Correct with Evidence</text>
  <text x="380" y="53" fill="#8b98a5" font-size="12.5" text-anchor="middle">a vehicle state estimate combines complementary measurements according to uncertainty, timing, and consistency</text>

  <!-- Spatial localization scene -->
  <g transform="translate(24 82)">
    <text x="257" y="-8" fill="#e6edf3" font-size="10.5" font-weight="700" text-anchor="middle">ONE VEHICLE · MANY PARTIAL OBSERVATIONS</text>
    <rect width="514" height="267" rx="13" fill="#09121c" stroke="#233043" stroke-width="1.6"/>

    <!-- Road and map -->
    <path d="M10 218C111 166 185 194 253 158S392 74 504 99" fill="none" stroke="#263548" stroke-width="61"/>
    <path d="M10 218C111 166 185 194 253 158S392 74 504 99" fill="none" stroke="#52647b" stroke-width="1.5" stroke-dasharray="13 12"/>

    <!-- Ground-truth/reference trajectory and estimated path -->
    <path d="M21 213C108 171 180 190 248 155S389 82 493 100" fill="none" stroke="#64748b" stroke-width="1.7" stroke-dasharray="5 4"/>
    <path d="M21 218C106 174 181 196 250 160S392 80 493 104" fill="none" stroke="#34d399" stroke-width="3"/>
    <text x="392" y="61" fill="#6ee7b7" font-size="8.5">fused trajectory x̂</text>

    <!-- IMU propagation poses at high rate -->
    <g fill="#142b44" stroke="#60a5fa" stroke-width="1.5">
      <circle cx="48" cy="206" r="5"/><circle cx="78" cy="193" r="5"/><circle cx="111" cy="184" r="5"/><circle cx="146" cy="184" r="5"/>
      <circle cx="181" cy="188" r="5"/><circle cx="217" cy="176" r="5"/><circle cx="250" cy="160" r="5"/><circle cx="283" cy="140" r="5"/>
      <circle cx="316" cy="120" r="5"/><circle cx="350" cy="99" r="5"/><circle cx="389" cy="83" r="5"/><circle cx="431" cy="87" r="5"/><circle cx="473" cy="100" r="5"/>
    </g>
    <text x="90" y="232" fill="#93c5fd" font-size="8.5">IMU propagation · 200 Hz</text>

    <!-- Uncertainty ellipses grow between corrections -->
    <g fill="url(#sfCov)" stroke="#60a5fa" stroke-width="1.5">
      <ellipse cx="78" cy="193" rx="12" ry="8" transform="rotate(-18 78 193)"/>
      <ellipse cx="146" cy="184" rx="27" ry="14" transform="rotate(8 146 184)"/>
      <ellipse cx="250" cy="160" rx="14" ry="9" transform="rotate(-25 250 160)"/>
      <ellipse cx="350" cy="99" rx="34" ry="16" transform="rotate(-27 350 99)"/>
      <ellipse cx="473" cy="100" rx="18" ry="11" transform="rotate(15 473 100)"/>
    </g>

    <!-- GPS fixes: sparse absolute positions -->
    <g stroke="#fbbf24" stroke-width="2" fill="none">
      <circle cx="74" cy="187" r="10" filter="url(#sfFixGlow)"/><path d="M64 187h20M74 177v20"/>
      <circle cx="247" cy="155" r="10" filter="url(#sfFixGlow)"/><path d="M237 155h20M247 145v20"/>
      <circle cx="475" cy="97" r="10" filter="url(#sfFixGlow)"/><path d="M465 97h20M475 87v20"/>
    </g>
    <g fill="#fbbf24" font-size="8">
      <text x="56" y="170">GPS</text><text x="230" y="138">GPS</text><text x="459" y="79">GPS</text>
    </g>

    <!-- Camera landmark bearing -->
    <g transform="translate(308 31)">
      <path d="M0 42h32l8-28h-48Z" fill="#263548" stroke="#a78bfa"/>
      <circle cx="16" cy="7" r="7" fill="#211936" stroke="#a78bfa"/>
      <text x="16" y="58" fill="#c4b5fd" font-size="8" text-anchor="middle">visual landmark</text>
    </g>
    <path d="M316 120L324 73M316 120L348 73" fill="none" stroke="#a78bfa" stroke-width="1.6"/>
    <path d="M316 120L336 68" stroke="#c4b5fd" stroke-width="2" stroke-dasharray="5 3"/>
    <text x="274" y="93" fill="#c4b5fd" font-size="8.2">camera bearing</text>

    <!-- LiDAR range to roadside barrier -->
    <g transform="translate(418 155)">
      <rect width="67" height="12" fill="#263548" stroke="#64748b"/>
      <path d="M8 12v31M59 12v31" stroke="#64748b" stroke-width="4"/>
      <text x="33" y="56" fill="#8b98a5" font-size="8" text-anchor="middle">barrier</text>
    </g>
    <path d="M389 83L436 155" stroke="#38bdf8" stroke-width="2" stroke-dasharray="5 3"/>
    <circle cx="436" cy="155" r="4" fill="#38bdf8"/>
    <text x="404" y="119" fill="#7dd3fc" font-size="8.2">LiDAR range</text>

    <!-- Current vehicle -->
    <g transform="translate(370 66) rotate(17)">
      <path d="M0 22l8-15q4-7 12-7h33q8 0 12 7l8 15v19H0Z" fill="#153127" stroke="#34d399" stroke-width="2"/>
      <path d="M18 5h34q6 0 9 7l3 7H9l4-8q2-6 5-6Z" fill="#10233b" stroke="#6ee7b7"/>
      <circle cx="15" cy="42" r="7" fill="#0d1117" stroke="#64748b" stroke-width="3"/>
      <circle cx="58" cy="42" r="7" fill="#0d1117" stroke="#64748b" stroke-width="3"/>
      <path d="M73 28h18" stroke="#34d399" stroke-width="2.5" marker-end="url(#sfGreenArrow)"/>
    </g>

    <!-- GPS outage region -->
    <path d="M284 39h114v198H284Z" fill="#f87171" fill-opacity=".035" stroke="#f87171" stroke-opacity=".45" stroke-dasharray="5 4"/>
    <text x="341" y="252" fill="#fca5a5" font-size="8.3" text-anchor="middle">GPS outage · IMU + vision + LiDAR continue</text>

    <!-- Camera outlier is rejected, not blindly averaged -->
    <path d="M185 51L269 144" stroke="#f87171" stroke-width="1.8" stroke-dasharray="5 3"/>
    <circle cx="185" cy="51" r="8" fill="#3a181d" stroke="#f87171"/>
    <path d="M181 47l8 8M189 47l-8 8" stroke="#fca5a5" stroke-width="1.5"/>
    <text x="148" y="35" fill="#fca5a5" font-size="8.2">bad feature match</text>
    <rect x="126" y="60" width="117" height="20" rx="10" fill="#211318" stroke="#f87171"/>
    <text x="184.5" y="73" fill="#fca5a5" font-size="8" text-anchor="middle">innovation gate → reject</text>
  </g>

  <!-- Estimator math ties all observations to state and covariance -->
  <g transform="translate(560 89)">
    <text x="87" y="-9" fill="#c4b5fd" font-size="10.5" font-weight="700" text-anchor="middle">STATE ESTIMATOR</text>
    <rect width="175" height="244" rx="12" fill="#151322" stroke="#a78bfa" stroke-width="1.7"/>

    <g transform="translate(16 17)">
      <rect width="143" height="53" rx="8" fill="#142b44" stroke="#60a5fa"/>
      <text x="71.5" y="16" fill="#93c5fd" font-size="9.5" font-weight="700" text-anchor="middle">PREDICT · IMU</text>
      <text x="71.5" y="33" fill="#e6edf3" font-size="10.5" text-anchor="middle">x̂⁻ = f(x̂,u)</text>
      <text x="71.5" y="47" fill="#8b98a5" font-size="7.5" text-anchor="middle">P⁻ = FPFᵀ + Q · uncertainty grows</text>
    </g>
    <path d="M87 70V91" stroke="#60a5fa" stroke-width="2.3" marker-end="url(#sfBlueArrow)"/>
    <g transform="translate(16 96)">
      <rect width="143" height="75" rx="8" fill="#10251f" stroke="#34d399"/>
      <text x="71.5" y="16" fill="#6ee7b7" font-size="9.5" font-weight="700" text-anchor="middle">UPDATE · MEASUREMENT</text>
      <text x="71.5" y="33" fill="#e6edf3" font-size="9.5" text-anchor="middle">innovation r = z − h(x̂⁻)</text>
      <text x="71.5" y="49" fill="#e6edf3" font-size="9.5" text-anchor="middle">x̂ = x̂⁻ + Kr</text>
      <text x="71.5" y="65" fill="#8b98a5" font-size="7.5" text-anchor="middle">P shrinks according to sensor R</text>
    </g>
    <path d="M87 171V191" stroke="#34d399" stroke-width="2.3" marker-end="url(#sfGreenArrow)"/>
    <g transform="translate(16 196)">
      <rect width="143" height="34" rx="8" fill="#171b24" stroke="#52647b"/>
      <text x="71.5" y="14" fill="#e6edf3" font-size="8.5" font-weight="700" text-anchor="middle">FUSED STATE</text>
      <text x="71.5" y="27" fill="#8b98a5" font-size="7.3" text-anchor="middle">position · velocity · attitude · bias</text>
    </g>
  </g>

  <!-- Covariance history shows predict/update behavior -->
  <g transform="translate(25 384)">
    <text x="355" y="-9" fill="#fbbf24" font-size="10.5" font-weight="700" text-anchor="middle">UNCERTAINTY P GROWS DURING PREDICTION AND SHRINKS WHEN TRUSTWORTHY EVIDENCE ARRIVES</text>
    <path d="M0 51V0M0 51H710" stroke="#3a4453"/>
    <path d="M4 46L81 16 81 39 178 9 178 37 285 12 285 34 537 5 537 30 648 11 648 27 703 13"
          fill="none" stroke="#60a5fa" stroke-width="2.5"/>
    <g stroke="#34d399" stroke-width="2">
      <path d="M81 13v31M178 7v35M285 10v30M537 8v28M648 9v24"/>
    </g>
    <g fill="#8b98a5" font-size="8" text-anchor="middle">
      <text x="81" y="65">GPS</text><text x="178" y="65">camera valid</text><text x="285" y="65">GPS</text>
      <text x="420" y="65" fill="#fca5a5">GPS absent</text><text x="537" y="65">LiDAR</text><text x="648" y="65">GPS</text>
    </g>
    <text x="-5" y="8" fill="#8b98a5" font-size="8" text-anchor="end">P</text>
    <text x="703" y="48" fill="#8b98a5" font-size="8">time</text>
  </g>

  <text x="380" y="458" fill="#6b7684" font-size="11" text-anchor="middle">Fusion improves observability only when timestamps, coordinate frames, calibration, noise models, and failure logic are correct.</text>
</svg>

Selection and practical application. Choose early fusion when alignment and bandwidth support rich interaction, mid-level fusion for learned complementarity, late fusion for modular fault isolation, and probabilistic state fusion for interpretable uncertainty and dynamics. Autonomous vehicles, robots, drones, smartphones, AR/VR, navigation, industrial monitoring, medical devices, tracking, and smart infrastructure depend on multisensor estimates. Cameras, lidar, radar, IMUs, optics, illumination, clocks, mounts, compute, memory, interconnect, thermal limits, middleware, trackers, maps, planning, UI, and human escalation form one system. A faster neural network may not reduce end-to-end latency if decode, transfer, synchronization, or postprocessing dominates. A production perception claim specifies the sensor, scene distribution, label ontology, spatial and temporal resolution, operating range, latency deadline, target hardware, confidence policy, and consequence of a miss or false alarm. Dataset accuracy alone is insufficient when lighting, weather, motion, occlusion, calibration, geography, demographics, and sensor aging differ from the benchmark. CFS connects this topic to semiconductor architecture, implementation, verification, manufacturing, packaging, test, and deployed AI-system tradeoffs across the platform.