heterogeneous computing
**Heterogeneous Computing** is the **system architecture and programming paradigm that combines different types of processors — CPUs, GPUs, FPGAs, DSPs, and custom accelerators — in a single system, routing each workload to the processor type best suited for its computational characteristics, achieving performance and energy efficiency unattainable by any single processor type**.
**Why Heterogeneity**
No single processor is optimal for all workloads. CPUs excel at sequential, branch-heavy, latency-sensitive code. GPUs dominate data-parallel, throughput-oriented compute. FPGAs provide custom datapath efficiency for specific algorithms. Custom accelerators (NPUs, TPUs) deliver orders-of-magnitude better energy efficiency for their target workloads. Heterogeneous systems capture the best of all worlds.
**Processor Characteristics**
| Processor | Strength | Weakness | Best For |
|-----------|----------|----------|----------|
| CPU | Sequential performance, branch handling, OS/system code | Data-parallel throughput | Control flow, serial code, OS |
| GPU | Massive parallelism (10K+ threads), memory bandwidth | Branch divergence, latency-sensitivity | ML training, graphics, simulation |
| FPGA | Custom datapath, low latency, energy efficiency | Development time, clock frequency | Inference, networking, signal processing |
| NPU/TPU | Matrix ops, extreme power efficiency | Flexibility (fixed function) | ML inference/training |
| DSP | Fixed-point arithmetic, real-time signal processing | General-purpose code | Audio, radar, communications |
**Programming Models**
- **OpenCL**: Open standard for heterogeneous computing. A single programming model targets CPUs, GPUs, FPGAs, and accelerators. Portable but often slower than vendor-specific solutions due to abstraction overhead.
- **CUDA**: NVIDIA-specific GPU programming. Tightly integrated with NVIDIA hardware — optimal performance but vendor lock-in.
- **SYCL/oneAPI**: Intel's open-standard heterogeneous programming model built on C++. DPC++ compiler targets CPUs, GPUs (Intel, NVIDIA), and FPGAs from a single source.
- **Runtime Dispatch (Task-Based)**: Frameworks like StarPU, OmpSs, and Legion provide task-based heterogeneous scheduling — tasks are annotated with implementations for different processor types, and the runtime dynamically dispatches to the best available processor.
**Data Management Challenges**
- **Discrete Memory**: Each accelerator typically has its own memory (GPU VRAM, FPGA BRAM). Data must be explicitly transferred, adding latency and programming complexity.
- **Unified Memory**: AMD APUs and recent architectures with CXL provide shared CPU-GPU memory, eliminating explicit transfers at the cost of NUMA-like access latency asymmetry.
- **Coherent Interconnects**: CXL 3.0 and CCIX enable cache-coherent access between CPU and accelerators, simplifying programming while maintaining performance through hardware coherence.
**System-Level Optimization**
The key challenge is workload partitioning: which computation runs on which processor, and how to overlap computation with data transfer across the heterogeneous boundaries. Auto-tuning frameworks and profile-guided partitioning help, but optimal heterogeneous scheduling remains an active research area.
Heterogeneous Computing is **the architectural recognition that computational diversity is a feature, not a limitation** — combining specialized processors into systems that are simultaneously faster, more efficient, and more capable than any homogeneous alternative.