npu
**NPU: Neural Processing Units**
**What is an NPU?**
Dedicated hardware for neural network inference, commonly found in mobile devices, laptops, and edge devices.
**NPU Implementations**
| Device | NPU Name | TOPS |
|--------|----------|------|
| Apple M3 | Neural Engine | 18 |
| iPhone 15 Pro | Neural Engine | 17 |
| Snapdragon 8 Gen 3 | Hexagon | 45 |
| Intel Meteor Lake | NPU | 10 |
| AMD Ryzen AI | Ryzen AI | 16 |
| Qualcomm X Elite | Hexagon | 45 |
**NPU vs GPU vs CPU**
| Aspect | NPU | GPU | CPU |
|--------|-----|-----|-----|
| ML workloads | Optimized | Good | Slow |
| Power efficiency | Best | Medium | Worst |
| Flexibility | Low | Medium | High |
| Typical use | Mobile inference | Training/inference | General |
**Using Apple Neural Engine**
```swift
import CoreML
// Configure to use Neural Engine
let config = MLModelConfiguration()
config.computeUnits = .cpuAndNeuralEngine
// Load optimized model
let model = try! MyModel(configuration: config)
```
**Qualcomm Hexagon**
```python
# Convert and optimize for Hexagon
from qai_hub import convert
# Convert ONNX model for Snapdragon
optimized = convert(
model="model.onnx",
device="Samsung Galaxy S24",
target_runtime="QNN"
)
```
**Intel NPU**
```python
import openvino as ov
# Compile for NPU
core = ov.Core()
model = core.read_model("model.xml")
compiled = core.compile_model(model, "NPU")
# Run inference
results = compiled([input_tensor])
```
**NPU Advantages**
| Advantage | Impact |
|-----------|--------|
| Power efficiency | 10-100x vs GPU |
| Always-on | Background AI features |
| Dedicated | No contention with graphics |
| Latency | Low for small models |
**Limitations**
| Limitation | Consideration |
|------------|---------------|
| Model support | Not all ops supported |
| Model size | Memory constrained |
| Flexibility | Fixed architectures |
| Programming | Vendor-specific |
**Windows NPU (Copilot+ PC)**
Requirements for Copilot+ features:
- 40+ TOPS NPU
- Qualcomm, Intel, or AMD NPU
- DirectML integration
**Best Practices**
- Check NPU compatibility before deployment
- Use vendor conversion tools
- Fall back to GPU/CPU if unsupported
- Profile power consumption
- Test with actual device NPUs