memory architecture design
**On-Chip Memory Architecture** is the **design discipline that organizes the hierarchy of registers, SRAM caches, and embedded memories within a processor or SoC — where memory access latency and bandwidth determine 50-80% of overall chip performance, making the capacity, organization, and placement of on-chip memory the most impactful architectural decision after the compute pipeline itself**.
**The Memory Hierarchy**
| Level | Size | Latency | Bandwidth | Technology |
|-------|------|---------|-----------|------------|
| Register File | 1-32 KB | 1 cycle | ~TB/s | Custom flip-flops |
| L1 Cache (I/D) | 32-64 KB | 3-5 cycles | 200+ GB/s per core | 6T/8T SRAM |
| L2 Cache | 256 KB-2 MB | 10-20 cycles | 100+ GB/s | 6T/8T SRAM |
| L3 Cache (LLC) | 4-256 MB | 30-60 cycles | 50-200 GB/s | SRAM or eDRAM |
| HBM/DDR (off-chip) | 16-192 GB | 100-300 cycles | 50-8000 GB/s | DRAM |
**SRAM Bitcell Design**
- **6T SRAM**: Standard bitcell with 6 transistors — two cross-coupled inverters for storage, two access transistors gated by the word line. Provides single-cycle read/write. Area: 0.020-0.030 μm² at 5nm node.
- **8T SRAM**: Adds a separate read port (2 transistors) to eliminate read disturb, improving read stability at low voltage. Enables operation at lower Vdd (0.5-0.6V) for power savings.
- **Bitcell vs. Periphery Area**: At advanced nodes, SRAM bitcell area stops scaling (limited by read/write stability margins), while periphery circuits (sense amplifiers, drivers, address decoders) contribute 30-50% of total memory area. Assist circuits (write-assist negative bitline voltage, read-assist positive word line underdrive) enable bitcell scaling at the cost of peripheral complexity.
**Cache Organization Architecture**
- **Associativity**: Higher associativity (8-way, 16-way) reduces conflict misses but increases tag comparison logic, area, and access latency. L1 caches typically use 4-8 way; L3 caches use 8-16 way.
- **Line Size**: 64 bytes is standard. Larger lines improve spatial locality exploitation but waste bandwidth on sparse access patterns.
- **Replacement Policy**: LRU (Least Recently Used) approximations (pseudo-LRU, RRIP — Re-Reference Interval Prediction) balance hit rate against hardware complexity.
- **Inclusive vs. Exclusive**: Inclusive L3 guarantees that L3 contains a superset of L1/L2 data (simplifies coherence). Exclusive L3 maximizes effective capacity (L1+L2+L3) but complicates coherence protocol.
**Embedded Memory Compilers**
Compilers (tools from ARM, Synopsys, foundry PDKs) generate optimized SRAM/ROM instances from parameterized specifications (word count, bit width, ports, muxing ratio). The compiler produces the layout (GDS), timing model (.lib), netlist, and verification views — enabling rapid integration of custom memory blocks into SoC designs.
On-Chip Memory Architecture is **the performance multiplier that determines whether a chip's compute units are fed or starved** — because even the most powerful ALU is useless if it spends 90% of its cycles waiting for data from a memory hierarchy that was designed with insufficient capacity, bandwidth, or proximity.