On-Chip Memory Architecture is the design discipline that organizes the hierarchy of registers, SRAM caches, and embedded memories within a processor or SoC — where memory access latency and bandwidth determine 50-80% of overall chip performance, making the capacity, organization, and placement of on-chip memory the most impactful architectural decision after the compute pipeline itself.

The Memory Hierarchy

SRAM Bitcell Design

• 6T SRAM: Standard bitcell with 6 transistors — two cross-coupled inverters for storage, two access transistors gated by the word line. Provides single-cycle read/write. Area: 0.020-0.030 μm² at 5nm node.
• 8T SRAM: Adds a separate read port (2 transistors) to eliminate read disturb, improving read stability at low voltage. Enables operation at lower Vdd (0.5-0.6V) for power savings.
• Bitcell vs. Periphery Area: At advanced nodes, SRAM bitcell area stops scaling (limited by read/write stability margins), while periphery circuits (sense amplifiers, drivers, address decoders) contribute 30-50% of total memory area. Assist circuits (write-assist negative bitline voltage, read-assist positive word line underdrive) enable bitcell scaling at the cost of peripheral complexity.

Cache Organization Architecture

• Associativity: Higher associativity (8-way, 16-way) reduces conflict misses but increases tag comparison logic, area, and access latency. L1 caches typically use 4-8 way; L3 caches use 8-16 way.
• Line Size: 64 bytes is standard. Larger lines improve spatial locality exploitation but waste bandwidth on sparse access patterns.
• Replacement Policy: LRU (Least Recently Used) approximations (pseudo-LRU, RRIP — Re-Reference Interval Prediction) balance hit rate against hardware complexity.
• Inclusive vs. Exclusive: Inclusive L3 guarantees that L3 contains a superset of L1/L2 data (simplifies coherence). Exclusive L3 maximizes effective capacity (L1+L2+L3) but complicates coherence protocol.

Embedded Memory Compilers

Compilers (tools from ARM, Synopsys, foundry PDKs) generate optimized SRAM/ROM instances from parameterized specifications (word count, bit width, ports, muxing ratio). The compiler produces the layout (GDS), timing model (.lib), netlist, and verification views — enabling rapid integration of custom memory blocks into SoC designs.

On-Chip Memory Architecture is the performance multiplier that determines whether a chip's compute units are fed or starved — because even the most powerful ALU is useless if it spends 90% of its cycles waiting for data from a memory hierarchy that was designed with insufficient capacity, bandwidth, or proximity.