Esp32 Xip Design
Execute-in-place (XIP) is the mechanism that lets the CPU run application code directly out of external SPI flash as if it were fast on-chip memory, and it only works because of a cache sitting between the CPU and that flash — without it, every single instruction fetch would stall the CPU for the full latency of an external SPI transaction, turning a 240 MHz core into something that performs far closer to the flash's own slow read speed. External flash is cheap and can hold megabytes of firmware the on-die SRAM could never afford to replicate at that cost, but a SPI flash read takes on the order of tens to hundreds of nanoseconds per access — compared to a CPU cycle at 240 MHz lasting about four nanoseconds, that gap is large enough that running code directly from flash with no cache would be unusably slow, which is exactly the problem the instruction cache exists to hide.
Cache hit rate, not raw CPU clock speed, is the number that actually determines how fast XIP code runs in practice, and that hit rate depends heavily on code locality — how tightly clustered the instructions a program actually executes are in memory. A tight inner loop that fits entirely within a few cache lines will run close to full CPU speed indefinitely once it's loaded once; code that jumps around unpredictably across a large function table, by contrast, can trigger a cache miss on nearly every fetch, turning that section of code's effective execution speed into something much closer to raw SPI flash latency — which is why performance-sensitive firmware deliberately places hot loops and interrupt handlers in on-chip SRAM rather than leaving them to run from flash through the cache, trading SRAM budget for execution speed exactly where it matters most.
Line-fill granularity matters because a cache always pulls in more than the single instruction that was actually requested, which is a deliberate bet that nearby instructions will be needed soon — a bet that pays off for sequential code and pays nothing for scattered jumps. When a miss occurs, the controller fetches an entire cache line — typically covering several consecutive instructions — in one SPI burst, rather than issuing a separate flash transaction for every single fetch; this amortizes the fixed overhead of a SPI transaction across multiple instructions for straight-line code, which is the common case, but delivers no benefit at all for code that jumps immediately to an unrelated address after only using one instruction from the line that was just fetched.
The decision to place a given function in flash (cached, XIP) versus in on-chip SRAM (always full speed, no cache behavior at all) is a real firmware architecture choice with a genuine capacity-versus-speed tradeoff, not just a default location a compiler picks. SRAM capacity on a chip like this is small and shared with every other runtime use — stack, heap, buffers, the Wi-Fi/BLE stack's own working memory — so placing an entire application in SRAM to guarantee cache-free speed everywhere simply isn't possible at any reasonable SRAM size; firmware developers instead profile for the specific hot paths — interrupt service routines, tight DSP-style loops, anything latency-critical — and deliberately link just those functions into SRAM while leaving the bulk of less performance-sensitive application code to run from flash through the cache, accepting its occasional miss penalty in exchange for not consuming scarce SRAM.
XIP and the instruction cache together are what make it economically viable to ship megabytes of firmware on cheap external flash while still running most of that code at speeds the application can actually use — and the real engineering discipline isn't the cache mechanism itself, which is largely fixed in hardware, but the firmware-side decision about which specific functions are worth the SRAM budget to guarantee full speed and which can tolerate the cache's hit-rate-dependent performance.