The Speed Gap
Why caches exist (it's physics, not preference)
Topic 1 of 10
Before we can understand caches, we need to understand why they exist. The answer isn't about clever algorithms or software design - it's about physics. The speed difference between computer components creates an unavoidable problem that caches solve.
Why do caches exist? It's not a design preference - it's a response to physics. The speed difference between computer components is staggering. Let's visualize just how big this gap is.
The Human Scale Analogy
If an L1 cache access took 1 second, then by comparison:
What does the CPU do while waiting?
A modern CPU runs at ~3 GHz - about 3 billion cycles per second. Click a layer to see how many cycles the CPU wastes waiting for data:
Key Insight
The speed gap between layers is 100 million times from top to bottom. If the CPU had to wait for disk on every operation, computers would be unusable. This is why caches exist at every layer boundary:
But how does a cache decide what to keep? It makes a bet about the future...