Why a website loads faster the second time you visit
A cache is a small, fast store that keeps copies of answers already fetched or computed. Your browser saves a site's logo and scripts on disk, so your next visit loads them in milliseconds instead of over the network.
Each lookup is a hit (the copy is there, so it's served fast) or a miss (it isn't, so you fetch it from the slow source and usually keep a copy).
A full cache needs an eviction policy, a rule for what to drop. LRU (least recently used) throws out whatever hasn't been touched for longest.
Systems design · 2 of 122 / 3
Caching
Real-life use case
Facebook answers billions of reads per second from Memcached
Memcached is an in-memory key–value cache: it keeps items in RAM, looked up by keys like "user:42:profile". Facebook runs it in front of its MySQL databases, serving billions of requests per second.
Facebook uses a pattern called cache-aside. The app asks the cache first; on a miss, it reads the database and stores the result in the cache.
When data changes, the app deletes the cached copy, so the next read fetches it fresh. Browsers use HTTP headers like Cache-Control: max-age=3600 to say how long a copy stays valid.
Systems design · 2 of 123 / 3
Caching
How it helps
A few popular items get most requests, so a small store answers them
Requests are skewed: a few popular items, like today's front page, get most of the traffic. Caching just those lets most requests come from RAM (about 0.1 ms) instead of a 10–100 ms database query.
So the hit rate, the share of requests the cache answers, sets average latency. Every hit is also a query the database never sees, which raises throughput.
The price is staleness: a cached copy can be out of date. So entries get a TTL (time to live, an expiry time) or are deleted when the source changes.
What it’s forEstimates the average time a request takes once a cache sits in front of a slow source.
T_avg = h · t_hit + (1 − h) · t_miss
h
hit rate, the share of requests found in the cache, e.g. 0.9
t_hit
time to answer from the cache, e.g. 1 ms
t_miss
time to answer when the cache misses and the slow source is used, e.g. 50 ms
T_avg
average time per request
What it givesThe average latency, which shows how much each extra point of hit rate is worth.
h = 0.9, t_hit = 1 ms, t_miss = 50 ms
T_avg = 0.9 · 1 + 0.1 · 50 = 0.9 + 5 = 5.9 ms
At h = 0.99: 0.99 + 0.5 = 1.49 ms, so the misses dominate