[00:02] requests hit your database at once. That's a cash stampede and most engineers only know one way to fix it. Let's go through four approaches. First, Let's go through four approaches. First, cash locking. A lock lets one request
[00:15] rebuild the cash while everyone else waits in line. Simple to implement, in your system. The downside of this is that if the rebuild is slow or fails, are going to be timing out. Another approach is to collapse requests instead
[00:31] of queuing them. Every duplicate in-flight request gets folded into one upstream call and everyone gets the same result back. Cloudflare does this at the CDN layer. Even if a million users hit the same key, your back-end sees at most
[00:44] one request per app server. We can also rely on randomness. As a cash entry ages rely on randomness. As a cash entry ages toward its TTL, each request has a small but growing chance of triggering a background refresh. So, at minute 50 of
[00:57] background refresh. So, at minute 50 of a 60-minute TTL, maybe 1% of requests refresh it. At minute 59, maybe 20%. Refresh is spread out naturally. The last is a background refresh. A dedicated worker proactively recomputes
[01:11] hot keys before they expire. This means zero stampede risk. The trade-off is you need to know what's hot ahead of time. Ticketmaster event pages are a perfect fit, but random traffic spikes are not. So, start with request coalescing. It's
[01:26] the easiest win and works at the CDN or app layer with no TTL changes. You can add a probabilistic early refresh for unpredictable spikes. Use background demand. Like and follow us for more system design tips.