Software Development

Cache Stampedes: Thundering Herds and the Fixes That Actually Work

Your cache hit rate is 99 percent, and the dashboard is green. Then one popular key expires. Within milliseconds, hundreds of requests miss together and rush to the database. That rush has a name, and it has a handful of well-known cures.

What a Stampede Looks Like

A cache stampede happens when a popular entry expires and every request that wanted it misses at once. As the Redisson glossary puts it, all of them then query the backing store together. The same thing can follow a cache flush, a node restart or a failover.

Notably, the damage lands on the database, not on the cache. The picture below shows the moment of expiry.

Three Fixes Worth Knowing

1. A mutex per key

First, let only one request rebuild the value. In Redis, a single command can act as that lock. The first caller gets OK, and everyone else gets nothing back, so they know to wait or poll. I verified the behavior below with redis-cli on Redis 7.0.15.

redis-cli SET lock:hot 1 NX PX 5000
redis-cli SET lock:hot 1 NX PX 5000
redis-cli PTTL lock:hot
redis-cli DEL lock:hot

Run it on any machine with Redis. The first command prints OK, the second prints an empty reply, the third prints a number just under 5000, and the last prints 1. The PX 5000 part matters, because the lock expires on its own if the rebuilding process crashes.

2. Stale while revalidate

Second, serve the old value while one worker refreshes it in the background. The idea is standardized for HTTP in RFC 5861, which describes returning a stale response while revalidating without blocking. As a result, users never wait for the rebuild.

3. Probabilistic early refresh

Third, refresh a little before expiry, with a probability that grows as the deadline nears. This is the XFetch algorithm from Optimal Probabilistic Cache Stampede Prevention (Vattani, Chierichetti and Lowenstein, VLDB 2015). However, the guarantee is only probabilistic. The Redisson glossary itself warns that, under enough concurrency, two nodes can still recompute at the same time.

FixCore ideaMain trade-off
MutexOne rebuilder, others waitWaiters pay the rebuild time
Stale while revalidateServe old data, refresh in backgroundData is briefly stale
Early refresh (XFetch)Refresh before expiry at randomNot strictly one rebuild

What the Numbers Say

Next, let’s measure. I wrote a small Python test against a real Redis 7.0.15. Fifty threads hammered one hot key with a 2 second TTL for 16 seconds per strategy, which produced roughly 100,000 requests each. The database was simulated by a 300 ms sleep, and I repeated the whole test twice.

. Redis 7.0.15 on loopback in a sandbox container, Python threads, simulated 300 ms database, key pre-warmed before each run. Database calls were 350, 7, 10 and 185 or 187 across the two runs; the last bar shows 186.

The gap is dramatic. Without protection, every expiry sent about 44 requests to the database, and 50 ran at the same moment. The mutex and stale-while-revalidate cut that to one call per expiry. Meanwhile, early refresh alone roughly halved the calls, but it did not remove the overlap.

Worst single request latency in ms, averaged over two runs (311 and 311, 325 and 321, 30 and 26, 311 and 310).
StrategyPeak concurrent DB callsWorst latency
No protection50about 311 ms
Mutex1about 323 ms
Stale while revalidate1about 28 ms
Early refresh (XFetch)9 to 14about 311 ms

Reading the results fairly

Several caveats apply. First, my simulated database does not slow down under load, so the real harm of 50 simultaneous queries is understated. Second, the XFetch result comes from my own simple implementation at a very high request rate. In theory it performs better, so treat this as a warning about using it alone. Third, the mutex made waiting requests pay the full rebuild time, which is why its worst case matched the unprotected one.

In practice, stale-while-revalidate gave the best experience here, because nobody waited. On the other hand, it serves slightly old data, so it suits values that tolerate a few seconds of lag. Combining early refresh with a lock is a sensible next step, but I did not measure that.

Stampede checklist

  • Identify your hottest keys, since a stampede needs heavy traffic on one entry.
  • Give every lock an expiry, so a crashed worker cannot block the key forever.
  • Decide how stale is acceptable before choosing stale-while-revalidate.
  • Plan for a cold cache too, because a restart empties every key at once.

What We Have Learned

A cache stampede is many requests rebuilding the same expired value at the same time. In our Redis test, doing nothing sent 350 queries to the database and up to 50 at once. A per-key mutex reduced that to 7 queries, but made waiters slow. Stale while revalidate used 10 queries and kept the worst latency near 28 ms, at the price of briefly stale data. Probabilistic early refresh alone was only partly effective in our setup. So choose by what you can tolerate, either waiting or staleness, and always give locks an expiry.

Eleftheria Drosopoulou

Eleftheria is an Experienced Business Analyst with a robust background in the computer software industry. Proficient in Computer Software Training, Digital Marketing, HTML Scripting, and Microsoft Office, they bring a wealth of technical skills to the table. Additionally, she has a love for writing articles on various tech subjects, showcasing a talent for translating complex concepts into accessible content.
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

0 Comments
Oldest
Newest Most Voted
Back to top button