ORIGINAL TECHNICAL PRACTICE
Prevent a cache stampede
A popular cache key expires and thousands of requests simultaneously recompute it. How would you reduce the load?
Written for independent practice. The model answer presents one approach; clarify constraints and explain alternatives.
A REASONED APPROACH
Model answer.
Coordinate recomputation and avoid synchronized expiry. A per-key single-flight mechanism can let one request refresh the value while others wait or receive an acceptable stale version.
Work through it
- Define the freshness requirement and whether serving stale data is allowed. Separate authoritative storage from the cache and decide how errors affect expiry.
- Use bounded per-key coordination with timeout and ownership safeguards. Add jitter to expiry times and consider background refresh before expiration for frequently requested keys.
- Measure hit ratio, refresh duration, stale responses and authoritative-store load. Test a failed refresher, lock expiry and a sudden burst after cache loss.
Tradeoffs to discuss
- Serving stale data improves availability for some content but is inappropriate when an old value violates a required invariant. Distributed coordination adds failure modes of its own.
Common pitfalls
- A lock without expiry can strand the key after a crash. Simultaneously warming every key after a restart can recreate the same overload at a larger scale.