Skip to main content

Concepts

Cache invalidation via leases

Intuition

Imagine a bulletin board in the meeting room: instead of someone running over there every time the information changes to erase and rewrite the right number by hand (risking writing the wrong version over the right one), the person who updates the original data simply tears the old notice off the board. The next time someone looks at the board and finds nothing, they go to the source, get the current value, and put up a new notice. This is simpler and safer than trying to edit the notice in place, because editing risks two people changing it at the same time and the slower one overwriting the faster one's work.

How it works

Facebook's Memcache uses a look-aside cache: on a read, the application queries memcached and, on a miss, fetches from the database and writes the result back to the cache; on a write, the application writes to the database and invalidates (deletes) the corresponding key in the cache, instead of trying to overwrite the cached value directly. The invalidation is triggered by watching the writes that reach the database (the role of mcsqueal), identifying which cache keys correspond to the changed data and deleting them in a targeted way, without flushing the entire cache. This design avoids two concurrency problems at scale: the 'thundering herd', in which an invalidated popular key causes a spike of simultaneous misses racing to recompute the same value in the database; and the delayed write, in which an old request that fetched an already stale value finishes processing after an invalidation and writes old data over the new. The lease mechanism solves both: when a miss is recorded, memcached hands a limited-validity token to the client that will recompute the value in the database; only the client with the most recent valid token is allowed to write the result back to the cache, and writes with an expired or invalidated token are silently discarded.

In practice

This pattern of invalidating instead of updating directly is the default in any architecture that combines a relational database with Redis or Memcached in front to absorb reads; leases (or variants such as recomputation locks with a short TTL) show up whenever a cache key is expensive to recompute and heavily accessed, to prevent a traffic spike on a single key from taking down the database behind it.

Use it in a design

Practice the cases that rely on this concept in the System Design school.