Concepts
Hedged and tied requests
Intuition
It is like ordering food from two different delivery apps at once when you are in a hurry: as soon as one of them arrives, you cancel the other order. You pay a little more (sometimes both charge something before you cancel), but you make sure you will not end up waiting for precisely the courier who got stuck in traffic. This does not fix the traffic or make any courier faster — it only protects you from depending on the bad luck of picking the wrong courier.
How it works
The Dean and Barroso paper describes two techniques that attack tail latency within a single request, instead of trying to eliminate the individual variability of each server (considered impractical). In 'hedged requests', the client sends the request to a primary replica and only fires a copy to a secondary replica if the first has not yet responded after a time close to the expected p95; it uses whichever response arrives first and cancels the other — since the second copy only goes out when the first is already taking longer than normal, the extra load is limited to a few percent of total traffic, even while greatly reducing the tail. In 'tied requests', the technique is more aggressive: the client enqueues the request on multiple servers simultaneously from the start, but tells each server about the existence of the copies on the others, so that as soon as one server starts processing (or finishes), it sends a cancel for the copies queued on the others — preventing duplicate work from piling up unnecessarily. In a real Google system, tied requests reduced median latency by ~16% and p99.9 by ~40%. The trade-off in both techniques is the same: more load (replicas processing, or at least queuing, redundant work) in exchange for a much shorter latency tail.
In practice
RPC frameworks widely used in industry today offer configurable hedging policies as part of the service-to-service communication library itself, so the application does not need to reimplement the logic by hand — a common choice when the cost of an occasional extra redundant call is much lower than the business cost of a slow response, but one that requires extra server capacity and care not to amplify load on systems already near their limit.
Use it in a design
Practice the cases that rely on this concept in the System Design school.