Concepts
Shuffle sharding
Intuition
Imagine a call center with eight agents, where any customer can end up with any of them: a single problematic customer locks up one agent after another and, in the end, nobody gets served. Now give each customer a pair of agents drawn from among the many possible combinations, so that any two pairs rarely coincide entirely. If a problematic customer locks up their pair, almost all the other customers still have at least one free agent and, if they try again, they get served.
How it works
Without sharding, all workers serve any request: this is efficient, but a poison request or a flood of them (as in a DDoS) takes down the first worker hit and then, in a cascade, the others, and the blast radius is 100%. With regular sharding, the fleet is divided into fixed shards (for example, eight workers into four shards of two) and the blast radius drops to 1/4, at the cost of more idle capacity per shard. With shuffle sharding, each customer (or whatever you want to isolate) is assigned a combination of workers as a virtual shard; with eight workers and two per shard there are 28 unique combinations. A problem with one customer hits their virtual shard and, in any other shuffle shard, at most one worker is affected. If the requesters tolerate failures and work around the problem, for example with retries, the customers on the other shuffle shards continue without interruption, because they still have at least one healthy worker. If there are hundreds of customers or more, each assigned to a shuffle shard, the blast radius is 1/28, versus 1/4 for regular sharding with four shards, that is, seven times better, and, according to the author, the numbers improve rapidly (he says "exponentially") with more workers and customers.
In practice
In Route 53, according to the 2019 article, capacity was organized into 2048 virtual name servers and each customer domain received a shuffle shard of four of them; the article cites 730 billion possible combinations, and the assignment guarantees that no domain shares more than two virtual servers with any other. When a domain suffers a DDoS, its four virtual servers receive the traffic spike, but no other domain notices. According to the author, AWS incorporated the technique into many of its other systems, and shuffle sharding is a way of arranging existing resources that, in general, has no additional cost. It makes sense when a failure can be triggered by a customer or request type and the requesters tolerate failures and retry.
Use it in a design
Practice the cases that rely on this concept in the System Design school.