Concepts
Cell-based architecture and blast radius
Intuition
A ship is divided into watertight compartments: if the hull is breached, the water is contained in one section and the ship keeps floating. A service can be divided the same way, into several complete, independent copies, each serving only a portion of the customers. That way, when one copy fails, only the customers of that copy feel the problem.
How it works
In a cell-based architecture, the service is split into cells, and each cell is a complete instance of the workload that, ideally, shares no state and makes no calls to other cells. A cell router, the thinnest possible layer, presents a single endpoint to clients and forwards each request to the right cell based on a partition key, such as the customer ID, aligned with the service's grain. A control plane provisions and removes cells and migrates customers between them, while the data plane (router and cells) performs the service's main function; thanks to static stability, the data plane keeps working even when the control plane is impaired. The blast radius is bounded by the number of cells: with 10 cells, a failure in one of them leaves 90% of requests unaffected, and with 100 cells each one represents 1% of the service. Shuffle sharding refines the assignment: instead of a fixed shard, each customer gets a combination of workers, and with eight workers and combinations of two there are 28 possible shuffle shards; with hundreds of customers or more, each in its own shuffle shard, the scope of a problem is 1/28, versus 1/4 in regular sharding with four shards. In this arrangement, one customer's problem hits at most one worker of any other shuffle shard (with larger shards, an overlap bound can be guaranteed, such as at most two servers in common between shards of four), and service to the others is guaranteed if the callers tolerate failures and work around the problem, for example with retries. At deployment time, releases roll out in waves, cell by cell or in sets, with the first cell as a canary and rollback at the first sign of failure. There are costs: more complexity, more infrastructure, and a router that is a single point of failure and must itself also be built in a cellular way; moreover, cells were not designed as failover domains or to mitigate dependency failures.
In practice
It makes sense for workloads that demand extreme resilience and accept the extra cost in complexity and infrastructure, as long as you know which key (customer, resource) defines who must be isolated from whom. In Route 53, according to the article on shuffle sharding, each domain gets a shuffle shard of four out of 2048 virtual servers, no domain shares more than two of them with another, and a DDoS against one domain goes unnoticed by the others. The same logic applies to deployments: publishing cell by cell, with the first one as a canary and rollback at the first sign of failure, keeps a bad release confined to a few customers.
Use it in a design
Practice the cases that rely on this concept in the System Design school.