Concepts
TrueTime and external consistency
Intuition
Imagine a notary office with branches on several continents, each with its own wall clock, and none of them showing exactly the same time. To stamp documents so that the order of the stamps always matches the real order in which things happened, each branch admits that its clock has a known margin of error, something like 'it is now 10:00, give or take up to 4 milliseconds'. After stamping, it waits for the margin's time to pass before announcing that it is done. That way, any document that starts after this announcement, at any branch in the world, will receive a later stamp. The cost is a small wait on each stamp, and the gain is that the order of the stamps never contradicts reality.
How it works
Spanner guarantees external consistency, equivalent to linearizability: if transaction T1 commits before T2 starts, T1's commit timestamp is smaller than T2's, even when the transactions are distributed. The foundation is the TrueTime API, which represents time as an interval with bounded uncertainty. TT.now() returns a TTinterval with endpoints earliest and latest, guaranteed to contain the absolute time of the call, and TT.after(t) and TT.before(t) say whether t has definitely already passed or has not yet arrived. The interval comes from time masters with GPS and atomic clocks, which have different failure modes, and from a daemon on each machine that polls several masters and advertises an uncertainty that grows between synchronizations (in production, epsilon is around 1 to 7 ms, averaging close to 4 ms). Two rules underpin the guarantee. In the Start rule, the coordinator leader assigns the write a commit timestamp no smaller than TT.now().latest. In the Commit Wait rule, the coordinator lets clients see the data only when TT.after(commit timestamp) is true, that is, when the timestamp is guaranteed to be in the past. The expected wait is at least twice the average uncertainty and usually overlaps with Paxos communication. Each tablet is replicated by a Paxos group. Transactions within a single group use only the leader, and transactions across several groups use two-phase commit (2PC) among the leaders: each participant logs a prepare via Paxos, the coordinator picks the final timestamp, waits out the commit wait, and logs the commit. Because every timestamp respects the serialization order, read-only and snapshot reads run at a timestamp without locks, on any replica whose safe time has already reached it. If the uncertainty increases, the system gets slower because it waits longer, but it remains correct.
In practice
It is the choice for global databases that need transactions and true ordering preserved across regions, such as F1, Google's ads backend, which uses five replicas on two US coasts and reports nearly invisible failover. In exchange, you pay write latency (about 14 ms in the paper's benchmarks), clock infrastructure with GPS and atomic clocks, and a recovery of about 10 seconds if an entire zone goes down, tied to the duration of the leaders' leases. Unlike the weak guarantees accepted in systems such as GFS, here the expensive coordination is accepted on purpose so that the application does not have to deal with ordering anomalies.
Use it in a design
Practice the cases that rely on this concept in the System Design school.