Concepts
Online migration: dual write, backfill, and read switchover
Intuition
Imagine replacing a store's foundation without ever closing its doors: you don't tear everything down at once; you build the new foundation next to it, start storing the new goods in both places, move the old ones over gradually, check that the two inventories match, and only then start serving customers from the new one. The store keeps selling the whole time, and the old building is demolished only when nobody depends on it anymore. The idea is to change where the data lives in small, verifiable steps, instead of in a single risky leap.
How it works
The pattern Stripe describes has four steps. In the first, dual writing, every new write updates both the old and the new store; since the extra write can weigh on the production database, the percentage of duplicated objects is increased gradually while watching operational metrics. Dual writing alone only covers new objects, so a backfill copies the existing ones: first lazily (copy when the object is updated) and then through a bulk copy, whose ID list can be generated offline over database snapshots so production is not overloaded with queries; rerunning, at the end, the job that generates that list confirms nothing was missed. In the second step, the read switch, an experiment (in this case, the Scientist library) reads from both paths, compares the results and alerts when they diverge, without affecting the application if the experimental path fails; only once everything matches does reading come from the new store. In the third, the order of the writes is reversed: writes go to the new store first and then to the old one, which requires refactoring the write paths; Stripe did it in small units, with changes of a few hundred lines at a time, and made access to the old field raise a hard error to reveal any forgotten use. In the fourth, once no code depends on the old data anymore, writing to it stops, it is removed, and the new store becomes the source of truth. Each phase exists to reduce a risk: dual writing keeps both sides in sync, the backfill completes the history, the compared read gives confidence before depending on the new store, and the reversal and cleanup only happen with that confidence. Migrating without a downtime window matters because Stripe does its infrastructure updates online, with no maintenance windows; and scale matters because, if each object took one second, copying a hundred million of them sequentially would take more than three years.
In practice
This pattern applies when you need to change the data model of a service that cannot stop, such as moving a list out of a record into its own table, and there are many places in the code that use the data. The cost is living for a while with two datastores, extra writes and transition code, in exchange for being able to advance, measure, and notice any divergence early, before reads and writes depend on the new side. In practice, the paper uses it to avoid a maintenance window and a one-shot switch of thousands of rows, in which edge cases would almost certainly slip through.
Use it in a design
Practice the cases that rely on this concept in the System Design school.