Concepts
Reconciliation and validation of financial data
Intuition
Think of a company with several departments, each recording money in its own notebook, at different paces. In the end, someone has to prove that the notebooks, taken together, tell the same story as the real world. It is not enough to trust that each notebook is right: you need to check that everything that happened was recorded, that it was recorded in time, and that the money that came in somewhere left from somewhere else, with nothing left sitting midway. When something does not add up, the check should also show where the account broke.
How it works
Stripe's Ledger models each producing system as a flow of funds between accounts, and reconciliation starts from a property of this model: in a flow that completed correctly, the intermediate (clearing) accounts must be zeroed out, so a query for clearing accounts with a non-zero balance points to where a missing, late or incorrect event broke the account. The Data Quality platform built on top of the Ledger evaluates each flow with three questions: clearing (did the flow complete? verified by the accounts that should be zeroed out), timeliness (did the data arrive on time? measured as the interval between the data entering the platform and reaching the Ledger, against a hard delivery-window limit) and completeness (does the Ledger fully represent the source system?). The last one, which explicitly watches for possible divergence between the Ledger model and the source data, uses cross-system checks, such as requiring every ID in the producer's database to have a corresponding event in the Ledger, plus automatic anomaly detection and per-account-type statistical models that estimate, from history, when the data should arrive; an event that does not show up is treated as possible missing data. The metrics are composed into a unified DQ score. Because the Ledger is an immutable log, the problematic data that any of these checks flags can be audited and reproduced exactly.
In practice
The Stripe post (an engineering write-up, not a peer-reviewed paper) reports about five billion events per day, 99.99% of the dollar volume fully ingested and verified within four days, and more than 99.9999% explainability of money movement, even with the data volume having grown 10 times. Dashboards show the score per team, with automatically generated tickets, and let you follow a point on the graph to its root cause. The lesson for those designing financial systems is to treat verification as part of the design, with balances that must net to zero, measured arrival deadlines, and completeness checks against the source, so that real-world imperfections (malformed reports, partner errors) stay bounded and explainable instead of hidden.
Use it in a design
Practice the cases that rely on this concept in the System Design school.