Real architecture · Senior
A bank on an immutable database: Datomic, Clojure, and Kafka at the core of Nubank
The audit domain justifies every choice: replay with offset reset, point-in-time queries, the "diplomat architecture", and standardizing the stack when growing from 50 to over 600 engineers.
Short answer
The audit domain justifies every choice: replay with offset reset, point-in-time queries, the "diplomat architecture", and standardizing the stack when growing from 50 to over 600 engineers. Nubank started from a premise of the financial domain: for a bank, you need the entire history, the complete audit trail of everything that is done.
01 · CONTEXT
Auditing as a starting requirement
Nubank started from a premise of the financial domain: for a bank, you need the entire history, the complete audit trail of everything that is done. The first technical choice, the database, came from there.
Datomic is described by Cavalcanti, of Nubank, as a niche database, an immutable ledger of facts: each update preserves the history and previous values are not lost. Instead of storing just the data, it stores the changes to the data, like a Git repository. When it came time to choose the database, this is the behavior the domain asked for.
The database also lets you query the data at any point in time, which helps both with auditing and with later debugging.
02 · THE STARTING STACK
Datomic, Clojure, Kafka, and AWS
Clojure came because of Datomic, which is built in Clojure; the team was already heading toward a functional language anyway. For messaging, the choice was Kafka, which was gaining traction in 2013 and keeps a message log that can be replayed. The cloud chosen was AWS, with DynamoDB as Datomic's backing database.
The client calls a domain service; the main language chosen was Clojure.
The service writes to Datomic, which stores the changes to the data like a Git repository, not just the current value.
Datomic uses DynamoDB as its backing database.
On reads, the service can query the database at any point in time, which serves auditing and debugging.
| Escolha | What the source points out |
|---|---|
| Datomic | Immutable ledger of facts; each update preserves the history |
| Clojure | Functional language on the JVM, with immutability by default |
| Kafka | Persistent message log, with TTL, and offsets that can be reset |
| Debt | Niche, not-yet-established technologies; hard to find people with experience |
03 · EVENTS
Kafka: the log you can reread
Cavalcanti describes Kafka as a persistent message log with TTL: it is not forever, but for a period you can inspect everything that was produced. There is also the possibility of resetting offsets and reprocessing old messages when necessary, something the team had to do a few times early on.
A service publishes messages to Kafka, the chosen messaging technology.
The log is persistent and has a TTL: it does not last forever, but for a period you can inspect what was produced.
In the diagram, the consumer reads from the log and writes the result to its own database.
By resetting the offset, old messages can be reprocessed when necessary.
04 · CODE AND INFRASTRUCTURE
Diplomat architecture and immutability
Nubank adopted microservices from the start, because the financial domain is very complex and containing that complexity in small services was important to the team.
To organize the code of each service, the team drew inspiration from hexagonal architecture, but gave the variant its own name, diplomat architecture, to avoid confusion with the original idea. In Cavalcanti's description, the control layer handles side effects and another layer holds the business logic, made of pure functions that receive maps and return maps.
| Aspecto | Hexagonal, according to Cavalcanti | Nubank's diplomat architecture |
|---|---|---|
| Where the idea goes | Cockburn's definition stops at the controller; after it, it is already business logic | Separates pure business logic from impure logic with side effects |
| Origin of the name | Ports and adapters | Its own name, inspired by ports and adapters, with a functional bias |
Immutability applies outside the code too. Cavalcanti says most of the mutable state lives in the database and that, in Datomic, whoever asks for the database receives a value: time stops, and no new transaction appears in that value. The infrastructure follows the same idea: instead of changing a running machine, you create a new one with the new configuration or the new code.
05 · DATA AND RECONCILIATION
Where the check comes in
With services that each have their own database, and shards with copies of those databases, the data ends up cut two ways: by service and by shard. An aggregator system was needed to gather the data from all services across all shards.
Each service has its own database, and there are copies of those databases in shards.
The ETL extracts the data from the services' databases across all shards.
The ETL output goes to a data warehouse.
The same ETL is used to check consistency between systems, since failed distributed transactions become a big problem.
According to the talk, with this volume of data and a distributed architecture, failed distributed transactions become a big problem, and the ETL is also used to check consistency between systems.
The decision not to use Kafka Streams for the ETL has a reason explained by Cavalcanti: when they needed an ETL, Kafka Streams was neither stable nor released.
06 · ORGANIZATION SCALE
Canonicity: from 50 to more than 600 engineers
When people ask about Nubank's stack, the answer is short: Clojure in production services, Kafka for asynchronous communication, Datomic for high-value business data, Scala in the analytics environment, and Flutter in the app.
After seven years building products, with more than 600 engineers, the Building Nubank blog asks whether preferring less variance in technologies results in a more efficient engineering organization. The authors say alignment was easier, though not easy, with 50 engineers, and that with 600 it is different and more complicated.
In practice, a team should be able to code, test, and deploy without depending on another team. When someone needs to work outside their domain, it is very likely that both teams use Clojure, Kafka, and Datomic, and there are standard libraries for talking to Datomic, DynamoDB, Kafka, and HTTP.
Standardization allows exceptions: the first ETL was built in Spark with Scala, and the machine learning services in Python.
Frequently asked questions
What is A bank on an immutable database: Datomic, Clojure, and Kafka at the core of Nubank?
The audit domain justifies every choice: replay with offset reset, point-in-time queries, the "diplomat architecture", and standardizing the stack when growing from 50 to over 600 engineers.
What does the A bank on an immutable database: Datomic, Clojure, and Kafka at the core of Nubank design cover?
It is organized in these sections: Auditing as a starting requirement, Datomic, Clojure, Kafka, and AWS, Kafka: the log you can reread, Diplomat architecture and immutability, Where the check comes in, and Canonicity: from 50 to more than 600 engineers.
How long is A bank on an immutable database: Datomic, Clojure, and Kafka at the core of Nubank and what level is it?
It takes about 17 minutes to read and is rated Senior level.
Now design it yourself
Run this case as an interview in the System Design school: you make the decisions, the tutor challenges them and you see how your design scores.