Sharding by customer and the shard 1 that grew too large

01 · CONTEXT

An architecture that did not last as long as expected

On the InfoQ podcast, Lucas Cavalcanti, of Nubank, says the initial deployment architecture comfortably handled 500,000 customers and the team thought it would live with it for a few years.

According to him, when the company started there was no Kubernetes or Docker, and the company grew faster than the project to migrate everything to Docker or to better structures on AWS. The sharding project was born from there, from the initial deployment choice on AWS.

The cloud lets you scale vertically and, in some cases, horizontally, but he says you reach a limit, and it was because of that limit that sharding had to be done. This is the starting point of the case: scaling vertically and horizontally has a ceiling.

1. The diagram shows a single deployment of the services serving all customers.

2. The same deployment reads from and writes to a single database.

3. Scaling this block vertically, or a few instances horizontally, hits a limit in the cloud, and the account ties the sharding project to that limit.

02 · THE DECISION

Splitting by customer

The sharding decision was to split by customer: a shard holds a set of customers and all of those customers' data, and another shard can have a different set.

1. In the diagram with shards, each group of customers has its own stack: services, database, and Kafka.

2. The second shard holds a different set of customers and all of their data.

3. The third shard repeats the same structure, so the design grows by copies.

In the talk published by InfoQ, he says sharding helped a lot with scaling in the consolidation phase, when a rare case affecting 0.1% of customers started to hit thousands of customers.

03 · THE DEBT

The first shard that grew too large

When the sharding project finished, the first shard was larger than it should have been. Cavalcanti explains that it is not only the number of customers that matters, but also how long each one has been a customer.

A startup's first customers are early adopters and the ones who use the product the most. The first shard ended up with the most engaged customers, more data, and more customers, because the sharding project took too long.

1. The first shard gathers the oldest customers, the early adopters, who use the product the most.

2. That shard's provisioning was four to five times that of the shards created later.

3. That shard required performance tuning.

4. The diagram shows the other shards with the same structure as the first.

5. Over time it became possible to make all shards a similar size.

According to him, the idea of sharding at the start was to avoid performance tuning, and even so the first shard required fine-tuning.

On outlier customers, the talk says some customers have 10x or 100x more transactions than the others. In his view, this basically affects that customer and not the rest, because most operations are already batched.

In the talk, he also says the first shard took longer than expected to set up and ended up with customers who were too old.

04 · ISOLATION

The shard as a unit of isolation and forecasting

A post on Nubank's engineering blog describes sharding going beyond the database: it reaches databases, key services, and entire clusters of microservices. Datomic runs several transactors on Kubernetes, each with a subset of the data, and there are several Kafka clusters for different shards, to spread the message load.

The post points to two goals. The first is to keep localized problems from spreading, because each shard is relatively independent; it is a way to isolate failures. The second is to forecast capacity in smaller, more predictable units, since one shard's usage helps project the others.

05 · QUOTAS

What blows up first in the cloud

The same post recalls that every cloud provider imposes limits or quotas, from load balancers and EC2 instances to storage, and that at Nubank's scale quotas can become a bottleneck if they are not monitored.

1. The diagram starts from Nubank's AWS accounts.

2. The AWS Limit Smoke Detector is an ETL pipeline that collects usage, quota, and cost from all AWS accounts and aggregates them in BigQuery.

3. The tool continuously compares actual usage against assigned quotas and flags problems before a critical limit is reached.

4. When usage rises quickly, the Smoke Detector alerts through dashboards and automated messages, and the team raises the quota or optimizes the service's consumption.

06 · IN THE INTERVIEW

How to use the case

When choosing a shard key, ask whether a shard's size depends only on the customer count or also on how they use the product, and whether there is a way to spread the most active customers across shards.

When estimating what blows up first, list the resources that have a quota or ceiling, compare current usage against each one, and say how you would be warned before reaching it.