WhatsApp: 2 million connections per server (Erlang/FreeBSD)

01 · CONTEXT

Two million connections on one machine

In early 2012, the WhatsApp blog showed a single server with more than two million open TCP connections, with CPU and memory to spare. Two years later, a talk summarized by High Scalability cited 465 million monthly users and peaks of 147 million phones connected at the same time.

The same summary gives the message volume: 19 billion coming in and 40 billion going out per day. This case looks beneath that volume: how a stack with Erlang and FreeBSD sustains persistent connections at scale, and what the team did so that a failure would not spread.

For the learner, the interest lies in two topics already covered in the track: communication protocols based on open connections, and the horizontal and vertical scaling of servers that hold connection state.

02 · THE PATH OF THE MESSAGE

One phone, one connection, few components

According to the summary, phones connect to chat and multimedia servers, and chat talks to a transient store and to account, profile, push, and group databases.

1. The phone connects to a chat server; it is through this open connection that it delivers the message.

2. While the message is in transit between users, the backend holds it in a transient store.

3. The chat server also queries account, profile, push, and group databases.

4. The server delivers to the recipient the text, the notifications, and the presence messages, such as typing or online.

5. Multimedia goes through its own servers, and the database only holds information about the media while it is being delivered.

There are three families of messages for the phone: the text itself, notifications such as a group subject change, and presence messages. Since messages and media are only kept while they are being delivered, the architecture revolves around throughput, caching, and partitioning, not durable storage.

03 · CONNECTIONS PER SERVER

Why the number dropped from two million to one million

The 2012 machine ran FreeBSD 8.2-STABLE with Erlang R14B03. Even with more than two million open sockets, the measurement showed about 42% idle CPU, which indicates plenty of headroom.

The 2014 summary explains the drop: with more users, the team preferred to keep more headroom per server for spikes; users also became more active; and functions that used to run outside these servers began running on them.

The resulting fleet had about 150 chat servers, with about 1 million phones each, and about 250 multimedia servers. The strategy remained big boxes and few nodes, because, according to the summary, operational complexity grows with the number of nodes, not the number of cores.

04 · ISOLATING BOTTLENECKS

Decouple so a problem does not become a cascade

The principle repeated in the talk is that tight coupling causes cascading failures. That is why everything is partitioned: if one partition gets into trouble, the others are not affected.

The specific villain is head-of-line blocking: the item at the front of the queue blocks all those behind it. The response was to separate queues by purpose and by destination.

05 · PARTITION AND REPLICATE

Partitions in pairs and a single owner per record

Services are split from 2 to 32 ways, most of them 32, and nodes run in pairs, one primary and one secondary. Access to a record is decided by hash and ends up in a single Erlang process.

1. Each record has an owning partition, chosen by hash, and every access to it goes through a single process.

2. Since each shard is written by only one node, replication from primary to secondary flows in a single direction.

3. If the primary of partition A goes down, the secondary starts serving both primary and secondary traffic.

4. Partition B does not notice A's problem: the partitioned design limits the damage to one piece.

The in-memory multimedia database is an example of the size of this: about 2 TB of RAM across 16 partitions held about 18 billion records. When a dispatcher process became a bottleneck, the team created gen_industry, a layer above gen_factory, with several dispatchers in parallel.

06 · LESSONS AND LIMITS

What to take into the interview

The summary records that Mnesia, Erlang's database, appeared to be a major source of problems at their scale, and that the biggest stress tests were world events, especially soccer, with vertical load spikes. The summary celebrates about 40 million users per engineer.

A short operational point: Erlang allows updating code in production without stopping the system, which fits servers holding millions of live connections.

In the interview, the real-time chat case covers the general design. This one comes in as a depth argument: persistent connections cost little per unit, what limits scale is the headroom for spikes, and the defense against cascades is to partition, decouple, and replicate in pairs.