OpenAI: a single Postgres primary for 800 million ChatGPT users

01 · CONTEXT

One writer for hundreds of millions of users

OpenAI describes how it sustains millions of queries per second for 800 million users with a single PostgreSQL primary that handles all writes and nearly 50 read replicas spread across several regions.

The database load grew more than tenfold in a year and keeps climbing. The design holds because the load is mostly reads; the company says that this, together with the optimizations, still leaves room to grow.

A single-primary architecture means one instance handles all writes while several replicas handle reads. This creates an inherent bottleneck, because writes cannot be distributed. The case shows how far this design can be pushed before moving to sharding.

02 · THE CHOICE

Why not shard the database

OpenAI's PostgreSQL remained unsharded. The central reason is that sharding the existing workloads would be complex and time-consuming, requiring changes to hundreds of endpoints and potentially taking months or even years. Since the load is mostly reads and there are extensive optimizations, the current architecture still offers headroom; the company does not rule out sharding in the future, but it is not a short-term priority.

03 · ARCHITECTURE

One writer, many replicas, and cache

OpenAI combined increasing the instance size (vertical scaling) with adding more read replicas (horizontal scaling). The diagram shows the resulting design.

1. All PostgreSQL writes arrive at the single primary.

2. Reads go to the replicas whenever possible; only the reads that are part of write transactions stay on the primary.

3. A cache layer serves most of the read traffic to relieve PostgreSQL.

4. Heavy writes that can be partitioned go to a sharded system, and new workloads are born there.

5. The primary streams the write-ahead log (WAL) to each read replica.

04 · RELIEVING THE PRIMARY

Writes, queries, and connections under control

On the write side, OpenAI fixed application bugs that generated redundant writes and introduced lazy writes, where appropriate, to smooth out spikes. On the query side, a query that joined 12 tables was behind serious incidents when its volume rose; the lesson was to avoid complex joins and, when they are necessary, to break up the query and move the join logic into the application.

Many problematic queries come from ORMs, so it is important to review the SQL they generate. Timeouts also need to be configured, such as idle_in_transaction_session_timeout, so that idle queries do not block autovacuum. These are examples of the performance anti-patterns in transactional databases.

Each instance has a maximum connection limit (5,000 on Azure PostgreSQL), and OpenAI has had incidents in which connection storms exhausted all available connections. The answer was PgBouncer, a proxy that pools and reuses connections; in benchmarks, the average connection time dropped from 50 ms to 5 ms.

1. Each database instance has a maximum connection limit, and connection storms have exhausted that limit in past incidents.

2. PgBouncer sits as a proxy between the applications and the database and reuses connections, reducing the clients' active connections.

3. Each replica has its own Kubernetes deployment with several PgBouncer pods, behind a single Service that balances traffic across the pods.

4. Proxy, clients, and replicas are in the same region to reduce network overhead.

05 · CASCADING FAILURES

Cache, isolation, and rate limits

Overload incidents tend to follow a pattern: an upstream problem causes a load spike on the database, such as widespread misses in the cache, a surge of expensive joins saturating the CPU, or a storm of writes from a new feature. As resource usage climbs, latency rises, requests time out, and retries amplify the load in a vicious cycle that can degrade the whole service.

1. The cache hit rate suddenly drops because of a failure in the cache layer.

2. The surge of misses pushes a large volume of requests straight to PostgreSQL.

3. With the key lock, only one request fetches the data from the database and repopulates the cache; the others wait for the update.

4. Low- and high-priority requests go to separate instances, so a low-priority spike does not degrade critical traffic.

Workload isolation also applies across products and services, so that one's activity does not affect another's reliability. In addition, OpenAI applies rate limiting at several layers (application, connection pooler, proxy, and query) and avoids retry intervals that are too short, which can cause retry storms.

The ORM layer was also extended to rate limit and, when necessary, completely block certain query patterns. This form of load shedding allows quick recovery from surges of expensive queries.

06 · AVAILABILITY AND LIMITS

The single point of failure and the ceiling of the design

With a single writer, if it goes down the entire service is affected. That is why critical reads were moved off the primary: if it goes down, they continue to be served by the replicas. Writes would still fail, but the impact is smaller.

1. Failure scenario: the primary goes down and writes start failing.

2. Critical reads continue to be served by the replicas.

3. The hot standby, continuously synchronized, can be promoted to reduce downtime.

4. With several replicas per region and spare capacity, the failure of one replica does not cause a regional outage.

The design has a ceiling. Since the primary streams the WAL to each replica, more replicas increase pressure on the network and CPU and make replication lag larger and more unstable. OpenAI says it cannot add replicas indefinitely without overloading the primary.

The way out under study is cascading replication, in which intermediate replicas pass the WAL on to the next ones, which would allow going beyond a hundred replicas. The cost is operational complexity, especially in failover, and the feature was still in testing.

There are limits in PostgreSQL itself as well. MVCC copies the entire row on every update, which amplifies writes, and reads have to walk through dead versions, which amplifies reads. Even a small schema change can rewrite the entire table; that is why only lightweight changes are allowed, with a strict five-second timeout, and rate-limited backfills can take more than a week.

07 · RESULTS

What the design delivered and where it hurts

OpenAI reports millions of queries per second on read-heavy workloads, nearly 50 replicas added with replication lag close to zero, low double-digit millisecond p99 latency at the client, and five nines of availability.

In twelve months there was only one PostgreSQL SEV-0 incident. It occurred at the launch of ChatGPT ImageGen, when write traffic rose more than tenfold with more than a hundred million new users in a week.

The company continues migrating the remaining write workloads, which are harder to shard, and says it will keep exploring other approaches, including sharded PostgreSQL or alternative distributed systems.