Netflix: what happens when you press play (Open Connect)

01 · CONTEXT

Why Netflix built its own delivery network

Open Connect is the name of the global network responsible for delivering Netflix. It is a CDN: a network that delivers internet content over HTTP and HTTPS, efficiently, bringing the content close to where people watch it.

The program began in 2011, in response to ever-growing scale, and for two reasons. The first: Netflix became a significant share of traffic on internet service providers' (ISPs') networks, and it needed to work with them directly and collaboratively. The second: the proactive, directed solution of cache is much more efficient than the standard CDN.

According to Netflix, this solution reduces overall demand on the upstream network, compared with the on-demand CDN, by several orders of magnitude.

02 · WHERE THE APPLIANCES SIT

Inside the provider or at the exchange point

The building blocks of the infrastructure are purpose-built appliances: cache appliances called Open Connect Appliances (OCAs). Their responsibility is to store the playable bits and make the content available to devices.

The OCA network is deployed in two ways. In the first, Netflix installs OCAs at internet exchange points (IXPs) in significant markets, and they interconnect with the providers present through settlement-free peering. In the second, it provides OCAs at no cost to qualified providers, and they sit directly inside the provider's network, with the same capabilities.

1. The embedded appliance sits in the provider's network, and it is the provider that controls which customers are routed to it.

2. At the exchange point, the appliance interconnects by peering with the providers present.

3. Both types of appliance have the same capabilities and report health, routes and files on disk to the control plane.

Each deployment site is tailored by the Open Connect team, based on characteristics of the local network and capacity-planning factors. For OCAs inside providers, Netflix supplies the server hardware and the provider supplies power, space and connectivity.

03 · THE PATH OF A PLAYBACK

Who chooses the appliance

Netflix's document says that an OCA does essentially two things: it reports its state to the control plane services, on AWS, and it serves content over HTTP and HTTPS when a device asks. It does not store customer data, such as viewing history, DRM information or subscriber data.

The control plane uses what the OCAs report (health metrics, learned BGP routes and files on disk) to direct clients, by URL, to the most suitable OCAs according to file availability, health and network proximity.

1. The appliances periodically report health, learned routes and content to the cache control services.

2. A user requests playback of a title.

3. The playback services check authorization and licensing and determine which files satisfy the request.

4. The steering service uses the information stored by cache control to choose the appliances and generate the URLs.

5. The playback services hand the URLs to the device and the appliance starts serving the requested files.

04 · OFF-PEAK FILL

Filling the appliances when the network is free

Netflix describes as a somewhat unique benefit being able to deploy most of the content and software updates proactively, in off-peak fill windows. It says it predicts with high accuracy what subscribers will watch and at what times.

Because it can predict this, Netflix uses off-peak bandwidth to download most of the content updates to the OCAs, in configurable time windows.

1. The control plane controls the appliances' fill behavior.

2. In the off-peak window, the appliances download the new content.

3. An appliance can also download updates from another, which minimizes the use of internet backbone capacity in the update cycle.

4. On playback, the control plane directs the device to an appliance, according to file availability, health and network proximity.

OCAs can also download updates from one another, which minimizes the use of internet backbone capacity in the update cycle. Providers with embedded OCAs also use peering to make overnight fill and updates efficient.

According to the High Scalability text, there are large OCAs, which store the entire catalog, and smaller ones, which store only part of it; the smaller ones are filled every day, during off-peak hours, by a process that Netflix names.

The control plane uses popularity algorithms and storage techniques so that content is distributed in a way that maximizes offload efficiency and quality of experience, minimizing churn, in the form of content updates.