Netflix: ML for streaming quality

01 · CONTEXT

Why use ML for streaming quality

The Netflix engineering blog opens with a question: why do we need machine learning to improve streaming quality? The answer involves diversity. The company serves more than 117M subscribers worldwide, and a one-size-fits-all solution becomes less and less suitable.

Diversity shows up in networks and devices. Cellular networks can be more volatile and unstable than fixed broadband, and each group of devices has different capabilities and connection quality because of hardware.

In each session, Netflix observes network and device conditions and the quality of experience delivered, and this makes it possible to use statistical modeling and machine learning. The article covers device-side challenges, and a third-party review summarizes the fronts as predicting network quality, detecting device anomalies, and allocating resources for predictive caching.

02 · NETWORK PREDICTION

Predicting the network is more than measuring the mean

Average bandwidth and round-trip time are well-known indicators of network quality, but the text states that stability and predictability make a big difference for video streaming.

The questions the article raises are concrete: can you predict what throughput will be over the next 15 minutes from the last 15 minutes of data? And, even without predicting the exact moment of a drop, can you at least characterize the expected throughput distribution from history?

Since measurements are observed at scale, the text sees room for more complex models that combine temporal pattern recognition with context indicators to better predict network quality.

03 · QUALITY ADAPTATION

Choosing the quality of each chunk

Movies and shows are usually encoded at several video qualities, to serve different networks and devices. Each version is split into fixed-duration chunks, and with each chunk downloaded the adaptive streaming algorithm decides at what quality to download it.

1. The player downloads video from the content server in fixed-duration chunks, and for each chunk decides at what quality to download it.

2. Each session leaves behind measurements of the network, the device, and the quality delivered to the user.

3. At scale, these observations enable models that combine temporal pattern and context to predict network quality.

4. A useful application of network prediction is adapting video quality during playback: the player uses the estimate when choosing the next chunk.

Quality of experience can be measured in several ways: the initial wait time, the overall video quality, how many times playback stopped to load more video into the buffer, and the perceptible quality fluctuation. These metrics compete with each other.

The difficulty is that the feedback signal for each decision is delayed and sparse. An aggressive switch to higher quality may have no immediate consequence, but gradually drain the buffer and lead to a rebuffer on some occasions. The text indicates that ML techniques, such as recent advances in reinforcement learning, have great potential for this problem.

04 · PREDICTIVE PREFETCH

Having the video on the device before play

Another front is predicting what the person will play in order to cache part of the video on the device before they press play, which allows starting faster or at higher quality. An example from the text: someone who has been watching a series tends to play the next unwatched episode.

1. The model combines aspects of viewing history with recent interactions and other context variables.

2. With this, the model estimates what the person will play; for someone following a series, the next unwatched episode is the natural candidate.

3. Part of the predicted video is placed in the device's cache before the person presses play.

4. When the person presses play, the video can start faster or at higher quality.

5. The model operates under resource constraints: the cache size and the available bandwidth.

The text frames the problem as supervised learning: maximize the chance of caching what the person actually ended up playing, respecting the cache size and the available bandwidth. According to the article, these models produced substantial reductions in the wait time for the video to start.

05 · DEVICE ANOMALIES

Prioritizing alerts across thousands of devices

Netflix operates on more than a thousand device types, from laptops to Smart TVs, and firmware updates or app changes can affect the experience. Detecting these changes is hard and laborious, and the criterion for labeling something a real problem is hard to define.

1. Alerts fire when measured conditions cross a criterion. A liberal trigger generates false positives and unnecessary manual investigation; a trigger that is too strict can let real problems through.

2. There is a history of the alerts fired and of the final human determination on whether the problem was real and actionable.

3. This history is used to train a model that predicts the probability that a set of measured conditions is a real problem.

4. According to the article, the use of predictive modeling has already brought a large reduction in alert volume, while keeping an acceptably low false-negative rate.

Even with the problem confirmed, finding the root cause is hard. The text says statistical modeling also helps with this, by controlling for covariates.

06 · WHAT TO TAKE INTO THE INTERVIEW

Why these problems call for ML

The article summarizes what makes these problems good candidates for ML: there is enough data, the data has many dimensions, and it is hard to hand-build the minimal set of informative variables for each problem.

In an interview, structure your answer along the data path: which signal is observed, which model turns it into a prediction, which decision it feeds, and which metric proves the gain. Each front in this case follows that path, and the success metric changes from one to the next.