Netflix: personalized rows, RecSysOps, and foundation model

01 · CONTEXT

Many models, a growing cost

Netflix's recommendation system is not a single model. The company itself describes a variety of models of machine learning specialized, each for a distinct need, such as “Continue Watching” and “Today’s Top Picks for You”.

As personalization algorithms expanded to meet new business needs, maintaining the system became expensive. In addition, it was hard to transfer innovations from one model to another, because most are trained independently even though they use common data sources.

1. Several models start from the same interaction data sources.

2. Each specialized model is trained independently, which makes it hard to transfer innovations from one to another.

3. Many of these models are limited to a short time window of history, due to serving latency constraints or training cost.

The answer described by the team is an architecture in which learning of member preferences is centralized and can be leveraged by multiple models.

02 · DATA AND TRAINING

Interactions become tokens

Netflix compares the volume of interactions to the token volume of large language models: with more than 300 million users at the end of 2024, that is hundreds of billions of interactions. To take advantage of this volume, interactions are tokenized, which identifies the meaningful events and reduces redundancy.

Tokenization merges adjacent actions into higher-level tokens; total watch duration, for example, may need to be summed. There is a trade-off: tokenization that is too lossy throws away valuable signal, and a sequence that is too detailed may exceed processing time and memory limits.

Even so, the history of active users can have thousands of events, beyond what transformers with standard attention can handle. At inference, the context window is usually kept at hundreds of events, not because of a model limit, but because these services require millisecond latency.

To work around this in training, the team uses sparse attention, which extends the window to several hundred events while keeping efficiency, and sliding-window sampling, with overlapping windows of interactions. At inference with multi-step decoding, KV caching reuses earlier computations.

To predict the next interaction, request-time attributes, such as login time, device, or location, are combined with post-action attributes from the previous step, such as the title interacted with and the duration.

The default objective is autoregressive next-token prediction, as in GPT, but with adjustments. Interactions do not carry the same weight: a 5-minute trailer should not count as a 2-hour movie watched in full. To capture long-term dependencies, several tokens ahead can be predicted at each step, and fields such as genre can serve as auxiliary targets, with the item ID as the main target.

03 · COLD START

New titles with no history

New titles enter the catalog frequently. The model needs to estimate members' preference for newly launched titles before anyone has interacted with them.

Two capabilities address this. First, incremental training: retraining on all members' history frequently is impractical, so new models are warm-started by reusing parameters from the previous one and initializing new parameters for the new titles. Second, the model combines learnable ID embeddings with learnable embeddings of the metadata.

The blend uses an attention mechanism based on the entity's "age" instead of a simple sum. With some randomness in training, the model is encouraged to learn from the metadata and not only from the ID. This way, newly launched or pre-launch titles have reasonable embeddings even without interaction data.

04 · HOW THE MODEL IS USED

Prediction, embeddings, and fine-tuning

Consuming teams have three paths. The first is to use the model as a direct predictor, which has several prediction heads, for example for genre preferences. The second is to use member and entity embeddings, computed in batch jobs and stored for offline and online use. The third is fine-tuning with application data, integrating the full model or subgraphs.

1. Training consumes the interaction history and also the title metadata.

2. Training predicts the next interaction and is warm-started from the previous model's parameters.

3. The foundation model now concentrates the learning of member preferences.

4. Batch jobs compute embeddings and store them for offline and online use.

5. Specialized models use the embeddings as features or to generate candidates.

6. They can also integrate the model or subgraphs and fine-tune with less data and compute.

Embeddings carry a risk for those who consume them: the space has arbitrary, non-interpretable dimensions and is incompatible across different training runs. To stabilize it, the team applies a low-rank orthogonal transformation, so that the dimensions keep their meaning when the base model is retrained.

According to Netflix, experiments confirm that the scaling law also holds for the foundation model, with consistent improvements as data and model size increase.

05 · OPERATIONS

RecSysOps: keeping the system healthy

Operating a recommendation system at scale requires high availability and throughput and involves many services and teams; new code and models go to production frequently. Netflix describes RecSysOps, a set of best practices for operating the system, which includes the end-to-end monitoramento and has four components.

Detection is the most critical component, because it triggers the others. Among the lessons are not relying only on partner teams' audits and auditing all inputs and outputs from your own perspective. On the member side, a chosen item that the ranking model did not put at the top is a potential problem; on the item side, teams pointed to concerns about cold start and production bias.

In prediction, cold start matters because each item is launched only once. So, using historical data, the team built a model that predicts cold-start problems a week or more in advance.

1. The team audits the service's inputs and outputs from its own perspective, without depending only on partners' audits.

2. When the member picks an item that the ranking did not put at the top, the case is a potential problem to analyze.

3. Prediction estimates before launch whether an item will have a cold-start problem.

4. To reproduce problems, the service logs candidates, context, features, and model id for a fraction of the traffic.

5. Diagnosis reproduces the case and checks whether the cause lies in the inputs or in the model itself.

6. The resolution picks a hotfix from a menu prepared in advance.

06 · DIAGNOSIS AND RESOLUTION

From cause to fix

In diagnosis, the first step is to reproduce the problem in isolation, which requires logging candidates, context, features, and the serving model id. To reduce cost, the log covers only a fraction of the traffic.

To validate the inputs, a simple solution is to compare the feature values with those of a comparable item or member. In one case, this exposed that an item's language was misconfigured in the upstream database. If the inputs are correct, the model and its training data are investigated.

In resolution, hotfixing an ML model is hard: models are heavily optimized, slow to train, and any manual change tends to worsen the recommendations. So it is better to have a menu of hotfixes prepared in advance, simple to apply. Detection and prediction checks should run automatically and regularly.