Recommendation system

01 · CONTEXT AND REQUIREMENTS

What a recommendation system needs to solve

A recommendation system picks, for each person, a few items from a huge catalog. This case uses the YouTube recommendations paper as its main reference, which points out three difficulties, and is a good exercise in ML system design.

On scale, the paper states that highly specialized distributed learning algorithms and efficient serving systems are essential. On freshness, the system must respond well to newly uploaded content and to the user's latest actions. On noise, ground truth about user satisfaction is almost never obtained, and so implicit and noisy signals are modeled.

02 · THE TWO-STAGE FUNNEL

Candidate generation and ranking

The paper describes the system as two neural networks: one for candidate generation and another for ranking. The first takes events from the user's activity history and retrieves, from a large corpus, a small subset of videos. The second assigns a score to each video, and the highest-scoring ones are presented to the user, ordered by that score.

The advantage of the split, according to the paper, is being able to recommend from a very large corpus while still being sure that the few items shown are personalized and engaging. The design also allows mixing in candidates from other sources.

1. In the diagram, the app connects to the recommendation service, which coordinates the two stages.

2. In the YouTube paper, candidate generation takes events from the user's activity history and retrieves a small subset of videos from a large corpus.

3. At serving time, scoring reduces to a nearest neighbor search in the dot product space, for which general-purpose libraries can be used.

4. Ranking assigns a score to each video according to an objective function, using a rich set of video and user features.

5. Since only a few hundred videos are scored, ranking has access to many more features of the video and of the user's relationship with it; in the diagram, they come from the online feature store.

6. The highest-scoring videos are presented to the user, ordered by score.

03 · GENERATING CANDIDATES

Embeddings, nearest neighbors, and choosing the label

To train candidate generation, the paper uses implicit feedback from watching: finishing a video is a positive example. The justification is that there is orders of magnitude more implicit history, which makes it possible to recommend items in the tail, where explicit feedback is extremely sparse.

An embedding is a mapping of sparse entities, such as videos and users, into a dense vector. At serving time, scoring millions of items under a latency of tens of milliseconds requires an approximate scheme that is sublinear in the number of classes; since calibrated likelihoods are not needed, the problem reduces to a nearest neighbor search in the dot product space.

The choice of what to predict also matters. The paper generates training examples from all plays, including videos embedded on other sites, and not only those made on the recommendations themselves; otherwise, it would be very hard for new content to surface and the recommender would be excessively biased toward exploitation. The paper also found much better performance when predicting the user's next play than a play held out at random.

04 · RANKING AND FEATURE STORE

Rich features, watch time, and consistency between training and serving

In ranking, only a few hundred videos are scored, instead of the millions in candidate generation; that is why the paper says there is access to many more features of the video and of the user's relationship with it. Ranking is also crucial for combining candidate sources whose scores are not directly comparable.

The final objective of ranking, according to the paper, is generally a simple function of expected watch time per impression. Ranking by click-through rate tends to promote deceptive videos that the user does not finish.

Features served live require dedicated infrastructure: the paper calls serving the history of impressions and plays up to date to the second an engineering feat, outside its scope, and considers it vital for responsive recommendations.

This is where the feature store comes in. The Feast documentation describes it, among its components, with an offline store for historical feature extraction used in model training, and an online store, which serves features with low latency in production systems. The same documentation says it avoids data leakage by generating point-in-time correct feature sets, so that future values do not leak into the models during training.

1. The diagram starts at the app, whose usage events go through a queue to a feature pipeline.

2. The Feast documentation states that it depends on upstream data pipelines to produce the feature values; here, that role belongs to the feature pipeline.

3. The offline store serves historical feature extraction used in model training.

4. Point-in-time correct feature sets prevent data leakage: future feature values do not reach the model during training.

5. The feature store pushes the values to the online store, which reduces feature retrieval latency.

6. The online store serves features with low latency in production; in the diagram, the ranking service reads them from there.

Feast does not orchestrate complex flows: it depends on data pipelines upstream to produce the feature values. In return, it offers a central catalog of feature definitions, a single source of truth, and aims to make features available consistently for training and for low-latency serving.

05 · FRESHNESS AND EVALUATION

New items, repetition, and how to know whether it improved

New content matters: the paper observes that users prefer recent content, but not at the expense of relevance. ML systems often have an implicit bias toward the past. To correct this, the model receives the age of the training example as a feature; at serving time, this feature is set to zero (or slightly negative), reflecting predictions made at the end of the training window.

Features about the frequency of past impressions are critical for introducing “churn”, that is, successive requests that do not return identical lists: if a video was recommended recently and the user did not watch it, the model naturally demotes it on the next page load.

In evaluation, the paper uses offline metrics, such as precision, recall, and ranking loss, to guide improvements, but for the final determination it turns to A/B tests with live experiments, because live A/B results do not always correlate with offline ones.

06 · EVOLUTION AND LIMITS

What this case does not cover

New users are a case apart: according to the paper, demographic features provide priors so that recommendations behave reasonably for them.

To broaden the design in an interview, it is worth discussing how to measure success, what to do when the neighbor index or the online store becomes unavailable, and how to include other candidate sources.