YouTube DNN Recommendations (2016)

01 · CONTEXT

A funnel of two networks

The case summarizes the paper "Deep Neural Networks for YouTube Recommendations", presented at RecSys in September 2016. According to the authors, YouTube went through a paradigm shift toward using deep learning as a general-purpose solution for nearly all learning problems.

The paper points to three challenges: scale, because algorithms that work well on small problems fail at this scale; freshness, because the corpus is dynamic and the system must model newly uploaded videos and the user's latest actions; and noise, because historical behavior is hard to predict and the available signal is noisy implicit feedback.

In terms of ML system design, the system is made up of two neural networks, one for candidate generation and another for ranking. The first takes events from the user's activity history and retrieves a small subset, in the hundreds, of videos from a large corpus, offering only broad personalization through collaborative filtering. The second scores each video with a rich set of video and user features, and the highest-scoring ones are shown in order.

1. The app requests recommendations from the recommendation service.

2. The candidate generation network takes events from the user's activity history and reduces the large corpus to a small subset, in the hundreds.

3. Candidates from other sources can be mixed in with those from the candidate generation network.

4. Ranking scores each candidate. Since there are only a few hundred videos, it is feasible to use many more features of the video and of the user's relationship with it.

5. The highest-scoring videos reach the app, ordered by score.

02 · CANDIDATES

Recommendation as extreme classification

The authors formulate recommendation as extreme multiclass classification: predicting which specific video will be watched at a given moment, among millions of videos (the classes). The network learns a user embedding, a function of history and context, useful for discriminating among videos with a softmax classifier.

To train, they use implicit feedback from watches, where the user finishing a video is a positive example. The choice is based on the quantity, orders of magnitude larger, of implicit history, which makes it possible to recommend well in the long tail, where explicit feedback is very scarce.

Training with millions of classes requires a shortcut: negative classes are sampled from the background distribution (candidate sampling) and the sampling is corrected with importance weighting; for each example, the cross-entropy of the true label and the sampled negative classes is minimized.

1. In training, watch history becomes examples: finishing a video is a positive example (implicit feedback).

2. To train with millions of classes, training samples negative classes and minimizes the cross-entropy of the true label and the sampled negatives.

3. The video embeddings are learned jointly with the other model parameters, and the network learns the user embedding as a function of history and context.

4. At serving time, the user network turns the history and context into a user vector.

5. Since the softmax's calibrated probabilities are not needed, scoring becomes a nearest-neighbor search, in dot product, between the user vector and the video vectors.

6. Scoring millions of items with latency of tens of milliseconds requires an approximate scheme that is sublinear in the number of classes.

At serving time, you need to get the most likely classes to choose the first ones to present. Scoring millions of items with latency of tens of milliseconds requires an approximate scheme that is sublinear in the number of classes. Since the softmax's calibrated probabilities are not needed, the problem becomes a nearest-neighbor search in the dot-product space, for which general-purpose libraries suffice. The authors report that A/B test results were not particularly sensitive to the chosen algorithm.

The input is the watch history, a sequence of video IDs mapped to a dense vector through embeddings. Search history is treated similarly, and demographic features provide priors so that recommendations behave reasonably for new users.

03 · FRESHNESS

The example's age

Recommending newly uploaded content is extremely important for YouTube as a product. The problem, according to the paper, is that machine learning systems tend to have an implicit bias toward the past, because they are trained to predict the future.

Video popularity is highly non-stationary, but the distribution the recommender produces reflects the average watch probability over the training window, of several weeks. To correct this, the authors feed the age of the training example as a feature during training.

04 · RANKING

Predicting watch time

The main role of ranking is to use impression data to specialize and calibrate the candidates' predictions for the user interface. Since only a few hundred videos are scored, instead of the millions in candidate generation, there is access to many more features of the video and of the user's relationship with it. Ranking is also crucial for combining candidate sources whose scores are not directly comparable.

The most important signals describe the user's previous interaction with the item itself and with similar items, for example how many videos they have already watched from that channel. Features about the frequency of past impressions are critical for introducing variation (churn) in the recommendations, that is, successive requests do not return identical lists.

To predict expected watch time, they use weighted logistic regression: positive (clicked) impressions are weighted by the observed watch time, and negative (not clicked) impressions get unit weight. At inference, the exponential function is the final activation and produces the odds, which closely estimate expected watch time.

Increasing the width and depth of the hidden layers improves results, but the cost is server CPU time at inference. The configuration that starts from a 1024-unit ReLU layer and ends in a 256-unit one gave the best results and still fit within the serving CPU budget.

05 · LESSONS

Label, context, and live tests

A central lesson is that the choice of the surrogate problem has a disproportionate impact on A/B test performance, but is very hard to measure with offline experiments. The authors note that live A/B results do not always correlate with offline ones, and the final determination of a model's effectiveness comes from live tests.

Many collaborative filtering systems hide a random item from the history and predict it from the others, . In the paper, the history is reversed: a random watch is chosen and only the actions prior to it are used as input. The authors found much better performance when predicting the next watch, and conclude that ranking a future watch did well in live metrics by capturing asymmetric co-watch behavior and avoiding leakage of future information.

The training examples come from all YouTube watches, including those embedded in other sites, and not just watches from recommendations; otherwise, it would be hard for new content to show up and the recommender would be too biased toward exploitation. Generating a fixed number of examples per user weights users equally in the loss and kept a small group of very active users from dominating the loss.

It is also necessary to withhold information from the classifier, so that it does not exploit the site's structure and overfit the surrogate problem. In the paper's example, serving the user's last search page as the homepage recommendation works very poorly; by discarding sequence information and representing searches as an unordered bag of tokens, the classifier no longer directly knows where the label came from.