Papers
Deep Neural Networks for YouTube Recommendations
Paul Covington, Jay Adams, Emre Sargin · RecSys 2016, 2016
The problem and the two-stage funnel
The authors describe the challenge of recommending videos on YouTube from three angles. Scale: algorithms that work on small problems stop working, and specialized distributed training and efficient serving are required. Freshness: the corpus is very dynamic, with many hours of video uploaded per second, so the system must react to new content and to the user's most recent actions, balancing novelty against established videos. Noise: historical behavior is sparse and rarely reveals true satisfaction, so the model learns from implicit feedback. The architectural answer is a funnel with two neural networks. Candidate generation takes the user's activity history and narrows a corpus of millions of videos down to a few hundred, focusing on precision and broad personalization through collaborative filtering, using coarse features such as IDs of watched videos, search tokens, and demographic data. Ranking then scores those hundreds with a much richer set of video and user features, and presents only a few dozen. The paper justifies the split: it makes it possible to recommend from a huge corpus while still being sure that the few items shown are personalized, and it also allows blending candidates coming from other sources. In practice, offline metrics (precision, recall, ranking loss) guide iteration, but the final decision comes from live A/B tests, because the two do not always correlate.
Candidate generation: recommendation as extreme classification
The paper formulates candidate generation as extreme multiclass classification: given the user and the context, predict which video, among millions of classes, will be watched at that moment. The network learns a user embedding (a function of history and context) and each video has its own embedding; a softmax classifier compares the two. Training uses implicit feedback, in which watching a video to completion is a positive example, because there is orders of magnitude more implicit history than explicit, which allows recommending long-tail items. To train with millions of classes, they use candidate sampling: they draw negatives from the background distribution and correct the sampling with importance weighting. A few thousand negative classes per example give, according to the paper, a speedup of over 100 times over the traditional softmax. They report that hierarchical softmax did not reach comparable accuracy. In the network, the sparse IDs of watched videos become embeddings (inspired by continuous bag of words language models, CBOW), are combined by averaging, concatenated with other features and passed through fully connected ReLU layers. At serving time, since calibrated probabilities are not needed, the problem becomes an approximate nearest-neighbor search in the dot-product space, which must answer in tens of milliseconds; the authors observed that A/B results were not very sensitive to the neighbor algorithm chosen.
Attributes, freshness, and the choice of label
One advantage the authors point out is that the neural network, seen as a generalization of matrix factorization, easily accepts arbitrary continuous and categorical features. Besides the video history, the inputs include search history (queries tokenized into unigrams and bigrams, embedded, and averaged), geographic region and device (embedded), and simple features such as gender, logged-in state, and age, which go straight into the network normalized to [0, 1]. The demographic features provide priors so that recommendations behave reasonably for new users. A central feature is the example age: machine learning systems tend to have an implicit bias toward the past, and the distribution produced by the recommender reflects the average watch probability over the training window, which spans several weeks. The paper feeds example age as a feature in training and sets it to zero (or slightly negative) at serving, simulating a prediction made at the end of the window. The choice of label matters too: examples come from all YouTube watches, including those on other sites, so that new content can surface and discovery can propagate through collaborative filtering. A fixed number of examples is generated per user, so that a few very active users do not dominate the loss. The authors also withhold information from the classifier: using the sequence of searches would make the model reproduce the last page of results, so searches enter as an unordered bag of tokens. Finally, they predict the next future watch, not a randomly held-out one, because that would leak information from the future and ignore asymmetric consumption patterns, such as series watched in sequence.
Ranking: rich attributes and expected display time
Ranking uses impression data to specialize and calibrate candidates for the specific interface. It can use many more features because it scores only a few hundred videos instead of millions, and it also serves to combine candidate sources whose scores are not directly comparable. The authors use hundreds of features, classified as univalent or multivalent and as item features (impression) or user/context features (query); query features are computed once per request and impression features for each scored item. The most important signals describe the user's previous interactions with the item itself and similar items, such as how many videos from the same channel they have watched. The authors also consider it crucial to pass information from candidate generation to ranking, such as which sources nominated the candidate and with what score. The frequency of past impressions introduces variation between requests. Categorical features become embeddings with simple vocabularies, built in one pass over the data before training, and features from the same ID space share the embedding. Continuous features are normalized by the cumulative distribution, approximated by linear interpolation over quantiles, and are also fed in squared and as square roots. The objective is to predict expected watch time, not clicks, because ranking by click-through rate often promotes deceptive videos that the user does not watch to the end, while watch time better captures engagement. They use weighted logistic regression: clicked impressions are weighted by the time watched and unclicked ones are weighted by one. At serving, the final activation is the exponential function, which produces the learned odds; since the fraction of clicked impressions is small (which holds in YouTube's case), those odds end up close to the expected watch time.
Evidence, trade-offs, and lessons
The experiments show that depth and additional features improve accuracy on holdout data. In candidate generation, the zero-depth model is equivalent to a linear factorization and performs similarly to the previous system; adding search, example age, and other features, together with ReLU layers in a tower shape (each layer with half the units of the previous one), improves MAP; width and depth were added until the incremental benefit diminished and convergence became difficult. In ranking, Table 1 measures the watch-time-weighted loss on next-day data: with no hidden layers the loss is 41.6%, and the configuration of 1024, 512, and 256 ReLU units reaches 34.6%. Larger layers improve the result, but they cost inference CPU, and this configuration was the best one that fit within the serving CPU budget. In this configuration, removing the powers of the continuous features raised the loss by 0.2%, and weighting positives and negatives equally raised it by 4.1%. The models have about a billion parameters and train on hundreds of billions of examples; most of the parameters sit in the high-cardinality embeddings. Among the authors' lessons: choosing the surrogate problem has an enormous impact on live results and is hard to measure offline, and keeping the history of impressions and views up to date to the second is an engineering feat that the paper leaves out of scope.
See the ideas at work
Practice the designs that use this paper in the System Design school.