Concepts
Two-stage recommendation: candidate generation and ranking
Intuition
Imagine picking a movie at a video rental store with millions of titles: no one evaluates each one carefully. First you quickly set aside a few hundred that match your taste, using simple cues; then you look at those few under a magnifying glass, comparing details, and keep only the best. Recommending this way lets you sweep a huge catalog without spending expensive analysis on each item.
How it works
In the YouTube paper, the funnel has two neural networks: candidate generation narrows a corpus of millions of videos down to a few hundred, and ranking scores those hundreds and presents only a few dozen. Candidate generation is formulated as extreme multiclass classification: the network learns a user embedding (a function of history and context), each video has its own embedding, and a softmax predicts which video will be watched; training uses implicit feedback and negative candidate sampling with importance weighting; with a few thousand negatives per example, the speedup is more than 100 times over the traditional softmax, according to the paper. It uses coarse features (IDs of watched videos, search tokens, demographic data) and aims for precision with broad personalization through collaborative filtering. At serving time, since calibrated probabilities are not needed, the problem becomes an approximate nearest neighbor search in dot-product space, which must respond in tens of milliseconds. Ranking, because it scores only a few hundred videos, can use hundreds of video and user features, combines candidates from several sources whose scores are not directly comparable, and optimizes expected watch time (weighted logistic regression) instead of clicks. In candidate generation, a key feature is the age of the example: it enters as a feature in training and is fixed at zero (or slightly negative) at serving, to correct the implicit bias toward the past and simulate a prediction made at the end of the training window.
In practice
The paper justifies the two-stage split because it makes it possible to recommend from a huge corpus while still being sure that the few items shown are personalized; it also makes it possible to blend in candidates from other sources, whose scores the ranking combines. The authors report that A/B results were not very sensitive to the choice of nearest-neighbor algorithm, and that the final decision about a model comes from live A/B tests, since offline metrics do not always correlate with them.
Use it in a design
Practice the cases that rely on this concept in the System Design school.