Spotify Discover Weekly / personalization

01 · CONTEXT

A new playlist every Monday

Discover Weekly is a personalized playlist of thirty songs the person has not heard yet. According to the source article, on a Monday more than a hundred million users found the new playlist, and this repeats every Monday. The article investigates the machine learning recommendation behind it: how Spotify picks those thirty songs for each person every week.

The source is a third-party article: the author wrote it after researching and talking to a Spotify data engineer, and she herself is looking for contacts among people who work there. For that reason the case describes the architecture at a high level, as the article reports it, and not as an official account from the company.

02 · BACKGROUND

Manual curation, tags and the mix of three models

Before Spotify, services tried different paths. Songza curated playlists by hand; Pandora tagged song attributes by hand; The Echo Nest used algorithms over the audio and the text of the song; Last.fm uses collaborative filtering.

Spotify did not adopt a single technique: it mixed strategies from other services. It bought The Echo Nest to get access to audio and text analysis data and algorithms, and uses collaborative filtering algorithms similar to Last.fm's. The result is three types of model, which the article presents next.

03 · COLLABORATIVE FILTERING

Taste without stars: implicit feedback and vectors

Unlike Netflix, Spotify has no stars with which the user rates a song. The data is implicit feedback, such as play counts, plus signals like saving the track to a playlist or visiting the artist's page after listening.

To compute this at scale, the article describes a matrix in which each row is a user and each column is a song in the catalog. Each cell marks whether the person has listened to the track or not, and the matrix is sparse. A factorization of this matrix yields one vector per user and one vector per song.

1. Listening signals, such as plays, saved tracks and visits to the artist page, form implicit feedback about each person's taste.

2. These records feed a users-by-songs matrix, in which each cell marks whether the person has listened to the track or not, and the matrix is sparse.

3. The factorization produces a vector that represents each user's taste and a vector that represents each song's profile.

4. The vectors mean nothing on their own, but they are useful for comparison: a user's against other users', a song's against the others'.

04 · TEXT AND AUDIO

New tracks: what is said and what is heard

The second type of model starts from words: track metadata, news, blogs and other text on the internet. Spotify crawls the web for what is said about artists and songs, and the most-used terms form what the article calls cultural vectors, or top terms. Each term has a weight that indicates the importance of the description, and the model uses terms and weights to create a representation of the song that lets it be compared to another.

The third type analyzes the audio. There is an additional reason: a new track, with few listeners, has little for collaborative filtering to compare it against and, if it has not yet been mentioned on the internet, the text model does not capture it either. Audio models do not tell new tracks from popular ones, and the article says that including them improves the service's accuracy.

The technique is a convolutional neural network, the same technology behind facial recognition, adapted from pixels to audio. The input is time-frequency representations of audio frames, concatenated into a spectrogram. A global temporal pooling layer computes statistics of the features learned across the song, and the output predicts characteristics such as tempo, whether the track is acoustic and danceability.

1. The text model crawls the web for texts about music and for what is said about artists and songs.

2. The most-used terms get weights and become a representation of the track, comparable to another's.

3. The convolutional network receives time-frequency representations of audio frames; according to the article, this model does not tell a new track from a popular one.

4. The output predicts characteristics of the track, such as tempo, whether it is acoustic and danceability.

05 · PUTTING THE PIECES TOGETHER

From raw data to the playlist

The article sums up the whole as follows: the three types of model feed the recommendation pipeline, which finally generates the Discover Weekly playlist. These models connect to a much larger ecosystem, with large amounts of data storage and Hadoop clusters to scale the recommendations over giant matrices, internet texts and audio files.

1. Data storage and Hadoop clusters are part of the larger ecosystem the models connect to.

2. The collaborative, text and audio models, the article's three types, connect to this ecosystem.

3. The outputs of the three types of model feed the recommendation pipeline.

4. The pipeline finally feeds the Discover Weekly playlist.

5. The person receives the renewed thirty-song playlist, as happens every Monday.