Similar listings on vacation rental platforms

01 · CONTEXT

The problem: similar listings and personalized search

According to the source, search ranking and the similar-listings carousel together account for 99% of booking conversions.

The core idea is to treat a person's sequence of actions as context, the same way word embeddings are trained from sentences. The result is listing vectors learned from search sessions, which make it possible to measure similarity between listings.

The case brings together machine learning systems e busca, and is divided into: training vectors from sessions, adapting the training to the marketplace, serving similar listings and personalizing search in real time.

02 · SESSIONS AND TRAINING

From clicks to vectors

The training data is click sessions. Each session is an unbroken sequence of listing ids clicked by the same person, and a new session starts whenever there is a pause of more than 30 minutes between two consecutive clicks.

Training uses negative sampling: the vector of the central listing is pulled closer to the vectors of the listings clicked before and after it, within a window, and pushed away from randomly drawn listings.

Each listing becomes a vector of 32 real numbers, which encode location, price, type, architecture and style. The source justifies the size as a balance between offline performance and the memory needed to store the vectors in the RAM of the search machines, for real-time similarity computations.

The reported scale is vectors for 4.5 million active listings, trained on more than 800 million click sessions.

03 · ADJUSTMENTS AND EVALUATION

Adapting the training to the marketplace

First adjustment: in sessions that end with the listing being booked, the booked listing is added as global context, so that at each step the training predicts not only the clicked neighbors but also the listing that ended up being booked.

Second adjustment: searchers tend to stay within a single market. As a result, the positive contexts are mostly from the same market, while the randomly drawn negatives are mostly from other markets. This imbalance led to suboptimal within-market similarities, and the fix was to add negatives drawn from the central listing's market.

New listings have no vector because they were not in the training data. The source creates the vector as the average of the 3 closest geographic neighbors that already have an embedding, of the same type and in the same price range as the new listing.

In the offline evaluation, cosine similarities are computed between the vectors of the clicked listing and the candidate listings, the candidates are ranked, and the position of the booked listing is observed. The variant with booking as global context and market negatives had a lower average position and outperformed the other two.

1. The diagram starts with the click and booking log.

2. Sessions are assembled from clicks; a new session starts when there is a pause of more than 30 minutes between two consecutive clicks.

3. Training pulls the central listing's vector closer to the surrounding clicked listings and pushes it away from randomly drawn listings.

4. In sessions that end in a booking, the booked listing is added as global context, and training gains negatives drawn from the central listing's market.

5. A new listing gets the average vector of nearby geographic neighbors that already have a vector, of the same type and in the same price range.

6. The offline evaluation ranks the candidates by cosine similarity to the clicked listing and observes the position of the booked listing.

04 · SIMILAR LISTINGS ON THE PAGE

Serving the similar-listings carousel

Each listing page has a carousel of similar listings that recommends comparable listings available for the same dates. At the time of the embeddings test, the existing algorithm called the main search ranking model for the same location and filtered by price range and type.

With the vectors, a listing's similar listings come from the cosine similarity between its vector and the vectors of all listings in the same market available for the same dates, when check-in and check-out are set. The 12 with the highest similarity appear in the carousel.

In the A/B test, the solution based on nearest neighbors in the embedding space increased the carousel CTR by 21% and led 4.9% more guests to discover, in the carousel, the listing they ended up booking.

1. The diagram starts with the listing page.

2. Each listing page has a carousel that recommends comparable listings available for the same dates.

3. The similar listings come from the cosine similarity between the listing's vector and the vectors of all listings in the same market available on the same dates.

4. The vectors are kept in the memory of the search machines, for real-time similarity computations.

5. The 12 listings with the highest similarity appear in the carousel; in the A/B test, the carousel CTR rose 21%.

6. The algorithm that existed at the time of the test called the main search ranking for the same location and filtered by price range and type.

05 · REAL-TIME PERSONALIZATION

Using the vectors inside search ranking

For each user, the source maintains in real time, with Kafka, two sets of recent events. One holds the listings clicked in the last 2 weeks. The other holds the skipped ones, defined as highly ranked listings that the person skipped in favor of a click at a lower position.

On each search, 2 similarity measures are computed for each candidate. EmbClickSim uses per-market centroids of the clicked listings and takes the maximum similarity; EmbSkipSim measures the similarity to the skipped listings.

Both measures enter as additional signals in the search ranking model. In the partial dependence plots, higher EmbClickSim leads to a higher model score and higher EmbSkipSim leads to a lower score.

In this architecture, it is worth thinking about message queues for the user events and about cache for the recent history.

1. The diagram starts with the search client.

2. For each user, the source maintains in real time, with Kafka, two sets of recent events.

3. One set holds the listings clicked in the last 2 weeks; the other holds the skipped ones, that is, highly ranked but skipped in favor of a click at a lower position.

4. On each search, 2 similarity measures are computed for each candidate: with the clicked ones, using per-market centroids and the maximum similarity, and with the skipped ones.

5. Both similarities enter as additional signals in the search ranking model.

6. The partial dependence plots showed that higher EmbClickSim raises the score and higher EmbSkipSim lowers it.

After a successful online experiment, the feature was launched to production in the summer of 2017, according to the source.