Table of Contents
This paper is an interesting industry paper from Airbnb. The most interesting part is that it’s closer to real-world application and it should the things they’ve tried and failed in Airbnb. I’ll need to re-read this paper to understand more.
Problem Definition
- Problem definition -> increase bookings
- Principle offline metrics: NDCG
Methods
- Gain of DNN: Earlier the focus was largely on feature engineering, but aer the move to deep learning, trying to do beer math on the features manually has lost its luster. It has freed us up to investigate problems at a higher level, like how can we improve our optimization objective, and are we accurately representing all our users?
- Model evolution
- Manually crafted score function
- GBDT
- Simple NN
- Lambdarank NN
- GBDT + FM
- Deep NN
- Feature engineeing
- 2 transformation - feature normalize to with the bulk of the distribution x in the {-1, 1} interval and the median mapped to 0.
- In addition to mapping features to a restricted numerical range, we ensured most of them had a smooth distribution.
TODO & Questions & Further Reading
- How can we test if the model is generalizing well beyond logged examples? e real test is of course online performance of the model, but we found the following technique useful as a sanity check: scaling all the values of a given feature in the test set, such as price to 2x, 3x, 4x etc. and observing changes in NDCG. We found that the model’s performance was remarkably stable over these values it had never seen before.
- what NN structure ??
- trained with pairwise loss ??
- toread
- Improving Deep Learning For Airbnb Search (industry, airbnb)
