150 successful machine learning models: 6 lessons learned at Booking.com
150 successful machine learning models: 6 lessons learned at Booking.com Bernadi et al., KDD’19
Here’s a paper that will reward careful study for many organisations. We’ve previously looked at the deep penetration of machine learning models in the product stacks of leading companies, and also some of the pre-requisites for being successful with it. Today’s paper choice is a wonderful summary of lessons learned integrating around 150 successful customer facing applications of machine learning at Booking.com. Oddly enough given the paper title, the six lessons are never explicitly listed or enumerated in the body of the paper, but they can be inferred from the division into sections. My interpretation of them is as follows:
- Projects introducing machine learned models deliver strong business value
- Model performance is not the same as business performance
- Be clear about the problem you’re trying to solve
- Prediction serving latency matters
- Get early feedback on model quality
- Test the business impact of your models using randomised controlled trials (follows from #2)
There are way more than 6 good pieces of advice contained within the paper though!
We found that driving true business impact is amazingly hard, plus it is difficult to isolate Continue reading
