Watch all the Transform 2020 sessions on-demand here.
Kayak processes more than 2 billion search queries a year. As its travel booking site has grown, Kayak has incorporated more sophisticated machine learning algorithms to help people refine their flight, hotel, and rental car searches.
The company saw AI as a way of making its products smarter and more efficient so customers could more easily plan and organize their trips. But it took years of learning to get to that point. At VentureBeat’s Transform 2019 conference, chief scientist and senior vice president of technology Matthias Keller talked about some of the lessons Kayak has learned.
One of the most important realizations was that despite headline-grabbing developments in AI, it shouldn’t be considered a ‘wonder weapon’ to solve all your problems. Keller used image processing as an example, explaining that while AI is pretty good at determining whether a picture is showing a dog or a cat, it’s not as accurate when it comes to other areas, like image classification.
Kayak uses algorithms to tag photos of hotels, determining whether they’re showing a beach, bedroom, bathroom, or gym, for example. But the tagging process isn’t perfect. Even if your AI achieves 95% accuracy, Keller said, 50 out of 1,000 predictions will still end up being wrong. For non-image problems, it’s hard enough to get 80% accuracy (where 200 out of 1,000 predictions are wrong).
June 5th: The AI Audit in NYC
Join us next week in NYC to engage with top executive leaders, delving into strategies for auditing AI models to ensure fairness, optimal performance, and ethical compliance across diverse organizations. Secure your attendance for this exclusive invite-only event.
“AI is about algorithms learning from already existing data. They’re not gonna generate any new solutions,” said Keller. “They basically give the best answer of something you have seen before in a training data set.”
He also warned the audience about the allure of using neural networks. Much has been said about how quickly they can learn when fed certain data sets. But it’s a whole different ballgame when you apply those networks to real-world business problems, where it might take weeks or months to crunch through that data — and still, most projects won’t succeed.
“And there are many reasons for that. We may have a problem that is just not within the wheelhouse of machine learning algorithms. We may have a lot of issues collecting our data together,” Keller said. “Data is very often the problem, because if we think about it, these image classifiers … were trained [on] hundreds of millions of images, and we just may not have that much data.”
Keller said companies have to carefully consider whether neural networks are right for them. If you’re intent on using them, he advised checking in with your team every couple of weeks to see if neural networks are something you should keep pursuing.
Overall, Keller advised staying flexible when using machine learning and AI because new or unseen data points can significantly affect their performance. This is something Kayak encounters frequently, as it has to factor in new airports and hotels popping up around the world.
“As much as we can try running our existing machine learning model with the belief that everything’s still gonna be fine, the better route is to continuously measure and retrain when needed,” Keller said.
