Mobile voice assistants continue to improve. Both Apple’s Siri and Google’s Now are a far distance from emotional and compelling Samantha, voiced by Scarlett Johannson in 2013’s double Oscar and Golden Globe winning Her. But both show clear and continuing improvement.
Piper Jaffray analyst Gene Munster, in new investor notes released this week, says that Google Now has bested Siri in search queries.
Munster’s team put the voices to the search test (renewing testing done late last year) by challenging Android and iOS devices with an 800-question test. Half of the questions were asked outdoors, half indoors. The environments were controlled for noise. Understanding the questions and then evaluating search responses to the queries were measured for each device.
Replicating actual user modalities, the large question set spanned OS commands, general information, local information, commerce and navigation.
June 5th: The AI Audit in NYC
Join us next week in NYC to engage with top executive leaders, delving into strategies for auditing AI models to ensure fairness, optimal performance, and ethical compliance across diverse organizations. Secure your attendance for this exclusive invite-only event.
Here are the key findings as given in the investor notes:
- Google Now slightly edged Siri in accuracy, delivering more accurate answers.
- Siri has improved her relationship with Bing and Apple Maps, using Google only 3 percent of the time (a sharp drop from 27 percent at the end of 2012).
- Siri is likely to continue to de-emphasize outside search engines going forward.
- Google Now’s navigation, local, and general-information skills were its strongest capabilities.
- Siri is stronger in interpreting OS commands, such as song play where the artist is mentioned in the query. However, the analysts expect Google Now to improve rapidly as Google pushes an increasing amount of voice-compatible hardware.
- Google Now’s advantage through the use of multiple simultaneous search channels (Google Search, Play, and Maps) is eroding. Siri is improving.
Pulling the camera back, the analysts gave Siri a B-, up from a C+ last year. Google Now earned a B, also up from a C+. Of course, everyone else gives Samantha an A+ (and then some).
