Watch all the Transform 2020 sessions on-demand here.
Telemedicine is becoming widespread, coinciding with the rise of conversational artificial intelligence (AI) tailor-made for health care. Over 71 percent of health providers now have bots, and for good reason: Studies show that hospitals could save over $100 million annually if they offered such apps and that people are roughly 53 percent more likely to patronize places they can message.
But not all bots are created equal — particularly when it comes to health services. In an effort to sort the wheat from the chaff, Mountain View, California-based user experience research firm UserTesting surveyed a group of 500 users about their opinions on five top health care bots — from Ada, Mediktor, Your.MD, Symptomate, and HealthTap. Respondents were given a list of symptoms for the common cold and food poisoning and were asked to use the bots to get a diagnose and then to evaluate their experience across several categories, including “ease of use,” “speed,” “credibility,” “aesthetics,” and “delight.” The questions they answered were used to produce quantitative percentage scores from 0 (worst) to 100 (best).
“Consumers have high expectations when it comes to their digital experiences. From ordering a car ride to managing finances to getting medical advice, consumers expect a fast, easy, and high-quality customer experience,” said Janelle Estes, chief insights officer at UserTesting. “Our study shows the importance of leveraging human insight to understand consumer sentiment and preference in order to create a superior customer experience — it can ultimately determine success or failure for next-gen apps like chatbots and conversational AI.”
Above: Mediktor came out on top with respect to handling complex diagnoses.
So how’d the bots fare? Not especially well, taken as a whole. None passed with flying colors, and several stand to improve with respect to usability and the ability to suss out complex conditions, according to UserTesting.
June 5th: The AI Audit in NYC
Join us next week in NYC to engage with top executive leaders, delving into strategies for auditing AI models to ensure fairness, optimal performance, and ethical compliance across diverse organizations. Secure your attendance for this exclusive invite-only event.
While users were satisfied with Mediktor and Ada’s diagnoses and treatments over 70 percent of the time, Your.MD’s responses mustered only a 65 percent rating, and Symptomate and HealthTap both scored below 50 percent. As for the user experience, bugs abounded in apps like HealthTap, which was rendered “nearly inoperable” for some users as a result of “loading issues.”
Credibility is another barrier bots have yet to overcome. UserTesting survey respondents expressed concern that some of the bots tested didn’t appear to comply with the Health Insurance Portability and Accountability Act, a 1996 law that provides data privacy and security to safeguard patients’ medical information. In-app grammatical errors and links to shady online resources didn’t contribute to perceived trustworthiness and led some survey participants to say they’d need additional information before they could be persuaded to take the bots’ suggestions.
Above: Ada had the highest credibility score.
A report from Pegasystems in October 2018 echoes those sentiments. It found that 65 percent of people would rather speak to a person than a chatbot because of the latter’s perceived lack of “intelligence,” and that 18 percent felt bots were “ineffective” or “annoying.” (That said, 49 percent rated chatbots highly when it came to providing basic information.)
It’s not all bad news for health care bots, though. Close to 46 percent of people surveyed said that prior to the study they’d interacted with a bot to look up medical symptoms. Overall, 73 percent said they found the five chatbots in the UserTesting’s study to be “helpful,” with only 12 percent reporting that they were “unhelpful.”
“The combined benefit to both consumers and health care providers indicates that it’s likely consumers will begin to see conversational interfaces more and more as providers rely on them to better service patients,” UserTesting wrote in its report. “The convenience and built-in empathy these apps can provide are priming consumers for global adoption.”
Above: Mediktor achieved high marks in usability.
It’s no wonder, then, that some analysts are predicting interactions with bots will exceed 2.8 billion by 2023 and that by 2025 the global bot market will reach $1.23 billion.
That said, poor user experiences threaten such continued expansion. According to a study conducted by the New York School of Medicine, 50 percent of people who download a health-related mobile app eventually abandon it because the experience is “too burdensome.”
“Chatbots have the potential to transform the way in which patients engage with their health care systems and go some way to take the pressure off overstretched staff,” UserTesting quoted Michael Larner, an analyst at Juniper Research, as saying.
