Do bot metrics change over time?
One question that still comes up after 2.4 billion messages at Dashbot is around buttons versus free-form communication, especially as platforms add more app-like functionality.
We first wrote about this after processing 70 million messages.
Let’s take a look at the latest insights.
June 5th: The AI Audit in NYC
Join us next week in NYC to engage with top executive leaders, delving into strategies for auditing AI models to ensure fairness, optimal performance, and ethical compliance across diverse organizations. Secure your attendance for this exclusive invite-only event.
A guy walks into a bar… and clicks ‘hi’?
The beauty of conversational interfaces is they enable users to communicate naturally in free form — to say what they want from a chatbot, and what they think of a chatbot, in their own words and voice. It’s the natural evolution of human-computer interaction.
Limiting the functionality in this medium to just button clicks can have a negative effect on user experience.
Is conversation dead?
When we previously looked at buttons and quick replies versus free-form conversation in Facebook, we saw there was a significant dropoff in engagement, as measured by sessions per user per month, when the percentage of buttons and quick replies became higher than the average.
Looking at the first three months of the year, we see an even more pronounced dropoff. The average percentage of messages across chatbots that are buttons and quick replies rose to about 50 percent. Chatbots with higher percentages tend to experience significantly less engagement.
Across Facebook chatbots in general, if the percentage of buttons and messages is more than one standard deviation above the average, the engagement is about half the engagement when it’s one standard deviation below the average.
People like to chat!
Where is chatting more popular?
Some categories tend to have significantly higher engagement with free-form communication over buttons.
Social makes the most sense — if you’re making a social chatbot, including dating chatbots, it’s better to allow free-form communication. Chatbots that skew more heavily to buttons in this category have significantly less engagement. In fact, for chatbots with button and quick reply percentages less than the average, the sessions per user per month are about 11 times greater than for chatbots with buttons and quick replies above the average.
Reference chatbots and fitness chatbots also do much better when they allow for more free-form communication. Reference chatbots have about 6.5 times the sessions per user per month when they have less than the average percentage of buttons versus above. Health and fitness chatbots have about 4 times more.
When are buttons more popular?
One of the most clear cases for which buttons have higher engagement is in games. Bot games tend to fit into two extremes — word-based games and button-based games. The engagement on button-based ones is about 3 times greater.
Shopping and entertainment (movies, videos, etc.) also tend to have higher engagement as the percentage of buttons and quick replies increases. In each case, the engagement is about 1.5 times greater in chatbots with buttons and quick reply percentages above the average versus below.
Finding a sweet spot
The right combination of free-form messaging versus buttons and quick replies can depend on the use case. Categories like travel, sports, and insurance tend to have higher engagement with the right mixture of buttons and quick replies versus free-form text.
It’s important to monitor the usage of your bot — reviewing the analytics can help you increase user engagement and retention.
Arte Merritt is the CEO and cofounder of Dashbot, a bot analytics platform for Facebook, Google Home, Alexa, Slack, Kik, and other conversational interfaces.
