Analytics are key to building a better chatbot. The following is a case study on how the Facebook bot Machaao uses Dashbot Analytics to increase user experience and engagement.
Finding their passion: Cricket
By the age of four, every Indian boy has been introduced to cricket. It’s a legacy passed on from generation to generation for over four decades.
But cricket is not only a passion for Indians; it’s one of the most popular sports in the world, with an estimated 2.5 billion fanbase.
The four founders of Machaao — Harshal Dhir, Aditya, Chavan, and Sunil Chiluvuri — all hail from India and have taken their passion for cricket to new heights and mediums, spreading cricket news and scores throughout the world with the Facebook Messenger bot Machaao.
June 5th: The AI Audit in NYC
Join us next week in NYC to engage with top executive leaders, delving into strategies for auditing AI models to ensure fairness, optimal performance, and ethical compliance across diverse organizations. Secure your attendance for this exclusive invite-only event.
Prior to launching their bot, the team was working on a contextual Messenger app that enabled users to access live scores, Twitter trends, YouTube videos, and Google images, as well as book tickets via group chat.
During the 2015 Cricket World Cup, the team and their friends were following India on the Machaao app. When India made it to the semifinals, the engagement stats made the team realize the power of contextual messaging, and they filed a patent for contextual messaging.
Once Facebook opened up Messenger to developers, the founders decided to integrate their cricket score functionality into Messenger — and Ganglia, Machaao’s bot for sports, was born.
Machaao is now one of the top bots on Facebook, with 200,000+ users. The company was recently accepted into Facebook’s FbStart program.
Analytics are key — listen to your users
Every day, Harshal and Aditya spend 30 minutes each looking at Dashbot reports.
They read through the messages to check the health of their bot, and to make sure the bot is responding correctly to any messages users send in. As they point out, if the bot doesn’t respond the way users want, they will tell you!
Dashbot provides top message reports, message funnels, and even the full transcripts to enable developers to traverse through the bot to see the messages users send in as well as the bot’s responses. These reports are particularly useful for improving response effectiveness.
New features based on Dashbot insights
Machaao created three new features directly from the insights they learned through the Dashbot Top Messages report.
1. Live scores
Users kept asking for live score updates. Before, users had to send in a team name to subscribe to get notifications. Now, users can click a quick reply button to get live score updates at any time. Machaao is finding that even if the users are subscribed, they still click the quick reply to get live updates.
2. Follow player
Machaao planned to add functionality to follow a player, but they weren’t sure when would be the right time to add it. Based on the messages in the Top Message Report, the feature jumped up on the priority list, as users were asking for player info.
3. Mute / pause
The mute functionality allows users to pause notifications without unsubscribing from the bot. This has helped to greatly increase user retention. Previously, if a match was one-sided and a user’s team was losing, they would get upset and unsubscribe so as not to see the score updates. This in turn cost Machaao more, as they would end up having to pay to re-acquire the user through ads.
Each of these features has helped to increase user engagement and retention.
Increasing engagement with better responses
Machaao strives to respond to any message a user may have. In addition to the abovementioned new features, the team also added more responses to users’ messages to increase engagement, based on analysis of Dashbot reports.
For example, they noticed users in Pakistan would message “Inshallah” (if God wills it) when their team is playing well. They hired someone who knew Urdu to come up with an appropriate response: “ameen summa ameen,” or “so be it.”
One of the tricks: When the bot doesn’t already have a response or if it receives more quirky messages, it will respond with a Giphy. This helps create a personality for the bot, which increases engagement as well.
Acquiring users via smart ad campaigns
As with any platform and service, user acquisition is incredibly important and can be quite challenging.
Machaao has found Facebook Ads for Messenger to be extremely effective. They have even seen a reduction in the user acquisition cost by 90 percent since they started six months ago.
However, it required a lot of work to get to this point. The team worked really hard to optimize their campaigns. They are very thoughtful in how they spend their money. They prefer researching and working hard over experimenting and getting it wrong.
Their hard work didn’t go unnoticed. The Facebook team was so impressed with how Machaao did their campaigns that they reached out and invited Machaao into the beta access program for Ads for Messenger.
Facebook has helped the team a lot with how to leverage Ad Manager to its fullest.
At the same time, Machaao also provides feedback and suggestions to Facebook on additional features, based on what they’re seeing through Dashbot reporting. If they see on Dashbot that a user wants something, they relay the information to Facebook that they need it.
Facebook Ads may not work the same for everyone, but it does work very well for cricket. Machaao found that if the ads enhanced content users were already consuming, they would lead to higher conversion.
Machaao doesn’t make use of an intermediate landing webpage, but instead directs users directly to the bot. The key is to have a really well done bot experience — a good “bot script,” as Aditya puts it — and to execute it well. Just like in normal conversation, as soon as there is dead air, people lose interest.
An additional benefit of skipping the web landing page: It gets the user directly into the bot, and you can re-engage them later if needed.
The future for Machaao
Machaao has lots of plans for the future for additional features and functionality, as well as expanding into additional sports.
They are taking a well thought out, methodical approach, making use of Dashbot Analytics to help with decisions. They want to serve the cricket experience to the fullest, and then they will expand.
Arte Merritt is the CEO and cofounder of Dashbot, a bot analytics platform for Google Home, Alexa, Facebook, Slack, Kik, and other conversational interfaces.
