testsetset
Google and its early-stage incubator, Jigsaw, have launched a new tool that uses machine learning to help publishers combat online abuse.
With Perspective, Google is offering online publishers an API they can integrate into any of their platforms that facilitate user comments. Perspective taps a human-generated database of comments that have already been tagged as abusive or toxic. Through the API, publishers can connect their own comments with the hundreds of thousands on Google’s database, and Perspective then rates them based on how similar they are to those previously flagged. Perspective facilities corrections from users, too, so it should improve as it receives more feedback from people using it.
The Perspective API essentially provides the data, and publishers can elect to use this data in different ways. Comments could be flagged by publishers to be manually reviewed by moderators or by members of the community. Alternatively, a publisher could decide to illustrate to a commenter in real time that what they’re writing is abusive:
Above: Perspective
Online abuse has been a problem since the advent of the web, with many publishers ultimately ditching comments sections due to the difficulties in policing them. Earlier this month, Amazon-owned IMDb shuttered its comments section after years of battling toxicity. The IMDb board has now resurfaced on Reddit (good luck with that).
June 5th: The AI Audit in NYC
Join us next week in NYC to engage with top executive leaders, delving into strategies for auditing AI models to ensure fairness, optimal performance, and ethical compliance across diverse organizations. Secure your attendance for this exclusive invite-only event.
While most comments sections have a community-led approach for flagging abuse, and many are already able to filter out comments that contain profanities, being able to tap a gargantuan database of pre-flagged comments should prove helpful. And as more publishers pick up on it, Perspective will gain access to additional comments, which should accelerate improvement. Perspective is also now part of Google’s open-source software library, TensorFlow, and its cloud Machine Learning platform.
Google’s latest initiative comes at a time when technology companies are increasingly investing in anti-abuse smarts. Just last week, Twitter unveiled a slew of new tools to battle trolls, while back in August Microsoft announced a similar initiative.
Google says it has been testing Perspective with the New York Times, which has a team dedicated to sifting through comments before they go live — manually fielding some 11,000 comments daily.
“This problem doesn’t just impact online readers,” explained Jigsaw president Jared Cohen, in a blog post. “News organizations want to encourage engagement and discussion around their content but find that sorting through millions of comments to find those that are trolling or abusive takes a lot of money, labor, and time. As a result, many sites have shut down comments altogether. But they tell us that isn’t the solution they want. We think technology can help.”
Though the focus of Perspective is on abuse, for now, the technology could also be appropriated for other types of comments. “Over the next year we’re keen to partner and deliver new models that work in languages other than English, as well as models that can identify other perspectives, such as when comments are unsubstantial or off-topic,” added Cohen.
