Watch all the Transform 2020 sessions on-demand here.
In recent years, artificial intelligence algorithms have become very good at recommending content to users — a bit too good, you might say. Tech companies use AI to optimize their recommendations based on how users react to content. This is good for the companies serving content, since it results in users spending more time on their applications and generating more revenue.
But what’s good for companies is not necessarily good for the users. Often, what we want to see is not necessarily what we should see. But how can companies whose business model depends on stealing more and more of our attention respect ethical norms while also delivering quality content to their users?
To address this challenge, a team of scientists at IBM Research, in collaboration with MIT Media Lab, has developed an AI recommendation technique that, while optimizing its results for the user’s preferences, also makes sure it stays conformant to other constraints, such as ethical and behavioral guidelines. Led by Francesca Rossi, AI Ethics Global Leader at IBM Research, the team of scientists has demonstrated the functionality of the AI in a movie recommendation system that allows parents to set moral constraints for their children.
There have been previous attempts to integrate ethical rules into AI algorithms, but they were mostly based on static rules. For instance, a user could designate a specific outlet or category of news that an algorithm should avoid recommending. While this approach can work in certain settings, it has its limits.
June 5th: The AI Audit in NYC
Join us next week in NYC to engage with top executive leaders, delving into strategies for auditing AI models to ensure fairness, optimal performance, and ethical compliance across diverse organizations. Secure your attendance for this exclusive invite-only event.
“It’s easy to define an explicit rule set,” said Nicholas Mattei, researcher at IBM. “But in a lot of the stuff on the internet, in areas with vast amounts of data, you can’t always write down exactly all the rules that you want the machine to follow.”
To solve this problem, the method Mattei and his colleagues have developed uses machine learning to define rules by examples. “We thought that the idea of learning by example what’s appropriate and then transferring that understanding while still being reactive to the online rewards is a really interesting technical problem,” Mattei said.
Teaching ethical rules by example
The researchers chose movie recommendations as a demo project because quite a bit of movie-related data already exists and it’s a domain in which the difference between user preferences and ethical norms are clearly visible. For instance, parents would not want video streaming services to serve inappropriate content to their children, even if their children show positive reaction to them.
The AI recommendation technique uses two different training stages. The first stage happens offline, which means it takes place before the system starts interacting with the end user. During this stage, an arbiter gives the system examples that define the constraints the recommendation engine should abide by. The AI then examines those examples and the data associated with them to create its own ethical rules. As with all machine learning systems, the more examples and the more data you give it, the better it becomes at creating the rules.
In the case of movie recommendations, in the offline training phase, the parent provides the AI with a set of examples of appropriate and non-appropriate content. “The system should be able to learn from those examples and use those examples to not recommend certain types of content while working with the other user, the child,” Mattei said.
The second stage of the training takes place online in direct interaction with the end user. Like a traditional recommendation system, the AI tries to maximize its reward by optimizing its results for the preferences of the user and showing content the user will be more inclined to interact with.
Since satisfying the ethical constraints and the user’s preferences can sometimes be conflicting goals, the arbiter can then set a threshold that defines how much priority each of them gets. In the demo IBM provided, a slider lets parents choose the balance between the ethical principles and the child’s preferences.
It’s easy to see how this system can help solve other problems that are tied to the way AI recommendation systems work. The IBM researchers also tested the technique in health care, where they used the algorithm to determine optimal drug dosage by creating a balance between drug success variables and quality of life constraints in a test environment.
The team has also worked on evolving the type of examples that the system can train on. “We also experimented with examples that were not just yes/no constraints, but were things like priorities, so that we can be a bit more articulate in the way we provide examples for ethical behavior,” said Rossi, the research team’s lead.
Not without limits
In the future, the research team will work on enabling the AI to work in more complicated situations. “In this use case, we focused on single recommendations, such as whether a particular movie should be recommended or not,” Rossi said. “We plan to expand this work to also be able to put constraints on sequences of actions.”
Such a change would enable the algorithm to tackle other types of problems such as filter bubbles and tech addiction, where a single harmless action (such as checking notifications on your phone or reading news from a biased source) can have an adverse effect when repeated over long periods or done in combination with other types of similar actions.
But while the method Rossi and her colleagues have developed works well in a parent-child or doctor-patient situation, it can run into limitations in settings where only one user is involved, which accounts for most applications we use on the internet. In such situations, users would be responsible for defining their own ethical guidelines and constraints.
“In some of the systems that are more personal, you might be able to express some high-level ideas like ‘I want to see more variety’ or ‘I want to spend less time on my phone,'” said Mattei. “But if you can express them, you can also override them. You have to express your virtues and stick to them.”
A solution might be to have users choose a friend of family member to set and control rules, as some tech execs have experimented with. This might be part of the questions that the IBM research team explores as it continues its collaboration with MIT Media Lab in a broader project focused on engineering moral reasoning.
The IBM researchers will be demonstrating their algorithm at the 27th International Joint Conference on Artificial Intelligence in Stockholm, Sweden, on July 17.
Ben Dickson is a software engineer and the founder of TechTalks, a blog that explores the ways technology is solving and creating problems.
