testsetset
In debates now raging in the U.S. Congress and in legislatures around the country, China has been repeatedly invoked as the boogeyman, including by a senator introducing national AI legislation this week and a San Francisco lawmaker ahead of a vote to ban facial recognition software last week. But fears of a dystopian AI future are also swirling around Amazon’s facial recognition software system Rekognition, which the company reportedly attempted to sell to U.S Immigration and Customs Enforcement (ICE) and which has been tested by law enforcement in Washington and Oregon.
While the public is currently focused on the use of Rekognition by law enforcement and government agencies, there is another issue to consider. If the technology becomes a standard part of the smart display experience, people may balk at the idea of installing Amazon’s technology in their homes.
AI assistants, smart displays, and facial recognition
In the race to be invited into the homes of consumers purchasing AI assistants, smart speakers, smart displays, and internet of things devices, both Amazon and Google have targeted personalization as a key part of their strategy.
In fact, Google’s Nest Hub Max, the company’s first smart display with facial recognition, is due out this summer.
June 5th: The AI Audit in NYC
Join us next week in NYC to engage with top executive leaders, delving into strategies for auditing AI models to ensure fairness, optimal performance, and ethical compliance across diverse organizations. Secure your attendance for this exclusive invite-only event.
Alexa commands “Play music” and “Play messages” began offering personalized results in December 2017, and last fall Amazon introduced Alexa Hunches, one of its first proactive ways to share personalized results.
This week, those efforts may have taken big steps forward. Sources told Bloomberg on Thursday that Amazon’s Lab126 and Alexa team are making a wearable device that detects emotion with voice sentiment analysis, though the source was unable to say whether the device is being developed for commercial use. The story came a day after reports emerged that Comcast plans to make an ambient listening device and health care-related sensors. And Amazon announced improvements to its voice emotion detection AI on Tuesday.
Alexa chief scientist Rohit Prasad told VentureBeat in 2017 that Amazon is first using emotion detection to identify frustration in user voices before moving to other feelings. Former Alexa AI senior project manager Ashwin Ram has also talked about assistants like Alexa having a positive impact on the global loneliness epidemic, while medical and AI researchers like Beyond Verbal are also working on detecting disease from people’s voices. An understanding of emotions could also shape music recommendations and smart home lighting and inform a range of other services, like ambient listening for health care services.
For Google, facial recognition is the key differentiator of its Nest Hub Max, a smart display due out this summer that will personalize what appears on its screen based on the person in front of it. Personalization will begin with podcasts, local events, and recipe results, and the screen will show your contacts when you’re making video calls.
Personalization may also be used to show you reminders, news video, and calendar appointments, as well as events drawn from emails — such as upcoming flights or tickets purchased for a movie, play, or concert.
You can envision Face Match biometric identification being used to quickly verify identity and complete purchases, like the kind of fast food delivery experience Google rolled out Thursday for Google Assistant on smartphones.
The Nest Hub Max uses on-device machine learning so facial recognition scans can’t be shared online. The Hub Max will be Google’s second device with facial recognition and follows the release of Nest Hello last fall. Welcoming something as intimate as facial recognition into your home could also open the door to AI-powered health services.
Facial recognition backlash
At the same time, facial recognition software is currently the subject of intense national scrutiny. Facial recognition regulation received bipartisan support in a Congressional hearing Wednesday, achieving the rare feat of uniting progressives like Rep. Alexandra Ocasio-Cortez (D-NY) and a number of conservative Republicans, who agreed that a national moratorium on the use of facial recognition technology may be necessary.
Also on Wednesday, Amazon shareholders rejected a proposal to halt the sale of Rekognition to governments, something at least one member of Congress said should increase the resolve to pass legislation. The shareholder decision makes action by Congress even more important, Rep. Jimmy Gomez (D-CA) said shortly after the vote.
It appears Amazon’s Ring home security company also faced some opposition this week, as New York state lawmakers proposed legislation that outlaws facial recognition to determine the “dangerousness” of a person. The law takes direct aim at the sort of tech found in patents Amazon filed earlier this year that is used to identify criminals and suspicious people.
Analysis by Algorithm Justice League founder Joy Buolamwini found Rekognition lacking in its ability to accurately identify women, young people, and people with dark skin. AWS AI and policy leaders attempted to discredit the research, an effort that was condemned by a prominent group of AI researchers.
“I don’t want to see an authoritarian surveillance state, whether it’s run by a government or whether it’s run by five corporations,” Ocasio-Cortez told BuzzFeed after the hearing Wednesday, adding that she’s concerned about the spread of facial recognition alongside a “global rise in authoritarianism and nationalism.”
Ocascio-Cortez may paint all five tech giants with the same brush, but people draw a distinction between companies selling facial recognition to law enforcement and ICE and companies using facial recognition to sell goods or personalize smart display results.
Google’s facial recognition is far from innocent; it was criticized long before Rekognition for misidentifying a black woman as a gorilla. Google also provides police with information to track suspects’ location history, and it tracks a whole lot of activity on Android smartphones. But Google isn’t selling facial recognition to police or to ICE. A Google VP said last December that the company doesn’t plan to make a facial recognition API available “before working through important technology and policy questions.”
Can Amazon sell facial recognition to police and put it in people’s homes?
We don’t yet know how consumers will respond to the Nest Hub Max’s facial recognition capabilities, but it could prove a key differentiator in the smart display market.
The unique nature of facial recognition — the intimacy of recognizing a human face and the way it can be used to track a person’s movement in real time — is what led Supervisor Aaron Peskin to refer to it as a “uniquely dangerous and oppressive technology.” If the Rekognition name continues to be sullied by the current backlash against facial recognition software, any efforts to roll out devices with the technology inside could put the company at a major disadvantage.
Privacy expectations vary from person to person, but every company planning to sell technology as intimate as ambient listening devices with emotional intelligence and facial recognition software will face a trust litmus test from consumers.
Companies may see themselves as made up of segregated departments, but if Amazon ever comes out with an Echo device that uses facial recognition, the average person may make no distinction between facial recognition sold to police and facial recognition powering personalization inside their home.
Alexa has violated user privacy in scary ways before, by recording and sending a conversation with the assistant to another party and by allowing employees to listen to people’s exchanges with Alexa. Unregulated facial recognition technology could raise those stakes to a whole new level, and Rekognition is rapidly becoming the poster child for surveillance capitalism.
