Watch all the Transform 2020 sessions on-demand here.
In a major about-face in public health policy, the Centers for Disease Control (CDC), U.S. Surgeon General Dr. Jerome Adams, and state and local health officials around the country recently began urging people to wear homemade face masks when they’re out in public. The directive is not meant to replace social distancing, but to reduce the spread of infection and ensure the most effective personal protective equipment goes to health care workers on the front line.
But it could also throw a wrench in a number of facial recognition applications, including those used to unlock smartphones.
Less than a year old, Google’s facial recognition system on Pixel 4 smartphones is built to recognize a person even if they’ve shaved their beard or are wearing sunglasses, but Face Unlock for Pixel 4 is rendered virtually useless by homemade face masks. A Google spokesperson told VentureBeat that Face Unlock isn’t made to recognize people wearing face masks and declined to say whether the company is working to add that capability to its system.
The Pixel 4 isn’t alone. Apple’s Face ID for iPhones launched in 2017 as one of the first facial recognition systems for smartphones. Some of the initial complaints about face masks rendering facial recognition inoperable were against Apple’s Face ID and came from Californians who kept their faces covered during the 2018 wildfire season, as well as people in parts of Asia, where it’s common for people to wear face masks when they’re sick.
June 5th: The AI Audit in NYC
Join us next week in NYC to engage with top executive leaders, delving into strategies for auditing AI models to ensure fairness, optimal performance, and ethical compliance across diverse organizations. Secure your attendance for this exclusive invite-only event.
Those frustrations have resurfaced with the emergence of COVID-19. As a workaround, in a video published last month a Tencent security employee demonstrated the ability to train Apple’s Face ID to recognize a smartphone user by doing a new facial scan with half of their face covered by a mask and the other half uncovered.
COVID-19 is expected to change the world in significant ways, from an increase in telehealth and video calls to shifts in economic and public health policy, but it may also lead to more facial recognition technology that’s capable of identifying people in masks. That technology will surely live on well after the pandemic to unlock your phone, enable purchases, and recognize people at protests or political rallies.
The new face mask recommendation from public health officials means facial recognition systems for smartphones and other settings must either adapt and grow more robust or be put on hold for a range of applications. Facial recognition is also in use in some workplaces for clocking in and clocking out of work, identity verification, and in parts of China for making purchases.
It’s not yet clear whether U.S. public health officials plan to use facial recognition in contact tracing, but questionable companies like Clearview AI are attempting to sell facial recognition to state agencies for the purpose of tracking people infected with COVID-19.
Mapping masked faces
The vast majority of AI systems today are designed for recognizing not just the area around your eyes, but also your nose, mouth, and the curvature of the lower half of your face.
Few facial recognition systems today recognize people wearing face masks, but some of the first to do so have emerged in China in recent weeks.
To create its face mask data set, Hanwang asked its employees to share photos of themselves wearing face masks. Based on those photos, the company generated thousands of simulated images of fake people in masks.
Now the company says its AI is capable of achieving 95% accuracy, but it’s designed for use in an office setting and for up to 50,000 employees, Hanwang CTO Huang Lei told ArsTechnica in March.
Also last month, researchers from Wuhan University released the Real World Masked Face Recognition data set, which they believe is the biggest masked face data set in the world. Using one of three real and simulated data sets, they claim they trained AI to achieve state-of-the-art performance, correctly recognizing people 95% of the time.
The portion of the data set that includes real people has 5,000 pictures of 525 different people wearing masks and 90,000 images of the same 525 subjects without masks. By contrast, the simulated data set is orders of magnitude larger and includes 500,000 images of 10,000 fake people.
Researchers open-sourced the collection of three data sets for the express purpose of making existing facial recognition systems around the world better at recognizing people in masks in public places like train stations or security checkpoints. They also support facial recognition for recognizing people who aren’t wearing face masks in public, which is illegal in China during the coronavirus pandemic.
“[I]t is necessary to improve the existing face recognition approaches that heavily rely on all facial feature points so that identity verification can still be performed reliably in the case of incompletely exposed faces,” a preprint paper on arXiv reads. “Our research has contributed scientific and technological power to the prevention and control of coronavirus epidemics and the resumption of production in industry. Furthermore, due to the frequent occurrence of haze weather, people will often wear masks, and the need for face recognition with masks will persist for a long time.”
In addition to making stronger facial recognition systems for use in public areas, researchers want their data sets to allow facial recognition to overtake identity authentication methods that require touch, like fingerprint scanners and keypads.
A masked future?
In an op-ed earlier this week, Northeastern University professor Woodrow Hartzog said face masks are a temporary speed bump for facial recognition. While they present a challenge, he believes face masks will not stand in the way of increased facial recognition use in the age of COVID-19.
Health officials are asking a majority of U.S. citizens and a sizable percentage of the global population to stay at home right now, but face masks could become more common when people return to work, at least until a suitable vaccine emerges, which could take more than a year.
The CDC said its face mask policy is voluntary, but in places like Riverside County, California, sheriff’s deputies will fine or jail people who do not wear face masks.
That’s all to say that as the country with the most confirmed cases of COVID-19 in the world, the United States might be a masked people for a while, and facial recognition that’s capable of recognizing individuals wearing masks may become an expected feature for systems that unlock phones, track COVID-19 outbreaks, or enforce quarantines.
Apple’s FaceID and Google’s Face Unlock don’t take face masks into account today, but don’t be surprised if COVID-19 leads to better AI for unlocking smartphones or paying for coffee, as well as for tracking COVID-19 cases and dissidents at protests. Before the novel coronavirus changed all our lives, facial recognition and fights over masks are most closely associated with protests and anti-face mask laws passed in Hong Kong last fall.
