Face masks are thwarting even the best facial recognition algorithms, study finds
Researchers from NIST found that face masks are causing facial recognition algorithms to fail as much as 50% of the time.
NIST digitally added face masks to immigration photos to test 89 facial recognition algorithms.
It turns out face masks aren't just effective at preventing the spread of airborne diseases like COVID-19 -- they're also successful at blocking facial recognition algorithms, researchers say.
In a report published Monday, the US National Institute of Standards and Technology found that face masks were thwarting even the most advanced facial recognition algorithms. Error rates varied from 5% to 50%, depending on an algorithm's capabilities.
Those results are troubling for the facial recognition industry, which has been scrambling to develop algorithms that can identify people through their eyes and nose alone as people turn to face masks amid the coronavirus pandemic.
Face masks are essential tools to limit the disease's spread, and governments across the US are requiring people to wear coverings. The masks have caused trouble for facial recognition software, prompting tech companies to adapt. Apple, for instance, has pushed an update so Face ID can work even when people are wearing covers.
Facial recognition algorithms rely on getting as many data points on a person's image as possible, and face masks tend to take away a lot of valuable identifying information. The algorithms are already finicky enough that improper lighting or a bad angle can fool the technology, and masks make matters worse, the study found.
One algorithm that would have an error rate of 0.3% surged to 5% when presented with images of people wearing masks, the study found. The study tested the effectiveness of 89 facial recognition algorithms against face masks.
The test looked at the algorithms' "one-to-one" matching capabilities -- essentially comparing one photo of a person to a different picture, but with a mask on. NIST used 6 million images for its research, and applied masks digitally, with different variations of the coverings.
Coronavirus updates
The study also found that the more of the nose that was covered, the more likely the mask was to stymie the algorithms. Black masks were also more likely to fool the algorithms than blue ones, the research showed.
NIST said this was the first of a series of tests for facial recognition and face masks. The agency plans to test algorithms that were created specifically for coverings later this summer.
"With the arrival of the pandemic, we need to understand how face recognition technology deals with masked faces," said Mei Ngan, a NIST researcher behind the report. "We have begun by focusing on how an algorithm developed before the pandemic might be affected by subjects wearing face masks."
Ngan said NIST expects algorithms to improve at detecting people wearing face masks. The study also acknowledged limitations because the masks used in the tests were added digitally rather than real, physical coverings.
It meant key differences like physical masks not fitting perfectly like a digitally edited photo is, or different textures and patterns that real masks have.
"We can draw a few broad conclusions from the results, but there are caveats," Ngan said. "None of these algorithms were designed to handle face masks, and the masks we used are digital creations, not the real thing."
Facial recognition researchers have been compiling photos of people wearing face masks as data for its algorithms to learn from -- in some cases, without people's knowledge.
The NIST study used photos of people applying for immigration benefits and digitally altered mask photos of travelers entering the US, according to the report.
