While there has certainly been no shortage of security solutions introduced as of late to help organizations mitigate threats related to COVID-19, be it thermal imaging for temperature monitoring or video analytics for mask detection, new research from the National Institute of Standards and Technology (NIST) has found that facial recognition algorithms are having a difficult time identifying people wearing face coverings.
According to a NIST Interagency Report, even the best of the 89 commercial facial recognition algorithms tested had error rates between 5% and 50% in matching digitally applied face masks with photos of the same person without a mask.
“With the arrival of the pandemic, we need to understand how face recognition technology deals with masked faces,” Mei Ngan, a NIST computer scientist and an author of the report, said in a statement. “We have begun by focusing on how an algorithm developed before the pandemic might be affected by subjects wearing face masks. Later this summer, we plan to test the accuracy of algorithms that were intentionally developed with masked faces in mind.”
In its research, the NIST team explored how well each of the algorithms was able to perform “one-to-one” matching, where a photo is compared with a different photo of the same person. The team tested the algorithms on a set of about six million photos used in previous studies. Researchers then digitally applied mask shapes to the original photos and tested the algorithms’ performance. Because real-world masks differ, the team also came up with nine mask variants, which included differences in shape, color and nose coverage.
“We can draw a few broad conclusions from the results, but there are caveats,” Ngan said. “None of these algorithms were designed to handle face masks, and the masks we used are digital creations, not the real thing.”
“With respect to accuracy with face masks, we expect the technology to continue to improve,” Ngan said. “But the data we’ve taken so far underscores one of the ideas common to previous FRVT tests: Individual algorithms perform differently. Users should get to know the algorithm they are using thoroughly and test its performance in their own work environment.”
