Facial recognition algorithms are getting better at recognizing faces in masks, according to data published on Tuesday by the National Institute for Standards and Technology (NIST). Drawing on independent testing of more than 150 separate facial recognition algorithms, the new report suggests masks may not be as big a problem for facial recognition systems as initially thought.
Vendors voluntarily submit their facial recognition algorithms to NIST for testing as part of the Facial Recognition Vendor Test (FRVT). The institute publishes results of those tests on a rolling basis as each algorithm is submitted. When NIST first examined masks’ effect on facial recognition in July, it found that algorithms weren’t great at identifying faces with masks. Unsurprisingly, it’s harder to recognize a face when the nose and mouth are covered.
NIST’s reports focus on false non-match rates (FNMR), a measure of how many matching faces slip through the algorithm without triggering an alert. In July, the error rate for some algorithms spiked to between 5 and 50 percent when they were confronted with images of masked people.
:no_upscale()/cdn.vox-cdn.com/uploads/chorus_asset/file/22133351/Screen_Shot_2020_12_01_at_9.04.08_AM.png)
But the pandemic has given developers plenty of time to focus on the mask problem, and NIST’s data shows that facial recognition algorithms are getting better at working with masked faces. Without masks, the best algorithms have a false match rate of roughly 0.3 percent — but that number still rises to 5 percent when high-coverage masks are worn.
“While a few pre-pandemic algorithms still remain within the most accurate on masked photos, some developers have submitted algorithms after the pandemic showing significantly improved accuracy and are now among the most accurate in our test,” the report reads.
NIST’s public leaderboard for facial recognition tests bears out this claim. Eight different algorithms now hold false non-match rates below 0.05 percent. Six of those eight were submitted to NIST after the first report was published in July.
The authors note a number of limitations to the study. In particular, while the tests drew on photos of real visa holders and actual border-crossing photos, they did not use actual images of masked faces. For the sake of expediency, NIST researchers instead applied masks digitally to ensure consistency across the sample. As a result, “we were not able to pursue an exhaustive simulation of the endless variations in color, design, shape, texture, bands, and ways masks can be worn,” the report notes. The digital mask was a blue surgical covering the full width of the face, but testers noted that performance varied considerably depending on how high the mask was placed on the face.
The US employs facial recognition at both land and air borders, matching travelers against their visa or passport photos as part of the biometric exit program. The NIST data is drawn from visa holders specifically who have few privacy rights over biometric information collected during the immigration process.
