Mall of America’s use of facial recognition technology in partnership with law enforcement raises critical civil liberties and civil rights concerns. The ACLU of Minnesota has continuously raised alarms on this particular surveillance technology and the threat it poses to individuals’ rights of privacy and due process. Our main concerns are the misleading nature of performance scores for this tech, the lack of transparency and proper government oversight, and the quick descent of our society becoming a surveillance state and the insidious effects that has on our democracy.
MOA states in their rollout that the particular software they’ll be using has a 99.3% accuracy rate, tested by the DHS and NIST. As ACLU National details in this letter to the federal government–while they serve as great marketing for vendors hoping to sell to private companies and government agencies, these performance scores can hide deeper disparities and don’t account for real world scenarios.
Testing labs often don’t have the exact same “matching databases” as law enforcement or other users, and cannot test the full range of variables that exist in real-world situations. Use of face recognition technology on real-time video feeds is also notoriously inaccurate. And federal testing by NIST shows that even face recognition algorithms that have relatively high accuracy rates in testing can have much higher rates of false positives for black men than white men. The inaccuracies of this technology have real-world consequences. Last year the ACLU-MN took up a case of false arrest involving MOA and the use of facial recognition, showcasing the real world harms of this technology. There are at least seven other known cases across the country of wrongful arrests due to police reliance on incorrect facial recognition results. All but one of those involves wrongful arrest of a Black person.
Even if the technology was as accurate as MOA claims (and it almost certainly is not), the Mall of America boasts 40 million annual visitors. Misidentifying even a small percentage of those people would still mean thousands of wrongful identifications. MOA must answer serious questions about how this technology performs in real-world conditions (not ideal conditions in a testing lab), whether it misidentifies people of color or women at higher rates, and what safeguards are in place to prevent discrimination and abuse.
MOA’s announcement raises other serious civil liberties concerns. MOA’s partnership with law enforcement to track “Persons of Interests” inspires many questions made worse by the lack of transparency from police. What categorizes someone as a “POI”? What’s the criteria and scope of this database? Does it go beyond those who have committed past crimes and include the undocumented, or migrants ICE is looking for? Just this year, the Federal Trade Commission barred Rite Aid from using face recognition technology for surveillance, citing harm to consumers who “were erroneously accused by employees of wrongdoing because facial recognition technology falsely flagged the consumers as matching someone who had previously been identified as a shoplifter or other troublemaker.”
What we know for sure is that innocent civilians who shop at MOA will be surveilled and tracked throughout their visit with no reasonable suspicion given by the police or MOA. The immense amount of foot traffic within MOA should not be dismissed. Thousands of Minnesotans and Americans visit MOA and this technology could lead to greater data harvesting where your image could be added to unknown databases. Remember that in order to catch the “bad guys” with facial rec you need to surveil everyone. This wholesale search and tracking without probable cause or consent poses a threat to some of our most fundamental civil liberties. When people go back to school shopping or looking for a graduation present or want to take their toddler on a carousel, they are not consenting to sharing that information with the government.
As more private and public spaces utilize facial recognition technology, what we will see is a growing patchwork of surveillance systems that could track any one of us as we go about our daily lives. The ACLU-MN does not view this as a positive state of affairs and without proper regulation or government oversight, the increasing use of surveillance technologies poses a grave threat to our democracy and constitutional rights.