Amazon sold facial recognition software to law enforcement. Here’s why some say it’s dangerous.

Some people raised concerns of a surveillance dystopia when Facebook first released its facial recognition feature in 2011. But truthfully, a little help tagging photos probably excited at least a few users.

Now, facial recognition technology is looking more tangibly sinister — or helpful, depending on who you ask.

At least 40 advocacy organizations signed a letter to Amazon CEO Jeff Bezos on Tuesday after it was discovered the company had sold its facial recognition software, Rekognition, to at least two different U.S. law enforcement agencies (including the city of Orlando, Florida, and the Washington County Sheriff’s Office in Oregon). The letter said the program is “primed for abuse in the hands of governments” and urged Bezos to stop selling Rekognition to government bodies.

“We are worried about the chilling effect this could have on dissent and we are against Amazon selling this to any law enforcement,” Chip Gibbons, policy and legislative counsel at a Washington, D.C., organization called Defending Rights and Dissent, said in an interview. “We have a right to privacy. If you have nothing to hide, it still doesn’t really trump that right.”

In a blog post in November, Amazon boasted that Rekognition is capable of identifying and tracking up to 100 faces in a crowd. It can also “accurately capture demographics and analyze sentiments for all faces in group photos, crowded events and public spaces such as airports and department stores.” Though there are likely innocent uses for Amazon’s artificial-intelligence-fueled software, envision an Orwellian future where cameras loom over intersections and catalogue pedestrian faces or identify political protesters in a crowd.

Imagine if a single corrupt police officer had access to the data and set up scans for a specific, law-abiding person of interest. Or simply look to Baltimore in 2015, when facial recognition software helped police identify people with outstanding warrants in photos taken at demonstrations in response to the death of Freddie Gray, a black man, in police custody.

As the letter to Bezos says, Rekognition raises “the possibility that those labeled suspicious by governments — such as undocumented immigrants or black activists — will be targeted.”

“I always say that minority communities are treated as the beta testers for these technologies,” Jonathan Capra, communications director at Restore the Fourth and host of the Privacy Patriots podcast, told Mic. Capra recalled when New York City police officers used license plate recognition software to scan cars as they pulled into a mosque for prayer.

“This is not theoretical. Once you start doing that with people’s faces in crowds, I think we know the color of the faces they’d be going after first,” Capra said.

Signatories of the Bezos letter are against the sale of Rekognition to government agencies, but perhaps there’s a future where the software’s uses could be limited for law enforcement. For example, police could try to identify faces in a crowd only to look for dangerous criminals in the most dire of situations — effectively limiting the situations to only the most urgent matters. Perhaps the use of Rekognition could also be barred from political protests.

“We want people to carry on with their lives without being under constant purview,” Capra said. “We want law enforcement to do their jobs, but we want their surveillance to at least be based on probable cause and, when applicable, be subject to a warrant.”