New Mexico Sues Snapchat Over AI-Generated Images Used to Catch Child Sex Offenders

The US state of New Mexico has filed a lawsuit against Snapchat, a popular messaging app among teenagers, after reports emerged that police are using fake human images created by AI to catch child sex offenders.

Prosecutors in New Mexico recently filed a lawsuit against Snap, the company that runs Snapchat, alleging that New Mexico police are using AI to capture images of a 14-year-old girl to catch child sex offenders. This new approach has become the norm with the advent of AI, replacing the use of photos of adults who looked young or photos of police officers when they were young.

New Mexico prosecutors created a fake Snapchat profile using an AI-generated image file of a 14-year-old girl called “Heather.” Despite setting the profile to private and not following anyone, they received recommendations for risky accounts. After following one of those accounts, they were recommended 91 inappropriate accounts and adult users. When they started chatting with one of those adults, they were soon offered to exchange photos with each other and were tempted to take and send explicit photos in exchange for money.

New Mexico alleges that Snapchat is effectively aiding and abetting child sex crimes through its algorithms. The Albuquerque, New Mexico police investigation began in 2022 after a rape that took place on Snapchat. A man named Alejandro Marquez was sentenced to 18 years in prison for meeting an 11-year-old girl through Snapchat and raping her. Recently, a man named Jeremy Guthrie was sentenced to prison for raping a 12-year-old girl.

According to an investigation by the New Mexico Department of Justice, Snapchat is the largest source of illegal images and videos on the dark web, with more than 10,000 child sexual exploitation videos allegedly created on Snap alone. Carrie Goldberg, a sex crimes attorney, said Snapchat may have a hard time winning its case in New Mexico, since Omegle was shut down last year for allegedly facilitating child abuse by randomly connecting adults with children.