Researchers say more than 250 ads containing AI-generated child sexual abuse material have appeared on Meta’s platforms including Facebook and Instagram since early August, weeks after the company removed dozens of similar ads following reporting from Wired.
Wired reports that the nonprofit Tech Transparency Project says that in early August, Meta deleted roughly 53 ads across Facebook, Instagram and Threads that contained child sexual abuse material, telling the outlet at the time that most of the abusive ads predated the rollout of new AI tools designed to “better detect and block” harmful content. Now, the same watchdog group says ads containing sexual imagery of children are back, some of which are identical to those previously removed.
The problem didn’t stop there. Researchers say Meta has published more than 350 abusive video ads since the end of last year. Many link to so-called “nudification” apps tied to Chinese developers. Unlike the initial batch of 53 ads, the new ones used images of real children, not stock or generic photos.
One ad used an official photograph of a minor from a European royal family, transformed into a video depicting a graphic sex act. Researchers declined to name the royal in order to protect the victim.
“Researchers identified the real-world identities of four minors used in the ads,” according to TTP, whose director is Katie Paul. Three of the four are from the United States. One is a teenager who runs a social media influencer account on a Meta platform. Another is a teen influencer with a public account. The third is a stock photo model, a real person, labeled as “pre-teen” on stock photo websites.
The ads typically open with an innocuous photo of a preteen or teen girl (only one involved a boy), paired with text claiming the image is “not just a photo” and that “there are no restriction” on how it can be used. The videos morph the children’s faces into explicit sexual content, often depicting sex acts. Clicking the ads sends users to download AI face-swapping or video apps from the Apple App Store or Google Play Store. Many carry the tagline: “This is the AI that men actually use.”
Researchers reported the ads to the National Center for Missing & Exploited Children’s CyberTipline and are consulting on how to notify those affected. According to Meta’s own ad library data, the ads were shown to more than 29,000 accounts across European Union countries and about 7,000 in the United Kingdom. More than 250 ads appeared in the US, 120 in Australia and 80 in India, though India lacks the granular ad performance data Meta provides elsewhere.
In a statement to Breitbart News, a Meta spokesperson wrote:
We don’t tolerate nudify apps or any kind of child exploitation, whether real or AI-generated. The criminals behind this activity know we’ve built strong defenses, including systems that block violating ads as well as ongoing monitoring in case we initially miss something: that’s why many of the ads flagged to us had already been removed, most had fewer than 200 impressions, and the total ad spend was under $5,000. These criminals constantly shift tactics to evade detection, which is why we keep strengthening our detection and enforcement.
The findings come as Meta is set to pay up to $16.7 billion to settle lawsuits alleging its platforms harmed children, and as it appeals a New Mexico court decision that would require improved reporting processes and human review in the state.
Some of the ads were posted by Chinese “reseller” companies, in a country where Meta’s platforms are banned. Meta has previously taken legal action against a Hong Kong-based company linked to a set of nudifier platforms. The ads pointed to nearly 50 apps combined on the Google Play Store and Apple App Store. Of the 350 ads researchers examined, about 300 directed users to AI image or video generator apps, mostly linked to developers based in China. Some of the apps had already been removed by the time Wired began reporting the story.
Breitbart News reported earlier this year that Apple and Google continue to offer “nudify” apps used to create vile deepfake pornography:
The research organization found that users can easily access these apps by searching for terms like “nudify” and “undress” in both the Apple App Store and Google Play Store. The applications can be used to digitally alter images of both celebrities and the general public to make them appear nude or partially undressed.
According to the report, the companies are not merely hosting these apps but are actively directing users to them through search results and advertisements. The Tech Transparency Project identified 18 apps with nudifying capabilities in the Apple App Store and 20 in the Google Play Store. Both platforms also use their autocomplete features to suggest names of additional nudifying apps as users type search terms.
The scale of the issue is substantial. Apps identified by the research group have been downloaded 483 million times collectively and generated approximately 122 million dollars in revenue, according to estimates from market researcher AppMagic. A spokesperson for AppMagic confirmed that the Tech Transparency Project’s work has resulted in several apps being removed and prompted others to modify their user policies
Read more at Wired here.
Lucas Nolan is a reporter for Breitbart News covering issues of AI, free speech, and online censorship.