A woman identified as “Jane Doe” sued Elon Musk’s xAI this week, alleging the company trained its Grok AI chatbot on child pornography depicting her, in what appears to be the first case accusing xAI of training its AI on child sexual abuse material (CSAM).
Ars Technica reports that the proposed class-action lawsuit filed against Musk’s xAI, now part of SpaceX, centers on abuse Doe suffered as a preschooler in the early 2000s, when adult men raped her to produce images later sold to pedophiles online. Those images were hashed by the National Center for Missing and Exploited Children (NCMEC) and the Canadian Centre for Child Protection, groups that track known child pornography so it can be identified and removed wherever it resurfaces.
Doe gets alerts through the U.S. Department of Justice Victim Notification System whenever her abuse material turns up somewhere new. The Canadian Centre for Child Protection told her that AI-generated CSAM depicting her had shown up on xAI. According to the complaint, offenders on online forums discussed “creating AI generated CSAM of Plaintiff and other similarly situated known, legacy, victims of CSAM.”
The lawsuit claims xAI stores images Grok generates and reuses them to further train the model. A press release from Doe’s lawyers described the material as “that same material,” referring to the CSAM depicting her that investigators say fed into Grok’s outputs. The complaint itself alleges that “CSAM depicting Plaintiff with its longstanding well-known hash values has been used as a part of the dataset used by xAI.”
Breitbart News previously reported on AI training datasets that were found to contain child pornography:
The Stanford Internet Observatory, in collaboration with the Canadian Centre for Child Protection and other anti-abuse charities, conducted a study that found more than 3,200 images of suspected child sexual abuse in the AI database LAION. LAION, an index of online images and captions, has been instrumental in training leading AI image-makers such as Stable Diffusion.
This discovery has raised alarms across various sectors, including schools and law enforcement. The child pornography has enabled AI systems to produce explicit and realistic imagery of fake children and transform social media photos of real teens into deepfake nudes. Previously, it was believed that AI tools produced abusive imagery by combining adult pornography with benign photos of kids. However, the direct inclusion of explicit child images in training datasets presents a more direct and disturbing reality.
Much of Doe’s legal argument turns on how Grok’s terms of service handle user content. The complaint says Grok treats public posts on X, along with the outputs Grok itself generates, as training data by default. As the filing puts it, “Because Grok’s terms treat public X posts and Grok’s own outputs as training data by default, publicly posting an image does not just expose it to viewers, but also feeds [it] directly into the pipeline xAI uses to train and improve its model and thereby generate further images.” xAI filters violent content out of its training data, but its terms do not specifically exclude CSAM, non-consensual intimate imagery, or other sexual or inappropriate material.
The complaint also raises a harder technical question: how do you undo the damage once abuse material has already shaped a trained model? It states that “because full removal of a training example’s influence from an already-trained model is technically difficult and not something that xAI has publicly claimed to have done, any CSAM ingested into training before takedown likely continued to shape the model’s outputs even after the original images were removed from public view.”
Doe’s suit accuses X of violating federal child pornography statutes and Masha’s Law, both of which let CSAM survivors sue over the production, possession, and distribution of abuse material. Margaret E. Mabie, one of Doe’s lawyers, said in a press release that “xAI did all three.”
The lawsuit seeks money damages on behalf of every victim who can prove Grok generated CSAM based on their real photos. It also asks the court to order xAI to destroy all Grok-generated CSAM currently stored or used in training, and to permanently block xAI from generating CSAM going forward. Getting there, the complaint suggests, would mean blocking all sexualized outputs from Grok, including non-consensual intimate imagery and the NSFW “bikini pics” that Elon Musk has promoted.
Doe’s lawyer Sarah London said in the press release that Doe “has lived for nearly two decades knowing that images of the worst thing that ever happened to her are circulating among predators online, and that they can resurface at any moment. xAI must be held responsible for knowingly training its models on images of the horrific abuse she suffered, and on the abuse images of every other survivor in this class.”
Read more at Ars Technica here.
Lucas Nolan is a reporter for Breitbart News covering issues of AI, free speech, and online censorship.