Civitai, an online marketplace for AI models, has recently introduced a controversial feature allowing users to post “bounties” for creating deepfake images of real people, including normal people without significant public presence. Creators earn money for completing the bounties, which may be used for deepfake porn or other nefarious purposes.
404 Media reports that Civitai, an emerging player in the generative AI marketplace, has been under fire for its latest feature, which permits users to request and financially incentivize the creation of deepfake images of real people. This practice, involving the use of ‘bounties’ and virtual currency, raises significant concerns regarding privacy and ethical use of AI technologies.
A deepfake of Mark Zuckerberg (The Washington Post / Getty)
Deepfake images are convincing AI-generated images and video that portray real people in fictional situations. For example, Breitbart News recently reported on a scandal at a New Jersey high school when students passed around deep fake pornography of a female classmate.
The feature allows Civitai users to ask the community to create AI models that generate specific styles or compositions, and notably, images of specific real people. Contributors who successfully meet these ‘bounty’ requests are rewarded with virtual currency, transferrable to real money. The unsettling aspect of this system is its openness to abuse, particularly concerning individuals with minimal or no public online presence.
The focus of many of these bounties has been skewed towards female celebrities and social media influencers. Civitai’s policy does not permit the creation of nonconsensual sexual images, but the platform’s structure and the ease of combining different AI models make it trivial to sidestep these restrictions. The situation becomes more concerning with the revelation of bounties placed on regular individuals, such as a case involving a woman with a minimal online footprint, raising alarm about the unchecked potential for personal harm and sexual abuse.
Civitai is ranked as the seventh most popular AI platform by Andreessen Horowitz. The platform’s monetization of these practices, which began informally in their community and now formalized through the bounties feature, demonstrates a clear gap in ethical guidelines governing AI-generated content.
Read more at 404 Media here.
Lucas Nolan is a reporter for Breitbart News covering issues of free speech and online censorship.