AI content platform Civitai has become a marketplace for celebrity deepfakes, which is prohibited by its own rules - study

Civitai, an online marketplace backed by venture capital firm Andreessen Horowitz (a16z), allows users to buy and sell instructional files for creating deepfakes of celebrities, including pornographic images that are prohibited by the platform's own rules. This is stated in a study by scientists from Stanford and Indiana universities who analyzed user requests on the site from mid-2023 to the end of 2024.

The researchers found that a significant portion of the requests were for deepfakes of real people, with 90% of them directed at women. The requests asked users to create “high-quality” models to generate images of public figures, such as blogger Charli D'Amelio or singer Gracie Abrams, often including links to their social networks. 86% of deepfake requests were for custom LoRA instruction files that “train” public AI models like Stable Diffusion to create content they weren’t originally designed for. The best entries were paid between $0.50 and $5, with almost 92% of requests completed.

The platform also provides educational materials on how to use external tools to further customize the image generators - for example, to change a person's pose. The researchers noted that the amount of pornography on the platform has increased, and the majority of weekly searches are now for NSFW content. Civitai automatically flags requests for deepfakes and prompts the people depicted to manually request removal of the content, but does not proactively moderate them.

In May 2025, the company announced it would ban all deepfake content, having previously only banned sexually explicit deepfakes. However, countless requests for deepfakes submitted prior to this ban are still available on the site, as is much of the material created as a result of those requests. That same month, the company's credit card processor stopped working with Civitai due to problems with non-consent-generated content, forcing users to switch to gift cards or cryptocurrency to purchase points for generating explicit content.

Civitai received a $5 million investment from Andreessen Horowitz in November 2023. The company's founder and CEO, Justin Mayer, described his goal at the time as creating a premier marketplace where people discover and share AI models for individual purposes. This isn't the first deepfake issue in a16z's investment portfolio—in February 2025, MIT Technology Review reported that another of the fund's companies, Botify AI, hosted AI companions resembling real-life actors under 18 who engaged in sexually charged conversations.

Earlier, the European Commission began an investigation into the Grok chatbot. It will assess whether the social network Twitter (X) complies with the requirements of EU digital legislation. Prior to this, the European Commission called images of real naked women and children created using AI Grok, distributed on Twitter, illegal and repulsive.