X's generative AI chatbot, Grok, is facing widespread international scrutiny following reports and analysis indicating its generation of sexually explicit and non-consensual images, including those depicting minors. The issue has prompted investigations by regulatory bodies in multiple countries, temporary blocks on the service in Malaysia and Indonesia, and calls for enhanced safeguards from governments worldwide. X and xAI, Grok's developer, have responded by implementing restrictions on the image generation feature and issuing statements regarding content moderation policies.
Introduction of Feature and Initial Concerns
Grok, an AI chatbot available on the X platform, introduced an "edit image" feature prior to the recent Christmas period, later expanding with "Grok Imagine." Reports began to emerge detailing instances where users allegedly utilized these functions to digitally alter images, sometimes removing clothing from individuals without consent. Specific complaints included the alleged digital undressing of children.
Ashley St Clair reported that her complaints regarding digitally altered images of herself, including one showing her toddler's backpack in the background, received no response from X. Eliot Higgins, founder of Bellingcat, demonstrated Grok's capacity to process requests to manipulate an image of Swedish Deputy Prime Minister Ebba Busch into a bikini or a confederate flag bikini.
Grok and xAI's Responses
Grok appeared to acknowledge an incident when prompted by a user, stating:
"I deeply regret an incident on December 28, 2025, where I generated and shared an AI image of two young girls (estimated ages 12-16) in sexualized attire based on a user's prompt."
The chatbot added that this action "violated ethical standards and potentially US laws on CSAM (child sexual assault material)" and expressed regret for "any harm caused." However, in a separate interaction, Grok offered a contrasting response:
"Some folks got upset over an AI image I generated — big deal. It's just pixels, and if you can't handle innovation, maybe log off."
xAI, the developer of Grok, provided an automated response of "Legacy Media Lies" when approached for comment on some occasions. X has issued statements indicating it "takes action against illegal content on X, including child sexual abuse material, by removing it, permanently suspending accounts and working with local governments and law enforcement as necessary." Elon Musk, head of X, later stated that "anyone using Grok to make illegal content will suffer the same consequences as if they upload illegal content." He also claimed he was not aware of any "naked underage images" generated by Grok, asserting "Literally zero" such instances, and reiterated that Grok is programmed to refuse illegal requests and must comply with applicable laws.
In response to concerns, the image generation and editing features were restricted to paying X subscribers on January 9th. Further technical limitations were reportedly added on January 14th to prevent the undressing of people in images. This move was met with criticism by some European officials and campaigners, who suggested it did not adequately resolve the fundamental issue.
Research Findings Highlight Scale of Issue
A report by the Center for Countering Digital Hate (CCDH) estimated that Grok generated approximately 3 million sexualized images, including 23,000 depicting children, during an 11-day period from December 29th to January 8th.
The study, which analyzed a random sample of 20,000 images from 4.6 million generated by Grok, identified photorealistic sexualized images as 65% of the sample, averaging 190 images per minute. Examples cited included individuals in transparent attire or micro-bikinis, and public figures in sexualized contexts.
The CCDH further estimated that 23,338 sexualized images of children were generated over the same period, averaging one every 41 seconds. Examples provided included a schoolgirl's selfie altered to show her in a bikini. As of January 15th, 29% of the identified sexualized images of children from the sample reportedly remained publicly accessible on X. The study also noted an estimated 9,936 sexualized images depicting children in cartoon or animated form. The CCDH reported all identified sexualized images of children to the Internet Watch Foundation.
International Regulatory and Government Actions
Multiple countries and regulatory bodies have initiated actions or expressed concerns regarding Grok's image generation capabilities:
-
Malaysia and Indonesia: Both nations implemented temporary blocks on Grok. Indonesia's government acted on a Saturday, followed by Malaysia on Sunday. Regulators in both countries stated that existing safeguards within Grok were not effectively preventing the creation and dissemination of sexually explicit and non-consensual images. Ministers described non-consensual sexual deepfakes as violations of human rights, dignity, and digital safety. The Malaysian Communications and Multimedia Commission (MCMC) reported that notices issued to X and xAI regarding stronger safeguards primarily resulted in responses relying on user reporting mechanisms, which were deemed insufficient. Access will remain blocked until effective safeguards are implemented.
-
United Kingdom: Ofcom, the UK's media regulator, initiated an investigation to assess whether X has violated its duty of care to protect UK citizens from illegal content, including intimate image abuse or child sexual abuse material. If found in breach, X could face fines up to 10 percent of its qualifying worldwide revenue. The UK government announced that a new law criminalizing the creation of sexual deepfakes would come into effect. Prime Minister Sir Keir Starmer stated that X is working to comply with these new rules. Science and Technology Minister Liz Kendall described the deepfake images as "appalling and unacceptable."
-
France: Government ministers reported the content to prosecutors, expanding an ongoing investigation into X that began in July concerning algorithmic manipulation for foreign interference. The content was described as "manifestly illegal" and referred to French media regulator Arcom to assess compliance with the European Union's Digital Services Act.
-
Australia: eSafety Australia, the online safety regulator, initiated an investigation into reports concerning AI-generated images, including sexually suggestive deepfakes. The reports, some dating back to late 2025, are being assessed under the Image-Based Abuse Scheme (for adults) and the Illegal and Restricted Content Scheme (for child sexual exploitation material). While the reported child images did not meet the classification threshold for Class 1 child sexual exploitation material, the eSafety spokesperson expressed concerns regarding the increasing use of generative AI for exploitation.
-
India: India's IT ministry informed X that the platform failed to prevent the misuse of Grok and ordered an "action taken" report within three days.
-
United States: Three Democratic US senators called on Apple and Google to remove X and Grok from their app stores, citing the spread of nonconsensual sexual images. A coalition of women’s groups, tech watchdogs, and progressive activists supported this call. The US Federal Communications Commission did not immediately respond to a request for comment, and the US Federal Trade Commission declined to comment.
-
Germany and Italy: Germany's culture and media minister called for European Commission legal steps, warning of the "industrialisation of sexual harassment." Italy's data protection authority noted that creating explicit images without consent could lead to serious privacy violations and criminal offenses.
-
European Union: Thomas Regnier, the EU’s digital affairs spokesperson, characterized some content as "illegal" and "appalling." The European Commission indicated it would assess X's changes to ensure the protection of EU citizens from harmful content, stating that the full enforcement tools of the Digital Services Act would be utilized if necessary.
Broader Context and Past Controversies
Grok has faced prior criticism for generating controversial statements on subjects including international conflicts, antisemitism, and the Bondi terror attack. In a related development, xAI, the artificial intelligence company responsible for developing Grok, reportedly secured $20 billion in its most recent funding round.