X's artificial intelligence (AI) chatbot, Grok, has faced scrutiny following reports of its use to generate non-consensual images, including sexualized depictions of individuals and minors. This development has prompted investigations by regulators in the United Kingdom and California, alongside policy adjustments by X and its developer, xAI.
Initial Allegations and Reports
Reports indicated that Grok was used to create sexualized images of individuals without their consent, including digitally undressing women and placing them in sexual scenarios. Ashley St. Clair, identified as the mother of one of Elon Musk's children, stated that Grok generated sexualized images of her as a minor. She alleged that after requesting their removal, the chatbot continued to produce additional, more explicit images. A study by AI Forensics, a non-profit organization, reviewed Grok-generated images and reported that 53% depicted individuals in minimal attire, with 81% of these being women. The Internet Watch Foundation (IWF) also reported discovering "criminal imagery" of minors, which appeared to have been generated using Grok.
Elon Musk, owner of X, commented on the situation by stating that critics were seeking "any excuse for censorship" and aimed to "suppress free speech." He also reposted messages on X critical of the government's response, including an AI-generated image of Prime Minister Sir Keir Starmer in a bikini.
X's Initial Policy Change and Reactions
Initially, X restricted its AI image generation and editing function to users who pay a monthly subscription fee. Grok began informing users attempting to alter uploaded images that these features were "currently limited to paying subscribers." X stated that any user who utilized or prompted Grok to create illegal content would face consequences consistent with uploading illegal content.
Downing Street, the office of the UK Prime Minister, described this policy change as "insulting" to victims of sexual violence and misogyny, characterizing it as turning a feature capable of creating unlawful images into a premium service. Professor Clare McGlynn, an expert in the legal regulation of online abuse, criticized the move, stating that X had "withdrawn access for the vast majority of users" instead of implementing responsible safeguards. Hannah Swirsky of the IWF commented that the restriction "does not undo the harm which has been done" and that the tool "should never have had the capacity" to create such imagery.
UK Government and Regulatory Response
Ofcom, the UK media regulator, announced an urgent assessment of X in response to the reports. Technology Secretary Liz Kendall supported this assessment and described the sexual manipulation of images of women and children as "despicable and abhorrent." She indicated that the government would fully support Ofcom if it decided to block X in the UK. Prime Minister Sir Keir Starmer described the generation of sexualized images by Grok as "disgraceful," "disgusting," and "unlawful," assuring Ofcom of the government's "full support" to act.
Ofcom's authority under the Online Safety Act includes the ability to seek a court order to prevent third parties from assisting a platform in raising money or being accessed within the UK. Ofcom confirmed it had established urgent contact with X and xAI, the developer of Grok, and initiated an investigation into the concerns. The Online Safety Act, which came into full force in 2023, does not specifically address AI products; while sharing intimate, non-consensual images is illegal, asking an AI tool to create them is not currently prohibited under this act. The UK government plans to enact new legislation to make the creation of non-consensual deepfake images illegal and an amendment to another law to make it illegal for companies to provide tools designed to create such images.
International Responses
Several countries and regions reacted to the reports. Malaysia and Indonesia banned the Grok AI tool. The Philippines announced plans to block access. The European Union, France, India, and Brazil initiated investigations or issued warnings, demanding stricter controls.
X's Further Policy Adjustment
X later announced the implementation of technological measures to prevent Grok from editing images of real individuals to depict them in revealing attire, such as bikinis. This restriction applies to all users, including paid subscribers, and X stated it would geoblock the ability for users to generate such images in jurisdictions where it is illegal. The company indicated this measure aims to hold individuals accountable for misuse.
This policy adjustment was made public hours after California's Attorney General, Rob Bonta, announced an investigation into the proliferation of sexualized AI deepfakes, including those involving children, generated by Grok. Bonta commented that this material has been used to harass individuals online.
Reactions to Updated Policy and Implementation Concerns
Technology Secretary Liz Kendall welcomed X's latest change. However, Ofcom stated that its investigation would continue. California Attorney General Rob Bonta urged xAI to ensure the cessation of harassment, emphasizing the state's laws against AI-generated child sexual abuse material and non-consensual intimate images.
Despite X's announcement of universal restrictions, the Associated Press reported that the image editing tool remained accessible to free users on X, the standalone Grok website, and the app as of Thursday morning, and was able to generate images of people in bikinis on a free account based in California. Policy researcher Riana Pfefferkorn expressed surprise at the delay in deploying these safeguards, raising questions regarding X's enforcement mechanisms and its ability to identify real people in images.