Grok, an artificial intelligence tool developed by xAI and associated with the social media platform X, is under international scrutiny following reports of its alleged use in generating sexually explicit imagery, including material depicting children. Regulatory bodies and governments in the United Kingdom, Australia, and the European Union have initiated investigations and issued warnings to X and xAI, while the platform has implemented some restrictions on Grok's image generation capabilities.
Allegations of Inappropriate Image Generation
The Internet Watch Foundation (IWF), a UK-based non-profit, reported discovering imagery on a dark web forum depicting girls aged 11 to 13. Users on the forum claimed this content, which included "sexualised and topless imagery," was generated using Grok. The IWF classified this material as Category C under UK law, representing the lowest severity of child sexual abuse material (CSAM). The IWF further observed instances where imagery initially attributed to Grok was reportedly used with other AI tools to produce more severe Category A CSAM. Ngaire Alexander, head of the IWF’s hotline, expressed concern regarding the apparent ease and speed with which photo-realistic CSAM could be generated.
Separately, researchers from AI Forensics, a European non-profit, analyzed 20,000 images generated by Grok between December 25 and January 1. Their findings indicated that 53 percent of these images depicted individuals in minimal attire, with 81 percent of those individuals identified as women and 2 percent appearing to be 18 or younger. The research also identified Nazi and ISIS propaganda material.
Prior to these specific reports, Ofcom, the UK regulator, had contacted X and xAI regarding allegations that Grok could be used to generate "sexualised images of children" and to digitally alter women's images without consent. The BBC had observed instances on the X platform where users requested the chatbot to modify real images of women into bikinis or sexual scenarios. Other reports indicated thousands of women experienced misuse, with some users requesting images with additional physical harm such as bruising, blood, or individuals tied up.
Platform Response and Restrictions
X has stated its policy is to address illegal content, including CSAM, by removing it, permanently suspending associated accounts, and cooperating with law enforcement and government bodies. The platform affirmed that any user attempting to generate or upload illegal content via Grok would face these consequences. Elon Musk, owner of X and xAI, publicly stated that individuals uploading such images would face prosecution.
In response to the growing concerns, Grok's image generation and editing capabilities were partially restricted. X made the function exclusive to paying subscribers, and the tool appeared to cease creating bikini images for public accounts. However, reports indicated that the Grok app, which does not publicly generate images, might retain the capability to create sexually explicit content from women's pictures.
Regulatory and Governmental Actions
United Kingdom:
- Ofcom is seeking urgent responses from X and xAI, with the UK government indicating support for the regulator to issue substantial fines or restrict access to the platform under the Online Safety Act if non-compliance with UK law is found. UK Technology Secretary Liz Kendall stated that ministers were considering the possibility of barring access to X in the UK.
- The House of Commons women and equalities committee announced it would cease using X for official communications due to concerns. Individual members of parliament, including Labour chair Sarah Owen and Liberal Democrat MP Christine Jardine, have also ceased or announced plans to cease using X.
- The Information Commissioner’s Office (ICO) has contacted X and xAI to clarify measures implemented to comply with UK data protection law and safeguard individuals' rights.
- Labour MP Jess Asato has advocated for expedited legislation to ban 'nudification' apps.
- Elon Musk publicly criticized UK Prime Minister Keir Starmer, alleging that the government's actions constituted an attempt to "suppress free speech."
Australia:
- Prime Minister Anthony Albanese publicly criticized Grok, calling the generation of sexualized images of women and children "unacceptable" and stating that Australians "deserve better." He confirmed the online safety regulator would investigate the issue.
- The eSafety Commissioner, Julie Inman Grant, initiated an investigation into the increase in sexually explicit images affecting both adults and children. The Commissioner issued a "please explain" letter to X, a measure that could lead to federal court fines.
- Australia is implementing new regulations, effective March 9, that will compel AI services to restrict children's access to sexually explicit, violent, and self-harm-related content. Australia has also recently implemented a ban preventing children under 16 from accessing social media platforms.
- Despite criticisms, Australian politicians largely continue to use X for official communications, citing the need to engage where the public and journalists are active. The office of the eSafety Commissioner and Commissioner Julie Inman Grant ceased posting on X in August of last year amidst ongoing legal disputes with the platform.
European Union and Global:
- The European Commission announced an investigation into "nudify" services and mandated X to retain all documents related to Grok as part of its compliance investigation.
- Several countries, including Sweden, Italy, France, Malaysia, and India, have issued warnings to AI companies regarding such content. The Italian regulator is collaborating with the Data Protection Commission, X's lead EU privacy authority. Sweden reported its Deputy Prime Minister was targeted by a Grok user's prompt.
Broader Context and Industry Responses
Concerns have been raised regarding a perceived lack of action from major app store providers, such as Apple and Google, in addressing content concerns on X. The Australian eSafety regulator retains the power to order app stores to remove applications found to host illegal content. Google responded to a report of an ad for a 'nudification' app on YouTube by stating such ads are prohibited and the advertiser's account has been permanently suspended.
The efficacy of X for government alerts and political statements is under review, with traditional roles in disaster alerts shifting to platforms like TikTok, dedicated emergency service applications, or traditional radio broadcasts.