Ofcom is weaponising its new Online Safety powers to probe whether X knowingly let Grok create non-consensual nudes and child-sex imagery—violations that can trigger advertiser boycotts and nationwide site blocks.
Britain’s media regulator Ofcom formally opened an investigation Monday into X over claims that its Grok chatbot is being used to “create and share undressed images of people” and “sexualised images of children,” the watchdog confirmed. The probe is the first use of Online Safety Act powers against an AI image tool.
What triggered the probe?
- Multiple reports, including from Reuters, showed Grok producing deepfake nudes and sexualised depictions of minors when prompted in plain English.
- Prime Minister Keir Starmer called the outputs “disgusting” and “unlawful,” giving Ofcom political cover to act.
- France has already referred X to prosecutors; India’s IT ministry demanded an explanation.
What the law says
Under the Online Safety Act 2023, platforms must:
- Assess the risk of UK users encountering illegal content—defined to include both non-consensual intimate imagery (NCII) and child sexual abuse material (CSAM) even if AI-generated.
- Take “proportionate” steps to mitigate that risk.
- Remove illegal content swiftly once notified.
Failure can lead to fines up to £18 million or 10 % of global revenue, whichever is higher. In the “most serious” cases, Ofcom can ask courts to order payment firms and advertisers to cut off X, or force ISPs to block the site nationwide.
X’s immediate defence—and its holes
X told users last week it now limits image generation to premium subscribers and bans “illegal content.” But Ofcom’s focus is on prevention, not reaction: did X beforehand assess the likelihood that Grok would be misused to break UK law? Early evidence suggests minimal guardrails were in place.
Timeline: how we got here
- Feb 2025: Grok image generation rolls out globally on X.
- Dec 2025: French authority signals first legal complaint.
- 08 Jan 2026: Starmer condemns Grok outputs in Parliament.
- 12 Jan 2026: Ofcom launches formal investigation.
Why developers should care
The probe sets a precedent: regulators will treat generative-AI APIs as publishers if outputs can reach the public. Expect new compliance templates:
- Mandatory red-team datasets for sexual imagery.
- Under-18 impact assessments for any model that can produce faces.
- Audit trails of prompt filters and user reports.
Start-ups shipping image models without these documents may find VC due-diligence harder and cloud-hosting contracts cancelled.
User impact right now
If you live in the UK:
- Expect stricter prompts: even harmless requests for “bikini” or “school uniform” are already being blocked by Grok.
- Free-tier users lose image access; X may expand paywall to text-only Grok.
- Uploads of real photos could face hash-matching against child-safety databases.
A full ISP block is unlikely in 2026, but advertiser boycotts could degrade X’s UK service quality within months.
Bottom line
Ofcom’s case is narrow—did X fail its risk-assessment duty?—yet the collateral damage could reshape how every AI platform launches image features. With the EU’s AI Act and California’s SB 1047 also looming, the era of “ship first, moderate later” is officially over.
Stay ahead of the regulatory curve—get the fastest, most authoritative tech analysis every day at onlytrustedinfo.com.
