According to TechCrunch, for the past two weeks, X has been inundated with AI-manipulated nude images created by its own Grok chatbot, affecting an alarming range of women including prominent figures, news personalities, and even world leaders. A December 31st research paper from Copyleaks initially estimated one image was posted per minute, but a sample from January 5th to 6th found a staggering rate of 6,700 per hour. The European Commission has taken the most aggressive action so far, ordering xAI to retain all documents related to Grok, a common precursor to a formal investigation. This follows CNN reporting suggesting Elon Musk may have personally intervened to prevent safeguards. Meanwhile, regulators in the UK, Australia, and India have issued stern warnings, with India’s MeitY demanding an “action-taken” report from X, threatening its safe harbor status in the country if unsatisfied. X’s public response has been to denounce the creation of illegal content and state that violators will face consequences.
The regulatory scramble
Here’s the thing: everyone is mad, and everyone is issuing statements, but what can they actually do? We’re seeing a painful lesson in real-time about the limits of tech regulation when it runs up against a platform owner who seems, at best, indifferent. The EU’s document preservation order is a serious move, but it’s just step one. Ofcom in the UK says it’s doing a “swift assessment,” and Australia’s eSafety commissioner notes a doubling in complaints. But these are largely procedural steps. The most concrete action is from India, with its 72-hour deadline—which, tellingly, got extended. It all feels like governments are bringing procedural knives to a disinformation gunfight. The scale of the problem is so vast and automated that traditional content moderation frameworks, built for human-posted material, are completely outmatched.
Musk’s role and X’s response
And then there’s the Elon of it all. The CNN report that he may have personally blocked safeguards is the elephant in the room. If true, it points to a core cultural problem at xAI and X that no regulatory warning can fix. The company’s official statement, posted by the X Safety account, condemns illegal imagery and promises consequences. But that’s a reactive, after-the-fact policy for a proactive, real-time technological failure. Removing the media tab from Grok’s account is a cosmetic fix. The real question is whether they’ve changed the model’s weights or implemented hard-coded blocks. Based on the continuing flood of images, it seems not. It’s a classic case of a company prioritizing “free speech” and raw capability over basic safety, and now the whole world is dealing with the toxic fallout.
Why this is different
Look, non-consensual deepfakes aren’t new. But this is different. This isn’t a fringe app or a dark web tool. It’s a feature integrated directly into one of the world’s largest social platforms, made by the platform owner’s own AI company. The barrier to abuse is practically zero. You don’t need technical skill; you just need a prompt. The Copyleaks research shows the system is being weaponized at an industrial scale. That changes everything. It turns a criminal act into a platform-enabled crisis. And it exposes the fundamental conflict when the entity that builds the tool, owns the distribution network, and sets the rules is also the one who seems least interested in limiting the tool’s most harmful outputs.
What happens next?
So what happens now? Regulators are in a bind. Fining X might not matter to Musk. Ordering reports, like India did, might just yield empty promises. The nuclear option is removing safe harbor protections, which India is hinting at. That would be huge—it would make X legally liable for every single piece of illegal content posted by its users in that jurisdiction. But would they actually pull that trigger? And if one country does it, do others follow? The other path is going after the AI model itself. The EU’s document order suggests they’re looking down that road, potentially under the AI Act. But that’s a slow process. In the meantime, as detailed in The Washington Post, the victims keep piling up. The whole mess is a forward-looking challenge that arrived about five years too early for any regulator’s comfort. They’re scrambling to catch up, and a lot of people are getting hurt in the gap.
