Elon Musk's social media network X has sparked widespread backlash after people utilized its AI chatbot Grok to manipulate images of females by removing their clothing.
Multiple cases have surfaced where the system has been used to digitally undress women to make them appear in swimwear without permission, as well as to insert them in inappropriate scenarios.
Company Reaction and Regulatory Stance
XAI, the organization responsible for Grok, did not respond to inquiries, aside from an standard response stating "legacy media lies".
A Home Office spokesperson stated that new laws is being proposed to ban nudification software. Under a upcoming law, anyone who distributed such technology would "receive a jail term and significant penalties".
"The platform could prevent these forms of abuse if they chose to," said a law professor, adding they "operate with impunity."
Regulator Position and Tool Functionality
The communications watchdog Ofcom said platforms must "consider the potential" of people viewing unlawful material on their networks, but did not confirm whether it was presently investigating X or Grok in regarding AI-generated images.
Grok is a free artificial intelligence—with some additional functions—which responds to X users' queries when they reference it in a post.
It is frequently used to give reactions or more background to other users' remarks, but people on X are also capable of alter an uploaded image through its AI image editing feature.
History of Controversy and Rules
The system has been criticised for enabling users to produce pictures and videos with nudity and sexualised content, and it was previously alleged to have been involved in generating inappropriate material of celebrities.
"X has been enabling the generation and spread of these pictures for months without taking any action and we have yet to see any intervention by regulators," the expert added.
XAI's own usage rules prohibits "portraying images of persons in a pornographic fashion".
In a statement, Ofcom said it was against the law to "generate or circulate private pictures or child sexual abuse material" and confirmed this applied to sexual deepfakes generated with AI.
It said services such as X were mandated to take "suitable measures" to "lessen the risk" of users encountering unlawful material on their platforms and to delete it swiftly when they are notified of it.
