The social media platform X has provoked significant backlash after users employed its artificial intelligence assistant Grok to manipulate photos of women by removing their attire.
Numerous instances have emerged where the technology has been used to virtually strip women to make them appear in swimwear without permission, as well as to put them in inappropriate situations.
Company Reaction and Legal Position
XAI, the organization that created Grok, remained silent to questions, except for an auto-reply response stating "traditional news outlets lies".
A government official stated that legislation is being introduced to prohibit nudification software. Under a upcoming law, anyone who distributed such technology would "face a prison sentence and heavy penalties".
"X or Grok has the ability to stop these forms of misuse if they chose to," said a law professor, adding they "seem to have impunity."
Oversight Body Position and Tool Functionality
The regulator Ofcom said tech firms must "evaluate the danger" of people accessing illegal content on their networks, but stopped short of confirming whether it was presently examining X or Grok in concerning AI-generated images.
Grok is a free AI assistant—with some premium functions—which reacts to X users' prompts when they reference it in a post.
It is often used to offer commentary or further background to other users' remarks, but people on X are also can edit an posted image through its AI image editing function.
History of Criticism and Rules
The chatbot has been faulted for allowing users to generate images and clips with nudity and sexually suggestive content, and it was earlier alleged to have been involved in creating inappropriate material of celebrities.
"X has been permitting the generation and spread of these pictures for months without taking any steps and we have yet to see any action by regulators," the expert added.
XAI's own terms of service bans "portraying likenesses of individuals in a pornographic way".
In a comment, Ofcom said it was against the law to "generate or distribute deepfakes or child sexual abuse material" and confirmed this applied to explicit AI images generated with AI.
It said networks such as X were mandated to take "appropriate steps" to "reduce the danger" of users coming across illegal material on their platforms and to take down it promptly when they become aware of it.
