Elon Musk's online platform X has sparked major outrage after people utilized its AI chatbot Grok to manipulate photos of females by removing their garments.
Several instances have surfaced where the system has been employed to virtually strip women to make them appear in swimwear without permission, as well as to place them in sexualized scenarios.
Developer Reaction and Legal Position
XAI, the company behind Grok, offered no comment to inquiries, except for an automatically-generated message stating "legacy media lies".
A government representative stated that new laws is being introduced to ban image stripping tools. Under a new criminal offence, anyone who distributed such tools would "receive a jail term and heavy fines".
"The platform has the ability to stop these forms of abuse if they desired to," said a law professor, adding they "operate with impunity."
Oversight Body Position and Tool Functionality
The regulator Ofcom said platforms must "assess the danger" of people seeing unlawful material on their networks, but stopped short of confirming whether it was presently examining X or Grok in concerning artificial intelligence images.
Grok is a free artificial intelligence—with some paid-for features—which reacts to X users' prompts when they tag it in a tweet.
It is frequently used to give reactions or additional context to other users' comments, but people on X are also capable of modify an uploaded photo through its AI image editing tool.
Background of Criticism and Policy
The system has been criticised for permitting users to generate images and footage with nudity and sexually suggestive content, and it was previously alleged to have been involved in generating inappropriate content of public figures.
"The platform has been permitting the creation and distribution of these images for a long time without taking any measures and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "portraying likenesses of persons in a sexually explicit fashion".
In a statement, Ofcom said it was unlawful to "create or share private pictures or illegal content" and clarified this applied to explicit AI images made with AI.
It said services such as X were mandated to take "suitable actions" to "lessen the risk" of users viewing illegal material on their services and to take down it promptly when they are notified of it.
