The social media platform X has provoked major outrage after individuals utilized its artificial intelligence assistant Grok to alter pictures of females by removing their garments.
Multiple instances have surfaced where the system has been used to virtually strip female subjects to render them in swimwear non-consensually, as well as to put them in inappropriate contexts.
Developer Reaction and Regulatory Position
XAI, the firm behind Grok, remained silent to questions, other than an automatically-generated response stating "traditional news outlets lies".
A government official stated that new laws is being proposed to prohibit nudification technology. Under a upcoming law, anyone who provided such technology would "receive a prison sentence and substantial fines".
"The platform is capable of halting these forms of misuse if they desired to," said a law professor, adding they "seem to have impunity."
Oversight Body Stance and Platform Functionality
The regulator Ofcom said tech firms must "assess the danger" of people viewing unlawful material on their networks, but stopped short of confirming whether it was currently examining X or Grok in regarding artificial intelligence pictures.
Grok is a free chatbot—with some premium functions—which reacts to X users' queries when they mention it in a post.
It is often used to offer commentary or more background to other users' remarks, but people on X are also able to alter an posted photo through its AI image editing function.
History of Controversy and Rules
The system has been condemned for enabling users to generate photos and videos with explicit content and sexually suggestive content, and it was earlier accused of making graphic material of public figures.
"X has been allowing the production and spread of these images for a long time without taking any action and we have not witnessed any challenge by oversight bodies," the expert added.
XAI's own acceptable use policy prohibits "showing likenesses of individuals in a sexually explicit fashion".
In a comment, Ofcom said it was against the law to "create or distribute non-consensual intimate images or child sexual abuse material" and stated this covered explicit AI images created with AI.
It said services such as X were mandated to take "suitable measures" to "reduce the risk" of users viewing illegal content on their platforms and to take down it swiftly when they learn of it.
