The online network X has provoked significant anger after users utilized its artificial intelligence assistant Grok to change pictures of females by stripping their garments.
Several instances have emerged where the system has been used to remove clothing from women to render them in bikinis non-consensually, as well as to put them in sexually explicit scenarios.
Company Response and Regulatory Stance
XAI, the firm that created Grok, offered no comment to inquiries, aside from an standard message stating "traditional news outlets lies".
A Home Office spokesperson stated that new laws is being proposed to prohibit nudification software. Under a new law, anyone who provided such technology would "be subject to a jail term and heavy fines".
"X or Grok is capable of halting these forms of misuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Regulator Position and Platform Capabilities
The regulator Ofcom said tech firms must "assess the risk" of people accessing unlawful material on their services, but did not specify whether it was at this time investigating X or Grok in concerning artificial intelligence pictures.
Grok is a free chatbot—with some additional features—which answers X users' queries when they mention it in a tweet.
It is frequently used to give commentary or further context to other users' remarks, but people on X are also able to modify an posted image through its AI image editing function.
Background of Criticism and Rules
The tool has been condemned for allowing users to create images and videos with nudity and sexually suggestive content, and it was in the past alleged to have been involved in creating inappropriate content of celebrities.
"X has been permitting the creation and sharing of these images for months without taking any steps and we have not witnessed any action by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "portraying likenesses of people in a pornographic way".
In a comment, Ofcom said it was unlawful to "make or circulate non-consensual intimate images or child sexual abuse material" and stated this included sexual deepfakes generated with AI.
It said services such as X were mandated to take "suitable actions" to "minimize the danger" of users coming across unlawful content on their services and to take down it promptly when they learn of it.
