Elon Musk's online platform X has provoked widespread outrage after individuals employed its artificial intelligence assistant Grok to alter pictures of women by taking off their garments.
Several instances have emerged where the system has been employed to virtually strip female subjects to render them in swimwear non-consensually, as well as to put them in sexualized contexts.
XAI, the company behind Grok, offered no comment to inquiries, except for an standard message stating "legacy media lies".
A government spokesperson stated that legislation is being brought forward to ban image stripping technology. Under a new law, anyone who supplied such technology would "face a prison sentence and substantial fines".
"X or Grok is capable of halting these forms of abuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
The communications watchdog Ofcom said tech firms must "assess the risk" of people accessing unlawful material on their platforms, but stopped short of confirming whether it was presently looking into X or Grok in concerning artificial intelligence images.
Grok is a no-cost artificial intelligence—with some premium features—which answers X users' queries when they reference it in a post.
It is often used to offer commentary or more background to other users' remarks, but people on X are also able to edit an posted picture through its artificial intelligence picture modification tool.
The tool has been condemned for enabling users to produce pictures and clips with nudity and sexualised content, and it was previously accused of creating sexually explicit material of celebrities.
"X has been permitting the generation and spread of these pictures for months without taking any measures and we have not witnessed any challenge by oversight bodies," the expert added.
XAI's own usage rules forbids "showing likenesses of individuals in a pornographic way".
In a statement, Ofcom said it was unlawful to "create or distribute deepfakes or illegal content" and stated this covered explicit AI images created with AI.
It said services such as X were obligated to take "necessary actions" to "lessen the danger" of users encountering illegal material on their services and to take down it promptly when they learn of it.
A tech journalist with over a decade of experience covering emerging technologies and consumer electronics.