The online network X has provoked major backlash after people utilized its artificial intelligence assistant Grok to alter pictures of females by stripping their garments.
Several cases have surfaced where the system has been used to virtually strip female subjects to render them in bikinis non-consensually, as well as to insert them in sexualized contexts.
Company Response and Regulatory Position
XAI, the company responsible for Grok, remained silent to requests for comment, other than an automatically-generated message stating "legacy media lies".
A government spokesperson stated that legislation is being proposed to ban nudification technology. Under a new criminal offence, anyone who distributed such tools would "be subject to a prison sentence and significant fines".
"X or Grok has the ability to stop these types of misuse if they wanted to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Position and Platform Functionality
The communications watchdog Ofcom said tech firms must "assess the potential" of people viewing unlawful material on their services, but did not confirm whether it was currently looking into X or Grok in concerning artificial intelligence pictures.
Grok is a no-cost AI assistant—with some additional features—which answers X users' queries when they tag it in a post.
It is frequently used to provide commentary or more context to other posters' comments, but people on X are also can modify an uploaded photo through its artificial intelligence picture modification function.
Background of Criticism and Policy
The tool has been condemned for allowing users to create pictures and videos with explicit content and sexualised material, and it was earlier accused of creating sexually explicit content of public figures.
"The platform has been allowing the generation and distribution of these images for months without taking any measures and we have not witnessed any challenge by regulators," Professor McGlynn added.
XAI's own terms of service bans "depicting likenesses of persons in a pornographic fashion".
In a statement, Ofcom said it was against the law to "create or share non-consensual intimate images or child sexual abuse material" and stated this included sexual deepfakes generated with AI.
It said platforms such as X were required to take "suitable measures" to "lessen the danger" of users viewing illegal material on their platforms and to delete it quickly when they learn of it.
