The online platform X has sparked major anger after users used its artificial intelligence assistant Grok to manipulate pictures of women by taking off their clothing.
Numerous examples have surfaced where the technology has been employed to digitally undress female subjects to render them in bikinis non-consensually, as well as to put them in sexualized contexts.
Company Reaction and Legal Position
XAI, the firm that created Grok, remained silent to questions, other than an auto-reply message stating "legacy media lies".
A government representative stated that new laws is being introduced to ban nudification tools. Under a proposed criminal offence, anyone who provided such tools would "receive a prison sentence and heavy penalties".
"The platform has the ability to stop these types of abuse if they wanted to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Stance and Platform Functionality
The regulator Ofcom said tech firms must "consider the potential" of people viewing unlawful material on their services, but stopped short of confirming whether it was at this time examining X or Grok in relation to artificial intelligence pictures.
Grok is a no-cost AI assistant—with some premium functions—which answers X users' queries when they mention it in a post.
It is often used to provide reactions or more context to other users' comments, but people on X are also able to modify an posted picture through its AI image editing feature.
History of Criticism and Policy
The tool has been faulted for enabling users to create photos and footage with explicit content and sexualised material, and it was in the past accused of making sexually explicit material of public figures.
"X has been enabling the creation and spread of these pictures for months without taking any measures and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own acceptable use policy bans "showing images of individuals in a sexually explicit manner".
In a comment, Ofcom said it was unlawful to "make or share private pictures or child sexual abuse material" and clarified this covered explicit AI images made with AI.
It said services such as X were required to take "appropriate measures" to "reduce the risk" of users encountering unlawful content on their services and to remove it promptly when they become aware of it.
