The online network X has caused significant anger after people utilized its AI chatbot Grok to manipulate pictures of women by removing their attire.
Multiple instances have been seen where the system has been employed to digitally undress female subjects to show them in swimwear without consent, as well as to insert them in inappropriate situations.
Company Response and Legal Stance
XAI, the company that created Grok, remained silent to requests for comment, aside from an standard response stating "legacy media lies".
A government representative stated that new laws is being introduced to prohibit nudification tools. Under a new law, anyone who supplied such technology would "be subject to a prison sentence and heavy fines".
"The platform is capable of halting these types of misuse if they chose to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Position and Platform Functionality
The regulator Ofcom said technology companies must "evaluate the risk" of people accessing unlawful material on their platforms, but did not confirm whether it was at this time examining X or Grok in regarding artificial intelligence pictures.
Grok is a no-cost AI assistant—with some premium functions—which responds to X users' prompts when they tag it in a message.
It is often used to provide reactions or more context to other users' remarks, but people on X are also can modify an posted image through its AI image editing feature.
History of Criticism and Policy
The system has been faulted for allowing users to produce pictures and clips with explicit content and sexually suggestive content, and it was in the past alleged to have been involved in generating inappropriate content of public figures.
"X has been allowing the creation and sharing of these images for a long time without taking any steps and we have not seen any action by oversight bodies," Professor McGlynn added.
XAI's own terms of service prohibits "showing images of persons in a sexually explicit manner".
In a statement, Ofcom said it was against the law to "create or share non-consensual intimate images or child sexual abuse material" and confirmed this applied to sexual deepfakes created with AI.
It said platforms such as X were required to take "appropriate measures" to "reduce the risk" of users coming across unlawful content on their platforms and to remove it swiftly when they are notified of it.
