Indonesian officials reported Saturday that they are temporarily blocking access to xAI's chatbot Grok, citing concerns over the generation of non-consensual, sexualized deepfakes. The move represents one of the most decisive governmental actions taken in response to the proliferation of AI-generated imagery, often depicting real women and minors in sexually explicit and abusive scenarios, which users have been able to generate via Grok on the social network X. Both X and xAI are under the same ownership.

Meutya Hafid, Indonesia's communications and digital minister, stated that "The government views the practice of non-consensual sexual deepfakes as a serious violation of human rights, dignity, and the security of citizens in the digital space," according to a statement shared with the Guardian and other publications. The ministry has also reportedly summoned X officials to discuss the matter.

Deepfakes, in this context, refer to AI-generated media, typically images or videos, that convincingly depict individuals doing or saying things they never actually did. Grok, xAI's chatbot, utilizes large language models (LLMs) to generate text and, in some cases, images based on user prompts. The concern arises when these LLMs are used to create realistic but fabricated sexual content without the consent of the individuals depicted.

The incident highlights the growing challenge of regulating AI-generated content and the potential for misuse of powerful generative AI tools. The industry impact could be significant, potentially leading to stricter regulations and increased scrutiny of AI companies' content moderation policies.

Other governmental bodies have also taken action. India's IT ministry ordered xAI to prevent Grok from generating obscene content. The European Commission has requested that the company retain all documents related to Grok, potentially setting the stage for a formal investigation. In the United Kingdom, the communications regulator Ofcom has commented on the issue.

The blocking of Grok in Indonesia remains temporary, pending further discussions between Indonesian officials and X representatives. The outcome of these discussions will likely influence the future availability of Grok in Indonesia and could set a precedent for how other countries address the issue of AI-generated deepfakes.