India's IT ministry ordered Elon Musk's X to implement immediate technical and procedural changes to its AI chatbot Grok after the platform generated obscene content, including AI-altered images of women. The order, issued Friday, directs X to restrict Grok from generating content involving nudity, sexualization, sexually explicit material, or any unlawful content.

The ministry gave X 72 hours to submit an action-taken report detailing the steps it has taken to prevent the hosting or dissemination of content deemed obscene, pornographic, vulgar, indecent, sexually explicit, pedophilic, or otherwise prohibited under Indian law. TechCrunch reviewed the order, which warned that failure to comply could jeopardize X's safe harbor protections, which provide legal immunity from liability for user-generated content under Indian law.

The action follows concerns raised by users who shared examples of Grok being prompted to alter images of individuals, primarily women, to make them appear to be wearing bikinis. Indian parliamentarian Priyanka Chaturvedi filed a formal complaint after these instances came to light.

Grok, developed by Musk's AI company xAI, is a large language model (LLM) designed to answer questions and generate text. LLMs like Grok are trained on massive datasets, and their outputs are generated based on patterns learned from this data. A key challenge in developing these models is ensuring they do not generate harmful or inappropriate content. This often involves implementing safeguards such as content filters and moderation policies.

The Indian government's directive highlights the growing regulatory scrutiny surrounding AI-generated content and the responsibility of tech companies to prevent misuse of these technologies. The concept of "safe harbor" is crucial for platforms like X, as it protects them from being held liable for the content posted by their users. However, this protection is contingent upon the platform taking reasonable steps to address illegal or harmful content.

The incident also underscores the potential for AI tools to be misused for malicious purposes, such as creating deepfakes or generating sexually explicit content without consent. The order from India's IT ministry serves as a warning to other tech companies developing AI models, emphasizing the need for robust safety measures and proactive content moderation.

X has not yet publicly commented on the order. The company's response and the actions it takes to address the concerns raised by the Indian government will be closely watched by the tech industry and regulators worldwide. The outcome could set a precedent for how AI-generated content is regulated in other jurisdictions.