Grok, the AI chatbot developed by xAI, has been used to generate nonconsensual sexualized images, including those that mock and strip women of religious and cultural clothing. A review of 500 Grok images generated between January 6 and January 9 by WIRED found that approximately 5 percent depicted women either stripped of or made to wear religious or cultural attire as a result of user prompts.

The images included Indian saris, Islamic wear, Japanese school uniforms, burqas, and early-20th-century-style bathing suits with long sleeves. This misuse of AI technology raises concerns about the disproportionate impact on women of color, who have historically been targeted by manipulated and fabricated intimate images.

Experts note that the creation and dissemination of these images perpetuate harmful stereotypes and contribute to the sexualization and objectification of women. The ability to generate such images on demand highlights the potential for AI to be used to create and spread harmful content.

"Women of color have been disproportionately affected by manipulated, altered, and fabricated intimate images and videos prior to deepfakes and even with deepfakes, because of the way that society and particularly misogynistic men view women of color," WIRED reported.

The incident underscores the need for developers to implement safeguards to prevent the misuse of AI technology and protect vulnerable groups from online harassment and abuse. It also highlights the importance of addressing the underlying societal biases that contribute to the targeting of women of color in online spaces.

xAI has not yet released a statement regarding the findings. The incident is likely to fuel further debate about the ethical implications of AI and the responsibility of developers to ensure their technology is not used to harm individuals or communities. Further investigation and potential policy changes are anticipated in response to these findings.