California Attorney General Rob Bonta issued a cease-and-desist letter to xAI on Friday, demanding the company immediately halt the creation and distribution of deepfake, nonconsensual intimate images and child sexual abuse material (CSAM). The action follows an earlier announcement this week that the California Attorney General's office was investigating xAI, Elon Musk's artificial intelligence startup, over reports that its chatbot, Grok, was being used to generate nonconsensual sexual imagery of women and minors.

The Attorney General's office alleges that xAI is facilitating the large-scale production of nonconsensual nudes, which are then being used to harass women and girls online. "Today, I sent xAI a cease-and-desist letter, demanding the company immediately stop the creation and distribution of deepfake, nonconsensual, intimate images and child sexual abuse material," Bonta said in a press release. "The creation of this material is illegal. I fully expect xAI to immediately comply. California has zero tolerance for CSAM."

The agency has given xAI five days to demonstrate the steps it is taking to address these issues. At the center of the controversy is Grok's "spicy mode" feature, which xAI created to allow for more uninhibited and potentially controversial responses from the AI.

Deepfakes, or synthetic media, utilize artificial intelligence, specifically deep learning techniques, to create realistic but fabricated images, audio, and video. The technology raises significant ethical concerns, particularly when used to generate non-consensual pornography or to spread misinformation. The relative ease with which these images can be created and disseminated online poses a significant challenge for law enforcement and content moderation efforts.

The investigation into xAI highlights the growing concerns surrounding the potential misuse of AI technologies and the responsibility of AI developers to prevent harm. As AI models become more sophisticated and accessible, the risk of malicious actors exploiting them for nefarious purposes increases. This incident underscores the need for robust safeguards and ethical guidelines to govern the development and deployment of AI systems.

The California Attorney General's office has not yet released details on the specific evidence it has gathered to support its allegations against xAI. xAI has not yet issued a public statement in response to the cease-and-desist letter. The next steps will likely involve xAI providing a response to the Attorney General's office within the stipulated five-day period, outlining the measures it is taking to address the concerns raised. The Attorney General's office will then evaluate xAI's response and determine whether further action is necessary.