California Attorney General Rob Bonta issued a cease-and-desist letter to xAI on Friday, demanding the company immediately halt the creation and distribution of deepfake, nonconsensual intimate images and child sexual abuse material (CSAM). The action follows an earlier announcement this week that the California Attorney General's office was investigating xAI over reports that its chatbot, Grok, was being used to generate nonconsensual sexual imagery of women and minors.
"Today, I sent xAI a cease-and-desist letter, demanding the company immediately stop the creation and distribution of deepfake, nonconsensual, intimate images and child sexual abuse material," Bonta said in a press release. "The creation of this material is illegal. I fully expect xAI to immediately comply. California has zero tolerance for CSAM."
The Attorney General's office claimed that xAI appeared to be facilitating the large-scale production of nonconsensual nudes, which are allegedly being used to harass women and girls online. The agency expects xAI to demonstrate within five days the steps it is taking to address these issues.
At the center of the controversy is Grok's "spicy mode" feature, which xAI created to allow for more unfiltered responses from the AI. This feature, designed to provide more engaging and provocative interactions, has seemingly been exploited to generate harmful and illegal content. Deepfakes, which are AI-generated synthetic media, can convincingly depict individuals doing or saying things they never did, raising serious concerns about defamation, privacy violations, and the potential for misuse in creating non-consensual pornography.
The legal and ethical implications of AI-generated content are becoming increasingly prominent as AI technology advances. The ability to create realistic and convincing fake images and videos raises questions about accountability and the responsibility of AI developers to prevent misuse of their technology. The California Attorney General's office is taking a firm stance, emphasizing the illegality of creating and distributing such material and demanding immediate action from xAI.
The investigation and cease-and-desist order highlight the challenges of regulating AI and ensuring that these powerful tools are not used to harm individuals or create illegal content. The case also underscores the need for AI companies to proactively address potential misuse of their technology and implement safeguards to prevent the creation and distribution of harmful content. The next five days will be critical as xAI responds to the Attorney General's demands and outlines its plan to address these serious concerns.
Discussion
Join the conversation
Be the first to comment