China is proposing new regulations to govern artificial intelligence (AI) with the aim of safeguarding children and preventing AI chatbots from providing advice that could lead to self-harm or violence. The Cyberspace Administration of China (CAC) published the draft rules over the weekend, outlining measures that would require AI firms to offer personalized settings and time limits on usage for younger users.

The proposed regulations also stipulate that AI firms must obtain consent from guardians before providing emotional companionship services to children. Furthermore, chatbot operators would be required to have a human take over any conversation related to suicide or self-harm and immediately notify the user's guardian or an emergency contact, according to the CAC. The rules also aim to prevent AI models from generating content that promotes gambling.

These regulations, once finalized, will apply to AI products and services operating within China. This marks a significant step toward regulating the rapidly expanding AI sector, which has faced increasing scrutiny regarding safety concerns throughout the year. The move follows a surge in the launch of AI chatbots both in China and globally.

The development reflects growing global concerns about the ethical implications and potential risks associated with AI, particularly generative AI, which can create text, images, and other content. Experts emphasize the need for responsible AI development and deployment to mitigate potential harms.

The CAC's proposed rules highlight the Chinese government's proactive approach to AI governance. The regulations address specific concerns related to child safety and mental health, reflecting a broader effort to ensure that AI technologies are used responsibly and ethically.

The draft rules are currently open for public comment, and the CAC is expected to consider feedback before finalizing the regulations. The implementation of these rules could set a precedent for other countries grappling with the challenges of AI governance. The regulations are expected to impact AI developers and service providers operating in China, requiring them to adapt their products and services to comply with the new requirements.