China is proposing new regulations to govern artificial intelligence (AI) products and services, with a focus on safeguarding children and preventing harmful content. The Cyberspace Administration of China (CAC) published the draft rules over the weekend, outlining measures to protect children and address potential risks associated with AI-driven interactions.

The proposed regulations target AI developers, requiring them to implement personalized settings and time limits on usage for younger users. Furthermore, AI firms would need to obtain consent from guardians before offering emotional companionship services to children. The rules also aim to prevent AI models from generating content that promotes gambling, reflecting broader concerns about the potential for AI to be used in ways that could be detrimental to society.

A key provision addresses the risk of AI chatbots providing advice that could lead to self-harm or violence. Under the proposed rules, chatbot operators would be required to have a human take over any conversation related to suicide or self-harm. The operator would also need to immediately notify the user's guardian or an emergency contact, according to the CAC.

These regulations come amid a rapid increase in the number of AI chatbots being launched both in China and globally. The move signals a significant effort to regulate the fast-growing AI sector, which has faced increasing scrutiny regarding safety concerns. Once finalized, the rules will apply to all AI products and services operating within China.

The development highlights the growing global debate surrounding the ethical implications of AI and the need for regulatory frameworks to mitigate potential risks. As AI technology becomes more sophisticated and integrated into daily life, governments and organizations are grappling with how to ensure its responsible development and deployment. The Chinese regulations represent one approach to addressing these challenges, focusing on specific areas of concern such as child safety and the prevention of harmful content. The CAC has not yet announced a timeline for when the draft rules will be finalized and implemented.