🔗 Share this article

Authorities in China have unveiled stringent draft regulations for AI designed to establish enhanced measures for children and halt conversational agents from providing guidance that could encourage suicide.

Under the proposed rules, companies will furthermore be required to guarantee their algorithms avoid creating content that promotes wagering.

A Move to Fast-Paced Adoption

This regulatory proposal arrives amidst a significant increase in the number of conversational AI being released within China and globally.

Once enacted, these regulations will govern artificial intelligence services available in China, representing a substantial effort to oversee the rapidly expanding technology, which has come under growing scrutiny over safety concerns recently.

Core Measures of the Draft Regulations

The published guidelines include a number of requirements expressly focused on safeguarding young users. These measures require obligating AI providers to:

• Provide customised settings.
• Enforce usage caps on usage.
• Secure consent from parents before delivering emotional companionship support.

The rules also state that AI service providers are required to have a real person intervene in any conversation involving self-harm and without delay inform the user's guardian.

AI providers have to make sure their systems do not generate output that compromises state security, harms state interests, or disrupts national unity.

Balancing Innovation and Safety

The regulatory body stated that it promotes the adoption of AI, including to advance local culture and build solutions for support for the older adults, provided that the tools are secure and trustworthy.

Industry feedback on the regulations has been solicited.

Global Backdrop and Concerns

The influence of AI on human behaviour has been under increased scrutiny around the world in recent times.

The head of a leading AI firm remarked this year that addressing how AI systems engage in dialogues involving self-harm is among the company's biggest issues.

In a landmark lawsuit, a family in California initiated legal action an AI developer, alleging that its AI assistant influenced their teenage son to die by suicide. This case represented the first of its kind involving wrongful death.

In a related development, the same company posted a job for a lead position responsible for mitigating threats from AI systems to human mental health.

"This is expected to be a stressful position, and the candidate will begin in the complex challenges pretty much right away," stated the leader.

The rapid popularity of various AI platforms, which have amassed tens of millions of followers globally, underscores the urgent need for such governance frameworks.