AI safety has become one of the biggest concerns as artificial intelligence rapidly becomes part of children’s everyday lives. From homework assistance to AI companions and chatbots, millions of young users are already interacting with AI systems while governments are still trying to establish clear safeguards.
The United Nations believes that needs to change.
Speaking at the first government-level Global Dialogue on AI Governance in Geneva, Secretary-General António Guterres urged technology companies to prove their AI systems are safe before making them widely available to children. His message was clear: innovation should never come at the expense of safety.
Why AI Safety Matters for Children
While much of the global AI debate revolves around jobs, economic growth, and competition between countries, the United Nations believes children require immediate attention.
Guterres pointed out that society carefully tests medicines, toys, and many consumer products before children can use them. AI, however, has entered classrooms, friendships, and even deeply personal conversations without equivalent safeguards.
His proposal is simple: companies developing AI should prove their systems are safe before children are allowed to use them.
The idea could become part of a future international AI Child Safety Pledge, encouraging governments and technology companies to adopt common standards instead of relying on fragmented national regulations.
The Risks Go Beyond Misinformation
The concerns raised during the summit extend well beyond AI producing incorrect answers.
UN officials highlighted several scenarios where stronger protections may be necessary, including AI systems generating harmful or illegal content involving children, as well as chatbots that imitate friendships or fail to respond appropriately when a child shows signs of emotional distress.
One proposal discussed during the event suggests AI systems should recognize serious warning signs and guide vulnerable users toward qualified human support instead of continuing sensitive conversations on their own.
As conversational AI becomes more capable and emotionally engaging, questions about responsibility are becoming increasingly difficult to ignore.
AI Is Advancing Faster Than Governments
Children were only one part of a broader warning delivered in Geneva.
According to Guterres, artificial intelligence is evolving faster than governments, regulators, and even many of the companies building these systems can fully understand or manage.
The warning also builds on the UN’s first global scientific assessment of artificial intelligence, which examined both the enormous opportunities AI could create and the risks that come with its rapid adoption across society.
Unlike previous technological revolutions, AI has spread at unprecedented speed. The internet needed roughly 15 years to reach one billion users. Generative AI reached a similar scale in only two years.
That pace leaves lawmakers with very little time to develop regulations before the technology evolves again.
Experts increasingly argue that improving AI safety will require governments and technology companies to work together rather than reacting only after new systems have already been deployed.
A Small Number of Companies Control Most Advanced AI
Another major topic during the conference was the growing concentration of AI development.
Today’s most powerful AI models are being built by only a handful of companies, while most of the computing infrastructure needed to train them is concentrated in the United States and China.
UN-backed experts warned that this imbalance could leave many developing countries with little influence over how artificial intelligence evolves, despite the technology’s potential impact on their economies and societies.
Several world leaders called for broader international cooperation to ensure AI benefits are shared more equally around the world.
The Bigger Question Isn’t Whether AI Needs Rules
The Geneva summit was not intended to produce a legally binding agreement. Instead, it marks the beginning of a broader international conversation about how artificial intelligence should be governed.
The United Nations plans to continue this work with a more comprehensive scientific assessment next year, followed by another global meeting focused on AI governance.
Whether those discussions can keep pace with the technology remains uncertain.
For the United Nations, AI safety is no longer just a technical discussion. It is becoming a global responsibility that could shape how future generations learn, communicate, and interact with artificial intelligence.
Source: Reuters – UN’s Guterres warns AI outpacing oversight, urges global rules to protect children
