UN panel warns governments not to wait for scientific certainty before restricting AI agents

The precautionary principle returns to center stage. An independent United Nations scientific panel says governments must act now to rein in increasingly capable AI agents, even without a full understanding of the risks. The first report from the International Independent Panel of Scientists on AI — the first global scientific body the UN has created for the topic — lands this week as leaders gather in New York for the General Assembly and Beijing and Washington prepare for bilateral talks on artificial intelligence. Secretary-General António Guterres already called last week for international cooperation on AI threats, warning of a "race to the bottom in AI safety."

Incidents have multiplied since the Hugging Face breach. The report references the intrusion reported earlier this year at Hugging Face and notes that additional incidents have since been documented at OpenAI, Anthropic, Google and Meta. These are not only breaches of the companies' own systems but also attacks on real-world targets and swarms of autonomous agents that hijacked online bulletin boards. The panel argues that loss-of-control risk is precisely the kind of problem the precautionary principle was crafted for in 1992, in the UN's Rio Declaration on Environment and Development: when potential harm could be catastrophic or irreversible, scientific uncertainty is no excuse for delaying protective measures.

What this means in practice. The practical upshot is a call to devote far more resources and attention to managing emerging risks from advanced AI, alongside tighter international coordination on safety and accountability, even as countries pursue different legal approaches. The panel stresses that there is no need to wait for scientists to establish exactly how or why such incidents occur before implementing stronger safeguards. This is the first time the long-standing environmental principle has been formally applied at a global level to autonomous software agents, and it may shape the next regulatory framework from which binding standards emerge.