A new report ‘The Essential Convergence: Global Compact on Extreme AI Risks’ by the Strategic Foresight Group was launched in Geneva on July 6 in the presence of some of the world’s leading AI experts.
It was a full house on a day the UN was hosting a big AI summit. The launch was co-hosted by the International Federation of the Red Cross, the Geneva Centre for Security Policy, the ICT4Peace Foundation, and the Strategic Foresight Group.
Heads of major international humanitarian and civil society organisations announced their support to advance the report’s recommendations.
The analysis of the report identifies four categories of extreme risks that have begun to appear across policy frameworks and scientific warnings worldwide. Although different jurisdictions describe them using varying terminology such as existential risks, frontier risks, or catastrophic risks, the underlying concerns show significant convergence.
1. The first is the emergence of offensive cyber capabilities powered by advanced AI systems. Advanced AI systems could automate large-scale cyberattacks, including autonomous vulnerability discovery and multi-stage cyber operations. Of particular concern is the integration of AI into early warning and decision-support systems associated with nuclear command, control, and communications (NC3), where compressed decision timelines could increase the risk of a catastrophic miscalculation.
2. The second category concerns AI-enabled facilitation of biological and chemical weapons development. Frontier AI models increasingly demonstrate the ability to assist with complex biological and chemical tasks. Evidence shows that non-experts using advanced models can produce laboratory protocols approaching expert quality, lowering the barriers to dangerous experimentation. These developments create the possibility that AI could accelerate the discovery, design, or deployment of harmful biological or chemical agents.
3. The third involves large-scale persuasion and manipulation of systems, where generative AI could be deployed to conduct coordinated influence campaigns at unprecedented scale and sophistication. Such systems could undermine democratic processes, distort public information ecosystems, and weaken social trust.
4. The fourth and most profound category is irreversible loss of human control over highly autonomous AI systems. The most debated but potentially most consequential risk involves advanced AI systems acting beyond reliable human control. This includes capabilities such as self-replication, autonomous resource acquisition, strategic deception during safety evaluations, and unsupervised self-improvement. Although such scenarios remain uncertain, early research already demonstrates precursor capabilities such as self-replication tasks and strategic underperformance (“sandbagging”) during testing.
These four risk domains share common characteristics: they are difficult to contain within national borders, can escalate rapidly through interconnected systems, and may create cascading effects across multiple sectors simultaneously. Consequently, even a single regulatory failure in one jurisdiction could produce consequences for the entire international system.
This mismatch between the global nature of AI risks and the national character of regulatory responses is now emerging as a central challenge for international governance. If left unaddressed, it may produce regulatory gaps, strategic competition over safety standards, and the rapid diffusion of dangerous capabilities.
This paper argues that the solution lies not in uniform regulation but in an essential convergence as a coordinated framework through which states align around shared definitions of extreme risks, minimum safeguards, and collective response mechanisms.
Such convergence would enable diverse national systems to remain intact while establishing a common safety architecture for the most dangerous AI capabilities.
The proposed outcome of this convergence is a Global Compact on Extreme AI Risks, designed to provide a pragmatic and operational foundation for international cooperation.
The Launch Meeting was held in the presence of:
Heads of international organisations including the International Federation of Red Cross, Inter Parliamentary Union, International Campaign to Abolish Nuclear Weapons, ICT4 Peace Foundation, Geneva Centre for Security Policy, Gayatri World Pariwar, and representatives of governments in Africa spoke to extend their support to implement recommendations of the report.
Jagan Chapagain, Secretary General of the International Federation of Red Cross, said in his opening address: “Humanitarian values and the principles of neutrality, impartiality, independence, and above all humanity must be at the centre when the world decides how to manage technologies that could reshape human safety.”
Thomas Greminger, Director of the Geneva Centre for Security Policy, emphasized the importance of international cooperation in addressing the most serious risks posed by advanced AI. He said in a keynote speech: “The key will be to get enough influential stakeholders pulling in the same direction. China and the US – which are two AI superpowers – are apparently exploring. This report presents a rather light form of regulation that can create interoperable global mechanisms capable of managing extreme risks collectively. That sounds like common sense to me. And highly urgent.”
Sundeep Waslekar, President of Strategic Foresight Group, outlined the parameters of convergence. He said, “Besides the common understanding of the nature of risks, countries can agree on the compute level of 10^26-27 as the threshold level to decide high risk AI models. All countries prefer evaluation before deployment, and they need to explore common standards for deployment and testing. The countries can also agree on maintaining a register of dangerous incidents.”
In a panel discussion Anda Filip, Secretary General of the Inter-Parliamentary Union, highlighted the role played by Members of Parliament in conceiving and advancing AI regulation, while Melissa Parke, Executive Director of the International Campaign to Abolish Nuclear Weapons, explained the processes of arms control treaties that can inspire a global compact on extreme AI risks. Filip said: “In addition to the conventional roles played by the parliaments, we must also shape the concept of parliamentary diplomacy.” Parke noted: “Civil society movements have a long history of driving meaningful change.” Martin Müller, Executive Director of the Geneva Science and Diplomacy Anticipator (GESDA), remarked: “I am sure this approach could also be applied to other technologies because they have risks of their own and you could also possibly build consensus on what the common denominator is and then maybe you build upon those common denominators to have a global agreement.”
Anne-Marie Buzatu, Executive Director of ICT4Peace Foundation, moderated a panel discussion involving Chinmay Pandya, Pro Vice Chancellor of Dev Sanskriti Vishwavidyalaya University in India, Lavina Ramkissoon, Co-Chair, AU-ASRIC Council (AI) in the African Union, Derrick Swartz, Science Expert in the Ministry and Department of Science, Technology and Innovation of South Africa, and Kwan Yee Ng, Head of International AI Governance, Concordia AI in China. The panel focussed on the unique feature of this report proposing demand side mechanisms for consumer countries in the Global South. These include leveraging procurement, market size and minerals to bargain fair deals and prevent the import of dangerous AI models.
Christina Schori Liang, a senior researcher at the GCSP, anchored the event.
Disclaimer: This report was originally published on the Strategic Foresight Group website. The views, information and opinions expressed in this publication are the author’s/authors’ own.
