Since the UK and US pioneered the establishment of national AI Safety Institutes, Concordia AI has been closely following and analyzing their evolution. This report analyzes the background, progress, comparisons, and challenges of current national AI Safety Institutes and their international network across multiple countries, aiming to provide reference for China’s policy formulation in global AI governance.
1. Background
Frontier AI systems like GPT-4 have demonstrated powerful emergent capabilities, driving numerous technological directions including multimodal large models, autonomous agents, scientific discovery agents, and embodied intelligence. These have approached or even surpassed human-level performance in multiple domains but have also raised new safety challenges. The “Bletchley Declaration” and “Seoul Declaration” issued by two successive Global AI Safety Summits promoted the establishment of national AI Safety Institutes to address technological risks and strengthen global governance.
2. Progress
The UK and US respectively established AI Safety Institutes at the first Global AI Safety Summit in 2023, followed by Japan, Singapore, Canada, South Korea, France, and the EU. Meanwhile, the US actively promotes the development of an international network of AI Safety Institutes. These institutions focus on AI safety evaluation, AI safety research, facilitating information exchange, and advancing standard development as core functions, and have initially established bilateral and multilateral collaborations.
3. Comparison
National AI Safety Institutes in different countries show diversity in institutional attributes, functional positioning, research priorities, and international cooperation. Based on detailed comparisons of the above information from eight national AI Safety Institutes, we conducted case studies focusing on leading AI Safety Institutes:
- UK AI Safety Institute: Adequate government funding support, attracting substantial technical talent, aiming to lead frontier AI safety evaluation and research; obtained pre-deployment evaluation authorization from OpenAI, DeepMind, and Anthropic; participated in organizing Global AI Safety Summits; has open-sourced the evaluation framework Inspect, providing testers with tools to assess specific capabilities of various models.
- US AI Safety Institute: Focuses on frontier AI risks while covering broader risk categories; established the AI Safety Research Consortium, leveraging the National Institute of Standards and Technology and collaborative networks; obtained access to new models from OpenAI and Anthropic both before and after release. Initially, focused on domestic security issues, but increasingly emphasized global cooperation after collaborating with the UK AI Safety Institute and announcing the establishment of an international network of AI Safety Institutes, aiming to coordinate parties in developing measurement science, voluntary guidelines, and rigorous testing standards for frontier AI. However, with Trump as president, the prospects for global cooperation remain uncertain.
- Other national AI Safety Institutes combine their own needs with various emphases on standardization, safety R&D, and regulatory enforcement.
4. Challenges
Although AI Safety Institutes and their international network play important roles in safety evaluation, safety research, and cooperation, they still need improvements in model access and evaluation permissions, information sharing and safety practices, standard development and regulatory frameworks, resource disparities and cooperation balance, and global inclusivity and international coordination to address the complex challenges that AI technology brings to global governance.