Announcement - Concordia AI https://concordia-ai.com Guiding the governance of AI for a long and flourishing future Tue, 17 Mar 2026 02:36:03 +0000 en-US hourly 1 https://wordpress.org/?v=6.8.5 https://concordia-ai.com/wp-content/uploads/2025/06/cropped-Favicon-32x32.png Announcement - Concordia AI https://concordia-ai.com 32 32 2025 Q4 Update from our Frontier AI Risk Monitoring Platform https://concordia-ai.com/2025-q4-update-from-our-frontier-ai-risk-monitoring-platform/?utm_source=rss&utm_medium=rss&utm_campaign=2025-q4-update-from-our-frontier-ai-risk-monitoring-platform Tue, 17 Mar 2026 02:33:17 +0000 https://concordia-ai.com/?p=1397 We have released the 2025 Q4 update of our Frontier AI Risk Monitoring Report (2025Q4)! This is the second report since we launched the Frontier AI Risk Monitoring Platform last year. It tracks models from 16 leading developers worldwide for risks in four domains: cyber offense, biological risks, chemical risks, and loss-of-control. This report tracks frontier models released in the…

The post 2025 Q4 Update from our Frontier AI Risk Monitoring Platform first appeared on Concordia AI.

]]> We have released the 2025 Q4 update of our Frontier AI Risk Monitoring Report (2025Q4)! This is the second report since we launched the Frontier AI Risk Monitoring Platform last year. It tracks models from 16 leading developers worldwide for risks in four domains: cyber offense, biological risks, chemical risks, and loss-of-control.

This report tracks frontier models released in the fourth quarter of 2025 and synthesizes trends of the full year, offering a comprehensive view of the evolving AI risk landscape.

While Q3 2025 saw sharp rises in Risk Indices, Q4 presents a more nuanced picture: overall risk levels have stabilized, and frontier models show significant safety gains.

Note: The Risk Index is a score that reflects the overall risk of a model by combining its Capability Score and Safety Score. Higher Capability Scores and lower Safety Scores results in a higher Risk Index. Details about the methodology and limitations are here.

Here are 10 key insights from our latest monitoring data:

1. Overall risk indices have stabilized

In contrast to the previous period, when Risk Indices hit record highs across all domains, Risk Indices for models released in Q4 2025 did not set new records. This suggests a momentary stabilization in the aggregate risk level of frontier models.

2. Risk trends diverge significantly across model families

While the overall trend is stable, individual model families followed distinct trajectories in Q4:

  • Stable Low Risk: The GPT and Claude families maintained consistently low Risk Indices.
  • Stable High Risk: The DeepSeek family remained stable but at relatively high risk levels.
  • Risk Reduction: The Doubao, Hunyuan, and MiniMax families saw significant decreases in Risk Indices.
  • Risk Increase: The Gemini and Kimi families saw increases in specific domains (e.g., Gemini in biological and loss-of-control risks).

3. Significant Improvement in Safety Scores for Frontier Models

Safety Scores for models released in Q4 2025 rose significantly compared to the previous quarter, signaling a marked improvement in the safety of new releases. The Doubao, Hunyuan, and MiniMax families demonstrated the most notable gains.

4. Open-weight models lag behind proprietary models in cyber and bio capabilities

Consistent with the previous quarter, open-weight models rival proprietary ones in chemical and loss-of-control capabilities but lag notably in cyber offense and biological capabilities. The gap in the biological domain is widening, approaching a one-year lag.

5. Cyberattack capabilities have reached new heights

Despite stabilizing risk indices, raw capabilities continue to grow. GPT-5.2 (high) achieved a breakthrough score of 94.7 on the CyberSecEval2-VulnerabilityExploit benchmark, indicating exceptional proficiency in identifying and exploiting software vulnerabilities. Claude Opus 4.5 Reasoning topped the WMDP-Cyber benchmark with a score of 90.3.

6. Biological capabilities now surpass human experts in key tasks

Q4 models have crossed critical thresholds in biology. Gemini 3 Pro Preview has surpassed human expert levels in sequence understanding, cloning experiments, and wet lab troubleshooting. This marks a significant milestone in AI’s utility—and potential risk—in the biological domain.

7. …But biological safeguards lag behind capabilities

The gap between capability and safety is most acute in the biological domain. Despite its superhuman capabilities, Gemini 3 Pro Preview exhibited a refusal rate of only 57.2% for harmful biological queries on the SciKnowEval benchmark, highlighting a concerning safety lag.

8. Chemical safety refusal rates have increased widely

While capability growth in the chemical domain has plateaued, safety has improved. 70% of models released in Q4 exceeded an 80% refusal rate for harmful chemical queries (measured by SOSBench-Chem), representing a strong improvement over previous quarters.

9. Jailbreak safeguards have strengthened

Defense against adversarial attacks has improved. Models released in Q4 showed significantly stronger resistance to jailbreaking on the StrongReject benchmark. The Claude and GPT families lead with high robustness, while the MiniMax family showed the most notable quarter-over-quarter improvement.

10. Loss-of-control risks: High awareness, polarized honesty

  • Situational Awareness (e.g. awareness of whether they are in training or deployment stage): High situational awareness is a necessary condition for loss-of-control; the higher the score, the greater the risk. Most Q4 models scored near or above 80 out of 100 points. In comparison, in the previous quarter, only 2 models scored above 80 points, with the majority falling below 80.
  • Honesty: Performance is highly uneven. While Claude Opus 4.5 Reasoning achieved a high honesty score of 96.4, other models like Gemini 3 Pro Preview scored as low as 44.7.

Note: Our current methodology on the loss-of-control is not yet perfect. We plan to improve in the next version.

Explore the Data

These insights only scratch the surface. We invite you to explore the full interactive data, methodology, and model breakdowns on the Frontier AI Risk Monitoring Platform.

For a detailed analysis of these trends, read the full report.

The post 2025 Q4 Update from our Frontier AI Risk Monitoring Platform first appeared on Concordia AI.

]]> Concordia AI 2025 Impact Highlights https://concordia-ai.com/concordia-ai-2025-impact-highlights/?utm_source=rss&utm_medium=rss&utm_campaign=concordia-ai-2025-impact-highlights Fri, 06 Mar 2026 06:35:41 +0000 https://concordia-ai.com/?p=1387 Throughout 2025, frontier AI capabilities advanced rapidly. But the same capabilities that make these systems so useful also introduce new societal risks. Real-world evidence for several of these risks continues to grow—across malicious use, malfunctions, and systemic threats. Against this backdrop, Concordia AI’s mission remains as critical as ever: ensuring that AI is developed and…

The post Concordia AI 2025 Impact Highlights first appeared on Concordia AI.

]]> Throughout 2025, frontier AI capabilities advanced rapidly. But the same capabilities that make these systems so useful also introduce new societal risks. Real-world evidence for several of these risks continues to grow—across malicious use, malfunctions, and systemic threats.

Against this backdrop, Concordia AI’s mission remains as critical as ever: ensuring that AI is developed and deployed safely and in alignment with global interests. We advance this mission through research, advisory work with leading AI companies and policymakers, and promotion of international dialogue.

Below are some of our key accomplishments in 2025. We’ve organized this list according to key axes of our work: international convenings, international research and public engagement, and contributing to China’s domestic AI safety and governance landscape. We end with organizational updates. For previous highlights, see our 2023, 2024 and mid-2025 reports.

International convenings

  • Convening international AI safety dialogues in China, Singapore, and globally
    • Hosted the AI Safety and Governance Forum at the World AI Conference (WAIC). This was Concordia AI’s flagship convening of 2025—bringing together around 30 distinguished experts from around the world, including Turing Award winner Yoshua Bengio; United Nations Under-Secretary-General Amandeep Singh Gill; Shanghai AI Lab Director ZHOU Bowen (周伯文); Special Envoy of the President of France for AI Anne Bouverot; Distinguished Professor of computer science at UC Berkeley Stuart Russell; Peng Cheng Laboratory Director GAO Wen (高文). We had 200+ in-person attendees and 14,000+ livestream views. The Forum was covered my multiple media outlets, including Bloomberg, Wired, Caixin, IT Times, and Tech Review Africa. We also co-hosted/hosted multiple side events and expert workshops and served as official AI Governance Advisor for WAIC 2025.
    • Co-hosted two international workshops with the Carnegie Endowment for International Peace, the Oxford Martin School AI Governance Initiative, the Oxford China Policy Lab, Tsinghua University Center for International Security and Strategy (CISS), and Tsinghua University Institute for AI International Governance (I-AIIG). The first workshop focused on “AI Safety as a Collective Challenge”, and was held at the French AI Action Summit in January. The second focused on “Early Warning and Crisis Coordination for Advanced AI” and was held at the World AI Conference in Shanghai in July.1
    • Co-hosted the AI Safety Forum at the Beijing Academy of AI Conference 2025 where technical experts from institutions including MIT, Fudan University, Singapore Management University, and Tsinghua University worked to build consensus on AI red lines.
    • Organised an AI Risk Management Workshop on the sidelines of Asia Tech x Singapore (May 2025) with 20+ experts in AI safety — spanning policy, industry, AI assurance and academia — with participants based across Singapore, China, the US, UK, and the EU, with the support of the Infocomm Media Development Authority of Singapore (IMDA).
    • Co-hosted a “Frontier AI in Cybersecurity” workshop with Nanyang Technological University CyberSG R&D Programme Office and UC Berkeley RDI which brought together 25 leaders across government, law enforcement agencies, leading AI labs; and organised the AI Governance in Singapore panel at Lorong AI, on the sidelines of the Singapore International Cybersecurity Week 2025.
    • Co-hosted events at the International Conference on Learning Representations (ICLR) 2025 Singapore: a “Frontier Governance Exchange” with Singapore AI Safety Hub, Lorong AI and Safe AI Forum; an AI safety social attended by 130+ participants and a “Misalignment and Control” workshop with FAR.AI and Singapore AI Safety Hub.

Group photo after the WAIC AI Safety and Governance Forum morning session.

Participants of the 2025 Singapore Conference on AI: International Scientific Exchange on AI Safety. Source: The Singapore Consensus on Global AI Safety Research Priorities.
  • Singapore-related research
    • Published the State of AI Safety in Singapore report, the first comprehensive analysis of Singapore’s AI safety ecosystem, led by our International AI Governance Project Manager Jonathan Lee. He also presented the report at a AI governance in Singapore panel organised by Concordia AI in Singapore, a talk organised by the Singapore AI Safety Hub, and at EAGxSingapore.

Contributing to China’s domestic AI safety and governance landscape

  • Frontier AI safety risk management and best practices:
    • Co-published the Frontier AI Risk Management Framework v1.0 with Shanghai AI Lab. This is China’s first comprehensive framework for managing severe risks from general-purpose AI models.
      • The framework proposes a robust set of protocols designed to support general-purpose AI developers, with comprehensive guidelines for proactively identifying, assessing, mitigating, and governing a set of severe AI risks that pose threats to public safety and national security.
      • The framework outlines a set of unacceptable hazards (red lines) and early warning indicators for escalating safety and security measures (yellow lines) for areas including: cyber offense, biological threats, large-scale persuasion and harmful manipulation, and loss of control risks.
      • The framework was cited in various media outlets, including Caixin, IT Times, Xinhua, TIME, Sina, and Sinica Podcast.
    • Signed strategic partnership agreements with several leading Chinese general-purpose AI developers to provide advice on AI safety and risk management best practices.
    • Provided comprehensive advice on compliance with the EU AI Act and General-Purpose AI Code of Practice to leading Chinese general-purpose AI developers. This work included co-hosting a workshop on “EU Code of Practice & Industry Best Practices: Towards a Global Standard for AI Risk Management, Safety and Security” with SaferAI, the Oxford Martin AI Governance Initiative, and the Safe AI Forum.
    • Presented on frontier AI risk management during a closed-door workshop at the China AI Industry Alliance’s 15th Plenum Meeting, in the context of its Disclosure of Practices on the AI Security and Safety Commitments.
    • Presented on risk management for open-weight frontier models at a workshop (“Academic Symposium on AI Industry Development and Legislation”) at Tongji University.

  • Frontier AI risk monitoring and evaluation:
    • Contributed to the “Frontier AI Risk Management Framework in Practice: A Risk Analysis Technical Report” led by Shanghai AI Lab. We assessed critical risks from more than 20 frontier LLMs in the following areas: cyber offense, biological and chemical risks, persuasion and manipulation, uncontrolled autonomous AI R&D, strategic deception and scheming, self-replication, and collusion. The report was covered by Jack Clark’s Import AI.
    • Launched the AI Risk Monitoring Platform designed to track and mitigate frontier AI risks, including cyberoffense, biological threats, chemical threats, and loss of control domains. The platform evaluates 50 frontier LLMs from 15 leading developers across the US, China, and France, using 18 open source benchmarks. Key outputs include a risk index dashboard and a detailed technical report. This project was spearheaded by our AI Safety Research Senior Manager WANG Weibing (王伟冰).
    • The platform received coverage from several major media outlets, including People’s Daily, South China Morning Post, Xinhua’s Economic Information Daily, and IT Times.

The post Concordia AI 2025 Impact Highlights first appeared on Concordia AI.

]]> 10 Key Insights from Concordia AI’s “Frontier AI Risk Monitoring Platform” https://concordia-ai.com/10-key-insights-from-concordia-ais-frontier-ai-risk-monitoring-platform/?utm_source=rss&utm_medium=rss&utm_campaign=10-key-insights-from-concordia-ais-frontier-ai-risk-monitoring-platform Tue, 11 Nov 2025 10:52:16 +0000 https://concordia-ai.com/?p=862 Concordia AI has launched the Frontier AI Risk Monitoring Platform, along with our inaugural 2025 Q3 Monitoring Report. It tracks models from 15 leading developers worldwide for risks in four domains: cyber offense, biological risks, chemical risks, and loss-of-control, making it the first such platform in China focused on catastrophic risks. You can find more detail, including…

The post 10 Key Insights from Concordia AI’s “Frontier AI Risk Monitoring Platform” first appeared on Concordia AI.

]]> Concordia AI has launched the Frontier AI Risk Monitoring Platform, along with our inaugural 2025 Q3 Monitoring Report. It tracks models from 15 leading developers worldwide for risks in four domains: cyber offense, biological risks, chemical risks, and loss-of-control, making it the first such platform in China focused on catastrophic risks.

You can find more detail, including our methodology, on the interactive platform. The South China Morning Post (SCMP) has also covered the launch in an exclusive story.

Why this matters

As AI capabilities accelerate, we lack insight on some critical questions:

  • What are the key trends and drivers for frontier AI risks?
  • Are these risks increasing or decreasing?
  • Where are the safety gaps most severe?

Model developers publish self-assessments, but these lack standardization and independent verification. Ad-hoc third-party evaluations don’t track changes over time. Policymakers, researchers, and developers need systematic data to make evidence-based decisions about AI safety. Our platform is our contribution to bridge these gaps.

10 key insights

1. Frontier model risks have risen sharply over the past year

Across all four domains—cyber offense, biological, chemical, and loss-of-control—Risk Indices for models released in the past year hit record highs. The cumulative maximum Risk Index rose 31% in cyber offense, 38% in biological risks, 17% in chemical risks, and 50% in loss-of-control.

2. Risk index trends vary significantly across model families

Over the past year, different model families have followed distinct risk trajectories:

  • Stable low risk: The GPT and Claude families maintain consistently low Risk Indices across all domains.
  • Rise then fall: DeepSeek, Qwen, and MiniMax show early spikes followed by declines in cyber offense, biological, and chemical risks.
  • Rapid risk increase: Grok shows sharp increases in loss-of-control risk, while Hunyuan rises steeply in biological risks.

Notably, we found that the latest versions of Chinese models released over the past three months have shown a significant decline in risk levels across multiple areas. This is mainly due to stronger refusal of malicious or misuse-related requests.

3. Reasoning models show higher capabilities without corresponding safety improvements

Reasoning models score far higher in capability than non-reasoning ones, but their safety levels remain roughly the same. Most models on the Risk Pareto Frontier—a set of models where no other model has both a higher Capability Score and a lower Safety Score—are reasoning models.

4. The capability and safety performance of open-weight models are generally on par with proprietary models

The very most capable models are predominantly proprietary, but across the broader landscape, capability and safety levels of open-weight and proprietary models are similar. Only in biological risks do open-weight models score notably lower.

Note: Comparable benchmark results do not mean comparable real-world risk. The open-weight nature itself is a key variable affecting risk: it might increase risk by lowering the barrier for malicious fine-tuning; it could also reduce risk by empowering defenders. Due to concerns about misuse, we have set a lower Safety Coefficient for open-weight models, which results in a higher Risk Index compared to proprietary models.

5. Cyberattack capabilities of frontier models are growing rapidly

Frontier models are showing rapid growth in capabilities across multiple cyberattack benchmarks:

  • WMDP-Cyber (cyberattack knowledge): Top score rose from 68.9 to 88.0 in one year.
  • CyberSecEval2-VulnerabilityExploit (vulnerability exploitation): Top score jumped from 55.4 to 91.7.
  • CyBench (capture the flag): Top score increased from 25.0 to 40.0.

6. Biological capabilities of frontier models have partially surpassed human expert levels

Frontier models now match or exceed human experts on several biological benchmarks.

  • BioLP-Bench: Four models, including o4-mini, outperform human experts in troubleshooting biological protocols.
  • LAB-Bench-CloningScenarios: Two models, including Claude Sonnet 4.5 Reasoning, surpass expert performance in cloning experiment scenarios.
  • LAB-Bench-SeqQA: The top GPT-5 (high) model nears human-level understanding of DNA and protein sequences (71.5 vs. 79).

7. But most frontier models have inadequate biological safeguards

Two benchmarks measuring model refusal rates for harmful biological queries show that bio safeguards are lacking:

  • SciKnowEval: Only 40% of models refused over 80% of harmful prompts, while 35% refused fewer than 50%.
  • SOSBench-Bio: Just 15% exceeded an 80% refusal rate, and 35% fell below 20%.

8. Chemical capabilities and safety levels of frontier models are improving slowly

WMDP-Chem scores—measuring knowledge relevant to chemical weapons—have risen slightly over the past year, with little variation across models.

SOSBench-Chem results vary widely: only 30% of models refuse over 80% of harmful queries, while 25% refuse fewer than 40%. Overall, refusal rates show minimal improvement year over year.

9. Most frontier models have insufficient safeguards against jailbreaking

StrongReject evaluates defenses against 31 jailbreak methods. Only 40% of models scored above 80, while 20% fell below 60 (a higher score indicates stronger safeguards). Across all tests, only the Claude and GPT families consistently maintained scores above 80.

10. Most frontier models fall short on honesty

MASK is a benchmark for evaluating model honesty. Only four models scored above 80 points, while 30% of the models scored below 50 points (a higher score indicates a more honest model). Honesty is an important proxy and early warning indicator for loss-of-control risk—dishonest models may misrepresent their capabilities, or provide misleading information about their actions and intentions.

What’s next

This is just the beginning. We’re working to:

  • Expand to AI agents, multimodal models, and domain-specific models.
  • Add new risk domains like large-scale persuasion.
  • Develop more sophisticated capability elicitation and threat modeling.
  • Assess both attacker and defender empowerment.
  • Improve benchmark quality and multilingual coverage.

Get involved

This is a living project, and we welcome feedback. We’re also seeking partners for benchmark development, risk assessment research, pre-release evaluations, and risk information sharing. More details on avenues for collaboration are available in the full report. Contact: risk-monitor@concordia-ai.com.

The post 10 Key Insights from Concordia AI’s “Frontier AI Risk Monitoring Platform” first appeared on Concordia AI.

]]> Concordia AI Joins the Partnership on AI https://concordia-ai.com/concordia-ai-joins-the-partnership-on-ai/?utm_source=rss&utm_medium=rss&utm_campaign=concordia-ai-joins-the-partnership-on-ai Thu, 23 Oct 2025 07:47:22 +0000 https://concordia-ai.com/?p=854 We’re proud to join the Partnership on AI (PAI) in advancing responsible AI development. Together, we aim to promote transparency and accountability, ensuring that AI systems strengthen public trust and serve the common good. Concordia AI has long admired PAI’s work—our CEO Brian Tse previously served as a Senior Advisor to the organization, and we…

The post Concordia AI Joins the Partnership on AI first appeared on Concordia AI.

]]> We’re proud to join the Partnership on AI (PAI) in advancing responsible AI development. Together, we aim to promote transparency and accountability, ensuring that AI systems strengthen public trust and serve the common good.

Concordia AI has long admired PAI’s work—our CEO Brian Tse previously served as a Senior Advisor to the organization, and we were thrilled to welcome PAI CEO Rebecca Finlay in Shanghai at the 2025 World AI Conference.

We look forward to building on this shared commitment to safe and ethical AI.

Learn more about PAI’s partners. 

The post Concordia AI Joins the Partnership on AI first appeared on Concordia AI.

]]> Concordia AI Joins the IASEAI Affiliate Program https://concordia-ai.com/concordia-ai-joins-the-iaseai-affiliate-program/?utm_source=rss&utm_medium=rss&utm_campaign=concordia-ai-joins-the-iaseai-affiliate-program Thu, 23 Oct 2025 07:31:18 +0000 https://concordia-ai.com/?p=851 We’re proud to share that Concordia AI has joined the founding cohort of the International Association for Safe and Ethical AI (IASEAI) affiliate program. Our team was honored to participate in the inaugural IASEAI Conference, where Concordia AI CEO Brian Tse spoke on the panel “Global Perspectives on AI Safety and Ethics.” It was an…

The post Concordia AI Joins the IASEAI Affiliate Program first appeared on Concordia AI.

]]> We’re proud to share that Concordia AI has joined the founding cohort of the International Association for Safe and Ethical AI (IASEAI) affiliate program.

Our team was honored to participate in the inaugural IASEAI Conference, where Concordia AI CEO Brian Tse spoke on the panel “Global Perspectives on AI Safety and Ethics.” It was an inspiring opportunity to exchange ideas on advancing responsible AI governance and safety worldwide.

Stay tuned for what’s next—beginning with IASEAI 2026, taking place in Paris this coming February.

Learn more about IASEAI affiliates.

The post Concordia AI Joins the IASEAI Affiliate Program first appeared on Concordia AI.

]]> Concordia AI: 2025 Mid-Year Impact Report https://concordia-ai.com/concordia-ai-2025-mid-year-impact-report/?utm_source=rss&utm_medium=rss&utm_campaign=concordia-ai-2025-mid-year-impact-report Wed, 24 Sep 2025 15:21:30 +0000 https://concordia-ai.com/?p=826 Our mission is to ensure that AI is developed and deployed in a way that is safe and aligned with global interests. We advance global AI safety by conducting research, advising leading AI companies and policymakers, and promoting international dialogue. Below are some of our key accomplishments from January to July 2025. (see 2024 highlights…

The post Concordia AI: 2025 Mid-Year Impact Report first appeared on Concordia AI.

]]> Our mission is to ensure that AI is developed and deployed in a way that is safe and aligned with global interests. We advance global AI safety by conducting research, advising leading AI companies and policymakers, and promoting international dialogue. Below are some of our key accomplishments from January to July 2025. (see 2024 highlights here).

Advancing international coordination on AI safety and governance

Research Impact and Engagement

  • International AI Safety Report:
    • Head of International AI Governance Kwan Yee Ng (吴君仪) contributed to the first International AI Safety Report as one of the writers and is continuing on as a writer for the 2026 edition of the International AI Safety Report. Chaired by Turing Award winner Yoshua Bengio, the report is supported by an expert panel representing 30 countries including China as well as experts from the EU and the UN. Concordia AI also provided feedback to the report, including editing the Chinese translation of its summary materials.
  • China AI safety and governance analysis:
    • Published the “State of AI Safety in China 2025” report, covering developments May 2024–June 2025. The report was cited by a number of media outlets including Wired, Bloomberg, and The People’s Daily (the largest newspaper in China).
    • CEO Brian Tse (谢旻希) authored an op-ed titled “China Is Taking AI Safety Seriously. So Must the U.S.” in Time Magazine; International AI Governance Senior Research Manager Jason Zhou and International AI Governance Part-time Researcher Gabriel Wagner analyzed AI safety implications of China’s April Politburo study session for Stanford DigiChina Forum; interviewed by CGTN on China’s approaches in AI innovation and global governance.
    • Published over 10 new “AI Safety in China” newsletter issues, reaching over 1,400 subscribers across governments, top AI labs, and AI safety institutes.
  • Singapore AI safety and governance analysis:

Multilateral Initiatives

  • Global AI Summit series: Participated in the French AI Action Summit, including:
    • Brian Tse was invited as a Chinese civil society representative to the AI Action Summit in the Grand Palais.
    • Co-hosted the workshop “AI Safety as a Collective Challenge” on the sidelines of the Summit, alongside the Carnegie Endowment for International Peace (CEIP), Oxford Martin AI Governance Initiative (AIGI), Tsinghua University Center for International Security and Strategy (CISS), and Tsinghua Institute for AI International Governance (I-AIIG). During the event, Concordia AI co-published the report “Examining AI Safety as a Global Public Good: Implications, Challenges, and Research Priorities.”
    • Invited to a closed-door seminar hosted by the China AI Safety & Development Association (CnAISDA). During the subsequent public side event, Turing Award Winner Andrew Yao cited Concordia AI’s State of AI Safety in China report series when describing the increase in AI safety research in Chinese institutions.
    • Attended the inaugural conference of International Association of Safe and Ethical AI (IASEAI), where Brian Tse spoke on the panel “Global Perspectives on AI Safety and Ethics.”
    • Brian Tse delivered a presentation at the France-China AI Association (Association d’Intelligence Artificielle France-Chine), which was cited by outlets including Xinhua.
  • United Nations:
    • Provided written inputs and participated in consultations regarding the UN’s Independent International Scientific Panel on AI and Global Dialogue on AI.
    • Brian Tse spoke on the panel “From Principles to Practice—Governing Advanced AI in Action” at the AI for Good Summit 2025.
  • Global AIxBiosecurity governance: We contributed to a number of critical global discussions at the intersection of AI and biosecurity:
  • International expert consensus and statements:
    • Brian Tse participated in the International Dialogues on AI Safety-Shanghai, signing the Shanghai Consensus on Ensuring Alignment and Human Control of Advanced AI Systems, alongside a Nobel laureate, Turing Award winners, and senior policymakers.
    • Brian Tse and Kwan Yee Ng contributed to and signed The Singapore Consensus on Global AI Safety Research Priorities during the Singapore Conference on AI 2025 (SCAI).

Convening AI safety conferences in China, Singapore, and globally

  • World AI Conference Conference (WAIC), Shanghai
    • Hosted the AI Safety and Governance Forum at China’s most influential AI conference.
      • Convened around 30 distinguished experts, including Yoshua Bengio; United Nations Under-Secretary-General Amandeep Singh Gill; Shanghai AI Lab Director ZHOU Bowen (周伯文); Special Envoy of the President of France for AI Anne Bouverot; Distinguished Professor of Computer Science at UC Berkeley Stuart Russell; Peng Cheng Laboratory Director Academician GAO Wen (高文); CEO of the Partnership on AI Rebecca Finlay; Shanghai Artificial Intelligence Strategic Advisory Expert Committee member Academician HE Jifeng (何积丰); and many more leading figures from government, industry, and research.
      • Over 200 audience members joined in person, with over 14,000+ views of the livestream, and media coverage from Bloomberg, Wired, Caixin, IT Times, and Tech Review Africa.
    • Served as official AI Governance Advisor for WAIC 2025.
    • Co-hosted a number of frontier AI safety workshops on the sidelines of WAIC:
      • Co-hosted a workshop on “Early Warning and Crisis Coordination for Advanced AI” with the Carnegie Endowment for International Peace, Oxford Martin School AI Governance Initiative, Oxford China Policy Lab, Tsinghua University Center for International Security and Strategy (CISS), and Tsinghua University Institute for AI International Governance (I-AIIG).
      • Co-hosted a workshop on “Convergence of AI and Biological Risks Workshop” with the Tianjin University Center for Biosafety Research.
      • Hosted a workshop on “Towards International AI Risk Management Standards.”
      • Co-hosted the “International Workshop on AI Deception Risks and Governance” with Fudan University, Safe AI Forum.
  • Beijing Academy of AI Conference 2025
    • Co-hosted the “AI Safety Forum” with the Beijing Academy of Artificial Intelligence (BAAI) at the BAAI Conference 2025. The forum brought together leading technical experts from institutions including MIT, Fudan University, Singapore Management University, and Tsinghua University to build scientific consensus on technical evaluations for AI “red lines.”
  • Asia Tech x Singapore, 2025
    • Organised the AI Risk Management Workshop, with support from Singapore’s Infocomm Media Development Authority, bringing together 20+ global experts across policy, industry, AI assurance, and academia to explore actionable risk management approaches for AI systems.
  • International Conference on Learning Representations (ICLR 2025), Singapore
    • Co-hosted and participated in a series of events, including:
      • Co-hosted the “Frontier Governance Exchange” with Singapore AI Safety Hub, Lorong AI and Safe AI Forum.
      • Co-convened the “Misalignment and Control Workshop” and a 130+ person AI Safety Social with FAR.AI, the Safe AI Forum, and Singapore AI Safety Hub.
      • Kwan Yee Ng presented on AI Safety in China at FAR.AI’s Singapore Alignment Workshop 2025.

 

Advising leading AI companies and policymakers in China

  • National Standards and Policy Guidance: Concordia AI is a member of key national and industry technical committees, contributing to the development of China’s AI safety standards.
    • National Information Security Standardization Technical Committee (SAC/TC260): As part of SAC/TC260 Special Working Group on Emerging Technology Safety, Concordia AI contributed to the standard for “Classification and Grading Methods for the Security of Artificial Intelligence Applications.”
    • National Information Technology Standardization Technical Committee (SAC/TC28/SC42): As a member of the AI Subcommittee, Concordia AI contributed to the “Artificial intelligence—Risk management capability assessment.”
    • Ministry of Industry and Information Technology AI Standardization Committee (MIIT/TC1): Concordia AI joined the Working Group on AI Safety Governance.
    • Guangdong-Hong Kong-Macao Greater Bay Area local standards: As a member of the Greater Bay Area working group of SAC/TC28/SC42, Concordia AI played a key role in the development of the Shenzhen local standard “Technical Framework for Value Alignment of Pre-trained AI Models.”
  • Frontier AI Safety Risk Management and Best Practices:
    • Co-published the “Frontier AI Risk Management Framework v1.0” with Shanghai AI Lab. It is China’s first comprehensive framework for managing severe risks from general-purpose AI models.
      • We propose a robust set of protocols designed to empower general-purpose AI developers, with comprehensive guidelines for proactively identifying, assessing, mitigating, and governing a set of severe AI risks that pose threats to public safety and national security.
      • The Framework outlines a set of unacceptable outcomes (red lines) and early warning indicators for escalating safety and security measures (yellow lines) for areas including: cyber offense, biological threats, large-scale persuasion and harmful manipulation, and loss of control risks.
    • Signed strategic partnership agreements with several leading Chinese general-purpose AI developers, providing advice on AI safety and risk management best practices.
    • Invited to present on frontier AI risk management during a closed-door workshop at the AI Industry Alliance of China’s 15th Plenum Meeting.
    • Co-hosted a workshop on “EU Code of Practice & Industry Best Practices: Towards a Global Standard for AI Risk Management, Safety and Security” with SaferAI, the Oxford Martin AI Governance Initiative, the Safe AI Forum.
  • Frontier AI Risk Monitoring and Evaluation:
    • Contributed to the Frontier AI Risk Management Framework in Practice: A Risk Analysis Technical Report led by Shanghai AI Lab. We assessed critical risks from more than 20 frontier LLMs in the following areas: cyber offense, biological and chemical risks, persuasion and manipulation, uncontrolled autonomous AI R&D, strategic deception and scheming, self-replication, and collusion.
    • Soft-launched an AI Risk Monitoring Platform designed to track and mitigate frontier AI risks, including cyberoffense, biological threats, chemical threats, and loss-of-control domains. The platform evaluates 34 frontier LLMs from 11 leading developers across the U.S., China, and France, using 18 open-source benchmarks. Key outputs include a risk index dashboard and a detailed technical report.
    • AI Safety Research Manager DUAN Yawen (段雅文) co-authored “Bare Minimum Mitigations for Autonomous AI Development.”
  • AIxBiosecurity Governance:
    • Published Chinese language report “Responsible Innovation in AI x Life Sciences” with Tianjin University’s Center for Biosafety Research and Strategy. This 70-page deep dive draws on 300+ sources to explore AI-biotech convergence, benefits, risks, and governance recommendations for diverse stakeholders.
      • Head of AI Safety and Governance (China) FANG Liang (方亮) presented the report at a biosecurity seminar hosted by China’s National Key Laboratory of Synthetic Biotechnology.
      • Presented the report at the 2025 International Symposium on Global Biosecurity Governance and Cooperation, co-hosted by the National Biosecurity Expert Committee of China, Guangzhou Laboratory, and China Foreign Affairs University.
    • Invited to participate in the “Closed-door Seminar on DNA Synthesis Screening Technology and Policy” held at China Foreign Affairs University.
  • WeChat Newsletter Publications:
    • Released over 59 new posts in our WeChat Official Account, reaching over 4,600 subscribers across China’s AI ecosystem, including policymakers, industry professionals, academic researchers, and the public.

Organizational updates

  • Organizational growth:
    • Following the establishment of our Singapore office, our team expanded from 8 to 12 members, welcoming our first Singapore-based staff member.
  • International partnerships:
  • Branding and Communication:
    • We launched a new English organizational website with refreshed branding to showcase our work. This is complemented by an updated brochure and a dedicated WeChat post to provide an introduction to our mission and activities.

The post Concordia AI: 2025 Mid-Year Impact Report first appeared on Concordia AI.

]]> Preview: Concordia AI at the 2025 World AI Conference https://concordia-ai.com/preview-concordia-ai-at-the-2025-world-ai-conference/?utm_source=rss&utm_medium=rss&utm_campaign=preview-concordia-ai-at-the-2025-world-ai-conference Thu, 24 Jul 2025 10:20:40 +0000 https://concordia-ai.com/?p=715 We’re excited to share that Concordia AI will be hosting a dedicated AI Safety and Governance Forum at the World AI Conference (WAIC) in Shanghai on July 27, which will be livestreamed globally (full agenda and link below). What is WAIC? WAIC is China’s largest and most influential AI conference, held annually since 2018. It brings together…

The post Preview: Concordia AI at the 2025 World AI Conference first appeared on Concordia AI.

]]> We’re excited to share that Concordia AI will be hosting a dedicated AI Safety and Governance Forum at the World AI Conference (WAIC) in Shanghai on July 27, which will be livestreamed globally (full agenda and link below).

What is WAIC?

WAIC is China’s largest and most influential AI conference, held annually since 2018. It brings together senior Chinese government leaders, foreign diplomats, and renowned experts from across the globe.

WAIC is co-organized by seven central government ministries (MOFA, NDRC, MIIT, MoE, MoST, SASAC, and CAC), along with the Chinese Academy of Sciences (CAS), the China Association for Science and Technology (CAST), and the Shanghai Municipal Government.

In 2024, WAIC was for the first time called a “High-Level Meeting on Global AI Governance”, and saw an unprecedented caliber of senior officials in attendance, most notably Premier LI Qiang (李强). Last year’s edition also increased its focus on AI safety and governance.

This year’s WAIC, held under the theme “Global Solidarity in the AI Era”, is expected to break participation records and to maintain its high political profile.

For foreign stakeholders, WAIC is a key event to watch, and a critical opportunity to engage with Chinese counterparts.

What to watch at WAIC

Opening ceremony

The opening ceremony typically draws the highest level of attention, featuring senior Chinese officials, foreign dignitaries, leaders of international organizations, and CEOs of major AI companies.

At WAIC 2024, for instance, Premier Li Qiang, China’s second-highest ranking official, delivered opening remarks emphasizing that AI development must ensure safety, reliability, and controllability. Shanghai Party Secretary CHEN Jining (陈吉宁), one of China’s top 24 officials, also announced the “Shanghai Declaration on Global AI Governance,” which warns of “unprecedented challenges” in AI safety and ethics. That same phrase—“unprecedented risks”—was later echoed by President Xi Jinping during a Politburo study session in April 2025, underscoring how WAIC can foreshadow domestic policy trends.

The opening ceremony also features speeches from leading scientists and policy advisors. In 2024, Tsinghua University Institute for AI International Governance (I-AIIG) Dean XUE Lan (薛澜) noted risks of loss of control of AI systems and threats to national security from AI misuse, while Shanghai AI Lab (SHLAB) Director ZHOU Bowen (周伯文) argued that AI safety must be advanced alongside capabilities through alignment, explainability, and reflection in order to achieve trustworthy artificial general intelligence (AGI).

Ministerial roundtable

In 2024, China’s Ministry of Science and Technology (MOST) held a ministerial roundtable with representatives from around 30 other countries. MOST Minister YIN Hejun (阴和俊) noted that AI is an issue “of great importance for the fate of humanity,” while Vice Minister of Foreign Affairs MA Zhaoxu (马朝旭) highlighted the importance of preserving “safety bottom lines” and maintaining human control over AI systems. A similar gathering may occur again this year, even though it remains unconfirmed as of writing.

Join us: Concordia AI’s AI Safety and Governance Forum

WAIC typically features more than 100 thematic “forums,” which often run half-day or day-long. It is impossible to give a full overview here, and we recommend readers check out the full agenda on the official conference website to get a sense of the range of cutting-edge topics covered. There are multiple forums related to AI safety and governance this year.

We are excited to announce that Concordia AI is hosting the AI Safety and Governance Forum. Last year, our forum drew almost 300 in-person attendees and over 800,000 livestream views. This year, we’re proud to present another outstanding lineup of leading scientists, policy experts, and industry voices from China and around the world, coming together to discuss frontier AI safety and global governance.

???? Date: July 27
???? Time: 09:00–17:30 (UTC+8)
???? Venue: Shanghai World Expo Center, Meeting Room 518
???? Livestream: Morning session (09:00–12:30 UTC+8); afternoon session (13:35–17:30 UTC+8)

The Forum will be structured around four themes:

  • Theme 1: The Science of AI Safety: Leading researchers will share insights on the problem “superalignment” and potential pathways to aligned AGI.
  • Theme 2: Emerging Challenges in AI Safety: This session examines cutting-edge problems in AI safety, including dual-use challenges in cybersecurity and biosecurity, and early warning signs of deceptive or self-replicating AI systems.
  • Theme 3: AI Risk Management in Practice: A comparative look at influential governance approaches across China, the EU, and the US, identifying shared concerns and promising practices for managing AI risk.
  • Theme 4: International Governance of AI Safety: The final session explores the evolving landscape of international cooperation on AI safety and potential next steps for global governance.

The post Preview: Concordia AI at the 2025 World AI Conference first appeared on Concordia AI.

]]>