2025年国际人工智能安全报告_298页_4mb

International AI Safety Report Summary

Core Content

The International AI Safety Report is a comprehensive scientific document that examines the capabilities and risks of general-purpose AI (AGI) and proposes methods for managing these risks. It is the result of a collaborative effort by 96 international AI experts, including a diverse Expert Advisory Panel nominated by 30 countries, the UN, EU, and OECD. The report aims to provide a shared scientific understanding of AI risks and help guide informed policymaking.

Main Contributors

• Chair: Prof. Yoshua Bengio (Université de Montréal / Mila – Quebec AI Institute)
• Scientific Lead: Soren Mindermann (Mila – Quebec AI Institute)
• Lead Writer: Daniel Privitera (KIRA Center)
• Writing Group: Includes experts from Stanford University, MIT, University of Oxford, and other leading institutions
• Senior Advisers: Daron Acemoglu (MIT), Geoffrey Hinton (University of Toronto), Stuart Russell (UC Berkeley), and many others

Key Findings

Capabilities of General-Purpose AI

• General-purpose AI has seen rapid improvements in recent years, particularly in programming, scientific reasoning, and abstract reasoning.
• It can now write computer programs, generate custom photorealistic images, and engage in extended open-ended conversations.
• Companies are increasingly developing general-purpose AI agents, which can autonomously plan and act with minimal human oversight.
• These agents have the potential to unlock new benefits and risks, including the ability to complete longer and more complex projects.

Risks of General-Purpose AI

1. Risks from Malicious Use

• Fake content can be generated, potentially harming individuals.
• Manipulation of public opinion is a growing concern, especially with the use of AI in information dissemination.
• Cyber offense and biological/chemical attacks could be enabled by advanced AI systems.
• Some AI systems may be used to conduct hacking or biological attacks, which could lead to societal-scale harm.

2. Risks from Malfunctions

• Reliability issues are a key concern, as AI systems may produce erroneous or harmful outputs.
• Bias in AI models can lead to discrimination against certain groups.
• Loss of control could occur if AI systems act beyond human oversight, especially with the rise of autonomous agents.

3. Systemic Risks

• Labour market risks: AI could displace workers or alter job structures.
• Global AI R&D divide: There is a gap in AI development between developed and developing nations.
• Market concentration and single points of failure: A few large companies dominate AI development, raising concerns about centralised control.
• Environmental risks: AI development consumes significant energy, potentially harming the environment.
• Privacy risks: AI systems may violate user privacy through data collection and processing.
• Copyright infringement: AI can generate content that infringes on intellectual property.

4. Impact of Open-Weight Models

• The use of open-weight models (like o3) may accelerate AI capabilities and reduce costs for inference.
• However, it also increases the risks associated with AI, as more people and organisations can access and use these models.

Risk Management and Policy Challenges

Technical Approaches

• Risk identification and assessment are essential for understanding the potential impact of AI.
• Mitigation techniques include training more trustworthy models, monitoring and intervention, and privacy-preserving methods.
• However, interpretability and reliability of AI models remain challenging for researchers and regulators.

Policy Challenges

• Evidence dilemma: Due to the rapid and unpredictable nature of AI advancements, policymakers often have to make decisions without full scientific evidence.
• Standardisation and coordination: There is a growing need for international standardisation and coordinated policy to manage AI risks effectively.

Conclusion

• The report highlights the importance of global collaboration in addressing AI safety.
• It acknowledges that while AI offers significant benefits, its risks must be managed to ensure safe and responsible development.
• The future of AI is uncertain, and the decisions made by governments and societies will determine whether it leads to positive or negative outcomes.
• The report serves as a foundation for future discussions at international summits, such as the AI Action Summit in Paris.

Recommendations

• Policymakers should prioritise evidence-based risk assessments.
• There is a need for international coordination and standardisation in AI risk management.
• Researchers and developers should focus on improving model reliability, interpretability, and safety.
• Civil society and industry should engage in ongoing dialogue to ensure that AI development aligns with ethical and societal values.

Additional Information

• The report does not recommend specific policies, but rather aims to inform and guide them.
• It is licensed under the Open Government Licence v3.0.
• The report is a synthesis of existing research, and not an official position of any government or organisation.
• It is supported by a wide range of stakeholders, including civil society groups, industry leaders, and international bodies.

References

• The report includes a list of acronyms and a glossary for clarity.
• It also provides instructions on how to cite the report.

Acknowledgements

• The Secretariat acknowledges the support and feedback from Angie Abdilla, Nitarshan Rajkumar, Geoffrey Irving, Shannon Vallor, Rebecca Finlay, and Andrew Strait.
• The UK Government provided operational support and ensured scientific independence for the report.

Final Note

• The stakeholders involved in the report are diverse and international, ensuring broader representation and scientific rigour.
• The report aims to promote consensus and collective action to manage AI risks and maximise its benefits.