> **来源：[研报客](https://pc.yanbaoke.cn)** # Summary of "Beyond Algorithms: The Real Risks of AI" ## Core Content This report provides a comprehensive analysis of the **technical, operational, and human risks** associated with the adoption of AI in organisations. It highlights the growing complexity of AI security, emphasizing the need for a holistic approach that integrates **technological innovation with robust risk management and algorithmic accountability**. The report is structured around three key levels: 1. **Technical Risks**: Focus on vulnerabilities in AI models, data, and infrastructure. 2. **Operational Risks**: Address the impact of AI on internal processes, business continuity, and organisational resilience. 3. **Human Risks**: Consider the role of error, ignorance, and manipulation in AI security. It also identifies the **actors** who exploit these risks, including **cybercriminals** and **state-sponsored groups**, and provides **practical recommendations** to mitigate these threats while aligning with **European regulatory frameworks** such as the AI Act, NIS2, ENS, and DORA. --- ## Main Attack Types and Risk Vectors | Attack Type | Affected Phase | Main Impact | Related to... | |------------------------------------|------------------------|---------------------------------------|----------------------------------------| | Prompt Injection | Input / Integrations | Integrity / Confidentiality | Insecure Output, Insecure Plugins, Excessive Agency | | Insecure Output Management | Inference / Operation | Integrity / Physical Security / Cost | Prompt Injection, Poisoning, Bias, Excessive Dependency | | Data Poisoning | Data / Training | Model Integrity | Supply Chain, Insecure Output | | Data Leakage | Training / Inference | Confidentiality / Compliance | Disclosure of Confidential Information, Excessive Dependency | | Denial of Service (DoS) | Operation / Integrations | Availability / Cost | Supply Chain, Excessive Agency | | Supply Chain Vulnerabilities | Data / Training / Operation | Integrity / Confidentiality / Availability | Poisoning, Insecure Output, Insecure Plugins | | Disclosure of Confidential Information | Integrations / Inference | Confidentiality / Reputation | Data Leakage, Insecure Output, Excessive Dependency | | Insecure Plugins | Integrations / Operation | Integrity / Confidentiality | Insecure Output, Excessive Agency, Excessive Dependency | | Excessive Agency | Operation / Business Logic | Systemic Risk | Prompt Injection, Insecure Output, Plugins, Excessive Dependency | | Excessive Dependency | Governance / Operation | Continuity / Sovereignty | Excessive Agency, Insecure Output, Disclosure | | Model Theft | Training / Inference | I.P. / Evasion | Data Leakage, Poisoning, Excessive Dependency | --- ## Key Risks and Impacts - **Prompt Injection**: Manipulates model inputs to alter behavior or extract sensitive data. Can bypass traditional security controls. - **Insecure Output Management**: Outputs are used in critical processes without validation, leading to errors, fraud, or compliance issues. - **Data Poisoning**: Introduces harmful data during training, affecting model integrity and potentially enabling attacks. - **Data Leakage**: Models may inadvertently expose confidential or private data through queries, attacks, or configuration errors. - **Denial of Service (DoS)**: Overwhelms AI models with traffic or manipulated data, affecting availability and increasing costs. - **Supply Chain Vulnerabilities**: Third-party components introduce inherited risks, potentially compromising the entire AI pipeline. - **Excessive Agency**: AI systems perform actions autonomously without sufficient human oversight, amplifying existing vulnerabilities. - **Excessive Dependency**: Over-reliance on AI reduces human control and increases exposure to external threats. These risks not only threaten the **technical integrity** of AI systems but also have **operational and legal consequences**, especially in regulated sectors such as healthcare, finance, and public administration. --- ## Actors Exploiting AI Risks - **Cybercrime Groups**: Such as FunkSec, GXC Team, Indrik Spider, Renaissance Spider. - **APT Groups**: Including APT28, Ember Bear, APT41, RedHotel, Sodium, Ta499, Imperial Kitten, Charming Kitten, APT42, Lazarus Group, Void Arachne. These actors are increasingly using AI to enhance their capabilities, including **Malware as a Service**, **Ransomware as a Service**, and **AI-based attacks**, which require advanced defences and awareness. --- ## Recommendations 1. **AI Security Audit**: Regularly assess AI systems for vulnerabilities, using adversarial prompt libraries and frameworks like the OWASP Top 10 for LLM Applications. 2. **Awareness and Cybersecurity Training**: Educate administrators and security teams to identify manipulation patterns and anomalous behavior. 3. **Input and Output Controls**: Implement strict input filtering, sandbox environments, and access restrictions to prevent injection and manipulation. 4. **Data Governance**: Validate data quality, apply privacy-enhancing technologies (PETs), and ensure traceability and version control in data pipelines. 5. **Supply Chain Management**: Audit and patch external dependencies, maintain SBOM practices, and ensure secure integration with third-party services. 6. **Human Oversight**: Ensure human validation in critical decisions, especially in high-impact environments, to prevent over-agency and over-dependency. 7. **Regulatory Compliance**: Align with the AI Act, GDPR, NIS2, and ENS to ensure ethical, secure, and traceable AI deployment. --- ## About S2GRUPO S2GRUPO is a company that provides **informational and professional insights** on AI security. The report reflects their **expert analysis** and aims to support **CISOs, technology managers, and cybersecurity teams** in understanding and managing AI-related risks. --- ## Conclusion The adoption of AI introduces new and complex security challenges that require a **multi-layered approach** to mitigate. By addressing **technical, operational, and human risks**, and by understanding the **actors** and **attack vectors**, organisations can better prepare for **emerging threats** and ensure **safe, ethical, and sustainable AI deployment**.