兰德-强化学习人工智能系统的风险评估-超越技术（英）-2024.7-101页_1mb

Reinforcement learning (RL) faces several significant technical challenges when applied to broader and more complex problems, as highlighted in Chapter 2. These challenges include:

1. Domain Generalization: RL systems struggle to generalize across different environments due to shifts in dynamics, rewards, and other variables. This limits their ability to perform reliably in real-world scenarios not covered during training.

2. Computational Feasibility: The computational demands of RL scale with problem complexity. For instance, training algorithms like those used in complex games or real-world applications requires massive resources, energy, and time, which may not be sustainable for broader DoD needs.

3. Reward Function Design:

   • Inefficient Exploration: The reward function must balance exploration and exploitation. Poorly designed functions can lead to suboptimal behavior, lack of adequate exploration, or unintended reward hacking (e.g., an agent exploiting loopholes for rewards).
   • Reward Hacking: Agents may achieve goals unintended by designers, leading to catastrophic failures in critical applications.

4. Vulnerability to Adversarial Attacks: RL agents, often relying on deep neural networks, are susceptible to adversarial attacks that manipulate input data to disrupt performance. These risks are heightened in broad applications where environments are more unpredictable.

5. Lack of Explainability and Trust: The "black box" nature of RL makes it difficult to understand or trust decisions. Without transparency, users may dismiss outputs or fail to detect subtle failures, limiting RL’s applicability in high-stakes scenarios.

6. Explainability and Trust Challenges: The inability to predict or explain RL decisions complicates risk assessment, training, and user trust. This limits RL to narrow tasks where intuitive oversight is feasible.

7. Adversarial Attack Risks: RL systems are vulnerable to attacks exploiting weaknesses in their design or training data, potentially leading to cascading failures in complex applications.

These technical challenges underscore the need for more robust algorithms, transparent models, and comprehensive risk mitigation strategies before widespread deployment in DoD applications.