2018年-WEF世界经济论坛_How_to_Prevent_Discriminatory_Outcomes_in_Machine_Learning_30页_1mb

Summary of "How to Prevent Discriminatory Outcomes in Machine Learning"

Core Content

This document, published by the World Economic Forum Global Future Council on Human Rights in 2018, explores the challenges and responsibilities associated with preventing discriminatory outcomes in machine learning (ML) systems. It emphasizes the ethical and human rights implications of ML in various sectors, such as finance, education, healthcare, and employment, and calls for proactive measures by businesses to ensure fairness, transparency, and accountability.

Main Points

Discriminatory Outcomes in Machine Learning

• Potential for Discrimination: ML systems can unintentionally or intentionally reinforce systemic bias and discrimination, often due to biased training data or flawed algorithm design.
• Impact on Human Rights: Discriminatory outcomes not only violate human rights but also undermine public trust in technology, which can lead to restrictive regulations that hinder ML's potential for social and economic benefit.
• Examples of Discrimination:
  • In Kenya, ML-based loan systems may unfairly exclude rural farmers due to limited digital footprints.
  • In Indonesia, ML models used for hiring may systematically disadvantage applicants from less developed regions.
  • In Mexico and China, there are documented cases of ML systems being used to exclude people with mental disabilities or to implement social credit scoring, which can have significant discriminatory effects.

Risks in Data and Algorithm Design

• Data Availability and Bias: Training data may be biased or incomplete, leading to discriminatory models. Data may be collected from urban populations, excluding rural or less privileged groups.
• Opaqueness of ML Systems: ML systems are often "black boxes," making it difficult to trace decisions back to their source or understand how they are made.
• Exclusiveness of ML Development: ML systems are typically developed by small, homogenous teams, often composed of men, and rely on proprietary data, which limits access and diversity in the development process.

Algorithm Design Concerns

• Model Selection: Choosing the wrong model for a specific context can lead to discrimination, as models that work well in one setting may not be appropriate in another.
• Inadvertent Discrimination: Features that appear neutral may encode bias if not carefully considered. For example, a model may assign higher importance to income levels, which can disproportionately affect women or other lower-income groups.
• Lack of Human Oversight: As ML systems become more autonomous, the absence of human involvement can result in the system overlooking important contextual factors, leading to unjust outcomes.
• Unpredictable Systems: The complexity of ML makes it difficult to anticipate and address all possible discriminatory impacts.

Key Principles for Businesses

To combat discrimination in ML, the document proposes four guiding principles:

1. Active Inclusion: ML applications should actively seek diverse inputs, especially from populations affected by the system's outputs.
2. Fairness: Companies should define and prioritize fairness in their ML systems, ensuring that the chosen definition of fairness is appropriate for the specific application.
3. Right to Understanding: ML systems involved in decision-making must be transparent and explainable, so that individuals can understand the reasoning behind decisions and that competent human authorities can review them.
4. Access to Remedy: Companies should identify potential negative impacts and provide clear mechanisms for redress if discrimination occurs.

Recommended Actions for Companies

The document outlines three key steps for businesses:

1. Identify Human Rights Risks: Companies should assess the adequacy of their training data and its potential biases through a multi-stakeholder approach.
2. Take Effective Action: Businesses should implement governance mechanisms to prevent and mitigate risks, including ethical compliance frameworks.
3. Be Transparent: Companies should monitor their ML applications, report findings, and work with third-party auditors in a manner similar to industries with high regulatory scrutiny, such as mining.

Conclusion

The document highlights the need for businesses to take the lead in addressing the human rights implications of ML, especially in the absence of sufficient government regulation. It advocates for a proactive, ethical approach to ML development and deployment, grounded in the principles of human rights and the Universal Declaration of Human Rights. The goal is to ensure that ML systems are fair, inclusive, and respectful of human dignity.

Appendix Highlights

• Appendix 1: Glossary of terms related to ML and human rights.
• Appendix 2: Summary of actions companies can take to address the challenges outlined in the document.
• Appendix 3: Ethical principles for the design and use of AI and autonomous systems.
• Appendix 4: Matrix outlining areas of action for human rights in ML.

Acknowledgements

The document acknowledges the contributions of various experts and organizations, including Microsoft, Google, and UNICEF Innovation, in the ongoing discourse on ethical AI and human rights.