Fairness in Machine Learning: The Principles of Governance  - Vector Institute for Artificial Intelligence

### Fairness in Machine Learning: The Principles of Governance #### Overview This article, part of the Vector Institute for Artificial Intelligence's Trustworthy AI series, delves into the concept of fairness in machine learning (ML) models. It aims to provide non-technical stakeholders with a clear understanding of ML-specific risks and principles, enabling them to effectively participate in ML model risk management. The article emphasizes the ethical, legal, and reputational implications of biased ML systems and outlines strategies to ensure fairness in various stages of ML development and deployment. #### Key Concepts of Fairness **Definition of Fairness**: In the context of this paper, 'fair' means non-discriminatory against protected groups, such as race, sex, gender, religion, and age. Ensuring fairness is essential for ethical reasons, alignment with organizational values, legal compliance, and maintaining public trust in AI technology. #### Sources of Bias in Machine Learning 1. **Bias in Historical Data**: - **Redlining Example**: Historical data can embed biases from past discriminatory practices, such as redlining, where services like mortgages or insurance are denied or overcharged based on community attributes. If an ML system is trained on such data, it can perpetuate these biases, leading to unfair outcomes for protected groups. 2. **Bias in Data Collection**: - **Sample Bias**: Data collection methods can result in non-representative samples. For example, a survey published in a magazine may attract a subset of readers who are not representative of the entire population. - **Measurement Bias**: Errors in data collection can distort datasets. Poorly trained surveyors might include irrelevant or out-of-date information, leading to biased results. 3. **Bias in Model Design**: - **Algorithmic Bias**: Erroneous assumptions or poor implementation by modelers can produce biased outcomes. For instance, excluding relevant financial criteria as features might disproportionately affect qualified borrowers, leading to higher interest rates or loan denials. 4. **Bias Due to Feature Correlation**: - **Proxy Features**: Even when sensitive features (e.g., race, gender) are removed, non-sensitive features (e.g., income, ZIP code) may correlate with them, effectively reintroducing bias. A notable example is the 2000 incident where a researcher re-identified nearly 90% of U.S. state employees using only their birthdate, sex, and ZIP code. #### Principles for Ensuring Fairness in ML 1. **Consider the Fairness Requirements for Each Use Case**: - **Context-Specific Fairness**: The approach to fairness should vary based on the model's intended use. Models impacting customers directly may require stricter fairness measures compared to those used for internal processes like staffing decisions. Assess the sensitivity of the use case to determine the appropriate level of scrutiny and the specific definition of fairness to apply. 2. **Prioritize Fairness at Every Stage**: - **Continuous Monitoring**: Fairness should be a concern throughout the entire ML pipeline, from task definition to dataset construction, model definition, training and testing, and deployment. Regularly monitor fairness, input data, and model performance to ensure ongoing compliance. 3. **Include Diverse Stakeholders**: - **Multiple Perspectives**: Involve stakeholders from diverse backgrounds and experiences to identify potential biases. Subtle biases are more likely to be spotted when a variety of perspectives are considered during the design, interpretation, and monitoring of ML models. 4. **Involve Humans When Necessary**: - **Human Oversight**: For high-stakes use cases, integrate human experts who can overrule model decisions if bias is detected or suspected. Human intervention is crucial in ensuring that ML systems do not inadvertently harm protected groups. #### Ethical, Legal, and Reputational Implications - **Ethical Concerns**: Modern societal values condemn marginalization based on protected attributes. Biased ML systems can perpetuate this marginalization, which is ethically unacceptable. - **Legal Risks**: Organizations can be held liable for unfair decisions made by their algorithms. Ensuring fairness is not only a moral imperative but also a legal one. - **Reputational Risks**: Highly publicized instances of unfair practices, even if unintentional, can damage an organization's reputation and lead to significant financial losses. - **Trust in AI**: Instances of unfairness can undermine public trust in AI technology, which is crucial for its widespread adoption and exploration of its potential value. #### Conclusion Fairness in machine learning is a multifaceted and complex issue that requires careful consideration and management. Non-technical stakeholders play a vital role in this process by understanding the key concepts of fairness and participating in the governance of ML models. By following the principles of context-specific fairness, continuous monitoring, diverse stakeholder involvement, and human oversight, organizations can deploy ML systems responsibly, ensuring they are both effective and ethical. Awareness and proactive management of fairness are essential for building trustworthy AI systems that align with societal and organizational values.