Algorithmic Bias in Artificial Intelligence and Mitigation Strategies

Krishnan Parthasarathy

Chief Architect, Product Management, TCS BaNCS, TCS

Industry

Capital Markets

Banking

Solution

TCS BaNCS™

Highlights

Bias in artificial intelligence can lead to unfair outcomes, perpetuating existing inequalities and affecting groups based on factors like gender, age or race.
AI bias can be addressed through a combination of transparency, fairness, and accountability, with strategies tailored to specific types of bias.
Key areas to focus on include mitigating bias in training data and addressing algorithmic bias that arises from design or decision-making processes.

Read the article here

Overview

The significant advancements in applying artificial intelligence (AI) to various domains have raised concerns about the fairness and bias of AI systems. Responses from such systems can result in unfair outcomes and carry forward existing inequalities. Drawing the line between using AI for decision making and avoiding accusations of bias requires a combination of transparency, fairness and accountability. This is an attempt to review aspects of these biases, general and specific strategies that can be employed to mitigate such biases.

Bias in Artificial Intelligence refers to systematic errors in an AI system that can lead to unfair, prejudiced, or unbalanced outcomes. These biases often reflect and amplify societal inequalities present in the data used to train AI models. Bias can manifest in various ways, affecting different groups unfairly based on factors like race, gender, age, or socio-economic status. Drawing the line between using AI for decision making and avoiding accusations of bias requires a combination of transparency, fairness, and accountability. Here, we look at the various aspects of bias and how they can be mitigated by choice of appropriate strategies to overcome each of them.

Bias classification

Here’s an overview of the aspects of bias and how they contribute to the overall problem of intervening with AI outcomes.

Doing this helps us understand the basis for bias and the problems it can cause and lead us to strategies for mitigation.

Data bias: Data bias occurs when the training data is not representative of the real-world population causing skewness. E.g. A bank is using AI to predict loan repayment capacity of a person before approving his loan application, if the trained model reflects historical biases (gender, race etc.), the results would be discriminatory.
Algorithmic bias: This happens when an AI model’s design or mathematical assumptions lead to biased outcomes. E.g. If the loan repayment capacity predictor model algorithm uses address as a feature, people from certain neighborhoods which may be racially segregated could be unfairly penalized with their applications getting rejected.
Selection bias: Selection bias arises when the data used to train the AI does not fully capture the diversity of the population. E.g. In the loan repayment capacity predictor, since the bank may not have enough data about young people or immigrants, suggesting their applications to be reject.
Confirmation bias: Confirmation bias is when AI systems reinforce existing beliefs rather than presenting diverse perspectives. E.g. In the loan repayment capacity predictor, since the bank may have a lot of data about people with accounts with them for a long time, the loan applications of only such customers would get recommended for approval.
Automation bias: Automation bias is when users overly rely on AI decisions, assuming they are always correct, without questioning potential errors. E.g. In the loan repayment capacity predictor, if a human doesn’t review all recommendations from the system and chooses to accept them directly, it would result in automation bias.
Measurement bias: Measurement bias is when incorrect or unfair labels are used in training data – when the correlations are incorrectly assumed. E.g. A credit scoring AI that uses ZIP codes as a proxy for financial reliability may disadvantage lower income neighborhoods.

General strategies in overcoming bias

The following is a listing of general strategies which can be followed in overcoming various bias.

Understand and mitigate bias in data: Bias often comes from the data used to train AI models. Ensure that training data is diverse, representative, and regularly audited for disparities. Use fairness metrics to detect and mitigate bias in AI outcomes.
Use explainable AI (XAI): AI decisions should be interpretable. Use explainability tools (e.g., SHAP, LIME) to provide reasons behind AI driven outcomes. Make AI generated recommendations easy to understand for stakeholders.
Human oversight and review: AI should assist, not replace, human decision makers, especially in high stakes scenarios (e.g., hiring, lending, medical diagnosis). Implement “human-in-the-loop” processes where necessary.
Transparency and documentation: Clearly communicate how AI models work, including data sources, methodologies, and limitations. Provide documentation on AI decision making frameworks, ensuring regulators and stakeholders can assess them.
Regular audits and bias testing: Conduct ongoing audits of AI models to identify and correct any emerging biases. Use fairness tests (e.g., disparate impact analysis) to measure bias in outputs.
Ethical guidelines and governance: Establish AI ethics policies aligned with regulatory frameworks (e.g., GDPR, AI Act). Set up an AI ethics committee to review sensitive AI applications.
User feedback and redress mechanisms: Allow affected individuals to challenge AI decisions and provide alternative inputs. Create pathways for users to report potential biases or errors in AI outputs.

Specific strategies to overcome various types of bias.

Next, we dwell into specific strategies using which bias can be overcome in AI – addressing each of the types of bias specifically.

Data Bias

Mitigating bias in training data is crucial for building fair and ethical AI systems.

When the data used to train AI models is not representative of the real-world population, it leads to biased outcomes.

Bias in AI isn’t intentional and can have serious consequences. It may not be able to completely avoid bias in training data. However, by actively managing them, one can build AI systems that are fair, ethical, and trustworthy. Here are some strategies to mitigate bias in data.

Collect diverse and representative data: Ensure training data includes a broad range of demographic groups (e.g., race, gender, age, geography, socioeconomic status). Avoid datasets that are skewed or overrepresent certain populations at the expense of others. Use multiple data sources to capture different perspectives. e.g. in facial recognition, include images of people from different ethnic backgrounds, lighting conditions, and facial expressions to improve accuracy across groups.
Reweighting: Assign higher weights to underrepresented data points to balance the dataset.
- Resampling: Over sample minority groups or under sample majority groups to create a balanced dataset.
- Synthetic data generation: Use AI techniques (e.g. GANs) to create synthetic but realistic data for underrepresented groups.

Remove proxy variables that introduce bias: Some features act as proxies for sensitive attributes like race, gender, or income level. Example of problematic variables: ZIP codes (which correlate with race/income), names (which can imply gender/ ethnicity), or college names (which may favor privileged applicants). Run statistical tests to detect unintended correlations between input features and protected attributes.
Use fairness aware algorithms: Train AI models with fairness constraints, ensuring they do not favor or disadvantage specific groups.
Implement techniques like:
- Adversarial debiasing: Train the model to minimize bias while maximizing accuracy.
- Fair representation learning: Encode data in a way that removes discriminatory factors.
- Fair regularization: Add penalties for biased outcomes in the model’s loss function.

Conduct regular bias audits and testing: Perform bias detection tests before and after model training.

Demographic parity: Check if different groups receive similar predictions.
Equal opportunity testing: Ensure all groups have equal chances of positive outcomes.
Disparate impact analysis: Verify that no single group is disproportionately at advantaged or disadvantage.
Ensure transparency and human oversight:
- Document AI model decisions to explain how predictions are made.
- Include a human in the loop to review AI generated decisions and override them if necessary.
- Provide feedback loops where users can report biased decisions for further investigation.
Continual monitoring and updating of data:
- Biases can evolve over time, so AI models should be retrained with fresh and unbiased data periodically.
- There should be a process to set up ongoing audits to detect bias drift.
- Use real world feedback to identify and correct biases as they emerge.