AWS Certified AI PractitionerAI/ML and Generative AI FundamentalsMedium
A data scientist is investigating a trained AI model that predicts loan default risk. They discover that the model consistently assigns higher risk scores to applicants from a particular demographic group, even when other financial indicators are similar to those of lower-risk applicants from other groups. What type of issue does this scenario highlight?
- AOverfitting
- BUnderfitting
- CData leakage
- DAlgorithmic bias
Show answer & explanationAnswer & explanation
Correct answer: D. Algorithmic bias
This scenario describes algorithmic bias, where the AI model produces systematically unfair or discriminatory outcomes against certain groups. This can stem from biases in the training data (e.g., historical lending practices), biased feature selection, or even the algorithm itself amplifying existing disparities, leading to inequitable risk assessments.
Why the other options are wrong
- A. Overfitting means the model performs well on training data but poorly on new data, not necessarily showing systematic discrimination.
- B. Underfitting means the model is too simplistic and performs poorly on both training and new data, not specifically discriminatory.
- C. Data leakage occurs when information from outside the training dataset is used to create the model, leading to overly optimistic performance estimates, not direct discrimination.
Algorithmic Bias
A systematic and repeatable error in a computer system that creates unfair outcomes, such as favoring certain groups over others.
- Can originate from biased training data, feature selection, or model design.
- Leads to discriminatory or prejudiced predictions/decisions.
- Often unintentional but has significant real-world consequences.
- Requires careful auditing, debiasing techniques, and diverse data.
Memory trick: Bias in AI: Unfair, Unintended Outcomes