AWS Certified AI PractitionerAI/ML and Generative AI FundamentalsMedium
A data scientist is evaluating the performance of a binary classification model that predicts fraud. The model has a high number of false positives, meaning many legitimate transactions are flagged as fraudulent. Which metric should the data scientist focus on to reduce these false positives without significantly missing actual fraudulent cases?
- APrecision
- BAccuracy
- CF1-Score
- DRecall
Show answer & explanationAnswer & explanation
Correct answer: A. Precision
False positives occur when the model incorrectly predicts the positive class (fraud) for a negative instance (legitimate transaction). Precision measures the proportion of true positive predictions among all positive predictions (True Positives / (True Positives + False Positives)). By optimizing for higher precision, the model will be more conservative in its positive predictions, thereby reducing false positives.
Why the other options are wrong
- B. Accuracy measures overall correct predictions but can be misleading in imbalanced datasets and doesn't specifically target false positives.
- C. F1-Score is the harmonic mean of precision and recall, providing a balance, but if the primary goal is reducing false positives, precision is the more direct metric to optimize.
- D. Recall (or sensitivity) measures the proportion of actual positives correctly identified, which would be important for not missing fraud, but optimizing it alone might increase false positives.
Precision
In classification, precision is the ratio of correctly predicted positive observations to the total predicted positive observations.
- Focuses on the quality of positive predictions.
- High precision means fewer false positives.
- Calculated as True Positives / (True Positives + False Positives).
Memory trick: Metrics 'measure' how 'accurate' or 'precise' our predictions are.