AWS Certified AI PractitionerAI/ML and Generative AI FundamentalsMedium
A financial institution is implementing a machine learning model to detect fraudulent transactions. They are particularly concerned about missing actual fraudulent transactions, even if it means flagging some legitimate transactions incorrectly. Which metric should they prioritize to evaluate the model's effectiveness?
- ARecall (Sensitivity)
- BF1-Score
- CAccuracy
- DPrecision
Show answer & explanationAnswer & explanation
Correct answer: A. Recall (Sensitivity)
Recall, also known as Sensitivity, measures the proportion of actual positive cases (fraudulent transactions) that were correctly identified by the model. In scenarios where missing a positive case (a fraudulent transaction) has severe consequences, maximizing recall is crucial, even if it leads to a higher number of false positives (legitimate transactions flagged as fraud).
Why the other options are wrong
- B. F1-Score is the harmonic mean of precision and recall, useful when both are important, but here, recall is paramount.
- C. Accuracy can be misleading in imbalanced datasets like fraud detection, as it doesn't differentiate between types of errors.
- D. Precision measures the proportion of positive identifications that were actually correct, which is less critical here than finding all fraud.
Recall (Sensitivity)
A metric that measures the proportion of actual positive cases that are correctly identified by a model.
- Calculated as True Positives / (True Positives + False Negatives).
- High recall indicates fewer false negatives (missed positives).
- Crucial in domains where missing positive cases is costly (e.g., fraud, disease detection).
- Also known as Sensitivity or True Positive Rate.
Memory trick: PRFA: Pick the Right Metric For the Application