AWS Certified AI PractitionerAI/ML and Generative AI FundamentalsMedium
A financial institution is implementing a machine learning model to detect fraudulent transactions. The model is designed to classify transactions as either 'fraudulent' or 'legitimate'. Given that detecting actual fraud is extremely critical, which evaluation metric should the institution prioritize to ensure that as many fraudulent transactions as possible are caught, even if it means flagging some legitimate transactions incorrectly?
- AF1-Score
- BRecall (Sensitivity)
- CPrecision
- DAccuracy
Show answer & explanationAnswer & explanation
Correct answer: B. Recall (Sensitivity)
Recall, also known as sensitivity, measures the proportion of actual positive cases (fraudulent transactions) that were correctly identified by the model. Prioritizing recall means minimizing false negatives, which is crucial when the cost of missing a positive case is high.
Why the other options are wrong
- A. F1-Score is the harmonic mean of precision and recall, balancing both. While useful, it might not be the absolute priority when one type of error (false negatives) is significantly more costly than the other.
- C. Precision focuses on the proportion of positive identifications that were actually correct, which is important but not the primary concern when missing fraud is highly costly.
- D. Accuracy measures overall correctness but can be misleading in imbalanced datasets like fraud detection, where legitimate transactions far outnumber fraudulent ones.
Recall (Sensitivity)
A metric that measures the proportion of actual positive cases that were correctly identified by a classification model. It is calculated as True Positives / (True Positives + False Negatives).
- Focuses on minimizing false negatives.
- Important when the cost of missing a positive is high (e.g., fraud, disease).
- Also known as 'Sensitivity' or 'True Positive Rate'.
Memory trick: Models Predict, Metrics Evaluate Performance.