Microsoft Azure AI Fundamentals (AI-900)Describe fundamental principles of machine learning on AzureHard
A team of data scientists is evaluating a machine learning model designed to predict customer churn. They have identified that the cost of incorrectly classifying a loyal customer as a churner (False Positive) is significantly lower than the cost of incorrectly classifying a churner as a loyal customer (False Negative), which leads to lost revenue. To optimize the model for business impact, which metric should they prioritize to minimize the latter type of error (missing actual churners)?
- ARecall
- BAccuracy
- CPrecision
- DSpecificity
Show answer & explanationAnswer & explanation
Correct answer: A. Recall
Recall (also known as Sensitivity or True Positive Rate) measures the proportion of actual positive cases that were correctly identified by the model. Prioritizing Recall minimizes False Negatives, meaning the model is better at identifying all actual churners, which is crucial when missing churners is more costly.
Why the other options are wrong
- B. Accuracy measures overall correct predictions and can be misleading in imbalanced scenarios or when error types have different costs.
- C. Precision measures the proportion of positive predictions that were actually correct. Prioritizing Precision would minimize False Positives, which is not the primary goal here.
- D. Specificity measures the proportion of actual negative cases that were correctly identified. Prioritizing Specificity would minimize False Positives, similar to Precision.
Recall (Sensitivity)
A classification metric that measures the proportion of actual positive cases that were correctly identified by the model. It is calculated as True Positives / (True Positives + False Negatives).
- Also known as Sensitivity or True Positive Rate.
- High Recall means fewer False Negatives (missing actual positives).
- Crucial when the cost of False Negatives is high (e.g., medical diagnosis, fraud detection).
Memory trick: When mistakes have different prices, choose wisely.