AWS Certified Machine Learning – SpecialtyModelingEasy
A data scientist is working on a binary classification problem to predict customer churn. They have trained several models and observed the following performance metrics on a held-out test set: Model A: Accuracy = 0.92, Precision = 0.85, Recall = 0.70, F1-score = 0.77 Model B: Accuracy = 0.88, Precision = 0.90, Recall = 0.65, F1-score = 0.76 Model C: Accuracy = 0.90, Precision = 0.80, Recall = 0.82, F1-score = 0.81 Model D: Accuracy = 0.93, Precision = 0.75, Recall = 0.90, F1-score = 0.82 The business team emphasizes that identifying as many churning customers as possible is critical, even if it means a higher rate of incorrectly flagging non-churning customers. Which model should the data scientist recommend?
- AModel C
- BModel B
- CModel D
- DModel A
Show answer & explanationAnswer & explanation
Correct answer: C. Model D
The business requirement emphasizes identifying as many churning customers as possible, which directly corresponds to maximizing Recall. Model D has the highest Recall score (0.90) among all models, making it the best choice for this specific objective.
Why the other options are wrong
- A. Model C has a good F1-score but its Recall (0.82) is lower than Model D, meaning it would miss more churning customers.
- B. Model B has the lowest Recall (0.65), making it unsuitable for the stated business goal.
- D. Model A has a lower Recall (0.70) compared to Model D, which does not meet the primary objective as well.
Recall (Sensitivity)
Recall measures the proportion of actual positive cases that were correctly identified by the model. It's crucial when the cost of false negatives is high.
- Calculated as True Positives / (True Positives + False Negatives).
- Focuses on minimizing false negatives.
- Important in scenarios like disease detection or fraud detection where missing a positive case is costly.
Memory trick: Remember 'Recall' as 'CATCH ALL' the positives, 'Precision' as 'BE PRECISE' with your positives.