AWS Certified Machine Learning – SpecialtyModelingMedium
A manufacturing company is using a machine learning model to predict equipment failure based on sensor data. The cost of a false negative (predicting no failure when one occurs, leading to production downtime) is significantly higher than the cost of a false positive (predicting failure when none occurs, leading to unnecessary inspection). The current model has an F1-score of 0.85. To optimize for the company's specific cost structure, which evaluation metric should the data scientist prioritize and aim to maximize?
- AAccuracy
- BRecall
- CPrecision
- DSpecificity
Show answer & explanationAnswer & explanation
Correct answer: B. Recall
A false negative (Type II error) in this scenario means missing an actual equipment failure, which is explicitly stated as having a significantly higher cost. Recall (also known as sensitivity) measures the proportion of actual positive cases that are correctly identified. Maximizing recall directly minimizes false negatives.
Why the other options are wrong
- A. Accuracy can be misleading in imbalanced datasets and doesn't differentiate between the costs of false positives and false negatives.
- C. Precision focuses on minimizing false positives (predicting failure when none occurs), which is less critical than false negatives in this scenario.
- D. Specificity measures the proportion of actual negative cases correctly identified (minimizing false positives), similar to precision's goal, but the problem prioritizes minimizing false negatives.
Recall (Sensitivity)
The proportion of actual positive cases that are correctly identified by the model.
- Calculated as True Positives / (True Positives + False Negatives).
- High recall indicates a low number of false negatives.
- Critical when the cost of missing a positive instance is high.
Memory trick: Costly misses need high Recall.