AWS Certified AI PractitionerResponsible AIHard
A research team is developing an AI model to detect early signs of a rare disease from medical images. Due to the rarity of the disease, positive cases (patients with the disease) are significantly underrepresented in the available training data. Which evaluation metric should be prioritized to ensure the model effectively identifies as many actual disease cases as possible, even if it means a higher rate of false positives?
- AF1-Score
- BPrecision
- CRecall (Sensitivity)
- DAccuracy
Show answer & explanationAnswer & explanation
Correct answer: C. Recall (Sensitivity)
In medical diagnosis for rare diseases, missing a positive case (false negative) can have severe consequences. Recall (Sensitivity) measures the proportion of actual positive cases that are correctly identified. Prioritizing high recall ensures that the model catches as many true disease cases as possible, even at the expense of potentially flagging some healthy patients (false positives) for further examination.
Why the other options are wrong
- A. F1-Score is the harmonic mean of precision and recall. While useful for balancing both, in this specific medical context, minimizing false negatives (maximizing recall) is paramount, even if it means a lower precision or F1-score.
- B. Precision measures the proportion of positive identifications that were actually correct (True Positives / (True Positives + False Positives)). High precision minimizes false positives, which is less critical than minimizing false negatives for a rare, serious disease.
- D. Accuracy measures the overall correctness of the model (True Positives + True Negatives) / Total. For imbalanced datasets, a high accuracy can be misleading if the model simply predicts the majority class (no disease) most of the time.
Recall (Sensitivity)
Recall, also known as Sensitivity, is an evaluation metric that measures the proportion of actual positive cases that are correctly identified by an AI model.
- Calculated as True Positives / (True Positives + False Negatives).
- Prioritized when minimizing false negatives is crucial (e.g., disease detection).
- High recall means fewer actual positive cases are missed.
Memory trick: Recall is like a wide net, trying to catch every single fish, even if some seaweed comes along.