AWS Certified Machine Learning – SpecialtyMachine Learning Implementation and OperationsMedium
A retail company uses a machine learning model to recommend products to customers. The model was initially trained on historical purchase data. Over time, the company observes a decline in recommendation accuracy, despite no changes in the underlying customer behavior or product catalog. This suggests that the relationship between features and targets might have changed. Which model monitoring technique should be employed to detect this specific issue and trigger retraining?
- AFeature attribution monitoring
- BModel bias monitoring
- CData quality monitoring
- DConcept drift detection
Show answer & explanationAnswer & explanation
Correct answer: D. Concept drift detection
Concept drift occurs when the relationship between input features and target variable changes over time, leading to degraded model performance even if input data distribution remains stable. Detecting this requires specialized monitoring.
Why the other options are wrong
- A. Feature attribution monitoring explains model predictions, but doesn't directly detect changes in the underlying data relationships.
- B. Model bias monitoring checks for fairness issues across different groups, not general performance degradation due to relationship changes.
- C. Data quality monitoring checks for issues like missing values or incorrect formats, not changes in feature-target relationships.
Concept Drift
A phenomenon where the statistical properties of the target variable, which the model is trying to predict, change over time in unforeseen ways.
- Leads to model performance degradation.
- Can occur even if input data distribution remains stable.
- Requires model retraining to adapt to new patterns.
Memory trick: Drifting concepts make models lose their grip on reality.