AWS Certified Machine Learning – SpecialtyMachine Learning Implementation and OperationsMedium
A retail company uses a machine learning model for dynamic pricing recommendations. The model's performance has recently degraded, leading to suboptimal pricing and reduced revenue. Upon investigation, the data science team found that the distribution of key input features, such as customer purchase history and product popularity trends, has significantly changed over the past month. Which MLOps practice should the team immediately implement to address this issue and prevent future occurrences?
- AOptimize the model's inference latency using SageMaker Neo.
- BRefactor the model training pipeline for faster execution.
- CSet up A/B testing for new model versions.
- DImplement SageMaker Model Monitor for data and model quality.
Show answer & explanationAnswer & explanation
Correct answer: D. Implement SageMaker Model Monitor for data and model quality.
The scenario describes data drift (change in input feature distribution), which is a key issue that SageMaker Model Monitor is designed to detect. Implementing Model Monitor would proactively identify such drifts and help prevent future performance degradation.
Why the other options are wrong
- A. Optimizing inference latency with SageMaker Neo addresses speed, not model performance degradation due to data drift.
- B. Refactoring the training pipeline might improve efficiency but won't detect or prevent data drift from occurring in the future.
- C. A/B testing compares model versions but doesn't directly address or detect the underlying data drift causing performance degradation.
SageMaker Model Monitor
A SageMaker capability that continuously monitors machine learning models in production for data drift, model quality degradation, and bias, providing alerts for timely intervention.
- Detects data drift (changes in feature distribution).
- Identifies model quality issues (e.g., accuracy drop).
- Supports bias detection and explainability monitoring.
Memory trick: Monitoring keeps your ML model healthy by catching drifts early.