AWS Certified Machine Learning – SpecialtyModelingMedium
A machine learning engineer is developing a real-time recommendation system. The initial model, trained on historical user data, performs well offline but shows degraded performance and slow response times when deployed in production. The engineer suspects the model is too complex for the inference environment. Which model optimization technique is most appropriate to address both performance and latency issues while maintaining reasonable accuracy?
- AUtilizing a larger training dataset with more features.
- BImplementing a more sophisticated hyperparameter tuning strategy like Bayesian optimization.
- CIncreasing the number of layers and neurons in the neural network.
- DApplying model quantization to reduce model size and computational requirements.
Show answer & explanationAnswer & explanation
Correct answer: D. Applying model quantization to reduce model size and computational requirements.
Model quantization reduces the precision of the numerical representations of weights and activations, leading to a smaller model size and faster inference. This directly addresses the issues of degraded performance and slow response times in a production environment due to model complexity.
Why the other options are wrong
- A. A larger training dataset and more features would likely increase model complexity and training time, and potentially inference time, not reduce it.
- B. Hyperparameter tuning optimizes model performance but does not directly address model size or inference latency due to complexity in the way quantization does.
- C. Increasing model complexity (more layers/neurons) would exacerbate the performance and latency issues, not solve them.
Model Quantization
Model quantization is a technique to reduce the precision of numbers used to represent a model's parameters and computations, typically from floating-point to lower-bit integers.
- Reduces model size significantly.
- Speeds up inference time due to simpler arithmetic operations.
- Can be applied during training (quantization-aware training) or post-training.
- May lead to a slight drop in accuracy, which needs to be balanced against performance gains.
Memory trick: Optimize models to make them 'LIGHT, FAST, and SMART' for deployment.