AWS Certified Machine Learning – SpecialtyModelingEasy
A machine learning engineer is training a deep learning model for image classification. During training, the model's accuracy on the training set steadily increases, reaching nearly 100%, but its accuracy on a separate validation set plateaus and then starts to decrease. The engineer suspects overfitting. Which of the following techniques would be most effective in mitigating this issue?
- ADecrease the batch size to provide more frequent updates.
- BAdd more layers to the neural network architecture.
- CIncrease the learning rate to speed up convergence.
- DImplement Dropout layers during training.
Show answer & explanationAnswer & explanation
Correct answer: D. Implement Dropout layers during training.
Overfitting occurs when a model learns the training data too well, including noise, leading to poor generalization. Dropout is a regularization technique that randomly deactivates neurons during training, preventing complex co-adaptations and encouraging robustness.
Why the other options are wrong
- A. Decreasing batch size can sometimes help with generalization but is not as direct and effective for overfitting as regularization techniques like Dropout.
- B. Adding more layers increases model complexity, which would likely exacerbate overfitting.
- C. Increasing the learning rate might cause the model to diverge or overshoot the optimal solution, potentially worsening overfitting or training instability.
Dropout
A regularization technique for neural networks that randomly sets a fraction of neuron outputs to zero at each update during training.
- Prevents complex co-adaptations between neurons.
- Acts as an ensemble of many thinned networks.
- Applied only during training, not inference.
Memory trick: Overfit models are tangled; cut connections with Dropout.