AWS Certified Machine Learning – SpecialtyModelingEasy
A machine learning engineer is training a deep neural network for image classification. They observe that the model's training loss is decreasing steadily, but the validation loss has started to increase significantly after a certain number of epochs. The model is exhibiting high variance. Which of the following techniques is most appropriate to address this issue?
- AIncreasing the model's complexity by adding more layers.
- BDecreasing the regularization strength (e.g., lower L2 penalty).
- CIncreasing the size of the training dataset.
- DReducing the number of epochs for training.
Show answer & explanationAnswer & explanation
Correct answer: D. Reducing the number of epochs for training.
When training loss decreases while validation loss increases, it's a clear sign of overfitting (high variance). Reducing the number of epochs, often through early stopping, is a direct and effective way to prevent the model from learning noise in the training data and improve generalization to unseen data.
Why the other options are wrong
- A. Increasing model complexity would also exacerbate overfitting, making the model more likely to memorize the training data.
- B. Decreasing regularization strength would make the model more prone to overfitting, exacerbating the problem.
- C. Increasing training data size is generally good but the question implies the problem is ongoing during training, and early stopping is a direct fix for the observed behavior.
Early Stopping
A form of regularization used during model training to prevent overfitting by monitoring validation loss and stopping the training process when the validation loss starts to increase.
- Prevents models from memorizing training data noise.
- Saves computational resources.
- Commonly used in deep learning.
Memory trick: Stop Early, Stay Smart, Avoid Over-learning.