AWS Certified Machine Learning – SpecialtyModelingEasy
A research team is training a deep neural network for medical image segmentation. They observe that the model performs exceptionally well on the training data, achieving very high Dice coefficients, but its performance drops significantly on unseen validation data. This indicates a strong sign of overfitting. To mitigate this, they decide to introduce a regularization technique that randomly sets a fraction of input units to zero at each update during training. Which regularization technique are they implementing?
- AL1 Regularization
- BBatch Normalization
- CDropout
- DL2 Regularization
Show answer & explanationAnswer & explanation
Correct answer: C. Dropout
Dropout is a regularization technique where randomly selected neurons are ignored during training. This forces the network to learn more robust features and prevents over-reliance on specific neurons, thereby reducing overfitting.
Why the other options are wrong
- A. L1 Regularization (Lasso) adds the absolute value of weights to the loss function, encouraging sparsity and feature selection, but it does not randomly drop units during training.
- B. Batch Normalization normalizes the inputs of each layer, stabilizing and speeding up training, but it is not a direct regularization technique for randomly deactivating neurons to prevent overfitting in the described manner.
- D. L2 Regularization (Weight Decay) adds the squared magnitude of weights to the loss function, penalizing large weights and preventing overfitting, but it does not randomly drop units during training.
Dropout
A regularization technique used in neural networks to prevent overfitting by randomly setting a fraction of neurons' outputs to zero during training.
- Forces the network to learn more robust features.
- Prevents co-adaptation of neurons.
- Effectively trains an ensemble of many thinned networks.
Memory trick: Dropout drops neurons, making the network think harder, like a puzzle with missing parts.