AWS Certified Machine Learning – SpecialtyModelingMedium
A data scientist is training a deep learning model for medical diagnosis. The model achieves high accuracy on the training set, but its performance on the validation set is significantly lower, indicating overfitting. To address this, the data scientist decides to add a penalty term to the loss function that is proportional to the sum of the absolute values of the model's weights. Which regularization technique is being applied?
- AL1 Regularization
- BL2 Regularization
- CDropout
- DEarly Stopping
Show answer & explanationAnswer & explanation
Correct answer: A. L1 Regularization
L1 Regularization (Lasso) adds a penalty term proportional to the absolute value of the weights. This encourages sparsity in the weights, effectively performing feature selection by driving some weights to exactly zero, thus reducing model complexity and mitigating overfitting.
Why the other options are wrong
- B. L2 Regularization (Ridge) adds a penalty proportional to the square of the weights, encouraging smaller but non-zero weights, which differs from the 'sum of absolute values' described.
- C. Dropout randomly deactivates neurons during training, which is different from adding a penalty term to the loss function based on weight magnitudes.
- D. Early Stopping halts training when validation performance degrades, but it does not add a penalty term to the loss function itself.
L1 Regularization (Lasso)
A regularization technique that adds a penalty equal to the sum of the absolute values of the model's weights to the loss function during training.
- Encourages sparsity by driving some weights to zero.
- Performs automatic feature selection.
- Reduces model complexity and overfitting.
Memory trick: L1's the one, absolute sum, making weights disappear, the sparse model's won.