AWS Certified AI PractitionerFoundation ModelsEasy
An AI research lab is developing a new foundation model with billions of parameters. During the training phase, they observe that the model's performance on the training dataset is exceptionally high, but its performance on unseen validation data is significantly lower. This indicates a problem where the model has learned the training data too well, including its noise and specific patterns, rather than general principles. What is this phenomenon called?
- AOverfitting.
- BBias-variance trade-off.
- CUnderfitting.
- DGradient vanishing.
Show answer & explanationAnswer & explanation
Correct answer: A. Overfitting.
Overfitting occurs when a model learns the training data too precisely, including noise and outliers, leading to poor generalization on new, unseen data. This is a common challenge with large models like foundation models due to their high capacity.
Why the other options are wrong
- B. The bias-variance trade-off is a concept that explains the relationship between model complexity, bias, and variance, but it's not the specific phenomenon observed here.
- C. Underfitting is when a model is too simple to capture the underlying patterns in the data, performing poorly on both training and validation sets.
- D. Gradient vanishing is a problem in training deep neural networks where gradients become too small, preventing effective weight updates, not a generalization issue directly.
Overfitting
Overfitting occurs when a machine learning model learns the training data too well, including its noise and specific details, leading to poor performance on new, unseen data.
- High training accuracy, low validation/test accuracy.
- Model is too complex for the data.
- Common in models with many parameters, like FMs.
Memory trick: Models can be too specific or too general.