AWS Certified AI PractitionerFoundation ModelsMedium
A startup is developing an application that uses a foundation model to summarize long technical documents. After initial testing, users report that the summaries occasionally contain plausible-sounding but factually incorrect information not present in the original document. Which term best describes this phenomenon?
- AHallucination
- BOverfitting
- CUnderfitting
- DCatastrophic Forgetting
Show answer & explanationAnswer & explanation
Correct answer: A. Hallucination
The phenomenon where a foundation model generates plausible-sounding but factually incorrect or fabricated information that is not supported by its source data or training is known as 'hallucination'. This is a common challenge with generative AI models, especially Large Language Models, and is a significant concern for applications requiring high factual accuracy.
Why the other options are wrong
- B. Overfitting occurs when a model learns training data too well and performs poorly on new data, not necessarily generating fabricated facts.
- C. Underfitting occurs when a model is too simple to capture the underlying pattern of the data, leading to poor performance.
- D. Catastrophic Forgetting is when a neural network forgets previously learned information upon learning new information, which is not the issue described.
LLM Hallucination
The phenomenon where a Large Language Model (or other generative AI) generates plausible-sounding but factually incorrect, fabricated, or unsupported information.
- Outputs are often confident and fluent, despite being false.
- Can be difficult to detect without external verification.
- A major challenge in applications requiring high factual accuracy.
Memory trick: When AI speaks, but facts are dreams.