AWS Certified AI PractitionerAI/ML and Generative AI FundamentalsMedium
A machine learning engineer is developing a real-time anomaly detection system for network intrusion. The system needs to identify unusual patterns in network traffic without prior examples of what an 'intrusion' looks like. The engineer has access to a large dataset of normal network behavior. Which type of machine learning approach is most suitable for this task?
- AUnsupervised Learning (Clustering)
- BSupervised Learning (Classification)
- CSupervised Learning (Regression)
- DReinforcement Learning
Show answer & explanationAnswer & explanation
Correct answer: A. Unsupervised Learning (Clustering)
The problem states that the system needs to identify 'unusual patterns... without prior examples of what an 'intrusion' looks like' and has 'a large dataset of normal network behavior'. This is a classic scenario for unsupervised learning, specifically clustering or density-based methods, where the model learns the structure of 'normal' data and flags deviations as anomalies.
Why the other options are wrong
- B. Supervised Classification requires labeled examples of both 'normal' and 'intrusion' data during training, which is explicitly stated as unavailable for intrusions.
- C. Supervised Regression is used to predict continuous numerical values based on labeled input-output pairs, which is not the goal here.
- D. Reinforcement Learning is used for decision-making in dynamic environments through trial and error, not typically for identifying patterns in static data for anomaly detection.
Unsupervised Learning (Anomaly Detection)
A machine learning approach used to identify rare items, events, or observations that deviate significantly from the majority of the data, without requiring labeled examples of anomalies.
- Learns patterns from unlabeled data.
- Identifies outliers or deviations from learned normal behavior.
- Common algorithms include K-Means, Isolation Forest, One-Class SVM.
- Useful when anomalies are rare or undefined.
Memory trick: Learning types: teacher, no teacher, or rewards.