AWS Certified Machine Learning – Specialty flashcards

Techniques like PCA that reduce the number of features in a dataset, which is particularly useful for mitigating overfitting when the number of features significantly exceeds the number of samples.

• Reduces computational cost.
• Helps combat the curse of dimensionality.
• Can improve model generalization by removing noise.

Memory trick: Reduce Dimensions, Retain Wisdom, Avoid Over-Fit.

A fairness metric that requires the true positive rate (recall) to be equal across different protected groups.

• Focuses on ensuring that the model is equally effective at identifying positive outcomes for all groups.
• Relevant when the cost of false negatives is high and fairness in 'being served' is paramount.
• Aims to prevent discrimination in access to benefits or detection of conditions.

Memory trick: Equal Opportunity means equal chances for true positives.

A non-parametric measure of the strength and direction of the monotonic relationship between two ranked variables. It assesses how well the relationship between two variables can be described using a monotonic function.

• Measures correlation between ranks, not raw values.
• Useful when the exact magnitude of prediction is less important than relative order.
• Ranges from -1 to +1, where +1 indicates a perfect monotonic relationship.

Memory trick: When the goal is ranking, Spearman's the one, predicting who's first, when the race is run.

A type of neural network designed to process sequential data by maintaining an internal state (memory) that captures information from previous elements in the sequence.

• Suitable for time series, natural language, and speech.
• Captures temporal dependencies.
• Suffers from vanishing/exploding gradients in vanilla form, often mitigated by LSTMs/GRUs.

Memory trick: RNNs remember the past, like a flowing river, each drop influencing the next.

Hybrid recommendation systems combine multiple recommendation approaches (e.g., collaborative filtering, content-based filtering) to leverage their respective strengths and mitigate weaknesses, such as the cold start problem.

• Address cold start by using content-based features for new users/items.
• Can improve overall recommendation quality and diversity.
• Common combination: matrix factorization (collaborative) + content-based.
• Different integration strategies: weighted, switching, mixed, or cascade.

Memory trick: For 'NEWBIES', don't be cold, 'MIX' your methods to warm them up!

Robustness testing evaluates how well a machine learning model performs when its input data is subjected to small, often imperceptible, perturbations or noise. Adversarial examples are inputs specifically crafted to cause a model to make an incorrect prediction with high confidence.

• Measures model's resilience to input variations and attacks.
• Crucial for safety-critical applications (e.g., autonomous driving, medical).
• Involves generating adversarial examples or injecting noise.
• Helps identify vulnerabilities and improve model reliability.

Memory trick: To check if your model is 'TOUGH', hit it with 'SMALL CHANGES' and see if it breaks.

LIME is an interpretability technique that explains the predictions of any black-box machine learning model by approximating it locally around the prediction with an interpretable model (e.g., linear model or decision tree).

• Model-agnostic: works with any ML model.
• Local interpretability: explains individual predictions, not the whole model.
• Provides human-understandable explanations (e.g., feature importance for a single instance).
• Helps build trust in complex models and debug their behavior.

Memory trick: To 'UNDERSTAND' your model, look at its 'GLOBAL' impact or 'LOCAL' decisions.

The dilemma in supervised learning where reducing one type of error (bias or variance) often increases the other.

• High bias (underfitting) means the model is too simple.
• High variance (overfitting) means the model is too complex.
• The goal is to find a balance for optimal generalization.

Memory trick: Simple models are biased, complex models are varied.

A data transformation technique (e.g., natural log or base-10 log) applied to numerical features to reduce skewness and potentially linearize non-linear relationships, making the data more suitable for models that assume linearity or normality.

• Effective for positively skewed data.
• Can stabilize variance.
• Helps linear models capture non-linear patterns.

Memory trick: Log It Down to Straighten the Line.

Synthetic Minority Over-sampling Technique, an oversampling method that generates synthetic samples for the minority class by interpolating between existing minority class instances and their nearest neighbors.

• Addresses class imbalance by increasing minority class size.
• Creates synthetic, not duplicate, samples.
• Helps prevent overfitting compared to simple oversampling.

Memory trick: SMOTE's the antidote, for tiny classes, it's the vote, creating new friends, so the model can quote.

An advanced ensemble technique where multiple base models are trained, and their predictions are then used as input features to a 'meta-learner' model, which learns to optimally combine these predictions to make the final output.

• Combines diverse models effectively.
• Meta-learner learns complex combination rules.
• Often yields higher performance than simple ensembles.
• Involves training multiple layers of models.

Memory trick: Stack the Models, Learn the Best Combine.

Model quantization is a technique to reduce the precision of numbers used to represent a model's parameters and computations, typically from floating-point to lower-bit integers.

• Reduces model size significantly.
• Speeds up inference time due to simpler arithmetic operations.
• Can be applied during training (quantization-aware training) or post-training.
• May lead to a slight drop in accuracy, which needs to be balanced against performance gains.

Memory trick: Optimize models to make them 'LIGHT, FAST, and SMART' for deployment.

Popularity bias in recommendation systems refers to the tendency of models to recommend disproportionately popular items, often overlooking niche or 'long-tail' items. Long-tail coverage is a metric that quantifies the percentage of items that are in the 'long tail' (less popular) that are actually recommended by the system.

• Common in collaborative filtering due to abundant data for popular items.
• Leads to lack of diversity and serendipity in recommendations.
• Metrics like Gini coefficient, catalog coverage, and long-tail coverage help detect it.
• Can be mitigated by re-ranking, boosting niche items, or hybrid models.

Memory trick: To see if your recommender is 'FAIR', check its 'DISTRIBUTION' and 'REACH'.

A form of regularization used during model training to prevent overfitting by monitoring validation loss and stopping the training process when the validation loss starts to increase.

• Prevents models from memorizing training data noise.
• Saves computational resources.
• Commonly used in deep learning.

Memory trick: Stop Early, Stay Smart, Avoid Over-learning.

Bayesian Optimization is a global optimization strategy for objective functions that are expensive to evaluate, often used for hyperparameter tuning. It uses a probabilistic model (surrogate model) to guide the search for optimal parameters.

• Builds a surrogate model (e.g., Gaussian Process) of the objective function.
• Uses an acquisition function to decide the next point to evaluate.
• Highly efficient for expensive functions and finding global optima with fewer evaluations.

Memory trick: To tune with 'Bayesian' smarts, you 'learn' from the 'past' to pick the 'best paths'.

Inputs to a machine learning model that an attacker has intentionally designed to cause the model to make a mistake, often by adding small, imperceptible perturbations.

• Used to evaluate and improve model robustness.
• Highlight model vulnerabilities and blind spots.
• Can be generated using various attack methods (e.g., FGSM, PGD).

Memory trick: Adversarial examples are tricksters, finding the model's weak spots like digital picksters.

Prioritizing the Recall metric in anomaly detection when the cost of missing an actual anomaly (false negative) is significantly higher than the cost of a false alarm (false positive).

• Minimizes False Negatives.
• Ensures critical events are not missed.
• Crucial in safety-critical applications (e.g., medical diagnosis, industrial failure).

Memory trick: Recall Every Anomaly, Avoid Catastrophe.

A hyperparameter optimization technique that samples hyperparameter combinations from specified distributions randomly, often proving more efficient than grid search for finding good parameter sets, particularly when some hyperparameters are more important than others.

• More efficient than Grid Search for high-dimensional spaces.
• Likely to find good parameters with fewer evaluations.
• Requires defining distributions for each hyperparameter.

Memory trick: Randomly Search, Efficiently Reach.

Evaluation of a model's performance under various real-world environmental conditions, including challenging and rare scenarios (e.g., adverse weather, different lighting, occlusions).

• Crucial for safety-critical applications like autonomous driving.
• Identifies performance degradation in 'edge cases' or non-ideal conditions.
• Often involves collecting or simulating diverse and challenging test data.

Memory trick: Robust models endure diverse, challenging conditions.

A standardized way of displaying the distribution of data based on a five-number summary: minimum, first quartile (Q1), median (Q2), third quartile (Q3), and maximum. It also identifies outliers.

• Visualizes distribution of a continuous variable.
• Effective for comparing distributions across different groups (categorical variable).
• Shows median, interquartile range (IQR), and potential outliers.
• Useful for quickly identifying differences in central tendency and spread between groups.

Memory trick: Continuous Age across Categories? Box Plot's the sage!

The process of transforming data from various inconsistent formats into a single, uniform, and consistent structure to ensure data quality and usability.

• Crucial for data integration from multiple sources.
• Ensures accurate comparisons and aggregations.
• Prevents errors in downstream analysis and model training.

Memory trick: Standardize: Make it all 'standard', like school uniforms.

A data transformation technique that replaces each data point with its logarithm. It is commonly used to reduce skewness and stabilize variance in positively skewed distributions.

• Effective for highly right-skewed data (e.g., income, house prices).
• Compresses large values more than small values.
• Helps achieve a more symmetrical, normal-like distribution.
• Requires all data points to be positive; add a constant if zeros or negatives exist.

Memory trick: Skewed income? Log it to make it look normal!

Techniques for identifying unusual data points in time-series data that exhibit repeating patterns (seasonality), often by decomposing the series into trend, seasonal, and residual components.

• Requires methods to account for seasonality to avoid false positives.
• Decomposition (e.g., STL) separates components for independent analysis.
• Anomaly detection is often most effective on the 'residual' component after removing trend and seasonality.

Memory trick: STL: 'Separate The Layers' for clear anomaly sight.

ANOVA is a statistical test used to compare the means of three or more groups to determine if there is a statistically significant difference between them.

• Compares means of a numerical variable across multiple categories.
• Requires the numerical variable to be approximately normally distributed within each group.
• Assumes homogeneity of variances (equal variances) across groups.

Memory trick: ANOVA: A Numerical variable Over Various Averages.