CompTIA Data+ (DA0-002)Data MiningEasy
A data analyst is working with a sales dataset that includes a 'TransactionAmount' column. They notice that a few entries are exceptionally high, skewing statistical measures like the mean and standard deviation. These extreme values are legitimate but rare. To mitigate their impact on statistical analysis without removing them entirely, the analyst decides to cap these values at the 99th percentile. What is this data cleansing technique called?
- ANormalization
- BStandardization
- CWinsorization
- DBinning
Show answer & explanationAnswer & explanation
Correct answer: C. Winsorization
Winsorization is the technique of replacing extreme values (outliers) with values at a specified percentile (e.g., 99th percentile) or a certain number of standard deviations from the mean. This caps the influence of outliers without removing them, preserving the original number of observations.
Why the other options are wrong
- A. Normalization scales data to a specific range (e.g., 0-1) but doesn't cap outliers.
- B. Standardization transforms data to have a mean of 0 and standard deviation of 1, not capping outliers.
- D. Binning groups continuous data into discrete intervals or 'bins'.
Winsorization
A statistical technique used to limit the influence of extreme values (outliers) in a dataset by replacing them with values at a specified percentile (e.g., 95th or 99th percentile) or a certain number of standard deviations from the mean.
- Differs from trimming, which removes outliers entirely.
- Preserves the original sample size.
- Useful when outliers are legitimate data points but disproportionately affect statistical measures.
Memory trick: Treating outliers is like managing a wild garden: Winsorization prunes, not uproots.