A data scientist is working with a time-series dataset where observations are recorded hourly. Missing values are present in key sensor readings. For a predictive model, it's crucial to fill these missing values in a way that reflects the typical daily and weekly patterns, rather than just simple linear interpolation or forward-fill. The dataset spans several years, and the missing data can occur in short bursts or longer periods. Which imputation strategy is most appropriate for capturing these cyclical patterns?
- ALast Observation Carried Forward (LOCF).
- BSeasonal decomposition and imputation using a seasonal Kalman filter or seasonal average.
- CMean imputation across the entire dataset.
- DLinear interpolation between adjacent non-missing values.
Show answer & explanationAnswer & explanation
Correct answer: B. Seasonal decomposition and imputation using a seasonal Kalman filter or seasonal average.
Seasonal decomposition methods combined with imputation techniques like seasonal Kalman filters or simply using seasonal averages (e.g., average value for that specific hour on that specific day of the week) are highly effective for time-series data with strong cyclical patterns. These methods explicitly account for seasonality, ensuring that imputed values align with typical patterns rather than just local trends or overall averages.
Why the other options are wrong
- A. LOCF (Last Observation Carried Forward) fills missing values with the last observed value. While simple, it can create step-like patterns and fails to capture daily/weekly cycles, especially for longer missing periods.
- C. Mean imputation across the entire dataset ignores time-series characteristics and cyclical patterns, leading to inaccurate imputations.
- D. Linear interpolation fills missing values by drawing a straight line between two known points. It captures local trends but often fails to accurately represent strong, recurring cyclical patterns over longer gaps or across seasonal boundaries.
Seasonal Time-Series Imputation
Imputing missing values in time-series data by leveraging and preserving inherent seasonal or cyclical patterns (e.g., daily, weekly, monthly cycles).
- Accounts for recurring patterns in data.
- More sophisticated than simple statistical imputations.
- Methods include seasonal averages, seasonal Kalman filters, Prophet models.
- Crucial for accurate modeling of cyclical time series.
Memory trick: To fill gaps in cycles, decompose and impute for seasonal smiles.