AWS Certified Machine Learning – SpecialtyData EngineeringMedium
A data scientist needs to prepare a dataset for a time-series forecasting model. The dataset contains sensor readings collected at irregular intervals, resulting in missing values. To ensure the model can be trained effectively, the data scientist needs to fill these missing values using a method that considers the temporal order and trends in the data. Which data preparation technique is most appropriate for handling missing values in this time-series context?
- AMode imputation
- BDeletion of rows
- CForward fill (LOCF)
- DMean imputation
Show answer & explanationAnswer & explanation
Correct answer: C. Forward fill (LOCF)
Forward fill (Last Observation Carried Forward - LOCF) is a common and often appropriate technique for time-series data. It imputes missing values by carrying forward the last observed valid value. This method respects the temporal order of the data, which is crucial for time-series analysis, and can be more suitable than methods that ignore the sequence, like mean or mode imputation.
Why the other options are wrong
- A. Mode imputation replaces missing values with the most frequent value, which is generally unsuitable for continuous time-series data and ignores temporal relationships.
- B. Deletion of rows with missing values (listwise deletion) can lead to significant data loss, especially in time-series data with many missing points, potentially reducing the dataset's size and representativeness.
- D. Mean imputation replaces missing values with the mean of the column, which disrupts the temporal patterns and can significantly bias time-series data.
Time-Series Imputation (Forward Fill)
A technique for handling missing values in time-series data by carrying forward the last valid observation to fill subsequent missing data points.
- Preserves the temporal order of data.
- Suitable for data collected at irregular intervals.
- Simple to implement and computationally efficient.
- Can be extended with more sophisticated methods like interpolation.
- Avoids introducing future information into past data (unlike backward fill).
Memory trick: For time series, just carry the last value forward, it's logical.