A data engineering team is building a new feature store to manage features for various machine learning models across different projects. They require a solution that can serve features with very low latency (milliseconds) for real-time inference and also provide historical feature values for model training and batch inference. The solution must support point-in-time correctness for reproducible training and backtesting. Which AWS service combination offers these capabilities?
- AAWS Glue Data Catalog for metadata and Amazon Athena for querying.
- BAmazon DynamoDB for online serving and Amazon S3 for offline storage.
- CAmazon Redshift for offline storage and Amazon ElastiCache for online serving.
- DAmazon SageMaker Feature Store with online and offline stores enabled.
Show answer & explanationAnswer & explanation
Correct answer: D. Amazon SageMaker Feature Store with online and offline stores enabled.
Amazon SageMaker Feature Store is a purpose-built service for managing ML features. It inherently provides both an online store for low-latency real-time inference and an offline store for historical data used in training and batch inference. Crucially, it supports point-in-time correctness, ensuring that features used for training accurately reflect the state at a specific historical moment, which is vital for reproducible ML.
Why the other options are wrong
- A. AWS Glue Data Catalog and Athena are for data discovery and querying, not for low-latency online serving or point-in-time feature management in a feature store context.
- B. While DynamoDB and S3 can be used, this is a custom solution. SageMaker Feature Store provides a managed, integrated service specifically for ML features, including point-in-time correctness, which is harder to implement manually.
- C. Redshift is a data warehouse, and ElastiCache is an in-memory cache; neither is purpose-built as a feature store nor offers integrated point-in-time correctness for ML features.
Amazon SageMaker Feature Store
A specialized repository for machine learning features that provides both an online store for low-latency serving and an offline store for training and batch inference.
- Ensures feature consistency between training and inference
- Supports point-in-time correctness for historical data
- Reduces feature engineering duplication
- Manages feature versions and lineage
Memory trick: Store features once, serve them fast, track them always.