AWS Certified Data Engineer – AssociateData Ingestion and TransformationMedium
A large manufacturing company uses thousands of IoT sensors on its factory floor. These sensors generate telemetry data (temperature, pressure, vibration) at a high frequency (tens of thousands of messages per second), each message being small (a few hundred bytes). The company needs to ingest this data, apply simple filtering rules (e.g., discard messages below a certain temperature threshold), and then route the filtered data to Amazon S3 for archival and Amazon DynamoDB for real-time dashboards. Which AWS service is best suited for this real-time ingestion and rule-based routing?
- AAmazon Kinesis Data Firehose
- BAWS IoT Core with IoT Rules Engine
- CAmazon Kinesis Data Streams
- DAWS Glue Streaming ETL
Show answer & explanationAnswer & explanation
Correct answer: B. AWS IoT Core with IoT Rules Engine
AWS IoT Core provides secure, scalable two-way communication for IoT devices. Its IoT Rules Engine allows you to filter, transform, and route messages from connected devices to various AWS services, such as S3 and DynamoDB, based on defined rules, making it perfect for this scenario.
Why the other options are wrong
- A. Kinesis Data Firehose can ingest streaming data to S3 and DynamoDB, but it doesn't offer the native device connectivity and rule-based filtering specific to IoT Core for direct device integration.
- C. Kinesis Data Streams is a generic real-time streaming service, but it lacks the direct device connectivity and built-in rules engine capabilities tailored for IoT devices that IoT Core provides.
- D. AWS Glue Streaming ETL can process streaming data but is typically used for more complex transformations on larger streams, not for simple rule-based routing directly from IoT devices.
AWS IoT Core + IoT Rules Engine
AWS IoT Core enables secure device connection and management, while the IoT Rules Engine processes and routes messages from connected devices to other AWS services based on defined rules.
- IoT Core: Connects billions of IoT devices securely.
- IoT Rules Engine: Filters, transforms, and routes device data.
- Supports various AWS service integrations (S3, DynamoDB, Lambda, Kinesis, etc.).
Memory trick: IoT devices connect, Rules Engine directs.