AWS Certified Data Engineer – AssociateData Ingestion and TransformationEasy
A large manufacturing company uses thousands of IoT sensors on its factory floor. These sensors generate telemetry data (temperature, pressure, vibration) at high frequency, sending small JSON messages. The company needs to ingest this data, apply basic filtering (e.g., discard data below a certain temperature threshold), and then route the filtered data to multiple destinations: an Amazon Kinesis Data Stream for real-time analytics and an Amazon S3 bucket for long-term archival. The solution must be fully managed and scalable.
- AAWS IoT Core with IoT Rules Engine
- BAmazon Kinesis Data Firehose with Lambda transformation
- CAWS Glue Streaming ETL
- DAmazon Kinesis Data Streams with AWS Lambda
Show answer & explanationAnswer & explanation
Correct answer: A. AWS IoT Core with IoT Rules Engine
AWS IoT Core is designed for connecting and managing IoT devices. Its IoT Rules Engine allows for real-time processing, filtering, and routing of messages from IoT devices to multiple AWS services, including Kinesis Data Streams and S3, without managing servers.
Why the other options are wrong
- B. Kinesis Data Firehose is for streaming data ingestion and delivery but isn't the primary service for IoT device connectivity and real-time rule-based routing from devices themselves. Lambda transformation would add complexity for simple filtering.
- C. AWS Glue Streaming ETL is for complex real-time ETL on data streams, typically after ingestion into Kinesis or Kafka, not for direct ingestion and simple filtering from IoT devices.
- D. Kinesis Data Streams can receive data, and Lambda can process it, but AWS IoT Core provides a more integrated and managed solution specifically for the IoT device ecosystem, including device management and security.
AWS IoT Core Rules Engine
A feature of AWS IoT Core that allows users to define rules that process and route messages from IoT devices to other AWS services based on message content, without needing to provision or manage servers.
- Enables filtering, transforming, and routing IoT messages.
- Supports SQL-like syntax for rule definition.
- Integrates with numerous AWS services as destinations (S3, Kinesis, Lambda, etc.).
Memory trick: IoT Core takes device messages, Rules Engine filters and routes them perfectly.