AWS Certified Data Engineer – AssociateData Storage and ManagementMedium
A data engineering team is designing a data lake on AWS. They need to store streaming data from various IoT devices for real-time analytics and subsequent batch processing. The solution must be fully managed, scalable, and capable of delivering data to Amazon S3, Amazon Redshift, and Amazon OpenSearch Service without requiring custom application development. Which AWS service is best suited for this requirement?
- AAWS IoT Core
- BAmazon Kinesis Data Firehose
- CAmazon Kinesis Data Streams
- DAmazon SQS (Simple Queue Service)
Show answer & explanationAnswer & explanation
Correct answer: B. Amazon Kinesis Data Firehose
Amazon Kinesis Data Firehose is a fully managed service for delivering real-time streaming data to various destinations like S3, Redshift, and OpenSearch Service. It handles scaling and data delivery without requiring custom application development, perfectly matching the requirements.
Why the other options are wrong
- A. AWS IoT Core is for connecting IoT devices and managing their interactions, not primarily for delivering streaming data to analytics destinations.
- C. Amazon Kinesis Data Streams is for building custom applications to process streaming data, which is not required here.
- D. Amazon SQS is a message queuing service, not optimized for continuous streaming data ingestion and delivery to analytics services.
Amazon Kinesis Data Firehose
A fully managed service for delivering real-time streaming data to data lakes, data stores, and analytics services.
- Automatically scales to match data throughput.
- Supports various destinations like Amazon S3, Amazon Redshift, Amazon OpenSearch Service, and Splunk.
- Requires no administrative overhead for data delivery.
Memory trick: Firehose funnels data fast to your analytics destinations.