AWS Certified Data Engineer – AssociateData Ingestion and TransformationMedium
A global e-commerce company needs to track user behavior on its website and mobile applications. The data consists of page views, clicks, and search queries, generated by millions of users worldwide. The total data volume is expected to be several terabytes per day, with high variability in traffic. This data needs to be delivered reliably and cost-effectively to Amazon S3 for long-term storage and to Amazon Redshift for analytical dashboards. The solution should minimize operational overhead and handle automatic scaling. Which AWS service is most appropriate for ingesting this data?
- AAWS DataSync
- BAmazon Kinesis Data Streams
- CAmazon Kinesis Data Firehose
- DAWS Database Migration Service (DMS)
Show answer & explanationAnswer & explanation
Correct answer: C. Amazon Kinesis Data Firehose
Amazon Kinesis Data Firehose is a fully managed service for delivering real-time streaming data to various destinations, including Amazon S3 and Amazon Redshift. It automatically scales, batches, compresses, and encrypts data, minimizing operational overhead and ensuring reliable delivery for high-volume, variable traffic.
Why the other options are wrong
- A. AWS DataSync is for large-scale data transfers between on-premises storage and AWS, not for real-time streaming data ingestion from applications.
- B. Kinesis Data Streams provides more granular control and supports custom consumers, but Firehose is simpler and more cost-effective for direct delivery to S3 and Redshift when complex real-time processing by custom applications is not the primary need.
- D. AWS DMS is for migrating databases, not for ingesting real-time user behavior data from applications.
Amazon Kinesis Data Firehose
A fully managed service for delivering real-time streaming data to destinations like Amazon S3, Amazon Redshift, Amazon OpenSearch Service, and Splunk, without managing servers.
- Automatically scales to match data throughput.
- Supports data transformation and conversion.
- Integrates with various AWS services for destinations.
Memory trick: Firehose funnels data, fast and free.