AWS Certified Data Engineer – AssociateData Storage and ManagementEasy
A data engineering team is building a data pipeline that ingests continuous streams of clickstream data from a website. This data needs to be processed in real-time for immediate analytics and then stored for batch processing later. The solution must be able to handle fluctuating data volumes, scale automatically, and deliver data to multiple consumers while ensuring data durability. Which AWS service is best suited for ingesting and temporarily storing this real-time streaming data?
- AAmazon DynamoDB
- BAmazon S3
- CAmazon Kinesis Data Streams
- DAmazon SQS
Show answer & explanationAnswer & explanation
Correct answer: C. Amazon Kinesis Data Streams
Amazon Kinesis Data Streams is a highly scalable and durable real-time data streaming service. It is designed to continuously capture and store gigabytes per second of data, making it ideal for ingesting clickstream data, handling fluctuating volumes, and delivering data to multiple real-time and batch consumers.
Why the other options are wrong
- A. Amazon DynamoDB is a NoSQL database, suitable for storing transactional data but not designed for continuous, high-throughput streaming data ingestion and fan-out to multiple analytics services.
- B. Amazon S3 is object storage, suitable for storing large volumes of data after ingestion, but not for real-time ingestion and continuous streaming to multiple consumers.
- D. Amazon SQS (Simple Queue Service) is a message queuing service, not optimized for high-throughput, continuous streaming of data to multiple consumers for real-time analytics.
Amazon Kinesis Data Streams
A real-time data streaming service that can continuously capture and store large streams of data records for real-time analytics and subsequent batch processing.
- Real-time data ingestion
- Highly scalable and durable
- Supports multiple consumers (fan-out)
- Ideal for clickstream, IoT, log data
- Data retention up to 365 days
Memory trick: Kinesis is the 'Kinetic' flow for your streaming data.