AWS Certified Solutions Architect – Associate (SAA-C03)Design High-Performing ArchitecturesMedium
A data engineering team is building a pipeline to ingest continuously flowing clickstream data from millions of users for real-time analytics. The data needs to be processed in order and persist for at least 24 hours for replayability. The solution must handle high throughput and be capable of integrating with other AWS services for further processing. Which AWS service is best suited for this task?
- AAmazon Kinesis Data Streams.
- BAmazon S3 Standard.
- CAmazon SQS (Simple Queue Service) Standard Queue.
- DAmazon DynamoDB.
Show answer & explanationAnswer & explanation
Correct answer: A. Amazon Kinesis Data Streams.
Amazon Kinesis Data Streams is designed for real-time processing of large, continuous streams of data. It ensures ordered delivery, offers data persistence for up to 365 days (default 24 hours), handles high throughput, and integrates seamlessly with other AWS services for consumption and analysis.
Why the other options are wrong
- B. S3 is object storage, suitable for batch processing or archival, but not for real-time, ordered ingestion and processing of continuous data streams.
- C. SQS Standard is a message queuing service that does not guarantee order and has a shorter retention period, making it less suitable for ordered real-time stream processing with replayability.
- D. DynamoDB is a NoSQL database, excellent for high-performance key-value access, but not primarily designed as an ingestion pipeline for continuous, ordered data streams for real-time analytics.
Amazon Kinesis Data Streams
Amazon Kinesis Data Streams is a massively scalable and durable real-time data streaming service that can continuously capture gigabytes of data per second from hundreds of thousands of sources.
- Real-time processing of streaming data.
- Ensures ordered delivery of records.
- Data persistence for 24 hours to 365 days (replayability).
- High throughput capabilities.
- Integrates with other AWS services for analytics.
Memory trick: Kinesis Streams are like a fast, orderly river for your data, always flowing and ready for analysis.