AWS Certified Cloud Practitioner (CLF-C02)Cloud Technology and ServicesHard
A data analytics company needs to process large streams of customer clickstream data in real-time. They require a highly scalable and durable messaging queue that can handle millions of messages per second, ensuring that no messages are lost and that messages are processed in the order they were received for each user session. Which AWS service is best suited for this use case?
- AAmazon SNS
- BAmazon SQS Standard Queue
- CAmazon Kinesis Data Streams
- DAmazon SQS FIFO Queue
Show answer & explanationAnswer & explanation
Correct answer: C. Amazon Kinesis Data Streams
Amazon Kinesis Data Streams is designed for real-time processing of large streams of data. It can handle millions of messages per second, ensures durability, and maintains the order of messages within a shard, which is critical for clickstream data where session order matters. SQS FIFO provides ordering but is capped at 3000 messages/second with batching, which is insufficient for 'millions of messages per second'.
Why the other options are wrong
- A. Amazon SNS is a pub/sub messaging service, primarily for notifications, and is not designed for streaming large volumes of ordered data for real-time processing.
- B. Amazon SQS Standard Queue does not guarantee message ordering or 'exactly-once' delivery, which is required for clickstream data where order and no loss are critical.
- D. Amazon SQS FIFO Queue guarantees ordering and exactly-once processing, but its throughput limit (3,000 messages/second with batching) is too low for 'millions of messages per second'.
Amazon Kinesis Data Streams
A real-time data streaming service capable of continuously capturing and storing gigabytes of data per second from hundreds of thousands of sources, ensuring durability and order within shards.
- Scales to millions of messages per second.
- Guarantees order of records within a shard.
- Data is available for processing within milliseconds.
Memory trick: Kinesis: Keep streams moving, ordered, and fast.