AWS Certified Data Engineer – AssociateData Storage and ManagementMedium
A data engineering team is building a real-time analytics pipeline for customer clickstream data. The data arrives in high volume and velocity and needs to be ingested, processed, and made available for querying within seconds. The solution must be fully managed and scale automatically to handle fluctuating traffic. Which AWS service is best suited for ingesting this continuous stream of data?
- AAWS Database Migration Service (DMS)
- BAWS Transfer Family
- CAmazon SQS
- DAmazon Kinesis Data Streams
Show answer & explanationAnswer & explanation
Correct answer: D. Amazon Kinesis Data Streams
Amazon Kinesis Data Streams is a fully managed, scalable service for real-time processing of large streams of data records. It can continuously capture and store GBs of data per second from hundreds of thousands of sources, making it ideal for high-volume, real-time ingestion.
Why the other options are wrong
- A. DMS is for migrating databases, not for real-time stream ingestion from applications.
- B. AWS Transfer Family is for file transfer over SFTP, FTPS, and FTP, not for real-time streaming data.
- C. Amazon SQS is a message queuing service, suitable for decoupling microservices, but Kinesis Data Streams is specifically designed for high-throughput, real-time data streaming and processing.
Amazon Kinesis Data Streams
Amazon Kinesis Data Streams is a fully managed and scalable service for real-time processing of streaming data. It can continuously capture gigabytes of data per second from hundreds of thousands of sources, making it ideal for operational monitoring, fraud detection, and live analytics.
- Real-time data ingestion
- High throughput and low latency
- Scalable and fully managed
- Data available for processing within seconds
Memory trick: Kinesis Keeps Klicking Kwickly.