AWS Certified Solutions Architect – ProfessionalContinuously Improve Existing SolutionsMedium
A startup is building a new real-time analytics platform. Their existing data ingestion pipeline uses Amazon Kinesis Data Streams to collect high-volume, high-velocity streaming data. They need to perform continuous SQL queries on these fast-moving data streams to generate real-time dashboards and trigger alerts based on specific data patterns, without managing any underlying servers or infrastructure. The solution must be highly available and scalable to handle spikes in data volume. Which AWS service should a Solutions Architect recommend?
- AAmazon EMR with Apache Spark Streaming.
- BAmazon Redshift with streaming ingestion.
- CAmazon Kinesis Data Analytics for Apache Flink.
- DAWS Lambda with custom stream processing logic.
Show answer & explanationAnswer & explanation
Correct answer: C. Amazon Kinesis Data Analytics for Apache Flink.
Amazon Kinesis Data Analytics for Apache Flink is a fully managed service that allows you to process and analyze streaming data in real time using Apache Flink. It supports continuous SQL queries directly on streaming data, is serverless, highly available, and automatically scales to handle varying data volumes, making it ideal for real-time analytics and alerting without managing infrastructure.
Why the other options are wrong
- A. EMR with Spark Streaming can perform real-time processing, but it is not a fully serverless solution; it requires managing EMR clusters and their underlying EC2 instances, which goes against the 'without managing any underlying servers or infrastructure' requirement.
- B. Redshift is a data warehouse for batch analytics, not optimized for continuous, real-time SQL queries on fast-moving streams without significant latency. While it has streaming ingestion, it's not a real-time stream processing engine for continuous SQL.
- D. AWS Lambda can process Kinesis streams, but writing and maintaining custom stream processing logic in Lambda requires significant development effort, and managing complex stateful computations for continuous SQL queries can be challenging compared to a dedicated service like Kinesis Data Analytics.
Amazon Kinesis Data Analytics for Apache Flink
A fully managed service that allows you to process and analyze streaming data in real time using Apache Flink, supporting continuous SQL queries.
- Serverless and fully managed service.
- Supports continuous SQL queries or Apache Flink applications.
- Processes data from Kinesis Data Streams and Firehose.
- Used for real-time analytics, dashboards, and anomaly detection.
Memory trick: Kinesis Data Analytics is the 'Stream Detective', constantly watching and reporting on your data flow.