AWS Certified Data Engineer – AssociateData Ingestion and TransformationHard
A global e-commerce company wants to analyze customer clickstream data in near real-time. The data is generated from their website and mobile applications, with peak volumes reaching several terabytes per hour. They need to enrich this raw data by joining it with customer profile information from a DynamoDB table and then store the enriched data in Amazon S3 for further analytics. Which AWS service is best suited for performing this near real-time data enrichment and transformation?
- AAWS Glue ETL
- BAmazon Kinesis Data Analytics for Apache Flink
- CAWS Step Functions
- DAmazon EMR with Apache Spark
Show answer & explanationAnswer & explanation
Correct answer: B. Amazon Kinesis Data Analytics for Apache Flink
Amazon Kinesis Data Analytics for Apache Flink is specifically designed for processing and analyzing streaming data in real-time. It can handle high throughput, perform complex transformations like joining streams with static data (DynamoDB), and deliver results to S3, perfectly matching the requirements for near real-time clickstream enrichment.
Why the other options are wrong
- A. AWS Glue ETL is primarily a batch ETL service, not optimized for continuous, near real-time streaming enrichment at terabytes per hour.
- C. AWS Step Functions is a serverless workflow orchestrator, not a data processing or transformation engine for streaming data.
- D. Amazon EMR with Apache Spark can process streaming data, but Kinesis Data Analytics for Apache Flink offers a more managed, serverless experience specifically for streaming analytics, reducing operational overhead.
Amazon Kinesis Data Analytics for Apache Flink
A fully managed service for processing and analyzing streaming data using Apache Flink.
- Enables real-time analytics and transformations on data streams.
- Supports SQL, Java, Scala, and Python for Flink applications.
- Automatically scales to match data volume and throughput.
Memory trick: Flink: The fast river for your stream data transformations.