AWS Certified Data Engineer – AssociateData Ingestion and TransformationMedium
A gaming company is developing a new mobile game and anticipates millions of concurrent players. They need to collect in-game telemetry data, such as player actions, scores, and session information, in real-time. This data must be immediately available for anomaly detection and live dashboards, and later archived to Amazon S3 for long-term analytics. The solution must handle unpredictable spikes in traffic and ensure high data durability. Which AWS service should be used to ingest this high-volume, real-time streaming data?
- AAmazon SQS
- BAmazon Kinesis Data Firehose
- CAWS IoT Core
- DAmazon Kinesis Data Streams
Show answer & explanationAnswer & explanation
Correct answer: D. Amazon Kinesis Data Streams
Amazon Kinesis Data Streams is best suited for this scenario. It provides real-time ingestion of high-volume streaming data, ensures durability, and allows multiple consumers (e.g., for anomaly detection, live dashboards) to process the data concurrently. Data can then be easily delivered to S3 for archiving and further analytics.
Why the other options are wrong
- A. Amazon SQS is a message queuing service, not optimized for high-volume, real-time streaming ingestion with multiple concurrent consumers for immediate processing.
- B. Amazon Kinesis Data Firehose is primarily for delivering streaming data to destinations like S3 or Redshift, but it does not allow multiple real-time consumers for immediate processing and anomaly detection directly from the stream.
- C. AWS IoT Core is for connecting IoT devices and managing their messages, not directly for general-purpose high-volume application telemetry from mobile games.
Amazon Kinesis Data Streams
A real-time data streaming service capable of ingesting and storing large amounts of data from various sources.
- Provides high throughput and low latency for data ingestion.
- Supports multiple concurrent consumers for real-time processing.
- Data is stored for up to 365 days (default 24 hours) for replay.
Memory trick: Kinesis Streams: The rapid river for your real-time game data.