AWS Certified Data Engineer – AssociateData Ingestion and TransformationEasy
A media company needs to process petabytes of video analytics data stored in Amazon S3. The data consists of billions of small JSON files, each containing metadata and events captured from video streams. Analysts need to run ad-hoc, interactive queries on this data to identify trends and anomalies. The solution must be serverless, cost-effective for intermittent querying, and support standard SQL. Which AWS service should be used?
- AAmazon EMR with Apache Hive
- BAmazon Athena
- CAmazon Redshift
- DAWS Glue ETL
Show answer & explanationAnswer & explanation
Correct answer: B. Amazon Athena
Amazon Athena is a serverless interactive query service that makes it easy to analyze data directly in Amazon S3 using standard SQL. It is ideal for ad-hoc queries on petabytes of data, paying only for the data scanned, which makes it cost-effective for intermittent querying.
Why the other options are wrong
- A. Amazon EMR with Apache Hive requires provisioning and managing a cluster, which is not serverless and can be more expensive for intermittent ad-hoc queries compared to Athena.
- C. Amazon Redshift is a fully managed data warehouse, designed for complex analytical queries on structured data, but it requires provisioning and managing a cluster, making it less cost-effective for intermittent, ad-hoc queries directly on S3 data.
- D. AWS Glue ETL is for data transformation and preparation, not for interactive querying of data. While it can prepare data for Athena, it's not the query engine itself.
Amazon Athena
A serverless interactive query service that makes it easy to analyze data directly in Amazon S3 using standard SQL.
- Serverless; no infrastructure to manage.
- Pay-per-query based on data scanned.
- Supports standard SQL.
- Ideal for ad-hoc analysis and querying data lakes in S3.
Memory trick: Athena 'sees' your S3 data with SQL eyes.