AWS Certified Solutions Architect – Associate (SAA-C03)Design High-Performing ArchitecturesMedium
A data analytics startup needs to process petabytes of historical data stored in Amazon S3 for daily batch analytics jobs. These jobs are computationally intensive and can take several hours to complete. The startup wants to optimize costs by only paying for compute capacity when needed and automatically scaling resources based on the workload. Which AWS compute solution would be most cost-effective and scalable for this requirement?
- AAmazon EC2 On-Demand Instances
- BAmazon EMR with EC2 Spot Instances
- CAWS Lambda functions
- DAWS Fargate on Amazon ECS
Show answer & explanationAnswer & explanation
Correct answer: B. Amazon EMR with EC2 Spot Instances
Amazon EMR is a managed cluster platform for big data processing using frameworks like Spark and Hadoop, ideal for petabyte-scale data in S3. Utilizing EC2 Spot Instances with EMR significantly reduces costs for fault-tolerant, flexible batch jobs, as Spot Instances allow bidding for unused EC2 capacity at a lower price.
Why the other options are wrong
- A. EC2 On-Demand instances provide flexibility but are not the most cost-effective for long-running, interruptible batch jobs compared to Spot Instances.
- C. AWS Lambda functions have a maximum execution duration of 15 minutes, making them unsuitable for jobs that take 'several hours' to complete.
- D. AWS Fargate on ECS is serverless for containers, but EMR with Spot Instances is typically more optimized for petabyte-scale, long-running big data processing with frameworks like Spark/Hadoop.
Amazon EMR with Spot Instances
A managed cluster platform for big data processing (e.g., Spark, Hadoop) that leverages EC2 Spot Instances for cost optimization on interruptible workloads.
- EMR simplifies big data framework deployment and management.
- Spot Instances offer significant cost savings (up to 90% off On-Demand).
- Ideal for fault-tolerant, flexible batch processing jobs.
- Integrates well with data stored in Amazon S3.
Memory trick: Big data crunch, EMR's punch, Spot saves cash, for the big lunch.