AWS Certified Data Engineer – AssociateData Storage and ManagementMedium
A data engineering team is building a serverless data processing pipeline using AWS Lambda. The Lambda functions need to access a shared, persistent file system to store intermediate processing results and configuration files that are too large to fit in Lambda's ephemeral storage. This file system must be accessible concurrently by multiple Lambda invocations and scale automatically. Which AWS storage service should they use?
- AAmazon EBS
- BAmazon S3
- CAmazon EFS
- DAmazon RDS
Show answer & explanationAnswer & explanation
Correct answer: C. Amazon EFS
Amazon EFS (Elastic File System) is a scalable, elastic, cloud-native NFS file system that can be mounted by Lambda functions. It provides shared, persistent storage that automatically grows and shrinks, meeting the requirements for concurrent access and large intermediate files.
Why the other options are wrong
- A. Amazon EBS (Elastic Block Store) provides block-level storage for single EC2 instances and cannot be directly attached to or shared concurrently by multiple Lambda functions.
- B. Amazon S3 is object storage, not a POSIX-compliant file system directly mountable by Lambda functions for shared, concurrent file access in the same way EFS is.
- D. Amazon RDS (Relational Database Service) is a managed relational database, not a file system for storing intermediate processing results or configuration files.
Amazon EFS for AWS Lambda
Amazon Elastic File System (EFS) provides scalable, shared, and persistent file storage that can be mounted by multiple AWS Lambda functions concurrently within a VPC.
- Enables Lambda functions to access and persist large amounts of data.
- Supports concurrent access from thousands of Lambda functions.
- Automatically scales storage capacity and performance.
Memory trick: EFS extends Lambda's reach.