AWS Certified Data Engineer – AssociateData Ingestion and TransformationHard
A pharmaceutical company needs to ingest experimental results from laboratory instruments. These instruments generate data in various proprietary binary formats and store them on a local network file share (NFS). Data needs to be periodically transferred to Amazon S3, where custom Lambda functions will perform format conversion and initial processing. The solution must provide low-latency access to the data on-premises for a short period before it's moved to S3. Which AWS service provides a hybrid cloud storage solution that fits this requirement?
- AAmazon S3 File Gateway
- BAWS Transfer Family
- CAWS Storage Gateway (File Gateway)
- DAWS DataSync
Show answer & explanationAnswer & explanation
Correct answer: C. AWS Storage Gateway (File Gateway)
AWS Storage Gateway (File Gateway) provides a file interface to S3, allowing applications to store and retrieve files from S3 using standard file protocols like NFS. It caches frequently accessed data locally, providing low-latency on-premises access, while asynchronously moving all data to S3, perfectly matching the hybrid storage and periodic transfer needs.
Why the other options are wrong
- A. Amazon S3 File Gateway is an incorrect service name; the correct service is AWS Storage Gateway (File Gateway).
- B. AWS Transfer Family provides managed SFTP/FTPS/FTP endpoints, primarily for external file exchange, not for integrating on-premises file shares with S3 for cached access.
- D. AWS DataSync is for high-performance bulk data transfers, not for providing a persistent, low-latency file share interface to S3 from on-premises.
AWS Storage Gateway (File Gateway)
A hybrid cloud storage service that provides a file interface to Amazon S3, allowing on-premises applications to store and retrieve files directly from S3.
- Supports NFS and SMB protocols for on-premises access.
- Caches frequently accessed data locally for low-latency access.
- Automatically uploads all files to S3, simplifying cloud integration.
Memory trick: File Gateway: Your local file drawer connected to the S3 cloud closet.