AWS Certified Data Engineer – AssociateData Ingestion and TransformationHard
A healthcare provider needs to ingest patient data from various on-premises systems, including electronic health records (EHR) and medical imaging archives. The data is stored in a combination of file shares (NFS) and object storage. The total data volume is over 500 TB, and new data is continuously generated. The migration needs to be secure, automated, and maintain data integrity, with ongoing synchronization of changes. Which AWS service is best suited for this large-scale, ongoing data ingestion?
- AAWS Storage Gateway (File Gateway)
- BAmazon S3 Transfer Acceleration
- CAWS Snow Family
- DAWS DataSync
Show answer & explanationAnswer & explanation
Correct answer: D. AWS DataSync
AWS DataSync is designed for large-scale, online data transfers between on-premises storage and AWS storage services like S3. It automates and accelerates transfers, ensures data integrity, and supports ongoing synchronization of changes, making it ideal for migrating and continuously ingesting 500 TB+ of file and object data from various on-premises sources.
Why the other options are wrong
- A. AWS Storage Gateway (File Gateway) provides on-premises applications with file-based access to Amazon S3, but its primary use case is to extend on-premises storage to S3, not to perform a one-time 500 TB+ migration with ongoing synchronization as efficiently as DataSync.
- B. Amazon S3 Transfer Acceleration speeds up S3 uploads over long distances but does not provide an agent for on-premises file/object storage integration, automation, or synchronization for diverse sources like NFS.
- C. AWS Snow Family (e.g., Snowball Edge) is primarily for offline data transfer of extremely large datasets where network transfer is impractical or too slow, not for continuous, online synchronization.
AWS DataSync for Large-Scale Migration & Sync
A service that automates and accelerates the online transfer of large amounts of data between on-premises storage and AWS storage services, supporting initial migration and continuous synchronization.
- High-performance, online data transfer
- Supports file shares (NFS, SMB) and object storage
- Automates, accelerates, and synchronizes data for ongoing changes
Memory trick: DataSync carries big data loads, on-prem to cloud, secure roads.