AWS Certified Data Engineer – AssociateData Ingestion and TransformationMedium
A healthcare provider receives patient data from various on-premises systems, including electronic health records (EHR) and medical imaging archives. This data, totaling hundreds of terabytes, needs to be securely and efficiently transferred to an Amazon S3 bucket in a central AWS account. Transfers occur periodically, often overnight, and must resume automatically if interrupted. Which AWS service is the most suitable for this large-scale, periodic data ingestion?
- AAWS Snowball Edge
- BAWS DataSync
- CAWS Transfer Family
- DAmazon Kinesis Data Streams
Show answer & explanationAnswer & explanation
Correct answer: B. AWS DataSync
AWS DataSync is designed for automated, accelerated, and secure data transfer between on-premises storage and AWS storage services like S3. It handles large datasets (hundreds of terabytes), can be scheduled for periodic transfers, and automatically resumes interrupted transfers, making it ideal for this scenario.
Why the other options are wrong
- A. Snowball Edge is for offline data transfer (physical appliance) or edge computing, not for online periodic transfers.
- C. AWS Transfer Family provides fully managed support for SFTP, FTPS, and FTP, primarily for external partners, not typically for internal large-scale data center migration to S3.
- D. Kinesis Data Streams is for real-time streaming data, not periodic, large-scale file transfers from on-premises.
AWS DataSync
A data transfer service that makes it easier for you to automate moving data between on-premises storage and Amazon S3, Amazon EFS, or Amazon FSx for Windows File Server.
- Accelerated transfer technology.
- Automates and orchestrates data transfer tasks.
- Supports incremental transfers and automatic recovery from interruptions.
Memory trick: DataSync moves on-prem data to cloud, fast and steady.