AWS Certified Data Engineer – AssociateData Ingestion and TransformationMedium
A global media company receives large video files (up to 50 GB each) from content creators located worldwide. These files arrive daily and need to be securely ingested into Amazon S3 for archival and subsequent processing by AWS Media Services. The content creators prefer using standard file transfer protocols. The ingestion solution must be highly available and eliminate the need for managing compute instances or complex network configurations. Which combination of AWS services should the company use?
- AAWS DataSync with a self-managed SFTP server
- BAWS Storage Gateway (File Gateway) with Amazon S3
- CAmazon Kinesis Data Firehose with an AWS Lambda function
- DAWS Transfer Family (SFTP) with Amazon S3
Show answer & explanationAnswer & explanation
Correct answer: D. AWS Transfer Family (SFTP) with Amazon S3
AWS Transfer Family provides a fully managed SFTP, FTPS, and FTP service that directly integrates with Amazon S3. This allows content creators to use their preferred file transfer protocols to upload large files securely into S3 without managing any servers.
Why the other options are wrong
- A. AWS DataSync is for bulk data transfer, often one-time or scheduled syncs, and does not provide a managed SFTP endpoint for external users. A self-managed SFTP server contradicts the 'eliminate managing compute instances' requirement.
- B. AWS Storage Gateway (File Gateway) provides on-premises applications with file-based access to cloud storage, but it does not offer an external, internet-facing SFTP endpoint for content creators.
- C. Amazon Kinesis Data Firehose is for streaming data ingestion, not for large file transfers via SFTP. Lambda functions are for compute, not direct SFTP endpoints.
AWS Transfer Family
A fully managed service that enables the transfer of files directly into and out of Amazon S3 using SFTP, FTPS, and FTP.
- Supports SFTP, FTPS, and FTP protocols
- Directly integrates with Amazon S3 and Amazon EFS
- Fully managed, serverless, and highly available
- Simplifies secure file transfers for external partners
Memory trick: Transfer Family makes file handoffs smooth and secure, like a trusted courier.