A retail company needs to process daily sales reports generated by point-of-sale (POS) systems across thousands of stores. Each store uploads a CSV file containing transaction data to a central SFTP server hosted in AWS. The company requires a serverless solution to automatically detect new files on the SFTP server, move them to an S3 bucket, and then trigger an ETL job to transform the CSV data into a Parquet format, partition it by store ID and date, and store it in another S3 bucket for analytics. Which AWS services should be used to automate this end-to-end pipeline?

Why the other options are wrong

• A. AWS Storage Gateway (File Gateway) provides file-based access to S3 from on-premises, but doesn't offer a managed SFTP server for external POS systems. Amazon SQS and Lambda could be used for parts of the automation, but the ingestion method is incorrect.
• C. AWS DataSync is for large-scale data transfers or syncs, not for providing a managed SFTP endpoint. S3 Event Notifications and AWS Glue ETL are appropriate for the latter parts of the pipeline, but DataSync doesn't fit the SFTP ingestion need.
• D. AWS Direct Connect is for network connectivity. AWS Batch and Amazon EMR are for batch processing and big data processing respectively, but they are not serverless for orchestration (EMR requires cluster management) and don't provide the SFTP ingestion or event-driven automation in a fully managed way for this scenario.