Professional Cloud ArchitectAnalyze and optimize technical and business processesHard
A global manufacturing company uses a legacy ERP system hosted on-premises. This system generates large daily reports (several TBs) that are critical for business intelligence. The company wants to migrate the reporting pipeline to Google Cloud to leverage BigQuery for analytics, but the legacy ERP system can only export data as flat files to a local NFS share, and it cannot directly connect to cloud APIs. The data transfer needs to be automated, secure, and handle large volumes efficiently. Which Google Cloud solution should be recommended for ingesting these daily reports into Google Cloud Storage before loading into BigQuery?
- AManually upload files using `gsutil cp` from a jump host in Google Cloud.
- BRefactor the ERP system to directly stream data into Cloud Pub/Sub.
- CImplement a Cloud Storage FUSE mount on a Compute Engine VM and copy files to it.
- DDeploy Storage Transfer Service for on-premises data transfer using a transfer agent.
Show answer & explanationAnswer & explanation
Correct answer: D. Deploy Storage Transfer Service for on-premises data transfer using a transfer agent.
Storage Transfer Service with a transfer agent is designed for automated, large-scale data transfers from on-premises file systems (like NFS shares) to Cloud Storage. The transfer agent handles synchronization, resumable transfers, and security, making it ideal for regularly moving large volumes of data from a legacy system that cannot directly use cloud APIs.
Why the other options are wrong
- A. Manual uploads are not scalable or automated for daily multi-TB transfers and would require significant operational overhead.
- B. Refactoring a legacy ERP system to directly stream data (especially to Pub/Sub, which is for messages, not raw files) is a significant and costly development effort, which the problem statement explicitly states the system 'cannot directly connect to cloud APIs' in the short term.
- C. While technically possible, using Cloud Storage FUSE for multi-TB daily transfers from an NFS share is not designed for robust, high-performance, automated transfers and might introduce reliability and performance issues compared to a dedicated transfer service.
Storage Transfer Service (on-premises)
A Google Cloud service for migrating large amounts of data into Cloud Storage from various sources, including on-premises file systems, using transfer agents.
- Supports transfers from NFS, SMB, and local file systems.
- Uses transfer agents for secure, efficient data movement.
- Handles large-scale, automated, and scheduled transfers.
- Provides data integrity checks and resumable transfers.
Memory trick: Transfer Service agents move the big files, automatically!