Professional Cloud ArchitectAnalyze and optimize technical and business processesMedium
A large enterprise is migrating its legacy data analytics platform to Google Cloud. They have petabytes of historical data stored in various on-premises databases and file systems. The migration needs to be performed with minimal impact on production systems, ensuring data integrity and security, and optimizing for large-scale, one-time transfer. Which Google Cloud service is most suitable for this initial data migration?
- ACloud Data Fusion for ETL pipelines.
- BCloud VPN for secure network connectivity.
- Cgsutil for command-line transfers.
- DStorage Transfer Service for one-time transfers from on-premises.
Show answer & explanationAnswer & explanation
Correct answer: D. Storage Transfer Service for one-time transfers from on-premises.
Storage Transfer Service is designed for large-scale, one-time or recurring data transfers from various sources, including on-premises, to Google Cloud Storage. It handles data integrity, security, and optimizes transfers, making it ideal for migrating petabytes of historical data efficiently.
Why the other options are wrong
- A. Cloud Data Fusion is an ETL service for data integration and transformation, not primarily for the initial large-scale bulk transfer of raw data from on-premises to cloud storage.
- B. Cloud VPN provides secure network connectivity but is not a data transfer service itself. While it might be used to secure the connection for a transfer, it doesn't perform the transfer or provide its management features.
- C. gsutil is a command-line tool, suitable for smaller transfers or scripting, but less optimized for petabyte-scale, managed, and monitored transfers from on-premises with retry mechanisms and integrity checks.
Storage Transfer Service (on-premises)
A fully managed Google Cloud service that enables large-scale, secure, and reliable data transfers from on-premises sources (or other cloud providers) to Google Cloud Storage.
- Supports transfers from NFS, HDFS, S3-compatible sources, Azure Blob Storage.
- Optimizes transfer bandwidth and performance.
- Provides data integrity checks and retry mechanisms.
- Managed service, reduces operational overhead.
Memory trick: Storage Transfer Service smoothly moves your mountain of data securely.