Professional Data EngineerBuilding and operationalizing data processing systemsMedium
A data analytics team is migrating an on-premises Hadoop cluster to Google Cloud. They have petabytes of diverse, unstructured, and semi-structured data (logs, images, videos, CSV files) that need to be stored cost-effectively, made accessible for various analytical tools (Dataproc, BigQuery, Dataflow), and serve as the single source of truth. Which Google Cloud service is the most appropriate foundational component for building this data lake?
- ACloud Storage
- BCloud SQL
- CCloud Bigtable
- DBigQuery
Show answer & explanationAnswer & explanation
Correct answer: A. Cloud Storage
Cloud Storage is an object storage service that provides a highly scalable, durable, and cost-effective solution for storing large volumes of unstructured and semi-structured data, making it the ideal foundation for a Google Cloud data lake.
Why the other options are wrong
- B. Cloud SQL is a relational database and not suitable for a data lake storing diverse unstructured data.
- C. Cloud Bigtable is a NoSQL database for specific high-throughput, low-latency operational workloads, not a general-purpose data lake.
- D. BigQuery is an analytical data warehouse, designed for structured data queries, not raw data lake storage.
Cloud Storage for Data Lakes
Google Cloud Storage serves as the foundational component for building data lakes on GCP, offering scalable, durable, and cost-effective object storage for diverse data types.
- Stores unstructured, semi-structured, and structured data.
- Highly scalable (petabytes to exabytes).
- Highly durable (99.999999999% annual durability).
- Integrates seamlessly with other GCP services like Dataproc, BigQuery, Dataflow.
Memory trick: For a lake of data, a cloud bucket's the base.