Professional Data EngineerDesigning data processing systemsHard

A healthcare provider is migrating its on-premises electronic health records (EHR) system to Google Cloud. The data contains highly sensitive patient information and must comply with HIPAA regulations. The data lake will store raw, unstructured, and semi-structured data from various sources. The design requires robust access controls, encryption at rest and in transit, and auditing capabilities to track all data access. Which Google Cloud storage service is the most appropriate foundational layer for this data lake, and what key security features should be configured?

  1. AFirestore with Security Rules and data masking for sensitive fields.
  2. BCloud SQL with Customer-Managed Encryption Keys (CMEK) and IAM roles for access control.
  3. CBigQuery with column-level encryption and authorized views for access control.
  4. DCloud Storage with Customer-Managed Encryption Keys (CMEK), VPC Service Controls, and Cloud Audit Logs.
Show answer & explanation

Correct answer: D. Cloud Storage with Customer-Managed Encryption Keys (CMEK), VPC Service Controls, and Cloud Audit Logs.

Cloud Storage is the ideal foundational layer for a data lake, capable of storing diverse data types at massive scale. CMEK ensures encryption at rest with customer-controlled keys, fulfilling a common compliance requirement. VPC Service Controls create security perimeters to prevent data exfiltration, crucial for HIPAA. Cloud Audit Logs provide comprehensive auditing for all data access and modifications, enabling compliance reporting.

Why the other options are wrong

  • A. Firestore is a NoSQL document database, not a data lake. Security Rules manage access at the document level but don't provide the comprehensive perimeter control or auditing for a large-scale, compliant data lake.
  • B. Cloud SQL is a relational database, not suitable as a foundational data lake for unstructured/semi-structured data. While CMEK and IAM are good, the core service is wrong.
  • C. BigQuery is a data warehouse, not a data lake for raw, diverse data types. Column-level encryption is a BigQuery feature, but it's not the foundational storage for a data lake.

Secure Data Lake on Cloud Storage

Building a compliant data lake on Google Cloud Storage involves using its object storage capabilities combined with robust security features like CMEK for encryption, VPC Service Controls for perimeter security, and Cloud Audit Logs for transparency and accountability.

  • Cloud Storage is scalable for diverse data lake data.
  • CMEK provides customer control over encryption keys.
  • VPC Service Controls protect against data exfiltration.
  • Cloud Audit Logs record administrative activity and data access.

Memory trick: Secure Lake, Controlled Access, Audited Trail.

More Designing data processing systems questions