Professional Data EngineerDesigning data processing systemsMedium
A financial institution processes sensitive customer transaction data daily. Due to stringent regulatory requirements, all data must be encrypted at rest and in transit, access must be strictly controlled and auditable, and data residency must be maintained within a specific geographic region. The solution must also allow for flexible, cost-effective long-term storage of historical data for compliance purposes. Which combination of Google Cloud services and features best addresses these requirements?
- ACloud Storage buckets with default encryption, security groups, and custom Python scripts for auditing.
- BCloud SQL with customer-managed encryption keys (CMEK), IAM roles, and VPC Service Controls.
- CBigQuery with customer-managed encryption keys (CMEK), IAM roles, data location settings, and audit logging.
- DDataproc clusters with local disk encryption, SSH access controls, and manual log review.
Show answer & explanationAnswer & explanation
Correct answer: C. BigQuery with customer-managed encryption keys (CMEK), IAM roles, data location settings, and audit logging.
BigQuery provides strong encryption capabilities (including CMEK), granular IAM for access control, explicit data location settings for residency, and automatic audit logging for compliance. These features collectively address all the stated requirements for sensitive financial data.
Why the other options are wrong
- A. Cloud Storage is suitable for data lakes, but 'default encryption' might not meet 'customer-managed encryption keys' (CMEK) requirement, 'security groups' are not a primary GCP access control mechanism, and 'custom Python scripts' are not a robust, integrated audit logging solution.
- B. Cloud SQL is not designed for petabyte-scale analytical data warehousing and lacks some of the compliance features like built-in audit logging and flexible long-term storage tiers of BigQuery.
- D. Dataproc is for processing, not primarily for secure data storage and querying. Local disk encryption and SSH are insufficient for comprehensive data at rest/in transit encryption, access control, and auditing requirements across a data warehousing solution.
Secure & Compliant Data Warehouse
Designing a data warehouse system on Google Cloud that adheres to strict regulatory requirements for encryption, access control, data residency, and auditability.
- BigQuery offers CMEK, IAM, data location, and audit logs.
- CMEK ensures encryption key control by the customer.
- IAM provides granular access management.
- Data location settings enforce residency requirements.
Memory trick: Encrypt, Control, Locate, Audit - The four pillars of compliance.