Professional Data EngineerDesigning data processing systemsHard
A financial institution needs to design a data processing system on Google Cloud to handle sensitive customer transaction data. The system must ensure that data at rest is encrypted, and access is strictly controlled based on the principle of least privilege. Data must be available for real-time fraud detection and also for daily batch reporting. The security team mandates that all encryption keys for sensitive data must be managed externally by a dedicated Hardware Security Module (HSM) and never directly exposed to Google Cloud services. Which Google Cloud service combination should be used to meet these requirements, specifically for key management and data storage?
- ACloud KMS for key management, Cloud Storage for data at rest, and Dataflow for processing.
- BCloud External Key Manager (EKM) with a customer-managed HSM, BigQuery for data warehousing, and Pub/Sub for real-time ingestion.
- CSecret Manager for key storage, Firestore for real-time data, and Cloud Functions for event-driven processing.
- DCloud HSM for key management, Cloud SQL for transactional data, and Dataproc for batch processing.
Show answer & explanationAnswer & explanation
Correct answer: B. Cloud External Key Manager (EKM) with a customer-managed HSM, BigQuery for data warehousing, and Pub/Sub for real-time ingestion.
Cloud External Key Manager (EKM) allows customers to manage encryption keys outside of Google Cloud, satisfying the requirement for external HSM. BigQuery is suitable for both real-time analytical queries (fraud detection) and large-scale batch reporting due to its architecture. Pub/Sub is ideal for real-time data ingestion.
Why the other options are wrong
- A. Cloud KMS manages keys within Google Cloud, not externally via a customer-managed HSM. Cloud Storage is object storage, not a data warehouse for complex queries.
- C. Secret Manager is for storing secrets, not for managing encryption keys with an external HSM. Firestore is a NoSQL document database, not ideal for complex analytical queries that BigQuery handles.
- D. Cloud HSM is Google-managed HSM, not customer-managed external HSM. Cloud SQL is a relational database, not typically used for large-scale analytical data warehousing.
Cloud External Key Manager (EKM)
A Google Cloud service that allows customers to protect their data at rest with encryption keys managed in an external key management system (KMS) or hardware security module (HSM) that they control.
- Provides cryptographic operations using externally-managed keys.
- Keys never leave the external KMS/HSM.
- Enhances compliance for highly regulated industries.
Memory trick: Keys Everywhere, Managed Securely.