Professional Data EngineerOperationalizing machine learning modelsMedium
A healthcare provider is developing a machine learning model to assist in disease diagnosis. The model processes sensitive patient data and must adhere to strict data privacy regulations (e.g., HIPAA). During the model training phase, the data engineering team needs to ensure that the training data is anonymized and that no personally identifiable information (PII) is exposed. Which Google Cloud service or technique should be prioritized to de-identify the sensitive data before training?
- AEncrypt the data at rest in BigQuery.
- BStore the data in a private Cloud Storage bucket with restricted access.
- CImplement custom Python scripts to manually remove PII fields.
- DUse Cloud DLP (Data Loss Prevention) to scan and de-identify sensitive data.
Show answer & explanationAnswer & explanation
Correct answer: D. Use Cloud DLP (Data Loss Prevention) to scan and de-identify sensitive data.
Cloud DLP is a specialized service designed to discover, classify, and de-identify sensitive data, offering various transformation techniques like tokenization, redaction, and format-preserving encryption, making it ideal for regulatory compliance with PII.
Why the other options are wrong
- A. Encryption at rest protects data from unauthorized access but does not de-identify the data for use in training, where the model could still learn from PII if decrypted during processing.
- B. Restricting access and using private buckets is a security measure but does not de-identify the data itself, meaning PII is still present.
- C. Manual scripting is error-prone, difficult to scale, and may not cover all PII types or adhere to best practices for de-identification.
Cloud DLP (Data Loss Prevention)
A Google Cloud service that helps discover, classify, and protect sensitive data across various Google Cloud products and on-premises environments, offering de-identification techniques.
- Detects over 150 types of sensitive data.
- Offers redaction, tokenization, format-preserving encryption.
- Crucial for regulatory compliance (e.g., HIPAA, GDPR).
Memory trick: DLP scrubs your data, making PII invisible before ML sees it.