Professional Data EngineerManaging and securing dataMedium
A financial institution is building a data warehouse in BigQuery. They need to ensure that data consumed by downstream analytical applications is trustworthy and that its origin and transformations can be tracked. Specifically, they need to visualize the path of data from its ingestion sources through various BigQuery transformations to its final destination tables. Which Google Cloud service provides this capability?
- ACloud Data Catalog Data Lineage
- BCloud Audit Logs
- CBigQuery Information Schema
- DCloud Trace
Show answer & explanationAnswer & explanation
Correct answer: A. Cloud Data Catalog Data Lineage
Cloud Data Catalog Data Lineage provides automated tracking and visualization of data movement and transformations across Google Cloud, allowing users to understand the origin and evolution of their data assets.
Why the other options are wrong
- B. Cloud Audit Logs record administrative activities and data access events, but they don't visualize data transformations or lineage.
- C. BigQuery Information Schema provides metadata about BigQuery datasets, tables, and views, but it does not visualize the lineage of data transformations.
- D. Cloud Trace is for monitoring application performance and latency, not for data lineage tracking.
Cloud Data Catalog Data Lineage
Cloud Data Catalog Data Lineage provides automated tracking and visualization of data movement and transformations across various Google Cloud services, helping users understand data origin and trust.
- Automates lineage collection.
- Visualizes data flow and transformations.
- Supports BigQuery, Pub/Sub, Dataflow, and more.
Memory trick: Lineage Links Latest Location.