Microsoft Azure Data FundamentalsDescribe core data conceptsMedium
A data analyst needs to combine customer demographic data from a relational database, product catalog information from a document database, and sales transaction logs from a data lake. Before loading this consolidated data into a data warehouse for reporting, the analyst must clean, transform, and normalize it. Which data processing concept describes this entire process?
- AStream Processing
- BBatch Processing
- CExtract, Transform, Load (ETL)
- DOnline Analytical Processing (OLAP)
Show answer & explanationAnswer & explanation
Correct answer: C. Extract, Transform, Load (ETL)
The process of extracting data from various sources, cleaning and transforming it, and then loading it into a target system like a data warehouse is precisely what Extract, Transform, Load (ETL) describes.
Why the other options are wrong
- A. Stream processing handles continuous data in real-time, not batch consolidation and transformation.
- B. Batch processing is a method of execution, but ETL is the specific concept encompassing the extraction, transformation, and loading steps.
- D. OLAP is for analyzing data in a data warehouse, not the process of preparing it.
Extract, Transform, Load (ETL)
A three-phase data integration process used to consolidate data from various sources into a single, consistent data store, typically a data warehouse or data mart.
- Extract: Reading data from source systems.
- Transform: Converting data into the desired format, cleaning, enriching, and applying business rules.
- Load: Writing the transformed data to the target data store.
- Crucial for data warehousing and business intelligence initiatives.
Memory trick: ETL: Get it, Fix it, Put it away!