Microsoft Azure Data FundamentalsDescribe an analytics workload on AzureMedium
A financial services company needs to process large volumes of historical transaction data, perform complex transformations, and then load the refined data into a data warehouse for reporting. This process needs to run on a daily schedule. Which Azure service is best suited for orchestrating and automating this data pipeline?
- AAzure Logic Apps
- BAzure Data Factory
- CAzure Event Hubs
- DAzure Stream Analytics
Show answer & explanationAnswer & explanation
Correct answer: B. Azure Data Factory
Azure Data Factory is a cloud-based ETL and ELT service that orchestrates and automates data movement and transformation, making it ideal for scheduled daily processing of large historical datasets.
Why the other options are wrong
- A. Azure Logic Apps is primarily for workflow automation and integration, not specialized for large-scale data transformation pipelines.
- C. Azure Event Hubs is for real-time event ingestion, not for orchestrating complex data transformations.
- D. Azure Stream Analytics is for real-time stream processing, not batch ETL/ELT.
Azure Data Factory (ADF)
A cloud-based ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform) service for orchestrating and automating data movement and transformation.
- Used for building complex data pipelines.
- Supports various data sources and destinations.
- Enables data integration and transformation at scale.
Memory trick: The data factory builds the data path.