Microsoft Certified: Azure Solutions Architect ExpertDesign data storage solutionsMedium
A manufacturing company uses several legacy on-premises systems that generate proprietary formatted data. They need to migrate this data to Azure for modernization and analytics. The data migration process involves extracting data from various sources, transforming it into a standardized format, and loading it into an Azure data lake. This process needs to be orchestrated, monitored, and scheduled regularly. Which Azure service is best suited to manage and orchestrate this ETL/ELT pipeline?
- AAzure Functions
- BAzure Data Factory
- CAzure Logic Apps
- DAzure Event Hubs
Show answer & explanationAnswer & explanation
Correct answer: B. Azure Data Factory
Azure Data Factory is a cloud-based ETL and ELT service that allows you to create, schedule, and orchestrate data workflows. It provides connectors to various data sources, enables data transformation, and supports monitoring of data pipelines, making it ideal for managing complex data integration scenarios.
Why the other options are wrong
- A. Azure Functions are serverless compute services for executing small pieces of code on demand, not for orchestrating full ETL/ELT pipelines.
- C. Azure Logic Apps is for workflow automation and integration of applications and services, but it's not specifically designed for large-scale data ingestion and transformation pipelines.
- D. Azure Event Hubs is a real-time data ingestion service for large volumes of events, not for orchestrating ETL/ELT processes.
Azure Data Factory
A cloud-based ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform) service for data integration. It allows you to create data-driven workflows for orchestrating data movement and transformation at scale.
- Orchestrates and automates data pipelines
- Supports ETL and ELT processes
- Connects to various on-premises and cloud data sources
- Provides monitoring and scheduling capabilities
Memory trick: Data Factory moves and molds, Logic Apps automate, Functions execute code.