Microsoft Azure Data FundamentalsDescribe an analytics workload on AzureHard
A data engineering team is designing a new analytics solution that requires ingesting data from various on-premises relational databases, cloud-based SaaS applications, and flat files. The data needs to be moved to Azure, transformed, and then loaded into an Azure Synapse Analytics Dedicated SQL Pool. They need a service that can orchestrate these complex data movement and transformation activities in a serverless and scheduled manner. Which Azure service should they choose?
- AAzure Stream Analytics
- BAzure Logic Apps
- CAzure Data Factory
- DAzure Functions
Show answer & explanationAnswer & explanation
Correct answer: C. Azure Data Factory
Azure Data Factory is a cloud-based ETL and data integration service that allows you to create, schedule, and orchestrate data workflows (pipelines) to move and transform data from diverse sources to various destinations, fitting the complex data movement and transformation requirement perfectly.
Why the other options are wrong
- A. Azure Stream Analytics is for real-time stream processing, not for scheduled batch data movement and transformation from diverse sources into a data warehouse.
- B. Azure Logic Apps are primarily for workflow automation and integration between applications, not for large-scale data movement and ETL orchestration.
- D. Azure Functions are serverless compute services for executing small pieces of code, not designed for orchestrating complex, end-to-end data integration pipelines.
Azure Data Factory
A cloud-based ETL (Extract, Transform, Load) and data integration service for orchestrating and automating data movement and transformation.
- Supports hybrid environments (on-premises and cloud).
- Provides visual tools for pipeline creation (no-code/low-code).
- Integrates with many Azure data services and external sources.
Memory trick: Data Factory is the factory floor for all your data pipelines.