Microsoft Certified: Fabric Analytics Engineer AssociatePrepare and transform data (20-25%)Easy
A data engineering team is tasked with ingesting data from a financial API that provides daily transaction records. The API requires authentication via an API key and returns data in JSON format. The team needs to apply basic transformations, such as filtering out test transactions and renaming a few columns, before loading the data into a Lakehouse table for further analysis. Which Microsoft Fabric tool is the most efficient and suitable for this ingestion and initial transformation process, given that the team prefers a low-code approach?
- AKQL (Kusto Query Language) script
- BNotebook with PySpark
- CDataflows Gen2
- DCopy Data activity in a Data Pipeline
Show answer & explanationAnswer & explanation
Correct answer: C. Dataflows Gen2
Dataflows Gen2 is ideal for low-code data ingestion and transformation, especially from various sources like APIs, offering a visual interface to apply transformations before loading data into a Lakehouse.
Why the other options are wrong
- A. KQL is primarily a query language for Kusto databases and is not designed for data ingestion or the specified transformations from an API source in this context.
- B. Notebooks with PySpark offer high flexibility but are a high-code solution, which goes against the team's preference for a low-code approach.
- D. While Data Pipelines can ingest data, Dataflows Gen2 offers a more robust and visual environment for the described transformations with a low-code approach.
Dataflows Gen2
A low-code data integration tool in Microsoft Fabric for ingesting, transforming, and preparing data from various sources.
- Uses Power Query for transformations.
- Supports a wide range of data sources.
- Can output directly to Lakehouse tables.
Memory trick: Flowing data, visually transformed, ready for the Lakehouse.