Microsoft Certified: Fabric Analytics Engineer AssociatePrepare and transform data (20-25%)Medium
A global e-commerce company uses Microsoft Fabric to analyze customer order data. They have multiple Dataflows Gen2, each ingesting order details from different regional sales systems. The data engineer needs to combine the results of these Dataflows into a single, unified table in a Lakehouse for comprehensive reporting. How should the engineer achieve this efficiently within Microsoft Fabric?
- ACreate a new Dataflow Gen2 and use 'Get Data' to connect to each existing Dataflow's output, then append them.
- BPublish each Dataflow Gen2's output to a separate table in the Lakehouse, then create a Power BI semantic model that combines these tables.
- CWrite a Spark Notebook to read the output of each Dataflow from the Lakehouse and union them programmatically.
- DUse a Data Pipeline with multiple Copy Data activities, one for each Dataflow's output, copying into the same Lakehouse table.
Show answer & explanationAnswer & explanation
Correct answer: A. Create a new Dataflow Gen2 and use 'Get Data' to connect to each existing Dataflow's output, then append them.
Creating a new Dataflow Gen2 and using 'Get Data' to connect to the outputs of existing Dataflows, followed by an 'Append Queries' operation, is the most direct and visually manageable way to combine data from multiple Dataflows within the Power Query environment for a unified output.
Why the other options are wrong
- B. This combines data at the reporting layer, not at the data integration layer in the Lakehouse, meaning the underlying Lakehouse table would still be fragmented, which is not what was requested.
- C. This involves more manual coding and might be overkill if the primary goal is simply appending already prepared tables, though it's a valid option for more complex transformations.
- D. While technically possible, using multiple Copy Data activities to append into the same table can be less efficient and harder to manage for transformations compared to a single Dataflow operation.
Dataflows Gen2 Append
Dataflows Gen2 can combine data from multiple sources, including other Dataflows, by using the 'Append Queries' transformation in Power Query.
- Connects to existing Dataflow outputs as data sources.
- Efficiently combines tables with similar schemas.
- Keeps data transformation logic within the Dataflow environment.
Memory trick: Combine data streams like rivers into one grand lake.