Microsoft Certified: Fabric Analytics Engineer AssociatePlan and implement data analytics solutions (10-15%)Easy
A data engineering team is planning to ingest data from an external relational database into a Microsoft Fabric Lakehouse. The source database contains several large tables, and the team needs to ensure that only new or updated records are ingested during subsequent runs to optimize performance and resource usage. Which ingestion strategy should the team implement?
- AFull load ingestion
- BIncremental load ingestion
- CSnapshot ingestion
- DBatch ingestion with full table overwrite
Show answer & explanationAnswer & explanation
Correct answer: B. Incremental load ingestion
Incremental load ingestion is the most efficient strategy for ingesting only new or updated records from a source system, saving time and resources. This approach avoids reprocessing unchanged data.
Why the other options are wrong
- A. Full load ingestion reloads all data every time, which is inefficient for subsequent runs with only new/updated records.
- C. Snapshot ingestion typically refers to taking a complete copy at a specific point in time, which is similar to a full load for the initial run and doesn't inherently handle subsequent updates incrementally.
- D. Batch ingestion with full table overwrite is essentially a full load and is inefficient when only new or updated records need to be processed.
Incremental Load
A data ingestion strategy that only processes and loads new or changed data from the source system since the last successful load.
- Optimizes performance by reducing data volume.
- Reduces resource consumption (compute, storage, network).
- Requires a mechanism to identify new/changed records (e.g., timestamp, version, change tracking).
Memory trick: Only new data, like a detective, finds what's changed.