A data engineer is designing a semantic model in Microsoft Fabric. The model will consume data from an operational database that is updated frequently throughout the day. Reports built on this model require data freshness within minutes, but only for the most recent data (e.g., the last 30 days). Historical data (older than 30 days) can be refreshed less frequently, perhaps daily. Which refresh strategy should the data engineer implement to efficiently balance data freshness and refresh performance?
- AUsing DirectQuery mode for all tables in the model.
- BScheduled refresh of the entire model once daily.
- CFull refresh of the entire model every 15 minutes.
- DIncremental refresh with a refresh policy configured for the last 30 days.
Show answer & explanationAnswer & explanation
Correct answer: D. Incremental refresh with a refresh policy configured for the last 30 days.
Incremental refresh is designed for scenarios where only a portion of the data (usually the most recent) needs to be refreshed frequently, while historical data is refreshed less often or not at all. By configuring an incremental refresh policy for the last 30 days, the model can achieve near real-time freshness for recent data without incurring the performance overhead of refreshing the entire historical dataset.
Why the other options are wrong
- A. While DirectQuery provides real-time data, it pushes all queries to the source, which might not be efficient for complex reports or a frequently updated transactional database, and it loses the performance benefits of cached data for historical reports.
- B. A daily scheduled refresh for the entire model would not meet the requirement of 'freshness within minutes' for recent data.
- C. A full refresh every 15 minutes for a large model would be highly inefficient and resource-intensive, especially if most of the data is historical and rarely changes.
Incremental Refresh
Incremental refresh in Microsoft Fabric semantic models allows for efficient data refresh by only processing a subset of data (e.g., the most recent partitions) instead of the entire table. This significantly reduces refresh times and resource consumption, especially for large tables.
- Refreshes only new or updated data partitions.
- Requires a date/time column for partitioning.
- Configuration involves defining a refresh policy (e.g., 'refresh data for last X days').
- Ideal for large, frequently updated tables where full refreshes are impractical.
Memory trick: Refresh smart, refresh incrementally, don't restart the whole cart.