Incremental refresh taking more time for refresh

[Vallirajap]

Hii all,
I have a report with 6 months of data, and I want to refresh the last 2 months only, so I enabled the incremental refresh option by setting the option in the below image and publishing it in the PBI service. but it takes 1 hour, 50 minutes, for refesh.

Incremental refesh is a way to reduce refesh time, but why does it behave like that?

Anyone who knows, please tell!

Thanks in advance!

[v-xiaoyan-msft · Accepted solution]

Hi  @Vallirajap ,

The time taken for incremental refresh could be related to the complexity of the model and the process of query folding. If the data source doesn’t support query folding, it could lead to longer refresh times.

If there’s a mismatch in data types between the RangeStart and RangeEnd parameters and the date column in your table, it could prevent the query from folding, leading to longer refresh times.Please ensure that the date/time column for the incremental refresh table is of Date/Time data type.

Also,If your table contains a large number of rows, a refresh can take a long time and consume a significant amount of resources.

You can use the Performance Analyzer in Power BI to measure the processing time required to update report elements.

For original post：

Power BI Incremental Refresh taking more time than... - Microsoft Fabric Community

Re: Power BI Incremental Refresh taking more time ... - Microsoft Fabric Community

Best regards.
Community Support Team_Caitlyn

[Rajeshkumaryarl]

Hi all. I also have a similar scenario. In our company, we did enable incremental refresh directly at dataflow and not in dataset in Power BI desktop. so i see refresh of the dataflow is done fastly. We configured incremental refresh at every 2 hours. We triggered dataset refresh an hour after dataflow refresh. Question is if dataflows incremental refresh is done fastly, the dataset refresh is consuming much more time closer to two hours. Could some one help me on this?? We are moving this to prod. So need to have a solution since in prod there will be maximum data coming through 

[lbendlin]

Drop the dataflow and implement Incremental Refresh for the dataset, but with live partition for "today".

[Rajeshkumaryarl]

did not get you. so you mean dataflow has to be configured for full load and dataset for incremental load ?? I dont think my manager will approve that scheme. 

[lbendlin]

Abandon the dataflow, it is not helping in your scenario.

Use this instead:  Incremental refresh for semantic models and real-time data in Power BI - Power BI | Microsoft Learn

[Vallirajap]

Hi @v-xiaoyan-msft,

For the first refresh, it takes 1:50 hours, but after 2 and 3 refreshes, it was finished in 45 minutes.

Why the time difference? What exactly happens in the incremental refesh?

Thanks in advance!

[lbendlin · Accepted solution]

Whenever you make structural (meta data) changes to your PBIX and publish that to the workspace it will reset the incremental refresh. All partitions will be removed and a single partition will be filled for the query where you have incremental refresh configured.  On the NEXT refresh your partitions will be recreated and filled according to your rules. On the NEXT refresh only your "hot"  partition will be refreshed.

To avoid that you either need to refrain from making meta data changes, or you need to use tools like ALM Toolkit to propagate the meta data changes without re-initializing the partitions.

[v-xiaoyan-msft · Accepted solution]

Hi  @Vallirajap ,

The time taken for incremental refresh could be related to the complexity of the model and the process of query folding. If the data source doesn’t support query folding, it could lead to longer refresh times.

If there’s a mismatch in data types between the RangeStart and RangeEnd parameters and the date column in your table, it could prevent the query from folding, leading to longer refresh times.Please ensure that the date/time column for the incremental refresh table is of Date/Time data type.

Also,If your table contains a large number of rows, a refresh can take a long time and consume a significant amount of resources.

You can use the Performance Analyzer in Power BI to measure the processing time required to update report elements.

For original post：

Power BI Incremental Refresh taking more time than... - Microsoft Fabric Community

Re: Power BI Incremental Refresh taking more time ... - Microsoft Fabric Community

Best regards.
Community Support Team_Caitlyn