Incremental refresh data changes on dataflow

[setis]

Dear all, 

I have a Data flow with the below incremental refresh settings. 

With this settings, will it detect and refresh data changes for the 2 years or only for the past 5 days?

I was assuming that it would detect data changes for the whole period and refresh no matter what the last 5 days.

Thanks in advance

[TomMartens · Accepted solution]

Hey @setis ,

unfortunately "detect data changes" does not work this way.

If you refresh today, the dataflow check for dates greater or equal 2020-06-23 (today - 5 days).

If a detect data changes column is configured Power BI stores the values of this column for the rows in the incremental refresh range. This can reduce the number of rows that will be refreshed.

This article explains this a little better: https://docs.microsoft.com/en-us/power-bi/transform-model/service-dataflows-incremental-refresh#:~:t...

Regards,

Tom 

[v-easonf-msft · Accepted solution]

Hi . @setis 

Incremental Refresh will make the part of the dataset to refresh much smaller, and as a result, the process would be much faster.  If you have a modified DateTime (or updated DateTime) in your table,one better way to do that  is to  enable the option of "Detect Data Changes", and then choose the modified date or update date from the table. Notice that this is different from the OrderDate or transaction date. And not all tables have such a field.Then the process of incremental refresh can monitor that field, and only get rows that their date/time is after the latest date/time in that field in the previous refresh. 

This video may better help you understanding the  "detect data changes"  in powerbi increment refresh.

https://www.youtube.com/watch?v=JsJWBr1_ktQ

In addition,  you can keep an eye on  this related thread.

https://community.powerbi.com/t5/Service/Premium-Incremental-Refresh-Detect-data-changes-How-does-it...

Ps. "detect data changes"  in the incremental refresh of dataflow should also be simlilar to that in the incremental refresh of  dateset.

Best Regards,
Community Support Team _ Eason

[TomMartens · Accepted solution]

Hey @setis ,

unfortunately "detect data changes" does not work this way.

If you refresh today, the dataflow check for dates greater or equal 2020-06-23 (today - 5 days).

If a detect data changes column is configured Power BI stores the values of this column for the rows in the incremental refresh range. This can reduce the number of rows that will be refreshed.

This article explains this a little better: https://docs.microsoft.com/en-us/power-bi/transform-model/service-dataflows-incremental-refresh#:~:t...

Regards,

Tom