Incremental refresh

[answeriver]

Hi all. Can somebody explain to me about this feature.
I have desktop report with huge data. Every day my data is updating of new data. Refreshing full report it so much time need.
Question is, do incremental refresh work with desktop version? And if the answer YES, I cant understand which date I need put on Range End?

[v-lionel-msft · Accepted solution]

Hi @answeriver ,

1. Incremental refresh is configured in Power BI Desktop and performed in Power BI Service.

2. The values of the two parameters (RangeStart & RangeEnd) can be set to any value, as they will be overwritten by subsequent settings.

"Then create two parameters of DateTime data type, with names of RangeStart and RangeEnd, set a default value for each too. The default value can be anything, but the name and the data type should be as mentioned here."

"After doing this action, your data in the table will be filtered based on the default values you have set for the RangeStart and RangeEnd parameters. However, don’t worry about that. these two parameters will be overwritten with the configuration you make in the Incremental Refresh setting of the Power BI Desktop."

For more, you can refer to this article, which is very detailed:

All You Need to Know About the Incremental Refresh in Power BI: Load Changes Only 

Best regards,
Lionel Chen

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[TomMartens]

Hey @answeriver ,

the simple answer is no, you need Power BI Desktop to setup and Power BI Service to perform the incremental refresh.

Here are two articles that explain

https://docs.microsoft.com/en-us/power-bi/service-premium-incremental-refresh

how to use incremental refresh and the announcement that incremental refresh is now available for Power BI Pro

https://powerbi.microsoft.com/en-us/blog/tag/incremental-refresh/

Hopefully, this helps to tackle your challenges.

Regards,

Tom

[jmg80525]

@TomMartens If you could kindly let me know if any of the above statement is incorrect that would be great.  Just looking for tierra firma in the ever-shifting sands of PBI.

[TomMartens]

Hey @jmg80525 ,

incremental refresh will be performed at the Power BI Service level. There is no chance to have incremental refresh at the Power BI Desktop level.

Whenever you "republish" the pbix file that contains the incremental refresh settings, the incremental refresh cycle will restart. For this reason I recommend to use 2 pbix files one that "just" contains the queries and the data model, and of course the incremental refresh settings. The 2nd pbix contains the content the visuals etc. This file does not connect to the "original" data source but instead connects to the data source Power BI datasets:

This approach allows to separate the development of reports, pages, and visuals from the model.

If you have to alter the Power BI data model by adding new tables and measures you update the data model without affecting the existing partitions of tables by using the external tools ALM Toolkit or Tabular Editor, but this requires some "practice" meaning you should practice updating a model without overwritng partitions.

Adding a calculated column to a table that is partitioned by incremental refresh will of course restart the incremental refresh cycle for this table.

Changing a query is more difficult, until now I do not know how to "update" the Power Query part without affecting existing partitions..

Hopefully, this answers some of your questions.

Regards,

Tom

[PQPBILessons]

Hi Tom,

Awesome clearly laid out help you provided here. 

Do you know why it is not possible to set up IR on PBI desktop? MS doesn't want to give this for free or that it is difficult to implement it?  I think this is one of the most important features of PBI.

Your idea of two PBIs to navigate around these limitations is fascinating. I have not tried it yet but I think it will address the problem I have of updating large-size files on a daily basis like the person you responded to. 

Thank you.

YY

[TomMartens]

Hey @PQPBILessons ,

incremental refresh is set up in Power BI Desktop, but performed at the Service. Next to that, incremental refresh is supported in almost any plans as listed in this article: https://learn.microsoft.com/en-us/power-bi/connect-data/incremental-refresh-overview

The only place where incremental refresh is not available from an execution point of view is Power BI desktop.
Personally I never missed it, because I'm working for an organization where sharing of insights is happening via the app and not by the pbix file.

If you have any further questions, please start a new thread.

Thanks,

Tom 

[jmg80525]

@TomMartens Thank you! That's very kind of you. Here is a video that cements me how "incremental refresh" works in addition to your help.

[jmg80525]

@TomMartens  Thanks for you clarification.  I've spent a better part of the day trying to get this figured out.  Have a time series dataset w/ 90 million plus rows and growing at 64k plus a day with a tremendous amount of cardinality.  I thought that by implementing incremental refresh I would prevent the need to reload the dataset inside the pbix every time I made a change in PQ and applied the transfromation.  From your comment to original poster above I now assume that is incorrect. Incremental refresh is about data refresh at the service level.