Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
po
Post Prodigy
Post Prodigy

Incremental Refresh in Pro?

Hi,

 

Believe we use power bi pro.

 

Have read incremebtal refresh is available in power bi premium

 

I  have a few questions and wondered if anyone could answer

 

1) Any idea when incremental refresh will be available to Pro users post over a year ago (2018) suggested would be coming to pro.

https://community.powerbi.com/t5/Desktop/Incremental-refresh/m-p/415403#M190722

 

 

2) If we used incremental refresh, we could just append to a large dataset with what has loaded in last day.

 

However, what if has been a change in data from say a few days agao - can pro detect this e.g pick up any updates to existing data as well as newly inserted data to base table importing.  Also what about deletes to base table are they reflected in incremental resfresh?

 

 

3)  Believe pro has a 10gb file size limit?

      Fact table concerned which would have incremental refresh would have several hundred million rows 

Any rough estimate as to largest fact table say under 10 columns e.g date_loaded, sales_qty, custtomer_no and product_no

 

Thanks

 

1 ACCEPTED SOLUTION
v-frfei-msft
Community Support
Community Support

Hi @po ,

 

For the 1st question, as the idea, MS are  definitely planning on adding Incremental refresh to Pro.

 

can pro detect this e.g pick up any updates to existing data as well as newly inserted data to base table importing.  Also what about deletes to base table are they reflected in incremental resfresh?

 

 


The answer should be no, as for the incremental refresh, datasets are filtered by using Power Query date/time parameters with the reserved, case-sensitive names RangeStart and RangeEnd.

 

3)  Believe pro has a 10gb file size limit?

      Fact table concerned which would have incremental refresh would have several hundred million rows 

Any rough estimate as to largest fact table say under 10 columns e.g date_loaded, sales_qty, custtomer_no and product_no

 

For the size of the fact table, the issue with that is Power BI uses a columnar compression algorithm to compress data. So, you would have to understand exactly how that algoritm works and exactly what data you have. For example, if you have lots of the same value in a column, that will compress very well, but if you have lots of unique values, that will compress less well.

 

 

 

 

Community Support Team _ Frank
If this post helps, then please consider Accept it as the solution to help the others find it more quickly.

View solution in original post

4 REPLIES 4
v-frfei-msft
Community Support
Community Support

Hi @po ,

 

For the 1st question, as the idea, MS are  definitely planning on adding Incremental refresh to Pro.

 

can pro detect this e.g pick up any updates to existing data as well as newly inserted data to base table importing.  Also what about deletes to base table are they reflected in incremental resfresh?

 

 


The answer should be no, as for the incremental refresh, datasets are filtered by using Power Query date/time parameters with the reserved, case-sensitive names RangeStart and RangeEnd.

 

3)  Believe pro has a 10gb file size limit?

      Fact table concerned which would have incremental refresh would have several hundred million rows 

Any rough estimate as to largest fact table say under 10 columns e.g date_loaded, sales_qty, custtomer_no and product_no

 

For the size of the fact table, the issue with that is Power BI uses a columnar compression algorithm to compress data. So, you would have to understand exactly how that algoritm works and exactly what data you have. For example, if you have lots of the same value in a column, that will compress very well, but if you have lots of unique values, that will compress less well.

 

 

 

 

Community Support Team _ Frank
If this post helps, then please consider Accept it as the solution to help the others find it more quickly.

Thanks for advice and link

 

Do you have any idea as to timelines ms plan to bring incremenatl refresh to pro ( e.g later this year, q2 next year etc?

Hi @po ,

 

As the PM said in the idea, https://ideas.powerbi.com/forums/265200-power-bi-ideas/suggestions/34198180-incremental-refresh-for-...

 

"Thanks for all the feedback on this! We are definitely planning on adding Incremental refresh to Pro. Do know though that it is a fair amount of work, so it may be a bit before we can give you a more firm timeline.

In the meantime, keep voting so you’re notified when we do have more information to share! "

 

 

 

 

Community Support Team _ Frank
If this post helps, then please consider Accept it as the solution to help the others find it more quickly.

Thanks for update - will do.

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.