Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
magnus_b
Advocate II
Advocate II

Reduce increment size for increment refresh in dataflows

Hi!

 

I have set up a dataflow that fetches data from an API using parameters similar to this: https://api.example.com/getData?fromDate=2021-09-08&toDate=2021-09-09

 

I have enabled incremental refresh, but the API request will fail since Power BI is trying to fetch 3 months of data at a time. Is there a way I can control the increment size to only fetching 1 month of data (or less) at a time?

 

When doing the API request manually in Postman I am able to run it successfully for 1 month of data, but 3 months of data will fail with error "Export result too large, please limit export criteria".

3 REPLIES 3
TomMartens
Super User
Super User

Hey @magnus_b ,

 

please check if this article provides what you are looking for: https://docs.microsoft.com/en-us/power-query/dataflows/incremental-refresh

 

If not, please provide detailed information on what makes your requirement special that is not met by the article.

 

Regards,

Tom



Did I answer your question? Mark my post as a solution, this will help others!

Proud to be a Super User!
I accept Kudos 😉
Hamburg, Germany

@TomMartens I have read the article, but I can't find any way to control the size of partions/increments use during inital load of data, can you?

Hey @magnus_b ,

 

from my knowledge it's very difficult to get the initial load done using dataflows, if the dataflows are not running on a Premium Capacity. Here you can use the enhanced compute engine (a SQL Server engine) that allows query folding.

Unfortunately, I have nothing that I can't share where the initial load is split into smaller chunks before the incremental refresh kicks in.

 

Regards,

Tom



Did I answer your question? Mark my post as a solution, this will help others!

Proud to be a Super User!
I accept Kudos 😉
Hamburg, Germany

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors