Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
Imrans123
Advocate V
Advocate V

Best Practice for dealing with WEB API Source refresh on DataFlows

Hey All, 

 

I was wondering what your work around is for refreshing WEB API for Dataflow. Basically I have multiple queries using difference calls, but all under the same API. So when they refresh parallely, I go beyond the rate limit that the API allows. 

 

Unlike the desktop file, there is no option for me to disable parallel loading. I am using invoke after and I am giving excessive buffer times to make sure that my data refreshes smoothly. BUt it's really surprising how I can't simply disable parallel loading.

 

Am I missing something here?

 

Edit: After a bit of research, it seems this option is not available on dataflows. If anyone has the same issue, https://ideas.powerbi.com/ideas/idea/?ideaid=9f65c4d8-0bf4-ec11-b5cf-281878de4cfd

3 REPLIES 3
otravers
Community Champion
Community Champion

You may want to split your queries into different dataflows and chain their sequential execution via Power Automate so that one starts just after the other is finished. For illustration, the left branch of the process below is sequential, the right one runs in parallel:

 

otravers_0-1656113133840.png

 

------------------------------------------------
1. How to get your question answered quickly - good questions get good answers!
2. Learning how to fish > being spoon-fed without active thinking.
3. Please accept as a solution posts that resolve your questions.
------------------------------------------------
BI Blog: Datamarts | RLS/OLS | Dev Tools | Languages | Aggregations | XMLA/APIs | Field Parameters | Custom Visuals

Woah, creative! I'll risk making my workspace really messy 😛 But it's a pretty full proof idea if all else fails. 

Yes, you do want to name your dataflows with meaningful and consistent names if you're going to have many of them. I like to prepend numbers in case I have many dataflows and they're meant to be chained in a sequence (e.g. staging vs. silver vs. gold stages).

------------------------------------------------
1. How to get your question answered quickly - good questions get good answers!
2. Learning how to fish > being spoon-fed without active thinking.
3. Please accept as a solution posts that resolve your questions.
------------------------------------------------
BI Blog: Datamarts | RLS/OLS | Dev Tools | Languages | Aggregations | XMLA/APIs | Field Parameters | Custom Visuals

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors