Data flow refresh order

[Drunken_Sailor]

Hello,
In a workspace, I have serval dataflows When dataflow 1 is refrershed, it also refreshes the next dataflow (2).  And when Dataflow 2 starts it starts dataflows 3a, 3b, and 3c.

I assumed that none of the the 3 (a, b and c) would start until 2 is done. and 2 wont start until 1 is done.  But when I look at the end refresh times of each
1) 13:07:48
2) 13:08:10
3a) 13:07:49
3b) 13:08:15
3c) 13:07:49

The 3s are small API calls, so they are very quick anyway.  Actually 3a and 3c, wont find any data at the moment, as they are just setup for future use.  But I still dont understand how they end before number 2 does.

Thanks in advance

[Drunken_Sailor]

Thanks to all for trying to provide a solution. However, after further investigation, what I have learned is that when dataflow 1 is triggered, either manually, scheduled, API  or Power Automate, Dataflows 2 and 3 (all of them) are also triggered in PARRALEL and not sequentially.  Basically, in my scenario, 3 finishes before 2 and 1.  Which makes no sense for me. There also seems to be no option to turn this "feature" off. In addition, the queries within the dataflow seem to run in parallel too even though some have dependecies on each other. 
The only way I see making this work is to come up with a convoluted mix of Dataflows and semantic models and use power automate to "orchestrate" the timing.  

The only way I see making this work is to come up with a convuluted mix of Dataflows and semantic models and use power automate to "orchestrate" the timing.  I hope someone can tell me I am wrong.

[TomMartens]

Hey @Drunken_Sailor ,

I'm wondering if you have Power BI Premium?
Chaining, is exactly what we are doing by using linked entities. Most of the time to create staging/transforming/loading chains, this works perfecly. So I'm wondering if you are using linked entites, but as I already mentioned - this is a Premium feature (but also available with Premium Per User).

This article explains the premium features of dataflows in more detail: Premium features of dataflows - Power BI | Microsoft Learn

Regards,

Tom

[SaiTejaTalasila]

Hi,

Using power automate is the perfect option for your requirement.

[Drunken_Sailor]

Sorry, not really. I'll get a two triggers then, which seem redundant.

[v-zhengdxu-msft]

Hi @Drunken_Sailor 

You can refer to the following official documentation for a description of the refresh order:

Understand and optimize dataflows refresh - Power BI | Microsoft Learn

After you delete the dataflow, if your dataflow contains many queries or some queries depend on others. If you cancel a refresh before you start evaluating a query that loads data into a target, no changes will occur to the data in the target of that query. There are a few instrumentation tools you can use to further observe the refresh order of your dataflow:
powerquery-docs/powerquery-docs/dataflows/monitor-dataflow-refreshes.md at main · MicrosoftDocs/powe...

Best Regards

Zhengdong Xu
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[Drunken_Sailor]

I read that too, which is what led me to orgnaizing the dataflows in such a way.  They used the term "chaining" which led me to believe it was sequential... but it isnt.

[Daryl-Lynch-Bzy]

Hi @Drunken_Sailor , are you using Scheduled Refresh to trigger the Dataflows.  I.e. start them all at 08:30.  If so, I would suggest switching to use Power Automate.  There is a Action (Start Dataflow Refresh) and Trigger (When Dataflow Refresh Ends) connectors that will allow you to run the Dataflows in a specific order or in parallel.  Here is video to get you started.
How to Refresh a Dataflow Using Power Automate 

[Drunken_Sailor]

Dataflow A is scheduled to start at 8:30.  That then triggers B and C to start at the same time.  If instead I use Power Automate to trigger A, i get the same result. B and C will also get triggered at the same time.

Sure, I could then use "when a dataflow refresh completes" to trigger B (which then triggers C) when A is finisshed, but that just means B gets refressed twice. If I then use another flow to trigger c when B is done, C get s triggered three times.  
In this scenario, it is not a big deal, as the refresh cycles are only a few seconds, but I have others that take 2 to 3 hours each. They make API calls to a service that also limits how many you can do concurrently. Hence why I need these to be in a series and not in parrallell.