cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Helper III
Helper III

scheduling ingestion dataflow, transformation dataflow, then final dataset

Hi, lets say i have an ingestion dataflow that takes 5 mins to run, then a transformation dataflow that has its source as the ingestion one (and it takes 5 mins to run) and then lastly a dataset that has its source as the transformation one (and it takes 5 mins to run).

 

Now, if you could somehow chain these together - it would take a total of 15 mins.

 

If you use the interface to schedule these refreshes, you have to keep to 30 min time boundaries so am i right that the quickest you could get these synched is 65 mins ?

 

Ingestion at time =0

Transforamtion at time = 30 mins

Data set at time = 60 mins (and then takes 5 mins to complete).

 

(this was not meant to sound like one of those facebook adding up quizzes...)

 

sb

 

 

 

1 REPLY 1
Community Support
Community Support

Hi @frano72 ,

 

Based on my refresh, we can only set scheduled refresh in 00/30 minutes under shared capacity, also notice that,  for the configured scheduled refreshthe target is to initiate the refresh within 15 minutes of the scheduled time slot, but a delay of up to one hour can occur if the service can't allocate the required resources sooner (30 mintues between each refresh should enough). So if set scheduled refresh for those dataflow and dataset, it may take 30+30+30+5=65 minutes or longer to complete entire refresh process,  if you want to ensure it can refresh success. So it seems combine some of them will be a greate idea. But we can also use Refresh Dataflow API and Refresh Dataset API (or this for app workspace) to trigger on-demand refresh at any time and execute right after api call, but notice that  In Shared capacities this call is limited to eight times per day (including refreshes executed via Scheduled Refresh)


Best regards,

 

Community Support Team _ Dong Li
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Helpful resources

Announcements
Community Conference

Power Platform Community Conference

Check out the on demand sessions that are available now!

Community Conference

Microsoft Power Platform Communities

Check out the Winners!

secondImage

Create an end-to-end data and analytics solution

Learn how Power BI works with the latest Azure data and analytics innovations at the digital event with Microsoft CEO Satya Nadella.

Top Solution Authors