cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
mcolb88
Helper III
Helper III

Diagnosing Cause of Wait Times in Data Flow Refresh

I have dataflows that will intermittently taking significantly longer than normal and I need to determine the casue so it can be fixed, if possible. I have a dataflow that reads items from Sharepoint lists, that are shared among several datasets. The dataflow ususally refreshing in under 20 seconds.  It's refreshed every 30 minutes between 7am and 7 pm. Starting today, it ran for 0:00:19, then 0:03:50, then 0:56:46, then 0:12:10, then 0:00:47 and then a killer at 5:14:27!!! 

 

I checked the log and see significant wait times with really short druations, no one query is more than 20 seconds, with wait times as long as 59 minutes.

 

mcolb88_0-1641504725939.png

This doe snot use a gateway and is soruced completely from SPO data. I have dataflow with long wait times to on-prem SQL and HANA that use a gateway. Any information on how to dig into these to determine the cauase is greatly appreciated. 

 

This is in a P3 premium capacity and I dont have Power BI admin access.

 

Thanks 

2 REPLIES 2
v-polly-msft
Community Support
Community Support

Hi @mcolb88 ,

There are many reasons for the slow refresh. Please check.

  • Insufficient CPU (refresh can be very CPU-intensive).
  • Insufficient memory, resulting in refresh pausing (which requires the refresh to start over when conditions are favorable to recommence).
  • Non-capacity reasons, including datasource system responsiveness, network latency, invalid permissions or gateway throughput.
  • Data volume - a good reason to configure incremental refresh, as discussed below.

You can refer to the following document about how to mitigate long refresh durations.

https://docs.microsoft.com/power-bi/transform-model/dataflows/dataflows-understand-optimize-refresh#...

 

Best Regards

Community Support Team _ Polly

 

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly

lbendlin
Super User
Super User

dataflows are notoriously hard to troubleshoot.  One alternative is to create an equivalent query in a dataset and then use the extensive diagnostics tools both in the Power Query editor and with the Event tracker tools presented by Phil Seamark,

Helpful resources

Announcements
August 2022 update 768x460.jpg

Check it Out!

Click here to learn more about the August 2022 updates!

August 1 episode 9_no_dates 768x460.jpg

The Power BI Community Show

Watch the playback when Priya Sathy and Charles Webb discuss Datamarts! Kelly also shares Power BI Community updates.

Power Platform Conf 2022 768x460.jpg

Join us for Microsoft Power Platform Conference

The first Microsoft-sponsored Power Platform Conference is coming in September. 100+ speakers, 150+ sessions, and what's new and next for Power Platform.

Top Solution Authors