Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
mcolb88
Helper III
Helper III

Diagnosing Cause of Wait Times in Data Flow Refresh

I have dataflows that will intermittently taking significantly longer than normal and I need to determine the casue so it can be fixed, if possible. I have a dataflow that reads items from Sharepoint lists, that are shared among several datasets. The dataflow ususally refreshing in under 20 seconds.  It's refreshed every 30 minutes between 7am and 7 pm. Starting today, it ran for 0:00:19, then 0:03:50, then 0:56:46, then 0:12:10, then 0:00:47 and then a killer at 5:14:27!!! 

 

I checked the log and see significant wait times with really short druations, no one query is more than 20 seconds, with wait times as long as 59 minutes.

 

mcolb88_0-1641504725939.png

This doe snot use a gateway and is soruced completely from SPO data. I have dataflow with long wait times to on-prem SQL and HANA that use a gateway. Any information on how to dig into these to determine the cauase is greatly appreciated. 

 

This is in a P3 premium capacity and I dont have Power BI admin access.

 

Thanks 

2 REPLIES 2
v-rongtiep-msft
Community Support
Community Support

Hi @mcolb88 ,

There are many reasons for the slow refresh. Please check.

  • Insufficient CPU (refresh can be very CPU-intensive).
  • Insufficient memory, resulting in refresh pausing (which requires the refresh to start over when conditions are favorable to recommence).
  • Non-capacity reasons, including datasource system responsiveness, network latency, invalid permissions or gateway throughput.
  • Data volume - a good reason to configure incremental refresh, as discussed below.

You can refer to the following document about how to mitigate long refresh durations.

https://docs.microsoft.com/power-bi/transform-model/dataflows/dataflows-understand-optimize-refresh#...

 

Best Regards

Community Support Team _ Polly

 

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly

lbendlin
Super User
Super User

dataflows are notoriously hard to troubleshoot.  One alternative is to create an equivalent query in a dataset and then use the extensive diagnostics tools both in the Power Query editor and with the Event tracker tools presented by Phil Seamark,

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors