dataflow long time to refresh and not responding to 'Cancel refresh' request

[anselmojg]

Hello,

One of our dataflows is taking a very long time to refresh today.. It normally takes less than 4 minutes but today it is still refrshing after more than FOUR hours (see capture below)

The Cancel Refresh manual request fails with this message:

Something went wrong
The request to cancel refresh couldn't be sent. This may be due to a server problem.
Please try again later or contact support. If you contact support, please provide these details.
Activity ID: 07d32449-88da-44d3-8b8c-bf70e05143cd
Request ID: 596bbb3c-7cba-9fd3-39b9-8445506be894
Correlation ID: 9d7550c9-f2a9-6f22-6c30-24bc9e56acb5
Status code: 500
Time: Mon Mar 25 2019 19:08:21 GMT+0100 (Central European Standard Time)
Version: 13.0.8885.144
Cluster URI: https://wabi-us-north-central-redirect.analysis.windows.net

[Anonymous]

If you are Premium would check if there is anything else refreshing at the sametime. If not the case then check your capacity memory usage. 

[Anonymous]

Hey Buddy,

Yes I am in Premium Capacity . However, there are lot of reports(Say at least 20)  that are scheduled to refresh multiple times in a day.  Is there a way to refresh data flows in parallel and not getting affected in terms of speed ? 

Please assist !!! 

[GilbertQ]

Hi there

You would need to make sure that there is enough capacity CPU & Memory available for the dataflow to run as fast as possible.

What happens if you refresh the dataflow when there is nothing else running?

[eduardosoares96]

What would be the minimum upload speed needed to update dataflows?

[Anonymous]

If you are on Premium capcaity try restarting the capacity

[Anonymous]

So  long as there is no time out on the data source end that would close the connection, you should be ok. I have never ran into a issue with uploads speeds even with a 5 meg upload internet connection. It may take longer but it works. Sometimes if you have incremental refresh set or big tranfermation steps in the dataflow that can cause long refresh times as well. Not sure if this was helpful 

[GilbertQ]

Hi there

It could possibly be with an issue on the On-Premise Data Gateway, there is a new version to install to if you installed the March version

[anselmojg]

Thanks Gilbert,

The gateway does not seem to be the culprit here since all data sources are excel workbooks hosted in SharePoint Online. The refresh continues and has now taken more than 18 hours!

The (I guess new) 'Cancel refresh' command still does not work.

I opted to create a new dataflow with the exact same entities and it just worked fine.... 

Somehow the original dataflow refresh has gone into some sort of infinite loop and does not seem to be timing out, which raises the question on how to stop manually a refresh????

thoughts?

[anselmojg]

The dataflow refresh finally completed after more than TWENTY ONE HOURS!!!

Just trigerred a manual refresh and took the usual time (under two minutes)

So the outstanding question is to understand why that particular dataflow refresh took so long and why it could not be stopped? It caused a significant issue as it affected the refresh of a couple of datasets that consume that dataflow. The remedy was to recreate another dataflow and re-connect those datasets to the new dataflow.... This is something that I would like to avoid....

[GilbertQ]

Hi there

Currently with it being V1 I am sure that things will improve.

I am also sure that there might be an idea to get dataflows to be stopped which you could find at https://ideas.powerbi.com

[Anonymous]

Team,

Did it ever got resolved ? I am also using dataflows and converted most of desktop report to have dataflow as datasource. 

Lot of users complain about the updates not being reflected in reports . When i looked at the issue, i found that data flows takes hours to complete refresh. Is it something that can be fixed ? Does it needed to change or look at the issue as resource consumption or does it require any extra hardward to compute things faster ?

Please suggest as lot of reports are not working fine because of this issue.