Dataset pulling from Dataflows running really slow in Service

[BertieGG]

Hi All,

I have a lovely Star Schema model running on a heap of DataFlows which takes about a minute to refresh all the data in the desktop (PBIX) but appears to be taking over an hour to refresh in the workspace.

I've found lots of materials on performance checking the PBIX, and articles about migrating to DataFlow to enhance speed, but nothing about the performance once live.  Any help would be much appreciated.

Thanks in advance

[Icey · Accepted solution]

Hi @BertieGG ,

Is the Dataset you are refreshing on a Pro, PPU or Premium Capacity Workspace?

If you're on a Pro workspace, you are sharing the Pro capacity with all other Pro users on the Microsoft Cloud. You don't see them or their data, and neither do they see you, but you're all using the finite resources Microsoft have put in place to service Pro users. I don't know how many users that is (Microsoft don't release numbers) but in 2016 it was over 5 million. That's why when you schedule a refresh your request is queued and may be run up to 1 hour later. It's not just your refresh that the service is running, but ALL the refreshes requested by ALL the Pro users. (Yes Microsoft splits this up geographically, but we don't know how many are on each Geography, just like we don't know how many in total). Thats why Refreshing on a Pro workspace takes a variable amount of time. Microsoft add servers and improve the service, but more users join and more features are provided that use up Server time. I guess for Microsoft it's a compromise between having millions of Servers standing idle just in case, or Users complaining that Refreshes take more than an hour.

Your alternatives are to use a Premium Capacity or Premium Per User (PPU) backed Workspace. Premium Capacity can be thought of as renting your own server from Microsoft in the cloud. It costs more, but you get a dedicated capacity that only you use, so you don't have these service contention problems. PPU offers many benefits of Premium Capacity, but at a lower per-user cost. PPU runs on faster servers and has fewer users, so in many cases will give you a faster refresh.

Reference: Solved: Re: (2MB) PBI Refresh via Webservice - Extrem slow - Microsoft Power BI Community

Best Regards,

Icey

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[BertieGG]

Many thanks for your response @Icey 

I think you've hit the nail on the head.  I just couldn't seem to find that info when I was looking for it originally.  I had heard about the "Noisy Neighbours" effect, but somehow slipped my mind in this context.