Dataflow refresh takes more time than usual dataset refresh for same query

[vmakhija]

Hello

I am trying out dataflows and building some POCs.

I have a dataflow where I have pasted the query from Desktop to create a query.

However, when refreshing the dataflow, it takes a lot more time than the dataset which is using the exact same query.

Note that dataset is not pointing to dataflow yet. Both are using the same query.

In fact, Dataflow refresh takes more than double the amount of time it takes for dataset refresh

So, I would like to know if there is something which is slowing down the dataflow refresh. Is it something to do with memory alloted for dataflow as compared to dataset?

Regards

[GilbertQ]

Hi there,

It sounds like the dataflow is running via your On-Premise data gateway?
If so you could ensure that it is running the latest version?
That it also has enough hardware to handle the potential load.
You can also monitor the Gateway to see if there are any issues: https://docs.microsoft.com/en-us/power-bi/service-gateway-onprem-tshoot

[vmakhija]

@GilbertQ 

thanks for your response.

Yes, dataflow is using the on-prem gateway (note that dataset refresh is also using the same gateway).

I will check with my admin if we are using the latest version of on-prem gateway.

By enough hardware, do you mean enough hardware on the data source side or the server handling my gateway?

My concern is that dataset refresh is using exactly same hardware and gateway, but is performing better than my dataflow refresh.

And the difference is substantial. 

Regards

[GilbertQ]

Hi there,

I do know that going into the dataflow can be a bit slower due the way it integrates into the Azure Datalake. I know that they are working on improvements.

Just to make sure that your hardware on your gateway can handle multiple data refreshes at once.

[vmakhija]

@GilbertQ 

thanks for your reply.

Good to know that MS is working on the improvements for dataflows.

Regarding hardware, yes, it can handle multiple refreshes as there are cases when more than datasets are being refreshed at the same time.

Regards