Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.

Reply
Anonymous
Not applicable

Speed and power of data gateway and data refresh

Dear all,

 

I need to resolve some issues in my mind. If I put the datagateway on a faster more powerful machine will that mean data refresh will be quicker than on a slower machine, as sometime we have some issues with queries timing-out.

 

Or is the speed of data refresh determined by the microsoft cloud.

 

Thank you.

1 ACCEPTED SOLUTION
Anonymous
Not applicable

The answer is "Maybe".  There are several factors that contribute to how long a refresh will take.  Yes a faster gateway will perform its operations faster.  The questions that matter here will come down to things like:

 

  • How big is the data source being refreshed?
  • How complex are the operations be performed on the data?
  • What resource bottlenecks is the Gateway facing?
  • Where is the Gateway compared to the data sources?
  • What network bottlenecks exist between the Gateway and its data sources?
  • What network bottlenecks exist between the Gateway and the Power BI Service?

 

As you can see, the power of your machine could be a bottleneck.  There could also be multiple bottlenecks. 

View solution in original post

4 REPLIES 4
v-shex-msft
Community Support
Community Support

Hi @Anonymous,

 

You can install your data gateway at the device which install the datasource, it will improve the refresh performance without update hardware.

 

Regards,

Xiaoxin Sheng

Community Support Team _ Xiaoxin
If this post helps, please consider accept as solution to help other members find it more quickly.
Anonymous
Not applicable

Thank you all for your responses. That answers my questions.

 

It is the processing that is taking the time I am pretty sure as I have a lot going on in power query, and also in DAX, doing row by row calculations of running totals to calculate cummulative totals that need to reset at the beginning of new events through a large table. And also needing to look up against the other tables to join the result of the before mentioned calculations against rows in another table based on whether a number of criteria, such as being within a certain date range while also being in a particular room, are matched.

 

The actually data pull doesn't take that long. We are going to put the enterprise gateway in our Data Centre as we move to a larger rollout, I have been using the Personal Gateway so far as we have tested the viability of Power BI for larger projects.

 

The main challenge using power BI is doing computations over large data sets in terms of refreshes timing out, especially as the data models grow and require more ETL to bring in different data sources. Maybe incremental data loading might help. But I also think I will need to move the analytic side of the building of the data model into a more robust system, and then use power bi to just point at already processed data, apart from where the processing is minimal in nature.

GilbertQ
Super User
Super User

Hi @Anonymous

 

One of the questions is when you process the data using Power BI Desktop does it take a long time to process the data?

 

And when it is processing the data, does your memory consumption and or your CPU max out?

 

If the above is the case, then yes having a higher specification server will ensure that it takes a lot quicker to run overall. 

 

You also need to check to ensure that your connectivity is good, and how far the data centre is from your server.

 

And how big is your Power BI Desktop file is, as that is the size that would need to be uploaded.





Did I answer your question? Mark my post as a solution!

Proud to be a Super User!







Power BI Blog

Anonymous
Not applicable

The answer is "Maybe".  There are several factors that contribute to how long a refresh will take.  Yes a faster gateway will perform its operations faster.  The questions that matter here will come down to things like:

 

  • How big is the data source being refreshed?
  • How complex are the operations be performed on the data?
  • What resource bottlenecks is the Gateway facing?
  • Where is the Gateway compared to the data sources?
  • What network bottlenecks exist between the Gateway and its data sources?
  • What network bottlenecks exist between the Gateway and the Power BI Service?

 

As you can see, the power of your machine could be a bottleneck.  There could also be multiple bottlenecks. 

Helpful resources

Announcements
Microsoft Fabric Learn Together

Microsoft Fabric Learn Together

Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City

PBI_APRIL_CAROUSEL1

Power BI Monthly Update - April 2024

Check out the April 2024 Power BI update to learn about new features.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors