Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.

Reply
alisoroush7
Frequent Visitor

refreshing a huge number of rows in a table

hi guys,

the problem is i have a table that contains 20 million of records. every time i want to refresh the data i should wait about 30 to 45 minutes to complete. the question is that is there any way to update 5 last million records of a table? or any other solution to avoid waiting too much?

Thanks

1 ACCEPTED SOLUTION
v-henryk-mstf
Community Support
Community Support

Hi @alisoroush7 ,

 

There are a few options you can consider to try and reduce the time it takes to refresh your data in Power BI:

  • Use DirectQuery mode: Instead of importing all of the data into Power BI, you can use DirectQuery mode to connect directly to the data source and query the data on demand. This can be faster, especially if you have a fast, efficient data source and a well-designed database. However, it may not always be possible to use DirectQuery mode, depending on the data source and the requirements of your report.
  • Optimize the data source: If you are using an on-premises data source, there are several ways you can try to optimize the data source to improve performance. For example, you can try indexing the tables in the data source, or partitioning the data to spread it across multiple tables or servers.
  • Use incremental refresh: If your data source supports it, you can use incremental refresh to only refresh the data that has changed since the last refresh. This can be a significant time saver, especially if you have a large dataset that changes frequently.
  • Use a faster computer: If you are using a slower computer or one with limited resources (e.g. CPU, RAM), this can contribute to the slow refresh times. Upgrading to a faster computer with more resources may help.
  • Use a faster connection: If you are using a slow internet connection, this can also contribute to slow refresh times. Using a faster connection (e.g. a wired Ethernet connection instead of WiFi) may help.

If the problem is still not resolved, please provide detailed error information and let me know immediately. Looking forward to your reply.


Best Regards,
Henry


If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

View solution in original post

3 REPLIES 3
v-henryk-mstf
Community Support
Community Support

Hi @alisoroush7 ,

 

There are a few options you can consider to try and reduce the time it takes to refresh your data in Power BI:

  • Use DirectQuery mode: Instead of importing all of the data into Power BI, you can use DirectQuery mode to connect directly to the data source and query the data on demand. This can be faster, especially if you have a fast, efficient data source and a well-designed database. However, it may not always be possible to use DirectQuery mode, depending on the data source and the requirements of your report.
  • Optimize the data source: If you are using an on-premises data source, there are several ways you can try to optimize the data source to improve performance. For example, you can try indexing the tables in the data source, or partitioning the data to spread it across multiple tables or servers.
  • Use incremental refresh: If your data source supports it, you can use incremental refresh to only refresh the data that has changed since the last refresh. This can be a significant time saver, especially if you have a large dataset that changes frequently.
  • Use a faster computer: If you are using a slower computer or one with limited resources (e.g. CPU, RAM), this can contribute to the slow refresh times. Upgrading to a faster computer with more resources may help.
  • Use a faster connection: If you are using a slow internet connection, this can also contribute to slow refresh times. Using a faster connection (e.g. a wired Ethernet connection instead of WiFi) may help.

If the problem is still not resolved, please provide detailed error information and let me know immediately. Looking forward to your reply.


Best Regards,
Henry


If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Greg_Deckler
Super User
Super User

@alisoroush7 Yes, you can use Incremental Refresh. https://www.bing.com/ck/a?!&&p=1eb4908b95b30592JmltdHM9MTY3MTIzNTIwMCZpZ3VpZD0zMWJiZmJkOC1lYzJlLTY2N...

 


@ me in replies or I'll lose your thread!!!
Instead of a Kudo, please vote for this idea
Become an expert!: Enterprise DNA
External Tools: MSHGQM
YouTube Channel!: Microsoft Hates Greg
Latest book!:
The Definitive Guide to Power Query (M)

DAX is easy, CALCULATE makes DAX hard...

but incremental refresh is for premium users only!

 

Helpful resources

Announcements
Microsoft Fabric Learn Together

Microsoft Fabric Learn Together

Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City

PBI_APRIL_CAROUSEL1

Power BI Monthly Update - April 2024

Check out the April 2024 Power BI update to learn about new features.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.