Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
eilina
Frequent Visitor

Dataset refresh process

Hi all,

After a long search in a lot of PBI documentation, I still couldn't find the answer on such question: "What is exactly happening with datasource when you press Refresh in PBI Service"? I understand the basic process (I hope at least) like that it sends PQ queries through the enterprise Gateway to the main datasource (we use Redshift) and then returns the results back and if refresh somehow fails, it magical way keeps the previous successfully refreshed data, but what about the details? What is happening with the dataset as an object in workspace during refresh: does it keep some copy of previous data refresh or it starts to update immidiately by getting the queries results - what if some error happens? How PBI recovers the previous dataset version?

Thank you in advance:)

Ekaterina.

1 ACCEPTED SOLUTION
GilbertQ
Super User
Super User

Hi there

 

At a very basic level this is how the Refresh process works

  • You click to start the refresh process.
  • In the background Power BI creates an empty copy of the table/dataset.
    • In this background table/dataset it loads all the new data.
  • When the refresh process is complete it then does a metadata switch (which happens in milliseconds) and switches the background table with the current table.
    • If there are queries happening it will wait for a specific amount of time before forcing the switch of tables.
  • That is why if the refresh fails the current table stays as is.

 

I hope that helps.





Did I answer your question? Mark my post as a solution!

Proud to be a Super User!







Power BI Blog

View solution in original post

1 REPLY 1
GilbertQ
Super User
Super User

Hi there

 

At a very basic level this is how the Refresh process works

  • You click to start the refresh process.
  • In the background Power BI creates an empty copy of the table/dataset.
    • In this background table/dataset it loads all the new data.
  • When the refresh process is complete it then does a metadata switch (which happens in milliseconds) and switches the background table with the current table.
    • If there are queries happening it will wait for a specific amount of time before forcing the switch of tables.
  • That is why if the refresh fails the current table stays as is.

 

I hope that helps.





Did I answer your question? Mark my post as a solution!

Proud to be a Super User!







Power BI Blog

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors