Scheduled Refresh not updating with latest data despite refresh completing sucessfully

[ruiz07s]

Hi everyone, long-time Power BI user. I have a Data Flow which connects to SQL Server with Gateway. This Data Flow is setup with Scheduled Refresh which successfully completes each morning. The issue I have is my Dataset published in Service (which uses the before-mentioned Data Flow as a source) does not pick up updates. When I refresh the report manually in Desktop however, I can see the updates which means the Data Flow does indeed refresh as expected. The issue I believe lies with Service / or the dataset in Service. Anyone have any ideas or encountered a similar issue?

**Update: Solution Found! (May 4, 2024): I ignored this issue until I could no longer. Through a long process of elimination, I found the issue / solution and it's a very simple one.

**Problem Recap: My dataflows were refreshing without error, but when I refresh my dataset in Power BI Service, my data did not change despite the dataset refreshing successfully. I note that some users suggested it could be an issue of scheduling, but this was not the case in my instance.

**Issue / Solution: The issue was that I intentionally disabled "Include in report refresh" for my fact table in Power Query. The reason I did this is my fact table contains a large number of transactions 100M+ rows and I have many dim / mapping tables which I prefer to refresh (separately from the fact table) for convenience. All tables are built in dataflows and I would typically refresh my  fact table manually in Power BI desktop. By unticking this, my fact table was excluded from refresh in Power BI Service (which makes total sense but which was something I completely overlooked. After ticking this and publishing out to service, my dataset refreshed as expected.

[v-yueyunzh-msft]

Hi , @ruiz07s 

According to your problem description, your dataset is connected to a dataflow created in a service, and the scheduled refresh of the dataflow is normal every day, but the data in your dataset is not refreshed, and you can refresh the data normally on the desktop. I think the root cause of your problem is that you didn't configure scheduled refresh for this dataset, because even if your dataflow data is kept up to date every day, your dataset needs to be refreshed periodically to get the latest data from the dataflow. So my point is that you only need to configure credentials for your dataset and turn on scheduled refresh to keep your data up to date every day.

For more information, you can refer to :
How to configure Power BI report scheduled refresh - Power BI | Microsoft Learn

Thank you for your time and sharing, and thank you for your support and understanding of PowerBI! 

Best Regards,

Aniya Zhang

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly

[ruiz07s]

Hi Aniya, thanks for your reply. Please let me clarify, I do in fact have the dataset refreshing on a daily schedule that's timed for 1hr following the dataflow refreshes. The issue is the dataset is not actually refreshing, despite the refresh showing as successful. Even if I choose on-demand refresh multiple times, it does not pull in the updated data from the dataflows. Is there anything else it could be?

[Anonymous]

Hi, have you figured out a solution for this problem? I am having the same issue and was wondering if you could share any insights. Thanks!

[ruiz07s]

Hi, I abandond this issue but recently revisited it and I found the issue and solution. I edited my original post with the solution, hopefully this solves your problem also.

[v-yueyunzh-msft · Accepted solution]

Hi , @ruiz07s 

According to your description, your dataset is connected to your Dataflow, and you configured your dataset refresh correctly an hour after the DataFlow refresh was completed. And the dataset refresh was successful.
Under normal circumstances, this should not be a problem. First of all, you need to check if your dataflow has new data generated, you can enter your dataflow after your dataflow has been successfully refreshed to see if new data has been generated.

Secondly, can you check if some functions that generate today's date are used in your dataset? Because scheduled refresh in Power BI Service is using the UTC+0 time zone. If so, you can connect to your dataflow directly using the Dataset, then publish and refresh it to see if the refreshed data is the latest data.

Finally, you need to check if refresh for that query is enabled in your Power Query.

For more information, you can refer to :
Managing query refresh - Power BI | Microsoft Learn

Thank you for your time and sharing, and thank you for your support and understanding of PowerBI! 

Best Regards,

Aniya Zhang

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly