I'm considering breaking one of my larger models into a series of Dataflows to improve the overall refresh performance of the model. I was wondering whether an individual dataflow had a size limit? Also if several dataflows are combined into a single model, will refreshing the model force a refresh of the associated dataflows or will they refresh independently based on their individual refresh frequencies?
The biggest limit I've run into is refresh time out (2 hours). I'm running a lot of Native Queries against AAS. These run slow as it is, but my feeling is that dataflows are slower than a regular service refresh. As a result, I've broken up a big model into entities in 5-6 different dataflows in the same workspace. Then I just staggered the refresh of each by two hours over the course of the night. Now everything runs fine.
Thank you. That confirms my thinking.
Does the refresh of the PBIX model force a refresh of the associated dataflows or will they refresh independently based on their individual refresh frequencies?
can you please explain the reason behind the suggestion not to append the results of the different dataflows / partitions?
I've successfully broken my large model out into a number of Dataflows and they are all refreshing on their own schedules 🙂 I am facing a new challenge in that any Dataflow that is over about 300MB will refresh in Dataflows and display in Power BI Desktop Query Editor but fail to load when I apply the query in Power BI Desktop. The error is:
Failed to save modifications to the server. Error returned: 'OLE DB or ODBC error: [DataSource.Error] Received an unexpected EOF or 0 bytes from the transport stream..'.
When applying it will happily load the data to a point - normally to the full size as it pauses for a long time at the same value and then crashes with the error. I'm going to break the larger items out into smaller Dataflows and then combine them as a new Table in Power BI Desktop. That should reduce the sizes being transferred over the wire.
Check out the News & Announcements to learn more.
Power Platform release plan for the 2021 release wave 2 describes all new features releasing from October 2021 through March 2022.
Microsoft received the highest score of any vendor in both the strategy and current offering categories.
DynamicsCon is a FREE, 4 half-day virtual learning experience for 11,000+ Microsoft Business Application users and professionals.