Earn a 50% discount on the DP-600 certification exam by completing the Fabric 30 Days to Learn It challenge.
I have a dataflow which queries a sql server database to extract 12 months worth of data. I use the native query option to run this so it fully folds.
The data is 1 minute interval and an incremental refresh runs several times a day to update.
As this is a large set of data and takes a long time to load into Power BI Desktop i wanted to create a subset containing 2 months of data. I would like this subset to query the dataflow created above and not the source sql server database.
Within the same workspace i created a dataflow from a linked entry and selected the above dataflow. I then selected reference and added a filter to the reference table. I also turned enable load off on the linked table as i don't want the user to see this.
A few questions,
1.Does this new dataflow query against the orginal instead of the source sql server database?
2. Do i need to schedule refreshes on the 2nd dataflow or will the data update when the 1st dataflow refreshes?
3. Is this the best practice for creating a subset and the most efficent in terms of performance?
I thought this should be a very simple process as there is minmal transformations but the Dataflow refreshes tend to fail regulalry with no helpful error messages.
Any help or feedback greatly appreciated.
Thanks for the response. This doesn't seem to be the case in my current process:
Latest data in 2nd dataflow
Latest data in 1st dataflow:
I was wondering if another part of my dataflow cause complications. I have several dataflows as inputs say dataflow a, b and c with separate refresh schedules, but each one relates to a different subset in dataflow d:
Should each filtered table automatically update when its related dataflow updates? If so this is not currently happening.
1. The new dataflow queries against the original dataflow (i.e. ADLS Storage Gen2 underneath), not the source SQL db.
2. If the linked entities are in the same workspace then refreshing dataflow #1 will trigger a refresh of its dependents: