Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and a 50 percent discount on exams.
Get startedEarn a 50% discount on the DP-600 certification exam by completing the Fabric 30 Days to Learn It challenge.
I have two dataflows pointing to the same set of data. Dataflow A is on SQL server (on prem) and dataflow B is in multiple .parquet files that need to be combined into a single table.
When I run the dataflow A, it takes about 1minute and 30seconds to do a select * from the table. Below is a screenshot of the refresh time
When I run dataflow B, it takes 9-10minutes to get the same amount of data. Below is the screenshot of the power query:
Below is the refresh time (the top is using [HierarchicalNavigation=true] and the bottom query is without it)
A couple of things:
Why is my ALDS gen 2 load so much slower than on prem?
Solved! Go to Solution.
Turns out our best solution will be to spin up a vm, install a gateway on it, and use it for the dataflow refresh from alds gen2 datalakes.
Don't use an on-prem gateway for your Azure data sources. You may not even need a VNet gateway.
Turns out our best solution will be to spin up a vm, install a gateway on it, and use it for the dataflow refresh from alds gen2 datalakes.
FYI you cannot use VNet gateway for power bi dataflows.
What is a virtual network (VNet) data gateway (Preview) | Microsoft Learn
I will look into other options and get back t oyou