Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.
I've set up a dataflow that pulls selected tables from our ERP SQL.
When I connect to the dataflow from PBI Desktop it loads several GB's of data even though I'm selecting only a few 100 lines from two tables.
If I connect directly to the SQL I can load the lines in seconds. Using the dataflow it takes several minutes.
Am I doing something wrong?
My approach in both cases is the same:
Solved! Go to Solution.
Yes. When you use a Dataflow as a source, to get folding, the Enhanced Compute Engine has to be turned on (premium capacity) and you need to use the Dataflows connector (not the Power BI Dataflows one).
Pat
To learn more about Power BI, follow me on Twitter or subscribe on YouTube.
Thanks @mahoneypat
Could you point me one step further?
Google is giving me lots of contradictory information about query folding regarding dataflows. I don't know the term, so don't know what I'm looking for.
One thing I see mentioned is the "Enhanced Compute Engine". The dataflow doesn't sit within a premium capacity. Is this the root of my issue, perhaps?
Yes. When you use a Dataflow as a source, to get folding, the Enhanced Compute Engine has to be turned on (premium capacity) and you need to use the Dataflows connector (not the Power BI Dataflows one).
Pat
To learn more about Power BI, follow me on Twitter or subscribe on YouTube.
Thanks. So outside of Premium, Dataflows isn't really a viable option, since any datasets will have to pull entire tables every time.
There is likely something breaking "query folding" in your query. The use of a SQL statement returns only the desired rows, but you should be able to get similar refresh time if you maintain query folding. There are indicators in the query editor for Dataflows to show if it is in place or not, and you can modify/rearrange your steps (e.g., do filtering and column selection first) to potentially maintain it.
Pat
To learn more about Power BI, follow me on Twitter or subscribe on YouTube.
Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City
Check out the April 2024 Power BI update to learn about new features.
User | Count |
---|---|
114 | |
99 | |
75 | |
73 | |
49 |
User | Count |
---|---|
145 | |
109 | |
109 | |
90 | |
64 |