Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
S_loke
Advocate I
Advocate I

Query folding using Dataflow as datasource - Is it possible? On Roadmap?

Similar to the issues reported on https://community.powerbi.com/t5/Desktop/Dataflows-don-t-support-Query-Folding/td-p/673500  Unfortunately, the tread seem to have gone into different direction, proving that query folding is possible on select datasources and how to use it but have not answered the original question of query folding using dataflows as a datasource.

 

We have implemented dataflows within a P1 capacity that represents a golden dataset.  As such, the size of the dataflow is rather large (6gb).  The intent is for users to apply a filter on the dataflow to retrieve only relevant results.

 

However, when using the on desktop, query folding doesn't seem possible as the data retrieved initially is the 6gb and the filter condition processed on desktop thereafter.  Obvious implication is the time/resources it takes to download the entire dataflow only to remove the majority when applying the filter.

 

The immediate questions

  1.   Is query folding using dataflows as source data possible (i.e. filter records on dataflow)
  2.   If query folding using dataflows is not currently possible, is this on the roadmap (MBAS suggest major performance improvements in dealing with large data volume within dataflows so hoping that query folding capability ships with it!)
1 ACCEPTED SOLUTION
v-shex-msft
Community Support
Community Support

HI @S_loke ,

Based on test, I haven't found that option on dataflow side and it means this feature still not available on dataflow.
In addition, I also checked plan list but haven't found it. You can consider to submit an idea to increase priority of this feature.

What’s new and planned for business intelligence

Regards,

Xiaoxin Sheng

Community Support Team _ Xiaoxin
If this post helps, please consider accept as solution to help other members find it more quickly.

View solution in original post

8 REPLIES 8
Lunar_sea
Frequent Visitor

Hi Microsoft,

 

I'm also looking for an offical update with respect to Query Folding for dataflows.

I'd really like this too. But how coudl it be done? isn't a dataflow just a text file on Azure Data Lake under the covers? I can see how you'd fold back to a relational dB, and (especially if your indexes / partitions were set up nicely in the back end) it would limit the rows at source. But how would you do that against a file on ADLS?

 

I wonder if they'd do something like a DirectQuery, where you'd target the Dataflow, but it would really fold the query back to the relational store behind the dataflow...

Hi,

 

I guess this is where the new engine kicks in.  The whitepaper suggest that when using dataflows, there are instances of SSAS that are deployed in the backend so in theory you could do query folding against that instance.  The question now is how to trigger this because it is a temporary instance.  So far in my testing, i haven't been able to isolate the steps to take advantage of this but hopefully more information will come through to assist in the near future.

nickyvv
Community Champion
Community Champion

Matthew Roche has an excellent post about Query Folding and Dataflows:

https://ssbipolar.com/2019/10/12/power-bi-dataflows-and-query-folding/

The summary:

  1. When accessing dataflow entities from Power BI Desktop, no query folding takes place, even if the enhanced compute engine is enabled.
  2. When accessing dataflow entities from other entities in the Power BI service, no query folding takes place unless the enhanced compute engine enabled.
  3. When accessing dataflow entities from other entities in the Power BI service, query folding will take place when the enhanced compute engine is enabled, because the linked entity’s query will be executed against the cached data in SQL, rather than the underlying CDM folder.


Did I answer your question? Mark my post as a solution!

Blog: nickyvv.com | @NickyvV


Thanks - the more information that is becoming avaliable, the more it makes sense.

 

Point 3 suggests that down the line query folding from desktop could be made possible.  PBI already have some examples of similar concepts (query caching for datasets)

 

fingers crossed that this somehow makes it onto their roadmap!!

v-shex-msft
Community Support
Community Support

HI @S_loke ,

Based on test, I haven't found that option on dataflow side and it means this feature still not available on dataflow.
In addition, I also checked plan list but haven't found it. You can consider to submit an idea to increase priority of this feature.

What’s new and planned for business intelligence

Regards,

Xiaoxin Sheng

Community Support Team _ Xiaoxin
If this post helps, please consider accept as solution to help other members find it more quickly.

Thank you - not the answer i was hoping for but at least i can now plan for alternative actions

nickyvv
Community Champion
Community Champion

Is there an official update on Query Folding for dataflows?



Did I answer your question? Mark my post as a solution!

Blog: nickyvv.com | @NickyvV


Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors