Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.

Reply
Anonymous
Not applicable

Filter through another Table without merging data?

Hi all,

 

Looking for some help on a question I cannot seem to find a direct answer to online.  I have a Excel 2016 workbook with PowerQuery connections that I am using to draw data from a ticketing system database.  In it, there are many instances where I need to filter one table, based on data from another table, before it is loaded into PowerPivot data model.  Is it possible in power query to filter without merging tables, or is that the only way?

 

Because the dataset is massive (over 1 - 2 mil records), I need to perform as much processing before I load it to PowerPivot, else it weighs down the excel workbook.

 

On a broader scale, this question really has to do with performance, both locally within the workbook as well as with regard to the SQL server.  I am a bit unclear as to the portion of processing that occurs on the SQL server side of things, and the portion that occurs in my local Excel client when I perform queries.  In a few of my workbooks, there are even local worksheets that I Power Query connect to for additional details to be entered as filters not provided by the database.

 

In providing PowerQuery as a solution for reporting needs to my manager, he asked that I identify how front-end/back-end heavy the queries are as they process.

 

To date, I have been merging tables left and right, then filtering as needed, but it got me wondering if I can filter data based off another table without having to merge.  Any info on the other questions would be greatly appreciated as well!

 

Sincerely,

1 ACCEPTED SOLUTION
ImkeF
Super User
Super User

Hi @Anonymous ,

if you're dragging data from a SQL server, then query folding should take place. That means that the data is filtered at the server side and only the filtered data gets loaded into PQ/PP. There are some limitations that are described here: https://www.mssqltips.com/sqlservertip/3635/query-folding-in-power-query-to-improve-performance/ 

But if you are inner joining against a non-SQL-source to filter the data, there is a catch and you need a workaround that I've described here: https://www.thebiccountant.com/2015/09/17/filter-sql-server-queries-with-excel-tables-query-folding-limitations-and-bug-workaround/

 

Imke Feldmann (The BIccountant)

If you liked my solution, please give it a thumbs up. And if I did answer your question, please mark this post as a solution. Thanks!

How to integrate M-code into your solution -- How to get your questions answered quickly -- How to provide sample data -- Check out more PBI- learning resources here -- Performance Tipps for M-queries

View solution in original post

1 REPLY 1
ImkeF
Super User
Super User

Hi @Anonymous ,

if you're dragging data from a SQL server, then query folding should take place. That means that the data is filtered at the server side and only the filtered data gets loaded into PQ/PP. There are some limitations that are described here: https://www.mssqltips.com/sqlservertip/3635/query-folding-in-power-query-to-improve-performance/ 

But if you are inner joining against a non-SQL-source to filter the data, there is a catch and you need a workaround that I've described here: https://www.thebiccountant.com/2015/09/17/filter-sql-server-queries-with-excel-tables-query-folding-limitations-and-bug-workaround/

 

Imke Feldmann (The BIccountant)

If you liked my solution, please give it a thumbs up. And if I did answer your question, please mark this post as a solution. Thanks!

How to integrate M-code into your solution -- How to get your questions answered quickly -- How to provide sample data -- Check out more PBI- learning resources here -- Performance Tipps for M-queries

Helpful resources

Announcements
Microsoft Fabric Learn Together

Microsoft Fabric Learn Together

Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City

PBI_APRIL_CAROUSEL1

Power BI Monthly Update - April 2024

Check out the April 2024 Power BI update to learn about new features.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors