Showing results for 
Search instead for 
Did you mean: 
Helper IV
Helper IV

Working with big datasets.



I need to create a new report and use incremental refresh in the future. For that I can not use custom SQL while pulling data from data warehouse because then incremental refresh will not work. Problem is I need data from tables that have data for 10+ years, I only need last 6 months worth of data. Can I filter the data in these tables without custom SQL before pulling the data? I can not overload this data warehouse pulling 10+ years of data and after that using filter. My only idea once again is creating a tabular model to another server but that's extra step for me for creating just 1 report.

Regular Visitor

Just create a sql view and select it like you would a table. Then incremental works just fine and can query fold succesfully.

If I create a view with a WHERE condition going back to almost a year doesn't that mean it still has to go through that same query that I am trying to avoid? This query takes too much time and resources on the data warehouse.

Super User
Super User

please provide more context for "I can not use custom SQL"


When you filter in Power Query it will attempt to push that filter upstream via "Query folding"  aka "native query".


please clarify " because then incremental refresh will not work " - that is not necessarily related.

With "Custom SQL" I meant when geting data. Get data-> Sql server. Under advanced options "SQL statement (optional, requires database)" I've learned that if I have anything written there then incremental refresh wouldn't work, so far this is the way I have been geting the data I need without downloading everything. Now I need another way since I have to get incremental refresh working.

Helpful resources

PBI_User Group Leader_768x460.jpg

Manage your user group events

Check out the News & Announcements to learn more.

Welcome Super Users.jpg

Super User Season 2

Congratulations, the new Super User Season 2 for 2021 has started!

Community Connections 768x460.jpg

Community & How To Videos

Check out the new Power Platform Community Connections gallery!