Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
rafaelmorenopbi
New Member

Incremental refresh and query folding with AWS redshift as data source

Hi

 

I have a dataset importing data from a Data Warehouse. So far we operate in a full refresh mode, and I was willing to change to incremental refresh. However I read the documentation and it states that query folding is a pre-requisite for incremental refresh (makes sense), but so far I could not achieve query folding with AWS redshift as a data source. 

 

Has any of you made it? If so, what did you do in order to achieve success? 

 

I just want to implement incremental refresh for this particular dataset importing data from this specific data warehouse in redshift. 

 

Thanks in advance, 

2 REPLIES 2
v-rongtiep-msft
Community Support
Community Support

Hi @rafaelmorenopbi ,

Most data sources that have the concept of a query language support query folding. These data sources can include relational databases, OData feeds (including SharePoint lists), Exchange, and Active Directory. However, data sources like flat files, blobs, and web typically do not.

In addition, in the Power Query Editor window, it is possible to determine when a Power Query query can be folded. In the Query Settings pane, when you right-click the last applied step, if the View Native Query option is enabled (not greyed out), then the query can be folded.

If the option is greyed out, maybe you could change another data source which supports query folding.

 

I have found a similar post, please refer to the following document to see if it helps you.

https://community.powerbi.com/t5/Service/Incremental-refresh-with-Amazon-Redshift/m-p/1925887/highli...

  • use the ODBC connection

https://www.fourmoo.com/2019/08/20/using-azure-analysis-services-to-connect-via-an-odbc-source-redsh...

 

Best Regards

Community Support Team _ Polly

 

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

lbendlin
Super User
Super User

"query folding is a pre-requisite for incremental refresh"

 

That is not accurate. Query folding is desired but not required. Incremental refresh can work with any data sources that allow for RangeStart/RangeEnd filters, regardless of query folding.

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors