Skip to main content
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Grow your Fabric skills and prepare for the DP-600 certification exam by completing the latest Microsoft Fabric challenge.

Reply
DarinSpence
Helper II
Helper II

Data Flow Gen2 Incremental Refresh

‎08-12-2023 06:06 PM

I have some dimension tables with a Time Updated column in Snowflake.  I'd prefer to SELECT from Snowflake only the rows with a Time Updated since the last refresh; and UPDATE the table in the Lakehouse.

 

Is there a method with Data Flow Gen2 to do something like Data Flow Gen1's incremental refresh?

Labels:
Message 1 of 3
230 Views
0
2 REPLIES 2
aj1973
Community Champion
Community Champion

‎08-12-2023 06:32 PM

Hi @DarinSpence 

In Power Query select the Date column and filtered to the last date.

Regards
Amine Jerbi

If I answered your question, please mark this thread as accepted
and you can follow me on
My Website, LinkedIn and Facebook

Message 2 of 3
217 Views
0
DarinSpence
Helper II
Helper II
In response to aj1973

‎08-12-2023 06:35 PM

When I tried that with Data Flow Gen2 Append, I got new; duplicate rows 😕

Message 3 of 3
206 Views
0

Helpful resources

Announcements
Europe Fabric Conference

Europe’s largest Microsoft Fabric Community Conference

Join the community in Stockholm for expert Microsoft Fabric learning including a very exciting keynote from Arun Ulag, Corporate Vice President, Azure Data.

RTI Forums Carousel3

New forum boards available in Real-Time Intelligence.

Ask questions in Eventhouse and KQL, Eventstream, and Reflex.

View All
Top Solution Authors
View All