Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn a 50% discount on the DP-600 certification exam by completing the Fabric 30 Days to Learn It challenge.

Reply
RobRayborn
Helper III
Helper III

Incremental Refresh

I have data that utilizes 4 SQL sources to create the final Table. Each one of those SQL sources has a Date column and the data for each of the 4 SQL sources go back years and years.
I only want the past 3 years total and only want the past two days updated.
Here's the question. Do I need to apply the 'RangeStart' and 'RangeEnd' to all 4 SQL sources or just the last Table that is Loaded?

2 REPLIES 2
TomMartens
Super User
Super User

Hey @RobRayborn ,

 

as always, it depends. When you apply the parameters "only" to the final SQL query, your dataset will contain exactly the data you want. But (as always), the preliminary queries will load all data. My assumption, the impact of incremental refresh on data load duration is limited and not as high as possible when the parameters are only set tot last SQL query.

 

Hopefully, this helps to tackle your challenge.

 

Regards,

Tom



Did I answer your question? Mark my post as a solution, this will help others!

Proud to be a Super User!
I accept Kudos 😉
Hamburg, Germany

I've attached the start and end date parameter to all the incoming data SQL tables that are transactional and will increae daily.  
I haven't had a proper test of this since one of my data sets is a Folder with Excel files and I can't seem to get the GD thing to utilize my On Premise gateway.
So I'll tell you how it goes once I can get that GD folder to refresh though my GD gateway or I decide to just cut that GD section out. 
Am I a little frustrated? GD right!

Helpful resources

Announcements
RTI Forums Carousel3

New forum boards available in Real-Time Intelligence.

Ask questions in Eventhouse and KQL, Eventstream, and Reflex.

LearnSurvey

Fabric certifications survey

Certification feedback opportunity for the community.

Top Solution Authors
Top Kudoed Authors