cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
fabiolamelo
Helper II
Helper II

Incremental Refresh by Sequential Number - Big Data

Hi everyone,

 

Is it possible use the Incremental Refresh using the sequencial number instead of date?

 

I am working in one project that using data with trillions of line and when the module is update the server crash.
I was using the transaction date as a parameter to do the incremental refresh and unfortunatelly for a lot of reason this date can not be using as a parameter. So, the DBA told me that the best away to refresh the data is by using the sequencial number as a paramter.


I need to find one solution to store the rows from the last 4 or 5 years and just refresh the data based on the last sequencial number. As I can show in the example below:

 

COMPANY |TRANSACTION DATE |SEQUENCIAL NUMBER|AMOUNT
TEST1 |01-01-17 |127953 | 1
TEST1 |01-01-17 |127954 | 1
TEST1 |01-01-17 |127955 | 1
TEST1 |01-01-18 |127956 | 1
TEST1 |01-01-18 |127957 | 1
TEST1 |01-01-18 |127958 | 1
TEST1 |01-01-19 |127959 | 1
TEST1 |01-01-19 |127960 | 1
TEST1 |01-01-19 |127961 | 1
TEST1 |01-01-20 |127962 | 1
TEST1 |01-01-20 |127963 | 1
TEST1 |01-01-20 |127964 | 1
TEST1 |01-01-21 |127965 | 1
TEST1 |01-02-21 |127966 | 1
TEST1 |19-04-21 |127967 | 2
TEST1 |19-04-21 |127968 | 2

 

TEST1 |20-04-21 |127969 | 2


On this cenario the last sequencial was 127968 and I whould like that the refresh just happend from
the next sequencial number which is "127969" . So the module should keep all the data from 127953 to 127968 and refresh just the next sequencial number.

 

I tried to use a incremental refresh by using the sequencial number as parameter list but it dont works, or I didnt did the correct steps. 

 

Is the are any solution for this case?

 

Can you please help me? I`m stuck trying to find this solution

Note: As the information is confidential I cant share the data.

 
Thanks

1 REPLY 1
v-kelly-msft
Community Support
Community Support

Hi @fabiolamelo ,

 

I'm afraid it can not be simply done by incremental refresh.Try to realize it in sql server or other database,then connect the updated data to power bi .

 

Best Regards,
Kelly

Did I answer your question? Mark my post as a solution!

 

Helpful resources

Announcements
PBI User Groups

Welcome to the User Group Public Preview

Check out new user group experience and if you are a leader please create your group!

MBAS on Demand

Microsoft Business Applications Summit sessions

On-demand access to all the great content presented by the product teams and community members! #MSBizAppsSummit #CommunityRocks

MBAS Attendee Badge

Claim Your Badge & Digital Swag!

Check out how to claim yours today!

secondImage

Are You Ready?

Test your skills now with the Cloud Skills Challenge.

Top Solution Authors