cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Mr_Glister Regular Visitor
Regular Visitor

Split and append table to reduce data load?

Hi,

I'm loading sales data from a SQL server. 

My sales data goes over many years but there is no need to load all the data every time with each refresh.

 

I thought I would duplicate my original sales table, filtering one only for the years >2017 and then renaming this one Sales_historic and disable "include in data refresh".

Final step then to append the two tables again, leaving me with the same table as I orginially had but only a small part of it reloads on refresh.

Would that work? I'm asking because I tried and it looks like it's still loading as many rows as before...

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted
Super User
Super User

Re: Split and append table to reduce data load?

No, that won't work that way. What you are doing just prevents having duplicate rows in two different tables (if you never load the first query that is). Essentially, the Append statement is still going to execute the query you specify for your historic data. What I would suggest is that you have your two queries. Load the first and second queries. Then disable Load for your first query (historic). Then, use the UNION function in DAX to create a New Table that UNION's your two tables together. In this way, you will only get updates when you refresh data but you will have a single table that appends both together.


I have book! Learn Power BI from Packt


Did I answer your question? Mark my post as a solution!

Proud to be a Datanaut!

View solution in original post

2 REPLIES 2
Highlighted
Super User
Super User

Re: Split and append table to reduce data load?

No, that won't work that way. What you are doing just prevents having duplicate rows in two different tables (if you never load the first query that is). Essentially, the Append statement is still going to execute the query you specify for your historic data. What I would suggest is that you have your two queries. Load the first and second queries. Then disable Load for your first query (historic). Then, use the UNION function in DAX to create a New Table that UNION's your two tables together. In this way, you will only get updates when you refresh data but you will have a single table that appends both together.


I have book! Learn Power BI from Packt


Did I answer your question? Mark my post as a solution!

Proud to be a Datanaut!

View solution in original post

v-jiascu-msft Super Contributor
Super Contributor

Re: Split and append table to reduce data load?

Hi @Mr_Glister,

 

Please vote up this idea. Unfortunately, "include in data refresh" only works in the Desktop. I have tested it. I also tried to switch the refreshing using R code. But failed.

 

Best Regards,

Dale

Community Support Team _ Dale
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Helpful resources

Announcements
Can You Solve These Challenge

Challenge: Can You Solve These?

Find out how to participate in the first Power BI 'Can You Solve These?' challenge.

Community News & Announcements

Community News & Announcements

Get your latest community news and announcements.

Virtual Launch Event

Microsoft Business Applications October Virtual Launch Event

Join us for an in-depth look at the new innovations across Dynamics 365 and the Microsoft Power Platform.

Community Kudopalooza

Win Power BI Swag with Community Kudopalooza!

Each week, complete activities and be qualified in the drawing for cool Power BI Swag.

Users Online
Currently online: 6 members 3,416 guests
Please welcome our newest community members: