Reply
Highlighted
Frequent Visitor
Posts: 10
Registered: ‎02-23-2017
Accepted Solution

Split and append table to reduce data load?

Hi,

I'm loading sales data from a SQL server. 

My sales data goes over many years but there is no need to load all the data every time with each refresh.

 

I thought I would duplicate my original sales table, filtering one only for the years >2017 and then renaming this one Sales_historic and disable "include in data refresh".

Final step then to append the two tables again, leaving me with the same table as I orginially had but only a small part of it reloads on refresh.

Would that work? I'm asking because I tried and it looks like it's still loading as many rows as before...


Accepted Solutions
Super User
Posts: 10,476
Registered: ‎07-11-2015

Re: Split and append table to reduce data load?

No, that won't work that way. What you are doing just prevents having duplicate rows in two different tables (if you never load the first query that is). Essentially, the Append statement is still going to execute the query you specify for your historic data. What I would suggest is that you have your two queries. Load the first and second queries. Then disable Load for your first query (historic). Then, use the UNION function in DAX to create a New Table that UNION's your two tables together. In this way, you will only get updates when you refresh data but you will have a single table that appends both together.


Did I answer your question? Mark my post as a solution!

Proud to be a Datanaut!


View solution in original post


All Replies
Super User
Posts: 10,476
Registered: ‎07-11-2015

Re: Split and append table to reduce data load?

No, that won't work that way. What you are doing just prevents having duplicate rows in two different tables (if you never load the first query that is). Essentially, the Append statement is still going to execute the query you specify for your historic data. What I would suggest is that you have your two queries. Load the first and second queries. Then disable Load for your first query (historic). Then, use the UNION function in DAX to create a New Table that UNION's your two tables together. In this way, you will only get updates when you refresh data but you will have a single table that appends both together.


Did I answer your question? Mark my post as a solution!

Proud to be a Datanaut!


Community Support Team
Posts: 7,619
Registered: ‎05-02-2017

Re: Split and append table to reduce data load?

Hi @Mr_Glister,

 

Please vote up this idea. Unfortunately, "include in data refresh" only works in the Desktop. I have tested it. I also tried to switch the refreshing using R code. But failed.

 

Best Regards,

Dale

Community Support Team _ Dale
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.