Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn a 50% discount on the DP-600 certification exam by completing the Fabric 30 Days to Learn It challenge.

Reply
AussieCoops
Regular Visitor

Datacleanse before or after appending tables or within same table?

I have five databases for my companies.  Each deals with a separate country/company.

Currently I Import the db from sqlserver one instance of SYSPRO with 5 db.

I am trying to speed things up a little.

So what is best?

Import each and do cleansing in each ===> then Append    OR

Import each without any cleanse and do all the cleaning up in the Appended table?

 

Currently I can View Native Query in the base imported tables right to the end of the query.

 

A third alternative is to write a longer single query bringing each db in, then Appending those within the same table using the technique mentioned by @v-sihou-msft  here:  https://community.powerbi.com/t5/user/viewprofilepage/user-id/6969 .   I have trialled this on a much smaller table and it seems fine.  Great tip btw.

Before I embark on a major writing exercise, any tips on speed issues:

1.  Import each table, cleanse then Append all in another table;

2.  Import basic tables, Append all in another table, then perform cleansing in the new table;

3.  Import using @v-sihou-msft method.

 

 

1 REPLY 1
az38
Community Champion
Community Champion

Hi @AussieCoops 

i have no investigated with digit but i think that 1 cleansing is better then 5, it should work a little bit faster as there will be less initializing procedures

and the second, and most important point - if you will discover some mistake or incompletness in your data after cleansing you will have to make change to the only datasource in the second option and not to the 5 different data sources with first option


do not hesitate to give a kudo to useful posts and mark solutions as solution
LinkedIn

Helpful resources

Announcements
PBI_APRIL_CAROUSEL1

Power BI Monthly Update - April 2024

Check out the April 2024 Power BI update to learn about new features.

Top Solution Authors
Top Kudoed Authors