cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
DataGuy2016 Regular Visitor
Regular Visitor

Table Merge creates duplicate records

Hi Folks,

 

I am stumped.

 

I am trying to merge two tables into a new table using a LEFT JOIN.  Both tables have unique records on each row.  When I join the tables, BI creates duplicate rows on some records for no apparent reason.  The duplicates are identical in every way.

 

Can anybody explain this?  Is this a bug?

1 ACCEPTED SOLUTION

Accepted Solutions
DataGuy2016 Regular Visitor
Regular Visitor

Re: Table Merge creates duplicate records

The problem lay with the Data.

 

The LEFT JOIN functionality of PowerBI works perfectly.

22 REPLIES 22
v-ljerr-msft Super Contributor
Super Contributor

Re: Table Merge creates duplicate records

Hi @DataGuy2016,

 

According to your description, I just tested it in my environment. The Merge Queries option works all fine for me without the issue.

 

I assume the issue is with your data. Could you post your table structure with some sample data which can reproduce the issue? It's better to share a sample pbix file. You can upload it to OneDrive or Dropbox and post the link here. Do mask sensitive data before uploading.Smiley Happy

 

In addition, here is a similar thread for your reference.Smiley Happy

 

Regards

DataGuy2016 Regular Visitor
Regular Visitor

Re: LEFT JOIN creates duplicate records

Hi,

 

The LEFT JOIN table Merge is creating duplicate records from the LHS table. 

 

I understand what a LEFT JOIN is. 

 

My LHS table only has unique rows.  Why is PowerBI creating duplicate records?  These are 100% identical in every respect.

 

 

 

SQL Joins.pngSQL JOINS

DataGuy2016 Regular Visitor
Regular Visitor

Re: Table Merge creates duplicate records

Hi there,

 

I am running the merge again and it is still creating duplicate records from the LHS table.  The LHS table has unique rows of data.  So I have 183,000 records merged into 187,000 records post join.

 

I cannot understand why BI would create an exact duplicate of the LHS row in the merged table?  Could it possibly be because the primary key is duplicated?  I know that this is not scientifically viable however this is the closest I can get to a primary key.

 

 

 

DataGuy2016 Regular Visitor
Regular Visitor

Re: Table Merge creates duplicate records

The problem lay with the Data.

 

The LEFT JOIN functionality of PowerBI works perfectly.

Highlighted
J_R_Cribb Regular Visitor
Regular Visitor

Re: Table Merge creates duplicate records

Hi @DataGuy2016

 

If you haven't already found a solution...

 

I had a similar issue where LEFT JOIN was creating duplicates, resulting over 1000 rows of data where I should have just 85.

 

I solved the issue by expanding the relative columns as an aggregate, instead of just an expansion.

 

Hope that this makes sense.

 

 

 

 

 

tera12 Regular Visitor
Regular Visitor

Re: Table Merge creates duplicate records

I am having the same problem with my data.  And no, I do not understand your answer to "expand as an aggregate," will you please clarify?

 

I know this is an old thread, but if someone else is still reading this thread and would like to offer help, I appreciate it!

 

 

J_R_Cribb Regular Visitor
Regular Visitor

Re: Table Merge creates duplicate records

Hi @tera12

 

I have created a quick sample that I hope helps explain my method. It may not work with your data, but it did fix the issue with mine!

 

I have two tables, I would like to merge 'Customer Type' with 'Customer Name'.

 

1.JPGCustomer Name Table

 

2.JPGCustomer Type Table

 

I would like to merge using the 'ID' column in both, to bring in the 'Cust Type' field from table 'Customer Type' into table 'Customer Name'.

 

 

 3.JPGMerge Options

 

 

Once successfully merged, I see the below:

6.JPG

 I now have the option to expand that column of the merged table to bring in data from the 'Customer Type' table. But this is where I choose to Aggregate a field rather than expand. You may require a different aggregate measure other than median, dependant on data type:

 

4.JPGExpand as Aggregate

 

The result from my data is below:

 

5.JPGResult

 

Please also see sample.pbix file which I hope can help you with your issue.

tera12 Regular Visitor
Regular Visitor

Re: Table Merge creates duplicate records

HI JR,

 

Thank you for the response.  Could you send me an email at trossgreene@stormsmart.com.  I want to talk with you more and share a sample file, but I can't figure out how to upload an excel file here.

 

Thanks,

 

Tera

Jbmolina93 Frequent Visitor
Frequent Visitor

Re: Table Merge creates duplicate records

I'm having the same issue thanks. What did you do to fix your data set?