Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.

Reply
kalimotxo
New Member

SELECT DISTINCT Returning Duplicate Strings

Hello All,

 

I have a query that I am running which is SELECT DISTINCT. It is a simple query with only one string field in the results. There are multiple WHERE conditions, and then a ORDER BY.

 

Once I get the result, I add a step for Rows: Remove Duplicates.

 

The table is connected to others through relationships. When I try to apply changes, I get an error stating "Column 'Location' in Table 'Location' contains a duplicate value' [Name Redacted]' and this is not allowed for columns on the one side of a many-to-one relationship or for columns that are used as the primary key of a table".

 

I manually checked in the preview list, and there are indeed what appear to be duplicate values. If I manually exclude it, a different location then also shows up in the error message, so there are multiple violations.

 

I have no idea how the above steps can result in duplicates. I specifically picked DISTINCT and added the step to remove duplicates. Any help appreciated!

 

Query Text:

SELECT DISTINCT LTRIM(RTRIM(OfficeName)) AS [Location]

 

FROM ADR

 

WHERE
   Market NOT LIKE '%Test%' AND Market NOT LIKE '%Inactive%'
   AND Name NOT LIKE '%MS%'
   AND PDN = 'P'
   AND CONVERT(date, Date) BETWEEN
      DATEFROMPARTS(YEAR(GETDATE())-2, 1, 1) AND
      DATEADD(dd, -1, DATEFROMPARTS(YEAR(GETDATE()), MONTH(GETDATE()), 1))
   AND ID IS NOT NULL
   AND WCID IS NULL
   AND (Type = 'S' OR Type = 'C' OR Type = 'E')
   AND Code NOT LIKE '%INACT%'
   AND OfficeName IS NOT NULL

 

ORDER BY LTRIM(RTRIM(OfficeName))

1 ACCEPTED SOLUTION
Rfranca
Resolver IV
Resolver IV

hi, @kalimotxo

 

Have you checked that the records in the LOCALIZATION column contain values with white spaces or special characters?

This is often the case, so there are duplications.
Use POWER EDITOR QUERY functions to perform this cleaning.

 

Please try and test using it. And please mark the right reply as answer if your issue has been resolved, otherwise, please feel free to ask if you have any other issue.

Best Regards,
Rfranca

View solution in original post

2 REPLIES 2
Rfranca
Resolver IV
Resolver IV

hi, @kalimotxo

 

Have you checked that the records in the LOCALIZATION column contain values with white spaces or special characters?

This is often the case, so there are duplications.
Use POWER EDITOR QUERY functions to perform this cleaning.

 

Please try and test using it. And please mark the right reply as answer if your issue has been resolved, otherwise, please feel free to ask if you have any other issue.

Best Regards,
Rfranca

Selecting text as the format, despite it already being the default, seems to have worked. I don't know why, but it did!

Helpful resources

Announcements
Microsoft Fabric Learn Together

Microsoft Fabric Learn Together

Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City

PBI_APRIL_CAROUSEL1

Power BI Monthly Update - April 2024

Check out the April 2024 Power BI update to learn about new features.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.