Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn a 50% discount on the DP-600 certification exam by completing the Fabric 30 Days to Learn It challenge.

Reply
zenisekd
Super User
Super User

"Remove duplicates" removes unique values in the format of whole number

Hi...

my issue is that I have a table, where the column ID is in the format of whole number (int), right from the beginning, without any transformations. The table has around 25k rows. 
When I use "remove duplicate rows" on ID column, it deletes, along with duplicates, certain unique numbers eg. 23682. 

 

zenisekd_2-1627306001302.png

When I revert the steps (first filter the number and then remove duplicates), the number is there ...

 

zenisekd_0-1627305951225.png

zenisekd_1-1627305963554.png

Can somebody tell me what the heck is going on?

I have read in different places that sometimes it helps to first convert the column to text, then trim, do upper or lower case, then remove duplicate rows and then convert it back to a number. This doesn't work for me, however...

1 ACCEPTED SOLUTION
v-kelly-msft
Community Support
Community Support

Hi  @zenisekd ,

 

After checking,I couldnt reproduce your senario,as it seems fine here:

vkellymsft_2-1627464824724.png

 

vkellymsft_1-1627464812930.png

 

So pls check the blog below to see whether you could find a solution:

https://radacad.com/remove-duplicate-doesnt-work-in-power-query-for-power-bi-here-is-the-solution

 

Best Regards,
Kelly

Did I answer your question? Mark my post as a solution!

View solution in original post

6 REPLIES 6
j_ocean
Helper V
Helper V

Are you sure the removed duplicates is pointing at the ID column? That's caught me before, pointing at some kind of title where it may coincidentally be the same across two entries.

v-kelly-msft
Community Support
Community Support

Hi  @zenisekd ,

 

After checking,I couldnt reproduce your senario,as it seems fine here:

vkellymsft_2-1627464824724.png

 

vkellymsft_1-1627464812930.png

 

So pls check the blog below to see whether you could find a solution:

https://radacad.com/remove-duplicate-doesnt-work-in-power-query-for-power-bi-here-is-the-solution

 

Best Regards,
Kelly

Did I answer your question? Mark my post as a solution!

Any ETA for this bug to be fixed?

I guess no ETA for a fix.

samdthompson
Memorable Member
Memorable Member

Hello, i have had this too. I am not sure of the techincal reason but i got around it by prefixing all the text values of whole numbers with a z then doing the duplicate removal, removing the z's and converting back to a whole number. A bit convoluted but it worked for me.

 

 

// if this is a solution please mark as such. Kudos always appreciated.

Got same problem here. Removing duplicates on a Whole Number column was in fact removing thousands of unique values. Changing type to text before removing duplicates and then back to Int has worked fine.

JoseRamon_0-1692256816126.png

 

Helpful resources

Announcements
LearnSurvey

Fabric certifications survey

Certification feedback opportunity for the community.

PBI_APRIL_CAROUSEL1

Power BI Monthly Update - April 2024

Check out the April 2024 Power BI update to learn about new features.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors