Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
SriKandimalla
Helper I
Helper I

How to remove duplicates using a condition

Hello Power BI Community,

 

I am trying to figure out couple of things:

1) Either to remove the duplicate rows 

OR

2) To avoid them in my count column

I have a dataset where 1 employee can be repeated many times but the task should not be repeated. Please review the data image below.
RemoveDuplicateQuestion.JPG

So I have to achieve the count of 1 for the combination (employee, task). 
If there is a way to delete the rows in power Query editor that would be great. Or if there is a way to write DAX count by eliminating the duplicate occurrences, I can use that as well.

 

help is much appreciated, Thanks in advance! @Anonymous @Nathaniel_C  @Mariusz @Greg_Deckler 

 

 

1 ACCEPTED SOLUTION

@SriKandimalla -

Are you trying to achieve:

1.PNG

Measure = COUNTROWS(DISTINCT(TableName))

 

Can also be accomplished as:

let
    Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WKk7MVdJRSjQ0VIrVgfOMjHDzjI3BvKxMEC8Jqg/Kg6qE8lBVEs8zMUHmmZqCeXmZyfk5qUhORQhAdSMEMLSA3BULAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [employee = _t, task = _t]),
    #"Changed Type" = Table.TransformColumnTypes(Source,{{"employee", type text}, {"task", type text}}),
    #"Removed Duplicates" = Table.Distinct(#"Changed Type")
in
    #"Removed Duplicates"

with

Measure 2 = COUNTROWS('TableName (2)')

2.PNG

 






Did I answer your question? Mark my post as a solution!
Did my answers help arrive at a solution? Give it a kudos by clicking the Thumbs Up!

Proud to be a Super User!



View solution in original post

6 REPLIES 6
Nathaniel_C
Super User
Super User

@SriKandimalla ,

Go to Power Query, Home tab, Remove Rows, Remove Duplicates.
Let me know if you have any questions.

If this solves your issues, please mark it as the solution, so that others can find it easily. Kudos are nice too.
Nathaniel





Did I answer your question? Mark my post as a solution!

Proud to be a Super User!




Hey @Nathaniel_C ,

Thanks for the response. I cannot delete the entire duplicates. I can assign the same task to multiple employees but the rule is each employee can have a particular task only once. So, I have to remove the rows where the count >1(excluding the 1st row with the duplicate, since I will lose the original row). But cannot remove all of them.

Hi @SriKandimalla ,

Go to Power Query, and try it. You can always reverse it.  Remove Duplicates, means after the first one.
Let me know if you have any questions.

If this solves your issues, please mark it as the solution, so that others can find it easily. Kudos are nice too.
Nathaniel





Did I answer your question? Mark my post as a solution!

Proud to be a Super User!




Hey @Nathaniel_C ,

 

I just tried it, it has left my table with unique values of task.
I need the duplicates as well to be able to assign the same to task to various employees. What I need to get rid of are the ones with duplicated of (Employee & task) combined. 

@SriKandimalla -

Are you trying to achieve:

1.PNG

Measure = COUNTROWS(DISTINCT(TableName))

 

Can also be accomplished as:

let
    Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WKk7MVdJRSjQ0VIrVgfOMjHDzjI3BvKxMEC8Jqg/Kg6qE8lBVEs8zMUHmmZqCeXmZyfk5qUhORQhAdSMEMLSA3BULAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [employee = _t, task = _t]),
    #"Changed Type" = Table.TransformColumnTypes(Source,{{"employee", type text}, {"task", type text}}),
    #"Removed Duplicates" = Table.Distinct(#"Changed Type")
in
    #"Removed Duplicates"

with

Measure 2 = COUNTROWS('TableName (2)')

2.PNG

 






Did I answer your question? Mark my post as a solution!
Did my answers help arrive at a solution? Give it a kudos by clicking the Thumbs Up!

Proud to be a Super User!



It solved the problem, Thank you @ChrisMendoza 

 

I have a follow-up question. What can we do if we have a unique column such as Row_ID before the employee column. How we will be distinct count then?


Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.