Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.

Reply
Birdjo
Resolver II
Resolver II

Power Query filter to only one row per day per user

Hello, 

I am developing an attendance report with access control system data.
I am doing calculations of the shift of employees by the time they clock-in.
But sometimes an employee clocks-in twice for a day, and that calculates him two shifts for that day.

clockins.png
I would like to filter the second, third or other clock-ins with power query and have only 1 record per day per employee.

Does someone know some M Code formulas which could I use to achieve that?

 

1 ACCEPTED SOLUTION

No need to merge: you can select multiple columns before chosing "Remove Duplicates".

 

Your #"Table Buffer" step just takes the result from the previous step without buffering.

(The step name is just a name and performs no action).

 

You didn't mention anything about sorting.

 

This is my code:

 

let
    Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
    #"Sorted Rows" = Table.Sort(Source,{{"Time", Order.Ascending}}),
    #"Changed type of Event Date to date" = Table.TransformColumnTypes(#"Sorted Rows",{{"Event Date", type date}, {"Time", type time}, {"Card Number", type text}}),
    Buffered = Table.Buffer(#"Changed type of Event Date to date"),
    #"Removed Duplicates" = Table.Distinct(Buffered, {"Event Date", "Card Number"})
in
    #"Removed Duplicates"

 

Specializing in Power Query Formula Language (M)

View solution in original post

6 REPLIES 6
MarcelBeug
Community Champion
Community Champion

You can select the Date and Employee columns and choose Remove Rows - Remove Duplicates.

 

This will not necessatily keep the first occurrence. If that would be required, then you need to buffer your table after sorting (if not already sorted), using Table.Buffer(<Name of the preceding step>) or just wrap the Table.Buffer around the code of the previous step.

Specializing in Power Query Formula Language (M)

Yes, I want to keep the first occurrence. Let's say it's min. time.
I don't know what exactly Table.Buffer, however I tried it and and applied Remove Duplicates aftert it. 
It removed the earliest entry, leaving the latest.

 

That is very strange. Would you mind sharing the relevant part of your code?

If there is no explicit sort in your code: how is the data sorted?

Specializing in Power Query Formula Language (M)

Here is what I did:
1. Merged date and card number so there should not be duplicates in this value:


 #"Inserted Merged Column" = Table.AddColumn(#"Changed type of Event Date to date", "Date time card", each Text.Combine({Text.From([Event Date], "bg-BG"), Text.From([Card Number], "bg-BG")}, " "), type text),

 

2. Then added Table.Buffer

 

#"Table Buffer" = (#"Inserted Merged Column"),

 

3. Then I remove the duplicates

 

#"Removed Duplicates" = Table.Distinct(#"Table Buffer", {"Date time card"})

 

And what happened is that not the earliest but the latest duplicate record was kept.

 

No need to merge: you can select multiple columns before chosing "Remove Duplicates".

 

Your #"Table Buffer" step just takes the result from the previous step without buffering.

(The step name is just a name and performs no action).

 

You didn't mention anything about sorting.

 

This is my code:

 

let
    Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
    #"Sorted Rows" = Table.Sort(Source,{{"Time", Order.Ascending}}),
    #"Changed type of Event Date to date" = Table.TransformColumnTypes(#"Sorted Rows",{{"Event Date", type date}, {"Time", type time}, {"Card Number", type text}}),
    Buffered = Table.Buffer(#"Changed type of Event Date to date"),
    #"Removed Duplicates" = Table.Distinct(Buffered, {"Event Date", "Card Number"})
in
    #"Removed Duplicates"

 

Specializing in Power Query Formula Language (M)

Well, I can't understand how exactly Table.Buffer function works but that did the job, so thank you very much!

Helpful resources

Announcements
Microsoft Fabric Learn Together

Microsoft Fabric Learn Together

Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City

PBI_APRIL_CAROUSEL1

Power BI Monthly Update - April 2024

Check out the April 2024 Power BI update to learn about new features.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors