Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.

Reply
Anonymous
Not applicable

If 2nd Date/Time Row, Remove Both

I'm pulling data from SQL for facility check-ins.  I want only a list of people who are still in the building.  In the attached screen shot, stationid 52 is a check-in station and stationid 192 is a check-out station.  If memid has an entry for the stationid 192 check-out staion, I want to filter out both rows for stationid 52 and stationid 192, leaving only the rows for memid who have checked-in and no rows for memid who have checked-out.  You can see in this screen shot that memid 98033 has a row for both stationid 52 and 192.  I want both rows for memid 98033 filtered out.Filter Checked Out Members.JPG

28 REPLIES 28
edhans
Super User
Super User

This is certianly doable in Power Query, but can you provide data?

 

How to get good help fast. Help us help you.
How to Get Your Question Answered Quickly
How to provide sample data in the Power BI Forum



Did I answer your question? Mark my post as a solution!
Did my answers help arrive at a solution? Give it a kudos by clicking the Thumbs Up!

DAX is for Analysis. Power Query is for Data Modeling


Proud to be a Super User!

MCSA: BI Reporting
Anonymous
Not applicable

OK, I typed the relevent columns directly into Excel and pasted that here...

chkinidmemidcheckinstationid
5438177980335/21/2020 2:17:49 PM52
54381781307325/21/2020 2:17:56 PM52
54381791348065/21/2020 2:18:06 PM52
5438180767955/21/2020 2:18:14 PM52
54381811030555/21/2020 2:18:20 PM52
5438182976405/21/2020 2:18:26 PM52
54381831130625/21/2020 2:18:35 PM52
5438184980335/21/2020 2:49:17 PM192

Is this what you need?
2020-05-21 14_51_08-Untitled - Power Query Editor.png

I didn't actually compare the station IDs. I just removed them if there were more than one present 

You might want to add additional Group By columns (Days, employee codes or whatever) if your data is a bit more complex. See the M code below.

 

1) In Power Query, select New Source, then Blank Query
2) On the Home ribbon, select "Advanced Editor" button
3) Remove everything you see, then paste the M code I've given you in that box.
4) Press Done

 

let
    Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("dZBLCsMwDAWvErwORF9L8h0K3Yfc/xqRU7qp1ZXgMYOkd55NhR3N2t7CgTmnHoQHAcFGA21IbO/XjKld+5f3DJDBmFZBeyHEI4hD/xV8QCE4ZGDdQlcepeBxLgAGLYScqzAvD+sCBV8dNKvB/LkvL/tgLQT506lEtvThMVK4bg==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [chkinid = _t, memid = _t, checkin = _t, stationid = _t]),
    #"Changed Type" = Table.TransformColumnTypes(Source,{{"chkinid", Int64.Type}, {"memid", Int64.Type}, {"checkin", type datetime}, {"stationid", Int64.Type}}),
    #"Grouped Rows" = Table.Group(#"Changed Type", {"memid"}, {{"All Rows", each _, type table [chkinid=number, memid=number, checkin=datetime, stationid=number]}}),
    #"Added Custom" = Table.AddColumn(#"Grouped Rows", "Keep", each Table.RowCount([All Rows]) < 2, type logical),
    #"Filtered Rows" = Table.SelectRows(#"Added Custom", each ([Keep] = true)),
    #"Expanded All Rows" = Table.ExpandTableColumn(#"Filtered Rows", "All Rows", {"chkinid", "checkin", "stationid"}, {"chkinid", "checkin", "stationid"}),
    #"Removed Other Columns" = Table.SelectColumns(#"Expanded All Rows",{"checkin", "memid", "chkinid", "stationid"})
in
    #"Removed Other Columns"

 



Did I answer your question? Mark my post as a solution!
Did my answers help arrive at a solution? Give it a kudos by clicking the Thumbs Up!

DAX is for Analysis. Power Query is for Data Modeling


Proud to be a Super User!

MCSA: BI Reporting

Hi @Anonymous 

Try this:

let
    Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("dZBLCsMwDAWvErwORF9L8h0K3Yfc/xqRU7qp1ZXgMYOkd55NhR3N2t7CgTmnHoQHAcFGA21IbO/XjKld+5f3DJDBmFZBeyHEI4hD/xV8QCE4ZGDdQlcepeBxLgAGLYScqzAvD+sCBV8dNKvB/LkvL/tgLQT506lEtvThMVK4bg==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [chkinid = _t, memid = _t, checkin = _t, stationid = _t]),
    #"Changed Type" = Table.TransformColumnTypes(Source,{{"chkinid", Int64.Type}, {"memid", Int64.Type}, {"checkin", type text}, {"stationid", Int64.Type}}),
    whos_checkedout_ = Table.SelectRows(#"Changed Type",each [stationid] = 192 )[memid],
    final_ = Table.SelectRows(#"Changed Type",each not List.Contains(whos_checkedout_,[memid]) )
in
    final_

 Please mark the question solved when done and consider giving kudos if posts are helpful.

Contact me privately for support with any larger-scale BI needs

Cheers 

SU18_powerbi_badge

Anonymous
Not applicable

@AlB and @edhans 

Thanks for this... some weirdness, though (and both methods got the exact same results):

  1. It's not removing all people who have scanned at check-out stationid 192 (see memid 76795).  It did remove memid 98033 who checked out.
  2. The checkin date from the query doesn't match when they actually scanned in.
  3. The stationid from the query doesn't match the stationid that they actually scanned in on.
  4. It's also not picking up any scans from memid 104958.

In the below data, I've included checkin and stationid from both the query and the table for comparision.  The table names are in all caps in each column heading.  

Also - I apologize for not mentioning this earlier, but stationid 56 is a secondary check in station within the facility and should not be counted as a check out when it occurs after initially scanning at stationid 52 upon entry.  Stationid 187 is an alternate initial entry to the building and should be treated the same as stationid 52.  

 

CHKINS checkinQUERY1 checkinQUERY1 chkinidQUERY1 memidSTATIONS stationidQUERY1 stationidSTATIONS stationname
5/25/2020 7:52:39 AM5/21/2020 2:18:26 PM54381829764018752Ground Level Entry
5/25/2020 12:08:48 PM5/21/2020 2:18:06 PM54381791348065252Main Check In
5/25/2020 12:08:56 PM5/21/2020 2:18:14 PM5438180767955252Main Check In
5/25/2020 12:09:09 PM5/21/2020 2:18:20 PM54381811030555252Main Check In
5/25/2020 12:59:05 PM5/21/2020 2:18:06 PM54381791348065652Health & Wellness Check In
5/25/2020 1:12:59 PM5/21/2020 2:18:14 PM54381807679519252Member Check-Out

FYI @Anonymous - neither solution above will work if you have the same memid check in on a different day. The below will handle it if it does. This speifically checks for the 52 and 192 codes as well, so code 171 would not cause this to get deleted for example.


But if you have multiple checkins a day, this will not work either, or a checkin at 11:50pm and a checkout at 12:03am the next day. We'd need to see a more comprehensive data set to see how to account for other possibilities, including additional columns if necessary.

 

1) In Power Query, select New Source, then Blank Query
2) On the Home ribbon, select "Advanced Editor" button
3) Remove everything you see, then paste the M code I've given you in that box.
4) Press Done

 

 

let
    Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("dZBLCsMwDAWvErwORF9L8h0K3Yfc/xqRU7qp1ZXgMYOkd55NhR3N2t7CgTmnHoQHAcFGA21IbO/XjKld+5f3DJDBmFZBeyHEI4hD/xV8QCE4ZGDdQlcepeBxLgAGLYScqzAvD+sCBV8dNKvB/LkvL/tgLQT506lEtvThMVK4bg==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [chkinid = _t, memid = _t, checkin = _t, stationid = _t]),
    #"Changed Type" = Table.TransformColumnTypes(Source,{{"chkinid", Int64.Type}, {"memid", Int64.Type}, {"checkin", type datetime}, {"stationid", Int64.Type}}),
    #"Added Custom" = Table.AddColumn(#"Changed Type", "Date Only", each DateTime.Date([checkin]), type date),
    #"Grouped Rows" = Table.Group(#"Added Custom", {"memid", "Date Only"}, {{"All Rows", each _, type table [chkinid=number, memid=number, checkin=datetime, stationid=number, Date Only=date]}}),
    #"Added Custom1" = Table.AddColumn(#"Grouped Rows", "Remove", each List.ContainsAll([All Rows][stationid],{52,192})),
    #"Filtered Rows" = Table.SelectRows(#"Added Custom1", each ([Remove] = false)),
    #"Expanded All Rows" = Table.ExpandTableColumn(#"Filtered Rows", "All Rows", {"chkinid", "checkin", "stationid"}, {"chkinid", "checkin", "stationid"}),
    #"Removed Other Columns" = Table.SelectColumns(#"Expanded All Rows",{"memid", "chkinid", "checkin", "stationid"})
in
    #"Removed Other Columns"

 

  



Did I answer your question? Mark my post as a solution!
Did my answers help arrive at a solution? Give it a kudos by clicking the Thumbs Up!

DAX is for Analysis. Power Query is for Data Modeling


Proud to be a Super User!

MCSA: BI Reporting
Anonymous
Not applicable

@edhans Some memid's will check in and out multiple times per day.  There shouldn't be anyone checking in one day and checking out the next.

Anonymous
Not applicable

@edhans @AlB 

Thanks for this!  Some weirdness, though (and both of your solutions provided the exact same results):

  1. Not all memid’s were removed on scan at check-out stationid 192. See memid 76795.  But memid 98033 was removed.
  2. The checkin date/time is different in the Query than what it actually is. The Table CHKINS is correct.
  3. Same for the stationid – Query is not correct, but table CHKINS is correct.
  4. Memid 104958 is not picking up any scan in the Query, but if I look at the raw data, is clearly scanning.

Also, and I apologize for not mentioning this earlier:

  • Stationid 187 is an alternate initial entry to the building and should be treated the same as stationid 52.
  • Stationid 56 is a secondary scan-in area of the building and should not be treated as a check-out.

In the data I’ve included below, the table names are in all caps in the heading.

CHKINS checkinQUERY1 checkinQUERY1 chkinidQUERY1 memidSTATIONS stationidQUERY1 stationidSTATIONS stationname
5/25/2020 7:52:39 AM5/21/2020 2:18:26 PM54381829764018752Ground Level Entry
5/25/2020 12:08:48 PM5/21/2020 2:18:06 PM54381791348065252Main Check In
5/25/2020 12:08:56 PM5/21/2020 2:18:14 PM5438180767955252Main Check In
5/25/2020 12:09:09 PM5/21/2020 2:18:20 PM54381811030555252Main Check In
5/25/2020 12:59:05 PM5/21/2020 2:18:06 PM54381791348065652Health & Wellness Check In
5/25/2020 1:12:59 PM5/21/2020 2:18:14 PM54381807679519252Member Check-Out

Ok, Can you provide some comprehesive data that includes your scenarios? Your original data didn't have any duplication. I don't need 10,000 rows, but 30-50 with some good examples of your true data needs would help. Please use the links below to provide data in a good format. Thanks!

 

How to get good help fast. Help us help you.
How to Get Your Question Answered Quickly
How to provide sample data in the Power BI Forum



Did I answer your question? Mark my post as a solution!
Did my answers help arrive at a solution? Give it a kudos by clicking the Thumbs Up!

DAX is for Analysis. Power Query is for Data Modeling


Proud to be a Super User!

MCSA: BI Reporting
Anonymous
Not applicable

@edhans 

Here is actual data, from when our buildng was still open before COVID-19 shutdown.  The check-out stationid 192 has not been actually put into place yet.  That's something new we're implementing when we get to re-open.  Stationid's 52 and 187 are 1st points of entry into the building and stationid 56 is an additional area within the building where members must scan again to gain entry after they've already scanned into either 52 or 187.

 

CHKINS checkinMEMBERS scancodeSTATIONS stationidSTATIONS stationname
3/3/2020 7:1533101187Ground Level Entry
3/3/2020 7:181666652Main Check In
3/3/2020 7:193410352Main Check In
3/3/2020 7:201666656Health & Wellness Check In
3/3/2020 7:203410356Health & Wellness Check In
3/3/2020 7:2033101187Ground Level Entry
3/3/2020 7:212897552Main Check In
3/3/2020 7:213416852Main Check In
3/3/2020 7:253218352Main Check In
3/3/2020 7:261770852Main Check In
3/3/2020 7:263245352Main Check In
3/3/2020 7:263468252Main Check In
3/3/2020 7:273245356Health & Wellness Check In
3/3/2020 7:272788152Main Check In
3/3/2020 7:283599952Main Check In
3/3/2020 7:282849756Health & Wellness Check In
3/3/2020 7:293237952Main Check In
3/3/2020 7:302788156Health & Wellness Check In
3/3/2020 7:302115052Main Check In
3/3/2020 7:303237956Health & Wellness Check In
3/3/2020 7:303599452Main Check In
3/3/2020 7:313218152Main Check In
3/3/2020 7:312115056Health & Wellness Check In
3/3/2020 7:311874152Main Check In
3/3/2020 7:323370652Main Check In
3/3/2020 7:331481752Main Check In
3/3/2020 7:333370656Health & Wellness Check In
3/3/2020 7:332949652Main Check In
3/3/2020 7:341481756Health & Wellness Check In
3/3/2020 7:371896352Main Check In
3/3/2020 7:393420952Main Check In
3/3/2020 7:403588552Main Check In
3/3/2020 7:403420956Health & Wellness Check In
3/3/2020 7:413588556Health & Wellness Check In
3/3/2020 7:422430752Main Check In
3/3/2020 7:442894552Main Check In
3/3/2020 7:453549952Main Check In
3/3/2020 7:473314352Main Check In
3/3/2020 7:482771152Main Check In
3/3/2020 7:482722652Main Check In
3/3/2020 7:493256952Main Check In
3/3/2020 7:502972052Main Check In
3/3/2020 7:503256956Health & Wellness Check In
3/3/2020 7:533253952Main Check In
3/3/2020 7:553468452Main Check In
3/3/2020 7:563322152Main Check In
3/3/2020 7:563484552Main Check In
3/3/2020 7:562348952Main Check In
3/3/2020 7:572348956Health & Wellness Check In
3/3/2020 7:583381252Main Check In

@Anonymous this data has nothing but stations 52, 187 (both entry) and 56 (to be ignored if I understand your post), so there is nothing here to test or eliminate.

 

Or I have totally lost the plot and am not sure what the goal is.



Did I answer your question? Mark my post as a solution!
Did my answers help arrive at a solution? Give it a kudos by clicking the Thumbs Up!

DAX is for Analysis. Power Query is for Data Modeling


Proud to be a Super User!

MCSA: BI Reporting
Anonymous
Not applicable

@edhans 

Check-Out stationid 192 doesn't actually exist yet.  We'll start using it once we re-open.  The goal is to build a report that shows who has scanned their badge at either stationid 52 or 187 and has not scanned at stationid 192 (ignoring stationid 56).  I manually added some test entries scanning at stationid 192.

CHKINS checkinMEMBERS scancodeSTATIONS stationidSTATIONS stationname
3/3/2020 7:1533101187Ground Level Entry
3/3/2020 7:162115052Main Check In
3/3/2020 7:181666652Main Check In
3/3/2020 7:193410352Main Check In
3/3/2020 7:201666656Health & Wellness Check In
3/3/2020 7:203410356Health & Wellness Check In
3/3/2020 7:2033101187Ground Level Entry
3/3/2020 7:212897552Main Check In
3/3/2020 7:213416852Main Check In
3/3/2020 7:242115056Health & Wellness Check In
3/3/2020 7:253218352Main Check In
3/3/2020 7:261770852Main Check In
3/3/2020 7:263245352Main Check In
3/3/2020 7:263468252Main Check In
3/3/2020 7:273245356Health & Wellness Check In
3/3/2020 7:272788152Main Check In
3/3/2020 7:283599952Main Check In
3/3/2020 7:2828975192Member Check-Out
3/3/2020 7:282849756Health & Wellness Check In
3/3/2020 7:2934103192Member Check-Out
3/3/2020 7:293237952Main Check In
3/3/2020 7:302788156Health & Wellness Check In
3/3/2020 7:303237956Health & Wellness Check In
3/3/2020 7:303599452Main Check In
3/3/2020 7:313218152Main Check In
3/3/2020 7:3516666192Member Check-Out
3/3/2020 7:4827881192Member Check-Out
3/3/2020 7:4935999192Member Check-Out
3/3/2020 7:5032183192Member Check-Out

@Anonymous 

I don't see what the problem is with my first solution then. That one eliminated all the members that had checked out (i.e, gone through 192). Froma what I understand that is exactly what you need here. 🤔

Please mark the question solved when done and consider giving kudos if posts are helpful.

Contact me privately for support with any larger-scale BI needs

Cheers 

SU18_powerbi_badge

 

Anonymous
Not applicable

@AlB 

Those are not the results I'm getting.  When I run the report, it's:

  • listing some people who have checked out
  • not listing some people who have checked in and not checked out
  • the checkin date/time field is stuck at 5/21 @ 2:18pm regardless of when they actually scanned
  • the stationid is stuck at 52 regardless of what stationid they scanned at.

I pasted what you wrote in the Advanced Editor.  Am I missing a step?  I'm very new to Power BI, maybe I'm doing something wrong?

 

I've attached a screnshot of my whole screen so you can see what I'm looking at that shows the results from your query vs all data and I've made notes in text boxes next to them each.  The results from your query shows only columns from your query and the results from all data shows only columns from my other tables.  The only filter that's on is a whole page filter of the date.

 

Thank you for your help!

 

AlB Query Results Screenshot.JPG

Anonymous
Not applicable

@AlB 

I also noticed that the chkinid's in your query are all from date 5/21/2020, not from date 5/27/2020 (the date on my page filter).  It's like the query is stuck on that one date and won't update to the date I'm running on the report.  I even deleted the query and re-added it and it still only pulled data from 5/21/2020.

@Anonymous 

I'm getting lost now. My query is stuck at 21st because the input for it (Source step) is what you initially posted and that only has data for the 21st. The query is intended to show the steps,  you would have to update the Source step and read your full input data instead of the example.

Please mark the question solved when done and consider giving kudos if posts are helpful.

Contact me privately for support with any larger-scale BI needs, tutoring, etc.

Cheers 

SU18_powerbi_badge

 

Anonymous
Not applicable

@AlB 

The original data had the 21st because that's what day it was then.  I need this to update real-time.  How do I update the source?  Like I said... brand spankin' new to Power BI, so I really don't know what I'm doing yet.  I just need this to work before Monday.

Thank you!

@Anonymous I somehow missed your earlier post this week with more data. I'll get back to this hopefully tonight. Perhaps @AlB will solve it before then, but if not, just wanted you to know I didn't bail. I just missed the notification with more data. 😁



Did I answer your question? Mark my post as a solution!
Did my answers help arrive at a solution? Give it a kudos by clicking the Thumbs Up!

DAX is for Analysis. Power Query is for Data Modeling


Proud to be a Super User!

MCSA: BI Reporting

Ok, @Anonymous, see if this works. Here is what I did:

  1. Duplicated your data and changed the date to the 4th so I had 2 days to play with to ensure it handled says right.
  2. Added a days only field (ignoring time)
  3. Grouped by day and ID
  4. filtered for any rows that had (187 or 52) and 192 and removed them
  5. kept the rest. I ignored any 56 records.

The PBIX file is here.

 

let
    Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("pZRLSwMxFIX/Sph1pXmcTBK3IiooLl3ULqoGKo4jTFvBf28y1RIVy72ZWWQ25577Sr7FojFzM9dSS+FOlW1mjTFKqvRX3qXzYnjb9U/iOr7HTpz32+GjWc5+BrVJppWyMv2tTsfN6rkXZ+v4+CKu+j9yn73b9NHkIZcEJQ1JrmXpno/LuOq2a3G/k1K34i52XR83m+MGh3zVBswZ6izWPjhL61Lti2w9TY5yQzU9jfdCK09cQk6hnJPE8trRHZbhbtB6TZO70r2m+WygnfeKli83bWwIgS7/3rwKoz6+PsRhH3Fyu9v+E4Lgajsq3xQx5xiijaN1ZWQ5s4oSR4NDvmqDtAbQClZfN5y2ZGMLzNAmCF+MhBgSiqtEC7GyeKlHQ1ADffCgDx70wYM+pkIfU6GPGuiDB33woI+p0AcP+uBBHzzogwd9TIU+eNAHD/rgQ/93CBv64EMfPOhjKvQxFfrgQR886IMPffChDz70wYL+8hM=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"CHKINS checkin" = _t, #"MEMBERS scancode" = _t, #"STATIONS stationid" = _t, #"STATIONS stationname" = _t]),
    #"Changed Type" = Table.TransformColumnTypes(Source,{{"CHKINS checkin", type datetime}, {"MEMBERS scancode", Int64.Type}, {"STATIONS stationid", Int64.Type}, {"STATIONS stationname", type text}}),
    #"Added Date" = Table.AddColumn(#"Changed Type", "Date", each DateTime.Date([CHKINS checkin]), type date),
    #"Grouped Rows" = Table.Group(#"Added Date", {"Date", "MEMBERS scancode"}, {{"All Rows", each _, type table [CHKINS checkin=datetime, MEMBERS scancode=number, STATIONS stationid=number, STATIONS stationname=text, Date=date]}}),
    #"Added Checked Out Validation" = 
    Table.AddColumn(
        #"Grouped Rows",
        "Checked Out", 
        each (
            List.ContainsAny([All Rows][STATIONS stationid], {187,52})
            and List.Contains([All Rows][STATIONS stationid], 192)
            )
        , type logical),
    #"Filtered out Checked Out IDs" = Table.SelectRows(#"Added Checked Out Validation", each ([Checked Out] = false)),
    #"Expanded All Rows" = Table.ExpandTableColumn(#"Filtered out Checked Out IDs", "All Rows", {"CHKINS checkin", "STATIONS stationid", "STATIONS stationname"}, {"CHKINS checkin", "STATIONS stationid", "STATIONS stationname"}),
    #"Removed Other Columns" = Table.SelectColumns(#"Expanded All Rows",{"CHKINS checkin", "MEMBERS scancode", "STATIONS stationid", "STATIONS stationname"})
in
    #"Removed Other Columns"

 

 

 



Did I answer your question? Mark my post as a solution!
Did my answers help arrive at a solution? Give it a kudos by clicking the Thumbs Up!

DAX is for Analysis. Power Query is for Data Modeling


Proud to be a Super User!

MCSA: BI Reporting
Anonymous
Not applicable

I'm sorry!  Your query still only shows data from the past, but I need this report to update with data real-time.  The purpose is to be able to pull this report at any time to see who is in the building at that time.

Helpful resources

Announcements
Microsoft Fabric Learn Together

Microsoft Fabric Learn Together

Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City

PBI_APRIL_CAROUSEL1

Power BI Monthly Update - April 2024

Check out the April 2024 Power BI update to learn about new features.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.