cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
xhead
Helper II
Helper II

How to correlate time-based events

I am gathering data from two different logging systems. Both have a user, event, and a log datetime. The logs are gathered on different machines and so may have slight differences in their system clocks and the events in both systems don't necessarily mean the same thing. (for example, a browser history log from a workstation and the IIS log from a web server)

 

I'm trying to answer the questions:

 

  • What events occurred at about the same time for the same user?
  • What events are recorded on one system for a user but not on the other system?

I'm not sure what "about the same time" is but it's probably within 5 seconds.

 

My first thought was to "round" the timestamps to the nearest "about" interval, like 0:00:00-0:00:05, 0:00:05-0:00:10, etc. This would group events in each log into 5-second intervals, but it wouldn't correlate an event logged in one source at 0:00:04 with an event logged on the other source at 0:00:06.

 

(It would be handy to have a "marker" event recorded in both logs, and use it to determine the clock difference between the two systems, but I don't think I have that).

 

It occurs to me that I guess I'd like to *cluster* events that happen within a certain fuzzy time interval.

 

Any ideas?

 

1 ACCEPTED SOLUTION

I like this idea.

 

I might massage it a bit because I think I'm not just correlating two events, but rather clustering multiple events that occur within the same time period. Anyway, your suggestion has allowed me to progress, thanks!

 

Mike

 

View solution in original post

2 REPLIES 2
v-ljerr-msft
Microsoft
Microsoft

Hi @xhead,

 

I'm just wondering if you can convert your time columns to number of seconds first, then use ABS function in the filter of your calculation in this scenario? Smiley LOL

EventInBothTable=
CALCULATE (
    LOOKUPVALUE ( table2[event], table2[event], EARLIER ( table1[event] ) ),
    FILTER ( table2, ABS ( table2[time] - EARLIER ( table1[time] ) ) <= 5 )
)

 

Regards

I like this idea.

 

I might massage it a bit because I think I'm not just correlating two events, but rather clustering multiple events that occur within the same time period. Anyway, your suggestion has allowed me to progress, thanks!

 

Mike

 

Helpful resources

Announcements
Microsoft Build 768x460.png

Microsoft Build is May 24-26. Have you registered yet?

Come together to explore latest innovations in code and application development—and gain insights from experts from around the world.

charticulator_carousel_with_text (1).png

Charticulator Design Challenge

Put your data visualization and design skills to the test! This exciting challenge is happening now through May 31st!

May UG Leader Call Carousel 768x460.png

What difference can a User Group make for you?

At the monthly call, connect with other leaders and find out how community makes your experience even better.

Power BI Dev Camp Session 22 768x460.jpg

Check it out!

Mark your calendars and join us on Thursday, May 26 at 11a PDT for a great session with Ted Pattison!