cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Microsoft
Microsoft

Share your thoughts on the new Anomaly detection feature (preview)

Hit Reply to tell us what you think about the new Anomaly detection feature so we can continue to improve.

-Power BI AI team

56 REPLIES 56
Helper I
Helper I

Why does this feature only work when users go against best practice?  Best practice being separating the model from the reporting layer. 

 

Would be nice to get to use these cool tools when following best practices.

Can you elaborate what you mean the feature go against best pratice? Can you elaborate what you mean anomaly detection didn't "seperating the model from the reporting layer. "? Do you mean cannot change sensitivity at Consumer mode?

 

 

New Member

Hello, could you please explain the process of calculating the expected range? I have a monthly series of data which has a run-rate of about 35k, but there is one month that has 64k. The expected range for this month is 52-62k. I don't see a reason for the expected range to be so high, I would expect something like 30-40k.

 

Thank you.

 

Výstřižek.PNG

Roughly, The expected value is calculated by 1) remove the anomaly points, 2) calculate an roughly average of adjacent values 2) apply frontier transform on the value from 2)

 

The expected range is just +/- static range based on the expected value and sensitivity.

 

Yet yes you are right, we are not sure why it returns the expected range > 50K not 30K-40K and we are investigating at our side, and will update the thread shortly.

 

Thank you for your feedback.

 

when I extend the time period of the chart (12 more months), it gives me the expected range.

 

Výstřižek.PNG

Yeah, the anomaly detection algorithm we use is based on frequency. So if the input is small, the model cannot give a very good accuracy (well, in fact I believe it applies to all statistic models). Yet we are checking and see if we can do anything to make the result better with limited input data like what you provided above.

Frequent Visitor

The expected min value when hovering over the line graph is impossible and showing negative numbers. The dataset we are using does not have any negatives.

In this case, if you could give us a repro pbix, we can try to involve Azure Anomaly Detection team to check why the negative numbers are shown.

 

Also, you may be interested in my reply on technical details in the same thread.

Helper III
Helper III

It seems to be software related. Asking a collegue to open the file gives explanation results, publishing the file also gives me the expected results. Weirdly enough, reinstalling did nothing for me to fix the issue.

What about re-install in a different folder? Or after un-install, and try delete the existing PBI Desktop folder (you can get it via right click on the PBIDesktop icon). 

Make sure you get the latest Desktop version.

lisahua46_0-1609359002368.png

Hope a clean re-install can fix the issue.

 

If still have issue, we probably need to check .Net version. https://docs.microsoft.com/en-us/dotnet/framework/migration-guide/how-to-determine-which-versions-ar... 

Helper III
Helper III

Everything is working for me, until I click on an anomaly:

 

TKA_4-1608212546662.png

 

 

TKA_0-1608212312126.png

 

Its a blank report with just date in the axis and revenue in the values.

 

TKA_1-1608212390782.png

TKA_2-1608212423012.png

TKA_3-1608212488049.png

 

 

This is without adding anything special to the overview. I tried it with 3 different reports that had different datasets, all with the same results. What am I doing wrong?

 

Version: 2.88.621.0 64-bit

 

Is it feasible to share a repro to us? 

Frequent Visitor

Hi to everyone,

I found some exemples on internet to test anomaly detenction.

I send you the link of one:

https://radacad.com/anomaly-detection-in-power-bi

 

I followed every step but the explaination panel alweys show me the same message:

"No explainations were found"

 

I send you the pbix that I used

https://1drv.ms/u/s!Aq9vs57jlUsQgxc3QKRh8CyXhaPJ

 

Can you tell me what's wrong?

Hello @I365SvcUsr , I see from your pbix that you are trying to explain by just two fields. If you remove these fields from the explain by field well, our automatic field selection will kick in and allow you to find explanations on more points. You can also try dragging more fields to the explain by field well.

 

Additionally, it is expected that some anomaly points won't return explanations, because there simply isn't enough information to explain that point with confidence.

Hi,

i tried to remove all values from the "explain by" field but nothing changes: the explanation panel is always empty.

I also selected the same date of the example that I sent you (27 october 2010).

Example.png

Can you show me what you see selecting this date into the pbix that i sent you?

Hello,

I did see that some edits were made to the data in this pbix so that it no longer matches the data given in the tutorial you linked, so some results may be different. However, I get quite a few explanations returned in the pbix you sent when I click on the anomaly point at 27 October, 2010 and have no fields in the explain by field well.

nneoma_0-1607104877954.png

Can you try closing the pane, removing the anomaly detection card, and applying anomaly detection again? Then check if you can get explanations for this point.

I365SvcUsr_retry_explanations.gif

If not, can you create a new pbix using the data directly from the tutorial at https://radacad.com/anomaly-detection-in-power-bi and try again? That way we can check if you are able to get explanations results at all.

Hello,

I wanna explain you every step I did this second time:

I uninstalled and installed PBI Dextop. After that, I downloaded again the .CSV from the linked tutorial and I imported it into a new pbix. I checked the option "Anomaly detenction" so I closed and opened the pbix.

At this time, I clicked on the line chart in the visualization panel and select thedate for Axis and temperature into Values, so I applied anomaly detection without fields in the explain by field.

The result is the same:

https://1drv.ms/u/s!Aq9vs57jlUsQgxhyFhRlDFHpDGBs

 

PBI ANOMALY DET.png

Nothing appears..

Thanks for reporting the issue, Looking into it..

Resolver I
Resolver I

Thank you. Much appreciated.

Microsoft
Microsoft

 

Anomaly detection tutorial - Power BI | Microsoft Docs

Check out this technical blog for more details about the Anomaly Detector algorithm.

User should get almost identical results of using Azure AnomalyDetection API vs PowerBI Anomaly Detection. 

 

The sensitivity is a way we allow users to tune the model with the understanding that different business or person could have different tolerance on even the same dataset. The higher the sensitivity, the narrower the band, as a result, you would get more anomalies.

 

The strength is defined as “Explanatory power”

“Explanatory power” is the percentage of change in the aggregate value at an anomaly point that is explained by the explanation value/time series. Concretely, it is calculated as the ratio of deviation (actual minus expected value) between the explanation/component time series (e.g., for Property type = Loft) and the aggregate time series for the anomaly point. See original paper

 

The implementation for AnomalyDetection and Explanation is open-sourced

Source code: AnomalyDetection Explanation

There is no relationship between sensitivity and strength.

 

Roughly speaking, for explain anomalies, we do:

  1. Given an “aggregate time series” (e.g., Listing count by date) with an anomaly on a given date, find anomalies in all “component time series” (Listing count by date where Property type = Loft, Listing count by date where Property type = Cabin, Listing count by date where Neighborhood = Capitol Hill etc.)
  2. Find the dimension (i.e., Property type or Neighborhood in the example) for which the component time series most likely show a root cause on the given date. This is based done using a decision tree (with classes being anomaly vs not anomaly) and the split criterion is Entropy Gain Ratio and additional filtering
  3. For the chosen dimension (e.g., Property type) rank dimension values (Loft, Cabin etc.) with anomalies on the given date according to an “explanatory power”

Helpful resources

Announcements
secondImage

Experience what’s next for Power BI

Join us for an in-depth look at the new Power BI features and capabilities at the free Microsoft Business Applications Launch Event.

secondImage

Power BI Women

Join our monthly meetings and learning sessions.

secondImage

Congratulations!

We are excited to announce the Power BI Super Users!

secondImage

The largest Power BI virtual conference

100+ sessions, 100+ speakers, Product managers, MVPs, and experts. All about Power BI. Attend online or watch the recordings.

Top Solution Authors
Top Kudoed Authors