Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.

Reply
Clint
Helper V
Helper V

Remove outliers- greater than 2 standard deviation from mean

Hello,

 

I have a dataset that includes service desk ticket info including the total time to close a ticket.  I have constructed a table (see image) that averages this across priority and by customer group.  What I've been asked to do is to filter out outlier values (highlighted) so that the averages aren't as skewed as they would be otherwise.    I've not used any statistical functions other than average so I'm not sure how to do this.  Any thoughts or insights are appreciated.

Call data averages.PNG

 

1 ACCEPTED SOLUTION

Hi @Clint ,

 

Does that meet your requirement?

Measure 2 = 
VAR k =
    SUMMARIZECOLUMNS ( 'Table'[category], "ave", AVERAGE ( 'Table'[Low] ) )
VAR a =
    CALCULATE ( STDEV.P ( 'Table'[Low] ), ALLSELECTED ( 'Table' ) ) * 2
VAR fk =
    FILTER ( k, [ave] < a )
VAR resu =
    IF ( AVERAGE ( 'Table'[Low] ) < a, AVERAGE ( 'Table'[Low] ), BLANK () )
RETURN
    IF (
        ISINSCOPE ( 'Table'[category] ),
        resu,
        DIVIDE ( SUMX ( fk, [ave] ), COUNTROWS ( fk ) )
    )

Capture.PNG

 

Pbix as attached.

 

Community Support Team _ Frank
If this post helps, then please consider Accept it as the solution to help the others find it more quickly.

View solution in original post

3 REPLIES 3
v-frfei-msft
Community Support
Community Support

Hi @Clint ,

 

I have created a sample for your reference. Please have a check as below.

Measure = 
VAR maxxa =
    MAXX ( ALLSELECTED ( 'Table' ), CALCULATE ( AVERAGE ( 'Table'[Low] ) ) )
VAR ave =
    AVERAGE ( 'Table'[Low] )
VAR resu =
    IF ( maxxa = ave, BLANK (), ave )
RETURN
    IF (
        ISFILTERED ( 'Table'[category] ),
        resu,
        DIVIDE (
            SUMX ( ALLSELECTED ( 'Table' ), resu ),
            DISTINCTCOUNT ( 'Table'[category] ) - 1
        )
    )

Capture.PNG

 

For more details, please check the pbix as attached.

 

Community Support Team _ Frank
If this post helps, then please consider Accept it as the solution to help the others find it more quickly.

Hi @v-frfei-msft ,

 

Thank you for this.  If I'm reading this right, this measure will replace the Max Average in each category w/a "blank"?  Assuming I understand this correctly, that's a good start but what I really need to do is figure out how to strip out all outlier values.  Ideally, I'd like to create a table this removes any values 2 standard deviations above the mean or, if that's not possible, any values above the 95 percentile.

Hi @Clint ,

 

Does that meet your requirement?

Measure 2 = 
VAR k =
    SUMMARIZECOLUMNS ( 'Table'[category], "ave", AVERAGE ( 'Table'[Low] ) )
VAR a =
    CALCULATE ( STDEV.P ( 'Table'[Low] ), ALLSELECTED ( 'Table' ) ) * 2
VAR fk =
    FILTER ( k, [ave] < a )
VAR resu =
    IF ( AVERAGE ( 'Table'[Low] ) < a, AVERAGE ( 'Table'[Low] ), BLANK () )
RETURN
    IF (
        ISINSCOPE ( 'Table'[category] ),
        resu,
        DIVIDE ( SUMX ( fk, [ave] ), COUNTROWS ( fk ) )
    )

Capture.PNG

 

Pbix as attached.

 

Community Support Team _ Frank
If this post helps, then please consider Accept it as the solution to help the others find it more quickly.

Helpful resources

Announcements
Microsoft Fabric Learn Together

Microsoft Fabric Learn Together

Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City

PBI_APRIL_CAROUSEL1

Power BI Monthly Update - April 2024

Check out the April 2024 Power BI update to learn about new features.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.