Automated ML Service with the Binary Model

[Anonymous]

Hello,

As I have mentioned before, it is the first time I am using this service with the binary model.

The data set that I use to make some tests is:

https://raw.githubusercontent.com/santoshc1/PowerBI-AI-samples/master/Tutorial_AutomatedML/online_sh...

The ideal solution for this dataset will be to obtain a calibrated model with high number of True Positive and True Negative and very few False Positive and False Negative.

Here’s it is not what we obtained. 

How can we decide which is the best Probability Threshold?

What can be done to improve the quality of the prediction that is to say, a model with very few False Positive and False Negative?

[v-joesh-msft · Accepted solution]

Hi  @Anonymous ,

I am not very familiar with ML. You can read the following document, which introduces the Cumulative Gains chart and the ROC curve, which are statistical measures of model performance, I hope to help.

https://docs.microsoft.com/en-us/power-bi/service-machine-learning-automated#binary-prediction-models

Best Regards,

Community Support Team _ Joey
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.