Machine Learning and Applied Statistics Lesson of the Day – The Line of No Discrimination in ROC Curves
June 23, 2014 Leave a comment
After training a binary classifier, calculating its various values of sensitivity and specificity, and constructing its receiver operating characteristic (ROC) curve, we can use the ROC curve to assess the predictive accuracy of the classifier.
A minimum standard for a good ROC curve is being better than the line of no discrimination. On a plot of
on the vertical axis and
on the horizontal axis, the line of no discrimination is the line that passes through the points
In other words, the line of discrimination is the diagonal line that runs from the bottom left to the top right. This line shows the performance of a binary classifier that predicts the class of the target variable purely by the outcome of a Bernoulli random variable with 0.5 as its probability of attaining the “Success” category. Such a classifier does not use any of the predictors to make the prediction; instead, its predictions are based entirely on random guessing, with the probabilities of predicting the “Success” class and the “Failure” class being equal.
If we did not have any predictors, then we can rely on only random guessing, and a random variable with the distribution is the best that we can use for such guessing. If we do have predictors, then we aim to develop a model (i.e. the binary classifier) that uses the information from the predictors to make predictions that are better than random guessing. Thus, a minimum standard of a binary classifier is having an ROC curve that is higher than the line of no discrimination. (By “higher“, I mean that, for a given value of , the of the binary classifier is higher than the of the line of no discrimination.)