Measuring the performance of classifiers
In this section, we'll see how to measure the performance of a classifier. In the example we saw in the previous chapter, a Decision Tree can predict that a new customer will not default, but actually he/she does default. We need a mechanism to evaluate the error rate of a decision tree; this mechanism is the confusion matrix or the error matrix.
Confusion matrix, accuracy, sensitivity, and specificity
Coming back to our loan example, imagine you have classified 1000 loans using a Decision Tree. For each loan, our classifier has added a label with the value yes or no, depending upon whether the algorithm predicts that the customer will default or not. In order to generalize, we will use the terms positive or negative classification. In our loans example, we have a positive classified observation when our classifier predicted that a customer will default, so the value of the Default? Attribute is Yes.
In this scenario, there are four types of predictions...
