We only know whether a model is successful if we can measure it, and it is worthwhile taking a moment to remember which metrics to use in which scenarios. Take, for example, a credit card fraud dataset where there is a large imbalance in the target variable because there will only be a, relatively, few cases of fraud among many non-fraudulent cases.
If we use a metric that just measures the percentage of the target variable that we predict successfully, then we will not be evaluating our model in a very helpful way. In this case, to keep the math simple, let's imagine we have 10,000 cases and only 10 of them are fraudulent accounts. If we predict that all cases are not fraudulent, then we will have 99.9% accuracy. This is very accurate, but it is not very helpful. Here is a review of the different metrics and when to use them.
