Testing the system
When you automate a task, sometimes you will have a more efficient and more effective method of achieving a necessary goal and it will be an overwhelming success. Other times you will have an outcome that is neither efficient nor effective, and it will be clear that you should return to the previous way of doing things and try again. Often, it will be somewhere in the middle. How do you test a system you have built? What is an acceptable error rate? How do you measure it?
First, we will talk about a framework for measuring test results known as the confusion matrix.
The confusion matrix
For each task, there are four possible outcomes. These four outcomes are collectively known as the confusion matrix. The following diagram is a visual representation of this simple concept:
Figure 8.3 – Confusion matrix
When testing systems, each outcome will fall into one of these categories. Here is a brief description of each:
- True...
