Chapter 6: Creating Test Validation Scripts

At one company that I worked at, I was trying to figure out the value of some of the test automation scripts that we had. In order to do this, I was analyzing the test results to see which scripts were giving us the most information. One of the rules I used to determine which scripts might not be adding value was to look at scripts that had never failed. My hypothesis was that if a script had been running for some time and had never failed, it was unlikely to fail in the future and so was not giving us valuable information. I had identified several test scripts that had never failed and was looking through them. Imagine my surprise when in several of them I found assertions that where checking things such as whether true==true or 5 == 5? No wonder the tests had never failed. It was impossible for them to fail.

Although these were egregious examples, the reality is that often a well-designed test suite will fail to deliver on its promise...