It would be nice if software errors had single causes. But the truth is, a bug is usually caused by the coincidence of several factors. Here's a demonstration of one such bug--and a demonstration of how the testing department at the Fantastic Computer Company misses it.
The following illustration shows the execution tree of FCC's new program:
Development has been completed, and the program is about to get sent to the testing department. Nobody at FCC knows about the potential divide-by-zero error we see lurking in the code, within the two instructions illustrated.
The testing department at FCC innocently chooses the test cases shown by the green and blue arrows, below:
The testing department did a pretty good job choosing test cases; their tests cover all the code in the program. Nevertheless, disaster awaits....
The test cases are run, and--nothing goes wrong.
No divide-by-zero errors occur, because none of the test cases executed the two instructions that combine to cause the error.
And so, the program gets the stamp of approval from the testing department--and then it gets shipped to the customer. One day, our unfortunate customer runs the following:
We'll skip the part about FCC employees getting fired.
The only reliable way to catch this type of bug (i.e. REAL coverage testing), is to test every possible PATH through the program. That is, test every single combination of events that could ever occur.
But that's impossible. Even in an extremely simple system, there can be millions or billions of possible paths. Testing that many paths is outside the operating budget of most companies.