Friday, December 22, 2006

The Importance of Test Design

"More than the act of testing, the act of designing tests is one of the
best bug preventers known. The thinking that must be done to create a useful test can discover and eliminate bugs before they are coded - indeed, test-design thinking can discover and eliminate bugs at every stage in the creation of software, from conception to specification, to design, coding and the rest."
-Boris Beizer, Software Testing Techniques

This is so true it's almost painful. I've seen projects where thousands of tests are run, but they aren't the right tests, and hence defects are missed. Creating massive automation infrastructures and purhasing expensive commercial testing programs (or "solutions", as they like to be called), is really a worthless exercise in time and money if you're not designing intelligent tests.

For example, lets say you have a function that takes in a string and returns the number of characters in that string. All the fancy test programs in the world aren't going to help you if your QA engineer doesn't test it with unicode characters, null strings, or strings with no null terminator.

Test design is absolutely one of the more interesting and creative aspects of QA. The simple process of coming up with possibile ways that one can break a component is a creative process on its own, but also one that requires technical expertise. How can you know the many ways to break a product on a Windows platform if you don't have indepth technical knowledge of Windows? It's almost a philosophical question - how can you know the things you don't know? There is only one answer: keep learning and expand your knowledge boundaries.

No comments: