Some of you may be aware that 2009 is the 200th anniversary of Charles Darwin’s birth and the 150th anniversary of the publication of his seminal work On The Origin Of The Species.
Looking back over my 10+ years experience in software testing, I see attitudes to testing have also progressed through a series of evolutionary phases, each rightly obeying Darwinist principles. Today we see a variety of species, most of whom are rare or barely surviving the Return On Investment (ROI) test.
Firstly, there was the “We Have To Do Testing” species. This unhappy creature still exists in some places, and can be found where career testers haven’t been given any chance of showing their real worth. Everyone knows they should do it, or are required to do it from above, but don’t know the reasons why they do it. I won’t say any more about this one, lest readers become even more disturbed at having envisioned this early form of life.
Next came the “Testing Is Insurance” beast. In this apparent step up, money is invested to avoid the perceived risk that it’ll all go wrong. Supposedly an enlightened view, it sees testing as a way of making sure that the pit you fall into isn’t a big one. Naturally, as with all insurance, it’s a sunk cost if the insured event never occurs. And if it does occur, well, that was the risk you took. You won’t see quality improving around here.
Somewhere amongst the menagerie are the “Testers Are Gate Keepers” animals. Here we see expectations that testing will prove the software is bug-free, and if it isn’t, then it’s their fault. Or maybe the developers can be whipped with the test results. Even apart from analogies regarding ambulances and cliffs, and lobbing and fences, this must be seen as folly as it places responsibility in the wrong quarter.
Testing will only ever find a certain percentage of defects, and then only those where the system doesn’t meet specifications. If defect rates are unnecessarily high, or the specifications were wrong or unclear, the number of defects that get through to production will be proportionally larger – and not because of poor testing.
More recently we have seen “Testing Is Risk Mitigation” flying around. This one is pretty common, and is actually quite useful. You can’t test everything (Cem Kaner). Risk-based testing asks: what would actually matter if it broke, and how bad would it be? Requirements are prioritised, linked through to test cases, and the limited testing budget directed where it really matters. Sounds pretty good! Note that you do need engagement from the business in order to perform the prioritisation step.
The most stakeholder-friendly members of this family are the “Testing Is An Information Service” specimens. We see these warm-blooded creatures interacting well with others around them, including stakeholders, project managers, business analysts, end users and developers.
They are part of Michael Bolton’s “brighter future”, where “Management recognizes testing’s value because testing provides service in the form of valuable information, right now, about the state of the program.”
Their purpose in life is to supply the facts the team needs in order to improve software quality, and that others need to be confident making project and business decisions. Having been clear from the start about what deliverables are going to be useful, they are less wasteful than their forebears and are less loaded down with documentation nobody reads – yet their outputs are no less sound.
They thrive in environments characterised by trust and communication, but they can also survive in, and bring a breath of fresh air to, other habitats. This does mean a higher level of engagement from the business, more even than before, as risk-based testing is just one of the techniques in their information services toolkit. The benefits of such engagement are exponential. With providing useful information as their overriding brief, we can see behaviours we’ve not seen before.
For example, you could ask about the testability of requirements before someone wastes time building from them. You could design the software by first asking what tests would tell you it’s working right. You could choose heuristic approaches that provide different, but still useful, information about the software product. Because performance, reliability, usability, security or legal conformance is more important to stakeholders, you might find yourself asking for information outside of the usual functional testing statistics you’re used to getting from your testers, because nobody ever asked them for anything else. And, what are the trends telling you about overall health of the project team’s outputs?
I could describe this last species further, but you can see that they are much more survivable when faced with the ROI challenge. Instead of being a must-have sunk cost, under the right conditions and with forethought testing can provide people with timely, meaningful answers to important questions about product quality and team dynamics – and with manageable cost vs. benefit, just as you’d expect from any service provider.
