|
Raising Expectations of Testing
Part 1: Managers
Cut your IT development delivery costs
by 30 percent! Does this sound like fiction to you? Read on, your goals
and targets may be way too conservative.
The mission and business-critical nature
of IT is no longer disputed. Yet many managers and project teams are not
aware or knowledgeable enough about a key component of IT differentiation,
namely software testing.
The IT industry is reaching adolescence,
with multiple factors driving its growing maturity. Globalisation, accompanied
by international competition and standardisation, require us to be aware
of worldwide trends as well as local factors. Increasing local competition,
litigation, industry experience, and user sophistication are some of the
key local drivers of the local IT maturation process.
It is no longer good enough to have a great
business idea. For success it is imperative that such an idea is supported
swiftly and accurately by a high-quality software implementation. If implementation
is delayed, or poor, one's whole business may be in jeopardy. Copycat
offerings (of higher quality) will surpass your own initiative within
three to twelve months.
To properly understand testing, one needs
to know that testing comprises two essential approaches, namely verification
and validation. Most managers have a reasonable, high-level understanding
of validation, which is executing the software and determining whether
it meets the users' requirements or not. Validation testing in isolation
however, is inherently flawed as a holistic approach to testing. At the
stage of validation testing, the product is already created at a fixed
quality, and the testing can only identify the numerous failings.
The milk is spilt and tears will be of
no avail. Damage control, in the form of expensive swat teams, additional
maintenance support, travel expenses, huge clean-up validation testing
initiatives, customer pacification and other remedies will be the real
cost of the lack of earlier investment in testing.
Furthermore, the correction of defects
at this late stage is inherently inefficient, with numerous documented
studies showing an approximately ten-fold improvement in software maintenance
productivity for software which has passed through effective verification
testing.
Verification testing is testing of the
software products that proceed execution of the code, for example, project
management plans, user requirements specifications, designs, and code
listings. The means by which your company undertakes its verification
testing is a significant differentiator.
Testing is a specialisation; in fact there
are specialisations within testing itself. The proper management of effective
testing within major organisations has significantly contributed towards
published results like that of Raytheon. In 1995, Raytheon reported more
than 30 percent cost reduction in its software non-conformances, as well
as a calculated 7.7 fold return on investment for the initiatives that
resulted in this.
As early as 1992, IBM reported an 82 percent
plus defect removal rate prior to development testing, as a result of
effective software verification techniques.
In 1981 Boehm studied 63 projects and was
able to show that for these projects, if the cost of fixing a defect in
the requirements stage was one hour, it would cost 15 to 70 hours to repair
the same defect, if it was only detected during validation testing (unit,
integration, system and acceptance execution of the code).
| Speaking with one
IT Manager, this past week, he expressed that testing was "a bit of a fuzzy
area", and this I have noted to be a common notion amongst the majority
of South African and even American IT managers. I'll be quick to point out
though, that the situation is rapidly changing because of the huge business
stakes involved. Testing need not be "fuzzy". The fact is, that effective
verification and validation testing methods, with accompanying reporting,
allow managers to have more regular management check points on all their
products necessary for software construction. This should happen throughout
the development cycle and is a factor which promotes IT project success.
If, as a manager (or tester), you can't
answer the following questions at any software development life cycle
(SDLC) stage, you need to do further work on your company/project testing
initiatives:
-
What are the detailed
testing tasks for this SDLC stage?
-
How many persons will
be required for the testing, and what skill sets should they have?
-
What is the detailed
testing schedule and budget, and how does this relate to deliverables?
-
What would the alternative
additional expense be if testing, was omitted or ineffective?
-
Exactly what percentage
of defects are being found by testing and when? Based on these measures
is the testing Cost effective?
-
What is the average
quality of our software documentation, in defects per page?
-
How many requirements
are there for testing in the XYZ subsystem?
-
How many test procedures
are ready for execution?
-
Is our testing effectiveness
static, improving, or worsening?
-
Which of our high-risk
requirements have related known, but unsolved defects?
-
Approximately how
many developers will be needed to repair new defects likely to be
found next month?
-
What are the ten most
common major defects that our team injects into the products?
-
Have we reduced the
introduction of the most common defect types over the past three months?
-
How many security
requirements are there, compared to usability requirements?
-
What is the cost-effectiveness
of our unit testing when compared to our code inspections?
-
When will the build
be ready for release?
-
What quality will
the build be at release?
-
And another 50 or
so other questions that the lack of space restricts us from asking.
A word of caution: The benefits to be derived
from effective testing result from knowledgeable and targeted management
and team focus over time. The nature of the gains is substantial, but
there is nothing easy about achieving such gains.
Our January edition will feature "Raising
Testing Expectations - Part 2: The Business".
Wayne Mallinson
info@testfocus.co.za
Reproduction & Copyright
The reproduction, adaptation or broadcast, without permission, of any articles
or photographs published in this
publication, is forbidden and copyright
is expressly reserved for Test and Data Services (Pty) Ltd.
Opinions
Opinions expressed by contributors are their own and do not necessarily express the opinions of the Publisher, Editor or members of the Editorial Panel.
|