Volume 8 Number 3 • May/June 2007 Feature Article

Reviews

Introduction

A Window of Opportunity

Reviews are effective because they can be conducted early (most effective when conducted before coding starts), offer cost-effective testing and reducing project timescales.

The Programmer’s Dilemma

Programmers are often given tight deadlines and are supplied with inaccurate, contradictory, incomplete, unclear business specifications. These specifications are the source of over half of the defects occurring within system applications. Not only is garbage-in responsible for garbage-out (GIGO), but also for wasted time, cost escalation, and stressful and unsatisfactory working conditions.

Group Wisdom

We accept sayings such as, “Two heads are better than one” because we know groups hold more information and insight.  Herein lies one clue to the power of review – focused teamwork devoted to improving a product.

Benefits of Reviews

• Detection and elimination of a high percentage of defects – often early ensuring cost- and time-effective results.
• Result in group participation and buy-in.
• Useful for training and stakeholder handover.
• Independent, objective assessment of products.
• Early testing, therefore reduction of expensive rework, as well as removal of testing and repair tasks from the critical path of projects.
• Facilitate early and accurate product quality assessment, thus better project decision-making and results.
• Reviews combined with other testing interventions increase quality-, effort-, and time-consistency of project results.
• Encourage critical thinking and may be used to direct and improve project culture.
• Can improve job satisfaction and thus staff retention.
• Provide accurate defect measures that support informed management decisions and process improvement within an organisation.
• Bring together IT role players that operate within silos, thus improving the quantity and quality of project communications and project results.

What can be Reviewed?

Any written document including code, test plans, installation instructions, business specifications, design documents, contract, and project management plans.

Types of Review

Four types of review are briefly described in this article. Reviews vary in method and outcome, therefore it’s important to select the types of review suited to the specific needs of the project. More than one type of review may be applied to a single product and one may be more or less suitable than alternate reviews at any given point.

Informal Review

• Although no studies have been done to record their value, they are likely to continue in organisations around the world.
• Easy to arrange, and really help the author of a product to identify defects,.
• Typically occurs when a friend or colleague is asked to ‘quickly’ check a document.
• Locate and correct defects or weak spots prior to the real target of the document spotting them, and the sense of comfort when an ‘all OK’ verdict is given to the document reviewed.
• Less threatening, faster, and easier to request and take part in than the other review types.

Technical Review

• Typically undertaken by peers who understand the same techniques and have similar skills.
• May check the work against standards for that work, such as a coding standard for Java.
• Other source documents may also be referred to during these reviews, such as a business specification.
• Normally done in small groups, one-on-one or two-on-one being common.
• Managers are typically not involved.
• Do find defects, but are also useful for learning and checking that work products adhere to standards.

Walkthrough

• The author of a document leads others through the product on a page-by-page basis, while they ask questions, pass comments, and occasionally discover defects in the product.
• Led by the author
• Good for knowledge transfer and learning
• Typically conducted in small groups of 2 - 7 persons.
• Less formal than inspections, but may be variously conducted.
• Will typically keep track of any discovered defects, but will variously use checklists and metrics.
• Will have different cultures of preparation or not, may easily discuss solutions to problems rather than just focusing on highlighting the problems.
• Good at getting buy-in or participation from participants.
• Recommended in less formal cultures, or where common understanding and participation are valued.

Inspections

• Most formal type.
• The added discipline of this review is used to achieve notable results rather than promote a sense of power.
• Companies that benefit most from inspections have already achieved some sense of process, represented by repeatable and defined successful behaviours.
• Successful at identifying a higher percentage of defects than other review types on an hour-for-hour, and thus a cost basis.
• Typically led by a moderator and not by the author of the document under inspection.
• Various role players inspect a portion of the document and compare it against source documents, standards, and a small set of rules that are chosen for maximum relevance.
• Reading times are slow and scientifically derived to yield maximum defects in a given time.
• Rules are kept to a small number for each participant inspector, where each has rules tailored to their distinct roles.
• Inspections unite the author, users, testers, developers, system architects and other IT players at the time of kick-off and defect logging, and later process improvement workgroups.
• Many companies have reported significant reductions in their rework as a result of inspections.
• Results are remarkable; however, for sustained good results, a mature team able to capture the method into routine operations, is required.
• Based on scientific principles, including statistical sampling and control, measurement, and a body of knowledge from experimentation and resulting empirical measurements.
• Companies embarking on this review technique should receive credible training and mentoring to learn and benefit from the subtleties of the technique.
• Leverage of the technique is achieved by prioritising and sampling portions of documents to inspect, as well as by the learning that authors bring to bear on the process by repeating fewer and fewer defect injection behaviours as they become experienced participants in inspections.

Return on Investment (ROI) for Reviews

The easiest way to calculate ROI for reviews is to record the number of defects found by a review and to compare this with the cost of missing them when they are first introduced to a product and finding them in a later project phase. For example, if we find and fix 100 major defects in a business requirements document soon after it is completed, we can calculate an approximate ROI for repairing the defects early, rather than finding and repairing them during system testing. To do this we would use industry benchmarks to get 100 x 1 unit of repair in contrast to 100 x 22.5 units of repair, which for similar test effort equates roughly to a 2250% ROI (reduction in repair costs for these 100 defects)..

Reviews Change Everything

Well-executed reviews will change everything about how you used to develop applications. They provide such insight, immediately pin-point areas where defects enter your products, and produce valuable metrics to allow management to take informed decisions. The result of this is that your whole software process will change (for the better).

Wayne Mallinson
Email: waynem@testdata.co.za

The Importance of Software Reviews

Standards, certifications, books, theses, research projects, magazine articles, case studies and numerous training courses worldwide all underline the growing importance of reviews in system development.

IEEE Standard 1028 of 1997
The Institute of Electrical Electronics Engineers (IEEE) has published a standard for software reviews, standard IEEE 1028 of 1997 which is best described by text from their own website:

“The IEEE standard for software reviews defines five types of software reviews, together with procedures required for the execution of each review type. This standard is concerned only with the reviews; it does not define procedures for determining the necessity of a review, nor does it specify the disposition of the results of the review. Review types include management reviews, technical reviews, inspections, walkthroughs, and audits”. http://stdsbbs.ieee.org/descr/1028-1997/

The Practitioner Certificate in Software Testing

The British Computer Society (BCS) Information Systems Examination Board (ISEB) Practitioner Certificate in Software Testing syllabus addresses informal reviews, walkthroughs, technical reviews and inspections.

Books

Handbook of Walkthroughs, Inspections, and Technical Reviews: Evaluating Programs, Projects, and Products by Daniel P. Freedman, 1990
Software Inspection by Tom Gilb and Dorothy Graham, 1993, Addison-Wesley
Software Inspection: An Industry Best Practice by Bill Brykczynski (Editor), Reginald N., Jr. Meeson (Editor), David A. Wheeler (Editor), 1996
Peer Reviews in Software: A Practical Guide by Karl E. Wiegers, 2001
High Quality Low Cost Software Inspections by Ronald A. Radice, 2004
Modern Software Review: Techniques and Technologies by Yuk Kuen Wong, IRM Press, 2006

Success Factors for Reviews

Karl Wiegers has written an article entitled, ‘Seven Deadly Sins of Software Reviews’ published on page 10 in this edition of Test Focus. He identifies seven areas which, if they are not well managed, undermine the effectiveness of software reviews. This article is well worth reading.

Taking this same list but now positively managing through the problem areas, we get success factors for software reviews:

1.         Ensure participants have a common understanding of the review process.
2.         Reviewers must critique the product, not the producer.
3.         Reviews must be planned.
4.         Review meetings should not drift into problem-solving meetings.
5.         Reviewers must be properly prepared.
6.         The correct people must participate in reviews.
7.         Reviewers must focus on substance and not on style.

To which I will add:

8.         Ensure the review process is embedded in the organisation’s software development process.
9.         Use statistical sampling and results evaluation to determine the frequency and types of review.
10.       Use process improvement workshops to reduce or eliminate known defects detected by the reviews.
11.       Publish review success metrics.
12.       Train all staff in the review process – it should form a core part of your business.

Wayne Mallinson
info@testfocus.co.za

Reproduction & Copyright

The reproduction, adaptation or broadcast, without permission, of any articles or photographs published in this publication, is forbidden and copyright
is expressly reserved for Test and Data Services (Pty) Ltd.

Opinions

Opinions expressed by contributors are their own and do not necessarily express the opinions of the Publisher, Editor or members of the Editorial Panel.