Private Inspections and the Logging Meeting.
Last month we discussed
private inspections and
mentioned the unusually slow reading rate required for effective defect
detection during inspections. We will now examine a case study that demonstrates
the economy of less haste and more speed.
During training, which I present, I often invite the course candidates to submit a page from their own project specifications for inspection. In one class on a three-day training course, I divided the class into four groups, assigned different roles to the five members of each group and assigned them to inspect a given page of their own specification for homework.
The results returned by the four groups of class 1 were 19, 14, 14, and 12 group-unique major defects found on that one page (refer to Figure 1). On analysis, many of the defects found in each group did not even overlap with the defects found by the other groups.
We then did some simple arithmetic to illustrate the quality gains, and the cost and time savings that could have accrued to the project from which the specification came.
Gross Saving: Average of 14.75 major defects found per group. Each defect was evaluated as saving 10 or more downstream project hours, a potential saving of 147.5 hours.
Costs: Kick-off meeting 0.5 hours x 5 persons = 2.5 hours used. Private inspection hours averaged one hour per person resulting in 5 hours per group. Logging meeting 0.5 hours x 5 persons = 2.5 hours used. Net hours used per group 10.0.
Net Saving: Saved hours (147.5 hours) less used hours (10.0 hours) averaged 137.5 hours per group!
Having found this exercise valuable I assigned the same specification page to an even more experienced class from the same company. However this group was attending a one-day course and so only had half an hour of private inspection time for the page.
Four groups of five persons from class 2 each now returned the following defect detection rates: 14, 9, 8, and 7 (refer to Figure 1). By similar arithmetic the average net savings was calculated to be 90 hours per group.
Figure 1
Conclusion: Groups in the second, more experienced class 'saved' time by halving the time for private inspections. This 'saving' makes sense in the short term (group members are often very busy with many demands on their time). However if we look at the net project saving in the second class, it was only 90 hours.
The first class by taking twice as long for private inspections, saved an additional (above class 2) net 50 hours for the project from one page of inspected specifications! If this was a one hundred page specification and similar results applied to each page, class 1 could save 13 750 hours or about seven-and-a-half person-years for the project!
If we now inspected the
same page in fifteen minutes, we could easily prove that even less net
savings would be achieved. With just a little extra extrapolation we will
be able to easily realise the poor quality, longer schedules and higher
costs that most of us are currently experiencing from radically shortening
inspection time! And the hours that we don't save will keep us busy!
Figure 2 shows defects found per measured unit against measured reading rates for code on a project. The shape of the curve is typical and one is able to clearly determine the optimal average reading rate for a particular software product within your company. If no previous optimal reading rate has been established, begin at one hour per page and certainly not faster than half-an-hour per page (remember the exercise above).
Figure 2
The Logging Meeting
Should junior staff take part in the inspection process? What value can they add? The correct answer is that they should take part as they can highlight jargon, general assumptions and other areas that persons familiar with the project may not realise will later cause maintenance problems for persons not intimately involved with the project. The junior members will also learn from the best practitioners in each role, what is acceptable and what constitutes a defect.
Inspections are massively parallel, highly focused and effective communication sessions around focused product areas. This is so significant that it bears repeating. Inspections are massively parallel, highly focused and effective communication sessions around focused product areas. Seventy nine percent of members of a large corporation practicing inspections reported synergy as having the largest influence on inspection effectiveness. Inspections are highly effective at detecting defects in work software work products (code, design, specifications, test plans etc.).
However an extremely important side effect of inspections is education of the participants of the meetings, not least of all, the authors of the products. It has been reported that authors tend to introduce 50% less defects into similar work products for each round of inspection attended! This means that after only two rounds of inspection, the same author will be introducing 75% fewer defects into their code or design or specific product. Imagine what this does for productivity, professionalism, career enhancement, and company payback!
During the logging meeting, the moderator first collects the number of major and minor errors collected by each person and the times they took for their private inspections. This immediately signals if the meeting should continue or be postponed due to insufficient preparation. If everyone has put in the necessary preparation the moderator can use the number of defects reported to plan the meeting timing. In one meeting I moderated this year, 140 major defects were found and we only had 90 minutes to record them. I decided to let participants only report the major and not the minor defects in the meeting.
The tight and efficient running of the logging meetings is critical to the long-term well being of the inspection process within a company. Any slackness or indiscipline here will destroy the effectiveness of the process and inspections risk being labeled yet another meeting. Please note that a disciplined meeting is not to be equated with no fun! Only that one must laugh quickly and then carry on.
The discussion of whether defects are major or minor should not be undertaken in the logging meetings. Only the author has final authority to classify/reclassify defect classes, and this takes place during update editing. Solutions to defects should not be discussed during the logging meetings. Imagine how long a meeting could become if 140 solutions were discussed, rather than merely reporting the defects.
Next month we will see why the inspection process is like a missile, rather than a cannon ball! How to find new rules and eliminate less-efficient rules, and more on this powerful testing lever!
Wayne Mallinson