May / June 2004 Feature Article

What is a Process?

"Process improvement should be done to help business, not for its own sake."
Anon

One of the many definitions describes a process as follows: "A process is a set of practices performed to achieve a given purpose; it may include tools, methods, materials, and/or people."

A process is also often described as one leg of the process-people-technology triad, yet it may also be considered as the "glue" that unifies the other aspects. It is therefore important to understand that having a motivated quality workforce and the latest technology won't lead to a testing organisation performing at its peak. We also require a roadmap that details, directs, and assigns activities and utilisation of scarce resources.

It follows that the determinants of product cost, schedule, and most importantly in the case of testing - the quality of the product - should reside in the process leg of the triad.
Why is it important to focus on process in a testing organisation?

This article attempts to answer this by using the process-people-technology triad.

Work Smarter, Not Harder

The testing workforce, on average, is only as good as it is trained to be, implying that if we don't have a structured approach to our work, we can't expect to train a new recruit to function productively. The constant focus on increased productivity has also forced most testing organisations to expect more from their resources. The solution? Not to work harder, but to work smarter - through process.

Technology and Process

If we shift our focus to the technology leg of the triad, it should be clear that applying expensive technology without a clear roadmap would yield no significant payoff. The fact is that technology will provide the most benefit if applied in the context of an appropriate process roadmap.

Focus on Quality

The quality of a system is significantly influenced by the quality of the process used to acquire, develop and maintain it. This implies indirectly that it is not possible to develop or test an application successfully if the process used to do this isn't designed with quality in mind. It also implies that the same amount of focus should be spent on the quality of the process, as is spent on the product itself. While this has been acknowledged in the manufacturing sector for some time, it has only recently become accepted in software development circles.

Where Did It All Start?

The theories of process management are a synthesis of concepts of Deming, Crosby, Juran and others, as discussed below:

  • Dr. W. Edwards Deming

    The story of Dr. W. Edwards Deming is a well-known one. An American industrial engineer whose radical methods were not accepted by US industry, they so impressed Japanese industry that the Japanese named their highest industrial award after Deming. In its absolutely simplest form, Deming's message was to focus as much or more on how something is done, rather than on what is done.

  • Philip Crosby

    Crosby's name is best known in relation to the concepts of Do it Right First Time and Zero Defect. He considers traditional quality control, acceptable quality limits and waivers of sub-standard products to represent failure rather than assurance of success. Crosby therefore defines quality as conformance to the requirements that the company itself has established for its products, based directly on the customers' needs. He believes that since most companies have organisations and systems that allow deviation from what is really required, manufacturing companies spend around 20% of their revenue doing things wrong and doing them over again. According to Crosby this can be 35% of the operating expenses for service companies.

  • Joseph M. Juran

    There are many aspects to Joseph M Juran's messages on quality. Intrinsic is the belief that quality does not happen by accident, it must be planned. Juran sees quality planning as part of the quality trilogy of quality planning, quality control and quality improvement.

    Over the last 30 years, the theories of Deming, Crosby, and Juran have been used to address quality problems common to many organisations. Many solutions have been discovered, but the gap exists between the state of the practice and the state of the art. Many of these concepts have been used to build process-improvement models.

  • The Process Model

    The concept of a process model requires more clarity. A model is a structured collection of elements that describes characteristics of effective processes. Processes included in these models are those proven by experience to be effective.
    How should we as a testing organisation use a process model?

    • A model will help set process improvement objectives and priorities.
    • A model will help to ensure stable, capable, and mature processes.
    • A model could be used as a guide for improvement of project and organisational processes.
    • A model that includes an appraisal methodology will provide a diagnosis of the state of improvement efforts.

    A process model is also important because it gives us a place from which to start our process journey. By using an existing model in our process definition activities, we benefit from the testing and process communities' prior experiences, and it provides our testing organisation with a common language and a shared vision. It is, however, important to remember the words of George Box: "All models are wrong; some are useful." It is very important to remember that process improvement should be done to help the business and not for its own sake.

  • The Capability Maturity Model

    Work done by different organisations has lead to the development of various process models. The SEI, ISO, EIA, and other organisations have generated models of their own, based on widely accepted practices with proven benefits.

    Software CMM® and ISO 9000 are examples of models created by the above organisations. The remainder of this discussion focuses mainly on the CMMI® model, which is the result of the work done by the SEI.

  • The CMMI® process model

    The CMMI® process model provides a testing organisation or department with guidance to improve the organisation's processes, and the ability to manage the service it delivers to the client or the organisation at large. The CMMI® model uses approaches with a proven track record by placing them in a structure that allows the testing organisation to examine the effectiveness of its existing processes, prioritise, and implement improvements.

    The CMMI® model was designed with input from industry, US Government and the SEI. The team members that contributed to the creation of the model have an average of 21 years' industry experience and are from organisations with a solid and sound process background.

Best practice

The strategy used in the development of the model was to base the content on current best practices. The models were further publicly reviewed and pilots conducted. Updated versions of the model are released based on the information and feedback received from the reviews and pilots.

In brief, the CMMI® model provides a testing organisation with a set of process improvement goals and priorities. It further provides guidance in the creation of quality processes, and a benchmark against which to measure a company or organisation.

The context

When a testing company evaluates its processes for improvement, it is important for the organisation to understand the context within which these process improvements will be viewed and assessed. The following three points provide a backdrop against which to view any process improvements:

  • What is our business goal as a testing organisation?
  • How will we go about measuring proposed process improvements?
  • What impact will these changes have on the bottom line of the testing company? (Return On Investment)

Categories of process improvement

We can basically categorise process improvements into the following categories:

  • Improved schedule and budget predictability.
  • Improved cycle time.
  • Increased productivity.
  • Improved quality.
  • Increased customer satisfaction.
  • Improved employee morale.
  • Increased return on investment.
  • Decreased cost of quality.

Some examples

To demonstrate the impact and the benefits of CMMI®, especially from a testing point of view, the following are some examples contained in a recent white paper released by the SEI.

  • Cost

    • Boeing, Australia has showed a 20% reduction in the cost to fix a defect.
    • Lockheed Martin M&DS has recorded a 20% reduction in unit software costs.
    • Lockheed Martin M&DS has recorded a 15% decrease in defect find and fix costs.

  • Schedule

    • Boeing, Australia has recorded a 60% reduction in work and fewer outstanding actions following pre-test and post-test audits.
    • General Motors has recorded an increase in the percentage of milestones met from approximately 50% to approximately 95%. They further decreased the average number of days late from about 50 to fewer than 10.
    • Lockheed Martin M&DS recorded a 30% increase in software productivity.
    • Northrop Grumman IT2 has reported meeting every milestone (25 in a row) on time, with a high level of customer satisfaction.
    • Bosch Gasoline Systems has recorded a 10% improvement in the first pass yield, leading to a reduction in rework.

  • Quality

    • Northrop Grumman IT 1 recorded only 2% of all defects found in production; they also reported a reduction in defects found from 6.6 per KLOC to 2.1 over 5 causal analysis cycles.
    • Sanchez Computer Associates INC, recorded a $2 million saving resulting from early detection and removal of defects.

  • Customer Satisfaction

    • Northrop Grumman IT2 earned a rating of "Exceptional" in every applicable category on their contractor performance evaluation survey.

  • CMMI® Implementation

    The testing organisation starting on the road to process improvement and using the CMMI® has two options with regard to implementation:

    • Continuous Representation

      By using this approach, the organisation is able to select the order of process improvement that would best meet its business objectives. It further allows a company to compare itself with other organisations in relation to specific process areas.

    • Staged Representation

      This approach provides a proven sequence of improvements, each serving as a foundation for the next. It allows organisations to compare competitors using maturity levels. It further provides a single rating that summarises appraisal results.

In Conclusion

Now that we have a basic understanding of the origin of process, what a process model is, and some initial discussions on the CMMI® process, the articles that follow will look at staged representation of the CMMI® in more detail. This will include a discussion on the five maturity levels within CMMI®, and discuss the process areas represented in each of these levels in more detail.

Mike Snyman and Dhani Kowlaser

CMM® and CMMI® are registered in the US Patent and Trademark Office by Carnegie Mellon University.

<< March / April 2004
July / August 2004 >>