January/February 2006 Feature Article

Did you count the red traffic lights?

This morning was one of those mornings again. I am sure that you will be able to relate to the following …

We are racing down Hans Strydom Drive (exceeding the speed limit as usual). The school is a mere 5 km away, yet I feel like a racehorse galloping towards the finish line. My heart is beating in my throat and my children are encouraging me to illegally overtake some cars in order to get them to school on time! But it doesn’t matter what speed I drive, ALL the traffic lights are RED! And why did the Traffic Department choose today of all days to fill in potholes that have been there for the past six months?

I believe that managers very often feel like racehorses in their daily attempts to deliver projects on time, with the least possible incidents. It is just as challenging to be able to estimate the testing effort of a project as it is to get your children to school on time during peak hour traffic. The only difference is that if you know of all the factors that might influence your testing effort the test estimation is so much easier! (Keep the red traffic lights in mind!)

Test Point Analysis (TPA^®) might be the answer to your testing estimation challenges. TPA^® is a technique that you can apply to estimate high level tests (system testing and user acceptance testing). The estimated effort (hours) can include all the effort for completing all the phases of testing (preparation, specification, execution and completion). It can also include the management and control portion by a test manager.

First things first

To start a TPA^® you first need to do a function point count – or use COSMIC-FFP ^b – that will indicate the functional size of the project to be tested. The functional size will be used as input to the TPA^®. The function point count is derived by a technique called Function Point Analysis (FPA). The International Function Point User Group (IFPUG, 1994) has developed an internationally accepted function point standard. The function point count will include the effort for low-level testing, e.g. unit and integration testing.

COSMIC-FFP (Common Software Measurement International Consortium – Full Function Points) is a method of sizing the functional requirements of software and has been approved as an International Standard (ISO/IEC 19761:2003). Both FPA and COSMIC-FFP are complete topics and, unfortunately, we will not be able to discuss them in this article (please see additional resources at the end of this article).

The risk-based test strategy

The next important step to take is to identify all the quality characteristics and to identify the importance of the quality characteristics. This information

• is included in the test strategy,
• will scope the testing work, and
• will indicate the volume of testing work necessary.

How do you do a TPA^®?

Figure 1 – Schematic presentation of the high level process for a TPA^® (adapted from chapter 7 ‘Test Point Analysis’, The Testing Practitioner, page 96).
[If COSMIC-FFP is used to size the system, read Cosmic Functional Size Units (Cfsu) instead of Function Points (FP)]

Step 1 – Calculate the Dynamic Test Points (TP_f)

A Dynamic Test Point is assigned to each individual function of the system to be tested. There are three factors that play a role in calculating the Dynamic Test Points:

• The function points (FP_f) (determined during the function point analysis) will indicate the size of each function.
• The function dependant factors (D_f) are defined per function and a weight will be assigned to each factor:
  • User importance (U_e) – how important is this function to the users (keep all stakeholders in mind)?
    • 3    The importance is low.
    • 6    The importance is normal.
    • 12  The importance is high.
    
    
  • User intensity(U_y) – how many users use this function and how often?
    • 2   The users use this function not so often (a few times per week).
    • 4   The users use this function about once a day.
    • 8   The users use this function constantly during the day.
    
    
  • Interfacing(I) – how much does one function affect other parts of the system? You will need to take into consideration the number of local data sets accessed by the applicable function as well as the functions that access these data sets.
    • 2   Low
    • 4   Normal
    • 8   High
    
    
  • Complexity(C) – how complex is the algorithm in a specific function? Take the IF statements and the CASE statements into consideration.
    •  3    Simple (0 – 5 conditions)
    •  6    Medium (6 – 11 conditions)
    • 12   Complex (more than 11 conditions)
    
    
  • Uniformity(U) – check the reusability of the code. When working with unique code the uniformity factor will be 1; dummy code or code that has been reused or cloned will have a uniformity factor of 0.6.

Equation 1: Calculating the function dependant factors (Df)
(The Testing Practitioner, page 102)

• Dynamic quality characteristics(Q_d) are also defined for each function. Quality characteristics can be measured explicitly or implicitly depending on the test type. Table 1 depicts the rating types and the weighing factor for each explicitly measured quality characteristic.

For each dynamic quality characteristic that will be tested implicitly a value of 0.02 is added to the Dynamic Test Point

Equation 2: Formula for Calculating the Dynamic Test Point (TP_f)
(The Testing Practitioner, page105)

Step 2 – Calculate the Static Test Points (Q_s)

Quality characteristics can be tested statically by means of a checklist. A value of 16 is added to the Static Test Point Count (Q_s) for each checklist that the test team uses to check certain qualities.

Step 3 – Calculate the total number of test points (TP)

The information that we have now is:

• The function point count (FP) for the complete system to be tested
• The dynamic test point count (TP_f) for each function in the system to be tested
• The static test point count (Q_s)

Equation 3 : Formula for calculating the total number of test points for the system (TP)
(The Testing Practitioner, page 106)

As the function point count indicates the size of the project under construction, the total number of test points indicates the size of the primary test work that needs to be done.

Step 4 – Determine the testers’ skills factor (S)

The skills factor indicates the level of experience that the testers and other test team members have. The less experienced the testers are, and the less product knowledge they have, the more time they will need to complete the testing. That is why it is so important to have a training plan to reduce the effort wasted due to insufficient skills. These skills include product knowledge, knowledge of tools that will be used during testing, testing knowledge, and analytical skills.

The skills factor has a value between 0.7 and 2.0. If the test team is highly skilled the skills factor will be 0.7 while a team with insufficient skills will have a value of 2.

Step 5 – Environmental Factors that might influence testing (E)

According to TPA^® the following environmental factors might affect the testing effort:

• Test tools– the rating for testing tools will indicate how much of the primary testing activities are automated. If no testing activities are automated it will take longer to complete testing.
  • 1   Tools are used for the test specification process and for testing.
  • 2   Tools are used for the test specification process or for testing.
  • 4   No tools are available.
• Preceding tests– the more thoroughly the preceding test (before the next, higher level test) is done, the less time is necessary to complete the higher level test. That is why it is important to have certain entry criteria, and to have insight into the test results of the previous lower level tests, before moving to the next test level.
  • 2   The preceding tests were executed according to a test plan and the test team is familiar with the test cases and test results of the preceding lower level tests.
  • 4   The preceding tests were executed according to a test plan.
  • 8   No test plan exists for the preceding lower level tests.
• Test basis– the test basis is all the system documentation on which the testing is based. This includes documents like system requirement specifications, detailed requirement specifications, data flow diagrams, etc. The more detailed (and higher quality) the documentation is the less time is necessary to prepare for testing.
  • 3     Templates and documentation standards are used for creating specification documentation. Formal inspections are done.
  • 6     Templates and documentation standards are used for creating specification documentation. No formal inspections are done.
  • 12   No templates or documentation standards are used for creating specification documentation.
• Development environment– the environment in which the code is developed plays an important role, as well as the way the code and the databases are designed.
  • 2   A forth generation programming language (4GL) language was used to develop the system. An integrated database management system (DBMS) was used.
  • 4   A 4GL language was used to develop the system. Some of the programs were developed using a 3GL programming language.
  • 8   A 3GL programming language was used.
• Test environment – the test environment can cause huge delays in testing. The risk is higher for a test environment that is newly set up than that for a test environment that has been used before for preceding tests.
  • 1   The test environment is stable and was used for testing before.
  • 2   Tests will be executed in a newly set up test environment, although the test environment will be similar to other well-utilised environments.
  • 4   Tests will be executed in a newly set up test environment.
• Testware– all testing documentation created during the testing process is referred to as testware. This includes artefacts such as test specifications, test scripts, test cases, environment specification, test data, etc.
  • 1   Test cases as well as usable data sets are available for the execution of tests.
  • 2   No test cases are available for testing; only usable data sets.
  • 4   No testware exists to assist with testing.

Step 6 – Calculate the primary test hours (PT)

The information that we have now is

• The total number of test points (TP) for the complete system under testing
• The skills factor (S)
• The environmental factor (E)

Equation 4 : Formula for calculating the primary test hours
(The Testing Practitioner, page 109)

The primary test hours indicates the effort for completing the primary testing activities: including specification, design, execution and completion. The primary test hours exclude management and control of the testing activities.

Step 7 – Size of the team (T)

The bigger the team the more effort it will take to manage the project. When sizing the team, also keep the contractors, the part time testers and the test manager in mind.

• 3   The team consists of up to 4 persons.
• 6   The team consists of between five and ten people.
• 12   The team consists of more than ten persons.

Step 8 – Planning and control of the testing activities (C)

The more tools are used to automate the management and planning process the less effort is necessary.

• 2   Tools are used for defect management and time tracking.
• 4   Either a defect management tool OR a time tracking tool is available and used.
• 8   No tools to support management or defect management are available.

Step 9 – Calculate the total number of test hours (TH)

The information that we have now is

• The primary test hours (PT)
• The team size factor (T)
• The planning and control factor (C)

If we know the team size factor and the planning and control factor we are able to calculate the test management allowance.

Equation 5 : Formula for calculating the allowance for test management (TM)
(adapted from The Testing Practitioner, page 110)

Equation 6 : Formula for calculating the total number of test hours (TH)
(adapted from The Testing Practitioner, page 110)

The green light indicates factors that have a low risk on a project; the amber light indicates factors that have average risk and the red light indicates possible risk areas. Can you see the value of knowing all the red traffic lights on the estimation road? I am sure that the next time you are waiting at a red traffic light you will think of all the delays and time wasted if you don’t know exactly what to look for when you estimate for a project. Test point analysis, together with function point analysis, provides a sound basis for testing estimation… good luck with your next estimation attempt!

Corné Kruger
cornek@testdata.co.za

® TPA^® is a registered trademark of Sogeti Nederland B.V.

Bibliography:

1. Ton Dekkers, Erik van Veenendaal, ‘Test Point Analysis’, p. 93 –  p. 114,The Testing Practitioner (Den Bosh, The Netherlands: UTN Publishers, 2002)
2. Martin Pol, Ruud Teunissen, Erik van Veenendaal, Software Testing: A Guide to the TMap^® Approach (Great Britain: Addison-Wesley Professional, 2002)

Additional Resources:

• IFPUG, Counting Practices Manual 4.2, U.S.A., IFPUG, 2004
• COSMIC Measurement Manual 2.2, Canada, copyright 2003 Common Software Measurement International Consortium
  The COSMIC Implementation Guide for ISO 19761 can be downloaded free-of-charge from www.lrgl.uqam.ca/cosmic-ffp/manual.html, 2003
• A free function point counting manual is available from www.softwaremetrics.com/freemanual.htm