|
July
2002 Feature Article
|
|
Test Management
by Metrics Measuring and Improving Your Testers’ Productivity Do you know if your testers are producing the goods? How effective are they, really? How can you improve their performance? If these are questions that you ask yourself, then read on…. To establish the answers to these questions, you will have to first establish a measurement programme that will allow you to monitor the productivity of your test team. Step 1. Determine the activities or tasks that are conducted by your testers. I suggest to a level that will determine tasks such as follow:
Step 2. You need to get testers to log time spent on these activities so that you can get a snapshot of the current situation. This information will be used to estimate averages or categories to estimate the testing effort required. Step 3. Divide the initial data into 3 categories that will account for the average time required to test large, medium and small sized projects (remember that the initial categories will be wrong, but will improve with time as you get more data). Step 4. Establish the processes that are involved in each of the categories in Step 1. This must be done so that you can measure the effectiveness of changes that will be brought about to improve productivity. This step should also include the decision of what will be used to measure each process i.e. cost, time, materials etc. |
Step 5. You now have a basis against which to evaluate the productivity of your team (this will also give you a handle on estimation for future projects!). A simple way to do this is to divide the product by the amount of hours used to produce it eg. Divide the total time taken to write all the test cases by the total number of test cases written – this will give you the average time to write a single test case (remember that this will also be influenced by the complexity of the requirement against which the analysis was conducted – this is another subject). Even though simplistic, I use this as a better rule of thumb.
I personally like to use what I call a tolerance chart when evaluating productivity. What I do is take the average time for each task and then add a 20% variance or tolerance to resolve issues such as complexity of requirements. I then plot all times on a graph and investigate items that are out of tolerance. I first subtract the average from each time measurement and plot the result (see Figure 1).
Step 6. Analyse the testing processes and determine which processes need to be changed or removed (this should be done to remove bottle necks or to streamline processes that are too bound up in red-tape). This is the first step to improving the quality of your testing effort.
Step 7. Monitor the time taken to conduct testing tasks to see if the change implemented in Step 6 is being effective (before scrapping the idea, you must consider if the change was clearly communicated and if additional training will be required to implement the change to the process).
I have found that these measures will lend themselves to performance appraisals, but I always am a bit hesitant in using them for such purposes as some managers will use this to beat up on under performing testers instead of trying to analyse the root cause and taking corrective action. If this happens, the testers may start producing false metrics to avoid such action or even to try to impress management. This will render the metrics effort totally useless.
Readers are welcome to forward any queries to me via info@testfocus.co.za. Peter Sage |
<< June 2002 |
August 2002 >> |