Test planning is an important part of software testing. This article seeks to apply test planning to both Agile and Traditional Software Development Life Cycles (SDLCs). It is written while bearing the IEEE Standard for Software and System Test Documentation (IEEE Std 829™-2008) in mind as it applies to test planning.
Firstly, Agile and Traditional SDLCs are compared and test planning within the Agile SDLC is discussed. Tables 1 and 2 both detail good and bad practices and are included to ensure that a fair basis of comparison is used when comparing Agile and Traditional SDLCs. Example sprint test planning documentation for Agile projects is derived from the IEEE standard mentioned above. This derived sprint test planning documentation should be used only with advice from experienced knowledgeable test practitioners or at your own risk, as the context of each project differs. Secondly, the Master Test Plan and Level Test Plan documents are briefly mentioned in the context of a traditional SDLC under the assumption that the reader is more familiar with the Master Test Plan and Level Test Plans of the traditional SDLC than the Agile release test plan and its associated sprint test plans.
Today, Agile development is rising in popularity in the UK and USA and likely throughout the world. This is not surprising, as for a long period traditional SDLCs have struggled with certain types of software development. The current rise of Agile is fitting in many cases, as it is addressing a needed balance between highly-defined software project scopes and those projects which are charting new territory and therefore are better subject to a bias toward empirical rather than defined controls.
Putting it another way, the requirements on some projects are well-understood, if not well-defined. This type of project is easier to plan and define up front, and then by careful project management to bring to pass roughly within original budget, schedule and cost. The trouble is that there are (many) other types of projects that defy early precise definition, projects whose detailed requirements are still emerging in the user’s minds when the project is over 50% completed, and for these projects the V-Model SDLC is not a good fit.
Much pain is experienced by all the stakeholders as project managers try their best, teams work incredible hours, the business does what it can, but still a project flounders. This is the pain of using the wrong ‘tool’ for the job that we all have experienced from time to time. It is often much better to take the time and money and purchase the correct tool - less damage is suffered by people and equipment and the end-result is achieved with unbelievable ease and efficiency.
However, no more excuses for companies who don’t equip their project managers and other stakeholders with the necessary skills. Important changes in software development have overtaken us at a pace, and project managers and all stakeholders need to be instructed in some of the new approaches such as SCRUM (a management technique to control empirical projects as opposed to plans and detailed Gantt charts for well-defined projects).
Real life is usually more complicated than ‘this or that choice’. In IT, a number of projects are hybrid and therefore best managed with approaches that suit both traditional and agile SDLCs. Human nature, however, often results in us dwelling on our own experiences rather than searching more widely for the truth, and so it can happen that we place ourselves in either the ‘Agile’ camp or the ‘traditional’ camp, which is about as clever as us saying the one tool we need is a hammer, and others arguing that it is a screwdriver, with neither side bothering to ask what the actual problem is that needs fixing – likely it needs both.
To compound matters, when we sit in one camp or the other we often make unfair comparisons, based on the possible (likely?) poor execution of the approach by those in the opposite camp. So some teams, rather than adhering to genuine Agile practices, use the name ‘Agile’ to code in an ad hoc manner and disregard any discipline or controls. Similarly, those using a traditional SDLC approach do not always use the necessary rigour to eliminate weaknesses in the approach e.g., not using prototyping where requirements are poorly defined or still emerging.
So without further ado, let’s be fair to both practices, learn better the one we know least and use them appropriately as the context of our current work dictates.
The Agile Manifesto (http://agilemanifesto.org/) reproduced in Figure 1 is a good place to start understanding Agile sentiment and practices.
The Agile Manifesto website outlines the history behind the manifesto from which I quote directly the opening paragraph (http://agilemanifesto.org/history.html), “On February 11 - 13, 2001, at The Lodge at Snowbird ski resort in the Wasatch mountains of Utah, seventeen people met to talk, ski, relax, and try to find common ground and of course, to eat. What emerged was the Agile Software Development Manifesto. Representatives from Extreme Programming, SCRUM, DSDM, Adaptive Software Development, Crystal, Feature-Driven Development, Pragmatic Programming and others sympathetic to the need for an alternative to documentation-driven, heavyweight software development processes convened”.
Figure 1: The Agile Manifesto (Source: http://agilemanifesto.org/)
SCRUM is a management technique suited to controlling empirical or agile projects. In SCRUM, an initial product backlog is identified and then partitioned into sprints. Figure 2 depicts a product backlog with a product release planning phase spanning 3 sprints worth of product backlog. The IEEE 829™:2008 master test plan document can be tailored to a product release test plan spanning the 3 sprints. For each sprint, a one-page sprint test plan can be tailored to meet the IEEE 829™: 2008 requirements for a level test plan, particularly if these sprint test plans are read in conjunction with the tailored release test plan.
Figure 2: Example Iterative (Agile) SDLC
During each sprint phase, the team members – namely, product owner, developers and testers - work together to meet the specific needs of the sprint.
Figure 3: ‘One Page’ Sprint Test Plan Designed with IEEE/829™: 2008 Level Test Plan in Mind
Just as Agile practices can be done well or poorly, so too can traditional practices vary. Table 2 compares good traditional (V-Model) practices with poor traditional (V-Model) practices.
Good Traditional (V-Model) Practices |
Poor Traditional (V-Model) Practices |
Master Test Plan Completed. High-level specification compiled by Joint Application Design (JAD) process and prototyped as necessary. |
A system or acceptance test plan is sometimes completed. Some high-level specification. Often of poor quality when measured by defects per page. |
Acceptance level test plan completed. Thorough reviews of the specification by business, testers, system architect and reaches an agreed quality before forwarding. |
Poorly reviewed, if at all. No testers involved at this stage. Business not trained in system and software testing. |
Detailed specification compiled by Joint Application Design (JAD) process and prototyped as necessary. |
Some detailed specification. Often of poor quality when measured by defects per page. |
System level test plan completed. Thorough reviews of the detailed specification by business analysts, testers, system architect and lead developers and reaches an agreed quality before forwarding. |
Poorly reviewed, if at all. No testers involved at this stage. System architect and business analysts not trained in system and software testing. |
High-level design compiled by system architects after modelling. |
Developers jump into the code right away. |
Architectural level test plan completed. Thorough reviews of the design and models by testers, the lead architect and development leads and reaches an agreed quality before forwarding. Tests of the design framework are executed by the system architect and development leads with possible help from testers. |
Poorly reviewed if at all. No testers involved at this stage. System architect and development leads not trained in system and software testing. |
Detailed design compiled by development leads. |
Developers jump into the code right away. |
Component level test plan completed. Thorough reviews of the detailed design by testers, developers, development leads and reaches an agreed quality before forwarding. |
Poorly reviewed if at all. No testers involved at this stage. Developers not trained in software testing. |
Coding. |
Coding. |
Component level test plan completed. Thorough reviews of the code by development peers and testers and reaches an agreed quality before forwarding. |
Poorly reviewed, if at all. No testers involved at this stage. Developers not trained in software testing. |
Testing is planned. Component testing by the developer of the code including the use of static and dynamic component testing tools. |
Little or no component testing. Developers not trained in software testing. |
Component integration. |
Little or no component integration. Builds often do not even compile correctly. Build configuration management poor. |
Component integration testing by involved developers and their development leads. It includes the use of static and dynamic component testing tools. |
Little or no component integration testing. Developers not trained in software testing. |
System testing executed by testers independent of the build team. It can include regression testing tools. |
System testing often done by testers or business persons untrained in software and system testing. This testing often combined with acceptance testing. |
Acceptance testing executed by the business with possible help from testers. |
Acceptance testing often done by users or business persons untrained in software and system testing. This testing often combined with system testing. |
Table 2: Good Traditional (V-Model) Practices and Poor Traditional (V-Model) Practices
Traditional software testing practices are best suited for projects where requirements are well-defined. Test planning on these projects can be thorough and can meet the requirements of the IEEE-829™ standard for test documentation. A master test plan is written to balance various project level test plans and possibly other integrated systems.
Agile software testing practices are best suited to empirical projects where requirements cannot be well-defined in advance. Here test planning must cater for sprint test plans and release test plans. These plans can be documented lightly if there is a requirement to meet the IEEE-829™ standard for test plan documentation.
When comparing Agile and traditional methods of software development, be sure to compare good practices in both SDLCs for a fair comparison.
Wayne Mallinson
info@testfocus.co.za
The reproduction, adaptation or broadcast, without permission, of any articles
or photographs published in this
publication, is forbidden and copyright
is expressly reserved for Test and Data Services (Pty) Ltd.
Opinions expressed by contributors are their own and do not necessarily express the opinions of the Publisher, Editor or members of the Editorial Panel.