November/December 2006 Feature Article

Test Strategy

Test strategy must help meet test policy objectives

The test strategy is a set of high-level approaches for testing in a company or a large division of a company.  The test strategy is a means to achieve certain testing goals. A company’s testing policy will direct the testing strategy towards the broader testing goals of the company.

Table 1: Example Relationships between Test Policy and Test Strategy Statements

Test strategy must help meet test policy objectives

The test strategy is a set of high-level approaches for testing in a company or a large division of a company.  The test strategy is a means to achieve certain testing goals. A company’s testing policy will direct the testing strategy towards the broader testing goals of the company.

Test strategy serves to communicate testing interests in the company

The test strategy can be at the level of a single test, a single project, or for all projects within a company. Thinking of one or more generic approaches to testing in the different development and test phases can save the cost and effort of planning specific approaches to project test plans or phase test plans multiple times. The strategy also communicates the broad interests of the company with respect to how testing should be conducted in each of the development or test phases.

Test strategy must reduce risk, increase opportunity, and keep business costs down

The current trend is to involve the test strategy with identifying project and product risks, and then apply suitable testing to mitigate each of these risks. This thinking is good, but it does not go far enough! Test strategy must consider value-based testing practices, as these will align better with business thinking. Risk would still be a factor, but now consideration of business opportunity and the cost of testing in addition to risk will foster better strategic thinking and fresh approaches to testing. Testing should be integrative and holistically considered within the company, and test strategies should accommodate this.

Test strategy should support company imperatives

It is often required that time, cost, or quality is optimised on projects; and specific test strategies are then selected depending on the current needs of the project. Quality itself is a collection of attributes that may include reduced defects, shorter time to market, faster response times, higher security, better ease of use, ease of maintenance, and other desirable attributes. Test strategy documents, at their best, should inform staff of ways to optimise and balance trade-offs for achieving these goals from a testing perspective.
A minimum test strategy

People, process, and context

A company’s staffing approach for testing is a key variable of strategy. Depending on the nature of testing decided upon, test staff would have to be highly skilled where sophisticated test techniques and automation were selected or less skilled where basic testing techniques and a predominantly manual testing approach were followed.

Test strategy should adjust to accommodate for shifts in the market, and also to allow for testing improvements as experience accrues. A well-defined collection of test processes will be easier to adjust than processes which were not standardised and documented as part of a testing strategy.

Business and other contexts of testing such as available skills, budgets, tools, and technology platforms will all influence test strategy and are the reason why test strategy must be tailored to each company situation as well as to keep it current.

Testing is a team effort where stakeholders include professional testers and other roles. Testing strategy must include developers, system analysts, database administrators, operations personnel, business analysts, product managers, and others for an optimised testing solution or approach at company level.

Technical points for strategic consideration for each development and testing phase

• Each development and testing phase has inputs, processes, and outputs. Quality checks in the form of entry and exit criteria for each phase should be in place. This ensures that testing detects defects early and that defects do not enter the next phase in the process.
• The testing techniques for use in each phase depend on the current project context. This allows the appropriate techniques to foster the project’s business aims.
• Standards that must be complied with include project, company, contractual, industry, national, and/or international standards. This will ensure that testing addresses regulatory and business needs correctly.
• Many testing exercises are once-off projects. By maintaining the test-ware (like documents, datasets, tool scripts, environment set-up instructions, and especially the test regression suites) much benefit can be gained in the form of better ongoing risk mitigation, faster testing cycles, and reuse of tests.
• Each development and testing phase produces valuable outputs like reports and other deliverable documentation, metrics, and incident logs. These all need sound management, both during and after each testing project.
  The degree of test independence is an important strategic consideration. In high-cost or high-risk projects, independence in testing is desirable so that parties separate from those who created the system should test the system. This separation is a sound strategic move when outsourcing software development – a different organisation should test the software at system level than the one supplying it. Businesses outsourcing software development need to develop very good practices during the requirements generation and the system acceptance phases.
• Test strategy must adjust to the software development life cycle chosen by any project. Varied integration approaches like top-down, depth-first, bottom-up, and incremental functional integration will also require specifically tailored testing approaches.
• In addition to other metrics, test completion guidelines belong in a strategy document so that testers have a clear indication as to when to stop testing.
• With labour costs being significant in information system development it is worth considering automation in any test strategy. Some companies have found that one tester keeping many automated tests running can cost less than a handful of testers doing manual testing. Managers need good insight here as not all tests are cost-effective to automate. A pervasive automation approach will also require system and code designs to support ease of testing.
• Re-use of software will reduce the need for some tests. In such cases it is also good to consider re-use of test plans, designs, cases, data, environments, procedures, and scripts where possible.
• Non-functional requirements can contribute greatly to project woes when they are poor or absent. Test strategy should ensure that testing pays early and sufficient attention to necessary non-functional tests such as security, maintainability, usability, performance, portability, and the like.
• A standardised test process should be part of any given strategy. The test process should cater for planning, specification, execution, and recording of tests and also should include test completion, closure, and review activities.
  Test improvement initiatives should form part of any test strategy.

Test strategy overlaps with test planning

Test strategy overlaps with test planning in as much as the test planning approach carries out the relevant test strategy in detail. The test planning confirms and sharpens elements from the test strategy such as which specific test techniques to use, test completion measures, and other overlaps.

Generic test approach

A table used in the test planning approach is useful - once the approach demonstrably conforms to company strategy it is suitable for re-use in future test plans (see table 2).

Table 2: Generic Testing Approach for Low, Medium, and High Risk Projects

Benefits of a company testing strategy

The benefits of a company testing strategy include:

• Guidance for all for carrying out company testing policies
• Common (re-usable) approach to testing
• Less training required
• Staff movement between projects facilitated by familiarity of approach across projects
• Regulates and governs testing within the company
• Provides guidelines for any project or phase test plans
• Prevents important testing areas from being ‘forgotten’
• Aligns the efforts of all staff with respect to testing
• Creates a benchmark expectation of testing during each development or testing phase
• Provides a communication path to senior management
• Saves time, effort, and money, and provides higher quality outputs from testing and development
• Can be tailored to various project sizes, risks, and technologies
• Forms a minimum high level testing baseline which may be extended and improved

Strategy next steps

If your company does not have an existing testing strategy, begin motivating for one.

• Do further research on test strategies on the Internet.
• Attend training on creating a testing strategy.
• Start discussing probable company test policy direction (strategy will have to follow this).

Wayne Mallinson
info@testfocus.co.za

Reproduction & Copyright

The reproduction, adaptation or broadcast, without permission, of any articles or photographs published in this publication, is forbidden and copyright
is expressly reserved for Test and Data Services (Pty) Ltd.

Opinions

Opinions expressed by contributors are their own and do not necessarily express the opinions of the Publisher, Editor or members of the Editorial Panel.