I must admit I was initially rather perplexed as to how to comment on the theme of testing from a customer’s perspective. For starters, we need to get to grips with identifying who the customer is. As an employee in a company offering professional services in the software testing arena, our customers are all too often those tasked with developing and delivering a solution to their business. To pen something from that perspective, however, would be to deny the reality that the real customer is that part of the business that needs a software solution to deliver their business offering to their customers.
A second area of quandary arises when one realises that the level of each customer’s understanding of software testing can vary widely. As this has already proved to be the case in the IT space, it must certainly be so in the business arena. The sad reality is that, although we can quite easily identify the real customer, it is far more difficult to surmise any kind of universal view that the customer may have concerning his software quality and the testing effort required.
My guess is that the majority of customers would take the view that their software ‘should work in the way we requested it should work, and it’s the job of the software supplier to ensure this’. Herein lies the rub, for it is far easier to request or demand quality in such an all-encompassing way than to ensure it, even more so when the one requesting it has little or no understanding of the implications that accompany such a request. Unfortunately, the real value of testing is all too often lost on a customer, especially if his solution provider is already doing a subjectively ‘reasonable’ job of testing. It usually takes a high-profile and costly defect to convince a customer that more time, money and effort should have been spent on testing before software release. Up to the point of failure, most customers will perceive testing to be an expensive evil, with it being very debatable as to whether it is necessary or not to test as much or as little as they do. This remains the case despite the fact that the same customers will all agree that the software must work.
So we can certainly identify that the customer wants software that works and expects that it has been sufficiently tested to meet all of his possible requirements. He certainly doesn’t want the testing effort adversely to affect the time to market, nor does he want the testing monetarily to cost very much, despite the fact that over 50% of a project’s budget should be spent on some or other form of verification and validation activity if these requirements are to be met. The contradiction of these wants is obvious and could be summed up as “the customer’s requirement is for a test team that costs next to nothing whilst managing comprehensively to test every aspect of the system in no time.”
In my mind, however, probably the greatest danger when it comes to in-house or custom-developed software is the customer’s expectation that it meets the same standards of quality we have come to expect from commercial software that has been built by massive software development teams for mass-market use.
One of the easiest ways to reduce the testing effort is to ensure that the analysis and requirements phases of the SDLC keep to realistic expectations. For example, requesting a sophisticated multi-level, highly flexible user profile sub-system is all well and good, but perhaps completely unrealistic if it implies costly and difficult programming and an even greater [time-consuming and costly] testing effort in order to assure it.
This is where the discussion must depart from testing per se and examine the roles of IT project management and business analysis in the SDLC for a moment, purely because they have a huge influence on how the customer will perceive the value of testing in his project. A phased approach to project costing has a far greater chance of forecasting realistic costs when compared to an approach that tries to project the full project cost prior to any detailed analysis. Similarly, a business analysis methodology that incorporates a detailed evaluation of functionality in terms of need and testability provides the project with better cost-cutting and decision-making ability at a later stage. The reintroduction of the acronym KIS (Keep It Simple) is long overdue. Too many in-house developed systems are over-sophisticated for the underlying business imperative.
In my experience, the real customer is seldom questioned to any meaningful depth as to what software quality criteria would be satisfactory vs. non-negotiable and, as a result, the majority of software testing that currently takes place is geared towards User Acceptance Testing (UAT) and Functional Testing, often to the total exclusion of other possible and important types of formal testing such as Performance, Security, Volume and Unit / Component Testing, to name but a few. Furthermore, the situation is unlikely to improve much in environments where testing is not integrated into the full SDLC as a set of necessary and ongoing activities. Bolting on testing at a later stage in a system’s development cycle is fraught with danger. Think of trying to design and integrate comprehensive security into an already-developed system. All too often I have seen project managers requesting testing services after the project budget has been determined without taking testing into account!
Conclusion:
Just as it is really important to ensure the right amount of time and effort is spent on the up-front specification of system requirements, irrespective of the SDLC methodology being pursued, it is equally important to determine what amount of effort will satisfy the owners of the system (the customer) that these requirements have been met. As the pressure to deliver more complex business solutions in shorter timescales continues to mount in the systems’ development arena, one certainty is starting to emerge when it comes to testing such solutions: smarter, quicker, leaner and more integrated testing processes will have to be found if those responsible for solution delivery are to have any hope of meeting the customer’s desire for quality software. These points give rise to the deliberation over an investment in automated testing products, processes, and people; a difficult value proposition while the value of testing remains unclear to any customer.
Dave Shaw
info@testfocus.co.za
The reproduction, adaptation or broadcast, without permission, of any articles
or photographs published in this
publication, is forbidden and copyright
is expressly reserved for Test and Data Services (Pty) Ltd.
Opinions expressed by contributors are their own and do not necessarily express the opinions of the Publisher, Editor or members of the Editorial Panel.