Software Quality Assurance

Formal SQA Definition

The correct definition of Software Quality Assurance goes something like:- The function of software quality that assures that the standards, processes, and procedures are appropriate for the project and are correctly implemented.

This definition is taken from Software Definitions at NASA The problem with this, and similar, definitions for commercial SQA practitioners are:- It tells us little about what SQA is other than repeating the definition. That is, it uses the defined terms 'assures' and 'software'. It doesn't provide a scope for someone responsible for Software Quality Assurance. It doesn't address the role, or relationship, with Software Testing. In its pure form under which a separate 'audit' style group needs to be established it is difficult to apply to a small development environment.

What is QA and QC?

The so called Quality Movement, which was first established in Japan in 1946 by the U.S. Occupation Force's, was based on W. Edwards Deming (USA) research and papers on Statistical Process Control (SPC). The various definitions and approaches to Quality Assurance come from Deming and other so called Quality Gurus.

The important point, for this discussion, is that these methods were applied to industrial production. That is the production of something tangible. Fujitsu's slogan Quality built-in, with cost and performance as prime consideration, typifies this approach. Under this approach the production process was broken down and specified. Each process would have an output (or component) that would have a required specification (measurement, weight etc.) and these would be verified as the main product was being built.

It would be a separate group Quality Control (QC) that would measure the components, at various manufacturing stages. QC would make sure the components were within acceptable 'tolerances', i.e. they did not vary from agreed specifications.

The Quality Assurance (QA) group would not take any part in the manufacture process itself (including Quality Control of the product) but would audit the process to make sure the established guidelines and standards were being followed. The QA group would then give input (metrics or measures) into a process of continuous improvement.

It is worth noting here that in manufacturing QC (or test and inspection) is easy to distinguish from Quality Assurance QA which is concerned with process compliance.

These QA and QC methods, in manufacturing, proved themselves to work (in Sales, Customer satisfaction and the right cost of production, i.e. PROFIT) and were adopted all over the world. QA and QC groups (for manufacturing) became the norm.

So far we are in great shape with QA and QC in the world of manufacturing, then someone came up with the bright idea Why don't we apply these proven Quality Management processes to Software?

The move to SQA and SQC

Hence SQA (and SQC) were born and with it came problems of definition and implementation.

The definition still refers back to the traditional manufacturing QA world. There are, however, some notable differences between software and a manufactured product. These differences all stem from the fact that the manufactured product is physical and can be seen whereas the software product is not visible. Therefore its function, benefit and costs are not as easily measured.

The following differences highlight some of the issues in taking the manufacturing QA\QC model and applying it to software development. The manufactured product is a physical realization of the customer requirements. The function of the product can be verified against this physical realization. The costs of manufacture, including rework, repairs, recalls etc., are readily categorized and visible. The benefit of the product to its user\customer are readily categorized and visible. In order to overcome these types of issues, and reap the benefit of QA\QC applied to software, other terms, models and paradigms needed to be (and were) developed.

In order to identify the Software Costs and Benefits, remembering Fujitsu's term with cost and performance as prime consideration, a number of Software Characteristics where defined. These characteristics are sometimes referred to as Quality Attributes, Software Metrics or Functional and Non-Functional Requirements.

The intention here is to breakdown the Software product into attributes that can be measured (in terms of cost benefit). Examples of these attributes are Supportability, Adaptability, Usability and Functionality.

There are many definitions of these Software Quality Attributes but a common one is the FURPS+ model which was developed by Robert Grady at Hewlett Packard. Under the FURPS model, the following characteristics are identified:-

Functionality The F in the FURPS+ acronym represents all the system-wide functional requirements that we would expect to see described. These functional requirements represent the main product features and answer the question What the product does for us rather than How does it do it.

The easiest way to think of functional requirements is to ask the question Why does this piece of software exist. This question of reason for being is distinct from security, look and feel and reliability concerns which are important but are not the concerned with the main function (or value add) of the software.

Usability

Usability includes looking at, capturing, and stating requirements based around user interface issues, e.g. issues such as accessibility, interface aesthetics, and consistency within the user interface.

Reliability Reliability includes aspects such as availability, accuracy, and recoverability, for example recoverability of the system from shut-down failure.

Performance Performance involves issues such as throughput of information, system response time (which also relates to usability), recovery time, and startup time.

Supportability This is a general bucket of requirements that address supporting the software: for example testability, adaptability, maintainability, compatibility, configurability, installability, scalability, localizability, and so on.

+ The "+" of the FURPS+ acronym allows for the specification of constraints, including design, implementation, interface, and physical constraints.

The specification of the FURPS+ characteristics needs to go into the Systems Requirements. The testing of these characteristics should be done by the SQC (testing team). Some of the FURPS+ characteristics, i.e. Functionality and Usability can be tested by executing the actual software. Some, however, like Supportability and Adaptability can only be verified by code inspection or dry running What if' scenarios.

It is important to note that neither the SQA nor SQC group should have the responsibility of putting the desired FURPS+ characteristic into the product.

The SQC group should only test the presence or absence of the FURPS+ characteristics, whilst SQA assures that everyone is following the correct procedures and standards during their process execution..

With an established practice of defining and measuring the FURPS+ (or similar model) characteristics it is possible to implement Software QA\QC along similar lines to manufacturing QA\QC.

Approaches such as the implementation of FURPS+ should, in theory, overcome the difficulties caused by the intangible nature of software, allowing each characteristic of the software to be measured by SQC.

By way of example, consider the Supportability FURPS+ characteristic. The effectiveness of the current (appropriate) standards and processes that impact Supportability can be measured by the length of time in takes to fix a defect. In order to improve this measure new coding standards could be implemented. In this scenario the SQC department would inspect the code to make sure that the coding standard was being implemented (in the code) and the SQA department would make sure the SQC and the development groups followed the correct (appropriate) process and standards. The SQA department would also collect and analyze the time needed to repair the defect (Supportability measure) in order to give input to the usefulness (aka the word appropriate taken from the standard QA definition) of the standards within a continuous process improvement initiative.