Software Testing Strategies

Software Testing Strategies

A strategy for software testing integrates software test case design techniques into a well-planned set of steps that cause the production of software. A software test strategy provides a road map for the software developer, the quality assurance organization, and the customer. Any testing strategy needs to include test planning, test case design, test execution, and the resultant data collection evaluation. A software test strategy should be flexible enough to promote the creativity and customization that are required to adequately test all large software-based systems.

A Strategic Approach to Software Testing

Testing is a group of activities that can be planned in advance and performed systematically. For this reason a set of stages that we can place particular tests case design techniques and test approaches should be developed for the software engineering procedure. A number of testing strategies have been identified, which provide a template for testing and all have the following features:

• Testing starts at the modular level and works outward towards the integration of the complete system.
• Diverse testing techniques are appropriate at diverse points in time.
• Testing is performed by the developer of the software and an independent test group.
• Testing and debugging ate diverse activities, but debugging must be included in any testing strategy.
  

A strategy for testing must include low-level tests that are required to verify that a small source code segment has been implemented correctly as well as high-level tests that that validate major system functions based on customer requirements.

Verifications and Validations

Software testing is one type of a broader domain that is known as verification and validation (V&V). Verification related to a set of operations that the software correctly implements a particular function. Validation related to a different set of activities that ensures that the software that has been produced is traceable to customer needs.

Organizing for Software Testing

For each software project, there is an inherent that happens as testing starts. The people who produce the software are required to test the software. Unfortunately, these developers have an interest in showing that the program is error free, it matches the customer’s needs and was completed on-time and within budget. The role of an independent test group (ITG) is to take out the inherent difficulty associated with allowing the builder to test the things that are built. The ITG works with the developer through out the project to ensure that the testing carried out is at the correct level. The ITG is part of the software development process in that it becomes involved during the specification stage and stays through out the project.

A Software Testing Strategy

The software engineering procedure can be seen as a spiral. Initially the systems engineering states the role of the software and lead the software requirement analysis, where the information domain, function, behaviour, performance and validation criteria for the software are identified. Moving inwards along the spiral, we come to design and finally coding.

A strategy for software testing may be to move upward along the spiral. Unit testing happens at the vortex of the spiral and concentrates on each unit of the software as implemented by the source code. Testing happens upwards along the spiral to integration testing, where the focus is on design and the production of the software architecture. Finally we perform system testing, where software and other system elements are tested together.

Criteria for Completion Testing

A fundamental question in software testing is how do we know when testing is complete. Software engineers need to have rigorous criteria for establishing when testing is complete. Musa and Ackerman put forward an approach based on statistical response that states that we can predict how long a program will go before failing with a stated probability using a certain model. Using statistical modeling and software reliability theory, models of software failure as a test of execution time can be produced. A version of failure model, known as logarithmic Poisson execution-time model, takes the form

where f(t) = cumulative number of failures that are anticipated to happen once the software has been tested for a particular amount of execution time t

= the initial failure intensity at the start of testing

p = the exponential reduction in failure intensity as errors are discovered and repairs produced.

The instantaneous failure intensity, l(t) can be derived by taking the derivative of f(t): (a)

Using the relationship noted in equation (a), testers can estimate the drop off of errors as testing progresses. The actual error intensity can be plotted against the estimated curve. If the actual data gained during testing and the Logarithmic Poisson execution-time model are reasonably close to another over a number of data points, the model can be used to estimate the total test time required to produce an acceptably low failure intensity.2.2 Unit Testing
Unit testing concentrates verification on the smallest element of the program – the module. Using the detailed design description important control paths are tested to establish errors within the bounds of the module.
2.2.1 Unit test considerations

The tests that are performed as part of unit testing are shown in the figure below. The module interface is tested to ensure that information properly flows into and out of the program unit being tested. The local data structure is considered to ensure that data stored temporarily maintains its integrity for all stages in an algorithm’s execution. Boundary conditions are tested to ensure that the modules perform correctly at boundaries created to limit or restrict processing. All independent paths through the control structure are exercised to ensure that all statements in been executed once. Finally, all error-handling paths are examined.

Unit test

2.2.2 Unit test procedures

Unit testing is typically seen as an adjunct to the coding step. Once source code has been produced, reviewed, and verified for correct syntax, unit test case design can start. A review of design information offers assistance for determining test cases that should uncover errors. Each test case should be linked with a set of anticipated results. As a module is not a stand-alone program, driver and/stub software must be produced for each test units. In most situations a driver is a “main program” that receives test case data, passes this to the module being tested and prints the results. Stubs act as the sub-modules called by the test modules. Unit testing is made easy if a module has cohesion.

2.3 Integration Testing

Once all the individual units have been tested there is a need to test how they were put together to ensure no data is lost across interface, one module does not have an adverse impact on another and a function is not performed correctly. Integration testing is a systematic approach that produces the program structure while at the same time producing tests to identify errors associated with interfacing.

2.3.1 Top-Down integration

Top-down integration is an incremental approach to the production of program structure. Modules are integrated by moving downwards through the control hierarchy, starting with the main control module. Modules subordinate to the main control module are included into the structure in either a depth-first or breadth-first manner. Relating to the figure below depth-first integration would integrate the modules on a major control path of the structure. Selection of a major path is arbitrary and relies on application particular features. For instance, selecting the left-hand path, modules M1, M2, M5 would be integrated first. Next M8 or M6 would be integrated. Then the central and right-hand control paths are produced. Breath-first integration includes all modules directly subordinate at each level, moving across the structure horizontally. From the figure modules M2, M3 and M4 would be integrated first. The next control level, M5, M6 etc., follows.

The integration process is performed in a series of five stages:

1. The main control module is used as a test driver and stubs are substituted for all modules directly subordinate to the main control module.
2. Depending on the integration technique chosen, subordinate stubs are replaced one at a time with actual modules.
3. Tests are conducted as each module is integrated.
4. On the completion of each group of tests, another stub is replaced with the real module.
5. Regression testing may be performed to ensure that new errors have been introduced.

2.3.2 Bottom-up Integration

Bottom-up integration testing, begins testing with the modules at the lowest level (atomic modules). As modules are integrated bottom up, processing required for modules subordinates to a given level is always available and the need for stubs is eliminated.

A bottom-up integration strategy may be implemented with the following steps:

1. Low-level modules are combined into clusters that perform a particular software subfunction.
2. A driver is written to coordinate test cases input and output.
3. The cluster is tested.
4. Drivers are removed and clusters are combined moving upward in the program structure.