Software testing

This information is to help students to make a plan for testing the software they have developed.  Testing is an important part of the marking schedule for 'O'-Level Computer Studies projects.  A good well executed plan is required.

The purpose of testing is to identify deficiencies in the system being developed. The testing activity takes place throughout the system development life cycle. Testing is to ensure quality.

Here are a few notes and suggestions for implementing a testing strategy.

We test to make sure that our software is of high quality. This means looking for two things

o Errors in system specifications. How can we go about making sure that our system specifications are accurate? Good communication with system stake-holders and meticulous attention to detail when constructing system model components are two approaches that will go a long way towards avoiding system specification errors. System specification errors lead to design errors that lead to programming errors that lead to a waste of time, goodwill and money.

o Errors in program code. (functions and procedures ) need to be tested destructively. After software modules have been tested the integrated system needs to be tested to ensure that modules work together. Strong coupling of procedures (which is bad programming) is a common source of errors that should be avoided. Syntax errors are easily and automatically detected by the compiler or interpreter that is being used. Semantic errors are errors that are much harder to detect. These types of errors produce unexpected results. The best testing strategies may not find all semantic errors because it is often too time consuming and too expensive to test every possible combination of inputs to make sure that processing will be carried out properly.

o Testing is a destructive process. This means that we have carried out the testing function successfully if we can show that an SDLC deliverable fails to meet requirements. We want to try and select inputs that will cause software failure. We want to try to prove that a system specification is wrong. It is clearly better to find these errors before a paying customer finds them!

The earlier errors are found the better. Errors found at the design phase are less costly to fix than after the software has been coded. This is because the later errors are found the further back we need to go in the SDLC to make a correction. Corrections to one aspect of design may have far reaching effects on other related system objects. For instance, a database table design change may mean that we must change 10 or 15 user interface screens and the associated code for each screen.

A simple testing strategy will address four levels of testing. These four levels are

1. Unit Testing
   This type of testing is to prove that modules do not operate according to design specifications. We must also prove that modules do not interface with each other.
   Example: A function designed to calculate the area of a triangle does not perform as expected when the inputs are abnormal – such as a height of -6 and base length of "quite big".
   
2. System Integration Testing
   The objective of this testing is to show that all the components of the system do not work together.
   Example: We create a user interface screen that has a save() function. After executing the save() function we move to a new screen and find we get an error message displayed. The problem being that the save() function of the previous screen changed the data environment for the information system, making it impossible to load information from a database view into the new screen. In this example the save() function of one screen interferes with the operation of the form_load() function of another screen. This particular example demonstrates a coupling problem, which we would be looking for in system integration testing. The example demonstrates how a module has failed to meet a programming requirement that the modules do not alter environmental or global parameters except in a predictable way.
3. User Acceptance Testing
   This testing is completed by end-users. It is to demonstrate that the system does not meet end – user requirements. Put another way, this testing is to show that the system does not meet or exceed user expectations.
   Example: A system is tried out by staff at a business where the new system is to be implemented. It is found that users have trouble navigating from one screen to another because buttons have unusual icons and there is no help function to guide users when they get stuck. This amounts to proof that the system is not ready for implementation. At this stage the system is tested in a controlled environment – not with live data.
4. Production Acceptance Testing
   The purpose of this testing is to prove that the system fails to operate according to user expectations in a real – life environment. This testing is normally carried out by system users with system developers acting as technical support staff.

1. Make a Gantt Chart that shows how the testing activities of your project are to be completed. Activities to be included are:
   Choosing test type and timing.
2. Making suitable test data sets that will exhaustively test the four categories above.

Here is a list of tests that might be included in a project. It is not necessary to complete many of these – though it is likely that all of them would be completed for a large expensive project.

Statement coverage test. This is a test to ensure that every line of code in your software has been exercised at least once.

Branch coverage test. To ensure that CASE and IF/ELSE statement alternatives have all been exercised at least once.

Loop coverage test. To ensure that loop statements start and end correctly (with the correct number of repetitions of code inside a loop).

Value sampling test. To make sure that a module works correctly when it receives data which falls in a normal range. For this test we use test data that is like that which we would expect to receive during normal operation of the system. We use this type of test when it is impossible to test for every possible type of data that might be used as input for a module.

Boundary value coverage tests. We use these tests to see if code works when input is at the maximum, minimum or outside boundary values for input. This test is also used to find errors in output. Will a report produced by a module display a very long name correctly on the page?

Program interface tests. To verify that different code modules in software integrate in a predictable way.

File handling tests. Does the program still work if a database table is empty or if the location of the database has been shifted to a new location?

Error message tests. Does the software provide helpful messages when predicted errors, such as user input errors, are detected by the software. The software would fail this test if an error caused the software to simply "crash".

Error-prone tests. Test the program with invalid data. This can be data that is of the wrong input field type. Enter a text value when a numerical value is expected.

Volume tests. To verify that the system can not handle the amount of information that it is supposed to handle as defined in the original specifications.

Performance tests. To verify that the system does not operate according to the stated response time.

Stress tests. To verify that the system can not handle unexpected situations. This test might be to used to verify that a program crashes when a connected LAN service is interrupted. The database being accessed might be on a LAN server. What happens if access to the server is interrupted because a cable has failed?

Usability test. To verify that the system is difficult for a human being to use.

Documentation tests. To verify that the technical and user documentation is satisfactory.

Compatibility tests. To demonstrate that the combination of equipment that is to be in use in the production environment is not going to work. Sometimes, with networks in particular, compatibility is a problem. Some NICs do not work with some types of cable. Some types of cable won’t work with different types of hub or switch and so on.

You should construct suitable tests for your project and provide (i) a list of the data that you have used (ii) the result of processing (iii) the expected result of processing (iv) a comment about the success or otherwise of the testing.  Remember that testing is successful when errors are found!

This short document is not a complete guide to testing. Any good system design and development textbook will provide a bigger list. The list above is based upon some of the tests suggested by Roger Fournier in his book called "Practical Guide To Structured system Development And Maintenance". Some ideas also came from a book called "Systems Analysis, Design and Implementation" by John G, Burch.