An Introduction to Testing

It is amazing in this day and age, that we are still asked why you should test. When you are really passionate about a subject it is hard, to say the least, to go right back to basics and justify why you are needed as part of the Software Lifecycle.

Lets look at some definitions of testing:

http://www.answers.com/topic/test?cat=technology
“a procedure for critical evaluation; a means of determining the presence, quality, or truth of something”

http://www.askoxford.com/concise_oed/test_1?view=uk
“a procedure intended to establish the quality, performance, or reliability of something.”

http://m-w.com/dictionary/test

“the procedure of submitting a statement to such conditions or operations as will lead to its proof or disproof or to its acceptance or rejection”

To put our own interpretation on this, we believe that testing helps to:-

- Find problems early

- Identify if build meets requirements

- Improve user perception

- Avoid unnecessary outages

- Reduce maintenance

- Establish quality of purchases

- Understand the risks

It is not the complete answer and will not ensure that your code is bug free before it goes live, but it will enable you to understand the risk of going live. Testing looks to perform sufficient checks to understand the quality: it cannot on it's own resolve the problems. Testing does not look at every single permutation, or we would be testing forever, but looks to test sufficient permutations to give confidence in the level of quality attained. In certain instances, such as safety critical applications, the volume of testing does increase as the confidence required needs to be higher.

The above graphic shows a series of testing types. This is often an area of confusion for someone new to testing, but each of these has a specific function. They can also often be references to the same thing, such as White Box and Unit from the above list. Part of the skills of the test manager are in knowing what test types to apply to a project and the level of intensity of the testing performed therein.

Different types of testing are performed by different people or roles associated with a project. The graphic below shows a series of testing types and the roles that would normally be expected to perform the testing. It is worth noting that the Test Manager associated with a project, would take responsibility for all testing to some greater or less degree.

From a Testing perspective, the testing is broken into 5 primary areas, Engagement, Planning, Preparation, Execution and Completion. A series of deliverables are produced through the life-cycle of a project

Strategy: A document defining the types of testing to be applied; the criteria that allow you to start or stop testing. The shaping activities such as risk identification.

TTRM: Traceability to Requirements Matrix, providing traceability from each test case/scenario back to the original requirement specification. Proving that testing is covering the full set of specifications.

Test Plan: A document defining the type of testing to be performed, the test cases/scenarios to be covered, when and by whom.

Test Schedule: A project management type schedule showing all activities performed by what resources, at what time and for what duration. Further dependencies on other activities will also be included.

Test Scripts: Detail of the tests to be performed, the steps to go through and the expected results, with reference back to the test cases/scenarios.

Defect: Detail of each script that has failed, why it has failed and under what circumstances. The priority of the defect, it’s severity to testing and additional information allowing the defect to be replicated.

Completion/Exit Report: A summary of the findings of the testing team, showing evidence of issues and stipulating an opinion of the readiness of the item under to test to be placed live.

Regression Pack: A suite of scripts that can be reused to test the primary functionality of the item in the future.

To summarize, we believe that testing is a discipline in the same manner as analysis, architecture, design or development. It helps the client understand the risks associated with the release of a newly developed or modified application. It is an essential means of introducing quality into the software development life-cycle. It protects the users of the system, looking to ensure that IT is delivering what was requested. Pre-defined outputs are produced naturally throughout the engagement.