Entry and Exit Criteria

As you pass from one phase of testing into the next, there is a need for control. For the purpose of this post, we will refer to the prior phase as the supplier and the current phase as the recipient. The supplier needs to retain control of their testing phase, until such time as it is deemed ready for release. The recipient needs to ensure that the testing performed by the supplier has achieved a sufficiently high standard as to be acceptable.

The means of achieving this is referred to as Exit Criteria for the supplier and Entry Criteria for the recipient. These criteria are documented in the test plan and define the standards that should be achieved entering and exiting, the test phase described by the document.

The criteria are set by the Test Manager or nominated delegate. They may take any form that the test manager deems necessary, although are more frequently based on defect volumes of certain severity and priority, along with test assets from the supplier.

Consideration should be given to the use of other information, drawing on experiences of dealing with particular project teams, development agencies etc. This requires the Test Manager to be able to think outside of the pure testing aspects of the project, to look and see what else may impact them. Closer to the heart of the Test Manager are subjects like the Test Environment/s and Configuration Management. The environment is one of the main enablers to testing and should be made available two weeks prior to test execution commencing. Without comprehensive Configuration Management the likelihood of controlled deliveries from development into testing is severely reduced. Such events and dependencies make excellent criteria and are sound reasons for not entering into a subsequent phase of testing.

Looking at the testing components only, Test Managers should ensure that the supplier of each testing phase is held accountable for the production of the associated test assets. The volume of defects is often seen as a point of contention. Test Managers should avoid making statements such as zero defects allowed, or setting the bar so high that entry or exit becomes impossible to achieve. Having to relax these criteria because they are impractical corrupts the integrity of the testing function and undermines the credibility of the Test Manager. Work with the Project Manager when setting these criteria, ensuring that they are bought into them and don’t rely on just a signature on a document. This gives a higher level of buy in and ensures that any future change to the criteria is going to be resisted by both the Test Manager and the Project Manager. Always ensure that such changes are formally change controlled.

It is recommended that on entry into Integration in the Large, there should be no top priority or top severity defects outstanding. This is not to say that there can be no defects, only that items which are so important to the business as to warrant a top priority, should be resolved as a priority. Defects of the highest severity are perhaps more debatable, but delivering an item with high severity defects indicates that elements of the testing cannot be completed and the deliverable is therefore significantly below the level of functionality that has been specified and is expected.

If circumstances dictate that entry or exit criteria are going to be overruled, as a minimum the risk register needs to be updated to reflect that this has occurred, detailing the impact and mitigating actions. It is worth noting that this is normally indicative of a project in trouble and that timescales are now seriously at risk.