Test Estimation (or) How Long is a Piece of String?

Estimation is an art form at the best of times, depending on the experience of the test manager, the stage of engagement in the software development life-cycle and many other considerations. Different organisations will have different expectations and it is important to understand this as a key consideration. E.g. If you start including every possible variety of non-functional testing in your estimate, in an organisation that does not normally look at this, you are going to have a result which is wildly inaccurate.

Some companies will have rules in terms of percentages of the total project cost; 20% Analysis and Design, 40% Development and 40% Testing, for example. This will vary depending on an organisation's focus on quality, but could be as low as 10% for testing.

We would suggest that the following aspects are considered:

General Risk : How mature is the relationship with the development agency? How often the application is being changed? Are more requirements anticipated? Do Service Level Agreements governign defect fix turn-around exist? Is the Configuration Management process mature?

Environments : Do they exist? Are there processes for environment management? Are the environments maintained in line with production? Will the environments be established and proven prior to test execution?

Quality : Have the requirements been static tested? How many defects or production issues are currently associated with the application/s? What level of unit and integration in the small testing is being performed? How many defects were found during earlier phases? Is the project confident that entry and exit criteria are being met? Is the project already slipping or maintaining schedule? How mature is the project team – is this something they are all familiar with doing? Is this a safety critical application/system?

Effort/Scale : How many requirements are to be tested? What percentage of the application/s is/are changing? How many cycles of execution are required? Do regression test packs already exist? What deliverables are needed/required?

Complexity : How many suppliers and development agencies are involved in the process? How many technologies and platforms are involved? How many systems are being changed? How many systems are interfacing? How many stakeholders are involved with the project/programme?

Timing : Is the project taking place over a drawn out period of time? Will the project stop and start? Is the testing expected to be performed over a very short time frame when compared to the effort required?

Any of the above have the ability to influence what is needed to complete the project, impacting different phases of the testing life-cycle, or the volume of effort that is required by different resources. Don’t forget that when project size or complexity increases, the strain on the test manager increases, and the introduction of additional resource such as defect managers may be necessary.

The base estimate should look at the volume of scripts that are required. From this you can calculate how many can be written and executed in a day and work out resource requirements accordingly. The management time can often equate to a further 33% effort. As a guideline a conservative estimate would look to write 5 scripts per day and execute 10 scripts per day. This is dependent on the content of the script and the experience of the testers, but can be significantly higher. Bear in mind that the cycles of execution will each act at as a multiplier to the volume of test execution. Two cycles should be the minimum. More than four would be indicative of problems.

TCL India have a mechanism for applying the above considerations to our estimates and weightings associated with each of the different elements.