A View on Test Automation

Testing has been a long and laborious task for many years, probably since its inception. I can’t comment on that, as I was not around. I have seen the introduction of automation as a means of reducing the time taken to test, but as with all new ideas, people jump on the band wagon and the results are not always as expected.

With vast sums of money invested, the cost of automation was high (when compared to a manual tester), a skill set which was hard to come by, but with the results of their labours there was an expectation that greater savings would be made. One man would be able to do the work of ten etc. The results however often failed to live up to expectation over a period of time. Early successes were not invested in and the result was a lot of automation started to gather dust.

One of the leading sellers of automation software once indicated that an automated script would take seven times longer to write, but would take one seventh of the time to execute. This would mean that a script that was automated would need to be run seven times without change in order to cover the cost of its creation. Here lies the main problem.

Initially automated scripts were often based on a record and playback capability, which whilst of extreme use had a finite life if the application was changed in any way. As time has moved on this has now become a basic form of automation and the capability is now more based around intelligence, not relying on the position in which it appears on the screen, but the fact that it is there as an object. It must still be understood that an application which is undergoing regular and significant change, is far less suitable than something which is relatively stable and unchanging.

Automation does however have a place. Regression testing, which relies on the same scripts being repeated, is an ideal candidate for automation. Another area is smoke testing or sanity testing, the process of ensuring that an application is performing as expected with a reduced regression set, perhaps for the purposes of checking that a test environment has been set up correctly. It should also be remembered that certain applications will go through many functional cycles of testing for a single release, again making them more suitable for automation.

This method of testing is often viable and the greater the capability of the automation engineer the more likely the success of the automation. Try to avoid a wholesale approach, which is likely to leave you with a lot of shelf-ware gathering dust. Look at each project and application individually and understand whether you are likely to obtain the returns you seek. Look at all the types of products that are available and select one that suits your purpose, but also for which the skills are available. Last, but by no means least, ensure that you plan the automation from the start, allowing for the extra time to develop the scripts. Once started, continue to evaluate the approach, bearing in mind that the scripts are likely to need updating with changes to some extent, generating a maintenance cost.

If you are looking at automation of regression packs, or for smoke testing purposes, consider the offshore option. The scripts are pre-defined and therefore the work understood. The cost is as always significantly lower.