Software Test Automation – Can Automated Test Tools Replace Human Testers?

Software Test Automation – Can Automated Test Tools Replace Human Testers?

Many smart executives have decided long ago to only sponsor projects in which execution can be either automated or outsourced.

At first glance, this sounds reasonable. However it must be noted that project setup for both automation and outsourcing is expensive, so a lot of planning must be completed before proceeding.

And then there is the matter of all the media attention surrounding the human costs of automation and outsourcing — namely job displacement, leading to retraining, attrition or layoffs.

As such, a good number of workers have been, and will likely remain, somewhat ambivalent about automation and outsourcing.

In his May 2007 Salon article entitled “You can’t stop a tidal wave with a fork“, David Silverman tells the story about how global market forces dismantled his 200 employee typesetting business in Iowa.

One interesting item Mr. Silverman mentioned was that some of his employees were stubbornly resistant to change:

quote openSome of our employees, rather than learn to use the computer effectively, would lean a stapler on the keyboard to ‘automate’ a task.”

Software Test Automation

Software test automation is the process of designing and executing software scripts to test commercial software before product release.

Software test automation is best suited to regression testing whereby the goal is to ensure that new features and bug fixes introduced into the latest release have not caused any new software defects. To regress is to go back to a previous (less developed) state.

Success

Software test automation has been applied with remarkable success to:

  • Regression testing
  • Performance testing
  • Stress testing
  • Load testing
  • Application Programming Interface (API) testing
How to Automate Software Testing

To get started developing a software test automation suite, consider the following areas first:

  1. Groups of test scenarios that organize logically based on similar requirements
  2. Tests that are repetitive, tedious or require high levels of computational cycles
  3. Test scenarios for software features and modules that are unlikely to change but still require on-going testing
Limitations of automation

The two biggest limitations of software test automation are with respect to the skills needed to (1) test the User Interface and (2) analyze test results.

User Interface Testing

Testing the User Interface (UI) is the key differentiator between automated testing by software tools and manual testing by human testers.

Automation is not appropriate if the UI is unstable, constantly changing or a candidate for a major redesign.

During the early stages of a software development project, the UI typically changes frequently. As new features are added, removed or changed the UI is changed as well. The impact to test automation is that even the slightest change to the UI causes automated test scripts to fail.

The lesson learned is that the time to start a software test automation project is when code changes affecting the UI have stabilized.

There are two caveats to the “freeze the UI before automating” guidance.

Firstly, if the automation test strategy leverages the API, then an unfinished or changing UI is not relevant because the UI is not used as the input for automated test commands.

Secondly, if software developers are responsible for not only delivering new features and bug fixes, but also tasked with delivering automation test script updates, then an automated test strategy can be started much earlier in the product development cycle. This is because each software release takes into account impacts to the UI so that new features and bug fixes do not “break” automation test scripts.

Test Results Analysis

In addition to the up front impact analysis required by software developers so that the investment in software test automation is preserved, results from automated testing require analysis by human testers.

Software Test Automation is, at its core, merely a tool enveloped in process. The assessment of the actions of the automation tool and the analysis of test results are higher level intellectual activities that have been, and will likely remain, the province of human testers.

Return on Investment

What is the return on investment for software test automation?

Setup costs for software test automation are high and only add to testing budgets.

If the goal of test automation is to reduce costs, then from where are the savings coming?

To justify investments in software test automation, executives are expected to measure the cost of defects, not merely in terms of their test budget, but more broadly with respect to the costs impact in resolving post production defects. This investment analysis should encompass the perspectives of both the software development company and its customers.

Test Automation vs. Human Testers

Not all tests can and should be automated.

Executives looking to slash test budgets are unlikely to succeed with the mindset that automation can replace human testers. There are however significant long-term advantages to software test automation that can not be ignored from a competitiveness perspective.

Developing a software automation test suite capable of both saving time and money requires a moderate investment.

The biggest advantage to developing an automated test suite is that software testers can be freed to design ever more creative test scenarios which, along with the proper tools and processes, are key to achieving overall software quality assurance.

To hire software test automation engineers and project managers that understand how to automate software testing, complete a staffing request with Global Technology Associates.

Tags: , , , , , , , , , , , , , , , , , , ,