Test Design Automation

There is no universal template for test design automation tools, but the majority leverage model-based testing to allow you to create or import a graphical representation (that is, a model) of the system under test. The details and complexity of this model vary by product, but in most cases, it will be a visual flow chart, where each step in the flow chart will be equipped with the necessary apparatus of testing, such as test data, test steps, and so on. Often, the model will also be able to link back to requirements generated either through the product or by a requirements management tool. The product can then generate test cases corresponding to different routes through your model, often in combination with distinct sets of test data. These test cases can then be executed within the product or exported as test scripts to a dedicated test execution framework. If the model is later changed, most products feature change or impact analysis to automatically modify or recreate affected test cases. In addition, many of the products in the space can be configured for continuous testing, so that this entire process can be done automatically.

Figure 2 – “Our automation efforts so far have already saved over five man-days of effort per release. That equates to around a 30% time-saving on testing every month.” Jet2.com

Manual testing is incapable of keeping pace with modern development. According to the Capgemini World Quality Report, 30% of IT spending is on testing, and this is expected to rise to 40% by 2019. Yet 44% of testing teams do not have time for core testing and, according to PAC, 89% of testing teams do not have sufficient budget to meet user expectations. These figures show that there is a desperate need for testing to be more efficient, both in time and cost. Automated testing will allow you to lower testing costs while increasing test coverage and raising the quality of your releases. For example, one major restaurant chain estimated that test design automation had saved it “2,000 man-hours of testing in the first year”, while a major telecom stated that it was able to use test design automation to achieve 100% test coverage across two applications that were previously stuck at 0% and 80% respectively.

Further research suggests that only 18% of testing teams believe they are delivering what the business wants and needs. The message is clear: testing as it currently stands is not producing a good experience for the end-user. If you want to meet customer expectations, you must invest in automated testing.

Quotes

“The quantitative benefit from [test design automation] was the time it took to create 1,200 test cases. It originally took 420 hours and with [test design automation] it took 70.”
Healthcare insurance company

“[Test design automation] will help us increase the efficiency of our business analysts 
by 10 percent and testers by more than 30 percent over the next three years.”
Rabobank

“The ability to be sure that a new web or mobile update is going to increase user satisfaction is worth millions.”
TUI

The key question for users will be how far to shift left. Fully-fledged test design automation products (which go furthest left) are relatively rare, as they represent the cutting edge of test automation. For example, you can choose between products that are essentially tabular (with an embedded object model), offerings that model the system under test and software that starts with requirements modelling. Further, there are other products in the test automation space that either enhance or overlap significantly with more advanced products, usually enabling a very significant amount of automation without automating test design per se.

Several spaces complement test design automation. Most products in this space support the import of requirements from relevant third-party tools, while test data management and service virtualisation products often integrate with test design automation. In addition, test automation itself works with test design automation. In fact, depending on your chosen product, automated test execution tools may be necessary to automate the whole of your testing process. Continuous testing is another natural way to extend test automation, and as such it may also be worth investigating continuous testing products.

What is the bottom line?

Test design automation provides a potentially massive boost to testing speed and efficiency. Properly leveraged, it can also dramatically improve the end-user experience.