Tricentis

Fig 01 - Test automation in Tricentis Tosca

Tosca allows you to build test case designs (more easily understood as object models) that represent the data entities contained within your application, and from them automatically generate qualitatively distinct sets of test data. The product also enables you to codelessly create modules, reusable pieces of UI interaction, by recording your actions in an application (which could be desktop, mobile, or browser based), by scanning the aforementioned application and selecting the appropriate objects and interactions manually, or by scanning a service or service definition (which can subsequently be virtualised instantaneously).

Your modules (in this case best thought of as test steps) can be assembled into test cases, which can in turn be equipped with test data, either manually or by linking them to a test case design. This creates a test case template, which will automatically generate a selection of test cases populated with the associated test data. Test cases themselves can be executed manually or automatically from within the product. This highly encapsulated design means that modifying any individual element of a test will rarely require additional changes to the rest of it. Even for those instances, primarily pertaining to test data, change management requires relatively little manual effort.

Fig 02 - Risk Based Requirements in Tricentis Tosca

Tosca also allows you to create and manage requirements and link them to your test cases, thus providing, among other things, risk-based test coverage. The product estimates the importance of each requirement via criticality values provided by the user, assigns it an appropriate weighting using a simple algorithm, and calculates test coverage for each requirement as a percentage of the total weight. In other words, Tosca measures test coverage for each requirement in terms of the risk posed by that requirement. This can be seen in Figure 2. In addition, the results from any test cases that are executed within the platform will be fed back into your requirements and displayed as part of the execution state. This is also weighted by risk in the same manner as test coverage.

Tosca provides several complementary capabilities apart from test automation, including test data management and service virtualisation. Other Tricentis products integrate with Tosca at various levels, providing performance/load testing, agile test management and automated, AI driven impact analysis. Tosca also supports over 160 technology specific adaptors as well as customer specific extensions, which allow it to integrate with all manner of development and application environments.

There are a number of reasons to recommend Tosca as a solution for continuous testing and test automation. To begin with, it makes it very easy to both create and maintain executable test cases. The approach taken to automation allows anyone to create test steps, and therefore test cases, because it doesn’t rely on writing code. What’s more, treating sets of test data, test steps and test cases as individual objects that can be combined to create your actual tests makes maintenance and change management much easier, since any change to the constituent parts of your test will, in most cases, automatically propagate to the whole.

Another strong point for Tosca is its breadth, as well as its capacity for integration. The various capabilities offered by Tosca and other products in Tricentis’ range allow it to form the core of a competent and comprehensive continuous testing solution that includes test data management, service virtualisation, performance testing, and agile test management, among other things. Moreover, the sheer quantity of technology adapters that Tricentis provides means that it is highly likely that Tosca will be able to fit into your existing environment and the technology that you are using within it, as well as enabling the possibility of end to end testing.

The risk-based approach taken by Tricentis is also interesting. Although the company is not alone in offering risk-based testing, there are few vendors in the space that position it as the crux of their offering in the same way as Tricentis. On the one hand, we appreciate the emphasis on real business outcomes that risk-based testing provides, particularly when it’s positioned so prominently. On the other, we would like to see more rigor in the methodology that Tricentis uses. In effect, it is not automated enough, with too much left up to the user. This is also a problem TestSuite has when it comes to test design: although individual test steps can be created and executed automatically, they must be assembled into a complete test case manually. We would like to see additional automation here.

The Bottom Line

Tricentis Tosca is a well-executed test automation product that, along with the rest of Tricentis’ products, is capable of providing a highly comprehensive continuous testing solution. If its standout features, most notably codeless test creation and its risk-based approach to testing, appeal to you, it is certainly worth your consideration.

Figure 1 – Automated message validation in Tricentis Tosca Orchestrated Service Virtualisation

OSV allows you to create virtual services by using the OSV Recorder to record requests and responses to the real service you would like to emulate. You can record interaction with the service either by interfacing with it manually or by running an automated UI test (for example, using Tricentis Tosca TestSuite). The result will be a recorded sequence of requests and responses that you can then group together to create an OSV scenario corresponding to the test (or sequence of manual inputs) that created it. In either case, your virtual service will now respond appropriately to requests that are specified in the recorded scenario, sending the appropriate responses at the appropriate times. Scenarios are stateful, so they won’t respond to valid requests if they do not come in a valid order. You can also configure your scenarios to parameterise their responses based on the preceding request. For example, your scenario could take an arbitrary date as input in a request, then return the following day as a date in the response. Additionally, instead of using the OSV Recorder, you can create virtual services from existing APIs using the built-in Tosca API Scan.

You can record and create scenarios for any number of tests. However, doing this for every possible test is not necessary. Instead, you can import a scenario, or set of scenarios, into TestSuite. This allows you to remove contextual data from your scenario and transform it into a test case template. You can then attach sets of test data to this template (created through TestSuite’s test data generation capabilities), generate test case instances from the combination of test case template and test data, and finally export those instances back into OSV as new scenarios for your virtual service. Test data that is attached in this way can leverage pattern matching (via wildcard operators) to allow for a variety of data in the request while still eliciting the same response. Moreover, since you can attach the same test data to both automated UI tests and imported scenarios, OSV and TestSuite together allow you maintain a single, centralised test data repository that provides test data to the entirety of your testing environment. Importing scenarios into TestSuite also gives you the option of viewing (and manipulating) them visually in a sequence flow diagram. In addition, virtual services created in OSV feature built-in, automated message validation. As a scenario is run, OSV will automatically analyse and validate all incoming data to make sure it is properly formed and sent in the correct order. Once the scenario is completed, any requests that do not conform to these standards will be flagged. This can be seen in Figure 1.

Testing is an extremely important part of the software lifecycle. Furthermore, test automation and continuous testing are a necessity in order to enable fast, efficient testing that can keep pace with modern development, particularly in an Agile environment. But you cannot have true continuous testing when your tests are dependent on services that may or may not be available at any given time, often resulting in bottlenecks in your development pipeline. This is where service virtualisation comes in. By constructing virtual services to test against in place of real services, you can create tests that are independent of any external systems and hence can be deployed at any time, thus enabling genuine continuous testing and removing the aforementioned bottlenecks.

OSV is particularly well placed to enable this capability, due to its tight integration with Tricentis Tosca TestSuite, a best-of-breed test automation and test design automation product with a notable capability for enabling continuous testing. Together, these products enable the rapid creation and integration of virtual services with automated UI tests, creating a new approach that Tricentis refers to as ‘Test-Driven Service Virtualisation’. This terminology is apt, with the products mentioned allowing you to drive the creation, automation and maintenance of your virtual services via your UI tests, themselves created and maintained through TestSuite. This test-driven approach places tests and testers at the heart of service virtualisation, and makes it much easier for you – and, particularly, your testers – to manage and maintain your virtual services.

The Bottom Line

OSV is a capable service virtualisation product whose standout (though by no means only) feature is its integration with Tricentis Tosca TestSuite, itself a best-of-breed test automation product. Conversely, the existence of OSV adds significantly to the appeal of TestSuite as a continuous testing platform. Ultimately, if you are already using TestSuite, we see very little reason not to leverage OSV as part of your testing solution.