Smartesting
Last Updated:
Analyst Coverage: Daniel Howard
Smartesting is a software company that specialises in model-based software testing and test design. It was founded in 2003 and is based in Paris. In the past, it has primarily operated in France, but is now increasingly offering its products internationally. Its latest products are Yest, released in 2017, and Orbiter, released in 2019.
Smartesting Yest and Orbiter
Last Updated: 6th February 2020
Yest is a test design automation product that allows you to build visual representations of the scenarios you wish to test, and from those representations automatically generate your testing assets. It distinguishes itself by focusing exclusively on test design, opting to leave test execution and test management to third party products.
Orbiter is a browser-based application that automatically generates models corresponding to your existing manual test suite, leverages AI and machine learning to sanitise that suite, then exports the generated models into Yest to either serve as the basis for your test design automation or simply to publish them elsewhere. In the latter case, Orbiter and Yest are effectively deployed as a solution solely for optimising your manual tests.
Yest models your test scenarios via a series of graphical application workflows (see Figure 1) where each workflow represents the behaviour of a particular process you wish to test. Each step in your workflow contains a decision table, which itself contains a series of ‘decisions’ that are used to describe the cases to test and to control movement within your workflow based on the data being passed through it. Each decision can also be equipped with additional, bespoke information, such as risk, priority, requirements, and so on. This can be used to link decisions, as well as the tests they are used in, to user stories in, for example, Micro Focus ALM or Atlassian JIRA. Each workflow also has a coverage view, which can be drilled down into to see which decisions aren’t covered by your existing tests. This coverage view can be displayed graphically as an overlay for your workflow.
Test scenarios are generated automatically, creating one test scenario for each decision within your decision tables. Each scenario has both a model view and a textual view, as shown in Figure 2. Both can be edited freely but are always synchronised: any changes made to one will automatically regenerate the other (and if this is not possible, Yest will tell you why). In addition, any wider-ranging changes made within a test scenario can be propagated out to your other testing assets automatically.
Test scenarios and test data can be combined to create test campaigns, which will generate all combinations of scenario and data automatically. This allows you to create complete, end to end tests supplied with a variety of data. Although you cannot execute test campaigns natively in Yest, you can export them to a variety of products, including Micro Focus ALM, Atlassian JIRA, Microsoft Excel, Cucumber, and so on. When exporting to, say, a test management product, the text of your test scenarios is exported directly, augmented with additional description if necessary.
Exporting to a test execution framework, on the other hand, will require you to automate your tests using Yest’s keyword system. In many cases, Yest is able to automatically create a mapping between Yest keywords and the keywords in your execution framework; in other cases, you will need to import a mapping manually. Either way, you will need to assign keywords and parameters to each action used in your test scenarios. From this, Yest will generate an appropriate, executable test script or feature file which can then be exported into your test execution framework.
Orbiter uses AI clustering and natural language analysis to a) refactor your manual tests by altering their structure, matching existing text to keywords, or replacing static data with data variables, and b) arrange them into broad, easily manageable groups. This serves to organise and streamline your testing suite, eliminate redundant tests, and ultimately prepare it for automation. Refactoring suggestions, complete with previews and one-click implementation, are provided for individual tests, groups of tests, and your suite as a whole. Finally, Orbiter will generate workflows (identical to the ones used by Yest) for each of your tests or groups of tests on the fly as you refactor them. These workflows make it easy to understand the structure of your tests, even as you change that structure, and, moreover, can be exported into Yest and used as the basis for your test design automation. All of the above can be seen in Figure 3.
Yest’s highly specialised structure, and integration with other products, provides significant flexibility in deployment. Since it is focused solely on test design, you can leverage its full capabilities in conjunction with a test execution and test management platforms of your choice. This makes it easy to slot it into your existing toolchain. It also suggests a separation of concerns, wherein testing is a collaboration between test design, handled by less technical users such as business analysts, and test execution, handled by developers or dedicated testers.
Orbiter is also a significant selling point for Yest. It provides a cost-effective way to organise your existing manual tests as well as a base for more advanced testing techniques. Alongside Yest, it significantly accelerates the creation of your models, and therefore your implementation of test design automation. Moreover, since it is a refactoring tool, this is the case even when your existing tests are not in a suitable state to support model creation. In short, Orbiter makes the transition to test automation or test design automation much smoother and easier, and should not be overlooked.
Yest also benefits from a particularly intuitive form of model-based testing. The workflows it uses are simple and transparent; in the case of test scenarios, they are fully synchronised with their textual description; and they are built around individual processes, rather than your entire system, meaning there is no pressure to model everything, or to model exhaustively. This all adds up to a product which is very easy to use.
The Bottom Line
Yest and Orbiter, in combination, are a highly effective way to either transition from manual testing to test design automation, or to add test design automation to your existing test automation process. Orbiter, in particular, will be highly valuable if you’re struggling with an inadequate manual testing suite.
Commentary
Coming soon.