Curiosity Software Test Data Automation

Curiosity Software Enterprise Test Data (ETD) is a solution for test data management and automation. This means that it helps you to both create viable test data sets and embed (and thus automate) the creation and delivery of those sets into your testing processes. That said, Curiosity takes the approach that the point of testing, and therefore test data, is to enable you to build high quality applications. Accordingly, although ETD is principally concerned with the creation and delivery of test data, it does so with a holistic view of application quality. This manifests in a significant amount of functionality that is perhaps slightly tangential, but ultimately very complementary, to the core issue of generating test data.

For example, the product is highly concerned with understanding your existing data so that it can generate test data that accurately represents said data and meets your testing needs. As such, it includes various data analysis and profiling capabilities. This includes the (automated) creation of a data dictionary, coverage and risk analysis, sensitive data discovery, and relationship detection, among other things. In addition, the resulting traceability and relationship information can be stored and represented as a graph network built using Neo4j. On the other hand, the product also devotes considerable resources to making sure that the test data it creates is available in the right place and at the right time. This means that it provides test data in a continuous, on-demand fashion that can be delivered to any number of consumers in parallel. At the same time, test data is kept up-to-date, and test data sets are kept as covering as necessary while minimising their overall size. On top of everything else, the product’s test data processes are highly automated to ensure good performance and avoid testing bottlenecks. Moreover, all of this is delivered as part of a single, integrated platform, offering all of the usual advantages that provides.

By itself, ETD provides a range of test data utilities, not least those mentioned above. Its repertoire for creating test data includes subsetting and masking as well as synthetic data generation. When used alongside its sister product, Curiosity’s Modeller, ETD can overlay onto a visual model of the system under test and generate test data at the same time as Curiosity’s Modeller generates your test scripts. ETD can also connect with a variety of environments and third-party products, and a partnership with Windocks (as well as tight integration with its solution) provides access to containerisation and database virtualisation.

Customer Quotes

“We found the software extremely flexible in allowing us to set up various techniques to automate our data generation. In fact we couldn’t find a use case that we couldn’t manage to cover. What most impressed us was the ability to access difficult mainframe data sources such as VSAM, IMS and DB2 and we were even able to generate data into a CICS program and an MQ queue. We haven’t seen that flexibility with other software.”
Ostia

ETD offers a wide range of services through a web-based self-service portal. This includes an extensible library of hundreds of synthetic data generation and data masking functions, a data subsetting capability, and various test data utilities, such as data cloning, data reservation, data ageing, data comparisons, and service virtualisation. The product’s synthetic data generation functionality is particularly sophisticated, and leverages machine learning-driven, multivariate pattern analysis to determine and replicate the distribution of your source data (this technique is also used in other areas of the product, such as sensitive data discovery). Other capabilities include the ability to create clustered synthetic data for the purposes of AI or graph database testing, as well as advanced support for generating message data (including time series messages) by working backwards from your desired outputs using a solving algorithm. It is also possible to create synthetic data using generative AI, generating data tables directly from your prompts. The product additionally provides a test data catalogue that centralises all of your test data related activities, including test data usage, requests for test data, and the fulfilment of those requests.

Fig 1 – Overlaying test data onto a model in Curiosity Modeller

Moreover, ETD working in concert with Curiosity’s Modeller is a powerful combination. Curiosity’s Modeller allows you to construct graphical models that mimic your production environment, then leverage those models to automatically generate a set of easily maintainable and maximally covering test scripts. ETD can overlay test data onto those models, as shown in Figure 1, meaning that whenever your test scripts are generated, they will gather whatever test data they need automatically. In other words, using ETD and Curiosity’s Modeller together intimately marries test data with your test automation processes, enabling much more effective overall test automation. Note as well that test data attached to Curiosity’s Modeller is delivered “just in time” to ensure that it is always as up to date as possible when it is leveraged in your tests. Curiosity’s Modeller’s value also extends into several other areas adjacent to TDM, including collaborative requirements engineering, test case optimisation, and risk/change management. All of this combined should allow you to shift your TDM efforts significantly to the left.

Fig 2 – Visual representation of a data structure in Enterprise Test Data

In addition, production messages, APIs, and databases can all be analysed to identify and visualise key characteristics and trends within your data. This analysis can then be used to ensure that your test data accurately represents these trends and characteristics. This process is dynamic and runs in real-time, meaning that it can execute automatically whenever you create your test data. In this way, it can be used to ensure that your test data is, and remains, meaningfully representative of your production data. This analysis even extends to such areas as master data management (MDM). The most relevant capability in this regard is a catalogue of data structures (including their lineage) that can be populated automatically via AI-driven analysis of your data (or, alternately, by importing Swagger files). Each such data structure is captured as a visual pipeline that can be used for driving test data creation (see Figure 2).

What’s more, as part of its efforts to accommodate an extremely wide set of data sources, and to generate sets of test data from the same, ETD offers something akin to a lightweight data integration/ETL capability. This feature allows you to build “minidbs”, that are accessed and managed within the ETD platform, by collating data from a multitude of different sources. The structures of your minidbs are generated automatically, and if any data in them is modified it can be published back to the original data source. The essential point of this exercise ties into the product’s data analysis capabilities: by moving data from many disparate sources into a single, controlled location, it becomes much easier to analyse it holistically, and thus generate an appropriately covering and representative set of test data.

Finally, thanks to the robust capability for building integration pipelines that underpins Curiosity’s offerings, ETD (and for that matter Curiosity’s Modeller) benefits from a huge variety of prebuilt connectivity and integration options. For one example, you can import Swagger/OpenAPI JSON specifications, along with the endpoints they contain, to leverage in your models. For another, test data jobs can be exposed through several different means, including APIs, to provide users with access to them even outside the Curiosity platform. Curiosity’s integration with Windocks (one of the company’s partners) is also notable, for providing containerised database virtualisation that can be executed within ETD.

ETD offers a number of standout features that we have already described. That said, there are two key points that distinguish ETD from its competition. This first is that it offers a wide-ranging test data solution, equipped with a wealth of capability derived from the comprehensiveness of Curiosity’s vision, within a single, integrated platform. This sets it apart from the various point solutions available in the space. The second is that despite the quantity (and quality) of capabilities that the product offers, the company is always striving to add more, to increase the breadth of the product and more fully realise its vision of holistic testing. In short, despite the success Curiosity has achieved with ETD and its other products, it continues to innovate and disrupt rather than rest on its laurels. On top of all of this, the product is highly visual, easy to use, collaborative, and promotes shifting left in the testing process.

It is also worth noting that Curiosity has implemented generative AI capabilities into its testing suite, over and above what we have already mentioned. This allows you to create and/or edit test models, as well as interrogate your existing data, using an AI copilot driven by a large language model of your choice. The platform also uses generative AI to convert natural language statements into data queries, and as part of the creation of your data dictionary (to generate natural language descriptions of your data types, for example).

The Bottom Line

Enterprise Test Data is an effective and innovative test data solution that is characterised by the scope of Curiosity’s vision and the continued expansion of its capabilities in order to meet that vision. It offers both an excellent solution now and the promise of an even better solution in the future.