BMC Compuware

Fig 01 - Data Visualizer in Topaz Workbench

The most basic functionality Topaz for Enterprise Data provides is the ability to easily browse and edit your data across a variety of data sources and file formats, including DB2, IMS, SQL Server, Oracle and VSAM, and in a variety of preset or custom layouts (including raw data, for those who need it). This includes a wide range of functionality, such as comparing data files, visualising data relationships (courtesy of the Data Visualizer, seen in Figure 1), and copying data from one LPAR to another. For TDM, Topaz for Enterprise Data offers data identification, masking, and subsetting, the latter via its ETL capabilities. The product also offers some data generation capability; however, this is recommended for use only as a complement to data masking and not as a standalone synthetic data capability.

To facilitate masking, you must abstract each type of data in your system (for instance, ‘first name’ data) into a ‘data element’ that can then have fields and masking rules attached to it (the former automatically via field name matching). A coverage view is available to display these associations, and a number of masking rule actions are available out of the box. Additional rule logic can be created via an expression builder. Masking rules can be collated and managed centrally, and each rule (or set of rules) is dynamically applied to its associated data element (or subset) at execution time.

Referential integrity is always maintained when masking, masking is propagated across relationships, and BMC Compuware’s patented ‘composite processing’ ensures that your data is masked consistently, even when it is present in multiple formats or contained within a larger field. For instance, Topaz for Enterprise Data will recognize that “Mary Jane Smith” and “Smith, Mary Jane” are the same name and mask them appropriately.

Fig 02 - Extract Visualizer in Topaz Workbench

Subsets are created via fairly standard ETL processes, albeit with a grip of extra features pertinent to TDM. Extracts are created using a driver table; referential integrity is maintained within the subset, as well as between the subset and any relevant applications (specified manually); and the makeup of your extract is presented to you visually, including a step-by-step breakdown of how your extract was created (see Figure 2). The transformation step is achieved by applying expressions, and loading via drag-and-drop column mapping. Both are, as mentioned, fairly standard.

Note that everything we’ve said in this section applies equally whether you’re working with the mainframe or with distributed systems. In addition, reports and audit logs are generated throughout.

BMC distinguishes itself via its extensive support for the mainframe. The fact that it will handle mainframe and distributed data sources consistently, within a single platform, using identical functionality and methods, and via a single, consistent, friendly UI, is outstanding (and not just within the TDM space).

In addition, the abstract nature of data elements in Topaz for Enterprise Data makes it easy to mask several different but conceptually similar fields using the same set of rules, and minimises maintenance issues if those rules are ever changed. That said, matching fields to data elements based on their names is not always ideal: in practice, fields tend to be named arbitrarily and not necessarily with forethought. However, this is mitigated somewhat by the ease of adding fields to data elements, as well as use of the coverage view to preview data identification and rule assignment.

Moreover, the platform is now well equipped to foster both self-service and collaboration. In this regard, it benefits from improvements to its sister product, BMC Compuware Topaz for Total Test, that allow your developers to create ‘placeholder’ virtual test environments that can be instantiated with test data by your testers at a later date. Similarly, masking rules are designed to be created ahead of time (by your subject matter experts, for example) and applied by your testers whenever they need them. These features, among others, allow your various developers, testers, and teams thereof, to work independently and asynchronously from each other, minimising the potential for blockers and/or bottlenecks.

The Bottom Line

In many ways, Topaz for Enterprise Data is the test data management solution for mainframe. If you want to leverage mainframe data for testing purposes – whether in conjunction with distributed data or not – you should at least consider it.

Fig 1 Viewing data relationships in Topaz for Enterprise Data

The most basic functionality Topaz for Enterprise Data provides is the ability to easily browse and edit data across a variety of data sources and file formats, including DB2, IMS, SQL Server, Oracle and VSAM, and in a variety of preset or custom layouts (including raw data, for those who need it). This includes a wide range of functionality, such as comparing data files, visualising data relationships (Figure 1), and copying data from one LPAR to another.

For test data management, Topaz for Enterprise Data offers data privacy (via data discovery and masking) and data subsetting (via ETL). It’s intended that you will use the former to create a gold copy of your data, then extract subsets via the latter. To achieve this, you are asked to abstract each type of data in your database (for instance, ‘name’ data) into a ‘data element’ that can then have masking rules applied to it. Fields in your system are matched to data elements (by matching the field name) and masking rules are defined for each element. Several masking rules are available out of the box, and additional rules can be created via an expression builder. This allows you to, for example, mask a credit card number by randomising its digits, then recalculate its check digit to match its new value.

Masking rules are dynamically built and applied to each data element at execution time. Moreover, Compuware’s patented ‘composite processing’ ensures that data is always masked consistently, even when it is present in multiple formats or contained within a larger field. For instance, Topaz for Enterprise Data will recognize that “Mary Jane Smith” and “Smith, Mary Jane” are the same name and mask them appropriately (for instance, to “London XXXX Klein” and “Klein, London XXXX”, respectively). A coverage view is available prior to execution that will display which fields are mapped to which data elements, if (and how) they are going to be masked, and allow you to adjust as appropriate. Relational integrity is always maintained and an audit log and masking report is generated upon execution.

Fig 2 Visualising data extracts in Topaz for Enterprise Data

Having created your gold copy, Topaz for Enterprise Data allows you to extract subsets via its ETL capabilities. Creating an extract (and hence a subset) requires you to select and include a singular driver table, as well as any relevant data in related tables and columns. The data and relationships in your extract can be viewed graphically, as shown in Figure 2.

Once you have created your extract, you can proceed to transform it and load it into your target database. This is achieved by specifying mappings from the data in your extract to the tables and columns in your target, as well as any necessary transform expressions. Several of the latter are available out of the box, and you also have the option of creating your own. Both the extraction and loading processes generate appropriate audit logs and reports.

Finally, Topaz for Enterprise Data also offers the ability to generate synthetic data. However, this is recommended for use in concert with data masking and not as a standalone capability.

Compuware distinguishes itself via its extensive support for the mainframe, which is outstanding in the test data management space. The fact that it will handle mainframe and distributed data sources consistently and within a single platform is particularly helpful. Moreover, Compuware’s test data management offering has historically been very easy to use – especially bearing in mind that it manages mainframe data – and this is even more true now that several important test data management capabilities have been consolidated within Topaz for Enterprise Data.

In addition, the abstract nature of data elements in Topaz for Enterprise Data makes it easy to mask several different but conceptually similar fields using the same set of rules, and minimises maintenance issues if those rules are ever changed. That said, matching fields to data elements based on their names is not always ideal: in practice, fields tend to be named arbitrarily and not necessarily with forethought. However, this is mitigated somewhat by the ease of adding fields to data elements and use of the coverage view to preview data identification and rule assignment.

The Bottom Line

If you want to leverage mainframe data for testing purposes – either in conjunction with distributed data or not – you should certainly be considering Topaz for Enterprise Data.