Delphix by Perforce

Delphix is a data automation and compliance platform designed for multi-cloud and DevOps environments (see Figure 1). It can be used to automate complex data operations, including compliance with data privacy regulations, by leveraging data (or, more properly, database) virtualisation. This enables you to create and manage virtual copies of your data sources for use in application development, testing, cloud migrations, reliability, analytics, machine learning, and other scenarios. Built-in, always-consistent data discovery and masking allow you to protect your sensitive data in these virtual environments, giving you the power to distribute compliant data sets en masse, breaking down data silos while retaining compliance. In particular, Delphix can provide a complete test data management solution that does not depend on either data subsetting or synthetic data generation.

Fig 1 – Delphix product offerings

The platform can be hosted on-premises or in the cloud, including multi-cloud environments. It leverages your existing storage and supports hybrid environments, wherein you can deploy Delphix and mask your data on-prem but provision that data in the cloud. Kubernetes deployment is also available. Over a hundred data sources are supported out of the box, including cloud data warehouses such as Databricks and Snowflake. Additional data sources can be added via an SDK. The platform’s HyperScale Compliance product offers further support for massive data sources, providing enhanced horizontal scaling that allows Delphix to operate much more performantly in those environments. This may be useful for, say, masking vast quantities of analytical data stored in a cloud data warehouse.

The platform is accessible either through a self-service portal or various data APIs, the latter of which can be used for leveraging Delphix functionality remotely and automatically (within an existing toolchain or a CI/CD pipeline, for instance). Pre-built integrations are available for a variety of third-party products, including popular development and testing tools (such as Jenkins and Git), production vendors, and analytics/AI solutions. In addition, Delphix Compliant Services (DCS), a multi-tenant SaaS implementation of Delphix, is available for Azure and Salesforce, using the aforementioned APIs to leverage Delphix in conjunction with those platforms.

Customer Quotes

“Delphix has enabled us to improve quality, reduce costs, and free up developers and testers to be more productive.”
Morningstar Retirement

“Data is the life blood of our business, and Delphix has been instrumental in empowering us to use our data faster and more productively.”
Morgan Stanley

Delphix keeps hold of a single, continuously updated copy of each of your production databases that you can provision virtual copies against as required. You can refresh, reset, or rewind your personal copies at any time, and you can also bookmark, branch and share them. Notably, provisioning is incremental, meaning that when your production database is updated, only those updates (and not entire datasets) will be pushed out to your virtual copies during a refresh. Even then, refreshes are strictly on demand: your users are given the freedom to decide if and when they need an update. In addition, multiple databases can be provisioned in a synchronised fashion. This could be used, for example, to facilitate integration testing for applications that use data from multiple sources. What’s more, the platform’s “data time machine” functionality provides a complete record of all production deltas, meaning that at any time you can look back and see exactly what changed, and when, in your production data.

If your data sources contain sensitive data, you will also need to discover and mask it. In Delphix, this functionality is generally applied when you first create a copy of any given production database, creating a masked version of it that desensitised virtual copies can subsequently be provisioned from. That said, it can also be used standalone. Both sensitive data discovery and masking are automated, and a library of masking algorithms is provided. They can be scheduled or run on-demand, apply to both structured and unstructured data, are consistent across all of your data (including data spread across multiple clouds) and come with preconfigured templates tailored to some common use cases, including applications (such as SAP) and regulations (such as GDPR). The platform supports tokenisation, as well as audit and compliance reporting, and if you already have a preferred data masking solution, you can use that instead.

The platform also includes the Data Control Tower, a data governance layer that lies on top of Delphix’s other capabilities, offering a single, central, and enterprise-wide location for controlling the totality of your Delphix deployment. In addition, Delphix provides a distributed data library that sits just under the Data Control Tower, which acts as a data catalogue that provides tagging functionality, versioning, and fine-grained, attributed-based access control, among other things. Together, they provide self-service access to your data, whether through the data library itself, APIs, or the provided CLI (Command Line Interface).

Data protection is available through the Continuous Vault addon, which will continuously synchronise your data with an immutable data vault in near real-time. It is certainly much faster than more traditional backup and restore, or automated recovery, solutions. Another addon, Elastic Data, adds elastic storage, allowing you to leverage cost-efficient object storage in addition to the more expensive (but also more performant) block storage already present. The latter can then be reserved for use as a performance cache. Notably, there are plans to bring this elastic storage capability to other Perforce offerings (such as the Helix Core version control solution), making it a good example of cross pollination between Delphix and the rest of the company’s product suite. Similarly, BlazeMeter can be used to inject synthetic data into your Delphix data sets. Add Perfecto, and you can create an end-to-end, continuous test automation solution, capable of providing high quality, masked, and virtualised data to your test environments, using only Perforce products.

Delphix’s core selling point is that it allows you to create sanitised, compliant data sets and distribute them efficiently across your enterprise from a central location. While test data management has historically been one of its most significant applications, it should be obvious from this framing that it is far from the only one. Indeed, the platform’s track record across a variety of use cases brings this point into sharp focus. The fact that it will discover and mask data consistently across all data sources, including multiple clouds, is a particularly major selling point, in that it allows you to ensure your data is compliant across your entire environment without creating “compliance silos”, wherein the act of making your data compliant renders it inoperable outside of its present location.

That said, in terms of test data management in particular, Delphix allows you to leverage a complete and compliant copy of your production data for testing. It lends itself well to the efficient provisioning and operationalisation of test data (and from there to DevOps), reinforced by the product integrations, comprehensive APIs, and self-service capabilities it provides, allowing you to easily integrate it into your existing CI/CD pipeline. Finally, its capabilities are automated, highly scalable, and provided as part of a single platform, making it a one-stop-shop for virtualisation-driven test data management.

Integration into the Perforce product suite has also done the platform a number of favours, from addons that only exist thanks to Perforce technology to wholesale integration with its new sibling products. Most notably, Delphix, BlazeMeter and Perfecto can be used together to create a comprehensive test automation solution entirely through Perforce.

The bottom line

Delphix is a compelling platform for enabling data compliance as well as a mature and effective solution for test data management. It has successfully broadened out from the latter, in part by identifying that many environments, far beyond just testing, require data that is both compliant and consistent – data that it is more than willing to provide.