Informatica acquires Compact Solutions

Readers with lengthy memories may recall that when I first started writing about data catalogues, I predicted that we would soon be in a position whereby we had siloed catalogues all over the place. As a result we were going to need a catalogue of catalogues. And there are a few vendors, of which Informatica is one, who are targeting this market.

The problem for any company wanting to offer this sort of enterprise data catalogue is that metadata is messy. More or less every environment you can think has its distinct metadata from every other and there is no commonality or agreed standards (but see below) about metadata across technologies. Worse, it’s hard to understand metadata. And this is where Informatica and Compact come in. While the two companies have been partners for a couple of years, and have a number of joint customers, bringing them together formally makes a lot of sense as Compact understands the metadata of environments – notably legacy mainframe environments (COBOL copybooks, JCL and so forth), dynamic SQL Stored Procedures and complex ecosystems like SAS and SAP – that Informatica has not. So this acquisition will significantly enhance Informatica’s position, not just for metadata management but also for the data lineage capabilities that that understanding of metadata supports.

More generally with respect to metadata the big problem is with the lack of standards, and readers may recall that I have previously extolled the virtues of ODPi Egeria (a Linux Foundation project). This is Informatica’s stated position with respect to this:

We continue to watch ODPi Egeria and its progress, but have no firm date yet on becoming part of the initiative. On the other hand, we are investing in deeply integrating with other metadata catalogs for our Catalog of Catalogs vision. An example is our integration with Apache Atlas where we can extract object and lineage metadata, reconcile that with metadata in our Enterprise Data Catalog to present a comprehensive enterprise view of the metadata. Other catalog connectors are AWS Glue and Cloudera Navigator, and connectors for Azure Data Catalog v2 and Google Data Catalog are in the pipeline.”

Fixing the metadata problem has been an issue for more than 30 years. Could we finally be getting somewhere? The danger, of course, is that we end up with multiple (well, a few) catalogues of catalogues, which don’t talk to each other. Of course, you may be able to interconnect via APIs (Informatica supports this) but this isn’t a deep level of integration. In any case I’m in favour of Egeria. Nevertheless, Informatica’s acquisition of Compact is to be welcomed as a step in the right direction.