The importance of a common data model

Before we even start to consider why a common data model might be important we had better be clear what it is. A data model defines the relationships between disparate data entities within a particular environment, thus establishing the context within which those entities have meaning. Thus you will have a data model underpinning your CRM system and your databases will each be predicated upon their own data model (in this case called a schema).

Going beyond vanilla flavoured data models you can have a federated data model that spans multiple databases. Thus suppliers of EII (enterprise information integration) platforms provide federated query capabilities that are based upon defining data relationships that span multiple data sources. However, the virtual schemas or views that these products use tend to be physical in nature rather than generalised or conceptual.

A common data model goes beyond all of these other more limited deployments and provides a data model that spans an enterprise’s applications and data sources. In other words it defines all the data relationships and meanings that exist within an organisation. As one might imagine such things are not simple to build and for this reason common data models are not common. However, their importance is increasingly being recognised and it is the telecommunications sector that is leading the way.

The TM Forum has published the Shared Information/Data telecommunications model for its industry. This is commonly known as “the SID” and it is increasingly being adopted by companies in that sector. As it proves its value it is likely that other industry groups will roll out similar models and, no doubt, there will be a variety of companies across all sectors that roll their own.

So, that’s what a common data model is but what do you do with it and why is it important? Well, in the first case it defines the terminology that the company uses for all of its data sources and the relationships that exist between different data items. Now, if you know all of that from a conceptual standpoint then you can map your existing applications and data definitions to the common data model. So, if you have different applications that define customers in different ways, for example, you can map each of these to the common data model so that, in effect, the common data model provides a bridge between the different meanings associated with each of the customer applications. Thus the common data model enables data interoperability between applications.

In practical terms this means that you can use the common data model as the basis for building data services as a part of your SOA (service oriented architecture) environment, you could use it to replace or augment (depending on your approach) your MDM (master data management) system, you could use it to enable data migration or ETL (extract, transform and load) process and you could probably use it for a whole bunch of other things that I haven’t though of yet.

Of course there are three main challenges to using a common model. First, there is getting a common data model in the first place. Secondly, there is the issue of defining all of those mappings between your applications and the common data model. And, thirdly, there is the management of change on an on-going basis: both the common model (and application-specific models) will evolve throughout the entire development and maintenance lifecycle to reflect changes in business requirements.

If you don’t happen to be a telco and don’t already have an internal architecture group creating a common model, you may have to build your own common data model (iteratively) or start from one of the industry models available from standards bodies or data warehousing vendors. However, from the mapping and change management perspective help is more immediately at hand because Progress Software offers a solution specifically for this purpose, called the DataXtend Semantic Integrator. As far as I know, this is the only such product of its kind available from anyone. This makes it a market leading product albeit that the market is in its infancy. Briefly, DataXtend allows you to import definitions for a common model, create the relevant mappings, govern the use of a common data model in SOA, and manage change. DataXtend SI also includes the ability to validate the semantic consistency of data exchanged between applications. That is to say, you can build data consistency rules into the mapping when required to ensure data interoperability. If you want more details look at www.progress.com/dataxtend/dataxtend_si.

Anyway, the bottom line is that if you are interested in a common data model then a discussion with the DataXtend people at Progress should be regarded as mandatory.