Pervasiveness

The idea of things being pervasive is making a comeback. First, IBM redefined the idea of pervasive BI and now Informatica, with the release of PowerCenter 9, has introduced the idea of pervasive data quality. And I would also argue that we should be thinking about the pervasive availability of data: what IBM calls (or called) information on demand and what Informatica is offering through its data services for SOA.

The important point about pervasiveness is that its about little things. Its about the little decisions (micro-decisions) that business people (or automated business processes for that matter) make hundreds of times every day: do we increase your credit rating, do we accept your insurance claim, do we make you a loan? To make those decisions with the minimum amount of stress you need the relevant information, you need to know that it is accurate, and you need to be able to see it right now. So, you need pervasive BI, pervasive data quality and pervasive availability.

Whats essentially new here is the concept of pervasive data quality. The key point here is that data quality touches large numbers of people in the organisation, even if they dont know it. For example, suppose that, as a line of business manager, you have to make some sort of business decision. And suppose further that you knew how reliable the information was that you were going to base that decision on. Would you make the same decision if you knew that the data was 95% accurate or 65% accurate? Almost certainly not: the balance of risk versus reward varies depending on how reliable your information is and with not very accurate data you are more likely to err on the side of caution.

Bottom line: anybody making decisions needs to be able to see data quality metrics (probably via a dashboard) about the information they are using.

In most cases, pervasive data quality wont go further than that. It may not seem like a big change but it is: in most companies, data quality is a back-office activity conducted by a handful of IT people in conjunction with a business analyst or two, plus maybe a data steward. What pervasive does is to take data quality out to the masses: not that they are expected to do data cleansing but that they should be able to see metrics when they need to and, more generally, that they are aware of the issues involved with respect to data that can be trusted, as opposed to not trusted. In time, one might hope that this will lead to a cultural shift whereby staff throughout the organisation recognise the value of high quality data and, indeed, that such a perception itself becomes pervasive.