Compliance in the telecommunications and associated industries

Compliance in the telecommunications and associated industries

The Regulation of Investigatory Powers Act (RIPA) 2000 is the UK act that requires relevant bodies such as telecommunications companies, internet service providers, e-mail providers and so on, to keep logs of all communications over their channels for a prolonged period of time and, moreover, that those companies provide mechanisms whereby relevant authorities (such as the police) can search through those logs as and when required. In addition, this storage and the associated retrieval capabilities must be provided at the company’s expense.

Now, before you consider how you might store and provide access to all of this (growing) data, consider what else you might want to use that information for. Let’s take call data records (CDRs) as an example. Your are not interested in any particular call but you would like to know what the pattern of calls is from any particular phone, not least because you want to help to prevent fraud in the event of a phone being stolen. In addition, you would probably like to know how much revenue and profit you earn from each customer so that, if you predict that an individual is likely to churn, you will know whether it is worth making a special offer in order to try to retain this customer. In other words you want to be able to analyse data from the CDRs but, in one way or another, this will be aggregated data rather than any requirement to store each CDR as a whole.

In other words, you have two requirements: the need to store raw data to meet the requirements of RIPA (or similar legislation in other countries) and the need to be able to analyse aggregated and summarised data which, presumably, will be in a data warehouse or data mart.

The question then becomes: does it make sense to store all the CDRs in the warehouse to serve both purposes or would it be better to have specific solutions for each requirement? Well, to begin with, for data protection and other legislative reasons, you may not actually be allowed to use this data for any other purposes, in which case it would be technically illegal to consolidate all of this data in a single place. But this may not be so in other jurisdictions. Thus the question remains: do you want to have all this data (if you are allowed to) in one place?

I think the answer must depend primarily on cost. If you put it all into the data warehouse then the warehouse will be substantially bigger and therefore cost more. If you have two separate systems then the RIPA (or equivalent) system needs to cost you less than the extra expense involved in putting it into a warehouse. Is this possible?

As a matter of fact it may well be. I have been looking at the latest offering from CopperEye, which addresses this market and has a RIPA compliant front-end called the Disclosure Request System: you can implement this, pre-installed on a 24Tb Sun X4500 for a list price of less than $250,000. Add this to, say, a Netezza data warehouse appliance for analysis purposes and you could make a considerable saving when compared to a conventional solution using Oracle or even a less well-known one (in the UK at least) from the likes of Sensage. Moreover, you can fully implement a CopperEye-based solution in just 10 days whereas you would be talking about weeks, months or even years by taking a more traditional route.

The Australians might claim that this was a ripper idea but I wouldn’t be so crass as to make such an awful pun.