SDRS and the EU Data Retention Directive

The EU Data Retention Directive requires that all EU governments must put in place laws requiring Telcos and ISPs to retain data on-line (as opposed to tape) for a period of between 6 months and 2 years so that the Police, counter-intelligence agencies and other bodies can request information about who has called or emailed whom, when, for how long, and so on. Now, some governments have been doing this anyway but many have not, or in a more ad hoc manner. What this Directive will ensure is that there is some sort of conformity across the EU. Also, it clarifies this as a potential target market for relevant vendors.

But who are relevant vendors? There are three basic approaches: you could use an OLTP or general-purpose database but these are predicated on the assumption that you have to support a roughly equal mix of read and write operations; or you could use a data warehouse, but these are specialised for environments where the emphasis is on reading data; or you can use a specially-designed environment where the focus is on writing the data. Given that Telcos and ISPs have very large volumes of data to load, index and store but only need to support relatively infrequent queries I know which I would choose.

One company with just such a product that is addressing this market is CopperEye, with the SDRS (Secure Data Retrieval Server) product developed in collaboration with Sun. This is shipped as an appliance (with all the benefits that one would expect of an appliance-based approach) based on the Sun Thumper (X4500) platform, with 48Tb of disk and running on Solaris. A couple of quick points: if you haven’t looked at Thumper, this is seriously cool hardware, designed for high performance computing; and, secondly, because the stored data is compressed you get the equivalent of 72Tb of storage for your data and indexes, where the latter are automatically created by CopperEye Greenwich as the data is loaded. And, as a further aside, CopperEye started life as an indexing specialist so it is very high performing both in terms of index creation and retrieval.

Typically, indexes would be created against access number (phone or email address), location and equipment used, though you can also store subscriber details (from, say, your CRM system) for legal discovery purposes.

In addition, within the appliance CopperEye is shipping its Disclosure and Request System (DRS) which provides the search mechanisms you need to find the relevant data about criminals, terrorists and the like. In addition, in its forthcoming release (August) CopperEye will be adding a web-based self-service capability that will allow relevant personnel, with appropriate authorisation, to search through records remotely. This is impressive and it is the first such capability I have seen offered by any vendor in this space. As it will greatly reduce the burden on the Telcos and ISPs I imagine it is something that will be greatly appreciated by them as well as by the police and other authorities.

There are other points to note: you can have two servers, with replication to support failover and index recovery; there is workflow built-in in order to support the different approaches that various countries will take towards implementing the EU directive; a full audit trail of all searches is maintained; there is optional integration with TruSeal for tamper proofing and you can integrate the solution with system monitoring tools such as HP OpenView.

Put this all together and you have what I think looks an impressive package. Add to that the fact that Sun’s sales force is working with CopperEye and it looks SDRS should be a winner.

(This article has been co-authored by Daniel Howard)