Harnessing big data for security - what are the key considerations and capabilities?

Volumes of data are expanding rapidly, and effectively harnessing the data volumes generated by organisations today brings both significant gains as well as challenges. In particular, big data security sets bring specialised challenges owing to the nature of the information that is produced, which must be stored in a sequential, time-stamped manner and which must be stored in its raw, unchanged format that proves its integrity; in other words, it has not been tampered with.

This requires the use of specialist tools. Traditional database and warehousing technologies do a good job of handling operational data, but were not designed with the specificities of security data in mind. They are also based on an architecture that is difficult to scale effectively to meet the challenges of huge data sets, such as the huge volumes and constant flow of security event information from systems throughout the organisation.

Even where the back end technology has been built with the needs of big data security event information in mind, effective analysis of that information requires a security intelligence platform be integrated at the front end so that information flows freely in an uninterrupted manner between all parts of the system. Many vendors offer partial solutions-either the back end technology or the front end security intelligence platform. This creates multiple integration and management challenges that are not only a challenge in the upfront implementation, but also in the ongoing management and extensibility of the system as new data sources are added. Few organisations have the resources or budget available to effectively overcome the challenges of integrating disparate technologies. Far better is to look for a system that was built from the ground up as one integrated system, designed specifically for the intricacies involved with security data.

This report discusses some of the challenges of harnessing big data security and outlines some of the key considerations and capabilities that organisations should consider when selecting a system that can handle the whole gamut of needs in a unified manner that is simple to integrate and manage.

Free Download (subject to terms)