Security analytics

Security Analytics helps an organisation manage risk and is an important enabler of good governance. Risk management is top of the mind for many executive officers today. It involves the identification, assessment and prioritisation of risks, as well as efforts to reduce the impact of those risks.

Regardless of whether the need is for early threat detection or historical analysis to track advanced threats over time (such as ‘low and slow’ attacks, which hackers hope will get through beneath the radar), today a security team must analyse massive volumes of data and disjointed processes in order to gather actionable security intelligence. It must incorporate event and log data from all parts of the enterprise, from internal networks systems to remote industrial equipment, in order to ensure no security gaps remain that could increase risk to the organisation. Done effectively, security analytics will aid the organisation by providing valuable insights that will help the organisation to manage and reduce the overall risks that it faces and will boost the overall security of its operations. It should also be served up in a variety of formats suited to a varied user community: some are ad-hoc investigators, others are people who look at dashboards and alerts to just determine if escalation needs to happen.

Further security intelligence capabilities include the ability to support forensic investigations, legal and e-discovery requests, and to aid in achieving corporate governance and regulatory compliance objectives. A key requirement of security intelligence technologies is that they have advanced analytics capabilities, capable of harnessing and making sense of massive volumes of data from an ever-growing range of disparate sources and can present the analysis to managers in a clear, integrated format that aids decision making.

You should care about security analytics if you are responsible for corporate security, corporate or IT governance, or audit. It is worth noting that logs are notoriously hard to read and interpret in their native format so you should expect security analytics products to make such data easier to read (for technologists) and more accessible and easier to understand, as exception reports (for managers). A number of suppliers allow organisations to monitor the use of hardware such as USB sticks and access control devices and can report on, possibly dysfunctional, temporal or locational patterns–so an alert can be raised if someone is accessing a data source from a place, or at a time, when he or she is unlikely to be authorised to do so.

Integrated security analytics technology addresses the corporate security issue and also supports compliance monitoring and reporting for regulations such as PCI DSS, Sarbanes-Oxley and so forth and is sometimes used in conjunction with things such as CDR (caller detail record) and IPDR (IP detail record) retention.

Actionable security intelligence will aid the IT organisation considerably by boosting its capabilities to manage and respond to security incidents arising from the very latest threats. This makes it of interest to C-level executives and will form an essential component of the overall risk management priorities for organisations that wish to shield themselves from harmful incidents and public disclosure of security incidents that could hurt not only the reputation and competitive positioning of the organisation, but could also hurt it financially as well.

Analytics for IT security has its roots in security information and event management (SIEM) systems with log management capabilities. However, the SIEM market become commoditised and overfocused on compliance. Security analytics vendors are adding ad-hoc SQL query functionality at the back end (and support for third party business intelligence tools), whilst also simplifying and extending their ability to correlate rules or introducing complex event processing engines, which can support low-latency analytics, at the front end.

The tools are thus evolving into integrated, centrally managed platforms that combine SIEM and log management with ancillary capabilities that include integrity monitoring and real time threat intelligence and reputation feeds for combating zero day and advanced persistent threats. Such platforms now provide analytics capabilities that scale to handle the needs of big data sets seen across organisations today, helping to make sense of the massive volumes of data generated. To provide the full range of analytics capabilities that organisations need, such platforms should be integrated with other security and IT technologies, including endpoint management controls, threat mitigation techniques, database activity monitoring, identity and access management systems, and application vulnerability scanning.

New is the appreciation that it’s a “big data” issue and the intelligence available from exploiting all the data available to an organisation can provide it not only with operational improvements, but can also improve its ability to detect and respond to increasingly sophisticated security threats and vulnerabilities. Increasingly, security analytics platforms are expanding to take in an ever-growing variety of feeds, such as those derived from machine-to-machine communications, from sensors built into devices such as mobile phones, smart energy meters, cars and industrial equipment, to provide a higher level of situational awareness across the organisation, which then improves operational decision making.

Near real-time threat protection needs low latency storage while the identification of longer-term attack patterns (hackers often resort to slow attacks in the hopes of evading detection) requires the longer-term storage of, possibly, very large amounts of data, so there is a trend towards using highly compressible file systems rather than databases per se. Only about 3 months of data used to be kept for batch analysis; now, certain threats not only require data to be held for many months but also require analysis in the context of real-time events.

A Bloor paper which discusses the increasing use of big data for security intelligence can be found here.

Many vendors in this area have a background in analytics, SIEM and log management and have been adding capabilities to expand their offerings into security intelligence platforms incorporating big data analytics for security. One recent change was the purchase by Cisco of Cognitive Security in January 2013.

Further back, Arcsight, RSA, Q1 Labs, LogLogic and others have all been acquired over the last few years (by, respectively, HP, EMC, IBM and TIBCO). While there are still a number of independent vendors in this space there are clearly fewer of them than there were before. Of these, LogRhythm probably has the largest presence.

There has been comoditisation, but the market is still attracting new vendors. For example, Red Lambda has entered the market with a solution based on complex event processing. In our view this is the right approach to take and we expect synergies from IBM with Streams and Q1 Labs in due course. Tier-3 already takes this approach.

This impact of big data will likely spread to the back-end. We have not seen any announcements to this effect yet but we expect vendors to be explorinhg the use of products such as Hadoop for cost-effective storage.