Collaborative SIEM solutions

With Infosec taking place next week my thoughts are naturally turning towards the security market in general and SIEM (security information and event management) in particular. Long-time readers will know that I have two major concerns about the SIEM market in general, the first being with respect to real-time analysis of event data and the second being the ability to perform analytics against log data.

The reason for the first of these emphases is that you need to identify attack types as soon as possible in order that you can react appropriately. That is, you need to be able to score incoming events in real-time, against recognised patterns of behaviour that define a particular attack type. The second emphasis derives from the first: you need sophisticated analytics in order to identify patterns and build suitable models that you can score against. In addition, log-based analytics can be used for other purposes such as fraud detection.

Most companies (there are exceptions) have not embraced either of these ideas. In terms of real-time event analysis the majority of vendors still employ rules-based systems that are complex to manage and maintain and relatively slow to identify threats. However, even where that is the case there is still no reason not to have sophisticated log-based analytics to support such environments. Log-based analytics can be used to identify the rules that need to be deployed in real-time.

Now, many companies in the SIEM market offer a single product approach to both real-time event management and log-based analytics. However, there are also a number that do not. For example, ArcSight offers its ESM product for event management and Logger for log management and analytics. Another example is netForensics, where large enterprises will typically need to license both SIM One and Cinxi One. In both cases these are separate products, even though they work together. In fact, it is arguable that even the vendors with a “single product” often do not really have such a thing. This is because it is commonplace to have two separate repositories for the data: typically a relational database for event data and a compressed, flat file system for log data. In our view, you need a (log) data warehouse (with compression) for storing log data so that you can perform appropriate analytics on those logs; flat files simply do not support the functionality necessary for sophisticated analytics.

Anyway, if the solutions offered by different vendors are effectively in two halves, especially when they are actually separate products, then you lose many of the advantages of having a single solution from a single vendor. It therefore may be appropriate to look at alternative approaches: adopting a collaborative solution that involves best-of-breed solutions from more than one supplier.

Unfortunately, there are not many vendors in this market attempting to be best-of-breed vendors for just a part of required SIEM functionality. However, there are a couple that are worth mentioning. One is Splunk, which provides operational business intelligence for machine-generated data and the other is Sensage, whose event warehouse is, in practice, a data warehouse for log-based data. While neither of these vendors is limited to the SIEM space both are active within it. Apart from their actual capabilities the main difference between them is that Splunk is usually used in addition to a full-blown SIEM solution whereas Sensage can be used to replace the log storage and analytics capabilities of SIEM products. In the case of Sensage this will make most sense when the SIEM capabilities of the vendor in question are split into separate products with separate licensing agreements. Thus implementing Sensage in conjunction with ArcSight ESM, for example, instead of ArcSight Logger, would make a lot of sense, as Logger does not have the analytic capabilities that Sensage offers.

There is a natural tendency, when a vendor presents you with what appears to be a complete solution, to accept that at face value. It is well worth looking again to see if a collaborative approach might be more appropriate.