What’s happening in CEP?

The complex event processing (CEP) market is evolving. It has got beyond the stage where people are worried about what to call it: should it be called CEP, ESP (event stream processing) or just plain event processing? Indeed EPTS (the event processing technical society) will shortly be releasing an official glossary of terms.

That isn’t to say there isn’t still debate. For example, the EPTS will be defining an event as “something that happens” which is fine but raises the question of whether when it happens is relevant: is this part of the event or metadata about the event? This is significant because on this depends whether you can have such a thing as a non-event. Now, what you and I would call a non-event (in real life that is) is a period of time in which nothing happens, but in a CEP environment what has happened is that an RFID scanner, say, has made multiple readings during which the only thing that has changed is the timestamp. So, is this an event or a non-event?

Anyway, enough of the theory. Arguably, the most interesting thing to have happened in the CEP space recently is the acquisition of AptSoft by IBM. Not that this is worrying Progress or StreamBase because AptSoft has not made much (if any) impact on the high performance end of the CEP sector in capital markets. And, indeed, IBM’s reference to AptSoft as BEP (business event processing) implies cognizance of this fact. Where we will see AptSoft in IBM terms is primarily as a part of the WebSphere suite where it will play a role in SOA implementations, for example. Progress Apama is also, incidentally, directed at this market as a part of the Progress SOA Suite. When Oracle finally introduces its own CEP product (my guess is that it will [and should] scrap BEA’s offering) it will no doubt be aimed in the same direction; and of course this is a natural for TIBCO.

As far as capital markets are concerned the three main players are clearly Progress Apama, StreamBase, Aleri and possibly Coral8, arguably in that order. However, what’s interesting is that there is an increasing demand for back-ending these systems with a data warehouse (when the market for RFID finally takes off the same is likely to apply there as well), an area which Sybase is focusing on with its Real-time Analytics Platform, as is Vertica. This sort of functionality is particularly in demand by hedge funds so Kx Systems’ implementations in these environments may be under threat. As an aside, and in case you are wondering why I haven’t mentioned any of the more well-known names in the data warehousing space, it is worth noting that Sybase IQ, Vertica and Kx all use column-based relational databases that are particularly suitable for the sort of high performance ad hoc analytics required in these environments (though this would also be true of the likes of Netezza).

Another major market for CEP is intelligence services, though this is largely untapped outside the United States. Here, StreamBase and AgentLogic are the leading vendors, in part because both of them are (partly) funded by (indirectly) the CIA. However, while this certainly represents an entrée in the States it doesn’t mean so elsewhere. Moreover, the nature of the applications involved is highly secret, so neither StreamBase nor AgentLogic would be able to claim very much in the way of hands-on experience in these environments. In other words, the opportunity is there for other vendors to penetrate non-US intelligence markets.

As I have commented previously there are a lot of use cases for CEP but not many markets. Certainly there have been a number of implementations in transport and logistics (where TIBCO has been doing well) but my suspicion is that the next major market will be in manufacturing and, particularly, in process manufacturing or in other process sectors (such as power stations). Demonstrable benefits can be achieved in productivity gains by employing CEP in these environments (Progress and Avaya can both quote examples) and, perhaps most important, shop-floor environments tend to understand the concept of events so that the sales cycle should not be so long.

The market for CEP is clearly alive and kicking but it is still a minority sport: Progress Apama has something over 75 customers, StreamBase approaching 50 and these are the two leading vendors. AptSoft had 19 prior to acquisition. A total market size of around 200 customers to-date means that CEP is still only tinkering at the fringes today, but the possibilities are huge.