Goodbye CEP, hello streaming and Cassandra

You may have missed it but Progress has sold its Apama division to Software AG and StreamBase has been acquired by TIBCO. This means that there are now no significant independent vendors of complex event processing (CEP) products. Now, Software AG is about integration and middleware while TIBCO, notwithstanding its ownership of Spotfire, is pretty much about the same things. No doubt both companies will continue to play in the capital markets space but I expect both of them to drop any pretensions to being general-purpose analytic platforms for high velocity data. Certainly, I don’t see either of them competing very successfully with either SAS or IBM for general-purpose environments, which is probably why the sellers were happy to sell.

Since neither SAS nor IBM refers to its respective product as complex event processing we may as well kiss that term goodbye: (event) streaming is where it is now at. However, it is worth considering whether event streaming is always the best approach. In particular, advocates of Cassandra would suggest that event streaming is fine if you just want to analyse what’s happening right now but that if you want to understand what’s happening now in the context of what was happening five minutes ago and five minutes before that – in other words, trending – then Cassandra is better option.

And, of course, there’s Impala, which is being touted as real-time Hadoop. Well, only if you conveniently ignore “near” or even possibly “far”. Cloudera Impala allows “real-time queries against data stored in HDFS or HBase” and, it is claimed, is an order of magnitude faster than Hive. Good. How long does it take to store the data in Hadoop? If we are talking about scads of high velocity data this isn’t going to compete with either streaming or Cassandra. It’s good, I like it, but don’t confuse it with either of these two, which have been engineered from the outset for real-time analytics against streaming data.

Of course, the other problem with streaming platforms, as opposed to Cassandra, is that they are very expensive. This has limited the market opportunities of companies in this space, more or less, to fraud, security and capital markets plus a few isolated use cases. Cassandra, it seems to me, has much greater potential to grab market share and until and unless the big boys with the big streaming platforms bring their prices down dramatically, what we are going to see is more and more Cassandra implementations, which must be good news for DataStax and Acunu.