Event processing moving forward

There are various issues that confront the event processing community. There now seems to be common accord that event processing as a term should best be used to encompass both complex event processing (CEP) and event stream processing (ESP). However, there remain other areas of contention and discussion.

On the language front this now seems to be less of an issue. It now seems clear that there is not going to be any sort of standard language (SQL or otherwise) for event processing, as I suggested last year. However, it is clear that at least some users want standards in this area before they will adopt anything other than a ‘roll your own’ approach, so the various vendors involved have agreed to work on a meta-language that can be mapped to the particular language used by that vendor. This is some way off becoming a reality. At present the focus is on defining a standard glossary.

Another issue is just exactly what event processing is, and how it differs (or does it?) from operational business intelligence. Last year, Forrester put out a paper in which it suggested that there were two distinct markets emerging: CEP (see first paragraph above) and business event management (BEM) where they contrasted the two by stating that the former was based on patterns and the latter on single events. The problem I have with this distinction is that single events only have meaning within context. And that context either has to be set by comparison with a pattern or with respect to a business rule. Leaving aside patterns as the domain of event processing you have to ask: where does a business rule come from? Put simply, it comes from experience. In other words, it comes from an understanding of past events. So where is the difference? Only that pattern-based recognition of important events is automated and comparison with business rules is only semi-automated, which would seem to be a retrograde step. While you can make distinctions about the latency of the decision-making process my own view is that event processing and operational BI form a continuum and trying to make distinctions is pretty much a waste of time.

A further vexed question is just how big the event processing market is. As far as I know, the only publicly quoted estimates are from the Tower Group, which estimates that it will be worth $67m in the Financial Services sector in 2007 and this will have increased tenfold by 2010. We could speculate that Financial Services represents say, one third of the market, and that potential service revenues might equal those of license revenues and you come up with a figure of something like $4bn in total by 2010. But there are a lot of ifs here. (Incidentally, thanks to IBM for the extrapolation).

Apart from the move towards standards, what will really drive the market will be two things, in my opinion. First, the entry of IBM and/or Oracle into this space. While other major vendors, such as Microsoft, are introduced in this space this is primarily through partners at present but when IBM and/or Oracle introduce products of their own then this will give a legitimacy and marketing presence to the market that it currently does not have. Note that it is not a question of if these vendors will enter the market but when, which should reassure any doubters amongst the user community even now.

Secondly, Progress is planning to roll-out its Apama event processing product to its partner base this year. Now, it is not as if other vendors are not planning (or already making) comparable efforts. Indeed, in the case of Coral8, this is the company’s major route to market and it is a natural for any American company without a European presence that wants to get into Europe’s MiFID market. However, Progress has a hugely bigger partner community, world-wide, than any of the vendors currently in the market. While the effects of this roll-out are more likely to be seen next year than this we should expect a significant increase in the number of applications that embed event processing technology, which will dramatically grow the market. Indeed, it is the embedded market (which, incidentally, doesn’t care much about standards) that is likely to be the biggest driver for growth in the event processing market.