Simplifying CEP

You can tell that complex event processing (CEP) was not initially developed by a company because no marketing man would ever have allowed a technology to be referred to as complex. In fact, the man generally regarded as the father of CEP, David Luckham, is a professor at Stamford University and he invented the term to describe the processing of complex events, as opposed to any idea that event processing is, in itself, particularly complex. He defines a complex event as “an event that could only happen if lots of other events happen”. For example, to use CEP (or any other technology for that matter) to recognise fraud means recognising the pattern of events that indicates that type of fraud: so this pattern consists of the “lots of other events”. To take a more prosaic example, the completion of an online shopping basket is a complex event that is dependent on a whole series of preceding steps.

Also, bear in mind that events are not limited to a single environment: for example, the availability of hotel rooms represents a complex event that will be influenced by such diverse individual events as the weather, time of year, whether there are any conferences in town and how many, the state of the economy and so on.

So, CEP is essentially about monitoring those individual steps or (micro)events or, in some cases, transactions, and then looking to see if they make up the complex event you are looking for. Or, of course, you might be looking for any one of a number of different complex events so there are multiple patterns against which the incoming event data must be tested against.

That’s it, basically. Of course there are bells and whistles that you can add to make the software faster or easier to use or look prettier but, essentially, CEP is about two things: monitoring events and then looking to see if those events fit into patterns that you are looking for or, sometimes, looking for events that don’t fit an expected pattern (anomaly detection). If you think about what the intelligence communities do, those responsible for squashing terrorist threats before anyone is affected, they are looking for a series of events to establish and identify a complex event that might be a threat. That’s the type of things they’ve been doing for years and businesses can leverage the same approach.

So, the question, naturally, is what can CEP do for your business? And the answers are as diverse as industry and commerce. Two generic examples are a) any environment in which you might want to prevent and/or detect fraud or criminal behaviour or any sort of unwanted behaviour (even if it is not actually illegal but is, perhaps, against corporate governance policies); and b) any network that you need to monitor, whether that be a road, rail, pipeline, computer or utility network. In its broadest sense you can even think of a shop-floor production line as a sort of network and certainly CEP has been employed on the shop-floor as it has in airports and by airlines.

An even more generic example is when you want to link events to a process of some sort. It is often the case that many business processes, for example, are embedded within application software while other processes have been formally modelled and are managed within a BPM (business process management) environment. And then, of course, informal processes abound. One of the issues that arises is how to link these together, and one answer to that is to use CEP, treating each step in a business process and each transaction as an event in its own right. However, this is not limited to business processes per se but any sort of environment where processes are involved, including process manufacturing (the shop-floor again), communications processes (not necessarily within communications companies), integration processes (witness Informatica’s acquisition of AgentLogic) and so on.

Finally, there lots of specific use cases: monitoring PC fleets for carbon emissions, monitoring stock ticks within capital markets, monitoring automated number plate recognition systems, monitoring patient heart rates and so on and so forth.

The bottom line is that if you need something monitored then you may need CEP. It might be because you need to detect a problem (typically, something anomalous has happened); or because there is an opportunity to buy stock or up-sell or cross-sell to an existing customer because you have recognised a particular pattern of events that predicts a higher than usual success rate (using CEP in conjunction with predictive analytics); or because you need to prevent, or at least detect, unwanted behaviour such as a security breach or potential fraud. But whatever your requirement is don’t get put off because it is called complex: it isn’t or, at least, it doesn’t have to be (with the right vendor).