For everything there is a season

All good things come to those that wait. That could be the position that Kx Systems finds itself in. For those that don’t know the company, Kx Systems was one of the first companies to deploy column-based database technology, back in the previous century. Like most other columnar products at that time (the exception was Sybase) it decided to focus on one particular market which, in the case of Kx Systems, was the financial sector and, specifically, trading platforms (the company has been especially successful with hedge funds).

To make this work, Kx Systems had to implement a number of other innovations in its product (Kdb+) not just columnar storage. To begin with it provides a form of stream processing. Just to clarify: complex event (and now stream) processing engines are fine if you want to do algorithmic trading solely on the basis of what’s happening now, but if you want to provide context to what is happening now then you need historic information as well. Kx Systems provides both, which has always been its differentiator compared to real-time processing that does not offer any persistence. Just to give you an idea of the scale (and performance) that Kdb+ works at, time stamps are accurate at the nanosecond level.

Note that this support for time stamps is another of the innovations I mentioned. Virtually no relational databases (IBM Informix is an exception) have time stamps and while NoSQL databases like Cassandra support time stamps they don’t offer either the performance or scale that you need for financial trading.

I haven’t finished yet. There are two more things that are top of mind nowadays: in-memory processing and vector (array) processing. Kdb+ does them both.

So, Kx Systems has all these cool technologies that IBM, SAP, Oracle and the rest are hyping to the skies and, what’s more, it’s had them for ages.

Of course, there’s a downside. The first is that Kx Systems built its own language (derived from APL) to enable vector processing, so it’s not quite the same thing that you would get from Actian in VectorWise or from IBM in BLU Acceleration. Historically, this has also meant that there were no BI or visualisation tools that were available to run with Kdb+. This, though, has changed; firstly through support for WebSockets and secondly because Kx Systems has partnered with Datawatch so that the latter’s Panopticon visualisation product now supports Kdb+.

The second downside is that Kx Systems is still relatively small: it has only around 100 direct customers, though some of these are major institutions, but it also has a number of OEM partners. For database enthusiasts the most interesting of these is 1010 Data, whose offering is based on Kx Systems technology.

The upside is that the technology that Kx Systems offers is really well-placed to serve the Internet of Things. The company has recognised this fact and has already got its first energy and pharmaceutical customers and is also pursuing asset management (preventative maintenance) opportunities. The financial markets that Kx Systems has historically addressed is lucrative but niche; the potential its technology offers for managing machine-to-machine environments is not just lucrative but huge. It may have taken some time but Kx Systems looks like it’s finally in the right place at the right time.