KX

At its core, KX (or, more properly, KX Streaming Analytics) is a highly performant streaming analytics platform layered on top of a time-series database. It allows you to analyse streaming, historical, in-memory, and time-series data rapidly and simultaneously. In turn, this can enable faster and better decision-making by providing you with up-to-date, comprehensive, and intelligent information at your fingertips via dashboards, apps, alerts, and so on (conceptually illustrated in Figure 1). The platform also offers real-time stream processing and an in-memory compute engine, as well as various industry-specific solutions. The platform’s design is such that it offers a broad base out of the box, which can then be built upon using these solutions to tailor it to your particular needs.

Fig 1 – KX conceptual architecture

The combination of stream processing, in-memory processing and persistent data storage means that KX provides what is effectively a Kappa architecture. Where it differs from many other potential providers of Kappa, or indeed Lambda, architectures – particularly where these are to be built on a variety of open-source offerings – is that KX offers a consistent, unified approach to supporting the combination of real-time and batch analytics that Lambda and Kappa architectures are aimed at, with a single code base.

Customer Quotes

“Through its ability to rapidly process vast amounts of time-series data, provide analytics in real-time, and integrate with oura machine learning pipelines, Kx is ideally placed to power our neuroscience platform.”
BrainWaveBank

“There are many valuable applications of satellite imagery across a range of industries, many of which are time sensitive and require powerful analytic processing. By combining our data and Kx technology we expect to be able to provide new and existing customers with unique and valuable insights.”
Airbus Defence and Space

KX is built on the Kdb+ database. Although KX refers to it (not without cause) as a time-series database, its under-the-hood architecture is columnar. On top of Kdb+, the company provides a built-in array processing language called q. The array-based nature of this processing means that it can work across both columns and rows simultaneously, making it much more efficient that traditional approaches. It is also worth noting the terseness of q, which is actually SQL-like. qSQL also exists as an explicitly SQL-like table querying syntax. Recent releases have also provided some support for SQL itself. In addition, a large number of connectors are available – more than a hundred in total – and you can call functions, methods and libraries written in Python, R, C and C++ from within q code. Integration with C# and Java is also provided, as are ODBC and JDBC drivers.

The combination of Kdb+ and array processing facilitates concurrent execution and parallelism, which feeds into the platform’s notably excellent benchmark performance. It’s suitable for large scale (100+ GB of data per day) deployments, and it has an impressively small footprint of less than a megabyte. This gives it significant scalability: it is equally capable of deploying to a centralised enterprise system as to the edge. Furthermore, it supports deployment across multiple machines with a distributed capability for scale-out in clustered environments. Resilience and automatic recovery are also provided, together with load balancing and replication.

KX has invested significantly into the cloud via KX Insights, which is essentially the platform’s cloud-first deployment option. Accordingly, it can be deployed across public and private clouds, including multi-cloud and hybrid environments, and it provides OpenShift-certified support for Docker and Kubernetes. It is available from the AWS, Google Cloud Platform, and Microsoft Azure cloud marketplaces as well as from KX directly.

Fig 2 – KX dashboards and visualisations

The company has also made a point of investing in machine learning (ML) and AI. To wit, KX provides an out-of-the-box ‘ML Toolkit’ that offers data pre-processing, model training, model scoring, and so on. Clustering, natural language processing, and support for external machine learning models are also available. Perhaps most notably, KX offers an ‘AutoML’ feature, which promises to help automate your machine learning processes through the use of customisable, end-to-end workflows. The platform also provides data preparation capabilities, native statistical libraries, and integration with various machine learning environments such as TensorFlow, Theano and others.

Other notable capabilities offered by the KX platform include real-time monitoring, alerting and network analysis. It also features dashboarding functionality and real-time visualisations, as shown in Figure 2.

There are a number of reasons to like KX. It boasts significant (and impressive) technical advantages, particularly in terms of performance and footprint. Although it was originally designed primarily to serve the financial services sector, these advantages apply just as well to (and the platform is compatible with) many different industries and environments, most notably IoT and the edge.

Moreover, KX can analyse multiple different kinds of data (most notably real-time and batch data) within a single platform, simultaneously; it leverages time-series capability as a core functionality (a rarity in the streaming space); it provides highly contextualised and intelligent insights on a continuous basis; and it offers substantial support for machine learning. This all results in the rapid delivery of analytics that are also robustly functional, comprehensive, and useful.

The Bottom Line

KX differentiates itself most notably with its high-end performance, miniscule footprint, and first-order time-series support. Accordingly, if you want to deploy a streaming solution to the edge, analyse streaming data (particularly sensor data) where time is a factor, or generate comprehensive analyses at speed across many different kinds of data, KX is more than worth a look.