McObject eXtremeDB

McObject’s eXtremeDB, whose architecture is illustrated in Figure 1, is an embedded (less than 200 KB footprint) hybrid in-memory and persistent database designed specifically to support time-series data. Unlike some other in-memory databases, it has specific capabilities designed to eliminate latency when transferring data between memory and CPU cache. Sequential data, such as time-series data, is stored in columnar format, while other data is represented in row format. There is an optional extension to allow data (transaction logging) to be persisted to disk as well as an optional SQL extension (to go along with the standard C/C++ API), which means that you can integrate visualisation tools such as Tableau or Qlik. Alongside this SQL capability there is a rule-based deterministic optimiser. As a general rule we prefer cost-based optimisers (especially if machine learning has been built-in) but in an embedded database it makes sense to take this approach as you do not want the overhead of gathering statistics. The database supports ACID properties and there are high availability and security features built-in, with support for inter-node replication and sharding. An XML extension is also available, as are compression capabilities.

Fig 01 eXtremeDB architecture

Customer Quotes

“eXtremeDB helped cut 18 programmer months from the development cycle.”
Boeing

“eXtremeDB-64 contributes significantly to trading platform robustness, with a proven database transaction mechanism that is optimized for high throughput.”
Dalian Commodity Exchange

For time series data, database indexes are implemented against timestamps. For row-oriented data, eXtremeDB also supports hash indexes, B-trees, Patricie Trie, KD-trees and R-trees. The last of these is particularly notable since this is a technique used for geo-spatial indexing and eXtremeDB specifically supports bounded rectangles (note that points and lines are both collapsed rectangles). This is important, because many IoT applications involving time-series also have geo-spatial requirements. On the other hand, this is not typically true in financial situations where you are dealing with tick data. For these environments, McObject provides a library of around 150 vector-based statistical functions to support financial, and virtually all other mathematical/statistical analyses. These functions can be pipelined into a sequential series of operations.

Fig 02 eXtremeDB Performance Monitor

For managing your environment, the product comes with the xPanel Dashboard, which provides a number of monitoring and management capabilities. These include the xSQL Configuration Tool, the Feed Handler (settings for financial market feeds) Configuration Tool, the SQL System Analyzer, a Performance Monitor (see Figure 2), the Tracer (tracking run-time activity), a Database Browser, NetViewer, and a graphical user interface (which they all are) for viewing web service connectors.

In financial markets eXtremeDB can not only out-compete other vendors focusing on processing tick data but it has the added advantage of supporting SQL and other conventional languages rather than proprietary ones, which is the case with some of its competitors. For other use cases within financial services, and possibly IoT also, we would like to see the product integrated with streaming technologies such as Kafka and Flink, but the company assures us that that would be easy to do.

In industrial and IoT environments the very small footprint of eXtremeDB is a major boon. While there are other databases with small footprints they do not generally have time-series and geo-spatial capabilities, so it comes as no particular surprise that eXtremeDB has so many millions of implementations.

The Bottom Line

McObject eXtremeDB is the leading database for embedded IoT and industrial environments. It is competitive in capital markets without being a dominant player in the same way that it is within (edge) devices and gateways.