InterSystems data technologies - the innovative database platform that you've never heard of

The subhead on this article is a bit unfair. Lots of people in Healthcare have heard of InterSystems, it has been profitable year-on-year since it started in the last century and it is used by highly visible projects like Gaia where the competition just can’t cope. The European Space Agency’s astrometry Gaia project involves a space observatory, which aims to catalog approximately 1 billion astronomical objects, mainly stars but also including planets, comets, asteroids, quasars etc. and that’s a lot of data to store and access efficiently.

So, why haven’t YOU (assuming that you don’t work in healthcare) heard of this, apparently very powerful,  data platform? Well, Intersystems is privately owned and run by an engineer who values engineering and happy customers above marketing (not priorities shared by some public companies I can think of). This isn’t to say that InterSystems’ marketing is poor; but it has been focused on the Healthcare sector – although it can’t be too bad, since InterSystems is successful and has always been profitable, year-on-year. Again, contrast this with some public companies that spend a lot of their time just making sure that Wall Street hears about them, rather than focusing on their customers.

However, I’m not a healthcare specialist particularly, so why was I at InterSystems’ UK Technology Summit near Birmingham recently? Well, because my background in DBA means that I appreciate a good database management system (DBMS) and I think that InterSystems Caché gets it more right than many do. It follows one of the key tenets of Relational database theory, by completely abstracting the underlying physical storage mechanism (an efficient sparse bitmap array) from the logical model seen by users and programmers – which can be an extremely fast SQL relational DBMS view. It’s fast, because it doesn’t store relational tables, it stores the underlying data in the most efficient way that it can.

In fact, however, Caché is a NO-SQL (in the sense “Not Only SQL”) database (“multimodel, as InterSystems puts it), which can just as easily and efficiently support an OO-DBMS (object database) model and others. This makes it an excellent match to modern data environments, which often include both relational operational data and document-oriented (hierarchical) data; as well as what is (misnamed) Unstructured Data or Big Data.

InterSystems, in reality, is offering a data platform rather than just a data store. Remember that term “DBMS”? The “MS” stands for “Management System” and the InterSystems platform offers management and other services that go well beyond just a data storage capability. You can use iKnow for processing “unstructured” natural language or text data; or embedded DeepSee real-time analytics; or add Ensemble integration technology, built on Caché (it’s a sort of “enterprise service bus on steroids”) to orchestrate existing services with new ones. It also includes effective “rapid application development” capabilities.

The InterSystems platform really is rather good, so surely there must be a downside that explains why it isn’t more widely known? Well, the big three database vendors do spend lots of money promoting the idea that the big three RDBMSs are the only safe choices for serious database work (and they are no longer “pure” relational databases either, although their addition of, say, OODBMS capabilities isn’t as elegant as the way Caché does it), but that isn’t all of it. The InterSystems platform has been a bit idiosyncratic and, possibly, a bit hard to get used to if you have been trained on other platforms. This is now being addressed as InterSystems updates its tools and interfaces, and integrates with OpenSource tools such as Apache Spark (see also my notes on the introduction of an Eclipse IDE last year).

I think that we’ll be hearing more about InterSystems outside of its traditional markets soon. This isn’t just because I think we are really going to need its capabilities as we move towards Mutable and IoT (which have scale and performance characteristics that the InterSystems platform is well-suited to), but also because I sense a new focus on expanding into new markets in the InterSystems managers I have been talking with.