illuminate: another data warehousing player

I have discovered yet another player in the data warehousing
market. While the product’s sweet spot is typically in
gigabytes rather than terabytes it is interesting, not just because
of its technology, but also because illuminate Solutions
effectively markets a BI solution with a warehouse built-in (which
makes it good for partners who want to embed it in their own
products) rather than a pure data warehouse.

Anyway, illuminate is a Spanish company with offices and/or
partners in France, the Netherlands, the United States, Caribbean
and South America. Having been founded in 2005 it has 30 direct
customers and 33 indirect ones (embedded solutions from partners).
That’s quite a lot for a company most readers will not have
heard of.

As far as the product is concerned, this uses something called
value-based storage. Basically, you store each value that any
record has: say “23”, just once. Clearly there may be
multiple records that would otherwise contain this value and this
is managed through the storage model and relevant metadata. As you
can imagine this makes the physical database very compact. So much
so, in fact, that the entire database is indexed (automatically,
when you load the data), though different types of techniques are
used (tokens, bit-mapping, vectors, Btrees) depending on the
datatype.

So the database is small and fast. Existing customers have not
gone beyond 200 to 300Gb in size (probably equivalent to a terabyte
or so for a more conventional product) so how far the product would
scale as a pure warehouse is unclear at present.

However, as I mentioned it is not the warehouse per se that is
important. What is important is the query functionality that
illuminate has built on top of the warehouse to exploit its
value-based nature. This allows you to ask queries that you could
not normally define using standard SQL. For example, it is trivial
to ask any questions based on value and to drill down
therefrom.

Now, you can use these capabilities directly through
illuminate’s own tools, which include dashboard and mapping
products, plus its data mining and pattern recognition software
(which, incidentally runs directly within the database with all the
performance benefits that that implies: though there are relatively
few algorithms supported at present). Or you can use illuminate
Explorer, which runs with both the company’s own products and
with third party products such as QlikTech, Cognos, BusinessObjects
and so forth to provide ad hoc query and interactive analysis
capabilities. There is also a conventional ODBC interface though
most companies deploying third party tools will probably want to
take advantage of the extended features that Explorer can provide
over and above standard SQL.

It is in its support for ad hoc and unpredictable queries that
illuminate is especially strong when compared to conventional
approaches. This is because everything is indexed so of course you
get better performance. Moreover, because the indexing is automatic
there is no tuning required and because of the value-based nature
of the storage there is never any need for warehouse restructuring
or database reorganisation—so this is a very low maintenance
solution as well as a high performing one—it should therefore
be no surprise that illuminate has gained a significant number of
customers in only a couple of years. I expect it to gain more.