At the 19th Hole

We don’t know what Sybase has to do with golf, but it seems to sponsor a golf tournament in New Jersey and use this as an excuse to tell analysts about what it is doing. I suppose anything done really well is interesting, and this was true here; but I think I’ve now seen all the golf I want to for a while…

On the other hand, there are a couple of neat Sybase products we really want to follow up on, starting with Sybase IQ (the other is PowerDesigner, see a subsequent article). Sybase has always had a strong following in the financial space but Sybase IQ just might be bringing it to more general attention. It’s a database with some interesting and innovative features for very large/complex analytical queries against very large volumes of data—we first met it years ago, about when Microsoft was buying SQL Server from Sybase and someone at Sybase was telling us that Microsoft had really bought the wrong database from it. And Microsoft might even agree, since the (really quite impressive) SQL Server today is a total rewrite of that purchase.

See Philip Howard’s “What’s cool about columns” paper for an explanation of the column-processing (as opposed to row-processing) technology available to IQ users.

Anyway, Sybase IQ has been bubbling along—selling pretty well to those that know what it can do but nothing very special—until now, when sales seem to have taken off. It’s an excellent decision support database with an SQL front end—so it is no harder to program for than any other SQL database (not that the general standard of SQL programming is that high, but that’s a different issue). And it scales to compete with the likes of Teradata and Netezza (well, Sybase has customer case studies where it was bought against this opposition).

We talked with Tom Traubitz, Director of Database Products ITSG about a trend we see these days, towards mixed workload on-line transaction processing (OLTP) and near-real-time decision support databases. This is often seen as the “one size fits all” database—which is definitely not a trend Traubitz sees, because sales of his specialist Sybase IQ database for very large systems are ramping up. And, to be fair, we don’t really see the mixed workload database as “one size fits all” either.

When we got down to technicalities, there was a great degree of agreement betwen us. One of the fundamental principles of relational database management systems (RDBMSs) is that the storage on disk is entirely abstracted from the logical table view presented to programmers. IQ largely achieves that, with an SQL programming interface on top of extremely clever storage and retrieval technology, including very efficient indexing (bitmap indexing for example, although bitmap indexes aren’t unique to IQ; these days, even Oracle has them).

Now, the database optimiser is what makes or breaks an RDBMS, a view with which Traubitz entirely concurs (he seems rather proud of Sybase’s optimiser in Adaptive Server Enterprise (ASE), Sybase’s OLTP relational database) and if you take a sophisticated RDBMS query optimiser and blend it with an intelligent storage management system (something Sybase hasn’t done yet, but let’s speculate), there is no reason why you shouldn’t fire off a query at a mixed ASE and IQ database and get back the answer from whichever database, or mix of databases, is most appropriate, with the programmer entirely unaware of where the data is actually stored.

Updates are an issue, since rebuilding IQ indices would probably be an unacceptable overhead, but a near-real-time decision support query would probably want to get actively changing data from the OLTP database anyway—and automatic replication could take care of time series data. And Philip Howard tells me that Sybase will address the IQ update issue in the next release anyway.

There is seldom only one “right answer” when implementing IT (as long as you get the underlying principles right) and this hypothetical mixed ASE/IQ database environment doesn’t sound a lot different in principle to a highly partitioned mixed-workload Oracle database. Interestingly, Traubitz says that back in 1993-1996 when IQ was brand new to most customers and early sales were mostly for fraud detection and the like (analysing data already stored in ASE Databases), there was discussion within Sybase as to whether IQ should be separately named or simply treated as an ASE “feature”.

Now, this piece is just
speculation on an interesting aspect of RDBMS design and, indirectly, explains
a bit of why a proper DBMS is so much more than just a simple database product.
However Phillip Howard reminds us that there’s a product called Sybase “RAP: The Trading Edition”
(it used to be Real-time Analytics Platform, and was used in a partnership
with Streambase, for example), which is a front-end for high volume real-time
financial processing based on in-memory transaction processing on ASE and time-series IQ analytics. So perhaps, in the end, it’s not all
that speculative after all…