Teradata Partners Oct 2007 – Part 1.

The Teradata Partners User Group conference in Las Vegas this year was on a bit of a high, claiming a record 3875 attendees—and there was some enthusiasm about its new status as a separate company outside NCR.

The technical highlight of the conference were the release of Teradata 12, supported by the 5500 “green” server shipped in August (which claims a 75% reduction in electricity consumption compared to Teradata servers of 3–5 years ago). We’ll discuss the technical capabilities of Teradata in Part 2 of this report.

Teradata Relationship Manager 6 was also released, improving Teradata’s marketing automation product. This is the foundation of Teradata’s successful customer management solution, which comes out well in Gartner’s Magic Quadrant for Multichannel Campaign Management.

Also announced was a new strategic relationship with SAS. SAS and Teradata seem to be a good fit culturally (perhaps more so than, say, SAS and Oracle; it is always possible that SAS’s Jim Goodnight and Oracle’s Larry Ellison simply don’t get along and SAS is traditionally very protective of its intellectual property). This may give SAS on Teradata a technical edge (it’s more than just a marketing thing) as SAS can now process data directly on the Teradata server, thus reducing expensive data movement overheads. In the past, you’d move data from Teradata into a SAS datamart for processing against the SAS algorithms; and possibly move the results back into the Teradata warehouse.

The main conference message seemed to be that tactical Active Data Warehousing (ADW)—mixed workload processing of near-real-time decision support information—was now a key focus for Enterprise Data Warehousing (EDW); although abandoning conventional strategic decision support wasn’t being contemplated.

Consider fraud detection for stolen credit cards, for example; the key patterns (a small non-customer-facing transaction, at a petrol pump perhaps, to test the card; followed by large purchases of easily sold goods) occur within 48 hours of theft, so you don’t want to wait 24 hours before you start looking for them and blocking stolen cards. This means ADW is important. On the other hand, some people do legitimately make small petrol purchases, although usually only at a few specific garages unlikely to be chosen by a card thief—so you also need long term customer behaviour profiles, which you get from a conventional EDW enquiry.

Teradata looks well able to cope with the largest, mixed workload, decision support applications; but it is either not interested in, or can’t, add OLTP applications to the mix. That’s not so surprising—its database is highly optimised for read performance, it logically rather than physically deletes records and it maintains referential integrity programmatically, after loads complete. Maintaining the real-time referential integrity, the 24×7 availability and the mean-time-to-recovery in minutes the best OLTP systems can manage, at the same time as handling large, complex, decision support enquiries, might stretch even Teradata (although Teradata 12 seems to be doing a good job with the availability necessary for mission-critical ADW). And it might not be necessary even if you do believe in the ideal of “a single company database”—perhaps the OLTP database is simply a transient “cache” feeding the EDW in low priority transactions.

However, if Teradata fully virtualises its hardware, can partition its workload across different devices as appropriate, can manage pre-emptive prioritisation well and even (using cylinder reads) scan small blocksize databases efficiently—and we believe it can—one starts to feel that it ought to be able to include OLTP in its mixed workloads. Is it so optimised for read performance that this really isn’t feasible? And might this give its competition an opportunity?

Could, for example, Oracle, DB2 or even Microsoft’s SQL Server, pull off this trick “well enough”, to take away Teradata’s lower end customers? A single database supporting both OLTP and near-real-time decision support queries would, presumably, save its owners money, compared with running both an OLTP system and a data warehouse—and moving data between them. Oracle already has one customer claiming to run a consolidated data warehouse and OLTP database and, by all accounts, it was part of Jim Grey’s vision for Microsoft SQL Server, although this vision may not entirely survive his unfortunate demise, of course. We shall see.

In the meantime, Teradata has more immediate competition, typified by Netezza (Greenplum and DATAllegro are further examples). Steve Brobst (Teradata’s CTO) apparently sees Netezza as “just” supplying a very large data mart appliance—Teradata has the skills and technology to build such appliances itself but apparently thinks they are a bit of a dead end. And yet Netezza’s technology isn’t standing still; its “zone maps” mean that it isn’t really limited to table scans, each release seems to be faster than the previous one and the virtualisation of its underlying proprietary FPGA (field programmable gate array) hardware gives it an industry-standard SQL interface (and although Brobst claims that Teradata’s use of industry standard Intel technology underneath, instead of custom hardware, gives it an edge in both flexibility and performance, Netezza does seem to be very fast). But, whatever you think of Netezza’s performance claims, perhaps it could run at least an entry level EDW well enough and, once again, nibble away at Teradata’s lower end customerbase.

The Teradata execs we talked to at the conference do apparently see HP NeoView as a real threat, as HP is definitely targeting the top-end Enterprise Data Warehouse marketplace where Teradata likes to feel secure (see here). However, Brobst claims that HP CEO Mark Hurd (ex Teradata COO and NCR CEO) brought marketing rather than technical knowledge about Enterprise Data Warehousing into HP—and that few, if any, real Teradata technicians moved across to HP.

Nevertheless, HP does have real database technology to build on, of course: Tandem’s highly scalable NonStop SQL (originally developed by Jim Gray, amongst others), which can effectively exploit parallel architectures as Teradata does, although it has had some issues with the SQL standard. Brobst claims that HP isn’t really admitting to the importance of NonStop to its solution; he says that HP is trying to present NeoView as more “open systems” than it really is (it claims to run on “industry standard servers and storage”, which seems fair enough to us, as far as it goes). He also seems to claim that NeoView’s competitive wins are rather more about cutting good deals than out-and-out technology victories—but perhaps it’s too early to decide on this, as some deals aren’t in the public domain yet.

Overall, however, we do think that Teradata may be in some denial about emerging threats to its market dominance, possibly led by its excellent performance in Gartner’s “Magic Quadrants”—it’s just been declared a Leader in the Magic Quadrant for Data Warehouse Database Management Systems, 2007. Magic Quadrants don’t always turn out to mean what people think they do—as Gartner says “Gartner… does not advise technology users to select only those vendors placed in the “Leaders” quadrant. The Magic Quadrant is intended solely as a research tool…” and sometimes the real competition doesn’t show up because it appears in a subtly different Quadrant, perhaps one not drawn yet.

We heard from several Teradata execs during this conference (Mike Koehler, President and CEO; Steve Brobst, CTO; Scott Gnau, Chief Development Officer) and while they claim to take all competition seriously, they didn’t seem to take it all that seriously. Perhaps time will show them to be right in their apparent confidence, but they weren’t very forthcoming to us about explicit measures dealing with the new competition, although that might just be natural caution on their part.