Breakthrough and instrumented applications

Breakthrough applications are the flip side of big data. They don’t necessarily involve huge amounts of data but they do typically involve some of the same sort of data that is often associated with big data. That is: sensor-based, web-based, log-based and similar data.

However, I don’t think that ‘breakthrough’ applications is very good term. After all, once they become commonplace, as they will, then the term ‘breakthrough’ will have become past its sell-by date. I think a more useful term is instrumented applications. I also prefer the expression ‘instrumented data’ as opposed to what some refer to as machine-generated data. Consider an application based on clickstream data, whereby you are, in part, monitoring user behaviour on your web site. I do not think that most people think of the Internet, or a web site, as a ‘machine’. Similarly, if you have an application providing location-based services to a mobile phone then I do not believe that most people think of their iPhone as a ‘machine’ – it simply conjures up other images. Conversely, instrumentation suggests monitoring of some sort, which is precisely what you are doing, so I prefer instrumented data as well as instrumented applications.

Instrumented applications typically consist of collecting instrumented data, possibly combined with transactional data, some analysis of that data and then the results presented within an operational environment, which may involve some broader business process or decision-making. In other words, these are usually hybrid analytic/operational environments.

A couple of good examples are InsureTheBox and Optalert. The former is a motor insurer that fits a box into your car, collects telemetry about your driving and rewards you if you drive well. However, it isn’t just a back-office application. The telemetry is monitoring the g forces on your car in real-time: if there is a particularly high g measurement and then you stop moving the company will assume that you have had a crash and call your mobile. If you don’t respond they will assume you are unconscious and call emergency services. So there is a real-time element to this application. This is common to many instrumented applications and also applies to Optalert which is an application designed for the drivers of lorries, trucks, mining equipment and so forth. The drivers wear special glasses that monitor how often the driver blinks as this is an indicator of tiredness. They can therefore alert drivers (and their employers) when it is time to take a break, in real-time, and the collected data can also be used to analyse and improve shift patterns and working hour agreements to minimise the effects of sleepiness.

The principle requirements for building instrumented applications are a database that you can fire up and forget about, support for the sort of instrumentation required and, for ISVs, relevant tooling to develop the relevant application. Suitable databases would be Caché from InterSystems (used by Optalert), IBM Informix (especially for time series and/or spatial applications), Ingres and Progress, as well as data warehousing products that don’t require tuning, such as IBM Netezza (used by InsureTheBox), Infobright and others.