TIBCO

The key TIBCO contribution is that the Edge, at scale, becomes an integral and native part of the whole corporate infrastructure. This brings two specific advantages: one allows Edge devices, particularly as they develop from passive sensors to become small, duplex systems that can be re-programmable – in the field or via the Web – to accommodate changes in the tasks they are required to perform. The second is that the data flow of a business process can be layered and widely distributed, reducing the overall time and costs of excessive data movement. Individual devices can process local raw data into something of value, which in turn can be processed in departmental/regional hubs or nodes, which then provide data of immediate value to the business up the chain to the main data centres. Reports can then be sent to executives in formats suitable for a wide range of end user clients – from laptops to smartphones to Raspberry Pi-based devices.

An important factor here is the increasing penetration of 5G mobile communications, which TIBCO sees as offering the potential to make the Edge increasingly smart, which in turn is likely to have the effect of increasing the value and importance of the Edge, moving the Centre of Gravity of the whole corporate network down towards the highly distributed end.

This trend is only likely to increase as AI is added to Edge capabilities. For example, AI is already finding real-world applications in areas where only limited real-time choices and options are possible and the speed of decision making is the primary value, an application type that can be widely expected to be found in edge computing operations.

The company sees the potential for applications such as these to be able to perform detailed, in-depth forensics, in real-time and at the point of occurrence, as would be possible with a major forensic application running on the main back-end data centre – and providing not only a result but also actions based on that result, far more quickly. In applications where the action concerns the location of a shipping container speed may not seem overly relevant, but when the application is the safe management of a loaded highspeed train, getting a result and applying it can prove vital.

Another important aspect, especially when it comes to the development of Edge systems and services as a new part of established enterprise environments, is the provision of effective governance services. TIBCOs long track record with large enterprises means that it already brings extensive skills and capabilities in providing and managing governance services at both scale and complexity. Some smaller businesses may find this capability important to them as they break into new markets.

Different sectors of business will obviously have different requirements when it comes to the specific tools that are required, and TIBCO has a simple policy for such areas, its goal is to seamlessly integrate with the tools and platforms the customer selects or already uses, on the basis of: ‘why develop it when you can connect to it’.

Fig 01 - A model flow in TIBCO Statistica

The core of model management in TIBCO Data Science is the model flow. As seen in Figure 1, this is essentially a form of workflow, created within Statistica, a component of TIBCO Data Science, using a no-code GUI. These flows contain data inputs, models and outputs (shown in Figure 1 in green, orange and purple, respectively) as well the standard flow controls. Business logic can be implemented within a workflow via a rules node, and the flows themselves can be saved as templates and reused inside other model flows. The latter capability is particularly valuable because it allows dedicated data scientists to create complicated, in-depth models and flows, before allowing business users to situate them within the appropriate business processes.

Models and model flows are centrally stored and managed within the platform and accessed via a file-tree structure. Models are versioned, with all previous versions stored within the platform, available to be accessed, deployed or reverted to, as necessary. Third party models (that is, models not created within the data science platform – all standard languages are supported) can also be imported into the platform.

Models and flows held within TIBCO Data Science can be deployed across the broader range of TIBCO products. For example, you could publish your models to the Artifact Management Server, a common model repository, to expose them to TIBCO Streaming (formerly known as StreamBase) for use with streaming analytics, or send them to Spotfire to drive data visualisations. You can also monitor and retrain models en masse via the Monitoring and Alerting Server (a component of TIBCO Data Science Operations), and models can be exposed for external use via OData.

TIBCO Data Science also features a number of collaborative capabilities between data scientists and business users that come into play with model management. Models and model flows (as well as other objects, such as projects) can be shared and worked on collaboratively across the platform. Moreover, models and model flows can be parameterised and tailored to business users before being exposed to them. This might include only exposing a subset of the model’s parameters, renaming fields to make them more easily understood, setting default values, and so on. The aim is to expose all the functionality that a business user needs, and no more, thereby making their interactions with the model as simple as possible. Once a model has been tailored in this way, it can be saved as a template and shared as needed. This is also tightly integrated with TIBCO Spotfire so citizen data scientists can run data science workflows directly from a Spotfire dashboard.

TIBCO Data Science is a platform for analytics in the broadest possible sense. It’s undeniable that machine learning, AI and data science are becoming ever more popular and important parts of analytics, and as a result, if you are leveraging analytics you are almost certainly leveraging analytical models. The sheer amount of these models is growing: we expect large organisations to be utilising thousands or even tens of thousands of models in the near future, if they aren’t already. At that quantity, model operations and management is essential, and TIBCO provides it along with all of its other capabilities.

What’s more, as modelling becomes more common and more popular, data scientists will be more needed (and therefore more in demand) than ever. While TIBCO (or other product) can eliminate the need for data scientists, its collaborative capabilities do enable more effective and efficient use of the data scientists and citizen data scientists you have, by enabling business users to take a larger role in the modelling process. This maximises the value of your data scientists, while providing a measure of self-service for your business users.

The Bottom Line

Model management will soon be essential for any serious analytics solution. TIBCO Data Science not only has it, but integrates it with a wealth of other analytics tools and products.

From a time-series perspective Spotfire works on the basis of time windows (various options) that you define. Specifically, it provides high level operators called Aggregate, Pattern, Join, Query and Gather, which can be combined and sequenced as required. Some 58 aggregate functions are built into the product, including both basic analytical functions (mean, median, standard deviation and so on) and more advanced statistical functions (slope, intercept, correlation, exponential moving average and so forth). A Java aggregate function API is available for users to define their own aggregate functions, if desired.

The Pattern operator has its own sub-language for sequential, value-based and temporal patterns, to which Boolean logic may be applied. Patterns within streams may also be detected using one or more operators in combination. The Join and Gather operators join streams either as two-way or multi-way joins respectively. Finally, the Query operator allows you to join a stream against a relational table, so that you can combine real-time and historic data. Streams may be buffered in such a way that arrival order is not material to the results of the operation so long as emission order is not material either.

Fig 02 Mapping the human body

Of course, for many IoT and industrial applications support for time-series analytics is not enough: you also need to combine this with location intelligence and Spotfire provides significant capabilities in this regard, supporting both geographical and non-geographical (see Figure 2) maps. Multi-layer mapping lets you zoom into successive levels of detail, for example, from the country to the state, county, city, neighbourhood, and house level. Moreover, you can choose to see whatever associations are relevant to you, with data specific to the level you are working at. The product provides worldwide address-level geocoding as well as route calculations with step-by-step directions.

More generally, the product uses native connectors, not just for its analytics but also for data preparation (TIBCO calls it wrangling) so that you can connect to and blend data from  a variety of relational and NoSQL databases; and to cloud environments like Amazon Redshift, Databricks, RDS, Microsoft Azure SQL Database, Google Analytics, and Salesforce.com. You can also build your own custom connectors. Natural Language Query capability – which is used throughout the product – lets you search for any data or connector.

From a more general standpoint, Spotfire supports AI/ML and predictive models developed in other TIBCO tools such as Statistica, as well as supporting languages such as R, Python and Java, and models developed in third party environments such as Spark MLlib or H20 models, or which can be imported via PMML (predictive modelling mark-up language). These can be scored on streaming data by operators in Spotfire. Also notable is the TIBCO Artefact Management Service, which supports governed deployment for these models. Once approved, then can be pushed out to Spotfire applications, or the applications can be set up to query for updated models at regular intervals, or under certain conditions. These models can be updated at runtime without requiring any application downtime.

Spatial analytics is commonplace and all the providers of streaming analytics platforms have the ability to process streaming data via time windows, in some sort of similar fashion to TIBCO Spotfire. In addition, there are lots of products that enable you to visualise time-series data. What is rare – even to the point of almost non-existence – is to find an analytics platform that has any sort of sophisticated functions for analysing historic (stored in a database) time-series data, whether as a stand-alone function or in conjunction with real-time data. There are specialised tools that focus only on time-series data but we know of no other general-purpose tool, apart from TIBCO Spotfire, that has this ability.

The Bottom Line

TIBCO Spotfire appears to be unique. Not only have we been unable to find any product with comparable capabilities but none of the database vendors with time-series capabilities could point us in any different direction. We don’t think anything more needs to be said.

Fig 1 - Creating an app in TIBCO Streaming

Streaming is a high-performance platform for rapidly building and deploying real time streaming analytics applications. It enables a variety of different user personas, ranging from professional developers to highly knowledgeable but non-technical domain experts, to build these applications using a browser-based, low/no-code visual interface, as shown in Figure 1. The ease of use this approach provides allows all of the aforementioned personas to leverage self-service streaming data as part of their analytics projects without undue difficulty, thus creating additional value from your data, reducing the time spent developing apps, and democratising your streaming capabilities.

A wide variety of accelerators for Streaming are freely available via the TIBCO community. Each one consists of a collection of pre-built components that have been optimised for a particular use case. Examples include financial fraud, case management, and insurance pricing. Notably, a number of accelerators are built to support the Internet of Things (IoT), including the Connected Vehicles Accelerator, and the IoT Drilling Accelerator. Combined with the embedded BI and analytics capabilities TIBCO provides, and the option to deploy on the edge, this allows Streaming to perform well in IoT environments.

Although Streaming integrates with several other TIBCO products, its integration with Spotfire, TIBCO’s general analytics platform, is particularly strong. For instance, Spotfire offers a shared environment for storing, managing and deploying assets across both products. Notably, this includes the application of predictive, machine learning models to streaming data, deployed within Streaming itself. On that front, Streaming supports models written in Java, Python and R (the latter via an accelerator), as well as PMML (Predictive Model Markup Language) and SparkML.

Fig 2 - A Spotfire analytics dashboard

For a further example of this integration, Live Datamart, previously a separate product but now part of Streaming, is an in-memory data warehouse equipped with a continuous query processing engine that drives alerting, user actions, and decision rules that can automatically take action under specified conditions. More to the point, it also provides live, real-time visualisations of your streaming data that can be accessed and assembled into dashboards using Spotfire (see Figure 2).

As a streaming analytics solution, Streaming is remarkable for two core reasons. First, by adopting low/no-code design principles and a friendly user interface, it makes the development of streaming apps faster, easier, and above all, more accessible. This enables self-service streaming app creation, in particular for business users and domain experts, who would otherwise typically have to go through a lengthy back-and-forth with IT to get a new streaming app built (by which point the window of opportunity for the original use case may have closed). In turn, this should allow you to analyse and react to streaming data with much greater agility.

Second, TIBCO places the emphasis not (or at least, not entirely) on streaming analytics as a solution in and of itself, but as one part of a comprehensive analytics suite. This is supported by the company’s other products, many of which tightly integrate with Streaming. As already discussed, Spotfire is particularly notable in this regard. This is a good approach: ultimately, analytics is about generating actionable insights, and it stands to reason that a more comprehensive and connected analytics pipeline is going to generate more useful, more complete, and more accurate information from which to generate those insights.

There are other reasons to care about TIBCO’s offering, of course: these are merely the most major. Other notable advantages include the product’s support for machine learning, its suitability for deployment to the edge or the cloud, and its range of accelerators, which frequently enable significantly greater time-to-value.

The Bottom Line

TIBCO Streaming makes it easy to build streaming applications. It is a complete streaming analytics platform, and together with the rest of the company’s ecosystem, can play an integral part in a comprehensive and holistic analytics solution. In short, it is certainly worth your consideration.