Right Insight

We first consider BI …

There are two distinct workflows in BI, as outlined below. The first could be described as prescriptive BI and the second as truly interactive BI.

Prescriptive BI

Typically the work flow is:

Target/Goal Setting —> Data Assembly —> Data Presentation —> Data Navigation —> Results Sharing —> BI Review:

Target/Goal Setting —>

Prescriptive BI for any specific part of a company begins with setting goals and targets that are relevant to the BI capability being provided: including KPIs, simple numerical targets, thresholds that warn of low or high values, thresholds that trigger action and so on.

Data Assembly —>

The data acquisition pipeline is constructed to provide the data that will be reported on. This can in some circumstances be a complex data pipeline.

Data Presentation —>

The data will then be presented to the group of BI users in an agreed form, which may be as simple as a structured report or may involve visual data representation such as, line graphs, bar charts, histograms, pie charts, pictographs, scatter diagram and geographical data maps.

Data Navigation —>

While the data provided and the ways of exploring it may be prescriptive, it may nevertheless provide slick drill down capability and include multiple ways to graphically represent and compare parts of the BI data.

Results Sharing —>

Typically it is possible to share Bi results with co-workers, which will prove useful if something unusual is noticed.

BI Review

Typically BI systems are reviewed on a regular basis and, may then be enhanced as a result of the review.

Prescriptive BI can be any one of three categories: hindsight (usually reporting with some graphics and drill down), oversight (usually in the form of purpose designed dashboards) and foresight (usually prescriptive trend projections).

Interactive (or Exploratory) BI

The intention behind Interactive BI is to provide specific “power users” (including business analysts) with a broader capability than offered by prescriptive BI — an ability to explore data (within specified data resources) with the expectation that they may discover useful new insights and a deeper understanding of the data..

Typically the workflow is as follows:

Data Assembly —> Data Analysis —> Results Sharing —> BI Review:

Data Assembly —>

Typically, within constraints determined to some degree by data governance, the user is given the ability to construct their own data collections to examine — possibly within a data lake or analytical database environment. Data acquisition tools like Splunk or network pipeline tools like Kafka may be put at their disposal (often with IT assistance or supervision). Possibly also, data preparation tools will be made available.

Data Analysis —>

In general the user is provided with a range of useful BI capabilities via a range of BI tools that have been approved and implemented. Users will, no doubt have been trained in their usage. The capability will almost certainly also include a full query capability.

The range of BI tools may include: Descriptive analytics, predictive analytics, graphical analysis, dashboards with drill-down, performance metrics and benchmarking, statistical analysis tools, data visualization tools, visual analysis, and more. the data using statistics such as how this trend happened and why

Results Sharing —>

It will be possible to share Bi results with co-workers, directly, and where the “power user” or business analyst creates something new and useful it may eventually be migrated to be included among the prescriptive BI capability.

BI Review

The results and achievements of explorative BI users should naturally be included in the BI systems review.

AI

Let us now consider AI. AI is not employed by all companies and when it is employed it is usually implemented in a manner that make it complementary to BI.

The primary role of BI is to enhance decision-making and thus refine the various processes of the business.

The purpose of AI is, as the term implies, to increase the intelligence and cognitive capabilities of a business and to identify opportunities for automation or enhancements of business processes.

In reality, the term “AI” is a catch-all for a wide variety of technologies that mimic human intelligence or generate knowledge in their own right. It includes statistical algorithms, machine learning, deep learning, natural language and semantic technology, logic processing and neural networks (which are especially powerful in all area of pattern recognition).

Nowadays those who manage projects in this area tend to be referred to as Data Scientists and can be thought of as working in the R&D area of the company.

We can think of AI projects as having the following (possibly overlapping) targets:

1. Knowledge discovery. AI projects which focus on big data from a multitude of data sources are likely to discover new knowledge which can then be embedded within systems.
2. Problem solving. Most businesses face specific challenges. Some AI projects are undertaken to explore possible solutions for known business problems. This may include the analysis of streaming data.
3. Process automation. This is the use algorithms and data analytics to improve the automation of business processes.
4. Process optimization. This involves modeling the interactions between business processes (in different time frames) to optimize the whole outcome for the business.
5. Business evolution. This tends to involve modeling markets, regulation, technology evolution, and the evolution of business dependencies and creativity.
6. Social intelligence. The focus here is in better understanding and responding to human behavior, involving staff, customers, agents, suppliers and stakeholders. Such activity is likely to depend on social media data and may involve streaming data.

The mainstream AI projects follow an iterative pattern of the form:

Data Acquisition and Assembly —> Data Analysis —> Results Review

When results emerge there will typically be a software development of some kind to implementation some change to one or more business processes.

It would be misleading to try to characterize all AI activities as following this pattern.

Ancillary Technology

BI and AI software are only part of the picture. The other part of the picture is data. It is often the case that data warehouse technology forms a central part of the BI and AI resource – requiring the associated data skills.

A great deal of data transfer may be involved in whatever architecture is implemented, involving messaging software, such as Kafka, and possibly also data lake technology. For more details read Data in Motion and Data at Rest.

In recent times the analysis of data in real-time has become a feature of corporate (see Streaming Analytics Platforms for details of this).

Graph databases and BI applications that feed from them are a relatively recent development – there has always been graph data applications but they were often niche. With the evolution of semantic technology this has begun to move towards the mainstream. The applications are mainly BI -related.

The data that is exploited by BI is mostly derived from operational systems in one way or another. Thus BI naturally falls under the umbrella of Data Governance. A number of related areas may be worth studying in this respect, particularly Data Discovery and Data Catalogues, Data Quality, Data as an Asset and Master Data Management (MDM).

All departments within an organization need to care about the BI capability available to them. Most will already be using BI in some way. If they are unaware of the full range of possibilities of BI  they should be encouraged to discover what is possible beyond what they currently employ.

Similarly if they are provided with the capabilities they need, but the service levels prove to be poor, they will rightly complain, even if they are not aware that the fault lies with the BI Platform.

Concern for BI is concern for the necessary feedback loop that is a vital part of all business activity, and hence not just middle management, but also C level executives need to care that this aspect of the business is optimized. A well-implemented BI Platform is required for that to be the case.

AI projects are more specifically likely to involve C-level sponsorship and interest and will usually have a specific target they attempt to achieve.

IT Departments are likely to have a section focused specifically on AI and BI systems.

There are multiple trends both in both AI and BI — as would be likely, given their broad areas of application.

AI

The big innovation that launched all those “Big Data” initiatives was Machine Learning, the use of a number of algorithms on very large data pools which were capable of discovering new knowledge in many areas. This naturally led to the creation of AI platforms. This growth of such Big Data activity has not stopped and often gives rise to innovations in technology or business processes. It is a focus of research in academia which often gives rise to spin-offs and new start-up companies.

In particular areas we can identify:

1. What is termed “democratization”—development of AI techniques that can be used by those who are not data scientists or even business analysts.
2. Creative AI — the use of AI for what are deemed creative activities (such as writing articles, conducting conversations, making videos, and generating knowledge).
3. Educative AI — this can best be described as AI which explains itself (rather than being a black-box) and hence helps to educate it users as well as assist them.

It is also worth noting that AI technology often bleeds into BI over time—when some useful idea is discovered that can be simply automated.

BI

We can identify:

1. As in AI so in BI in respect of democratization. This aligns with another BI trend: Improving users skills in the sense of data literacy, is a growing part of the BI world.
2. Real-time/streaming data capabilities have been added in recent times and this trend is set to continue—as the world inevitably moves enthusiastically towards the impossible goal of the “real-time processing of everything.”
3. The Mobile BI market is substantial and growing. Many BI tools have provided specific implementations for mobile devices. This trend is set to continue.
4. The fastest growing BI segment is currently cloud analytics. The trend to the cloud is a strong in this area as in most other areas of IT.
5. Data discovery — providing greater ability to users to explore and use data in their BI tools.
6. BI integration at the back end — providing capabilities to securely and reliably source external data as well as internal data.
7. At the front-end there is a clear trend to what could be called personalization, where the user is given an improved ability to control their interface. This combines with another similar trend: BI self-service.
8. Data Storytelling is an emerging idea that aims to improve the effective communication of what is discovered by BI tools.
9. BI self-service has extended even towards the point of being able to assemble user portals by taking and integrating elements form multiple BI tools. The leads towards the idea of a company maintaining a BI catalog of capabilities.
10. Natural language processing (an AI focus area) is gradually being added to BI tools. This is just one aspect of semantic technology.
11. Data semantics has matured in recent times leading to the creation of effective data ontologies (semantic structures) which have yet to make an appear in the BI World but will likely do so in time.
12. Embedded BI (where the BI capability is fully embedded in applications) has been a trend for quite a while and continues to grow. This leads into the idea (also a trend) of Agile BI development on the developer side.
13. Data governance should long ago have been an aspect of BI. It has become prominent in recent times partly due to data regulation.

The global BI market is estimated to be worth about $25 billion, roughly a third of which is believed to be mobile BI.

An estimated 67% of the global workforce has access to BI tools and the average organization uses an estimated 3.8 different BI tools.

Historically there has always been BI vendors that dominated the market. It used to be Cognos and Business Objects in the early days, supersedes by Tableau and Qlik, and we have recently witnessed Microsoft’s Power BI become dominant. According to TrustRadius, Power BI has a 36% market share followed by Tableau (20%) and Qlik Sense (11%).