Automating Business Process Discovery

I can remember the many hours of heartache trying to work out what actually happened when I was trying to document a business process for a customer. Invariably, Joe tells you one thing, Sarah tells you something completely different and “Old Bob” tells you that for “X” we do it that way. Trying to untangle these conflicting issues to understand the business rules and the process paths is hard work but vital.

So it was with a lot of enthusiasm (as well as scepticism) that I agreed to be briefed by Teemu Lehto, QPR Software’s Vice President of Business Development about their new product ProcessAnalyzer v2, described as an Automated Business Process Discovery (ABPD) tool.

So what is Automated Business Process Discovery? ABPD is an emerging field that discovers the business processes based on examining the electronic footprints the users leave in the IT assets supporting the process. This allows the business process to be automatically discovered and documented in near real time. Approaching the business process “bottoms-up” from the detailed facts of instances of the process provides a detailed depiction of the business process, complete with all the nuances of the process, complete with detailed statistical information on how often different variations of the process are executed, how long it takes, what data conditions give rise to process variations, and what variations there are between different users or groups.

Why should you be interested? Well, Jim Sinur, Gartner Research VP, listed the topic as number 2 in their The Top Ten BPM Technologies in March 2010. Gartner have seen ABPD as a means to overcoming the following issues in the way we currently model processes. It is a costly and time-consuming process that is vulnerable to human interpretation, where we come across inadequate business knowledge and a lack of objective validation techniques. Pretty hard on those of us who have made a living out of working through this morass to get to the heart of the matter—the actual process. To give Gartner their due they do recognise that ABPD will be an aid to making the process quicker and less costly and I would certainly agree with this. Forrester have also introduced ABPD as a new evaluation criterion in their Forrester Wave: Business Process Management Suites, Q3 2010. So the two senior industry analyst houses see this as an important area. From Bloor’s viewpoint we also see that ABPD is an important technique to aid in the discovery of processes which are already in operation.

So what about the new entrant, QPR Software’s ProcessAnalyzer v2? Lehto explained that QPR had started work on the product back in 2009 and, during last year (2010), they put a first release out to some of their customers to test the waters. These initial tests proved that the functionality was right, so QPR then redeveloped to make it easier to use. ProcessAnalyzer can extract the necessary information it requires from a number of different data sources including SQL databases, ODBC, web services (XML) and OLAP. The first striking thing about ProcessAnalyser is the user interface—Microsoft Excel—thereby making it very familiar and easy to use for a large majority of users. Now we all know that currently Excel has a limit of 1million rows, so I asked Lehto what happened if you were dealing with more than 1million rows of data. QPR then used SQL Server to hold the data.

Figure 1: QPR Software ProcessAnalyzer capability summary (Source: QPR Software)

In the demonstration I was shown, an Excel spreadsheet with patient number, process step, organisation unit, start time and end time was used to generate a process flow diagram with swim lanes based on the organisation unit data and each process step contained the number of instances as well as the total and average duration of the step. When moving to the flow view, incoming and outgoing processes are shown with green and red arrows respectively. The flow arrow that is most used is thickened and on each flow the volume of the flow and its duration is shown. The process visualization capability allows the user to drill down to individual cases. The chart shows the average duration of each activity within the swim lane (organisation unit for the demonstration). The resource load analysis report shows the resources used on a timeline. Simultaneous cases are shown on top of each other. The colours classify the cases based on their duration. The case comparison report is a graph describing the total duration of cases. As in the previous graph, the width of a box describes the average duration of the activity and the colours classify the cases based on their duration.

Lehto outlined what QPR Software saw as an example project. The first step would be to analyse the project specification and then set the objectives and define the scope. The next step would involve the identification of relevant IT systems and specifying analysis data. The data then has to be gathered before the analysis is carried out. The various outputs from the tool are then presented and discussed. The project is completed by fine tuning the analysis based on feedback.

QPR ProcessAnalyzer Desktop version is priced very attractively at €780/month/user and can be purchased directly from the QPR website. A person familiar with log files and Microsoft Excel learns the usage with the provided tutorial and can make the analysis himself. There is also a  Starter Pack, which includes 1 month usage for 1 person plus 2 days of training and consulting service.

Who else offers ABPD solutions? Here are 4 vendors and their products that I have identified:

• With the acquisition of ARIS, Software AG offer the ARIS Process Performance Manager. ARIS has had a very close relationship with SAP and is used by the latter in documenting the processes within the R/3 package. This provides similar capabilities to what I have described for QPR.
• Fujitsu, with their latest release of Interstage BPM V11, have incorporated an automated discovery engine, originally developed by their services division for client projects. This tracks all the ad hoc subtasks, and can make suggestions on improving the process based on how the process was actually executed including the user-created subtasks, rather than how it was originally designed. It uses Flash to view how many times each path was traversed in the process. The product uses system logs as input.
• StereoLOGIC, based in Toronto, Canada, offer StereoLOGIC Discovery Analyst. It is niche company only offering this tool.. The discovered business processes are published in standard BPMN and XML formats and can be exported into Microsoft Visio and IBM WebSphere Business Modeler.
• Pallas Athena of Apeldoorn, The Netherlands, offer ReflectOne, which uses event logs from applications to extract the information to produce process models.

ABPD is a very useful component to establish exactly what actually happens in a business process so that bottlenecks and unusual exceptions can be identified and improved. By automating the steps, time and effort, as well as human error, are reduced. If you add the ability to simulate and pass through to a BPMS engine to control in the future, we are talking about serious time saving.

Organizations have been looking for this for a long time; QPR has ventured into the market in a much bigger way than anyone else in terms of what you can do with QPR ProcessAnalyzer and how easy to use the product is.