process mining

What is process mining?

Process mining is a technique that interprets logs from enterprise applications -- like customer resource management (CRM) and enterprise resource planning (ERP) platforms -- to identify the steps in common business processes, detect variations and prioritize areas of improvement. It uses a statistical approach and various techniques, including machine learning, data mining and predictive analytics, to identify patterns in the underlying data.

The term was first coined in the 1990s by Prof. Wil van der Aalst at the Eindhoven University of Technology, whose students have launched several different startups to help automate and refine the process. The first generation of tools only analyzed one process at a time, such as quote-to-cash, customer onboarding, accounts payable, order management or procurement. Recent innovations let businesses assess the interconnection between multiple processes for more holistic optimization.

Process mining is only one way to view enterprise activities. Complementary manual approaches include business process analysis and business process mapping. Task mining and task capture tools record how users interact with software. Collectively, these techniques are referred to as process intelligence. Increasingly, vendors are expanding their tools to support these different capabilities within a common platform.

Process mining and process intelligence tools are often combined with low-code/no-code development tools and robotic process automation (RPA) to speed business process development and business reengineering efforts.

Why process mining is essential in the enterprise

Process mining is important because it automatically analyzes data from underlying systems and provides insight into opportunities for improvement. Examples include the following:

• Process conformance analysis that compares existing processes to desired workflows to improve auditing or compliance.
• Benchmarking capabilities that compare process differences across regions, teams or business units. Simulation capabilities help teams assess the impact of proposed changes.
• Process mining insights that automate the deployment of new processes across cloud services, ERP systems and internal business apps.
• Identifying opportunities to use automation tools such as RPA to automate process parts.
• Decomposing existing processes into tasks that use generative AI tools.

Types of process mining

The four broad categories of process mining capabilities include the following:

Process discovery

Process discovery analyzes ERP and CRM event logs and maps them as process workflow diagrams. The output consists of various formats common in business process analysis, including activity diagrams, Business Process Modeling Notation (BPMN) diagrams, Petri nets to represent places and transitions, state diagrams, event-driven process chain diagrams and social network graphs.

Conformance checking

Conformance checking compares an existing process with the ideal process based on governance, risk management and compliance constraints. For example, it can detect deviations that might result in fraud or the absence of controls that might be required by regulators.

Performance analysis

Performance analysis analyzes data about cycle times, waiting times, costs, processing time and steps to identify opportunities for improvement. It can also highlight outliers across regions or teams as possible models for others.

Task mining

Task mining, sometimes called task capture, extracts data from an employee's interaction with business systems using machine vision and natural language processing. Task capture data is sometimes combined with process mining to improve and contextualize manual tasks involved in business processes. Task mining can also be used to find ways to improve employee experience or as a starting point for RPA development.

Benefits of process mining

Some of the top benefits of process mining include the following:

• Reduces manual effort required to understand common business processes.
• Provides a view of how existing processes run in practice.
• Real-time analysis alerts managers about changes in process behavior for investigation.
• Prioritizes automation opportunities.
• Automates risk management and compliance tracking.
• Identifies bottlenecks in customer experience.

Drawbacks of process mining

There are also many drawbacks and challenges with process mining that enterprises should consider. These include the following:

• Process analysis can be constrained to log data produced by enterprise systems.
• It can miss manual aspects and human workarounds in broken processes.
• Poor quality data can confuse and mislead process analysis efforts.
• Setting up process mining is complex and might require new staff or skills.
• Additional time and resources might be required to collect, clean and analyze data to produce good results.
• Employees might push back against suggested areas of improvement.

Process mining use cases

Process mining use cases include the following:

• Discovering process bottlenecks to increase efficiency.
• Streamlining the rollout of process improvement projects by interpreting existing processes and verifying results.
• Helping data science teams understand the end-to-end process in complex data preparation workflows to work faster and discover opportunities for new tools or processes.
• Allowing auditors to identify and rectify compliance issues more efficiently, such as systemic lapses in regulatory compliance requirements.
• Making it easier to discover and prioritize automation opportunities.
• Identifying the root cause of delays within certain processes.
• Comparing existing processes against best practices to identify and fix performance gaps.

The future of process mining

The field of process mining has been around for nearly 30 years. Now, academics, vendors and enterprises are exploring how improvements in cloud infrastructure, data processing techniques and AI can help evolve the field to further improve business process management (BPM) efforts. Some examples of recent process mining innovations include the following:

• Granular process analysis techniques could help break down complex processes into smaller steps. This could allow enterprises to apply innovations in generative AI to automate more aspects of business operations with minimal disruption.
• Decision mining can help identify the chain of decisions involved in a process to identify dependencies in routing a case. This can help teams understand the various types of tacit knowledge and expertise required for a particular chain of decisions.
• Process observability is a newer technique that applies data observability techniques with BPM-specific large language models to contextualize process data. It takes advantage of specialized LLMs called large process models that interpret unstructured data across enterprise systems as part of business process analysis. The generative AI interface can help business analysts ask questions about current processes and ways to improve processes.
• Object-centric process mining is another newer technique that transforms events log data into an object-centric event log to relate events to multiple objects, such as sales orders, items and production orders. This format can represent the relationships between multiple objects. It helps view all operational activities from various perspectives from a single source of truth.
• Digital twins of an organization could take advantage of improvements in process mining to construct a live model of the organization. This could improve understanding between different stakeholders, optimize the interplay of multiple processes and run what-if scenarios.