ktsdesign - stock.adobe.com
How pattern matching in machine learning powers AI
Pattern matching may sound like a simple idea, but it's being used to create some highly advanced AI tools, powering everything from image recognition to natural language applications.
Much of the power of machine learning rests in its ability to detect patterns. Much of the basis of this power is the ability of machine learning algorithms to be trained on example data such that, when future data is presented, the trained model can recognize that pattern for a particular application. If you can train a system on a pattern, then you can detect that pattern in the future. Indeed, pattern matching in machine learning -- and its counterpart in anomaly detection -- is what makes many applications of AI work, from image recognition to conversational applications.
As you can imagine, there are a wide range of use cases for AI-enabled pattern and anomaly detection systems. In particular, pattern recognition -- one of the seven core patterns of AI applications -- is being applied to fraud detection and analysis, finding outliers and anomalies in big stacks of data; recommendation systems, providing deep insight into large pools of data; and other applications that depend on identification of patterns through training.
Fraud detection and risk analysis
One of the challenges with existing fraud detection systems is that they are primarily rules-based, using predefined notions of what constitutes fraudulent or suspicious behavior. The problem is that humans are particularly creative at skirting rules and finding ways to fool systems. Companies looking to reduce fraud, suspicious behavior or other risk are finding solutions in machine learning systems that can either be trained to recognize patterns of fraudulent behavior or, conversely, find outliers and anomalies to learned acceptable behavior.
Financial systems, especially banking and credit card processing institutions, are early adopters in using machine learning to enable real-time identification of potentially fraudulent transactions. AI-based systems are able to handle millions of transactions per minute and use trained models to make millisecond decisions as to whether a particular transaction is legitimate. These models can identify which purchases don't fit usual spending patterns or look at interactions between paying parties to decide if something should be flagged for further inspection.
Cybersecurity firms are also finding significant value in the application of machine learning-based pattern and anomaly systems to bolster their capabilities. Rather than depending on signature-based systems, which are primarily oriented toward responding to attacks that have already been reported and analyzed, machine learning-based systems are able to detect anomalous system behavior and block those behaviors from causing problems to the systems or networks.
These AI-based systems are able to adapt to continuously changing threats and can more easily handle new and unseen attacks. The pattern and anomaly systems can also help to improve overall security by categorizing attacks and improving spam and phishing detection. Rather than requiring users to manually flag suspicious messages, these systems can automatically detect messages that don't fit the usual pattern and quarantine them for future inspection or automatic deletion. These intelligent systems can also autonomously monitor software systems and automatically apply software patches when certain patterns are discovered.
Uncovering insights in data
Machine learning-based pattern recognition systems are also being applied to extract greater value from existing data. Machines can look at data to find insights, patterns and groupings and use the power of AI systems to find patterns and anomalies humans aren't always able to see. This has broad applicability to both back-office and front-office operations and systems. Whereas, before, data visualization was the primary way in which users could extract value from large data sets, machine learning is now being used to find the groupings, clusters and outliers that might indicate some deeper connection or insight.
In one interesting example, through machine learning pattern analysis, Walmart discovered consumers buy strawberry Pop-Tarts before hurricanes. Using unsupervised learning approaches, Walmart identified the pattern of products that customers usually buy when stocking up ahead of time for hurricanes. In addition to the usual batteries, tarps and bottled water, it discovered that the rate of purchase of strawberry Pop-Tarts also increased. No doubt, Walmart and other retailers are using the power of machine learning to find equally unexpected, high-value insights from their data.
Automatically correcting errors
Pattern matching in machine learning can also be used to automatically detect and correct errors. Data is rarely clean and often incomplete. AI systems can spot routine mistakes or errors and make adjustments as needed, fixing data, typos and process issues. Machines can learn what normal patterns and behavior look like, quickly spot and identify errors, automatically fix issues on its own and provide feedback if needed.
For example, algorithms can detect outliers in medical prescription behavior, flag these records in real time and send a notification to healthcare providers when the prescription contains mistakes. Other automated error correction systems are assisting with document-oriented processes, fixing mistakes made by users when entering data into forms by detecting when data such as names are placed into the wrong fields or when other information is incomplete or inappropriately entered.
Similarly, AI-based systems are able to automatically augment data by using patterns learned from previous data collection and integration activities. Using unstructured learning, these systems can find and group information that might be relevant, connecting all the data sources together. In this way, a request for some piece of data might also retrieve additional, related information, even if not explicitly requested by the query. This enables the system to fill in the gaps when information is missing from the original source, correct errors and resolve inconsistencies.
Industry applications of pattern matching systems
In addition to the applications above, there are many use cases for AI systems that implement pattern matching in machine learning capabilities. One use case gaining steam is the application of AI for HR and staffing. AI systems are being tasked to find the best match between job candidates and open positions. While traditional HR systems are dependent on humans to make the connection or use rules-based matching systems, increasingly, HR applications are making use of machine learning to learn what characteristics of employees make the best hires. The systems learn from these patterns of good hires to identify which candidates should float to the surface of the resume pile, resulting in more optimal matches.
Since the human is eliminated in this situation, AI systems can be used to screen candidates and select the best person, while reducing the risk of bias and discrimination. Machine learning systems can sort through thousands of potential candidates and reach out in a personalized way to start a conversation. The systems can even augment the data in the job applicant's resume with information it gleans from additional online sources, providing additional value.
In the back office, companies are applying pattern recognition systems to detect transactions that run afoul of company rules and regulations. AI startup AppZen uses machine learning to automatically check all invoices and receipts against expense reports and purchase orders. Any items that don't match acceptable transactional patterns are sent for human review, while the rest are expedited through the process. Occupational fraud, on average, costs a company 5% of its revenues each year, with the annual median loss at $140,000, and over 20% of companies reported losses of $1 million or more.
The key to solving this problem is to put processes and controls in place that automatically audit, monitor, and accept or reject transactions that don't fit a recognized pattern. AI-based systems are definitely helping in this way, and we'll increasingly see them being used by more organizations as a result.
