AI for decision-making shows promise, but worker trust an issue
Companies like Walmart are using AI for decision-making in business processes, but the discipline is new and workers are wary. Experts offer ways to build trust.
Enterprises are starting to infuse AI in business processes to improve -- or replace -- the decision-making traditionally done by humans. At the Re•Work Applied AI conference in San Francisco, executives from Walmart and other companies explored some of the challenges and benefits of using AI to automate decisions.
Sudha Viswanathan, a senior software engineer at Walmart Labs, said that companies like Walmart have been using various BI tools to improve decision-making for some time. "Today, things have been changing and it is faster paced," she said.
Executives need to find the right balance between the decision algorithms and the humans who work with them, according to the AI experts at the conference. The use of AI for decision-making will require building trust in the algorithms, especially in the early days: accuracy, explainability and a good UX are the chief ways to gain employee faith in the algorithms, they said. Enterprises also can improve the use of AI automation in business processes by coming up with good metrics to quantify the impact of AI decision-making tools on business operations.
Walmart tests AI for decision-making
AI capabilities like natural language processing and machine learning are enabling Walmart to make decisions faster and from more types of data, Viswanathan said. The AI capabilities allow Walmart to optimize more -- and different-- kinds of decisions than were possible with SQL queries of the past. But it is still early days for using AI in business process automation, Viswanathan stressed. More work is needed to develop new models.
Some of the early AI automation work at Walmart has focused on improving the retailer's online operations by using machine learning and other AI capabilities to optimize which advertisements and recommendations to show consumers. In the stores, Walmart is looking to use the machine learning models to improve decisions on how to stock shelves and price items.
Viswanathan said Walmart has also been working on developing a platform that solves common problems associated with decision-making workflows. The platform is being designed to able to support multiple machine learning models on a shared infrastructure, so that data scientists can focus on making sense of the data and applying insights rather than be distracted by the engineering side.
Keep humans in the loop on AI decision-making
As AI starts to get better, executives can think about replacing some human decision workflows with automation.
Volodymyr Kuleshov
Volodymyr Kuleshov, co-founder and CTO of Afresh, an AI supply chain for food, said enterprises should find ways to fully automate business process decisions when the AI can perform as well or better than humans. The automation will give the human workforce more time to do what they are good at.
Or, at least, that is the ideal with using AI for decision-making.
Kuleshov cautioned the audience that the state-of-the-art AI tools being deployed today to optimize decisions are still not there, either in terms of accuracy or explainability -- or both. And, even if the AI works well, accuracy can degrade for certain types of cases.
As for which business process decisions are good candidates for automating with AI, that will depend not only on the capabilities of the current AI tools, but also an enterprise's tolerance for risk and its ability to meet the rigors of training and testing the AI.
Danielle Deibler
"There are [processes] like making a flatbread that I would be perfectly happy to give to a machine," said Danielle Deibler, co-founder and CEO at Marvelous.ai, an augmented analytics platform. If the AI degrades, the business is out a few loaves of bread. Driving a plane or a car is another matter, she added.
Even low-risk decision workflows might need to be occasionally vetted by humans to identify any emerging problems or clean up the data going to the decision-making engine. "You might need a human early on in the collection of data or further down in the process of making the decision," Deibler said.
Accuracy vs. explainability in AI decision-making
The need for explainability grows as the accuracy of an algorithm goes down. Kuleshov said workers might not ask too much about explainability when an AI algorithm is 99.9% accurate. But a solution that is only 90% accurate might lead to a greater call for explainability. And accuracy can vary. "It may only be accurate on one subset of loads or might have lower accuracy for certain populations," Kuleshov said.
The variation in accuracy could be caused by a lack of training data for certain kinds of populations or transactions, Walmart's Viswanathan said. If that is the case, the data scientists need to find ways to fill in the gaps in the training data.
In the meantime, companies should focus on pushing out their AI recommendations to users along with an explanation of why a particular decision was made. This will require improving the application interface, so that workers can see the explanation within the application. But it will help build trust for the kinds of cases where the tool does well. The workers will have the context they need to decide whether they go with the decision recommended by the algorithm or look for more data.
Jessica Groopman
Explainability is often seen as a data science challenge, but it's also a broader UX challenge, stressed Jessica Groopman, industry analyst and founding partner at Kaleido Insights.
Develop better metrics for assessing AI decision-making
It's important to understand the impact on business results of using AI for decision-making.
Afresh's Kuleshov said companies should "come up with a well-defined set of metrics." This requires working with different departments to identify metrics that reflect the objectives the organization.
Marvelous.ai's Deibler agreed that keeping humans in the loop is important. They should be part of the process of disagreeing or agreeing with the results.
After that, companies must measure the difference between machine decisions and human decisions and use the results to improve the algorithm.
"This can help ease it into their workflow," she said, "so they can see this is something that augments rather than replaces them."
