Measuring AI becomes key pursuit as enterprises seek ROI

Grammarly creates AI value assessment approach

Suha Can, CISO at Grammarly, a San Francisco company that provides an AI writing tool, echoed that assessment. Can, speaking on AI ROI measurement at the conference, said the enterprise adoption cycle over the last two years has transitioned from scrambling to adopt AI to grappling with its value.

"Right now, the problem we are all facing is to figure out whether all this AI we have all manically deployed is actually useful for our businesses," he said.

The truth is that you've been in the mud for the past year, working hard to find all those [enterprise AI] benefits.

Can said the gap in understanding AI's benefits has created a "pilot purgatory," where AI deployments remain stuck while organizations ponder whether they're worth expanding.

Against this backdrop, Grammarly earlier this year created an AI assessment playbook to guide its deployments. Can said the assessment focuses on compliance and security, quality, employee experience and impact -- that is, how does a given AI technology affect the organization's KPIs?

Those evaluation factors are sequential and interlocking. For example, an AI tool might nail compliance and perform as advertised to meet the quality check, but prove difficult for employees to use, Can said. Another one might fail the compliance test out of the gate. An organization that adopted a similar process could drop a tool at that point, before spending more effort on the technology.

"What we did find in our assessment is that going through ... tools in sequence will actually save your teams time," he added.

Tools that pass the first three assessment stages can move on to the impact phase. Here, for example, Grammarly tested an AI tool with its customer support team to gauge the tool's influence on customer satisfaction scores, Can said. The test found that a control group of employees who didn't use the tool was three times more likely to have a negative customer satisfaction score than a test group of employees who used the tool, he noted. This result, Can said, gave him confidence that the tool would provide value.

Kroger AI use case helps employees, customers

Kroger, a grocery retailer based in Cincinnati, also assesses how AI deployments drive important business metrics. A key theme for data science and AI at the company is making things easier for both store workers and customers. For example, Kroger's retail data science subsidiary, named 84.51 degrees after Cincinnati's longitudinal location, built an algorithm that has reduced the distance an associate must walk in a grocery store to find and pack items for customer pickup orders, said Todd James, chief data and technology officer at the subsidiary. Fewer steps have shortened the lead time for pickup orders from three hours to less than two hours across Kroger's highest-volume stores.

"It takes less time to pick the order, therefore the lead time doesn't need to be as much," said James, who spoke at the conference on delivering value through AI.

The data science company designed the algorithm to extend across Kroger's 2,500-plus stores. It was also built to be applied in other use cases. In one such example, the algorithm is being repurposed to help shrink the travel time for trucks delivering grocery items from distribution centers to stores.

The Kroger unit's AI and data science technology foundation is built on Google Cloud, Microsoft Azure and Databricks' data analytics platform. It's also working on a switchboard technology that will automate the selection of the optimal large language model for a given use case, taking the cost and capability of multiple models into account, according to James.

John Moore is a writer for TechTarget Editorial covering the CIO role, economic trends and the IT services industry.