Success with automation and AI requires a high 'RQ'

Toronto moments

Automation and AI technologies are on a spectrum from more deterministic, where A always leads to B, to more probabilistic, where A could lead to B but could also lead to C or to D.

And these probabilistic systems create a new wrinkle for companies: No matter how swanky the user interface or how cutting-edge the technology, probabilistic systems can produce incorrect -- and even illogical -- results that can erode the trust humans have in the machine's abilities.

Gownder pointed to IBM Watson as an example. During its Jeopardy! debut in 2011, Watson answered a final question about U.S. cities with "Toronto," causing the audience to gasp. When researchers did a post-mortem, it became clear that even Watson doubted the response. Using probabilistic judgement, the machine determined that Toronto had only a 30% chance of being correct, but it was the best answer it could come up with at the time.

These "Toronto moments," as Forrester now refers to them, "teach us something about the intersection between human beings and AI and the trust that is part of this," Gownder said.

The more probabilistic a system is, the more human intervention it might need. But designing systems and processes that strike a balance between trust and intervention will be a challenging step for companies. That's where Forrester believes RQ will come in handy.

What is RQ?

The robotics quotient is a self-assessment that "measures the ability of individuals and organizations to learn and adapt to and collaborate with automated entities," Gownder said. It's composed of 39 characteristics that Forrester regards as a collection of automation and AI best practices.

J.P. Gownder

The higher the score, the more prepared a company is to tackle the new challenges that come with automation and AI technologies. But RQ doesn't just measure readiness, according to Gownder. It also enables CIOs to "identify gaps or areas where you need to prioritize resources before you make a big bet on automation and AI," he said.

The 39 characteristics fall into one of three categories -- people, leadership and organizational structure. People, for example, are measured across different dimensions -- such as facilitation, which considers how effective an employee might be at communicating with an automated entity, and perception, which includes things like basic digital literacy and "constructive ambition," or an eagerness to learn.

For leaders, the RQ highlights vision, adaptability, the ability to inspire trust and influence. The final category refers to IT employees and beyond; CIOs will need to influence the C-suite and even the board of directors to secure the budget, buy-in and support that automation and AI tools can demand. "The CIO is no longer a benign dictator who has all the power," Gownder said. "This is the creation of an ecosystem across business units with lots of participation from the workers themselves."

Organizational structures will also need to adapt. Automation and AI may require new titles such as bot manager, new training and mentoring opportunities for humans and machines alike, new processes that encourage human-machine team creation, and new metrics. "After all, we can have all the good intentions, and the well-educated employees and the leaders who are on board," Gownder said, "but if we do not create structures, processes and budgets -- the b word -- we're going to have a hard time getting this through."

Don't forget about trust

The categories of people, leadership and the organization are then measured against one final category -- trust. Gownder called trust "a multiplier in this model." Automation and AI technologies exist on a spectrum from transparent to opaque, and where the technology falls on that spectrum will influence employee trust.

"If you're implementing something that is very transparent, that is very deterministic, your employees will bring a high level of inherent trust to the machine. They're used to these sorts of systems," Gownder said. "If you're using probabilistic systems, where the machine is often uncertain of its results, then you're going to have a higher burden of RQ investment."

Forrester's model breaks down the complexity of trust by providing a numeric value for how deterministic the technology is, how transparent the technology is and how much change the technology could have on the workplace.

The changes that automation and AI will have on the workplace could be a sensitive area for leaders, especially as automation and AI instigate changes in the workforce. "As you might imagine, when employees are losing their jobs as part of a deployment of automation, you magnify the mistrust among remaining employees," Gownder said. "It raises the bar for the change management."

But the efforts could be worthwhile. As repetitive tasks become automated, job satisfaction generally goes up, Gownder said. And although AI remains in its early stages, it is poised to transform how companies operate and interact with customers.

Whether companies choose Forrester's RQ method or not, Gownder argued that an organizational competency in AI and automation is needed.

"If you want to be successful in creating a mixed workforce that incorporates digital workers, human workers, lots of automated processes, lots of probabilities, lots of real-time data and AI, you're going to have to measure your people, your leaders, your organization and the inherent trust that is associated with technology," he said.