Recent reports from KPMG, McKinsey & Co. and Goldman Sachs indicate that generative AI is immature and carries a high price tag, with no clear path to ROI.
Listen to this article. This audio was generated by AI.
Enterprises trying to tap generative AI for higher productivity and revenue growth are battling an expensive, immature technology with an elusive return on investment.
That's according to recent reports from KPMG, McKinsey & Co. and Goldman Sachs that reflect the GenAI experiences of many large organizations. A critical challenge with the technology is generating a return that justifies the high cost of deploying GenAI infrastructure at scale.
A recent AI survey from management consultancy McKinsey showed that only 11% of companies polled have deployed GenAI broadly, according to Aamer Baig, a senior partner at the firm, who spoke to attendees of the MIT Sloan CIO Symposium in May. Also, only 15% of the respondents reported earnings improvements from GenAI.
Justifying cost is critical because AI infrastructure is expensive. For example, Nvidia's previous-generation H100 AI GPU costs roughly $30,000, and the number needed can range from a couple of hundred to thousands, depending on the AI model and its size.
The technology is very new; it barely works, and it's very costly.
Anshul ChaturvediManaging director, World Wide Technology
"The technology is very new; it barely works, and it's very costly," said Anshul Chaturvedi, managing director at IT services provider World Wide Technology (WWT).
A large language model today is a black box that's difficult to make accurate and ensure consistent responses, said Rob Mason, CTO of Applause. The company tests the results of AI in software from the user's perspective.
"Inside of the companies that make GenAI, people are studying what they built to figure out what it will do, as opposed to what they want it to do," Mason said.
A Fortune 100 company that hired AI application specialist Trustbit, acquired this year by IT consultancy Timetoact Group, had to deploy three different models internally to ensure at least 95% response accuracy, Trustbit technical consultant Rinat Abdullin said.
If the three models provided the same answer, it was considered accurate. Less than that, and a human would decide which answer was correct. Accuracy was critical because customers could hold the business liable for the wrong answer.
"Companies don't always need 100% accuracy, but they want to be certain that when the model provides an answer, if it's not sure, then they will know about that," Abdullin said.
The shift to revenue-generating AI
Some enterprises are strategically repositioning their use of GenAI from employee productivity to revenue generation, according to KPMG's second-quarter survey of 100 U.S.-based C-suite and business executives. The respondents were from organizations with annual revenue of $1 billion or more.
In the first quarter, using GenAI to grow employee productivity was the top ROI metric for 51% of respondents, the KPMG survey found. In the second quarter, revenue generation became No. 1 at 52%, while improving productivity fell to No. 3 at 40%.
"Leaders are beginning to view GenAI investment and adoption as table stakes," said Steve Chase, KPMG's vice chair of AI and digital innovation. "Now, they're focused on how to translate those investments into a competitive advantage."
Chaturvedi, who has noticed a similar shift among WWT customers, said he believes it's more out of confusion over how to get the most value out of GenAI. He recommends starting with employee productivity and customer service applications before tackling pricier revenue-generating use cases.
"Using it for employee productivity is to have a nice little safe playground where if you mess up, it doesn't really matter," Chaturvedi said.
Model immaturity
The hype around GenAI's potential to transform business hides the immaturity of the technology and the need for a lot more research. Daron Acemoglu, an economics professor at MIT, said he predicts GenAI won't be ready to change business operations dramatically for at least 10 years.
"Many tasks that humans currently perform, for example in the areas of transportation, manufacturing, mining, etc., are multifaceted and require real-world interaction, which AI won't be able to materially improve anytime soon," Acemoglu said in a Goldman Sachs GenAI report.
The current architecture used for LLMs will have to change to mimic the many types of cognitive processes of humans, their ability to process sensory inputs or their reasoning capabilities, Acemoglu said.
"Large language models (LLMs) today have proven more impressive than many people would have predicted, but a big leap of faith is still required to believe that the architecture of predicting the next word in a sentence will achieve capabilities as smart as HAL 9000 in 2001: A Space Odyssey," he said in the report.
Antone Gonsalves is an editor at large for TechTarget Editorial, reporting on industry trends critical to enterprise tech buyers. He has worked in tech journalism for 25 years and is based in San Francisco. Have a news tip? Please drop him an email.
