Growing AI energy use, tech workloads constrain IT

ORLANDO, Fla. -- The availability and cost of energy is becoming a constraint on technology use -- one IT managers must plan to address as enterprise AI applications increase computing needs in organizations.

The electricity required to power expanding IT workloads, especially considering rising AI energy use, was an undercurrent at Gartner IT Symposium/Xpo 2024, which ends today. The annual event also featured discussions of emerging technologies, such as agentic AI, and an IT future that could see multiple developments, such as quantum computing and neuromorphic computing, arrive in the same time span -- developments that are expected to drive the need to orchestrate deeper layers of technology resources.

But conference speakers pointed to energy consumption as a here-and-now issue that's unlikely to be resolved anytime soon.

Jensen Huang, president and CEO at Nvidia, said progress against Moore's law -- which holds that CPU performance effectively doubles every two years with minimal added cost -- distantly lags the amount of data organizations process these days. The slowing rate of performance increases began a decade ago and is now "to the point where it is non-existent," Huang said during his keynote session. The result is spiraling compute costs and energy use.

"If your compute demand continues to grow exponentially, but general-purpose computing performance does not, then you should be experiencing computing cost inflation and, very importantly, computing energy inflation," Huang said. "In fact, energy consumption is going up exponentially."

Huang, whose company is a leading supplier of GPUs, said accelerated computing addresses energy efficiency. "You have to use acceleration wherever you can," he said.

Daryl Plummer, an analyst at Gartner, pointed to a different approach in his discussion of the consulting and market research firm's strategic predictions for 2025. He said Fortune 500 companies, through 2027, will shift $500 billion from energy Opex to microgrids in a bid to counter "chronic energy risks and AI demand." Microgrids provide independent energy systems to meet the needs of a company or group of companies.

"What we are saying, essentially, is we don't generate enough energy and what we do generate can't be delivered over our fragile distribution infrastructure, which hasn't been modernized enough," Plummer said.

Enterprises that use a lot of electricity are starting to "generate their own energy" and share that power with close corporate partners through microgrids, he added. "This is something, if you are a big consumer of energy, to think about today."

In fact, energy consumption is going up exponentially.

Plummer said cloud data center operators are already buying "mini nuclear power plants" and noted Microsoft's planned move to purchase electricity from the Three Mile Island nuclear plant in Pennsylvania. Nuclear power is one source of energy that it and other hyperscalers are looking to tap into as the power demands of generative AI continue to expand.

Enterprises might have to pursue energy risk mitigation strategies for several years before they see relief. Frank Buytendijk, an analyst at Gartner, predicted it will take five to 10 years to deliver on the "promise of energy efficient computing."

During that time horizon, several still-embryonic technologies could emerge, according to Buytendijk, who spoke on the future of computing at the symposium.

"Quantum; neuromorphic; and optical, or photonic, computing are significantly more energy efficient and, in a number of cases, very fast as well," Buytendijk said. "It's going to take some time, but we will get there. The idea of energy efficient computing is just a little bit further out."

John Moore is a writer for TechTarget Editorial covering the CIO role, economic trends and the IT services industry.