putilov_denis - stock.adobe.com
IT ops pros grapple with glut of AI assistants
After nearly two years of hype, it's still early for enterprise GenAI. That said, skepticism -- and outright pessimism -- have begun to emerge about AI assistants for IT automation.
AI assistants have established a stronger foothold in app development than in IT automation so far. Amid dawning skepticism about the technology in general, some IT ops pros have begun to question what the long-term value will be.
It's not unusual for a new technology to hit what Gartner calls the "trough of disillusionment" after its initial introduction, when real-world results replace theoretical beliefs about how effective it will be. But generative AI assistants for code completion and test automation such as GitHub Copilot have gained a significant foothold in developer workflows. In IT ops, by contrast, the sheer number of AI assistants introduced over the last year for IT infrastructure automation, the lingering qualms about AI risks and the unresolved question of how they all fit into a cohesive workflow have been trickier stumbling blocks.
In the same week last month, for example, two IT ops vendors introduced new AI assistants: Platform management vendor Pulumi unveiled Pulumi Copilot, while federated search and data management vendor Cribl previewed Cribl Copilot. The previous week, Splunk added to an existing roster of AI assistants with Configuration Assistant for IT Service Intelligence, AI Assistant in Observability Cloud and AI Assistant in Security.
When they become generally available, these tools will enter a market teeming with dozens of similar offerings for IT infrastructure automation from every major cloud provider as well as most other large enterprise IT vendors, such as IBM and Red Hat. Many come with lofty claims about eliminating toil for IT ops teams and turbocharging automation.
Users can augment generative AI with supplemental technologies such as retrieval-augmented generation (RAG) to refine results, and large language models (LLMs) are improving as they mature. But for now, some IT ops pros would like vendors to tone down their rhetoric about this early generation of AI assistants.
"I've experimented with a few copilots -- I think it's a natural progression of tool evolution, especially if these copilots have domain-specific knowledge," said Steve Koelpin, lead Splunk engineer for a Fortune 1,000 company in the Midwest. "But [vendors] have to be careful about setting expectations. It has huge potential, but it's not there yet. People will see it as a magic bullet and then be disappointed when it's not."
Jason Bloomberg
Even where RAG and other data management techniques can improve results, vendors developing AI assistants also face high model training and development costs, said Jason Bloomberg, an analyst at Intellyx.
"We're absolutely in a generative AI bubble," he said. "It's unclear that anybody is going to actually build something that has a rational business model [where] they can charge enough to cover the costs."
Another industry analyst said he's skeptical that AI assistants will ever make a major productivity impact for IT ops pros.
Andi Mann
"IT ops is not predictable, it is not full of known knowns, it does not use a predefined language -- so ML and GenAI is definitionally going to be of limited use in IT ops compared to, say, dev/test," said Andi Mann, global CTO and founder of Sageable, a tech advisory and consulting firm in Boulder, Colo. "For me, that just raises the question of why most IT ops copilots are being built at all, except for show. I am singularly unimpressed by AI from any vendor that only helps junior staff for a few weeks until they build their skills."
AI agents to the rescue?
Not everyone is quite so pessimistic about the long-term outlook for AI assistants in IT automation. Atlassian, Google, Microsoft and some early adopter platform engineers have begun to invest in AI agents -- orchestrated groups of chatbots backed by LLMs that perform multistep workflows.
For one IT pro experimenting with AI assistants, there is potential worth exploring in generative AI, even if downsides remain.
"We're a very lean and small shop, and we need our senior engineers focused on bigger things," said Kevin Keeler, vice president of DevOps, QA and architecture at A+E Networks. "We have managed services where we don't have a whole lot of control over the people that are coming in, so getting them ramped up quickly does add some value. ... Where we're really focused is on infrastructure operations and maintenance, and how we can decrease response times for end users without having a human directly involved."
A+E Networks is considering AI assistants and chatbots from Pulumi, Ansible, Moveworks and other IT ops vendors, as well as experimenting with virtual agents deployed as Slack chatbots using a tool from startup Kubiya.
"Kubiya allows us to create these virtual agents through our own code," Keeler said. "We're working on automating our major incident management processes -- when our customer services team wants to open incidents, they can talk to the virtual agent, and it'll hit PagerDuty, it'll create tickets, post on status pages, create the bridges for us to jump on those calls right away and get the teams assembling a lot faster."
However, this is scaled back from the original vision at the broadcasting company, which was to have AI-driven virtual agents perform IT automation directly, Keeler said.
"When people get into [AI automation], they get these grandiose ideas of what the bots are going to do, like they're going to replace a senior DevOps engineer," he said. "When we first started, we were telling the bots, 'Hey, kick this process off, do this, do this,' and it didn't work well because we ended up having to train bots a heck of a lot more ... on our business context."
Now, his team's focus is evaluating AI assistants as a potentially more convenient interface to access IT services for end users and to bring various information sources together during troubleshooting. But the "heavy lifting" of IT automation will be done with existing IT service management tools, Keeler said.
"If you're a user, going to a web portal [to make a request] might take too long for you," he said. "But if you go to a bot, it'll ask you a couple of questions, it'll know which remote location you're in, it'll know a lot of things about you already and start kicking off the process a little bit faster, and you don't have to key in everything."
However, it's been hard to quantify the value of replacing web portals with chatbots so far, especially given the amount of model training required, Keeler said. It remains to be seen how effectively AI assistants can learn about a complex IT environment and how it corresponds to a unique business lexicon -- but that training process has been getting shorter, he said.
"The models seem to be getting better and better," Keeler said. "And we're getting used to it -- our understanding of AI has also gotten better."
Pulumi mulls AI orchestration
To Intellyx's Bloomberg, the best path forward for AI assistants in IT ops is deeper integration between generative AI and other data analytics tools, rather than broader integration between AI assistants.
"That sort of reminds me of Furbys from years ago, where you could take multiple Furbys and they would talk to each other," he said. "I don't know if it worked very well then, and it's sort of unclear if it's going to work very well now. I don't know if you can have one AI talk to another and get good results."
Instead, Bloomberg cites Pulumi Copilot, released in public beta June 12, as an example of an AI assistant strategy with more potential upside for IT pros. Pulumi Copilot builds in Pulumi AI, which was shipped as an experimental feature in 2023 and trained by early adopters since that launch. Pulumi Copilot is grounded in Pulumi Insights, including a knowledge graph about cloud infrastructure based on the vendor's infrastructure-as-code integrations.
This means Pulumi Copilot can answer IT ops teams' questions about their organization's cloud usage, team activities and documentation, as well as offer platform engineers guidance on how to optimize cloud costs, compliance and security. It can also answer questions about why a specific cloud update failed and guide users to relevant parts of the infrastructure for troubleshooting -- and automatically generate the infrastructure code to take action.
Pulumi's advantage comes from its initial focus on building imperative language interfaces in programming languages such as Java and TypeScript for declarative infrastructure-as-code frameworks, Bloomberg said. In this context, natural language used by AI assistants can function as just another kind of imperative language.
"You can describe the state of infrastructure you desire using any language you want, and then the technology is able to support that," he said. "AI, then, is just a supporting player for [Pulumi]."
But Joe Duffy, Pulumi's CEO, didn't rule out a future foray into virtual agents to address the industry's emerging copilot sprawl.
Pulumi's knowledge graph, which it brands as Cloud Supergraph, is extensible by partners. But the vendor will also likely build its own virtual agents to tie together AI-driven automation for its platform engineering products, Pulumi Deployments and Environments, Secrets and Configuration (ESC), according to Duffy, in an interview with TechTarget Editorial last month.
"We have access to your cloud account, with controls and governance and proper security, but ESC allows us to actually interact with your cloud resources," Duffy said. "So as you're asking questions, it's combining data from your cloud and logs and metrics, in addition to all of your Pulumi Supergraph data, so it understands that entire context."
Beth Pariseau, senior news writer for TechTarget Editorial, is an award-winning veteran of IT journalism covering DevOps. Have a tip? Email her or reach out @PariseauTT.
