Open source AI agent frameworks add enterprise ops polish

Kagent: An agentic catalog for CNCF projects

Kagent, released March 17, consists of agents, a declarative API and a controller based on Microsoft's AutoGen version 0.4, and pre-integrated plugins based on the open source Model Context Protocol. At launch, plugins were available for Argo, Helm, Istio, Kubernetes, Prometheus and a cloud-native expert knowledge base.

The idea for Kagent came from repeated consulting engagements by expert developers at Solo.io to help customers set up complex cloud-native infrastructure, according to Lin Sun, director of open source at Solo.io.

Lin Sun

"Sometimes when our customer-facing engineers talk to customers, they don't have the answer, and that means we have to pull in an additional expert," Sun said. "And we started thinking about, 'What if we can clone some of these top developers so they could potentially help our support engineering team more effectively, without impact [to] their daily development work, because we want them focused on building new features?'"

The project's goal is a donation to the Cloud Native Computing Foundation, Sun said. There, it would serve as a framework for other CNCF projects to contribute their own developer clones as virtual agents to help end users with tricky configuration, troubleshooting, observability and network security tasks.

Kagent is designed for Kubernetes and cloud-native projects, but users could run it outside of Kubernetes if they choose, Sun said.

"We believe Kubernetes' container runtime and declarative nature allow users to run Kagent easily anywhere Kubernetes is supported," she said. "In Kubernetes, you can simply configure autoscaling and pod replicas and retries to increase an agent's resiliency."

If Kagent takes off, it could help provide a standard set of connectors between multiple large language models (LLMs) autonomously capable of using tools in multiple clouds using existing standards from Kubernetes to Model Context Protocol, according to Paul Nicholson, an analyst at IDC.

"Anyone who can provide frameworks and consistency to help businesses accelerate those deployments and then solve that complexity of getting to the LLMs, no matter where they exist ... that is a good thing," Nicholson said.

Dapr Agents extend Dapr Workflow

Meanwhile, one CNCF project launched its own AI agent framework this month that has significant overlaps with AutoGen. On March 12, the Dapr microservices orchestration project launched Dapr Agents, based on an extension of its Dapr Workflow tool first created by a security researcher at Microsoft.

When Roberto Rodriguez, now a Dapr Agents core maintainer and no longer working for Microsoft, created the Dapr Workflow extension, AutoGen was in version 0.2. This version was constrained within a single environment and not meant for use on production infrastructure.

Rodriguez needed something more robust to help with threat hunting. At the time, AutoGen focused on a "group chat" approach to AI agent coordination rather than a series of ordered steps. Rodriguez wanted more control over the order in which the system executed agentic AI tasks. He started to write his own Python package for this, but discovered Dapr Workflow and realized that it already contained a framework to orchestrate microservices applications, including built-in observability and resilience features such as retries if a component failed.

"As I was building [my own] system, to me it was pretty clear that it was not going to scale ... [while] Dapr's battle-tested and proven to scale," Rodriguez said during a weekly Dapr community meeting March 19. "A lot of the well-known [AI agent frameworks] out there, to be honest, they're really great to do demos. They're amazing when they show you everything happening on one screen, but if you want to decouple all of that and also be able to swap components of the agentic system with different cloud providers and different databases, it is not that easy."

Kubernetes is an obvious place [to run agents] because it is declarative [and] offers automation and observability. ... But there are a lot of agent-specific lifecycle management ... interactions that do not exist in any other application development runtime framework. So Dapr is also a natural choice.

Dapr Workflow, released in 2024, added microservices orchestration to the distributed application framework, which was also originally created at Microsoft in 2019 and donated to the CNCF in 2021. Dapr connects microservices applications to other elements of the IT environment, such as caching and data pipeline tools, through a set of standard APIs. Workflow adds the ability to configure how and in what order microservices workflows are executed.

According to Rodriguez, Dapr contained most of the foundation for an AI agent framework, but needed a few tweaks to better integrate with AI agents, such as outputs in certain schemas optimized as LLM prompts, awareness of the presence of LLMs and their tool choices. His initial extension of Dapr Workflow, called Floki, has now become Dapr Agents with updates from Dapr's other maintainers.

Dapr Agents introduces a new agent object to Dapr Workflow, designed to be a long-running process that can assess which tools to use for different tasks and evaluate the outputs it receives from other agents rather than requiring workflow steps to be hardcoded upfront.

Dapr Agents expands Floki's focus on ordered chains of tasks to support group chat AI agent workflow patterns, fan-in and fan-out, or a mix of multiple patterns. These patterns can be managed as a single logical workflow within Dapr, including its original infrastructure and services APIs, resilience features, and built-in metrics and traces for monitoring.

"Kubernetes is an obvious place [to run agents] because it is declarative [and] offers automation and observability and all that stuff," Deng said. "But there are a lot of agent-specific lifecycle management, security monitoring and LLM interactions that do not exist in any other application development runtime framework. So Dapr is also a natural choice."

Mark Fussell

As an early-stage project built around an emerging technology, Dapr Agents has limitations, according to Dapr co-creator Mark Fussell, founder and CEO at Diagrid, a startup vendor founded in 2021 that hosts Kubernetes and Dapr API managed services.

"This is still in an early preview stage, and only in an SDK inside Python. ... Within a few months, the goal is to have all the SDKs and the runtime have an agent syntax across Java, .NET, Python, JavaScript and Go," Fussell said in an interview with Informa TechTarget. "Another powerful thing is that we're ... going to be multilanguage like we've done with Workflow."

AutoGen, SKPF and Dapr: Ships in the night?

Since Rodriguez began work on Floki, Microsoft integrated a previously separate project, the Semantic Kernel Process Framework (SKPF), into AutoGen 0.4, released in January. This added scalability, resilience, workflow orchestration and integration with enterprise applications that are more similar to Dapr Agents.

In fact, early demos of SKPF included integrations with open source Dapr along with the Microsoft Orleans distributed systems framework. Microsoft continued to tout Dapr compatibility in a blog post about the roadmap for AutoGen 0.4 this month. There is a GitHub issue open in Microsoft's AutoGen repository about coordinating efforts with the Dapr community, but it shows little activity.

"The Dapr community is not engaged with Microsoft on SKPF," Fussell said in an email to Informa TechTarget last week. "SKPF could, like Dapr Agents, build on Dapr APIs ... but it also looks like they have built their own process management for state."

Microsoft did not comment as of press time.

Beth Pariseau, senior news writer for Informa TechTarget, is an award-winning veteran of IT journalism covering DevOps. Have a tip? Email her or reach out @PariseauTT.