Sysdig Sage early adopters kick the tires on CNAPP AI agents
AI agents in Sysdig Sage add more sophisticated multi-step reasoning than is available with generic LLMs, but it's meant to assist humans, not replace them.
Security engineers with early access to a new AI agent-based feature of Sysdig's CNAPP said it's helped them sift through alerts and better customize policies but keeps humans in control in important ways.
Sysdig Sage, in controlled availability with 15 customers of the cloud-native application protection platform (CNAPP), is built on AI agents, an architecture that adds a layer of sophistication beyond a large language model-based chatbot. AI agents are extensions of LLMs that perform specialized tasks and break multi-step queries down into their component parts. Together, they can coordinate fetching data to answer complex or open-ended questions from users.
"This is just a pervasive issue in the realm of security, that you have a lot of information and not enough context … that you feel confident about," said Cat Schwan, security engineer and team lead at Apree Health, a primary care and tech company in Seattle that's used Sysdig Sage in production over the last month. "You can just ask Sage any question, even just like, 'Why am I getting this alert? Why is this important? What should I do?' It helps give analysts of all levels [of experience] a springboard to start taking action."
The importance of a human in the loop
An emphasis on human action assisted by AI analytics also attracted another Sysdig user, e-commerce company BigCommerce, based in Austin, Texas. BigCommerce also signed on to participate in the controlled availability program for Sage.
"This is what AI should have been doing for us all along," wrote Dan Holden, CISO at BigCommerce, in a public comment on LinkedIn Aug. 1. "It's not about magical detection capability (our math is better than their math BS), it's about making the human's response capability better and faster."
So far, BigCommerce has focused on using Sage to evaluate and fine-tune custom security policy rules, according to Jordan Bodily, team lead for infrastructure security engineering at the company.
"We're not looking to cut out every event from Sysdig, but what we are interested in is obtaining custom events and amending criticality [ratings] for our needs," Bodily said. "In our case, we've been interacting with Sage to answer several questions: Why was there a spike in event X? [Does] Sage have recommendations to reduce the quantity of an event, such as creating exclusions for what we deem normal?"
Previously, this kind of fine-tuning required duplicating an out-of-the-box rule as a template for experimentation, then digging through documentation for the open source Falco policy engine Sysdig uses, then testing changes to the duplicate rule for correct syntax, Bodily said.
"For me, that's not a big deal because I'm a heavy user of Sysdig and I'm comfortable with it," he said. "However, there are certainly those that don't interact with it daily and this is where Sage saves us time and frustration."
Engineers with less experience in application security will be able to ask Sage questions to better learn how to use Sysdig, without requiring internal experts to spend time answering those questions, according to both Bodily and Schwan.
"It really helps bridge the gap in knowledge between security and DevOps for us," Schwan said. "It's helped us figure out exactly what we need to say to our [infrastructure-as-code] team, like, 'Hey, we are seeing this event -- is this something that we should expect to be seeing?'"
Neither early user has called on Sysdig Sage in a live threat response scenario yet, although both said they believe it will be helpful in that case.
"I do expect that we will naturally shift [into] its ability to help during triage," Bodily said. "We've used it to get more information on some events to help us answer the question of, 'Is this normal activity?' Or 'Have we seen this before?'"
Addressing LLM, AI agent risks
The development of Sysdig Sage comes nearly two years into generative AI hype, which over the last six months has given way to some disillusionment about whether LLMs can fulfill their early promise in a cost-effective manner, while containing risks such as data privacy, security, possible copyright violations and inaccurate results.
Sysdig officials said the LLM that backs Sage has been trained and tested repeatedly on application security data to ensure accurate results, but that the company doesn't mine customer data for that training.
Sysdig also regularly tests AI agents working together to ensure there are no hallucinations caused by AI-to-AI interactions, said co-founder and CTO Loris Degioanni in an email to TechTarget Editorial.
"To date, we haven't recorded any diminished quality or accuracy," Degioanni said.
In some AI agent architectures, multiple LLMs might feed each other data, which can raise the risk of hallucinations, but Sysdig Sage appears to be taking a different, less risky approach, said Andy Thurai, an analyst at Constellation Research.
"Ultimately, everybody will build personalized virtual agents, which can lead to a situation of a bot talking to another bot trained on a different context, and they will try to game each other, thinking they're dealing with a human, in an infinite loop," he said. "In this particular case, [Sysdig] has one LLM trained with domain-specific information and uses agents as extensions … that take a specific task and use complex reasoning and multi-shot prompting."
Sysdig Sage's answers have sometimes been high-level and generic, Schwan said, but it has shown more accuracy and given more useful answers than generic chatbots she's seen.
"Is it 1,000% perfect? No," Schwan said. "I don't think I've seen an LLM that is yet, though."
Using AI still warrants caution, but it has become a necessity with increasingly complex cloud infrastructures that generate more events and alerts than humans can process. Given that necessity, Sysdig has plenty of competitors, including integrations between Cisco and Splunk. But IT organizations with primarily cloud-native apps have favored Sysdig's CNAPP for the speed of its threat detection and response while capturing comprehensive log data.
"Response times are critical in cloud threat detection and response, and the impact of [an incident] is more severe when the time is longer between the detection and response and remediation," said Melinda Marks, an analyst at TechTarget's Enterprise Strategy Group. "Sysdig Sage is a good application of generative AI to save analyst time for faster response because it can use data from eBPF and contextual information to more quickly process and analyze data than humans."
In addition to a refresh of its eBPF-based data collector this week that boosts its performance, Sysdig added Cloud Identity Insights. This feature links user account information to events to pinpoint which entities within a cloud infrastructure made certain changes, such as escalating privileges, that might indicate a threat.
"As a security team, we sometimes struggle because our entire environment is [based on HashiCorp] Terraform, so we'll get an alert that a service account is doing something, [but] someone could impersonate a service account in GCP [Google Cloud Platform]," Schwan said. "I could see that being a powerful tool to [see] who initiated … an exact command within a container."
Beth Pariseau, senior news writer for TechTarget Editorial, is an award-winning veteran of IT journalism covering DevOps. Have a tip? Email her or reach out @PariseauTT.
