Sysdig Sage early adopters kick the tires on CNAPP AI agents

The importance of a human in the loop

An emphasis on human action assisted by AI analytics also attracted another Sysdig user, e-commerce company BigCommerce, based in Austin, Texas. BigCommerce also signed on to participate in the controlled availability program for Sage.

"This is what AI should have been doing for us all along," wrote Dan Holden, CISO at BigCommerce, in a public comment on LinkedIn on Aug. 1. "It's not about magical detection capability (our math is better than their math BS), it's about making the human's response capability better and faster."

So far, BigCommerce has focused on using Sage to evaluate and fine-tune custom security policy rules, according to Jordan Bodily, team lead for infrastructure security engineering at the company.

Jordan Bodily

"We're not looking to cut out every event from Sysdig. But what we are interested in is obtaining custom events and amending criticality [ratings] for our needs," Bodily said. "In our case, we've been interacting with Sage to answer several questions: Why was there a spike in Event X? [Does] Sage have recommendations to reduce the quantity of an event, such as creating exclusions for what we deem normal?"

Previously, this kind of fine-tuning required duplicating an out-of-the-box rule as a template for experimentation, then digging through documentation for the open source Falco policy engine Sysdig uses, then testing changes to the duplicate rule for correct syntax, Bodily said.

"For me, that's not a big deal because I'm a heavy user of Sysdig and I'm comfortable with it," he said. "However, there are certainly those that don't interact with it daily, and this is where Sage saves us time and frustration."

Engineers with less experience in application security will be able to ask Sage questions to better learn how to use Sysdig without requiring internal experts to answer those questions, according to both Bodily and Schwan.

Cat Schwan

"It really helps bridge the gap in knowledge between security and DevOps for us," Schwan said. "It's helped us figure out exactly what we need to say to our [infrastructure-as-code] team, like, 'Hey, we are seeing this event. Is this something that we should expect to be seeing?'"

Neither early user has called on Sysdig Sage in a live threat response scenario yet. But both said they believe it will be helpful in that case.

"I do expect that we will naturally shift [into] its ability to help during triage," Bodily said. "We've used it to get more information on some events to help us answer the question of, 'Is this normal activity?' or, 'Have we seen this before?'"

Addressing LLM, AI agent risks

The development of Sysdig Sage comes nearly two years into generative AI hype. Over the last six months, that hype has given way to some disillusionment about whether LLMs can fulfill their early promise in a cost-effective manner while containing risks such as data privacy, security, possible copyright violations and inaccurate results.

Sysdig officials said the LLM that backs Sage has been trained and tested repeatedly on application security data to ensure accurate results but that the company doesn't mine customer data for that training.

Sysdig also regularly tests AI agents working together to ensure there are no hallucinations caused by AI-to-AI interactions, co-founder and CTO Loris Degioanni said in an email to TechTarget Editorial.

"To date, we haven't recorded any diminished quality or accuracy," Degioanni said.

In some AI agent architectures, multiple LLMs might feed each other data, which can raise the risk of hallucinations. But Sysdig Sage appears to be taking a different, less risky approach, said Andy Thurai, an analyst at Constellation Research.

"Ultimately, everybody will build personalized virtual agents, which can lead to a situation of a bot talking to another bot trained on a different context, and they will try to game each other, thinking they're dealing with a human, in an infinite loop," he said. "In this particular case, [Sysdig] has one LLM trained with domain-specific information and uses agents as extensions … that take a specific task and use complex reasoning and multi-shot prompting."

It's not about magical detection capability … it's about making the human's response capability better and faster.

Sysdig Sage's answers have sometimes been high-level and generic, Schwan said. But it has shown more accuracy and given more useful answers than generic chatbots she's seen.

"Is it 1,000% perfect? No," Schwan said. "I don't think I've seen an LLM that is yet, though."

Using AI still warrants caution, but it has become a necessity with increasingly complex cloud infrastructures that generate more events and alerts than humans can process. Given that necessity, Sysdig has plenty of competitors, including integrations between Cisco and Splunk. But IT organizations with primarily cloud-native apps have favored Sysdig's CNAPP for the speed of its threat detection and response while capturing comprehensive log data.

"Response times are critical in cloud threat detection and response, and the impact of [an incident] is more severe when the time is longer between the detection and response and remediation," said Melinda Marks, an analyst at TechTarget's Enterprise Strategy Group. "Sysdig Sage is a good application of generative AI to save analyst time for faster response because it can use data from eBPF and contextual information to more quickly process and analyze data than humans."

In addition to a refresh of its eBPF-based data collector this week that boosts its performance, Sysdig added Cloud Identity Insights. This feature links user account information to events to pinpoint which entities within a cloud infrastructure made certain changes, such as escalating privileges, that might indicate a threat.

"As a security team, we sometimes struggle because our entire environment is [based on HashiCorp] Terraform. So we'll get an alert that a service account is doing something, [but] someone could impersonate a service account in [Google Cloud Platform]," Schwan said. "I could see that being a powerful tool to [see] who initiated … an exact command within a container."

Beth Pariseau, senior news writer for TechTarget Editorial, is an award-winning veteran of IT journalism covering DevOps. Have a tip? Email her or reach out @PariseauTT.