Monte Carlo launches first agents for data observability
Though the vendor already provides monitoring and troubleshooting capabilities, generative AI-powered agents for each aim to make instantaneous tasks that previously took days.
Monte Carlo has entered the agentic AI era, launching Observability Agents on Thursday to help enterprises ensure data quality.
Observability Agents are a set of generative AI (GenAI)-powered agents that take on aspects of data observability, which is the process of monitoring and understanding the health of data.
Monte Carlo's first generally available Observability Agent is a monitor that recommends rules and thresholds to maintain data quality. A troubleshooting agent that investigates the root causes of data quality issues and provides recommended steps for resolving them is planned for general availability before July.
Monte Carlo already provides root cause analysis capabilities, and users can already establish rules and thresholds. However, the agents turn what previously were lengthy processes into nearly instantaneous ones, according to Lior Gavish, the vendor's co-founder and chief technology officer.
Using AI to address data quality and improve efficiency are two top areas of investment in data management, said Stewart Bond, an analyst at IDC, which conducted a survey last summer on the subject.
"Monte Carlo is meeting both of these trends with the introduction of the monitoring and root cause detection agents," he said. "Creating agents that sit on top of the data observability capabilities in Monte Carlo is a natural evolution for the product portfolio."
Based in San Francisco, Monte Carlo is a data observability specialist whose tools enable users to monitor data throughout its lifecycle to ensure its quality for analytics and AI-based analysis.
Observability with agents
Many enterprises have increased their investments in AI development since OpenAI's November 2022 launch of ChatGPT marked a significant improvement in generative AI technology.
Simultaneously, many data management and analytics vendors have added tools that aid AI development -- given that data provides the intelligence in AI -- as well as developed GenAI capabilities to simplify the use of their complex platforms.
Over the past year, those generative AI capabilities have evolved to include agents, which are applications that not only respond to prompts from users but also are capable of acting autonomously to surface insights and take on specific repetitive processes.
Data management and analytics vendors that have launched agents to aid customers that include Tableau and Google Cloud. Now, Monte Carlo is following suit, providing users with agents meant to improve what they already are doing with the vendor's data observability tools, according to Gavish.
What the agents do is accelerate tasks that are extremely time-consuming for data teams and make it instantaneous.
Lior Gavish Co-founder and chief technology officer, Monte Carlo
"What the agents do is accelerate tasks that are extremely time-consuming for data teams and make it instantaneous," he said.
The monitoring agent uses generative AI to identify patterns and relationships across a data set that humans might otherwise miss, automatically generates monitors and thresholds that can be easily understood and recommends them to data teams. To date, the recommendations made by the monitoring agent have a 60% acceptance rate, according to Monte Carlo.
The troubleshooting agent discovers data and AI quality anomalies, investigates the cause across hundreds of different hypotheses, such as factual inaccuracies from the data source and coding errors, and provides explanations for the anomalies. The result is an 80% reduction in the time it takes to resolve incidents, according to Monte Carlo.
Like Bond, Kevin Petrie, an analyst at BARC U.S., noted that his firm's research shows data quality as the top potential application for agentic AI, outpacing data integration and documentation. As a result, Monte Carlo is addressing users' needs with its new agents.
"This announcement is right on target in terms of market demand," Petrie said.
The biggest benefits of the agents are likely efficiency and greater productivity, he continued. In addition, Monte Carlo is wise to enable the agents to deliver recommendations that require human approval before they are implemented.
"They save time with root cause analysis and identify issues that humans might otherwise have overlooked," Petrie said. "The caveat, of course, is that these agents still require expert human oversight and inspection. You don't want to auto-configure an invalid alerting threshold or misdiagnose a data quality issue."
TechTarget
Bond similarly cited efficiency as a significant benefit for the agents, along with comprehensiveness beyond what humans may think about researching. However, alert fatigue could be an unintended consequence.
"Will the frequency and volume of alerts result in people ignoring those alerts?" Bond said. "It's great that issues can be identified earlier. Perhaps because of the additional guidance these agents are providing for resolution, the risk of fatigue may be mitigated by this incremental value the agents are providing."
Regarding Monte Carlo's decision to make the monitoring and troubleshooting agents the first Observability Agents, a combination of user feedback and the vendor's own data about what customers most need from the vendor provided the impetus, according to Gavish.
"It was an easy decision to apply AI and agentic workflows to those areas," he said.
From a competitive perspective, Monte Carlo is one of the first to deliver agents that target data observability, Bond continued. However, he said he expects more to follow in the near future.
One that has already delivered such capabilities is Boomi, whoseDataHub Agent with AWS Bedrock addresses data observability, Petrie said.
Looking forward
While the monitoring and troubleshooting agents are Monte Carlo's first Observability Agents, they won't be the last, according to Gavish. Agents for collecting and managing metadata, as well as performance optimization details, are possibilities.
"For the past five or six years, we've been collecting information [about customers], and now we're putting the agents on top of that," Gavish said.
Beyond agentic AI, as data teams evolve into data and AI teams, adding observability for unstructured data and AI tools are other ways Monte Carlo might expand, he added.
Bond noted that many data management vendors are prioritizing agentic AI. As a result, Monte Carlo is wise to focus on developing more Observability Agents.
"As with many other vendors in the data intelligence and integration software market, what's next is more agents applying AI and automation to data and supporting data preparation for AI," Bond said.
Eric Avidon is a senior news writer for Informa TechTarget and a journalist with more than 25 years of experience. He covers analytics and data management.
