Acceldata unveils AI-powered data observability tools
The vendor's update includes features such as error detection with root cause analysis and schema reconciliation that use AI to automate finding and fixing data integrity problems.
Acceldata on Tuesday unveiled a series of AI-driven data observability features aimed at enabling customers to more easily and effectively fix data integrity problems, including smart matching to compare data sets and rule creation capabilities to speed reconciliation.
With enterprises increasing their reliance on real-time analytics and AI-driven tools, data observability that addresses the quality of the data used to inform real-time and AI-powered applications is gaining importance, according to Kevin Petrie, an analyst at BARC U.S.
"Companies have long struggled with data quality issues," he said.
Frequent problems include data that is incomplete, inconsistent, stale or inaccurate, Petrie continued. Those problems, in turn, can result in inaccurate analytical outputs, project delays and cost overruns.
AI initiatives raise the stakes because without trustworthy inputs, AI and machine learning models can automate bad decisions and misguided actions that damage the business. Data quality observability tools help data teams identify, assess, and resolve or prevent issues.
Kevin PetrieAnalyst, BARC U.S.
"AI initiatives raise the stakes because without trustworthy inputs, AI and machine learning models can automate bad decisions and misguided actions that damage the business," Petrie said. "Data quality observability tools help data teams identify, assess, and resolve or prevent issues."
Based in Campbell, Calif., Acceldata is one of a group of data observability specialists that also includes Monte Carlo, Metaplane and Soda Data.
The vendor raised $50 million in funding in February 2023 plus another $10 million the following October. Since then, it has acquired Bewgle to add data observability capabilities for generative AI and large language model pipelines, developed new automated remediation capabilities, and unveiled conversational AI capabilities that make it easier and more efficient for users to monitor their data.
New capabilities
Data observability is simply monitoring data as it moves through pipelines to ensure its quality.
It used to be a relatively straightforward process. For decades, all of an organization's data was kept in on-premises databases overseen by teams of data experts. When analysts and other data consumers wanted to use data to inform decisions, they submitted requests to the data team, who then developed reports, dashboards and other data products, often on a weekly, monthly or quarterly basis.
Now, however, data observability is a daunting endeavor for humans.
The advent of the cloud changed the paradigm by enabling far greater storage capacity than on-premises databases, as well as the ability to collect and ingest massive amounts of data, which continues to grow exponentially. In addition, the rear-facing weekly, monthly and quarterly reports that were once sufficient no longer meet the needs of enterprises.
Real-time analytics is a necessity, and AI is becoming one too. Both require the immediate operationalization of data as it is collected, which, in conjunction with data volume reaching a petabyte scale for individual organizations, makes checking for data quality untenable for even large teams of humans.
Data observability vendors whose platforms automate monitoring data -- and alert engineers and other experts when problems arise -- make it possible for enterprises to maintain the data quality needed to ensure the accuracy of real-time and AI applications.
In March 2023, Acceldata introduced automated incident detection and remediation capabilities along with intelligent alerting so that users can understand which alerts to prioritize.
The vendor's latest update advances those automated capabilities. Particularly noteworthy are AI-powered features aimed at making it easier and more efficient to discover and resolve data integrity problems, according to Petrie.
"Acceldata is improving its ability to find data problems, fix them and meet auditability requirements," he said.
Acceldata's update includes the following new capabilities:
Smart matching for complex data sets, which uses AI to automate column alignment across data sets to enable precise data comparisons and reduce the need for manual interventions.
Bulk reconciliation policy creation, a tool that automates rule creation to ensure consistent validation and remediation across multiple data sources and data formats.
Error detection with root cause analysis, which not only alerts users to data that needs remediation, but also provides AI-powered recommendations for resolving data quality problems.
Auditable processes, a feature that provides audit-ready documentation to maintain data integrity and accountability across data operations.
Table schema reconciliation to automatically adapt data to database schema changes, preserving consistency and accuracy during transformations.
Nested data structure to reconcile problems with JavaScript nested arrays, which are groupings of related data in JavaScript.
UPSERT handling, which enables users to manage Delta records, batch updates and multiload merges to maintain data integrity across large environments.
Flexible reconciliation across environments, a feature aimed at fixing data integrity problems within high-volume tables without requiring a complicated data infrastructure.
Chunk-based hash equality, which reduces data movement and maintains precision by reconciling high-volume data sets, breaking them down into smaller, more manageable chunks and comparing their tables through hashing.
Regarding the impetus for developing the new features that make up Acceldata's latest update, Ashwin Rajeeva, the vendor's co-founder and CTO, said a combination of customer feedback and market observations were determining factors.
For example, customers often mentioned problems scaling traditional reconciliation methods, particularly when migrating, integrating and transforming data, which led Acceldata to try to assist through automation. Meanwhile, as organizations' data deployments grow more complex, including hybrid environments that feature multiple clouds along with on-premises databases, being able to ensure data quality across environments is critical.
"Acceldata's AI-powered capabilities are the result of blending customer insights with our commitment to staying ahead of the data landscape's evolving demands," Rajeeva said.
Petrie, meanwhile, noted that the update helps Acceldata remain competitive with its primary peers. While the individual features add to the comprehensiveness of the vendor's platform, the platform itself now measures up well against those from other data observability specialists.
"This [update] makes it more competitive with data quality tools such as Monte Carlo," he said.
Monte Carlo, which has raised $236 million in funding, including $135 million in 2022, has raised more than twice the $105.6 million raised by Acceldata. In addition, Monte Carlo and Metaplane both recently released updates. Monte Carlo added new generative AI tooling, and Metaplane unveiled a native application for Snowflake that tightens the bond between the two.
Acceldata is keeping pace by providing a broad array of data observability tools, according to Petrie. While Monte Carlo's latest release featured two new tools, and Metaplane's latest was centered on its relationship with Snowflake, Acceldata's update includes nearly 10 new features.
"Acceldata differentiates itself with the depth of its pipeline observability features," Petrie said. "More than other data quality tools, Acceldata helps data teams measure and optimize the performance of data pipelines and their underlying infrastructure."
Looking ahead
Following the introduction of its latest update, Acceldata's product development focus will continue to be building AI-powered tools to automate data observability tasks such as anomaly detection and root cause analysis, according to Rajeeva.
"This will help organizations identify and resolve data issues more efficiently," he said.
In addition, Acceldata plans to improve its user interface as well as add and enhance integrations with data platforms such as Databricks and Snowflake to simplify connectivity between its platform and where customers keep their data.
"By focusing on these areas, Acceldata is committed to providing enterprises with the tools they need to ensure data integrity, compliance and operational excellence in the age of AI," Rajeeva said.
Eric Avidon is a senior news writer for TechTarget Editorial and a journalist with more than 25 years of experience. He covers analytics and data management.
