6 trends that shaped data management, analytics in 2024

Emergence of agentic AI

While 2024 began with vendors finally delivering some of the generative AI-powered assistants they introduced in 2023, as the year progressed, mere assistants were becoming passe.

Assistants enable natural language interactions. But they're reactive. They require the user to engage with the natural language interface by asking questions. Some assistants are sophisticated enough to suggest follow-up questions that lead to deeper analysis, while others at least let the user ask follow-up questions without the assistant losing track of the thread.

Throughout the second half of 2024, agentic AI rose quickly to prominence.

The market is moving toward agentic AI and agentic analytics. The notion of building agents rather than relying on dashboards represents a paradigm shift in how organizations utilize data. Rather than data appearing in dashboards with data interpretation left to the viewer, agents can initiate actions based on the data.

Agentic AI tools are proactive, rather than reactive. Unlike the assistants that preceded them, they can act autonomously.

"The market is moving toward agentic AI and agentic analytics," said David Menninger, an analyst at ISG's Ventana Research, in September. "The notion of building agents rather than relying on dashboards represents a paradigm shift in how organizations utilize data. Rather than data appearing in dashboards with data interpretation left to the viewer, agents can initiate actions based on the data."

Similarly, Yasmeen Ahmad, Google Cloud's managing director of strategy and outbound product management for data, analytics and AI, said that developing AI agents is the next phase in the evolution of AI in the enterprise.

Google Cloud is among those now developing AI agents and providing tools for customers to do the same. Looker, the tech giant's primary analytics platform, is taking an agentic approach to generative AI.

So are Databricks, Qlik, Salesforce, Snowflake and ThoughtSpot, among numerous others.

For example, ThoughtSpot unveiled Spotter, a generative AI-powered agent that understands context and learns continuously to better understand a business's operations, in November. Tableau, a subsidiary of Salesforce, in September unveiled Tableau Einstein, an entirely new version of its BI platform with agentic AI at its core. And Databricks in June launched the Mosaic AI Agent Framework to enable agentic AI development.

"Rather than a human coming to data and asking for an insight or for data quality to be improved, a data agent is monitoring data, looking for anomalies, surfacing insights, suggesting semantic modeling metrics to monitor," Ahmad said in August. "We're moving from a reactive world to one where generative AI is proactive in supporting the data analytics lifecycle."

The evolving data platform

Not long ago, the main purpose of data management and analytics was to enable customers to prepare and analyze data.

Data management vendors such as Databricks and Snowflake provided cloud-based platforms for storing data, making it easy for customers to access their data for analysis. And vendors such as MicroStrategy and Qlik then provided the platforms for creating and viewing the reports and dashboards that led to insights and decisions.

Now, they're all becoming AI platforms.

Since OpenAI launched ChatGPT, the key focus for many data management and analytics vendors has been to develop environments that enable customers to build generative AI-powered models and applications.

"All the data platform vendors are making significant investments in providing AI/ML capabilities," Menninger said in November.

For example, Databricks acquired MosaicML for $1.3 billion in 2023 to provide a foundation for AI development. Over the past two years, it also developed integrations with large language model (LLM) developers such as Mistral AI and Anthropic, built its own LLM, and unveiled model quality and AI governance capabilities.

Similarly, archrival Snowflake has developed integrations with LLM providers, built its own LLM and, with Cortex AI, created an environment for customers to create AI tools. Cortex AI includes features such as AI observability and containerized storage to securely manage AI models.

In addition, tech giants AWS, Google Cloud and Microsoft, as well as specialists from Accenture to Zoho have made AI development a focal point of their product development.

However, with surging interest and investments in AI development still so new, the past two years were only the beginning of a new era for data management and analytics, according to Baris Gultekin, Snowflake's head of AI.

"When it comes to AI, I would say that everyone is at the beginning of the journey, and we are moving incredibly fast," he said in March. "Overall, the speed of development -- the pace of development -- is incredible."

AI potential to reality

About three months after the initial release of ChatGPT, the first generative AI moves from data management and analytics vendors started to trickle out.

Vendors such as Pyramid Analytics, ThoughtSpot and Sisense unveiled integrations with LLM vendors. With the integrations, they planned develop AI-powered assistants that enable customers to engage with their data using natural language rather than code.

Many others followed suit. They promised the types of tools that would make decision intelligence capabilities available to virtually any employee and take on onerous coding and documentation tasks that would improve the efficiency of experts.

Most, however, weren't even in preview when they were first introduced. Throughout 2023, there was a lot of promise but little actual delivery on those promises.

"We have seen many applications of generative AI in data intelligence software emerge in the past year, but mostly in the R&D lab and in demos that may or may not have had some smoke and mirrors involved," said Stewart Bond, an analyst at IDC, in April.

That started to change in late, 2023 when MicroStrategy was one of the first vendors to make generative AI-powered capabilities generally available. Many others followed suit throughout 2024, and the promise of generative AI began to be realized.

Data management specialists such as Informatica and Dremio launched AI-powered assistants, as did analytics specialists including Tableau and Qlik.

In addition, the data platform vendors -- Databricks and Snowflake -- and tech giants all made generative AI-powered capabilities generally available, along with features aimed at aiding customers as they develop their own generative AI applications.

One enterprise that has taken advantage of the AI development capabilities launched throughout 2024 is PowerSchool. The education technology vendor's platform is used by 17,000 school districts in the U.S and abroad.

Using technology from Snowflake and Microsoft, the company developed PowerBuddy, a generative AI-fueled assistant that enables users to interact with data using natural language.

"Any user can ask a natural language question," said Shivani Stumpf, PowerSchool's chief product and innovation officer, in November. "The notion is that everybody in education, whether you are a parent, a student, an administrator, a counselor, a principal, could use a buddy that provides them the information at their fingertips that's relevant to them."

The need for data quality

As AI development surges, for AI assistants, agents and other applications to be of any worth to an organization, the data used to train and inform the application needs to be of high quality.

As a result, data quality is taking on greater importance.

AI models and applications are trained by data. It's what gives them their intelligence. As a result, the models and applications are only as good as the data that feeds them.

If the data is inaccurate, inconsistent, incomplete or stale, the outputs delivered by the models and applications will reflect that and won't be trustworthy. Consequences could range from AI applications simply going unused because their outputs can't be trusted to financial loss, regulatory violations and significant embarrassment if the applications are used and decisions made based on bad outputs.

However, if the data is of high quality, if it's complete, consistent, accurate and fresh, model and application outputs will be more likely to be correct and trustworthy. Benefits include widespread use of data to inform decisions, which has been shown to spur growth, and greater efficiency, which also helps an organization's bottom line.

"[Data quality] is really important in a world where you're going from hand-created dashboards and reports to a world where you want AI to do [analysis] at scale," Saurabh Abhyankar, chief product officer at MicroStrategy, said in September. "But you can't scale unless you have a system in place so [the AI application] can be precise. ... To do that, the data quality simply has to be there."

Ensuring data quality, however, has long been a challenge for enterprises. The exponentially increasing volume of data now collected by enterprises, in conjunction with the growing complexity of data, make it even more of a challenge.

To come as close as possible to ensuring that only high-quality data is used to train AI tools, automated processes – such as vector search, retrieval-augmented generation and data observability -- overseen by humans who can intervene when necessary are required, according to Donald Farmer, founder and principal of TreeHive Strategy.

"[Data quality] puts an emphasis on processes that can be automated, identifying data-cleaning processes that require less expertise than before," he said in September. "That's where it's changing. We're trying to do things at much greater scale, and you just can't have a human in the loop at that scale. Whether the process can be audited is very important."

AI governance emerges

Although a heightened emphasis on data quality helps ensure that AI models and applications deliver appropriate outputs, organizations still need to ensure that they use the AI tools appropriately.

Just as bad data can lead to organizational harm, so can improper use of AI models and applications.

For decades, data was kept in on-premises databases and overseen by organizations' IT departments, with analysts having to submit requests for reports and dashboards to be developed. With limited access to data, there was no need for data governance.

Then came the era of self-service analytics with vendors such as Tableau and Qlik providing platforms that enabled non-experts to access and analyze data. Once access to data was no longer limited to trained experts, organizations needed data governance frameworks to enable business users to confidently work with data as well as set limits on their use of data to protect the organization from accidental harm.

Now, the same evolution is taking place with AI.

Machine learning, predictive analytics and other forms of traditional AI were long the domain of data science teams. Generative AI has changed that, enabling more widespread use of AI to inform decisions. Virtually any employee is now able to ask questions of an organization's data.

If not governed with proper practices and policies, improper use of AI tools could have some of the same consequences as poorly trained models and applications, including poor accuracy, biased outputs, noncompliance with regulations and financial loss, according to Kevin Petrie, an analyst at BARC U.S.

"If those risks are not properly controlled and mitigated, you can wind up with ... regulatory penalties or costs related to compliance, angry or alienated customers, and you can end up with operational processes that hit bottlenecks because the intended efficiency benefits of AI will not be delivered," he said in September.

With the rising interest in AI development still so new, many organizations have yet to develop AI governance frameworks, Petrie continued. However, numerous data management vendors, such as Alation and Collibra, are now adding AI governance tools that enable customers to better ensure the proper use of AI.

"Being a data-driven activity, the development of AI must be governed as rigorously as we govern our data, so it's a natural extension of data governance programs," Doug Henschen, an analyst at Constellation Research, said in October. "Organizations need help with these challenges, so it's good to see … vendors adding functionality to address AI-specific risks and emerging regulatory requirements."

Return of funding

Investors once loved data management and analytics vendors.

With analytics becoming a more popular means of making decisions throughout the 2010s, financiers saw opportunity. That increased in 2020, when the COVID-19 pandemic struck and real-time analysis became critical, with data providing the intelligence for businesses to make decisions that would enable them to survive amid constant change.

In September 2020, Snowflake set a record for the largest initial public stock offering in history by a technology vendor. Following that, in 2021 alone, 10 data management or analytics vendors executed funding rounds of $100 million or more, including Databricks, raising $1 billion in February of that year, and Confluent, raising $828 million.

Such vendors continued to attract capital in the early months of 2022, with Sigma Computing and Pyramid Analytics topping $100 million with their funding rounds.

But then a confluence of events, including Russia's invasion of Ukraine, repeated supply chain problems, rising interest rates and increasing fear of a recession led to economic uncertainty.

Tech stocks fell hard as the overall stock market declined. Layoffs increased. And venture capital funding dried up.

Only Databricks and a select few other data management and analytics vendors were able to raise funding in the second half of 2022 and throughout 2023.

In 2024, while funding isn't flowing into the data sector nearly as freely as it was a few years ago, it picked up with Cribl, Aerospike and Sigma each raising more than $100 million and others such as Ocient and Coalesce attracting about $50 million.

"Raising venture capital has been somewhat difficult in recent years given the overall economic environment," Matt Aslett, an analyst at ISG's Ventana Research, said in April. "But funding is still available … for analytics and data software providers that have an attractive, differentiated value proposition."

To some degree, that value proposition is AI.

AI vendors themselves are raising copious amounts of capital. For example, OpenAI has raised more than $10 billion in 2024, while Anthropic -- which develops the Claude line of generative AI Models -- raised about $7 billion this year.

Data management and analytics vendors that provide tools that can be used to develop AI models and applications have tended to be the ones attracting investors, according to Stephen Catanzano, an analyst at Informa TechTarget's Enterprise Strategy Group.

"[The vendors attracting funding] are all adding AI enablement capabilities to chase that market of AI workloads," he said in April. "I think vendors look at the massive AI market projections and can easily show they can get a piece, but they need more money and are getting attention." 

Eric Avidon is a senior news writer for TechTarget Editorial and a journalist with more than 25 years of experience. He covers analytics and data management.