Generative AI a key tech for analytics, but within limits

Generative AI is all the rage.

In the 14 months since OpenAI released ChatGPT, which marked a significant improvement in the capabilities of large language models (LLMs), a bevy of analytics and data management vendors have unveiled generative AI capabilities.

Most remain in some stage of preview as vendors work to ensure the reliability of their tools, but some are already generally available. Meanwhile, many organizations have begun incorporating generative AI models -- trained by combining their own proprietary data with LLM technology -- into their decision-making processes.

One reason vendors and enterprises are prioritizing generative AI is its potential to make existing data experts more efficient with increased process automation capabilities and natural language interfaces that eliminate -- or greatly reduce -- the need to write code. Another is that the natural language interfaces enabled by generative AI have the potential to expand analytics use beyond trained experts to virtually all employees within an organization.

But generative AI has its drawbacks and needs to be deployed with extreme care, according to Howard Dresner, founder and chief research officer at Dresner Advisory Services.

Generative AI models, no matter how carefully trained, often deliver incorrect outputs called hallucinations. And sometimes those outputs seem so plausible that they lead to bad decisions with terrible consequences. As a result, Dresner said, while generative AI can be a transformative technology, its use needs to be carefully governed, just as organizations carefully govern their data.

In a recent interview, Dresner delved deeply into generative AI and other analytics trends. In Part II of the discussion, he speaks about what generative AI does well and what it does not, as well as how generative AI will evolve. In Part I, he discusses analytics trends beyond generative AI, including the increasing importance many enterprises are placing on business intelligence.

Editor's note: This Q&A has been edited for clarity and conciseness.

Now a little more than a year since OpenAI released ChatGPT and generative AI became the top analytics trend, what are the GenAI capabilities that BI vendors are developing?

Howard Dresner

Howard Dresner: The common denominator for most BI and analytics vendors is that they're integrating with the dominant large language models. Some are going with the various OpenAI approaches. Some are using Google. Some are using open source. But at a minimum, they're all doing some sort of integration so users have a better natural language interface in [the vendor's] environment.

That's fine if they're [also] integrating that natural language interface with [a data application] that is able to take advantage of that interface. But if they're just using generative AI against raw data, good luck. Hallucinations are rampant when dealing with just data. One of our user organizations built their own large language models. When the LLMs are going against certain things like documents, they work very well. When they're trying to use them against raw data, they get lots of hallucinations.

Does that mean LLMs should never be used to query raw data?

Dresner: It's not impossible. Some organizations use multiple LLMs, and they're driving toward a consensus. They're finding ways to compensate for hallucinations. What people need to understand is that neural networks don't really know anything. They're mathematical models that weigh things and approximate. They don't actually know what the answer ought to be. That's why you can have a dialogue with ChatGPT, and even after a lot of prompting, it will say things that are absurd.

Eventually, that will get fixed.

In its current state, what is generative AI's greatest benefit as it relates to analytics?

Dresner: There are many areas where it can do great stuff. Summarization [of reports and documents] is certainly one. It can also look across things -- give perspective by looking across multiple bodies of information. It's really good at dealing with tech sources, and not just a single source, but combining data from multiple sources and synthesizing that information in a way that would be really hard for a human. That's extraordinarily powerful.

In addition, it replaces what we've traditionally thought of as natural language processing because of its facility with language, and not just English, but all language. That's always been one of the bugaboos with natural language interfaces. Typically, they only understood English and they required a lot of maintenance. Generative AI does not. It may require a lot of prompting and other things to ensure that you don't have hallucinations, but you don't have to do all that other work. That's why having a GenAI interface in an existing application is going to become the norm.

Does that mean generative AI will eventually be part of all analytics processes?

It has its uses. But there has to be someone in charge of how it's used. There have to be policies and processes, and there have to be some things that are taboo that you cannot do.

Dresner: It's appropriate for certain use cases, but not all use cases. Some people are still just going to want to report, some are going to want a dashboard, some are going to want some embedded insights in an existing application. We shouldn't think of GenAI as a panacea. It's not. It is really cool, and everybody says they're doing it and has a prototype that's in production. No organization says they're doing nothing with it. The problem is that none of it is coordinated [within the organization] -- everyone is doing their own thing, and that's chaotic.

One of the things we said at the beginning of last year when we first started studying GenAI is that you've got to govern this like anything else. It has to be governed, controlled. There have to be policies and processes. There has to be someone in charge of this, because otherwise it's just helter-skelter.

It has its uses. But there has to be someone in charge of how it's used. There have to be policies and processes, and there have to be some things that are taboo that you cannot do.

In its current state, what are GenAI's shortcomings as it relates to analytics?

Dresner: Once again, you have to be concerned about hallucinations.

It takes a great deal of prompting and refinement, and then you still have to review what it gives you. People are thinking that they can outsource tasks to generative AI, but someone still has to read what it generates. Sometimes, it will do things that look completely plausible. Someone will read it and think, 'That is awesome.' But then when they go to verify it, they discover it was completely erroneous.

If integrations with generative AI platforms and enabling natural language interactions are common capabilities analytics vendors are now providing, what do you think will be the next wave of GenAI capabilities vendors will develop?

Dresner: In general, you will see broad usage [beyond just analytics], especially in consumer-facing applications.

As it relates to analytics, I think you'll see generative AI show up -- and it already is -- in requests for information and requests for documentation, especially by IT. Generative AI will be included and will be an option [in platforms] whether it's analytics, performance management -- the question is, how deep will it be part of the platforms? In many cases, and rightly so, vendors are playing it relatively safe because they don't want to run into issues where they are generating inaccurate results. The whole industry is going to have to evolve so the core technologies become more reliable and more accurate, and as a result, the core technologies will take more advantage of generative AI.

It's really early days. We'll definitely see more of generative AI, both for good and for bad.

Eric Avidon is a senior news writer for TechTarget Editorial and a journalist with more than 25 years of experience. He covers analytics and data management.