Getty Images
How companies are tackling AI hallucinations
AI hallucinations can cause real damage, so leading tech companies use automated reasoning, new data sets and retrieval-augmented generation to reduce them.
When typing a prompt into an AI model, there is a general expectation that the answer will be truthful. After all, that is why people turn to the tool in the first place. However, it is all too common for someone to ask a question and be given a response that seems believable on the surface – but turns out to be completely inaccurate. This is called an AI hallucination. For companies that want to use AI to support their operations, figuring out how to address AI hallucinations is a growing priority.
AI models respond to prompts using the same method each time, but AI hallucinations only occur occasionally; data from generative AI startup Vectara estimates that AI hallucinations occur between 0.7% and 29.9% of the time, depending on the large language model (LLM) used. This results from the AI detecting patterns or objects within its trained data set that don't exist, thereby producing an answer that isn't factually correct. Since false answers are generated and presented the same way as truthful answers, it is not immediately apparent when an AI hallucination occurs.
AI hallucinations have been around for as long as LLMs and generative AI programs have, but they have become more noticeable as more people adopt the technology. There are also more casual users interacting with AI now, thanks to the mass availability of tools such as ChatGPT and DeepSeek, many of whom may not have the same understanding of how AI works. Similarly, businesses are now using AI in more ways within their operations, not necessarily with a complete grounding of how the AI works, which increases hallucinations' visibility – and potential damage.
How AI hallucinations occur
Although not perfect, there is an expectation that AI tools will produce truth in response to human queries. The name "artificial intelligence" implies that the tool is a source of genuine knowledge; many users take that literally. More casual users of AI may not know precisely how the technology works and, therefore, don't realize that correct answers result from pattern detection rather than sentient understanding. This means many users don't know there is a need to fact-check the responses for AI hallucinations.
This issue is compounded by the fact that AI models are often asked questions about topics that the user has insufficient knowledge of, so they are turning to the model in the first place. Without a substantial grounding on the subject, detecting even quite apparent errors in response can be difficult. The AI is also pulling its answer from a large data set that includes factual material, meaning there are likely some elements of truth or similarities to the truth within the AI hallucination. This can be sufficiently convincing. While the user can do independent research and fact-check the answer, this can be time-consuming and undermines the convenience of asking the AI in the first place.
Another complicating factor is that should the questioner ask the AI to provide evidence for its answer; the model will often manufacture additional inaccurate material. The evidence looks convincing on the face, so the user doesn't realize an AI hallucination has occurred. Once again, it is simply using the data set to find patterns, so this is not malicious – but the effects are still accurate. This happened notably to a legal team working on a claim of aviation injury in 2023 who used ChatGPT for research and unknowingly filed fictitious material in their argument. They were ultimately fined for acting in bad faith.
AI hallucinations can occur quickly and be difficult to spot, especially if there is no fact-checking process. While some hallucinations may ultimately be benign and harmless, others can have significant ramifications – mainly when used in a business environment.
The potential pitfalls of AI hallucinations in business
An individual user receiving an incorrect piece of trivia can be irritating but not ultimately harmful. When an employee uses AI and receives a hallucination, the impacts can be both far-reaching and damaging. This is particularly true if the results support business strategy or convince external players toward a path of action. For instance, if an AI hallucination provides false figures but the employee does not realize their inaccuracy, these numbers could be used in business collateral that misleads investors, shareholders or clients about the reality of business performance. This can erode trust in the organization and damage the company’s reputation in the market.
Generating inaccurate information through an AI hallucination can drain resources and delay operations. These mistakes will likely be caught somewhere, requiring employees to review and correct information before processes can resume manually, but only after a delay. By not seeing the error at the start, both time and energy are wasted. When this happens at scale, the effect can substantially slow down performance and negatively affect revenue. This is precisely the opposite outcome that most AI programs are implemented for: to drive greater efficiency and free up employee time.
Due to the potential ramifications of AI hallucinations, many decision-makers are growing wary of using AI for essential use cases. This hesitation can lead to businesses losing out on the benefits of AI in their efforts to avoid the pitfalls. For this reason, tackling the issue of AI hallucinations will also positively affect broader corporate AI strategy.
How companies are addressing AI hallucinations
AI hallucinations are an unavoidable byproduct of LLM processing, but that doesn't mean companies cannot reduce their occurrence or effect. There are a few different approaches to this problem, with some focusing on improving the original data set and others looking into the fact-checking aspect of their AI tools.
Automated reasoning
One approach is to deploy automated reasoning to verify AI results as they are generated. Amazon has been a prominent player in this aspect of AI hallucination solutions through the introduction of its Automated Reasoning checks product. Automated reasoning uses mathematical, logic-based algorithms and reasoning processes to "prove" the truth of the result, giving users higher confidence in the veracity of their outputs. Since the process is automated, customers don't need to delay their operations or manually conduct fact-checking. Each customer chooses their policies and creates a personalized "absolute truth" against which the AI results are verified so that the results are relevant to their specific use cases. PwC is currently deploying tools within its portfolio--such as the Maestro Medical Legal Review solution--to validate that AI results comply with regulations.
Improved Data Quality
Another method is to address the original data set that the LLM is trained on to reduce the risk of pulling from inaccurate data. Inaccurate data could mean unverified or faulty data points or refer to subjective and opinion-based material. At Google, they have approached improving the veracity of the AI output by restricting the amount of user-generated content factored into an AI result. User-generated content (UGC) includes social media posts or Reddit contributions, which may reflect opinion rather than fact. By limiting the balance of UGC in the AI processing, Google hopes to reduce the risk of AI hallucinations.
Retrieval-Augmented Generation
Finally, retrieval-augmented generation (RAG) is a growing popular method in addressing AI hallucinations. RAG supplements LLMs' datasets by connecting them to specific repositories of information relevant to their use cases. This allows for greater depth and accuracy when submitting queries tied to more specific or technical topics, such as medicine or law. RAG can link to any external resource, so these can be highly customized for different purposes. Since the answers are checked against these (often highly technical) references, it reduces the risk of an AI hallucination and enables greater confidence in the results.
Other methods to mitigate AI hallucinations
Since this is such a widespread phenomenon, companies are trying a range of other methods to minimize the effects of hallucinations. Some of these include:
- Introducing guardrails on how AI is used within the business. This could involve limiting the use cases within which AI is deployed or implementing specific constraints in regard to outputs.
- Maintaining human visibility over AI outputs. By keeping humans in the loop at every stage of AI generation, there is less chance of a hallucination going undetected.
- Rigorously testing LLM models before deployment. This enables engineers to identify and resolve any recurring issues before the technology is rolled out more widely.
- Establishing a culture of transparency and educating employees on AI limitations. Since AI hallucinations can’t be eradicated entirely, it is critical to teach users about the possibility of hallucinations and the importance of verification.
Madeleine Streets is a senior content manager for WhatIs. She has also been published in 'TIME,' 'WWD,' 'Self' and Observer.'
