Fostering Trust in Generative AI: Tackling Hallucinations for Business Success
Fostering Trust in Generative AI: Tackling Hallucinations for Business Success
Introduction
Generative Artificial Intelligence (Generative AI) has emerged as a transformational technology that allows the generation of novel content that ranges from text and images to audio and video. It has been deployed for various use cases such as Machine Translation, Knowledge Discovery, Text Summarization, Code Generation, Image/Video Generation, and many other applications.
At Aimon, we have talked to over 50 companies that are on their path to adopting Generative AI. Most of them are spending millions of dollars on building Generative AI applications to gain a competitive advantage in their market. One of the biggest hurdles and an intriguing facet of Generative AI technology is its inclination to produce what has been termed "Hallucinations" - unexpected, untrue, or unintended outputs that diverge from reality or learned context. Here is an example of GPT 3.5 hallucinating:
Here is the same example with Google’s Bard model:
You might remember this news article that came out in April about an Australian mayor preparing to sue Open AI for portraying him as an ex-convict.
What are Hallucinations?
If I were to describe it in two words, hallucinations are plausible nonsense. In other words, they are deviations from the expected output, often taking the form of bizarre, nonsensical, or unintended content. Generative AI models “make stuff up” due to various reasons (we cover those reasons below in the next section). This could range from minor inconsistencies to fabricated facts and contradictory statements.
There are two main types of hallucinations:
Intrinsic Hallucinations are hallucinations where the models manipulate original information and slightly misrepresent true facts. For example, “Eiffel Tower is the tallest building in France'' or “The first man on the moon was Yuri Gagarin”.
Extrinsic Hallucinations are the ones where models completely fabricate new false information. This can’t be inferred from or traced back to original sources. For example, “Cristiano Ronaldo is a Cricket player”
What is the business impact of Hallucinations?
Misleading Information
If a model is used to generate customer support summaries, marketing content, or other content that is intended to be informative or persuasive, hallucinations in the model's outputs can mislead users including employees, customers, and partners.
Reputation or Brand value impact
Generated content accessible by external users that is factually incorrect can harm the company's brand. This can lead to negative media coverage, boycotts, and other damage to the company's reputation. Recently, a law firm was fined after one of its attorneys used ChatGPT, to create a legal brief that included fake court cases with made-up citations and quotes.
Diminishing Trust
Most of the companies we talked to are spending millions of dollars to build their Generative AI solutions. Most of them identify Hallucinations as their biggest hurdle to launching their features. The reason is simple - they are worried that their solution may not be reliable and be perceived as untrustable by their customers.
Retraining costs
Once companies realize their models hallucinate about specific topics, the next thing is to go through a retraining process. This may entail processes such as fine-tuning or prompt engineering.
Legal and Compliance Risks
Hallucinated outputs might lead to legal challenges or compliance issues, especially in regulated industries where accurate and reliable information is critical.
Why do Generative AI models hallucinate?
Generative AI models have typically been trained on a massive amount of data from the internet, books, questions and answers, and other sources like Wikipedia. Driven by probability, Large language models (LLMs) that are a subset of generative AI models are really good at predicting what words should come next in a sentence.
But here's the thing: these models don't have a sense of what's right or wrong, true or false like we do (or do we?). Instead, they make guesses based on how likely they think a word should come next. It's like making educated guesses!
Sometimes, these educated guesses can lead to very surprising results. For example, they might appear to have impressive knowledge and be able to answer questions on legal or medical topics but their responses are influenced by what they've seen in their training data. A few other reasons why generative AI models may hallucinate:
Data Limitations
Inadequate or biased training data can result in the AI model learning insufficient, incomplete, or inaccurate patterns, leading to the generation of hallucinated outputs that mirror the data's limitations or biases.
Complexity of Patterns
In an attempt to capture intricate patterns within the data, Generative AI models can become overly complex. Consequently, when faced with new data or scenarios, the model may generate hallucinated outputs based on its training. Overfitting is a condition in which a model exhibits an excessive fit to the training data, but it struggles to generalize effectively to new data. On the other hand, underfitting describes a model that inadequately captures the intrinsic patterns within the training data, resulting in suboptimal predictive performance.
Ambiguity in Input Data
If the input data provided to the generative model is ambiguous or contains contradictory information, the model might struggle to generate accurate outputs. For example, one Gym membership document says the price for Premium membership is $49.99/mo and the other states it is $39.99/mo.
How do we reduce Hallucinations in Generative AI models?
Generative AI hallucinations can be reduced by tackling and preventing the above-mentioned issues such as insufficient data. However, due to the cost and scale at which Generative AI foundational models (proprietary or open source) are trained, it is not feasible for companies adopting them to mitigate Hallucinations entirely for the topics in their business domain. Here is a little guide on how such companies can approach minimizing Hallucinations for their specific use cases:
Detect Hallucinations in real-time: Consider this - when you roll out a Generative AI feature to your users, how do you know if and when the model hallucinates? There are various proposed solutions for detecting Hallucinations such as Rouge N, SelfCheckGPT, and cross-model verification. Additionally, Aimon offers a proprietary state-of-the-art Hallucination detector.
Improve Training Data: Once you know the scenarios in which your model hallucinates, you can then focus on improving that cohort of your training data. This helps your model gain a more robust understanding of the context and reduces the likelihood of producing unintended outputs. You could then use the improved data for the next step.
Improve the model: Real-world feedback can help guide improvements in model accuracy. One effective approach is Prompt Engineering which can be employed to instruct the model about what to exactly do, to not do, and how. For example, a prompt like such: Write an article about Paris and what her name means. Can be prompt-engineered to give a more desired output: Write an article about Paris and what the name means. Don't talk about the city but the singer. The base models can also be improved with factual updates with techniques such as RLHF and Fine Tuning. This feedback can help edit knowledge around specific topics but involves changing the model weights.
Human-in-the-Loop Approaches: Integrating human oversight into the generative AI process allows experts to review and filter outputs, ensuring that the content aligns with desired standards and outcomes. Ideally, human effort should be guided and optimized to better focus on higher-priority Hallucination topics.
Continuously monitor Hallucinations and Accuracy: Lastly, automate the continuous improvement of Generative AI models. The accuracy and how your users consume the model may change due to internal or external factors. Monitoring will help guide accuracy improvements, reducing manual efforts.
The Future
Future efforts around Hallucination Detection are likely to focus on refining models to minimize unintended outputs, developing more effective training methodologies, creating point models that do fewer things but do them well (enough), and creating frameworks for ethical and responsible AI use. With these developments, model Accuracy and Hallucinations will remain a very important focus area for Enterprise Generative AI adoption.
About Us
Model Accuracy and Hallucinations are one of the biggest hindrances preventing organizations from innovating with Generative AI. The exploration of generative AI hallucinations is an ongoing area of research and companies like Aimon are building innovative solutions to help companies unlock the value of Generative AI with confidence.
In a future with numerous model providers, and various frameworks, encapsulating and encoding the enterprise's truth layer is crucial. The team at Aimon Labs has developed innovative features that can help you detect and correct hallucinations thereby improving accuracy in your Generative AI models on an ongoing basis. We also offer solutions for Toxicity, Sensitive Data (PHI, PII, etc), and Custom confidential data protection.
Please reach out to us at info@aimon.ai if you would like to learn more about our solution.
#generativeai #ai #ml #chatgpt #bard #llm #machinelearning #hallucination