Unveiling the Technological Truth Behind AI Hallucinations and Coping Strategies: Exploring the Future of Artificial Intelligence—Learn AI Slowly 042

Introduction

• Can AI truly distinguish between reality and fiction?
  • Would you feel like crawling into a hole if your AI assistant quoted a fabricated legal precedent during a crucial meeting?

• Are we ready to bear the consequences of AI’s errors?
  • When an AI “diagnosis” could turn doctors into “killers” in an instant, how much can we trust its advice?
• Can we completely eliminate AI hallucinations?
  • Could technology advance to a point where human supervision is unnecessary?
  • Or will humans always have to patch things up behind AI?
• How can we maintain control over AI’s outputs while using it?
  • How should companies strike a balance between AI and human review?
  • After all, AI can also get “distracted”!
• With risks and opportunities, how should we navigate AI hallucinations?
  • Can we view AI hallucination as a springboard for innovation, rather than a pitfall?
• For the average person, AI hallucinations can obscure the judgment of conclusions in unknown areas.
  • Caution should be exercised: boldly hypothesize, carefully verify.

AI hallucination is one reason why many people are deterred from deeply engaging with AI; it can spout nonsense with remarkable seriousness. There were once prompt techniques that allowed AI to respond based on its training data, which could somewhat sidestep temporal hallucinations; however, they can never be completely eradicated due to the nature of generative AI.
This article aims to explore AI hallucinations alongside human cognitive biases, the background of their emergence, current efforts to address these issues, and finally, observe AI hallucinations from a positive perspective to find ways to coexist with AI.

AI Daydreaming? — An Unsettling Case of AI Hallucinations

“When Lawyers Start Spouting Nonsense”—AI Inventing False Legal Cases

Imagine a lawyer confidently citing a case provided by AI in court, only for the judge to point out that the case is entirely fabricated—what an awkward situation that would be! This isn’t some movie plot; it’s a real occurrence within AI hallucinations.
Despite possessing extensive legal knowledge, AI often fabricates nonexistent cases, creating plausible names, judges, and even judgment dates without any truth. This raises serious concerns.

“Heart Diabetes”?—AI Doctor, Are You Serious?

The emergence of AI medical diagnosis opens up hope for addressing healthcare shortages and improving diagnostic efficiency. However, AI doctors can also make laughably absurd, even life-threatening mistakes.

Some AIs in medical fields might create unheard-of medical terms when answering queries. For example, they could rigidly combine “heart failure” and “diabetes” into a bizarre new diagnosis—“heart diabetes”! Such absurd “creativity” not only highlights AI’s lack of understanding of medical knowledge but can mislead doctors, delay treatment for patients, or even result in irreversible consequences. AI doctor, are you sure you’re not joking?

It seems that the issue of AI hallucinations is extraordinarily distressing and almost unbearable to look at—could it really be so? Let’s examine another case.

AI’s “Creative Sparks” — A Shortcut to Scientific Discoveries?

AlphaFold3 is a protein structure prediction method used to study biomolecular structures. It can predict how different molecules interact, helping scientists understand disease mechanisms and develop new drugs.

For instance, AlphaFold3 can be used to study how antibodies bind to viruses. This information can support the design of new vaccines.

Here are some potential applications of this method:

• Studying how proteins interact with other molecules, such as drugs or target molecules.
• Predicting protein structure and function, which can assist scientists in designing new medications and treatments.
• Investigating mechanisms of diseases, which can lead to the development of new diagnostic and therapeutic methods.

AlphaFold3 represents a powerful new tool with the potential to revolutionize how we understand biomolecules and treat diseases.

When AlphaGo defeated Lee Sedol with a perplexing move, it left nearly everyone human astounded. However, humanity lost! It’s hard to argue against calling this an “illusion” created by humanity’s vanity, under which we might increasingly face the consequences of our own misguided confidence.

AI Hallucinations: Why So Confusing? What’s the Difference With Errors?

Honestly, it’s hard to be sure without checking when someone speaks so authoritatively.

People often refer to certain erroneous outputs from AI models as “hallucinations” for a few key reasons:

The “Rationality” of Output

AI hallucinations differ from typical grammatical or spelling errors; they refer to cases where the model generates grammatically correct, semantically smooth, and even plausibly reasonable sentences, but these sentences contradict facts or deviate from user expectations. This kind of “rationality” makes it challenging to immediately detect the underlying errors, creating a feeling of being “fooled.”

The “Confidence” of Output Tone

AI typically presents responses with a strong sense of confidence, even when these responses are inaccurate. This “confidence” can easily lead one to believe that the model genuinely “knows” the answer, thereby lowering vigilance and making it easier to trust its outputs.

Drawing Parallels With Human Hallucinations

Originally, the term “hallucination” refers to perceptual experiences wherein individuals perceive things without external stimuli, such as seeing or hearing things that do not exist. By calling AI’s erroneous outputs “hallucinations,” we draw a parallel to human cognitive phenomena, suggesting that AI models may also create “false perceptions” similar to humans.

Distinguishing Hallucinations from Errors: Examples

	Errors	Hallucinations
Nature	Errors in grammar, spelling, calculations, these mistakes are relatively easy to identify and correct	Errors arising from misunderstanding knowledge, reasoning, or alignment with human expectations, these errors are deeper, less detectable, and harder to correct
Manifestation	Lead to incomplete, awkward, or obviously unreasonable outputs	Lead to outputs that appear reasonable and fluent, even exuding strong confidence, but upon deeper examination, they reveal logical flaws or factual inaccuracies
Causes	Due to random errors occurring during model training or decoding	Due to the model’s inherent limitations in knowledge, data biases, or insufficient reasoning capabilities

General Errors

• Question: “Translate this sentence into French: ‘The weather is nice today.’”
• Model Output: “Aujourd’hui est un beau temps.” (Grammatical error; the correct translation should be “Il fait beau aujourd’hui.”)

AI Hallucinations

• Question: “Translate this sentence into French: ‘The weather is nice today.’”
• Model Output: “Paris is the capital of France.” (Seems reasonable but contradicts the user’s instruction)

AI Hallucinations

• Question: “If Mount Everest were 500 meters shorter, which mountain would be the highest in the world?”
• Model Output: “If Mount Everest were 500 meters shorter, it would still be the highest mountain in the world.” (Seems reasonable, but the reasoning is flawed)

This serves as a wake-up call, raising concerns about AI hallucinations. Could it be that AI can actually create hallucinations akin to human ones? What are the distinctions and connections between AI hallucinations and human cognitive biases? To better understand this issue, we need to conduct a detailed comparison of both.

AI Can Also Make Mistakes? — A “Mirror” to Human Cognitive Biases

In the Crowd, There Are You and Me: Human Misinterpretations

Many years ago, in the hallways of a college dormitory, the roar of the Black Panther was often heard:

In the sea of people, there are you and me, meeting, getting to know each other, and reflecting on one another.

No one reminded anyone else of the misspeaking, nor did anyone find it disconcerting. Only these two lines were sung every time, until I later looked closely at the lyrics and discovered the error. After realizing the mistake, I never sang it again. I suspect that those listening at the time likely also assumed that the line was correct. With such passionate roaring, how could there be an error in the lyrics?

We tend to confidently affirm certain facts, only to face discrepancies with reality. This happens frequently in life, such as with internet rumors spreading—initially accepted without question only to be disproven later for a myriad of reasons.

We’ve all experienced embarrassing moments of “seeing incorrectly” or “misunderstanding.” Are these not manifestations of human cognitive biases? So does AI make similar mistakes? Are AI hallucinations akin to cognitive biases in the AI world? To solve this mystery, we should also conduct a thorough comparative analysis of both.

Definitions and Mechanisms of Generation: “Cousins” Rather Than “Twin Brothers”

AI hallucinations and human cognitive biases both point to the distortion or misunderstanding of information. However, there are fundamental differences in their mechanisms of production:

• Human Cognitive Biases: Stem from human psychological and physiological mechanisms.
  • For example, inattention, memory distortion, emotional fluctuations, and inherent cognitive biases. Human perception is not flawless; our brains interpret information based on prior experiences and expectations, making us susceptible to various cognitive biases.
• AI Hallucinations: Arise from the technical limitations of AI systems, including:
  • Data Defects: Training data deficiencies, biases, and noise may hinder AI model’s ability to accurately learn the patterns of the real world.
  • Model Limitations: Oversimplified model structures, unreasonable parameter settings, or inadequate training methods can result in poor generalization ability, easily leading to hallucinations.
  • Reasoning Defects: Even with sufficient knowledge, an AI model may still generate logical errors or inference mistakes when confronted with complex issues.

Thus, AI hallucinations and human cognitive biases are more like “cousins” than “twin brothers.” They share similar manifestations, but their underlying causes are distinctly different.

Manifestation and Impact Scope: From “Individual Fallacies” to “Collective Hallucinations”

The student who thinks they bombed the exam believes they nailed it. While screwing in, they wonder, “I answered correctly, didn’t I?”

Both AI hallucinations and human cognitive biases exhibit many similarities in their manifestations, such as:

• Information Distortion: Both can lead to twisted or misunderstood information, like misremembering events, misinterpreting others’ intents, or erroneous perceptions of numbers or statistics.
• Logical Errors: Both may result in faulty reasoning processes, leading to poor judgments, incorrect conclusions, or unreasonable suggestions.

However, the scope of influence is drastically different:

• Human Cognitive Biases: Generally only affect individual judgment and behavior, represent “individual fallacies.” For instance, due to confirmation bias, an investor might develop excessive optimism about a specific investment project, ultimately leading to personal financial losses.
• AI Hallucinations: Given the widespread usage of AI systems, hallucinations can impact thousands of users, or even society at large, representing “collective hallucinations.” For example, if a news recommendation algorithm is biased, it could lead to the large-scale dissemination of false information, triggering social panic or manipulation.

		Human Cognitive Biases	AI Hallucinations
Essence	Distortion of information	Shortcuts taken by the brain to conserve cognitive resources, these shortcuts enhance efficiency but can also lead to misinterpretation and distortion	Overreliance on statistical patterns in training data, making it unable to accurately understand and generate information when faced with new situations
Manifestation	Diverse and often undetected	Confirmation bias (focusing only on supportive information), availability bias (easier recall of recent or prominent information), anchoring effect (overreliance on initially acquired information)	Creating nonexistent people, places, or events, or providing incorrect descriptions of known facts.
Causes	Related to experience and knowledge	Dependent on personal growth experiences, cultural backgrounds, and knowledge structures. Different experiences and knowledge create different cognitive models, leading distinct interpretations of the same information	Depending on the quality of training data, model structure, and training strategies. If the training data has biases or errors, the model learns these biases and errors, reflecting them in its outputs
Impact	Can lead to erroneous decisions	May cause incorrect judgments and choices in life. For instance, if an investor is affected by availability bias, they may overestimate recent trends in the stock market, leading to poor investment decisions	Can mislead users, disseminate false information, or even lead to safety incidents. For example, if a medical diagnostic AI system hallucinates, it might provide incorrect diagnoses, delaying patient treatment

AI Hallucinations: A “Magnifying Glass” for Technical Defects

Although AI hallucinations and human cognitive biases exhibit many similarities, it’s crucial to recognize the uniqueness of AI hallucinations. AI hallucinations are not indicative of AI having consciousness or subjective intent; instead, they reflect technical deficiencies within AI systems.

The emergence of AI hallucinations serves as a reminder that AI technology remains in development; its reliability and security require ongoing attention and improvement. We shouldn’t equate AI systems with humans, nor should we simply blame hallucinations on AI itself. A deep understanding of the essence of AI hallucinations can better equip us to address their challenges, allowing artificial intelligence to become a genuine partner rather than a potential threat.

Again, it’s clear that AI hallucinations are not subjective fabrications, but rather a manifestation of its technical limitations, fundamentally distinct from human cognitive biases. However, in the face of these cognitive “traps,” humanity has developed strategies over a long evolutionary process to counteract them. So how does humanity counteract cognitive biases and avoid falling into these thinking pitfalls? What insights can these strategies provide for tackling AI hallucinations?

Cognitive Training: Keeping the Brain Alert

In simpler terms: just keep learning!

The brain operates like a sophisticated instrument that requires constant learning and training for optimal performance. To avoid “assuming” errors, we must continuously enhance cognitive abilities, like regularly upgrading a system or patching its bugs.

• Recognizing “Cognitive Traps”: Just as we learn to identify phishing emails, we also need to understand common cognitive biases like:
  • Confirmation Bias: We tend to seek out information that supports our existing views while overlooking contradictory evidence. For example, those who believe in astrology might pay more attention to aspects of their horoscope that align with their experiences while dismissing inconsistencies.
  • Anchoring Effect: We can also be heavily influenced by first impressions, even if erroneous. For example, a business may mark a product with a high initial price, then offer a discount, leading consumers to feel they’re getting a great deal—even if the discounted price is still above market value.
• Cultivating Logical Thinking: Just like learning math, we need to understand logical reasoning to identify flawed arguments. For example, the statement “All swans are white because the swans I’ve seen are white” is logically flawed, as it doesn’t account for the existence of black swans.
• Mastering Data Analysis Skills: In an age overflowing with information, we encounter vast amounts of numbers and statistics every day. Learning basic statistical concepts can help improve our ability to analyze data and avoid misleading narratives. For example, if an advertisement claims a health product is effective 90% of the time without specifying sample size or methodology, we must remain cautious not to take it at face value.

Just as you wouldn’t play around with dosage, besides self-improvement, there are many tools to help improve cognitive biases.

Structured Thinking: Using Tools to Aid Judgment

Thinking tools expand our brain’s computational power and storage capacity.

Even when we strive to maintain alertness, our brains may occasionally “slack off” and fall into “assuming” errors. In those instances, structured thinking tools can help reduce mistakes.

• Decision Matrix: When faced with multiple options, we can list the pros and cons of each choice in a table, quantifying scores to help make a more rational decision.
  • For example, when choosing a travel destination, we may score attributes like scenery, transportation, and costs instead of relying solely on gut feelings.
• Checklists: When executing complex tasks, we can use checklists to ensure that each step is carried out as planned, avoiding omissions or mistakes.
  • For instance, pilots meticulously follow checklists before taking off to ensure all aircraft systems are functioning well.
• Risk Assessment Models: When making significant decisions, we might use risk assessment models to analyze the risks of different options and formulate contingency plans.
  • For instance, conducting a risk assessment before investing can help manage finances better, reducing potential losses. This is analogous to doctors using standardized checklists to minimize misdiagnoses. Such structured thinking tools act like training wheels for our minds, enabling us to make more precise judgments when facing complex problems.

Collective Intelligence: Harnessing the Wisdom of the Group

More importantly, as social animals, we continuously create new understanding through information exchange.

As the saying goes, “Three cobblers with their wits combined can come up with a Zhuge Liang.” When facing complex challenges, relying solely on individual strengths may not yield the best solutions. Here, we can tap into collective intelligence to compensate for the limitations of individual cognition.

• Brainstorming: By pooling ideas, we can approach problems from different perspectives, igniting creativity, and finding more comprehensive solutions.
  • For example, during a product design meeting, team members are encouraged to share various ideas without being confined to their own thought processes.
• Debates and Discussions: By engaging in debate and discussion, we can clash and refine different viewpoints, helping us better grasp the essence of problems and formulate more reasonable solutions.
  • For instance, in court proceedings, lawyers attempt to persuade judges and juries through logical arguments and evidence showcased during the debate process.
• Voting and Negotiation: When collective decisions are needed, we can integrate everyone’s opinions through voting or negotiation to arrive at options acceptable to the majority.
  • For instance, a community owners’ meeting can vote on management plans for the neighborhood.

Collective wisdom can be viewed as combining multiple “processors,” forming a powerful “computing network” capable of tackling even more complex issues.

From Humans to AI: Secrets to Making AI Smarter

The ways humans tackle cognitive biases provide valuable insights for addressing AI hallucination challenges. By adapting these methods to develop corresponding technical solutions, we can help AI understand the world better and make more accurate judgments.

• Data “Cleanup”: Just as humans require cognitive training, we need to carry out a “cleanup” of the training data for AI models—removing errors, filling in gaps, and balancing biases to ensure AI models learn more accurate and comprehensive knowledge.
• Opening AI’s “Black Box”: Similar to how humans use structured tools to aid in thinking, we should make AI’s “thinking processes” more transparent to facilitate human understanding and oversight.
  • For example, explainable AI technologies can help us make sense of how AI models arrive at their conclusions, avoiding erroneous conclusions derived from faulty logic or data. (Research by the Anthropic team in 2024 aims to address the “black box” issue, and is still in the exploratory phase.)
• Creating an AI “Advisory Board”: Just as humans utilize collective decision-making to maximize brainstorming, we can have multiple AI models collaborate to solve problems, mitigating errors resulting from limitations of individual models. (Recent discussions highlighted beneficial explorations such as the multi-agent model by Andrew Ng.)

Managing AI hallucinations is a challenging journey that requires continual learning from human wisdom, transforming experiences in combating cognitive biases into momentum for advancements in AI technology.

Humanity has gained valuable experience from the long-standing struggle against cognitive biases. This experience teaches us to enhance our cognitive abilities and harness tools for aids in judgment, leveraging collective intelligence to minimize errors and make more rational decisions. For AI systems, do similar “cognitive biases” exist? What precisely are AI hallucinations, and what forms and categories do they encompass?

AI Hallucinations: When Artificial Intelligence “Earnestly Spews Nonsense”

Having understood how humans skillfully navigate cognitive biases, do AI systems face similar “cognitive pitfalls”? The answer is undoubtedly yes: this is “AI hallucination.”

AI Hallucinations: A Maze of Reality and Imagination

In simple terms, AI hallucination refers to information generated by AI systems that does not align with facts or expectations, much like how artificial intelligence “earnestly spews nonsense.” These “nonsense outputs” aren’t intentional; they result from technical limitations.

We can define AI hallucination as follows: The outputs generated by AI systems appear rational and fluent but contradict the input information, contextual environment, or objective facts, lacking logical or experiential support.

AI Hallucinations: “Multiple Faces”

AI hallucinations can manifest in various forms, as illustrated in the diagram below:

The Risks Behind AI Hallucinations

While AI hallucinations may seem like “minor errors,” in practice, they can pose significant risks.

• Misleading Users: AI hallucinations can lead users to acquire incorrect information, resulting in erroneous judgments.
  • For example, a medical AI assistant that provides incorrect diagnostic recommendations may delay patient treatment.
• Dissemination of False Information: AI hallucinations may facilitate the creation and dissemination of misinformation, misdirecting the public and affecting social stability.
  • For instance, AI could generate fake news reports or social media posts used for political propaganda or commercial hype.
• Damaging Credibility of AI Systems: AI hallucinations can erode users’ trust in AI systems, hindering the promotion and application of AI technology.
  • For example, if users discover that AI frequently “spouts nonsense,” they may lose faith in its judgment and even reject AI products.

AI Hallucinations: A Duality of Opportunities and Challenges

The emergence of AI hallucinations rings alarm bells, reminding us to focus on security and reliability while developing AI technology. However, we should not dismiss the positive implications of AI hallucinations.

• Advancing Technical Progress: The emergence of AI hallucinations reveals existing technical repressions, driving researchers to explore new methods and technologies, improving the performance and reliability of AI systems.
• Inspiring Human Creativity: AI hallucinations can sometimes produce unexpected and novel outputs, offering inspiration for human artistic creation and scientific inquiry, facilitating breakthroughs in conventional thinking as we explore the unknown.

AI hallucinations serve as a double-edged sword: both a challenge and an opportunity. We need to confront the risks posed by AI hallucinations while actively exploring their potential value, ensuring that artificial intelligence serves humanity more effectively.

AI hallucinations, much like a “ghost” haunting AI systems, may wear the guise of reality but could lead us astray. What causes these “earnest nonsense” phenomena? Delving into the origins of AI hallucinations will be crucial to finding effective countermeasures, enabling artificial intelligence to become a genuinely trusted partner for humanity.

Can AI “Go Bad”? — Investigating the Hidden Drivers of AI Hallucinations

Having witnessed various bewildering behaviors of AI hallucinations, one cannot help but ask: what causes AI to venture down the path of “earnestly spewing nonsense?” Only by unveiling the mystery behind AI hallucinations can we identify effective antidotes to prevent AI from straying further down the wrong path.

The Misinformation of “Poor Textbooks”: The “Pitfalls” of Data Quality

In early information construction, a common joke was, “Garbage in, garbage out.” This is also a significant reason for many information projects failing; AI’s learning process is no exception.

If the training data for an AI model is flawed, it might “go bad” and produce various hallucinations.

• Information Pollution:
  • If AI’s source materials contain falsehoods, it may remember and output these errors as “truth.” AI models can easily be influenced by erroneous information in their training data, producing “mimicking errors.”
  • For example, If the training data is filled with false information claiming “the Earth is flat,” the AI model might confidently declare the Earth is flat when questioned.
• Bias Trap:
  • If training data carries biases, AI may internalize these biases and express them in its outputs. For instance, if a majority of programmers in the training data are male, AI might assume programmers are male while neglecting the existence of female programmers.
  • This bias could cause AI outputs to lack objectivity and fairness, potentially exacerbating existing biases and discrimination in society.
• Knowledge Gaps:
  • An AI’s knowledge base comes from its training data; if the data lacks knowledge in certain domains or isn’t current, the AI may “gropingly” respond to queries by “making things up.”
  • For example, if a medical AI model has never been trained on a rare disease, when faced with relevant cases, it may provide incorrect diagnoses or invent nonexistent medical terms.

“Insufficient Brainpower”?—Limitations of the Model Itself

Even if training data is flawless, AI models can still hallucinate due to internal limitations.

• Simple Structures, Insufficient Understanding:
  • AI models resemble students; if their “brains” are overly simplistic, their comprehension capabilities will be limited, leading to misunderstandings or misinterpretations when faced with complex text or images.
  • For example, a simple image recognition model may struggle to differentiate between a cat and a tiger since they share similar characteristics.
• Lack of Focus, Missing the Key Points:
  • AI models must concentrate their attention when processing information. If they “lose focus,” they might overlook crucial information while overemphasizing irrelevant data, resulting in outputs that “miss the mark.”
  • For example, when translating a news report, AI might focus excessively on a detail while disregarding the overarching theme, leading to translations that diverge significantly from the original message.
• Insufficient Reasoning, Leading to Logical Confusion:
  • AI models need adequate reasoning abilities to grasp the logical relationships behind texts or images. If they lack these skills, they risk generating logically confused text, even contradicting themselves.
  • For example, a conversation bot might start by saying “Today is sunny,” only to follow up with “Don’t forget your umbrella, it’s going to rain today.”

“Inadequate Training”?—Flaws in Training Methods

Training AI models is akin to students learning; effective methodologies and sufficient time are necessary for successful outcomes. If there are defects in training methods, AI models may struggle and easily hallucinate.

• Rigid Learning Modes, Lacking Flexibility: Traditional AI training methods generally require models to repetitively learn a massive sample of data until they can accurately identify or produce those samples. However, this method lacks flexibility, leading the model to flounder when encountering new, unseen samples and resorting to guesses, resulting in a hallucination. For example, an AI model that only learned images of cats might misclassify a dog’s image as a cat or “imagine” a dog based on typical cat traits.
• Overreliance on “Standard Answers”, Lacking Creativity: In the training process, AI models often receive a “standard answer” to reach, requiring continuous adjustments until their outputs align with the standard. However, excessive dependency on “standard answers” can stifle an AI model’s creativity, rendering it unable to produce novel or creative answers when confronted with open-ended problems, leaving them only capable of restating existing knowledge or “cobbling” based on prior information.

Does AI Also “Misspeak”?—Randomness in Reasoning Processes

When generating text or images, AI models typically choose the most probable words or pixels based on probability distributions. This randomness can boost the creativity and variability of AI models but may lead to “misspeaking,” producing illogical or inaccurate statements.

For example, when an AI writing model constructs a narrative, it might select pronouns “he” or “she” to refer to a character. If the model hasn’t correctly grasped the character’s gender, it may result in “gender confusion,” like referring to a female character as “he” in a female-focused story.

The emergence of AI hallucinations represents a complex process, likely arising from numerous factors simultaneously. Only by fully understanding the origins of AI hallucinations can we find effective countermeasures to ensure systems remain reliable and secure.

From data quality to model structure, training methodologies, and reasoning mechanisms, the emergence of AI hallucinations resembles a domino effect—interconnected and complex. To better understand this issue, we must reflect on the history of AI hallucinations and explore how they evolved alongside advancements in AI technology.

A Brief History of AI Hallucinations: From “Artificial Idiocy” to “Earnest Nonsense”

Having understood the multi-faceted interplay behind AI hallucinations, how has this “stubborn” problem evolved alongside AI technology’s advancement? Let’s revisit the brief history of AI hallucinations to uncover potential governance clues.

Expert Systems Era: The “Cage” of Rules

Early AI systems predominantly relied on rules and logic set by human experts for reasoning and decision-making, much like actors performing only according to the prescribed script. These systems, known as “expert systems,” excelled in addressing specific domain problems, such as diagnosing certain diseases or performing simple arithmetic calculations.

However, the fatal weakness of expert systems lies in their limited knowledge bases. When faced with situations exceeding their predefined rules, they would become “stumped” and make basic errors, akin to a student who can only recite from the textbook but for whom anything beyond the textbook is a complete mystery.

For instance, early medical expert systems might misdiagnose or suggest incorrect treatments when encountering unfamiliar diseases. Such circumstances, viewed today, resemble “artificial idiocy,” but at the time reflected the true level of AI technology development.

Machine Learning Era: From “Brute Memorization” to “Extrapolation”

With advancements in computing technology and exponential data growth, AI entered the “machine learning” era. Machine learning algorithms could autonomously discover patterns from vast quantities of data and make predictions and decisions, much like students summarizing problem-solving techniques from numerous exercises.

Compared to expert systems, machine learning models exhibit enhanced generalization abilities, allowing them to handle more complex and dynamic tasks. Nevertheless, they still rely on data quality and quantity. If training data contains biases or inadequacies, AI models may still “learn biased” perspectives and produce various hallucinations.

For example, if a machine translation model has learned only limited language samples, it might generate semantic errors or logical chaos while translating complex sentences, mirroring a student who can only “memorize” vocabulary without comprehending deeper meanings.

Deep Learning Era: Secrets Within the “Black Box”

In recent years, deep learning technology has made breakthrough advancements, becoming the “star of the AI field.” Deep learning models possess more intricate structures and parameters, enabling them to learn finer features from massive datasets, resulting in more accurate predictions and decisions.

However, deep learning models often lack interpretability, with their internal decision-making process functioning as a “black box,” making it difficult for humans to understand. This complicates our ability to determine if an AI model genuinely “understands” something or merely retains information; when confronted with novel situations, hallucinations may emerge, often hidden beneath a seemingly plausible facade.

As AI model complexity escalates, the variety of AI hallucinations expands, making their manifestations and detection increasingly challenging. For instance, an AI writing model might produce a syntactically correct, logically coherent article, but the content could be entirely fictitious, fabricated with nonexistent characters, events, or theories.

AI Hallucinations: “Side Effects” of Technological Progress

Reflecting on the history of AI hallucinations reveals that this issue isn’t new; it has been evolving alongside advancements in AI technology. From early “artificial idiocy” to today’s “earnest nonsense,” the complexity and obscurity of AI hallucinations continue to escalate.

The evolution of AI hallucinations also reflects the growth of AI technology levels—shifting from reliance on manual rules to data-driven models and now deep learning, making AI systems increasingly “intelligent,” yet simultaneously confronting more intricate and unpredictable challenges. AI hallucinations are indeed a “side effect” of technological progress, reminding us to remain vigilant about safety and reliability as we enhance AI capacity.

The issue of AI hallucinations has accompanied AI technology like a ghost on its journey—from the “artificial idiocy” of early expert systems to the “earnest nonsense” generated by deep learning models, its complexity and obscurity continue to evolve. While AI abilities are becoming stronger, its tendency towards “filling in gaps” with sincere nonsense raises growing concerns. As a significant challenge bound to AI progress, what countermeasures do humans have? What steps have researchers taken to “tame” AI hallucinations and make AI more safe and reliable?

AI Hallucinations: Not Without Remedy

The history of AI hallucinations tells us this challenge has been a persistent shadow over AI development. So, in confronting AI hallucinations, are we doomed to be helpless? Not at all! In recent years, researchers have devised various techniques to attempt to “tame” this elusive “ghost,” making AI increasingly trustworthy.

Data “Health Check”: Building a Solid Foundation for AI

As mentioned earlier, low-quality training data is a significant contributor to AI hallucinations. Hence, providing “clean” and “healthy” training data for AI models is essential—similar to giving AI a comprehensive “health check”—to prevent hallucinations.

• Data Cleaning: Just as a doctor purges toxins from a patient, data scientists employ various techniques to conduct a “cleaning” of AI training data, eliminating erroneous information, filling in missing data, correcting inconsistencies, and striving to mitigate biases in the data.
• Data Augmentation: To ensure AI models learn comprehensive knowledge, we need to provide a more extensive and richer training dataset—akin to supplying varying types of practice problems to students, assisting them in mastering diverse knowledge points and problem-solving techniques. For example, when training an image recognition model, we might modify existing images through rotation, scaling, or cropping, generating new samples to enhance the model’s generalization capabilities.

“Rewiring the Brain”: Optimizing AI Models

In addition to providing high-quality data, we can lower the risks of AI hallucinations by optimizing the AI models themselves.

• Model Editing: If we identify deficiencies in certain areas, such as a model’s propensity for specific types of hallucinations, we can utilize “model editing” techniques to amend the structure or parameters, akin to a doctor performing surgery to fix bodily flaws.
• Prompt Engineering: AI models require instructions to operate; these “prompts” serve as commands given to AI models. By thoughtfully designing prompts, we can guide AI models to better interpret our intentions and generate outputs that align more closely with our expectations, consequently reducing hallucinations. For example, when using an AI writing model, we might set parameters for writing style, themes, or keywords to steer the model towards producing suitable text.

“Knowledge Plugins”: RAG Technology

To address potential knowledge deficiencies in AI models, we can supply external knowledge support, similar to offering students reference materials. Retrieval-Augmented Generation (RAG) is a method that provides AI models with “knowledge plugins.”

We can summarize how RAG works in a few steps:

1. Understanding Queries: The AI model needs to comprehend user questions or instructions initially.
2. Retrieving Relevant Knowledge: Based on its understanding of the questions, the AI model searches external knowledge bases for pertinent information. For instance, if a user inquires, “How tall is the Eiffel Tower?”, the AI model would retrieve related data from its knowledge base.
3. Integrating Knowledge and Reasoning: The AI model combines retrieved knowledge with its reasoning capacity to produce the final answer or content.

The advantages of RAG technology include extended AI model knowledge, increased accuracy in responses, and improved content generation by accessing reliable information.

Advantages of RAG Technology:

• Improved Accuracy: By retrieving relevant information, generated content is more fact-based, reducing the chances of fabrication and hallucinations.
• Increased Consistency: Integrating retrieved information with generated models ensures coherence in context and logic.
• Strong Adaptability: RAG technology can be applied to various generation tasks, including text generation, Q&A systems, translation, etc., indicating wide application prospects.

Limitations of RAG Technology

While RAG technology effectively reduces AI hallucinations, it possesses limitations:

• Dependence on Retrieval Quality: RAG’s performance highly depends on the quality of retrieved information. If errors or biases exist in the knowledge base, generated content may also be affected.
• High Computational Resource Requirements: The combination of retrieval and generation requires significant computational resources, which might restrict its application in resource-limited environments.
• Slower Processing Speeds: Given the requirement for information retrieval before content generation, RAG technology tends to operate more slowly, potentially rendering it unsuitable for real-time applications.

“Adversarial Training”: Making AI More Resilient

During training, AI models may encounter “malicious” sample data, such as deliberately erroneous information or noisy datasets. These “malicious” samples may mislead the AI model, leading to hallucinations. To enhance the robustness of AI models, we might apply adversarial training techniques, allowing them to deal with these “malicious” samples and learn how to identify and resist them.

Adversarial training resembles “live exercises” for AI models, preparing them to remain calm and make correct judgments when faced with various “attacks.”

Precise “Control”: Utilizing Prompts to Subdue AI Hallucinations

To avoid letting AI fall into the trap of hallucinations, we can cleverly apply “prompt engineering,” like a seasoned animal trainer using precise commands to guide AI to produce more reliable content.

The AI’s ability to understand prompts is closely tied to hallucination occurrence. Clear and specific prompts help AI better grasp our intentions, thereby reducing “assuming” errors.

For example, when we ask AI about “the key dates of World War II,” simply posing the question may elicit responses based solely on memory that could be somewhat off. However, if we explicitly instruct AI to “answer according to credible historical literature” and set a timeline, AI is more likely to provide accurate responses.

Here are some prompt tactics to help avoid AI hallucinations:

• Clearly Require AI to Cite Reliable Sources:
  • For example, when inquiring about historical events, ask AI to reference authoritative historical documents;
  • When seeking scientific facts, request citations from published research papers;
  • When asking about legal provisions, require references from official legal documents.
• Request Detailed Reasoning Processes from AI:
  • This can enable us to comprehend how AI forms conclusions and assess their reasonableness.
  • For instance, when asking about a mathematical formula, one could request AI to show the derivation process;
  • When questioning code functionality, it helps to ask AI to explain each line’s meaning.
• Explicitly Set Boundaries for AI Outputs:
  • For instance, when querying famous quotes, specify the individual’s name and context;
  • When asking about news events, define the timeframe and relevant keywords.

By implementing these techniques, we can make prompts clearer, more specific, and targeted, thereby guiding AI to produce more accurate and trustworthy content. Nevertheless, prompt engineering is merely a supportive approach; fundamentally addressing AI hallucination issues requires continued efforts across data, models, training methodologies, and more.

Addressing AI hallucinations presents a multifaceted and prolonged challenge; existing technologies cannot entirely mitigate the issue. Ongoing exploration of new methods and techniques, along with stringent regulation and assessment of AI systems, is essential to ensure AI remains secure, reliable, and trustworthy.

AI Hallucinations: An Inescapable Reality

We have identified various techniques for addressing AI hallucinations, akin to “magical weapons” in our struggle against this stubborn adversary. However, the harsh reality is: AI hallucinations cannot be completely avoided.

Theoretical “Constraints”: Boundaries of AI’s Abilities

“Hallucination is Inevitable: An Innate Limitation of Large Language Models” (Xu et al., 2024) — This paper reveals the fundamental reason AI hallucinations cannot be entirely eliminated: the boundaries of AI models’ abilities.

• AI’s Knowledge is Data-Dependent: AI derives its knowledge from training data and cannot exceed the limits of this data, similar to how a student cannot answer questions outside their textbooks.
• Limited Reasoning Abilities in AI: Even with sufficient knowledge, AI’s reasoning abilities are finite; it cannot engage in infinite reasoning or associations like humans do.

Thus, even with continuous improvements to data quality and model optimization, AI will never master all knowledge nor possess limitless reasoning capabilities.

Case Studies Indicating AI’s “Stubbornness”

The AI hallucination cases we discussed earlier indicate AI’s “stubbornness.” Be it the fabricated legal cases from ChatGPT or misdiagnoses by medical AIs, these instances illustrate that even the most advanced AI models cannot entirely avoid errors.

AI Hallucinations: An Objective Limitations

The inevitability of AI hallucinations is an objective reality rooted in the inherent limitations of AI technology, rather than human negligence or error. We must face this reality with greater caution in our approach to AI technology.

• AI is Not Omnipotent: We should not expect AI systems to resolve all issues, nor regard their judgments as absolute truths.
• Exercise Caution with AI Applications: Particularly in critical areas like healthcare, finance, and law, we need to be exceptionally cautious and implement necessary measures to mitigate the risks posed by AI hallucinations.
• Ongoing Improvement of AI Technology: The inevitability of AI hallucinations does not equate to the abandonment of technological pursuit; rather, we must strive to enhance AI technology continually, ensuring its reliability and security for better service to society.

The Other Side of AI Hallucinations: Unexpected Surprises

Despite the myriad risks associated with AI hallucinations, we mustn’t dismiss them altogether. AI hallucinations provide an opportunity for unexpected surprises and may even drive advancements in human society.

AI’s “Creative Sparks”: Igniting Human Creativity

As noted in “Confidently Nonsensical?”: A Critical Survey on the Perspectives and Challenges of ‘Hallucinations’ in NLP” (Narayanan Venkit et al., 2023), AI hallucinations can occasionally yield novel, curious, and even “beyond human imagination” outputs, stimulating artistic creation and scientific research.

• New Avenues for Artistic Creation: AI hallucinations can assist artists in transcending traditional methodologies, allowing exploration of innovative styles and forms of expression.
  • For instance, certain AI artists leverage the hallucination effects of image-generating models, leading to fantastically vibrant creations that captivate visually.
• Fresh Perspectives in Scientific Discovery: AI hallucinations may unveil hidden rules and patterns in data; insights that might go unnoticed by human scientists could carry significant scientific value.
  • For example, when analyzing astronomical data, an AI model might unexpectedly discover unexplained anomalies, prompting scientists to develop new physical theories to account for these findings and advance our understanding.

AI’s “Trial and Error”: Accelerating Technological Progress

The emergence of AI hallucinations reflects the trial-and-error journey of AI technology. Each hallucination signifies a “learning” and “evolution” instance for AI models. By analyzing the causes of AI hallucinations, we can better recognize the models’ limitations and enhance their designs and training strategies, propelling AI’s advancement.

For instance, early machine translation systems often produced nonsensical translations—errors that drove researchers to refine translation algorithms, ultimately leading to significantly improved translation quality. Today, thanks to deep learning, the quality of machine translations has yielded significant enhancements, though AI hallucinations remain an ongoing concern.

AI Hallucinations: A Double-Edged Sword

AI hallucinations wield dual influences: they can serve as motivation for human progress or present unpredictable risks. We must adopt a nuanced perspective on AI hallucinations, embracing the pursuit of AI technology while remaining aware of its potential hazards.

• Leverage Strengths, Avoid Weaknesses: We should harness the positive facets of AI hallucinations to inspire creativity and drive technological advancement. Concurrently, we must research countermeasures for AI hallucinations to minimize associated risks.
• Human-Machine Collaboration: The inescapability of AI hallucinations necessitates human involvement in its technology’s development. We should bolster oversight and guidance on AI systems to ensure that artificial intelligence ultimately serves humanity’s goals and values.

AI hallucinations represent an obstacle in the evolution of AI technology but also an essential path toward a smarter future. Through human wisdom and effort, we can cross this hurdle and reach a harmonious coexistence of AI and humanity.

Dancing with AI: Seeking Opportunities Amid Challenges

AI hallucinations reflect both the light and shadows in the journey of artificial intelligence development. We must acknowledge the associated risks and challenges while recognizing the embedded opportunities and hope. How we approach AI hallucinations will shape the future trajectory of artificial intelligence.

AI Hallucinations: Coexisting Challenges and Opportunities

Reflecting on past content, we understand that AI hallucinations represent a manifestation of AI system limits, and their elimination isn’t feasible. AI hallucinations can mislead users, disseminate false information, and undermine the credibility of AI systems—factors requiring vigilance and preventive strategies.

However, AI hallucinations can yield unexpected surprises, stimulating human creativity, fostering technological advancement, and even paving the way for discovering new scientific truths. AI hallucinations represent two sides of the same coin, possessing both negative effects and positive values.

Human Wisdom: Guiding AI Towards Good

The development of AI technology relies on human intelligence and guidance. In grappling with AI hallucinations, we should not dismiss them merely as technical issues but think about how to propel AI technology’s healthy development from an ethical and societal perspective—ensuring that AI truly benefits humanity.

• Establishing AI Ethical Norms: Clear ethical guidelines must be formulated to steer AI’s research and applications, ensuring that AI systems align with human values and moral standards, thereby preventing AI from being weaponized against humanity.
• Enhancing AI Oversight: We need to strengthen regulation of AI systems, establishing comprehensive evaluation mechanisms that promptly detect and correct errors and biases in AI, preventing severe consequences of hallucinations.
• Improving Public AI Literacy: Enhancing public understanding of AI technologies’ principles and limitations should be prioritized, allowing individuals to rationally view AI hallucinations and avoid being misled by AI.

The Future of AI: Cautious Optimism

The evolution of AI technology presents immense opportunities and challenges for society. We believe that humanity can overcome the challenges posed by AI hallucinations, fully realizing AI’s potential, transforming it into a formidable engine for societal progress.

On the journey toward an intelligent future, we should maintain an optimistic disposition to actively explore AI’s infinite possibilities while being cautious to remain vigilant about its inherent risks. In this way, we can dance harmoniously with AI, creating a brighter future together.

Conclusion

The origin of this exploration stems from discovering a summarization prompt that, given a public account article link, provides a suitable summary. During my research on AI hardware, the output somewhat mirrored the actual content; however, opening the web page revealed notable discrepancies—indicating that it completely failed to access the webpage. In writing this article, I encountered similar issues and pondered if I may have misclassified it under AI hallucination for a few reasons:

• Strategically, summary-type articles, it’s likely that AI does not read the entire article but skims the beginning and end while randomly selecting portions of the middle.
• Using Python’s request library to read webpage speed is inherently slow, whereas large language models can respond remarkably quickly; in reality, many LLMs do not truly search the web.

Considering the explanations above, it resembles more of the developers of large language models being “clever,” placing the blame on “AI hallucinations.”

Gemini	Actual Page
The page doesn’t mention PaLM at all, yet it’s not denying its lack of internet access, instead earnestly concocting a narrative.

References

• Navigating Hallucinations for Reasoning of Unintentional Activities
• Exploring the Mechanisms and Countermeasures of AI Hallucinations
• Analysis of Data Bias and Its Impact on AI Systems
• The Impact of Data Quality on AI Hallucinations
• Application of AI in the Medical Field and Its Hallucination Issues
• Application of RAG (Retrieval-Augmented Generation) Technology in Reducing AI Hallucinations
• Mechanisms of Non-Factual Hallucination in Language Models
• “Confidently Nonsensical?”: A Critical Survey on the Perspectives and Challenges of ‘Hallucinations’ in NLP
• Sorry, Come Again (SCA) Prompting – Enhancing Comprehension and Diminishing Hallucination with [PAUSE] -injected Optimal Paraphrasing
• HILL: A Hallucination Identifier for Large Language Models