Hallucination
person concept tool org talk claim — click a node to jump to its page; hover an arrow for the relation
Hallucination refers to the tendency of large language models to generate outputs that are factually incorrect, fabricated, or ungrounded — a phenomenon widely treated as a reliability problem in AI systems.
Hallucination as Feature, Not Bug
Frank Coyle (UC Berkeley) offers a counterintuitive reframing: rather than treating hallucination as a defect to be eliminated, he argues it is an intrinsic and fundamental property of how large language models operate. Probabilistic text generation — the same mechanism that produces hallucinations — is constitutive of LLM capability itself, not an implementation flaw. In his words: "People worry about hallucinations, but that's the feature. That's actually a feature of large language models." ↗
This position implies that efforts to fully suppress hallucination may be misguided or even self-defeating, since the stochastic generative process underlies the model's broader usefulness. The argument surfaces in the context of Coyle's case for ontologies in agentic systems: if LLMs are inherently probabilistic and unconstrained in their outputs, external knowledge structures become necessary to ground and validate what models produce.
Context in Agent Engineering
The reframing has practical implications for agentic systems design. Treating hallucination as a permanent characteristic of the underlying technology — rather than a bug awaiting a fix — shifts the engineering question from "how do we stop the model from hallucinating?" to "how do we build systems around a probabilistically generating component?" This motivates architectural choices such as retrieval augmentation, evals, and ontology-backed validation layers as complements to LLM generation rather than replacements for model improvement.
Points of Disagreement
The broader AI engineering community largely treats hallucination as a problem to be mitigated through techniques such as retrieval-augmented generation, fine-tuning, and chain-of-thought prompting. Coyle's framing — that hallucination is the feature — stands in tension with this consensus, though the graph currently contains no speaker directly rebutting his position.