LLM

A Large Language Model (LLM) is a probabilistic neural network system capable of processing and generating natural language, situated within the AI landscape alongside rule-based Symbolic AI and Neurosymbolic AI hybrid approaches. In agentic system architectures, LLMs serve as reasoning and language components but carry inherent limitations due to their probabilistic nature.

Probabilistic Nature as a Core Limitation

Frank Coyle (UC Berkeley) consistently foregrounds the probabilistic character of LLMs as a source of unreliability in agentic and domain-constrained applications. When domain logic must be enforced precisely — for example, ensuring an order is refunded only once or that a payment goes only to a buyer — expressing those constraints purely through natural language prompts is fragile: "when you're in the pure text world, this can get funky because the LLMs are again probabilistic". Similarly, prompt engineering in English cannot reliably prevent domain errors: "it's very tricky to do that in English. The LLMs are again probabilistic and return some crazy stuff".

Relationship to Symbolic and Neurosymbolic AI

Coyle positions LLMs within a broader taxonomy of AI approaches. Symbolic AI — which subsumes rule-based systems — is framed as an alternative paradigm to LLMs, one that offers formal, deterministic representations rather than probabilistic outputs: "symbolic AI, which rule-based systems come under that category". Neurosymbolic AI is presented as a convergent approach that combines the strengths of both: "the convergence of something that is probabilistic, the agents, the LLMs, with the more formal representations that you have with ontologies", and is characterized concretely as "neural networks tied into symbolic AI".

Proposed Mitigation: External Ontology Validators

Rather than abandoning LLMs, Coyle advocates placing an ontology-based validator outside the LLM in an agent tool-use loop. OWL functional and disjoint properties can enforce domain constraints that natural language prompts cannot reliably express, with the validator acting as a deterministic check on the LLM's probabilistic outputs. This architectural pattern — ontology as external constraint layer rather than internalized prompt instruction — is Coyle's central proposal for making LLM-based agents more reliable in structured domains.