A large language model (LLM) benefits most from an ontology in domains where hallucinations and errors can have serious consequences, or where compliance rules demand explainable outputs.
For AI applications using natural language generation where accuracy is less important, specialised LLM training and techniques such as hybrid RAG can reduce hallucination or semantic drift to an acceptable level.
What is an LLM?
LLMs are AI models that are trained on vast data-sets to understand, generate, and process human-like text. Using statistical probability and context, they predict what word ought to follow another. While LLMs excel at fluency, their outputs are not usually verified, consistent or accountable to a high standard.
What is an ontology?
When an enterprise’s knowledge is mapped and organised in a way that machines can read, and stakeholders can agree on, an ontology is born. It sets out what key concepts mean, using controlled vocabulary, and how these concepts relate to one another. Unlike a basic taxonomy, which only defines what things are rather than how they are related, an ontology captures multiple dimensions of meaning and detail. This allows intelligent systems to compare, combine, and reason with data.
Ontologies can be used to constrain LLMs and knowledge graphs, operating as semantic guard-rails. This can prevent LLMs from making things up (hallucinating), or drifting from what important concepts are supposed to mean while performing a task.
Why are ontologies back in the conversation?
Until recently, ontologies were often dismissed by software engineers as too fragile, too difficult to scale, or too onerous to develop and reach consensus on. Scientific research and advanced manufacturing were the exceptions; for these sectors, ontologies were indispensable to preserve shared meaning for unifying data and multilingual collaboration. This justified the time and effort required to develop and maintain ontology standards.
In the age of AI, however, ontologies help govern everything from sophisticated military systems to drug development and digital twins. Microsoft and Palantir now see grounded intelligence, based on ontological structures, as key to delivering on the productivity promise of AI.
At the World Economic Forum in Davos, on Jan 20, 2026, Palantir CEO Alex Karp said: “If you just buy LLMs off the shelf and try to do any of these things that are regulated, it’s not precise enough. What you’re going to see, especially in America, is people trying to do something like Ontology by hand. Once you build a software layer to orchestrate and manage the LLMs in a language your enterprise understands, you actually can create value.”
What alternatives to ontologies are there?
Techniques such as retrieval augmented generation (RAG) have been applied with some success to improve the contextual awareness of LLMs and enhance the quality of their outputs. Precise prompting can also help. In domains where trust and interoperability are essential, specialised LLMs such as Claude for Life Sciences are being intensively trained in narrow verticals to improve their accuracy and compliance.
However, in many cases these specialised models are linking up to existing ontologies, which suggests that setting loose an LLM on an important knowledge base without ontological guardrails, carries risks that few regulated domains are willing to countenance.
Where is the evidence that LLMs lack grounding?
A 2025 benchmark study by Zhang et al. makes the case that LLMs exhibit significant limitations in ontological reasoning and learning when evaluated across a broad benchmark of symbolic tasks. This suggests that ontologies provide essential grounding for structured understanding.
Another 2025 study by Nananukul et al. addresses the challenges of using LLMs for high-assurance reasoning and how these can be addressed by grounding the original natural language problem into a dual neuro-symbolic context.
What kinds of ontology tools are out there?
In general, ontology tools divide into three types. There are paid-for enterprise solutions and consultancy packages that are often focused on one or two domains such as healthcare or government. Then there are free tools such as Protégé that have been around for a decade or more. They tend to require specialist skills in ontology engineering.
Recently, a third category of ontology tool has emerged. Ontology as a service (OaaS) platforms and semi-automated ontology generators claim to make the process of initiating and building a working ontology more practical for non-technical people – and less expensive than enterprise solutions. Some include consensus-building approaches such as the Delphi Method.