Building an AI agent that can plan, reason and take action usually requires your domain’s knowledge to be structured using a vector database, a graph database, an ontology – or a combination of all three. The decision about which to choose depends on how capable and trustworthy your AI service needs to be, and what resources you have to implement and run it.
What does a vector database do?
Vectors arrange and store knowledge according to things that have similar meanings, rather than just using keywords. So for example, if a medical AI agent is asked about ‘heart medication’, the vector database can surface documents about beta blockers or statins, even if the documents don’t literally contain the words ‘heart medication’.
Vectors help with improving an AI agent’s recall, especially in domains with large quantities of unstructured data. They are also relatively convenient to implement. However, using the previous example, a vector on its own cannot reliably explain what a heart medication is, how it works, or how it connects to other concepts.
A 2025 arXiv survey explored the performance of vector databases, including the use of retrieval augmented generation (RAG).
What does a graph database do?
Whereas vectors can provide fast if imprecise recall, graph databases provide the basis for structured reasoning. With graph, an AI agent can automate critical or high value tasks that demand accuracy and reliability.
Graph databases are based, firstly, on entities (nodes), which are individual examples within a category. An entity might be a person’s name in the customer category of a garage database, or a specific car within the vehicle category.
Second, a knowledge graph maps out how entities relate to one another (edges). This means they can link a customer to a car in this case, and also to the mechanic performing the repair.
One benefit of a graph database is that it can reduce the errors and hallucinations which are a feature of large language models (LLMs) and vectors. It allows the agent’s reasoning to be explained (XAI) by tracing its source data, which is known as data lineage. Graph also enables the AI agent to work within the rules and constraints set by the domain, often in the form of an ontology.
What does an ontology do?
Ontologies sit above a vector and graph database. They provide structure to govern the domain’s knowledge contained in the graph. An ontology can be used to impose high level rules such as ethical or legal guidelines, providing AI agents with the contextual understanding they need to be domain intelligent.
Ontologies also use controlled vocabulary, which sets out the precise meanings of words in a domain. This enables the domain’s knowledge to be shared across systems and teams with less chance of misunderstandings or errors. Ontology standards play an important part in helping systems to work smoothly together, known as interoperability.
Which database is best for my AI app?
Vectors are useful if an agent’s primary job is recall, such as suggesting movies based on similar films that a user has viewed. Vectors are also used in cybersecurity for identifying suspicious patterns of behaviour. They are less suited to applications that require high levels of accuracy and reliability.
Graph databases work best with interconnected data and complex queries, as seen in supply chain logistics, for example. They need careful monitoring and maintenance, however, and are not as simple to scale as vector databases.
Does my database need an ontology?
Ontologies are needed wherever critical data is widely shared across domains, or unified to provide intelligent systems with advanced capabilities. Medicine, finance and defence applications have traditionally relied on ontologies – although their use is spreading as more sectors with complex data begin to roll out intelligent systems.
Bear in mind however that agreeing on the definition of an ontology requires domain experts to reach consensus first, which can require the assistance of a modern ontology as a service platform.