Graph RAG: when relationships are the key evidence

In the field of artificial intelligence applied to business, retrieval-augmented generation (RAG) has become established as a technique for agents to access external knowledge. Vector search, based on semantic similarity, is the most widespread method. However, not all questions are answered with similar text fragments. When the answer depends on relationships between facts, similarity alone is not enough. That is where Graph RAG comes into play.

The limit of vector search

Vector search is effective for finding documents or records whose meaning resembles the query, even with different words. For example, a question about "canceling a subscription" can retrieve a support article about "account cancellation". It works well with documentation, help articles, or product descriptions, where the answer is usually in one or two passages. Similarity has a clear role: ranking the likely evidence. But it does not establish ownership, dependency, authorization, or the scope of a policy. Two records can be semantically close and have no operational connection. Therefore, hard metadata constraints (such as tenant boundary, effective date, or account identifier) must be applied independently of the vector ranking. A similarity score is an opinion about relevance; a tenant boundary is a condition that the system must fulfill.

Questions that require relationships

When an agent leaves a self-contained knowledge base, questions arise about connected systems: which services depend on this component? Who can approve an exception for this account? Which customers are affected by this deployment? What controls apply to the data in this environment? A typical example is that of a vulnerability: a package advisory identifies a library; a deployment record says that a service uses that library; a service catalog connects the service with a customer's environment; and a contract connects that customer with a notification policy. The answer requires four records and three relationships. The text can describe each step, but an agent that assembles the chain implicitly can link the vulnerable library with an unintended service version, or associate a shared service with the wrong customer. It can also retrieve an expired policy or miss an updated dependency after the documentation is published.

A graph makes those connections explicit by turning each record into a node. Typed and directed edges record that a service USES a library or SUPPORTS an environment. A GOVERNED_BY edge links the contract to the policy. Each connection carries its source and owner, with confidence scores or effective dates when they matter. The same four records are retrieved as similar passages above and connected as typed and traceable edges below.

Graph RAG: a term with nuances

Graph RAG is an overloaded term. Microsoft's GraphRAG proposal uses an LLM to extract a knowledge graph from unstructured text, builds a hierarchy of communities, and supports queries at both corpus and entity level. This article refers to something more specific: a graph built from relationships that your systems already record, where each edge traces back to an operational source instead of being inferred from a model's interpretation. The approach combines content retrieval with traversal over known relationships. A practical flow begins by finding candidate entities and passages for the question. The application resolves those candidates to specific records, follows only the allowed relationships, and then retrieves the source documents needed to explain the result. That resolution step is the hardest: a question about "the payment service" must land on the correct record when the catalog lists three services with similar names, and an incorrect match can affect all subsequent hops. When the match is ambiguous, the application should show the candidates instead of choosing the closest name.

The graph answers "what is connected?". The source material answers "what does the policy say?". You need both.

Implications for SMEs and IT teams

For an SME, adopting Graph RAG does not mean building a graph of the entire company. The advice is to start with the types of relationships that answer an important question. Mapping the whole company is a long detour with no guarantee of a useful destination. If your business depends on dependency chains (suppliers, customers, contracts, policies), a lightweight graph can make a difference. Vector search remains useful for locating documents and records, but add a graph when similarity alone cannot prove why one fact applies to another. At ForgeNEX we recommend first evaluating whether your critical questions need relational evidence. If so, identify the key relationships, ensure that each edge has a traceable source, and combine content retrieval with graph traversal. Do not replace vector search: accompany it. The key is that the graph answers "what is connected" and the text answers "what does the policy say". You need both.

If you want to delve deeper into how to fit this architecture into your SME, you can consult our previous article Graph RAG: When Relationships are the Evidence, where we explore its impact on cybersecurity and DevOps. You may also be interested in AI accelerates exploits: your CVE spreadsheet is no longer useful, because vulnerability management is a clear use case for Graph RAG.

Source: The New Stack. Analysis and adaptation: ForgeNEX.