GraphRAG uses a lengthy prompt with examples to make the LLM identify all entities (name, type, description) from given entity types, then identify pairs of related source/target entities with a relationship description and an integer strength score from 1 to 10, which is later used when merging the subgraphs.

A knowledge graph is a knowledge base that uses a graph structure data model to represent and operate on data, storing interlinked descriptions of entities (objects, events, situations, or concepts) while encoding the free-form semantics or relationships underlying those entities.

Representing unstructured data as a graph is itself a good visualization aid because humans understand interconnected ideas and relationships, allowing one to see larger concepts, offshoots, and the overall structure of the data.

Traditional RAG benefits from simpler implementation/deployment, suits straightforward retrieval tasks, handles structured text data well, and has lower computational overhead, but loses structural information when chunking, can break related content, has limited ability to capture relationships, and struggles to reason across disparate facts—risking incomplete or fragmented answers.

Knowledge graph RAG addresses the limitation of semantic similarity retrieval by introducing a structured hierarchical organization that lets the system start at and traverse different paths along the graph to retrieve the right information more effectively.

In a local search for tools for model initialization, GraphRAG found relevant information beyond the document's explicit tools list (Hugging Face, PyTorch, TensorFlow), surfacing Nvidia Nemo and Optimum by Hugging Face through graph traversal—demonstrating it can pull related entities the explicit text section did not list.

Communities create a structured way of understanding different levels of granularity in the knowledge graph, ranging from broad overviews of concepts at the top level to very detailed local clusters at lower levels, aiding organization and navigation during retrieval.

Drift search (Dynamic Reasoning and Inference with Flexible Traversal) combines global and local search by using hypothetical document embedding (HyDE) to start at the global level, generating an initial answer plus follow-up questions, then running local searches for each follow-up to create a pseudo-refinement loop—with relevance-scored follow-ups guiding traversal—aggregating all intermediate responses via map-reduce into a final ranked answer.

GraphRAG's indexing data flow loads and splits text into chunks, extracts entities and relationships with an LLM per chunk, merges and summarizes the per-chunk subgraphs, creates embeddings for the nodes, performs community detection, and then generates Community Reports (summaries of information within communities) which are also embedded.

After building the basic graph, GraphRAG applies the Leiden algorithm to group nodes into hierarchical communities at different levels of granularity, and embeds nodes with the node2vec algorithm, so that communities capture obvious connections while embeddings capture subtle patterns.

When the same RAG-vs-fine-tuning-vs-PEFT query was run through a simple ChromaDB vector retrieval setup (chunked identically to GraphRAG), it returned a sparse, less specific list lacking detailed supporting arguments, inferior to GraphRAG's global, local, and drift responses.

Knowledge graphs mirror human cognition by explicitly mapping relationships between objects, concepts and ideas through both semantic and relational connections, paralleling how brains understand information not as isolated facts but as interconnected concepts.

Knowledge graphs are not a new concept—they were introduced by Google and power much of the advanced Google search used today—but traditionally their creation was very resource intensive, requiring either manual building and curation or conversion of relational databases into graph structures.

As you scale a knowledge base by adding topics or more text on the same topic, traditional RAG gets bottlenecked at the retrieval step because retrieving just N relevant chunks based on the query misses connections between similarities across the data and broader themes the information conveys.

Traditional retrieval augmented generation works by chunking documents, embedding them into a vector database, and at runtime embedding the user query, running a semantic similarity calculation to retrieve the top N relevant chunks, and passing those plus the original query to the language model as context.

With the advent of large language models, knowledge graphs can now be created automatically by using LLMs' reasoning capabilities to read unstructured text (Word documents, PDFs, blog posts) to identify entities, infer relationships, and structure information—and LLMs can also update and maintain those graphs.

The three main components of knowledge graphs in GraphRAG are entities (distinct objects/persons/places/events/concepts extracted from text chunks, forming nodes), relationships (connections between entities, also extracted by LLM analysis), and communities (clusters of related entities identified through hierarchical community detection, typically the Leiden algorithm).

When merging per-chunk subgraphs, entities with the same name and type are merged by combining their descriptions into an array, relationships with the same source and target are likewise merged into an array, and then these arrays are summarized one final time by an LLM into a single explanation given all context across chunks.

GraphRAG by default splits text into chunks of 1,200 tokens with 100-token overlaps, which for the fine-tuning guide produced about 53 chunks.

Global search retrieves the most similar community reports at a specified hierarchical level, then uses a map-reduce style approach: an LLM extracts key points from each report with relevance scores, ranks and filters out the least relevant points, and returns the most important points as context for generation.