A company updates its support policy, but its chatbot keeps quoting the old version. Someone asks which clause in a service agreement covers weekend work and receives a convincing answer with no page reference. Everything sounds reasonable. Nobody can check where the claim came from.
This is the kind of problem that makes retrieval-augmented generation worth considering. Language models can explain things fluently, but company records change, exceptions matter, and different people may be allowed to read different files. Training a model once and expecting it to remember every document forever is an awkward way to manage that knowledge.
What exactly is RAG?
RAG stands for Retrieval-Augmented Generation. It combines document search with language generation. Before a model writes an answer, the application retrieves potentially useful passages from a collection of documents. It then supplies those passages to the model as context for the answer.
The key distinction is that the information does not need to have been part of the model's training data. You can index an updated product specification, a revised contract or an internal handbook, then make that material available to a question-answering application under the appropriate access rules.
The approach was described in a 2020 research paper by Patrick Lewis and colleagues. Since then, RAG has become a broad term. Some products attach a search result to a prompt and stop there. Others maintain document histories, combine multiple search methods, enforce permissions and check whether an answer cites the sources actually retrieved.
A source reference is particularly useful. It gives the reader a way to inspect the relevant evidence instead of relying solely on the model's confidence. That still does not guarantee a correct interpretation. A model can cite the right paragraph and misunderstand its meaning.
What happens between a question and an answer?
Consider a hypothetical question: “Does our maintenance agreement include support at weekends?” The work starts well before anybody types that sentence.
First, the application collects approved documents, extracts readable text and splits it into manageable pieces, usually called chunks. Each chunk retains its origin. The system may also compute an embedding, a numerical representation used to compare the meaning of passages and questions. Search indexes make these chunks retrievable later.
When the question arrives, the application searches the permitted collection, ranks matching passages and constructs a limited context. The language model sees both the question and those passages. It drafts its answer, and the surrounding application may reject it if required citations are missing or invalid.
The details matter. An agreement may distinguish standard support from emergency on-call coverage. A simple keyword search for “weekends” might overlook sections referring to Saturdays, public holidays or out-of-hours duty. A purely semantic search might identify a relevant-sounding clause from the wrong agreement. Retrieval must work across both wording and meaning, inside the correct document collection.
If the right passage never reaches the model, even excellent generation cannot reliably recover it.
Building a RAG system: the decisions a prototype doesn't reveal
It is easy to demonstrate RAG with a handful of clean PDFs. Building a service for changing, sensitive documents calls for a different level of care.
Start with questions rather than technology
“Let users ask anything about our business” is not a useful specification. Gather the actual questions people need answered, find the documents that should support those answers and record what a correct response would look like. For a support team, that might include service hours, excluded work, contractual response targets and differences between customer plans.
Add deliberate no-answer cases. What should happen when the requested information is absent, an agreement has expired or the user lacks permission? A RAG system needs to handle those cases without filling the gaps with plausible fiction.
Ingest source material carefully
PDFs, web pages, office documents and databases do not provide equally clean text. A table extracted from a PDF can lose the association between a value and its column. A web page may contribute navigation labels and cookie notices that have nothing to do with its subject. Scanned files may require a separate recognition stage.
A source adapter should know how to fetch or receive approved material, normalize it and preserve useful metadata: title, original location, version and access information. Keeping this adapter separate from the retrieval engine is often the cleaner design. The core search service should not have to know the peculiarities of every website or storage platform.
Split without losing meaning
Chunk size affects which evidence will be available at question time. Cut a contractual provision at the wrong point and its exception may end up in another chunk. Store an entire lengthy manual as one passage and it becomes difficult to identify the relevant section.
Good segmentation respects headings, paragraphs and tables where possible. Location metadata matters just as much. “Page 18, section 4.2” gives someone a much better verification path than “found in policy.pdf”.
Combine precise matching with semantic retrieval
Two search techniques solve slightly different problems. Lexical search matches words and expressions; it is useful for model numbers, legal references and exact product names. Vector search compares embeddings, helping retrieve related passages that use different vocabulary.
Combining them is often worthwhile. If a user asks about part number AX-204, a search engine should not bury the exact match under passages that are semantically similar but concern another component. Conversely, an employee who asks about “working outside office hours” should be able to find a document that uses the term “on-call duty”.
The embedding model, document language, indexing policy and ranking strategy should be tested against representative questions. Buying a vector database does not settle the question of relevance.
Make the model accountable to the evidence
The retrieved passages must fit within an explicit context budget. A generation policy then decides whether there is enough supporting material to answer. When the available evidence is weak, the system should be allowed to abstain rather than manufacture a confident explanation.
Retrieved documents are also untrusted input. A malicious instruction embedded in a document should remain document content, not acquire authority over the assistant. Separating system instructions, user requests and source text is a necessary part of the design.
Keep access control in the retrieval path
In a multi-customer application, permission checks cannot be a cosmetic filter applied after searching the entire database. A query must operate within the caller's authorized scope from the start. Additional checks in the database provide a separate line of defence.
Document updates require similar discipline. When a policy is revised, the active version must be unambiguous. Historical copies may remain useful for audits, but they must not silently compete with the current policy when somebody asks what applies today.
Evaluate retrieval and refusal, not just writing quality
A fluent answer is a poor benchmark. More revealing questions include: did the correct passage appear in the top results? Was it ranked highly enough to reach the model? Did the query expose material from another customer's collection? Did the system decline a question whose answer was unavailable?
Build an evaluation set with expected evidence, near-duplicate documents, obsolete revisions and intentionally unanswerable questions. Repeat those checks after changing chunking, embeddings or ranking. This work is less visible than the chat interface, but it determines whether a deployment is dependable.
Where RAG helps, and where it doesn't
A business with a small, stable set of structured records might be better served by a normal database query or search interface. RAG introduces operational work: ingestion, storage, indexing, model calls, monitoring and maintenance. It is not automatically the cheapest answer to every information problem.
It becomes more attractive when useful knowledge is scattered across lengthy, frequently revised material. Technical manuals, internal procedures, large product catalogues and collections of historical publications are all plausible candidates. A public institution might use a document assistant to help people locate regulations and publications, while retaining links to the official text.
The limitation should be visible to visitors. If the institution's documents do not specify an opening time or a legal requirement, the chatbot should not invent one. For consequential questions, the source itself remains the authority.
RAG is also different from fine-tuning. Fine-tuning changes a model's behaviour through training. RAG retrieves external knowledge during a request. They can be combined, but keeping a changing set of documents up to date is usually a retrieval and information-management problem before it is a training problem.
Why we built PerseoRAG
When developing document-based applications, the same engineering problems keep coming back: ingestion, document versions, retrieval, permissions, evidence formatting and citation checks. Implementing that logic afresh for each customer makes security mistakes more likely and complicates testing.
We built PerseoRAG to make those parts reusable. The open-source 1.1.0 release is a Python library with FastAPI orchestration, PostgreSQL storage and pgvector support. It is available under the Apache 2.0 licence.
Its boundaries are deliberate. PerseoRAG is not a finished chatbot, a universal crawler or a bundle tied to one particular language model. It also does not implement the complete user-authentication system for every application that might use it. The integrating application supplies the relevant connectors, model providers and user-specific authorization rules.
That separation has practical value. One application can present a public-facing interface; another can serve internal documents. Their user experiences and access policies differ, while the retrieval and evidence-handling contracts remain consistent.
What PerseoRAG does under the hood
Document history without duplicate ingestion
Documents are stored inside an authorization scope and a collection. After normalization, their content and metadata are fingerprinted. Importing an unchanged document again is idempotent. A changed document produces a new immutable version, while a separate document head identifies the version that is currently active.
Default retrieval joins through that active head. It does not mix an obsolete procedure into today's answers merely because an old revision still exists in storage. Keeping history is useful for inspection, but selecting the current version is a separate responsibility.
Hybrid retrieval with PostgreSQL and pgvector
PerseoRAG uses PostgreSQL full-text search for lexical matching and pgvector for dense embedding search. It combines their rankings using Reciprocal Rank Fusion. This method considers where a passage appears in each result list rather than assuming that lexical and vector similarity scores share a common scale.
Version 1.1 uses exact vector search. Approximate-nearest-neighbour indexing, where appropriate, is left to the deployment because collection size, embedding dimensions and performance constraints vary. A library used across different projects should not pretend that a single index configuration is universally optimal.
The 1.1 release also supports distinct document and query embedding methods for providers that require them, while keeping compatibility with the earlier, simpler embedding interface.
Isolation enforced before ranking
The hosting application resolves the authorized scope from the authenticated request. The caller cannot simply choose another scope by putting an identifier into the query body. Candidate retrieval runs inside that scope, with PostgreSQL Row-Level Security providing an independent enforcement layer.
This avoids the dangerous pattern of ranking material from every customer and removing unauthorized records only afterwards. The boundaries apply while potential answers are being selected.
Grounded context, abstention and citation checks
Retrieved passages become a bounded context containing source references, chunk identifiers, document versions and local citation IDs. The system filters duplicates and rejects evidence that crosses scope boundaries.
An evidence policy runs before text generation. It can decline to generate when the available material is insufficient. After generation, PerseoRAG validates cited identifiers against the evidence actually supplied. An unknown citation, missing citation or empty response is not accepted as a grounded answer.
These mechanisms are useful safeguards, but they cannot prove that the final interpretation is factually correct. Legal, medical and contractual applications still need domain-specific validation and a clear human review process.
PerseoRAG includes retrieval evaluations such as Recall@K, Mean Reciprocal Rank and cross-scope leakage checks. The synthetic fixtures detect regressions in the library; they are not blanket accuracy claims for a customer's documents.
How it relates to Gnosis and Sherlock
There is a distinction between a RAG engine and an application built around one. Gnosis and Sherlock are not components inside PerseoRAG. Applications can integrate a reusable retrieval engine as a dependency and provide their own workflows, interfaces and permission systems. That does not turn those applications into modules of the engine itself.
Similarly, the library's provider-independent design allows an integration to choose local or external models. Data residency and confidentiality depend on the actual infrastructure and providers used. Installing PerseoRAG does not, by itself, guarantee that data never leaves a customer's server.
The sensible way to begin a RAG project
It is difficult to quote a meaningful fixed price without examining the source material. A collection of publicly accessible pages is different from thousands of confidential, badly scanned documents with individual permissions and regular updates. The effort lies in preparing, securing and maintaining the knowledge base as much as in connecting a language model.
We would begin with a representative sample of documents, questions, expected evidence and users with different permissions. That gives us something to measure before choosing hosting, models or the interface visitors will see. A small pilot that correctly refuses unsupported questions is more useful than a polished demonstration that gets the exceptions wrong.
At PerSeoDesign, our work on web applications and document retrieval meets at exactly this point. If you are considering a knowledge assistant for a website or an internal team, the most useful first conversation is about which documents people should be able to consult, who may see them and how they will verify an answer.
The PerseoRAG source code and technical documentation are public. For an integration discussion, use our contact page.