# RAG vs fine-tuning: a decision guide for business knowledge

Canonical: https://gptnavi.com/guides/rag-vs-fine-tuning
Sources checked: 2026-10-22

## Should a business use RAG or fine-tuning for its knowledge base?

Use retrieval augmented generation when an answer needs current or private source material. Consider fine-tuning when the model needs more consistent behavior on a defined task. First test whether a clear prompt and supplied context solve the problem. Retrieval and tuning can be combined, but neither substitutes for access control or evaluation.

## Diagnose a missing fact before changing the model

Take a failed question and manually supply the correct policy paragraph. If the model now answers accurately, the first problem is likely finding and presenting evidence. Inspect ingestion, chunk boundaries, date filters, and the query itself. If the answer is still wrong with the exact source in view, investigate instruction clarity and task difficulty. This simple intervention prevents an expensive training project from becoming the default response to every failure.

## Separate retrieval success from answer success

For each question, record the required source passage, whether it was retrieved, whether the answer used it correctly, and whether the citation actually supports the claim. A fluent answer can hide a retrieval miss. Include questions for which the library contains no answer and questions whose answer changed after a policy update. Keep the test questions separate from examples used to adjust prompts or train a model.

## Test access boundaries as a release condition

Create two test identities with different document permissions and ask the same question. An answer for one identity must not reveal text, titles, or inferred facts from the other's restricted material. Re-run after changing access rights and deleting a document. The scorecard below records this as a blocking test even if ordinary answer accuracy is high. The checklist is an authored evaluation protocol, not a claim that any named product passed it.

## Proposed pilot

1. Prepare 20 questions with expected sources, including five unanswerable cases.
2. Compare a supplied-context baseline with your retrieval system using the same questions.
3. Repeat under two permission profiles, then run the evaluation scorecard.

## Sources

- [Microsoft Learn: retrieval augmented generation and indexes](https://learn.microsoft.com/en-us/azure/foundry/concepts/retrieval-augmented-generation): RAG grounds answers in retrieved data; fine-tuning addresses behavior or task performance.

This is a decision framework, not a vendor performance benchmark.