Retrieval-augmented generation

Retrieval-augmented generation, usually shortened to RAG, means fetching relevant documents before answering rather than relying on what the model memorised during training.

In practice: Your question is used to search a document store, the top results are pasted into the prompt alongside your question, and the model answers from those. This is why a tool can answer questions about your own files without having been trained on them.

Why it matters when picking a tool

Any tool that claims to answer questions about your documents is doing this. The differences that matter are how it splits documents, whether it shows which source it used, and what happens when the search finds nothing relevant. A good tool says it does not know; a bad one answers anyway.