ragornot

RAG or not — decide.

Do you actually need RAG?

Eight quick questions, one clear recommendation — RAG, lexical search, long-context, fine-tuning, or nothing at all — each with the real cost and energy tradeoff. The logic is a fixed decision tree you can inspect, and it's happy to tell you that you don't need RAG. Curious how the options compare live? Run the Benchmark.

0/8
1.What do you actually need out of this?

Be honest about the end goal — it's the biggest fork in the road.

2.Does answering require private, proprietary, or frequently-updated data the model wasn't trained on?
3.Do you need citations or verifiable grounding?

i.e. users must be able to trace an answer back to a source.

4.How much reference material is relevant to a typical question?
5.How often does that reference material change?
6.Is very low latency (sub-second) critical?
7.Is cost-per-query tightly constrained?

High volume, thin margins, or a hard budget per request.

8.Roughly how many queries per day?

Answer all 8questions to see a recommendation. There's no wrong answer — the tool is happy to tell you that you don't need RAG, or don't need AI at all.

Do I need RAG? The short answer

RAG (retrieval-augmented generation) is worth it when your answers must reflect private, current, or large and changing data and you need to cite sources. It isn't always the right tool: if the relevant knowledge fits in a prompt and rarely changes, long-context or a good system prompt can be enough; if you just need to find a document, lexical (BM25) search is cheaper and faster; and if you need exact lookups of structured records, a database beats an LLM.

Answer honestly about your use case. If you need exact lookups of structured records (IDs, prices, rows), use a database — not AI. If you need to find the right document or passage, use lexical (BM25) search. If you need generated natural-language answers: choose RAG when the knowledge is private, current, large, or changing, or when you need citations; consider fine-tuning — or a strong system prompt — for a small, stable, high-volume workload that doesn't need citations; and long-context is fine when everything relevant fits in a prompt, rarely changes, and volume is low.

The cost and energy figures behind these tradeoffs are sourced on the methodology page, and you can see the modes compared live on the benchmark.

RAG or not — FAQ

Do I need RAG?
Use RAG when your answers must reflect private, current, or large/changing data and you need to cite sources. If your knowledge is small and rarely changes, long-context (pasting it into the prompt) or a good system prompt can be enough. If you only need to find the right document, lexical (BM25) search is cheaper and faster. If you need exact lookups of structured records, use a database — not an LLM.
RAG vs long-context — which is cheaper?
Retrieval is far cheaper. Stuffing a long document into the context window costs roughly 20–24× more than RAG and about 100× the energy (~40 Wh vs ~0.3 Wh per query), because you pay for those tokens on every call. Long-context is fine at small scale; switch to RAG as your documents or query volume grow.
RAG vs fine-tuning — when should I fine-tune?
Fine-tuning adapts a model's style, format, or task behavior and can shrink prompts at high volume, but it doesn't cite sources and must be redone whenever your data changes. For fresh, citable knowledge, RAG stays current without retraining. Reach for fine-tuning only for a small, stable, high-volume workload that doesn't need citations.
When is RAG not worth it?
When your knowledge fits in a prompt and rarely changes (use long-context), when you just need to locate a passage (use lexical search), when you need exact structured lookups (use a database), or when you don't actually need generated answers at all. RAG adds a retrieval system and a per-query LLM cost — worth it only when grounding on your own, changing data is the point.