RAG or not — compare. learn. decide.
Live Benchmark
Run 7 real AWS-docs queries through four retrieval strategies and get empirical numbers on cost, latency, answer quality, and carbon — so you can decide whether RAG is worth it for your organisation.
Advanced: Benchmark on your own docs
Upload your own PDFs or TXTs and benchmark retrieval against them with up to 10 iterations. Requires an access key — request one below if you don't have one.
Request access to advanced benchmark
Leave your email and we'll send you an access key to run benchmarks on your own documents. Your email is used only to send the key — we don't share it or use it for anything else. See our Privacy Policy.
Mode Comparison (illustrative — run benchmark for live data)
Numbers below are representative of the demo corpus runs. Hit “Run Benchmark” above to replace these with your own live results.
RAG vs Flat baseline: adds $0.00041 per query in LLM cost — free retrieval modes cost nothing beyond Lambda compute.
| Mode | Relevance (proxy)Retrieval relevance — a lexical (BM25 / query-term) match confidence for how relevant the retrieved chunks are, NOT whether the final answer is correct. For live runs: avg_confidence × 100 (0–1 → %). LLM-only has no retrieval step so it shows N/A. Static numbers are illustrative from demo corpus runs. End-answer evaluation (correctness, faithfulness) is a planned future metric. | Latency measuredEnd-to-end time measured in the browser with performance.now() around the full fetch() call — includes network round-trip, not just Lambda execution. | Cost / query measureddata.llm_stats.cost_usd from the Lambda — Bedrock token billing cost. Exactly $0.00000 for Flat and Hierarchical (no LLM call). For LLM-only and RAG, reflects input + output token pricing. | What it means for your org |
|---|---|---|---|---|
| Flat (Lexical) | 71% | 6ms | $0.00000 | Fast and free, but a single global ranking can surface off-topic chunks from unrelated docs. |
| Hierarchical | 79% | 11ms | $0.00000 | Narrowing to top documents first improves precision over Flat at a small extra compute cost. |
| LLM-only | N/A | 690ms | $0.00013 | No grounding — fluent, but frequently generic or wrong on service-specific details. |
| RAG | 91% | 860ms | $0.00041 | Retrieval grounds the answer before generation — highest retrieval relevance, but the slowest and priciest mode. |
Impact Analytics
How we calculate this →Sourced per-query estimates — each figure links to its coefficient. Run the benchmark above for your own live numbers. Latency, tokens, and cost are measured (from the Bedrock API); energy, water, and CO₂ are modeled from token count (~0.6Wh per 1,000 tokens, anchored to Epoch AI's short-query figure), then × grid intensity and × PUE. All modeled figures are order-of-magnitude estimates, not measurements — see the methodology.
The order-of-magnitude that matters
Retrieval ≈ 0.3 Wh vs long-context ≈ 40 Wh — ~100× the energy
A retrieval-grounded answer uses about 0.3 watt-hours; attaching a roughly 100,000-token document to the query uses about 40 watt-hours — around 133 times more energy.
Retrieval sends the model only what's relevant, so each answer stays near a short-query energy budgetsource: Epoch AI, "How much energy does ChatGPT use?" (Feb 2025) (opens in a new tab). Attaching a large document to every prompt pays the full long-context tax on every callsource: Epoch AI, "How much energy does ChatGPT use?" (Feb 2025) (opens in a new tab). Same answer, ~100× the energy — the whole reason to ask “RAG or not?” before reaching for a giant context window. Order-of-magnitude estimate, not a measurement.
AssumptionsModeled energy/water/CO₂ recompute live as you change these. energy = tokens→Wh × (PUE / 1.2); CO₂ = energy(kWh) × grid intensity.
Grid: Roughly the US average carbon intensity. · PUE: Typical hyperscaler — assumed in the base figure. · Uncertainty band spans Epoch's 0.1–0.6 Wh short-query range.
Modeled per-query impact modeled — estimate
| Mode | Wh |
|---|---|
| Flat (Lexical) | <0.001 Wh |
| Hierarchical | <0.001 Wh |
| LLM-only | 0.300 Wh |
| RAG | 0.300 Wh |
RAG ≈ 0.300 Wh (0.100 Wh–0.600 Wh)source: Epoch AI, "How much energy does ChatGPT use?" (Feb 2025) (opens in a new tab). Lexical modes are retrieval-only (no LLM call).
| Mode | grams CO₂ |
|---|---|
| Flat (Lexical) | <0.001 g |
| Hierarchical | <0.001 g |
| LLM-only | 0.120 g |
| RAG | 0.120 g |
RAG ≈ 0.120 g (0.040 g–0.240 g). energy(kWh) × 400 gCO₂/kWhsource: Ember / IEA grid-intensity averages (opens in a new tab).
| Mode | mL water |
|---|---|
| Flat (Lexical) | <0.01 mL |
| Hierarchical | <0.01 mL |
| LLM-only | 1.20 mL |
| RAG | 1.20 mL |
RAG ≈ 1.20 mL (0.40 mL–2.40 mL)source: Jegham et al., "How Hungry is AI?" (arXiv, May 2025) — full-scope (incl. electricity generation) (opens in a new tab).
Ranges are low–high uncertainty bands from Epoch AI's 0.1–0.6 Wh short-query spread, scaled by your PUE selection.
Two water numbers, two scopesScope-1 counts only water evaporated by on-site data-center cooling. Full-scope adds the water used to generate the electricity — roughly 4× larger. We chart full-scope and label it as such.
Scope-1 (on-site cooling): vendors disclose ~0.32 mL (OpenAI)source: OpenAI (vendor-disclosed) — scope-1 (on-site cooling only) (opens in a new tab) and ~0.26 mL median (Gemini)source: Google (vendor-disclosed, median) — scope-1 (on-site cooling only) (opens in a new tab) per query. Full-scope: a short GPT-4o query is ~1.2 mL once you count electricity generationsource: Jegham et al., "How Hungry is AI?" (arXiv, May 2025) — full-scope (incl. electricity generation) (opens in a new tab). The charts above show full-scope. RAG ≈ 1.20 mL full-scope / 0.32 mL scope-1 per query.
Flat and Hierarchical make no Bedrock call — ~0 marginal LLM energy, water, or cost beyond Lambda compute.
RAG cost measured
$123.00/mo
$1496/yr
RAG CO₂ modeled — estimate
36.00kg/mo
(12.00kg–72.00kg) · 438.0kg/yr · 400 gCO₂/kWh
RAG water (full-scope) modeled — estimate
360.0 L/mo
(120.0 L–720.0 L) · Lexical modes ≈ 0
Annual RAG CO₂ ≈ 2.2k km driven source: EPA / EEA typical passenger-car average (opens in a new tab) or 55k phone chargessource: EPA equivalencies (approx.) (opens in a new tab). Rough equivalents for intuition only.