ragornot

RAG or not — compare. learn. decide.

Live Benchmark

Run 7 real AWS-docs queries through four retrieval strategies and get empirical numbers on cost, latency, answer quality, and carbon — so you can decide whether RAG is worth it for your organisation.

Flat & Hierarchical: near-zero cost, millisecond latencyRAG: highest accuracy, ~$0.00041/queryLLM-only: no grounding — the control baseline

Advanced: Benchmark on your own docs

Upload your own PDFs or TXTs and benchmark retrieval against them with up to 10 iterations. Requires an access key — request one below if you don't have one.

Request access to advanced benchmark

Leave your email and we'll send you an access key to run benchmarks on your own documents. Your email is used only to send the key — we don't share it or use it for anything else. See our Privacy Policy.

Mode Comparison (illustrative — run benchmark for live data)

Numbers below are representative of the demo corpus runs. Hit “Run Benchmark” above to replace these with your own live results.

RAG vs Flat baseline: adds $0.00041 per query in LLM cost — free retrieval modes cost nothing beyond Lambda compute.

Accuracy, latency, and cost per query for each retrieval mode
ModeAccuracyFor live runs: avg_confidence × 100, where confidence is the retrieval model's score for how relevant the top chunks were (0–1). LLM-only has no retrieval step so it shows no confidence. Static numbers are illustrative from demo corpus runs.LatencyEnd-to-end time measured in the browser with performance.now() around the full fetch() call — includes network round-trip, not just Lambda execution.Cost / querydata.llm_stats.cost_usd from the Lambda — Bedrock token billing cost. Exactly $0.00000 for Flat and Hierarchical (no LLM call). For LLM-only and RAG, reflects input + output token pricing.What it means for your org
Flat (Lexical)
71%
6ms$0.00000Fast and free, but a single global ranking can surface off-topic chunks from unrelated docs.
Hierarchical
79%
11ms$0.00000Narrowing to top documents first improves precision over Flat at a small extra compute cost.
LLM-only
53%
690ms$0.00013No grounding — fluent, but frequently generic or wrong on service-specific details.
RAG
91%
860ms$0.00041Retrieval grounds the answer before generation — highest accuracy, but the slowest and priciest mode.

Impact Analytics

Illustrative — run the benchmark above to replace with live data. Grid intensity at 400 gCO₂/kWh. LLM/RAG energy derived from cost at ~2,615 Wh/$. All are order-of-magnitude estimates, not measurements.

Estimated token cost per query
Estimated token cost per query
ModeUSD
Flat (Lexical)$0.00000
Hierarchical$0.00000
LLM-only$0.00013
RAG$0.00041
Estimated CO₂ per query
Estimated CO₂ per query
Modegrams CO₂
Flat (Lexical)0.0002g
Hierarchical0.0004g
LLM-only0.14g
RAG0.16g
RAG vs Flat baseline deltaHow much more each RAG query costs and emits versus pure lexical (Flat) retrieval — the near-zero baseline.
Cost: +$0.00041/queryCO₂: +0.164g/query

Flat and Hierarchical cost nothing beyond Lambda compute — no Bedrock call, no LLM billing.

Org-scale projectionRAG cost scales linearly with query volume. Lexical modes cost ~$0 regardless of scale — only Lambda compute.
/day

RAG cost

$123.00/mo

$1496/yr

Flat / Hierarchical cost

~$0/mo

Lambda compute only

RAG CO₂ (est.)

49.20kg/mo

598.6kg/yr

Annual RAG CO₂ ≈ 3.0k km driven or 75k phone charges. Rough order-of-magnitude only (200 gCO₂/km, 8 gCO₂/charge).