RAG vs. Long-Context Caching: An Explicit Token-Cost Scenario

RAG and long-context caching solve different parts of a workload. Retrieval chooses information for a question; caching can reduce repeated-prefix charges. To compare them fairly, hold the model, response size and authorized data constant, then include cache creation, new question tokens, reranking and storage. The example below uses documentation-linked prices and explicitly chosen token counts rather than measured end-to-end performance.

Amazon Bedrock describes a reranker as a second relevance-ranking step over retrieved candidates. Retrieving 30–50 candidates and forwarding five is one possible architecture; it is not a universal quality setting. Chunk length, corpus, metadata filtering and queries determine how much useful context those five passages contain.

State the token and service assumptions first

Assume 1,000 queries, Claude Sonnet 5.5 at $2/MTok uncached input, $0.10/MTok cache reads and $10/MTok output, with 400 output tokens per query. Naive retrieval forwards 15,000 input tokens/query. A reranked variant forwards 2,500 input tokens/query and is assigned an illustrative $1 total reranker charge. The cache variant reuses a 100,000-token prefix, adds 500 fresh tokens/query, and pays for one 5-minute cache warm-up at $2.50/MTok. Every subsequent prefix access is assumed to hit the cache.

Reducing uncached input from 15,000 to 2,500 tokens saves 83.3% of that input component. At the stated Sonnet rate, it saves $25 per 1,000 queries before reranking, retrieval and storage charges. This does not prove an 83.3% reduction in the entire bill or a quality improvement. Replace the $1 allowance with the price for your actual reranker, region and document size.

def compare_1000_queries(reranker_usd=1.0, warmup_mtok=0.1,
                         regional_claude_scenario=False):
    count = 1000
    input_rate, read_rate, output_rate, write5m_rate = 2.0, 0.10, 10.0, 2.50
    premium = 1.10 if regional_claude_scenario else 1.0
    output = count * 400 / 1_000_000 * output_rate
    naive = count * 15000 / 1_000_000 * input_rate + output
    reranked = count * 2500 / 1_000_000 * input_rate + output
    cached = (count * 100000 / 1_000_000 * read_rate
              + count * 500 / 1_000_000 * input_rate
              + warmup_mtok * write5m_rate + output)
    return {"naive": naive * premium,
            "reranked": reranked * premium + reranker_usd,
            "cached": cached * premium}
# Default: naive $34.00; reranked $10.00; cached $15.25.
# Embeddings, vector DB, retrieval and storage costs are additional.

Compare complete bills and test answer quality

The table keeps generated output constant and shows the fresh-query input and warm-up explicitly. It assumes the chosen cache prefix is reusable for the same authorized data and that later requests hit it. Extra misses or a different reranker price change the totals.

Use the Amazon Bedrock pricing page for the selected reranking model and region; the $1 figure above is an adjustable example, not a quotation for every API. Embedding updates, vector storage, retrieval operations and reranker latency can change the result. A claim that reranking is faster or more accurate needs a measured query set and timing/quality results.

Cache reuse depends on an identical eligible prefix and reuse within the refresh/TTL rules. A cache may be useful for a shared static handbook, while query-specific retrieval can suit a large or changing corpus. There is no universal 100K/250K cutoff that makes one architecture always cheaper. Different users can reuse an authorized shared prefix; private or tenant-specific content must be isolated and filtered before it enters a prompt.

The optional 10% Claude partner-cloud regional scenario multiplies the eligible model inference component. It does not automatically add 10% to every vector database or reranker service, and it is not a Gemini Developer API rule. Check the actual platform pricing and contract instead of treating first-party token rates as a complete cloud bill.