How to Cut Your AI/LLM Costs in 2026: Caching, Cheaper Models, and Multi-Agent Routing

AI features ship fast and then the bill arrives. The good news: most LLM spend is avoidable waste - the same prompt paid for a thousand times, a frontier model doing work a cheap one could handle, tokens generated that nobody reads. Here are six levers that cut real money, ordered by how much they typically save, with numbers and code you can paste today.

Lever 1: Prompt caching (the biggest single win)

Most AI apps resend the same large prefix on every request - a system prompt, tool definitions, a knowledge base, few-shot examples. That prefix is often 80% of your input tokens, and you are paying full price for it every single time. Prompt caching bills the cached prefix at roughly one tenth of the input price on a hit. Writing to the cache costs about 1.25x, so the break-even is two requests - after that it is close to 90% off the cached portion.

The rule is that caching is a prefix match: anything before your cache breakpoint must be byte-identical across requests. Put the stable content first (frozen system prompt, deterministic tool list) and the volatile content (the user's actual question, timestamps, per-request ids) last. A single datetime.now() interpolated into the system prompt invalidates the entire cache on every request - the most common reason a cache hit rate sits at zero.

Lever 2: Right-size the model

The reflex to route everything to the smartest, most expensive model is the second biggest source of waste. Model tiers differ by 5x or more in price, and most requests in a real app - classification, extraction, short answers, routing - do not need the top tier. The trick is matching the model to the job, not to your ambition.

Prices move, so check current rates - but the ratios hold: the small tier is roughly 5x cheaper than frontier for input and output alike. And do not sleep on open-source models. Llama, Mistral, DeepSeek, and Qwen are available through the same gateway, and for high-volume, well-scoped tasks (summarization, extraction, internal tools) they can undercut hosted frontier models dramatically. Route the boring 80% of your traffic to a cheap or open model and reserve the expensive tier for the requests that actually need it.

Lever 3: Multi-agent routing (the cascade pattern)

Levers 1 and 2 combine into the highest-leverage architecture: a cheap model classifies each incoming request, and only the genuinely hard ones escalate to an expensive model. This is the cascade - a fast, cheap triage step in front of your real work. In most apps the classifier costs a rounding error and diverts the majority of traffic away from the frontier tier.

import { generateText } from 'ai'

async function answer(userMessage: string) {
  // 1. A cheap model triages the request
  const { text: difficulty } = await generateText({
    model: 'anthropic/claude-haiku-4.5',
    prompt: 'Classify as SIMPLE or HARD. Reply with one word only.\n\n' + userMessage,
  })

  // 2. Route to the tier that fits
  const model =
    difficulty.trim().toUpperCase() === 'HARD'
      ? 'anthropic/claude-sonnet-4.6'
      : 'anthropic/claude-haiku-4.5'

  const result = await generateText({ model, prompt: userMessage })
  return result.text
}

The same idea scales up: specialized agents per job (a cheap extractor, a mid-tier writer, a frontier planner) instead of one expensive generalist doing everything. You pay top-tier prices only for the steps that are genuinely top-tier work.

Lever 4: Response and semantic caching

Prompt caching (Lever 1) discounts the shared prefix. Response caching skips the model call entirely when the same request comes in again. For static-knowledge queries - FAQs, policy lookups, translations, extraction from identical documents - cache the whole response at the gateway and serve repeats for free.

import { generateText } from 'ai'

const result = await generateText({
  model: 'openai/gpt-5.4',
  prompt: 'Summarize our refund policy in two sentences.',
  providerOptions: {
    gateway: { cacheControl: 'max-age=3600' }, // serve repeats for 1 hour
  },
})

The cache key is the model plus the prompt plus the generation parameters, so identical requests hit and anything that varies (a real conversation) misses and is billed normally. Do not cache user-specific conversations; do cache the repetitive, deterministic lookups that make up a surprising share of most apps.

Lever 5: Gateway-level cost controls

Routing every call through a single gateway is what makes the levers above governable instead of theoretical. From one place you get spend attribution by feature and user, automatic failover to a cheaper provider or model when the primary is down or rate-limited, per-user rate limits so one abuser cannot run up your bill, and budget alerts before you blow the month.

import { gateway } from 'ai'

const result = await generateText({
  model: gateway('anthropic/claude-sonnet-4.6'),
  prompt,
  providerOptions: {
    gateway: {
      order: ['anthropic', 'bedrock'],     // failover order
      models: ['openai/gpt-5.4'],          // fallback model if the primary fails
      user: userId,                         // per-user tracking and rate limits
      tags: ['feature:support', 'env:prod'], // cost attribution
    },
  },
})

Because the gateway charges provider list price with zero markup, you get all of this - failover, tracking, per-user limits, budgets - without paying more per token than calling the provider directly. Tags turn a mystery invoice into a per-feature cost breakdown, which is how you find the one endpoint quietly burning half your budget.

Lever 6: Token hygiene

The unglamorous lever that adds up. You pay per token in and per token out, so trim both.

• Use structured output (a schema) instead of asking for JSON in prose - shorter, parseable, and no wasted tokens explaining the format.
• Cap output with maxOutputTokens. A model with no ceiling will happily write three paragraphs where you needed one sentence.
• Trim the context you resend - old tool results and stale history are pure cost. Summarize or drop them.
• For anything not real-time (nightly reports, bulk classification, embeddings backfills), use the Batch API - it runs asynchronously at roughly half price.

Putting it together

None of these are exotic. Stacked, they routinely take an AI feature from alarming to boring on the cost side. A representative before-and-after for a support assistant handling a mix of simple and hard questions:

The exact numbers depend on your traffic mix, but the shape is real: the same product, an order of magnitude cheaper to run, with no drop in quality on the requests that actually matter - because the frontier model still handles those. The waste you cut was never doing useful work.