Skip to content

Meet DeepintShield at Bengaluru Tech Summit 2026, 17 to 19 November, BIEC Hall 1, booth H1A106. Book 20 minutes

Start For Free

Set LLM budgets for each team

Enforce budgets and rate limits at customer, team, Virtual Key and provider level, then cut waste with caching and routing.

Here is how to set LLM budgets for each team: enforce them at the gateway, before a request reaches a provider. DeepintShield applies independent budgets and rate limits at customer, team, Virtual Key and provider level, so every request must pass every limit, and spend is visible by team as it happens.

The challenge

Model spend behaves differently from most cloud costs. It grows with usage, prompt length and model choice, and it can spike in minutes:

  • One bill, many teams. Provider invoices do not tell you which product or team spent the money.
  • Alerts arrive late. Monthly reports show an overspend after it has happened.
  • Loops and retries. An agent stuck in a loop or a bad retry policy can burn through a budget overnight.
  • Wrong prices. Negotiated discounts and new models make simple cost estimates inaccurate.
  • Limits in code drift. Each application enforces its own caps, if any, in its own way.

The approach

Set limits where requests pass, not in each application. Mirror your organisation in a hierarchy so a team limit covers all its applications. Use accurate prices so the numbers match the invoice. Reduce waste before it reaches the budget, then watch spend in the tools your operations team already uses. Savings from caching and routing depend on the workload, so measure against your own baseline. Agree the hierarchy with finance first: who owns each budget, what should happen at the limit, and who can raise it. Use rate limits as well as budgets: a rate limit slows a runaway loop straight away, while a budget caps total spend for the period.

How to set LLM budgets for each team with DeepintShield

  1. Build the budget hierarchy. The hierarchy runs Customer, Team, Virtual Key, then Provider Config. Each level has its own budget and rate limits, all of them must pass, and cost is deducted at every level. Providers that are over budget are dropped from routing. Limits cover requests and tokens, with reset periods from one minute to one month. Teams and Customers, and per-model budgets in Model Hub, need the Team plan or above. See budgets and limits.
  2. Get the prices right. In Model Hub, set model overrides in USD per million tokens, apply pricing adjustments as multipliers (for example 0.8 for a 20% negotiated discount), and sync a custom pricing datasheet on a schedule. DeepintShield adds no token markup. See model catalogue and pricing.
  3. Cut waste before it reaches the budget. Cost Optimization offers 12 techniques. Five run on every plan: provider prompt-cache hints, request coalescing, TTFT prefix ordering, the guardrail evaluation cache and parallel-tool safety. Team and above adds semantic caching, cascade routing, reasoning-effort throttling, prompt compression, RAG optimisation, conversation summarisation and batch eligibility analysis. Semantic caching can save up to 60% on repetitive workloads; results depend on the workload.

What you can show afterwards

  • Spend and usage per customer, team and Virtual Key, with the requests refused at each limit.
  • Prometheus metrics such as deepintshield_cost_total and deepintshield_cache_hits_total, labelled by Virtual Key, team and customer.
  • Cache hits, saved tokens and saved cost per key in the Agentic Cache report (Business plan and above).
  • The price rules that produced each cost figure.
  • Which teams are close to their limits, so owners can act before requests are refused.

Before go-live, set a small limit on a test key and exceed it. Confirm requests are refused and that the application shows a useful error instead of retrying in a loop. Then compare the gateway’s cost figures with the provider invoice for the same period.

Read 12 ways to reduce wasted LLM work before choosing techniques.

Start For Free