Token Budgets in Multi-Agent Systems: How to Enforce Limits Without Breaking Reasoning
In 2025, a market research pipeline entered an infinite loop and kept running for 11 days before a billing alert finally caught it. Nobody noticed because there was no per-agent budget ceiling, no circuit breaker, and no mechanism to halt execution before the next API call. By the time the investigation started, the cost was significant.
This is not a cautionary tale about rogue AI. It is a cautionary tale about treating token budgets as an operational afterthought. Multi-agent systems consume up to 15x more tokens than single-agent chat, with context windows routinely growing from 15K to 150K tokens over a single run. Without deliberate budget enforcement baked into the architecture from day one, runaway costs are not a question of if but when.
The good news: enforcing token budgets in a multi-agent pipeline does not require sacrificing reasoning quality. Done right, it tends to improve it.
Why Bigger Context Windows Do Not Solve the Problem
The natural response to context overflow is to reach for a bigger window. Production models now support 200K to 1M tokens, which sounds like it removes the constraint entirely. It does not.
Research on what practitioners call "context rot" (the progressive degradation of model reasoning as context length grows) and the closely related "Lost in the Middle" phenomenon shows that models do not use context uniformly. When critical information is buried in the middle of a long context, models produce hallucinations and logical errors at significantly higher rates than when the same information sits near the beginning or end. Performance degrades measurably past a certain inflection point, and that inflection point is lower than most teams expect.
The practical consequence: expanding your context window does not linearly improve multi-agent reasoning. Beyond a threshold, longer contexts degrade output quality while simultaneously driving up latency and cost. Token budgets are not just a cost lever; they are a quality lever. The goal is enforcing discipline, not merely tracking spend.
The Compounding Overhead of Multi-Agent Pipelines
Before you can enforce budgets effectively, you need to understand where the tokens are actually going.
Each agent in a pipeline carries fixed overhead: a system prompt (often 3,000 or more tokens, with research suggesting roughly 40% redundancy across agents in the same pipeline), tool definitions that repeat on every call, and conversation history that accumulates across turns. A deployment with four to eight specialized agents running agent-to-agent coordination can see tokens compound five to ten times compared to a single agent solving the same task.
This overhead is not incidental; it is the primary cost lever. Optimizing system prompts and eliminating redundant tool definitions often delivers more impact than any architectural redesign. Teams that audit their prompt overhead routinely find they can cut per-agent context by 20 to 40% without losing any capability.
Budget Enforcement as an Architectural Primitive
The core insight that separates well-run multi-agent systems from the ones that produce 11-day billing surprises: token budgets must be enforced before execution, not reported after it.
Effective budget systems use an atomic reservation pattern. Before an agent call goes out, the system estimates token cost, attempts to reserve that amount from a shared budget, and only proceeds if the reservation succeeds. After the call, actual usage is reconciled against the estimate.
def attempt_agentic_call(agent, task, budget_manager):
estimated_tokens = estimate_tokens(agent, task)
if not budget_manager.reserve(estimated_tokens):
raise BudgetExhausted(
f"Need {estimated_tokens}, {budget_manager.remaining} available"
)
result = agent.run(task)
actual_tokens = count_tokens(result.api_calls)
budget_manager.reconcile(estimated_tokens, actual_tokens)
return result
For multi-process systems where multiple agents run concurrently, reservations need to be atomic at the infrastructure level. Redis-backed Lua scripts are a common approach: because Lua scripts in Redis execute as a single atomic operation, two agents cannot simultaneously draw down the same budget pool.
The reservation model also surfaces something useful. When an agent cannot reserve enough tokens to proceed, the system has a natural decision point. It can degrade gracefully (route to a cheaper model, compress its context, or defer the task) rather than silently accumulating cost.
Context Compaction: Preserving Reasoning While Cutting Tokens
Evicting old context entirely is a blunt instrument. An agent that loses its earlier reasoning steps often has to re-derive conclusions it already reached, which costs more tokens, not fewer.
Structured summarization is the more effective approach. Rather than discarding older turns, compaction compresses them into a summary that preserves decision-relevant information while eliminating redundant detail. In practice, this achieves 60 to 80% token reduction on conversation history with little or no loss in task performance. Agents with compacted contexts often match or exceed the performance of agents with full history, because compression forces elimination of noise and reduces the cognitive load the model carries into each new turn.
def compact_conversation(history, threshold=100_000):
"""Summarize older turns when context exceeds threshold."""
if count_tokens(history) < threshold:
return history
recent_turns = history[-5:]
older_turns = history[:-5]
# summarize_turns wraps an LLM call that condenses older_turns into one entry
summary = summarize_turns(older_turns)
return [summary] + recent_turns
Beyond simple history compaction, staged loading is a complementary technique. Detailed procedural guidance for a specific task (a complex SQL generation schema, or a domain-specific regulatory framework) enters the context only when a matching task pattern is detected. The rest of the time, that material stays out of the window entirely.
Tiered Allocation and Model Routing
Not every agent in a pipeline needs the same budget, and not every task needs the most capable model.
Tiered allocation formalizes what experienced teams discover empirically: high-reliability agents that produce correct outputs consistently reach confidence quickly and can operate with smaller budgets. Experimental or less reliable agents benefit from more tokens to reason through edge cases and recover from errors. Allocating uniformly across agents means a single low-reliability bottleneck consumes the entire budget while fast, reliable agents sit underutilized.
Per-agent budget tracking with explicit model assignments is straightforward to implement:
agent_budgets = {
"researcher": {"tokens": 50_000, "model": "haiku"},
"drafter": {"tokens": 100_000, "model": "sonnet"},
"reviewer": {"tokens": 80_000, "model": "sonnet"},
}
for agent_name, config in agent_budgets.items():
tokens_used = run_agent_with_budget(
agent_name, config["tokens"], config["model"]
)
if tokens_used > config["tokens"]:
log_alert(
f"{agent_name} exceeded budget: {tokens_used}/{config['tokens']}"
)
Model routing extends this further. Lightweight tasks such as classification, formatting, and routing decisions go to smaller models like Claude Haiku. Complex multi-step reasoning tasks get routed to larger models like Claude Sonnet or GPT-4o. Implementing a cost multiplier per model lets you denominate the entire pipeline budget in dollars rather than raw tokens, which makes allocation decisions far more intuitive. Teams that have adopted this pattern report 40 to 60% cost reductions without measurable capability loss on the tasks that matter.
Circuit Breakers and Observability
Budget reservations prevent overspend on any single call. Circuit breakers protect against a deeper failure mode: agents in a loop, making individually valid calls that collectively consume far more than any reasonable task should require.
A spend-rate monitor compares recent token consumption against a historical baseline. When the ratio crosses a threshold, the circuit breaker halts execution before the next call rather than after:
def should_continue_agent_loop(agent_name, spend_history, window_secs=300):
recent_spend = sum_tokens_last_n_seconds(agent_name, window_secs)
baseline_spend = median_tokens_in_window(agent_name, spend_history)
if recent_spend > 2 * baseline_spend:
return False # Halt: spend rate is anomalously high
return True
Circuit breakers depend on observability. Every agent call should be logged with tokens consumed, model used, task type, and outcome. Without that data, you cannot establish a meaningful baseline, and the circuit breaker has nothing to compare against. Real-time dashboards that surface spend rates deviating more than two standard deviations from baseline are the earliest practical warning system for loops and runaway agents. Post-hoc analysis of per-agent token utilization across successful and failed runs gives you the data you need to tune budgets over time.
Externalizing Context with Structured Tools
One underused lever for reducing per-call context is keeping large resources outside the context window entirely. The Model Context Protocol (MCP) and structured tool definitions allow agents to reference external databases, file systems, and APIs without loading full payloads into the prompt. Instead of pulling a 50KB dataset into context, an agent queries specific rows. Instead of loading an entire document, it fetches the relevant section.
This externalization can reduce per-call context by 50 to 70% on data-intensive tasks. It also has a secondary benefit: agents that cannot load everything at once are pushed toward incremental, targeted reasoning, which tends to produce better-structured outputs than agents that receive everything upfront and must decide what to pay attention to.
Pairing MCP-style resource references with hard context limits creates a productive constraint. The agent knows it cannot load the whole dataset, so it asks better questions about what it actually needs.
Putting It Together
The five-layer structure that well-engineered pipelines converge on:
- Per-request reservations: atomic budget checks before each agent call
- Per-session ceilings: hard caps on total tokens consumed across a full pipeline run
- Per-agent allocations: tiered budgets that reflect each agent's reliability and role
- Model routing: lightweight tasks to smaller models, complex tasks to larger ones
- Circuit breakers: spend-rate monitors that halt execution on anomalous activity
Teams that have implemented all five layers consistently report 30% or greater cost reductions without sacrificing task completion rates. More importantly, they gain the observability infrastructure to know what is normal and detect when something has gone wrong before a billing alert does.
The Real Value of the Constraint
Token budgets are not a tax on capability. They are the mechanism that makes multi-agent systems reliable and predictable in production. The "Lost in the Middle" research, the infinite-loop incidents, and the compounding overhead of multi-agent pipelines all point toward the same conclusion: more tokens is not always better reasoning, and uncontrolled token spend is not a side effect of powerful agents. It is a sign of agents that have not been designed carefully.
Building budget enforcement into the architecture from day one, paired with context compaction, tiered allocation, and real-time observability, turns token limits from a constraint into a design discipline. Systems built under that discipline tend to be more modular, more verifiable, and more reliable than those that treat the context window as free real estate.