By Marvin — 12 Jul 2026

Why Trajectory-Level Observability Breaks Down for Production AI Agents

The traces looked perfect. Every tool call was logged, every reasoning step was captured, every LLM response was recorded in sequence. The agent's trajectory was a model of clean, efficient execution. And for 48 hours, it quoted stale Q1 prices to every customer who asked for a shipping estimate.

Nobody noticed, because nobody was looking at the answers. The team was looking at the traces.

This is the central failure mode of trajectory-level observability as it's currently practiced: it tells you how your agent ran, but it cannot tell you whether what the agent produced was correct, whether the run cost three times what it should have, or whether a visual element on a web page silently steered the agent off course. Trajectory observability is necessary infrastructure. It is not sufficient for production.

Industry projections estimate that over 40% of agentic AI projects will be canceled by 2027, with runaway costs and inadequate visibility cited as leading causes. The problem is not that teams are failing to instrument their agents. The problem is that the instrumentation they have built answers the wrong questions.

This article explains where trajectory-level observability fails in production, why its three most critical blind spots (semantic correctness, cost attribution, and multimodal fidelity) require fundamentally different tooling, and what a more complete observability model looks like.

What Trajectory Observability Actually Captures

When engineers talk about trajectory observability, they mean capturing the full sequence of events inside an agent loop: the initial prompt, each LLM call and its response, every tool invocation and its result, any internal reasoning steps, and the final output. Frameworks like LangGraph, CrewAI, and AutoGen emit some version of this. OpenTelemetry's GenAI conventions provide a standardized span model for it.

This is genuinely valuable. Before trajectory tracing, debugging a multi-step agent failure meant staring at a final output and guessing. Now you can replay the execution, see which tool was called when, and identify the step where the reasoning diverged from what you expected. For many categories of failure, that's enough.

But "enough to debug" and "enough for production monitoring" are different standards. Production requires not just the ability to investigate a failure after it happens, but the ability to detect failures in real time, prevent regressions, attribute costs accurately, and maintain confidence in outputs that cannot be manually reviewed at scale.

Trajectory observability, on its own, fails all four of those requirements.

The Semantic Gap: Clean Traces, Wrong Answers

The most dangerous category of production agent failure is what the research community calls "corrupt success": the agent completes its task without triggering a single error, executes a logical sequence of tool calls, and returns a plausible-looking answer that is factually wrong.

The stale-price example above illustrates this cleanly. The agent calls the pricing API, receives a response, formats it, and returns it. The trace shows a well-structured, non-redundant execution. What the trace cannot show is that the pricing data came from a cache that hadn't been invalidated after a rate change. The trajectory is correct. The answer is wrong.

This gap exists because trajectory observability and semantic evaluation are fundamentally different instruments. A trace captures process quality: was the tool call sequence efficient? Did the agent avoid redundant steps? Did reasoning follow from premises to conclusions without obvious breaks? An evaluation captures output quality: is the answer correct? Is it consistent with ground truth or an authoritative reference?

These two signals are orthogonal. An agent can have a poor trajectory (redundant calls, unnecessary hops, sloppy reasoning) and still produce a correct answer by luck. It can have a pristine trajectory and produce a confident, well-reasoned wrong answer. Production teams that instrument only for the trajectory are flying on process data while outcome quality degrades invisibly.

The fix is not to replace trajectory tracing with semantic evaluation. It's to run both, and to log evaluations as first-class telemetry alongside traces:

# Integrate correctness eval with trajectory trace
trace_text = format_trace_for_eval(trace)
eval_result = evaluator.judge(
    query=trace.initial_query,
    agent_response=trace.final_output,
    trajectory=trace_text,
    evaluation_rubric="correctness, safety, cost_efficiency"
)

# Emit eval result as a span event so it appears in your trace viewer
span.add_event("semantic_eval", attributes={
    "correctness_score": eval_result.correctness,
    "safety_score": eval_result.safety,
    "eval_model": "claude-sonnet-4-6",
    "eval_cost_usd": eval_result.cost
})

The key architectural point here is that the evaluation result lives inside the trace. When you filter your observability dashboard for runs where correctness dropped below threshold, you can immediately inspect the trajectory that caused it. Process and fidelity become linked signals rather than separate dashboards.

Structural Failures That Return HTTP 200

Semantic errors are one category of silent failure. Research on agent fault detection identifies a broader taxonomy of failure modes that share the same diagnostic surface: HTTP 200, a plausible output, and nothing in the logs.

Consider redundant tool calls. An agent tasked with retrieving customer order history might call the same API endpoint three times in a single loop because its context window grew stale between steps. Each call succeeds. The final result is correct (the third call overwrites the first two). The trace looks verbose but functional. Without automated fault classification, you would never flag this as a problem. In production at scale, those redundant calls might triple your API costs and push latency from 1.2 seconds to 3.7 seconds.

A lightweight fault detector applied to the trace catches this immediately:

def detect_redundant_calls(trace):
    tool_calls = [e for e in trace.events if e.name == "tool_call"]
    seen = {}
    faults = []
    for call in tool_calls:
        key = (call.tool_name, call.input_args)
        if key in seen and (call.timestamp - seen[key]).seconds < 5:
            faults.append({
                "type": "redundant_tool_call",
                "tool": call.tool_name,
                "first_at": seen[key],
                "repeated_at": call.timestamp,
                "severity": "warning"
            })
        seen[key] = call.timestamp
    return faults

The pattern extends across a wider fault set: hallucinated function invocations (the agent calls a tool that doesn't exist, fails silently, and infers a result anyway), tool selection drift (the agent starts the trajectory using tool A for a task type, then switches to tool B partway through without apparent reason), and reasoning breakdown (the agent contradicts its own stated plan between steps). None of these raise exceptions. All of them are detectable by static analysis of the trace.

The gap in current observability platforms is that they collect traces but don't analyze them against known fault patterns. Teams are left to write ad hoc post-mortems. Automated fault classification would catch these at runtime, route them to alerting, and build the dataset needed to understand which fault types cluster with which downstream quality problems.

Cost Attribution: A FinOps Problem That Is Also a Design Problem

A multi-step agent running multiple models and dozens of tool calls per request can consume 5 to 20 times more tokens than a single LLM call. At scale, that multiplier turns into significant monthly spend. The question is not whether the cost is high; it's whether anyone on the team can tell where the cost is going.

Most observability platforms stop at token counting at the request level. You can see that total spend this month was $18,400. You cannot see that $11,200 of that came from one agent running in a feedback loop, that the web-scraping tool is responsible for 40% of context tokens, or that one enterprise tenant's usage pattern is dramatically less efficient than all the others.

Without attribution, the team's only feedback signal is the monthly bill. By the time a runaway agent has been running for a week, the damage is done.

Cost attribution requires tagging at the span level, from the start:

with tracer.start_as_current_span("agent_loop", attributes={
    "agent_id": "pricing_checker",
    "feature": "quote_generation",
    "tenant_id": "customer_acme"
}) as span:
    for step in agent.steps():
        span.add_event("step_complete", attributes={
            "tool": step.tool_name,
            "tokens_input": step.tokens_in,
            "tokens_output": step.tokens_out,
            "model": step.model,
            "cost_usd": step.cost
        })

With those attributes in place, the aggregation query becomes trivial:

SELECT agent_id, feature, SUM(cost_usd) as total_cost
FROM trace_events
WHERE name = 'step_complete'
  AND timestamp > NOW() - INTERVAL 7 DAYS
GROUP BY agent_id, feature
ORDER BY total_cost DESC;

But cost attribution is not just a FinOps reporting exercise. It is a design feedback loop. When you can see that the "document summarization" step of your research agent accounts for 60% of total cost, you have a concrete optimization target: can that step use a smaller, cheaper model without meaningful quality loss? When you can see that one tenant's queries are triggering 8-step agent loops while everyone else averages 3, you have an anomaly to investigate before it becomes a support ticket.

Cost transparency at the agent and feature level becomes a control plane for system design. Teams that treat it as optional reporting are flying without instruments.

Current trajectory observability is designed for text-in, text-out agent loops. The trace captures the LLM's textual response to each prompt, the structured arguments passed to each tool, and the text returned by each tool. That model works reasonably well for agents that operate purely in text.

It fails for the agents that are actually being deployed at scale: agents that scrape web pages and take screenshots, agents that read PDFs and extract tables, agents that process voice transcripts or analyze images. These agents have intermediate state that is fundamentally non-textual. A vision-based agent navigating an enterprise UI might encounter a red warning banner that changes its behavior. The trace captures the LLM's next action. It does not capture what the agent saw.

This creates a diagnostic dead end. You know the agent failed at step 6. You can see the tool call it made at step 6 and the reasoning that preceded it. But that reasoning was responding to a visual input that never made it into the trace. You cannot reconstruct the failure because the trace is missing a dimension.

The problem scales with the adoption curve. Vision-based and computer-use agents are among the fastest-growing deployment categories, and they are exactly the category that existing observability tooling handles worst. Teams building these agents need to treat screenshots and document images as first-class trace artifacts, stored alongside text spans and retrievable for replay.

Rare Failures and the Sampling Trap

Observability platforms handling high request volumes typically apply sampling: capture 100% of traces below some threshold, then drop to 10% or 1% above it. This is a sensible approach for stateless request-response services where all requests are structurally similar.

For agents, it is dangerous. Agent failure modes are not uniformly distributed across the request space. A particular combination of query type, context length, and tool availability might produce a hallucinated function invocation 0.1% of the time and clean results the other 99.9%. At 1% sampling, you might see that failure once every 100,000 requests. At 10% sampling, once every 10,000. With full sampling across a targeted evaluation dataset that deliberately exercises known edge cases, you see it in the first week.

The deeper issue is that current trajectory observability is a forensic tool: you hit a bug, you find the trace, you replay it. This is useful and it should exist. But production systems in high-stakes domains need predictive observability: continuous evaluation of a representative sample, including adversarial edge cases, against a known failure taxonomy. The goal shifts from "why did run X fail?" to "what failure modes does this agent have, and how often do they surface in production traffic?"

Connecting production traces to continuous evaluation pipelines is the infrastructure step that most teams skip because it is complex and expensive. It is also the step that separates teams that discover bugs through customer complaints from teams that discover them in dashboards.

A Three-Layer Model for Agent Observability

The pattern that emerges from these failure modes is a pyramid with three layers, each necessary and none sufficient alone.

The process layer is trajectory tracing: what steps did the agent take, in what order, with what inputs and outputs? This is what most teams have today, and it is the foundation. Without it, the other two layers have nothing to attach to. OpenTelemetry's GenAI conventions, supplemented by framework-specific instrumentation, are the right infrastructure here.

The fidelity layer is semantic evaluation and cost attribution: was the output correct, and what did it cost? This layer runs evaluation models against sampled production traces, logs correctness and safety scores as span events, and aggregates cost by agent, feature, and tenant. Without this layer, the process layer tells you how the agent ran but not whether it should be trusted or what it consumed.

The safety layer is fault detection, multimodal capture, and rare-failure coverage: are known fault patterns appearing in production, what are the agents actually seeing in non-text environments, and which edge cases in the evaluation dataset are surfacing in real traffic? This is the most operationally expensive layer to build, but it catches the failures that the other two miss.

Teams should build in order: process first, then fidelity, then safety. But they should plan for all three from the start, because retrofitting attribution tags and evaluation hooks into a production agent pipeline instrumented only for tracing is significantly more painful than designing for all three layers upfront.

What This Means for Teams Building Agents Now

The practical implications sort into three categories.

For teams just starting to instrument agents: design your span schema to include agent ID, feature, and tenant ID from day one. These attributes cost nothing to add at write time and are expensive to reconstruct later from undifferentiated trace data.

For teams with existing trajectory instrumentation: the fastest path to fidelity-layer coverage is to integrate an evaluation model call at the end of each agent run and log its output as a span event. This requires no changes to existing tracing infrastructure and immediately surfaces correctness trends that trajectory data cannot show.

For teams operating agents in multimodal environments: treat screenshots and document images as trace artifacts, not as ephemeral intermediate state. The storage cost is real but modest; the diagnostic value of replaying exactly what the agent saw is substantial.

On the tooling side, fragmentation across instrumentation standards in frameworks like LangGraph, CrewAI, and AutoGen creates genuine operational risk. OpenTelemetry's GenAI conventions provide a portable foundation, but adoption is uneven, and the cost attribution and fault-detection layers are largely not covered by any current standard. Teams that instrument heavily against a single vendor's observability platform are accumulating porting debt.

Conclusion

Trajectory observability was the right first step. It brought rigor to a domain that previously had almost none, and it made the debugging of complex multi-step agent failures tractable for the first time. Teams that invested early in tracing infrastructure are ahead of those that didn't.

But the production bar is higher than "we can debug failures after the fact." The agents going into production today are running autonomously, at scale, in high-stakes domains, with token costs that compound rapidly and failure modes that don't surface as errors. Trajectory tracing alone is not a production monitoring system for that environment. It is one layer of three.

The teams that ship reliable, cost-efficient agents in the next two years will be the ones who build the fidelity and safety layers before they need them, not after their first costly production incident makes the gaps impossible to ignore.