A 200 OK doesn't mean the agent succeeded
Infrastructure can be healthy while an agent's task goes wrong, so agent observability leans on execution traces, not only logs and latency.
The idea in brief
For a conventional ML service, logs and latency say a lot about health. An agent can return a successful response while picking the wrong tool, using stale context, or retrying more than it should. That shifts attention to the execution trace.
Points from the Conf42 talk
Emerson Braun's Conf42 Observability 2026 talk, "Make Agents Observable: Tracing Decisions, Tool Calls and Recovery", makes several points along these lines:
- An agent reporting that a task is done is not proof that it is done.
- A valid HTTP response doesn't mean the intended change happened.
- Evidence should cover context, decisions, actions and outcomes, with credentials and personal data redacted before export.
- Cost includes tool calls, retry loops and approval waits, not only model tokens.
What standard telemetry already provides
OpenTelemetry's 2026 walkthrough of GenAI observability shows the structure the GenAI conventions give an agent run: an invoke_agent span with nested chat spans for model calls and execute_tool spans for each tool invocation. The gen_ai.response.finish_reasons attribute can show when a model stopped to request a tool.
That covers which tools were called and how long each step took. Whether the task actually succeeded usually needs a check against the target system, which no generic convention can supply.
What each signal can and can't tell you
| Signal | Tells you | Doesn't tell you |
|---|---|---|
| HTTP status | The request completed | Whether the task was done |
| Latency | How long it took | Whether the right tool was used |
execute_tool spans | Which tools ran, and in what order | Whether their results were correct |
| Outcome check | Whether the change actually happened | Needs a domain-specific check |
Questions a trace should answer
- Which tools were called, in what order, with what result?
- How many retries or fallbacks happened?
- Where was the run blocked, if anywhere?
- Was the outcome verified, and how?
Limitations
Tool arguments are sensitive.
In OpenTelemetry, tool arguments and results only appear when content capture is enabled, which is off by default.
Traces are not evaluations.
A complete trace shows what happened, not whether the answer was good.
More telemetry, more cost.
Recording every step of long agent runs adds up.
Sources
Cite this post
Adam, Ammar. (Sep 2026). A 200 OK doesn't mean the agent succeeded. www.ammaradam.dev. https://www.ammaradam.dev/posts/agent-success-is-not-a-200/.
BibTeX
@article{adam2026agentsuccessisnota200,
title = {A 200 OK doesn't mean the agent succeeded},
author = {Adam, Ammar},
journal = {www.ammaradam.dev},
year = {2026},
month = {Sep},
url = {https://www.ammaradam.dev/posts/agent-success-is-not-a-200/}
} 
Comments
Sign in with GitHub to like this post or join the discussion.