AI Agent Observability: How to Monitor Agents in Production
AI agent observability connects traces, tool calls, policy decisions, costs, failures, and business outcomes so teams can understand and control production behaviour.
Explore this article
- Why normal application monitoring is not enough
- Use the business task as the top-level trace
- Monitor three layers: execution, control, and outcome
- Design privacy and security into the telemetry path
- Alert on risk and service degradation, not model personality
- Connect observability to evaluations and release decisions
- Make every serious alert lead to an incident path
- Multi-agent systems need one parent task, not isolated dashboards
Why normal application monitoring is not enough
Traditional monitoring answers valuable questions: Is the API available? Did the database query fail? How long did the request take? An agent adds a different class of uncertainty. The application may be healthy while the agent chooses the wrong tool, asks for excessive data, retries an irreversible action, receives a policy denial, or finishes with an outcome that the user quietly corrects later.
That does not make conventional telemetry obsolete. It means the agent trace must connect ordinary service signals to the task's changing state. HTTP calls, queues, database spans, model requests, retrieval operations, tool executions, approval events, and downstream state changes belong to one observable story. OpenTelemetry's semantic conventions provide a common vocabulary for traces, metrics, logs, and events; the developing GenAI conventions extend that direction for model and agent operations.
The deeper lesson is the same one behind why AI agents fail in production: a capable model is only one component. Production reliability depends on integrations, permissions, controls, recovery, ownership, and evidence. Observability is the evidence layer that shows whether those parts worked together.
- AI agent observability
- A system's ability to explain and measure an agent-run business task from initial intent through decisions, tool interactions, controls, state changes, and final outcome.
It combines traces, structured events, metrics, logs, evaluation results, and business records. It should support live operations, debugging, security review, cost management, and later audit without becoming a database of every sensitive prompt and response.
Use the business task as the top-level trace
A chat session is rarely the right top-level unit. One conversation can contain several tasks, and one task can continue asynchronously after the conversation ends. Start a trace when the system accepts a defined job: reconcile this invoice, investigate this support case, prepare this renewal, or update this record. Give that job a stable task ID and carry its trace context across services and queues.
Inside that trace, create spans for operations with duration: a model request, retrieval query, policy evaluation, tool call, approval wait, external API call, or state transition. Use structured events for meaningful points in time such as an approval granted, a budget threshold crossed, a fallback selected, a user correction received, or a kill switch activated. OpenTelemetry distinguishes spans, which represent operations with boundaries and duration, from events, which represent named occurrences at a point in time.
The task trace should answer five questions without requiring an engineer to correlate unrelated dashboards manually: What outcome was requested? What versioned system attempted it? What evidence and controls influenced execution? What changed in the outside world? Was the result accepted, corrected, reversed, or abandoned? This is related to, but not identical with, an audit trail. Auditing AI agent actions explains the durable evidence needed for accountability; observability adds live diagnosis, aggregation, alerting, and operational feedback.
| Signal group | Record | Why it matters |
|---|---|---|
| Identity | Task, trace, tenant, user or service identity | Correlates activity while preserving access boundaries |
| Purpose | Requested outcome, task class, risk tier | Lets teams judge success against the right objective |
| Versions | Agent, prompt, model, tool, policy, retrieval index | Makes regressions and rollbacks reproducible |
| Decisions | Selected action, alternative category, confidence or uncertainty signal | Explains routing without storing hidden chain-of-thought |
| Tools | Tool name, authorised scope, redacted inputs, result status, latency | Shows what the agent attempted and what actually happened |
| Controls | Policy result, approval ID, denial reason, budget and rate-limit state | Proves that authority was checked outside the model |
| Outcome | Business state change, acceptance, correction, reversal, abandonment | Separates task success from technical completion |
| Economics | Model usage, tool cost, elapsed time, human-review effort | Supports unit economics and capacity decisions |
Monitor three layers: execution, control, and outcome
The first layer is execution health. Track latency, timeouts, provider errors, tool failures, queue delay, retries, context size, token usage, cache behaviour, and dependency health. These signals reveal whether the machinery can complete work consistently, but not whether the work was sensible.
The second layer is control health. Track permission denials, approval requests, approval expiry, policy versions, high-risk action attempts, budget stops, tool-call depth, repeated retries, escalation, fallback use, and emergency stops. A rising denial rate may mean the guardrail is doing its job, the agent is drifting toward disallowed behaviour, or the task design no longer matches its authority. The metric creates a question; trace inspection and outcome data provide the answer. The AI agent guardrails guide covers the deterministic controls that should produce these events.
The third layer is outcome health. Did the invoice reconcile correctly? Was the ticket resolved without reopening? Did the proposed change pass review? Did a human accept, edit, reject, or undo the result? How much elapsed time and reviewer effort did the task consume? Outcome instrumentation is domain-specific, which is precisely why generic model dashboards cannot finish the job. A token chart may explain cost movement. It cannot tell a finance owner whether duplicate payments were prevented.
A seven-step instrumentation sequence
- 01
Define the task contract
Name the business outcome, valid terminal states, risk tier, owner, and evidence that proves success. If success is undefined, the dashboard will optimize convenient technical proxies.
- 02
Create one trace envelope
Generate a task and trace identifier at intake. Propagate it through model calls, tools, queues, services, approvals, and business records without using sensitive content as an identifier.
- 03
Version every behaviour-changing dependency
Record agent, prompt, model, tool schema, policy, workflow, and retrieval-index versions. A timestamp alone cannot explain which behaviour was active.
- 04
Wrap model, retrieval, and tool boundaries
Measure duration, status, error type, retry count, and safe operation metadata. Capture redacted payloads only when they are necessary for diagnosis and permitted by policy.
- 05
Emit controls as first-class events
Record permission checks, approvals, denials, rate limits, budget decisions, validation failures, fallbacks, and circuit breakers with their policy version and outcome.
- 06
Join the final business result
Update the trace when the downstream result becomes known, even if that happens hours later. Record acceptance, correction, reversal, escalation, or abandonment rather than treating a generated answer as success.
- 07
Turn evidence into action
Create dashboards by task and risk tier, define alerts with owners and runbooks, sample traces for review, and promote recurring failures into evaluation and regression suites.
const task = startTaskTrace({
taskType: "invoice-reconciliation",
riskTier: "medium",
agentVersion,
promptVersion,
policyVersion
});
await task.span("tool.lookup-invoice", async (span) => {
span.setAttribute("tool.name", "lookup-invoice");
span.setAttribute("authorization.result", "allowed");
return lookupInvoice(redactedReference);
});
task.event("approval.requested", { approvalId, actionClass: "write" });
task.finish({ status: "completed", businessOutcome: "matched-for-review" });Design privacy and security into the telemetry path
Telemetry often becomes a second copy of the most sensitive workflow data. An agent may see customer messages, contracts, credentials, financial records, source code, or internal decisions. If raw inputs and outputs flow into a broadly accessible logging platform, the monitoring system can undermine the controls around the production system.
Define an allowlist of fields for each event rather than sending arbitrary objects and redacting later. Store stable references, classifications, hashes, counts, and decision summaries where those are sufficient. Separate operational telemetry from restricted evidence. Apply encryption, access controls, tenant isolation, retention, deletion, and audit logging to the observability platform itself. OWASP's AI Agent Security Cheat Sheet recommends monitoring decisions, tool calls, outcomes, costs, and security events while redacting sensitive data.
For high-risk actions, record the action class, authorization result, approval identifier, policy version, execution result, and safe parameters needed for reconstruction. Do not record a model's private reasoning. A concise decision summary such as “selected the approved refund tool because the order met policy conditions” is more operationally useful than a large opaque transcript. Apply the same minimum-access principle described in giving agents access to company data safely to the people and systems that can query traces.
What the first production dashboard should answer
- How many tasks reached each terminal business state?
- Which task classes have the highest correction, rejection, or reversal rate?
- Where is time spent: model, retrieval, tool, queue, approval, or human review?
- Which tool and policy versions correlate with new failures?
- What is the cost per accepted outcome, not only cost per model request?
- Which denials, retries, or escalations are increasing by risk tier?
- Are sensitive telemetry fields being redacted and retained correctly?
- Who owns each alert, and which runbook defines the response?
Alert on risk and service degradation, not model personality
An alert should indicate a condition that a named person or automated control can act on. Useful candidates include an irreversible tool called without the expected approval, repeated attempts after a policy denial, a sudden rise in privileged-tool usage, cross-tenant access attempts, a budget or recursion limit reached, a material fall in accepted outcomes, or a failure pattern concentrated on a new version.
Start with deterministic conditions around authority, money, privacy, and irreversible state. Add anomaly detection where a baseline is meaningful, but do not let an anomaly score become an unexplained enforcement oracle. Define the event, scope, threshold, evaluation window, severity, owner, runbook, and safe automated response. Then test the alert using controlled failures.
Rate alerts by business impact. A transient model timeout on a retryable research task may belong in a service-level dashboard. A successful transfer to an unapproved destination needs immediate containment even if latency and error rate look perfect. The control should be able to pause the affected tool, tenant, workflow, or version without disabling every agent. That containment design is part of the production operating model required to close the pilot-to-production gap.
| Measurement layer | Primary question | Example evidence |
|---|---|---|
| System health | Could the workflow execute? | Availability, latency, dependency errors, retries |
| Agent behaviour | Did it choose and use capabilities appropriately? | Tool selection, validation, policy and approval events |
| Task quality | Was the produced result acceptable? | Evaluation criteria, reviewer decision, user correction |
| Business outcome | Did the work create the intended effect? | Resolved state, reversal, downstream KPI, elapsed effort |
| Governance | Can the organisation explain and control the system? | Versions, ownership, audit evidence, incident actions |
Connect observability to evaluations and release decisions
Offline evaluations answer whether a candidate version handles known cases. Production observability reveals cases the team did not anticipate: ambiguous requests, changing data, unusual tool responses, adversarial content, approval friction, and quiet human corrections. The two systems should exchange evidence.
Promote representative failures, near misses, denials, corrected outputs, and costly traces into a curated evaluation set. Preserve the input safely, expected control behaviour, acceptable outcome, version context, and reason the case matters. Before releasing a new prompt, model, tool, workflow, or policy, run the relevant regression suite. After release, compare outcome and control signals by version and risk tier.
Do not turn every production trace into training data automatically. Consent, privacy, licensing, confidentiality, retention, and data quality still apply. A trace selected for evaluation should pass a deliberate review and sanitisation process. NIST describes AI risk management as a lifecycle activity covering the design, development, use, and evaluation of AI systems. Observability supplies evidence for that loop; it does not replace risk ownership or human judgment.
Make every serious alert lead to an incident path
When an agent crosses a meaningful boundary, the first job is containment. Disable or narrow the affected capability, preserve relevant trace and business evidence, stop unsafe retries, and identify whether any external state changed. Do not begin by editing the prompt in production and hoping the behaviour disappears.
Next, reconstruct the task. Confirm the initiating identity and intent, active versions, retrieved evidence references, tool requests, policy decisions, approvals, external responses, retries, and final state. Determine whether the issue was model behaviour, ambiguous task design, stale data, tool semantics, missing idempotency, permission scope, integration failure, or human-process breakdown. Several causes can coexist.
Correct the narrowest responsible layer. A tool-authorisation defect belongs in the policy or identity layer. Duplicate execution needs idempotency and transaction design. Poor routing may require an evaluation-backed prompt or model change. Ambiguous ownership needs an operational change. Add the case to tests, document residual risk, release through the normal gate, and monitor the corrected version. Small teams can keep this lightweight, but they still need the ownership and incident path described in AI agent governance for small business.
Multi-agent systems need one parent task, not isolated dashboards
A multi-agent workflow becomes unreadable when each agent produces a separate session with no shared context. Keep one parent task trace and create child spans for orchestration, delegation, specialist work, inter-agent messages, tool calls, reconciliation, and final commitment. Record the sender, recipient, message type, trust boundary, delegated scope, and result without copying unnecessary sensitive payloads.
The orchestrator should own the terminal business state. A specialist agent may complete its assignment successfully while the overall workflow fails during reconciliation or execution. Measure both levels, but do not count child-agent completion as business success. Watch for cascading retries, circular delegation, privilege accumulation, conflicting results, and context growth across handoffs.
This is another reason to use multiple agents only when separation creates a measurable benefit. The architectural guidance in multi-agent systems explained applies directly: one orchestrator should own workflow state and shared policy while bounded specialists perform work that truly needs separate context, tools, controls, or scaling.
Full trace retention versus risk-based sampling
Advantages
- Complete traces improve reconstruction for rare and distributed failures.
- Consistent evidence makes version comparison and incident review easier.
- High-risk or state-changing tasks may justify stronger retention.
Trade-offs
- Full payload retention increases privacy, security, storage, and access risk.
- High-volume low-risk tasks can create noise that hides important cases.
- Retention without ownership and review creates cost rather than learning.
Frequently asked questions
What is the difference between AI agent monitoring and observability?
Monitoring checks known conditions through metrics, dashboards, and alerts. Observability combines traces, events, logs, metrics, versions, and outcomes so a team can investigate conditions it did not predict. A production programme needs both: monitoring for timely action and observability for explanation.
Should we store every prompt and response?
No. Store only what is necessary, permitted, protected, and useful for a defined purpose. Prefer classifications, hashes, references, redacted fields, decision summaries, and sampled evidence. Never place credentials, unnecessary personal data, or hidden chain-of-thought in general telemetry.
Which metrics should we start with?
Start with task completion state, acceptance or correction rate, elapsed time, human-review effort, tool and policy failures, retries, approval behaviour, cost per accepted outcome, and incidents by task and risk tier. Add model-level metrics where they help explain those outcomes.
Can OpenTelemetry monitor AI agents?
OpenTelemetry provides vendor-neutral foundations for traces, metrics, logs, events, context propagation, and semantic conventions. Its GenAI conventions are developing. Use the standard where it fits, add controlled domain fields for your task and policy data, and version those extensions rather than locking critical evidence into one vendor dashboard.
How long should agent traces be retained?
There is no universal period. Choose retention by task risk, legal and contractual duties, incident needs, privacy, storage cost, and the time required to discover downstream outcomes. Retain high-risk evidence deliberately, sample lower-risk activity where appropriate, and enforce deletion rather than keeping everything indefinitely.
Who should own AI agent observability?
Engineering should own instrumentation quality, but the operating owner must define successful outcomes and response priorities. Security and privacy teams define sensitive-event handling; risk or compliance functions define required evidence where applicable. One named service owner should be accountable for the complete production loop.
Sources and implementation references
- AI Risk Management FrameworkNIST · Accessed August 2026
- Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence ProfileNIST · 2024
- OpenTelemetry Semantic ConventionsOpenTelemetry · Accessed August 2026
- OpenTelemetry Generative AI semantic conventions projectOpenTelemetry · Accessed August 2026
- AI Agent Security Cheat SheetOWASP Cheat Sheet Series · Accessed August 2026
Guardrails 101: Keeping AI Agents From Going Off-Script
AI agent guardrails work when deterministic controls limit authority, validate actions, cap impact, require approval, detect anomalies, and stop execution safely.
AI Agents vs RPA: Do You Still Need Both?
RPA is strongest on stable, repetitive steps. AI agents handle ambiguity and variable context. Most businesses need a governed combination rather than wholesale replacement.
Auditing AI Agent Actions: A Practical Starting Point
An AI agent audit trail should reconstruct intent, evidence, decisions, tool calls, approvals, results, cost, and versions—not merely store chat transcripts.