What belongs in an AI agent trace schema?

Scope

This schema is for agent runtimes that combine model calls, retrieval, tools, policy checks, and evaluators. It is a logical contract, not a requirement to place every field on every OpenTelemetry span. Decide which data belongs in resource attributes, span attributes, events, metrics, logs, and protected evidence stores.

Why it happens

Instrumentation usually grows component by component. The model gateway records tokens, the retriever records query latency, the tool service logs HTTP requests, and the product database stores the final state. Each component may be healthy while the overall run is impossible to reconstruct. Common names such as request_id, session_id, and user_id also acquire incompatible meanings.

A trace schema should begin from the questions operators and evaluators must answer, then map those questions to stable fields and relationships.

Diagnosis

Review existing telemetry for four kinds of ambiguity:

1. Identity ambiguity: a run, attempt, conversation, tool transaction, and external effect share or lack identifiers.
2. Version ambiguity: the model alias, prompt, policy, tool schema, retrieval index, or evaluator changed without an immutable reference.
3. State ambiguity: telemetry cannot distinguish requested, authorized, attempted, returned, confirmed, compensated, and unknown.
4. meaning ambiguity: success, cost, confidence, or result has no documented definition and unit.

If a field cannot be defined in one sentence with its unit, allowed values, owner, privacy class, and compatibility rule, it is not ready to become a production contract.

Solution

Organize the schema into nine groups:

Group	Minimum fields
Envelope	schema_version, run_id, started_at, ended_at, environment
Relationships	trace_id, span_id, parent_span_id, typed links for retries or handoffs
Task	task_type, task_version, tenant_ref, requested_outcome, risk_class
Executable versions	agent, model, prompt, policy, tool-contract, retrieval-index, evaluator versions
Model operation	provider, operation, model, request settings, token counts, finish or error reason
Evidence	source reference, version or observed time, query reference, rank, admission decision
Tool operation	tool, action, authorization, idempotency key, attempt, response, effect confirmation
Economics	latency, provider units, currency, allocated cost, cache status, retry waste
Outcome and governance	execution state, evaluation state, business outcome, stop reason, privacy class, redaction

Use OpenTelemetry trace context for distributed correlation and established semantic attributes where their definitions fit. Put local concepts under a documented namespace. Version the envelope and publish compatibility rules: additive optional fields, deprecated fields, semantic changes, and required migrations.

Do not put unbounded arrays or complete documents in span attributes. Emit bounded events or store a protected object and record its immutable reference, hash, classification, and retention policy.

Artifact

Start with the Agent Run Envelope JSON Schema. Validate envelopes at ingestion and maintain a representative compatibility fixture for every released schema version.

Common mistakes

• Calling every operation agent.step and placing meaning in free-form text.
• Recording a mutable model or prompt alias without a resolved version.
• Mixing requested tool calls, provider responses, and confirmed external effects.
• Putting high-cardinality or sensitive payloads into metric labels.
• Using success: true when only execution completion is known.
• Adding fields without a data owner, privacy class, unit, or evolution rule.