Signal Studio

Question-led guide · diagnostic

Why did my agent perform the same side effect twice after a timeout?

A retry-safe effect protocol using operation identity, idempotency, effect receipts, reconciliation, and explicit unknown completion states.

Author
Leo J. Li
Published
Updated
Review by

Direct answer

The timeout told the caller that it did not receive a conclusive response; it did not prove the server failed to apply the effect. If the agent retries with a new operation identity, the downstream system can perform the action again. Generate one stable operation key for the intended effect, persist it before execution, require an idempotent endpoint or reconciliation query, and record the returned effect receipt before claiming success.

Scope

This guide applies to durable effects such as payments, refunds, tickets, messages, configuration changes, and record creation. It assumes retries can occur after network loss, process failure, queue redelivery, or human resume. Read-only queries may still produce audit or billing events, but the most serious risk is repeating non-idempotent business effects.

Why it happens

Distributed systems cannot always distinguish “the server did nothing” from “the server completed the effect but the response was lost.” Agent frameworks add another layer: the model may interpret a timeout message as permission to try again, while the tool wrapper creates a fresh request and key. The transcript then shows two reasonable attempts but not the downstream reality.

Diagnosis

  1. Find the first durable record that represents the intended business effect.
  2. Compare operation or idempotency keys across attempts.
  3. Determine whether the downstream service committed before the response path failed.
  4. Inspect queue redelivery, workflow replay, task lease expiry, and manual resume events.
  5. Verify whether deduplication and the business effect share an atomic boundary.
  6. Check key retention against the maximum retry and reconciliation window.
  7. Identify whether the agent claimed failure, success, or uncertainty before receiving an effect receipt.

Solution

Use a two-phase protocol:

  1. Trusted code creates an effect proposal with subject, resource, action, parameters, policy, and stable operation key.
  2. Authorization approves or denies the exact proposal.
  3. The executor sends the stable key to an idempotent endpoint or records the key atomically with the effect.
  4. On timeout, workflow state becomes effect_unknown, not effect_failed.
  5. A reconciliation query looks up the operation key or business identifier.
  6. The workflow records an effect receipt, then transitions to succeeded, failed, compensated, or needs review.

The model may explain the state and propose a next step, but it does not decide whether an unobserved effect occurred.

Artifact

Field Example meaning
operation_key Stable identity for one intended effect across retries
proposal_hash Canonical subject, resource, action, and parameters
policy_version Authorization rules applied to the proposal
attempt_id Unique transport attempt for diagnosis
status proposed, authorized, executing, effect_unknown, succeeded, failed, compensated
downstream_reference Provider or system identifier used for reconciliation
effect_receipt Immutable reference showing the observed result
next_action retry, reconcile, compensate, request review, or stop

Common mistakes

  • Generating a new idempotency key on every retry.
  • Marking a timeout as failure and immediately issuing another effect.
  • Deduplicating only in the agent process rather than at the effect boundary.
  • Reusing one key with changed parameters.
  • Claiming exactly-once behavior without testing crash and concurrency boundaries.

Evidence

  1. Automatic retry is safe only when request semantics are idempotent or the client can establish that the original request was not applied.

    RFC 9110 defines idempotent methods and cautions against automatically retrying non-idempotent requests without additional knowledge.

    Primary source · standard · checked Aug 25, 2026

    Limit: HTTP method semantics do not guarantee that every implementation or secondary side effect is actually deduplicated.

  2. An application can use an idempotency key to make repeated create or update requests return the original result rather than create a duplicate.

    Stripe documents saving the first result for an idempotency key and returning it for subsequent requests with the same key.

    Primary source · official-doc · checked Aug 25, 2026

    Limit: Stripe's exact retention, validation, and endpoint behavior is provider-specific and should not be copied as a universal protocol.

  3. Agent workflows need an explicit unknown-effect state and a reconciliation path.

    The effect-ledger pattern separates request failure, response failure, and effect completion so recovery does not depend on model inference.

    Signal Studio author framework · reviewed Aug 25, 2026

    Limit: The downstream system must expose enough identity or lookup capability for reliable reconciliation.

Limitations

Idempotency does not automatically reverse partial multi-system effects or guarantee exactly-once delivery. Some operations require reconciliation, compensation, or human review. Retention windows and concurrency behavior must be designed for the downstream system and business risk.

FAQ

Is a UUID enough to prevent duplicates?
Only if the same intended effect reuses the same key and the enforcement point persists and checks that key atomically with the effect. A new UUID on every retry defeats deduplication.
Should the model choose the idempotency key?
No. Trusted workflow code should derive or issue the operation identity from the business intent and state transition, then keep it stable across retries.

Continue within Context, memory, and state, or use one of these adjacent diagnostics:

English editorial review: Codex native-English editorial review, .