Why did my agent perform the same side effect twice after a timeout?

Unknown outcome is a first-class state

This guide applies to durable effects such as payments, refunds, tickets, messages, configuration changes, and record creation. It assumes retries can occur after network loss, process failure, queue redelivery, or human resume. Read-only queries may still produce audit or billing events, but the most serious risk is repeating non-idempotent business effects.

Why a timeout invites duplicate action

Distributed systems cannot always distinguish “the server did nothing” from “the server completed the effect but the response was lost.” Agent frameworks add another layer: the model may interpret a timeout message as permission to try again, while the tool wrapper creates a fresh request and key. The transcript then shows two reasonable attempts but not the downstream reality.

The email sent before timeout

The mail provider accepts a message but the client loses the response. Before retrying, the workflow queries by operation key, finds the provider message ID, records the effect, and continues without sending a second email.

At the application boundary, operation identity should be unique within the business scope:

create table effect_ledger (
  tenant_id text not null,
  operation_key text not null,
  state text not null check (state in ('planned','attempted','confirmed','unknown','compensated')),
  receipt_ref text,
  updated_at timestamptz not null,
  primary key (tenant_id, operation_key)
);

The constraint prevents two ledger identities; reconciliation is still needed to discover an external effect committed before a response was lost.

Follow one operation across attempts

1. Find the first durable record that represents the intended business effect.
2. Compare operation or idempotency keys across attempts.
3. Determine whether the downstream service committed before the response path failed.
4. Inspect queue redelivery, workflow replay, task lease expiry, and manual resume events.
5. Verify whether deduplication and the business effect share an atomic boundary.
6. Check key retention against the maximum retry and reconciliation window.
7. Identify whether the agent claimed failure, success, or uncertainty before receiving an effect receipt.

Reconcile through an effect ledger

Use a two-phase protocol:

1. Trusted code creates an effect proposal with subject, resource, action, parameters, policy, and stable operation key.
2. Authorization approves or denies the exact proposal.
3. The executor sends the stable key to an idempotent endpoint or records the key atomically with the effect.
4. On timeout, workflow state becomes effect_unknown, not effect_failed.
5. A reconciliation query looks up the operation key or business identifier.
6. The workflow records an effect receipt, then transitions to succeeded, failed, compensated, or needs review.

The model may explain the state and propose a next step, but it does not decide whether an unobserved effect occurred.

Effect-ledger template

Field	Example meaning
operation_key	Stable identity for one intended effect across retries
proposal_hash	Canonical subject, resource, action, and parameters
policy_version	Authorization rules applied to the proposal
attempt_id	Unique transport attempt for diagnosis
status	proposed, authorized, executing, effect_unknown, succeeded, failed, compensated
downstream_reference	Provider or system identifier used for reconciliation
effect_receipt	Immutable reference showing the observed result
next_action	retry, reconcile, compensate, request review, or stop

Idempotency claims that stop at request IDs

• Generating a new idempotency key on every retry.
• Marking a timeout as failure and immediately issuing another effect.
• Deduplicating only in the agent process rather than at the effect boundary.
• Reusing one key with changed parameters.
• Claiming exactly-once behavior without testing crash and concurrency boundaries.