How should an AI agent tool-permission system work?

Scope

Use this design for tools that read private data, send messages, modify records, deploy software, spend money, change permissions, or trigger physical or operational effects. Public read-only tools still need rate, tenant, provenance, and data-handling controls, but consequence should determine rigor.

Why it happens

Many agent frameworks expose a list of callable functions and assume that listing a function grants permission. The runtime may hold one long-lived service credential, so every user and task inherits the application’s full reach. Arguments are checked for syntax but not for ownership, current state, business rules, or purpose.

This collapses four separate questions: can the model describe an action, may this principal perform it, should it happen under current conditions, and did the intended effect actually occur?

Diagnosis

Inventory every tool action rather than every tool name. For each action, record:

• principal and delegation chain;
• tenant and target resource;
• data classification and effect severity;
• allowed argument ranges and prohibited combinations;
• credential audience, scope, and lifetime;
• confirmation and separation-of-duty rules;
• idempotency, retry, verification, and recovery behavior;
• telemetry fields and evidence retention.

Flag any tool where the executor accepts free-form model text, uses a cross-tenant credential, or cannot state whether a timeout happened before or after the external commit.

Solution

Define five capability stages: read, propose, simulate, execute, and verify. A run receives only the stages it needs. The model emits a proposal conforming to a strict schema. The application resolves resource identifiers, normalizes arguments, and sends authenticated facts to a policy decision point.

Issue a short-lived capability envelope that binds principal, tenant, resource, action, normalized argument constraints, purpose, expiration, and a unique action identifier. The executor rejects any mismatch and uses a credential whose audience and scope fit only the target service.

For risky actions, require step-up authentication, a second approver, or confirmation showing a deterministic preview. After execution, return a receipt containing provider transaction ID, before-and-after state references, idempotency key, and verification result. A model-generated sentence is not an effect receipt.

Artifact

Apply the permission and effect columns in the Prompt-Injection Control Matrix. The same matrix makes it visible when an untrusted input can influence an overpowered capability.

Common mistakes

• Treating function exposure as authorization.
• Passing one application service token through every user and tenant workflow.
• Checking the tool name but not the target resource or normalized arguments.
• Asking the model to enforce a spending, deletion, or data-access policy stated only in prose.
• Reusing an approval after the resource state or arguments have changed.
• Recording an HTTP 200 response as proof that the intended business effect is correct.