Why should an agent platform limit admission before scaling workers?

A worker is not a unit of downstream capacity

An agent worker often spends much of its time waiting for model inference, a search service, or a tool response. Low CPU can coexist with a saturated dependency. Increasing replicas increases the number of callers; it does not increase the number of requests that dependency can complete.

Admission asks whether accepting another task is compatible with the service promise. Scheduling asks when accepted work may run. Autoscaling asks how many executors should be available. These decisions should share capacity evidence, but they do not have interchangeable meanings.

Find the binding resource in the next task

Estimate demand separately for model calls, input and output tokens, tool operations, storage, and human review where relevant. Protect a measured reserve for already accepted obligations and incident recovery. A single global “active tasks” limit is useful only if task sizes are sufficiently comparable or it is deliberately conservative.

Kubernetes HPA reacts periodically to metrics. Metric delay, scheduling, and worker startup all occur after the load signal. A bounded admission policy must already exist during that interval.

Twenty replicas behind a sixty-call limit

Take a deliberately simplified hypothetical workload: every task needs four model calls, the usable provider allowance is 60 calls per minute, and other constraints are absent. Its arithmetic ceiling is 15 tasks per minute. Raising workers from five to twenty does not change that ceiling.

If requests arrive at 25 tasks per minute for ten minutes, the excess is 100 tasks before accounting for variability. A five-minute completion promise is already in jeopardy. The arithmetic does not model token quotas or tails; it is enough to show why replica count cannot solve this particular bottleneck.

Fill the admission headroom worksheet

Record one interval and task class at a time. Use observed conservative demand ranges when task costs vary.

Resource	Usable budget per minute	Demand per task	Approximate task ceiling	Admission implication
Model calls	60	4	15	Binding in this constructed workload
Tool reads	240	8	30	Does not justify accepting above model limit
Evidence writes	600	12	50	Monitor storage latency separately
Review completions	Not required	0	Not applicable	Reassess if review becomes mandatory

The smallest applicable ceiling is a starting bound, not a safe target. Subtract uncertainty and protected work, then validate the resulting policy under burst and impairment tests. Add token budgets when they bind independently of call count.

Give each rejected or waiting task a precise meaning

Before acceptance, return a clear capacity decision and the offered service level. After acceptance, preserve the task identity and a bounded waiting state with a deadline and wake condition. Avoid returning success merely because a request entered an unbounded queue.

Use separate outcomes for rejected demand, accepted work waiting for capacity, expired work, and completed tasks. This prevents a dashboard from reporting improved availability while users accumulate work that can no longer finish on time.

Increase workers only when they can convert waiting into progress

Scale the eligible runnable population, not every task that is waiting for approval or an external event. Verify that additional workers increase completed work without excessive throttling, queue-age growth, or downstream latency. If the provider is constrained, hold concurrency and consider an explicitly supported lower-cost task mode or refusal.

Google’s overload guidance supplies the general principle of bounded degradation and rejection. An agent-specific degradation still needs a usefulness decision. Omitting decisive evidence to fit capacity may violate the task promise even if it makes the request faster.

Revisit the bound when task shape changes

A prompt or tool revision can double the average number of calls without changing request volume. Version the demand assumptions with the release and compare predicted versus observed consumption. Provider purchasing belongs in on-demand versus provisioned capacity; repeated attempt amplification belongs in why retries worsen an incident. The full operating context is in AI Agent Platform Engineering.