Agent test runs get clear 402 errors on balance and plan tier checks
All four agent test-run endpoints now document 402 for a depleted balance or an exhausted spend limit, the single-run path finally takes the admission gate its siblings already had, and an explicit over-tier model override is refused instead of being honoured unchecked.
Running a single agent test when the workspace was out of credit, or past a
spend cap, used to start the run anyway. POST /v1/agents/tests/{test_id}/runs
skipped the admission check its three sibling paths already applied. It now
takes the same gate, and returns 402 before the run row is created.
The other three run paths already refused - what changed for them is that the
402 is now documented. All four are explicit about it:
| Endpoint | |
|---|---|
POST /v1/agents/tests/{test_id}/runs | run one - newly gated |
POST /v1/agents/{agent_id}/tests/runs | run all - already gated |
POST /v1/agents/tests/runs/batch | batch - already gated |
POST /v1/agents/tests/suite-runs/{suite_run_id}/resubmit | resubmit - already gated |
So if you integrate the batch endpoint, your funding errors are documented now too, and the shape hasn’t changed.
Model access is enforced on every one of them. An explicit override - the
per-test model_override or the run-level config_override.model - has to name
a catalogued model your plan includes, and returns 402 before any run is
queued if it doesn’t. Previously an explicit override went through with no
catalog check and no plan check at all, so a test could quietly run on a model
above the plan and hand back a verdict for a model production would never use.
One deliberate asymmetry: an over-tier model stored on the agent is clamped to the platform default rather than refused, because a live call clamps it the same way and a test has to predict production. Only an explicit ask gets refused - silently clamping one would give you a verdict for a model you never asked for. A resubmit re-checks the parent suite’s stored override against today’s plan rather than replaying it, so a downgrade can’t be replayed around.
See the docs changelog entry.