Outside callers of local inference

Outside callers

The engine's one bearer is the fleet's own. A caller from outside the fleet gets its own key, its own limits and, when one is declared, its own price, and reaches the deployment only through the gateway:

stado inference callers add <deployment> --caller <name> \
  --requests-per-minute N --tokens-per-day N \
  [--usd-per-million-prompt P --usd-per-million-generated G] [--json]
stado inference callers revoke <deployment> --caller <name>
stado inference callers list <deployment>
stado inference callers usage <deployment> [--caller <name>] [--since-hours 24]
stado inference gateway <deployment> --listen ADDR:PORT \
  --flush-seconds N --refresh-seconds N

add writes a new key to the vault item inference-caller-<deployment>-<caller> (field token), prints it once, and keeps only its SHA-256 in the caller's declaration (inference/callers/<deployment>/<caller>.json in the fleet store). It refuses an unknown deployment, a caller already declared, a name that is not lowercase letters, digits and inner hyphens, a limit of zero, and a negative price. revoke keeps the declaration with its revocation time; the gateway refuses the key from its next refresh.

gateway refuses to start for a deployment whose model license was never read or whose model chain declares no license, naming the repositories. It answers POST /v1/... only. Each request must send Authorization: Bearer <key>; the key is compared by hash with every declared caller. It answers 401 for no key, an unknown key or a revoked one; 429 naming the limit for a caller past its requests in the last 60 seconds or its tokens for the UTC day; 400 for a body that is not JSON or asks for "stream": true (a streamed answer's tokens cannot be counted for billing); 502 naming the cause when the engine does not answer or its answer cannot be read. An admitted request is sent to the engine with the deployment's own bearer, and the answer's usage tokens are added to the caller's hour; an answer that states no usage is counted as usage_unread, never as zero. Usage is kept per caller and hour at inference/usage/<deployment>/<caller>/<hour>.json, written every --flush-seconds and when the hour changes; a restarted gateway reads the day's kept usage, so no caller gets its day again. usage prints each hour's requests, prompt and generated tokens, unread answers, refusals and the charge at the caller's price (no price declared without one). Desktop's Registry screen lists, issues, revokes and reads callers.

One call without an account

A deployment can also sell one call to a caller who has no key and no account, such as an AI agent: the gateway states a price, a payment by a method the operator declares settles it, and the call runs with a key good for that one call.

stado inference offer set <deployment> \
  --usd-per-million-prompt P --usd-per-million-generated G \
  --max-prompt-tokens N --max-generated-tokens N --quote-seconds N \
  --method NAME=VERIFY_URL[,VAULT_ITEM] [--method ...] [--json]
stado inference offer show <deployment>
stado inference offer withdraw <deployment>
stado inference offer settlements <deployment> [--since-hours 24]

No payment method is built in. Each --method names a verification endpoint (https, or this host's loopback 127.0.0.1) the gateway asks whether a payment settled a quote: it is sent POST {"quote", "deployment", "usd", "reference"} and must answer {"settled": true|false, "usd": <paid>}; with ,VAULT_ITEM the item's token field is sent as its bearer. set refuses a negative price, a bound or quote time of zero, a method that is not NAME=URL[,ITEM], a name that is not lowercase letters, digits and inner hyphens, a repeated name, and an unknown deployment. The offer is kept at inference/pay-per-call/<deployment>/offer.json and the gateway reads it on its next --refresh-seconds.

With an offer, a request to the gateway without Authorization is answered 402 Payment Required with {"error": {"type": "payment_required"}, "quote": {id, usd, price, max_prompt_tokens, max_generated_tokens, methods, expires_at}}. The quote's usd is the most one call can cost: every token the offer allows at its price. The payer pays it by any listed method and sends POST /v1/pay with {"quote", "method", "reference"}. The gateway refuses an unknown quote (404 unknown_quote), an expired one (410 quote_expired), a method the offer does not list (400 unknown_method), a method endpoint that does not answer, refuses or answers without settled and usd (502 payment_unverified, with its cause), a payment not settled or below the quote (402 payment_insufficient), and a reference already exchanged for a key (409 reference_used). Otherwise it answers {"key", "quote", "usd_paid", "calls": 1}; only the key's SHA-256 is kept.

The call with that key is spent before the engine is asked, so a second call with it is refused (401 spent). Its max_tokens is held to the quote's generated bound. When it returns, its settlement is kept at inference/pay-per-call/<deployment>/settlements/<quote>.json: the method, what was paid, served or failed, the engine's status, the tokens its answer stated, their cost at the quote's price, and the refund due (the whole payment for a failed call, the payment less the cost for a served one). A served answer that states no usage keeps its cost and refund unread, never zero; a prompt that cost more than was paid records the unpaid amount. settlements lists every paid call with what was earned and what is owed back, and stado market earnings and stado market compare count each paid call's payment less its refund as inference sold. Without an offer, a request without a key is refused 401 as before, and /v1/pay answers 404 not_sold. Desktop's Registry screen sets, shows and withdraws offers and lists settlements.