Planning and applying a deployment

Planning and applying

The lifecycle is deliberately two-step and generation-fenced:

stado inference plan chat-primary \
  --host gpu-host \
  --image 'vllm/vllm-openai@sha256:<image-digest>' \
  --cache-dir /srv/stado/inference/chat-primary \
  --model 'example/model' \
  --revision '<model-revision>'
stado inference apply <plan-id>

plan inventories the host, requires Docker, NVIDIA tooling and ip (iproute2), reads every global IPv4 address of the host's interfaces that are up, and binds the endpoint on the first one the chosen --visibility holds; no Tailscale client is needed unless --visibility tailscale is chosen. It then saves an immutable plan bound to the current registry digest, locally under ~/.stado/inference-plans/<id>.json. A host with no address on the chosen network is refused naming it, the addresses it holds, and the visibility that would bind on one of them: <host> has no address the <visibility> visibility binds an endpoint on; its interfaces hold <addresses>: choose --visibility <other>, or connect the host to the <visibility> network (or only connect the host to the <visibility> network when no other visibility fits). The registry refuses a deployment whose endpoint.visibility is no visibility (… is no visibility; it is one of lan, tailscale), whose host is not an address that visibility binds on, whose protocol is not openai-chat, or whose host:port is another deployment's endpoint.

plan also reads, from the Hugging Face hub, the license the model repository declares at the planned revision and the license of each base model its card names, using the token of the Skarbiec role huggingface when one is stored. It records them in the deployment as model.license and prints license=<repo>: <license>, <base>: <license>; a repository or base model that declares none is printed as none declared with a license_warning= line, because serving it to outside callers is not permitted until it declares one. The license decides whether the model may be sold, not whether it may run, so a hub that cannot answer (a repository that does not exist, one gated beyond the stored token, or no network) does not stop the plan: it prints license_warning=the license of <repo> was not read (<cause>); …, records no license, and --json carries the cause as license_unread. stado inference list prints the recorded license in its LICENSE column, and not read; plan the deployment again for a deployment whose license was not read.

apply refuses, before touching the host, a deployment that is sold outside the fleet while its license was not read or its chain declares none: sold means a caller of it has a price (stado inference callers), it carries a pay-per-call offer, or a listed network node serves it (stado market node --deployment). The refusal names each sale and what the license lacks (<name> is sold outside the fleet (caller <c> pays for it; …) and its model chain declares no license for <repo> (…)); declare the license on the repository and plan again, or withdraw the sale. A deployment only the fleet uses is applied with the plan's warning.

apply executes one persisted plan only if that registry precondition still matches: it installs the container, waits for an authenticated readiness probe, and only then commits the deployment to the registry. A failed runtime, readiness check, or registry compare-and-swap restores the prior runtime.

Install refuses a host whose GPU has an unmanaged active compute process, an endpoint port already in use by anything other than this deployment's own container, and a host that already carries another inference reservation.

Who owns the board

--gpu-mode decides who owns the board. exclusive (the default) keeps the board reserved for inference, and no decision takes it. yieldable makes the local Stado agent the lifecycle owner, and the board goes to the use worth most per board-hour: when an eligible GPU job is queued, the agent values the job at the income its submitter stated (stado submit --income-usd-per-hour … --income-source …), else at the autonomy policy's budgets.internal_usd_per_gpu_hour, and the deployment at what outside callers paid for it (priced caller usage and paid calls) over the policy's freshness.pricing_max_age_seconds, per hour. The job takes the board only when it is worth more; on a tie the deployment keeps it. A side whose value cannot be read — a job with no income and no declared internal value, a caller with no price, a paid call whose answer stated no usage, no policy — is a refused candidate named with its cause and never chosen, so the board stays as it is. When the job wins, the agent pauses the inference container, advertises the released capacity, and resumes inference only after queued and active GPU work has cleared; an admitted job already running keeps the board until it ends. There is no timeout-based eviction, and the route's ordered provider fallback remains available while the local container is yielded.

Each decision is kept under market/decisions/<worker>/ with every candidate, its value and source or why it was refused, the use the board held, the one chosen and why, and the agent's diagnostics carry the latest as board_decision. stado market decisions --target WORKER [--since-hours 24] [--json] (stado.market-decisions.v1) prints them; Desktop's Registry screen runs it.

A deployment holds one board, named by the driver's UUID in resources.gpu: --gpu GPU-<uuid> (as stado inference blockers prints it) names it, and without it plan takes the board with the most free memory from the inventory's BOARDS line. The container is started with --gpus "device=<uuid>", the reservation carries the same UUID, and on a host with several boards an exclusive reservation withdraws only that board from admission: GPU jobs and rentals stay eligible on the others, and the agent's capacity line says exclusive inference reservation '<name>' holds board <uuid>; GPU claims stay eligible on the host's other <n> board(s). A deployment written before boards were named has no gpu and still holds every board of its host; so does a reservation whose board the driver does not report (inference_gpu_error in the agent's diagnostics names it). plan refuses --gpu naming a board the host does not have, listing its boards, a host that reports no board, and the registry refuses a gpu that is not GPU-<hex> or a board another running deployment of the same host already holds.