Configuration

First-run config and deployment profiles

stado config init creates only a schema-versioned local queue profile: local compute, local primary and backup stores, one deployment identity, and a loopback dashboard. It contains no Wisent service routes, production clients, cloud locators, or credentials. Existing legacy files migrate explicitly with stado config migrate; the exact prior file is preserved beside the migrated document.

STADO_CONFIG selects the operator-owned deployment profile. Stado ships no company deployment profile: stado config init creates a neutral local profile, and an operator extends that file with provider order, explicit provider fences, storage, object/release/service verifiers, workload grants and service routes. Those decisions remain outside the public source tree and cloud CLI state.

storage.backend selects the write authority and first read target. storage.backup configures a disaster-recovery mirror: a primary mutation commits first and is mirrored afterwards. Ordinary storage clients may retry a failed primary read against that mirror; authority-sensitive readers, including the object API server, use primary-only mode instead. A successful primary answer that an object is absent never falls through to the backup. A null or empty storage.backup.backend disables the Stado-managed mirror. Changing the write authority still requires an explicit fenced migration or the fixed-root reconciliation described in disaster recovery; configuration and read fallback never promote the backup.

Only route-local or process-local values should be overridden:

Var	Purpose
STADO_CONFIG	Readable deployment profile path.
STADO_API_URL	Operator-selected canonical HTTPS Stado origin, including /api/release/object; overrides api.url. Plain HTTP is accepted only on loopback.
STADO_RELEASE_ORIGIN	Object-service origin used by the stado-landing gateway. Public callers still use STADO_API_URL; the gateway's upstream is selected independently.
STADO_API_TOKEN	Dedicated caller token for its mapped object namespace.
STADO_MACHINE_API_TOKEN	Machine submit/status/cancel token.
STADO_SERVICE_API_TOKEN	Caller-specific deployer token; accepted only for mapped service names/actions.
STADO_REGISTRY_API_URL	Desktop's local source and the endpoint assigned to its registry API credential.
STADO_REGISTRY_API_TOKEN_FILE	Desktop's owner-only raw registry client bearer file; defaults to ~/.stado/registry-api-desktop-token.
WC_REGISTRY_API_CLIENTS	Exact registry API client mapping; overrides registry_api.clients.
WC_REGISTRY_SKARBIEC_URL	Server-side Skarbiec origin for the registry API verifier.
WC_REGISTRY_SKARBIEC_CONSUMER	Exactly stado-registry-api-verifier.
WC_REGISTRY_SKARBIEC_TOKEN_FILE	Dedicated server verifier grant; defaults to ~/.stado/stado-registry-api-verifier-skarbiec-token.
STADO_RELEASE_VERSION	Required exact immutable Stado runtime version.
STADO_RELEASE_PLATFORM	Required exact Stado runtime platform for dispatched agents.
STADO_ALERT_CHANNELS	Explicit comma-separated optional adapters: slack, telegram, sendgrid, gcp-pubsub.
STADO_HOST_HEALTH_API_URL	Authenticated Stado host-health origin.
STADO_HOST_HEALTH_SKARBIEC_URL	Skarbiec origin for the route-only host-health publisher.
STADO_HOST_HEALTH_SKARBIEC_CONSUMER	Exactly stado-host-health-beacon.
STADO_HOST_HEALTH_SKARBIEC_TOKEN_FILE	Owner-only grant scoped only to stado-host-health-api.
STADO_CREDENTIALS_STORE	Requested credential backend (skarbiec, skarbiec://<https-origin>, or file://<absolute-path>). A value different from credentials.store is a pending migration.
STADO_CREDENTIALS_ADMIN_URL	Canonical Skarbiec origin used only for store administration and migration; defaults to the shared Skarbiec URL.
STADO_CREDENTIALS_ADMIN_CONSUMER	Skarbiec bootstrap consumer used only for store administration and migration.
STADO_CREDENTIALS_ADMIN_TOKEN_FILE	Owner-only bootstrap grant for the credential-store administrator.

What each bearer opens, and how every token is verified against its own Skarbiec item: security.

The public origin is independent of host-control routes and network providers. Changing the selected HTTPS endpoint does not require changing a release client. A gateway's upstream is selected by its deployment configuration, not by detecting a tunnel program.

Web products select their edge separately. A product with edge: "stado" requires web_api.edge with target, address and contact; stado web edge declare records that selection. Without a valid declaration, stado web status returns edge-unconfigured, the exact edge_error, and a non-zero exit code. It does not treat an unknown expected address as a healthy public connection.

Cloud-provider locators and credentials are not caller overrides. An enabled provider adapter receives its exact profile and provider-plugin identity; a workload-agent grant contains only the provider-neutral application items in agent.skarbiec.items. It must never contain stado-gcp, stado-azure, or stado-aws, and no bootstrap, health, recovery, or release path invokes gcloud, gsutil, or az.

Product data enters through stado://<namespace>/<key> and the authenticated Stado object boundary. Immutable artifact manifests may additionally reference provider-native az://, gs://, and s3:// locations, plus hf:// and HTTPS; access still resolves through authenticated provider adapters, and embedded credentials or sensitive query parameters are rejected.

How the gateway resolves a namespace and authorizes each key and action: object store.

Optional alerts are disabled when alerts.channels is absent or empty. Enabling a channel authorizes only its own credential lookup and network route. The Pub/Sub topic and SendGrid recipient are inert unless their adapters are also enabled. An alert failure is isolated from scheduling, execution, health, and the other channels.

Credential store

One selector owns every application credential, including cloud/provider credentials, service tokens, and SSH host keys:

{
  "credentials": {
    "store": "skarbiec",
    "admin": {
      "url": "http://127.0.0.1:8787",
      "consumer": "local-operator",
      "token_file": "~/.stado/local-operator-skarbiec-token"
    }
  }
}

credentials.store is the committed source of truth. STADO_CREDENTIALS_STORE is its process-level override. Supported locators are skarbiec, skarbiec://<https-origin>, and file://<absolute-path>. The file backend is an owner-only local/offline manager; Skarbiec adds encryption, scoped grants, audit, recovery recipients, and remote HTTPS access.

Changing the environment selector does not make Stado read an empty backend. It creates a fail-closed pending migration:

export STADO_CREDENTIALS_STORE=file:///secure/stado-credentials.json
stado secrets migrate

Without an environment override, stado secrets migrate --to <locator> performs the same change. Migration snapshots every active item and its type, requires an empty destination, copies and reads every value back, commits credentials.store, and only then removes the source items. A failed copy, verification, config write, or source cleanup rolls back to the previous store. Normal reads and writes remain blocked while the environment and committed selectors differ.

All stado secrets CRUD, provider reads, scoped verifier reads, and stado fleet key operations use this selector. There is no OpenSSH-file fallback for host channels. Only a backend's own bootstrap credential remains outside the selected store: putting the grant needed to unlock a manager inside that same manager would be circular. For Skarbiec, this is the owner-only admin token file named above.

Registry API access

Desktop's registry, inventory, cleanup, convergence and storage-reconciliation requests use one source-bound client bearer, not its Wisent account session. Configure the server's deployment profile with an exact client and only the actions that client should perform:

{
  "registry_api": {
    "clients": {
      "stado-desktop": {
        "item": "stado-desktop-registry-api",
        "actions": [
          "policy-read",
          "converge-read",
          "converge-apply",
          "storage-reconcile-read",
          "storage-reconcile-apply"
        ]
      }
    },
    "skarbiec": {
      "url": "http://127.0.0.1:8787",
      "consumer": "stado-registry-api-verifier",
      "token_file": "~/.stado/stado-registry-api-verifier-skarbiec-token"
    }
  }
}

The Skarbiec item must be named <client>-registry-api; its token field holds the client's raw bearer. The server's separate verifier grant reads that field. Do not put the verifier grant or an account session token in Desktop's client file. registry_api.skarbiec.url defaults to the deployment's Skarbiec origin, and the verifier grant must have its own file.

policy-read permits the projected registry and host inventory; policy-write permits policy updates. cleanup-read and cleanup-run permit the janitor's report and action. converge-read permits the Services report; converge-apply permits delivery. storage-reconcile-read permits only the storage transaction's Status request; storage-reconcile-apply independently permits Run, Resume, Rollback and Finalize. An apply-only client does not gain read permission, and a read-only client cannot deliver a release or change a storage transaction.

The storage action names and their dashboard route must be deployed together. Older Stado servers reject these names as unknown configuration instead of ignoring them. Upgrade the server before adding the two actions; before rolling back to an older server, remove them or restore that server's prior client action list. Leaving the new grants in an older profile is not a compatibility shim.

In Desktop Settings → Registry API access, save the exact source endpoint and an absolute token-file path. Desktop rereads the file on each request. The two STADO_REGISTRY_API_* environment variables override the saved fields; when either is present the settings are read-only. The default client path is ~/.stado/registry-api-desktop-token. A missing, empty or unreadable client file is reported locally; a credential assigned to another endpoint is not sent.

An absent client mapping or a missing, wrong or action-ineligible bearer returns HTTP 401. Malformed mapping or unreadable verifier state returns 503, not permission to use another credential. A completed convergence or storage reconciliation product result returns HTTP 200 with exit_code and the complete report, even when that exit code is nonzero; storage reconciliation also retains refusal. See Services and storage root authority handoff.

Registry

stado-rs/data/registry.json is an intentionally empty bundled seed. A release binary therefore knows no operator host, service, route, release key or workload until the operator creates a registry with stado config init or publishes one through the selected store. stado registry push and stado registry pull resolve that store from STADO_CONFIG, preserve generation fencing, and surface an unreachable store as failure rather than silently switching providers.

Each target entry:

{
  "name": "my-workstation",
  "kind": "local",
  "ssh": "user@host-or-ip",       // used by `wc bootstrap` to install the agent
  "gpu_type": "nvidia-tesla-t4",  // SKU label the agent broadcasts
  "vram_gb": 96,                  // total GPU VRAM
  "gpu_power_limit_watts": 300,   // persistent board power cap, reconciled by the agent
  "env_overrides": { "WISENT_DTYPE": "auto" },
  "agent_args": ["--gpu-type", "nvidia-tesla-t4"]
}

A target declares capabilities and policy, not a concurrency limit. The worker measures live CPU, RAM, disk, running jobs, and per-device VRAM every poll and admits as many compatible jobs as those resources allow. Legacy slots, max_concurrent, and WC_LOCAL_SLOTS values are discarded when a registry is read and are never written back.

gpu_power_limit_watts is valid only on local targets and must be a positive integer. Set it with stado host gpu-power-limit TARGET WATTS: Stado writes the generation-fenced declaration, applies it immediately, and the local agent reconciles it at startup and every five minutes. A declared cap that the NVIDIA driver cannot apply closes admission on that host instead of silently running new jobs outside the policy.

A coordinator entry pins the scheduling-tick driver:

{
  "name": "local-control-plane",
  "runtime": "daemon",
  "host": "https://stado.wisent.com",
  "interval_seconds": 180,
  "state_uri": "stado://system/registry",
  "active": true
}

A registry document may also carry the fleet's central enrollment and communication catalog. enrollment declares which registration paths are allowed — allow_join for machine-initiated stado fleet join/approve, allow_enroll for control-plane stado fleet enroll, and require_verified_hostname for the verified-identity contract. channels declares how machines reach the control plane. Both sections are additive: a document without them is unrestricted, which stado fleet catalog reports explicitly. The stado fleet commands enforce the catalog in their preflights, before any write.

{
  "enrollment": {
    "allow_join": true,
    "allow_enroll": true,
    "require_verified_hostname": true
  },
  "channels": {
    "control_plane": ["loopback"],
    "notes": "any address that resolves: LAN, mDNS, tailnet"
  }
}

Logical services and local resolvers

The optional top-level service_directory is the fleet routing contract. authority names the target and absolute Stado binary that serve canonical versioned snapshots and commit placement changes; other hosts use their local registry only to bootstrap that SSH path. generation is monotonic. Each logical service declares its active host, host-relative loopback origin on every eligible host, optional placement profile, and the exact consumers and capabilities allowed to resolve it.

Per-target service_resolver policy declares the loopback resolution API and stable compatibility adapters for workloads on that host:

{
  "service_directory": {
    "authority": {
      "target": "control-host",
      "command": "/opt/stado/bin/stado"
    },
    "generation": 7,
    "services": {
      "model-router": {
        "placement_profile": "router-with-credentials",
        "active_host": "control-host",
        "endpoints": {
          "control-host": {"url": "http://127.0.0.1:8080"},
          "worker-host": {"url": "http://127.0.0.1:8080"}
        },
        "consumers": {
          "application": {"capabilities": ["model-routing"]}
        }
      }
    }
  },
  "targets": [{
    "name": "worker-host",
    "kind": "local",
    "service_resolver": {
      "api_bind": "127.0.0.1:17600",
      "refresh_seconds": 5,
      "max_stale_seconds": 15,
      "adapters": [{
        "service": "model-router",
        "bind": "127.0.0.1:17601",
        "consumer": "application"
      }]
    }
  }]
}

All resolver and adapter binds must be loopback. The authority target must have registry SSH transport, and the authority command must be an absolute, component-normal path. Remote host-relative endpoints require targets[].ssh; they are never rewritten into client configuration. The resolver refuses an unknown consumer, an active placement transaction, a rolled-back directory generation, or a cache older than the configured limit.

Signed product release control

The optional top-level release_control object owns product release trust and desired state. trusted_keys contains Ed25519 public keys only. Each product binds one logical service, exact archive paths, schema versions, a blue-green policy, and registered host policy. desired and previous are written only by stado release promote|rollback; their platform references point beneath the same immutable stado://releases/<product>/<version>/<platform>/ coordinate.

Each target uses two private candidate ports and one loopback stable bind. Its state, runtime, log, and install roots are absolute. A legacy launchd label and plist are optional one-time migration inputs: the agent starts and proves the candidate before disabling the legacy service, then owns the stable port.

Registry validation rejects unknown products/services/targets, non-loopback stable binds, reused ports, unsafe paths, untrusted desired keys, incomplete platform sets, non-monotonic zero generations, and rollback windows shorter than drain.

A block that declares trusted_keys and an empty products map is accepted. It states what the document trusts and nothing about rollout, which is exactly what stado scratch create emits into a lease's registry: the fleet's schema_version, generation and trusted_keys, no products. A block that declares neither is the only one refused, as registry.release_control must declare trusted_keys or products.

Local inference

The optional top-level inference section is the single desired-state and routing catalog for Stado-managed vLLM. gateway_target is the registered host running Brama; deployments run on registered local GPU targets and expose their OpenAI-compatible endpoint only on a Tailscale IPv4 address.

{
  "inference": {
    "gateway_target": "my-brama-host",
    "deployments": [],
    "routes": {}
  }
}

The lifecycle is deliberately two-step and generation-fenced:

Create the shared bearer without exposing it in argv, output, or a file:

stado inference init-credential

Bootstrap the gateway snapshot before Brama's first managed start:

stado inference route set example-client/chat/primary \
  --to openai/gpt-4.1-mini \
  --expected absent --gateway gateway-host

stado inference plan chat-primary \
  --host gpu-host \
  --image 'vllm/vllm-openai@sha256:<image-digest>' \
  --cache-dir /srv/stado/inference/chat-primary \
  --model 'example/model' \
  --gpu-mode yieldable \
  --max-model-len 16384 \
  --revision '<model-revision>'
stado inference apply <plan-id>
stado inference doctor chat-primary
stado inference verify chat-primary
stado inference route set example-client/chat/primary \
  --to chat-primary \
  --fallback openai/gpt-4.1-mini \
  --expected openai/gpt-4.1-mini \
  --gateway gateway-host

The immutable model revision and container image digest prevent silent model or runtime replacement.

kv_cache_memory_gb sets a fixed vLLM KV-cache allocation in GiB. Omitting it preserves the pinned image's own memory policy. A smaller cache releases VRAM without changing model weights or output quality, but lowers the number of tokens and long requests that can remain active concurrently.

gpu_mode defaults to exclusive, which keeps the GPU reserved for inference. yieldable makes the local Stado agent the lifecycle owner: it pauses the inference container when an eligible GPU job is queued, advertises the released capacity, and resumes inference only after queued and active GPU work has cleared. Eligibility includes provider, accelerator, host pin, centralized assignment, and capacity constraints; there is no timeout-based eviction. The ordered provider fallback remains available while the local container is yielded. route set therefore accepts a non-ready yieldable deployment only as the primary of a route with at least one ordered fallback. It still requires an exclusive primary and every local fallback to be ready, so an unavailable deployment cannot be published as the route's only destination.

If plan or apply reports an unmanaged GPU workload, inspect it through the same target-scoped host channel instead of opening an ad hoc SSH session:

stado inference blockers --host gpu-host
stado inference release --host gpu-host \
  --identity <PID:START_TICKS>

blockers reports the executable, owner, VRAM use, cgroup, and an identity made from both PID and /proc start ticks. release refuses a stale identity, sends TERM, and waits for exit. Add --force only to escalate that same verified process to KILL; it never accepts a bare PID.

A cancelled or failed pre-commit plan can leave a runtime or root-owned model cache without a registry deployment. Clean that exact saved plan through the managed channel:

stado inference plan-logs <plan-id>
stado inference abort <plan-id> --purge-cache

plan-logs reads the not-yet-committed container logs through the same managed host channel.

abort never changes the registry. It stops only the runtime described by the immutable local plan, removes its cache through the pinned container runtime, and consumes the plan after successful cleanup. Set --cache-dir during plan when the target's home filesystem is not the intended model volume.

plan inventories the host, requires Docker, NVIDIA tooling, and a live Tailscale address, then saves an immutable plan bound to the current registry digest. apply rechecks that precondition, installs the digest-pinned vLLM container under Docker's unless-stopped supervisor, waits for an authenticated readiness probe, and only then commits the deployment. A failed runtime, readiness check, or registry compare-and-swap restores the prior runtime.

The deployment and Brama use one centrally stored credential item: provider:local-openai, containing a non-empty token field. stado inference init-credential generates and stores it without printing the token, and refuses to overwrite an existing item. Deliberate rotation can use stado credentials put provider:local-openai with JSON on standard input, but must be coordinated with runtime replacement; never place the token in argv or registry data. Route changes require --expected, probe the destination first, stage an owner-only route snapshot on the gateway, compare-and-swap the registry, and then atomically commit the snapshot. Brama reloads that file per request, so cutover needs no backend restart. Ordered --fallback destinations are attempted when the primary provider fails; An external provider fallback therefore preserves the same model contract while local inference is unavailable. rollback reinstalls the recorded prior deployment; retire refuses while any primary or fallback route still selects the deployment and retains model cache unless --purge-cache is explicit.

Quotas

The provider-neutral config/quotas.json object contains reservation overlays, not invented capacity. A reservation subtracts operator-owned capacity from a live provider limit before dispatch. Missing or unreachable live quota data is reported as unavailable; Stado does not reinterpret it as zero usage or unlimited capacity.

GCP quota reads are part of the preview compute adapter and use the Rust provider boundary. Azure supports configured reservations with incomplete live VM-family coverage. Live AWS quota management is planned and unavailable.

Optional GCP adapter prerequisites

The GCP compute adapter is preview. An operator must explicitly enable it and supply an already provisioned project, canonical store, scoped managed identity or Skarbiec service account, quota overlay, network, image, ownership labels, cost policy, and immutable Stado release. Runtime compute and storage operations stay inside the Rust provider adapters; install, bootstrap, release, health, and recovery paths do not invoke a cloud CLI.

Zone candidates and machine compatibility belong to the selected deployment profile. Resource exhaustion may advance to the next allowed candidate; an authorization, ownership, invalid configuration, or ambiguous provider error must not. GCP is not stable until a release-scoped live test creates an owned VM, boots the pinned agent, runs and collects a workload, exercises cancellation and recovery, and reaps all paid resources.

The active deployment workflow publishes immutable native releases and installs the Rust coordinator. It has no Cloud Function scheduler, mutable package upgrade, provider CLI authentication, or ambient credential fallback.

Workload runtime ownership

Stado does not pin Python, CUDA framework, Hugging Face, NumPy, or application package versions as part of the control-plane contract. A workload declares and owns its runtime, image, command, source revision, artifacts, and verification hook. Provider bootstrap templates may install prerequisites required by a specific workload profile, but those pins are deployment data and do not become dependencies of the Rust agent or local onboarding path.