Fleet-owned agent sandboxes

stado sandbox fleet operates isolated execution environments on a registered Linux server, including a bare-metal server rented outside Stado. It does not purchase a Hetzner or OVH server. Enroll an existing server through stado fleet enroll, then inspect its sandbox prerequisites.

These environments are gVisor containers, not virtual machines. Docker must register the runsc runtime and use cgroup v2. Stado refuses to fall back to ordinary runc. The host also needs flock from util-linux and permission to use Docker and the host lock directory. Stado does not install these prerequisites or restart production services as a side effect.

Isolation and resources

Each sandbox requires an immutable OCI image (IMAGE@sha256:DIGEST) containing /bin/sh and sleep with infinity support. CPU cores, RAM bytes and the process ceiling are explicit positive integer inputs. Docker enforces these limits; setting the memory-swap ceiling equal to RAM disables swap. The root filesystem is read-only, capabilities are dropped, privilege escalation is disabled and networking is none unless egress is declared (Declared egress). Each environment has its own writable volume at /workspace; no host directory or Docker socket is mounted. A GPU reaches a sandbox only when declared: create --gpus N gives it N of the host's NVIDIA GPUs through gVisor's nvproxy (docker create --runtime runsc --gpus N), which requires runsc registered in Docker with --nvproxy and a host NVIDIA driver that runsc nvproxy list-supported-drivers lists. Creation refuses with runsc is registered without --nvproxy before the sandbox container is made when the runtime lacks it, removing the workspace it had made, reads back the GPU request (GPU request verification failed) and the NVIDIA devices inside the sandbox (no NVIDIA device reached the sandbox through nvproxy); the com.wisent.stado.sandbox.gpus label records the count, and fork gives the new sandbox the same count. gVisor exposes only /dev/nvidiactl, /dev/nvidia-uvm and /dev/nvidia#, and its supported GPUs, drivers and ioctls are gVisor's (https://gvisor.dev/docs/user_guide/gpu/).

CPU limits are shared-host quotas, not reserved physical cores or a capacity reservation. Stado does not provision a hypervisor or promise dedicated vCPUs through this command family.

Lifetime

create --stop-after-seconds N declares how long the sandbox runs after each start: its main process is sleep N instead of sleep infinity, so the sandbox stops by itself N seconds after create or resume, with its workspace kept as stop keeps it, and resume starts the count again. The value is the caller's and is recorded in the com.wisent.stado.sandbox.stop-after-seconds label; creation reads back both the label and the main process and refuses with lifetime declaration verification failed or lifetime enforcement verification failed when either differs. Without the option the sandbox runs until stop or delete, with the label empty: Stado declares no lifetime of its own. Zero, negative and malformed values are refused before any host is contacted. An automatic stop ends every process in the sandbox, detached executions included, exactly as stop does.

Declared egress

Without --egress a sandbox has no network at all. create --egress HOST (repeatable, lowercase host names) with --egress-proxy-image IMAGE@sha256:DIGEST, an immutable image holding tinyproxy, lets it reach those hosts over HTTP and HTTPS and nothing else. The sandbox is attached only to its own internal Docker network (stado-sandbox-NAME-egress, created --internal, with no route out of the host); the one container on that network and the default bridge is its egress proxy (stado-sandbox-NAME-proxy, under gVisor, read-only, no capabilities), whose configuration denies by default and passes each declared host name exactly (FilterDefaultDeny Yes, FilterType ere, FilterURLs Off, one anchored pattern per host) and allows CONNECT only to port 443. The sandbox's HTTP_PROXY and HTTPS_PROXY name that proxy; a program that ignores them, or speaks another protocol, reaches nothing. Stado changes no firewall of the host. The proxy's log (docker logs stado-sandbox-NAME-proxy) shows what was asked and refused; the sandbox's com.wisent.stado.sandbox.egress and com.wisent.stado.sandbox.egress-proxy-image labels record the declaration. stop stops the proxy after the sandbox, resume starts it before, fork gives the new sandbox the same declaration with a proxy of its own, and delete removes the proxy, its configuration volume and the network and confirms they are absent. A malformed host, --egress without a proxy image, a proxy image without --egress, and a proxy image without a digest are refused before any host is contacted; an existing proxy, network or configuration of the same name is refused rather than adopted.

Use images containing the tools and dependencies a build needs: a sandbox without declared egress cannot download packages. This is not yet a public multi-tenant sandbox sales service, customer billing system or microVM hypervisor. Provider costs do not establish a retail price or a claim that it is cheaper than every competing service.

Placement across the fleet

--host is optional on every operation except inventory. Without it, Stado reads every registered Linux host at once with one read-only program and decides from what each host answered about itself, never from another product's store:

• create puts the sandbox on a host that has gVisor runsc and cgroup v2 (and runsc --nvproxy when --gpus is declared) and has free at least the declared cores, RAM and GPUs; of those, the one with the most free RAM, then the most free cores, then by name. Free cores are the host's nproc less the CPU limits of its running fleet sandboxes; free RAM is the kernel's MemAvailable, but no more than MemTotal less the RAM limits of its running fleet sandboxes; free GPUs are the GPUs nvidia-smi --list-gpus lists less those running sandboxes requested. A stopped sandbox holds nothing until it is resumed, so a host can be full when one is resumed there. CPU limits stay shared-host quotas, as above.
• Every named operation (show, usage, update, exec, logs, stop, resume, fork, delete) runs on the host that keeps that sandbox. Names are kept unique across the fleet: create and fork refuse a name another host already keeps (sandbox NAME already exists on HOST).
• list lists every host's fleet sandboxes, each entry carrying host.

The receipt carries placement: for create, chosen, the demand and, for every host, fits, the reason it cannot take the sandbox and the room it reported (cores, memory_available_bytes, memory_total_bytes, gpus and what running sandboxes hold); for a named operation, located and whether each host holds it. Registry targets that are not Linux hosts reachable through their channel are listed with why they were not asked. A host that cannot be read is named with its own failure and does not stop the others; list then prints what it read and fails naming the hosts it lacks, so a partial listing is never taken for the whole fleet. Refusals:

• no registered Linux host has room for sandbox NAME (C cores, M bytes of RAM, G GPUs); no sandbox was created: HOST: reason; ...
• no registered Linux host keeps sandbox NAME; asked HOSTS; not read: HOST: reason (stated not_found)
• sandbox NAME exists on HOST and HOST; name the one meant with --host

Commands

Once the host operation returns, Stado prints a JSON receipt containing schema, host, operation, ok, exit_code, stdout, stderr and result. Runtime failures retain the actual Docker operation's output and nonzero exit status. Argument validation, registry authorization and transport failures can happen before a host receipt exists; these fail through Stado's CLI error diagnostics and must not be interpreted as an empty fleet or successful operation. An inventory response is not proof of a working sandbox: creation also starts gVisor and executes a filesystem probe inside it.

stado sandbox fleet inventory --host example-host
stado sandbox fleet list --host example-host
stado sandbox fleet create example-build --host example-host --image "$SANDBOX_IMAGE" --cpus "$SANDBOX_CPUS" --memory-bytes "$SANDBOX_MEMORY_BYTES" --pids-limit "$SANDBOX_PIDS_LIMIT" [--stop-after-seconds "$SANDBOX_STOP_AFTER_SECONDS"]
stado sandbox fleet show example-build --host example-host
stado sandbox fleet usage example-build --host example-host
stado sandbox fleet prove example-build --host example-host --connect-seconds "$SANDBOX_CONNECT_SECONDS"
stado sandbox fleet update example-build --host example-host --cpus "$SANDBOX_CPUS" --memory-bytes "$SANDBOX_MEMORY_BYTES" --pids-limit "$SANDBOX_PIDS_LIMIT"
stado sandbox fleet exec example-build --host example-host --command 'printf build-result > /workspace/result'
stado sandbox fleet exec example-build --host example-host --command 'your-worker-command' --detach
stado sandbox fleet logs example-build --host example-host --execution "$EXECUTION_ID"
stado sandbox fleet stop example-build --host example-host
stado sandbox fleet resume example-build --host example-host
stado sandbox fleet stop example-build --host example-host
stado sandbox fleet fork example-build --host example-host --to example-build-copy
stado sandbox fleet delete example-build --host example-host --confirm example-build

Without --host, the same operations place and find the sandbox themselves (Placement across the fleet):

stado sandbox fleet create example-build --image "$SANDBOX_IMAGE" --cpus "$SANDBOX_CPUS" --memory-bytes "$SANDBOX_MEMORY_BYTES" --pids-limit "$SANDBOX_PIDS_LIMIT"
stado sandbox fleet exec example-build --command 'your-worker-command'
stado sandbox fleet list
stado sandbox fleet delete example-build --confirm example-build

Supply SANDBOX_IMAGE, CPU, RAM and process limits from your workload requirements, not these examples. show returns Docker's actual inspection document, including State, HostConfig, Mounts and image identity. list reads only containers carrying Stado's fleet sandbox ownership label.

update changes the three resource ceilings and reads them back. A Docker update that fails returns the actual observed sandbox configuration beside the error; it does not claim that a partially applied change was rolled back. The creation labels record original requested values; HostConfig is the current enforced configuration.

usage reads what a running sandbox has used since its last start, from its own cgroup v2 counters: cpu_usage_usec from cpu.stat, memory_peak_bytes from memory.peak, memory_current_bytes, processes from pids.current, and workspace_kib, the workspace size du -sk reports, with since, the start the counters begin at. It is a measurement, not a bill: no price is applied. A stopped sandbox has no counters and is refused with sandbox NAME is not running; its usage counters exist only while it runs since its last start; a kernel without memory.peak is refused naming the cgroup directory rather than reporting an invented peak.

Foreground execution returns the command's output and fails when that command fails. Detached execution returns a unique execution UUID; acceptance means Docker accepted the execution request, not that the command started or completed successfully. Its stdout, stderr, state and exit code are retained under /workspace/.stado-executions/<UUID>. Read those through logs while the sandbox is running. Stopping the sandbox ends all its processes, not just a single detached job. It preserves the workspace, not process memory; resume starts the environment again and does not resume interrupted processes. A state file still saying running after interruption is not evidence that the old process survived.

exec --credential NAME=ITEM#FIELD (repeatable) hands a credential field to the command inside the sandbox, read exactly as stado credentials run --env reads it: from the selected store, or with --route ROUTE --consumer ID --grant-file FILE under that consumer's own Skarbiec grant, never the store administrator. A field the grant does not cover is refused before anything reaches the host. The value travels in the host program on standard input, is exported in the host shell and handed to docker exec -e NAME by name, so the command alone has it in its environment: it is in no argv, file, receipt or Stado output, and a detached execution's retained files hold only what the command itself writes. A sandbox has no network, so this is the way a secret reaches it.

fork makes a new sandbox from a stopped one. The source must be stopped so its workspace is copied whole. The new sandbox takes the source's image, its currently enforced CPU, RAM and process ceilings and its declared lifetime. The copy runs in a gVisor container from the source's image with no network and no capabilities, reading the source workspace read-only. A digest over every file's name and content is computed on both workspaces; the new sandbox is built and started only when the two digests are equal. The receipt prints workspace digest: ... and then the new sandbox's inspection. When they differ, or any step fails, the new workspace and container are removed and the source is untouched. Empty directories are copied but are not part of the digest.

Delete permanently removes the managed container and its workspace and verifies that both are absent. It refuses an object without Stado's ownership label. An occupied container or volume name is refused instead of adopting unrelated data. Creation, forks, resource updates and lifecycle changes are serialized through a host-side lock; execution and inspection do not hold that lock, so a running command does not prevent stopping its sandbox.

Refusals and diagnostics

• --image must name an immutable IMAGE@sha256:<64 lowercase hex digits>; no sandbox was created
• --to must name a new sandbox other than the source: a lowercase letter, then lowercase letters, digits and hyphens; nothing was forked
• sandbox NAME is running; stop it before forking so its workspace is copied whole
• the copy of NAME's workspace differs from it (...); no sandbox was kept
• --command must not be empty
• --env "SPEC": ...; write it as NAME=ITEM#FIELD or NAME=role:ROLE#FIELD for a malformed --credential, and the store's own refusal, naming the variable and the sandboxed command, when a field cannot be read or the grant does not cover it
• --execution must be the UUID returned by exec --detach
• --confirm must equal the sandbox name; nothing was deleted
• sandbox names must begin with a lowercase letter and contain only lowercase letters, digits and hyphens
• gVisor runsc is not registered in Docker; no runc fallback is allowed
• Docker CLI is missing; install the sandbox runtime on this host
• flock is missing; install util-linux before sandbox operations
• sandbox NAME is not running; start it before proving its isolation
• the sandbox has no bash with timeout, so no connection could be tried from inside it; nothing was proven
• the host's own addresses could not be read; nothing was proven, and the same for its listening ports, or ip/ss missing
• prove exits non-zero and lists in result.problems every route and setting that fails: the sandbox opened a connection to the host's ADDRESS:PORT (an address of the host's interfaces or its default gateway, on a port the host listens on), runtime is not runsc, root file system is writable, capabilities are not all dropped, no-new-privileges is not set, mounts KIND SOURCE from the host for anything but the sandbox's own workspace volume and its tmpfs. A sandbox handed to anyone outside the fleet must pass it first; result.connections_tried counts the attempts.

Resource arguments reject zero, negative and malformed values before contacting a host. Unsupported host platforms name the selected host and observed platform. Docker failures retain both output streams rather than reporting an empty fleet or a successful create.

Desktop and API

Compute providers → Fleet sandboxes exposes prerequisite inspection, listing, creation with every resource input and the optional stop-after lifetime, resource updates, state inspection, foreground/background execution, execution logs, stop, resume, fork into a new sandbox and confirmed deletion. Every operation but prerequisite inspection takes the host as optional: left empty, Stado places or finds the sandbox as --host omitted does on the command line, and the receipt shows placement. The complete native operation receipt retains errors. It uses the authenticated Stado operator API, not a separate desktop shell implementation. Mutation requests use the existing RUN_MUTATION confirmation contract. Existing Boat sandbox commands and controls are separate and unchanged.

Real qualification

Run node tests/sandbox/fleet.mjs from the Stado checkout with these inputs supplied explicitly:

Input	Meaning
STADO_BIN	Actual Stado binary to qualify
STADO_TEST_SOURCE_REVISION	Clean committed source revision identified by that binary
STADO_SANDBOX_TEST_HOST	Registered isolated Linux qualification host
STADO_SANDBOX_TEST_IMAGE	Immutable BusyBox-compatible image digest
STADO_SANDBOX_TEST_CPUS	Initial positive CPU quota
STADO_SANDBOX_TEST_MEMORY_BYTES	Initial positive RAM ceiling
STADO_SANDBOX_TEST_PIDS_LIMIT	Initial positive process ceiling
STADO_SANDBOX_TEST_NANO_CPUS	Expected Docker NanoCpus value for the initial quota
STADO_SANDBOX_TEST_UPDATED_CPUS	Different positive CPU quota for the update
STADO_SANDBOX_TEST_UPDATED_MEMORY_BYTES	Different positive RAM ceiling for the update
STADO_SANDBOX_TEST_UPDATED_PIDS_LIMIT	Different positive process ceiling for the update
STADO_SANDBOX_TEST_UPDATED_NANO_CPUS	Expected Docker NanoCpus value after the update
STADO_SANDBOX_TEST_STOP_AFTER_SECONDS	Positive lifetime for the second sandbox, which must stop by itself and keep its workspace
STADO_SANDBOX_TEST_CREDENTIAL	ITEM#FIELD of a disposable credential the selected store holds; only its digest, computed inside the sandbox and by credentials run, is retained
STADO_SANDBOX_TEST_PROXY_IMAGE	Immutable image holding tinyproxy, the egress proxy of the third sandbox
STADO_SANDBOX_TEST_EGRESS_HOST	A host serving HTTP the third sandbox declares and must reach through its proxy
STADO_SANDBOX_TEST_DENIED_HOST	Another host serving HTTP the third sandbox does not declare and must not reach

The image must provide BusyBox-compatible sh, sleep, cat, test, mkfifo, cp, find, sort and sha256sum. The runner drives the actual candidate binary on the declared Linux host, inspects configured resource ceilings, exercises workspace persistence and filesystem restrictions through real commands, tests meaningful refusals, stops and resumes, forks a stopped sandbox and reads the copied workspace and limits from the new one, lets a declared lifetime stop a second sandbox by itself, deletes and verifies external absence. The configuration readback alone is not a resource-exhaustion or adversarial isolation qualification. Its report under .build/sandbox/lifecycle-*/report.json retains source and binary identities, commands, statuses and actual results. Missing dependencies and a dirty or mismatched source identity remain blocked or failed, never passed. A build, CLI help output or runtime inventory does not count as a passing lifecycle test.

GPU passthrough has its own journey, run on a registered Linux host with an NVIDIA GPU and runsc registered with --nvproxy: node tests/sandbox/gpu.mjs with STADO_BIN, STADO_TEST_SOURCE_REVISION, STADO_SANDBOX_GPU_HOST, STADO_SANDBOX_GPU_IMAGE (an immutable CUDA-capable image), the CPU, RAM and process inputs above, STADO_SANDBOX_GPU_COUNT and STADO_SANDBOX_GPU_COMMAND (a CUDA sample or nvidia-smi -L the image holds). It creates a sandbox with --gpus, reads the label and runtime back, runs the declared GPU command inside it, retains its output, deletes the sandbox and confirms absence; its report is under .build/sandbox/gpu-*/report.json.