On this page

The complete test plan

Stado is tested completely when every command and every leaf subcommand that stado --help lists has a test of its own, and that test drives the real stado binary through the whole flow it claims. This page says what such a test is, how it is isolated and run, and which command groups each test area under tests/ defends, so a missing test is found by reading one table.

What one test is

  • One command and its story. A test drives one command: its success and every refusal the command can give, a missing argument, malformed input, an unknown object, a duplicate, an occupied state, an unreadable backend and a wrong type. The test name says which command it defends.
  • The real binary. A Rust test runs CARGO_BIN_EXE_stado through Command; a Node journey runs the binary STADO_BIN names. No mock, stub, canned response, simulated provider or dry run counts.
  • State, not output. Assertions read what the command changed: the registry document, the object written to the store, the file on disk, the exit status. Standard output is supporting evidence.
  • Exact refusals. Each refusal is asserted with its whole sentence, and the test also judges whether the sentence names what failed, the state it observed and what would fix it. A refusal that does not is a defect the test exposes.
  • The fleet lifecycle. A fleet test creates a real isolated fleet through the CLI, checks it was created, edits it and checks the stored change, then deletes it and confirms it is gone.

How a test is isolated and run

Each case creates its own root under the checkout's build directory (CARGO_TARGET_TMPDIR, or .build/<area>-<unique> for a Node journey), passes its own HOME, and sets WC_STORAGE_BACKEND=local, WC_LOCAL_STORAGE_PATH inside that root and STADO_CONFIG naming a file that does not exist, so no test reads or writes the operator's registry, vault or configuration. The root is removed when the case ends.

Every run keeps a report beside it: the source revision, the binary's digest, each command with its exit status, and the result. A run whose real flow could not happen is reported blocked, never passed.

A new test is written only after the operator approves its path with tama tests approve <path> in his own terminal; the block-unapproved-tests gate refuses any other.

Which area defends which command group

The table names, for every command group of stado --help, the test areas whose tests invoke it. An area defends a group only as far as its tests drive the group's leaves: the second column is where to look, not a proof that every leaf is covered.

Command group Test areas under tests/ The plan adds
onboarding none tests/onboarding/walkthrough.rs: the walkthrough and the registry-v2 import with its refusals
capabilities capability
overview capability, overview
impact none tests/impact/inventory.rs: one dependency's inventory and an unknown dependency
resources providers
optimize autonomy
billing billing
cloud none tests/cloud/role.rs: the role contract check and repair, refused without a provider
tunnel none tests/tunnel/routes.rs: a route added, read and removed, an unknown provider refused
waits wait, host_silence, install, mcp, worker_start
availability public_status
machine machine and five areas that read its interface
mcp mcp
submit, status, results, cancel, job submission, status, cancellation, queue, scheduler and others
quality quality
serve service_serving, http, host_health and others
network network
quota none tests/quota/inspect.rs: quota read per provider and an unconfigured provider refused
profiles placement
config config
artifact product
build build, build_status
release release, release_delivery, release_destinations, release_install, release_publishers
ci ci
schedule schedule, scheduler
cost cost
registry capability, delivery and the host areas
fleet fleet, fleet_flush
identity none tests/identity/holders.rs: which host holds which identity, and a stale holder reported
host host_activity, host_exec, host_gates, host_health, host_silence
bootstrap service
market market
instances space_reclaim
sandbox sandbox, providers
recovery storage
storage storage
contributions contributions
credentials credentials, credentials_move, credentials_unlock, vault_authority
queue queue
alerts alerts
service service, service_answering, service_directory, service_images, service_update
egress egress
product product, install, dependencies
placement placement
resolver resolver
database database
web web
dns none tests/dns/records.rs: a record set, read and removed in a zone Stado manages, an unmanaged zone refused
inference none tests/inference/plan.rs: a deployment planned against a real host and an unknown model refused
stream none tests/stream/session.rs: a display session provisioned and closed, a host without a display refused
doctor capability, host_exec, service_directory
workload workload
capacity scheduler, host_health
repair service_answering
runner none tests/runner/profiles.rs: a declared runner profile listed and an unknown profile refused
space space_reclaim, host_gates, privacy
route cli_surface tests/route/directory.rs: a route read and operated without naming a product
scratch storage tests/scratch/lease.rs: a target leased and destroyed when the run ends

The order the plan is built in

  1. A test for every command group marked none, one file per group at the path the table names, each covering success and every refusal of the group's leaves.
  2. For every other group, each leaf subcommand the group's areas never invoke gets its own case in the area that already defends the group.
  3. The groups whose only coverage is incidental (an area that reads the group while testing another) get a test of their own: route and scratch first.

Each test lands with the command it defends, approved and committed in the same change, and runs in the release manifest's quality gates on both supported platforms.

Source: this website