Keeping installed surfaces current

A host keeps its installed surfaces current inside its one Stado process, com.wisent.stado; no separate launch agent runs sync. Declare the cadence and each surface on that process:

stado serve --product-sync-interval-seconds 3600 --product-sync-surface cli --product-sync-surface desktop

Every period the process runs stado product --workspace ~/.stado/products/workspace sync --surface SURFACE --fetch --clone-missing for each declared surface, in order. That workspace is Stado's own: each product's source is cloned into it from its GitHub origin the first time the product is reconciled and fetched on every later pass. The background sync therefore reads nothing under ~/Documents, which macOS lets a program read only after the person at the Mac allows it, and uncommitted work in the operator's own checkouts never holds a product. A surface whose sync fails or exits non-zero is written to the process's standard error as [stado serve product-sync] SURFACE sync failed: <error> or ... sync exited with status N, and the next surface still runs. --product-sync-interval-seconds without a surface, or a surface without the interval, is refused by the argument parser.

Product sync is the process's delivery role: it installs the release that replaces the one running, so nothing else in the process may end it. It starts before every other role, and when another role stops (a resolver that refuses its configuration, a startup read that fails) the process does not end with it as it does otherwise. It writes stado serve pid=PID failed: <the stop>; it keeps product-sync running so a release that replaces this one can still be installed, every later stop of another role on its own line, and keeps syncing at its cadence. A release whose resolver refuses to start therefore does not take the delivery of its own repair down with it on every restart. After each pass the role compares the program the process runs (its length and modification time) with what it started as; when a pass installed a release over it, the role ends with product sync installed a new release over <program>, the program this process runs; the service ends so its init system starts the installed release, and with it the process, as a failure, so launchd or systemd starts the installed release. The real journey tests/delivery/broken_resolver.rs (cargo test --test delivery-broken-resolver -- --ignored with STADO_TEST_SYNC_INTERVAL_SECONDS) holds the resolver's API bind so the real resolver fails, observes the process keep product sync running through a pass, installs the built program over the copy it runs and observes the process end naming it.

stado product schedule and the hourly com.wisent.products.cli-sync and com.wisent.products.desktop-sync launch agents it installed are removed. The catalog lists both as units com.wisent.stado retires once it carries --product-sync-interval-seconds, so the next stado service ensure stado --host HOST boots them out.

Replacing a program other products call

Every stado invocation records, once per caller and command, which executable started it and which command it ran, under ~/.stado/callers/. Nobody writes that list: it is what the installed programs did.

Before stado product install or stado product update places a program, and before stado release local install replaces one, Stado reads the records whose callee is the program being replaced. It keeps those whose caller is another product's recorded installed path and has not changed since it was recorded, and asks the incoming program for each recorded command with <command> --help. A command the incoming program does not answer refuses the installation:

<incoming> does not answer commands installed products still run, so it would break them; update these products to a revision that no longer runs them, then install again: oko cli (installed at ~/.stado/bin/oko, source revision <revision>) runs `~/.stado/bin/stado secrets get`

Update the named product first, then install again. A record whose caller has since been replaced or removed is deleted when it is read, and a caller that is no product's installed path (a shell, a script) is never a reason to refuse. A caller that cannot be read is written to standard error as stado: the program that ran this command was not recorded: <error> and the command still runs.

A record this account cannot read was not written by this account's programs, so it says nothing about what they run: it is named on standard error as caller record <path> cannot be read by this account (<error>); it was not written by this account's programs and is left out of the check, kept, and the installation goes on. A Stado run under another effective user than the owner of HOME (sudo stado …) records nothing, so it leaves no such record behind.

For Stado's resident unit, stado product install stado --surface cli uses the prepared candidate to reconcile the existing native unit from the candidate's catalog before replacing the executable. It checks the installed unit's actual argv with stado quality units --candidate <prepared-stado> --program stado --host <registry-host>. A parser refusal names the host, unit file, argv, candidate and error, and leaves the executable untouched. stado quality parse -- <argv> checks the same parser without running that command. Once files are placed, the real resident process must serve its declared port through stado service serving stado --host <registry-host> --wait --json; an activation failure restores the exact recorded product backup, restarts its unit and reports the observed predecessor state. Stado Desktop → Products keeps the install refusal and status receipt; Services → Managed services shows the unit's last exit and stderr evidence.

An installation that stopped part way

An installation that stopped part way leaves its receipt installing, and the next stado product install of that surface resumes the plan it retained. The receipt is bound to the host it was installing for; an install for another host (or for this machine without --host, when it was bound to one) is refused as refused input, naming the receipt and the one command that undoes it on the machine that holds it:

unfinished installation is bound to another host: id="<product>", surface="<surface>", host=<recorded host>, started_at="<time>", receipt="<home>/.stado/products/<product>/<surface>.json"; requested_host=<requested host>. On the machine holding this receipt, roll back this incomplete installation to its retained predecessor with `env HOME='<home>' stado product --workspace '<workspace>' rollback '<product>' --surface '<surface>'`, then repeat the requested installation. This recovery changes owned files and, for a service, its recorded host's service; it is not run automatically. Do not add --host to the recovery command: that would forward it to another machine instead of resolving this receipt

The recovery command carries the receipt's own HOME, workspace and, when it is not the embedded one, --catalog. A service surface with no retained predecessor is undone with remove instead of rollback, and the sentence says "remove this incomplete surface and its managed service". Stado never runs the recovery itself; run it on the machine holding the receipt, then install for the host you meant.

A receipt retains exactly one predecessor under previous, the state rollback restores, and that predecessor carries no history of its own. Receipts written by older Stado releases nested every earlier installation, so a surface reinstalled a few hundred times was deeper than the JSON parser's recursion limit and every stado product command refused it; such a receipt is read whole, trimmed to one predecessor, and written flat by the next installation or removal.

An installation another one waits on

An installation holds ~/.stado/products/<product>/<surface>.lock during preparation and placement. Its record names the guarded product, PID, operating-system process birth, command, phase, step and acquisition time. The guarded product comes from the lock path, not the command line: an installer can hold its dependency's lock while its own command names a different product. stado product locks --json lists the recorded and observed identities and whether the OS lock is held; Stado Desktop's Products screen shows those rows and refusals. A released lock is reclaimed on the next install even if its stale record names another product, with its recorded product, command, PID and observed birth printed on stderr. Only a held lock whose record names another guarded product is refused with both names and shown as an error in the locks list; it is never overwritten or signalled. A held lock with unverified process identity is not signalled. A queued installation prints czekam with the validated holder and its step; stado waits --json shows the open process, host-channel and lock waits. stado product status <product> --surface <surface> --json reports the holder or a post-placement step under operation with observed running state. The isolated dedicated-host journey is tests/install/resident/locks.mjs.

stado product status stado --surface cli --json

status reports the holder, its phase and step, or the post-placement operation with its recorded PID and process birth time. A stuck holder is not ended by hand: the next installation of the same surface supersedes it, as described below, and status read again shows the result.

A catalog recipe's after_install steps run after the receipt says installed, without holding the surface lock. Stado's release local converge resolves this machine from its local registry record and recycles readers of the replaced binary here. The receipt records the process identity and declared steps as after_install_running until the outcome is saved; each wait is registered with its host and operation and exposed by stado waits --json. The lock is taken back only to record after_install or after_install_failure, and a newer receipt is never overwritten. Stado Desktop's Products inspector shows the current operation while it is running. The real dedicated-host journey is tests/install/resident/converge_wait.mjs; it verifies the exact candidate revision and that the surface lock is free while the reader step stands.

Reader convergence calls the ordinary service update --from-archive --refresh-image with a local-only guard. A registry change between reader selection and update cannot redirect the step to SSH: the update refuses with the selected host and says no remote command was started. The same refusal and the retained after_install_failure appear in CLI status and Stado Desktop → Products.

Once its files are placed and the receipt says installed, the holder records phase placed. A newer installation without --wait supersedes a holder placed the way it supersedes one still preparing — it is told to stop (SIGTERM), superseding the installation <holder>: it is placed, not placing files — and waits only for one placing. A holder from an older Stado never records placed and keeps placing through its after-install steps; when one of its children runs one of the recipe's after_install steps (the step's program, then its literal words in order), it is past placement too: superseding the installation <holder>: it is past placement, running its after-install step ``, followed by each process told to stop.

An installation running its after-install steps does not hold the lock, so a newer installation of the same surface takes it at once. Holding it, the newer one reads the receipt's after_install_running record; when the process table still names that pid with the recorded birth time, it and its children are told to stop (SIGTERM), each named on stderr after superseding the installation of <product>/<surface> by pid <pid>: it is running its after-install steps <steps>, so the older converge cannot finish after the newer one's. A recorded pid with another birth time belongs to another process and nothing is signalled. With --wait the newer installation stops nothing.