Service

A managed service joins a registry declaration to a launchd or systemd unit. The declaration states what should run; host observations show what actually runs. Stado compares both before reporting or changing service state.

What it is

A managed service is one entry in a target's services[] array: the launchd/systemd unit, program, arguments and host-side unit path Stado is required to keep running (architecture). A unit absent from that managed set is not a managed-service lifecycle target. Inspect it with service list --undeclared or service unit show; service adopt can record a proven existing unit before normal managed lifecycle operations.

A declaration carries the name the CLI addresses it by, the unit identity (launchd label or systemd unit), the unit-file path, and — when the declaration is the source of the unit rather than a pointer at a plist somebody installed by hand — program and args. That pair is what makes the declaration reinstallable from the document alone: service ensure renders the unit from them, so a host that lost its unit file can be made to run the right thing again (deploy/service/model/managed/service.rs, ManagedService). A declaration that names only a path cannot be reinstalled from the document; repair records declaration_incomplete and alerts until the entry carries its program and args — read the truth with stado service show <name> and write it into the entry (operations).

The units Stado renders for the host agent, for a deployed service and for the object API that a storage transaction restores carry one declared program search path in their PATH, never the installing shell's: on macOS /opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin, on Linux /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin (primitives/tool_path), so a tool such as gpg is found by every one of them on a host or by none of them. A precheck runner's launcher sets its own PATH in its installer.

Sources: registry, recovery, product

The managed set has three sources, and the distinction is load-bearing (deploy/service/model/vocabulary.rs):

Source	Meaning
registry	Declared in the target's services[]. What adopt/retire/deploy edit, and what stado registry doctor diffs against live host state.
recovery	The fixed list every stado repair stado --step host pass restarts. Genuinely managed, but by that fixed program and not by the document — so it can be neither adopted nor retired, and is never silently converted into a registry service.
product	Located by a shipped product declaration naming both the label and the unit file, addressable without a registry record.

The product name and its exact native unit identity select the same catalog entry, including its required environment. Addressing the host Stado process by its unit com.wisent.stado therefore keeps the same defaults as addressing it by stado: that one process serves the object API, the release API and every Stado role, so there is no separate object API unit to address. Its physical recovery writer preserves other installed unit fields, such as resource limits, instead of erasing what the shared service renderer declared.

One shipped entry is worth naming, because it is a repair nothing else can perform. brama-grant-sync runs brama subscription sync on a cadence, so every provider grant the harnesses on that machine hold joins the gateway's pool. It exists for the state a gateway cannot leave on its own: every credential it holds is dead, and its automatic sign-in cannot replace them because the provider asks for an authenticator code the vault does not hold. A machine that is still signed in then repairs the gateway. The entry carries no port and no secret — the gateway is resolved through the service directory and the console bearer is read from the vault at every pass.

stado service ensure brama-grant-sync --host <workstation> \
  --as-launch-agent --reason "<why this machine repairs the pool>"

Who observes it

Two deliberately separate halves (deploy/service/mod.rs). The read side — stado service list — joins declarations against the latest host beacons and issues no ssh, so it stays answerable while a host is wedged; a stale beacon yields unknown, never a confident active or missing. The write side (restart, deploy, ensure, retire, logs, env) rides the approved host channel with a fixed, narrow remote program. The autonomy cycle joins beacon unit state with a fresh stado service verify reachability sweep and repairs from that evidence (operations).

Stado Desktop → Services → Managed services keeps the beacon's failed row when the follow-up stado service status <name> --json cannot read its exit evidence. service status reads the last launchd exit code and last stderr line by default; --lines requests a larger log window. The alarm and inspector show both, or the exact host-read refusal when either is unavailable. The beacon's state, time and detail remain visible separately; a failed status read does not turn the fleet-wide list into an empty fleet. A failed release activation reports the unit's last exit and stderr before restoring the retained executable.

stado service serving stado --host <host> --wait --json waits for the declared unit to own its directory port, or refuses when its process exits or settles on another port. It accepts no --port with --wait: there is no declared readiness to prove for an arbitrary caller-supplied endpoint. The refusal includes the last native exit and stderr evidence. Product installation uses this same read before accepting a resident replacement; on failure it restores the recorded exact backup and checks the predecessor's serving state.

Failed service: evidence and repair

Managed stdout and stderr stay on the fleet host. On macOS the launchd unit's StandardOutPath and StandardErrorPath name their files; when stdout has no declared path, Stado reads $HOME/.stado/logs/<unit>.log. On Linux the system or user journal holds the unit streams. Stado does not forward them to a central support inbox or repository maintainer. Read a bounded window over the approved host channel:

stado service logs --lines <LINES> <NAME> --host <HOST>
stado service status <NAME> --lines <LINES> --json
stado optimize status

For a loaded unit that is not declared as a managed service, use stado service unit logs <LABEL> --host <HOST> --lines <LINES> instead. Neither command sends logs to a maintainer by itself.

stado service status --lines includes that window of a failed unit's stderr; without it the status names the service logs command to read the stream. stado optimize status reads the latest reconciliation outcome, not an assertion that application code was repaired.

The default autonomy mode is report: it records a plan without mutation. In an authorized enforcing mode, the reconciler can restore an eligible declared unit from its program and arguments when host and endpoint evidence justify it. It cannot infer or patch a bug in a repository, and it does not submit private logs to a public issue automatically. A missing declaration, unreachable host or inconclusive evidence is a recorded refusal, not a successful repair (autonomy, operations).

Separately, a catalog-declared answering probe can trigger host-local repair steps and a restart when its API request stands unanswered. Inspect stado service status <NAME> --json and stado service recoveries --service <NAME> --json for the actual attempt, refusal and subsequent answer; a running unit or an attempted restart alone does not prove the service recovered.

Where it lives in the store

In the registry document, as targets[].services[]. Every mutation goes through the same validated compare-and-swap write path as stado registry push, so a mutation that would produce an invalid document is refused with nothing uploaded (cli).

Ensure

Creating, repairing or restarting a declared unit, one product to one process per host under its one unit: Ensure.

Adopt

Claiming a unit a host already runs: Adopt.

Retire

Stopping a unit and withdrawing its declaration: Retire.

Converge

Reconciling a service's declared placement with what its hosts run, from the CLI, the API or Desktop: Converge.

A service that accepts and never answers is recovered where it runs

For an operational reading of a standing request, see diagnosing an unanswered service.

A service that accepts every connection and answers none passes a readiness read and fails everyone: its /health can answer while the path the service exists for holds every request, every client's resolver records the waits, and nothing on the host that runs the service acts, because nothing there asks the question the clients are asking. A product's catalog service declares that question as answering: the method and path of one request on the service's own API, the JSON body to send, and the repair step of its repair declaration (stado-rs/data/catalog/repair-catalog.json) to run before a restart. Brama declares POST /v1/chat/completions. Skarbiec declares GET /readyz with the crypto and audit-lock repairs: its readiness decrypts the vault's canary item, so a gpg standing behind a held keyring lock leaves the question unanswered, while a listing such as POST /v1/items/list decrypts nothing and answers as fast as ever. Stado declares GET /api/release/object?uri=stado://releases/answering-probe with the object-api repair: the public release route answers before host trust and Skarbiec and reads the store, so a healthy host answers at once that the object is absent, while an object API whose store holds every read leaves the question unanswered and its declared repair runs.

service:
  answering:
    method: GET
    path: /readyz
    repair:
    - crypto
    - audit-lock

On every beacon pass the host process sends the question to the service's recorded listen port on this host. The request runs to its end in a task of its own and its start is recorded, the way the resolver records every wait; the next pass reads the record. A question unanswered longer than this host's declared directory refresh interval (service_resolver.refresh_seconds, the same measure the resolver applies to a held adapter) is a service that stands: the host runs each declared repair step through the same path stado repair <service> --step <name> --target <host> --apply takes, then the restart through the same path stado service restart takes, forgets the question so the next pass asks the new process afresh, attempts an alert on the configured channels, and appends the recovery to ~/.stado/service-recoveries.jsonl when writable. No manual step: the host that runs the service acts first. A refused restart remains a refusal in the beacon. Alert delivery is attempted on the configured channels, with each channel's outcome retained; the answering beacon does not present the attempt as recovered.

The alert says recovery was attempted, names the repair and restart outcomes, and never claims the register already contains the event: the append happens after the alert and may refuse. The beacon reports that refusal with its file operation and cause.

Once the declared repair steps and restart succeed, the host retains that attempt for the continuous failed-answering episode. A later standing reading reports already_attempted with the last recovery instead of repeating the restart. The unanswered question is retired so the next beacon can ask afresh and observe an answer. A successful API answer ends the episode; a later failure can recover again. A refused repair step or restart does not count as complete recovery: the next beacon retries it; the beacon and alert name each refusal, and a writable register retains it. Replacing the managed unit or changing its declared listen port, method, path or body retires the old in-flight question and answer. The next pass asks the new endpoint rather than treating an old wait or HTTP result as its state. Changing the repair steps starts a new recovery episode if the question still stands, so an earlier attempted repair does not suppress the new declaration. Removing the question or being unable to read its directory endpoint retires its pending request and last answer; the beacon names an unreadable endpoint instead of carrying stale readiness.

stado service status <service> --json and stado service list --json carry the beacon's answering.recovery beside the failed API question; the plain status prints the same answering block. Stado Desktop → Services → Inspector shows the attempt, its repair-step refusals, restart cause, register refusal, and failed alert deliveries on the service row. An absent recovery for a standing question is shown as missing evidence, not as a successful restart. stado service recoveries --service <service> reads the recoveries the host could persist. If a restart ran but the register append failed, answering.recovery keeps the observed restart and recording_refusal names the file operation and cause. The host does not restart the same failed episode again merely because that record could not be persisted; the recovery-history command cannot prove the attempt, so inspect the beacon's refusal.

When the host that runs it does not act

A host that runs an older catalog never asks the question that would find its service standing, and a host whose own Stado stands on the service it would judge never reaches the judgement. The other hosts see it anyway: their resolvers hold the service's requests in phase: answer and refuse new ones. So every host's beacon pass also reads its own resolver's waiting list (~/.stado/resolver-waiting.json): a service on another host that has held this host's requests for the first byte of an answer longer than the declared refresh interval triggers a recovery attempt from this host: the same repair steps and restart run on the host that serves it over its registry SSH channel. The steps are fixed host payloads and the restart is the unit's own init system command, so neither needs that host's Stado, which may be the very process that stands. A wait still opening its channel says the host or its SSH daemon does not answer, not the service, and is not acted on.

One standing is acted on once after its declared repairs and restart succeed. The record keeps stood_since, when the oldest held request was accepted, and the complete answering declaration used for that attempt. A later pass reports already_recovered only when the same oldest request still stands, the managed unit and current declaration match that record, and every declared repair step was applied. A changed declaration or an older record without its snapshot cannot prove the current attempt; the peer acts again. A refused repair step or restart is retried on the next pass. A restart ends the held connections with the process that held them, so a service that stands again is a new standing.

The beacon carries peer_recoveries: one block per service and host with since, waited_seconds, bound_seconds, held (how many requests stand), oldest (the oldest wait's sentence: client, request line, endpoint) and one of recovery, already_recovered, or not_recovered with the reason (the catalog declares no answering question or no such service, the registry declares no unit for it on that host, the register cannot be read). A waiting list that cannot be read is carried as unreadable.

If a peer's register append fails after a restart, its beacon names recording_refusal and retains that outcome for the same oldest held request while the observing host process runs. That is not durable history; if the observer restarts before the register becomes writable, a later pass may attempt the standing again.

The Skarbiec crypto step reads readiness before and after skarbiec recover-daemons within the same declared refresh interval, which the caller hands it as SKARBIEC_READY_WITHIN_SECONDS; a caller that cannot read one is refused naming it. A readiness read without a bound stood as long as the stranded gpg children held the vault, and the restart that ends them was never reached.

The beacon carries each probe under the unit's answering: asked (the first question was just sent), waiting with since, waited_seconds and the newest finished probe, stood with since, waited_seconds and bound_seconds plus the recovery record, or finished with answered (status, the HTTP library's successful verdict, elapsed_ms, at, and the actual response detail when the HTTP status is not successful) or refused (the transport's own cause). A fast HTTP 503 is an answer, not a standing request: its response detail diagnoses the failed readiness operation, but the standing-recovery path does not restart the service for that status. The CLI's service list/status prints the beacon's failed successful verdict and response detail. Desktop's Services fleet table and inspector mark the failed answer as needing attention and show that detail; an older beacon missing successful is not shown as healthy. A port the directory does not record, or a resolver that declares no interval, is said in the block as unreadable, never guessed. A service whose catalog entry declares no question carries no block.

stado service recoveries [--service NAME] [--json] reads this host's register, oldest first: when, which service on which host and its unit, the question that stood and the exact answering declaration used (older rows lack this snapshot), since when and for how long against the bound, the requests this host's resolver held unanswered to the service at the time, the ordered repair-step reports (empty when none is declared; older rows may hold one report or null), what the restart answered and whether it ended well, what each alert channel answered, and for an attempt on another host's service observed_by, the host whose resolver observed it. The plain CLI still says recovered from <host> even for a refused repair or restart, and counts an unreadable resolver-list diagnostic as one held request; inspect restarted, the repair statuses and held_requests in --json before treating that line as a recovery or a request count.

A register that is not there holds no recovery; one that cannot be read, or holds a line that is not a recovery, is refused naming the file and the line. The catalog refuses an answering whose method is not in capitals, whose path does not start with /, whose repair is not a list of nonempty step names, or that carries a field outside method, path, body, repair. Omitted or empty repair steps are valid when only the restart is declared. Stado Desktop's Routes operations carry Read this host's service recovery attempts, including refusals, which runs the same history command on the selected host.

The internal probe exercise tests/service_answering/stands.rs (service-answering-stands) asks the question of a real listener on this machine that accepts and never writes a byte and reads it as standing once the fixture registry's declared refresh interval has passed, asks it of one that answers successfully and another that promptly returns HTTP unavailable with a diagnostic body, classifies the latter as unsuccessful without losing the body, forgets the standing one and sees the next pass ask afresh, and writes a recovery to a register under an isolated home, reads it back whole and sees a line that is not a recovery refused. It does not run a managed host's repair or restart. tests/service_answering/cli.mjs reads an actual fresh host beacon through stado host health, service status and service list, plus the unmanaged service refusal. It requires a declared live host/service and a clean source revision, but does not exercise a declaration change or standing recovery.

The resolver-list journey tests/service_answering/peer.rs (service-answering-peer) publishes a waiting list as a running resolver does under an isolated home. It shows that an undeclared standing service is not_recovered with its cause, while a wait still opening its channel, a wait on this host's service, and a wait younger than the bound are not standings. A list whose publisher no longer runs names nothing, and a malformed list is unreadable naming the file. It does not qualify a peer repair or restart: that requires a managed service on a real other host, its current declaration, and the resulting recovery and service state.

Commands that act on it

stado service list
stado service show <name>
stado service ensure <name> --host <host> --reason <why>
stado service adopt <unit> --host <host>
stado service retire <unit> --host <host>
stado service restart <name>
stado service logs <name>
stado release version show --host <host> --json
stado release version converge --host <host> --json

Full flags and the deploy/declare contract in the cli reference.

service logs and status resolve a declared service; logs can also use a product's catalog unit. retire and remove require a registry declaration. For an exact unit name with no matching declaration, stado service logs <unit> --host <host> --json refuses with not a registry-managed service on <host> and failure code not_found. Repeating service remove after its first successful removal has the same class. That is an absent declaration, not an unattributed transport failure. stado service unit show <unit> --host <host> --json can still inspect the host's loaded unit without a declaration. When the unit is loaded with no process and its last run exited nonzero, unit show prints a failing row (failing in --json): <unit> runs no process: its last run exited <code> and its supervisor has started it <runs> time(s) — a release that cannot serve under a keep-alive supervisor, restarted without end. A host release delivery whose restart cannot prove a fresh process on the activated image appends the same sentence to its refusal, <unit>: restart did not prove a fresh pid on the activated immutable image; … ; <unit> runs no process: …, so the delivery names the crash loop it caused.

The real journey tests/service/failing_unit.mjs (STADO_BIN, STADO_SERVICE_TEST_HOST, STADO_SERVICE_TEST_EXIT_CODE) loads two launchd jobs in this account's GUI domain from plists under .build/: one whose program exits with the stated code, one still running. It reads the first through unit show until launchd reports the run ended and checks the failing verdict in --json and the text row, checks that the running job carries none, boots both out and checks that unit show then refuses the label. It writes report.json with the source revision, the binary's digest and every command with its exit status.

Not to be confused with

• The unit itself. The unit is host state; the service is the fleet's declaration of it. service restart <name> and service restart <unit-id> address the same declaration when those identities belong to the same managed service.
• A service directory entry. stado://service/<name> routing — who may call a service and where it currently answers — is the directory, a different primitive in the same document.
• A release. Installing signed product bytes is the release flow; the service declaration is what must keep running regardless of which bytes are current.