On this page

A service that accepts and never answers

A service that accepts and never answers is recovered where it runs

For an operational reading of a standing request, see diagnosing an unanswered service.

A service that accepts every connection and answers none passes a readiness read and fails everyone: its /health can answer while the path the service exists for holds every request, every client's resolver records the waits, and nothing on the host that runs the service acts, because nothing there asks the question the clients are asking. A product's catalog service declares that question as answering: the method and path of one request on the service's own API, the JSON body to send, and the repair step of its repair declaration (stado-rs/data/catalog/repair-catalog.json) to run before a restart. Brama declares POST /v1/chat/completions. Skarbiec declares GET /readyz with the crypto and audit-lock repairs: its readiness decrypts the vault's canary item, so a gpg standing behind a held keyring lock leaves the question unanswered, while a listing such as POST /v1/items/list decrypts nothing and answers as fast as ever. Stado declares GET /api/release/object?uri=stado://releases/answering-probe with the object-api repair: the public release route answers before host trust and Skarbiec and reads the store, so a healthy host answers at once that the object is absent, while an object API whose store holds every read leaves the question unanswered and its declared repair runs.

service:
  answering:
    method: GET
    path: /readyz
    repair:
    - crypto
    - audit-lock

On every beacon pass the host process sends the question to the service's recorded listen port on this host. The request runs to its end in a task of its own and its start is recorded, the way the resolver records every wait; the next pass reads the record. A question unanswered longer than this host's declared directory refresh interval (service_resolver.refresh_seconds, the same measure the resolver applies to a held adapter) is a service that stands: the host runs each declared repair step through the same path stado repair <service> --step <name> --target <host> --apply takes, then the restart through the same path stado service restart takes, forgets the question so the next pass asks the new process afresh, attempts an alert on the configured channels, and appends the recovery to ~/.stado/service-recoveries.jsonl when writable. No manual step: the host that runs the service acts first. A refused restart remains a refusal in the beacon. Alert delivery is attempted on the configured channels, with each channel's outcome retained; the answering beacon does not present the attempt as recovered.

The alert says recovery was attempted, names the repair and restart outcomes, and never claims the register already contains the event: the append happens after the alert and may refuse. The beacon reports that refusal with its file operation and cause.

Once the declared repair steps and restart succeed, the host retains that attempt for the continuous failed-answering episode. A later standing reading reports already_attempted with the last recovery instead of repeating the restart. The unanswered question is retired so the next beacon can ask afresh and observe an answer. A successful API answer ends the episode; a later failure can recover again. A refused repair step or restart does not count as complete recovery: the next beacon retries it; the beacon and alert name each refusal, and a writable register retains it. Replacing the managed unit or changing its declared listen port, method, path or body retires the old in-flight question and answer. The next pass asks the new endpoint rather than treating an old wait or HTTP result as its state. Changing the repair steps starts a new recovery episode if the question still stands, so an earlier attempted repair does not suppress the new declaration. Removing the question or being unable to read its directory endpoint retires its pending request and last answer; the beacon names an unreadable endpoint instead of carrying stale readiness.

stado service status <service> --json and stado service list --json carry the beacon's answering.recovery beside the failed API question; the plain status prints the same answering block. Stado Desktop → Services → Inspector shows the attempt, its repair-step refusals, restart cause, register refusal, and failed alert deliveries on the service row. An absent recovery for a standing question is shown as missing evidence, not as a successful restart. stado service recoveries --service <service> reads the recoveries the host could persist. If a restart ran but the register append failed, answering.recovery keeps the observed restart and recording_refusal names the file operation and cause. The host does not restart the same failed episode again merely because that record could not be persisted; the recovery-history command cannot prove the attempt, so inspect the beacon's refusal.

When the host that runs it does not act

A host that runs an older catalog never asks the question that would find its service standing, and a host whose own Stado stands on the service it would judge never reaches the judgement. The other hosts see it anyway: their resolvers hold the service's requests in phase: answer and refuse new ones. So every host's beacon pass also reads its own resolver's waiting list (~/.stado/resolver-waiting.json): a service on another host that has held this host's requests for the first byte of an answer longer than the declared refresh interval triggers a recovery attempt from this host: the same repair steps and restart run on the host that serves it over its registry SSH channel. The steps are fixed host payloads and the restart is the unit's own init system command, so neither needs that host's Stado, which may be the very process that stands. A wait still opening its channel says the host or its SSH daemon does not answer, not the service, and is not acted on.

One standing is acted on once after its declared repairs and restart succeed. The record keeps stood_since, when the oldest held request was accepted, and the complete answering declaration used for that attempt. A later pass reports already_recovered only when the same oldest request still stands, the managed unit and current declaration match that record, and every declared repair step was applied. A changed declaration or an older record without its snapshot cannot prove the current attempt; the peer acts again. A refused repair step or restart is retried on the next pass. A restart ends the held connections with the process that held them, so a service that stands again is a new standing.

The beacon carries peer_recoveries: one block per service and host with since, waited_seconds, bound_seconds, held (how many requests stand), oldest (the oldest wait's sentence: client, request line, endpoint) and one of recovery, already_recovered, or not_recovered with the reason (the catalog declares no answering question or no such service, the registry declares no unit for it on that host, the register cannot be read). A waiting list that cannot be read is carried as unreadable.

If a peer's register append fails after a restart, its beacon names recording_refusal and retains that outcome for the same oldest held request while the observing host process runs. That is not durable history; if the observer restarts before the register becomes writable, a later pass may attempt the standing again.

The beacon carries each probe under the unit's answering: asked (the first question was just sent), waiting with since, waited_seconds and the newest finished probe, stood with since, waited_seconds and bound_seconds plus the recovery record, or finished with answered (status, the HTTP library's successful verdict, elapsed_ms, at, and the actual response detail when the HTTP status is not successful) or refused (the transport's own cause). A fast HTTP 503 is an answer, not a standing request: its response detail diagnoses the failed readiness operation, but the standing-recovery path does not restart the service for that status. The CLI's service list/status prints the beacon's failed successful verdict and response detail. Desktop's Services fleet table and inspector mark the failed answer as needing attention and show that detail; an older beacon missing successful is not shown as healthy. A port the directory does not record, or a resolver that declares no interval, is said in the block as unreadable, never guessed. A service whose catalog entry declares no question carries no block.

stado service recoveries [--service NAME] [--json] reads this host's register, oldest first: when, which service on which host and its unit, the question that stood and the exact answering declaration used (older rows lack this snapshot), since when and for how long against the bound, the requests this host's resolver held unanswered to the service at the time, the ordered repair-step reports (empty when none is declared; older rows may hold one report or null), what the restart answered and whether it ended well, what each alert channel answered, and for an attempt on another host's service observed_by, the host whose resolver observed it. The plain line derives its words from what the register holds: an attempt from another host says recovered from <host>, whose resolver held its requests only when the restart ended well, and tried from <host>, whose resolver held its requests, and not recovered when it was refused; a resolver waiting list that could not be read is said as the resolver waiting list was unreadable (<cause>) instead of being counted as one held request.

A register that is not there holds no recovery; one that cannot be read, or holds a line that is not a recovery, is refused naming the file and the line. The catalog refuses an answering whose method is not in capitals, whose path does not start with /, whose repair is not a list of nonempty step names, or that carries a field outside method, path, body, repair. Omitted or empty repair steps are valid when only the restart is declared. Stado Desktop's Routes operations carry Read this host's service recovery attempts, including refusals, which runs the same history command on the selected host.

The internal probe exercise tests/service_answering/stands.rs (service-answering-stands) asks the question of a real listener on this machine that accepts and never writes a byte and reads it as standing once the fixture registry's declared refresh interval has passed, asks it of one that answers successfully and another that promptly returns HTTP unavailable with a diagnostic body, classifies the latter as unsuccessful without losing the body, forgets the standing one and sees the next pass ask afresh, and writes a recovery to a register under an isolated home, reads it back whole and sees a line that is not a recovery refused. It does not run a managed host's repair or restart. tests/service_answering/cli.mjs reads an actual fresh host beacon through stado host health, service status and service list, plus the unmanaged service refusal. It requires a declared live host/service and a clean source revision, but does not exercise a declaration change or standing recovery.

The resolver-list journey tests/service_answering/peer.rs (service-answering-peer) publishes a waiting list as a running resolver does under an isolated home. It shows that an undeclared standing service is not_recovered with its cause, while a wait still opening its channel, a wait on this host's service, and a wait younger than the bound are not standings. A list whose publisher no longer runs names nothing, and a malformed list is unreadable naming the file. It does not qualify a peer repair or restart: that requires a managed service on a real other host, its current declaration, and the resulting recovery and service state.

Source: this website