Reading admission and diagnostic failures
Reading admission and diagnostic failures
stado host gates HOST --json reads the registry, current disk usage, janitor
state and snapshots, effective agent storage configuration, storage client,
published capacity and queue. Each read retains its result or concrete error
and reports its operation, source, state, start and finish time, elapsed
milliseconds and error in observations. Elapsed time describes the read;
the host-gate observer does not use it to cancel the operation. Independent
reads run together. A failed host read does not remove a completed capacity
or queue reading.
complete: false means the diagnostic could not finish; claiming is then
null, and the command exits non-zero. Observation states distinguish
complete, partial, absent, cached, error and skipped. A cached
registry is identified as historical, not presented as a successful current read.
An unread source is not the same thing as an absent object.
Every capacity read the job store answers is kept on this machine in
~/.stado/capacity-publications.json (STADO_CAPACITY_COPY_FILE moves it),
the same copy stado workload start places from. When the store refuses the
read or its client does not open (an object API whose vault authorization does
not answer, as while the vault is being moved), the capacity read is
cached: its detail is the store's failure followed by read from the copy of them this machine kept when it last read them from <store> at <time> (<n>s ago), each row judged by the promise its agent made, and the host's kept row
is reported as its capacity. A kept row past its author's next_by still
reports capacity_publication_stale. With no copy the read is error, ending
this machine keeps no copy of an earlier read either. A cached read is not a
current one, so complete stays false and claiming null; the copy only
keeps the host's last word visible.
agent_declared_not_loaded is a blocker when the host's newest beacon does not
report its declared agent unit active. A beacon that carries no units map
says nothing about the agent, so it is that blocker too, naming the beacon's
object path, and a unit the beacon lists without a state is reported with the
entry the beacon carried rather than as an empty state. A host with no beacon
at all is the separate no_capacity_publication silence.
The memory: line reports what memory this host has left — available X of Y GiB, swap N% used, each GiB figure shown to one decimal while the JSON
memory_available_gb and memory_total_gb carry them unrounded — and memory evidence: names where that reading came from:
capacity_publication while the host's agent is publishing, and
host_memory_measurement when it is not and the command read vm_stat and
sysctl (or /proc/meminfo) off the host itself. Memory is a reading only and
never a blocker. The second source is what makes the line useful on the host an
operator looks at when it has stopped answering: the one whose agent has no
memory left to publish with would otherwise print memory: not observed.
A figure the host's capacity publication does not carry — free or total
cores, RAM or VRAM, the running job count, its age, a reservation's fields —
prints as not reported on the capacity: and reserved: lines, and a reading
this command could not take (disk use, memory, its evidence, the local snapshot
count) prints as not observed; neither is ever printed as zero, - or ?.
Figures print as the host published them, and waiting jobs are named by their
whole job id.
The queue read walks the priority index — queue_priority/, which names
queued jobs and nothing else — and returns the pinned jobs it finds, oldest
first, up to 200. It is partial when that window filled, with the detail
200 pinned jobs filled the window; more may be queued for this host; a
partial read still decides the verdict. Reading every object under queue/
instead would include one transition sentinel per job that ever left the
queue, and on a busy queue every host's read would time out and every verdict
would be claiming: unknown. The coordinator's reaper retires a sentinel once its
job has settled in a terminal prefix and the object is a day old, at most 100
per prefix per tick, and logs reaper: queue/ settled sentinels retired=N kept=M inspected=K when it removed any.
When placed workloads hold the host, the text carries a reserved: line —
N placed workload(s) hold C core(s), R GiB RAM, V GiB VRAM; the capacity line is net of them — and one line per hold naming the kind, product, holder
and acquisition time; capacity in the JSON carries running_workloads,
reserved and reservations. The accelerator: line says who holds the
card, from the same publication (space).
disk.free_bytes is measured by the host's filesystem reader; free_gb is its
GiB display value. used_percent, headroom_gb (what may still be written
before the disk-full rule deletes; negative past it), full_percent (80) and
full judge that measurement against the rule.
These fields are null when the measurement is unavailable, never zero.
pressure_source distinguishes the agent's current capacity_publication
from host_disk_measurement; capacity.diagnostics retains the agent's actual
admission reason and reported state. Low free space and the agent's decision
are different readings. Neither a slow queue read nor an unavailable disk
reader proves that the disk is full.
Stado Desktop shows the same results in Hosts → selected host → Gates:
completed measurements, unknown admission when reads are incomplete, each
source and error, and the agent's published diagnostics. Reading this screen or
running host gates does not clean a disk, restart a service or change policy.
Source: this website