Stand-ins: how one is found, and what replaces it

Written 2026-09-09, from the pass that replaced every test area in stado-rs/tests built on one. The instruction behind it was three words long — replace them all with real tests — and it took a night, because a stand-in is not one thing. It has four shapes, each found a different way and each with its own replacement. This page is that process, so the next pass does not rediscover it.

The rules a real test obeys are on Test suite, which also carries the counts this pass moved. This page is about finding the tests that break those rules.

The four shapes

A machine that does not exist. const HOST: &str = "macbook-fake", a registry declaring mini-fake and laptop-fake, a beacon written by hand for fake-mini. Found by reading, not by intuition:

git grep -n -i -E '"[a-z0-9_./-]*(fake|mock|dummy)[a-z0-9_./-]*"' \
  -- 'stado-rs/tests/*'

Replaced by the machine running the test, declared by its own kernel host name — /bin/hostname, trimmed and lowercased, because the registry refuses a host name it would have to normalise. That one change makes the product's current-host resolution execute instead of an ssh path nobody dials, which is the difference between testing the command and testing the fixture. A name the registry does not declare survives in exactly one place: a case whose subject is the refusal of an unknown host, and the comment says so.

A system tool the test wrote itself. A shell script named ssh, launchctl, plutil, pgrep or pmset, placed on PATH so the product finds it instead of the real program. Found with git grep -ln '#!/bin/sh' -- 'stado-rs/tests/*' and then reading every hit, because the same construct is sometimes the subject rather than the stand-in: tests/host_run writes a shell script on purpose — it is the workload the product is asked to run. The replacement is the machine's own tool answering for real, and a launchd case owns a com.wisent.test.* label in this login's own domain and removes it before the run ends.

A case that never runs the product. No CARGO_BIN_EXE_stado anywhere in the area, so it calls library functions and proves nothing about a command anybody types. Found by set difference:

git grep -l CARGO_BIN_EXE_stado -- 'stado-rs/tests/*' | cut -d/ -f3 | sort -u
git ls-tree --name-only -d HEAD:stado-rs/tests | sort

Sixteen areas were on the wrong side of that difference on 2026-09-08. The replacement is not a wrapper: find the surface an operator reaches for by walking --help on the built binary, drive that, and read the state it left. Where no command exposes the behaviour, say so — naming the function and the missing surface is a finding, and inventing a command to make a test possible is not.

A case that cannot run at all. #[ignore = "…"] looks like coverage in the tree and executes for nobody. Twenty-six on 2026-09-08, most of them deferred with the words "Probierz records the real … journey" — which stopped being a reason when Probierz became optional rather than the required route. An ignore survives only when it names the exact resource and the exact command that runs it, and only when somebody has run it with that resource present and recorded the result.

Two structural rules

Both were learned the hard way, and both are about keeping a repair possible at all:

• 300 lines. A file past that limit cannot be edited in this workspace at all. declaration/main.rs was 566 lines and had a case asserting that a corrected registry document validates while it was failing; the repair was impossible until the file was split, so the area sat red. The changelog hit the same wall at 414 lines and stopped receiving entries. An oversized area is a frozen area.
• Split by what a piece defends, never by line count: fixture.rs for the seeded state and the product invocation, refusals.rs for the sentences, one file per capability. A folder of fragments is a worse file.

What the pass found in the product

A test that touches the real thing finds real defects, which is the argument for doing this at all. The release route returned a body shorter than the Content-Length it had just declared and called it an archive. StorageError::Auth announced "GCP authentication failed" on a machine with no GCP configuration, for a refusal about a local token file's mode. repair --target exited 0 for a host outside the registry. Every one was found by a case that reached the component instead of imitating it.

The state a test may touch

One shape does not show up in any of the greps above, and it is the worst of them: a test that reaches the operator's own machine. tests/claimability spawned the product without overriding HOME, and stado status records a last-known-good registry into $HOME/.stado/cache/registry-last-good.json, so every run of that area wrote the operator's real cache with a toy three-host document. It fires only when the queue listing shows a row, which is why it went unnoticed for weeks.

So isolation is not a convention, it is part of what makes a test a test: a tempdir, HOME inside it, WC_STORAGE_BACKEND=local, WC_LOCAL_STORAGE_PATH inside it, and STADO_CONFIG at a path that does not exist. env_clear() with an explicit PATH is the strongest form. And because a rule nothing measures is a rule that decays, tests/documentation refuses a revision in which an area spawns the product without passing HOME.