Building Rust with kompilator

Rust builds go through kompilator

Every Rust build Stado runs itself compiles with kompilator, never Cargo. Stado carries kompilator's sources and compiles kompilator once per source digest with the machine's own rustc. The carried sources are every file under kompilator/src, listed by stado-product's build script as Stado builds, so a file kompilator gains is carried by the next build of Stado without anyone adding it to a list. kompilator keeps the units it compiled in a store of its own and reuses them in the next build of the same sources, so no compiler cache is installed or put in front of a compiler; the Kache cache and its compiler-cache commands are gone. That covers:

• the owner-local stado product install of a kind: kompilator recipe, which runs kompilator build --revision <commit> --release --bin …, keeps the checkout's units in its build area and places each binary from the path kompilator printed; the receipt's kompilator field names the program, its digest, the unit store and the paths answered, and Stado Desktop's Products screen shows it;
• stado product kompilator stage --bin NAME [--features A,B] on a release builder, a kompilator build that takes only --bin and --features (stage builds with kompilator, which takes --bin NAME and --features A,B; remove … otherwise) and runs the kompilator KOMPILATOR names when it is set (KOMPILATOR names <path>, and no kompilator is there… when nothing is there);
• every step of a release builder job, which finds kompilator first on PATH with KOMPILATOR and KOMPILATOR_CACHE (its unit store for the product and platform under the build cache) set, and no CARGO_TARGET_DIR or compiler wrapper; its log says [release-worker] kompilator: <path> (<digest>); units kept at …. Every job the agent runs carries KOMPILATOR_JOBS, the cores the job asked for, and kompilator compiles with that many jobs when no --jobs is given (a value that is not a positive whole number is refused naming the variable);
• stado quality check and stado quality format, which run their gates and build with the kompilator KOMPILATOR names when the caller set it, else the one compiled from the sources this Stado carries; so a check still runs when the carried sources do not compile, including the check of the change that repairs them, and a refusal says cannot run kompilator for this check: the one KOMPILATOR names when it is set, else the one Stado compiles from the sources it carries into …;
• stado host upgrade and stado host kompilator … build, below.

kompilator is compiled into ~/.stado/products/kompilator/<digest>/kompilator through a staging directory beside it, staging-<pid>-<uuid>, named by the process compiling there and renamed into place once rustc succeeds, so no build runs a half-written program. A compile that is refused, or killed before it ends, keeps its staging so its sources and what rustc said can be read. Each use of kompilator then removes every staging whose process no longer runs, of every digest under ~/.stado/products/kompilator/ and not only the one in use (a digest whose sources the toolchain refuses is never placed, and one no installed Stado carries is never compiled again), and the older staging-<uuid> ones an earlier Stado left, saying on stderr removed the kompilator staging <path> its ended compile left, or why one cannot be removed; a staging whose process still runs is left to it. Each digest directory is tagged CACHEDIR.TAG as well, so the janitor may reclaim a whole digest under disk pressure, and the next use compiles it again.

No cargo check runs before a release build. kompilator tags its output and unit store with CACHEDIR.TAG, so the janitor reclaims them under disk pressure, and leaves one whose .lock a build holds or whose .in-use the run that created it still holds; it also still reclaims a Kache store an older Stado left.

A release manifest cannot ask for Cargo either. A quality gate, test step or build of .wisent-release.json whose program is cargo (the bare name or any path ending in it) is refused when the manifest is read — by stado quality check, stado release catalog sync and enroll, stado build submit and stado release submit — as stado.release.manifest.rule-broken, naming the step: platforms.<platform>.quality.<name>.argv (or build.argv) runs "cargo", and Stado runs no Cargo: build a Rust crate with stado product kompilator stage --bin NAME and run a test with kompilator test --manifest-path Cargo.toml --test NAME; a cargo fmt or cargo clippy step has no replacement, since no Wisent gate formats or lints, so remove it. Replace or remove the step as the sentence says and run the check again; it changes nothing.

What kompilator answers besides a build

Every Rust step of a product's own release, gates and scripts calls kompilator, never Cargo:

• kompilator test --manifest-path <dir>/Cargo.toml [--lib] [--test NAME]… runs, as cargo test does, the unit tests inside the library (<out>/unittests/lib-<crate>) and each binary (<out>/unittests/bin-<name>) beside every integration test; --lib and --test NAME select. A binary under src/bin/, and src/main.rs beside a declared [[bin]], are built and tested as Cargo finds them.
• kompilator metadata --manifest-path <dir>/Cargo.toml prints the workspace in the shape of cargo metadata --no-deps --format-version 1: its root, its members and each target's kind and absolute source. Tama's tama hooks stage --kompilator <path> reads its binaries there.
• A Git dependency is read from vendored sources when a build is handed them: the directory KOMPILATOR_GIT_SOURCES names, else the sources/ of the private-cargo-sources input whose mount WISENT_INPUT_PRIVATE_CARGO_SOURCES_DIR names; nothing is fetched while either is set, so a worker with no Git credential builds Spis, Ster, Skrzynka, Skarbiec or Jeden. A vendored directory that lacks a locked package is refused naming the package, the directory and the variable, and an entry vendored for another commit than the lock pins is refused naming both.
• kompilator vendor --manifest-path <dir>/Cargo.toml --out DIR writes every Git package the lock names as DIR/<name>-<version>, the package's repository at its locked commit, files and directories only, so a package that inherits from its repository's workspace is read against it; its .kompilator-source names the lock's source line it was vendored from. stado release catalog input pin --cargo publishes the private-cargo-sources input with it (vendor-refused names what kompilator said when a locked commit cannot be fetched).
• kompilator relock --manifest-path <dir>/Cargo.toml writes into the lock the version each workspace member now declares, changing only those entries and the dependency lines naming them, with nothing fetched; stado release loop's version step runs it after it bumps a Cargo.toml version. A member the lock has no entry for, and a version the lock already holds from another source, are refused with the lock unchanged.
• kompilator lock --manifest-path <dir>/Cargo.toml writes into the lock every package a member newly asks for, with the packages those need, so a product that adopts a dependency is locked without Cargo: a crates.io one at the highest version the lock holds, else crates.io publishes, that meets its requirement (with the index's checksum); a Git one at the commit its rev, branch or tag names now (git ls-remote). Optional dependencies are locked as the features asked of them turn them on, and a package the lock already holds keeps its entry. A workspace with no lock gets one. A rev that is not a full commit, a path outside the workspace and a requirement nothing published meets are refused with the lock unchanged.
• kompilator run --manifest-path <dir>/Cargo.toml [--bin NAME | --example NAME] [-- ARGS] builds one binary or example as build does and runs it with ARGS in the caller's directory, the program replacing kompilator, so its output, signals and exit status are its own: what a fleet job ran as cargo run --release -- … and a reader as cargo run --example NAME. Several binaries and no --bin, two --bin, and a --bin with an --example, are refused; a program its sources do not compile (exit 65) never runs.
• --result FILE (to build, test and run) writes what the command came to as one JSON document: state, the command and the error that ended it, the manifest_path and sources built (pinned to the commit --revision named) and the findings, each compiler error rustc pointed into the sources with its header, code and <file>:<line>:<column>. It holds the fields Stado's retained Cargo results held, so Oko charges its findings to the commits that wrote those lines (oko sessions scorecard charges observe FILE). A document that cannot be written fails the command, naming the file.
• --example NAME (to build, run and where) links an example: a [[example]] table, or examples/NAME.rs and examples/NAME/main.rs unless autoexamples = false, as a program against the package's library and its dev-dependencies, written as <out>/examples/NAME. A build of examples alone links no binary of the package. An example the package does not declare is refused naming those it does; one declared with a crate-type other than bin refuses only a build that asks for it; test does not take --example.
• kompilator update --manifest-path <dir>/Cargo.toml --package NAME@FROM --precise TO moves one crates.io package of the lock to TO, read from the crates.io index (checksum, yanked, dependencies), moving a locked dependency the new version needs higher to the highest published version every requirement on it admits, and adding nothing the lock lacks; stado product advisories apply runs it. A dependency it cannot place, a dependent whose requirement excludes the move and an unpublished or yanked version are refused with the lock unchanged.
• kompilator build of a package that is a library alone compiles it and answers no path, as a gate that compiles every member of a workspace needs.
• Cargo's profile overrides in the environment (CARGO_PROFILE_DEV_DEBUG=line-tables-only, CARGO_PROFILE_<NAME>_<KEY>, CARGO_PROFILE_<NAME>_BUILD_OVERRIDE_<KEY>) apply over the manifest's [profile]; a value its setting does not take is refused naming the variable. --out and the unit store are made absolute before a build, so a --revision build's scripts read an absolute CARGO_MANIFEST_DIR.
• CI takes kompilator from quality-control's action (wisent-ai/quality-control/.github/actions/kompilator): the runner's own, else compiled from Stado's main with rustc through a token that reads it; the shared rust-gates workflow builds every package with it.
• Every git kompilator starts leaves out the repository variables of its caller (git rev-parse --local-env-vars: GIT_DIR, GIT_INDEX_FILE, …), so a --revision build started from a Git hook reads the manifest's own repository.

stado product kompilator has one operation, stage, above; any other word is refused by the command line (exit 2), naming stage. A product's build or test is kompilator build or kompilator test itself, and its workspace's packages and targets are kompilator metadata. It was named stado product cargo while it ran no Cargo; that name is gone, and so is its results, which read only the receipts an earlier Stado's Cargo runs left.

Before the build admission, a stage is refused as usage (exit 2) when --manifest-path names no file (stado.product.kompilator.options.manifest-absent), when a forwarded argument is neither --bin nor --features (stage-argument), and when no --bin names a binary (stage-binary-missing). Once admitted it is refused when WISENT_OUTPUT_DIR is unset (output-unset) or relative (output-relative), and when a built binary cannot be copied there (stado.product.kompilator.stage.staged-unplaced, naming the binary, the destination and the system's answer). kompilator's own refusals are stado.product.kompilator's.

Building on another host

stado host kompilator TARGET build --manifest-path PATH --revision REV --bin NAME… [--deadline SECONDS] [--json] builds a committed revision on a fleet host with kompilator. Past --deadline the processes the run started on the host are sent SIGTERM and the build fails (ran past the declared --deadline of <N>s and was stopped), naming the manifest, the binaries and what kompilator printed last; stado host upgrade builds through the same run with no deadline. stado host kompilator TARGET gate --manifest-path RELEASE_MANIFEST --revision REV --gate NAME --platform PLATFORM [--json] runs one quality gate the revision's release manifest declares, exactly as the submit queue runs it. PATH is a crate's Cargo.toml inside its canonical checkout on this machine and REV a commit that checkout holds; the manifest is read from the commit, never from the working tree.

The commit is exported (no uncommitted edit is read) and delivered, with the kompilator sources Stado carries, into a fresh run directory in TARGET's run area: <work root>/runs when the registry declares a work root, else $HOME/.stado/work/runs. TARGET compiles kompilator there once per source digest with its own rustc — rustc could not compile kompilator from the sources Stado delivered, so nothing was built; … Select a Rust toolchain that compiles it (rust-toolchain.toml, or rustup default) when it cannot — and puts it first on PATH. build runs kompilator build --release of each --bin into the run's out; kompilator keeps its units in host-build/kompilator-units of the host's build cache, shared by every run there. Each binary kompilator answered is pulled back to <manifest directory>/target/host-kompilator/<commit>/<NAME>, and the run directory is removed whatever the build answered; the exit status is kompilator's. gate runs the declared argv in the delivered tree's root with the release worker's contract (WISENT_SOURCE_DIR, WISENT_OUTPUT_DIR, an empty WISENT_INPUTS_DIR, WISENT_VERSION, WISENT_PLATFORM) and the host's installed Stado as STADO_BIN, first on PATH; the submit queue runs every declared gate this way.

Refused before anything is exported, each naming its fix:

• build without --bin: a host build names the binaries it brings back; add --bin NAME for each;
• any argument after --: a host build runs kompilator, which takes no forwarded arguments; remove … (stado.host.runs.kompilator-request);
• a path that is not a Cargo.toml, or one outside any checkout on this machine (exit 2);
• a commit the checkout does not hold (fetch or commit it first), and a commit that holds no such manifest;
• a host whose build-cache volume cannot hold a compile: the volume measured is the one holding the build cache (below the declared work root when there is one), and its free bytes must cover the janitor's margin — the largest build measured there — plus what the running jobs hold; where that need is not known, the janitor's disk-full threshold decides. The refusal names the free bytes, the mount and the work root, with stado host gates TARGET or stado space report TARGET to read what holds the room (exit 69).

A binary kompilator printed no path for is refused as kompilator answered no path for <binary> in <out>, so nothing of it was brought back. Every cause is listed by stado errors show stado.host.run.build.

The build receipt (--json, schema stado.host-kompilator-receipt.v1) carries the target, checkout, commit and archive_sha256 (the WISENT_SOURCE_SHA256 the build was given), the manifest, the binaries asked for, kompilator's stdout, stderr and exit_code, each brought-back binary with its path, bytes and sha256, free_bytes_before and free_bytes_after (or free_bytes_after_error) of the build-cache volume, and the removed run_directory. A gate's receipt carries the commit, archive_sha256, the argv that ran, its streams and exit_code, the run_directory, and stado_revision, the Stado that exported, delivered and compiled the run.