Testing

619 tests. cargo test runs all of them in about twenty seconds — most of that is tests/tftp.rs, which waits on real UDP timeouts because that is what it is testing.

cargo test does not run the harnesses that matter most: the boot rig, the DSM package’s three, and the loader build. See The package is tested too and the boot rig.

cargo test                                # everything
cargo test <name>                         # one, by substring
cargo test -- --nocapture                 # show stdout
cargo test --all-features                 # what CI runs

Where a test belongs

tests/common/mod.rs — the fixtures every suite shares

Answers are stored as a directory per identity, so a fixture cannot just be a filename any more. seed() takes the name a test thinks in — 98fa9b50d810.toml, groups/rack-a.toml, default.toml — and writes it through StoreWrite, so it lands exactly where a write from the admin API would and cannot drift from the layout. A name the store would refuse (an extension nobody serves) is written literally instead, because those fixtures exist precisely to prove that a stray file answers nothing.

One copy, not one per suite — the same reasoning that makes loaders.rs a single table read by both TFTP and the DHCP snippet. Four copies of a mapping are four chances for a fixture to land somewhere the server does not look, and a test that seeds nothing passes for the wrong reason.

tests/stores.rs — the conformance suite

Every behavioural case runs twice, once per store, and asserts the identical outcome. That suite is what keeps two backends from drifting.

A new behaviour belongs there, not in a store-specific test. A test that covers one backend proves half of what it claims — and the half it does not cover is exactly where a divergence hides.

tests/cli.rs — the commands people are told to run

check is what deploy.sh runs before it ships anything, and what the documentation tells people to put in CI — so its exit code is a contract, not a convenience. render’s stdout/stderr split is another: the document goes to stdout so render … > answer.toml yields a usable file, and the provenance line goes to stderr so it does not end up inside it.

Also pinned here: the import → export round trip is byte-identical, comments and formatting included. That is what makes the database safe to adopt and safe to leave; if it ever stops being exact, export is no longer a way back out.

tests/integration.rs — against the real binary

It starts the actual binary on an ephemeral port and talks HTTP to it. The binary prints the address it bound, so there is no port race and no sleep-and-hope.

This suite exists because some failures are invisible to unit tests. The clearest example: hyper panics at runtime if header_read_timeout is set without .timer(…). It compiles. Only a real connection finds it.

Explicitly covered:

• a truncated request, and one with no Content-Length;
• an aberrant Content-Length — and a chunked body that outgrows the cap while streaming, which is the other way in and trips the limit mid-read;
• an unknown method, an empty body, a 1 MB body, a body that is not valid UTF-8;
• the connection cap: over it, a prompt 503 rather than a queue — and the permit coming back afterwards;
• and, after each of those, that the server still answers. That last assertion is the one that matters — the abuse is only interesting if the server survives it.

cargo test does not rebuild target/debug/rescriptum. A manual check against a stale binary once “reproduced” a bug that had already been fixed. Rebuild before poking at the binary by hand.

tests/tftp.rs — a transfer is a conversation

Nothing here can be proved from inside a function. Blocks, acknowledgements, retransmission, the empty packet that ends a transfer — every bug worth catching lives in the turn-taking, and the first run found two of the “works by hand, never after a reboot” kind. A file whose length is an exact multiple of the block size must end with an empty data packet; leave it out and the client waits forever for a final block that never comes.

It also owns the one listener failure in this server that is not fatal. A TFTP port that cannot be bound must not take answers and media down with it — measured on DSM, where the capability is granted outside the package and an upgrade drops it — so the test holds the port with a squatter, then asserts three things at once: the server came up, it warned and said what still works, and boot check still exits non-zero.

That last one first passed for the wrong reason: three missing loaders were already failing the command. The fixture now writes every loader the table names, and a control run with TFTP off proves the directory is otherwise clean.

tests/media.rs — boot media against the real binary

Both listeners up, and every abuse case ends by proving the server still answers. One case proves the property the separate socket exists for: answers keep succeeding while four image transfers are in flight.

There is deliberately no binary ISO fixture in this repository. boot::iso::build writes images in memory, behind the test-support feature so it never reaches a release binary.

The boot chain belongs in the rig

packaging/boot-rig/run.sh is not Rust and cargo test does not run it. It boots a claimed and an unclaimed machine in QEMU under TCG, on a private bridge with no uplink, and asserts four markers: the DHCP handoff answered from our own generated snippet, a loader fetched over TFTP, the unclaimed machine fell through to its local disk, and the claimed machine reached its own answer. CI runs the same thing plus a deliberate break.

A QEMU guest bridged into a container has a MAC of its own, and Docker Desktop’s virtual switch does not forward frames from a MAC it did not assign — measured, which is why the primary rig is one container rather than four on a Docker network.

Check that a test can fail

A test that passes for the wrong reason is worse than no test: it reports coverage that does not exist. Before trusting a new one, break the thing it guards and watch it go red.

One in this suite did not survive that check. It claimed to protect the version.is_some() clause in the listing cache; removing the clause left it green, because with either store a version is unreadable only when the store is also empty, so the clause cannot currently fire at all. The test proves something real — a directory that appears after startup is served on the very next request — and now says so instead.

Assertions worth copying

• Assert on parsed values, not on formatting. Replacing a table with a scalar leaves the key’s original decor, so the output can read value= 3 — valid TOML, different text. A string comparison there fails for the wrong reason, or passes for one.
• Cache-invalidation tests must share one Answers instance. A test that constructs a fresh one per call bypasses the cache entirely and silently proves nothing.
• Config::from_lookup takes a closure, so configuration tests never touch the process environment — and therefore never race each other under a parallel test runner.
• Assert the old text was found before writing. Two python/sed patches in this project’s history silently matched nothing and were only caught by checking test counts afterwards.

The example answers are a test too

RESCRIPTUM_ANSWERS_DIR=examples cargo run -- check

examples/ holds a worked example of every format, and it is the only place they are shown composing together. Two of them caught real bugs — a missing doctype and an unpaired pass attribute. Keep them working.

The package is tested too, in three places

cargo test does not touch the DSM package, because none of it is Rust. Three harnesses do, and each proves something the others cannot.

packaging/dsm/lifecycle-test.sh                     # the first .spk in dist/ that runs here
docker compose -f packaging/dsm/vm/docker-compose.yml up -d   # a DSM 7.2 machine
packaging/dsm/vm/on-dsm.sh admin@<host> -p 2222     # against it
packaging/dsm/vm/on-dsm.sh admin@nas                # the verdict

The VM is vdsm/virtual-dsm, which installs Synology’s own Virtual DSM release — no loader image to find. KVM makes it fast rather than possible: without /dev/kvm it emulates, about ten times slower, which is what docker-compose.emulated.yml is for. It does want 14 GiB free for the storage, hardcoded in the image.

The last one is destructive on purpose — it upgrades over a hand-edited env file and a canary in the shared folder, then uninstalls, then checks both survived. Those two guards are the most expensive things in the package to get wrong, and the first published .spk is the one whose uninstall scripts will run during everybody’s first upgrade. packaging/dsm/vm/README.md is the rig: what it is evidence about, and what it is not.

The same rule as everywhere else applies to these: break the thing they guard and watch them go red. Reverting the postinst upgrade guard, making postuninst delete the share, returning 1 for a stopped package and refusing prestart turns 33 green checks into 25 green and 8 red — which is how we know the harness is testing anything at all. Today it is 85 checks in lifecycle-test.sh, 28 in check-spk.sh and 52 on the machine; the three most recently added were each watched red the same way — by putting RESCRIPTUM_TFTP_ADDR=off back, by deleting the panel’s report of the TFTP state, and by making it claim to be serving with nothing bound.

CI

.github/workflows/ci.yml, on every push to main and develop and on every pull request:

The cross job is not redundant. SQLite is compiled from source, and armv7-musl is the least forgiving target shipped — it is where a C dependency breaks first. Catching that on a push beats catching it while cutting a release.

The audit job is the other half of the rule that adding a dependency needs a reason: a reason to add one is not a reason to keep it. --deny warnings fails on an unmaintained or yanked crate too, not only on a vulnerability. When something appears with no fix, add --ignore RUSTSEC-… with a line saying why rather than dropping the flag.

Every action used is an official actions/* one, and both Zig and cargo-audit are installed directly rather than through a third-party action. That is deliberate: this toolchain vets and links a binary people run as root.

The docs site has its own gate — see the docs site.