Skip to content

Commit 3b72387

Browse files
authored
feat(build): mutation testing sized to the change, with its scratch space on disk instead of memory (#1636)
## Goal Keep workspace copies and disposable Cargo targets on disk. Use by-hand mutation checks for every delivery. Run a project's mutation command only when the task asks for that campaign by name. **Spec:** `.ai/specs/SPEC-600-mutation-testing-sized-to-the-change-wit.md` ## Plan - [x] Bound session tmpfs and provide run-owned disk scratch - [x] Reap interrupted builds and verify current-run baseline evidence - [x] Record independent before-and-after measurements - [x] Install the instance's by-hand delivery policy and bootstrap contract - [x] Preserve Cargo cancellation and inherited jobserver tokens - [x] Reclaim attempt scratch after forced host termination - [x] Deliver the by-hand mutation policy and bootstrap - [ ] Deliver disk build scratch, the tmpfs cap and on-request campaign cleanup - [ ] Deliver disposable Cargo targets and request staging ## Steps - Bound session tmpfs and provide run-owned disk scratch (done) - Reap interrupted builds and verify current-run baseline evidence (done) - Record independent before-and-after measurements (done) - Install the instance's by-hand delivery policy and bootstrap contract (done) - Preserve Cargo cancellation and inherited jobserver tokens (done) - Reclaim attempt scratch after forced host termination (done) - Deliver the by-hand mutation policy and bootstrap (done) - Deliver disk build scratch, the tmpfs cap and on-request campaign cleanup (pending) - Deliver disposable Cargo targets and request staging (pending) --- Delivered with Yulius task #583: https://yulius.quartio.ai/s/yulius/t/583 <!-- codesmith:footer --> --- <a href="https://dashboardbackend.blacksmith.sh/track/run-proof?expires=1797888262&installation_model_id=441550&pr_number=1636&ref=codesmith_pr_footer&repository=Quartio-Studio%2Fyulius&return_to=https%3A%2F%2Fgithub.com%2FQuartio-Studio%2Fyulius%2Fpull%2F1636&variant=prove_it_works&signature=552f7ebed37f1f7fe1a48cfb0a5677a2a7e58aa83686066089ac2b45a4994c67"><picture><source media="(prefers-color-scheme: dark)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/proof-prove-it-works-v2-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/proof-prove-it-works-v2-light.svg"><img alt="Prove it works" src="https://pr-comments-assets.blacksmith.sh/codesmith/proof-prove-it-works-v2-dark-3x.png" width="129" height="28"></picture></a> <sup><a href="https://app.blacksmith.sh/Quartio-Studio/codesmith/yulius/pr/1636?autoLogin=true&ref=codesmith_pr_footer">Open in [code]smith</a> · Need help on this PR? Tag <code>@codesmith-bot</code> with what you need.</sup> <!-- codesmith:autofix:disabled --> <!-- /codesmith:footer -->
1 parent 14de9b0 commit 3b72387

29 files changed

Lines changed: 220 additions & 128 deletions

File tree

‎.agents/skills/yulius-git/SKILL.md‎

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,9 +28,7 @@ the task's existing authorization. Only a pre-existing bug, a flaky test the cha
2828
did not cause or a cosmetic nit left after 3 rounds is
2929
filed, saying why it is outside the change.
3030

31-
Require a successful `yulius/ci` status on the exact resulting dev SHA. A task's work
32-
ends there; with a merge-first staging target (Yulius's own `[cd.dev]`, ADR-0436), once its
33-
staging request (`cd_deploy`, when `delivery_status` says it is due) was answered. Promotion (dev → main, a merge commit, never a squash) is a person's, on the CD
31+
Merge on green Yulius CI for the exact head. For merge-first staging (Yulius's `[cd.dev]`, ADR-0436), ask once with `cd_deploy` when `delivery_status` says it is due. The task ends after asking. Yulius reports a failed gate, deploy or verification for a fix forward. Promotion (dev → main, a merge commit, never a squash) is a person's, on the CD
3432
page (ADR-0110): a task that needs it asks with `needs_you`. Keep both permanent branches. Do not
3533
claim CI proves a deployed app works.
3634

‎.agents/skills/yulius-review/SKILL.md‎

Lines changed: 3 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -43,14 +43,11 @@ test the change did not cause, or a cosmetic nit left after 3 rounds becomes
4343
an issue, saying why it is outside the change ("present on dev before this change",
4444
with the evidence). The PR answers every finding; deferring an own-code finding
4545
makes your verdict request changes (`docs/engineering/workflow.md`, Review output).
46-
Before asking for another round, answer each finding of the last report with the
47-
test that fails without its fix (or why no test can), check every finding of
48-
that report against the new head, and run the project's mutation command on the
49-
lines of the fix where it can take them. Here: `scripts/dev mutants --base
50-
<reviewed head>`.
46+
Before asking for another round, answer each finding of the last report with the test that fails
47+
without its fix (or why no test can) and check every finding of that report against the new head.
5148
After an approval, fix the remaining nits within 20 product lines and merge
5249
without another round: `delivery_merge` with the approved head counts them
5350
(tests and documents do not count). More than that, or a file leaving the
5451
change, needs a new round. A usage limit can cut a reviewer off before its report: resume that same reviewer by its agent id, never start a new one for the same round. A reviewer runs at high effort, never max.
5552

56-
Mutation testing runs beside the review: when you ask for a pull request's first review, start the project's mutation command in the background and let the reviewer read meanwhile; when it ends, put its summary line in the spec's Evidence and fix each missed mutant test-first before the merge. After a later round's fix, run it on that fix's lines alone. A project without one undoes by hand the few lines its new tests are meant to hold and says what failed. The reviewer may undo a single line it doubts and runs no campaign of its own. Nobody writes a mutation script by hand. A test that cannot fail is still fixed in the same pull request (ADR-0141). Here the command is `scripts/dev mutants`.
53+
Mutation checks are by hand, in every project: undo the few lines your new tests are meant to hold, watch those tests fail, restore the lines, and say in the spec's Evidence what failed. Do not start a mutation run or campaign for a delivery (not beside the review, not between rounds, not before the merge), also where a project's own documents still ask for one: run the project's mutation command only when the task's request asks for it by name (instance policy of 2026-11-21). The reviewer may undo a single line it doubts and runs no campaign of its own. Nobody writes a mutation script by hand. A test that cannot fail is still fixed in the same pull request (ADR-0141). Here the on-request command is `scripts/dev mutants`.

‎.agents/skills/yulius-verify/SKILL.md‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -56,8 +56,6 @@ Yulius CI is unrun until its run for the exact head exists; local equivalence is
5656
a claim that `yulius/ci` passed. Leave product acceptance open if its evidence
5757
is absent, even when the harness passes.
5858

59-
For published delivery, also verify the required PR check and the `yulius/ci` status
60-
of the exact post-merge dev commit per `docs/engineering/git.md`. Local green checks do
61-
not replace that post-merge gate.
59+
For published delivery, verify the required PR check on the exact head. After the merge, request staging once. Yulius records integration CI on the landed revision and reports a failed gate, deployment or verification. The task ends after asking.
6260

63-
Mutation testing runs beside the review: when you ask for a pull request's first review, start the project's mutation command in the background and let the reviewer read meanwhile; when it ends, put its summary line in the spec's Evidence and fix each missed mutant test-first before the merge. After a later round's fix, run it on that fix's lines alone. A project without one undoes by hand the few lines its new tests are meant to hold and says what failed. The reviewer may undo a single line it doubts and runs no campaign of its own. Nobody writes a mutation script by hand. A test that cannot fail is still fixed in the same pull request (ADR-0141). Here the command is `scripts/dev mutants`.
61+
Mutation checks are by hand, in every project: undo the few lines your new tests are meant to hold, watch those tests fail, restore the lines, and say in the spec's Evidence what failed. Do not start a mutation run or campaign for a delivery (not beside the review, not between rounds, not before the merge), also where a project's own documents still ask for one: run the project's mutation command only when the task's request asks for it by name (instance policy of 2026-11-21). The reviewer may undo a single line it doubts and runs no campaign of its own. Nobody writes a mutation script by hand. A test that cannot fail is still fixed in the same pull request (ADR-0141). Here the on-request command is `scripts/dev mutants`.

0 commit comments

Comments
 (0)