pmacs/docs/ci-crdt-coverage-framing.md

679 lines
34 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Framing — the CRDT half of the test corpus is dark in CI
**Revision 4.** Status: **PR #209 open, all 14 checks green on the first
run.** Branch `ci-crdt-coverage`, based on `githubsucks/main` @
`4223dd3` (#208). Approved at revision 2.
**Revision 3 → 4** records the first CI run and one review finding:
- **Acceptance 8 holds against the real run.** `Test (crdt)` reported
**3,717 passed / 0 failed / 30 ignored** — 3,746 census (with
`basedpyright` not skipped as it is locally) + 1 doc test 30
ignored. The job compiled and ran the corpus; it did not report green
over nothing. §1.7's warning that the local green proved nothing about
hosted runners is now **discharged by evidence** rather than still
outstanding.
- **A gap `feature-census` structurally could not see**: pmacs-protocol
has its own `crdt` feature. It gates no tests — the census is 17
either way — but it changes `cfg!(feature = "crdt")` *expressions*
inside the capability defaults, so those 17 tests exercise different
runtime values under it, and CI had only ever run the non-crdt ones.
Closed by an explicit `-p pmacs-protocol --features crdt` step. **A
feature can matter to a crate a per-test census scores as
unaffected**, and the script's header now says so.
**Revision 2 → 3** records implementation findings, not a new design
round. Three things changed:
- **§1.1's target-column claim was wrong**, and `scripts/feature-census`
found it. "Eight test binaries contain zero tests" merged two
different true statements; the corrected reading is in §1.1.
- **The census now has a tool** (§1.9). The ledger's standing "re-measure,
don't quote" instruction had never had one.
- **Acceptance 9 is revised** — the deliberate-break bite is replaced by
a structural coverage assertion plus the CI count reconciliation, with
what each does and does not prove stated explicitly. See §4.
**Revision 1 → 2** recorded the decisions on Q#CC3, Q#CC4 and Q#CC7, and
**corrected revision 1's classification of `m10_10_perf`**, which was
wrong in a way that would have made the lane worse (§1.3a). It also
completed the disposition accounting: all 279 dark tests assigned, 275
recovered and 4 excluded for named reasons (§1.2a).
`.github/workflows/ci.yml` never enables the `crdt` feature anywhere.
Every `#[cfg(feature = "crdt")]` test is therefore **not compiled** in
CI — not skipped, not filtered, not reported. **279 tests have never
executed in CI, and 186 of them are in the library**, whose
`cargo test --lib --features crdt` invocation `CLAUDE.md` lists as a
**required** pre-PR gate. CI has never once run a required gate.
This lane was named in `docs/active-work.md` under "NEEDS A LANE" and
has had no branch and no owner since it was found while gating #166.
---
## 0. Coherence impact (COHERENCE §20)
- **Journey steps touched:** none directly. This lane adds no user-facing
surface and no command.
- **Interaction islands added:** none.
- **Config registry adoption:** none.
- **Background-work attribution:** none.
- **Why it belongs on the board anyway:** it is the release-readiness
prerequisite for **P8 (Distribution, §17)**. §17's grade is "missing —
zero release machinery exists," and the first thing release machinery
must do is produce an artifact from a tree whose tests actually ran.
Shipping binaries over a corpus where half of the library's tests have
never been compiled by CI converts an invisible gap into a shipped one.
This lane does not advance a coherence concern; it protects the one
that comes next.
---
## 1. Ground truth (measured at `4223dd3`, 2026-08-01)
Every number below was **re-measured on this tree**, not quoted forward.
The ledger is explicit that the figure moves with every merge, and it
did: the previous reading was 273 at `74301d1`.
### 1.1 The census
Under CI's exact flags versus the same flags plus `crdt`:
| | tests | targets with ≥1 test |
|---|---:|---:|
| `--no-default-features --features luajit` | 3,467 | 93 |
| `--no-default-features --features luajit,crdt` | 3,746 | 101 |
| **dark** | **279** | **8 targets gain tests** |
**Corrected in revision 3.** Revisions 1 and 2 read the target column as
"eight test binaries contain zero tests under CI's flags." That is not
what the column measures, and `scripts/feature-census` (§1.9) found the
error by reporting both figures separately. Two different true
statements had been merged:
- **Eleven** targets run under CI's flags with **zero tests**. They
build, run, and report `ok` with nothing in them.
- **Eight** of those eleven **gain tests under `crdt`** — the ones this
lane recovers. The other three are helper binaries (`pmacs_audit`,
`pmacs_fake_lsp`, `pmacs_fake_mcp`) with no tests in either
configuration, which is correct and not a gap.
A green result from the eight is not weak evidence, it is no evidence.
The three are fine. Conflating them inflates the defect.
Per target, every row with a nonzero delta:
| dark | CI | full | target |
|---:|---:|---:|---|
| 186 | 1,899 | 2,085 | **the library itself** (`src/lib.rs`) |
| 21 | 15 | 36 | `m5_5_acceptance` |
| 14 | 1 | 15 | `gpu_invocation_acceptance` |
| 14 | 1 | 15 | `gpu_initial_target_acceptance` |
| 8 | 0 | 8 | `m10_11_acceptance` |
| 6 | 0 | 6 | `m10_2_perf` |
| 6 | 0 | 6 | `auto_pair_crdt_acceptance` |
| 4 | 5 | 9 | `vterm_stage3_acceptance` |
| 4 | 0 | 4 | `m10_10_perf` |
| 3 | 2 | 5 | `bottom_panel_stage2b_gpu_acceptance` |
| 3 | 0 | 3 | `compile_mode_crdt_acceptance` |
| 2 | 22 | 24 | `theme_faces_acceptance` |
| 2 | 0 | 2 | `m11_5_semantic_acceptance` |
| 1 | 18 | 19 | `terminal_copy_mode_acceptance` |
| 1 | 10 | 11 | `gpu_font_acceptance` |
| 1 | 9 | 10 | `vterm_stage1_acceptance` |
| 1 | 7 | 8 | `statusline_segments_acceptance` |
| 1 | 0 | 1 | `m10_11_perf` |
| 1 | 0 | 1 | `auto_indent_crdt_acceptance` |
The rows sum to 279. This is the whole census, not its head.
`bottom_panel_stage2b_gpu_acceptance` is new since the previous reading.
### 1.2 "279 dark" overstates what a plain `crdt` leg recovers
**Eleven of the 279 are `#[ignore]`d**, so adding `--features crdt` to
the `test` job does not run them — `--ignored` does, and the `test` job
does not pass it. Measured by listing the ignored set under the crdt
build and differencing against the CI build:
| dark | of which ignored | recovered by a plain leg | target |
|---:|---:|---:|---|
| 186 | 1 | 185 | the library |
| 8 | 3 | 5 | `m10_11_acceptance` |
| 6 | 6 | **0** | `m10_2_perf` |
| 1 | 1 | **0** | `m10_11_perf` |
| — | — | — | all other rows: nothing ignored |
**A plain `crdt` leg recovers 268 tests, not 279.** The remaining 11
need either an `--ignored` invocation or a deliberate exclusion.
Quoting 279 as the lane's deliverable would overstate it by the exact
set that is hardest to place.
### 1.2a Full disposition of all 279
Every dark test is assigned. Nothing is residue.
| count | disposition | mechanism |
|---:|---|---|
| 268 | **recovered by the plain `crdt` leg** — including `m10_10_perf`'s 4, which stay unignored (§1.3a) | new ubuntu/luajit `crdt` job |
| 7 | **recovered by a new `m10-perf-gates` job**: `m10_2_perf` (6) + `m10_11_perf` (1) | `--release --ignored --features crdt`, per the `m5`/`m6` precedent |
| 3 | **deliberately excluded**: `m10_11_acceptance`'s PTY-doubled tests, marked *"operator-invoked before tagging, not CI-default"* | unchanged; documented, not a gap |
| 1 | **deliberately excluded**: `buffer::tests::proptests::crdt_undo_of_an_identity_replace_reports_a_no_op_edit_carrying_an_op` — the **#157 CRDT undo repro**, an `#[ignore]`d marker for a known open defect | unchanged; arming it is that defect's lane, not this one |
| **279** | | |
**The lane recovers 275 of 279.** The other four are excluded with
stated reasons. This distinction matters for the PR description: a lane
that says "268 of 279" invites the question of what the 11 are, and two
of the four answers are "already correct."
### 1.3 The deliberate/accidental classification — the lane's stated first task
`docs/active-work.md` says this classification "is not finished and is
the lane's first task." It is finished here.
**Deliberate, and belonging in a perf job**`#[ignore]`d because they
are release-mode benches or budget gates, exactly like
`m5_perf_acceptance` and `m6_perf_acceptance`:
- `m10_2_perf` — 6 dark, all 6 ignored: *"perf bench; release-mode-only
via `--ignored --nocapture`"*.
- `m10_11_perf` — 1 dark, ignored: *"perf gate; requires release build"*.
**But their placement is accidental even so.** Grepping `ci.yml` for
`--test` yields exactly four named suites: `acceptance`,
`m4_acceptance`, `m5_perf_acceptance`, `m6_perf_acceptance`.
**`m10_2_perf` and `m10_11_perf` have no CI job at all**, with or
without `crdt`. Their being `#[ignore]`d is deliberate; their being
unreferenced by any workflow is not. Q#CC4 fixes the second.
**Deliberate, and belonging nowhere in CI** — a third disposition,
distinct from the above and easy to collapse into it:
- `m10_11_acceptance` — 3 of its 8 dark are ignored, marked *"PTY-doubled
tests are operator-invoked before tagging, not CI-default"*. These are
not benches awaiting a job; they are **deliberately manual**, run by an
operator before tagging a release. Giving them a job would contradict
the reason they are ignored. Its other 5 dark tests are ordinary
coverage and are recovered by the plain leg.
**Accidental**`m10_10_perf`: 4 dark, **zero `#[ignore]` markers**, and
no CI job naming it. Accidental in that *nothing runs it*. See §1.3a for
why its unignored state is nevertheless deliberate and must be
preserved.
Everything else in the table is accidental: ordinary correctness
coverage that has simply never been compiled.
### 1.3a `m10_10_perf` is not a perf gate — revision 1 got this wrong
Revision 1 classified `m10_10_perf` as a perf suite that "a plain `crdt`
leg would start running inside the general correctness job... a hazard
the fix must handle," and the natural remedy — add `#[ignore]`, give it
a perf job — was proposed and accepted on that basis. **The suite's own
header falsifies it:**
> The numbers are recorded to stdout via eprintln (visible under
> `cargo test -- --nocapture`) and asserted against **generous bounds
> that exist to catch catastrophic regressions, not to verify a tight
> perf claim.**
Its four tests are catastrophic-regression tripwires with deliberately
loose bounds. **The absence of `#[ignore]` is the design, not an
oversight**, and the contrast with its siblings is explicit in their
ignore reasons:
| suite | `#[ignore]` reason | what it is |
|---|---|---|
| `m10_2_perf` | *"perf bench; release-mode-only via `--ignored --nocapture`"* | benchmark |
| `m10_11_perf` | *"perf gate; requires release build"* | budget gate |
| `m10_10_perf` | **none** | regression tripwire |
Three consequences:
1. **Adding `#[ignore]` would demote a deliberate CI-default tripwire
into an operator-invoked bench** — a coverage *reduction* shipped
inside a coverage lane.
2. **The hazard revision 1 named does not apply to this suite.** Perf
assertions are dangerous in a shared correctness job when their
bounds are tight; generous bounds designed to catch only catastrophic
regressions are precisely what is safe there.
3. **`m10_10_perf` therefore needs no job of its own.** It is recovered
by the plain `crdt` leg, and its 4 tests are inside the 268.
The general lesson, which is the reusable part: *a suite's `#[ignore]`
state is a claim about how it should be invoked, and the file that makes
the claim is the authority.* Revision 1 classified three suites by their
filename suffix (`_perf`) and their marker counts, and got the one whose
name and markers disagreed with its purpose exactly backwards.
### 1.4 The clippy obstacle is real, and the ledger's inventory is stale
`cargo clippy --workspace --all-targets --features crdt -- -D warnings`
**fails on `main`.** The ledger recorded this and correctly warned that
its list was "a lower bound, not an inventory," because clippy abandons
remaining targets once one fails.
**`--keep-going` is what converts the lower bound into an inventory.**
That flag was not used before; with it, the complete set at `4223dd3` is
**eight findings across four files**:
| file:line | lint |
|---|---|
| `src/daemon.rs:4464` | `useless_conversion` to the same type: `u64` |
| `src/daemon.rs:4544` | item in documentation is missing backticks |
| `src/daemon.rs:4551` | `too_many_lines` (112/100) |
| `tests/auto_indent_crdt_acceptance.rs:42` | missing doc backticks |
| `tests/bottom_panel_stage2b_gpu_acceptance.rs:509` | `too_many_lines` (104/100) |
| `tests/vterm_stage3_acceptance.rs:637` | `too_many_lines` (122/100) |
| `tests/vterm_stage3_acceptance.rs:816` | `too_many_lines` (132/100) |
| `tests/vterm_stage3_acceptance.rs:866` | redundant `continue` |
**The ledger's list is wrong in both directions**, which is why it had to
be re-measured rather than carried forward: the `unneeded mut` at
`src/daemon.rs:4965` is **gone** (fixed incidentally by later work), a
finding in `bottom_panel_stage2b_gpu_acceptance.rs` is **new**, and every
`src/daemon.rs` line number has moved. A stale lint inventory is worse
than none, because it invites fixing lines that no longer exist.
None of the eight is a correctness defect. All are lint-policy findings,
and none requires a behavioral change — which is what makes them safe to
clear in a preparatory commit rather than a design round.
### 1.5 `PMACS_REQUIRE_GPU` does not cover the suites the fix wants to move
The proposed fix routes four GPU-requiring `crdt` suites onto the
existing `gpu-render` job, on the grounds that it already has lavapipe
and `PMACS_REQUIRE_GPU=1`. **That job runs `cargo test -p pmacs-gpu` — a
different package.** All four target suites live in the root `pmacs`
package's `tests/`, so moving them means adding a *new* root-package
invocation to that job, not extending an existing one.
And the guard is **not uniform across the four**. Grepping every
reference to `PMACS_REQUIRE_GPU`:
- `tests/vterm_stage3_acceptance.rs` — two sites, both binary-presence
skips promoted to failures.
- `tests/bottom_panel_stage2b_gpu_acceptance.rs` — one site, same shape.
- `pmacs-gpu/src/main.rs` — adapter presence, a different condition.
- **`gpu_invocation_acceptance` and `gpu_initial_target_acceptance`
reference it nowhere.** Setting the variable does not arm them.
So `PMACS_REQUIRE_GPU=1` is necessary for a37 and the panel suite and
**insufficient as a blanket guarantee** that all four ran. Whatever
proves these suites executed has to be per-suite, not one environment
variable assumed to cover the set.
### 1.6 The a37 vacuum, restated with its current mechanism
`a37_real_daemon_real_pty_and_headless_gpu_render_one_terminal_session`
derives the frontend binary path from `CARGO_BIN_EXE_pmacs`'s sibling and
**returns `ok` after an `eprintln!` when it is absent**. The ledger
measured 9/9 in 0.17 s having never run it, against ~4 s for a real run.
`PMACS_REQUIRE_GPU` is the only thing that promotes that skip to a
failure, and `CLAUDE.md` applies that flag to `cargo test -p pmacs-gpu`,
a different package — so **the required local gate does not cover a37
either.** This is unchanged and re-confirmed at the grep above.
### 1.7 The local sweep is green, and what that does and does not mean
A full serialized sweep under CI's flags plus `crdt`
`cargo test --all-targets --no-default-features --features luajit,crdt -- --test-threads=1 --skip basedpyright`
— completed on this machine at `4223dd3`:
**104 suites reporting, 3,715 passed, 0 failed, 30 ignored.**
It **reconciles exactly** with §1.1's census, which is the check that
matters: 3,715 passed + 30 ignored + 1 `basedpyright` test filtered by
`--skip` = **3,746**, the crdt-flags census figure. No test binary was
silently absent from the sweep, and no suite reported `ok` for a
population smaller than the census predicted. A sweep that did not
reconcile would be the a37 vacuum at corpus scale.
Seven targets report `ok. 0 passed`. Three are helper binaries with no
tests (`pmacs_audit`, `pmacs_fake_lsp`, `pmacs_fake_mcp`) — expected.
**The other four make §1.3's asymmetry visible in the run itself:**
`m5_perf_acceptance` and `m6_perf_acceptance` report zero *and have
`--ignored` CI jobs*; `m10_2_perf` and `m10_11_perf` report identically
and **have none**. The same output line means "covered elsewhere" for two
of them and "covered nowhere" for the other two, which is exactly why
Q#CC4 cannot be answered by looking at a test run.
**What this establishes:** the 268 recoverable tests are not hiding
correctness defects. Bet 1 holds locally.
**What it does not establish, and the distinction is the lane's whole
risk:** the failures the ledger predicts are *hosted-runner timing and
concurrency* failures — real PTY behavior on CI runners, wgpu under
lavapipe, daemon sockets at unfamiliar concurrency. A green serialized
run on a 16-thread developer machine removes one class of explanation
(the tests are wrong) and leaves the class actually expected (the
environment differs) entirely untested. **This result must not be quoted
as evidence that the CI leg will be green.**
### 1.8 What is NOT established
- **No macOS reading exists.** Every measurement here is Linux. The
ledger's expectation that these suites will be flakier on hosted
runners than on a developer machine is an expectation, not a
measurement.
- **This machine is a laptop with an integrated Radeon (Phoenix1), 16
threads.** It is a real Vulkan device rather than lavapipe, but it is
shared-memory and thermally constrained, which is precisely the
condition under which the ledger records a37 producing
`last_frame_text` all spaces with nonzero `rendered_nonuniform_frames`.
**A red a37 here is ambiguous by construction.** Universum (7900 XTX,
available remotely) is where that ambiguity gets settled — and settling
it there proves the *test* is sound, not that the *CI job* will pass,
because the CI job runs on lavapipe regardless.
---
### 1.9 The census now has a tool
`docs/active-work.md` says the dark-test figure "moves with every merge
and must be re-measured, not quoted." **That instruction had no tool**,
so every re-measurement was a hand-rolled `--list` pipeline written from
scratch — including this lane's, whose first attempt filtered on
`/ : test$/` when libtest prints `name: test` with no space before the
colon. It matched nothing, reported zero targets, and looked like a
clean run.
`scripts/feature-census <config-a> <config-b> [--covers <test>]` is that
tool. It reproduces every figure in §1.1 and §1.2 independently, and its
header records each parsing trap, all of which were hit while writing
it. Two are worth repeating here because they change results rather than
merely breaking a run:
- **A target with zero tests prints its `Running` line and nothing
else.** Counting only test lines drops it from the diff entirely —
losing precisely the finding worth surfacing. This is how the §1.1
error above survived two revisions.
- **Both configurations need an `--ignored` pass, not just the richer
one.** Counting only B's ignored set attributes pre-existing ignores
to the feature: `rope::tests::perf_smoke_*` are ignored under both and
are not "dark and ignored." The difference is what turns a flat
"279 dark" into §1.2's "268 recovered, 11 needing `--ignored`."
It is **fail-closed on a build failure** (exit 3). A configuration that
does not compile yields no test list, which is indistinguishable by
counting from "this configuration has no tests" and would render as an
entirely false "every test is dark." That is not a small error; it is a
number that would get quoted.
## 2. Questions
- **Q#CC1 — does the `crdt` leg go on the existing `test` matrix or its
own job?** The matrix is already 4 legs (2 OS × 2 Lua flavors) and is
the CI critical path at an observed 17-minute max. A `crdt` leg on all
four doubles the most expensive job. A separate ubuntu-only job costs
one leg. **Leaning: separate job, ubuntu-only, per the ledger's "start
ubuntu-only and decide about macOS from evidence."**
- **Q#CC2 — is `crdt` matrixed over Lua flavor?** The feature is
orthogonal to the Lua VM. `m5-perf-gates` and `m6-perf-gates` already
set the precedent of luajit-only with a written justification.
**Leaning: luajit only, with the reason recorded in the workflow the
way the existing perf jobs do.**
- **Q#CC3 — where does `m10_10_perf` go? DECIDED: it stays unignored and
runs in the plain `crdt` leg.** The decision was first taken as "give
it a perf job," on revision 1's classification; §1.3a then established
that classification was wrong — the suite is a deliberate CI-default
regression tripwire with generous bounds, not a bench, and adding
`#[ignore]` would reduce coverage inside a coverage lane. **The
decision was reversed on the evidence, not on preference**, and it is
a one-line change to reinstate the original answer if the tripwire
reading is rejected.
- **Q#CC4 — do `m10_2_perf` and `m10_11_perf` get a job in this lane, or
a follow-on? DECIDED: in this lane.** They are dark for a second,
independent reason (unreferenced by any workflow) that predates the
`crdt` gap, so this PR does fix two causes at once. That is accepted
deliberately: leaving them would ship a lane headlined "the dark tests
now run" with 7 still dark, and the second cause is one job block, not
a second investigation. They go in a new `m10-perf-gates` job shaped
like the existing `m5-perf-gates` / `m6-perf-gates`.
- **Q#CC5 — what proves each GPU suite actually executed?** §1.5
establishes `PMACS_REQUIRE_GPU` covers only two of the four. Does the
lane add the guard to the other two, or assert execution some other
way (a test-count floor per suite)?
- **Q#CC6 — do the clippy fixes ride this PR or precede it?** Eight
lint-policy findings across four files, none behavioral. Riding along
means the PR that adds CI coverage also touches `src/daemon.rs`.
**Leaning: a separate preparatory commit on the same branch**, so the
diff reads as two intents, but not a separate PR — the lints are
unreachable-by-CI today and have no independent reason to be fixed.
- **Q#CC7 — is a red first run acceptable to merge behind? DECIDED:
fix in-lane by default, with one bounded escape hatch (below).**
First, one of the three options offered in revision 1 was not real.
**"Land the leg non-required until it is green" assumed the first CI
run happens after merge. It does not** — the new job runs on the pull
request, so the entire first-run failure set is visible *during
review*, before anything reaches `main`. There is no window in which
`main` carries a red required check, and therefore nothing to protect
against by landing non-required. The genuine choice is only
fix-in-lane versus quarantine-with-follow-on, taken per failure once
observed.
**Default: fix in-lane.** The escape hatch is keyed on *cause class*,
not on effort, because effort is what turns a scoped lane into an
unbounded one:
1. **The lane's own configuration** (wrong flags, missing build step,
a suite that skipped vacuously) — **always in-lane.** It is this
PR's defect.
2. **Reproduces locally** under §1.7's sweep or on Universum —
**in-lane.** A real test defect the lane surfaced is the lane
working as intended.
3. **Environment-only** — green serialized here, green on Universum,
red only on a hosted runner — **named follow-on lane.** This is the
class that historically consumed multiple review rounds on this
project: the signal lane had three tolerance rules rejected across
three revisions, each concluding something about a process from
something that was not about that process. Committing in advance to
resolve that class *inside a workflow-configuration PR* is
committing to an unbounded investigation in a lane whose whole
value is being small and mergeable.
**§1.7 makes class 3 the most likely red and class 2 the least**, which
is the uncomfortable direction: the corpus is already green serialized
on a developer machine, so a CI red is by elimination an environment
difference. The escape hatch exists precisely because the measurement
points that way.
---
## 3. Bets
- **Bet 1 — the 268 recoverable tests mostly pass. RESOLVED: they all
do, locally.** The serialized sweep is 3,715/3,715 with an exact
census reconciliation (§1.7). This bet is settled for the developer
machine and **explicitly not settled for CI**, which is the
environment the lane is actually changing.
- **Bet 2 — the failures that do appear concentrate in the process/PTY
and GPU suites**, not in the library. The library's 185 are pure logic
over a CRDT backend; the flake surface the ledger documents is
uniformly real-process and real-device. *Unresolved: with zero local
failures there is nothing yet to concentrate. This bet now resolves
only on the first CI run, and is the reason §7 sequences the GPU
commit last.*
- **Bet 3 — the eight clippy findings are the complete blocker.** With
`--keep-going` there is no remaining truncation, so no further lint
surprises appear once these are cleared. *Falsified if clearing them
reveals findings in targets that failed to build for a non-lint
reason.*
- **Bet 4 — `PMACS_REQUIRE_GPU` alone will not prove the GPU suites ran.**
§1.5 already all but establishes this; the bet is that a per-suite
execution assertion is needed and that adding it finds at least one
suite silently skipping.
---
## 4. Acceptance
Written against §1's measurements, with the local sweep (§1.7) in hand.
Criteria 8 and 9 exist specifically because that sweep came back green:
a green pre-measurement is the condition under which a vacuous CI job is
easiest to ship unnoticed.
1. `cargo clippy --workspace --all-targets --features crdt -- -D warnings`
exits zero on the branch, and the check is run **with `--keep-going`**
so its success is an inventory rather than a first-failure abort.
2. The dark census is **re-measured on the branch** and the workflow runs
a leg that compiles them. **275 of 279 recovered**, with the four
exclusions named individually (§1.2a) — never one headline number
with an unexplained remainder.
2a. `m10_10_perf` is **unmodified** by this lane: no `#[ignore]` added,
no job of its own, its 4 tests recovered by the plain leg. A diff
touching `tests/m10_10_perf.rs` fails this criterion (§1.3a).
3. Every GPU-requiring suite added to a job **proves it executed**, per
Q#CC5 — a suite that skips its body reports failure, not `ok`.
`PMACS_REQUIRE_GPU` alone does not satisfy this, because it is absent
from two of the four suites (§1.5).
4. `a37` specifically: a run that does not build `pmacs-gpu` **fails**
rather than reporting 9/9 in 0.17 s.
5. The perf suites' placement is explicit and each is justified in the
workflow text: `m10_2_perf` and `m10_11_perf` in a new
`m10-perf-gates` job matching the `m5`/`m6` precedent;
`m10_10_perf` deliberately in the correctness leg, **with its
generous-bounds rationale written into the workflow comment** so a
later reader does not "fix" the inconsistency by ignoring it.
6. The new job carries `timeout-minutes`, per the workflow's own standing
rule that every job does.
7. `docs/active-work.md`'s "NEEDS A LANE" block is replaced by this
lane's state, and its stale figures (273 dark, the seven-item clippy
list) are corrected rather than left beside the new ones.
8. **The new job's test count reconciles.** Its reported passed +
ignored + filtered must equal the branch's re-measured crdt census
for the suites it runs, the way §1.7 reconciles to 3,746. A job that
runs fewer tests than the census predicts has found a silently-absent
binary, and that is the defect class this whole lane exists to end.
9. **The leg is proven load-bearing, structurally rather than by
breakage.** *Revised in revision 3 — the original criterion asked for
a deliberately-broken test, and it is worth saying plainly why that
was dropped rather than quietly meeting a weaker bar.*
The original: break a `crdt`-gated library test, confirm the job goes
red, revert. It proves two things at once — that the leg compiles
crdt tests, and that a failure propagates to a red job. The second is
generic cargo/libtest behavior, not anything this lane changes, and
buying it costs a mutation of tracked source restored by a shell
trap, plus a broken commit in the PR's history.
What replaces it, in two parts:
- **Structural, and stronger on the point that matters:**
`scripts/feature-census luajit luajit,crdt --covers <test>` asserts
a named test is present under the new job's flags and **absent
under the old job's**. That is a claim the break test does *not*
make — a red job proves the leg caught something, but not that the
existing job could never have. Verified for
`crdt_apply_edit_keeps_invariant_basic`: present under B, absent
under A, exit 0. The negative cases are exercised too (a test in
both configs, and a misspelled name, both exit 1).
- **Empirical, from the first CI run:** criterion 8's count
reconciliation. A job reporting ~3,715 passed has demonstrably
compiled and run the crdt corpus; a vacuous leg reports a number
near the `test` job's.
**What is no longer proven, stated rather than glossed:** that a
*failing* crdt test turns this specific job red. Nothing on this
branch demonstrates it. It rests on cargo returning non-zero on test
failure and GitHub Actions failing a step on non-zero — both
universal, neither lane-specific. If review wants that proven
directly, the break test is still the way, and it belongs on a
throwaway PR rather than in this one's history.
---
## 5. Parked
- **macOS.** Ubuntu-only first, by the ledger's own instruction. A macOS
`crdt` leg is a follow-on decided from the first run's evidence.
- **The `--lib --features crdt` flake.** `process::tests::setsid_escapee_is_not_reaped_and_teardown_reclaims_readers`
failing ~1 in 5 under parallel full-suite load, with
`active_reader_probe` returning `None`. The ledger's leading
explanation — `drain_until`'s tick reaping the leader before the probe
— is an inference from control flow, **not a falsified root cause**, and
no serial full-suite bite has been run to separate parallelism from
another whole-suite effect. Discriminating it is named as belonging to
this lane; it should be its own PR, because it is a product defect
hypothesis and everything else here is workflow configuration.
- **The two unattributed CRDT failures from #178's round-2 gating.** No
test names were captured, so there is nothing to reproduce.
- **`basedpyright`.** Still hangs forever, still `--skip`ped, still
deliberately not installed in CI. Unchanged by this lane.
---
## 6. Gates
The standing suite from `CLAUDE.md`, plus the two this lane exists to
make meaningful:
- `cargo fmt --check`
- `cargo clippy --workspace --all-targets -- -D warnings`
- **`cargo clippy --workspace --all-targets --features crdt --keep-going -- -D warnings`** (new, and the lane's own subject)
- **`scripts/feature-census luajit luajit,crdt`** (new; re-measures the
census rather than quoting it forward, per the ledger's own rule)
- `cargo test --lib`
- `cargo test --lib --features crdt`
- the touched acceptance suites
- `cargo test --test m4_acceptance -- --skip basedpyright`
- `PMACS_REQUIRE_GPU=1 cargo test -p pmacs-gpu`
- `git diff --check`
Verified green on this machine at `4223dd3` **before any change**: fmt,
diff-check, `--lib` (1,896 passed), `--lib --features crdt` (2,081
passed, 4 ignored), required GPU (221 passed), and the full serialized
crdt sweep (3,715 passed, 0 failed, 30 ignored, reconciling exactly to
the 3,746 census — §1.7).
Recording the pre-change baseline matters here more than usual: this
lane's subject *is* the test corpus, so without a baseline any red on
the branch is unattributable between the lane's changes and the tests it
newly compiles.
---
## 7. Branch plan
One branch, `ci-crdt-coverage`. Commits in this order, because each
earlier one is a precondition for the next being observable. **As
landed:**
1. **Clear the eight clippy findings** (§1.4) — `7a9cf5b`. Nothing can
compile the `crdt` targets under `-D warnings` until this lands, so
no CI change was testable before it.
2. **Add the `m10-perf-gates` job** for `m10_2_perf` and `m10_11_perf`
(Q#CC4) — `06abbac`. **No source file changes**: per Q#CC3 and
§1.3a, `m10_10_perf` is not touched and gets no `#[ignore]`.
3. **Add the `crdt-test` job**`7a8746d`. Recovers 268 tests,
`m10_10_perf`'s 4 among them.
4. **Give the census a tool**`a776bc3`, `scripts/feature-census`
(§1.9). This replaced the planned break-test as the non-vacuity
proof; see acceptance 9 for what was traded away and why.
5. **Update `docs/active-work.md` and `docs/agent-handoff.md`** per
acceptance 7.
**Two departures from the plan as approved, both deliberate:**
- **The GPU suites did not get a separate commit against `gpu-render`.**
The plan's step 5 assumed they could be added to that job; §1.5
established it runs a different package, so co-locating them there
would have meant a new invocation rather than an extension, plus a
standing requirement to classify every future suite as GPU-requiring
or not. They are covered by `crdt-test` instead, which installs
lavapipe and sets `PMACS_REQUIRE_GPU=1` for the whole corpus. **One
job cannot develop the hole that splitting invites.**
- **The break-test became a coverage assertion plus a tool** (acceptance
9). The proof got stronger on the claim specific to this lane — that
the old job *structurally cannot* see these tests — and weaker on a
generic one it no longer makes.
**Universum is still where the GPU question settles.** Everything above
was verified on this laptop, whose integrated Radeon renders the GPU
suites but is thermally constrained, so a red `a37` here is ambiguous by
construction (§1.8). What Universum's 7900 XTX can establish is that the
tests are sound; it cannot establish that the lavapipe CI job will pass,
and the first PR run is the only thing that can. Budget for that gap
rather than assuming it away.