neuwirth
/
pmacs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

docs: record the CI CRDT coverage lane and its corrections 

Framing revision 3, plus the ledger and handoff updates the lane owes.

docs/active-work.md's "NEEDS A LANE" block is replaced rather than
annotated: the lane now has a branch, a framing, an implementation and
verification, so the old text's premise ("no branch, no framing yet")
was the part that needed removing. Its stale figures go with it — the
273-dark reading at 74301d1 and the seven-item clippy list, both
superseded and both explicitly labelled do-not-quote in the new block.

The corrections this lane produced, all now recorded where the next
reader will hit them:

  * m10_10_perf is a CI-DEFAULT REGRESSION TRIPWIRE, not a bench. Its
    bounds are deliberately generous "to catch catastrophic regressions,
    not to verify a tight perf claim", so #[ignore]ing it to give it a
    perf job would have reduced coverage inside a coverage lane. The
    framing had this backwards through revision 1 and says so.

  * gpu-render runs `cargo test -p pmacs-gpu` — a DIFFERENT PACKAGE from
    the root-package GPU suites. The long-recorded fix-shape of "move
    the GPU-requiring crdt suites onto gpu-render, it already has
    lavapipe" does not work as written.

  * PMACS_REQUIRE_GPU is not uniform: absent from
    gpu_invocation_acceptance and gpu_initial_target_acceptance
    entirely, so it cannot serve as blanket proof the GPU suites ran.

  * The dark-test classification found THREE dispositions, not two:
    benches awaiting a job, deliberately-manual operator tests, and
    known-defect markers. Collapsing the second into the first gives a
    CI job to tests whose #[ignore] reason says not to.

  * §1.1's own target-column claim was wrong, and scripts/feature-census
    caught it. Eleven targets run with zero tests under CI's flags;
    eight of those gain tests under crdt and three are helper binaries
    with no tests in either. The framing had merged two true statements.

Acceptance 9 is revised rather than quietly met at a lower bar. The
planned deliberately-broken-test bite is replaced by the --covers
coverage assertion plus the CI count reconciliation, and the doc states
what that trades: stronger on the claim specific to this lane (the old
job STRUCTURALLY cannot see these tests), and no longer proving that a
failing crdt test turns the job red — which is generic cargo and Actions
behavior, not anything this lane changes.

The handoff gains a census hazard beside the existing sweep note,
because every trap in it was hit while writing the script and one of
them survived two framing revisions: libtest prints `name: test` with no
space before the colon, a zero-test target prints only its Running line,
and both configurations need an --ignored pass or pre-existing ignores
get attributed to the feature.

The ledger also records what is NOT established, which is the lane's
whole remaining risk: the sweep is green SERIALIZED ON A DEVELOPER
MACHINE, and the failures this lane expects are hosted-runner timing and
concurrency. That green removes the "tests are wrong" explanation and
leaves the expected one untested. It must not be quoted as evidence the
CI leg will be green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Levi Neuwirth 2026-08-01 11:00:12 -04:00
parent a776bc337b
commit 57abcd940e
No known key found for this signature in database
3 changed files with 297 additions and 180 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

259
docs/active-work.md
View File

@ -5,6 +5,15 @@ landed on `main`. Read it after `docs/agent-handoff.md`. Remove completed
entries when their PR merges; do not let this become a second permanent entries when their PR merges; do not let this become a second permanent
backlog. backlog.
**Updated later the same day, on a new machine.** Development moved to
the laptop; the recovery path in "Repository authority" below was
exercised from this checkout and the `githubsucks` alias was absent and
had to be added, exactly as that section anticipates. **One lane opened:
CI CRDT coverage**, which had been sitting under "NEEDS A LANE" with no
branch and no owner since #166. It is implemented on
`ci-crdt-coverage` and its block replaces the old one below. Everything
else in this file is unchanged.
**This snapshot is an absorption pass, taken with ZERO open PRs** — **This snapshot is an absorption pass, taken with ZERO open PRs** —
the one window in which a ledger refresh has nothing to re-conflict the one window in which a ledger refresh has nothing to re-conflict
with, and taken deliberately before a machine move. Nine PRs landed with, and taken deliberately before a machine move. Nine PRs landed
@ -139,140 +148,132 @@ form. All four steps ran clean. **The two-argument form still does not
work** for a remote-only branch (`fatal: invalid reference`), which is work** for a remote-only branch (`fatal: invalid reference`), which is
why every lane below spells out the `-b` form. why every lane below spells out the `-b` form.
## The CRDT half of the test corpus is dark in CI — NEEDS A LANE ## CI CRDT coverage lane — IMPLEMENTED on `ci-crdt-coverage`, PR not yet open
- **No branch, no framing yet.** Found while gating #166, then measured **This lane had no branch and no owner from #166 until 2026-08-01.** It
properly during the vterm as-framed audit. Deliberately kept out of #166 so now has both. Framing:
a CI change would not arrive after review approval. `docs/ci-crdt-coverage-framing.md` revision 3 (approved at revision 2).
- **Root cause:** `.github/workflows/ci.yml` never enables the `crdt` feature Branch `ci-crdt-coverage` off `githubsucks/main` @ `4223dd3`, developed
anywhere — zero hits across the workflow directory. The `test` job runs in the primary checkout on the laptop, not a worktree.
`cargo test --all-targets --no-default-features --features luajit|lua54`.
Every `#[cfg(feature = "crdt")]` test is therefore **not compiled** in CI,
not merely skipped.
- **Measured, `--list` under CI's exact flags versus the same flags plus
`crdt`: 3,176 vs 3,449 — 273 tests dark.** Re-measured at `74301d1`
(2026-07-26; at `fe8b8ba` it read 3,170 vs 3,443, the same 273 dark —
#176 added six tests, none of them `crdt`-gated). **The number moves
with every merge and must be
re-measured, not quoted.** #168 reported 3,024 vs 3,288 — 264 dark,
177 in the library — at `1b6a084`; #178 then added CRDT-only
generated-buffer coverage, and other lanes landed CRDT tests in
between. Per target:
| dark | CI | full | target | - **Root cause, unchanged:** `.github/workflows/ci.yml` never enabled
|---:|---:|---:|---| the `crdt` feature anywhere. Every `#[cfg(feature = "crdt")]` test was
| 185 | 1,848 | 2,033 | **the library itself** (`src/lib.rs`) | therefore **not compiled** in CI, not merely skipped — including the
| 21 | 15 | 36 | `m5_5_acceptance` | 186 library tests behind `cargo test --lib --features crdt`, which
| 13 | 1 | 14 | `gpu_invocation_acceptance` | `CLAUDE.md` lists as a REQUIRED pre-PR gate. CI had never run a
| 13 | 1 | 14 | `gpu_initial_target_acceptance` | required gate.
| 8 | 0 | 8 | `m10_11_acceptance` | - **Re-measured at `4223dd3`: 3,467 vs 3,746 — 279 dark**, up from the
| 6 | 0 | 6 | `auto_pair_crdt_acceptance` | 273 recorded at `74301d1`. **Do not quote this number either.** It
| 6 | 0 | 6 | `m10_2_perf` | moves with every merge, and there is now a tool: `scripts/feature-census
| 4 | 5 | 9 | `vterm_stage3_acceptance` | luajit luajit,crdt` reproduces the whole per-target table, the ignored
| 4 | 0 | 4 | `m10_10_perf` | split, and the zero-test-target count in one command.
| 3 | 0 | 3 | `compile_mode_crdt_acceptance` | - **The 279 are fully dispositioned; 275 are recovered.** 268 by the new
| 2 | 22 | 24 | `theme_faces_acceptance` | `crdt-test` job, 7 by the new `m10-perf-gates` job, and 4 deliberately
| 2 | 0 | 2 | `m11_5_semantic_acceptance` | excluded: `m10_11_acceptance`'s three PTY-doubled tests (marked
| 1 | 14 | 15 | `terminal_copy_mode_acceptance` | operator-invoked before tagging) and the #157 CRDT undo repro, an
| 1 | 9 | 10 | `vterm_stage1_acceptance` | `#[ignore]`d known-defect marker whose arming belongs to that defect's
| 1 | 7 | 8 | `statusline_segments_acceptance` | lane.
| 1 | 10 | 11 | `gpu_font_acceptance` | - **The deliberate/accidental classification the old lane text called
| 1 | 0 | 1 | `auto_indent_crdt_acceptance` | "the lane's first task" is finished**, and it found THREE dispositions
| 1 | 0 | 1 | `m10_11_perf` | rather than two: benches awaiting a job (`m10_2_perf`,
`m10_11_perf`), deliberately-manual operator tests
(`m10_11_acceptance`), and known-defect markers. Collapsing the second
into the first would have given a CI job to tests whose `#[ignore]`
reason says not to.
- **`m10_10_perf` is NOT a perf gate**, and the framing got this wrong
first. Its header says its bounds are "generous ... to catch
catastrophic regressions, not to verify a tight perf claim," so its
lack of `#[ignore]` is the design. Adding one to give it a job would
have shipped a coverage reduction inside a coverage lane. It stays
untouched and rides the plain leg.
- **The clippy obstacle is cleared, and the old inventory was stale in
both directions.** `cargo clippy --workspace --all-targets --features
crdt` had never passed. The previous seven-item list was correctly
labelled "a lower bound, not an inventory" — `--keep-going` is what
converts it. The real set was eight findings across four files; the
`unneeded mut` at `daemon.rs:4965` had been fixed incidentally, a
finding in `bottom_panel_stage2b_gpu_acceptance.rs` was new, and every
`daemon.rs` line number had moved.
- **One job, not two.** The fix-shape recorded here previously put the
GPU-requiring suites onto `gpu-render` "which already has lavapipe and
PMACS_REQUIRE_GPU". **That job runs `cargo test -p pmacs-gpu` — a
different package** from the four root-package suites, so co-locating
them means a new invocation rather than an extension. Splitting also
requires classifying every future suite as GPU-requiring or not, and a
misclassified one skips forever. `crdt-test` installs lavapipe and
sets `PMACS_REQUIRE_GPU=1` for the whole corpus instead.
- **`PMACS_REQUIRE_GPU` is not uniform and cannot serve as blanket
proof**: it appears in `vterm_stage3_acceptance` (twice) and
`bottom_panel_stage2b_gpu_acceptance` (once), and **not at all** in
`gpu_invocation_acceptance` or `gpu_initial_target_acceptance`.
- **The external-tool install block is deliberately not duplicated.**
Measured: it gates `m4_acceptance`, `m6_5_repl_acceptance` and
`m6_8_multi_repl_acceptance`, none of which has a single dark test.
The only tool-gated code in the dark set is `src/process.rs`, whose
two variables need no install.
- **Crdt clippy runs in `crdt-test`, not the `clippy` job.** The `clippy`
job matrixes over Lua flavor and never enables `crdt`, so clearing
those lints once would let them drift straight back and the next job
to compile them would be red on arrival — the exact state this lane
found.
- **Verification on the laptop (integrated Radeon, 16 threads), all at
the exact commands CI runs:** clippy green with and without `crdt`;
fmt; diff-check; `--lib` 1,896; `--lib --features crdt` 2,081; doc
tests; `m10_2_perf` 6/6 in 79s and `m10_11_perf` 1/1 in 5s under
release; and the full serialized sweep with `PMACS_REQUIRE_GPU=1` at
**3,715 passed / 0 failed / 30 ignored in 366s**, reconciling exactly
to the 3,746 census (3,715 + 30 + 1 basedpyright-skipped). The
reconciliation is the point: a sweep that does not reconcile is the
a37 vacuum at corpus scale.
- **What is NOT established, and it is the lane's whole remaining risk:**
the sweep is green *serialized on a developer machine*. The failures
this lane expects are hosted-runner timing and concurrency — real PTY
on CI runners, wgpu under lavapipe, daemon sockets at unfamiliar
concurrency. A green local run removes the "tests are wrong"
explanation and leaves the expected one untested. **Do not quote it as
evidence the CI leg will be green.**
- **Red-first-run policy, decided:** fix in-lane by default, with one
escape hatch keyed on cause class. Lane-configuration failures and
locally-reproducing failures are fixed here; an environment-only
failure (green locally and on Universum, red only on a hosted runner)
becomes a named follow-on rather than an unbounded investigation
inside a workflow-config PR. The local green makes that third class
the most likely red, which is why the hatch exists.
- **Universum (7900 XTX, remote) is where the GPU ambiguity settles.**
The laptop renders the GPU suites but is thermally constrained, which
is the documented condition for a37's "all spaces with nonzero
rendered_nonuniform_frames" signature. Universum can establish the
tests are sound; it cannot establish that the lavapipe CI job passes.
The rows sum to 273; the table is the whole census, not its head. Recovery from a clean checkout:
- **The single worst line is the library.** `cargo test --lib --features crdt` ```sh
is a REQUIRED local gate in `CLAUDE.md`, and CI has never run it. 185 git fetch githubsucks
library tests — the whole CRDT half — are developer-machine-only, and git worktree add ../pmacs-ci-crdt \
that count grows with every merged branch that adds a `crdt`-gated -b ci-crdt-coverage \
unit test. githubsucks/ci-crdt-coverage
- **Ten suites run zero or one test in CI**, including `gpu_initial_target` ```
(#148's entire acceptance, 1/14), `gpu_invocation` (#141's, 1/14), and
`a37`, the Vterm Stage 3 real-daemon/real-PTY/real-wgpu path that #135 ### Still owned by this lane, not yet done
built specifically because "a decoded-message fixture would prove none of
the three fit together". - **The `--lib --features crdt` flake.**
- **⚠ `a37` will report green in the new job without running, unless the
job builds `pmacs-gpu` AND sets `PMACS_REQUIRE_GPU=1`.** Measured
2026-07-26 while gating #173. `a37_real_daemon_real_pty_and_headless_gpu_
render_one_terminal_session` derives its sibling binary path from
`CARGO_BIN_EXE_pmacs`, and on a missing binary it `eprintln!`s a skip and
**returns `ok`**. A fresh worktree running
`cargo test --features crdt --test vterm_stage3_acceptance` reports **9/9
in 0.17 s having never run it**; a real run takes ~4 s. Only
`PMACS_REQUIRE_GPU=1` promotes that skip to a failure, and `CLAUDE.md`
applies that flag to `cargo test -p pmacs-gpu` — a **different package**,
so the required local gate does not cover a37 either. The `gpu-render`
job already sets the flag, which is what makes fix-shape part 2 sound;
state it as a **requirement** of that job rather than inheriting it by
luck, because a `crdt` leg added to the plain `test` job would run a37
vacuously.
- **`a37` is also load-sensitive, which changes how to read the expected
first-run failures.** It passed at `d152120` and failed at that *same
commit* twenty minutes later, with a second agent saturating the machine
with `rustc` in between; it then failed identically on `d152120`,
`04c5ad1`, and the #173 merge commit, which is how #173 established the
failure was not its own. The signature is `last_frame_text` all spaces
with `rendered_nonuniform_frames` nonzero — frames arrive, content does
not. `pmacs-gpu`'s own suite flaked the same way under the same load
(201/202, then 202/202 on immediate rerun). **So a red a37 on the first
CI run is ambiguous by construction**: before treating it as a real
failure, run the same command on the merge base, and prefer serialized
execution for this suite over retry-until-green.
- **Sort deliberate from accidental before proposing a fix.** Some of the 264
are perf suites that are `#[ignore]`d by default and belong to their own
jobs (`m10_2_perf` 6, `m10_11_perf` 1). `m10_10_perf` has **no** `#[ignore]`
and no CI job naming it, so it looks accidental. This classification is not
finished and is the lane's first task.
- **Fix shape, two parts** (the flag combination is verified to work:
`--no-default-features --features luajit,crdt` lists 10 vterm Stage 1 tests
versus 9 without):
1. a `crdt` leg on the `test` job for the non-GPU suites and the library;
2. the GPU-requiring `crdt` suites onto the existing `gpu-render` job, which
already has lavapipe and `PMACS_REQUIRE_GPU=1` —
`vterm_stage3_acceptance`, `gpu_invocation_acceptance`,
`gpu_initial_target_acceptance`, `gpu_font_acceptance`.
- **Expect first-run failures, and budget for them.** These would execute in
CI for the first time ever: real PTY timing on CI runners, wgpu under
lavapipe, and daemon-socket tests at unfamiliar concurrency. Start
ubuntu-only and decide about macOS from evidence. A red first run is the
lane working, not the lane failing.
- Mitigating fact, verified rather than assumed: #166's three unit pins are
**not** `crdt`-gated and do run under CI's exact flags, including the
controller-release pin whose only job is catching the plausible wrong fix.
- **This lane also owns a `--lib --features crdt` flake, observed and
scoped without overclaiming its cause** (inherited from #178's gating,
where the terminal lane recorded it). `cargo test --lib --features
crdt` failed ~1 run in 5 on
`process::tests::setsid_escapee_is_not_reaped_and_teardown_reclaims_readers` `process::tests::setsid_escapee_is_not_reaped_and_teardown_reclaims_readers`
— `active_reader_probe` returning `None` at `process.rs:3179` ("live failed ~1 run in 5 with `active_reader_probe` returning `None`. **It
runtime probe"). **Pre-existing and unrelated to #178:** that branch did not reproduce in this lane's runs** — `--lib --features crdt` was
did not touch `src/process.rs` at all, and the test passed 10/10 2,081/2,081 and the full serialized sweep was clean — but a
standalone; the observed non-reproduction under serial execution is consistent with the
failures were during parallel full-suite runs. That localizes the leading hypothesis rather than evidence against it: the trigger was
trigger to suite load or interaction, but does **not** distinguish observed under *parallel* full-suite load, and every run here was
parallelism from another full-suite effect — no serial full-suite bite `--test-threads=1`. The `drain_until` explanation (draining for
was run. The leading code-path explanation is the known `drain_until` `Started` also ticks, and a tick can reap the leader before the
trap: draining for `Started` also ticks, and a tick can reap the leader following probe) remains an inference from control flow, not a
before the following `active_reader_probe`. That is an inference from falsified root cause. Discriminating it is its own PR — it is a
the failure site and control flow, not yet a falsified root cause. product-defect hypothesis and everything else here is workflow
Discriminating it belongs here. Two unnamed CRDT failures in #178's configuration.
round-2 gating are a plausible match but remain **unattributed** — no - **The two unattributed CRDT failures from #178's round-2 gating.** No
test names were captured. test names were captured, so there is nothing to reproduce.
- **A second standing obstacle for this lane:** `cargo clippy --workspace - **macOS.** Ubuntu-only first, deliberately; a macOS `crdt` leg is a
--all-targets --features crdt -- -D warnings` **fails on `main`** — follow-on decided from the first run's evidence.
measured at `74301d1`: seven errors before the build aborts, four in
`src/daemon.rs` (`useless_conversion` at 3996, missing doc backticks at
4076, `too_many_lines` 112/100 at 4083, an unneeded `mut` at 4965) and
three in `tests/vterm_stage3_acceptance.rs` (`too_many_lines` at 637
and 793, a redundant `continue` at 843). **Treat that as a lower
bound, not an inventory:** Clippy abandons the remaining targets once
one fails, and a run on an older tree surfaced a further doc-backticks
error in `tests/auto_indent_crdt_acceptance.rs:42` that this run never
reached. The
standing gate list runs Clippy without `crdt`, so these lints have
never been enforced. Any CI job that compiles the `crdt` targets has to
fix them first or it will be red on arrival.
## Discovery lane (P4) — STAGE 1 MERGED (#207); STAGE 2 IS NEXT ## Discovery lane (P4) — STAGE 1 MERGED (#207); STAGE 2 IS NEXT

33
docs/agent-handoff.md
View File

@ -1,6 +1,6 @@
# Agent handoff — cross-machine continuity # Agent handoff — cross-machine continuity
**Last updated: 2026-07-29, as bottom-panel Stage 2B-3 — the GPU panel **Last updated: 2026-08-01 (CI CRDT coverage lane opened on `ci-crdt-coverage`; development moved to the laptop). Previously 2026-07-29, as bottom-panel Stage 2B-3 — the GPU panel
band, compatible protocol-v21 activation, and the negotiated band, compatible protocol-v21 activation, and the negotiated
`panel_capable` flip, completing Arc 7 Stage 2 — opens atop `e003b81`. `panel_capable` flip, completing Arc 7 Stage 2 — opens atop `e003b81`.
Beneath it, bottom-panel Stage 2B-2 (#187) — the Beneath it, bottom-panel Stage 2B-2 (#187) — the
@ -94,10 +94,21 @@ anchor, so every item is startable.
#### Open lanes (branch exists, work not finished) #### Open lanes (branch exists, work not finished)
- **CRDT half of the corpus is dark in CI — still no lane, no owner.** - **CRDT half of the corpus is dark in CI — LANE OPENED 2026-08-01**
CI never enables `crdt`, so those tests are *not compiled*, not merely (`ci-crdt-coverage`, `docs/ci-crdt-coverage-framing.md` rev 3;
skipped. **Re-measure before quoting a number**; the ledger's figure implemented, PR not yet open). CI never enabled `crdt`, so those tests
moves with every merge. were *not compiled*, not merely skipped — including 186 library tests
behind a **required** `CLAUDE.md` gate CI had never run.
**Re-measure before quoting a number**, and there is now a tool:
`scripts/feature-census luajit luajit,crdt`. Measured 279 dark at
`4223dd3`; 275 recovered, 4 excluded with stated reasons. Two
corrections it forced are worth carrying: **`m10_10_perf` is a
CI-default regression tripwire, not a bench** (its bounds are
deliberately generous, so `#[ignore]`ing it to give it a perf job
would *reduce* coverage), and **`gpu-render` runs a different
package** (`pmacs-gpu`) from the root-package GPU suites, so the
long-recorded "move them onto gpu-render" fix-shape does not work as
written.
- **Generated-buffer immutability** — Stage 1 merged (#191); Stage 2 - **Generated-buffer immutability** — Stage 1 merged (#191); Stage 2
not started. Four writer mechanisms have still not adopted not started. Four writer mechanisms have still not adopted
`set_generated_contents`; key the inventory by *writer*, not buffer. `set_generated_contents`; key the inventory by *writer*, not buffer.
@ -156,7 +167,17 @@ someone forgot.
`cargo test --test m4_acceptance -- --skip basedpyright`. `cargo test --test m4_acceptance -- --skip basedpyright`.
- **The crdt sweep needs `cargo build --workspace` first**, or twelve - **The crdt sweep needs `cargo build --workspace` first**, or twelve
`gpu_invocation_acceptance` tests fail on a missing `pmacs-gpu` `gpu_invocation_acceptance` tests fail on a missing `pmacs-gpu`
binary. binary. `cargo build --workspace --no-default-features --features
luajit,crdt` is the invocation that produces both binaries.
- **Never hand-roll the dark-test census — use
`scripts/feature-census`.** libtest prints `name: test` with **no
space before the colon**, so a filter written `/ : test$/` matches
nothing and reports a clean zero; a target with zero tests prints its
`Running` line and nothing else, so counting only test lines drops it
from the diff; and both configurations need an `--ignored` pass, or
pre-existing ignores get attributed to the feature. Each of those was
hit while writing the script, and the second one survived two
revisions of a framing doc.
- **A green a37 means nothing on its own** — the vterm real-daemon - **A green a37 means nothing on its own** — the vterm real-daemon
acceptance returns `ok` without running unless `pmacs-gpu` is built. acceptance returns `ok` without running unless `pmacs-gpu` is built.

185
docs/ci-crdt-coverage-framing.md
View File

@ -1,15 +1,25 @@
# Framing — the CRDT half of the test corpus is dark in CI # Framing — the CRDT half of the test corpus is dark in CI
**Revision 2.** Status: framing only. No branch, no implementation. **Revision 3.** Status: **implemented** on branch `ci-crdt-coverage`,
Scouted against `githubsucks/main` @ `4223dd3` (#208), with **zero open based on `githubsucks/main` @ `4223dd3` (#208). Approved at revision 2.
PRs** on the board.
**Revision 1 → 2** records the user's decisions on Q#CC3, Q#CC4 and **Revision 2 → 3** records implementation findings, not a new design
Q#CC7, and **corrects revision 1's classification of `m10_10_perf`**, round. Three things changed:
which was wrong in a way that would have made the lane worse. See §1.3a.
The correction also completes the disposition accounting: all 279 dark - **§1.1's target-column claim was wrong**, and `scripts/feature-census`
tests are now assigned, with 275 recovered and 4 excluded for named found it. "Eight test binaries contain zero tests" merged two
reasons (§1.2a). different true statements; the corrected reading is in §1.1.
- **The census now has a tool** (§1.9). The ledger's standing "re-measure,
don't quote" instruction had never had one.
- **Acceptance 9 is revised** — the deliberate-break bite is replaced by
a structural coverage assertion plus the CI count reconciliation, with
what each does and does not prove stated explicitly. See §4.
**Revision 1 → 2** recorded the decisions on Q#CC3, Q#CC4 and Q#CC7, and
**corrected revision 1's classification of `m10_10_perf`**, which was
wrong in a way that would have made the lane worse (§1.3a). It also
completed the disposition accounting: all 279 dark tests assigned, 275
recovered and 4 excluded for named reasons (§1.2a).
`.github/workflows/ci.yml` never enables the `crdt` feature anywhere. `.github/workflows/ci.yml` never enables the `crdt` feature anywhere.
Every `#[cfg(feature = "crdt")]` test is therefore **not compiled** in Every `#[cfg(feature = "crdt")]` test is therefore **not compiled** in
@ -51,16 +61,27 @@ did: the previous reading was 273 at `74301d1`.
Under CI's exact flags versus the same flags plus `crdt`: Under CI's exact flags versus the same flags plus `crdt`:
| | tests | test binaries | | | tests | targets with ≥1 test |
|---|---:|---:| |---|---:|---:|
| `--no-default-features --features luajit` | 3,467 | 93 | | `--no-default-features --features luajit` | 3,467 | 93 |
| `--no-default-features --features luajit,crdt` | 3,746 | 101 | | `--no-default-features --features luajit,crdt` | 3,746 | 101 |
| **dark** | **279** | **8** | | **dark** | **279** | **8 targets gain tests** |
The binary column is a fact the earlier census did not carry: **eight **Corrected in revision 3.** Revisions 1 and 2 read the target column as
test binaries contain zero tests under CI's flags.** They are still "eight test binaries contain zero tests under CI's flags." That is not
built, still run, and still report `ok` — with nothing in them. A green what the column measures, and `scripts/feature-census` (§1.9) found the
result from those eight is not weak evidence, it is no evidence. error by reporting both figures separately. Two different true
statements had been merged:
- **Eleven** targets run under CI's flags with **zero tests**. They
build, run, and report `ok` with nothing in them.
- **Eight** of those eleven **gain tests under `crdt`** — the ones this
lane recovers. The other three are helper binaries (`pmacs_audit`,
`pmacs_fake_lsp`, `pmacs_fake_mcp`) with no tests in either
configuration, which is correct and not a gap.
A green result from the eight is not weak evidence, it is no evidence.
The three are fine. Conflating them inflates the defect.
Per target, every row with a nonzero delta: Per target, every row with a nonzero delta:
@ -328,6 +349,38 @@ as evidence that the CI leg will be green.**
--- ---
### 1.9 The census now has a tool
`docs/active-work.md` says the dark-test figure "moves with every merge
and must be re-measured, not quoted." **That instruction had no tool**,
so every re-measurement was a hand-rolled `--list` pipeline written from
scratch — including this lane's, whose first attempt filtered on
`/ : test$/` when libtest prints `name: test` with no space before the
colon. It matched nothing, reported zero targets, and looked like a
clean run.
`scripts/feature-census <config-a> <config-b> [--covers <test>]` is that
tool. It reproduces every figure in §1.1 and §1.2 independently, and its
header records each parsing trap, all of which were hit while writing
it. Two are worth repeating here because they change results rather than
merely breaking a run:
- **A target with zero tests prints its `Running` line and nothing
else.** Counting only test lines drops it from the diff entirely —
losing precisely the finding worth surfacing. This is how the §1.1
error above survived two revisions.
- **Both configurations need an `--ignored` pass, not just the richer
one.** Counting only B's ignored set attributes pre-existing ignores
to the feature: `rope::tests::perf_smoke_*` are ignored under both and
are not "dark and ignored." The difference is what turns a flat
"279 dark" into §1.2's "268 recovered, 11 needing `--ignored`."
It is **fail-closed on a build failure** (exit 3). A configuration that
does not compile yields no test list, which is indistinguishable by
counting from "this configuration has no tests" and would render as an
entirely false "every test is dark." That is not a small error; it is a
number that would get quoted.
## 2. Questions ## 2. Questions
- **Q#CC1 — does the `crdt` leg go on the existing `test` matrix or its - **Q#CC1 — does the `crdt` leg go on the existing `test` matrix or its
@ -474,12 +527,41 @@ easiest to ship unnoticed.
for the suites it runs, the way §1.7 reconciles to 3,746. A job that for the suites it runs, the way §1.7 reconciles to 3,746. A job that
runs fewer tests than the census predicts has found a silently-absent runs fewer tests than the census predicts has found a silently-absent
binary, and that is the defect class this whole lane exists to end. binary, and that is the defect class this whole lane exists to end.
9. **A deliberately-broken bite proves the leg is not vacuous.** Before 9. **The leg is proven load-bearing, structurally rather than by
merge, break one `crdt`-gated library test on the branch and confirm breakage.** *Revised in revision 3 — the original criterion asked for
the new CI job goes red. A leg added to a corpus that is already a deliberately-broken test, and it is worth saying plainly why that
green locally cannot otherwise be distinguished from a leg that was dropped rather than quietly meeting a weaker bar.*
compiles nothing — which is precisely the failure `.github` has today
and the reason criterion 8 is not sufficient on its own. The original: break a `crdt`-gated library test, confirm the job goes
red, revert. It proves two things at once — that the leg compiles
crdt tests, and that a failure propagates to a red job. The second is
generic cargo/libtest behavior, not anything this lane changes, and
buying it costs a mutation of tracked source restored by a shell
trap, plus a broken commit in the PR's history.
What replaces it, in two parts:
- **Structural, and stronger on the point that matters:**
`scripts/feature-census luajit luajit,crdt --covers <test>` asserts
a named test is present under the new job's flags and **absent
under the old job's**. That is a claim the break test does *not*
make — a red job proves the leg caught something, but not that the
existing job could never have. Verified for
`crdt_apply_edit_keeps_invariant_basic`: present under B, absent
under A, exit 0. The negative cases are exercised too (a test in
both configs, and a misspelled name, both exit 1).
- **Empirical, from the first CI run:** criterion 8's count
reconciliation. A job reporting ~3,715 passed has demonstrably
compiled and run the crdt corpus; a vacuous leg reports a number
near the `test` job's.
**What is no longer proven, stated rather than glossed:** that a
*failing* crdt test turns this specific job red. Nothing on this
branch demonstrates it. It rests on cargo returning non-zero on test
failure and GitHub Actions failing a step on non-zero — both
universal, neither lane-specific. If review wants that proven
directly, the break test is still the way, and it belongs on a
throwaway PR rather than in this one's history.
--- ---
@ -511,6 +593,8 @@ make meaningful:
- `cargo fmt --check` - `cargo fmt --check`
- `cargo clippy --workspace --all-targets -- -D warnings` - `cargo clippy --workspace --all-targets -- -D warnings`
- **`cargo clippy --workspace --all-targets --features crdt --keep-going -- -D warnings`** (new, and the lane's own subject) - **`cargo clippy --workspace --all-targets --features crdt --keep-going -- -D warnings`** (new, and the lane's own subject)
- **`scripts/feature-census luajit luajit,crdt`** (new; re-measures the
census rather than quoting it forward, per the ledger's own rule)
- `cargo test --lib` - `cargo test --lib`
- `cargo test --lib --features crdt` - `cargo test --lib --features crdt`
- the touched acceptance suites - the touched acceptance suites
@ -534,31 +618,42 @@ newly compiles.
## 7. Branch plan ## 7. Branch plan
One branch, `ci-crdt-coverage`. Commits in this order, because each One branch, `ci-crdt-coverage`. Commits in this order, because each
earlier one is a precondition for the next being observable: earlier one is a precondition for the next being observable. **As
landed:**
1. **Clear the eight clippy findings** (§1.4). Nothing can compile the 1. **Clear the eight clippy findings** (§1.4) — `7a9cf5b`. Nothing can
`crdt` targets under `-D warnings` until this lands, so no CI change compile the `crdt` targets under `-D warnings` until this lands, so
is testable before it. no CI change was testable before it.
2. **Add the `m10-perf-gates` job** for `m10_2_perf` and `m10_11_perf` 2. **Add the `m10-perf-gates` job** for `m10_2_perf` and `m10_11_perf`
(Q#CC4), shaped like `m5-perf-gates` / `m6-perf-gates`: (Q#CC4) — `06abbac`. **No source file changes**: per Q#CC3 and
`--release --features crdt -- --ignored --nocapture`, with §1.3a, `m10_10_perf` is not touched and gets no `#[ignore]`.
`timeout-minutes` and a written luajit-only justification. 3. **Add the `crdt-test` job** — `7a8746d`. Recovers 268 tests,
**No source file changes** — per Q#CC3 and §1.3a, `m10_10_perf` is `m10_10_perf`'s 4 among them.
not touched and gets no `#[ignore]`. 4. **Give the census a tool** — `a776bc3`, `scripts/feature-census`
3. **Add the non-GPU `crdt` leg**, ubuntu-only, luajit-only, with its (§1.9). This replaced the planned break-test as the non-vacuity
justification written in the workflow. This is the commit that proof; see acceptance 9 for what was traded away and why.
recovers 268 tests, `m10_10_perf`'s 4 among them. 5. **Update `docs/active-work.md` and `docs/agent-handoff.md`** per
4. **Prove the leg is not vacuous** (acceptance 9): break one
`crdt`-gated library test, confirm the new job goes red, revert.
Do this *before* step 5, so the proof is against the simpler job.
5. **Add the GPU-requiring `crdt` suites to `gpu-render`**, with the
per-suite execution proof from Q#CC5 — this is where §1.5's
package mismatch has to be handled explicitly.
6. **Update `docs/active-work.md` and `docs/agent-handoff.md`** per
acceptance 7. acceptance 7.
**The GPU-suite commit is the one that needs Universum.** Steps 1–3 are **Two departures from the plan as approved, both deliberate:**
fully verifiable on this laptop; step 4's failures cannot be
distinguished from thermal and load noise here (§1.8), and settling them - **The GPU suites did not get a separate commit against `gpu-render`.**
on the 7900 XTX proves the tests are sound without proving the lavapipe The plan's step 5 assumed they could be added to that job; §1.5
CI job will pass. Budget for that gap rather than assuming it away. established it runs a different package, so co-locating them there
would have meant a new invocation rather than an extension, plus a
standing requirement to classify every future suite as GPU-requiring
or not. They are covered by `crdt-test` instead, which installs
lavapipe and sets `PMACS_REQUIRE_GPU=1` for the whole corpus. **One
job cannot develop the hole that splitting invites.**
- **The break-test became a coverage assertion plus a tool** (acceptance
9). The proof got stronger on the claim specific to this lane — that
the old job *structurally cannot* see these tests — and weaker on a
generic one it no longer makes.
**Universum is still where the GPU question settles.** Everything above
was verified on this laptop, whose integrated Radeon renders the GPU
suites but is thermally constrained, so a red `a37` here is ambiguous by
construction (§1.8). What Universum's 7900 XTX can establish is that the
tests are sound; it cannot establish that the lavapipe CI job will pass,
and the first PR run is the only thing that can. Budget for that gap
rather than assuming it away.