From 40709fafb93a199ae8a65e264eaad1a45aeebbf0 Mon Sep 17 00:00:00 2001 From: Levi Neuwirth Date: Sat, 25 Jul 2026 17:17:23 -0400 Subject: [PATCH] docs: measure the CI coverage gap and record the vterm as-framed audit The lane opened in the previous commit was scoped to the Vterm Stage 3 acceptance. Measuring it properly shows the problem is much larger and not vterm-specific. Comparing cargo test --list under CI's exact flags against the same flags plus crdt: 3,024 versus 3,288. 264 tests are dark in CI, and the single worst line is the library itself at 177 -- cargo test --lib --features crdt is a required local gate that CI has never run. Ten suites run zero or one test, including gpu_initial_target (#148's entire acceptance, 1 of 14), gpu_invocation (#141's, 1 of 14), and a37, the Stage 3 real-daemon/real-PTY/real-wgpu path that #135 built precisely because a decoded-message fixture would prove none of the three fit together. The lane now carries the per-target table, the verified flag combination for the fix, a two-part fix shape (a crdt leg on the test job, plus the GPU-requiring suites onto the existing gpu-render job that already has lavapipe), and an explicit instruction to sort deliberate exclusions from accidental ones first -- some of the 264 are perf suites that are ignored by default and belong to their own jobs, while m10_10_perf has no ignore attribute and no job naming it. docs/vterm-framing.md gains an as-framed audit section. The arc is structurally complete and every test named in the Stage 2 verification map exists, but criterion 22's "without thrash" clause was never pinned anywhere -- the word appears nowhere in src or tests -- and that clause describes exactly the defect #166 fixed. Of the nine Stage 3 tests, only three drive a real daemon, so the six that construct EditorState directly could never see a dispatcher-loop defect; a31 passes on the broken tree for that reason. Four of the nine, including a37 and Stage 3 review round 1's own presence regression guard, do not run in CI at all. The section also records what was not audited: section 11's blanket claim about deferral safety covers roughly twenty items and none were spot-checked. docs/gpu-terminal-input-framing.md scores bet B2 true now that the reporter has confirmed typing works, and retracts Q#GT5. The bash fixture behind it does not reproduce in real use and was almost certainly measuring its own timing rather than a product behaviour; it is marked retracted rather than deleted so nobody re-derives it from an earlier revision. docs/agent-handoff.md section 5 gains the lesson the confirmation cost: a daemon-side fix is not deployed until the daemon is restarted from a tree containing it, and rebuilding a binary does nothing to a running process. No code changes. --- docs/active-work.md | 82 +++++++++++++++++++++--------- docs/agent-handoff.md | 9 ++++ docs/gpu-terminal-input-framing.md | 26 +++++++--- docs/vterm-framing.md | 42 +++++++++++++++ 4 files changed, 129 insertions(+), 30 deletions(-) diff --git a/docs/active-work.md b/docs/active-work.md index 5624d24..5f2f142 100644 --- a/docs/active-work.md +++ b/docs/active-work.md @@ -227,32 +227,68 @@ If it does not, stop and repair the remote/fetch configuration. buffer a directory should resolve *to*, and `pmacs .` should route into it rather than growing a second directory surface. -## Stage 3 acceptance is dark in CI — NEEDS A LANE +## The CRDT half of the test corpus is dark in CI — NEEDS A LANE -- **No branch, no framing yet.** Found while gating #166; deliberately kept - out of it so a CI change would not arrive after review approval. -- `.github/workflows/ci.yml` **never enables the `crdt` feature** (grep the - workflow directory: zero hits). Every `#[cfg(feature = "crdt")]` acceptance - test is therefore not merely skipped in CI — it is **not compiled**. -- That covers the whole Vterm Stage 3 real-path acceptance, including `a37` - (real daemon + real PTY + real wgpu), which has been dark since #135, and - the two tests #166 added beside it. -- The `gpu-render` job is the only one with lavapipe and - `PMACS_REQUIRE_GPU=1`, and it runs `cargo test -p pmacs-gpu`, which never - reaches the `pmacs` crate's acceptance suites. -- The shape of the fix is one step on the `gpu-render` job: - `cargo test --features crdt --test vterm_stage3_acceptance -- --test-threads=1`. - It needs its own lane rather than a drive-by because it would run `a37` - under lavapipe **for the first time**, and neither its timing budgets nor - its wgpu path have ever been exercised on that adapter or on macOS CI. -- Worth auditing at the same time: which *other* `crdt`-gated acceptance - suites are dark for the same reason. This is a coverage question about the - gate list itself, not about any one suite. +- **No branch, no framing yet.** Found while gating #166, then measured + properly during the vterm as-framed audit. Deliberately kept out of #166 so + a CI change would not arrive after review approval. +- **Root cause:** `.github/workflows/ci.yml` never enables the `crdt` feature + anywhere — zero hits across the workflow directory. The `test` job runs + `cargo test --all-targets --no-default-features --features luajit|lua54`. + Every `#[cfg(feature = "crdt")]` test is therefore **not compiled** in CI, + not merely skipped. +- **Measured, `--list` under CI's exact flags versus the same flags plus + `crdt`: 3,024 vs 3,288 — 264 tests dark.** Per target: + + | dark | CI | full | target | + |---:|---:|---:|---| + | 177 | 1,832 | 2,009 | **the library itself** (`src/lib.rs`) | + | 21 | 15 | 36 | `m5_5_acceptance` | + | 13 | 1 | 14 | `gpu_invocation_acceptance` | + | 13 | 1 | 14 | `gpu_initial_target_acceptance` | + | 8 | 0 | 8 | `m10_11_acceptance` | + | 6 | 0 | 6 | `auto_pair_crdt_acceptance` | + | 6 | 0 | 6 | `m10_2_perf` | + | 4 | 5 | 9 | `vterm_stage3_acceptance` | + | 4 | 0 | 4 | `m10_10_perf` | + | 3 | 0 | 3 | `compile_mode_crdt_acceptance` | + | 2 | 22 | 24 | `theme_faces_acceptance` | + | 2 | 0 | 2 | `m11_5_semantic_acceptance` | + | 1 | 9 | 10 | `vterm_stage1_acceptance` | + | 1 | 7 | 8 | `statusline_segments_acceptance` | + | 1 | 10 | 11 | `gpu_font_acceptance` | + | 1 | 0 | 1 | `auto_indent_crdt_acceptance` | + | 1 | 0 | 1 | `m10_11_perf` | + +- **The single worst line is the library.** `cargo test --lib --features crdt` + is a REQUIRED local gate in `CLAUDE.md`, and CI has never run it. 177 + library tests — the whole CRDT half — are developer-machine-only. +- **Ten suites run zero or one test in CI**, including `gpu_initial_target` + (#148's entire acceptance, 1/14), `gpu_invocation` (#141's, 1/14), and + `a37`, the Vterm Stage 3 real-daemon/real-PTY/real-wgpu path that #135 + built specifically because "a decoded-message fixture would prove none of + the three fit together". +- **Sort deliberate from accidental before proposing a fix.** Some of the 264 + are perf suites that are `#[ignore]`d by default and belong to their own + jobs (`m10_2_perf` 6, `m10_11_perf` 1). `m10_10_perf` has **no** `#[ignore]` + and no CI job naming it, so it looks accidental. This classification is not + finished and is the lane's first task. +- **Fix shape, two parts** (the flag combination is verified to work: + `--no-default-features --features luajit,crdt` lists 10 vterm Stage 1 tests + versus 9 without): + 1. a `crdt` leg on the `test` job for the non-GPU suites and the library; + 2. the GPU-requiring `crdt` suites onto the existing `gpu-render` job, which + already has lavapipe and `PMACS_REQUIRE_GPU=1` — + `vterm_stage3_acceptance`, `gpu_invocation_acceptance`, + `gpu_initial_target_acceptance`, `gpu_font_acceptance`. +- **Expect first-run failures, and budget for them.** These would execute in + CI for the first time ever: real PTY timing on CI runners, wgpu under + lavapipe, and daemon-socket tests at unfamiliar concurrency. Start + ubuntu-only and decide about macOS from evidence. A red first run is the + lane working, not the lane failing. - Mitigating fact, verified rather than assumed: #166's three unit pins are - **not** `crdt`-gated and do run under CI's exact flags - (`--no-default-features --features luajit|lua54`), including the + **not** `crdt`-gated and do run under CI's exact flags, including the controller-release pin whose only job is catching the plausible wrong fix. - The regression protection is live; the real-daemon evidence is local-only. ## Bottom-panel lane (window placement + side windows) — Stage 1 IN REVIEW diff --git a/docs/agent-handoff.md b/docs/agent-handoff.md index e28911f..165f06a 100644 --- a/docs/agent-handoff.md +++ b/docs/agent-handoff.md @@ -745,6 +745,15 @@ final variant — its own round-trip cannot detect a discriminant shift. ## 5. Hard-won ops lessons +- **A daemon-side fix is not deployed until the daemon is restarted from a + tree that contains it.** #166's reporter rebuilt and saw no change: the + running daemon had been started from a shared checkout still on a pre-fix + branch, and `pmacs --gpu` attaches to whatever process already owns the + socket. Rebuilding a binary does nothing to a running process. When + validating a daemon-side fix by hand, check the running process's binary + path and start time against the tree you think you fixed — + `ps -eo pid,lstart,args | grep '[p]macs --daemon'` — before concluding the + fix failed. - **Two operations that must be alternatives are not made alternatives by being adjacent.** The dispatcher applied its grid and semantic terminal-layout syncs to every attached frontend; a semantic session diff --git a/docs/gpu-terminal-input-framing.md b/docs/gpu-terminal-input-framing.md index 0bbccef..aa1bea4 100644 --- a/docs/gpu-terminal-input-framing.md +++ b/docs/gpu-terminal-input-framing.md @@ -81,7 +81,11 @@ character: | | frames for a static screen | typed `Z` ever visible at the prompt | |---|---|---| | `main` today | **730** in a 20 s window | **no** | -| with the guard | **2** | (see Q#GT5 — a separate question) | +| with the fix | **2** | yes | + +Bet B2 is **scored TRUE**: with the fix deployed, the reporter confirmed +typing into a GPU terminal works. The earlier caveat here pointed at Q#GT5, +which is now retracted — see "Deferred (named)". The TUI is unaffected: a grid session has no semantic terminal declaration, so only one arm ever runs for it. This is a **frontend-kind** defect, which is @@ -281,11 +285,12 @@ change. Stays v20. snapshot that signals the switch-away). Hence the split in Q#GT1. Recorded rather than deleted: the failure mode is one a reviewer or a future simplification will re-propose. -- **B2.** The user's reported symptom is this defect. *Partially scored: the - storm is proven and GUI-only, and its shape (line editor unusable, output - still flowing) matches the report. Not fully scored until the user, or an - acceptance running the **user's own shell**, confirms typing works after the - fix. Q#GT5 is the reason this bet is stated rather than assumed.* +- **B2 — SCORED TRUE 2026-07-25.** "The user's reported symptom is this + defect." Confirmed in real use after the fix was deployed: typing into a GPU + terminal works. The confirmation needed a daemon **restart** built from a + tree containing the fix — the first attempt reported no change because a + pre-fix daemon still owned the socket, which is worth remembering whenever a + daemon-side fix is being validated by hand. - **B3.** No other pair of per-frontend-kind daemon operations is applied as siblings rather than alternatives. *Scored by an explicit audit of the dispatcher's per-frontend loop during implementation — this defect's shape @@ -294,7 +299,14 @@ change. Stays v20. ## Deferred (named) -- Interactive-shell echo on a raw-mode PTY (Q#GT5) — its own scout. +- ~~Interactive-shell echo on a raw-mode PTY (Q#GT5)~~ — **RETRACTED + 2026-07-25.** The observation behind it (a `bash --norc -i` fixture not + echoing typed characters) does not reproduce in real use: with the fix + deployed, typing into a GPU terminal echoes normally. The fixture was almost + certainly measuring its own timing — polling a published screen snapshot + before readline had finished initialising — not a product behaviour. Recorded + as retracted rather than deleted so nobody re-derives it from the framing's + earlier revision and spends a scout on it. - **A geometry change appears to clear the visible screen.** Observed while building acceptance 4: after the probe's deliberate 25×92 → 20×71 resize, the next frame's visible grid is entirely blank even though the content diff --git a/docs/vterm-framing.md b/docs/vterm-framing.md index b1124c0..b86bcd0 100644 --- a/docs/vterm-framing.md +++ b/docs/vterm-framing.md @@ -1674,6 +1674,48 @@ GPU assertions remain in `pmacs-protocol` and `pmacs-gpu` respectively. - **37:** one real-daemon/real-PTY/headless-wgpu acceptance path; it is not replaced by a decoded-message fixture. +### 0.12 As-framed audit, 2026-07-25 (after #166) + +Prompted by a GPU terminal input defect that shipped in Stage 3 and was fixed +in #166. The arc is structurally complete — all 37 criteria have +implementations, and every test named in the Stage 2 verification map exists — +but the audit found two gaps worth recording against the criteria themselves. + +**Criterion 22's "without thrash" was never pinned.** The criterion reads +"unchanged, zero, passive, and failed resize cases preserve prior geometry +*without thrash*". The word appears nowhere in `src/` or `tests/`. The suite +pinned the four enumerated single-arm cases and never the cross-arm +interaction — which is exactly where the thrash lived: the daemon applied +both the grid and the semantic terminal-layout sync to every attached +frontend, so a semantic session's PTY was resized twice per tick forever. +Criterion 31's "only the exact durable controller changes PTY geometry" was +violated in the same event, in spirit rather than letter: the controller was +the right frontend, but the geometry came from the grid projection. #166 adds +the settle pins; the gap was open from #135 (2026-07-22) until then. + +**Why the Stage 3 suite could not see it.** Of its nine tests, only three +drive a real daemon; the other six construct `EditorState` directly and never +execute the dispatcher loop where the defect lived. `a31`, which is about two +semantic frontends sharing one session, therefore passes on the broken tree. +The same structural blindness explains why `bottom_panel_stage1_acceptance` +was unaffected. A criterion about *dispatcher* behavior needs a test that +runs the dispatcher. + +**Four of the nine Stage 3 tests do not run in CI at all**, because they are +`#[cfg(feature = "crdt")]` and the workflow never enables that feature: +`a37`, the two added by #166, and +`terminal_mode_keeps_reporting_presence_so_peers_drop_the_stale_caret` — which +is Stage 3 review round 1's own regression guard. Stage 1's +`read_only_empty_crdt_bootstrap_is_immutable_against_remote_content`, the CRDT +half of criterion 14, is dark for the same reason. Stage 2 is fully covered +(6/6). This is not a vterm problem: 264 tests workspace-wide are dark, +including 177 in the library. It has its own lane in `docs/active-work.md`. + +**Not audited:** §11's blanket claim that "deferral means graceful ignore or +documented absence, never escape leakage, panic, unbounded allocation, or +child leak". That covers roughly twenty deferred items and none were +spot-checked. It remains an unproven claim rather than a known gap. + ## 10. Gates and bite verification Every PR runs the standing full gates from `AGENTS.md`, sequentially, plus its