diff --git a/docs/active-work.md b/docs/active-work.md index 93b07d8..d77db78 100644 --- a/docs/active-work.md +++ b/docs/active-work.md @@ -295,20 +295,36 @@ from #171 and #215. - **`panel-mapping-generation` (§5b) is HELD BEHIND THIS LANE** by explicit instruction. That lane is code-complete at `5174f73` with its own fifteen stages green; its sixteenth stage is this defect. -- **Reproduction is 5/5 in the full sweep and 0/N in every reduction.** - All 37 preceding targets plus the suite run green, and other - packages' targets run after the failure. That paradox is the shape of - the problem, not an aside. +- **Reproduction is 5/5 in the full sweep; every reduction R1–R10 is + green.** Each run is enumerated with exact command, worktree, HEAD, + cleanliness and log digest in `docs/probe-sigint-evidence.md` — "0/N" + is not a record. +- **But R9 did NOT run the same binaries as the sweep.** It executed + `…-91f51d0b…` / `…-6b4b8223…`; the sweeps executed `…-5d9105cb…` / + `…-d4dae4f0…`, and those artifacts are byte-different. Command shape + changes Cargo's fingerprint. R9 establishes **same target names and + order**, not same binaries. What the evidence supports is an + **interaction**: prior targets alone (R9) green, workspace artifacts + alone (R10) green, both together (F1–F5) red. `--workspace` selection + is **not sufficient by itself and not ruled out** — later-selected + packages can affect the build graph before their tests ever run, so + "their targets execute after the failure" does not exonerate them. - **Ruled out by measurement — do not re-run:** machine load; tmpfs starving RAM (settled by experiment, not argument — `/tmp` 21G→1.2G, available 27G→45G, still red); leaked daemons; inotify; `--workspace` feature unification; any specific preceding test. -- **Ground truth that DEEPENS rather than explains it.** Neither binary - handles signals: `run_gpu` (`src/main.rs:324`) blocks in - `command.status()` with no handler, and grepping the whole of - `pmacs-gpu/src` for signal machinery returns nothing. The probe polls - at 50ms (`pmacs-gpu/src/main.rs:1065`). Two processes with default - `SIGINT` disposition should both die at once. +- **Ground truth, and what it does NOT establish.** Neither binary + contains signal-handling code: `run_gpu` (`src/main.rs:324`) blocks + in `command.status()` with no handler, and grepping all of + `pmacs-gpu/src` for signal machinery returns nothing. The probe's + **event loop** wakes at least every 50ms + (`pmacs-gpu/src/main.rs:1065`) — but the process is **not** bounded: + its stdin reader blocks in `read_to_end` (`:1109`) and, once ready, + the loop leaves only when stdin closes (`:1212`). + **No claim is made that either process holds the DEFAULT + disposition** — absence of handler code cannot establish that, and + inherited ignore is the leading hypothesis precisely because the + source is silent. - **TWO retracted claims, both mine.** (a) "Mechanism located" — launcher in `do_wait`, probe child in `futex_do_wait`. (b) The retraction of (a), which argued the failing launcher "must live ≥8s". @@ -319,24 +335,37 @@ from #171 and #215. correct. (a) is therefore not refuted by (b); it stays **unproven for a different reason** — the suite spawns launchers from five call sites, so command line alone cannot attribute one to this test. - **Do not key on process age. Key on the PID the test records.** + Under `--features crdt` that is **six** sites (`:509, :534, :544, + :574, :725, :1097`, all inside `#[cfg(feature = "crdt")] mod crdt`); + the other two `--gpu` arguments sit under `#[cfg(not(…))]` and are + compiled out. **Do not key on process age. Key on the PID the test + records.** - **Diagnostics must DISCRIMINATE** blocked delivery, inherited ignore, and an escaped process group: snapshots **before and after** the signal, for test parent / launcher / probe; **per-thread** `SigBlk` from `/proc//task/*/status`; `SigPnd`/`ShdPnd`; and `PID`/`PPID`/`PGID`/`SID`. A post-failure snapshot cannot prove inheritance. -- **"Neither binary handles signals" does NOT mean default - disposition.** `SIG_IGN` is inherited across `fork` and survives - `exec`, so an inherited non-default disposition is the leading - hypothesis precisely because the source is silent. Revision 1's - "two processes with default disposition" claim contradicted its own - hypothesis and is withdrawn. -- **Reduction evidence is enumerated** in the framing §4 with command, - run count and log for each of R1–R10 and F1–F5, with the logs - preserved off the tmpfs at - `/home/jeans/build/pmacs-gate-targets/probe-sigint-evidence/` — - `/tmp` is a tmpfs and these were nearly lost to a cleanup mid-lane. +- **Why that matters:** `SIG_IGN` is inherited across `fork` and + survives `exec`, while handlers do not. So a runtime disposition can + arrive from the test harness, `cargo`, or the invoking shell without + appearing anywhere in the source. Revision 1's "two processes with + default disposition" contradicted its own hypothesis and is + withdrawn; the assertion no longer appears above it either. +- **Run provenance is a pushed document**, `docs/probe-sigint-evidence.md`: + exact command, worktree, HEAD, cleanliness, artifact family, result + and log digest per physical run. Three caveats stated there rather + than smoothed over — **R1 and R2 have no preserved log** (revision 2 + double-counted one log as both R2 and R6), **cleanliness is UNKNOWN** + for every pre-manifest run, and **R1–R10 ran in the + `panel-mapping-generation` worktree**, not at `main`. Log bodies are + machine-local under + `/home/jeans/build/pmacs-gate-targets/probe-sigint-evidence/`; `/tmp` + is a tmpfs and they were nearly lost to a cleanup mid-lane. +- **D0 precedes every other diagnostic:** re-run the matrix at `main` + under a harness that captures provenance **and the artifact hashes + actually executed**, since command shape silently changed the binary + once already. - **Coherence: journey step 12(a), "closing is clean", IS touched** — Ctrl-C teardown of a GPU session is that step, grade movement or not. Revision 1 claimed no journey step, reasoning from grade movement, diff --git a/docs/gpu-probe-sigint-framing.md b/docs/gpu-probe-sigint-framing.md index 9bc587f..d9b0614 100644 --- a/docs/gpu-probe-sigint-framing.md +++ b/docs/gpu-probe-sigint-framing.md @@ -1,11 +1,19 @@ # GPU launcher / probe SIGINT teardown — framing -Revision 2. Status: **awaiting approval. No implementation.** +Revision 3. Status: **awaiting approval. No implementation.** -Revision 1 was rejected on five findings. Each is answered below, and -the two that changed the technical picture — the lifetime arithmetic -(§5) and the disposition claim (§3) — are recorded as corrections -rather than quietly rewritten. +Revisions 1 and 2 were each rejected on five findings. Every correction +is recorded in place rather than quietly rewritten, because three of +them were claims this document itself had advanced: + +- r1 → r2: the ">6 s selector" and the "≥8 s lifetime" arithmetic + (§5); "two processes with default disposition" (§3); "119 binaries + green, one red" (§1); an unobtainable A2 (§8); "journey steps + touched: none" (§9). +- r2 → r3: **"R9 ran the same binaries" — it did not** (§4); "the probe + never blocks indefinitely" (§3); the launcher call-site count (§5); + reduction provenance, now in `docs/probe-sigint-evidence.md`; and + ledger corrections that had not been made portable (§11). ## 1. The problem, stated as what is observed @@ -52,9 +60,14 @@ stages green and is held behind this lane by explicit instruction. `waitpid` — with no handler installed. Grepping `SIGINT|signal_hook|sigaction|ctrlc|set_handler|pthread_sigmask|sigprocmask` across `pmacs-gpu/src` returns nothing. -- **The probe never blocks indefinitely.** +- **The probe's event loop wakes at least every 50 ms.** `run_headless_managed_probe` (`pmacs-gpu/src/main.rs:1065`) loops on - `event_rx.recv_timeout(Duration::from_millis(50))`. + `event_rx.recv_timeout(Duration::from_millis(50))`. **Revision 2 said + "never blocks indefinitely", which is false**: the probe's stdin + reader thread blocks in `read_to_end` (`:1109`) with no timeout, and + once `ready` the loop has **no deadline of its own** — it leaves only + when stdin closes (`:1212`). So the process is not bounded; only the + event wakeup is. - **The daemon *does* handle signals, deliberately.** `src/daemon.rs:629-641` registers `SIGTERM`/`SIGINT` via `signal_hook::flag`. The daemon is the process the test asserts must @@ -73,31 +86,67 @@ establishes is narrower: **neither binary sets a disposition itself**, so whatever disposition they hold at runtime was inherited, and that is measurable rather than arguable. -## 4. Reductions attempted — each with command, count, and log +## 4. Reductions attempted -Preserved off the tmpfs at -`/home/jeans/build/pmacs-gate-targets/probe-sigint-evidence/`, because -`/tmp` is a tmpfs and these were nearly lost to a cleanup mid-lane. +**Full provenance lives in `docs/probe-sigint-evidence.md`**, which is +pushed with this branch: exact command, worktree, HEAD, cleanliness, +the artifact family actually executed, result, and log digest for every +physical run. Log bodies stay machine-local under +`/home/jeans/build/pmacs-gate-targets/probe-sigint-evidence/` — `/tmp` +is a tmpfs and they were nearly lost to a cleanup mid-lane. -| # | reduction | runs | result | log | +Three provenance caveats are recorded there rather than smoothed over: +**R1 and R2 have no preserved log** (revision 2 cited `gpu3.log` for +both R2 and R6; that log is R6's three-suite run alone, and counting +one run as two was wrong); **cleanliness is `UNKNOWN` for every +pre-manifest run**, because it was not recorded at the time and is not +inferrable; and **R1–R10 ran in the `panel-mapping-generation` +worktree**, not at `main`. `D0` re-runs the matrix under a harness that +captures all of it, at `main`, before any row here is relied on. + +All rows carry `--features crdt`. Full argv, worktree, HEAD, +cleanliness and artifact family per run: `docs/probe-sigint-evidence.md`. + +| # | reduction (after `cargo test`) | runs | result | log | |---|---|---|---|---| -| R1 | `cargo test --features crdt --test gpu_invocation_acceptance ctrl_c_on_launcher_group` | 3 | green, 0.15–0.17 s | *(console; superseded by R2)* | -| R2 | `cargo test --features crdt --test gpu_invocation_acceptance` (whole suite) | 1 | green, 15 passed | `gpu3.log` | -| R3 | `cargo test --workspace --features crdt --no-fail-fast -- --skip basedpyright ctrl_c_on_launcher_group` | 1 | green — every binary runs, only this test executes | `filtered.log` | -| R4 | `cargo test --features crdt --lib --test gpu_invocation_acceptance --no-fail-fast` | 1 | green, 2145 + 15 | `two.log` | +| R1 | `--test gpu_invocation_acceptance ctrl_c_on_launcher_group` | 3 | green, 0.15–0.17 s | **no log preserved** | +| R2 | `--test gpu_invocation_acceptance` (whole suite) | 1 | green, 15 passed | **no log preserved** | +| R3 | `--workspace --no-fail-fast -- --skip basedpyright ctrl_c_on_launcher_group` | 1 | green — every binary runs, only this test executes | `filtered.log` | +| R4 | `--lib --test gpu_invocation_acceptance --no-fail-fast` | 1 | green, 2145 + 15 | `two.log` | | R5 | `--test gate_script_acceptance --test gpu_invocation_acceptance` | 1 | green | `suspect.log` | -| R6 | three GPU suites in sweep order (`gpu_font`, `gpu_initial_target`, `gpu_invocation`) | 1 | green | `gpu3.log` | -| R7 | targets 1–19 (incl. `--lib --bins`) + the suite | 1 | green | `half1.log` | -| R8 | targets 20–37 + the suite | 1 | green | `half2.log` | -| R9 | **all 37 preceding targets** + the suite | 1 | green | `prefix.log` | -| R10 | `--workspace` with only `gpu_initial_target` + `gpu_invocation` | 1 | green | `wsonly.log` | -| F1–F5 | full `cargo test --workspace --features crdt --no-fail-fast -- --skip basedpyright` | 5 | **red, 5/5** | `base-sweep.log` (at `72da24a`), `postclean.log`, `sweep-inst.log`, `sweep-diag.log`, gate `…-2144707` | +| R6 | `--test gpu_font_acceptance --test gpu_initial_target_acceptance --test gpu_invocation_acceptance` | 1 | green | `gpu3.log` | +| R7 | `--lib --bins` + `--test`×14 (targets 6–19) + the suite | 1 | green | `half1.log` | +| R8 | `--test`×18 (targets 20–37) + the suite | 1 | green | `half2.log` | +| R9 | `--lib --bins` + `--test`×32 (targets 6–37) + the suite | 1 | green (**reduction artifacts**) | `prefix.log` | +| R10 | `--workspace ... --test gpu_initial_target_acceptance --test gpu_invocation_acceptance` | 1 | green (**workspace artifacts**) | `wsonly.log` | +| F1–F5 | full `--workspace --no-fail-fast -- --skip basedpyright` | 5 | **red, 5/5** | `base-sweep.log` (at `72da24a`), `postclean.log`, `sweep-inst.log`, `sweep-diag.log`, gate `…-2144707` | -**R9 is the shape of the problem.** The same binaries, in the same -order, with the same tests before it, pass as a subset and fail as part -of the whole. R10 rules out `--workspace` feature unification; other -packages' targets run at log lines 4848+, after the failure at 3066, so -they cannot be implicated either. +**Correction to revision 2: R9 did not run the same binaries.** It +executed `gpu_initial_target_acceptance-91f51d0b5303ff9f` and +`gpu_invocation_acceptance-6b4b8223dea45247`; the failing sweeps +executed `-5d9105cb7047aab8` and `-d4dae4f01bcdef62`. Those artifacts +are **byte-different** (sha256/16 `36912fa2…` vs `1b3cc86c…`, and +`858d7148…` vs `ede0c07d…`; see `docs/probe-sigint-evidence.md`). +Cargo's target selection changes the fingerprint, so command shape +changes the executable. R9 therefore establishes **same target names +and order**, not same binaries. + +What the evidence actually supports is an **interaction**, and only +that: + +| prior targets execute | workspace artifact family | result | +|---|---|---| +| yes | no | R9 green | +| no | yes | R10 green | +| yes | yes | **F1–F5 red** | + +Neither factor alone reproduces it. So `--workspace` artifact +selection is **not sufficient by itself** — and, importantly, **not +ruled out either**, which is how revision 2 phrased it. Later-selected +packages can influence Cargo's build graph and fingerprints *before* +their test executables ever run, so "their targets execute after the +failure at line 3066" does not exonerate them. The same applies to the +claim that other packages "cannot be implicated": withdrawn. Also refuted, by measurement: machine load (red on a quiet box, load 2.77); tmpfs starving RAM (**tested by experiment** — `/tmp` 21 G → @@ -121,10 +170,15 @@ observed. A ">6 s" selector would therefore have captured **nothing**, repeating the very sampling error it was written to correct. So A is **not** refuted by B. A remains **unproven for a different -reason**: the suite spawns launchers from **five** call sites -(`:38, :65, :509, :534, :544, :574, :725, :1097` — eight `--gpu` -arguments across the file), so a launcher captured by command line -alone cannot be attributed to *this* test. The `do_wait` / +reason**: under `--features crdt` the suite spawns root launchers from +**six** call sites — `:509, :534, :544, :574, :725, :1097`, all inside +`#[cfg(feature = "crdt")] mod crdt` (`:88`). Eight `--gpu` arguments +appear in the file, but `:38` and `:65` sit under +`#[cfg(not(feature = "crdt"))]` (`:26`) and are compiled out of the +failing configuration. Revision 2 said "five" while citing eight, which +was wrong twice over. Six is the number; what matters is that it is +more than one, so a launcher captured by command line alone cannot be +attributed to *this* test. The `do_wait` / `futex_do_wait` pair is consistent with the failing instance and consistent with a healthy sibling, and nothing recorded distinguishes them. @@ -153,6 +207,11 @@ diagnostic only, and it must **discriminate** the three live candidates: blocked delivery, inherited ignore, and an escaped or wrong process group. +- **D0 — re-run the §4 matrix with captured provenance**, at `main`, + recording the artifact hashes actually executed. Revision 2's + strongest claim collapsed because command shape silently changed the + binary; no further reduction should be trusted until each row names + the executable it ran. - **D1 — key on the PID this test records.** The test already owns `launcher.id()`. Capture around its own `kill`, not by scanning for age or command line. @@ -223,3 +282,19 @@ Written now so the fix cannot quietly become "make the test pass". - `gpu_initial_target_acceptance` including the suite as a module. It is why the failure reds twice, and it is a tidiness question, not a correctness one. + +## 11. Record corrections owed to other ledgers + +A correction is not made until it is portable. Two were outstanding +when revision 2 was reviewed, and both are closed by this revision: + +- **This branch's ledger** asserted that two default-disposition + processes "should both die at once" and then withdrew that same claim + further down. The assertion is removed; only the withdrawal and its + reasoning remain. +- **`panel-mapping-generation` @ `16cf3a2`** still carried "119 + binaries green, one red", the ≥8 s arithmetic, the "default action" + claim and the ">6 s selector". Pushing `16cf3a2` made the *retraction* + portable but not the *correction*. That ledger is corrected on its own + branch and pushed, so the held lane no longer transports falsified + claims. diff --git a/docs/probe-sigint-evidence.md b/docs/probe-sigint-evidence.md new file mode 100644 index 0000000..0f33128 --- /dev/null +++ b/docs/probe-sigint-evidence.md @@ -0,0 +1,87 @@ +# GPU probe SIGINT lane — run manifest + +Every physical run behind +`docs/gpu-probe-sigint-framing.md` §4. Pushed so the evidence travels; +the log bodies stay machine-local under +`/home/jeans/build/pmacs-gate-targets/probe-sigint-evidence/` and are +identified here by SHA-256 prefix and byte count. + +## Provenance honesty + +These runs were made **before** this manifest existed, so their +provenance is **reconstructed, not captured**. Specifically: + +- **Commands** are exact — they are the literal invocations issued. +- **Worktree** is exact. +- **HEAD** is given as a range where the run cannot be pinned to one + commit, and marked `~`. It is never guessed at single-commit + precision. +- **Cleanliness** was not recorded at the time and is therefore + `UNKNOWN` for every pre-manifest run. It is not inferred. + +**D0 (below) exists because of this.** No conclusion in §4 should rest +on a `UNKNOWN`-cleanliness row once D0 has replaced it. + +## The artifact-identity column, and why it exists + +Reduction and sweep runs did **not** always execute the same compiled +test executables. Cargo's target selection changes the fingerprint, so +`--test a --test b` and `--workspace` can produce byte-different +binaries for the same source. Verified: + +| family | `gpu_initial_target_acceptance` | `gpu_invocation_acceptance` | +|---|---|---| +| reduction (R7–R9) | `-91f51d0b5303ff9f`, sha `36912fa25a72ffc7` | `-6b4b8223dea45247`, sha `858d71486d0b66f0` | +| workspace (R10, F1–F5) | `-5d9105cb7047aab8`, sha `1b3cc86cbb8d6092` | `-d4dae4f01bcdef62`, sha `ede0c07dd9abb456` | + +They are byte-different. Any claim of the form "the same binaries pass +as a subset" is therefore **unsupported by these runs**. + +## Runs + +All in worktree `pmacs-mapping-gen` unless stated. `WT=mg` is +`/home/jeans/Repos/personal/pmacs-mapping-gen`; `WT=main` is +`/home/jeans/Repos/personal/pmacs` at `72da24a`. All carry +`CARGO_TARGET_DIR=/home/jeans/build/pmacs-gate-targets/pmacs-mapping-gen-8cb089c8` +except `WT=main`, which uses `…/pmacs-fdccc423`. + +| id | exact command (after `cargo`) | WT | HEAD | clean | artifacts | result | log (sha256/16, bytes) | +|---|---|---|---|---|---|---|---| +| R1 | `test --features crdt --test gpu_invocation_acceptance ctrl_c_on_launcher_group` ×3 | mg | ~`724b785`–`5174f73` | UNKNOWN | reduction | green, 0.15–0.17 s | **none preserved** | +| R2 | `test --features crdt --test gpu_invocation_acceptance` | mg | ~`724b785`–`5174f73` | UNKNOWN | reduction | green, 15 passed | **none preserved** | +| R3 | `test --workspace --features crdt --no-fail-fast -- --skip basedpyright ctrl_c_on_launcher_group` | mg | ~`724b785`–`5174f73` | UNKNOWN | workspace | green | `e09a96512035284e` 33113 | +| R4 | `test --features crdt --lib --test gpu_invocation_acceptance --no-fail-fast` | mg | ~`724b785`–`5174f73` | UNKNOWN | reduction | green, 2145 + 15 | `89050c702de22d57` 158812 | +| R5 | `test --features crdt --no-fail-fast --test gate_script_acceptance --test gpu_invocation_acceptance` | mg | ~`5174f73`–`b72843a` | UNKNOWN | reduction | green | `31b3e5249b475479` 3706 | +| R6 | `test --features crdt --no-fail-fast --test gpu_font_acceptance --test gpu_initial_target_acceptance --test gpu_invocation_acceptance` | mg | ~`5174f73`–`b72843a` | UNKNOWN | reduction | green, 11+15+15 | `332693a39c73731a` 4569 | +| R7 | `test --features crdt --no-fail-fast --lib --bins` + `--test` ×14 (targets 6–19) + `--test gpu_invocation_acceptance` | mg | ~`b72843a` | UNKNOWN | reduction | green | `9e1ebc59ed9f0dd4` 187531 | +| R8 | `test --features crdt --no-fail-fast --test` ×18 (targets 20–37) + `--test gpu_invocation_acceptance` | mg | ~`b72843a` | UNKNOWN | reduction | green | `8b26ebfcf5f871b4` 28677 | +| R9 | `test --features crdt --no-fail-fast --lib --bins` + `--test` ×32 (targets 6–37) + `--test gpu_invocation_acceptance` | mg | ~`b72843a` | UNKNOWN | **reduction** | green | `b31d98ee2f427eca` 214566 | +| R10 | `test --workspace --features crdt --no-fail-fast --test gpu_initial_target_acceptance --test gpu_invocation_acceptance -- --skip basedpyright` | mg | ~`b72843a` | UNKNOWN | **workspace** | green | `81b48fd7a0e261dc` 3553 | +| F1 | `build --workspace --no-default-features --features luajit,crdt && test --workspace --features crdt --no-fail-fast -- --skip basedpyright` | **main** | `72da24a` | clean (verified `git status --porcelain` empty) | workspace | **red** | `10b55b8ba8741125` 334446 | +| F2 | `test --workspace --features crdt --no-fail-fast -- --skip basedpyright` | mg | ~`b72843a` | UNKNOWN | workspace | **red** | `474f88f0dad581fe` 338555 | +| F3 | same as F2, with resource sampler | mg | ~`b72843a` | UNKNOWN | workspace | **red** | `7b8519e7300e8bb3` 338555 | +| F4 | same as F2, with process sampler | mg | ~`b72843a` | UNKNOWN | workspace | **red** | `5ccdefc5d89eece3` 338555 | +| F5 | gate stage 15 of `./scripts/gate --protocol --acceptance ×6` | mg | `5174f73` + docs | UNKNOWN | workspace | **red** | gate log `20260817T172537Z-2375685/15-sweep-crdt.log` | + +Supporting, not a reduction: `9b8a01076b44bb7c` 98838 +(`proc-sample.log`) is the process-table sampler output behind the +retracted "mechanism located" claim. + +**R2 and R6 are distinct runs.** Revision 2's §4 cited `gpu3.log` for +both; that log is R6's three-suite run only, and R2's log was never +preserved. R1 likewise has no log. Both are marked accordingly rather +than backfilled. + +## D0 — re-run the matrix with captured provenance + +Before any §4 row is relied on for a conclusion, re-run the reductions +under a harness that records, per run and at the time: exact argv and +environment, worktree, `git rev-parse HEAD`, `git status --porcelain` +emptiness, the artifact hashes actually executed, the result, and the +log digest. Two constraints learned the hard way: + +- run them **at `main`**, not on a feature branch — R1–R10 ran in the + `panel-mapping-generation` worktree, which carries §5b changes; +- record the **artifact hash per run**, since command shape changes it, + which is the whole reason R9's result did not mean what it appeared + to mean.