Merge pull request #244 from levineuwirth/parse-budget-diagnosability

Parse budgets report the measurement that failed
This commit is contained in:
Levi Neuwirth 2026-08-30 00:04:17 +02:00 committed by GitHub
commit a85205a9f6
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
5 changed files with 348 additions and 3 deletions

View File

@ -270,6 +270,91 @@ hazard in a shape that looks committed. **A documented error message
that never appears is worse than no documentation**, because the reader
waits for a signal that is not coming.
## Parse-budget diagnosability — ACTIVE
**Written with the branch's FIRST commit**, per the standing correction
from #171 and #215.
- **Branch `parse-budget-diagnosability`**, base `githubsucks/main` @
**`3557779`** exactly, in worktree
`/home/jeans/Repos/personal/pmacs-parse-budget`.
**`githubsucks/parse-budget-diagnosability` is the authoritative
tip** (the ref, not a SHA). Recover with `git fetch githubsucks &&
git checkout parse-budget-diagnosability`.
- **Framing `docs/parse-budget-diagnosability-framing.md`, revision 2 —
APPROVED.** Revision 1 was reviewed and had two defects worth
keeping: it claimed `dispatch_parse_round_trips_a_rust_source_file`
was the codebase's **sole** measurement-omitting assertion (false —
`tests/m4_acceptance.rs:244` is the same budget on the same
measurement), and it bundled `workflow_dispatch`, which this ledger
had already recorded as its own lane.
- **What it does:** both `duration_ms < 100` assertions now report the
observed value and the budget. **Neither budget moves.**
- **Why:** `dispatch_parse_round_trips_a_rust_source_file` redded twice
on macOS/`lua54` — U11, then again on #243 — and **both margins were
unrecoverable**, so the second red could not be compared with the
first. A 1ms overshoot and a 900ms overshoot are different failures
and produced identical logs.
- **D1/D2 verified against REAL PANIC MESSAGES**, by forcing only the
comparison bound to `0` in a scratch build:
- `trivial parse should be fast: took 0ms against a 100ms budget`
- `200-line parse should be quick: took 11ms against a 100ms budget`
The second is the stronger demonstration: a non-zero observed value
cannot be mistaken for a literal.
- **THE SCRATCH PANIC PROVES ONLY HALF.** It exercises a budget of
**0** while printing `100ms`, so it says nothing about the committed
threshold. **D3 carries that half separately** by pinning the literal
`100` in both files. The two are a proof together; neither
substitutes for the other.
- **NOT an assertion-hygiene audit.** The framing's §3 withdraws
revision 1's completeness claim rather than repairing it: a sweep
wide enough to be complete also catches `Instant::now() < deadline`
loop guards and `eval::<bool>` turbofish, and a sweep narrow enough
to be accurate proves nothing about completeness.
- **GATE RUN 1 IS NOT A RESULT.** It ran in a background task that was
killed at 314s; `07-sweep.log` ends in `Terminated`. Stages 1–5 and 8
were green and stage 6 (`gpu`) genuinely failed — that one completed
and reported — but the run as a whole proves nothing and must not be
read as a gate outcome.
- **Those `gpu` failures are R7's SIXTH and SEVENTH occurrences**, in
`docs/ci-red-signatures.md`. All three required fragments, isolated
selector green three times, and **this lane touches no `pmacs-gpu`
file at all** — its whole diff is two `assert!` message strings and
three docs.
- **GATE RUN 2: 7/8, `gpu` red on R7 again** (head `45d438c`, log
`20260829T150011Z-429115`). `sweep` ran fully green this time.
**Two consecutive in-gate R7 failures prompted a narrowing**: 17 green
runs outside the gate, in four configurations, never reproduced it.
**A third in-gate run was then GREEN**, so "in-gate always fails" is
false and the registry's first write-up — which called those runs
*exclusions* — is corrected there: nothing outside the gate has ever
reproduced this, so matching one gate condition at a time outside it
cannot isolate an in-gate cause. In-gate is 2 failures in 4;
out-of-gate is 0 in 17, over a **bounded observation window** the
registry defines so the ratio cannot drift with review activity.
Recorded there; **not this lane's to solve**.
- **PR #244** — https://github.com/levineuwirth/pmacs/pull/244.
- **Every review round ends with a head-exact 8-stage gate**, green
each time, with `HEAD` and a clean worktree captured before and after
and the result read from the eight stage logs rather than inferred
from stage exits. **The PR body carries the current head and log
id**; they are not duplicated here, because a docs commit answering a
review moves both, and a ledger line naming them is stale the moment
it is written — the lesson §5b learned twice.
- **Review follow-up found U13, separately recorded.** On signed head
`756c2b8`, gate `20260829T171606Z-1087848` was 7/8: `07-sweep`
reddened only
`gate_script_acceptance::skipped_directories_are_reported_with_a_reason`.
The child command's stdout was empty, but the row discards its stderr
and status, so the mechanism is unrecoverable. The exact selector and
the full 36-test binary both passed immediately afterwards —
intermittence only. This lane changes neither the gate script nor
that acceptance binary; diagnostic hardening is a separate lane.
- **Still owed, separately:** `workflow_dispatch` on `ci.yml`, and U9's
discriminating control — pin test-binary concurrency to 1, then load
a lone `--lib` binary — which has been named since 2026-08-09 and
never run.
## Panel-pointer replay (parent acceptance 48) — MERGED as #243 (`6c9bae6`)
- **MERGED 2026-08-29T10:37:10Z** at approved head `b8c51b7`, merge

View File

@ -496,12 +496,97 @@ Stage 4; the lane touches no `pmacs-gpu` code at all.
| **selector** | `-p pmacs-gpu attach::tests::managed_retry_survives_transients_and_uses_the_successful_stream` |
| **job / flavor** | local (Linux), `cargo test --workspace --features crdt --no-fail-fast`, i.e. under full-sweep load |
| **required fragments** | `transient sequence must attach` + `Handshake(Io(` + `BrokenPipe` (or `code: 32`) |
| **status** | **FIFTH OCCURRENCE 2026-08-15 — causal status still UNRESOLVED.** The fifth carries the strongest tree exclusion this row has had: a **documentation-only diff** |
| **status** | **SEVENTH OCCURRENCE 2026-08-29 — causal status still UNRESOLVED.** The sixth and seventh came back to back on one lane and are written up together below; the fifth carries the strongest tree exclusion this row has had, a **documentation-only diff** |
| **what IS established** | **three** occurrences at `pmacs-gpu/src/attach.rs:1680`, the second and third with all three fragments **verified** rather than inferred; the test drives a scripted transient-then-success sequence over a real socket pair. **The added GPU test is not the mechanism** — see the third-occurrence control below |
| **what is NOT** | whether the broken pipe is the *fixture's* writer closing early or a real retry-path defect. **This row is not a claim that it is harmless** |
| **rerun evidence** | occurrence 1: 6 isolated runs green, plus a full `--workspace --features crdt` sweep green (113 targets). Occurrence 2: **30 green on the observing branch** (15 isolated selector, 15 full `-p pmacs-gpu`) **plus a 15-run merge-base control, also green**. Occurrence 3: 5 isolated selector runs green, 10 full `-p pmacs-gpu` runs green **with** the added test, and **1 failure in 10 with the added test `#[ignore]`d** — the first rerun in this row's history that reproduced anything. Per the rerun rule the green runs establish intermittence only; the red control run is what carries the exclusion |
| **retirement** | hardening that removes the named mechanism plus a discriminating witness — or a diagnosis showing the fixture, not the code, closes the pipe |
**Sixth occurrence — the parse-budget diagnosability lane, 2026-08-29,
local (Linux), `gpu` step.** All three required fragments present in the
durable log
(`pmacs-parse-budget-9c27ecfe/gate-logs/20260829T144541Z-350549/06-gpu.log`):
```
transient sequence must attach: Attach(Handshake(Io(Os { code: 32,
kind: BrokenPipe, message: "Broken pipe" })))
```
at `pmacs-gpu/src/attach.rs:1889`, `283 passed; 1 failed`.
* **The tree exclusion is as strong as the fifth's.** The observing
lane's entire diff is `src/async_runtime.rs`,
`tests/m4_acceptance.rs` and three docs — **no `pmacs-gpu` file, and no
file `pmacs-gpu` links against beyond the workspace it always did.**
The change is two `assert!` message strings.
* **Rerun: isolated selector green three times** (`1 passed`, 0.01 s
each), which is this row's established control shape.
* **THE SAME RUN'S `sweep` STAGE WAS TRUNCATED, and is not evidence.**
The gate was running in a background task that was killed at 314s;
`07-sweep.log` ends in `Terminated`. That stage's absence says
nothing, and the run as a whole is **not** a gate result. Only the
`gpu` stage's failure is, because it completed and reported.
**A SEVENTH OCCURRENCE FOLLOWED IMMEDIATELY**, on the next gate run of
the same worktree at head `45d438c`
(`20260829T150011Z-429115/06-gpu.log`), same selector, same three
fragments, `283 passed; 1 failed`. **That run's other seven stages were
green**, `sweep` included and complete — 121 result lines, none with a
failure — so this pair is not confounded by a truncation the way the
sixth was.
**Two consecutive in-gate failures is new for this row**, whose prior
five were spread across lanes and months. It prompted a narrowing, and
the narrowing is the useful part.
**A THIRD IN-GATE RUN WAS GREEN** (head `68a16f9`, log
`20260829T152024Z-563254`, all eight stages, zero failures anywhere).
So "in-gate always fails" is **false**, and the paragraph below was
written before that run and is corrected rather than deleted.
**Seventeen green runs outside the gate, in four configurations, all at
head `45d438c` on the failing worktree:**
| condition reproduced outside the gate | runs | result |
|---|---|---|
| isolated selector | 3 | green |
| full `-p pmacs-gpu` binary | 6 | green |
| full binary under a gate-shaped 61-character `TMPDIR` (tested because this project already knows socket-path length matters) | 6 | green |
| `m4` then `gpu` back to back, as the gate orders them | 2 pairs | green |
**THESE ARE NOT EXCLUSIONS, and an earlier version of this entry called
them that.** The reasoning was wrong: **nothing outside the gate has
ever reproduced this failure**, in 17 runs across four configurations —
so matching one gate condition at a time *outside* the gate cannot
isolate an in-gate cause. All these runs establish is that none of the
four conditions **by itself** reproduces the failure. They do not show
that any of them is uninvolved when the gate supplies the rest.
**THE OBSERVATION WINDOW IS BOUNDED, deliberately.** It is the first
four in-gate runs of 2026-08-29 —
`20260829T144541Z`, `T150011Z`, `T152024Z`, `T152824Z` — plus the 17
out-of-gate runs taken between them. **Later head-exact verification
gates on this lane are NOT part of it and do not move these numbers.**
Without that boundary the tally re-counts itself every time a review
round adds a docs commit and the gate is re-run, which is a ratio that
drifts with review activity rather than with the phenomenon.
**What the window supports**, at the strength it carries: in-gate
**2 failures in 4**; out-of-gate **0 failures in 17**. That asymmetry
is suggestive and it is not a clean split,
because the third in-gate run passed.
**The method for the next occurrence follows from that.** Varying
conditions outside the gate cannot answer this question. It has to be
varied INSIDE — the gate's ambient root, its exported environment, and
process state carried across stage boundaries are the uneliminated
candidates, and each would need a gate run with that one thing changed.
**Causal status: still UNRESOLVED.** What these occurrences add is a
sharper question and a method, not a cause: previous entries compared
lanes and trees, and these locate the asymmetry in the *runner* while
showing that the obvious way to probe it — reproducing gate conditions
outside the gate — cannot work.
**Fifth occurrence — panel cell-mapping generation (§5b) framing,
2026-08-15, local (Linux).** The `scripts/gate` **`gpu` step** again,
the same flavor as occurrence 2, inside a `--protocol` run
@ -991,3 +1076,20 @@ discriminating control U9 named is STILL UNRUN**: pin test-binary
concurrency to 1, and separately load a lone `--lib` binary. Four
incidents is enough evidence that the family will keep costing review
rounds until someone runs it.
### U13 — gate prune-reporting row receives empty child stdout in `sweep`
Recorded during PR #244 review, 2026-08-29, on signed head `756c2b8`.
The first six gate stages were green; `07-sweep` produced this one red;
`08-diff-check` remained green.
| field | value |
| --- | --- |
| **selector** | `--test gate_script_acceptance skipped_directories_are_reported_with_a_reason` |
| **job / step** | local (Linux), `scripts/gate` step `07-sweep`, log `20260829T171606Z-1087848` |
| **required fragments** | `skipped_directories_are_reported_with_a_reason` + `the unmarked directory must be named with its reason; output was:` followed by empty output |
| **status** | **one occurrence; exact selector and the full 36-test binary both green on immediate rerun** |
| **what IS established** | the test observed empty stdout from its `scripts/gate --prune` child even though its fixture had created an unmarked lookalike which a successful prune must name. The row discards the child's stderr and success status (`let (out, _, _)`), so the durable failure cannot distinguish wrong successful output from a refused or failed child invocation |
| **what is NOT** | any mechanism. Another row in the same test binary mutates shared git-worktree metadata, but concurrent execution is only a candidate; this occurrence did not capture the child status or stderr needed to support it |
| **why this branch is excluded** | the lane changes two assertion-message strings and three docs. `scripts/gate` and `tests/gate_script_acceptance.rs` are byte-identical to the base; the `async_runtime` edit is inside its unit-test module, and the other Rust edit belongs to a different integration-test binary |
| **next occurrence** | preserve the child success status and stderr in the assertion before rerunning. A green rerun establishes intermittence only; it cannot reconstruct why stdout was empty |

View File

@ -0,0 +1,150 @@
# Parse-budget diagnosability — reporting the measurement that failed
**Status: revision 2 — APPROVED 2026-08-29, IMPLEMENTED, PR #244.**
Revision 2 answers review of 1. Three of its four changes are
corrections, and one is a scope reduction:
- **SPLIT.** Revision 1 bundled this with adding `workflow_dispatch` to
`ci.yml`. `docs/active-work.md` already records those as "two
follow-ups … both their own lanes", and a shared thesis is not an
atomic feature boundary. **`workflow_dispatch` is no longer in this
framing**; it gets its own.
- **The census was WRONG.** Revision 1 called
`dispatch_parse_round_trips_a_rust_source_file` the sole outlier.
`tests/m4_acceptance.rs:244` asserts the same measurement against the
same budget and also omits it. **Both are now in scope.**
- **The completeness claim is WITHDRAWN**, not repaired. See §3.
- The concurrency reasoning revision 1 got backwards belonged to the
other change and leaves with it.
## 1. What this fixes
Two assertions compare a parse duration against a 100ms budget and
report **nothing** about what they measured:
| where | assertion | message |
|---|---|---|
| `src/async_runtime.rs:3328` | `duration_ms < 100` | `"trivial parse should be fast"` |
| `tests/m4_acceptance.rs:244` | `duration_ms < 100` | `"200-line parse should be quick"` |
When either reds, the log says the parse was slow and stops there.
## 2. Why now, and the evidence
The first has redded **twice** on `Test (macos-latest / lua54)`, and
**both margins are unrecoverable**:
| occurrence | where | outcome |
|---|---|---|
| U11 | PR #242, run `32393462318`, job `96504773333` | red, green on rerun |
| U11's recurrence | PR #243, run `33156571314`, job `98800645872` | red, green on rerun |
`src/async_runtime.rs` was **byte-identical to `main`** for both — blob
`9310ce3fca8c5fd8ebd39a68c29ad6985e256049`. U11's own row predicted the
cost:
> *A recurrence is not another instance of this row. Because the margin
> was never captured, a second red cannot be compared with the first.*
That came true once. Nothing prevents a third.
**A 1ms overshoot and a 900ms overshoot are different failures** — one
says a threshold is marginal, the other says something stalled — and
today they produce identical logs.
## 3. Scope, and a claim this framing does NOT make
**In scope: the two assertions in §1.** They are the same measurement
against the same budget, in the same subsystem, and it would be strange
to fix one and leave the other to produce the next unreadable red.
**Out of scope, and deliberately: everything else.** Revision 1 claimed
these were the only measurement-omitting assertions in the codebase.
**That claim is withdrawn and is not replaced by a corrected one.** A
sweep wide enough to be complete also catches `Instant::now() <
deadline` loop guards and `eval::<bool>` turbofish, which are not
budget assertions at all; a sweep narrow enough to be accurate proves
nothing about completeness. **This lane is not an assertion-hygiene
audit and should not be read as one.**
Several nearby budget assertions do already report their measurements —
`composition_overhead_under_ten_percent`, `criterion_1_end_of_line_typing…`
(`optimistic.rs:989`), `dired_open_renders_10k_entries_under_200ms`,
`m6_2_pty_streaming_respects_byte_ceiling` (`observed {in_flight}`).
They are cited as **precedent for the shape**, not as evidence that the
set is exhausted.
## 4. What lands
```rust
assert!(
duration_ms < 100,
"trivial parse should be fast: took {duration_ms}ms against a \
100ms budget"
);
```
and, in `tests/m4_acceptance.rs`:
```rust
assert!(
duration_ms < 100,
"200-line parse should be quick: took {duration_ms}ms against a \
100ms budget"
);
```
**Both budgets stay at 100ms.** Widening is what R1 already rejected,
and it would discard the signal these reds carry.
## 5. Acceptance
| # | contract | witness | mutation |
|---|---|---|---|
| D1 | the `async_runtime` assertion reports the observed ms **and** the budget | a scratch build with the comparison bound forced to `0` panics with a message containing the observed value and `100ms` | restore the bare message → the row cannot separate a 1ms overshoot from a 900ms one |
| D2 | the `m4_acceptance` assertion does the same | same method, same row | same |
| D3 | both budgets are still `100` | the comparison literal is unchanged in both files | widen either → R1's rejected remedy returns |
**D1 and D2 are asserted against a real panic message, not by reading
the source.** A row that greps the format string would pass while the
assertion it describes had been deleted — which is the same
read-the-code-not-the-effect failure this project has repeatedly
caught.
**How the fault is injected, and what each half proves.** Temporarily
force **only the comparison bound** to `0` in a scratch build. The
resulting panic then proves two things and no more: that the observed
value is interpolated, and that the budget text reads `100ms`.
**It does NOT prove the exercised budget was 100ms — it was 0.** The
scratch build compares against `0` while the message still says `100ms`,
so that panic alone says nothing about the committed threshold. **D3
carries that half separately**, by pinning the comparison literal in
both files. The two rows are only a proof together, and neither
substitutes for the other.
This is the **smallest deterministic fault injection** available, not
the only conceivable one. A row that waited for a genuine 100ms
overshoot would be exactly as intermittent as the thing it documents.
## 6. Coherence impact (`COHERENCE.md` §20)
- **Journey steps touched: NONE.** No product behaviour changes. What
moves is **evidence quality** — whether a failing check can be
reasoned about — the same axis the `scripts/gate` SIGINT guard sat
on.
- **Interaction islands: none added.**
- **Config registry: no entry.** A test budget is not a user-tunable.
- **Background work: none started**, and no attribution moves.
## 7. What this does NOT do
- **It does not fix the intermittence**, and makes no claim about
cause. It makes the next occurrence *comparable*.
- **It does not widen, relax or `#[ignore]` any budget.** Two
intermittent reds are not evidence a threshold is wrong.
- **It does not add `workflow_dispatch`.** That is its own lane, per
the ledger's recorded decision.
- **It does not run U9's discriminating control**, which remains
unrun and is a third, separate piece of work.

View File

@ -3325,7 +3325,11 @@ mod tests {
.expect("parse handoff must hold a bundle on Complete");
match rt.take_result(id) {
Some(JobOutcome::Complete(JobResult::Parse { duration_ms })) => {
assert!(duration_ms < 100, "trivial parse should be fast");
assert!(
duration_ms < 100,
"trivial parse should be fast: took {duration_ms}ms \
against a 100ms budget"
);
}
other => panic!("unexpected outcome: {other:?}"),
}

View File

@ -241,7 +241,11 @@ fn m4_1_dispatch_parse_round_trips_via_runtime() {
.expect("parse handoff holds bundle on Complete");
match rt.take_result(id) {
Some(JobOutcome::Complete(JobResult::Parse { duration_ms })) => {
assert!(duration_ms < 100, "200-line parse should be quick");
assert!(
duration_ms < 100,
"200-line parse should be quick: took {duration_ms}ms \
against a 100ms budget"
);
}
other => panic!("unexpected outcome: {other:?}"),
}