LeVCS/crates
Levi Neuwirth 5cfa95f179 Implement the partial B4 crash-matrix and benchmark slice
Scope 6.6 deliverables 1-3, partial. This is evidence, not a freeze: no
Wave B deliverable set is complete and the throughput figure below is not
a publishable bundle.

The revert-and-observe-red acceptance for cd37f8b's two generators was
never actually performed -- the tests asserted it in their doc comments,
which is a claim. It is performed now, in a throwaway copy, one mutation
at a time, and each reddens exactly one test for the right reason. With
segment frame validation reverted to pushing footer offsets straight into
the adopted set, the store still opens and recovery still completes: it
reports recovery_ok=true and adopts the corrupted frame as authority, so
the red is exit 0 where 65 is required rather than a store that failed to
open. With the shard-index check removed from journal binding, the moved
journal is adopted whole and root_uuid alone does not catch it -- the Wave
A blocker reproduced.

The eight Wave B failpoint rows now drive StoreEngine::submit in-process
with the full six-field expectation asserted against the frozen oracle,
field by field rather than by struct comparison. No pending-wave-b row
remains. The four fields Wave A could not reach come from four independent
observations: the store naming itself poisoned, the two-root status read,
a receipt obtainable at all, and a different transaction submitted to the
same shard before any reopen. The last two are not the same question -- a
shard can be unpoisoned and still refuse a later append because its writer
thread died, which is exactly what phase-aware panic ownership fixed and
what this now checks from outside. AfterRootCasBeforeWaiterWake is
observed as a genuinely hung submit whose receipt is still retrievable,
so a waiter that never wakes is proved to be a hung request rather than an
absent transaction. Every row also asserts its group's fence count against
the public durability snapshot.

Two flake campaigns were measured rather than rerun: 4 failures in 40, then
7 in 40, from two distinct causes. One is a finding -- publishing a group
adds an index delta layer and none are sealed, so submit refuses after
exactly max_index_runs publications for the life of an engine. Both fixed
structurally; 200/200 and 40/40 after.

store-bench emit-skeleton now defaults to the submit path, with the journal
seam retained under --path drive for comparison. The signer is real, the
ref CAS is evaluated by the sequencer, and objects_new is summed from the
store's own receipts rather than multiplied out of the transaction count.

Explicit blockers, retained rather than worked around:

  - StoreEngine::open still refuses startup state 1, so the benchmark seeds
    its root by a non-production path. Seeding a store off the production
    path in order to measure the production path is the charter item 8
    smell; the disclosure is recorded in the fixture, a const doc, and the
    module docs, and a test asserts open still refuses so it cannot go
    stale in the safe direction.
  - P2 is blocked three ways -- checkpointing disabled, no steady state
    under the index-run ceiling, and no warmup/repetition/trim protocol.
    The rate emitted is a debug build on tmpfs, marked preliminary.
  - The 100 SIGKILL cycles still drive the journal seam, so kill -9 never
    lands inside a real publication.
  - Four schema claims became earnable and are requested, not emitted;
    bench/result-schema.json is lead-owned.
  - Reopen after close needs a bounded, measured, reported wait, because
    the root LOCK outlives StoreEngine::drop. Diagnosed since as fork/exec
    inheritance of the lock file description; the fix belongs in the lock
    primitive, and this wait is removed when it lands.

scripts/check-phase1.sh GATE_EXIT=0; verify-store-recovery.sh 100 cycles,
recovery_failures=0, acknowledged_loss=0, torn_transactions=0,
repeated_adoptions=0, bundle=schema-valid.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 20:55:02 -04:00
..
levcs-cli Robustness fixes 2026-05-03 20:56:46 -04:00
levcs-client integrate CI and docs 2026-05-01 11:29:18 -04:00
levcs-core integrate CI and docs 2026-05-01 11:29:18 -04:00
levcs-identity Freeze Wave A: Phase 1 storage spine 2026-07-26 19:47:03 -04:00
levcs-instance Robustness fixes 2026-05-03 20:56:46 -04:00
levcs-merge integrate CI and docs 2026-05-01 11:29:18 -04:00
levcs-protocol Amend Wave B storage contracts after adversarial review 2026-07-28 18:01:05 -04:00
levcs-store Implement the partial B4 crash-matrix and benchmark slice 2026-07-28 20:55:02 -04:00
levcs-tui integrate CI and docs 2026-05-01 11:29:18 -04:00