B4's harness measured intermittent AlreadyLocked on reopening a store it
had just dropped -- up to 29 retries over 147ms, 7 failures in 40 runs,
reproducing on a single-shard engine. StoreEngine::drop was not the cause:
it closes every channel and joins every writer, and looked correct in
every single-process test.
An flock is held by the open file description, not by a descriptor. A
concurrently forked child transiently inherits that description, and
FD_CLOEXEC closes it at exec, not at fork -- so during that window the
parent closing its own descriptor releases nothing. Scope 3.1 makes
AlreadyLocked a refusal and never a wait, so every consumer that closes
and reopens a root -- a recovery drill, an in-place restart, the Phase 2
migrator -- could be refused with no defined retry.
Releasing must therefore be an explicit act. lock_root returns an RAII
RootLock that issues LOCK_UN in Drop before the file closes, and both
owners -- RecoverySession and ShardDrive -- hold that guard. The unlock
lives in sys.rs beside try_lock_exclusive so the locking syscalls stay in
the one funnel. StoreEngine needs no change: it holds a RecoverySession,
never a File, so it inherits the fix.
Drop is the only release path. A release() and a file() accessor were
written and deleted before landing: neither had a caller, and an uncalled
second way to release a lock is exactly the decoy the charter names. A
failed LOCK_UN in Drop cannot be returned and must not be swallowed, so it
increments a counter the regression asserts unchanged.
The rejected alternative was fixing this in StoreEngine::drop alone. That
leaves ShardDrive and the drive's one-shot session exposed and makes
correctness depend on a descriptor lifetime that fork can extend.
The regression is synchronized rather than timed: the child forks while
the lock is held, signals ready on one pipe, and blocks on a second until
after the parent has released and attempted its reopen, so the inherited
descriptor is provably open across the whole window and the reopen is
asserted on its first attempt. With the explicit unlock removed it fails
10/10; as landed it passes 40/40. Measured under load -- 200 close-reopen
cycles against 72,255 concurrent forks -- 0 refusals, worst case 1 attempt
and 10.8ms; the same load kills the pre-fix behaviour within 0.05s, so the
load reproduces the defect rather than merely being weak.
Contract review 2026-07-28-B records the amendment. Two carry-forwards are
recorded in scope 6.6: the verify-store-recovery SIGKILL cycles still drive
the journal seam rather than submit, so kill -9 never lands inside a real
publication and the acknowledged-crash-recovery criterion is only partly
earned; and B4's bounded reopen retry must become a one-attempt assertion
now that the defect it compensates for is gone.
scripts/check-phase1.sh GATE_EXIT=0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The B2 review of the first B1/B3 slice found five defects whose fixes were
not available to the packages that had to make them: each needed a change
to a surface those packages do not own. Rather than let them restate a
frozen fact locally, the lead amends the surfaces and the packages consume
them. Contract review 2026-07-28-A records all five with the weaker
alternative that was rejected for each.
- CommitEvidenceSigner::sign_event's parameter becomes signing_digest.
The message is SignedCommittedTransactionV1::signing_digest, which
binds key epoch and durability result around the canonical event; the
bare event digest is chain identity, not signature material. A doc line
was not enough: misuse is undetectable until after durability, and is
first observed by a mirror on another instance.
- TransactionEvidenceV1::actor() is new, exhaustive over all six
variants. A destination event's actor must restate the evidence's
source instance, and deriving it a second way store-side is what
produced the defect it closes.
- format::object_type_code becomes pub(crate) and recovery.rs's private
twin is deleted, so one table exists where three did.
- RefRecord gains from_target/target() in terms of RefTarget, with the
code table stated once per direction and an unknown kind refused by
value as CheckpointError::RefKind. Defaulting an unknown kind would
launder it into the next checkpoint within one interval.
- max_projection_objects is capped at MAX_CANONICAL_ITEMS and its default
lowered to it; max_projection_chunks likewise, and max_projection_bytes
against the transitive per-chunk allowance. The old default described a
projection no manifest could encode. Supporting a hundred million
objects requires a versioned chunked or indexed manifest design, not a
larger hostile-decode ceiling.
Both governing documents are updated where they now misstate a frozen
fact, including §6.4's claim that signing covers the event digest. §6.5
records the two carry-forwards from B3's first slice — production staging
use is incomplete, and expire() has no scheduler — and records that the
requested StoreEngine staging accessor is a pending amendment which must
not be a bare Arc<ProjectionStaging>, since a clone could outlive
EngineShared, survive release of the root LOCK, and keep serving a root
this process no longer holds.
No package file is touched: engine.rs, transaction.rs, and staging.rs
consume these in their own commits.
levcs-protocol 6 consumer tests, levcs-store 109 library tests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The freeze commit cannot contain its own hash, so it is recorded here.
Wave A is frozen at 5ee9c6b78b with the gate
evidence measured at that commit, in both the plan and the scope document.
Restates what the freeze does and does not cover: the two carry-forwards are
B-wave work and outside it, and doc/swarm-fabric-roadmap-exploration.md
(e6a058d) stays non-binding and outside it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QNy4Ve7mogg4X1ezJTnFxG
Deliberately a separate commit, outside the Wave A freeze (5ee9c6b). This
document decides nothing and is not part of any frozen contract: it is a place
to hold the thinking on whether LeVCS should become a change fabric for
multi-agent work, pending its own review.
Records where the proposal maps onto the existing plan, where it cuts against
it -- levcsd must not own ref transactions, the ephemeral stratum invariant,
delegation being identity-shaped rather than protocol-shaped, attestations not
fitting TransactionEvidenceV1, and the canonical benchmark workload being the
wrong shape for swarm traffic -- and where the proposal is simply wrong.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QNy4Ve7mogg4X1ezJTnFxG
Freezes the Wave A interfaces, frame format, durability ordering, and
crash/fault fixtures per doc/phase1-storage-spine-scope.md section 5. Wave B
may now start against these.
D0 lands crates/levcs-store: the sealed public API, the file-ownership split,
the single durability syscall funnel with its counters and fault hooks, the
17-entry failpoint registry in compiler-enforced correspondence with
oracle::AppendFailpoint, and the journal-level drive seam. Wave A lands the
frame codec and journal/segment lifecycle (A1), the recovery index and
checkpoints (A2), and the durable-ingest benchmark and crash harness (A3).
The adversarial review found five defects behind a green gate, three of them
blockers, all closed here. Two were the same shape: drive::reopen_through_
recovery had reimplemented a simplified recovery and called none of recovery.rs
-- so it adopted segment footer sequences without validating frame bytes,
journal_id, or root_uuid, and it double-adopted interrupted seals. The seam
between two packages was untested precisely because each package's own tests
passed. reopen_through_recovery now delegates rather than decides, and
DriveRecovery carries the recovery report verbatim so tests assert the
disposition and not merely its effect: a double adoption and a correct replay
produce the same adopted set, which is how the defect stayed invisible.
Also closed: store-bench now schema-validates its own emitted artifact with a
five-mutation negative control instead of matching JSON substrings; the ACK
reconciler distinguishes duplicates and regressions from forward gaps;
checkpoint writes and a journal truncation are routed through the durability
funnel, whose guard now covers writes and truncations rather than only sync,
rename, and unlink.
bench/result-schema.json is amended (contract review 2026-07-24-B, second and
third amendments): per-gate latency ceilings conditional on outcome so a failed
run is representable, and the verification claims split per gate so a storage
run cannot certify an object graph it never touches. Not-applicable claims are
forbidden rather than falsified; applicable-but-not-performed report false.
Every relaxation is re-pinned in the else branch and asserted member by member,
after an edit in this series silently un-pinned all eleven validation flags and
was caught only by revalidating against constructed bundles.
Arming the fault registry now requires a FaultSerial token, so the invariant is
a compile error rather than a comment. The file where this was diagnosed
carried a header saying it was deliberately the only test in it, and a second
test had been added under that comment anyway -- an 8-in-40 failure rate that
read as flakiness.
Evidence at this commit: check-phase1.sh GATE_EXIT=0 across all four feature
configurations, 124 test binaries, zero failures; verify-store-recovery.sh
--cycles 100 with recovery_failures=0, acknowledged_loss=0,
torn_transactions=0, repeated_adoptions=0, bundle=schema-valid; recovery_eio
40/40 at four test threads; golden corpus byte-stable; fmt clean.
Carry-forwards, explicitly not Wave A blockers and recorded in scope section 5:
extend the crash matrix to generate sealed-frame corruption and cross-shard
journal movement, since it structurally cannot express the class the first
blocker belonged to; and wire GroupBuilder through B1's production path, since
deliverable 4-A1.2 is presently asserted only over a type nothing calls.
Charter item 9 -- ask every package what of its work is correct but uncalled --
is accepted for every subsequent wave. It, not the review, found the class.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QNy4Ve7mogg4X1ezJTnFxG
Levi Neuwirth