LeVCS/crates/levcs-store
Levi Neuwirth e050b6dddd Release the root lock explicitly rather than by closing a handle
B4's harness measured intermittent AlreadyLocked on reopening a store it
had just dropped -- up to 29 retries over 147ms, 7 failures in 40 runs,
reproducing on a single-shard engine. StoreEngine::drop was not the cause:
it closes every channel and joins every writer, and looked correct in
every single-process test.

An flock is held by the open file description, not by a descriptor. A
concurrently forked child transiently inherits that description, and
FD_CLOEXEC closes it at exec, not at fork -- so during that window the
parent closing its own descriptor releases nothing. Scope 3.1 makes
AlreadyLocked a refusal and never a wait, so every consumer that closes
and reopens a root -- a recovery drill, an in-place restart, the Phase 2
migrator -- could be refused with no defined retry.

Releasing must therefore be an explicit act. lock_root returns an RAII
RootLock that issues LOCK_UN in Drop before the file closes, and both
owners -- RecoverySession and ShardDrive -- hold that guard. The unlock
lives in sys.rs beside try_lock_exclusive so the locking syscalls stay in
the one funnel. StoreEngine needs no change: it holds a RecoverySession,
never a File, so it inherits the fix.

Drop is the only release path. A release() and a file() accessor were
written and deleted before landing: neither had a caller, and an uncalled
second way to release a lock is exactly the decoy the charter names. A
failed LOCK_UN in Drop cannot be returned and must not be swallowed, so it
increments a counter the regression asserts unchanged.

The rejected alternative was fixing this in StoreEngine::drop alone. That
leaves ShardDrive and the drive's one-shot session exposed and makes
correctness depend on a descriptor lifetime that fork can extend.

The regression is synchronized rather than timed: the child forks while
the lock is held, signals ready on one pipe, and blocks on a second until
after the parent has released and attempted its reopen, so the inherited
descriptor is provably open across the whole window and the reopen is
asserted on its first attempt. With the explicit unlock removed it fails
10/10; as landed it passes 40/40. Measured under load -- 200 close-reopen
cycles against 72,255 concurrent forks -- 0 refusals, worst case 1 attempt
and 10.8ms; the same load kills the pre-fix behaviour within 0.05s, so the
load reproduces the defect rather than merely being weak.

Contract review 2026-07-28-B records the amendment. Two carry-forwards are
recorded in scope 6.6: the verify-store-recovery SIGKILL cycles still drive
the journal seam rather than submit, so kill -9 never lands inside a real
publication and the acknowledged-crash-recovery criterion is only partly
earned; and B4's bounded reopen retry must become a one-attempt assertion
now that the defect it compensates for is gone.

scripts/check-phase1.sh GATE_EXIT=0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 21:20:52 -04:00
..
benches Freeze Wave A: Phase 1 storage spine 2026-07-26 19:47:03 -04:00
examples Freeze Wave A: Phase 1 storage spine 2026-07-26 19:47:03 -04:00
src Release the root lock explicitly rather than by closing a handle 2026-07-28 21:20:52 -04:00
tests Implement the partial B4 crash-matrix and benchmark slice 2026-07-28 20:55:02 -04:00
Cargo.toml Implement D0-B storage publication interfaces 2026-07-27 22:31:32 -04:00