Commit Graph

9 Commits

Author SHA1 Message Date
Levi Neuwirth 4c68f08446 Implement the partial B3 staging-session slice
Scope 6.5 deliverables 1-5: what a staging session is on disk, what it
costs, when it dies. The instance-layer half of plan §8 — identity,
policy, ProjectionCore, the v2 routes — is deliberately absent; B3 binds
and exposes the fields those checks key on and evaluates none of them.

Two findings from the B2 review shaped the result more than the original
deliverables did.

Restart was in-memory handle reuse. open() built a fresh registry and
never read the filesystem, so after a real reopen an existing session ID
was admitted as new -- and if materialization then found the old
directory, error cleanup could unlink a durable session. That is data loss
reachable from an ordinary restart plus one error. open() now scans,
validates, and reconstructs sessions, chunk indexes, and the whole
occupancy account before returning, so a reconstructed ID is occupied and
refused as a conflict long before the error path; and that path no longer
calls remove_dir_all. Either change alone closes the loss.

A session is final iff its directory holds a valid session record for its
own ID, installed by rename_noreplace over fenced, digest-checked bytes,
so the final name only ever appears atomically over complete data. A
directory without one is abandoned materialization -- a crash between
mkdir and that rename, unaccounted and unreferenceable. The two states
share no code path, and reclaimed abandonments count on their own counter
so they can never be read as aborts or expiries.

Global quotas were per-handle. Every open() built an independent registry
outside the root LOCK, so two handles admitted twice the global limit and
the atomic-insertion work bought nothing across them. Construction now
requires proof of the held root lock and refuses a second in-process
instance, making two accountants on one root inexpressible rather than
discouraged.

Also: bounds are enforced atomically with insertion under one mutex with
no read-then-decide path, refused as typed LimitExceeded or Overloaded and
never by eviction; the directory-sync test pinned two syncs when the first
session in a shard needs three, a counter assertion that encoded the bug;
cleanup now validates a whole directory before unlinking anything, rather
than discovering a surprise midway through destroying a live session; and
artifact I/O moved off the registry mutex onto maintenance workers, with
the calling thread asserted to hold no guard rather than documented not to.

Deliverables 6-8 are explicit NotImplemented naming themselves.
StagedSessionState omits Finalizing, so deliverable 6 will fail to compile
at exactly the expiry and abort sites that must learn about a pin.

Carry-forwards recorded in §6.5, not closed: no production path begins a
session, so the sealed-invisibility acceptance stays ignored with both
blockers named; and expire() has no scheduler, so session age is a bound
enforced when asked and never asked.

116 library tests, 27 staging tests, 1 intentionally ignored.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 18:01:25 -04:00
Levi Neuwirth ef7d263fed Amend Wave B storage contracts after adversarial review
The B2 review of the first B1/B3 slice found five defects whose fixes were
not available to the packages that had to make them: each needed a change
to a surface those packages do not own. Rather than let them restate a
frozen fact locally, the lead amends the surfaces and the packages consume
them. Contract review 2026-07-28-A records all five with the weaker
alternative that was rejected for each.

  - CommitEvidenceSigner::sign_event's parameter becomes signing_digest.
    The message is SignedCommittedTransactionV1::signing_digest, which
    binds key epoch and durability result around the canonical event; the
    bare event digest is chain identity, not signature material. A doc line
    was not enough: misuse is undetectable until after durability, and is
    first observed by a mirror on another instance.

  - TransactionEvidenceV1::actor() is new, exhaustive over all six
    variants. A destination event's actor must restate the evidence's
    source instance, and deriving it a second way store-side is what
    produced the defect it closes.

  - format::object_type_code becomes pub(crate) and recovery.rs's private
    twin is deleted, so one table exists where three did.

  - RefRecord gains from_target/target() in terms of RefTarget, with the
    code table stated once per direction and an unknown kind refused by
    value as CheckpointError::RefKind. Defaulting an unknown kind would
    launder it into the next checkpoint within one interval.

  - max_projection_objects is capped at MAX_CANONICAL_ITEMS and its default
    lowered to it; max_projection_chunks likewise, and max_projection_bytes
    against the transitive per-chunk allowance. The old default described a
    projection no manifest could encode. Supporting a hundred million
    objects requires a versioned chunked or indexed manifest design, not a
    larger hostile-decode ceiling.

Both governing documents are updated where they now misstate a frozen
fact, including §6.4's claim that signing covers the event digest. §6.5
records the two carry-forwards from B3's first slice — production staging
use is incomplete, and expire() has no scheduler — and records that the
requested StoreEngine staging accessor is a pending amendment which must
not be a bare Arc<ProjectionStaging>, since a clone could outlive
EngineShared, survive release of the root LOCK, and keep serving a root
this process no longer holds.

No package file is touched: engine.rs, transaction.rs, and staging.rs
consume these in their own commits.

levcs-protocol 6 consumer tests, levcs-store 109 library tests.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 18:01:05 -04:00
Levi Neuwirth cd37f8bc04 Close the Wave A matrix carry-forward (B4 deliverable 1)
The Wave A review recorded that the crash matrix structurally could not
express its own first blocker: no failpoint corrupts a frame inside a
sealed segment, and none moves a journal between shards of one root. A
green matrix on a defect it cannot represent is the same trap as a helper
nothing calls, so the finding was carried forward rather than closed.

Add the two physical crash-image generators as a `damage` subcommand on
the crash driver, and two tests that drive them through the same
production `reconcile` path the matrix and the recovery script use.

  - sealed-frame-corruption flips one payload byte in the first frame of
    a segment the drive actually sealed and installed. The footer stays
    structurally valid, so only frame verification can reject it.
  - cross-shard-journal-movement relocates shard 1's active journal under
    shard 0 of the same root. Root UUID validation cannot see this; the
    journal header's shard index must be bound to the directory being
    opened.

Both generators mutate a production-written image and fence the mutation;
neither synthesizes a footer, frame, manifest, or checksum, so a passing
test cannot be an artifact of the harness agreeing with itself. Each
source image is required to be unambiguous — exactly one segment or
journal — so the result does not depend on directory iteration order.

The assertions pin the refusal to its own cause rather than to any
non-zero exit: the frame case must fail on the digest recomputation and
the movement case on the shard binding, and neither may publish a partial
adoption result.

crash_matrix: 26 passed. scripts/check-phase1.sh green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 14:25:07 -04:00
Levi Neuwirth 5111d655da Implement D0-B storage publication interfaces 2026-07-27 22:31:32 -04:00
Levi Neuwirth e85a159525 Scope Wave B storage work 2026-07-27 21:31:52 -04:00
Levi Neuwirth 5ee9c6b78b Freeze Wave A: Phase 1 storage spine
Freezes the Wave A interfaces, frame format, durability ordering, and
crash/fault fixtures per doc/phase1-storage-spine-scope.md section 5. Wave B
may now start against these.

D0 lands crates/levcs-store: the sealed public API, the file-ownership split,
the single durability syscall funnel with its counters and fault hooks, the
17-entry failpoint registry in compiler-enforced correspondence with
oracle::AppendFailpoint, and the journal-level drive seam. Wave A lands the
frame codec and journal/segment lifecycle (A1), the recovery index and
checkpoints (A2), and the durable-ingest benchmark and crash harness (A3).

The adversarial review found five defects behind a green gate, three of them
blockers, all closed here. Two were the same shape: drive::reopen_through_
recovery had reimplemented a simplified recovery and called none of recovery.rs
-- so it adopted segment footer sequences without validating frame bytes,
journal_id, or root_uuid, and it double-adopted interrupted seals. The seam
between two packages was untested precisely because each package's own tests
passed. reopen_through_recovery now delegates rather than decides, and
DriveRecovery carries the recovery report verbatim so tests assert the
disposition and not merely its effect: a double adoption and a correct replay
produce the same adopted set, which is how the defect stayed invisible.

Also closed: store-bench now schema-validates its own emitted artifact with a
five-mutation negative control instead of matching JSON substrings; the ACK
reconciler distinguishes duplicates and regressions from forward gaps;
checkpoint writes and a journal truncation are routed through the durability
funnel, whose guard now covers writes and truncations rather than only sync,
rename, and unlink.

bench/result-schema.json is amended (contract review 2026-07-24-B, second and
third amendments): per-gate latency ceilings conditional on outcome so a failed
run is representable, and the verification claims split per gate so a storage
run cannot certify an object graph it never touches. Not-applicable claims are
forbidden rather than falsified; applicable-but-not-performed report false.
Every relaxation is re-pinned in the else branch and asserted member by member,
after an edit in this series silently un-pinned all eleven validation flags and
was caught only by revalidating against constructed bundles.

Arming the fault registry now requires a FaultSerial token, so the invariant is
a compile error rather than a comment. The file where this was diagnosed
carried a header saying it was deliberately the only test in it, and a second
test had been added under that comment anyway -- an 8-in-40 failure rate that
read as flakiness.

Evidence at this commit: check-phase1.sh GATE_EXIT=0 across all four feature
configurations, 124 test binaries, zero failures; verify-store-recovery.sh
--cycles 100 with recovery_failures=0, acknowledged_loss=0,
torn_transactions=0, repeated_adoptions=0, bundle=schema-valid; recovery_eio
40/40 at four test threads; golden corpus byte-stable; fmt clean.

Carry-forwards, explicitly not Wave A blockers and recorded in scope section 5:
extend the crash matrix to generate sealed-frame corruption and cross-shard
journal movement, since it structurally cannot express the class the first
blocker belonged to; and wire GroupBuilder through B1's production path, since
deliverable 4-A1.2 is presently asserted only over a type nothing calls.

Charter item 9 -- ask every package what of its work is correct but uncalled --
is accepted for every subsequent wave. It, not the review, found the class.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QNy4Ve7mogg4X1ezJTnFxG
2026-07-26 19:47:03 -04:00
Levi Neuwirth d22f48ae08 Robustness fixes 2026-05-03 20:56:46 -04:00
Levi Neuwirth 1996db37b2 integrate CI and docs 2026-05-01 11:29:18 -04:00
Levi Neuwirth 21c6056ae6 levcs 0.1.0 - initial core 2026-05-01 11:14:36 -04:00