Levi Neuwirth
bc06706e41
FORMAT_MAJOR becomes 1 and FORMAT_MINOR restarts at 0. The decoder stops being exact-major-only: it classifies three ways through a named FormatEpoch carried on FixedHeader, so major 0 is decoded deliberately as legacy rather than refused. Old readers already fail closed on an unknown major, so that half needed no mechanism — which is why the major is the right carrier, and why the header's immutability, fatal to FORMAT_MINOR as a provenance field, is what makes it sound as an epoch field. The matrix: a major-0 bundle with no base may open; one carrying a base is refused; one attempting to add a base is refused and told to repack. That last row is the non-inheritance rule. Three errors, none of which degrades to read-only: two permanent legacy/repack errors, and ReductionAuthorityUnavailable, which is temporary, names P13-S27, and must not say repack — a major-1 container is already the right epoch. Until P13-S27 lands, both major-1 base boundaries are closed: opening a major-1 bundle that already carries a base, and committing one into it. Neither may be left open while the epoch asserts a validation that never ran. Text projection cannot mint a base. serialize_document staged a carried base into a fresh bundle and build_manifest wrote it, so an old or hand-authored document could be laundered straight through the boundary. All three sides now refuse: projection, parsing, and a new dedicated SerializeError variant — none of which existed to be "retained". COMPANION_VERSION moves to 0.14.0 and the corpus is rebuilt to 20 vectors and ten rejection classes, with canonical_bases reach dropping 2 -> 0. That is a real capability loss and is recorded as one. Corruption keeps precedence in both epochs: a corrupt major-1 base fails as malformed, never as the temporary authority error a user would reasonably retry. All 11 mutations were run and observed, not reasoned about. M4 is the signing one — with the legacy commit refusal removed, a legacy container gains a base in place, which is exactly the counterexample that killed FORMAT_MINOR. M11 confirms the third error is distinct while test 3 stays green, proving the mutation stayed inside the major-1 branches. M7 fails on both epoch halves. M8 falls through to SerializeError::Bundle(ReductionAuthorityUnavailable), confirming the text layer's own refusal is what the test asserts. Two touch-table gaps surfaced during execution, both the same shape: a .tex requirement addition moves hardcoded counts in requirement_labels.rs, and the companion bump moves a second normative version literal spelled version~0.13.0 rather than (0 13 0). Neither file was in any touch table; the second was caught only because a test exists for exactly that failure. P13-S27 is unblocked — its pin 2a is resolved from outside, as its own prohibition required — and inherits three obligations: both interim refusals converted to validation, M8's deferred laundering demonstration, and pin 3c's two suspended conformance assertions. P13-S16 remains blocked on S27. Workspace green at 1569. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QjsEnYhm1gPpf6ii2iFxFV |
||
|---|---|---|
| .. | ||
| examples | ||
| src | ||
| tests | ||
| Cargo.toml | ||
| DECISIONS.md | ||
| README.md | ||
README.md
epiphany-bundle
The Epiphany .musc file format, implementing the normative requirements of
Chapter 8 (File Format) of the core specification (spec/core_spec.pdf).
This is Agent D's crate per spec/QUICKSTART.md. It depends on
epiphany-determinism (Agent A) and on nothing else — not on epiphany-core
(Agent B) or epiphany-ops (Agent C):
bundles handle bytes, ops handles semantics. A canonical-base snapshot from the bundle's perspective is opaque bytes plus a frontier DVV; only
epiphany-opsinterprets it. — QUICKSTART
A bundle is a single file: a fixed 64-byte header at offset 0, two 256-byte superblock slots, then a body of immutable, content-addressed chunks. The superblocks are the only mutable on-disk objects. A commit appends new chunks, writes a new manifest chunk, then flips the active superblock by writing the inactive slot and durably flushing it — that flush is the commit point. Because commits only ever append and touch the inactive slot, a crash can never corrupt the active state.
What's here
| Area | Items | Spec |
|---|---|---|
| Prelude | FixedHeader (64 B, CRC-32C), Superblock/CommitState (256 B, CRC-32C), select_active |
Ch. 8 §"The Bundle Layout", §"Superblock Selection" |
| Atomic commit | Bundle::create/open/commit, the 7-step protocol, cold-open path |
Ch. 8 §"The Atomic Write Protocol", §"Streaming Reads" |
| Content addressing | chunk_content_hash/chunk_id, ChunkRef, ChunkKind, CompressionAlgorithm, domain separation |
Ch. 8 §"Content Hashing", §"Chunks" |
| Manifest | Manifest (canonical_base ≠ acceleration_snapshots), SnapshotRef, BlobRef, ProfileDeclaration, ExtensionDeclaration |
Ch. 8 §"The Manifest" |
| Retention | RetentionPolicy (first-class), ProfileConstraints |
Ch. 8 §"Garbage Collection and Retention" |
| Op blocks | pack_operation_blocks (1 MiB soft target), encode_block/decode_block |
Ch. 8 §"Operation Envelope Blocks" |
| Storage | BlockStore, MemStore, FileStore (real fsync), FaultStore (crash sim) |
Ch. 8 §"Durable Writes" |
| Gates | fuzz::run_crash_recovery_fuzz, fuzz::exhaustive_crash_check, fuzz::run_manifest_selection_harness |
QUICKSTART acceptance |
The crash-recovery contract (the acceptance gate)
Kill the process between any two syscalls in the commit protocol; reopen; the bundle must be valid in 100% of runs, and must recover to the previous generation when the crash precedes the durable flush. This is the most important single test in the entire prototype. — QUICKSTART, Agent D
Killing a real process between syscalls cannot be made deterministic, so the
fuzzer drives the commit against a FaultStore that distinguishes live
(page-cache) bytes from durable (survives-a-crash) bytes and can crash after
any chosen syscall — optionally tearing the in-flight superblock write, the
case the slot CRC must catch. After every simulated crash the bundle is reopened
from the durable image and must:
- open successfully (never corrupt);
- be at the previous generation or the new one, never anything else;
- if the commit returned
Ok, be at the new generation; and if the crash was clean (the in-flight flush persisted nothing) and the commit did not complete, be at the previous generation — the exact "recover to the previous generation when the crash precedes the durable flush" property. (A torn final flush may at a full prefix legitimately persist the whole superblock — the genuine post-commit case — so the torn branch admits either generation.) - report no integrity anomaly;
- have every canonical chunk present and hash-intact.
Two drivers exercise this: a randomized 10,000-iteration sweep, and an exhaustive per-commit sweep that tests every syscall boundary crossed with every tear point (clean, and torn at prefixes around the 252-byte CRC offset and the 256-byte slot size). The second leaves no step of the protocol untested.
The companion manifest_selection gate asserts the Chapter 8 superblock-
selection rule across every corruption scenario the QUICKSTART enumerates: slot A
corrupt + B valid (and vice versa), both valid at generation+1, both valid at the
same generation (equivalent, and divergent), a generation gap > 1, a
non-committed slot, a manifest-hash mismatch, and neither valid.
Building and testing
cargo test -p epiphany-bundle # unit + the two gates
cargo clippy -p epiphany-bundle --all-targets -- -D warnings
cargo run --release --example fuzz_crash -- 1000000 # extended crash soak
Hand-off criteria (QUICKSTART, Agent D)
cargo testclean.- Crash-recovery fuzzer passes 10,000 iterations
(
crash_recovery_fuzz_ten_thousand_iterations, two seeds; extended soak via the example binary; exhaustive per-syscall sweep inexhaustive_sweep_across_base_states_and_commit_shapes). - Manifest-selection harness handles every corruption scenario
(
every_selection_scenario_holds). - Real-filesystem
fsyncround-trip (file_store_real_fsync_round_trip).
Scope boundaries (per QUICKSTART "Don't do these")
v0 writes only uncompressed chunks (compression on the write path is deferred),
but reading zstd-compressed chunks and blobs is supported, per the spec's
§Compression MUST (the manifest is mandatory-uncompressed regardless, and a
compressed manifest is rejected). It carries the text-projection root but does
not implement the s-expression projection content, and it preserves extension
declarations and chunks but does not evaluate edit barriers — barrier operands
(OperationKindTag, ObjectKind, EditBarrier) are owned by Agents C and E.
Operation envelopes, snapshots, and causal frontiers are opaque bytes here.
See DECISIONS.md for the prototype byte-layout choices that anticipate the
deferred Binary Format companion, and the batched Pass 11 candidates.