epiphany/crates/epiphany-bundle/DECISIONS.md

epiphany-bundle — decisions and Pass 11 candidates

This file records (a) the implementation decisions the QUICKSTART asked each agent to make once and document, and (b) the ambiguities discovered while building epiphany-bundle, batched as Pass 11 candidates for the spec rather than improvised in code (QUICKSTART, Process notes: "Ambiguities go into a batch, not into code … Don't open Pass 11 until you have at least three such items batched.").

Implementation decisions (QUICKSTART "Decisions you'll need to make")

1. Replica ID entropy source — N/A to this crate (Agent B/epiphany-core).

2. Event-arena storage — N/A to this crate (Agent B).

3. Chunk store backend for v0 — a positioned single-file BlockStore, append-at-EOF. The bundle is the file format, so chunks are addressed by file offset within one file (the spec's ChunkRef::offset), not a side BTreeMap<ChunkId, Bytes>. The BlockStore trait abstracts positioned reads/writes plus an explicit durable flush; three implementations back it:

   • MemStore — an in-memory byte image (flush is a no-op); the default v0 backing and the recovered-image reader.
   • FileStore — a real file whose flush is fsync (the production durability path; unix-only, via positioned pread/pwrite).
   • FaultStore — the crash simulator behind the acceptance gate. The QUICKSTART suggested an in-memory BTreeMap for v0 and deferring the mmap'd file backend "until Agent D's crash fuzzer is green." The crash fuzzer is this crate's gate, and it drives FaultStore; memory-mapping is left as the deferred optimization (the format's chunk immutability makes it safe later, per Chapter 8 §"Memory Mapping"). The body is allocated append-at-EOF, which trivially satisfies "MUSTNOT overwrite any currently-reachable chunk."

4. Async or sync — sync only. No async traits anywhere; BlockStore is sync (decision 4). A thin async wrapper crate can come later, as the QUICKSTART suggests, without touching this type system.

5. MSRV — workspace 1.77. No exotic features. std::io::Error::other (stable since 1.74) is used; overflow-checks stay on in release so offset and length arithmetic faults loudly rather than wrapping.

Additional local decisions

• A prototype canonical byte layout precedes the Binary Format companion. Chapter 8 §"Binary Format Companion" defers the byte-level encoding to a separate spec that does not yet exist (an explicit openquestion). To make the atomic-commit, crash-recovery, and re-serialization guarantees testable now, this crate defines a concrete, fixed-convention encoding: little-endian integers (matching the spec's preimage convention), u32-length-prefixed variable fields, 0/1 option-presence bytes, and the fixed prelude offsets documented in the header/superblock module tables. This is the bundle's analogue of epiphany-core's P11-4 and is provisional — see P11-D2.

• CRC-32C is hand-rolled (Castagnoli, table-driven const fn). Chapter 8 specifies CRC-32C for the header and each superblock; a ~30-line table-driven implementation avoids a second hashing dependency (the workspace keeps blake3 as the sole content-hash dependency) and pins the "123456789" → 0xE3069283 check vector.

• Blobs hash bare; chunks/manifests hash structured. Following Agent A's ContentHash::of_blob, a blob id is BLAKE3("MUSCBLOB" || payload). Non-blob chunks use the structured Chapter 8 preimage domain || kind || schema || uncompressed_length || payload, with the manifest under MUSCMANI and all other kinds under MUSCCHNK. See P11-D3.

• commit_timestamp is written as 0. It is advisory (selection is by generation, never timestamp), and a fixed value keeps commits byte-reproducible for the fuzzer. A real editor would stamp wall-clock time here.

• Crash model. FaultStore separates live (page-cache) bytes from durable bytes; only a successful flush promotes live → durable, and a crash on a flush may tear the most-recent (single in-flight) write to a prefix. This is faithful because the protocol's flush points isolate dependencies: the superblock-slot write is the only pending write at the commit-point flush, so a torn superblock is the only torn write that can affect selection, and the CRC catches it. Earlier-step torn writes only ever produce unreachable garbage.

• Indeterminate commit-point flush poisons the bundle. If the final flush (the commit point) returns an error, the new superblock may or may not have reached durable storage, so the on-disk active generation is unknown. commit marks the in-memory bundle read-only and returns the error; the caller must reopen from storage to resync. (Earlier flush errors are safe — the active slot is untouched, so the bundle remains validly at the old generation.)

• Reader resource limits. Untrusted lengths (a superblock's manifest_length, a manifest's chunk/blob lengths) are checked against policy caps (MAX_MANIFEST_BYTES, MAX_CHUNK_BYTES, MAX_BLOB_BYTES, plus a BlobRef's own declared_max) before any allocation, so a large/sparse hostile file cannot drive an OOM or a 32-bit truncation. The values are generous v0 defaults; a production reader would make them configurable.

• Writer/reader symmetry. create/commit refuse to emit anything their own open would reject: at least one declared profile; the canonical base's profile declared and its reduction-version consistent; every canonical root present and of the right kind/shape (operation roots decode and fit the profile's max block size; the canonical base is a snapshot; every blob root resolves and hash-verifies); the encoded manifest within MAX_MANIFEST_BYTES; no initial canonical roots/blobs at create. After emitting, the in-memory manifest is normalized (decoded back from its own canonical bytes), so bundle.manifest() matches a reopen (duplicate roots already collapsed), and a required extension makes even a freshly created bundle read-only. open additionally checks the manifest lies in the body, its generation matches the superblock, the superblock's profile is declared, and profile ids are distinct. The active profile (and thus the max block size enforced on reads) is the one the selected superblock names — a bundle opened under Lite reads under Lite's limits, not the canonical-first profile's.

• Profile support model (emittability vs editability). A profile is editable (Full/Lite, exact major, block bound ≤ the reader's MAX_CHUNK_BYTES), understood but read-only (ReadOnly), or unsupported (Custom registry profiles, a mismatched major, or a block bound the reader cannot allocate). Emittability and editability are separate. A bundle is emittable as long as it declares at least one understood profile — so a sole ReadOnly profile produces a valid read-only bundle (the spec describes ReadOnly-produced bundles). The active profile a writer names prefers the canonical-first editable profile (so [ReadOnly, Lite] is emitted under Lite, editable), falling back to the first merely understood one (a sole ReadOnly → read-only bundle). That selected declaration is computed once and drives canonical-root limits, the superblock profile, and the live read-only state, so commit-time validation cannot disagree with the reopened bundle. open mirrors this: an understood profile opens read-only-if-ReadOnly, an unsupported one opens read-only with an UnsupportedProfile anomaly. A profile major must match exactly. The spec's SHOULD upgrade-the-profile-on-first-edit is deferred — v0 opens a ReadOnly bundle read-only rather than rewriting the profile.

• Numeric SemVer ordering. Declarations carrying a SemVer (profiles, extensions) sort by an explicit numeric (id, version) key, not by encoded bytes — the little-endian version integers would otherwise sort byte-wise (256.0.0 before 1.0.0), violating Appendix D's "ascending by … semantic version."

• Blob media types are validated as the narrow RFC 6838 §4.2 restricted name type/subtype (ASCII, ≤127 chars per component, alphanumeric-first, the restricted-name alphabet — not the broader HTTP token set) on both decode and emit, keeping arbitrary or non-NFC bytes out of canonical manifests (ASCII is already NFC).

• Generation exhaustion returns BundleError::GenerationExhausted rather than overflow-panicking at u64::MAX.

Known v0 limitations (deliberately deferred, not defects)

These are bounded by v0 scope (QUICKSTART "Don't do these" / "decisions you'll need to make") rather than spec ambiguities. Each is honest about what is not yet enforced so a later integration knows where to extend.

• Retention/GC is a type, not an engine. RetentionPolicy is modeled as the QUICKSTART asks, and rollback over the two fixed slots is structurally supported, but there is no retained-manifest catalog, deterministic retention selection, GC reachability pass, or rollback operation. The spec itself frames GC as "a conservative, optional, deferred operation" that "MUSTNOT run as part of a commit's critical path" (Chapter 8 §"Garbage Collection and Retention"), and the body is append-only in v0, so nothing is reclaimed yet — manifests older than one generation are simply retained. A policy requesting more than one retained manifest cannot be honored for reclamation until the GC engine lands, but no manifest is lost either.

• Content-address dedup is current-manifest scoped, not whole-history. A commit reuses a chunk (or blob) already referenced by the active manifest rather than re-appending it. It does not dedup against older retained manifests or unreferenced garbage, and it trusts the existing reference's location without re-verifying the chunk (the chunk was verified when first committed and is re-verified on read). Whole-history dedup needs the same body-wide content index as the deferred GC engine.

• Operation-envelope block summary metadata is omitted. Chapter 8's OperationEnvelopeBlock carries dvv_summary, min_stamp, and max_stamp. These are semantic — a DVV and OperationStamps computed by reading the envelopes, which belong to epiphany-ops (Agent C). The bundle treats a block as opaque envelope bytes, so it cannot compute them. At C/D integration these become opaque, ops-supplied fields prefixed to the block payload; v0 stores only the envelopes (block::encode_block). read_operation_block does enforce the chunk kind and the active profile's maximum block size.

• Schema negotiation is major-gate only. A canonical chunk or manifest at an unsupported schema major is refused (BundleError::UnsupportedSchemaVersion); v0 defines only schema 0.x, so there is no minor-version back-compat matrix to exercise yet. Non-canonical opaque chunks at unknown majors are carried verbatim (they are never parsed).

• Extensions: required → read-only; opaque preservation is partial. An unknown required extension forces read-only (v0 understands no extensions, so all are unknown). Optional-extension preserved_chunk_roots are carried in the manifest, but the bundle does not yet enforce that a commit's builder closure preserves them, nor evaluate edit barriers / the unsafe-edit path — barrier operands (OperationKindTag, ObjectKind, EditBarrier) are owned by Agents C/E. The commit closure is, however, validated to never publish dangling or mismatched canonical roots.

Pass 11 candidates (ambiguities for the spec, not resolved in code)

P11-D1 — Superblock selection has no tie-break for equal generations

Chapter 8 §"Superblock Selection" says "the slot with the higher generation is active" but specifies no rule for two valid slots at the same generation (which the QUICKSTART nonetheless lists as a scenario the harness must handle). This crate resolves it deterministically: equal generation and equivalent load-bearing fields (manifest_hash, manifest_schema_version, reduction_algorithm_version, profile_id — the advisory commit_timestamp and the physical manifest offset/length are excluded) → the slots are equivalent, pick A; equal generation that differs in any of those → an IntegrityAnomaly::DivergentSameGeneration, opened read-only (two different committed states cannot share a generation under a conforming writer). The spec should adopt or override this.

P11-D2 — The Binary Format companion is not yet written

The concrete byte layout in this crate (prelude field offsets, integer endianness, length-prefix widths, option/enum-discriminant encodings) is provisional, standing in for the deferred Binary Format companion specification. When that companion lands, reconcile this crate's header, superblock, chunk, and manifest encodings with it (a failing cross-implementation round-trip would be the trigger, per the QUICKSTART process notes). This is the file-format analogue of epiphany-core's P11-4.

P11-D3 — Blob hashing shape is ambiguous

Chapter 8 §"Blobs" says blobs are "content-addressed identically to chunks (BLAKE3 of uncompressed payload, with the MUSCBLOB domain tag)." "Identically to chunks" implies the structured preimage (which commits to kind, schema version, and length); "BLAKE3 of uncompressed payload with the domain tag" implies a bare MUSCBLOB || payload. The two disagree. This crate follows Agent A's ContentHash::of_blob (bare MUSCBLOB || payload), the only spec content hash documented as a bare domain || payload. The spec should state explicitly whether a BlobId commits to a kind/schema/length or is bare.

P11-D4 — Enum discriminant values entering canonical state are unspecified

ChunkKind::canonical_bytes() appears in the chunk hash preimage (Chapter 8 §"Domain-Separated Preimages"), so each ChunkKind's numeric discriminant is normative — yet Chapter 8 fixes only the shape of the preimage, not the discriminant table (exactly the situation epiphany-core P11-1 flags for TypedObjectId). This crate assigns ChunkKind discriminants by declaration order (OperationEnvelopeBlock = 0 … Manifest = 8), as a single byte, and likewise fixes ProfileId (0–3) and CompressionAlgorithm (0–2) discriminants. The spec should pin these, since the ChunkKind value in particular changes content hashes.

P11-D5 — ManifestId derivation inputs are undefined

Chapter 8 §"The Manifest" says "Each commit produces a new ManifestId" and the deferred-types table assigns it the MUSCMNIF domain tag, but no derivation preimage is given. This crate derives trunc128(BLAKE3("MUSCMNIF" || document_id || generation || manifest_body)), where manifest_body is the canonical manifest encoding with the manifest_id field excluded (to avoid self-reference). The spec should fix the canonical input list so two conforming writers derive identical manifest ids.

P11-D6 — Where the RetentionPolicy lives is not shown

Chapter 8 §"Garbage Collection and Retention" requires that "the active conformance profile MUST declare a RetentionPolicy," but the ProfileDeclaration / ProfileConstraints structs shown in §"Format Profiles" do not include the field. This crate places retention_policy inside ProfileConstraints. The spec should show the field explicitly (and confirm whether a bundle declaring multiple profiles resolves retention from the first declared profile, as this crate does).