Push 5 / P5: the missing inverse -- bytes back to an operation envelope

Text Projection requires decoding operation envelopes. No such decoder existed,
and the hole was bigger than the task: the format was WRITE-ONLY for operations.
A bundle's envelope blocks decoded to opaque byte strings, OperationKind had an
encoder and no decoder, nothing outside epiphany-bundle even called decode_block,
and nothing anywhere reconstructed an OperationEnvelope. Chapter 6 holds that a
score's canonical state IS the set of operations committed to it -- so a bundle
could be written and its score never reopened. The envelope's byte layout was
fully pinned in the Binary Format companion. Nobody wrote the inverse.

epiphany_ops::decode_envelope is that inverse. The first thing built on it is
testkit/tests/bundle_reopen.rs: create a bundle from 400 generated envelopes,
commit, take the bytes, reopen from nothing but bytes, decode every envelope,
rebuild the OperationSet, reduce -- and get the same canonical state. That test
could not have been written before this commit.

Strict in two layers, per the P2 lesson. A whole-envelope re-encode-and-compare
guard, sound here because every sequence in this encoding is normalized by its
encoder. Plus per-site checks where the rule deserves its own error and a future
encoder change must not silently relax it: TransposeInterval.targets is a SET
(seq-strictly-increasing; a duplicate is rejected, never absorbed by the BTreeSet
it collects into), and the frozen Transpose.targets is a MULTISET (non-decreasing,
duplicates preserved). That is the rule Push 4a wrote into the wire table and left
for whoever built this decoder. And a bounded count(): a declared count past the
bytes remaining is rejected before it can drive an allocation.

Coverage measured, not assumed -- again. The obvious oracle (gen_envelope_set,
4000 envelopes) reaches only 28 of 31 kinds and 1 of 4 payload variants.
ChangeRegionTimeModel, DeclareTransaction, Registered and all three meta payloads
were untouched, and they hold the trickiest decoders: PositionRemapping, NFC
strings, ResolutionAction, EnvelopeHash. So the exhaustive test drives a match on
OperationKindTag, and the compiler forces a sample for every future kind.

Two mutations verified. Removing the seq-strictly-increasing check still rejects
-- the guard is a real backstop there -- but with the wrong error, so the per-site
check earns its place on the error rather than the verdict. Removing the
whole-envelope guard leaves every round-trip test green, because round-trips only
ever feed canonical bytes; an_unsorted_sequence_is_rejected_by_the_whole_envelope_guard
is the test that locks it, and it fails under that mutation.

A trap worth remembering: PitchId::new(ReplicaId(7), 1) and
OperationId::new(ReplicaId(7), 1) have identical canonical bytes -- typed ids
share their byte form -- so a byte-patching test that searches for an id finds the
envelope's own leading id first.

Gate: fmt clean, clippy 0, 31 targets / 1031 passed / 0 failed, docs 0 under
-D warnings, conformance 8/8, zero golden churn.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

crates/epiphany-ops/DECISIONS.md

@@ -1611,3 +1611,56 @@ vector, and MUST re-encode an accepted value to its own bytes — and that
 **accepting a `reject` vector and then normalizing it is accepting it**. Binary
 Format 0.8.0 → 0.9.0. The wire-format fuzzer stays an implementation deliverable.
 The corpus header now cites the requirement rather than asserting one.
+
+## Push 5 / P5 — the missing inverse: bytes back to an operation envelope
+
+Text Projection is blocked on this, but the blocker was bigger than the task.
+
+**The format was write-only for operations.** A bundle's envelope blocks decoded
+to opaque byte strings; `OperationKind` had an encoder and no decoder; nothing in
+the workspace called `decode_block` outside `epiphany-bundle` itself; and nothing
+anywhere reconstructed an `OperationEnvelope`. Chapter 6 holds that a score's
+canonical state *is* the set of operations committed to it — so a bundle could be
+written and its score could never be reopened. The envelope's byte layout was
+fully pinned (`binary_format.tex` §"Operation Envelope Encoding"); nobody wrote
+the inverse.
+
+`epiphany_ops::decode_envelope` is that inverse, and the first thing built on it
+is `testkit/tests/bundle_reopen.rs`: create a bundle from 400 generated
+envelopes, commit, take the bytes, reopen from nothing but bytes, decode every
+envelope, rebuild the `OperationSet`, reduce — and get the same canonical state.
+That test could not have been written before.
+
+**Strictness, in two layers, per the P2 lesson.**
+- A whole-envelope re-encode-and-compare guard. Sound *here* because every
+  sequence in this encoding is normalized by its encoder (`sorted_canonical`,
+  `BTreeSet`, `BTreeMap`), so non-canonical bytes cannot survive a round trip.
+- Per-site checks where the rule deserves its own error, and where a future
+  encoder change must not silently relax it: `TransposeInterval.targets` is a
+  **set** (`seq^⇑`, strictly increasing — a duplicate rejected, never absorbed by
+  the `BTreeSet` it collects into), and the frozen `Transpose.targets` is a
+  **multiset** (non-decreasing, duplicates preserved). This is the rule Push 4a
+  wrote into the wire table and left for whoever built this decoder; it is now
+  enforced.
+- A bounded `count()`: a declared count past the bytes remaining is rejected
+  before it can drive an allocation.
+
+**Coverage, measured rather than assumed — again.** The obvious oracle
+(`gen_envelope_set`, 4,000 envelopes) reaches only **28 of 31 kinds and 1 of 4
+payload variants**. `ChangeRegionTimeModel`, `DeclareTransaction`, `Registered`,
+and all three meta payloads were untouched — which happen to hold the trickiest
+decoders (`PositionRemapping`, NFC strings, `ResolutionAction`, `EnvelopeHash`).
+`every_operation_kind_and_payload_round_trips` drives an exhaustive `match` on
+`OperationKindTag`, so the compiler forces a sample for every future kind.
+
+**Mutations verified.** Removing the `seq^⇑` check still rejects (the guard is a
+genuine backstop for that field) but with the wrong error — so the per-site check
+earns its place on the error, not the verdict. Removing the whole-envelope guard
+leaves every round-trip test green, because round-trips only ever feed canonical
+bytes; `an_unsorted_sequence_is_rejected_by_the_whole_envelope_guard` is the test
+that locks it, and it fails under that mutation.
+
+**A trap worth remembering.** `PitchId::new(ReplicaId(7), 1)` and
+`OperationId::new(ReplicaId(7), 1)` have *identical canonical bytes* — typed ids
+share their byte form. A byte-patching test that searches for an id finds the
+envelope's own leading id first. Patch by framing, or from the end.

crates/epiphany-ops/src/lib.rs

@@ -83,6 +83,7 @@ mod conflict;
 mod decode;
 mod effect;
 mod encode;
+mod envdecode;
 mod envelope;
 mod migrate;
 mod opset;
@@ -111,6 +112,7 @@ pub use effect::{
     NoOpReason, OperationEffect, PreconditionFailureReason, ReanchorReason, ReanchorResult,
     RepairKind, RepairRecord, TupletCompensationKind,
 };
+pub use envdecode::{decode_envelope, EnvelopeDecodeError};
 pub use envelope::{
     peek_operation_id, well_formed, EnvelopeHash, OperationEnvelope, WellFormednessError,
 };

crates/epiphany-testkit/tests/bundle_reopen.rs

@@ -0,0 +1,79 @@
+//! The document round trip the format could not do until Push 5 / P5: write a
+//! bundle, close it, reopen it from bytes, and rebuild the operation set the
+//! score's canonical state *is* (Chapter 6 §"Design Principles").
+//!
+//! Before `epiphany_ops::decode_envelope` existed, a bundle's operation blocks
+//! decoded to opaque byte strings and stopped there. Everything below the
+//! `decode_block` call was unreachable: no `OperationEnvelope`, no
+//! `OperationSet`, no reduction, no score.
+
+use epiphany_bundle::{
+    decode_block, pack_operation_blocks, Bundle, DocumentId, FileUuid, Manifest, MemStore,
+    StagedChunk,
+};
+use epiphany_determinism::fuzz::SplitMix64;
+use epiphany_ops::{decode_envelope, OperationSet};
+
+fn append_roots(ctx: &epiphany_bundle::CommitContext) -> Manifest {
+    let mut m = ctx.previous_manifest.clone();
+    m.operation_roots.extend(ctx.new_chunks.iter().copied());
+    m
+}
+
+#[test]
+fn a_bundle_round_trips_through_bytes_back_into_a_reduced_score() {
+    // A real, varied operation set: every kind the generator reaches.
+    let mut rng = SplitMix64::new(0x00DE_C0DE_0B00_C1E5);
+    let envelopes = epiphany_ops::fuzz::gen_envelope_set(&mut rng, 400);
+    assert!(envelopes.len() > 300, "a meaningful document");
+
+    let expected = {
+        let mut set = OperationSet::new();
+        set.accept_all(envelopes.clone());
+        set.reduce().canonical_bytes()
+    };
+
+    // Write it.
+    let payloads: Vec<Vec<u8>> = envelopes
+        .iter()
+        .map(epiphany_determinism::CanonicalEncode::to_canonical_bytes)
+        .collect();
+    let staged: Vec<StagedChunk> = pack_operation_blocks(&payloads)
+        .into_iter()
+        .map(StagedChunk::operation_block)
+        .collect();
+    assert!(!staged.is_empty());
+
+    let mut bundle = Bundle::create(
+        MemStore::new(),
+        FileUuid([3; 16]),
+        Manifest::empty(DocumentId([9; 16])),
+    )
+    .expect("create");
+    bundle.commit(&staged, append_roots).expect("commit");
+    let image = bundle.into_store().into_bytes();
+
+    // Close it, reopen it from nothing but the bytes.
+    let reopened = Bundle::open(MemStore::from_bytes(image)).expect("reopen");
+
+    let mut recovered = Vec::new();
+    for chunk in reopened.manifest().operation_roots.clone() {
+        let payload = reopened.read_chunk(&chunk).expect("chunk reads");
+        for envelope_bytes in decode_block(&payload).expect("block frames") {
+            // The inverse that did not exist.
+            recovered.push(decode_envelope(&envelope_bytes).expect("envelope decodes"));
+        }
+    }
+
+    assert_eq!(recovered.len(), envelopes.len(), "every envelope came back");
+
+    // Same operations, and therefore the same canonical state. Reduction is
+    // permutation-invariant, so this compares the document, not the file layout.
+    let mut set = OperationSet::new();
+    set.accept_all(recovered);
+    assert_eq!(
+        set.reduce().canonical_bytes(),
+        expected,
+        "the reopened bundle reduces to the same canonical state"
+    );
+}