Land whole-score codec (item 5) and flip the M3 full-Score gate green

Implements a total, reversible canonical byte form for the entire
epiphany_core::Score graph, unblocking the byte-level full-Score serialization
gate that M3 had to defer.

epiphany-core/src/codec.rs:
- Score::canonical_bytes() / Score::decode_canonical() with a validating
  ScoreDecodeError and a forward-only Reader cursor.
- A local Codec trait with generic combinators (Option/Vec/BTreeSet/BTreeMap/
  tuple) and macros (struct_codec!/cstyle_enum_codec!/unit_codec!/
  catalog_id_codec!) so encode and decode stay symmetric across ~110 types
  spanning graph.rs, event.rs, pitch.rs, time.rs, tempo.rs.
- Uniform form: LE integers, one discriminant byte per tagged union, u32
  counts/length-prefixes, every variable-width leaf length-prefixed, raw UTF-8
  for free text (so decode(encode(x)) == x for any valid score; catalog ids are
  already NFC). EventArena round-trips via iter_canonical + insert.
- Two pub(crate) accessors added for the codec: EventOrderingDAG::edges_ref,
  SpellingPrecedence::order_ref.
- Tests: generator-score corpus (valid_score + valid_score_rich), exotic
  event/pitch variants the generators omit, distinctness, and decoder
  rejection of trailing/truncated/empty bytes.

epiphany-testkit:
- roundtrip::assert_score_serialization_stable: encode the real Score, store it
  as a bundle Snapshot, reopen + hash-verify, decode to an equal Score, and
  assert a byte-identical re-encode.
- convergence::materialized_score builds a real ~50-bar reduce_onto
  materialization for the gate.
- criterion_4_full_score_byte_roundtrip flips from #[ignore] to a live gate;
  wired into the conformance suite. Docs (lib.rs, README, core DECISIONS P11-4)
  updated to reflect the landed codec.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Levi Neuwirth 2026-06-21 18:15:07 -04:00
parent 9b0d3e8e2c
commit f5aaa96b11
11 changed files with 2335 additions and 39 deletions

View File

@ -95,10 +95,25 @@ exist. To make round-trip serialization testable now (v0 acceptance criterion
4), this crate defines a concrete canonical byte form for its primitives —
notably `RationalTime` (sign + length-prefixed big-endian numerator and
denominator magnitudes, always reduced) and the wall-clock integers
(little-endian, matching `QuantizedCoord`). These are deterministic and
reversible but provisional: when the Binary Format companion lands, reconcile
this crate's `CanonicalEncode`/`CanonicalDecode` with it (a failing cross-crate
round-trip test would be the trigger, per the QUICKSTART process notes).
(little-endian, matching `QuantizedCoord`).
**M3 follow-up — the whole-score codec (item 5).** `src/codec.rs` now composes
those primitives into a total, reversible canonical byte form for the *entire*
`Score` graph (`Score::canonical_bytes` / `Score::decode_canonical`), so the
materialized graph — not only the Chapter 6 `MaterializedState` bookkeeping —
round-trips byte-identically (Agent F's `criterion_4_full_score_byte_roundtrip`
drives it through a real bundle snapshot). The form is deliberately uniform:
little-endian integers, a single discriminant byte per tagged union, `u32`
counts/length-prefixes, every variable-width leaf length-prefixed, and raw
(non-NFC-folded) UTF-8 for free-text fields so `decode(encode(x)) == x` for every
valid score (catalog ids are already NFC at construction). The two private-field
accessors the codec needs (`EventOrderingDAG::edges_ref`,
`SpellingPrecedence::order_ref`) are `pub(crate)`.
These are deterministic and reversible but provisional: when the Binary Format
companion lands, reconcile this crate's `CanonicalEncode`/`CanonicalDecode` and
the whole-score `codec` with it (a failing cross-crate round-trip test would be
the trigger, per the QUICKSTART process notes).
### P11-5 — Scope boundary: the Chapter 4 tuning catalog is referenced, not defined here

File diff suppressed because it is too large Load Diff

View File

@ -286,6 +286,14 @@ impl EventOrderingDAG {
self.edges.get(&event).map(|v| v.as_slice()).unwrap_or(&[])
}
/// The raw adjacency map, for the canonical codec (which must serialize the
/// full ordering, not only the events reachable from a query).
pub(crate) fn edges_ref(
&self,
) -> &std::collections::BTreeMap<crate::ids::EventId, Vec<crate::ids::EventId>> {
&self.edges
}
/// Every event the ordering names — DAG nodes (sources) and edge targets.
/// Used by the invariant checker to confirm the ordering only references
/// events that exist in the region (invariant 10).

View File

@ -42,6 +42,7 @@
//! event-arena storage via `slotmap` (decision 2), fully sync (decision 4),
//! current stable Rust (decision 5). `unsafe` is forbidden crate-wide.
mod codec;
mod event;
mod graph;
mod ids;
@ -106,6 +107,8 @@ pub use tempo::{
INVERSION_MAX_ITERATIONS, INVERSION_TOLERANCE_WHOLE_NOTES,
};
pub use codec::ScoreDecodeError;
pub use indexes::ScoreIndexes;
pub use invariants::{check_invariant, check_invariants, GraphInvariant, InvariantViolation};

View File

@ -643,6 +643,11 @@ impl SpellingPrecedence {
Some(SpellingPrecedence { order })
}
/// The precedence order, for the canonical codec.
pub(crate) fn order_ref(&self) -> &[SpellingSourceKind] {
&self.order
}
/// The rank of a source kind: lower wins. `0` is the highest precedence.
pub fn rank(&self, kind: SpellingSourceKind) -> usize {
self.order

View File

@ -107,13 +107,14 @@ Criterion 4 has three tiers — two asserted now, one pending item 5:
structural equality across delivery orders) — the determinism precondition a
byte codec depends on.
- **The full-`Score` byte round-trip is pending item 5 (Agent B).** No
whole-score canonical codec (`CanonicalEncode`/`CanonicalDecode for Score`)
exists yet, so `criterion_4_full_score_byte_roundtrip` is marked `#[ignore]`
(visible as *ignored*, never falsely green) rather than asserted on the
bookkeeping projection and passed off as a whole-Score gate. When item 5 lands
the codec, drop the attribute and assert the real byte cycle through a bundle
snapshot.
- **The full-`Score` byte round-trip** (`criterion_4_full_score_byte_roundtrip`,
via `assert_score_serialization_stable`): item 5's whole-score codec
(`epiphany_core::Score::canonical_bytes` / `decode_canonical`) has landed, so a
real ~50-bar `Score` — materialized through Agent C's `reduce_onto` — now
`encode → decode → re-encode`s byte-identically through a real bundle snapshot
(hash-verified on reopen), with the decoded `Score` structurally equal to the
original. This is the whole musical graph (arena, voices, regions,
cross-cutting, tombstones), not the bookkeeping projection.
## Decisions (per QUICKSTART "Make each one once and document it")
@ -132,14 +133,13 @@ Criterion 4 has three tiers — two asserted now, one pending item 5:
Per the QUICKSTART, implementation-discovered gaps are batched, not improvised:
- **Whole-graph (`epiphany_core::Score`) wire format — pending item 5 (Agent B).**
Criterion 4 is a real decode round-trip at the canonical Chapter-6
`MaterializedState` layer; the materialized `Score` is shown *reproducible*
today. A direct canonical byte codec for the richer core `Score` does not exist
yet (it is item 5's "whole-score codec", to be reconciled with the Binary
Format companion), so the whole-`Score` byte round-trip
(`criterion_4_full_score_byte_roundtrip`) is an explicit `#[ignore]`'d gate
rather than a falsely-green assertion.
- **Whole-graph (`epiphany_core::Score`) wire format — landed (item 5).** A
direct canonical byte codec for the core `Score` now exists
(`epiphany_core::Score::canonical_bytes` / `decode_canonical`), and
`criterion_4_full_score_byte_roundtrip` exercises it on a real `reduce_onto`
materialization through a bundle snapshot. The prototype byte form predates the
Binary Format companion specification and is to be reconciled with it (see
`epiphany-core/DECISIONS.md`, P11-4).
- **Layout harness re-pointed.** `epiphany-layout-ir` has landed, so `layout_stub`
now drives the real IR types behind the same `round_trip` signature (done). IR
coordinates are f32 staff spaces, quantized only when serializing canonical

View File

@ -38,15 +38,17 @@ fn main() {
eprintln!("[1/8] canonical round-trip corpus: {iters} iters");
roundtrip::run_roundtrip_corpus(iters, 0x00C0_FFEE_1234_5678);
// 1b. Bundle manifest + reducer-bookkeeping serialization stability.
// (Full-Score *byte* round-trip is pending item 5's whole-score codec.)
eprintln!("[1b ] manifest + reducer-bookkeeping serialization stability");
// 1b. Bundle manifest + reducer-bookkeeping serialization + full-Score byte
// round-trip (item 5's whole-score codec, via reduce_onto).
eprintln!("[1b ] manifest + bookkeeping + full-Score serialization stability");
for seed in 0..n(64) {
roundtrip::assert_manifest_roundtrip(&roundtrip::committed_manifest(seed));
let mut rng = Rng::new(seed.wrapping_mul(0x0100_0193).wrapping_add(17));
roundtrip::assert_manifest_roundtrip(&generators::rich_manifest(&mut rng));
let session = generators::operation_envelopes(&mut rng, 40, 3, 6, 6);
roundtrip::assert_reduction_serialization_stable(&session, seed);
let (score, frontier) = convergence::materialized_score(seed.wrapping_add(101));
roundtrip::assert_score_serialization_stable(&score, &frontier, seed);
}
roundtrip::assert_content_mutation_changes_serialization();

View File

@ -281,6 +281,19 @@ pub fn run_graph_convergence(orders: usize, seed: u64) {
assert_graph_convergence(&base, &envelopes, &targets, orders, &mut rng);
}
/// Builds a two-voice base, authors a real ~50-bar edit session, reduces it onto
/// the base via [`OperationSet::reduce_onto`], and returns the materialized real
/// [`Score`] together with the causal frontier it covers. Used by the
/// full-Score serialization gate (criterion 4, whole-graph tier).
pub fn materialized_score(seed: u64) -> (Score, Vec<u8>) {
let base = two_voice_base(seed);
let mut rng = Rng::new(seed ^ 0x5C0E_5E51_A11A_B1E5);
let (_targets, envelopes) = crate::generators::graph_edit_session(&base, &mut rng);
let materialization = materialize_onto_in_order(&base, &envelopes);
let frontier = crate::generators::frontier_bytes(&envelopes);
(materialization.score, frontier)
}
#[cfg(test)]
mod tests {
use super::*;

View File

@ -84,9 +84,11 @@
//! Criterion 1's reducer-bookkeeping counterpart ([`convergence::assert_convergence`])
//! and criterion 4's bookkeeping-projection serialization
//! ([`roundtrip::assert_reduction_serialization_stable`]) are retained under
//! honest names. The full-`Score` **byte** round-trip is pending item 5 (Agent
//! B): no whole-score codec exists yet, so that one gate is marked `#[ignore]`
//! in `tests/acceptance.rs` rather than asserted falsely.
//! honest names. The full-`Score` **byte** round-trip
//! ([`roundtrip::assert_score_serialization_stable`]) is now live, driving Agent
//! B's whole-score codec ([`epiphany_core::Score::canonical_bytes`] /
//! [`epiphany_core::Score::decode_canonical`]) on a real `reduce_onto`
//! materialization through a bundle snapshot.
pub mod rng;

View File

@ -35,6 +35,7 @@ use epiphany_bundle::{
MemStore, ProfileId, ReductionAlgorithmVersion, SchemaVersion, SlotParse, SnapshotId,
SnapshotRef, StagedChunk, Superblock,
};
use epiphany_core::Score;
use epiphany_determinism::{CanonicalDecode, CanonicalEncode};
use epiphany_ops::{MaterializedState, OperationEnvelope, OperationSet};
@ -314,6 +315,87 @@ pub fn assert_reduction_serialization_stable(envelopes: &[OperationEnvelope], se
assert_manifest_roundtrip(reopened.manifest());
}
/// **Full-`Score` canonical serialization stability** (acceptance criterion 4,
/// the whole-graph tier — item 5's whole-score codec). The real
/// [`epiphany_core::Score`] encodes to canonical bytes, survives
/// content-addressed storage as a `Snapshot` chunk in a real bundle
/// (hash-verified on reopen), decodes back to an **equal** `Score`, and
/// re-encodes byte-identically. Unlike [`assert_reduction_serialization_stable`]
/// (which round-trips the Chapter 6 bookkeeping projection), this round-trips the
/// whole musical graph — the arena, voices, regions, cross-cutting, and
/// tombstones — through [`Score::canonical_bytes`] / [`Score::decode_canonical`].
///
/// `frontier` is the causal frontier the snapshot materializes (so the snapshot
/// reference is semantically consistent, not falsely empty).
pub fn assert_score_serialization_stable(score: &Score, frontier: &[u8], seed: u64) {
let canonical = score.canonical_bytes();
// Determinism: re-encoding the same score is byte-identical.
assert_eq!(
canonical,
score.canonical_bytes(),
"re-encoding the same score changed its bytes"
);
// serialize: stage the canonical score as a real Snapshot chunk referenced
// from the manifest's canonical_base.
let mut rng = Rng::new(seed);
let uuid = FileUuid(rng.array16());
let doc = DocumentId(rng.array16());
let mut bundle =
Bundle::create(MemStore::new(), uuid, Manifest::empty(doc)).expect("create bundle");
let snapshot = StagedChunk {
kind: ChunkKind::Snapshot,
schema_version: SchemaVersion::V0,
payload: canonical.clone(),
};
let frontier = frontier.to_vec();
bundle
.commit(&[snapshot], |ctx| {
let mut m = ctx.previous_manifest.clone();
let root = ctx.new_chunks[0];
let mut sid = [0u8; 16];
sid.copy_from_slice(&root.hash.as_bytes()[..16]);
m.canonical_base = Some(SnapshotRef {
snapshot_id: SnapshotId(sid),
covers_causal_frontier: FrontierBytes::from_bytes(frontier.clone()),
reduction_algorithm_version: ReductionAlgorithmVersion(0),
profile_id: ProfileId::Full,
hash: root.hash,
root,
});
m
})
.expect("commit snapshot");
let image = bundle.into_store().into_bytes();
// load: reopen, hash-verify, read back byte-identically.
let reopened = Bundle::open(MemStore::from_bytes(image)).expect("reopen bundle");
reopened
.verify_canonical_chunks()
.expect("canonical chunks intact");
let base = reopened
.manifest()
.canonical_base
.as_ref()
.expect("a canonical base");
let loaded = reopened
.read_chunk(&base.root)
.expect("read snapshot chunk back");
assert_eq!(
loaded, canonical,
"score bytes were not preserved through content-addressed storage"
);
// deserialize → equal → reserialize byte-identically.
let decoded = Score::decode_canonical(&loaded).expect("loaded score must decode");
assert_eq!(&decoded, score, "decoded score changed");
assert_eq!(
decoded.canonical_bytes(),
loaded,
"decoded score did not reserialize byte-identically"
);
}
/// Confirms criterion 4 is *musically sensitive* in the strongest form: a score
/// whose operations keep **identical identities and ordering metadata** but whose
/// payload *content* changes must reduce to **different** canonical bytes. This

View File

@ -153,22 +153,19 @@ fn full_score_materialization_is_reproducible() {
}
}
/// Criterion 4 (full-Score byte round-trip) — **pending item 5 (Agent B).** A
/// whole-`epiphany_core::Score` / `GraphMaterialization` `encode → decode →
/// re-encode` byte round-trip requires the whole-score canonical codec
/// (`CanonicalEncode`/`CanonicalDecode for Score`), which does not exist yet:
/// today only the bookkeeping `MaterializedState` and the A/B typed values have
/// codecs. This gate is intentionally `#[ignore]`'d (visible as *ignored*, never
/// falsely green) until item 5 lands the codec; then drop the attribute and
/// assert the real byte cycle through a bundle snapshot.
/// Criterion 4 (full-Score byte round-trip) — **the whole-graph tier (item 5).**
/// A real ~50-bar `epiphany_core::Score`, materialized through Agent C's
/// `reduce_onto`, is `encode → decode → re-encode`d byte-identically through the
/// whole-score canonical codec and a real bundle snapshot (hash-verified on
/// reopen). This is the honest full-Score serialization gate the bookkeeping
/// projection ([`reducer_bookkeeping_serialization`]) only approximated.
#[test]
#[ignore = "pending item 5 (Agent B): whole-score codec (CanonicalEncode/Decode for Score) does not exist yet"]
fn criterion_4_full_score_byte_roundtrip() {
unimplemented!(
"blocked on item 5: epiphany_core::Score has no canonical byte codec. \
When it lands, reduce_onto a base, encode the Score, decode, and assert \
a byte-identical re-encode through a real bundle snapshot."
);
for seed in 0..24u64 {
let seed = seed.wrapping_mul(0x9E37_79B9).wrapping_add(13);
let (score, frontier) = convergence::materialized_score(seed);
roundtrip::assert_score_serialization_stable(&score, &frontier, seed);
}
}
/// Criterion 5 — **Reduction determinism.** A randomized 1,000-envelope set,