Land whole-score codec (item 5) and flip the M3 full-Score gate green
Implements a total, reversible canonical byte form for the entire epiphany_core::Score graph, unblocking the byte-level full-Score serialization gate that M3 had to defer. epiphany-core/src/codec.rs: - Score::canonical_bytes() / Score::decode_canonical() with a validating ScoreDecodeError and a forward-only Reader cursor. - A local Codec trait with generic combinators (Option/Vec/BTreeSet/BTreeMap/ tuple) and macros (struct_codec!/cstyle_enum_codec!/unit_codec!/ catalog_id_codec!) so encode and decode stay symmetric across ~110 types spanning graph.rs, event.rs, pitch.rs, time.rs, tempo.rs. - Uniform form: LE integers, one discriminant byte per tagged union, u32 counts/length-prefixes, every variable-width leaf length-prefixed, raw UTF-8 for free text (so decode(encode(x)) == x for any valid score; catalog ids are already NFC). EventArena round-trips via iter_canonical + insert. - Two pub(crate) accessors added for the codec: EventOrderingDAG::edges_ref, SpellingPrecedence::order_ref. - Tests: generator-score corpus (valid_score + valid_score_rich), exotic event/pitch variants the generators omit, distinctness, and decoder rejection of trailing/truncated/empty bytes. epiphany-testkit: - roundtrip::assert_score_serialization_stable: encode the real Score, store it as a bundle Snapshot, reopen + hash-verify, decode to an equal Score, and assert a byte-identical re-encode. - convergence::materialized_score builds a real ~50-bar reduce_onto materialization for the gate. - criterion_4_full_score_byte_roundtrip flips from #[ignore] to a live gate; wired into the conformance suite. Docs (lib.rs, README, core DECISIONS P11-4) updated to reflect the landed codec. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
9b0d3e8e2c
commit
f5aaa96b11
|
|
@ -95,10 +95,25 @@ exist. To make round-trip serialization testable now (v0 acceptance criterion
|
|||
4), this crate defines a concrete canonical byte form for its primitives —
|
||||
notably `RationalTime` (sign + length-prefixed big-endian numerator and
|
||||
denominator magnitudes, always reduced) and the wall-clock integers
|
||||
(little-endian, matching `QuantizedCoord`). These are deterministic and
|
||||
reversible but provisional: when the Binary Format companion lands, reconcile
|
||||
this crate's `CanonicalEncode`/`CanonicalDecode` with it (a failing cross-crate
|
||||
round-trip test would be the trigger, per the QUICKSTART process notes).
|
||||
(little-endian, matching `QuantizedCoord`).
|
||||
|
||||
**M3 follow-up — the whole-score codec (item 5).** `src/codec.rs` now composes
|
||||
those primitives into a total, reversible canonical byte form for the *entire*
|
||||
`Score` graph (`Score::canonical_bytes` / `Score::decode_canonical`), so the
|
||||
materialized graph — not only the Chapter 6 `MaterializedState` bookkeeping —
|
||||
round-trips byte-identically (Agent F's `criterion_4_full_score_byte_roundtrip`
|
||||
drives it through a real bundle snapshot). The form is deliberately uniform:
|
||||
little-endian integers, a single discriminant byte per tagged union, `u32`
|
||||
counts/length-prefixes, every variable-width leaf length-prefixed, and raw
|
||||
(non-NFC-folded) UTF-8 for free-text fields so `decode(encode(x)) == x` for every
|
||||
valid score (catalog ids are already NFC at construction). The two private-field
|
||||
accessors the codec needs (`EventOrderingDAG::edges_ref`,
|
||||
`SpellingPrecedence::order_ref`) are `pub(crate)`.
|
||||
|
||||
These are deterministic and reversible but provisional: when the Binary Format
|
||||
companion lands, reconcile this crate's `CanonicalEncode`/`CanonicalDecode` and
|
||||
the whole-score `codec` with it (a failing cross-crate round-trip test would be
|
||||
the trigger, per the QUICKSTART process notes).
|
||||
|
||||
### P11-5 — Scope boundary: the Chapter 4 tuning catalog is referenced, not defined here
|
||||
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load Diff
|
|
@ -286,6 +286,14 @@ impl EventOrderingDAG {
|
|||
self.edges.get(&event).map(|v| v.as_slice()).unwrap_or(&[])
|
||||
}
|
||||
|
||||
/// The raw adjacency map, for the canonical codec (which must serialize the
|
||||
/// full ordering, not only the events reachable from a query).
|
||||
pub(crate) fn edges_ref(
|
||||
&self,
|
||||
) -> &std::collections::BTreeMap<crate::ids::EventId, Vec<crate::ids::EventId>> {
|
||||
&self.edges
|
||||
}
|
||||
|
||||
/// Every event the ordering names — DAG nodes (sources) and edge targets.
|
||||
/// Used by the invariant checker to confirm the ordering only references
|
||||
/// events that exist in the region (invariant 10).
|
||||
|
|
|
|||
|
|
@ -42,6 +42,7 @@
|
|||
//! event-arena storage via `slotmap` (decision 2), fully sync (decision 4),
|
||||
//! current stable Rust (decision 5). `unsafe` is forbidden crate-wide.
|
||||
|
||||
mod codec;
|
||||
mod event;
|
||||
mod graph;
|
||||
mod ids;
|
||||
|
|
@ -106,6 +107,8 @@ pub use tempo::{
|
|||
INVERSION_MAX_ITERATIONS, INVERSION_TOLERANCE_WHOLE_NOTES,
|
||||
};
|
||||
|
||||
pub use codec::ScoreDecodeError;
|
||||
|
||||
pub use indexes::ScoreIndexes;
|
||||
|
||||
pub use invariants::{check_invariant, check_invariants, GraphInvariant, InvariantViolation};
|
||||
|
|
|
|||
|
|
@ -643,6 +643,11 @@ impl SpellingPrecedence {
|
|||
Some(SpellingPrecedence { order })
|
||||
}
|
||||
|
||||
/// The precedence order, for the canonical codec.
|
||||
pub(crate) fn order_ref(&self) -> &[SpellingSourceKind] {
|
||||
&self.order
|
||||
}
|
||||
|
||||
/// The rank of a source kind: lower wins. `0` is the highest precedence.
|
||||
pub fn rank(&self, kind: SpellingSourceKind) -> usize {
|
||||
self.order
|
||||
|
|
|
|||
|
|
@ -107,13 +107,14 @@ Criterion 4 has three tiers — two asserted now, one pending item 5:
|
|||
structural equality across delivery orders) — the determinism precondition a
|
||||
byte codec depends on.
|
||||
|
||||
- **The full-`Score` byte round-trip is pending item 5 (Agent B).** No
|
||||
whole-score canonical codec (`CanonicalEncode`/`CanonicalDecode for Score`)
|
||||
exists yet, so `criterion_4_full_score_byte_roundtrip` is marked `#[ignore]`
|
||||
(visible as *ignored*, never falsely green) rather than asserted on the
|
||||
bookkeeping projection and passed off as a whole-Score gate. When item 5 lands
|
||||
the codec, drop the attribute and assert the real byte cycle through a bundle
|
||||
snapshot.
|
||||
- **The full-`Score` byte round-trip** (`criterion_4_full_score_byte_roundtrip`,
|
||||
via `assert_score_serialization_stable`): item 5's whole-score codec
|
||||
(`epiphany_core::Score::canonical_bytes` / `decode_canonical`) has landed, so a
|
||||
real ~50-bar `Score` — materialized through Agent C's `reduce_onto` — now
|
||||
`encode → decode → re-encode`s byte-identically through a real bundle snapshot
|
||||
(hash-verified on reopen), with the decoded `Score` structurally equal to the
|
||||
original. This is the whole musical graph (arena, voices, regions,
|
||||
cross-cutting, tombstones), not the bookkeeping projection.
|
||||
|
||||
## Decisions (per QUICKSTART "Make each one once and document it")
|
||||
|
||||
|
|
@ -132,14 +133,13 @@ Criterion 4 has three tiers — two asserted now, one pending item 5:
|
|||
|
||||
Per the QUICKSTART, implementation-discovered gaps are batched, not improvised:
|
||||
|
||||
- **Whole-graph (`epiphany_core::Score`) wire format — pending item 5 (Agent B).**
|
||||
Criterion 4 is a real decode round-trip at the canonical Chapter-6
|
||||
`MaterializedState` layer; the materialized `Score` is shown *reproducible*
|
||||
today. A direct canonical byte codec for the richer core `Score` does not exist
|
||||
yet (it is item 5's "whole-score codec", to be reconciled with the Binary
|
||||
Format companion), so the whole-`Score` byte round-trip
|
||||
(`criterion_4_full_score_byte_roundtrip`) is an explicit `#[ignore]`'d gate
|
||||
rather than a falsely-green assertion.
|
||||
- **Whole-graph (`epiphany_core::Score`) wire format — landed (item 5).** A
|
||||
direct canonical byte codec for the core `Score` now exists
|
||||
(`epiphany_core::Score::canonical_bytes` / `decode_canonical`), and
|
||||
`criterion_4_full_score_byte_roundtrip` exercises it on a real `reduce_onto`
|
||||
materialization through a bundle snapshot. The prototype byte form predates the
|
||||
Binary Format companion specification and is to be reconciled with it (see
|
||||
`epiphany-core/DECISIONS.md`, P11-4).
|
||||
- **Layout harness re-pointed.** `epiphany-layout-ir` has landed, so `layout_stub`
|
||||
now drives the real IR types behind the same `round_trip` signature (done). IR
|
||||
coordinates are f32 staff spaces, quantized only when serializing canonical
|
||||
|
|
|
|||
|
|
@ -38,15 +38,17 @@ fn main() {
|
|||
eprintln!("[1/8] canonical round-trip corpus: {iters} iters");
|
||||
roundtrip::run_roundtrip_corpus(iters, 0x00C0_FFEE_1234_5678);
|
||||
|
||||
// 1b. Bundle manifest + reducer-bookkeeping serialization stability.
|
||||
// (Full-Score *byte* round-trip is pending item 5's whole-score codec.)
|
||||
eprintln!("[1b ] manifest + reducer-bookkeeping serialization stability");
|
||||
// 1b. Bundle manifest + reducer-bookkeeping serialization + full-Score byte
|
||||
// round-trip (item 5's whole-score codec, via reduce_onto).
|
||||
eprintln!("[1b ] manifest + bookkeeping + full-Score serialization stability");
|
||||
for seed in 0..n(64) {
|
||||
roundtrip::assert_manifest_roundtrip(&roundtrip::committed_manifest(seed));
|
||||
let mut rng = Rng::new(seed.wrapping_mul(0x0100_0193).wrapping_add(17));
|
||||
roundtrip::assert_manifest_roundtrip(&generators::rich_manifest(&mut rng));
|
||||
let session = generators::operation_envelopes(&mut rng, 40, 3, 6, 6);
|
||||
roundtrip::assert_reduction_serialization_stable(&session, seed);
|
||||
let (score, frontier) = convergence::materialized_score(seed.wrapping_add(101));
|
||||
roundtrip::assert_score_serialization_stable(&score, &frontier, seed);
|
||||
}
|
||||
roundtrip::assert_content_mutation_changes_serialization();
|
||||
|
||||
|
|
|
|||
|
|
@ -281,6 +281,19 @@ pub fn run_graph_convergence(orders: usize, seed: u64) {
|
|||
assert_graph_convergence(&base, &envelopes, &targets, orders, &mut rng);
|
||||
}
|
||||
|
||||
/// Builds a two-voice base, authors a real ~50-bar edit session, reduces it onto
|
||||
/// the base via [`OperationSet::reduce_onto`], and returns the materialized real
|
||||
/// [`Score`] together with the causal frontier it covers. Used by the
|
||||
/// full-Score serialization gate (criterion 4, whole-graph tier).
|
||||
pub fn materialized_score(seed: u64) -> (Score, Vec<u8>) {
|
||||
let base = two_voice_base(seed);
|
||||
let mut rng = Rng::new(seed ^ 0x5C0E_5E51_A11A_B1E5);
|
||||
let (_targets, envelopes) = crate::generators::graph_edit_session(&base, &mut rng);
|
||||
let materialization = materialize_onto_in_order(&base, &envelopes);
|
||||
let frontier = crate::generators::frontier_bytes(&envelopes);
|
||||
(materialization.score, frontier)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
|
|
|||
|
|
@ -84,9 +84,11 @@
|
|||
//! Criterion 1's reducer-bookkeeping counterpart ([`convergence::assert_convergence`])
|
||||
//! and criterion 4's bookkeeping-projection serialization
|
||||
//! ([`roundtrip::assert_reduction_serialization_stable`]) are retained under
|
||||
//! honest names. The full-`Score` **byte** round-trip is pending item 5 (Agent
|
||||
//! B): no whole-score codec exists yet, so that one gate is marked `#[ignore]`
|
||||
//! in `tests/acceptance.rs` rather than asserted falsely.
|
||||
//! honest names. The full-`Score` **byte** round-trip
|
||||
//! ([`roundtrip::assert_score_serialization_stable`]) is now live, driving Agent
|
||||
//! B's whole-score codec ([`epiphany_core::Score::canonical_bytes`] /
|
||||
//! [`epiphany_core::Score::decode_canonical`]) on a real `reduce_onto`
|
||||
//! materialization through a bundle snapshot.
|
||||
|
||||
pub mod rng;
|
||||
|
||||
|
|
|
|||
|
|
@ -35,6 +35,7 @@ use epiphany_bundle::{
|
|||
MemStore, ProfileId, ReductionAlgorithmVersion, SchemaVersion, SlotParse, SnapshotId,
|
||||
SnapshotRef, StagedChunk, Superblock,
|
||||
};
|
||||
use epiphany_core::Score;
|
||||
use epiphany_determinism::{CanonicalDecode, CanonicalEncode};
|
||||
use epiphany_ops::{MaterializedState, OperationEnvelope, OperationSet};
|
||||
|
||||
|
|
@ -314,6 +315,87 @@ pub fn assert_reduction_serialization_stable(envelopes: &[OperationEnvelope], se
|
|||
assert_manifest_roundtrip(reopened.manifest());
|
||||
}
|
||||
|
||||
/// **Full-`Score` canonical serialization stability** (acceptance criterion 4,
|
||||
/// the whole-graph tier — item 5's whole-score codec). The real
|
||||
/// [`epiphany_core::Score`] encodes to canonical bytes, survives
|
||||
/// content-addressed storage as a `Snapshot` chunk in a real bundle
|
||||
/// (hash-verified on reopen), decodes back to an **equal** `Score`, and
|
||||
/// re-encodes byte-identically. Unlike [`assert_reduction_serialization_stable`]
|
||||
/// (which round-trips the Chapter 6 bookkeeping projection), this round-trips the
|
||||
/// whole musical graph — the arena, voices, regions, cross-cutting, and
|
||||
/// tombstones — through [`Score::canonical_bytes`] / [`Score::decode_canonical`].
|
||||
///
|
||||
/// `frontier` is the causal frontier the snapshot materializes (so the snapshot
|
||||
/// reference is semantically consistent, not falsely empty).
|
||||
pub fn assert_score_serialization_stable(score: &Score, frontier: &[u8], seed: u64) {
|
||||
let canonical = score.canonical_bytes();
|
||||
// Determinism: re-encoding the same score is byte-identical.
|
||||
assert_eq!(
|
||||
canonical,
|
||||
score.canonical_bytes(),
|
||||
"re-encoding the same score changed its bytes"
|
||||
);
|
||||
|
||||
// serialize: stage the canonical score as a real Snapshot chunk referenced
|
||||
// from the manifest's canonical_base.
|
||||
let mut rng = Rng::new(seed);
|
||||
let uuid = FileUuid(rng.array16());
|
||||
let doc = DocumentId(rng.array16());
|
||||
let mut bundle =
|
||||
Bundle::create(MemStore::new(), uuid, Manifest::empty(doc)).expect("create bundle");
|
||||
let snapshot = StagedChunk {
|
||||
kind: ChunkKind::Snapshot,
|
||||
schema_version: SchemaVersion::V0,
|
||||
payload: canonical.clone(),
|
||||
};
|
||||
let frontier = frontier.to_vec();
|
||||
bundle
|
||||
.commit(&[snapshot], |ctx| {
|
||||
let mut m = ctx.previous_manifest.clone();
|
||||
let root = ctx.new_chunks[0];
|
||||
let mut sid = [0u8; 16];
|
||||
sid.copy_from_slice(&root.hash.as_bytes()[..16]);
|
||||
m.canonical_base = Some(SnapshotRef {
|
||||
snapshot_id: SnapshotId(sid),
|
||||
covers_causal_frontier: FrontierBytes::from_bytes(frontier.clone()),
|
||||
reduction_algorithm_version: ReductionAlgorithmVersion(0),
|
||||
profile_id: ProfileId::Full,
|
||||
hash: root.hash,
|
||||
root,
|
||||
});
|
||||
m
|
||||
})
|
||||
.expect("commit snapshot");
|
||||
let image = bundle.into_store().into_bytes();
|
||||
|
||||
// load: reopen, hash-verify, read back byte-identically.
|
||||
let reopened = Bundle::open(MemStore::from_bytes(image)).expect("reopen bundle");
|
||||
reopened
|
||||
.verify_canonical_chunks()
|
||||
.expect("canonical chunks intact");
|
||||
let base = reopened
|
||||
.manifest()
|
||||
.canonical_base
|
||||
.as_ref()
|
||||
.expect("a canonical base");
|
||||
let loaded = reopened
|
||||
.read_chunk(&base.root)
|
||||
.expect("read snapshot chunk back");
|
||||
assert_eq!(
|
||||
loaded, canonical,
|
||||
"score bytes were not preserved through content-addressed storage"
|
||||
);
|
||||
|
||||
// deserialize → equal → reserialize byte-identically.
|
||||
let decoded = Score::decode_canonical(&loaded).expect("loaded score must decode");
|
||||
assert_eq!(&decoded, score, "decoded score changed");
|
||||
assert_eq!(
|
||||
decoded.canonical_bytes(),
|
||||
loaded,
|
||||
"decoded score did not reserialize byte-identically"
|
||||
);
|
||||
}
|
||||
|
||||
/// Confirms criterion 4 is *musically sensitive* in the strongest form: a score
|
||||
/// whose operations keep **identical identities and ordering metadata** but whose
|
||||
/// payload *content* changes must reduce to **different** canonical bytes. This
|
||||
|
|
|
|||
|
|
@ -153,22 +153,19 @@ fn full_score_materialization_is_reproducible() {
|
|||
}
|
||||
}
|
||||
|
||||
/// Criterion 4 (full-Score byte round-trip) — **pending item 5 (Agent B).** A
|
||||
/// whole-`epiphany_core::Score` / `GraphMaterialization` `encode → decode →
|
||||
/// re-encode` byte round-trip requires the whole-score canonical codec
|
||||
/// (`CanonicalEncode`/`CanonicalDecode for Score`), which does not exist yet:
|
||||
/// today only the bookkeeping `MaterializedState` and the A/B typed values have
|
||||
/// codecs. This gate is intentionally `#[ignore]`'d (visible as *ignored*, never
|
||||
/// falsely green) until item 5 lands the codec; then drop the attribute and
|
||||
/// assert the real byte cycle through a bundle snapshot.
|
||||
/// Criterion 4 (full-Score byte round-trip) — **the whole-graph tier (item 5).**
|
||||
/// A real ~50-bar `epiphany_core::Score`, materialized through Agent C's
|
||||
/// `reduce_onto`, is `encode → decode → re-encode`d byte-identically through the
|
||||
/// whole-score canonical codec and a real bundle snapshot (hash-verified on
|
||||
/// reopen). This is the honest full-Score serialization gate the bookkeeping
|
||||
/// projection ([`reducer_bookkeeping_serialization`]) only approximated.
|
||||
#[test]
|
||||
#[ignore = "pending item 5 (Agent B): whole-score codec (CanonicalEncode/Decode for Score) does not exist yet"]
|
||||
fn criterion_4_full_score_byte_roundtrip() {
|
||||
unimplemented!(
|
||||
"blocked on item 5: epiphany_core::Score has no canonical byte codec. \
|
||||
When it lands, reduce_onto a base, encode the Score, decode, and assert \
|
||||
a byte-identical re-encode through a real bundle snapshot."
|
||||
);
|
||||
for seed in 0..24u64 {
|
||||
let seed = seed.wrapping_mul(0x9E37_79B9).wrapping_add(13);
|
||||
let (score, frontier) = convergence::materialized_score(seed);
|
||||
roundtrip::assert_score_serialization_stable(&score, &frontier, seed);
|
||||
}
|
||||
}
|
||||
|
||||
/// Criterion 5 — **Reduction determinism.** A randomized 1,000-envelope set,
|
||||
|
|
|
|||
Loading…
Reference in New Issue