Text Projection: the operation layer, and the tests that can see it fail

Projection and strict parse for the whole Chapter-6 operation vocabulary: the 31
kind productions, the envelope with its stamp and causal context, the four
payload variants, and the sub-vocabularies for actions, undo policies, tuplet
compensation, cross-cutting values and position remapping. Written against the
grammar, per `req:textproj:operation-vocabulary`, calling `TextValue` only where a
production says `value`.

Names are generated, never spelled: `OperationKindTag::catalog_name()` comes from
the same vocabulary macro as the wire discriminant and the decoder, and `parse`
dispatches through an exhaustive match, so a kind added to the vocabulary and not
to the projector fails to compile.

Six sequences are order-constrained because their encoders normalize. Both halves
of each are enforced and both halves are tested, which turned out to matter. The
rejecting half was straightforward -- `TransposeOp.targets` mirrors the frozen
multiset exactly, rejecting a strict decrease while accepting a duplicate, and
getting that backwards would silently break a frozen operation's replay. The
*normalizing* half was written correctly by every agent and tested by none: every
fixture was already sorted, so all five outbound sorts survived deletion with the
suite green. The consequence was real -- with one removed, the projector emits
descending targets that its own parser then rejects, and that disagree with the
canonical bytes.

`textproj_conformance.rs` closes that. Each of the five builds a value unsorted in
memory, asserts the fixture pair really is descending before relying on it, then
checks the projection sorts, parses, and matches what encode-then-decode produces
-- pinning text and bytes to one normalization rather than merely to each other.

It also carries a structural injectivity sweep: 3319 mutants of projected
envelopes, of which 259 parse, all re-projecting byte-identically. That is
`req:textproj:roundtrip`'s second equation, and it is the evidence that no
whole-line re-project guard is needed -- every path that could normalize is
pre-empted by a per-site check. The sweep asserts its own reach, because one that
rejected everything would prove nothing. Deleting any per-site order check makes
it fail independently of the dedicated test.

And it locks the companion's worked example byte-for-byte. It was correct and
nothing kept it so, which is how a "machine-checked" claim became true of one run
and of nothing durable at 0.3.0.

All seven checks mutation-verified, each killed by exactly its own named test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Levi Neuwirth 2026-07-21 13:43:09 -04:00
parent 4ded63a583
commit fdb4a57885
9 changed files with 2389 additions and 2 deletions

View File

@ -653,7 +653,7 @@ pub fn decode_envelope(bytes: &[u8]) -> Result<OperationEnvelope> {
}
#[cfg(test)]
mod tests {
pub(crate) mod tests {
use super::*;
use crate::valuegen;
use epiphany_core::{BeamId, MusicalDuration, RationalTime, SlurId, StaffId, TimeSignatureId};
@ -679,7 +679,7 @@ mod tests {
/// `OperationKindTag`**, so a new kind cannot be added without giving the
/// round-trip a sample of it — which is the compile-time half of the same
/// guarantee `operation_kind_tag_vocabulary!` gives the decoder.
fn sample_kind(tag: OperationKindTag) -> OperationKind {
pub(crate) fn sample_kind(tag: OperationKindTag) -> OperationKind {
match tag {
OperationKindTag::InsertEvent => OperationKind::InsertEvent(InsertEventOp {
staff_instance: si(),

View File

@ -92,6 +92,11 @@ mod reduce;
mod slot;
mod stamp;
mod support;
#[cfg(test)]
mod textproj_conformance;
mod textproj_envelope;
mod textproj_kind;
mod textproj_leaf;
mod v0;
mod validate;
pub mod valuegen;
@ -140,6 +145,7 @@ pub use support::{
RepairKindRegistryId, ReplicaAnomalyRegistryId, ResolutionRegistryId,
SerializedCanonicalInputs,
};
pub use textproj_envelope::{parse_envelope, project_envelope};
pub use undo::{UndoPolicy, UndoTransactionPayload};
pub use v0::V0OperationEnvelope;

View File

@ -49,6 +49,32 @@ macro_rules! registry_id {
}
}
impl CanonicalByteOrder for $name {}
/// Projects this opaque registry identifier as its canonical 16 bytes.
impl epiphany_core::textvalue::TextValue for $name {
fn project(&self) -> epiphany_core::textvalue::Sexp {
epiphany_core::textvalue::Sexp::Bytes(self.canonical_bytes().to_vec())
}
fn parse(
s: &epiphany_core::textvalue::Sexp,
) -> Result<Self, epiphany_core::textvalue::TextError> {
let epiphany_core::textvalue::Sexp::Bytes(bytes) = s else {
return Err(epiphany_core::textvalue::TextError::Expected {
expected: stringify!($name),
found: crate::textproj_leaf::class_of(s),
});
};
let bytes: [u8; 16] = bytes.as_slice().try_into().map_err(|_| {
epiphany_core::textvalue::TextError::NotCanonical(concat!(
"a ",
stringify!($name),
" is exactly 16 bytes"
))
})?;
Ok(Self(u128::from_be_bytes(bytes)))
}
}
};
}

View File

@ -0,0 +1,288 @@
//! Cross-layer conformance tests for operation text projection.
//!
//! These tests deliberately join the text projector, strict parser, and binary
//! decoder. Keeping them together avoids duplicating full-envelope fixtures in
//! the leaf and kind projection modules.
use std::collections::BTreeSet;
use epiphany_core::textvalue::{read_sexp, Sexp, TextValue};
use epiphany_core::{
EventId, MusicalPosition, OperationId, PitchId, RationalTime, ReplicaId, TransactionId,
TranspositionInterval, TupletId, WallClockTime,
};
use epiphany_determinism::CanonicalEncode;
use crate::causal::CausalContext;
use crate::envdecode::{decode_envelope, tests::sample_kind};
use crate::envelope::OperationEnvelope;
use crate::payload::{
OperationKind, OperationKindTag, OperationPayload, PositionRemapping, TransposeIntervalOp,
};
use crate::stamp::{HybridLogicalClock, OperationStamp};
use crate::support::AuthorId;
use crate::textproj_envelope::{parse_envelope, project_envelope};
use crate::TupletCompensation;
fn envelope(kind: OperationKind) -> OperationEnvelope {
let id = OperationId::new(ReplicaId(7), 1);
OperationEnvelope {
id,
author: AuthorId(0x1122_3344),
stamp: OperationStamp::new(HybridLogicalClock::new(WallClockTime(42), 7), id),
causal_context: CausalContext::new()
.with_seen(ReplicaId(1), 3)
.with_seen(ReplicaId(2), 5)
.with_dot(OperationId::new(ReplicaId(3), 9))
.with_dot(OperationId::new(ReplicaId(4), 11)),
transaction: Some(TransactionId::new(ReplicaId(7), 5)),
payload: OperationPayload::Primitive(kind),
}
}
fn projected_kind(text: &str) -> Sexp {
let tree = read_sexp(text).expect("the projector emits one valid s-expression");
let Sexp::List(envelope) = tree else {
panic!("an envelope projection is a list")
};
let Sexp::List(payload) = &envelope[6] else {
panic!("the envelope payload is a list")
};
assert_eq!(payload[0].as_symbol(), Some("primitive"));
payload[1].clone()
}
fn list(s: &Sexp) -> &[Sexp] {
let Sexp::List(items) = s else {
panic!("the selected production field is a list")
};
items
}
fn assert_parses_and_matches_binary(env: &OperationEnvelope, text: &str) {
parse_envelope(text).expect("the projector must emit text its strict parser accepts");
let via_binary =
decode_envelope(&env.to_canonical_bytes()).expect("the canonical binary form decodes");
assert_eq!(
project_envelope(&via_binary),
text,
"text and binary projection must apply identical normalization"
);
}
fn position(n: i32) -> MusicalPosition {
MusicalPosition(RationalTime::from_int(n))
}
#[test]
fn transpose_targets_are_normalized_on_projection() {
let high = PitchId::new(ReplicaId(9), 900);
let low = PitchId::new(ReplicaId(9), 2);
assert!(high.to_canonical_bytes() > low.to_canonical_bytes());
let mut kind = sample_kind(OperationKindTag::Transpose);
let OperationKind::Transpose(op) = &mut kind else {
panic!("the exhaustive sample matches its requested tag")
};
op.targets = vec![high, low];
let env = envelope(kind);
let text = project_envelope(&env);
let kind = projected_kind(&text);
assert_eq!(list(&kind)[1], vec![low, high].project());
assert_parses_and_matches_binary(&env, &text);
}
#[test]
fn declared_incompatible_events_are_normalized_on_projection() {
let high = EventId::new(ReplicaId(9), 900);
let low = EventId::new(ReplicaId(9), 2);
assert!(high.to_canonical_bytes() > low.to_canonical_bytes());
let mut kind = sample_kind(OperationKindTag::ChangeRegionTimeModel);
let OperationKind::ChangeRegionTimeModel(op) = &mut kind else {
panic!("the exhaustive sample matches its requested tag")
};
op.declared_incompatible = vec![high, low];
let env = envelope(kind);
let text = project_envelope(&env);
let kind = projected_kind(&text);
assert_eq!(list(&kind)[3], vec![low, high].project());
assert_parses_and_matches_binary(&env, &text);
}
#[test]
fn rewrite_tuplets_are_normalized_on_projection() {
let high = TupletId::new(ReplicaId(9), 900);
let low = TupletId::new(ReplicaId(9), 2);
assert!(high.to_canonical_bytes() > low.to_canonical_bytes());
let mut kind = sample_kind(OperationKindTag::DeleteEvent);
let OperationKind::DeleteEvent(op) = &mut kind else {
panic!("the exhaustive sample matches its requested tag")
};
op.tuplet_compensation = TupletCompensation::RewriteTuplets {
tuplets: vec![high, low],
};
let env = envelope(kind);
let text = project_envelope(&env);
let kind = projected_kind(&text);
let compensation = list(&kind)[2].clone();
assert_eq!(list(&compensation)[1], vec![low, high].project());
assert_parses_and_matches_binary(&env, &text);
}
#[test]
fn cascade_delete_tuplets_are_normalized_on_projection() {
let high = TupletId::new(ReplicaId(9), 900);
let low = TupletId::new(ReplicaId(9), 2);
assert!(high.to_canonical_bytes() > low.to_canonical_bytes());
let mut kind = sample_kind(OperationKindTag::DeleteEvent);
let OperationKind::DeleteEvent(op) = &mut kind else {
panic!("the exhaustive sample matches its requested tag")
};
op.tuplet_compensation = TupletCompensation::CascadeDeleteTuplets {
tuplets: vec![high, low],
};
let env = envelope(kind);
let text = project_envelope(&env);
let kind = projected_kind(&text);
let compensation = list(&kind)[2].clone();
assert_eq!(list(&compensation)[1], vec![low, high].project());
assert_parses_and_matches_binary(&env, &text);
}
#[test]
fn remapping_entries_are_normalized_on_projection() {
let high = EventId::new(ReplicaId(9), 900);
let low = EventId::new(ReplicaId(9), 2);
assert!(high.to_canonical_bytes() > low.to_canonical_bytes());
let mut kind = sample_kind(OperationKindTag::ChangeRegionTimeModel);
let OperationKind::ChangeRegionTimeModel(op) = &mut kind else {
panic!("the exhaustive sample matches its requested tag")
};
op.remapping = PositionRemapping::Reassign(vec![(high, position(9)), (low, position(2))]);
let env = envelope(kind);
let text = project_envelope(&env);
let kind = projected_kind(&text);
let remapping = list(&kind)[4].clone();
assert_eq!(
list(&remapping)[1],
vec![(low, position(2)), (high, position(9))].project()
);
assert_parses_and_matches_binary(&env, &text);
}
#[test]
fn worked_example_envelope_is_byte_exact() {
let id = OperationId::new(ReplicaId(7), 1);
let env = OperationEnvelope {
id,
author: AuthorId(0x1122_3344),
stamp: OperationStamp::new(HybridLogicalClock::new(WallClockTime(42), 7), id),
causal_context: CausalContext::new()
.with_seen(ReplicaId(1), 3)
.with_dot(OperationId::new(ReplicaId(2), 9)),
transaction: Some(TransactionId::new(ReplicaId(7), 5)),
payload: OperationPayload::Primitive(OperationKind::TransposeInterval(
TransposeIntervalOp {
targets: BTreeSet::from([
PitchId::new(ReplicaId(7), 1),
PitchId::new(ReplicaId(7), 2),
]),
interval: TranspositionInterval {
diatonic_steps: 4,
chromatic_steps: 7,
},
},
)),
};
let spec = "(envelope #x00000000000000070000000000000001 #x00000000000000000000000011223344 (stamp 42 7 #x00000000000000070000000000000001) (causal ((#x0000000000000001 3)) (#x00000000000000020000000000000009)) (some #x00000000000000070000000000000005) (primitive (transpose-interval (#x00000000000000070000000000000001 #x00000000000000070000000000000002) (transposition-interval 4 7))))";
assert_eq!(
project_envelope(&env),
spec,
"the companion's worked envelope must remain byte-exact"
);
}
fn mutants(s: &Sexp, out: &mut Vec<Sexp>) {
if let Sexp::List(items) = s {
for index in 0..items.len().saturating_sub(1) {
let mut mutant = items.clone();
mutant.swap(index, index + 1);
out.push(Sexp::List(mutant));
}
for index in 0..items.len() {
let mut mutant = items.clone();
mutant.insert(index, items[index].clone());
out.push(Sexp::List(mutant));
}
for index in 0..items.len() {
let mut mutant = items.clone();
mutant.remove(index);
out.push(Sexp::List(mutant));
}
for (index, child) in items.iter().enumerate() {
let mut child_mutants = Vec::new();
mutants(child, &mut child_mutants);
for child_mutant in child_mutants {
let mut mutant = items.clone();
mutant[index] = child_mutant;
out.push(Sexp::List(mutant));
}
}
}
}
#[test]
fn every_accepted_structural_mutant_reprojects_byte_exactly() {
let mut checked = 0usize;
let mut accepted = 0usize;
let mut violations = Vec::new();
for tag in OperationKindTag::PAYLOAD_FREE {
let env = envelope(sample_kind(*tag));
let text = project_envelope(&env);
assert_eq!(
project_envelope(&parse_envelope(&text).expect("canonical text parses")),
text
);
let tree = read_sexp(&text).expect("canonical text is one s-expression");
let mut structural_mutants = Vec::new();
mutants(&tree, &mut structural_mutants);
for mutant in structural_mutants {
let mutant_text = mutant.render();
checked += 1;
if let Ok(parsed) = parse_envelope(&mutant_text) {
accepted += 1;
let reprojected = project_envelope(&parsed);
if reprojected != mutant_text {
violations.push((mutant_text, reprojected));
}
}
}
}
println!(
"mutants={checked} accepted={accepted} violations={}",
violations.len()
);
assert!(checked > 3_000, "the sweep must retain structural reach");
assert!(
accepted > 100,
"the sweep must exercise successful parses, not only rejection; accepted {accepted}"
);
assert!(
violations.is_empty(),
"{} accepted mutants re-projected differently; first violation: {:?}",
violations.len(),
violations.first()
);
}

View File

@ -0,0 +1,368 @@
//! Grammar-directed text projection for operation envelopes and their
//! envelope-level productions.
use std::collections::{BTreeMap, BTreeSet};
use epiphany_core::textvalue::{read_sexp, Sexp, TextError, TextValue};
use epiphany_core::{OperationId, ReplicaId};
use crate::causal::CausalContext;
use crate::envelope::OperationEnvelope;
use crate::payload::{
OperationKind, OperationPayload, ResolveConflictPayload, ResolveEquivocationPayload,
};
use crate::stamp::{HybridLogicalClock, OperationStamp};
use crate::undo::UndoTransactionPayload;
fn class_of(s: &Sexp) -> &'static str {
match s {
Sexp::List(_) => "list",
Sexp::Symbol(_) => "symbol",
Sexp::Int(_) => "integer",
Sexp::Bytes(_) => "byte string",
Sexp::Str(_) => "string",
}
}
fn split_production(s: &Sexp) -> Result<(&str, &[Sexp]), TextError> {
let items = s.as_list().ok_or(TextError::Expected {
expected: "production",
found: class_of(s),
})?;
let constructor = items
.first()
.and_then(Sexp::as_symbol)
.ok_or(TextError::Syntax(
"a production is headed by its constructor",
))?;
Ok((constructor, &items[1..]))
}
fn expect_fields<'a>(
fields: &'a [Sexp],
type_name: &'static str,
expected: usize,
) -> Result<&'a [Sexp], TextError> {
if fields.len() != expected {
return Err(TextError::Arity {
type_name,
expected,
found: fields.len(),
});
}
Ok(fields)
}
/// The grammar's flattened `(stamp physical-time logical-counter id)`
/// production.
impl TextValue for OperationStamp {
fn project(&self) -> Sexp {
Sexp::List(vec![
Sexp::sym("stamp"),
self.hlc.physical_time.project(),
self.hlc.logical_counter.project(),
self.id.project(),
])
}
fn parse(s: &Sexp) -> Result<Self, TextError> {
let fields = s.expect_struct("stamp", 3)?;
Ok(OperationStamp::new(
HybridLogicalClock::new(
epiphany_core::WallClockTime::parse(&fields[0])?,
u32::parse(&fields[1])?,
),
OperationId::parse(&fields[2])?,
))
}
}
/// The grammar's dotted-version-vector `(causal (replica-seen*) (bytes*))`
/// production.
impl TextValue for CausalContext {
fn project(&self) -> Sexp {
Sexp::List(vec![
Sexp::sym("causal"),
self.vector.project(),
self.dots.project(),
])
}
fn parse(s: &Sexp) -> Result<Self, TextError> {
let fields = s.expect_struct("causal", 2)?;
// Parse through the core collection implementations: they reject a
// duplicate or decrease before BTreeMap/BTreeSet can normalize it.
Ok(CausalContext {
vector: BTreeMap::<ReplicaId, u64>::parse(&fields[0])?,
dots: BTreeSet::<OperationId>::parse(&fields[1])?,
})
}
}
/// The grammar's four operation-payload productions, with their records inlined.
impl TextValue for OperationPayload {
fn project(&self) -> Sexp {
match self {
OperationPayload::Primitive(kind) => {
Sexp::List(vec![Sexp::sym("primitive"), kind.project()])
}
OperationPayload::ResolveConflict(payload) => Sexp::List(vec![
Sexp::sym("resolve-conflict"),
payload.target.project(),
payload.action.project(),
]),
OperationPayload::UndoTransaction(payload) => Sexp::List(vec![
Sexp::sym("undo"),
payload.target.project(),
payload.policy.project(),
]),
OperationPayload::ResolveEquivocation(payload) => Sexp::List(vec![
Sexp::sym("resolve-equivocation"),
payload.target.project(),
payload.chosen.project(),
]),
}
}
fn parse(s: &Sexp) -> Result<Self, TextError> {
let (constructor, fields) = split_production(s)?;
match constructor {
"primitive" => {
let fields = expect_fields(fields, "OperationPayload::Primitive", 1)?;
Ok(OperationPayload::Primitive(OperationKind::parse(
&fields[0],
)?))
}
"resolve-conflict" => {
let fields = expect_fields(fields, "OperationPayload::ResolveConflict", 2)?;
Ok(OperationPayload::ResolveConflict(ResolveConflictPayload {
target: TextValue::parse(&fields[0])?,
action: TextValue::parse(&fields[1])?,
}))
}
"undo" => {
let fields = expect_fields(fields, "OperationPayload::UndoTransaction", 2)?;
Ok(OperationPayload::UndoTransaction(UndoTransactionPayload {
target: TextValue::parse(&fields[0])?,
policy: TextValue::parse(&fields[1])?,
}))
}
"resolve-equivocation" => {
let fields = expect_fields(fields, "OperationPayload::ResolveEquivocation", 2)?;
Ok(OperationPayload::ResolveEquivocation(
ResolveEquivocationPayload {
target: TextValue::parse(&fields[0])?,
chosen: TextValue::parse(&fields[1])?,
},
))
}
found => Err(TextError::UnknownConstructor {
type_name: "OperationPayload",
found: found.to_owned(),
}),
}
}
}
/// The grammar's complete operation-envelope production.
impl TextValue for OperationEnvelope {
fn project(&self) -> Sexp {
Sexp::List(vec![
Sexp::sym("envelope"),
self.id.project(),
self.author.project(),
self.stamp.project(),
self.causal_context.project(),
self.transaction.project(),
self.payload.project(),
])
}
fn parse(s: &Sexp) -> Result<Self, TextError> {
let fields = s.expect_struct("envelope", 6)?;
Ok(OperationEnvelope {
id: TextValue::parse(&fields[0])?,
author: TextValue::parse(&fields[1])?,
stamp: TextValue::parse(&fields[2])?,
causal_context: TextValue::parse(&fields[3])?,
transaction: TextValue::parse(&fields[4])?,
payload: TextValue::parse(&fields[5])?,
})
}
}
/// Projects one envelope as its canonical single-line s-expression.
///
/// The returned string deliberately has no trailing LF; the enclosing document
/// projection owns line separators.
pub fn project_envelope(envelope: &OperationEnvelope) -> String {
envelope.project().render()
}
/// Parses one complete canonical envelope line.
///
/// Strictness is enforced where information could otherwise be lost: the
/// reader rejects alternate lexical spellings, and the nested collection
/// parsers reject disorder and duplicates before constructing ordered maps or
/// sets. A whole-envelope re-project guard was mutation-tested and removed
/// because every accepted parse already preserves its input exactly.
pub fn parse_envelope(input: &str) -> Result<OperationEnvelope, TextError> {
OperationEnvelope::parse(&read_sexp(input)?)
}
#[cfg(test)]
mod tests {
use core::fmt::Debug;
use epiphany_core::textvalue::{read_sexp, TextValue};
use epiphany_core::{
EventId, OperationId, ReplicaId, TransactionId, TypedObjectId, WallClockTime,
};
use super::{parse_envelope, project_envelope};
use crate::causal::CausalContext;
use crate::conflict::{ConflictId, ResolutionAction};
use crate::envdecode::tests::sample_kind;
use crate::envelope::{EnvelopeHash, OperationEnvelope};
use crate::payload::{
OperationKindTag, OperationPayload, ResolveConflictPayload, ResolveEquivocationPayload,
};
use crate::stamp::{HybridLogicalClock, OperationStamp};
use crate::support::{AuthorId, OperationKindRegistryId};
use crate::undo::{UndoPolicy, UndoTransactionPayload};
fn round_trip<T>(value: &T)
where
T: TextValue + PartialEq + Debug,
{
let text = value.project().render();
let sexp = read_sexp(&text).expect("projected text must be readable");
let parsed = T::parse(&sexp).expect("projected value must parse");
assert_eq!(&parsed, value, "value round-trip");
assert_eq!(parsed.project().render(), text, "text round-trip");
}
fn envelope(payload: OperationPayload) -> OperationEnvelope {
let id = OperationId::new(ReplicaId(7), 1);
OperationEnvelope {
id,
author: AuthorId(0x1122_3344),
stamp: OperationStamp::new(HybridLogicalClock::new(WallClockTime(42), 7), id),
causal_context: CausalContext::new()
.with_seen(ReplicaId(1), 3)
.with_seen(ReplicaId(2), 5)
.with_dot(OperationId::new(ReplicaId(3), 9))
.with_dot(OperationId::new(ReplicaId(4), 11)),
transaction: Some(TransactionId::new(ReplicaId(7), 5)),
payload,
}
}
fn round_trip_envelope(value: &OperationEnvelope) {
round_trip(value);
let text = project_envelope(value);
assert!(!text.ends_with('\n'));
assert_eq!(
parse_envelope(&text).expect("canonical envelope line must parse"),
*value
);
}
#[test]
fn stamp_and_causal_productions_round_trip() {
let id = OperationId::new(ReplicaId(7), 1);
round_trip(&OperationStamp::new(
HybridLogicalClock::new(WallClockTime(42), 7),
id,
));
round_trip(
&CausalContext::new()
.with_seen(ReplicaId(1), 3)
.with_seen(ReplicaId(2), 5)
.with_dot(OperationId::new(ReplicaId(3), 9))
.with_dot(OperationId::new(ReplicaId(4), 11)),
);
}
#[test]
fn every_primitive_and_all_four_payload_variants_round_trip() {
for tag in OperationKindTag::PAYLOAD_FREE {
let payload = OperationPayload::Primitive(sample_kind(*tag));
round_trip(&payload);
let value = envelope(payload);
round_trip_envelope(&value);
}
let registered = OperationPayload::Primitive(sample_kind(OperationKindTag::Registered(
OperationKindRegistryId(1),
)));
round_trip(&registered);
round_trip_envelope(&envelope(registered));
let conflict = OperationPayload::ResolveConflict(ResolveConflictPayload {
target: ConflictId(0xDEAD_BEEF),
action: ResolutionAction::Reanchor {
new_target: TypedObjectId::Event(EventId::new(ReplicaId(7), 3)),
},
});
round_trip(&conflict);
round_trip_envelope(&envelope(conflict));
let undo = OperationPayload::UndoTransaction(UndoTransactionPayload {
target: TransactionId::new(ReplicaId(7), 5),
policy: UndoPolicy::BestEffort,
});
round_trip(&undo);
round_trip_envelope(&envelope(undo));
let equivocation = OperationPayload::ResolveEquivocation(ResolveEquivocationPayload {
target: OperationId::new(ReplicaId(7), 2),
chosen: EnvelopeHash([9; 32]),
});
round_trip(&equivocation);
round_trip_envelope(&envelope(equivocation));
}
#[test]
fn causal_collections_reject_duplicates_and_disorder() {
for noncanonical in [
"(causal ((#x0000000000000002 5) (#x0000000000000001 3)) ())",
"(causal ((#x0000000000000001 3) (#x0000000000000001 5)) ())",
"(causal () (#x00000000000000020000000000000005 #x00000000000000010000000000000003))",
"(causal () (#x00000000000000010000000000000003 #x00000000000000010000000000000003))",
] {
let sexp = read_sexp(noncanonical).expect("fixture is well-formed text");
assert!(
CausalContext::parse(&sexp).is_err(),
"must reject rather than normalize {noncanonical}"
);
}
}
#[test]
fn payload_shapes_are_exact() {
for noncanonical in [
"(primitive)",
"(primitive dismiss)",
"(resolve-conflict #x000000000000000000000000deadbeef accept-loser extra)",
"(undo #x00000000000000070000000000000005)",
"(resolve-equivocation #x00000000000000070000000000000002)",
"(undo-transaction #x00000000000000070000000000000005 best-effort)",
] {
let sexp = read_sexp(noncanonical).expect("fixture is well-formed text");
assert!(OperationPayload::parse(&sexp).is_err(), "{noncanonical}");
}
}
#[test]
fn whole_line_noncanonical_spelling_is_rejected() {
let value = envelope(OperationPayload::UndoTransaction(UndoTransactionPayload {
target: TransactionId::new(ReplicaId(7), 5),
policy: UndoPolicy::BestEffort,
}));
let canonical = project_envelope(&value);
let doubled_space = canonical.replacen(' ', " ", 1);
assert!(parse_envelope(&doubled_space).is_err());
assert!(parse_envelope(&(canonical + "\n")).is_err());
}
}

View File

@ -0,0 +1,678 @@
//! Grammar-directed Text Projection for primitive operation kinds.
//!
//! Operation payload records are an implementation detail: the grammar inlines
//! their fields after the Operation Catalog name, in canonical encoder order.
use epiphany_core::textvalue::{Sexp, TextError, TextValue};
use epiphany_determinism::{sorted_canonical, CanonicalEncode};
use unicode_normalization::UnicodeNormalization;
use crate::payload::{
ChangeRegionTimeModelOp, CreateCrossCuttingOp, CreateRegionOp, CreateRepeatStructureOp,
CreateStaffInstanceOp, CreateStaffOp, CreateVoiceOp, DeleteCrossCuttingOp, DeleteEventOp,
DeleteIdentifiedPitchOp, DeleteRegionOp, DeleteRepeatStructureOp, DeleteStaffInstanceOp,
DeleteVoiceOp, InsertEventOp, InsertIdentifiedPitchOp, ModifyCrossCuttingOp, ModifyEventOp,
ModifyIdentifiedPitchOp, OperationKind, OperationKindTag, RespellPitchOp, SetMetadataOp,
SetMetricGridOp, SetStaffLayoutOp, SetTempoSegmentOp, SetTimeSignatureOp, SetUserPageBreakOp,
SetUserSystemBreakOp, TransactionDescriptor, TransposeIntervalOp, TransposeOp,
};
use crate::support::OperationKindRegistryId;
/// Builds one operation production from its generated catalog name and its
/// positionally inlined payload fields.
fn production(tag: OperationKindTag, fields: Vec<Sexp>) -> Sexp {
let mut items = Vec::with_capacity(fields.len() + 1);
items.push(Sexp::sym(tag.catalog_name()));
items.extend(fields);
Sexp::List(items)
}
fn class_of(s: &Sexp) -> &'static str {
match s {
Sexp::List(_) => "list",
Sexp::Symbol(_) => "symbol",
Sexp::Int(_) => "integer",
Sexp::Bytes(_) => "byte string",
Sexp::Str(_) => "string",
}
}
/// Resolves a production head through the generated tag vocabulary. Registered
/// kinds carry their real id in the first field, so a zero-valued tag is used
/// only to select the exhaustive parsing arm.
fn production_tag(s: &Sexp) -> Result<OperationKindTag, TextError> {
let items = s.as_list().ok_or(TextError::Expected {
expected: "operation kind",
found: class_of(s),
})?;
let head = items
.first()
.and_then(Sexp::as_symbol)
.ok_or(TextError::Syntax(
"an operation kind is headed by its catalog name",
))?;
if let Some(tag) = OperationKindTag::PAYLOAD_FREE
.iter()
.copied()
.find(|tag| tag.catalog_name() == head)
{
return Ok(tag);
}
let registered = OperationKindTag::Registered(OperationKindRegistryId::from_raw(0));
if registered.catalog_name() == head {
return Ok(registered);
}
Err(TextError::UnknownConstructor {
type_name: "OperationKind",
found: head.to_owned(),
})
}
fn fields(s: &Sexp, tag: OperationKindTag, arity: usize) -> Result<&[Sexp], TextError> {
s.expect_struct(tag.catalog_name(), arity)
}
/// Reads the grammar's opaque `bytes` terminal. `Vec<u8>`'s generic TextValue
/// implementation denotes a sequence of integers, which is a different
/// production and therefore must not be used for registered payload bytes.
fn parse_bytes(s: &Sexp) -> Result<Vec<u8>, TextError> {
match s {
Sexp::Bytes(bytes) => Ok(bytes.clone()),
_ => Err(TextError::Expected {
expected: "byte string",
found: class_of(s),
}),
}
}
/// Projects and strictly parses all 31 `kind` productions from
/// `spec/text_projection.tex`.
impl TextValue for OperationKind {
fn project(&self) -> Sexp {
match self {
OperationKind::InsertEvent(op) => production(
self.tag(),
vec![op.staff_instance.project(), op.event.project()],
),
OperationKind::DeleteEvent(op) => production(
self.tag(),
vec![op.event.project(), op.tuplet_compensation.project()],
),
OperationKind::RespellPitch(op) => {
production(self.tag(), vec![op.pitch.project(), op.spelling.project()])
}
OperationKind::CreateCrossCutting(op) => {
production(self.tag(), vec![op.structure.project()])
}
OperationKind::ChangeRegionTimeModel(op) => production(
self.tag(),
vec![
op.region.project(),
op.new_time_model.project(),
sorted_canonical(op.declared_incompatible.clone()).project(),
op.remapping.project(),
],
),
OperationKind::SetUserSystemBreak(op) => production(
self.tag(),
vec![
op.region.project(),
op.anchor.project(),
op.present.project(),
],
),
OperationKind::DeclareTransaction(op) => production(
self.tag(),
vec![
op.id.project(),
Sexp::Str(op.label.nfc().collect()),
op.category.project(),
],
),
OperationKind::Registered(id, bytes) => {
production(self.tag(), vec![id.project(), Sexp::Bytes(bytes.clone())])
}
OperationKind::ModifyEvent(op) => production(self.tag(), vec![op.event.project()]),
OperationKind::Transpose(op) => production(
self.tag(),
vec![
sorted_canonical(op.targets.clone()).project(),
op.chromatic_steps.project(),
],
),
OperationKind::InsertIdentifiedPitch(op) => {
production(self.tag(), vec![op.event.project(), op.pitch.project()])
}
OperationKind::DeleteIdentifiedPitch(op) => {
production(self.tag(), vec![op.pitch.project()])
}
OperationKind::ModifyIdentifiedPitch(op) => {
production(self.tag(), vec![op.pitch.project(), op.value.project()])
}
OperationKind::DeleteCrossCutting(op) => {
production(self.tag(), vec![op.structure.project()])
}
OperationKind::ModifyCrossCutting(op) => {
production(self.tag(), vec![op.structure.project()])
}
OperationKind::CreateRegion(op) => production(self.tag(), vec![op.region.project()]),
OperationKind::DeleteRegion(op) => production(self.tag(), vec![op.region.project()]),
OperationKind::CreateStaffInstance(op) => {
production(self.tag(), vec![op.region.project(), op.instance.project()])
}
OperationKind::DeleteStaffInstance(op) => {
production(self.tag(), vec![op.staff_instance.project()])
}
OperationKind::CreateVoice(op) => production(
self.tag(),
vec![op.staff_instance.project(), op.voice.project()],
),
OperationKind::DeleteVoice(op) => production(self.tag(), vec![op.voice.project()]),
OperationKind::SetMetadata(op) => production(self.tag(), vec![op.metadata.project()]),
OperationKind::SetMetricGrid(op) => {
production(self.tag(), vec![op.region.project(), op.grid.project()])
}
OperationKind::SetUserPageBreak(op) => production(
self.tag(),
vec![
op.region.project(),
op.anchor.project(),
op.present.project(),
],
),
OperationKind::CreateStaff(op) => production(self.tag(), vec![op.staff.project()]),
OperationKind::SetTimeSignature(op) => production(
self.tag(),
vec![
op.region.project(),
op.anchor.project(),
op.time_signature.project(),
],
),
OperationKind::SetTempoSegment(op) => production(
self.tag(),
vec![
op.region.project(),
op.start.project(),
op.segment.project(),
],
),
OperationKind::SetStaffLayout(op) => production(
self.tag(),
vec![
op.staff_instance.project(),
op.instrument_override.project(),
op.staff_lines_override.project(),
op.visible.project(),
],
),
OperationKind::CreateRepeatStructure(op) => {
production(self.tag(), vec![op.repeat.project()])
}
OperationKind::DeleteRepeatStructure(op) => {
production(self.tag(), vec![op.repeat.project()])
}
OperationKind::TransposeInterval(op) => production(
self.tag(),
vec![op.targets.project(), op.interval.project()],
),
}
}
fn parse(s: &Sexp) -> Result<Self, TextError> {
let tag = production_tag(s)?;
Ok(match tag {
OperationKindTag::InsertEvent => {
let [staff_instance, event] = fields(s, tag, 2)? else {
unreachable!("the arity-2 check returned two fields")
};
OperationKind::InsertEvent(InsertEventOp {
staff_instance: TextValue::parse(staff_instance)?,
event: TextValue::parse(event)?,
})
}
OperationKindTag::DeleteEvent => {
let [event, tuplet_compensation] = fields(s, tag, 2)? else {
unreachable!("the arity-2 check returned two fields")
};
OperationKind::DeleteEvent(DeleteEventOp {
event: TextValue::parse(event)?,
tuplet_compensation: TextValue::parse(tuplet_compensation)?,
})
}
OperationKindTag::RespellPitch => {
let [pitch, spelling] = fields(s, tag, 2)? else {
unreachable!("the arity-2 check returned two fields")
};
OperationKind::RespellPitch(RespellPitchOp {
pitch: TextValue::parse(pitch)?,
spelling: TextValue::parse(spelling)?,
})
}
OperationKindTag::CreateCrossCutting => {
let [structure] = fields(s, tag, 1)? else {
unreachable!("the arity-1 check returned one field")
};
OperationKind::CreateCrossCutting(CreateCrossCuttingOp {
structure: TextValue::parse(structure)?,
})
}
OperationKindTag::ChangeRegionTimeModel => {
let [region, new_time_model, declared_incompatible, remapping] = fields(s, tag, 4)?
else {
unreachable!("the arity-4 check returned four fields")
};
let declared_incompatible: Vec<epiphany_core::EventId> =
Vec::parse(declared_incompatible)?;
// The encoder sorts this sequence. Reject disorder before storing
// the order-preserving Vec, rather than accepting text that the
// next projection would silently normalize.
if declared_incompatible
.windows(2)
.any(|w| w[0].to_canonical_bytes() > w[1].to_canonical_bytes())
{
return Err(TextError::NotStrictlyIncreasing(
"declared incompatible event ids",
));
}
OperationKind::ChangeRegionTimeModel(ChangeRegionTimeModelOp {
region: TextValue::parse(region)?,
new_time_model: TextValue::parse(new_time_model)?,
declared_incompatible,
remapping: TextValue::parse(remapping)?,
})
}
OperationKindTag::SetUserSystemBreak => {
let [region, anchor, present] = fields(s, tag, 3)? else {
unreachable!("the arity-3 check returned three fields")
};
OperationKind::SetUserSystemBreak(SetUserSystemBreakOp {
region: TextValue::parse(region)?,
anchor: TextValue::parse(anchor)?,
present: TextValue::parse(present)?,
})
}
OperationKindTag::DeclareTransaction => {
let [id, label, category] = fields(s, tag, 3)? else {
unreachable!("the arity-3 check returned three fields")
};
let label = String::parse(label)?;
// TransactionDescriptor's encoder normalizes this sole text
// field. Mirror envdecode's inequality guard so parsing rejects
// a spelling that the next projection would normalize.
if label.nfc().collect::<String>() != label {
return Err(TextError::NotCanonical("transaction label is not NFC"));
}
OperationKind::DeclareTransaction(TransactionDescriptor {
id: TextValue::parse(id)?,
label,
category: TextValue::parse(category)?,
})
}
OperationKindTag::Registered(_) => {
let [id, bytes] = fields(s, tag, 2)? else {
unreachable!("the arity-2 check returned two fields")
};
OperationKind::Registered(TextValue::parse(id)?, parse_bytes(bytes)?)
}
OperationKindTag::ModifyEvent => {
let [event] = fields(s, tag, 1)? else {
unreachable!("the arity-1 check returned one field")
};
OperationKind::ModifyEvent(ModifyEventOp {
event: TextValue::parse(event)?,
})
}
OperationKindTag::Transpose => {
let [targets, chromatic_steps] = fields(s, tag, 2)? else {
unreachable!("the arity-2 check returned two fields")
};
let targets: Vec<epiphany_core::PitchId> = Vec::parse(targets)?;
// This frozen payload is a sorted multiset, not a set. Mirror
// envdecode's strict-decrease comparison so duplicate targets
// remain legal and are replayed repeatedly.
if targets
.windows(2)
.any(|w| w[0].to_canonical_bytes() > w[1].to_canonical_bytes())
{
return Err(TextError::NotStrictlyIncreasing("transpose targets"));
}
OperationKind::Transpose(TransposeOp {
targets,
chromatic_steps: TextValue::parse(chromatic_steps)?,
})
}
OperationKindTag::InsertIdentifiedPitch => {
let [event, pitch] = fields(s, tag, 2)? else {
unreachable!("the arity-2 check returned two fields")
};
OperationKind::InsertIdentifiedPitch(InsertIdentifiedPitchOp {
event: TextValue::parse(event)?,
pitch: TextValue::parse(pitch)?,
})
}
OperationKindTag::DeleteIdentifiedPitch => {
let [pitch] = fields(s, tag, 1)? else {
unreachable!("the arity-1 check returned one field")
};
OperationKind::DeleteIdentifiedPitch(DeleteIdentifiedPitchOp {
pitch: TextValue::parse(pitch)?,
})
}
OperationKindTag::ModifyIdentifiedPitch => {
let [pitch, value] = fields(s, tag, 2)? else {
unreachable!("the arity-2 check returned two fields")
};
OperationKind::ModifyIdentifiedPitch(ModifyIdentifiedPitchOp {
pitch: TextValue::parse(pitch)?,
value: TextValue::parse(value)?,
})
}
OperationKindTag::DeleteCrossCutting => {
let [structure] = fields(s, tag, 1)? else {
unreachable!("the arity-1 check returned one field")
};
OperationKind::DeleteCrossCutting(DeleteCrossCuttingOp {
structure: TextValue::parse(structure)?,
})
}
OperationKindTag::ModifyCrossCutting => {
let [structure] = fields(s, tag, 1)? else {
unreachable!("the arity-1 check returned one field")
};
OperationKind::ModifyCrossCutting(ModifyCrossCuttingOp {
structure: TextValue::parse(structure)?,
})
}
OperationKindTag::InsertRegion => {
let [region] = fields(s, tag, 1)? else {
unreachable!("the arity-1 check returned one field")
};
OperationKind::CreateRegion(CreateRegionOp {
region: TextValue::parse(region)?,
})
}
OperationKindTag::DeleteRegion => {
let [region] = fields(s, tag, 1)? else {
unreachable!("the arity-1 check returned one field")
};
OperationKind::DeleteRegion(DeleteRegionOp {
region: TextValue::parse(region)?,
})
}
OperationKindTag::InsertStaffInstance => {
let [region, instance] = fields(s, tag, 2)? else {
unreachable!("the arity-2 check returned two fields")
};
OperationKind::CreateStaffInstance(CreateStaffInstanceOp {
region: TextValue::parse(region)?,
instance: TextValue::parse(instance)?,
})
}
OperationKindTag::DeleteStaffInstance => {
let [staff_instance] = fields(s, tag, 1)? else {
unreachable!("the arity-1 check returned one field")
};
OperationKind::DeleteStaffInstance(DeleteStaffInstanceOp {
staff_instance: TextValue::parse(staff_instance)?,
})
}
OperationKindTag::CreateVoice => {
let [staff_instance, voice] = fields(s, tag, 2)? else {
unreachable!("the arity-2 check returned two fields")
};
OperationKind::CreateVoice(CreateVoiceOp {
staff_instance: TextValue::parse(staff_instance)?,
voice: TextValue::parse(voice)?,
})
}
OperationKindTag::DeleteVoice => {
let [voice] = fields(s, tag, 1)? else {
unreachable!("the arity-1 check returned one field")
};
OperationKind::DeleteVoice(DeleteVoiceOp {
voice: TextValue::parse(voice)?,
})
}
OperationKindTag::SetMetadata => {
let [metadata] = fields(s, tag, 1)? else {
unreachable!("the arity-1 check returned one field")
};
OperationKind::SetMetadata(SetMetadataOp {
metadata: TextValue::parse(metadata)?,
})
}
OperationKindTag::SetMetricGrid => {
let [region, grid] = fields(s, tag, 2)? else {
unreachable!("the arity-2 check returned two fields")
};
OperationKind::SetMetricGrid(SetMetricGridOp {
region: TextValue::parse(region)?,
grid: TextValue::parse(grid)?,
})
}
OperationKindTag::SetUserPageBreak => {
let [region, anchor, present] = fields(s, tag, 3)? else {
unreachable!("the arity-3 check returned three fields")
};
OperationKind::SetUserPageBreak(SetUserPageBreakOp {
region: TextValue::parse(region)?,
anchor: TextValue::parse(anchor)?,
present: TextValue::parse(present)?,
})
}
OperationKindTag::InsertStaff => {
let [staff] = fields(s, tag, 1)? else {
unreachable!("the arity-1 check returned one field")
};
OperationKind::CreateStaff(CreateStaffOp {
staff: TextValue::parse(staff)?,
})
}
OperationKindTag::SetTimeSignature => {
let [region, anchor, time_signature] = fields(s, tag, 3)? else {
unreachable!("the arity-3 check returned three fields")
};
OperationKind::SetTimeSignature(SetTimeSignatureOp {
region: TextValue::parse(region)?,
anchor: TextValue::parse(anchor)?,
time_signature: TextValue::parse(time_signature)?,
})
}
OperationKindTag::SetTempoSegment => {
let [region, start, segment] = fields(s, tag, 3)? else {
unreachable!("the arity-3 check returned three fields")
};
OperationKind::SetTempoSegment(SetTempoSegmentOp {
region: TextValue::parse(region)?,
start: TextValue::parse(start)?,
segment: TextValue::parse(segment)?,
})
}
OperationKindTag::SetStaffLayout => {
let [staff_instance, instrument_override, staff_lines_override, visible] =
fields(s, tag, 4)?
else {
unreachable!("the arity-4 check returned four fields")
};
OperationKind::SetStaffLayout(SetStaffLayoutOp {
staff_instance: TextValue::parse(staff_instance)?,
instrument_override: TextValue::parse(instrument_override)?,
staff_lines_override: TextValue::parse(staff_lines_override)?,
visible: TextValue::parse(visible)?,
})
}
OperationKindTag::CreateRepeatStructure => {
let [repeat] = fields(s, tag, 1)? else {
unreachable!("the arity-1 check returned one field")
};
OperationKind::CreateRepeatStructure(CreateRepeatStructureOp {
repeat: TextValue::parse(repeat)?,
})
}
OperationKindTag::DeleteRepeatStructure => {
let [repeat] = fields(s, tag, 1)? else {
unreachable!("the arity-1 check returned one field")
};
OperationKind::DeleteRepeatStructure(DeleteRepeatStructureOp {
repeat: TextValue::parse(repeat)?,
})
}
OperationKindTag::TransposeInterval => {
let [targets, interval] = fields(s, tag, 2)? else {
unreachable!("the arity-2 check returned two fields")
};
OperationKind::TransposeInterval(TransposeIntervalOp {
// CanonicalSet's TextValue parser performs the grammar's
// strictly-increasing set check before BTreeSet construction.
targets: TextValue::parse(targets)?,
interval: TextValue::parse(interval)?,
})
}
})
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::envdecode::tests::sample_kind;
use crate::payload::{OperationKind, OperationKindTag};
use epiphany_core::textvalue::read_sexp;
fn all_tags() -> impl Iterator<Item = OperationKindTag> {
OperationKindTag::PAYLOAD_FREE
.iter()
.copied()
.chain(std::iter::once(OperationKindTag::Registered(
OperationKindRegistryId::from_raw(0),
)))
}
fn round_trip(value: &OperationKind) {
let text = value.project().render();
let sexp = read_sexp(&text).expect("the projection is valid canonical text");
let parsed = OperationKind::parse(&sexp).expect("the projection parses");
assert_eq!(&parsed, value);
assert_eq!(parsed.project().render(), text);
}
fn swap_first_two_in_sequence(s: &mut Sexp, field: usize) {
let Sexp::List(production) = s else {
panic!("operation projection is a list")
};
let Sexp::List(sequence) = &mut production[field] else {
panic!("selected field is a sequence")
};
assert!(sequence.len() >= 2, "fixture has two ordered elements");
sequence.swap(0, 1);
}
fn duplicate_first_in_sequence(s: &mut Sexp, field: usize) {
let Sexp::List(production) = s else {
panic!("operation projection is a list")
};
let Sexp::List(sequence) = &mut production[field] else {
panic!("selected field is a sequence")
};
assert!(sequence.len() >= 2, "fixture has two ordered elements");
sequence[1] = sequence[0].clone();
}
#[test]
fn every_operation_kind_round_trips_with_canonical_text() {
let tags: Vec<_> = all_tags().collect();
assert_eq!(tags.len(), 31, "the grammar has 31 kind productions");
for tag in tags {
round_trip(&sample_kind(tag));
}
}
#[test]
fn operation_kind_rejects_unknown_constructor_and_wrong_arity() {
let unknown = read_sexp("(unknown-operation #x00)").expect("well-formed text");
assert!(OperationKind::parse(&unknown).is_err());
let sample = sample_kind(OperationKindTag::DeleteRegion);
let Sexp::List(mut items) = sample.project() else {
panic!("operation projection is a list")
};
items.push(Sexp::int(0));
assert!(OperationKind::parse(&Sexp::List(items)).is_err());
}
#[test]
fn transaction_label_projects_nfc_and_rejects_non_nfc_text() {
let mut sample = sample_kind(OperationKindTag::DeclareTransaction);
let OperationKind::DeclareTransaction(descriptor) = &mut sample else {
panic!("tag fixture constructs its corresponding operation")
};
descriptor.label = "e\u{301}".to_owned();
let Sexp::List(mut items) = sample.project() else {
panic!("operation projection is a list")
};
assert_eq!(items[2], Sexp::Str("\u{e9}".to_owned()));
items[2] = Sexp::Str("e\u{301}".to_owned());
assert!(OperationKind::parse(&Sexp::List(items)).is_err());
}
#[test]
fn registered_payload_uses_and_requires_the_bytes_terminal() {
let sample = sample_kind(OperationKindTag::Registered(
OperationKindRegistryId::from_raw(0),
));
let Sexp::List(mut items) = sample.project() else {
panic!("operation projection is a list")
};
assert!(matches!(
items.as_slice(),
[Sexp::Symbol(_), Sexp::Bytes(_), Sexp::Bytes(_)]
));
items[2] = Sexp::List(vec![Sexp::int(1), Sexp::int(2)]);
assert!(OperationKind::parse(&Sexp::List(items)).is_err());
}
#[test]
fn transpose_multiset_rejects_strict_decrease_but_accepts_duplicate() {
let sample = sample_kind(OperationKindTag::Transpose);
let mut decreasing = sample.project();
swap_first_two_in_sequence(&mut decreasing, 1);
assert!(OperationKind::parse(&decreasing).is_err());
let mut duplicated = sample.project();
duplicate_first_in_sequence(&mut duplicated, 1);
let parsed = OperationKind::parse(&duplicated).expect("duplicates are legal in a multiset");
assert_eq!(parsed.project(), duplicated);
}
#[test]
fn change_region_declared_incompatible_rejects_decrease_but_accepts_duplicate() {
let sample = sample_kind(OperationKindTag::ChangeRegionTimeModel);
let mut decreasing = sample.project();
swap_first_two_in_sequence(&mut decreasing, 3);
assert!(OperationKind::parse(&decreasing).is_err());
let mut duplicated = sample.project();
duplicate_first_in_sequence(&mut duplicated, 3);
let parsed = OperationKind::parse(&duplicated).expect("non-decreasing permits duplicates");
assert_eq!(parsed.project(), duplicated);
}
#[test]
fn transpose_interval_targets_reject_duplicate_and_decrease() {
let sample = sample_kind(OperationKindTag::TransposeInterval);
let mut duplicated = sample.project();
duplicate_first_in_sequence(&mut duplicated, 1);
assert!(OperationKind::parse(&duplicated).is_err());
let mut decreasing = sample.project();
swap_first_two_in_sequence(&mut decreasing, 1);
assert!(OperationKind::parse(&decreasing).is_err());
}
}

View File

@ -0,0 +1,627 @@
//! Grammar-directed text projections for operation-layer leaves and sub-vocabularies.
use epiphany_core::textvalue::{Sexp, TextError, TextValue};
use epiphany_core::{Beam, EventId, MusicalPosition, Rest, Slur, Spanner, Tie, TupletId};
use epiphany_determinism::sorted_canonical;
use crate::conflict::{ConflictId, ResolutionAction};
use crate::envelope::EnvelopeHash;
use crate::payload::{
CrossCuttingValue, PositionRemapping, TransactionCategory, TupletCompensation,
};
use crate::support::AuthorId;
use crate::undo::UndoPolicy;
/// The lexical class of `s`, used by impls in this module and by the registry-id
/// declaration macro. This mirrors the core text layer's private diagnostic helper.
pub(crate) fn class_of(s: &Sexp) -> &'static str {
match s {
Sexp::List(_) => "list",
Sexp::Symbol(_) => "symbol",
Sexp::Int(_) => "integer",
Sexp::Bytes(_) => "byte string",
Sexp::Str(_) => "string",
}
}
fn constructor<'a>(
s: &'a Sexp,
type_name: &'static str,
) -> Result<(&'a str, &'a [Sexp]), TextError> {
let items = s.as_list().ok_or(TextError::Expected {
expected: type_name,
found: class_of(s),
})?;
let Some(head) = items.first().and_then(Sexp::as_symbol) else {
return Err(TextError::Syntax(
"a constructor list is headed by a symbol",
));
};
Ok((head, &items[1..]))
}
fn expect_arity<'a>(
fields: &'a [Sexp],
expected: usize,
type_name: &'static str,
) -> Result<&'a [Sexp], TextError> {
if fields.len() != expected {
return Err(TextError::Arity {
type_name,
expected,
found: fields.len(),
});
}
Ok(fields)
}
fn unknown(type_name: &'static str, found: &str) -> TextError {
TextError::UnknownConstructor {
type_name,
found: found.to_owned(),
}
}
fn parse_sorted_sequence<T>(s: &Sexp, what: &'static str) -> Result<Vec<T>, TextError>
where
T: TextValue + Ord,
{
let values = Vec::<T>::parse(s)?;
// These payload fields are Vecs whose binary encoders sort without removing
// duplicates. Check before constructing the enum so parsing never normalizes
// a descending input; equal neighbours remain legal in the frozen wire form.
if values.windows(2).any(|pair| pair[0] > pair[1]) {
return Err(TextError::NotCanonical(what));
}
Ok(values)
}
fn parse_reassign_entries(s: &Sexp) -> Result<Vec<(EventId, MusicalPosition)>, TextError> {
let entries = Vec::<(EventId, MusicalPosition)>::parse(s)?;
// The encoder sorts by EventId alone and its stable sort preserves the order
// of duplicate keys. Compare only IDs: equal IDs with any positions are
// therefore canonical, while a strict decrease would be normalized.
if entries.windows(2).any(|pair| pair[0].0 > pair[1].0) {
return Err(TextError::NotCanonical(
"Reassign entries must be non-decreasing by EventId",
));
}
Ok(entries)
}
/// Projects an author identifier as its canonical 16 big-endian bytes.
impl TextValue for AuthorId {
fn project(&self) -> Sexp {
Sexp::Bytes(self.canonical_bytes().to_vec())
}
fn parse(s: &Sexp) -> Result<Self, TextError> {
let Sexp::Bytes(bytes) = s else {
return Err(TextError::Expected {
expected: "AuthorId",
found: class_of(s),
});
};
let bytes: [u8; 16] = bytes
.as_slice()
.try_into()
.map_err(|_| TextError::NotCanonical("an AuthorId is exactly 16 bytes"))?;
Ok(Self(u128::from_be_bytes(bytes)))
}
}
/// Projects a conflict identifier as its canonical 16 big-endian bytes.
impl TextValue for ConflictId {
fn project(&self) -> Sexp {
Sexp::Bytes(self.canonical_bytes().to_vec())
}
fn parse(s: &Sexp) -> Result<Self, TextError> {
let Sexp::Bytes(bytes) = s else {
return Err(TextError::Expected {
expected: "ConflictId",
found: class_of(s),
});
};
let bytes: [u8; 16] = bytes
.as_slice()
.try_into()
.map_err(|_| TextError::NotCanonical("a ConflictId is exactly 16 bytes"))?;
Ok(Self(u128::from_be_bytes(bytes)))
}
}
/// Projects an envelope hash as its canonical 32 bytes.
impl TextValue for EnvelopeHash {
fn project(&self) -> Sexp {
Sexp::Bytes(self.0.to_vec())
}
fn parse(s: &Sexp) -> Result<Self, TextError> {
let Sexp::Bytes(bytes) = s else {
return Err(TextError::Expected {
expected: "EnvelopeHash",
found: class_of(s),
});
};
let bytes: [u8; 32] = bytes
.as_slice()
.try_into()
.map_err(|_| TextError::NotCanonical("an EnvelopeHash is exactly 32 bytes"))?;
Ok(Self(bytes))
}
}
/// Implements the grammar's `action` production in canonical discriminant order.
impl TextValue for ResolutionAction {
fn project(&self) -> Sexp {
match self {
ResolutionAction::AcceptLoser => Sexp::sym("accept-loser"),
ResolutionAction::KeepWinner => Sexp::sym("keep-winner"),
ResolutionAction::Override { override_operation } => {
Sexp::List(vec![Sexp::sym("override"), override_operation.project()])
}
ResolutionAction::Reanchor { new_target } => {
Sexp::List(vec![Sexp::sym("reanchor"), new_target.project()])
}
ResolutionAction::Dismiss => Sexp::sym("dismiss"),
ResolutionAction::Registered(id) => {
Sexp::List(vec![Sexp::sym("registered"), id.project()])
}
}
}
fn parse(s: &Sexp) -> Result<Self, TextError> {
if let Some(name) = s.as_symbol() {
return match name {
"accept-loser" => Ok(Self::AcceptLoser),
"keep-winner" => Ok(Self::KeepWinner),
"dismiss" => Ok(Self::Dismiss),
_ => Err(unknown("ResolutionAction", name)),
};
}
let (name, fields) = constructor(s, "ResolutionAction")?;
let fields = expect_arity(fields, 1, "ResolutionAction")?;
match name {
"override" => Ok(Self::Override {
override_operation: TextValue::parse(&fields[0])?,
}),
"reanchor" => Ok(Self::Reanchor {
new_target: TextValue::parse(&fields[0])?,
}),
"registered" => Ok(Self::Registered(TextValue::parse(&fields[0])?)),
_ => Err(unknown("ResolutionAction", name)),
}
}
}
/// Implements the grammar's `policy` production.
impl TextValue for UndoPolicy {
fn project(&self) -> Sexp {
match self {
UndoPolicy::StrictInverse => Sexp::sym("strict-inverse"),
UndoPolicy::BestEffort => Sexp::sym("best-effort"),
UndoPolicy::Cascade => Sexp::sym("cascade"),
}
}
fn parse(s: &Sexp) -> Result<Self, TextError> {
match s.as_symbol() {
Some("strict-inverse") => Ok(Self::StrictInverse),
Some("best-effort") => Ok(Self::BestEffort),
Some("cascade") => Ok(Self::Cascade),
Some(name) => Err(unknown("UndoPolicy", name)),
None => Err(TextError::Expected {
expected: "UndoPolicy",
found: class_of(s),
}),
}
}
}
/// Implements the grammar's `tuplet-comp` production in canonical discriminant
/// and payload-field order.
impl TextValue for TupletCompensation {
fn project(&self) -> Sexp {
match self {
TupletCompensation::NotInTuplet => Sexp::sym("not-in-tuplet"),
TupletCompensation::ReplaceWithRest { rest } => {
Sexp::List(vec![Sexp::sym("replace-with-rest"), rest.project()])
}
TupletCompensation::RewriteTuplets { tuplets } => Sexp::List(vec![
Sexp::sym("rewrite-tuplets"),
sorted_canonical(tuplets.clone()).project(),
]),
TupletCompensation::CascadeDeleteTuplets { tuplets } => Sexp::List(vec![
Sexp::sym("cascade-delete-tuplets"),
sorted_canonical(tuplets.clone()).project(),
]),
}
}
fn parse(s: &Sexp) -> Result<Self, TextError> {
if let Some(name) = s.as_symbol() {
return match name {
"not-in-tuplet" => Ok(Self::NotInTuplet),
_ => Err(unknown("TupletCompensation", name)),
};
}
let (name, fields) = constructor(s, "TupletCompensation")?;
let fields = expect_arity(fields, 1, "TupletCompensation")?;
match name {
"replace-with-rest" => Ok(Self::ReplaceWithRest {
rest: Rest::parse(&fields[0])?,
}),
"rewrite-tuplets" => Ok(Self::RewriteTuplets {
tuplets: parse_sorted_sequence::<TupletId>(
&fields[0],
"RewriteTuplets ids must be non-decreasing",
)?,
}),
"cascade-delete-tuplets" => Ok(Self::CascadeDeleteTuplets {
tuplets: parse_sorted_sequence::<TupletId>(
&fields[0],
"CascadeDeleteTuplets ids must be non-decreasing",
)?,
}),
_ => Err(unknown("TupletCompensation", name)),
}
}
}
/// Implements the grammar's `cross-cutting` production, retaining the canonical
/// discriminant order of `CrossCuttingValue::encode_canonical`.
impl TextValue for CrossCuttingValue {
fn project(&self) -> Sexp {
match self {
CrossCuttingValue::Tie(value) => Sexp::List(vec![Sexp::sym("tie"), value.project()]),
CrossCuttingValue::Slur(value) => Sexp::List(vec![Sexp::sym("slur"), value.project()]),
CrossCuttingValue::Beam(value) => Sexp::List(vec![Sexp::sym("beam"), value.project()]),
CrossCuttingValue::Spanner(value) => {
Sexp::List(vec![Sexp::sym("spanner"), value.project()])
}
}
}
fn parse(s: &Sexp) -> Result<Self, TextError> {
let (name, fields) = constructor(s, "CrossCuttingValue")?;
let fields = expect_arity(fields, 1, "CrossCuttingValue")?;
match name {
"tie" => Ok(Self::Tie(Tie::parse(&fields[0])?)),
"slur" => Ok(Self::Slur(Slur::parse(&fields[0])?)),
"beam" => Ok(Self::Beam(Beam::parse(&fields[0])?)),
"spanner" => Ok(Self::Spanner(Spanner::parse(&fields[0])?)),
_ => Err(unknown("CrossCuttingValue", name)),
}
}
}
/// Implements the grammar's `remapping` production. Each reassign entry delegates
/// to the core pair projection for `(EventId, MusicalPosition)`.
impl TextValue for PositionRemapping {
fn project(&self) -> Sexp {
match self {
PositionRemapping::PreserveTime => Sexp::sym("preserve-time"),
PositionRemapping::Reassign(entries) => {
let mut entries = entries.clone();
entries.sort_by_key(|(event, _)| event.canonical_bytes());
Sexp::List(vec![Sexp::sym("reassign"), entries.project()])
}
}
}
fn parse(s: &Sexp) -> Result<Self, TextError> {
if let Some(name) = s.as_symbol() {
return match name {
"preserve-time" => Ok(Self::PreserveTime),
_ => Err(unknown("PositionRemapping", name)),
};
}
let (name, fields) = constructor(s, "PositionRemapping")?;
if name != "reassign" {
return Err(unknown("PositionRemapping", name));
}
let fields = expect_arity(fields, 1, "PositionRemapping")?;
Ok(Self::Reassign(parse_reassign_entries(&fields[0])?))
}
}
/// Implements the five-variant transaction-category sub-vocabulary in canonical
/// discriminant order.
impl TextValue for TransactionCategory {
fn project(&self) -> Sexp {
match self {
TransactionCategory::NoteEntry => Sexp::sym("note-entry"),
TransactionCategory::Structural => Sexp::sym("structural"),
TransactionCategory::Layout => Sexp::sym("layout"),
TransactionCategory::Import => Sexp::sym("import"),
TransactionCategory::Registered(id) => {
Sexp::List(vec![Sexp::sym("registered"), id.project()])
}
}
}
fn parse(s: &Sexp) -> Result<Self, TextError> {
if let Some(name) = s.as_symbol() {
return match name {
"note-entry" => Ok(Self::NoteEntry),
"structural" => Ok(Self::Structural),
"layout" => Ok(Self::Layout),
"import" => Ok(Self::Import),
_ => Err(unknown("TransactionCategory", name)),
};
}
let (name, fields) = constructor(s, "TransactionCategory")?;
if name != "registered" {
return Err(unknown("TransactionCategory", name));
}
let fields = expect_arity(fields, 1, "TransactionCategory")?;
Ok(Self::Registered(TextValue::parse(&fields[0])?))
}
}
#[cfg(test)]
mod tests {
use std::fmt::Debug;
use epiphany_core::textvalue::read_sexp;
use epiphany_core::{
AnchorOffset, BeamId, EventId, MusicalDuration, RationalTime, ReplicaId, SlurId, SpannerId,
TieId, TimeAnchor, TupletId, VoiceId,
};
use super::*;
use crate::support::{
ConflictKindRegistryId, ExtensionPreconditionId, IntegrityAnomalyRegistryId,
OperationKindRegistryId, PreconditionFailureRegistryId, ReanchorReasonRegistryId,
RepairKindRegistryId, ReplicaAnomalyRegistryId, ResolutionRegistryId,
};
fn round_trip<T>(value: T)
where
T: TextValue + PartialEq + Debug,
{
let rendered = value.project().render();
let read = read_sexp(&rendered).expect("projected text is valid s-expression");
let parsed = T::parse(&read).expect("projected value parses");
assert_eq!(parsed, value);
assert_eq!(parsed.project().render(), rendered);
}
fn event(counter: u64) -> EventId {
EventId::new(ReplicaId(7), counter)
}
fn tuplet(counter: u64) -> TupletId {
TupletId::new(ReplicaId(7), counter)
}
#[test]
fn byte_leaves_round_trip() {
round_trip(AuthorId(0x0011_2233_4455_6677_8899_aabb_ccdd_eeff));
round_trip(ConflictId(0xffee_ddcc_bbaa_9988_7766_5544_3322_1100));
round_trip(EnvelopeHash([0x5a; 32]));
round_trip(OperationKindRegistryId(1));
round_trip(ConflictKindRegistryId(2));
round_trip(ResolutionRegistryId(3));
round_trip(RepairKindRegistryId(4));
round_trip(ReanchorReasonRegistryId(5));
round_trip(ReplicaAnomalyRegistryId(6));
round_trip(IntegrityAnomalyRegistryId(7));
round_trip(ExtensionPreconditionId(8));
round_trip(PreconditionFailureRegistryId(9));
}
#[test]
fn byte_leaves_reject_noncanonical_lengths() {
let short = read_sexp("#x00").unwrap();
assert!(AuthorId::parse(&short).is_err());
assert!(ConflictId::parse(&short).is_err());
assert!(EnvelopeHash::parse(&short).is_err());
assert!(OperationKindRegistryId::parse(&short).is_err());
assert!(ConflictKindRegistryId::parse(&short).is_err());
assert!(ResolutionRegistryId::parse(&short).is_err());
assert!(RepairKindRegistryId::parse(&short).is_err());
assert!(ReanchorReasonRegistryId::parse(&short).is_err());
assert!(ReplicaAnomalyRegistryId::parse(&short).is_err());
assert!(IntegrityAnomalyRegistryId::parse(&short).is_err());
assert!(ExtensionPreconditionId::parse(&short).is_err());
assert!(PreconditionFailureRegistryId::parse(&short).is_err());
}
#[test]
fn action_round_trips_every_variant() {
let values = [
ResolutionAction::AcceptLoser,
ResolutionAction::KeepWinner,
ResolutionAction::Override {
override_operation: epiphany_core::OperationId::new(ReplicaId(1), 2),
},
ResolutionAction::Reanchor {
new_target: epiphany_core::TypedObjectId::Event(event(3)),
},
ResolutionAction::Dismiss,
ResolutionAction::Registered(ResolutionRegistryId(4)),
];
for value in values {
round_trip(value);
}
}
#[test]
fn action_rejects_noncanonical_productions() {
for bad in ["accept-winner", "(override)", "(reanchor #x00 #x01)"] {
let sexp = read_sexp(bad).unwrap();
assert!(ResolutionAction::parse(&sexp).is_err(), "{bad}");
}
}
#[test]
fn policy_round_trips_every_variant() {
for value in [
UndoPolicy::StrictInverse,
UndoPolicy::BestEffort,
UndoPolicy::Cascade,
] {
round_trip(value);
}
}
#[test]
fn policy_rejects_noncanonical_productions() {
for bad in ["strict", "(cascade)"] {
let sexp = read_sexp(bad).unwrap();
assert!(UndoPolicy::parse(&sexp).is_err(), "{bad}");
}
}
#[test]
fn tuplet_comp_round_trips_every_variant() {
let rest = crate::valuegen::rest_value(
event(9),
VoiceId::new(ReplicaId(7), 1),
MusicalDuration(
RationalTime::new(1, 4).expect("one quarter has a nonzero denominator"),
),
);
for value in [
TupletCompensation::NotInTuplet,
TupletCompensation::ReplaceWithRest { rest },
TupletCompensation::RewriteTuplets {
tuplets: vec![tuplet(1), tuplet(2)],
},
TupletCompensation::CascadeDeleteTuplets {
tuplets: vec![tuplet(2), tuplet(2)],
},
] {
round_trip(value);
}
}
#[test]
fn tuplet_comp_rejects_noncanonical_productions() {
for bad in ["rewrite-tuplets", "(replace-with-rest)", "(unknown ())"] {
let sexp = read_sexp(bad).unwrap();
assert!(TupletCompensation::parse(&sexp).is_err(), "{bad}");
}
}
#[test]
fn tuplet_rewrite_rejects_descending_ids() {
let sexp = read_sexp(
"(rewrite-tuplets (#x00000000000000070000000000000002 #x00000000000000070000000000000001))",
)
.unwrap();
assert!(TupletCompensation::parse(&sexp).is_err());
}
#[test]
fn tuplet_cascade_rejects_descending_ids() {
let sexp = read_sexp(
"(cascade-delete-tuplets (#x00000000000000070000000000000002 #x00000000000000070000000000000001))",
)
.unwrap();
assert!(TupletCompensation::parse(&sexp).is_err());
}
#[test]
fn cross_cutting_round_trips_every_variant() {
let start = event(1);
let end = event(2);
let spanner = Spanner {
id: SpannerId::new(ReplicaId(7), 1),
start: TimeAnchor::Event {
id: start,
offset: AnchorOffset::Musical(MusicalDuration::zero()),
},
end: TimeAnchor::Event {
id: end,
offset: AnchorOffset::Musical(MusicalDuration::zero()),
},
staves: Vec::new(),
kind: Default::default(),
style: Default::default(),
};
let values = [
CrossCuttingValue::Tie(crate::valuegen::tie(
TieId::new(ReplicaId(7), 1),
start,
end,
)),
CrossCuttingValue::Slur(crate::valuegen::slur(
SlurId::new(ReplicaId(7), 1),
start,
end,
)),
CrossCuttingValue::Beam(crate::valuegen::beam(
BeamId::new(ReplicaId(7), 1),
vec![start, end],
)),
CrossCuttingValue::Spanner(spanner),
];
for value in values {
round_trip(value);
}
}
#[test]
fn cross_cutting_rejects_noncanonical_productions() {
for bad in ["tie", "(tie)", "(unknown #x00)"] {
let sexp = read_sexp(bad).unwrap();
assert!(CrossCuttingValue::parse(&sexp).is_err(), "{bad}");
}
}
#[test]
fn remapping_round_trips_every_variant() {
round_trip(PositionRemapping::PreserveTime);
round_trip(PositionRemapping::Reassign(vec![
(event(1), MusicalPosition::origin()),
(
event(2),
MusicalPosition(
RationalTime::new(3, 4).expect("three quarters has a nonzero denominator"),
),
),
(event(2), MusicalPosition::origin()),
]));
}
#[test]
fn remapping_rejects_noncanonical_productions() {
for bad in ["reassign", "(reassign)", "(unknown ())"] {
let sexp = read_sexp(bad).unwrap();
assert!(PositionRemapping::parse(&sexp).is_err(), "{bad}");
}
}
#[test]
fn remapping_rejects_descending_event_ids() {
let sexp = read_sexp(
"(reassign ((#x00000000000000070000000000000002 (ratio 0 1)) (#x00000000000000070000000000000001 (ratio 0 1))))",
)
.unwrap();
assert!(PositionRemapping::parse(&sexp).is_err());
}
#[test]
fn transaction_category_round_trips_every_variant() {
for value in [
TransactionCategory::NoteEntry,
TransactionCategory::Structural,
TransactionCategory::Layout,
TransactionCategory::Import,
TransactionCategory::Registered(OperationKindRegistryId(5)),
] {
round_trip(value);
}
}
#[test]
fn transaction_category_rejects_noncanonical_productions() {
for bad in ["noteentry", "registered", "(registered)", "(unknown #x00)"] {
let sexp = read_sexp(bad).unwrap();
assert!(TransactionCategory::parse(&sexp).is_err(), "{bad}");
}
}
}

View File

@ -0,0 +1,149 @@
# Contract: the Text Projection operation layer
Repo root `/home/jeans/Repos/active/epiphany`. Read this in full before writing a
line. The plan it implements is `spec/PLAN_TEXTPROJ_OPS.md`.
## The ruling: this layer is grammar-directed
`req:textproj:value-projection` — the mechanical struct/enum/newtype rule — does
**not** govern the operation vocabulary. It governs exactly the `value` positions
*inside* the grammar's productions.
The grammar is in `spec/text_projection.tex`, Chapter "Grammar", in the single
`lstlisting` block. **It is the specification for your work.** Find your
production, implement exactly what it says. Where it says `value`, call
`TextValue`; where it says anything else, follow the production.
Three consequences you will meet immediately:
* An operation kind **inlines** its `*Op` payload record. `(insert-event #x0a
<event>)` — *not* `(insert-event (insert-event-op #x0a <event>))`. The record
exists so each enum variant can name a type; the binary form adds no bytes for
it (`OperationKind`'s encoding writes the tag then delegates), so the text adds
no wrapper either.
* Envelope-level productions use the grammar's names, not kebabbed Rust names:
`envelope`, `stamp`, `causal`, `undo`.
* `stamp` flattens `HybridLogicalClock` into three arguments. That is the
grammar's shape and it is deliberate.
## Where field order comes from
Every operation payload type has an explicit
`impl CanonicalEncode for T { fn encode_canonical(&self, …) }` in
`crates/epiphany-ops/src/payload.rs` (or `undo.rs` / `conflict.rs`). It reads its
fields in order. **That order is the ratified declaration order — mirror it
exactly.** Never infer an order from the struct declaration, a doc comment, or
the grammar's argument names; read `encode_canonical`.
`crates/epiphany-ops/src/envdecode.rs` is the binary *inverse* and is the best
cross-reference in the repo: it already decodes every one of these in wire order.
## Strict parsing is the point
`req:textproj:strict-parse`: a parser MUST reject text that is not the canonical
projection of the value it denotes. It MUST NOT normalize — not whitespace, not
case, not an out-of-order sequence, not a duplicate in a set-typed field.
The rules that follow from that, learned expensively on the core layer:
* **If a constructor normalizes, check before constructing.** Returning the
normalized value *is* accepting the bad input.
* **If a constructor only validates** (rejects rather than adjusts), the `None`
it returns is the whole of the strictness. Do **not** add a
re-project-and-compare guard as a backstop: on the core layer four such guards
were written, mutation-tested, found unable to fire, and removed. A check that
cannot fail is worse than no check — it invites weakening the real one.
* **Order-preserving `Vec`s need per-site checks.** A guard that re-projects
cannot see an order it faithfully preserves.
* **Exercise every outbound normalization.** For every value normalized on the
way out, construct a non-normalized input and prove that normalization happens.
Five such normalizations were written across two agents and all five were
untestable by omission because every fixture was already sorted. A projection
that emits text its own parser rejects is a defect ordinary round-trip tests
cannot see.
### The six order-constrained sequences
Each one's encoder normalizes, so `project` must apply the same normalization and
`parse` must reject text that is not already in that order.
| field | rule | note |
|---|---|---|
| `TransposeOp.targets` | **non-decreasing multiset** | duplicates are **legal**; the wire form is frozen. Reject only a strict *decrease*. |
| `TransposeIntervalOp.targets` | strictly increasing | a `CanonicalSet<PitchId>`, which is a type alias for `BTreeSet<PitchId>`; the existing core impl already does this check |
| `ChangeRegionTimeModelOp.declared_incompatible` | non-decreasing | encoder applies `sorted_canonical` |
| `TupletCompensation::RewriteTuplets.tuplets` | non-decreasing | encoder applies `sorted_canonical` |
| `TupletCompensation::CascadeDeleteTuplets.tuplets` | non-decreasing | shares one encoder arm with `RewriteTuplets`, but is a **separate** arm in the projector |
| `PositionRemapping::Reassign` entries | non-decreasing by `EventId` | encoder does `sort_by_key(|(e, _)| e.canonical_bytes())` |
**This table said "three" until an agent found the other three.** The scoping
script behind it searched `pub struct` bodies for sequence fields and never
looked inside enum variants, so every constrained sequence living in an enum was
invisible to it. If you are looking for sites of some kind, say out loud what
shapes your search can and cannot see before you trust its answer.
Getting `TransposeOp` backwards — rejecting a duplicate — silently breaks a
frozen operation's replay. It is the single highest-risk line in this work.
## Names are generated, never spelled
`OperationKindTag::catalog_name()` is production code, generated by
`operation_kind_tag_vocabulary!` from the same list that generates the wire
discriminant and the decoder. **Use it.** Writing `"insert-event"` as a literal
creates a list parallel to the enum — the exact shape that has cost this project
four bugs, most recently a kind that encoded to tag 30 and whose own decoder
rejected it.
For the same reason, `parse` must dispatch through an **exhaustive `match` over
`OperationKindTag`** (no `_` arm), so a kind added to the vocabulary and not to
the projector fails to compile. Every `match` in a `project` is likewise
exhaustive with no `_` arm.
## Style
* Match the surrounding code. Doc comments on every `impl` and every non-obvious
decision — especially *why* a parse rejects rather than normalizes. No comment
that merely restates the next line.
* `rustfmt` clean. No `clippy` warnings. No `#[allow(...)]`.
* No `unwrap`/`expect` on a parse path, except where an invariant was just
checked — and then name the check in the message.
* Touch **only** the file you are told to create. Do not edit `payload.rs`,
`envelope.rs`, `textvalue*.rs`, or another agent's file.
* Do **not** run `cargo fmt --all` — it rewrites files other agents hold. Run
`rustfmt --edition 2021 crates/epiphany-ops/src/<your file>`.
## How to verify, and how not to mislead yourself
Sibling files may be empty stubs while you work, so the crate may not link. That
is expected. Your file is done when **no compiler error points into it**:
```
cargo check -p epiphany-ops --all-targets 2>&1 | grep -n '<your file>'
```
must print nothing. Errors located in other files, or in `payload.rs` from a
macro expansion, are not yours.
A previous agent on this project reported "verification passes" when errors did
in fact point into its own file. **Paste the actual command and its actual
output in your report.** If the crate does not link, say so plainly and say your
tests were therefore not executed.
### Tests you must write
Under `#[cfg(test)] mod tests` in your own file:
1. **Round-trip** for every production you implement: build a value, `project()`,
`render()`, `read_sexp()`, `parse()`, assert equal to the original, and assert
the re-rendered text is byte-identical.
2. **Rejection** for every strictness rule you implement — a text that is
well-formed but not canonical must be rejected, not normalized. Assert the
rejection, not the message.
3. **Mutation-verify every order check and every guard you write.** For each:
assert the anchor text is present, delete or invert the check, confirm a named
test of yours fails, restore. A `str.replace` that matches nothing looks
exactly like a passing test. **Report the result per check, with output.** If
a check survives deletion, say so — that is a finding, not a failure, and it
means the check is dead or the test is blind.
Use `epiphany_core::textvalue::read_sexp` to parse test text.

245
spec/PLAN_TEXTPROJ_OPS.md Normal file
View File

@ -0,0 +1,245 @@
# Text Projection — the operation layer: scope and plan
Status: **scoping complete, one ruling needed before dispatch.**
Prepared against `master` @ `cf81074`. Every claim below was checked against the
code; where I ran a probe I say so.
---
## 1. The one decision that must be made first
**Is the operation vocabulary projected by the grammar's productions, or by
`req:textproj:value-projection`?**
They disagree, systematically. The grammar spells out `envelope`, `stamp`,
`causal`, `payload`, `kind`, `action`, `policy`, `tuplet-comp`, `cross-cutting`,
`remapping`, `reassign-entry`. What the mechanical value rule would produce for
the same types is different in three ways:
| grammar says | value rule would say |
|---|---|
| `(envelope …)` | `(operation-envelope …)` |
| `(stamp 1700 0 #x…)` — HLC flattened, 3 args | `(operation-stamp (hybrid-logical-clock 1700 0) #x…)` — 2 args |
| `(causal (…) (…))` | `(causal-context (…) (…))` |
| `(undo #x… best-effort)` | `(undo-transaction (undo-transaction-payload #x… best-effort))` |
| `(insert-event #x… <event>)` | `(insert-event (insert-event-op #x… <event>))` |
The last row is the one that matters most: **every one of the 31 kind
productions inlines its `*Op` payload record.** Under a literal reading of
clause 1 (`*Op` is a struct with named fields, so not a clause-2 newtype), the
doubled head would be required.
### Recommendation: grammar-directed, and say so
The companion already implies it. `req:textproj:value-projection`'s preamble
scopes itself: *"An operation payload **embeds** canonical values from the core
specification's Chapter 5 … It states one rule for turning any of them into
text."* The rule is for the **embedded Chapter-5 values**, and the `value`
nonterminal marks exactly where it applies. If the rule governed the operation
vocabulary too, the grammar's own productions would be redundant and partly
wrong.
What is missing is one normative sentence, because clause 1 is general enough to
be misread. Suggested shape — a new requirement, or an extension of
`req:textproj:schema-directed`:
> Chapter 6's operation vocabulary — the envelope, its stamp and causal context,
> the payload, the operation kinds and their sub-vocabularies — is projected by
> the productions of Chapter~\ref{ch:grammar}, not by
> `req:textproj:value-projection`. That rule governs exactly the `value`
> positions those productions contain.
>
> An operation kind's payload record is **inlined** into its production. The
> record exists so each variant can name a type; it is not a modelling
> distinction, and the binary form agrees — `OperationKind`'s encoding writes the
> kind tag and then delegates to the record, adding no bytes for the wrapper. It
> adds no text here either, for the same reason clause 2 makes a newtype
> transparent.
**Consequence for the implementation:** the ops projector is written *against the
grammar*, explicitly, and calls `TextValue` only at `value` positions. That is
less work than a mechanical derivation and produces far more readable text.
### One inconsistency to fix while we are here
`transpose-interval` inlines its interval as `(interval <d> <c>)`. But
`TranspositionInterval` is a core type with a `TextValue` (from `struct_codec!`)
that projects as `(transposition-interval <d> <c>)`. **Two names for one type**,
and the other one is what appears at any `value` position.
Options: (a) change the production to `"(transpose-interval (" bytes* ") " value
")"` and let the value rule name it — my recommendation, it removes the special
case; or (b) rename the inline head to `transposition-interval`.
---
## 2. What I verified (so nobody re-derives it)
**The grammar is accurate.** I mechanically compared each of the 31 `kind`
productions' argument list against its payload struct's field list: **31/31
match**, in both arity and shape (`bytes` ↔ id, `value` ↔ Chapter-5 type,
`option``Option<…>`, `bytes*``Vec<Id>`). No production needs changing
apart from the interval naming above.
**Core is ready — with two gaps.** A compile probe confirmed every Chapter-5 type
embedded in an operation payload already has a `TextValue`: `Event`, `Pitch`,
`IdentifiedPitch`, `PitchSpelling`, `Region`, `RegionTimeModel`,
`RepeatStructure`, `ScoreMetadata`, `Staff`, `StaffInstance`, `Voice`,
`TimeAnchor`, `TimeSignature`, `MetricGrid`, `TempoSegment`,
`StaffLineConfiguration`, `TranspositionInterval`, `Tie`, `Slur`, `Beam`,
`Spanner`, plus every embedded id.
Two do **not**: `TransactionId` and `TypedObjectId`. Both live in
`epiphany-core`, so `epiphany-ops` cannot implement the trait for them — the
orphan rule makes this a **core prerequisite**, not an ops task.
**`CanonicalSet<T>` is a type alias for `BTreeSet<T>`.** So
`TransposeIntervalOp.targets` is already covered by the existing `BTreeSet`
impl, per-site strict-increase check included. No new impl, no orphan problem.
**The three order-constrained sequences**, and exactly what each requires:
| field | type | encoder | text rule |
|---|---|---|---|
| `TransposeOp.targets` | `Vec<PitchId>` | `sorted_canonical` (**no dedup**) | non-decreasing **multiset** — duplicates legal, frozen wire form |
| `TransposeIntervalOp.targets` | `CanonicalSet<PitchId>` | set iteration | strictly increasing |
| `ChangeRegionTimeModelOp.declared_incompatible` | `Vec<EventId>` | `sorted_canonical` | non-decreasing |
Note the third: it is **not** in `envdecode.rs`'s per-site checks, because the
binary whole-envelope guard catches it (the encoder sorts, so a mis-ordered input
re-encodes differently). The text projector must likewise emit
`sorted_canonical(...)` for it, or text and binary will disagree.
---
## 3. Prerequisite (blocks everything; ~20 lines)
**P0 — `TextValue` for `TransactionId` and `TypedObjectId`, in `epiphany-core`.**
`TypedObjectId` already implements `CanonicalEncode + CanonicalDecode`, so it
drops straight into the existing `bytes_text_value!` list. `TransactionId` is
generated by `graph_id!`, like `EventId` and `PitchId`, and gets the same
traits.
**But do not just add two lines.** `bytes_text_value!` is a hand-maintained list
of 30 ids sitting parallel to the `graph_id!` invocations — the exact shape that
has cost this project four bugs, and `TransactionId`'s absence is that latent bug
*already biting*. The fix is to generate the `TextValue` impl **from `graph_id!`
itself**: every graph id is a byte-string leaf by definition, so the list that
declares them should be the list that projects them. `bytes_text_value!` then
keeps only the genuine non-`graph_id!` leaves (`ContentHash`, `TypedObjectId`).
Gate: the compile probe in §2 must pass for all four types.
---
## 4. Work breakdown
All of it lands in `epiphany-ops`, in a new `textproj` module group. The envelope
belongs with the crate that owns the envelope; the eventual `epiphany-textproj`
crate handles *document* lines (header, profile, extension, blob, canonical-base)
and needs `epiphany-bundle`, which is a later phase.
File boundaries are one-per-agent, and the type lists below are the **transitive
closure**, precomputed. Do not let an agent discover its dependencies from
compiler errors: `cargo check` reports only the frontier, which is how
`AnchorOffset`, `VoiceSelector`, `PowerOfTwo`, `OctaveOffset` and `NonZeroU16`
were each missed in the core fan-out.
### A — `textproj_leaf.rs` (one agent)
Ops-local leaves and the grammar's sub-vocabularies.
* **Leaves → byte strings.** `AuthorId` (`u128`), `ConflictId`, `EnvelopeHash`,
and the nine `registry_id!` types. As in P0, generate these **from the
`registry_id!` macro**, not from a parallel list.
* **Sub-vocabulary productions**, written to the grammar:
* `action``ResolutionAction`, 6 variants: `accept-loser`, `keep-winner`,
`dismiss`, `(override <bytes>)`, `(reanchor <bytes>)`,
`(registered <bytes>)`. Verified: names are the kebabs of the variants.
* `policy``UndoPolicy`: `strict-inverse`, `best-effort`, `cascade`.
* `tuplet-comp``TupletCompensation`, 4 variants.
* `cross-cutting``CrossCuttingValue`: `(tie <value>)` etc.
* `remapping``PositionRemapping`, with
`reassign-entry ::= "(" bytes " " ratio ")"` (a `(EventId, MusicalPosition)`
pair — the generic tuple impl in core already covers it).
* `TransactionCategory` — 5 variants including `Registered`.
Every `match` in a `project` must be exhaustive with no `_` arm.
### B — `textproj_kind.rs` (one agent; the largest)
The 31 `kind` productions, each inlining its `*Op` record positionally in
`CanonicalEncode` order.
* Drive the name from `OperationKindTag::catalog_name()` — production code,
generated by `operation_kind_tag_vocabulary!`. **Never** spell a kind name as a
literal; that list is already the single source and a parallel copy would
reintroduce the P4 defect.
* `parse` must dispatch on an exhaustive `match` over `OperationKindTag` so a kind
added to the vocabulary and not to the projector **fails to compile**.
* The three per-site order checks from §2. `TransposeOp`'s is a **multiset**:
reject strictly-decreasing, accept equal neighbours. Getting this backwards
silently breaks a frozen operation's replay.
### C — `textproj_envelope.rs` (one agent)
`envelope`, `stamp`, `causal`, `payload`, and the public entry points
`project_envelope(&OperationEnvelope) -> String` and
`parse_envelope(&str) -> Result<OperationEnvelope, …>`.
* `stamp` flattens `HybridLogicalClock` into three arguments — this is the
grammar's shape, and the ruling in §1 is what authorises it.
* `causal`'s `vector` is a `BTreeMap<ReplicaId, u64>` → map entries; `dots` is a
`BTreeSet<OperationId>`. Both already carry per-site strict-increase checks
from the core impls.
* **The whole-line guard.** `parse_envelope` should end with
`if parsed.project().render() != line { reject }`, mirroring
`decode_envelope`'s `to_canonical_bytes() != bytes`. Unlike in core, this one
may well be **live**, because several fields normalize. Whether it survives is
a question for mutation testing, not assertion — see §5.
---
## 5. Verification contract (non-negotiable)
Every phase gates on: `cargo fmt --all --check`; `cargo clippy --workspace
--all-targets` → 0; full workspace tests; `RUSTDOCFLAGS="-D warnings" cargo doc
--workspace --no-deps` → 0; `cargo run -q -p epiphany-testkit --example
conformance_suite` → 8/8; zero golden churn; `latexmk -xelatex` clean for any
touched spec document.
Beyond that, three things this track has learned the hard way:
1. **Mutation-verify every check**, asserting the anchor is present before
substituting — a `str.replace` that matches nothing looks exactly like a
passing test. Delete each order check and each guard in turn and confirm a
named test fails. In the core layer this found four guards that could never
fire; they were removed. Expect the same scrutiny here, and expect a different
answer for the whole-line guard.
2. **Exhaustive coverage, not sampled.** `gen_envelope_set` reaches only 28 of 31
kinds and 1 of 4 payload variants — a round-trip test built on it would prove
almost nothing. **Reuse `envdecode.rs`'s `sample_kind`**, which is an
exhaustive `match` over `OperationKindTag` and already builds a valid payload
for every kind, plus the three meta payloads. This is the single biggest
de-risking asset available.
3. **Two blind spots no round-trip test can see**, both already closed in core
and both reappearing here:
* *Field order* — a `project`/`parse` pair that agrees with itself on a wrong
order round-trips perfectly. Close it by mechanically diffing the identifier
sequence in each `CanonicalEncode::encode_canonical` against each `project`,
as was done for the 44 hand-written core impls.
* *Constructor names*`(insert-evnt …)` round-trips too. Here it is closed
for free **if** every name comes from `catalog_name()`; that is the reason
for the rule in §B.
---
## 6. Not in this phase
The document lines — `header`, `document`, `lineage`, `profile`, `extension`,
`canonical-base`, `blob` — plus `req:textproj:derived-ordering`'s sort-and-dedup
by projected form. Those need `epiphany-bundle`'s `Manifest`, `BlobRef`,
`ChunkRef`, `ProfileDeclaration`, `ExtensionDeclaration`, `SnapshotRef`, and
belong in a new `epiphany-textproj` crate. The text vector corpus and the
conformance step land with them, since a conformance vector is a whole document.