epiphany/crates/epiphany-ops
Levi Neuwirth a41596d329 Push 5 / P4: the decode conformance corpus, and the tag it could not read back
spec/vectors/decode_vectors.txt -- 37 committed byte strings across five
surfaces, each with its normative accept/reject verdict. The reference
implementation's fuzzers prove its own decoders self-consistent, which says
nothing about whether a foreign decoder agrees with the format. This is what one
is checked against. Gated in the conformance suite as [7d], and drift-locked:
the committed file must equal vectors::render(), so a wire-format change lands
in the diff.

It found a real defect on its first run. OperationKindTag::TransposeInterval
encoded to [30] and its own decoder REJECTED it -- Push 4a added the variant to
discriminant() and never to decode_canonical. OperationKindTag is what edit
barriers persist, so a barrier prohibiting TransposeInterval could be written and
never read back. Silent data loss on reopen.

Four things should have caught it. None did, and two made it worse:

  The round-trip test enumerated DISCRIMINANTS -- (0u8..30).map(decode_canonical)
  -- starting from bytes the decoder already knew, so it structurally could not
  notice a variant the decoder was missing. It now enumerates VARIANTS from one
  all_tags() list, with a completeness check in both directions.

  The distinctness test's hand-written variant list omitted it too. Same list now.

  operation_kind_tag_decode_rejects_malformed_bytes asserted that tag 30 is
  REJECTED, and layout-ir's decode_rejects_unknown_discriminants asserted the
  same at the barrier surface. Both were locking the bug in place and made it
  look deliberate. Both now name 31, and a new barrier test round-trips a barrier
  prohibiting every tag -- the persistence surface where this actually bites.

  The P2 decode fuzzer fed valid corpus bytes to the tag decoder and tallied the
  failure as a REJECTION, like any garbage input. It never asserted that an
  unmutated corpus entry decodes. Both fuzzers now do, as a pre-pass.

The harness had the same disease as the code. `check` collapsed "rejected" with
"accepted but does not re-encode", so a decoder that silently normalizes
non-canonical bytes PASSED the reject vectors it was written to catch. Verified:
removing the whole-state guard, and restoring the lenient compression codec, both
left the corpus green. `check` now returns Ok(injective) for accept and Err for
reject and never conflates them -- silently normalizing non-canonical bytes IS
accepting them. With that fixed, all four defect mutations fail the corpus, each
naming its class.

The corpus pins one vector per class this repo has shipped a bug in:
non-canonical-map-order (a guard catches it; no per-site check exists),
non-canonical-vec-order (only a per-site check catches it; a guard is blind),
lenient-sub-codec (a guard masked it in the manifest; the index had none), plus
trailing-bytes, truncated, unknown-discriminant, count-exceeds-remaining. A test
fails if one goes missing.

Gate: fmt clean, clippy 0, 30 targets / 1024 passed / 0 failed, docs 0 under
-D warnings, conformance 8/8 (now including [7d]), zero golden churn.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 20:16:15 -04:00
..
examples A B C D F 2026-06-19 12:42:31 -04:00
src Push 5 / P4: the decode conformance corpus, and the tag it could not read back 2026-07-09 20:16:15 -04:00
tests Schema major 2 Phase B: snapshot side + honest stamps (data-model fills) 2026-07-07 16:30:07 -04:00
Cargo.toml A B C D F 2026-06-19 12:42:31 -04:00
DECISIONS.md Push 5 / P4: the decode conformance corpus, and the tag it could not read back 2026-07-09 20:16:15 -04:00
README.md Land M1 + M2 (Agent C): framework edge fixes and real-Score graph integration 2026-06-21 16:37:51 -04:00

README.md

epiphany-ops

The Epiphany concurrent semantics: the operations through which the score graph becomes a live model, and the deterministic reduction by which a set of operations becomes a materialized score state. Implements the normative requirements of Chapter 6 (Semantic Operations and Concurrent Reduction) of the core specification (spec/core_spec.pdf). This is Agent C's crate per spec/QUICKSTART.md, building on Agent A's epiphany-determinism and Agent B's epiphany-core.

A score's state is defined by the set of operations committed to it. Any materialized graph is a deterministic reduction of that set; caches, snapshots, and partial reductions are acceleration structures, never the source of truth. — Chapter 6, Design Principles

The thesis in one paragraph

The replicated operation set is a grow-only CRDT: replicas accumulate envelopes and converge on the same set. The materialized graph is not a CRDT — it is the deterministic reduction of that set in a single canonical order (causal-first, then the HLC tuple). Any permutation of the same envelopes reduces to byte-identical materialized state. That property is the determinism heart of the architecture, and the reduction fuzzer is its tripwire.

What's here

Area Items Spec
Stamps HybridLogicalClock, OperationStamp, the reduction & monotonicity tuples Ch. 6 §"Operation Identity and Stamps"
Causal context CausalContext (dotted version vector), covers, the missing-predecessor signal Ch. 6 §6.2
Payloads OperationKind, the discriminator-only OperationKindTag, OperationPayload, the §6.10 representative ops Ch. 6 §"Operation Envelopes", §6.10
Envelopes OperationEnvelope, EnvelopeHash (MUSCENVH), well_formed (incl. stamp.id == id) Ch. 6 §6.4
Slots OperationSlot::{Single, Equivocated}, the order-independent (Pass-10) transitions Ch. 6 §6.5
Anomalies AnomalousReplicaSegment, IntegrityAnomaly/Kind, the HLC-monotonicity detector Ch. 6 §6.6; Ch. 5 §"System-Derived Counter Collisions"
Effects OperationEffect, NoOpReason, the typed PreconditionFailureReason, RepairRecord/RepairKind Ch. 6 §6.3.2, §6.5
Conflicts ConflictRecord, ConflictKind, content-derived ConflictId (derive_conflict_id), the registry, resolution Ch. 6 §6.4
Transactions / undo TransactionDescriptor with the causal-prior-descriptor rule, UndoTransactionPayload / UndoPolicy Ch. 6 §6.6, §6.8
Operation set OperationSet: accept pipeline (well-formedness → slot → causal), grow-only Ch. 6 §"Envelope Acceptance"
Reduction canonical_reduction_order (single function), MaterializedState, the reduction driver Ch. 6 §6.3

The determinism this crate enforces

  1. A single reduction-order function. canonical_reduction_order performs deterministic causal topological ordering, using the intrinsic stamp tuple (physical, logical, replica, counter) only among ready operations.
  2. Order-independent equivocation. A duplicate OperationId with different canonical bytes transitions its slot to Equivocated regardless of which envelope arrived first (Pass 10). Equivocated slots contribute nothing to reduction; dependents are held pending.
  3. Content-derived facts. ConflictId and IntegrityAnomalyId are derived from content, so two replicas reducing the same set agree on every conflict and anomaly id — the conflict registry and anomaly register are deterministic materialized facts, not local bookkeeping.
  4. Byte-identical materialized state. MaterializedState::canonical_bytes serializes the effect log, conflict registry, anomaly register, object existence, spellings, and LWW fields in their normative orders.
  5. Real graph materialization. OperationSet::reduce_onto(&base_score) returns GraphMaterialization { state, score }. The graph is mutated in the same canonical order and compares by canonical event identity, independent of arena storage order.

Hand-off gates

Run the gate harnesses (QUICKSTART, Agent C):

cargo test -p epiphany-ops
cargo run --release -p epiphany-ops --example fuzz_reduction          # 10k iters, seed 0
cargo run --release -p epiphany-ops --example fuzz_reduction 100000 7 # soak, seed 7
  • Reduction determinism — every randomized envelope set reduces to byte-identical materialized state under any acceptance order (v0 acceptance criteria 1 and 5).
  • Equivocation order-independence — every duplicate-id-with-different-bytes scenario equivocates regardless of arrival order (v0 acceptance criterion 3).

The integration tests (tests/concurrent_reduction.rs) exercise these plus transaction atomicity, descriptor precedence, anomaly exclusion, and forward undo through the public API.

Scope and decisions

Chapter 6 specifies the framework and a representative selection of operations; the full ~6080-primitive catalog is an explicit open question (§6.11) deferred to the Operation Catalog companion. This crate implements the framework in full and the representative operations, which is sufficient to exercise every reduction discipline. The representative operations can also reduce onto an epiphany_core::Score: insert/delete, voice promotion, supported cross-cutting structures, system breaks, migration checks, transaction rollback, and undo mutate the real graph while preserving Agent B's invariants. reduce() remains the base-free CRDT/bookkeeping API; reduce_onto() is the graph-aware editing path. See DECISIONS.md for remaining payload boundaries.

Per QUICKSTART "Don't do these": undo is the spec's forward compensating operation, never inverse-based; unsafe is forbidden; everything is sync.