The audit's fourth push: the biggest outstanding Phase-2 item plus the performance gate. 793 workspace tests pass; clippy -D warnings, fmt, and rustdoc (deny-warnings) clean; all three spec documents build with zero undefined references. Binary Format companion (spec/binary_format.tex, v0.1.0 — Agent J's deliverable, 43 pages): - Twelve chapters transcribed from the golden-locked implementation: encoding conventions (the three prefix/endianness regimes, a normative no-varint rule, reject-never-normalize decode discipline), identifiers imported from the core spec's Canonical Byte-Layout Reference, primitive value encodings, the whole-Score positional codec ratified as the schema-major-0 wire form, operation wire forms (envelope field order with the normative id-leads property, the OperationPayload 0..=3 and OperationKind 0..=23 tables, effects/conflict/anomaly/MaterializedState vocabulary), the bundle physical layout (64-byte header, 256-byte superblock, chunk preimages and framing, ChunkRef, manifest body order), the operation-index payload, and the extension-blob/edit-barrier byte forms. - Ratifies P12-D1 (req:binfmt:opindex), P12-E1 (req:binfmt:ext-blobs), P12-E2 (req:binfmt:condition-depth, MAX_CONDITION_DEPTH = 64 normative), and P12-E3 (req:binfmt:object-kind-open) — batch rows struck through; discharges the provisional-codec notes in core (P11-4), ops, and bundle (P11-D2/D4/D5) DECISIONS with ratification cross-references. - Pins the frozen-layout schema-evolution keystone: within schema major 0 every positional struct layout is frozen; a field-set change is a schema-major change with migration — formally grounding the data-model-expansion staging decision. Open questions kept honest in-document: SnapshotId derivation, index-refresh threshold, u64/u32 prefix unification at the next major. - Not yet delivered from J's charter: the cross-implementation decoder test and the wire-format fuzzer (follow-up harnesses). F1 benches (crates/epiphany-testkit/benches/, per the F0 decision): - criterion 0.5.1 (workspace dev-dependency; MSRV 1.77 respected with documented transitive pins: clap 4.5.53, half 2.4.1). - reduction bench at 1K/10K/50K envelopes with the Chapter-10 budget (>10,000 envelopes/second cold) written in the bench as a Pass/Xfail gate; bundle benches for the typical-edit commit (<=50 ms; measured ~14.7 ms on real disk after catching that tmpfs neuters fsync) and the open/bootstrap read (<=200 ms; measured ~60 us). - CI: quick budget gates in the conformance job, full gates nightly. Subquadratic canonical_reduction_order (the F-surfaces/K-fixes handshake, closing K's 10K-envelope acceptance gate): - The bench documented the failure (50K at ~1.7K env/s, a 29 s cold reduction; two O(n^2) loops); the fix replaces pair enumeration with threshold/frontier readiness per replica plus explicit-dot dependent lists and a stamp-tuple binary heap — O((n + sum(context)) log n), never materializing covered pairs. - Byte-identical order: same edge relation, same ready predicate, same total order; the old implementation is retained as a test-only oracle with element-for-element order-equality property tests over fuzz sets, adversarial sets, and directed shapes (2,000-envelope full-coverage chains, dot cycles, duplicate-id stamp ties), mutation-tested for sensitivity. - Measured: 1K 155K->674K env/s, 10K 12.5K->257K, 50K 1.7K->87K; all three scale points now pass and the 50K row is promoted from Xfail. Also: fixed nine rustdoc private/unresolved intra-doc links that had accumulated across the pushes (the CI deny-doc-warnings job would have failed on them). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NEs4aYiu8MXjdYdMxw8PTd |
||
|---|---|---|
| .. | ||
| examples | ||
| src | ||
| tests | ||
| Cargo.toml | ||
| DECISIONS.md | ||
| README.md | ||
README.md
epiphany-ops
The Epiphany concurrent semantics: the operations through which the score
graph becomes a live model, and the deterministic reduction by which a set of
operations becomes a materialized score state. Implements the normative
requirements of Chapter 6 (Semantic Operations and Concurrent Reduction) of
the core specification (spec/core_spec.pdf). This is Agent C's crate per
spec/QUICKSTART.md, building on Agent A's epiphany-determinism and Agent B's
epiphany-core.
A score's state is defined by the set of operations committed to it. Any materialized graph is a deterministic reduction of that set; caches, snapshots, and partial reductions are acceleration structures, never the source of truth. — Chapter 6, Design Principles
The thesis in one paragraph
The replicated operation set is a grow-only CRDT: replicas accumulate envelopes and converge on the same set. The materialized graph is not a CRDT — it is the deterministic reduction of that set in a single canonical order (causal-first, then the HLC tuple). Any permutation of the same envelopes reduces to byte-identical materialized state. That property is the determinism heart of the architecture, and the reduction fuzzer is its tripwire.
What's here
| Area | Items | Spec |
|---|---|---|
| Stamps | HybridLogicalClock, OperationStamp, the reduction & monotonicity tuples |
Ch. 6 §"Operation Identity and Stamps" |
| Causal context | CausalContext (dotted version vector), covers, the missing-predecessor signal |
Ch. 6 §6.2 |
| Payloads | OperationKind, the discriminator-only OperationKindTag, OperationPayload, the §6.10 representative ops |
Ch. 6 §"Operation Envelopes", §6.10 |
| Envelopes | OperationEnvelope, EnvelopeHash (MUSCENVH), well_formed (incl. stamp.id == id) |
Ch. 6 §6.4 |
| Slots | OperationSlot::{Single, Equivocated}, the order-independent (Pass-10) transitions |
Ch. 6 §6.5 |
| Anomalies | AnomalousReplicaSegment, IntegrityAnomaly/Kind, the HLC-monotonicity detector |
Ch. 6 §6.6; Ch. 5 §"System-Derived Counter Collisions" |
| Effects | OperationEffect, NoOpReason, the typed PreconditionFailureReason, RepairRecord/RepairKind |
Ch. 6 §6.3.2, §6.5 |
| Conflicts | ConflictRecord, ConflictKind, content-derived ConflictId (derive_conflict_id), the registry, resolution |
Ch. 6 §6.4 |
| Transactions / undo | TransactionDescriptor with the causal-prior-descriptor rule, UndoTransactionPayload / UndoPolicy |
Ch. 6 §6.6, §6.8 |
| Operation set | OperationSet: accept pipeline (well-formedness → slot → causal), grow-only |
Ch. 6 §"Envelope Acceptance" |
| Reduction | canonical_reduction_order (single function), MaterializedState, the reduction driver |
Ch. 6 §6.3 |
The determinism this crate enforces
- A single reduction-order function.
canonical_reduction_orderperforms deterministic causal topological ordering, using the intrinsic stamp tuple(physical, logical, replica, counter)only among ready operations. - Order-independent equivocation. A duplicate
OperationIdwith different canonical bytes transitions its slot toEquivocatedregardless of which envelope arrived first (Pass 10). Equivocated slots contribute nothing to reduction; dependents are held pending. - Content-derived facts.
ConflictIdandIntegrityAnomalyIdare derived from content, so two replicas reducing the same set agree on every conflict and anomaly id — the conflict registry and anomaly register are deterministic materialized facts, not local bookkeeping. - Byte-identical materialized state.
MaterializedState::canonical_bytesserializes the effect log, conflict registry, anomaly register, object existence, spellings, and LWW fields in their normative orders. - Real graph materialization.
OperationSet::reduce_onto(&base_score)returnsGraphMaterialization { state, score }. The graph is mutated in the same canonical order and compares by canonical event identity, independent of arena storage order.
Hand-off gates
Run the gate harnesses (QUICKSTART, Agent C):
cargo test -p epiphany-ops
cargo run --release -p epiphany-ops --example fuzz_reduction # 10k iters, seed 0
cargo run --release -p epiphany-ops --example fuzz_reduction 100000 7 # soak, seed 7
- Reduction determinism — every randomized envelope set reduces to byte-identical materialized state under any acceptance order (v0 acceptance criteria 1 and 5).
- Equivocation order-independence — every duplicate-id-with-different-bytes scenario equivocates regardless of arrival order (v0 acceptance criterion 3).
The integration tests (tests/concurrent_reduction.rs) exercise these plus
transaction atomicity, descriptor precedence, anomaly exclusion, and forward
undo through the public API.
Scope and decisions
Chapter 6 specifies the framework and a representative selection of
operations; the full ~60–80-primitive catalog is an explicit open question
(§6.11) deferred to the Operation Catalog companion. This crate implements the
framework in full and the representative operations, which is sufficient to
exercise every reduction discipline. The representative operations can also
reduce onto an epiphany_core::Score: insert/delete, voice promotion, supported
cross-cutting structures, system breaks, migration checks, transaction
rollback, and undo mutate the real graph while preserving Agent B's invariants.
reduce() remains the base-free CRDT/bookkeeping API; reduce_onto() is the
graph-aware editing path. See DECISIONS.md for remaining payload boundaries.
Per QUICKSTART "Don't do these": undo is the spec's forward compensating
operation, never inverse-based; unsafe is forbidden; everything is sync.