0.1.0 left kind, action, policy, constraints and barrier derived-but-unwritten
and admitted it. All are now written. The grammar has no undefined nonterminal
(machine-checked) and 31 operation-kind productions in exact discriminant order,
cross-checked against envdecode.rs.
The one real decision was how embedded Chapter-5 values appear. An operation
payload carries an Event, a Pitch, a Region, a TimeSignature, and there were
three ways to write them:
Forty hand-written productions would restate the entire Chapter-5 data model in
a second normative document -- two normative listings of one struct, which is
the exact drift P13-I1 was opened to close.
Opaque canonical-value byte strings would be lossless and zero-drift, but a
pitch would be unreadable without binary tooling, failing the core spec's own
"format inspection and debugging" use case.
One mechanical rule (req:textproj:value-projection, ratified): a struct is
(<type-name> <field>...) with fields positional in the ratified declaration
order; a newtype is transparent, exactly as in the binary form; a tagged union
is (<variant> <field>...); an option is () or (some v); a sequence keeps the
binary form's order. A rule cannot drift from what it reads.
Two leaf decisions follow from canonicality rather than taste. A rational is
(ratio n d), lowest terms, sign on the numerator. A CanonicalF64 is the byte
string of its eight canonical IEEE-754 bytes and never a decimal: decimal float
text is not canonically unique -- shortest-round-trip and 17-significant-digit
forms both round-trip, and -0.0 has two spellings -- so a decimal tempo would
break req:textproj:canonical-text at the first tempo mark.
Operation-kind names follow the Operation Catalog's sections (create-region,
create-staff), not OperationKindTag's (InsertRegion, InsertStaff), which renamed
three pairs for reasons internal to the tag space. The projection follows the
semantics.
One deferral, stated as such rather than left to inference:
affected_object_kinds and edit_barriers have ratified structured shapes AND
canonical byte encodings, and the bundle stores them opaquely. At 0.3.0 the
projection does the same, on the principle that it interprets nothing the bundle
does not. A later revision may project them structurally; their canonical bytes
are unchanged by that, so it changes the text and not the document.
Also corrected: the extension line now writes its fields in the ratified
declaration order, which had preserved_chunk_roots before affected_object_kinds
and edit_barriers.
Gate: clippy 0, 31 targets / 1031 passed / 0 failed, conformance 8/8, zero golden
churn; core_spec, binary_format and text_projection all build with no undefined
references. Grammar closure machine-checked: no undefined nonterminal.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A review found the 0.1.0 companion lossy for documents that are valid today. Its
claim to preserve the manifest's canonical roots was false three ways:
A canonical blob had no representation at all. blob_roots referenced by
canonical operations or reduced state ARE canonical roots, and the document
structure had no blob line. An embedded image, font or recording would vanish
from a projection silently -- the operations referencing it still there,
pointing at a blob id the text no longer contained.
An ExtensionDeclaration lost its semantic version and its affected_object_kinds
outright, and left its preserved_chunk_roots undefined.
ProfileId::Custom(ProfileRegistryId) was unrepresentable: a symbol was required
where sixteen registry bytes are carried.
All three share one cause I had not named. A ChunkRef and a BlobRef are PHYSICAL
references -- offset, compressed length, compression -- which the projection may
not preserve; and they carry DERIVABLE identities -- ChunkId, ContentHash, BlobId
-- which it may not duplicate. Having no rule for either, I dropped the
references and took their contents with them.
req:textproj:derive-or-carry states it, and it is the same rule
req:textproj:reduced-state-derived already applied one level up: carry exactly
what the document does not determine, and nothing it does. Physical attributes
never appear; derivable identities never appear; content and semantic attributes
always do. The one non-derivable identity in schema major 0 is SnapshotId, which
the Binary Format companion pins as opaque and forbids readers to derive -- an
exception for a stated reason rather than an oversight. The grammar now visibly
contains no offset, no length, no compression, no chunk id, no hash, no blob id.
So: req:textproj:canonical-blobs (canonical blobs projected, non-canonical ones
not), req:textproj:profile-id ((custom #x...)), req:textproj:extension-declaration
(every field; chunks as kind + schema + payload, never as a ChunkRef -- the
projection has no file to point into), and req:textproj:base-snapshot-inline
extended to say what the inlined payload IS and that the root ChunkRef and the
SnapshotRef hash are re-derived from hash(Snapshot, schema, payload), never read.
The gap started upstream. core_spec's own list of what the projection preserves
omitted canonical blobs while classifying blob_roots as canonical roots -- an
inconsistency inside one document. Corrected there, along with withdrawing the
permission to reference a base snapshot "externally", which the inline
ratification had already made untenable.
Also: the generated PDF metadata and page header still said Operation Catalog,
inherited from the copied preamble.
The four 0.1.0 ratifications stand unchanged. Implementation stays deferred: a
gate that is lossy is not a gate.
Gate: clippy 0, 31 targets / 1031 passed / 0 failed, conformance 8/8, zero golden
churn; core_spec, binary_format, operation_catalog and text_projection all build
with no undefined references.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
core_spec Chapter 8 declares the text projection normative and leaves the form
unwritten; binary_format excludes it as "the Text Projection companion's". That
companion did not exist. spec/text_projection.tex v0.1.0 supplies it. No
implementation -- this is the gate.
It was blocked on P5. The projection's normative requirement is bidirectionality
WITH THE BINARY FORM, which needs bytes -> OperationEnvelope, and that decoder
did not exist until 3baf8d0.
Four ratified calls:
Reduced state is preserved by DETERMINING it, never by a second literal copy. It
is a deterministic function of the operation set and the canonical base; a text
carrying both would hold two sources of truth for one fact and nothing could stop
them disagreeing. core_spec's "all canonical reduced state" now carries that
reading inline rather than being silently reinterpreted.
A canonical base snapshot is INLINED as one opaque byte string. This is the call
with teeth. A base exists precisely so prior operations need not be retained, and
where they are pruned the base is derivable from nothing else -- so a
reference-only projection of a compacted document would be LOSSY, and the text
would not determine the document it claims to project. core_spec permits "encoded
compactly or referenced externally"; inline is the choice that keeps archival
honest, and core_spec now says why.
Lowercase hex everywhere: one rule, no alphabet or padding to canonicalize,
greppable. Base64 would buy a quarter of the bytes of the one body nobody reads,
at the price of a second encoding and a rule for which applies where.
One envelope per line: the stated use case is that merge conflicts surface at the
envelope level, and one line per envelope makes a three-way merge conflict
exactly an envelope conflict -- never a conflict inside one, yielding an operation
neither side wrote. It also removes all indentation, so canonicality has nothing
to hide in. Readability is a pretty-printer's job; what a pretty-printer must not
do is write its output back and call it a projection.
Strict parsing is stated in the terms P2-P5 taught: normalizing non-canonical
text IS accepting it. The rationale names both hazards this repo hit in binary --
a re-encode guard is blind to order-preserving sequences, and a guard on an outer
value can mask a lenient inner codec -- and prescribes the same total defence.
Conformance requires both directions: a projector alone cannot be checked.
One gap is stated in the document rather than left to be discovered: the atom
productions and line shapes are normative, while kind/action/policy/constraints/
barrier are derived from the Operation Catalog and the wire table rather than
spelled out. That is the difference between a design gate and a finished
companion.
Gate: clippy 0, 31 targets / 1031 passed / 0 failed, conformance 8/8, zero golden
churn; core_spec, binary_format and the new companion all build with no undefined
references.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two review findings after P4. Both real.
The tag-omission failure could recur. After the P4 fix there were STILL four
hand-maintained lists -- a test-local all_tags(), a barrier test spelling
0u8..=30, a fuzz corpus naming five tags, a vector corpus naming four -- plus a
malformed-bytes test asserting that 31 rejects. A future tag 31 added to
discriminant() and omitted from decode_canonical() would have left every one of
them green, and the malformed test would have LOCKED it, exactly as the 30
version did two commits ago.
operation_kind_tag_vocabulary! is now the single source. It generates
discriminant, from_discriminant, and OperationKindTag::PAYLOAD_FREE from one
list, and the generated discriminant match is exhaustive over the enum -- so a
variant added to the enum and not to the macro fails to COMPILE. Everything
downstream reads PAYLOAD_FREE: the decoder, the fuzz corpus (all 31 tags, not
five), the conformance vectors (an accept vector per tag: 65 vectors, not 37),
and the edit-barrier round-trip. Every "one past the vocabulary" constant is
computed, never spelled; a spelled constant is the trap that springs on whoever
appends the next tag.
Verified end to end with a hypothetical tag 31. Added to the enum alone: compile
error. Added to the enum and the macro: it compiles, decodes, and every derived
check passes because they read PAYLOAD_FREE -- while the committed corpus's
drift lock AND its now-stale "one past the vocabulary" reject vector both fail,
forcing the new vectors into the diff. There is no path where a new tag leaves
everything green.
Second finding: the corpus called itself normative while the spec said it was
deferred. Binary Format's "About This Companion" listed the cross-implementation
decoder test among things the document does not cover, and the Golden Anchor
Registry called it "the deferred conformance harness" whose literal-byte vectors
a future test "should add". Both now ratify it. New req:binfmt:decode-vectors
and a "The Decode Vector Corpus" section: a conforming decoder MUST accept every
accept vector for a surface it implements, MUST reject every reject vector, and
MUST re-encode an accepted value to exactly its bytes -- and accepting a reject
vector and then normalizing it IS accepting it. Binary Format 0.8.0 -> 0.9.0.
The wire-format fuzzer stays an implementation deliverable. The corpus header
now cites the requirement instead of asserting one.
Gate: fmt clean, clippy 0, 30 targets / 1024 passed / 0 failed, docs 0 under
-D warnings, conformance 8/8 with [7d] at 65 vectors, zero golden churn,
binary_format rebuilds with no undefined references.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
spec/vectors/decode_vectors.txt -- 37 committed byte strings across five
surfaces, each with its normative accept/reject verdict. The reference
implementation's fuzzers prove its own decoders self-consistent, which says
nothing about whether a foreign decoder agrees with the format. This is what one
is checked against. Gated in the conformance suite as [7d], and drift-locked:
the committed file must equal vectors::render(), so a wire-format change lands
in the diff.
It found a real defect on its first run. OperationKindTag::TransposeInterval
encoded to [30] and its own decoder REJECTED it -- Push 4a added the variant to
discriminant() and never to decode_canonical. OperationKindTag is what edit
barriers persist, so a barrier prohibiting TransposeInterval could be written and
never read back. Silent data loss on reopen.
Four things should have caught it. None did, and two made it worse:
The round-trip test enumerated DISCRIMINANTS -- (0u8..30).map(decode_canonical)
-- starting from bytes the decoder already knew, so it structurally could not
notice a variant the decoder was missing. It now enumerates VARIANTS from one
all_tags() list, with a completeness check in both directions.
The distinctness test's hand-written variant list omitted it too. Same list now.
operation_kind_tag_decode_rejects_malformed_bytes asserted that tag 30 is
REJECTED, and layout-ir's decode_rejects_unknown_discriminants asserted the
same at the barrier surface. Both were locking the bug in place and made it
look deliberate. Both now name 31, and a new barrier test round-trips a barrier
prohibiting every tag -- the persistence surface where this actually bites.
The P2 decode fuzzer fed valid corpus bytes to the tag decoder and tallied the
failure as a REJECTION, like any garbage input. It never asserted that an
unmutated corpus entry decodes. Both fuzzers now do, as a pre-pass.
The harness had the same disease as the code. `check` collapsed "rejected" with
"accepted but does not re-encode", so a decoder that silently normalizes
non-canonical bytes PASSED the reject vectors it was written to catch. Verified:
removing the whole-state guard, and restoring the lenient compression codec, both
left the corpus green. `check` now returns Ok(injective) for accept and Err for
reject and never conflates them -- silently normalizing non-canonical bytes IS
accepting them. With that fixed, all four defect mutations fail the corpus, each
naming its class.
The corpus pins one vector per class this repo has shipped a bug in:
non-canonical-map-order (a guard catches it; no per-site check exists),
non-canonical-vec-order (only a per-site check catches it; a guard is blind),
lenient-sub-codec (a guard masked it in the manifest; the index had none), plus
trailing-bytes, truncated, unknown-discriminant, count-exceeds-remaining. A test
fails if one goes missing.
Gate: fmt clean, clippy 0, 30 targets / 1024 passed / 0 failed, docs 0 under
-D warnings, conformance 8/8 (now including [7d]), zero golden churn.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Audit correction. The manifest test was documented -- in its own doc comment, in
DECISIONS.md, and in P3's commit message -- as "exhaustive single-byte
perturbation". It tried three XOR deltas per byte. The claim was false as
executed, and the "guard is total" conclusion leaned on it.
Both tests now do what their names say:
every_single_byte_replacement_of_a_manifest_is_rejected -- each byte, each of
the 255 other values. 0.15s.
compression_none_rejects_a_non_zero_parameter_byte -- every one of the 255
non-zero parameter bytes, plus a round-trip of all 256 values through Zstd and
Reserved, so the strictness is shown to be confined to None.
And the totality claim is re-seated where it belongs: on the argument, not on a
finite test. manifest_id is derived from the body, so a body edit fails the id
check and an id edit fails the derivation; encode_body sorts and deduplicates
every vector, so an out-of-order or duplicated encoding cannot round-trip. The
test is evidence for that argument over ONE constructed manifest, and is blind
to multi-byte perturbations entirely. Both the doc comment and DECISIONS.md now
say so.
Worth recording: restoring the leniency fails the codec test and the index test,
and leaves the manifest test GREEN -- the guard rejects those bytes whatever the
sub-codec does. That is not a weak test. It is the asymmetry that hid the bug,
and it locks the guard rather than the codec. A suite where every test fails on
every mutation would be telling us less.
No codec or wire-format change; the strict branch was already correct.
Gate: fmt clean, clippy 0, 30 targets / 1012 passed / 0 failed, docs 0 under
-D warnings, conformance 8/8, zero golden churn.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A wire-decode fuzzer over Bundle::open, Manifest::decode, OperationIndex::decode,
decode_block and envelope_offsets. The existing crash-recovery fuzzer corrupts an
image the way a CRASH does -- torn writes at syscall boundaries. This one
corrupts it the way an attacker or a bit-rotted disk does: arbitrary bytes,
anywhere.
It found a real defect. CompressionAlgorithm::None read its parameter byte and
DISCARDED it, while encode writes zero. So [0, 0xFF] and [0, 0] both decoded to
None, and the first re-encoded to the second: a lenient, non-injective codec,
inherited by every structure embedding a ChunkRef.
Whether that was visible depended entirely on the embedder:
Manifest::decode has a whole-value re-encode guard, and it is TOTAL -- proved
by exhaustive single-byte perturbation, every one rejected. It caught this.
OperationIndex::decode has no guard; it validates per-site. It accepted both
byte strings, while its own doc promised to "reject (never normalizing) any
non-canonical form". That promise was false.
That is the same two-layer lesson P2 recorded one commit ago, from the other
side: a re-encode guard is complete only where the encoder normalizes, and its
completeness can MASK a lenient sub-codec rather than fix it. Fixed at the
source, not papered over at the index. An exhaustive sweep -- every byte, every
value, plus an 8-byte extreme-integer window -- finds no remaining non-injective
site.
The fix contradicted ratified spec text, which said the byte was "present but
zero, and ignored on read". Escalated rather than fixed unilaterally. The user
ratified strict decode: core spec's clause is superseded, Binary Format gains
req:binfmt:compression-none-parameter and moves 0.7.0 -> 0.8.0. No wire layout
changed, and no conforming writer emits a non-zero byte, so this rejects only
corrupt or adversarial input -- no existing file changes meaning.
Coverage was the harness's problem again. The fuzzer's first run reached the
operation index's accept path ZERO times -- random bytes never decode as an
index -- so every assertion under it was vacuous. It found the bug only once the
index corpus was built from real OperationIndex::build output. The smoke tests
now assert on a WireFuzzCoverage so that cannot silently regress. 1.5M inputs
across five seeds, ~1s each, clean after the fix.
Three regressions, each mutation-verified by restoring the leniency: the codec
itself, the index that exposed it, and the manifest guard's totality -- which is
the asymmetry that hid it.
Gate: fmt clean, clippy 0, 30 targets / 1012 passed / 0 failed, docs 0 under
-D warnings, conformance 8/8, zero golden churn, both spec documents rebuild
with no undefined references.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Review findings on 794400c. The criterion-4 harness stored current
(v2) Score bytes stamped V0 in the canonical_base slot and decoded
them with the unversioned decoder — bypassing the schema-major
snapshot contract Phase B built, and misusing the base role. Fixed
substantively: the harness stages a properly-roled ACCELERATION
snapshot (ChunkKind::Snapshot stamped for_major(2), referenced from
Manifest::acceleration_snapshots) and decodes through
Score::decode_canonical_versioned keyed by the stamped major — the
versioned seam now exercised end-to-end through the bundle, giving
the snapshot role its first real producer/consumer.
Consequences: max_supported_major(Snapshot) -> 2; and since the
per-kind gate no longer implies it, the canonical-base-stays-major-0
rule is enforced per ROLE (mis_stamped_canonical_base, consulted at
open and commit -> read-only + UnsupportedCanonicalChunkMajor;
regression test covers commit and reopen).
Also: three stale-doc fixes (decode_canonical now "current (schema
major 2)"; CreateRegionOp + StagedChunk::operation_block +
stage_operation_block docs describe minimal stamping, not just "v1
CreateRegion").
Full gate green: fmt, clippy -D warnings, rustdoc -D warnings, 30
workspace suites, conformance scale 1 (8/8).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NEs4aYiu8MXjdYdMxw8PTd
The nine type bodies fill to the ratified Ch5 shapes and the wire moves
to the Binary Format §Schema Major 2 layouts, review-hardened (high-
effort review; 8 findings, 7 fixed + 1 deferral sharpened).
epiphany-core:
- 19 new leaf types in graph.rs (SlurKind/CurveDirection/
CurvatureOverride/SpanStyle/LineStyle/SpaceUnit/SubBeam/
BeamGeometryOverride/SpannerKind+5 payload types/RepeatKind/Volta/
StaffBracketKind/MetadataEntry/MetadataValue/Timestamp/
SoundConfiguration/TranspositionInterval/UnpitchedMember) with the
ratified discriminants (cstyle_enum_codec! reused; tagged unions
hand-written); nine struct fills appended in wire order.
- The frozen wire forms generalized to a SHARED sub-codec layer
(enc_/dec_*_v1, v0==v1 for every type major 2 changed; vec framing
through enc_/dec_vec_v1) used by the new decode_v1_score/
encode_v1_score AND the rerouted v0 pair — major 2 touched types the
v0 walk had treated as unchanged (metadata, staves, cross_cutting,
staff instances transitively). Strict-canonical guards on every
versioned path; decode_canonical_versioned dispatches {0,1,2} with
composed default-fill migration.
- Invariants extended to the new reference-bearing fields (REVIEW
FIX): Beam.sub_beams events, RepeatKind DaCapo/DalSegno anchors,
Volta spans now covered by CrossCuttingRefsResolve + the anchor
model walk, with negative tests.
- Tests: v1 migration size-anchor (v1 omits exactly the appended
default bytes — the frozen encoder cannot drift), a non-default
round-trip covering every new field and every SpannerKind/RepeatKind
wire arm, fuzzer corpus gains genuine-v1 forms + the major-2 seam
with enforced must-decode-Ok on unmutated frozen forms.
epiphany-ops + epiphany-bundle (Phase C's semantic core, landed here
deliberately — the live codec flip makes CrossCutting/Staff/Metadata
payload bytes v2 immediately; shimming nine transitively-embedded
types the major-1 D1 way was throwaway):
- Minimal-stamping OperationKind::schema_major per the ratified rule
(CrossCutting/CreateStaff/SetMetadata always 2; CreateRegion 2 iff a
carried instance bears Some(staff_lines_override) else 1;
CreateStaffInstance/SetStaffLayout 2 iff Some else 0), unit-locked.
- Bundle op-block accept-set [0,2]; SchemaVersion::V2;
beyond-accept-set tests moved to major 3; testkit V2
stamp-derivation test.
- the_canonical_base_is_byte_identical_across_data_model_majors:
pinned blake3 of a seeded reduction — the companion's SHOULD that
the canonical base never moves across data-model majors.
- The op-payload migrate-on-read deferral restated precisely in
DECISIONS (no consumer byte-reconstructs op payloads today; the
first one must bring per-type frozen payload decoders).
Zero golden churn (fixtures deliberately carry v2 defaults).
Instrument::new consolidates the sweep's default fills. Full gate:
fmt, clippy -D warnings, rustdoc -D warnings, 30 workspace suites,
conformance scale 1 (8/8).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NEs4aYiu8MXjdYdMxw8PTd
The canonical op-block side of Region.permits_spanning_slurs: CreateRegion now
encodes at schema major 1, blocks carrying one are stamped major 1, the reader
admits them per-role, and a bundle whose op history is beyond this reader's
accept-set opens read-only. The migrate-on-read primitive is deferred (op
payloads are never reconstructed-to-values from bytes, so it has no consumer).
- ops: CreateRegionOp::encode_canonical embeds the region's full (v1) canonical
bytes; OperationKind/OperationPayload/OperationEnvelope::schema_major report
the payload's binary-format major (CreateRegion => 1, else 0). Removed D1's
transitional Region::canonical_bytes_v0 (dec_region_v0 stays for snapshots).
- bundle: max_supported_major(kind) raises the OperationEnvelopeBlock role to
[0,1] (every other role stays exact-0); the read gate is now
major > max_supported_major(r.kind). StagedChunk::operation_block_versioned +
SchemaVersion::for_major project a derived block major to a version.
- bundle: commit-time canonical-root validation checks structure without the
accept-set (a newer writer's higher-major root is publishable); the accept-set
is a read concern. Both open and commit consult
unsupported_operation_root_major and go read-only (+ the new
IntegrityAnomaly::UnsupportedCanonicalChunkMajor) when a canonical op root
exceeds the accept-set, so the live bundle refuses further commits at once.
- testkit: stage_operation_block derives a block's schema version from its
operations (max schema_major); staged_envelope_blocks routes through it so a
generated CreateRegion stream is never mis-stamped v0.
Tests: CreateRegion payload is v1 and carries the flag; the op reports major 1;
a derived CreateRegion block stamps V1 and reopens read-write; a major-2 block
opens read-only (open and post-commit); the per-role accept-set shape. Full gate
green (workspace tests, clippy -D warnings, fmt, rustdoc -D warnings).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NEs4aYiu8MXjdYdMxw8PTd
Stands up the schema-evolution machinery as a tested behavioral no-op, the
load-bearing one-way-door piece the later phases build on.
- SchemaVersion::V1 (bundle/ids.rs) -- infrastructure, an identity today.
- The core dispatch seam Score::decode_canonical_versioned(bytes, major)
(core/codec.rs), with the decode_v0_score / migrate_v0_score scaffold. It is
the identity at major 1's introduction (v0 layout == v1 layout), with
explicit "Phase C/D freeze this by value + default-fill the new field"
contracts baked into the doc comments so the freeze is a clean edit later.
Unit-tested by versioned_decode_is_identity_across_majors.
A first-pass review caught that the initial gate widening over-reached: it
admitted major 1 for every chunk kind, but the bundle's own op-block decoder
(block::decode_block) and manifest decoder are unversioned, so a spec-valid
major-1 op block would have passed the gate and then been mis-read rather than
migrated / opened read-only. The accept-set ran ahead of the decoders.
Corrected: the gates stay EXACT to major 0 in this phase -- the manifest gate
to Manifest::SCHEMA.major (the manifest never grows a v1 layout in this bump),
the generic-chunk gate to SUPPORTED_SCHEMA_MAJOR = 0. Admission of major 1 is
raised PER CHUNK ROLE by the phase that adds that role's versioned decode or
discard path (snapshot -> C, op block -> D, layout cache -> E), never as a
blanket accept-set ahead of a decoder that can read it. The roundtrip
seam-exercise was reverted too (it conflated the canonical-base MaterializedState
role with the acceleration-snapshot Score role); the acceleration-snapshot read
path + the first usable_* wrapper land in Phase C.
So Phase B is version infrastructure + the dispatch seam only; the gate
widening, usable_* wrappers, and ops symmetry move to the phases that exercise
them.
863 workspace tests pass; clippy -D warnings, fmt --check, rustdoc -D warnings
all clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NEs4aYiu8MXjdYdMxw8PTd
The audit's fourth push: the biggest outstanding Phase-2 item plus the
performance gate. 793 workspace tests pass; clippy -D warnings, fmt,
and rustdoc (deny-warnings) clean; all three spec documents build with
zero undefined references.
Binary Format companion (spec/binary_format.tex, v0.1.0 — Agent J's
deliverable, 43 pages):
- Twelve chapters transcribed from the golden-locked implementation:
encoding conventions (the three prefix/endianness regimes, a
normative no-varint rule, reject-never-normalize decode discipline),
identifiers imported from the core spec's Canonical Byte-Layout
Reference, primitive value encodings, the whole-Score positional
codec ratified as the schema-major-0 wire form, operation wire
forms (envelope field order with the normative id-leads property,
the OperationPayload 0..=3 and OperationKind 0..=23 tables,
effects/conflict/anomaly/MaterializedState vocabulary), the bundle
physical layout (64-byte header, 256-byte superblock, chunk
preimages and framing, ChunkRef, manifest body order), the
operation-index payload, and the extension-blob/edit-barrier byte
forms.
- Ratifies P12-D1 (req:binfmt:opindex), P12-E1 (req:binfmt:ext-blobs),
P12-E2 (req:binfmt:condition-depth, MAX_CONDITION_DEPTH = 64
normative), and P12-E3 (req:binfmt:object-kind-open) — batch rows
struck through; discharges the provisional-codec notes in core
(P11-4), ops, and bundle (P11-D2/D4/D5) DECISIONS with ratification
cross-references.
- Pins the frozen-layout schema-evolution keystone: within schema
major 0 every positional struct layout is frozen; a field-set change
is a schema-major change with migration — formally grounding the
data-model-expansion staging decision. Open questions kept honest
in-document: SnapshotId derivation, index-refresh threshold, u64/u32
prefix unification at the next major.
- Not yet delivered from J's charter: the cross-implementation decoder
test and the wire-format fuzzer (follow-up harnesses).
F1 benches (crates/epiphany-testkit/benches/, per the F0 decision):
- criterion 0.5.1 (workspace dev-dependency; MSRV 1.77 respected with
documented transitive pins: clap 4.5.53, half 2.4.1).
- reduction bench at 1K/10K/50K envelopes with the Chapter-10 budget
(>10,000 envelopes/second cold) written in the bench as a Pass/Xfail
gate; bundle benches for the typical-edit commit (<=50 ms; measured
~14.7 ms on real disk after catching that tmpfs neuters fsync) and
the open/bootstrap read (<=200 ms; measured ~60 us).
- CI: quick budget gates in the conformance job, full gates nightly.
Subquadratic canonical_reduction_order (the F-surfaces/K-fixes
handshake, closing K's 10K-envelope acceptance gate):
- The bench documented the failure (50K at ~1.7K env/s, a 29 s cold
reduction; two O(n^2) loops); the fix replaces pair enumeration with
threshold/frontier readiness per replica plus explicit-dot dependent
lists and a stamp-tuple binary heap — O((n + sum(context)) log n),
never materializing covered pairs.
- Byte-identical order: same edge relation, same ready predicate, same
total order; the old implementation is retained as a test-only
oracle with element-for-element order-equality property tests over
fuzz sets, adversarial sets, and directed shapes (2,000-envelope
full-coverage chains, dot cycles, duplicate-id stamp ties),
mutation-tested for sensitivity.
- Measured: 1K 155K->674K env/s, 10K 12.5K->257K, 50K 1.7K->87K; all
three scale points now pass and the 50K row is promoted from Xfail.
Also: fixed nine rustdoc private/unresolved intra-doc links that had
accumulated across the pushes (the CI deny-doc-warnings job would have
failed on them).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NEs4aYiu8MXjdYdMxw8PTd
Two audit pushes whose code edits interleave line-by-line in the same
files (reduce.rs, bundle.rs, the DECISIONS logs), committed together so
the tree at every commit builds. Gate: 784 workspace tests pass, clippy
-D warnings clean, fmt clean.
Push 1 — the true MUST violations, all fixed:
- bundle: zstd read support on both read paths, output bounded by the
declared uncompressed_length, typed decompression errors, explicit
CompressedManifest rejection (zstd 0.13 workspace dep; write path
stays uncompressed per the Phase-3 deferral).
- ops: system-derived counter collision check — mint registry seeded
from the base graph, canonical-order pre-walk, halt via the new
PendingReason::HaltedBySystemCollision (discriminant 4, additive)
with transaction-atomicity and causal-dependent closure; neither
input set occupies a collided counter. canonical_pitch_bytes made
pub in core for the MUSCSPCH preimage.
- ops: Transpose skips tombstoned targets per the catalog; missing
targets still refuse the whole operation.
- ops: marker re-anchoring recorded as a RepairRecord in the
triggering operation's effect; ResolveConflict meta-conflicts name
both resolvers; base-free pitch-id freshness; reserved effect
vocabulary annotated.
- core: decomposition pre-pass honors authored attachments
(resolve_decomposition, spec-default precedence); inversion
tolerance typed as a TempoIntegration-class Tolerance.
- CONFORMANCE.md: the determinism conformance statement required by
Appendix D — all seven declarations.
Push 3 — wiring the types-only machinery:
- layout-ir/engrave: to_constrained emits real constraints (successive
notehead no-collision chains, per-glyph region containment, soft
user-break constraints); ConstraintStrength{Required, Preferred}
with strength-by-rule; Preferred violations surface as warnings, not
failures; StubSolver reworked honest-but-renderable. SVG goldens
byte-identical; snapshot constraint counts regenerated (0->90/15).
- layout-ir: to_logical projects user system/page breaks as anchored
EngravingOverrides with paired UserOverride-sourced decisions
(OverrideKind::SystemBreak/PageBreak carry TimeAnchor, ratified in
the spec alongside).
- layout-ir/ops/editor-core: edit-barrier bridge — decode mirrors for
the whole barrier tree (reject-never-normalize, NFC revalidation,
MAX_CONDITION_DEPTH = 64), golden-locked blob codec for the
ExtensionDeclaration fields, a barrier gate in apply and
apply_transaction backed by a Score oracle and real containment
contexts, and apply_unsafe recording the crossed extensions in
extensions_requiring_tombstone() for the next bundle write.
- ops: ResolveEquivocation meta-operation per the newly ratified
catalog entry — payload discriminant 3 (appended), set-level
earliest-resolve-governs promotion, ResolveConflict-mirrored
meta-conflicts, permutation-invariance fuzz; the missing golden
locks on the OperationKind/OperationPayload wire tables added.
- ops/editor-core: validation modes — ValidationMode + a non-canonical
advisory layer (validate.rs), an authoring gate before minting, and
reduction pinned as replay mode by construction (canonical bytes
untouched).
- bundle: the operation index (opindex.rs) — provisional golden-locked
payload, binary-search locate, staleness defined as full-ChunkRef
set equality against operation_roots, and the reject-and-rebuild
discipline (a defective index is never bundle corruption).
- ops: re-anchoring rule table completed — the four-key "nearest"
ordering computed from base-free ledger indices; markers re-anchor
to the nearest live event in the same staff instance (replacing the
Push-1 region-start stand-in); cue-source cascade; graphic-gesture
Events/Range/Free rows; comment and analytical-annotation orphaning.
Zero appended discriminants.
Spec enablers ratified with Push 3: catalog §ResolveEquivocation
(0.3.0 -> 0.4.0) and anchored break overrides; 16 new Pass-12 rows
filed (C1-C4, K5-K7, I4-I6, D1, E1-E5). The data-model payload
expansion (SlurKind, beam geometry, voltas, instrument bodies,
metadata) is deliberately staged to the Binary Format companion — the
positional graph codec has no value-level versioning, so filling those
structs is a schema-major break that should land once, with J.
Also carries the pre-existing editor-track increment: the atomic
tuplet overwrite (CascadeDeleteTuplets prunes decomposition
attachments naming the cascaded tuplet).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NEs4aYiu8MXjdYdMxw8PTd
Independent audit of b2f2e20 / a7adbdc. The canonical, document-state byte
layouts were already correct and golden-locked; this closes the one real
spec-vs-code gap (in the non-canonical layout namespace) and the smaller
doc/test gaps around it.
LayoutObjectId (item 2.6): the spec normatively stated the id "is derived ...
with MUSCLOID" and the ratification log / layout DECISIONS header called it
"pinned" -- but no code uses MUSCLOID (stable_layout_id and
manifestation_layout_id are untagged; synthesized_layout_id borrows MUSCCONF),
and MUSCLOID cannot even be constructed: DomainTag::from_bytes accepts only
built-ins or MUSCS-prefixed tags, so wiring it requires changing the frozen
determinism crate (out of scope for a spec pass, and the spec itself says
MUSCLOID is not a canonical system tag). Kept MUSCLOID as the pinned Track-A
target and made every artifact honest that the v0 code is provisional:
- spec: "is derived" -> "MUST be derived" (forward contract) + a note that
the prototype mints provisional ids; changelog "pinned" -> "specified ...
as the Track-A target".
- records: ratification-log line 2.6 and layout DECISIONS header/body now
state spec-pinned-but-code-provisional; stale "Pass 11 candidate 3"
pointer now cites the ratified requirement.
- provenance.rs comments name the MUSCLOID target and label the current
derivation provisional.
Other fixes:
- epiphany-core/DECISIONS.md: the Tuplet bullet still claimed degenerate
ratios are caught by runtime invariant 16 "since a Tuplet is a plain
struct" -- stale after the construction-time TupletRatio change. Rewritten.
- codec.rs: added degenerate_tuplet_ratio_is_rejected_on_decode, guarding
the TupletRatio::dec re-validation branch that no test exercised (the
constructor was tested, the decode path was not).
- spec: integrity-anomaly snippet kind.canonical_bytes() ->
to_canonical_bytes() (the actual method).
- ids.rs / reduce.rs: clarifying comments (ManifestId's intentional,
golden-locked document_id/generation duplication; compute_promotions
bucketing by voice == (staff_instance, original_voice) via Invariant 5).
Verification: cargo test --workspace (434 pass, +1), clippy --all-targets clean
(0 warnings), fmt clean; spec rebuilds (lualatex, 0 undefined refs, 254pp).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Audit follow-up to b2f2e20. The ratification was byte-faithful, but the
audit found a gap between the protection the worklist claims ("a
golden-bytes test already locks every byte-layout item") and the
protection actually in place for several newly-normative tables, plus
three small inaccuracies in the ratified spec text.
Golden locks (close the round-trip-vs-golden gap):
- ChunkKind: chunk_kind_discriminants_are_golden pins the literal 0..=8.
ChunkKind::canonical_bytes() is in the chunk hash preimage, so the
prior round-trip-only test would let a coordinated renumbering silently
change every chunk content address while passing.
- CompressionAlgorithm: compression_algorithm_encoding_is_golden pins the
exact bytes (None -> [0,0], not a bare tag).
- ProfileId (load-bearing superblock field): profile_id_discriminants_are_golden
pins the u32 discriminants and the fixed 20-byte encoding.
- ResolutionAction / TransactionCategory / ObjectKind: *_discriminants_are_golden
pin the canonical discriminants (ObjectKind feeds the anomaly id;
ResolutionAction/TransactionCategory feed operation content hashes).
- IntegrityAnomalyId: integrity_anomaly_id_byte_form_is_locked golden-locks
the MUSCSANM-derived id (cross-replica agreement is a conformance
property; it previously had no byte-form golden).
Spec-text fixes (core_spec.tex):
- CompressionAlgorithm: "None = 0 (no payload)" was wrong; the code writes
a fixed two bytes (discriminant + always-present parameter byte). Text
now states the fixed-width framing.
- ProfileId: "a single discriminant followed by any variant payload" was
wrong; it is a u32-LE discriminant + a fixed 16-byte registry id (zero
unless Custom), 20 bytes total. Text now matches the only encoding.
- TupletRatio listing showed `pub` fields (freely constructible by struct
literal), contradicting req:time:tuplet-ratio-construction. Listing now
shows private fields + the checked `new`/`actual()`/`notated()`, matching
the code.
Test honesty:
- testkit resolution_action generator now emits Dismiss (rng.below(6)); it
previously skipped the variant, leaving the Dismiss path unfuzzed.
Verification: cargo test --workspace (433 pass, +7), clippy -D warnings
clean, fmt clean; spec rebuilds (lualatex/latexmk, 0 undefined refs, 254pp).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Spec-revision pass (architecture unchanged) converting the v0
implementation's provisional, golden-locked choices into ratified
core_spec.tex text, so durable byte layouts are fixed before the
next-phase build-outs. Worklist: spec/PASS11_WORKLIST.md; per-item
dispositions: spec/PASS11_RATIFICATION_LOG.md.
Adopt-and-pin (bytes): TypedObjectId 16-bit BE discriminant table
0..=27 (added the 5 variants the code carried); promoted-voice
(MUSCSVCE), synthetic-pitch (MUSCSPCH, tuning always in identity),
and integrity-anomaly (MUSCSANM, now a reserved built-in tag)
derivations; ChunkKind/ProfileId/CompressionAlgorithm discriminants;
ManifestId preimage (manifest_id excluded); RationalTime/scalar
layouts + the codec convention baseline the Binary Format companion
inherits.
Decide-and-pin: tempo Linear interpolates speed (not bpm);
StructuralFieldCollision tags the winner Conflicted; lifted the
>2-way / partial-overlap voice-promotion rule to normative; pinned
TransactionCategory and ObjectKind core vocabularies; added
ResolutionAction::Dismiss so the Dismissed state is reachable by an
authored op; pinned the (non-canonical) LayoutObjectId derivation
(MUSCLOID).
Fixes: blob hashing is bare MUSCBLOB||payload (deleted the
contradictory "identically to chunks" phrasing); equal-generation
superblock rule (DivergentSameGeneration); defined ProfileConstraints
with the required RetentionPolicy + first-declared precedence; made
the DVV zero-based floor normative; reconciled the invariant count to
19 and named the three construction-time MUSTs — TupletRatio now
rejects degenerate ratios at construction (zero term or
actual==notated), enforced by a checked constructor + codec decode
validation.
Code changes carry regression tests; byte-layout golden tests now
cite their ratified requirements. Workspace green: cargo test, clippy
-D warnings, fmt; spec builds (lualatex, 253pp). Per-crate DECISIONS
files annotated with the ratification status.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011giSRaHCFCGm1Z2SWv6JHt
Chapter 8's OperationEnvelopeBlock carries dvv_summary/min_stamp/max_stamp so a
reader can select or skip a block by causal frontier / stamp range without
decoding it. These are semantic (ops-computed); the bundle carries them opaquely.
Bundle (Agent D):
- OperationBlockSummary { dvv_summary: FrontierBytes, min_stamp, max_stamp } and
Manifest.operation_block_summaries: BTreeMap<ChunkId, OperationBlockSummary>,
keyed by the block's chunk id, encoded/decoded in canonical (ChunkId-ascending)
order and accessible via Manifest::operation_block_summary. Optional and
non-canonical; preserved across reopen by the manifest round-trip.
- Round-trip + selectability test.
Testkit (Agent F, the C/D integration point):
- roundtrip::operation_block_summary computes the summary from envelopes using
ops (causal frontier + min/max OperationStamp canonical bytes).
- assert_operation_block_summary_survives_storage commits a real operation block
+ its summary, reopens, and selects the summary by block id without decoding
the payload. Wired into acceptance + the conformance suite.
bundle DECISIONS updated (summary metadata now carried, not omitted); fixed a
stale "pending item 5" doc on criterion 4 (the whole-score codec has landed).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
E-D (layout-ir, honest solver tier): add SolverTier::Stub (a non-conformance rung
below Minimal) and have StubSolver report it instead of falsely claiming the
Minimal conformance tier; the passthrough evaluates no constraints and computes
no quality metrics.
E-C (layout-ir, constraint/reference validation): ConstrainedLayoutIR::validate()
now also checks the LayoutConstraint vector — NoCollision/Align/PositionWithin
must name glyphs in the set, SystemBreakAt/PageBreakAt must name existing slots,
PositionWithin regions must be finite/non-negative — rejecting dangling
references instead of silently accepting them.
E-B (layout-ir, content-sensitive ScoreVersion): derive ScoreVersion from the
whole score's canonical bytes (Agent B's whole-score codec) rather than the
layout projection's object identities, so a pure content edit that changes no
identifier still changes the version — required for correct incremental-layout
cache invalidation.
D-A (bundle, extension-root preservation): Bundle::commit now enforces
preservation — after the builder closure runs, every prior extension declaration
it did not re-declare (by extension_id) is carried forward verbatim, so an
extension-unaware writer cannot silently orphan an unknown extension's
preserved_chunk_roots. An extension-aware writer that re-declares its id keeps
control.
Each fix has a regression test; per-crate DECISIONS updated. (Item-6 remainder:
D-B operation-block summaries next; E-A real time-axis deferred per request.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Levi Neuwirth