FORMAT_MAJOR becomes 1 and FORMAT_MINOR restarts at 0. The decoder
stops being exact-major-only: it classifies three ways through a named
FormatEpoch carried on FixedHeader, so major 0 is decoded deliberately
as legacy rather than refused. Old readers already fail closed on an
unknown major, so that half needed no mechanism — which is why the major
is the right carrier, and why the header's immutability, fatal to
FORMAT_MINOR as a provenance field, is what makes it sound as an epoch
field.
The matrix: a major-0 bundle with no base may open; one carrying a base
is refused; one attempting to add a base is refused and told to repack.
That last row is the non-inheritance rule. Three errors, none of which
degrades to read-only: two permanent legacy/repack errors, and
ReductionAuthorityUnavailable, which is temporary, names P13-S27, and
must not say repack — a major-1 container is already the right epoch.
Until P13-S27 lands, both major-1 base boundaries are closed: opening a
major-1 bundle that already carries a base, and committing one into it.
Neither may be left open while the epoch asserts a validation that never
ran.
Text projection cannot mint a base. serialize_document staged a carried
base into a fresh bundle and build_manifest wrote it, so an old or
hand-authored document could be laundered straight through the boundary.
All three sides now refuse: projection, parsing, and a new dedicated
SerializeError variant — none of which existed to be "retained".
COMPANION_VERSION moves to 0.14.0 and the corpus is rebuilt to 20
vectors and ten rejection classes, with canonical_bases reach dropping
2 -> 0. That is a real capability loss and is recorded as one.
Corruption keeps precedence in both epochs: a corrupt major-1 base fails
as malformed, never as the temporary authority error a user would
reasonably retry.
All 11 mutations were run and observed, not reasoned about. M4 is the
signing one — with the legacy commit refusal removed, a legacy container
gains a base in place, which is exactly the counterexample that killed
FORMAT_MINOR. M11 confirms the third error is distinct while test 3
stays green, proving the mutation stayed inside the major-1 branches. M7
fails on both epoch halves. M8 falls through to
SerializeError::Bundle(ReductionAuthorityUnavailable), confirming the
text layer's own refusal is what the test asserts.
Two touch-table gaps surfaced during execution, both the same shape: a
.tex requirement addition moves hardcoded counts in
requirement_labels.rs, and the companion bump moves a second normative
version literal spelled version~0.13.0 rather than (0 13 0). Neither
file was in any touch table; the second was caught only because a test
exists for exactly that failure.
P13-S27 is unblocked — its pin 2a is resolved from outside, as its own
prohibition required — and inherits three obligations: both interim
refusals converted to validation, M8's deferred laundering
demonstration, and pin 3c's two suspended conformance assertions.
P13-S16 remains blocked on S27.
Workspace green at 1569.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QjsEnYhm1gPpf6ii2iFxFV
Kind and tag 39 at schema-minor epoch 12, in both unaligned discriminant
spaces. Measure is schema major 0 -- struct_codec! is a plain walk and
TimeAnchor's Codec has no version branching -- so schema_major() gains no arm
and OperationEnvelopeBlock stays at 3.
CreateMeasure is a nested-container mint following CreateStaffInstance, not
G3a's root-level shape: it carries the owning StaffInstanceId beside the
value, and measure_values carries that parent because Measure has no
back-pointer and the graph-removal arm will need it. Append-only, with
referential preconditions on the parent, the signature, and every non-wall-clock
start referent.
The comparable relation is five exact shapes with an identical boundary
selector; ordering across Start/End is unsound while measure length is
unresolved, so it stays unverifiable. Boundary distance needs a musical delta,
which only same-referent same-selector Musical offsets supply. The effective-grid
oracle reconstructs inheritance from metric_grid_chain and meter_change_chain by
write recency -- not by always overlaying per-key on whole-grid -- folds in
prospective overrides, and runs identically in both reduction modes, so an
instance_grid ledger keeps base-free reduction honest.
Three precondition reasons at 16-18. Repairs a live bug found in review: the
materialized-effect decoder stopped at 13, so reasons 14 and 15 already encoded
without decoding, and the generator drawing below(14) could not see them.
Executed against spec/CONTRACT_GENESIS_G3B_MEASURE.md, mutations M1-M33,
M64-M66, M67-M70. text_projection.tex moved into this packet: four tests read
it live, so the companion bump cannot be split from the grammar.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QjsEnYhm1gPpf6ii2iFxFV
CreateStaffGroup (35), CreatePartDefinition (36), CreateAnalysisLayer (37),
CreateView (38) -- all schema major 0, all schema-minor epoch 11, on
CreateStaff's set-union mint pattern with byte-identical re-carry
idempotence. Completes the four remaining root-level Score entity vectors.
No epiphany-bundle change of any kind: schema_major() gained NO arm (all
four fall through the catch-all _ => 0), so the op-block accept-set stays at
3 where G2b left it. The OperationKind golden lock stays at [(OperationKind,
u8); 30] -- kinds 35-38 remain outside it by design, per P13-S15's own rung.
Closes the satisfiability half of the from-empty defect: every G3 object
kind previously became Live only through base ingest, so CreateStaff's group
precondition was unsatisfiable from empty. It does NOT make a consistent
group authorable -- §1.1 disposition B rules Staff.group the sole authority
and StaffGroup.members a non-authoritative projection G3a stores without
maintaining, with both stale forms permitted. That ruling now reaches all
four normative sites: both field doc comments (neither field was documented
before), core_spec.tex, and both the new CreateStaffGroup and the existing
CreateStaff sections of operation_catalog.tex.
Carried-value maps for all four families thread through seven sites each,
including the base seed that fails silently -- a re-carry test reducing only
from empty cannot see a missing seed, which is why t5b reduces onto a
populated base.
Verified independently, not relayed. Full gate re-run: 1429 tests / 0
failed, clippy 0, fmt clean, git diff --check clean. Two mutations the agent
had not observed were run here and observed: reverting COMPANION_VERSION to
0.11.0 kills t11 (the agent had only reasoned this), and skipping the
staff_group_values base seed kills t5b. Both reversed by editing back, with
the baseline green.
Two findings recorded rather than silently absorbed:
- valuegen::analysis_layer's name format was accidentally exactly 16 bytes,
the same width as AnalysisLayerId's encoding, which made a field-swap
mutation on the two-field AnalysisLayer struct byte-invisible. Shortened
and documented -- a fixture that makes a mutation undetectable is the same
failure class as a guard that cannot fail.
- binary_format.tex has NO Revision History row for G2b. G2b's own contract
required one (touch row 27: "version, Revision History row"); it changed
the payload-layout row, tag row, accept-set text and §2373 history but
never bumped the version or added the row. The document went 0.12.0 (G2a)
-> 0.13.0 (G-minor) -> 0.14.0 (G3a) with the accept-set raise unrecorded.
G3a adds its own row without retroactively backfilling G2b's.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QjsEnYhm1gPpf6ii2iFxFV
The corpus fixture stamped the manifest at {0,10} because SetTuningContext
needs an operation block at {3,10}. Those are separate domains: the document
line carries the manifest's aggregate SchemaVersion and projection discards the
block's schema by design. With no edit barriers the manifest stays baseline V0,
so the fixture was locking an over-stamped manifest into the corpus while
appearing to prove the operation epoch - the exact inference text_projection.tex
tells readers not to make. The block's stamp is proven where it lives, by the
staged-and-reopened roundtrip test.
t10 still read only bundle.rs. Correcting ids.rs and adding a prose
cross-reference did not make the pair travel together; sharing one guard does.
It now iterates both sources. Two incidental discoveries while extending it:
include_str! pulls in the test's own text, so both the needle and the assertion
message must avoid the phrase they search for - which is why the original split
its needles with concat().
Four stale comments: manifest.rs's barrier-tag range 24-33, payload.rs's "ten
events"/"thirty-four payloads", and the 30..=33 ranges in generators.rs and
layout_stub.rs.
Mutation sweep, each run and observed:
- t1 kind space 34->35, and tag space 34->35, separately. Both kill t1.
- t2/t3 schema_major 3->0: kills t3 and the staged/reopened test ({0,10} vs
{3,10}).
- t6 predecessor restore dropped: kills t6 (442 vs 441).
- t8 kind epoch 10->9 and tag epoch 10->9, separately. Both kill t8; the tag
mutation additionally kills s1's tag table, the kind mutation does not,
which is why both tables needed the entry.
- t10 stale claim reinjected into bundle.rs and into ids.rs, separately. Each
kills the guard, naming the offending file.
t5 and t7 were signed earlier; t4 by the cap-to-2 run.
Gate: 1410 tests, clippy 0, fmt clean, git diff --check clean, 14 textproj
vectors, 105 decode vectors.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QjsEnYhm1gPpf6ii2iFxFV
Closes four coverage gaps found in review. No behavioral change to the reducer,
codec, or wire.
ids.rs carried the same falsehood bundle.rs did - major 3 is snapshot-only, no
operation payload embeds the tuning context, the op-block accept-set is
untouched. t10 grepped only bundle.rs, so it could not see it. Rewritten, with
a pointer to its sibling so the pair stays together.
The text-projection corpus covered no set-tuning-context document: every
accepted document built DeleteRegion envelopes, and G2b only regenerated
headers to 0.11.0. Adds a real one, carried at minor 10. The typed all-kind
round-trip proves the production parses; only a committed vector proves the
emitted text is stable across implementations, which is what that corpus is
for.
Fixing it exposed a latent fragility worth more than the fix: the negative
vectors bound their source documents by positional index into the accept list,
so inserting a document silently repointed every reject builder at the wrong
text. Converted to by-name binding, which is what made the insertion safe.
The generator guard stopped at kind 33 - the very test written because
TransposeInterval and CreateInstrument had gone missing from every corpus while
downstream suites stayed green. It now records and asserts SetTuningContext, so
removing or rerouting the new arm cannot leave it green.
t3 and t4 did not test block behavior. t3 computed max(schema_major()) over
bare OperationKinds without building a block; t4 asserted only the ceiling
constant. Neither would notice a mis-derived stamp or a gate that rejected the
block it now admits. Adds one integration test that stages a real envelope
through stage_operation_block and reopens the bundle, asserting {3, 10} and
read-write. Verified by mutation: reverting the accept-set to 2 fails it.
Also three stale counts, each the hand-maintained-parallel-list shape: the
textproj production count said 31 and had been stale since the original
text-projection commit (now 35), the G-minor epoch table omitted kind 34, and
the corpus count was pinned at 13.
Gate: 1410 tests, clippy 0, fmt clean, conformance 8/8, 14 textproj vectors,
105 decode vectors, git diff --check clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QjsEnYhm1gPpf6ii2iFxFV
Kind and tag 34, schema major 3, minor epoch 10. The last rung before G3, and
the one that closes P13-S13: the tuning context becomes operation-authored, so
it finally has a canonical carrier. The closure argument is the metadata
precedent, not the canonical base - the base embeds no graph values for any
field, and metadata has been durable purely through its operations since M2d.
The payload carries epiphany_core::TuningContextSettings, a five-field subset
of ScoreTuningContext, not the full graph type. ScoreTuningContext's codec
deliberately drops accidental_extensions, so a full-value payload would have
diverged between a live session, where accept stores the envelope as a value,
and the same document reloaded, where the field decodes empty. canonical_value!
could not have caught that: it compares bytes and never the originating value,
so a field that never reached the bytes is structurally invisible to it. The
subset makes the divergence unrepresentable instead of relying on a
normalization step nothing can enforce, and it costs no wire design - the
encoding is byte-identical to the existing five-field walk, which
tuning_context_settings_canonical_bytes_match_score_tuning_context asserts
directly. Reduction leaves accidental_extensions untouched.
SetTuningContext is the sole genesis payload born at major 3, because minimal
stamping is a function of each payload's value, so the accept-set raise is
charged to this one surface: OperationEnvelopeBlock 2 to 3. The doc comment
above it did not merely record the cap, it asserted that no operation payload
embeds the tuning context - a sentence this rung falsifies - so it is rewritten
rather than left beside a corrected constant.
Undo restores the seeded base settings, default or not, and the
never-authored versus authored-to-default distinction stays unobservable. An
earlier draft of the contract had that backwards; PLAN_GENESIS_OPS section 5
trap 5 withdrew it, and SetMetadata is the disproof.
Fixes two undefined references the interrupted run had not yet reached:
operation_catalog.tex referenced sec:evolution:major3, a label defined in
binary_format.tex, which LaTeX cannot resolve across documents. Replaced with
the sectionsc convention already used for every other cross-companion citation
in that file.
Gate: 1409 tests, clippy 0, fmt clean, conformance 8/8 including [7f], both
vector corpora regenerated, all four PDFs at 0 undefined references. The t5 and
t7 mutations were re-run independently and observed to fail as specified; the
remaining eight are not signed off, because the implementing run was stopped
before it reported them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QjsEnYhm1gPpf6ii2iFxFV
Implements the schema-minor MUST (binary_format.tex, Schema Versioning) that no
writer had ever honoured: a writer raises the chunk minor when it emits a
discriminant appended after the minor it otherwise declares, so an
unknown-discriminant decode failure is attributable to version skew rather than
corruption.
introduced_minor() lands on the five vocabularies with post-baseline variants -
OperationKind, OperationKindTag, OperationPayload, ReanchorReason, and
PreconditionFailureReason - each exhaustive with no wildcard arm, so a future
variant cannot compile without being assigned an epoch. The tag epochs live
inside operation_kind_tag_vocabulary! rather than beside it, because a sibling
match is the parallel list that macro exists to prevent. The sentinel is
Option<u16>, not 0, since 0 is a real baseline minor for V1-V3 and conflating
them would make the max read correctly only by accident.
An envelope's required minor is the max over every discriminant it actually
emits; a block's is the max over its envelopes; major and minor derive
independently. Baselines are not normalised - V0 keeps minor 1.
The manifest seam keeps epiphany-bundle opaque: no ops or layout-ir dependency,
and the aggregate version is supplied by the producer rather than derived, with
CommitContext carrying the previous one so unchanged barrier content preserves
it. The version rides the superblock slot that already exists; Manifest gains no
field, which would have been schema-major and would have defeated the rung.
bundle.rs's superblock check stays major-only - tightening it to full-version
equality is a conformance regression, and s11 locks that.
textproj carries the manifest SchemaVersion and never derives it, so
COMPANION_VERSION moves 0.9.0 to 0.10.0 with the corpus regenerated. Not because
of op-block stamping, which remains projection-invisible. A new normative
requirement records the carry-never-derive rule in the companion itself; its
rationale names layout-ir, which textproj genuinely lacks, rather than the
operation vocabulary, which it has.
Gate [7f] adds an independent oracle over decodable in-tree barrier fixtures,
requiring exact equality rather than >=. Equality is load-bearing: >= catches
under-stamping but not stale over-stamping after the sole maximum contributor is
removed. Undecodable blobs are reported not-checkable, never as a pass.
Also repairs binary_format.tex's stale claim that OperationKind and
OperationKindTag append at 30 with a history stopping at 29, while 30-33 are
taken and the normative tables already carry them.
Gate: 1399 tests, clippy 0, fmt clean, conformance 8/8 with [7f] at four
fixtures checked and one not-checkable, 102 decode vectors byte-identical
(verified, not assumed), 13 text-projection vectors.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QjsEnYhm1gPpf6ii2iFxFV
SetCanvasLayoutDefaults and SetSpellingPrecedence, kinds and tags 32/33, on the
SetMetadata LWW pattern: advisory last-writer-wins, no conflict, no idempotence
short-circuit, seeded from the base so a value-restoring undo of the first
operational write reaches the pre-operational value. Both carried types already
had Codec impls, so the packet designs no wire layout -- canonical_value!
delegation and one push_lp_bytes each, inheriting strict canonical-form
rejection rather than writing it.
Both stamp schema major 0 unconditionally, staying in the catch-all arm. The
op-block accept-set is untouched at 2; epiphany-bundle has an empty diff. The
raise to 3 belongs to G2b, and the chunk schema minor -- a MUST no writer has
ever honoured -- belongs to G-minor, filed as P13-S14 and deliberately not
fixed here.
Most of this commit is documentation the vocabulary had outgrown. G1 shipped
five normative falsehoods because its contract declared the wire surfaces out
of scope, and Push 4a left TransposeInterval out of both of core_spec's
normative listings. So binary_format's payload-layout and tag tables now reach
33 rather than 30, its snapshot-only bullet records that it is empty and why,
two spelled-out payload counts move from eight to nine, and core_spec's
listings gain all four missing kinds. The Pass-12 K8 doctrine is narrowed
rather than reversed: the root and canvas remain structural givens no operation
mints, and only the claim that the score's contents arrive outside the
operation set is superseded.
Two generators were stale by two tranches and fed corpora other suites treat as
exhaustive. layout_stub's is now derived from PAYLOAD_FREE with an explicit
Registered append instead of a hand-written match, so future built-ins follow
structurally; s10 asserts PAYLOAD_FREE union {Registered} so deleting that
append cannot pass.
Companion 0.8.0 -> 0.9.0, binary format 0.11.0 -> 0.12.0, catalog 0.9.0 ->
0.10.0 (covering G1 retroactively). 102 decode vectors, regenerated.
Coordinator verification: 1371 passed / 0 failed, clippy 0 warnings, fmt clean,
conformance 8/8 and 9/9, requirement_labels 6/6, four PDFs at 0 undefined
references, goldens byte-identical, accept-set confirmed still 2. Mutations
s3/s5/s7/s10 re-run independently and each observed to kill its test.
Two findings beyond the contract. max_supported_major is not reachable outside
epiphany-bundle -- mod bundle is private and the symbol is absent from its
pub use -- so the gate's "assert it in code" was unsatisfiable without touching
a crate the contract forbids; verified by reading the unedited source instead.
And core_spec carries two P12-K8 references, not one; the contract named only
the first, so the second is annotated here for consistency.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QjsEnYhm1gPpf6ii2iFxFV
Score::empty plus operations alone now materializes a note-bearing Score. The
chain CreateInstrument -> CreateStaff -> CreateRegion -> CreateStaffInstance ->
CreateVoice -> InsertEvent needed exactly one new link: CreateStaff already
demanded a live Instrument and nothing could create one.
Instrument is a root with no outbound references, so the operation carries no
referential preconditions -- only mint and byte-identical re-carry, on the
CreateStaff template. It designs no wire layout: Instrument joins
canonical_value! and the payload is one push_lp_bytes over the existing Codec,
so strict canonical-form rejection is inherited rather than written. Kind 31 and
tag 31 agree; schema_major is unconditionally 2 (Instrument's major-2 appends
are mandatory, not Option-hidden); bundle.rs is untouched and the op-block
accept-set stays 2, since that raise belongs to G2.
Two cross-cutting items the ruling required. Reduction now writes identity for
the first time, deriving next_counter from the log rather than trusting the
seed -- and the implementation is broader than contracted, covering minted
entity ids as well as operation ids, which is right: both burn counters. And the
from-empty path is pinned to reduce_operation_set_onto, since the base-free mode
skips referential preconditions by design; a test documents that asymmetry as
designed rather than as a bug to fix.
The contract's parallel-safety claim was WRONG and this commit corrects it.
Extending OperationKind is not containable to core+ops: Rust exhaustiveness
forces an arm in editor-core's barriers.rs, and because testkit depends on
editor-core, that one missing arm blocked conformance and requirement_labels
too. Three more downstream sites had 31 or a kind-count baked in as a literal --
layout-ir's barrier decode test, testkit's grammar vocabulary count, and the
textproj corpus generator. The subagent found the first two, reverted its
out-of-bounds edit, and reported rather than working around; the user authorized
the boundary crossing. Each literal now carries a comment saying it must move
with every tag append.
The text projection needed a companion bump, which the contract never
anticipated. Adding create-instrument to the kind production while holding
0.7.0 would leave two incompatible grammars claiming one version -- precisely
what the single-version gate exists to prevent -- so COMPANION_VERSION is now
0.8.0, the first kind appended since the header was gated. Cached projections do
not migrate and are not expected to: a TextProjection chunk is a non-canonical
accelerator, so a stale one is regenerated. The negative "wrong version" vector
had to flip, since 0.8.0 was the version it used as its future-and-therefore-
rejected example; it now names 0.7.0, which tests the deferred migrate-on-read
posture better anyway. Test headers that were literals now assert against the
constant.
Gate, all observed: fmt clean; clippy --workspace --all-targets 0 warnings;
1359 passed / 0 failed; requirement_labels 6/6; conformance 8/8 and 9/9 with
golden-gate, 96 decode vectors and 13 textproj vectors, every verdict agreed.
max_supported_major(OperationEnvelopeBlock) verified still 2. Both PDFs rebuilt.
Mutations i1, i3 and i5 re-run independently rather than taken on report: the
spine collapses to TargetMissing without the instrument, an unseeded
instrument_values misreports a base re-carry as RecreateContentMismatch, and a
seed-returning cursor yields 0 where 12 is required.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QjsEnYhm1gPpf6ii2iFxFV
The cross-implementation corpus (req:binfmt:decode-vectors) covered ops and
bundle but nothing from epiphany-core — the oldest and most load-bearing wire
in the repo had no vectors at all. 65 -> 93 vectors, 5 -> 19 surfaces.
This delivers a revision the companion had already ordered. binary_format.tex
asked for the corpus to extend to the representative layouts "which remain
round-trip locked rather than literal-byte locked" — a description, written in
advance, of the exact defect found in tranche 3b-i: swapping smufl and
overrides in BOTH halves of ScoreTuningContext's codec silently moved a
permanently frozen layout and passed all 1283 tests and 8/8 conformance. An
encoder and decoder that agree on the wrong order round-trip perfectly. Only
literal bytes can see it.
That mutation is now the mandatory regression vector
(core.score_tuning_context/overrides_before_smufl), and re-applying the swap
was verified to fail three ways: the corpus reports "declared reject, but was
ACCEPTED", the committed corpus goes stale, and the conformance suite fails.
Both the full workspace suite and conformance now catch what previously passed.
Almost no new machinery was needed: CanonicalValue was already public and
already did decode + finish() + re-encode + reject-on-mismatch, and already
covered four of the five representative layouts. RationalTime joins it (the
fifth, and the exemplar of the spec's own warning that an outer guard can mask
a lenient inner codec — it reduces to lowest terms on decode, so only
hand-written bytes reach that path), along with the five schema-major-3 types.
check() therefore calls real production API, not a harness-side wrapper: a
leaf reject vector exercises the code an external implementer must match.
Codec and Reader stay pub(crate); the only new public item is `pub mod
vectors`.
Two things deliberately shaped: core is appended LAST in the testkit chain, so
the regenerated corpus diff is purely additive — 56 insertions, 0 deletions,
which PROVES no existing vector's bytes moved and therefore that no existing
wire form changed. And the per-major Score vectors assert canonical_bytes()
equality only at major 3: majors 0-2 migrate by design, rewriting the bytes,
and their canonicality at their own major is already established by the frozen
per-major encoder's re-encode guard. Comparing there would fail on every
vector and invite "fixing" it by gutting the vector's meaning.
Also rebuilds binary_format.pdf, which commit 68b08ad should have carried and
did not — every other binary_format.tex commit includes it. The rebuild picks
up both that tranche's Schema Major 3 section and this one. latexmk -xelatex
twice: 0 errors, 0 undefined references, 60 pages.
Gate: fmt clean, clippy 0, 1321 passed / 0 failed, doc 0, conformance 8/8 with
gate [7d] reporting 93 vectors, requirement labels 6/6 at 212/282/282 (no new
req: labels).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Thirteen drift-locked document vectors -- four accept, nine reject, one per
implemented rejection class -- gated as conformance step [7e] beside [7d]'s 65
decode vectors.
The checker distinguishes three outcomes, and that distinction is the whole
point. `Rejected`, `AcceptedCanonical`, and `AcceptedNonCanonical` are separate
verdicts, and every one of the four wrong pairings is a failure: an accept vector
that merely normalizes fails, and a reject vector that is accepted -- whether as
canonical or after normalization -- fails. The decode corpus learned this the
expensive way, where collapsing "rejected" with "accepted but normalized" let a
corpus pass against the exact decoders it was written to catch.
Accept vectors assert the byte-checkable equation `project(serialize(parse(T)))
== T`, which quantifies over TEXTS. The comments say plainly that it must not be
replaced with bundle identity, and name the three deliberate binary losses --
duplicate blobs collapsing under derived-ordering, regenerated physical layout,
and dropped accelerators -- so the next reader does not "fix" it into a bundle
comparison and then chase a phantom.
Reach is asserted exactly, not merely non-zero: two documents each carrying an
extension, a canonical base, a custom profile, a lineage, and more than one
envelope; and every reject class must have exactly one vector the parser really
refused. A mislabelled reject drops its class to zero rather than collecting
credit from its label.
The corpus was verified by breaking the implementation, not the checker: making
`parse_document` silently tolerate a missing trailing LF is caught three ways,
and the suite reports "declared reject, but was ACCEPTED and normalized".
Review fixed one defect in the semantic harness. It claimed to check
`semantics(parse(project(B))) == semantics(B)` but compared `parse(project(B))`
against `document_from_bundle(B)` -- and `project_bundle` *is*
`project_text_document(document_from_bundle(..))`, so both sides flowed through
one function and any bug in it cancelled out. Dropping an envelope there passed.
It now compares against the envelopes the test itself serialized in, which is
independent of the projection path; the same mutation fails.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two review findings after P4. Both real.
The tag-omission failure could recur. After the P4 fix there were STILL four
hand-maintained lists -- a test-local all_tags(), a barrier test spelling
0u8..=30, a fuzz corpus naming five tags, a vector corpus naming four -- plus a
malformed-bytes test asserting that 31 rejects. A future tag 31 added to
discriminant() and omitted from decode_canonical() would have left every one of
them green, and the malformed test would have LOCKED it, exactly as the 30
version did two commits ago.
operation_kind_tag_vocabulary! is now the single source. It generates
discriminant, from_discriminant, and OperationKindTag::PAYLOAD_FREE from one
list, and the generated discriminant match is exhaustive over the enum -- so a
variant added to the enum and not to the macro fails to COMPILE. Everything
downstream reads PAYLOAD_FREE: the decoder, the fuzz corpus (all 31 tags, not
five), the conformance vectors (an accept vector per tag: 65 vectors, not 37),
and the edit-barrier round-trip. Every "one past the vocabulary" constant is
computed, never spelled; a spelled constant is the trap that springs on whoever
appends the next tag.
Verified end to end with a hypothetical tag 31. Added to the enum alone: compile
error. Added to the enum and the macro: it compiles, decodes, and every derived
check passes because they read PAYLOAD_FREE -- while the committed corpus's
drift lock AND its now-stale "one past the vocabulary" reject vector both fail,
forcing the new vectors into the diff. There is no path where a new tag leaves
everything green.
Second finding: the corpus called itself normative while the spec said it was
deferred. Binary Format's "About This Companion" listed the cross-implementation
decoder test among things the document does not cover, and the Golden Anchor
Registry called it "the deferred conformance harness" whose literal-byte vectors
a future test "should add". Both now ratify it. New req:binfmt:decode-vectors
and a "The Decode Vector Corpus" section: a conforming decoder MUST accept every
accept vector for a surface it implements, MUST reject every reject vector, and
MUST re-encode an accepted value to exactly its bytes -- and accepting a reject
vector and then normalizing it IS accepting it. Binary Format 0.8.0 -> 0.9.0.
The wire-format fuzzer stays an implementation deliverable. The corpus header
now cites the requirement instead of asserting one.
Gate: fmt clean, clippy 0, 30 targets / 1024 passed / 0 failed, docs 0 under
-D warnings, conformance 8/8 with [7d] at 65 vectors, zero golden churn,
binary_format rebuilds with no undefined references.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
spec/vectors/decode_vectors.txt -- 37 committed byte strings across five
surfaces, each with its normative accept/reject verdict. The reference
implementation's fuzzers prove its own decoders self-consistent, which says
nothing about whether a foreign decoder agrees with the format. This is what one
is checked against. Gated in the conformance suite as [7d], and drift-locked:
the committed file must equal vectors::render(), so a wire-format change lands
in the diff.
It found a real defect on its first run. OperationKindTag::TransposeInterval
encoded to [30] and its own decoder REJECTED it -- Push 4a added the variant to
discriminant() and never to decode_canonical. OperationKindTag is what edit
barriers persist, so a barrier prohibiting TransposeInterval could be written and
never read back. Silent data loss on reopen.
Four things should have caught it. None did, and two made it worse:
The round-trip test enumerated DISCRIMINANTS -- (0u8..30).map(decode_canonical)
-- starting from bytes the decoder already knew, so it structurally could not
notice a variant the decoder was missing. It now enumerates VARIANTS from one
all_tags() list, with a completeness check in both directions.
The distinctness test's hand-written variant list omitted it too. Same list now.
operation_kind_tag_decode_rejects_malformed_bytes asserted that tag 30 is
REJECTED, and layout-ir's decode_rejects_unknown_discriminants asserted the
same at the barrier surface. Both were locking the bug in place and made it
look deliberate. Both now name 31, and a new barrier test round-trips a barrier
prohibiting every tag -- the persistence surface where this actually bites.
The P2 decode fuzzer fed valid corpus bytes to the tag decoder and tallied the
failure as a REJECTION, like any garbage input. It never asserted that an
unmutated corpus entry decodes. Both fuzzers now do, as a pre-pass.
The harness had the same disease as the code. `check` collapsed "rejected" with
"accepted but does not re-encode", so a decoder that silently normalizes
non-canonical bytes PASSED the reject vectors it was written to catch. Verified:
removing the whole-state guard, and restoring the lenient compression codec, both
left the corpus green. `check` now returns Ok(injective) for accept and Err for
reject and never conflates them -- silently normalizing non-canonical bytes IS
accepting them. With that fixed, all four defect mutations fail the corpus, each
naming its class.
The corpus pins one vector per class this repo has shipped a bug in:
non-canonical-map-order (a guard catches it; no per-site check exists),
non-canonical-vec-order (only a per-site check catches it; a guard is blind),
lenient-sub-codec (a guard masked it in the manifest; the index had none), plus
trailing-bytes, truncated, unknown-discriminant, count-exceeds-remaining. A test
fails if one goes missing.
Gate: fmt clean, clippy 0, 30 targets / 1024 passed / 0 failed, docs 0 under
-D warnings, conformance 8/8 (now including [7d]), zero golden churn.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Levi Neuwirth