Audit correction. The manifest test was documented -- in its own doc comment, in
DECISIONS.md, and in P3's commit message -- as "exhaustive single-byte
perturbation". It tried three XOR deltas per byte. The claim was false as
executed, and the "guard is total" conclusion leaned on it.
Both tests now do what their names say:
every_single_byte_replacement_of_a_manifest_is_rejected -- each byte, each of
the 255 other values. 0.15s.
compression_none_rejects_a_non_zero_parameter_byte -- every one of the 255
non-zero parameter bytes, plus a round-trip of all 256 values through Zstd and
Reserved, so the strictness is shown to be confined to None.
And the totality claim is re-seated where it belongs: on the argument, not on a
finite test. manifest_id is derived from the body, so a body edit fails the id
check and an id edit fails the derivation; encode_body sorts and deduplicates
every vector, so an out-of-order or duplicated encoding cannot round-trip. The
test is evidence for that argument over ONE constructed manifest, and is blind
to multi-byte perturbations entirely. Both the doc comment and DECISIONS.md now
say so.
Worth recording: restoring the leniency fails the codec test and the index test,
and leaves the manifest test GREEN -- the guard rejects those bytes whatever the
sub-codec does. That is not a weak test. It is the asymmetry that hid the bug,
and it locks the guard rather than the codec. A suite where every test fails on
every mutation would be telling us less.
No codec or wire-format change; the strict branch was already correct.
Gate: fmt clean, clippy 0, 30 targets / 1012 passed / 0 failed, docs 0 under
-D warnings, conformance 8/8, zero golden churn.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A wire-decode fuzzer over Bundle::open, Manifest::decode, OperationIndex::decode,
decode_block and envelope_offsets. The existing crash-recovery fuzzer corrupts an
image the way a CRASH does -- torn writes at syscall boundaries. This one
corrupts it the way an attacker or a bit-rotted disk does: arbitrary bytes,
anywhere.
It found a real defect. CompressionAlgorithm::None read its parameter byte and
DISCARDED it, while encode writes zero. So [0, 0xFF] and [0, 0] both decoded to
None, and the first re-encoded to the second: a lenient, non-injective codec,
inherited by every structure embedding a ChunkRef.
Whether that was visible depended entirely on the embedder:
Manifest::decode has a whole-value re-encode guard, and it is TOTAL -- proved
by exhaustive single-byte perturbation, every one rejected. It caught this.
OperationIndex::decode has no guard; it validates per-site. It accepted both
byte strings, while its own doc promised to "reject (never normalizing) any
non-canonical form". That promise was false.
That is the same two-layer lesson P2 recorded one commit ago, from the other
side: a re-encode guard is complete only where the encoder normalizes, and its
completeness can MASK a lenient sub-codec rather than fix it. Fixed at the
source, not papered over at the index. An exhaustive sweep -- every byte, every
value, plus an 8-byte extreme-integer window -- finds no remaining non-injective
site.
The fix contradicted ratified spec text, which said the byte was "present but
zero, and ignored on read". Escalated rather than fixed unilaterally. The user
ratified strict decode: core spec's clause is superseded, Binary Format gains
req:binfmt:compression-none-parameter and moves 0.7.0 -> 0.8.0. No wire layout
changed, and no conforming writer emits a non-zero byte, so this rejects only
corrupt or adversarial input -- no existing file changes meaning.
Coverage was the harness's problem again. The fuzzer's first run reached the
operation index's accept path ZERO times -- random bytes never decode as an
index -- so every assertion under it was vacuous. It found the bug only once the
index corpus was built from real OperationIndex::build output. The smoke tests
now assert on a WireFuzzCoverage so that cannot silently regress. 1.5M inputs
across five seeds, ~1s each, clean after the fix.
Three regressions, each mutation-verified by restoring the leniency: the codec
itself, the index that exposed it, and the manifest guard's totality -- which is
the asymmetry that hid it.
Gate: fmt clean, clippy 0, 30 targets / 1012 passed / 0 failed, docs 0 under
-D warnings, conformance 8/8, zero golden churn, both spec documents rebuild
with no undefined references.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two audit pushes whose code edits interleave line-by-line in the same
files (reduce.rs, bundle.rs, the DECISIONS logs), committed together so
the tree at every commit builds. Gate: 784 workspace tests pass, clippy
-D warnings clean, fmt clean.
Push 1 — the true MUST violations, all fixed:
- bundle: zstd read support on both read paths, output bounded by the
declared uncompressed_length, typed decompression errors, explicit
CompressedManifest rejection (zstd 0.13 workspace dep; write path
stays uncompressed per the Phase-3 deferral).
- ops: system-derived counter collision check — mint registry seeded
from the base graph, canonical-order pre-walk, halt via the new
PendingReason::HaltedBySystemCollision (discriminant 4, additive)
with transaction-atomicity and causal-dependent closure; neither
input set occupies a collided counter. canonical_pitch_bytes made
pub in core for the MUSCSPCH preimage.
- ops: Transpose skips tombstoned targets per the catalog; missing
targets still refuse the whole operation.
- ops: marker re-anchoring recorded as a RepairRecord in the
triggering operation's effect; ResolveConflict meta-conflicts name
both resolvers; base-free pitch-id freshness; reserved effect
vocabulary annotated.
- core: decomposition pre-pass honors authored attachments
(resolve_decomposition, spec-default precedence); inversion
tolerance typed as a TempoIntegration-class Tolerance.
- CONFORMANCE.md: the determinism conformance statement required by
Appendix D — all seven declarations.
Push 3 — wiring the types-only machinery:
- layout-ir/engrave: to_constrained emits real constraints (successive
notehead no-collision chains, per-glyph region containment, soft
user-break constraints); ConstraintStrength{Required, Preferred}
with strength-by-rule; Preferred violations surface as warnings, not
failures; StubSolver reworked honest-but-renderable. SVG goldens
byte-identical; snapshot constraint counts regenerated (0->90/15).
- layout-ir: to_logical projects user system/page breaks as anchored
EngravingOverrides with paired UserOverride-sourced decisions
(OverrideKind::SystemBreak/PageBreak carry TimeAnchor, ratified in
the spec alongside).
- layout-ir/ops/editor-core: edit-barrier bridge — decode mirrors for
the whole barrier tree (reject-never-normalize, NFC revalidation,
MAX_CONDITION_DEPTH = 64), golden-locked blob codec for the
ExtensionDeclaration fields, a barrier gate in apply and
apply_transaction backed by a Score oracle and real containment
contexts, and apply_unsafe recording the crossed extensions in
extensions_requiring_tombstone() for the next bundle write.
- ops: ResolveEquivocation meta-operation per the newly ratified
catalog entry — payload discriminant 3 (appended), set-level
earliest-resolve-governs promotion, ResolveConflict-mirrored
meta-conflicts, permutation-invariance fuzz; the missing golden
locks on the OperationKind/OperationPayload wire tables added.
- ops/editor-core: validation modes — ValidationMode + a non-canonical
advisory layer (validate.rs), an authoring gate before minting, and
reduction pinned as replay mode by construction (canonical bytes
untouched).
- bundle: the operation index (opindex.rs) — provisional golden-locked
payload, binary-search locate, staleness defined as full-ChunkRef
set equality against operation_roots, and the reject-and-rebuild
discipline (a defective index is never bundle corruption).
- ops: re-anchoring rule table completed — the four-key "nearest"
ordering computed from base-free ledger indices; markers re-anchor
to the nearest live event in the same staff instance (replacing the
Push-1 region-start stand-in); cue-source cascade; graphic-gesture
Events/Range/Free rows; comment and analytical-annotation orphaning.
Zero appended discriminants.
Spec enablers ratified with Push 3: catalog §ResolveEquivocation
(0.3.0 -> 0.4.0) and anchored break overrides; 16 new Pass-12 rows
filed (C1-C4, K5-K7, I4-I6, D1, E1-E5). The data-model payload
expansion (SlurKind, beam geometry, voltas, instrument bodies,
metadata) is deliberately staged to the Binary Format companion — the
positional graph codec has no value-level versioning, so filling those
structs is a schema-major break that should land once, with J.
Also carries the pre-existing editor-track increment: the atomic
tuplet overwrite (CascadeDeleteTuplets prunes decomposition
attachments naming the cascaded tuplet).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NEs4aYiu8MXjdYdMxw8PTd
Audit follow-up to b2f2e20. The ratification was byte-faithful, but the
audit found a gap between the protection the worklist claims ("a
golden-bytes test already locks every byte-layout item") and the
protection actually in place for several newly-normative tables, plus
three small inaccuracies in the ratified spec text.
Golden locks (close the round-trip-vs-golden gap):
- ChunkKind: chunk_kind_discriminants_are_golden pins the literal 0..=8.
ChunkKind::canonical_bytes() is in the chunk hash preimage, so the
prior round-trip-only test would let a coordinated renumbering silently
change every chunk content address while passing.
- CompressionAlgorithm: compression_algorithm_encoding_is_golden pins the
exact bytes (None -> [0,0], not a bare tag).
- ProfileId (load-bearing superblock field): profile_id_discriminants_are_golden
pins the u32 discriminants and the fixed 20-byte encoding.
- ResolutionAction / TransactionCategory / ObjectKind: *_discriminants_are_golden
pin the canonical discriminants (ObjectKind feeds the anomaly id;
ResolutionAction/TransactionCategory feed operation content hashes).
- IntegrityAnomalyId: integrity_anomaly_id_byte_form_is_locked golden-locks
the MUSCSANM-derived id (cross-replica agreement is a conformance
property; it previously had no byte-form golden).
Spec-text fixes (core_spec.tex):
- CompressionAlgorithm: "None = 0 (no payload)" was wrong; the code writes
a fixed two bytes (discriminant + always-present parameter byte). Text
now states the fixed-width framing.
- ProfileId: "a single discriminant followed by any variant payload" was
wrong; it is a u32-LE discriminant + a fixed 16-byte registry id (zero
unless Custom), 20 bytes total. Text now matches the only encoding.
- TupletRatio listing showed `pub` fields (freely constructible by struct
literal), contradicting req:time:tuplet-ratio-construction. Listing now
shows private fields + the checked `new`/`actual()`/`notated()`, matching
the code.
Test honesty:
- testkit resolution_action generator now emits Dismiss (rng.below(6)); it
previously skipped the variant, leaving the Dismiss path unfuzzed.
Verification: cargo test --workspace (433 pass, +7), clippy -D warnings
clean, fmt clean; spec rebuilds (lualatex/latexmk, 0 undefined refs, 254pp).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Spec-revision pass (architecture unchanged) converting the v0
implementation's provisional, golden-locked choices into ratified
core_spec.tex text, so durable byte layouts are fixed before the
next-phase build-outs. Worklist: spec/PASS11_WORKLIST.md; per-item
dispositions: spec/PASS11_RATIFICATION_LOG.md.
Adopt-and-pin (bytes): TypedObjectId 16-bit BE discriminant table
0..=27 (added the 5 variants the code carried); promoted-voice
(MUSCSVCE), synthetic-pitch (MUSCSPCH, tuning always in identity),
and integrity-anomaly (MUSCSANM, now a reserved built-in tag)
derivations; ChunkKind/ProfileId/CompressionAlgorithm discriminants;
ManifestId preimage (manifest_id excluded); RationalTime/scalar
layouts + the codec convention baseline the Binary Format companion
inherits.
Decide-and-pin: tempo Linear interpolates speed (not bpm);
StructuralFieldCollision tags the winner Conflicted; lifted the
>2-way / partial-overlap voice-promotion rule to normative; pinned
TransactionCategory and ObjectKind core vocabularies; added
ResolutionAction::Dismiss so the Dismissed state is reachable by an
authored op; pinned the (non-canonical) LayoutObjectId derivation
(MUSCLOID).
Fixes: blob hashing is bare MUSCBLOB||payload (deleted the
contradictory "identically to chunks" phrasing); equal-generation
superblock rule (DivergentSameGeneration); defined ProfileConstraints
with the required RetentionPolicy + first-declared precedence; made
the DVV zero-based floor normative; reconciled the invariant count to
19 and named the three construction-time MUSTs — TupletRatio now
rejects degenerate ratios at construction (zero term or
actual==notated), enforced by a checked constructor + codec decode
validation.
Code changes carry regression tests; byte-layout golden tests now
cite their ratified requirements. Workspace green: cargo test, clippy
-D warnings, fmt; spec builds (lualatex, 253pp). Per-crate DECISIONS
files annotated with the ratification status.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011giSRaHCFCGm1Z2SWv6JHt
Levi Neuwirth