A review found the 0.1.0 companion lossy for documents that are valid today. Its
claim to preserve the manifest's canonical roots was false three ways:
A canonical blob had no representation at all. blob_roots referenced by
canonical operations or reduced state ARE canonical roots, and the document
structure had no blob line. An embedded image, font or recording would vanish
from a projection silently -- the operations referencing it still there,
pointing at a blob id the text no longer contained.
An ExtensionDeclaration lost its semantic version and its affected_object_kinds
outright, and left its preserved_chunk_roots undefined.
ProfileId::Custom(ProfileRegistryId) was unrepresentable: a symbol was required
where sixteen registry bytes are carried.
All three share one cause I had not named. A ChunkRef and a BlobRef are PHYSICAL
references -- offset, compressed length, compression -- which the projection may
not preserve; and they carry DERIVABLE identities -- ChunkId, ContentHash, BlobId
-- which it may not duplicate. Having no rule for either, I dropped the
references and took their contents with them.
req:textproj:derive-or-carry states it, and it is the same rule
req:textproj:reduced-state-derived already applied one level up: carry exactly
what the document does not determine, and nothing it does. Physical attributes
never appear; derivable identities never appear; content and semantic attributes
always do. The one non-derivable identity in schema major 0 is SnapshotId, which
the Binary Format companion pins as opaque and forbids readers to derive -- an
exception for a stated reason rather than an oversight. The grammar now visibly
contains no offset, no length, no compression, no chunk id, no hash, no blob id.
So: req:textproj:canonical-blobs (canonical blobs projected, non-canonical ones
not), req:textproj:profile-id ((custom #x...)), req:textproj:extension-declaration
(every field; chunks as kind + schema + payload, never as a ChunkRef -- the
projection has no file to point into), and req:textproj:base-snapshot-inline
extended to say what the inlined payload IS and that the root ChunkRef and the
SnapshotRef hash are re-derived from hash(Snapshot, schema, payload), never read.
The gap started upstream. core_spec's own list of what the projection preserves
omitted canonical blobs while classifying blob_roots as canonical roots -- an
inconsistency inside one document. Corrected there, along with withdrawing the
permission to reference a base snapshot "externally", which the inline
ratification had already made untenable.
Also: the generated PDF metadata and page header still said Operation Catalog,
inherited from the copied preamble.
The four 0.1.0 ratifications stand unchanged. Implementation stays deferred: a
gate that is lossy is not a gate.
Gate: clippy 0, 31 targets / 1031 passed / 0 failed, conformance 8/8, zero golden
churn; core_spec, binary_format, operation_catalog and text_projection all build
with no undefined references.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
core_spec Chapter 8 declares the text projection normative and leaves the form
unwritten; binary_format excludes it as "the Text Projection companion's". That
companion did not exist. spec/text_projection.tex v0.1.0 supplies it. No
implementation -- this is the gate.
It was blocked on P5. The projection's normative requirement is bidirectionality
WITH THE BINARY FORM, which needs bytes -> OperationEnvelope, and that decoder
did not exist until 3baf8d0.
Four ratified calls:
Reduced state is preserved by DETERMINING it, never by a second literal copy. It
is a deterministic function of the operation set and the canonical base; a text
carrying both would hold two sources of truth for one fact and nothing could stop
them disagreeing. core_spec's "all canonical reduced state" now carries that
reading inline rather than being silently reinterpreted.
A canonical base snapshot is INLINED as one opaque byte string. This is the call
with teeth. A base exists precisely so prior operations need not be retained, and
where they are pruned the base is derivable from nothing else -- so a
reference-only projection of a compacted document would be LOSSY, and the text
would not determine the document it claims to project. core_spec permits "encoded
compactly or referenced externally"; inline is the choice that keeps archival
honest, and core_spec now says why.
Lowercase hex everywhere: one rule, no alphabet or padding to canonicalize,
greppable. Base64 would buy a quarter of the bytes of the one body nobody reads,
at the price of a second encoding and a rule for which applies where.
One envelope per line: the stated use case is that merge conflicts surface at the
envelope level, and one line per envelope makes a three-way merge conflict
exactly an envelope conflict -- never a conflict inside one, yielding an operation
neither side wrote. It also removes all indentation, so canonicality has nothing
to hide in. Readability is a pretty-printer's job; what a pretty-printer must not
do is write its output back and call it a projection.
Strict parsing is stated in the terms P2-P5 taught: normalizing non-canonical
text IS accepting it. The rationale names both hazards this repo hit in binary --
a re-encode guard is blind to order-preserving sequences, and a guard on an outer
value can mask a lenient inner codec -- and prescribes the same total defence.
Conformance requires both directions: a projector alone cannot be checked.
One gap is stated in the document rather than left to be discovered: the atom
productions and line shapes are normative, while kind/action/policy/constraints/
barrier are derived from the Operation Catalog and the wire table rather than
spelled out. That is the difference between a design gate and a finished
companion.
Gate: clippy 0, 31 targets / 1031 passed / 0 failed, conformance 8/8, zero golden
churn; core_spec, binary_format and the new companion all build with no undefined
references.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Levi Neuwirth