docs: frame tab-width rendering parity

Define the fixed cross-frontend tab-stop contract, GPU source projection and mapping rules, core display-width consolidation, minimap behavior, and acceptance gates.

docs/tab-width-parity-framing.md

@@ -0,0 +1,422 @@
+# Tab-width rendering parity - side quest
+
+**Status:** Revision 1 framing; awaiting user approval.
+
+**Base:** `githubsucks/main` at `40111dc` (landed-state documentation for
+locals-query processing #134); protocol v18.
+
+## Problem
+
+Pmacs has no single tab-rendering contract:
+
+- `src/text_view.rs`, `src/highlight.rs`, `src/diag.rs`, and
+  `src/completion.rs` independently hard-code an 8-column tab stop.
+- `src/overlay.rs` repeats the same 8-column arithmetic as literals.
+- `pmacs-gpu/src/main.rs::advance_minimap_col` uses 4 columns and counts every
+  non-tab character as one column.
+- The GPU code buffer sends raw `\t` bytes to cosmic-text. Its visible width is
+  therefore whatever the selected font's tab glyph happens to provide, not a
+  pmacs tab stop.
+
+The disagreement is observable. The same buffer can place text, syntax faces,
+diagnostic squiggles, completion popups, selections, carets, and minimap marks
+at different columns between the TUI and GPU frontends. Merely defining an
+`editor.tab-width` config key would not fix the GPU: tab expansion, source-byte
+mapping, styling, and hit testing all happen inside the frontend after the raw
+semantic frame arrives.
+
+This is the remaining top-ranked item in `docs/side-quest-backlog.md:123-130`
+and `:245-248`.
+
+## Goal
+
+Define one fixed 8-column tab-stop invariant, make every shipped buffer-text
+renderer honor it, and preserve byte-addressed editor semantics while the GPU
+shapes an expanded display projection.
+
+For a tab beginning at logical display column `c`, its width is
+
+```text
+8 - (c mod 8)
+```
+
+so a tab at an already aligned column advances a full eight columns. Source
+text remains byte-for-byte unchanged.
+
+## Scope
+
+### In
+
+- One canonical tab-stop constant shared by the core and GPU crates.
+- One core display-column utility used by plain text, syntax styling,
+  diagnostics, completion placement, and generic buffer-style overlays.
+- Tab expansion in the GPU code-buffer projection before cosmic-text shaping.
+- Correct projected-to-source and source-to-projected mapping for clicks,
+  carets, selections, diagnostic geometry, wrapping, and inline adornments.
+- GPU minimap width/indent accounting using the same tab and Unicode-width
+  rules as the code view.
+- Focused regression coverage at tab-stop boundaries, after wide Unicode, and
+  through styled and selected tab bytes.
+
+### Out
+
+- A user-configurable `editor.tab-width` setting. This change deliberately
+  chooses the already-shipped TUI behavior, 8, and makes it universal.
+- Per-buffer or per-language tab widths, indentation policy, soft-tab
+  insertion, tab-to-spaces conversion, or retabbing existing files.
+- Changing what the Tab key inserts. A literal tab remains one source byte.
+- Expanding tabs in protocol payloads or mutating `SemanticFrame` byte ranges.
+- Tabs in statusline, minibuffer, menu, or other non-buffer UI strings.
+- A wire schema or protocol-version change.
+
+## Ground truth and contracts to preserve
+
+### Core renderers are byte-addressed but paint in display columns
+
+`TextView` already expands a tab to spaces at the next multiple of 8 and maps
+the one source byte to that display interval. Syntax highlighting and
+diagnostics independently translate byte ranges into display columns.
+Completion computes its popup anchor from a byte offset. Generic
+`BufferStyleSpan` overlays compute both ends from the line start. All five
+paths require the same prefix-width operation; today they implement it
+separately.
+
+The current TUI inverse mapping rounds every display column inside an expanded
+tab forward to the source boundary after the tab. That behavior is observable
+and remains the cross-frontend rule.
+
+### GPU code text is already a source-preserving projection
+
+`projected_rich_chunks` interleaves source text, foreground style spans, and
+inline adornments. `line_from_chunks` is the only content fed to cosmic-text.
+`line_chunk_cache` drives source-byte-to-layout-cursor conversion, while
+`current_hit_runs` and `projected_line_starts` convert cosmic-text hit results
+back into source bytes. Incremental line reshaping and full slice rebuilds both
+consume the same chunk construction path.
+
+Tab expansion belongs at this projection boundary. Expanding the daemon's text
+would invalidate every protocol byte range; asking cosmic-text to interpret raw
+tabs would retain font-dependent behavior.
+
+### GPU geometry currently assumes source bytes equal shaped bytes
+
+Foreground colors are attached to chunks and therefore naturally survive a
+projection when the chunk provenance is retained. Background selections,
+current-line washes, and diagnostic squiggles are different:
+`push_glyph_extent_rects` currently compares source-relative decoration bytes
+directly with cosmic-text glyph byte offsets. That equality already needs
+special handling for adornments and becomes definitively false once one tab
+byte projects to multiple spaces. The geometry path must use the same
+source/projection mapping as caret placement and hit testing.
+
+### The protocol transports raw text and raw byte ranges
+
+`pmacs-protocol` owns the types shared by the daemon and GPU. `SemanticFrame`
+continues to carry unmodified text plus byte-addressed spans, decorations, and
+adornments. A tab-stop constant is a rendering semantic for those existing
+fields, not a serialized field. Adding it changes neither postcard encoding nor
+version negotiation.
+
+## Decisions
+
+### Q#TW1 - The canonical tab stop is fixed at eight columns
+
+Add a documented public constant named `TAB_STOP_COLUMNS: u32 = 8` to
+`pmacs-protocol` and re-export it through the crate root. Both the pmacs core
+and `pmacs-gpu` consume that constant.
+
+The shared protocol crate is the narrow existing dependency common to both
+frontends. A second rendering crate is unjustified, while two frontend-local
+constants would preserve the drift this work is meant to remove. The constant
+is normative metadata for interpreting raw text already carried by the
+semantic protocol; it is not serialized.
+
+Do not add a config-registry key. A future configurable width would need a
+buffer-effective value in every semantic frame (or another versioned frontend
+fact), cache invalidation when it changes, and tests across reconnects. That is
+a separate feature, not hidden scope in this parity fix.
+
+No `PROTOCOL_VERSION` bump: no message variant, field, encoding, capability, or
+negotiation rule changes. Protocol v18 gains a compiled rendering invariant for
+a previously unspecified raw-tab case.
+
+### Q#TW2 - One core module owns display-column arithmetic
+
+Add `src/display_width.rs` and export it from `src/lib.rs`. It owns:
+
+- `TAB_STOP_COLUMNS` consumption from `pmacs_protocol`;
+- advancing a logical column by one character, including tabs and
+  `unicode-width` handling;
+- the width of a valid UTF-8 string from a specified starting column;
+- the display column at a byte boundary in a line; and
+- the display-column pair for a half-open byte range.
+
+Byte helpers clamp to the input length and use the longest valid UTF-8 prefix
+when a stale/asynchronous range lands inside a code point. They do not allocate.
+Tabs are always evaluated from the line's logical column zero, not from the
+viewport edge or a range's start.
+
+Migrate `text_view`, `highlight`, `diag`, `completion`, and `overlay` to this
+module. Delete their constants and private copies rather than leaving aliases
+or wrapper functions. `TextView` may still special-case tab painting, but its
+pad count comes from the shared column advance.
+
+### Q#TW3 - GPU expands tabs in the rich-chunk projection
+
+After source/style/adornment boundaries have produced `RichChunk`s, run one
+projection pass before either full-slice or per-line shaping. The pass walks
+chunks and code points in display order while tracking a logical display
+column:
+
+- ordinary characters retain their text and provenance and advance by
+  `unicode-width`;
+- newline resets the logical column to zero;
+- a source tab becomes `TAB_STOP_COLUMNS - (column % TAB_STOP_COLUMNS)` ASCII
+  spaces carrying explicit provenance for that one source byte;
+- a tab inside an adornment also becomes spaces but retains the adornment's
+  anchor provenance; and
+- zero-width characters do not advance the logical column.
+
+A chunk with no tab is retained rather than copied again. Chunks containing
+one or more tabs are split only at those tab boundaries. The existing visible
+slice and per-line caches therefore bound both allocations and work; the
+frontend never expands the whole file merely to draw one viewport.
+
+`line_from_chunks`, `build_hit_runs`, incremental line replacement, and the
+full rebuild all consume the expanded chunks. No alternate shaping path may
+feed raw buffer tabs to cosmic-text.
+
+### Q#TW4 - A projected tab run has first-class source provenance
+
+Extend `ChunkSource` with a source-tab form containing the tab's
+slice-relative byte offset. The derived `ProjectedRun` then represents three
+semantics:
+
+1. source text is byte-linear;
+2. adornment text snaps to its anchor; and
+3. all projected spaces for a tab correspond to one source byte.
+
+Boundary rules are explicit:
+
+- source offset at the tab byte maps to the first projected space;
+- source offset immediately after the tab maps after the final projected
+  space;
+- a projected hit exactly at the tab's leading boundary maps before the tab;
+- any hit inside its expanded interval maps after the tab, matching
+  `TextView::display_to_pos`; and
+- a hit at the following projected boundary maps to the following source
+  boundary without crossing an adornment's established left-gravity rule.
+
+Factor the per-line source-to-projected conversion out of
+`State::code_byte_to_projected` so caret placement, decoration geometry, and
+unit tests use the same boundary implementation. Keep projected-to-source in
+the run map built from those exact chunks. Do not infer positions from counts
+of spaces after shaping.
+
+### Q#TW5 - Tab stops use the final visible logical column
+
+The projection pass counts all visible content before a tab, including wide
+Unicode and inline-adornment text. This makes the expanded tab end on a visible
+8-column boundary instead of overlapping or drifting when an inlay hint occurs
+before it.
+
+Cosmic-text remains responsible for glyph shaping and pixel geometry. The tab
+rule controls how many monospace spaces are supplied; it does not replace
+shaping with manual pixel placement. Code font fallback may vary in pixels,
+but logical columns remain deterministic.
+
+Add `unicode-width = "0.2"` as a direct `pmacs-gpu` dependency. Do not reach
+through another crate's transitive dependency.
+
+### Q#TW6 - Styles and decorations cover the full projected tab
+
+Foreground styling is preserved by assigning every expanded source-tab chunk
+the color of the source chunk that contained the tab. A style span covering
+`[tab, tab + 1)` therefore colors every projected space; a span ending at the
+tab colors none of them.
+
+For background selections and diagnostic squiggles, convert each source-range
+intersection on a shaped line to projected byte boundaries before comparing it
+with `LayoutGlyph::{start,end}`. The conversion uses that line's cached chunks
+and the Q#TW4 boundary rules. Do not rewrite protocol ranges, and do not use
+source line offsets as if they were projected byte offsets.
+
+This same conversion covers own selections, peer selections, current-line
+geometry where applicable, and diagnostic ranges. Gutter diagnostic signs are
+line-presence indicators and remain source-line based; they need no horizontal
+projection.
+
+### Q#TW7 - The minimap uses the same logical-width rule
+
+Replace the hard-coded 4-column `advance_minimap_col` branch with the shared
+`TAB_STOP_COLUMNS` value. Ordinary characters advance by `unicode-width`
+instead of unconditionally by one; zero-width characters advance by zero and
+wide characters by two.
+
+The minimap remains a density abstraction rather than shaped text, but its
+indent and content extents now agree with the code view's logical columns.
+Clipping and pixel compression are unchanged.
+
+### Q#TW8 - Projection invalidation follows existing text/chunk invalidation
+
+Tab width is fixed at compile time, so it introduces no runtime invalidation
+source. Text edits, style/adornment updates, font changes, resizes, scrolling,
+and buffer switches already rebuild or replace the affected chunk cache. Tab
+projection runs inside those existing paths.
+
+`try_reshape_line` must regenerate the expanded chunks for the edited line;
+`rebuild_lines_reusing_scroll` may retain an unchanged line and its already
+expanded cache. `hit_map_dirty` continues to mark when the whole-slice reverse
+map must be rebuilt. No new generation counter or whole-file cache is needed.
+
+## Data flow
+
+```text
+                         daemon / TUI core
+source bytes ───────────────────────────────────────────────┐
+   │                                                       │
+   ├─ display_width helpers ──> TUI glyph/style columns    │
+   │                                                       │
+   └─ SemanticFrame { raw text, byte ranges } ─────────────┤
+                                                           v
+                                                     pmacs-gpu
+                                                           │
+                     style + adornment boundaries ─────────┤
+                                                           v
+                                              source-rich chunks
+                                                           │
+                                         expand tabs to spaces
+                                         + preserve provenance
+                                                           │
+                          ┌────────────────────────────────┼─────────────┐
+                          v                                v             v
+                    cosmic-text                    hit/caret map   decoration map
+                    shaping/render                   ↕ source       source → glyph
+                          │                                │             │
+                          └────────────────────────────────┴─────────────┘
+```
+
+The invariant is that only the display projection expands a tab. Every editor,
+protocol, edit, selection, syntax, diagnostic, and adornment coordinate remains
+a source-byte coordinate.
+
+## Bets
+
+1. **Eight is the correct parity target.** It is the established TUI behavior
+   and existing tests already encode it. This work removes divergence rather
+   than introducing a new preference.
+2. **ASCII spaces are the stable shaping input.** The code font is measured as
+   monospace and spaces participate in wrapping, hit testing, and glyph ranges
+   that the existing GPU architecture already understands.
+3. **Visible-slice expansion is cheap enough.** The GPU already allocates owned
+   rich chunks for the shaped viewport. Scanning them once and allocating only
+   around actual tabs is below shaping cost and avoids a whole-file projection.
+4. **Forward rounding inside a tab is acceptable.** It matches the shipped TUI
+   inverse mapping and avoids inventing fractional positions inside one source
+   byte.
+5. **A fixed semantic constant does not require protocol v19.** There is no wire
+   representation change. If external v18 clients exist, they remain decodable
+   but must adopt the documented invariant to obtain visual parity.
+
+## Acceptance criteria
+
+1. **Canonical rule:** one exported `TAB_STOP_COLUMNS = 8` definition is shared
+   by the pmacs core and GPU; no renderer-local tab-width literals remain in
+   the touched buffer-rendering paths.
+2. **Source preservation:** inserting/opening `"\t"` leaves one tab byte in the
+   buffer, semantic frame, edits, undo history, and saved file. Rendering never
+   replaces source text.
+3. **TUI boundary behavior:** tabs beginning at columns 0, 7, and 8 end at
+   columns 8, 8, and 16 respectively in plain rendering and position mapping.
+4. **Core overlay parity:** syntax foreground spans, diagnostic underlines,
+   completion popup anchors, and generic buffer-style spans all resolve the
+   same byte boundary after tabs and wide Unicode to the same display column.
+5. **GPU shaping input:** code-buffer chunks presented to cosmic-text contain
+   no raw tab from source text or text adornments. The equivalent expanded
+   spaces end at the next logical 8-column boundary.
+6. **GPU visual geometry:** for `"\tx"`, `"1234567\tx"`, and
+   `"12345678\tx"`, the GPU lays out `x` at logical columns 8, 8, and 16.
+   A case with a width-2 Unicode character before the tab also lands on the
+   mathematically correct stop.
+7. **Caret mapping:** source carets immediately before and after a tab render at
+   the leading and trailing edges of the expanded interval, including when the
+   line wraps near that interval.
+8. **Hit testing:** clicking the tab's leading boundary resolves before the tab;
+   clicking within its expanded spaces resolves after it; clicking following
+   text resolves to its original source byte. No click returns a synthetic
+   space offset.
+9. **Styled tab:** a source foreground span exactly covering a tab colors every
+   expanded space and does not color the following source character.
+10. **Selected/diagnostic tab:** own and peer selections and diagnostic
+    squiggles whose source range covers a tab span the full projected interval
+    and remain aligned with following text.
+11. **Adornment interaction:** an inline text adornment before a source tab
+    contributes to the visible logical column, the tab still ends at the next
+    8-column stop, and source/adornment hit gravity remains deterministic.
+12. **Minimap parity:** leading and interior tabs use 8-column stops; width-2
+    and zero-width Unicode affect minimap logical columns by 2 and 0 rather
+    than 1.
+13. **Edit freshness:** inserting or deleting a tab on a visible line updates
+    shaping, caret position, hit testing, styles/decorations, and minimap shape
+    on the next normal refresh without switching buffers or forcing a full
+    rebuild.
+14. **No scope creep:** `pmacs.config` gains no tab-width key,
+    `PROTOCOL_VERSION` remains 18, and no wire message shape changes.
+15. **Quality gates:** focused default/Lua 5.4 tests, the touched acceptance
+    suite, both GPU unit and required hardware-backed tests, the standard
+    project gates, workspace sweep, and `git diff --check` pass.
+
+## Verification plan
+
+Focused checks should exercise the shared arithmetic and the two real render
+paths rather than inspecting source text:
+
+- core unit tests for valid-prefix handling, Unicode widths, and tab starts at
+  0/7/8;
+- existing and extended `TextView`, highlight, diagnostic, completion, and
+  overlay tests using byte ranges that cross tabs;
+- GPU projection-map tests for tab expansion and both mapping directions;
+- GPU layout/offscreen tests for caret, selection/diagnostic geometry,
+  wrapping, adornments, and edit freshness;
+- minimap shape tests for tabs and Unicode; and
+- `tests/tab_width_acceptance.rs` rendering one tabbed fixture through the
+  core-facing path while the GPU suite proves the frontend projection.
+
+Required commands before a PR:
+
+```text
+cargo fmt --check
+cargo clippy --workspace --all-targets -- -D warnings
+cargo test --lib
+cargo test --lib --features crdt
+cargo test --no-default-features --features lua54 --lib
+cargo test --test tab_width_acceptance
+cargo test --test m4_acceptance -- --skip basedpyright
+PMACS_REQUIRE_GPU=1 cargo test -p pmacs-gpu
+cargo test --workspace -- --skip basedpyright
+git diff --check
+```
+
+Run strict Clippy as its own command. Any known timing-only failure must be
+rerun isolated per `docs/agent-handoff.md`; a rerun is evidence only when the
+failure matches a documented flaky test.
+
+## Expected files
+
+- `pmacs-protocol/src/lib.rs` - canonical tab-stop rendering constant and
+  protocol-level documentation.
+- `src/display_width.rs` and `src/lib.rs` - shared core display-column logic and
+  module export.
+- `src/text_view.rs`, `src/highlight.rs`, `src/diag.rs`, `src/completion.rs`,
+  and `src/overlay.rs` - remove duplicated arithmetic and consume the helper.
+- `pmacs-gpu/Cargo.toml`, `Cargo.lock`, and `pmacs-gpu/src/main.rs` - direct
+  Unicode-width dependency, tab projection/provenance, geometry mapping,
+  minimap parity, and focused tests.
+- `tests/tab_width_acceptance.rs` - focused observable TUI/core parity across
+  plain text and byte-addressed overlays.
+- `docs/agent-handoff.md`, `docs/active-work.md`,
+  `docs/side-quest-backlog.md`, and this framing document - updated only after
+  implementation is proven and published according to their protocols.
+
+No Lua runtime, config-registry, syntax-query, theme, or serialized protocol
+file should need a behavior change.