pmacs/docs/folding-framing.md

# Folding — framing (Arc 6)

**Revision 3 — 2026-07-22. Status: framing only, on branch `folding`
(off canonical `main` @ `cac4961`); no implementation.** Rev 1 passed a
ground-truth review; rev 2 fixed round 1's seven findings; rev 3 fixes round
2's five majors and four minors. See §0 for the per-round changelog.

## 0. Revision history

### Round 1 (rev 1 → rev 2)

- **F1** the grid TUI is daemon-rendered and never receives `FoldState`; its
  collapse is instance-side in the daemon grid renderer. Staging reworked.
- **F2** stored range pinned to a line-aligned interior.
- **F3** the store's translation is the instance-side buffer-attached `View`
  (`BufferStyleSpanTranslator` pattern), not the frontend `translate_byte_range`.
- **F4** stale-tree fold creation refuses.
- **F5** multi-frontend point / edit-vs-fold pinned.
- **F6** explicit-buffer Lua API + validation.
- **F7** authoritative-empty `FoldState`.

### Round 2 (rev 2 → rev 3)

- **R2-1 (major).** The block-kind heuristic picked a **body line** as the
  fold head on indentation grammars — tree-sitter-python's `block` starts on
  the first statement line, so `def foo():` was left above a headless fold
  (verified in review; brace languages escaped only because `{` shares the
  introducer line). Fixed with a **head-selection ascend rule** (Q#FD1, §3),
  a no-op for brace languages. Acceptance 1 now tests both languages.
- **R2-2 (major).** The interactive-vs-programmatic split cannot live in the
  store's `View`: `View::on_edit(&Buffer, &Edit)` (`src/overlay.rs:248`) and
  `Edit` (`src/rope.rs`) carry no source frontend and no "point was inside"
  signal — only optional `crdt_op`. The `View` does **translate + drop only**;
  the **unfold is a pre-edit step at the dispatch/command layer** that knows
  the authenticated frontend and its point (Q#FD5, §5). This is the
  handoff's deferred "origin-pinned `buffer.after-edit` fan-out" gap.
- **R2-3 (major).** "CRDT op = remote = translate" misclassifies GPU typing
  (a GPU user types via CRDT ops but is editing at their own point inside a
  rendered fold). The classifier is the **authenticated source frontend's
  point, not the transport**. Stage 1 implements the unfold for the command
  path; **CRDT-origin unfold is a named Stage 3 obligation** (that is when a
  GPU user can type into a rendered fold). Q#FD5, §5, §8.
- **R2-4 (major).** Q#FD8 recreated the #120 stale-mirror trap: revert drops
  the store, emits `BufferSnapshot`, and resets the producer baseline, so the
  now-empty store is suppressed as "initial empty" and a GPU keeps rendering
  pre-revert folds unless its snapshot arm **clears the fold mirror**. That
  frontend clear is load-bearing; named as a Stage 3 obligation and pinned in
  acceptance 7 (Q#FD8, §5). Same class as [[message-gating-on-active-state]].
- **R2-5 (major).** A line-aligned tail hid non-member text on shared-closer
  lines (`} else {`, `}, [deps])`). Fixed by **keeping a closing-delimiter
  line visible** (Q#FD3, §5); delimiter-less (indentation) nodes still hide
  through their last body line. Decided, not bet.
- **Minors.** (a) unfold is **plural** — every fold containing the point.
  (b) a shared head line (`foo(() => {`) toggles **innermost-first**.
  (c) Q#FD9's rejection reason corrected: `(0,0)` is in-bounds; the reject
  comes from the ≥1-hidden-line rule. (d) Stage 2/3 re-framings must address
  three named interactions (§8): fold-aware `LineNumbers`, visible-line
  viewport/scroll accounting, and hidden-line signs/presence clamp-or-drop.

## 1. Problem and what ships

Pmacs cannot fold. `FoldState` was declared in the M11.1 semantic-frontend
design but has never been produced — the producer says "pmacs has no
instance-side fold source yet," and a test pins it is never emitted.

Arc 6 gives pmacs a fold engine (instance-side fold store + a fold source +
Lua commands), renders collapsed regions with a gutter fold marker in both
frontends, and produces `FoldState` for semantic (GPU) sessions. It is the
roadmap's "keystone gutter rider."

**`FoldState` needs no protocol bump** — the variant has been in the encoding
since M11.1 and both frontends already decode it (the TUI drops it, the GPU
has a decode arm); Arc 6 only starts *producing* it.

**Git gutter markers are a SIBLING rider, not this arc** (§11).

## 2. Ground truth (scouted 2026-07-22, `main` @ `cac4961`; verified across two review rounds)

- **`FoldState { buffer_id, folds: Vec<ByteRange> }`** —
  `pmacs-protocol/src/message.rs:886`, gated on `semantic_render`,
  DECLARED-BUT-UNPRODUCED. `semantic_render.rs:4002` pins it is never emitted.
- **`BlockAdornments`** (also unproduced) is the declared home for
  folded-region placeholders. Arc 6 does not produce it (Q#FD7).
- **No fold source exists** — the bundled grammars export
  `HIGHLIGHTS`/`INJECTIONS`/`LOCALS`/`TAGS` only, no fold query, no
  `folds.scm`. Fold source is Q#FD1. **tree-sitter-python's `block` node
  starts on the first statement line, not the `def` line** (R2-1) — the reason
  the head-selection rule is required.
- **Two frontend render paths (F1).** Grid TUI: daemon-rendered
  (`render_states` → `render_state.render_frame`, `src/daemon.rs:1106`),
  advertises `semantic_render: false` (`src/frontend.rs:385`), never receives
  `FoldState`. GPU: semantic session (`semantic_states` → `sem.render_frame`,
  `:1091`), does. Collapse is daemon-side for the TUI, wire-fed for the GPU.
- **Gutter signs are frontend-derived, not a wire channel** — fold markers
  follow suit per path; no new wire type.
- **Instance-side edit translation** is the buffer-attached `View`
  (`BufferStyleSpanTranslator`, `src/overlay.rs:235`; hook
  `on_edit(&Buffer, &Edit)` at `:248`), which sees every edit once,
  provenance-blind (R2-2): `Edit` carries only `crdt_op`, no source frontend.
- **Staleness is detectable** — `ParseViewHandle::current()`
  (`src/syntax.rs:696`) is `None` before first settle; `pending_edit_count()`
  (`:706`) is nonzero while edits await settle.
- **Greenfield** Lua/commands.

## 3. Fold source (Q#FD1) — structural node folding with head selection and closer-aware tail

The grammars ship no fold queries, so pmacs defines "what is foldable." v1 is
**structural node folding for grammar-backed buffers** (reuses the parse trees
for every bundled grammar and injection layer, zero per-language authoring).
Indentation folding (grammarless fallback) and curated per-language queries
(quality pass) are DEFERRED (§11).

The source, at a point:

1. **Match** the nearest enclosing NAMED node `B` spanning **≥2 source lines**
   (source lines, not display rows — soft wrap is frontend-only and unknowable
   instance-side), biased to block-like kinds (`block`, `body`, `*_list`,
   `declaration_list`, `statement_block`, brace/bracket-delimited nodes).
2. **Head selection (R2-1).** Ascend: while `B`'s parent introduces `B` (a
   `function_definition` / `if_statement` / … whose block child is `B`) **and**
   `parent.start_line < B.start_line`, take the parent as the head node. This
   makes the **introducer line the head** — `def foo():` on Python, where the
   `block` starts a line lower. It is a **no-op for brace languages**, where
   `{` shares the introducer's line (`parent.start_line == B.start_line`), so
   the head node stays `B` and the result is identical.
3. **Tail selection (R2-5).** The hidden interior is a whole-line range. Its
   first hidden line is `head_line + 1`. Its last hidden line is:
   - if `B`'s last line begins with `B`'s **closing-delimiter token**
     (`}`/`)`/`]`, and `end`-style closers later) — a brace/bracket node —
     then `B.last_line - 1`, **keeping the closer line visible**. This is what
     keeps `} else {` and `}, [deps])` on screen with their trailing siblings.
   - else (a delimiter-less node, e.g. a Python `block`) — `B.last_line`,
     hiding through the last body line.

The stored range is the byte range `[end of head_line, end of last-hidden
line]` (§5). A node that yields **zero** hidden lines (e.g. `fn f() {}` on two
lines, empty body) is **not foldable**.

**Stale-tree rule (Q#FD10, F4).** The source reads
`ParseViewHandle::current()`; if it is `None` (no settle yet) or
`pending_edit_count() > 0` (settled coordinates are stale), the fold command
**refuses with a status message and stores nothing** — a fold is durable state
and must not be computed against stale coordinates. Settle is a main-thread
pump, so the window is sub-frame. Translate-through-pending is a §11
refinement.

The block-kind heuristic (step 1) remains a taste bet (Bet B1); step 2 fixed
the *determinable* Python defect, which was not taste.

## 4. Where fold state lives (Q#FD2)

Instance-side, per the wire contract. A **per-buffer fold store** (a set of
byte ranges) lives beside the buffer, attached as a `View` (F3). Commands
mutate it; the daemon grid renderer reads it directly (Stage 2); the semantic
producer ships it as `FoldState` to GPU sessions (Stage 3). Nested folds are
allowed; the store is **shared by every attached frontend** (Emacs parity).

## 5. Fold model semantics (Q#FD3, Q#FD5, Q#FD6, Q#FD8)

**Stored range = line-aligned hidden interior (Q#FD3).** A fold is identified
by its **head line** (the introducer, §3 step 2), which stays visible with a
frontend-drawn ellipsis. The stored byte range is `[end of head line, end of
the last hidden line]`, where the last hidden line is chosen by §3 step 3 —
so a **closing-delimiter line stays visible** (fixing `} else {`), while a
delimiter-less node hides through its last body line. One normalized form is
computed by the source and seen identically by the store, the grid renderer,
the wire, and `folds()`.

**Point and folds (Q#FD3, F5).**
- Folding a range containing the **invoking frontend's** point moves that
  point to the head line.
- The store is shared, so another frontend's cursor may sit inside a newly
  folded range. "No cursor inside a fold" is a **per-cursor, render-time**
  invariant (its caret clamps to the head on that frontend's next frame — a
  Stage 2/3 concern). In **Stage 1** there is no motion-awareness, so the
  invariant is **creation-time-only**; acceptance is scoped to that so it
  cannot self-contradict.

**Edits and folds — two separated mechanisms (Q#FD5, Q#FD6, R2-2, R2-3).**
- The store's buffer-attached `View` does **translation and drop only**, and
  is **provenance-blind**: on every edit it translates each fold's range, and
  **drops** any fold whose head/tail the edit destroys or whose interior
  collapses below one line. It cannot unfold-on-typing because `Edit` carries
  no frontend and no point (R2-2).
- **Unfolding on an interactive edit is a pre-edit step at the dispatch layer**
  (which holds the authenticated frontend and its point): before applying an
  edit that a frontend is making at its point, unfold **every** fold
  containing that point (plural — minor a). The classifier is the
  **authenticated source frontend's point, not the transport** (R2-3): a GPU
  user's CRDT-op insert at a point inside a fold is interactive and must
  unfold, even though it arrives as a CRDT op. **Stage 1 implements this for
  the command path** (daemon `dispatch_key` self-insert/delete, which has the
  frontend + point); **CRDT-origin unfold is a Stage 3 obligation**, wired
  when the GPU renders folds and a GPU user can type into one.

**Store lifecycle vs producer baseline (Q#FD8, F3, R2-4).** Three coupled
resets, kept distinct:
- The **per-session producer suppression baseline** resets on `BufferSnapshot`
  so the fold set is re-shipped to a (re)joining semantic session.
- The **per-buffer store** is dropped on buffer **content replacement**
  (revert/reload) — its ranges name bytes that no longer exist (revalidation
  is a §11 refinement).
- **The frontend fold mirror must clear on `BufferSnapshot` (R2-4, Stage 3
  obligation).** Revert simultaneously drops the store, emits a snapshot, and
  resets the baseline, so the producer sees an empty store with a fresh
  baseline and correctly suppresses it as "initial empty" — which means the
  GPU keeps rendering pre-revert folds **unless its snapshot arm clears fold
  state**, exactly as it already clears spans/decorations. The
  empty-after-snapshot suppression is correct **only because** the snapshot
  clears the frontend mirror; this pairing is load-bearing and is pinned in
  acceptance 7.

## 6. Lua command surface and validation (Q#FD4, Q#FD11)

**Interactive commands** (resolve to the invoking frontend's active-window
buffer — command context, not ambient resolution). On a head line shared by
more than one fold, they act **innermost-first** (minor b):

- `fold.toggle`, `fold.close`, `fold.open`, `fold.close-all`, `fold.open-all`.
- `close-all` folds **top-level** foldable regions only (Emacs `hs-hide-all`
  parity — nested regions are not auto-folded; feeds B2). `open-all` clears.

**Data API (Q#FD4, F6): explicit buffer, no ambient resolution** (matching
#127): `pmacs.fold.fold(buffer, range)`, `unfold(buffer, range)`,
`folds(buffer)`, `toggle(buffer, pos)`.

**Validation (Q#FD11, F6).** `fold(buffer, range)` rejects unless: the buffer
is a normal document buffer; both endpoints are UTF-8 boundaries; and the
range normalizes to **≥1 hidden line**. Q#FD9 (terminals never fold) follows
from the last clause (minor c): `(0,0)` on an empty terminal identity buffer
is technically in-bounds, but it normalizes to zero hidden lines and is
rejected — no special case.

Bindings remain the user's call (Emacs has no single convention).

## 7. Frontend collapse + gutter marker (Q#FD7)

- **Grid TUI — daemon-rendered.** The daemon grid renderer reads the store and
  omits each fold's hidden lines, showing the head line with an ellipsis and a
  gutter fold glyph. No wire (the vterm Stage 2 shape).
- **Semantic GPU — wire-fed.** The GPU receives `FoldState`, excludes the
  hidden bytes from its shaped slice, shows the head + ellipsis + fold glyph,
  and makes caret/hit-test fold-aware. It also clears its fold mirror on
  `BufferSnapshot` (R2-4).

The placeholder is frontend-local (not a `BlockAdornment`); the gutter marker
is derived per path like the diagnostic sign bars — no new wire type.

## 8. Staging and scope

- **Stage 1 — fold engine (instance), headless.** The per-buffer store + its
  translating/dropping `View`; the structural source with head selection,
  closer-aware tail, and the stale-tree rule; the Lua data API + interactive
  commands + validation; the **command-path pre-edit unfold**; `FoldState`
  production (authoritative-empty, diff-suppressed); headless acceptance. No
  rendering. **Approval-critical.**
- **Stage 2 — grid (daemon-rendered) collapse + gutter marker.** The daemon
  grid renderer collapses folded interiors and draws the TUI gutter glyph +
  head placeholder; caret clamps to the head. **Must also make the
  daemon-computed `LineNumbers` family fold-aware** (skipped lines; relative
  distance measured across a fold), **count visible lines in viewport/scroll
  accounting**, and **clamp-to-head-or-drop diagnostic signs on hidden lines**
  (minor d).
- **Stage 3 — GPU collapse + gutter marker.** The GPU consumes `FoldState`,
  excludes folded bytes, draws the glyph, makes caret/hit-test fold-aware at
  TUI parity, **clears the fold mirror on `BufferSnapshot`** (R2-4), wires
  **CRDT-origin interactive unfold** (R2-3), and applies the same
  hidden-line rules to **peer-presence rects and line numbers** (minor d).

Stages 2–3 are sketched; each is re-framed in detail after the prior stage
lands. This framing asks approval for the architecture and Stage 1's detail.

## 9. Numbered decisions

- **Q#FD1** Structural node folding: match block-like node ≥2 source lines →
  **ascend to the introducer head** → **closer-aware tail** (closing-delimiter
  line kept visible; delimiter-less nodes hide through the last body line);
  stale/absent tree refuses. Indentation and curated queries deferred. (§3)
- **Q#FD2** Fold state is instance-side, per-buffer, a set of ranges, shared
  by all frontends; nested allowed. (§4)
- **Q#FD3** Stored range = line-aligned hidden interior; head line visible;
  closer line visible for closer-terminated nodes; invoking point moves to the
  head; no-cursor-inside is per-cursor render-time, creation-time-only in
  Stage 1. (§5)
- **Q#FD4** Interactive commands (invoking frontend's buffer, innermost-first
  on shared heads); data API takes an explicit buffer, no ambient resolution;
  bindings decided by the user. (§6)
- **Q#FD5** The store `View` translates + drops only (provenance-blind); the
  **pre-edit interactive unfold** lives at the dispatch layer, keyed on the
  authenticated source frontend's point (not transport), unfolding **every**
  fold containing it; Stage 1 = command path, CRDT-origin = Stage 3. (§5)
- **Q#FD6** A fold whose head/tail an edit destroys is dropped, not
  re-anchored. (§5)
- **Q#FD7** Placeholder + gutter marker are frontend-local per path (TUI
  daemon-rendered, GPU wire-fed); `BlockAdornments` stays unproduced; no new
  wire type. (§7)
- **Q#FD8** `FoldState` (semantic sessions only) is whole-buffer,
  authoritative-empty (initial empty suppressed until a fold exists; unchanged
  suppressed; non-empty→empty emits exactly one empty frame); its per-session
  baseline resets on `BufferSnapshot`; the STORE drops on content replacement;
  **the GPU fold mirror must clear on `BufferSnapshot`** or empty-after-revert
  suppression leaves stale folds (#120 class). (§5, §8)
- **Q#FD9** Terminals never fold — from the ≥1-hidden-line validation, not
  from bounds. (§6)
- **Q#FD10** Fold creation against a `None`/stale parse tree refuses. (§3)
- **Q#FD11** `fold(buffer, range)` validates buffer kind, UTF-8 boundaries,
  ≥1 hidden line; rejects otherwise. (§6)

## 10. Bets

- **B1** The block-kind heuristic (§3 step 1) picks a fold *target* users find
  natural. FALSIFIABLE on real Rust/Python; fallback is curated Tier-1
  queries. (Head selection and closer-aware tail are now decided, not bet.)
- **B2** Whole-buffer `FoldState` is cheap: folds are a handful, `close-all`
  is top-level only, so the set is O(top-level blocks). No viewport scoping.
- **B3** Both collapse paths reuse existing machinery (daemon cell painting;
  the GPU's projected↔source map) — no new layout engine.

## 11. Deferred (named)

- Indentation folding for grammarless buffers (Q#FD1 (C)).
- Curated per-language fold queries (Q#FD1 (A)).
- Translate-a-node-range-through-pending-edits so creation need not refuse on a
  stale tree (Q#FD10 refinement).
- Fold-store revalidation against new content on revert/reload (v1 drops it).
- Persisted folds across sessions; `fold.hide-level N`; auto-fold-on-open.
- `BlockAdornments` production (rich placeholders, diff zones, blame bands).
- **Git gutter markers** — the sibling gutter rider; separate diff source.
- Search revealing folds (a match inside a fold auto-unfolds) — Stage 2+.

## 12. Acceptance — Stage 1 (engine)

1. **Head selection, both grammar shapes (R2-1).** In Rust `fn foo() { … }`,
   a point in the body folds with head line `fn foo() {`. In Python
   `def foo(): / body`, a point in the body folds with head line `def foo():`
   — **not** a body line. `close-all` on each yields the introducer as head.
2. **Stale/absent tree (Q#FD10).** With `current() == None`, and with
   `pending_edit_count() > 0` after an edit before settle, `fold.toggle`
   refuses and stores nothing; after settle it succeeds.
3. **Commands.** `fold.toggle` folds the enclosing region and unfolds on a
   head; `close-all` folds top-level regions only (a nested inner region is
   not auto-folded); `open-all` clears.
4. **Range semantics (Q#FD3, R2-5).** For a brace node the closing-delimiter
   line is **outside** the stored range (stays visible) and a `} else {` case
   keeps `else {` visible; for a Python node the last body line is **inside**
   the range (hidden). `folds(buffer)` returns exactly the normalized ranges.
5. **Point (Q#FD3).** Folding a range containing the invoking point moves it
   to the head; Stage 1 does not prevent later motion into a fold.
6. **Edits — separated mechanisms (Q#FD5/Q#FD6, R2-2/3).** The store `View`
   translates a fold across a programmatic edit inside it and drops a fold
   whose head an edit deletes — with no knowledge of source. A command-path
   self-insert at a point inside a fold (or inside **nested** folds) unfolds
   **all** of them before the edit applies. (CRDT-origin unfold is asserted in
   Stage 3.)
7. **`FoldState` production (Q#FD8, F7, R2-4).** The flipped pin test asserts
   all three transitions to a semantic session — nothing until a fold exists,
   nothing when unchanged, exactly one empty frame after `open-all` — while
   `BlockAdornments` is still never emitted; the per-session baseline resets on
   `BufferSnapshot`. The test documents that empty-after-snapshot suppression
   is correct only paired with the Stage 3 frontend-mirror clear.
8. **Store lifecycle.** Buffer content replacement (revert) drops the store.
9. **Nested folds.** Folding an inner then an outer region yields two ranges;
   `open-all` clears both; a shared head line toggles innermost-first.
10. **Injected layer.** A fold sourced inside an injected layer — a fenced
    code block in a markdown buffer — returns the inner block's range, proving
    the source walks injection layers, not just the root tree.
11. **Lua data API (Q#FD4/Q#FD11).** `pmacs.fold.*` with an explicit buffer
    drives the above and round-trips `folds()`; an out-of-bounds,
    non-boundary, or sub-one-line range is rejected; a fold on a terminal
    identity buffer is rejected via the ≥1-hidden-line rule (Q#FD9).

## 13. Gates (Stage 1)

`cargo fmt --check`; strict workspace Clippy; `cargo test --lib` and
`--features crdt`; `tests/folding_acceptance.rs` (default + CRDT); `cargo test
--test m4_acceptance -- --skip basedpyright`; `PMACS_REQUIRE_GPU=1 cargo test
-p pmacs-gpu`; the workspace sweep; `git diff --check`. New behavioral
acceptance is bite-verified with `scripts/bite`.

## 14. Branch and PR plan

Branch `folding`, worktree `../pmacs-folding`, off canonical `main` @
`cac4961`. This framing (rev 1 → rev 3) is its opening commits. After
approval, Stage 1 implements on this same branch and opens as the first
folding PR. Stages 2 and 3 are separate branches/PRs off the main resulting
from the prior stage, each with its own detailed framing.