22 KiB

Raw Blame History

Folding — framing (Arc 6)

Revision 2 — 2026-07-22. Status: framing only, on branch folding (off canonical main @ cac4961); no implementation. Revision 1 passed a ground-truth review (every scouted claim verified against the tree) but the reviewer found one architectural mis-framing and six spec gaps. Revision 2 fixes all of them; see §0 for the changelog.

0. Revision 2 — review round 1 resolutions

F1 (architectural). R1's Stage 2 said "the TUI consumes FoldState." It cannot: the grid TUI advertises semantic_render: false (src/frontend.rs:385), so the per-session outgoing filter never sends the FoldState family to it, and the TUI has no layout of its own — the daemon renders its cell grid (render_states vs semantic_states, src/daemon.rs:875/881; the grid path is render_state.render_frame(...) at :1106). The TUI collapse is instance-side rendering in the daemon grid renderer, reading the fold store directly — no wire. FoldState on the wire serves ONLY semantic (GPU) sessions. Staging reworked accordingly (§8): Stage 2 is grid/daemon rendering; Stage 3 is the wire-fed GPU.
F2. Stored range semantics pinned: the store holds line-aligned interior ranges (Q#FD3, §5); the head line stays visible, the interior including the closing-delimiter line is hidden. §7 rewritten to match.
F3. The fold store's edit-translation is the instance-side buffer-attached View (BufferStyleSpanTranslator pattern, src/overlay.rs:235), which sees every real edit regardless of source — not the frontend-side translate_byte_range. The two "resets" are split: per-session producer baseline vs per-buffer store lifecycle (Q#FD8, §5).
F4. Stale-tree fold creation pinned: read ParseViewHandle::current() (src/syntax.rs:696); if it is None or pending_edit_count() > 0 (:706), refuse with a status message in v1 (Q#FD10, §3).
F5. Multi-frontend point + edit-vs-fold rules pinned (Q#FD3, Q#FD5, §5): the invoking frontend's point moves to the head; interactive-point-inside edits unfold, programmatic/remote edits translate; the invariant is creation-time-only in Stage 1.
F6. Lua surface takes an explicit buffer (no ambient resolution, per #127), with full range validation (Q#FD4, Q#FD11, §6) — which is also what makes Q#FD9 (terminals never fold) hold.
F7. FoldState follows the authoritative-empty discipline; the non-empty→empty transition (open-all) emits exactly one empty frame (Q#FD8, acceptance 4).
Minors. "≥2 source lines" not display rows; close-all folds top-level only (Emacs hs-hide-all parity, feeds B2); an injected-layer fold acceptance added; and an explicit note that FoldState needs no protocol bump.

1. Problem and what ships

Pmacs cannot fold. The FoldState wire family was declared in the M11.1 semantic-frontend design but has never been produced — the producer says in so many words that "pmacs has no instance-side fold source yet," and a test pins that FoldState is never emitted.

Arc 6 gives pmacs a fold engine (instance-side fold model + a fold source + Lua commands), renders collapsed regions with a gutter fold marker in both frontends, and produces FoldState for semantic (GPU) sessions. It is the roadmap's "keystone gutter rider": it lights up an already-declared wire family, adds the fold-marker rider beside the existing diagnostic signs, and is a visible feature.

FoldState needs no protocol bump. The variant has been in the wire encoding since M11.1 and both frontends already decode it (the TUI drops it, the GPU has a decode arm); Arc 6 only starts producing it. No PROTOCOL_VERSION change, no SUPPORTED change.

Git gutter markers are a SIBLING rider, not this arc. They ride the same gutter but need a diff source, unrelated to folding. Named as a deferral (§11), framed separately.

2. Ground truth (scouted 2026-07-22, `main` @ `cac4961`; verified in review)

FoldState { buffer_id, folds: Vec<ByteRange> } exists in pmacs-protocol/src/message.rs:886, gated on semantic_render, DECLARED-BUT-UNPRODUCED. Doc: "the instance's authoritative fold set as document facts. Folding is an instance command-semantics concern (Lua can fold); the visual collapse is a frontend layout concern."
semantic_render.rs:4002 (block_adornments_and_fold_state_still_never_emitted) asserts FoldState/BlockAdornments are never sent. Stage 1 flips this to assert FoldState IS produced while BlockAdornments stays unemitted (F7).
BlockAdornments (also unproduced) is the declared home for "folded-region placeholders." Arc 6 does not produce it — the fold placeholder is frontend-local (Q#FD7).
No fold source exists. The bundled grammars export HIGHLIGHTS/INJECTIONS/LOCALS/TAGS queries only — no fold query and no folds.scm (LanguageEntry has exactly those fields). The roadmap's "tree-sitter fold ranges" is not free; the fold source is Q#FD1.
Two frontend render paths, not one (F1). The grid TUI is daemon-rendered (render_states → render_state.render_frame, src/daemon.rs:1106) and advertises semantic_render: false (src/frontend.rs:385), so it never receives FoldState. The GPU is a semantic session (semantic_states → sem.render_frame, :1091) and does receive it. Fold collapse is therefore daemon-side for the TUI and wire-fed for the GPU.
Gutter signs are frontend-derived, not a wire channel. The GPU's collect_gutter_sign_rects computes diagnostic sign bars locally; the TUI's are painted daemon-side. Fold markers follow the same model per path — no new wire type.
Instance-side edit translation is a solved pattern (F3). Compile-mode's BufferStyleSpanTranslator (src/overlay.rs:235) is a buffer-attached View that sees every real edit — commands, CRDT ops, LSP workspace edits, Lua — once per edit, fragment-preserving. The fold store attaches the same kind of View. The frontend-side translate_byte_range (pmacs-gpu/src/main.rs) is a different thing (the GPU translating its own received copies across optimistic edits) and is not the store's mechanism.
Staleness is detectable (F4). ParseViewHandle::current() (src/syntax.rs:696) returns None before the first settle and the latest settled bundle otherwise; pending_edit_count() (:706) is nonzero while edits await settle.
Greenfield Lua/commands. No existing fold surface or commands.

3. Fold source (Q#FD1) — the load-bearing decision

The grammars ship no fold queries, so "what is foldable" must be defined by pmacs. Three options:

(A) Curated per-language fold queries (nvim-treesitter's @fold-capture model). Highest quality — but per-language authoring plus ongoing maintenance, exactly the work the grammars declined to ship.
(B) Structural node folding. At a point, fold the nearest enclosing NAMED node that spans ≥2 source lines (F-minor: source lines, not display rows — soft wrap is frontend layout and unknowable instance-side), biased to block-like kinds by a small shared heuristic on node-kind names (block, body, *_list, declaration_list, statement_block, brace/bracket-delimited nodes). Reuses the parse trees already present for every bundled grammar AND every injection layer. Zero per-language authoring.
(C) Indentation folding. Language-agnostic, works with no grammar, predictable, but ignores syntax.

Recommendation: (B) structural node folding for grammar-backed buffers as the v1 engine. Reuses tree-sitter, no per-language work, covers all bundled grammars and injection layers day one. (C) is the grammarless fallback, DEFERRED so Stage 1 stays scoped to grammar buffers; (A) is a later quality pass, DEFERRED.

Stale-tree rule (Q#FD10, F4). The source reads ParseViewHandle::current(). If it is None (no settle yet) or pending_edit_count() > 0 (the settled tree's coordinates are stale relative to the current buffer), a fold command refuses with a status message and stores nothing — a fold is durable state and must not be computed against stale coordinates. Settle is sub-frame, so the refuse window is tiny. Translate-the-node-range-through-pending-edits is a named refinement (§11).

The block-kind heuristic (B) is the part most likely to feel wrong ("it folded the tiny inner block, not the function"); Bet B1 (§10) states it and names the fallback (curated Tier-1 queries).

4. Where fold state lives (Q#FD2)

Instance-side, per the wire contract. A per-buffer fold store (a set of byte ranges) lives beside the buffer, attached as a View (F3). Commands mutate it; the daemon grid renderer reads it directly to collapse the TUI (Stage 2); the semantic producer ships it as FoldState to GPU sessions (Stage 3). Nested folds are allowed — the store is a set, consumers collapse the union.

The store is shared by every attached frontend (Emacs parity): folds are a document-level view fact, not per-window. Per-cursor consequences of that sharing are pinned in §5 (F5).

5. Fold model semantics (Q#FD3, Q#FD5, Q#FD6)

Stored range = line-aligned interior (Q#FD3, F2). A fold is identified by its head line H. The stored/shipped byte range is the hidden interior: from the newline that terminates H through the end of the last source line the folded region spans. So:

H (with its opener, e.g. fn foo() {) stays visible, with a frontend-drawn ellipsis at its end.
The interior lines and the closing-delimiter line (}) are hidden — the range ends at the end of the line containing the region's last byte.
The structural source yields a raw node span [node.start, node.end); the store normalizes it to this line-aligned interior before anything else (renderer, wire, folds()) sees it. One normalized form, one meaning, everywhere — resolving the R1 §5/§7 contradiction.

Point and folds (Q#FD3, F5).

Folding a range that contains the invoking frontend's point moves that point to the head line H (Emacs hs-minor-mode).
The store is shared, so another frontend's cursor may already sit inside a newly folded range. "A cursor cannot sit inside a fold" is a per-cursor, render-time invariant: on that frontend's next frame the caret clamps to H (a Stage 2/3 rendering concern). In Stage 1 there is no motion-awareness (deferred to Stage 2/3), so the invariant is creation-time-only: folding moves the invoking point out, but later motion re-entering a fold is not yet prevented. Acceptance is written to that scope so it cannot self-contradict.

Edits and folds (Q#FD5, Q#FD6, F5). The store's buffer-attached View (F3) sees every edit:

An interactive edit at the invoking frontend whose point is inside a fold unfolds that fold first — you cannot type into hidden text you cannot see.
A programmatic or remote edit (a peer CRDT op, an LSP workspace edit, a Lua buffer edit) translates the fold through the View, keeping it folded — it is not a person typing into the hidden region.
A fold whose head or tail an edit destroys (e.g. the head line deleted, or the range collapses below one hidden line) is dropped, not re-anchored — a fold is view state, never data.

Store lifecycle vs producer baseline (Q#FD8, F3). Two distinct resets, previously conflated:

The per-session producer suppression baseline resets on BufferSnapshot so the fold set is re-shipped to a (re)joining semantic session — the established producer discipline.
The per-buffer fold store is dropped or revalidated on buffer content replacement (revert/reload): the ranges describe bytes that no longer exist, so revert clears the store (revalidation against the new content is a §11 refinement).

6. Lua command surface and validation (Q#FD4, Q#FD11)

Interactive commands (resolve to the invoking frontend's active-window buffer — command context, not ambient resolution):

fold.toggle — fold the enclosing foldable region at point, or unfold if point's line is a fold head.
fold.close / fold.open — explicit fold/unfold at point.
fold.close-all / fold.open-all — fold every top-level foldable region (Emacs hs-hide-all parity — nested regions are not auto-folded; see B2) / clear the fold set.

Data API (Q#FD4, F6): explicit buffer, no ambient resolution (matching #127's deliberate refusal of ambient-buffer lookup):

pmacs.fold.fold(buffer, range), unfold(buffer, range), folds(buffer) -> {range,...}, toggle(buffer, pos).

Validation (Q#FD11, F6). fold(buffer, range) validates and rejects otherwise: the buffer exists and is a normal document buffer; the range is in-bounds; both endpoints are UTF-8 char boundaries; the range normalizes (§5) to at least one hidden line. This validation is what makes Q#FD9 hold: a terminal identity buffer is empty, so every range is out-of-bounds and rejected — no fold can be stored on a terminal even from Lua, with no special case.

Bindings are left for this review round — Emacs uses C-x C-z / hs-* / outline C-c @; pmacs has no precedent, so the binding is the user's call.

7. Frontend collapse + gutter marker (Q#FD7)

Two paths (F1):

Grid TUI — daemon-rendered. The daemon grid renderer reads the fold store directly and omits each fold's hidden interior from the cells it paints, showing H with an ellipsis; it draws the gutter fold glyph on H. No wire, same shape as vterm Stage 2's daemon-painted terminal cells.
Semantic GPU — wire-fed. The GPU receives FoldState, excludes the hidden bytes from its shaped code slice, shows H with an ellipsis, and draws the fold glyph on H. Caret/hit-test gain a fold-aware step (the largest per-frontend cost, and why the GPU is its own stage).

In both paths the placeholder is frontend-local (an ellipsis / ⋯ N lines), not a BlockAdornment — Q#FD7 keeps BlockAdornments unproduced. The gutter marker is derived from the fold set per path, like the diagnostic sign bars — no new wire type.

8. Staging and scope

Mirrors vterm; reworked for F1 (the TUI path is daemon-side, not a wire consumer).

Stage 1 — fold engine (instance), headless. The per-buffer fold store + its buffer-attached translating View; the structural fold source with the stale-tree rule; the Lua data API + interactive commands + validation; FoldState production for semantic sessions (authoritative-empty, diff-suppressed); and headless acceptance. No rendering — folds are asserted in the store and on the wire, not on screen. Approval-critical.
Stage 2 — grid (daemon-rendered) collapse + gutter marker. The daemon grid renderer collapses folded interiors and draws the TUI gutter fold glyph
- head placeholder; caret handling clamps to H. Instance-side rendering work; no wire change.
Stage 3 — GPU collapse + gutter marker. The GPU consumes FoldState, excludes folded bytes from its shaped slice, draws the fold glyph and makes caret/hit-test fold-aware, at TUI parity.

Stages 2–3 are sketched here and re-framed in detail after Stage 1 lands. This framing asks approval for the architecture and Stage 1's full detail.

9. Numbered decisions

Q#FD1 Fold source: structural tree-sitter node folding (v1); indentation fallback and curated queries deferred. (§3)
Q#FD2 Fold state is instance-side, per-buffer, a set of ranges, shared by all frontends; nested folds allowed. (§4)
Q#FD3 Stored range is the line-aligned hidden interior (head line visible, closing-delimiter line hidden); the invoking point moves to the head; "no cursor inside a fold" is a per-cursor render-time invariant, creation-time-only in Stage 1. (§5)
Q#FD4 Interactive commands fold.toggle/close/open/close-all/open-all (invoking frontend's active buffer); data API pmacs.fold.* takes an explicit buffer, no ambient resolution; bindings decided in review. (§6)
Q#FD5 Interactive edit with point inside a fold unfolds it first; programmatic/remote edits translate the fold. (§5)
Q#FD6 A fold whose head/tail an edit destroys is dropped, not re-anchored. (§5)
Q#FD7 Placeholder + gutter marker are frontend-local per path; the TUI path is daemon-rendered, the GPU path wire-fed; BlockAdornments stays unproduced; no new wire type. (§7)
Q#FD8 FoldState (to semantic sessions only) is whole-buffer, authoritative-empty (initial empty suppressed until a fold exists; unchanged suppressed; non-empty→empty emits exactly one empty frame), and its per-session baseline resets on BufferSnapshot. The per-buffer STORE is a separate lifecycle, dropped on buffer content replacement. (§5, §8)
Q#FD9 Terminal identity buffers never fold — guaranteed by validation (empty buffer ⇒ out-of-bounds ⇒ rejected), not a special case. (§6)
Q#FD10 Fold creation against a None or stale (pending_edit_count() > 0) parse tree refuses with a message; no fold is stored. (§3)
Q#FD11 Explicit-fold(buffer, range) validates buffer kind, bounds, UTF-8 boundaries, and ≥1 hidden line; rejects otherwise. (§6)

10. Bets

B1 "Nearest enclosing block-like node spanning ≥2 source lines" is predictable enough for a v1 fold without curated queries. FALSIFIABLE: if the review finds the fold target surprising on real Rust/Python, fall back to curated queries for Tier-1 languages.
B2 Whole-buffer FoldState is cheap: folds are a handful, and close-all folds top-level only (Q#FD4), so the set is O(top-level blocks), far below the style-span volume the producer already ships. No viewport scoping.
B3 The two collapse paths reuse existing machinery: the daemon grid renderer already paints cells from instance state (vterm Stage 2), and the GPU already has a projected↔source map for adornments — neither needs a new layout engine.

11. Deferred (named)

Indentation folding for grammarless buffers (Q#FD1 (C)).
Curated per-language fold queries (Q#FD1 (A)).
Translate-a-node-range-through-pending-edits so fold creation need not refuse on a stale tree (Q#FD10 refinement).
Fold-store revalidation against new content on revert/reload (Q#FD8: v1 drops the store).
Persisted folds across sessions (saveplace-style).
fold.hide-level N / outline-style folding by depth; auto-fold-on-open.
BlockAdornments production (rich placeholders, diff zones, blame bands).
Git gutter markers — the sibling gutter rider; separate diff source, separate framing.
Search revealing folds (a match inside a fold auto-unfolds) — Stage 2+.

12. Acceptance — Stage 1 (engine)

Structural source. At a point inside a multi-line block, the source returns the enclosing block-like node normalized to its line-aligned interior; at top level it returns the enclosing item; in a grammarless buffer it returns nothing (fallback deferred).
Stale/absent tree (Q#FD10). With current() == None, and with pending_edit_count() > 0 after an edit before settle, fold.toggle refuses and stores nothing; after settle it succeeds.
Commands. fold.toggle folds the enclosing region and unfolds on a fold head; close-all folds every top-level block-like region (nested not auto-folded); open-all clears.
Range semantics (Q#FD3). The stored/shipped range is the line-aligned interior: the head line's bytes are outside it, the closing-delimiter line is inside it; folds(buffer) returns exactly the normalized ranges.
Point (Q#FD3). Folding a range containing the invoking point moves it to the head; the creation-time-only scope holds (Stage 1 does not prevent later motion into a fold).
Edits (Q#FD5/Q#FD6). An interactive edit at a point inside a fold unfolds it; a programmatic edit inside a fold translates it; an edit deleting the head drops it; each leaves a consistent set.
FoldState production (Q#FD8, F7). The flipped pin test asserts all three transitions to a semantic session — nothing until a fold exists, nothing when unchanged, exactly one empty frame after open-all — while BlockAdornments is still never emitted; the per-session baseline resets on BufferSnapshot.
Store lifecycle. Buffer content replacement (revert) drops the store.
Nested folds. Folding an inner then an outer region yields two ranges; open-all clears both.
Injected layer (§3 injection-coverage claim). A fold sourced inside an injected layer — a fenced code block in a markdown buffer — returns the inner block's range, proving the source walks injection layers, not just the root tree.
Lua data API (Q#FD4/Q#FD11). pmacs.fold.fold/unfold/folds/toggle with an explicit buffer drive all of the above and round-trip folds(); an out-of-bounds, non-boundary, or sub-one-line range is rejected; a fold on a terminal identity buffer is rejected (Q#FD9).

13. Gates (Stage 1)

The standing suite: cargo fmt --check; strict workspace Clippy; cargo test --lib and --features crdt; the new tests/folding_acceptance.rs (default + CRDT); cargo test --test m4_acceptance -- --skip basedpyright; PMACS_REQUIRE_GPU=1 cargo test -p pmacs-gpu; the workspace sweep; git diff --check. New behavioral acceptance is bite-verified with scripts/bite.

14. Branch and PR plan

Branch folding, worktree ../pmacs-folding, cut from canonical main @ cac4961. This framing (rev 1 → rev 2) is its opening commits. After approval, Stage 1 implements on this same branch and opens as the first folding PR. Stages 2 and 3 are separate branches/PRs off the main resulting from the prior stage, each with its own detailed framing.

22 KiB Raw Blame History Unescape Escape