`main` moved from c49a8c7 (folding Stage 1, #142) to 47581f4 (web
grammars, #146) while Stage 2 was in framing and implementation. The
text merge is clean, but it is NOT semantically clean: #146 added three
new `Viewport { .. }` literals to `src/highlight.rs`'s unit tests, and
Stage 2 gives `Viewport` a `folds` field. Merged alone, `cargo test
--lib` fails to compile — so the carry-over is resolved here rather
than left for CI to discover.
Merged (not rebased) so the four framing revisions the review rounds
cite by SHA (59410c0, e221f13, 8160d66, 4222ffa) stay reachable.
Full gate suite re-run on the merged tree.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q5BkezMppbpCgGAYk2ftxV
Implements docs/folding-stage2-framing.md rev 4. The daemon grid
renderer now consults the fold store: hidden lines are omitted, rows
below shift up, and every consumer that assumed
`display_row = source_line - view_top` routes through one shared
projection. No wire schema change and no protocol bump (Bet B6) — the
collapse is entirely daemon-side; the GPU path is Stage 3.
The spine (Q#FD12) is `src/fold_view.rs`: a `VisibleLineMap` derived
from `FoldRegistry::folds` plus a window's line offsets and never
stored. Its unit is a merged **hidden component** — overlapping OR
adjacent hidden intervals unioned, each keeping the one visible
`head_line` and that line's exact `head_position`. Adjacent intervals
merge because the later fold's head is itself hidden, which is what
makes nesting, shared heads, and crossing overlap all resolve to a
head that can actually render (round-3 F2).
Instances are short-lived and built **per rendered window** and **per
command/event operation**, never once per frame: `paint_frame` renders
several windows that may show different buffers, so a singleton would
leak one pane's folds into another (round-2 F2). The render instance
rides on a lifetime-bearing `Viewport<'a>` as `Option<&'a
VisibleLineMap>` — a shared ref is `Copy`, so `Viewport` stays `Copy`
(Bet B7).
Rendering:
- `TextView::render` walks visible lines; the head line gets a
trailing content-area ellipsis (Q#FD13).
- The gutter walks visible lines too: Absolute keeps the raw `line+1`,
Relative/Hybrid measure VISIBLE distance anchored on the cursor's
visible head (Q#FD14). The fold glyph takes the col-0 sign cell only
when a gutter exists — line numbers default to Off, so with no gutter
the ellipsis is the sole marker (Q#FD20, round-1 F3). A diagnostic
clamped onto the head wins that cell by paint order.
- A diagnostic on a hidden line clamps its SIGN to the outermost
visible head (most-severe merge); the squiggle needs a real row, so
only the sign clamps (Q#FD15).
- Caret, local selection endpoints, and peer cursors project via
`visible_position_of` — the head row AND the head's end-of-content
column, never an arbitrary column (round-2 F3). Peer presence derives
the RECIPIENT window's map.
- Style/search/completion overlays route through
`Viewport::row_offset_of`; the mode-line indicator reckons in
visible-line space.
Command/event time is scoped per frontend (Q#FD21): a
`fold_projection` flag on `FrontendView`, set at attach from the
negotiated `semantic_render` bit (grid ⇒ true, semantic ⇒ false until
Stage 3, LOCAL ⇒ true) and never inferred from a `FrontendId` (Bet
B8). Without it, shared `EditorCore` motion would make a simultaneous
unfolded GPU session's cursor skip lines it still displays. The map's
two axes stay separate (round-3 F1): the acting frontend supplies the
policy, the operation's TARGET window supplies the buffer — a wheel
event names a pane without activating it.
Motion (Q#FD17, ruled: include), paging, wheel, the click inverse, and
the auto-scroll clamp all step by visible lines under that gate;
motion from a hidden logical cursor normalizes to the visible head
first. `view_top` stays a source-line index (Bet B5), set only via
`clamp_view_top` so it never rests hidden.
Unfold widening (Q#FD19): the pre-edit unfold moves to the top of
`apply_active_edit` — one funnel that subsumes the six primitives'
calls and covers yank + query-replace, both of which place point at
the edit site first. Interactive Lua mutators hook the common
`run_buffer_edit`, above the managed/bypass split, gated on
`InteractiveCommandOrigin` AND the edit targeting that frontend's
active-window buffer. The remote/optimistic-CRDT path stays excluded
(Stage 3); undo/redo unfold stays deferred.
Acceptance: `tests/folding_stage2_acceptance.rs`, 35 tests asserting
on the real `paint_frame` cell grid, covering framing items 1–14
including crossing folds, a nested deeply-hidden cursor, a split of
two different buffers with an inactive-pane wheel, and simultaneous
grid+semantic motion.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q5BkezMppbpCgGAYk2ftxV
Review found that adding ("attribute", fg(3)) for HTML/CSS also colours the
@attribute capture three already-bundled grammars emit — rust (attribute_item),
lua (<const>), yaml (directives) — which were previously unpainted. Verified on
a Rust buffer: #[derive(Debug)] now paints uniformly yellow (fg 3), an
improvement over unpainted and the distinct-attribute convention most editors
follow.
Name this retro-paint as intended in the framing (Q#WEB4, rev 4) and pin it with
rust_attribute_repaints_via_shared_attribute_capture so it is a chosen effect,
not incidental. @tag is unaffected (HTML/CSS only).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Register `html` (.html/.htm/.xhtml) and `css` (.css) entries in
BUILTIN_LANGUAGES, backed by the official tree-sitter-html 0.23 and
tree-sitter-css 0.25 grammars over the tree-sitter-language shim (ABI-fine, no
overlay — both export their query constants). The single `extensions` field
wires detection ahead of the LSP filetype map.
HTML's crate-exported INJECTIONS_QUERY lights up <script> -> javascript
(already registered) and <style> -> css (registered here) via the #122
injection engine — the north-star injection consumer.
The only capture reconciliation (Q#WEB4): the two web captures both grammars'
queries use that pmacs did not recognize — ("tag", fg(5)) and
("attribute", fg(3)) — added to highlight.rs's table; @tag.error prefix-walks
to tag, and everything else already maps.
Tests: table guards; load-and-parse smokes (roots document/stylesheet);
highlights-resolve (node-name compat gate, asserts @tag present); extension
resolution; a tag+attribute paint test (the attribute assertion is
load-bearing); and the injection payoff — an HTML buffer with embedded
<style>/<script> paints a CSS property and a JS keyword INSIDE the injected
regions.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The shipped latex_highlights_resolve covered only the compile / capture-name
gate; the approved framing's acceptance 3 also requires parsing a document and
asserting painted output (the grid_paints_injected_child_keyword template).
tree-sitter compiles a pattern that uses a valid field on the wrong node and
then silently never matches, which the compile gate cannot catch — a real risk
given this grammar cut's structural drift from what the nvim query targets.
latex_grid_paints_section_and_command parses `\section{Intro}` + `\foo{bar}` and
asserts the \foo command_name paints @function (non-default) and the section
title text paints @keyword.control (bold), proving the reconciled query actually
matches, not merely compiles.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Share one fixed eight-column tab-stop contract across core and GPU renderers. Consolidate byte-to-display-column accounting, expand GPU code tabs with source provenance, align caret/hit/decoration geometry, and refresh minimap projection on edits.
Compile grammar locals metadata, resolve lexical definitions and references
once per settled layer, and apply local property predicates in both highlight
producers. Restore non-shadowed JavaScript builtins while suppressing local
shadows, with lexical, viewport, render, and edit-freshness regressions.
Add the strict pmacs.statusline provider registry, deterministic
borrow-released per-window evaluation, context-scoped failure latches,
and a pure built-in LSP provider.
Preserve the legacy TUI modeline while composing faced custom runs,
and append authoritative complete StatuslineSegments replacements for
semantic frontends. Expand dynamic ThemeFacts, reset producer/frontend
baselines symmetrically, and gate all provider work off protocol v18.
Teach the GPU to atomically validate, resolve, shape, clip, and cache
custom modeline runs without displacing the protected status suffix.
Document the public Lua lifecycle, wire ownership, snapshot semantics,
and the fully gated Arc 4 stage-3 delivery state.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Teach the syntax engine that one buffer can hold more than one
language. After the root parse, run the grammar's injections.scm, parse
each embedded region with the injected language, and merge every
layer's highlight spans. First consumer: markdown fenced code + inline
(zero new grammars — the block grammar already ships an injection query
and the injected langs already have grammars from #118).
Engine (src/syntax.rs):
- ParseTreeBundle now holds Vec<Layer> (root layer 0 + injected
children, depth-ascending); installed atomically so the existing
Arc::ptr_eq style gate and highlight cache keep working (Q#IJ1).
- run_parse builds layers on the worker: run injections.scm, resolve
the injected language, compute Vec<Range> (exclude NAMED children,
intersect the parent's ranges), set_included_ranges cold-parse,
recurse — bounded by depth (3), a layer backstop (4096), and a
(lang,ranges) visited guard; any child failure drops that child only
(Q#IJ3/IJ5). LanguageEntry gains injections_query; markdown_inline is
registered (retires the M9.7 block-only floor); markdown/rust carry
injection queries.
- Injected languages resolve off the static BUILTIN_LANGUAGES table
(Send loaders + query sources), preserving lazy loading. Dynamic
fence names go through a case-folded alias map seeded with defaults
and Lua-extensible via pmacs.parse.injection_aliases, snapshotted into
ParseRequest at dispatch so the worker never touches the Rc registry
or a Lua table (Q#IJ2/IJ4). Highlight queries are resolved at settle
(resolve_layer_queries), keeping query compilation main-thread/cached.
Producers:
- SyntaxHighlightView (grid) iterates layers shallow-to-deep so a
deeper layer's styling wins within its region (Q#IJ6/IJ7).
- scoped_style_spans (wire) flattens all layers into DISJOINT effective
spans via a boundary sweep, since the GPU re-sorts spans by start
(replace_style_spans / merge_style_spans) and would otherwise destroy
producer order. The GPU source_color_at consumer is fixed to fold all
covering spans (matching semantic_client's effective_style_at) rather
than returning the first.
Named-children exclusion: content ranges exclude only NAMED children
(matching tree-sitter-md's own inline splitter) — excluding a block
inline node's anonymous text tokens would shred the paragraph into
unparseable fragments.
13 acceptance gates (framing docs/multi-language-injections-framing.md):
layer structure, absolute child offsets, alias resolution (static +
case-folded dynamic + unknown-skip + Lua-async override), multi-range
inline, recursion bounds, wire + grid + GPU producers, incremental edit
/ new fence, many-paragraph settle budget with tail coverage, and the
single-layer regression guard.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
Arc 4 stage 1 (docs/theme-faces-framing.md, revision 4). Faces are
theme entries under the reserved ui/ui.* namespace -- zero new Lua
API. Theme::face() resolves with the dotted-prefix walk but never
falls back to default_style; each face applies owns-surface within
its stage-1 component mask, identical on both frontends.
Substrate: two monotonic theme mutation counters (syntax/face) with
transactional set/merge/clear/default (parse before locking, commit
all-or-nothing, bump from the prior value); the StyleGate and the
minimap summary key on the counters -- fixing the pre-existing bug
where a mid-session pmacs.theme.set never re-shipped StyleSpans --
with the summary gaining payload-equality suppression that still
advances its key on computation.
Wire: InstanceMessage::ThemeFacts appended after CompletionPopup
(postcard discriminants are ordinal; a byte pin guards placement),
PROTOCOL_VERSION 15 -> 16, daemon-gated >= 16, one authoritative
table per attachment (None-seeded baselines), TUI silent-drop arm.
Grid: paint_frame resolves ui.modeline / ui.statusline /
ui.minibuffer(.candidate) / ui.gutter / ui.selection faces;
SearchView and DiagnosticView take the theme handle through the real
attachment paths (EditorCore injection, install_diag threading); the
canonical severity color resolves ui.diag.* with the Default ->
built-in policy that keeps the minimap presence encoding sound.
GPU: exact-name face table applied per draw with the Q#TH5 Default
mapping (plain text / window bg, reverse swap), local/peer wash
split, candidate-dropdown glyph site, and the status-band
shaping-cache invalidation without which a diag-face recolor with
constant counts kept stale counter colors.
Tests: 18-test acceptance suite (grid, wire, daemon gate, atomicity,
monotonicity, late join), 7 GPU headless tests incl. decoded vertex
colors, units for the face walk / transactional commits / producer
caches; protocol pins for v16 + the CompletionPopup byte pin.
Bites vs 3cbb9de (scripts/bite): semantic_render.rs (8 runtime test
failures), editor.rs (5 runtime), daemon.rs (v15 gate, runtime);
lua_bindings/mod.rs, pmacs-gpu/main.rs, search.rs, diag.rs, and
highlight.rs bite as compile failures (weaker evidence, disclosed).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VoiEyuPjoBhvwACf8HAnLB
Sub-arc 2 of the UX arc, TUI half. When a window reserves a line-number
gutter, lines with diagnostics get a severity-colored sign glyph (E/W/I/H)
in the gutter's leading column — closing the last deferred Task #23 item.
No protocol/daemon change: the per-line severity is already frontend-side
(the diag store the DiagnosticView already reads).
- `Viewport` gains `gutter_w` so overlays can reach the gutter's leading
column at `cell_origin.col - gutter_w`; the text area is already shifted
past it, so viewport-relative painters stay gutter-agnostic.
- The gutter's number pass now runs *before* the overlays (was after), so
the DiagnosticView can draw its sign into the gutter's blanked leading
column without the number pass erasing it.
- DiagnosticView: with a gutter, draw the severity sign glyph colored by
`underline_color()`; without one, keep the legacy column-0 background
marker (the "fake gutter" that predates a real gutter column). Extracted
to `paint_line_markers` to keep `render` under the line cap.
The number never reaches column 0 (>=1 leading pad by construction), so
sign and number coexist. Diagnostic signs currently ride the line-number
gutter (visible when line numbers are on); a signs-without-numbers mode is
deferred.
Test: gutter_sign_replaces_the_column_marker_when_a_gutter_is_reserved.
Validated: fmt + clippy --all-targets clean both flavors; 1441 lib + 22
diag tests pass. Needs a TUI eyeball before the GPU half.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014TXbAwk27agwhrNNrhLi2U
Drops the policy-A exclusivity that left grammar-backed languages
without LSP semantic refinement. Adds tree-sitter-c (.c/.h) and
tree-sitter-cpp (.cpp/.cc/.cxx/.hpp/...) to the bundle so the grid
TUI gets lexical highlighting (keywords / strings / operators) on
first open. The Lua attach in builtin/runtime/lsp.lua now pushes
LspStyleView whenever an LSP server is up, regardless of grammar
presence; with both views attached the cell-painter pipeline runs
SyntaxHighlightView first (lexical) then LspStyleView (semantic)
and their styles compose through crate::overlay::merge_styles. The
result is the VSCode / Zed "TextMate + LSP semantic tokens" model
on a terminal grid: keywords colored by tree-sitter, identifiers
refined by clangd's semantic tokens.
`.h` is ambiguous C / C++; the `c` BUILTIN_LANGUAGES entry claims it
to match the LSP filetype map's default. Users who want `.h` parsed
as C++ can override via Lua (extension → language map).
Note the tree-sitter-c / -cpp crates expose `HIGHLIGHT_QUERY`
(singular), matching tree-sitter-md's `HIGHLIGHT_QUERY_BLOCK`
convention; tree-sitter-rust / -lua use `HIGHLIGHTS_QUERY` (plural).
Same bundled highlights.scm either way.
Regression guard: builtin_languages_include_c_and_cpp asserts the
language entries exist and claim their canonical extensions. The
LspStyleView module doc rewritten to reflect dual-authority
composition; the existing headline test's comment updated (the
test fixture still attaches only LspStyleView directly, so its
asserted cells reflect the LSP authority alone — Lua-level
attach_buffer is what exercises composition end-to-end).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The grid TUI's M_B1 LspStyleView shipped functional but the default
theme only covered tree-sitter capture names. clangd / rust-analyzer
/ gopls emit LSP-spec SemanticTokenTypes that didn't intersect — so
`LLAMA_LOG_ERROR`, namespace names, parameters, fields, etc. fell
through to default and rendered uncolored. This brings the default
theme up to the LSP vocabulary so what the LSP layer ships actually
paints.
Theme additions (Theme::default_dark): macro, namespace, parameter,
property, class, struct, enum, interface, enumMember, modifier,
decorator, regexp, typeParameter. Refactored default_dark to a
data-driven (name, style) table — cuts the function from ~170 lines
to ~70, and adding a new theme entry now means appending one row
rather than 6 lines of `by_capture.insert(...)`.
LspStyleView::render now builds the lookup name as `<type>.<first-
modifier>` when modifiers are set, else just `<type>`. Theme::lookup's
dotted-prefix walk falls back to base if a refined entry isn't
defined, so the change is a strict refinement: themes that want to
target e.g. `function.defaultLibrary` (clangd's standard-library
modifier) can, themes that don't see no behavior change. Allocation
is skipped in the no-modifier case via `Cow::Borrowed`.
Tests: default_dark_covers_lsp_token_types regression-guards the
LSP-vocabulary coverage; lsp_style_view_uses_modifier_in_capture_lookup
proves a modifier-refined theme entry wins over the base.
The bigger polish (real C++ keyword/string coloring) needs
tree-sitter-cpp + dropping policy A's exclusivity so grammar-backed
languages get both views — separate thread, intentionally out of
scope here.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the visible "C++ has no syntax coloring in the grid TUI" gap.
Sibling of SyntaxHighlightView: a View impl that paints LSP semantic
tokens as cell styles, attached for buffers with no bundled
tree-sitter grammar. Same policy A (one styling authority per buffer)
the semantic-frontend producer arc enforces, applied to the grid
renderer the user actually uses today.
Mechanics: every render re-derives the buffer's URI from
buf.file_path() and pulls (encoding, legend) via the existing
LspManager::semantic_style_context plus tokens via for_uri. Per
visible line, tokens are converted from LSP encoding units to byte
ranges via char_to_byte, then to display columns via the existing
byte_range_to_display_cols (UTF-8 + tab aware). Theme::lookup
resolves token type names through the same dotted-prefix mechanism
the tree-sitter capture names use, so "function", "variable",
"type", "keyword" land on the existing theme vocabulary with no new
style names. Default-styled spans skip the per-cell loop, matching
SyntaxHighlightView's short-circuit.
Wiring: pmacs.lsp._attach_style binding pushes the overlay on the
active window (mirrors pmacs.parse._attach_highlight). install_lsp
and make_lsp_manager take SharedSyntaxRegistry so the binding can
hand the LspStyleView the shared ThemeHandle; editor.rs caller
updated. builtin/runtime/lsp.lua's attach_buffer attaches the view
when pmacs.parse.language_for_path returns nil (grammar-less
signal), dedup'd via a styled_buffers set that mirrors syntax.lua's
highlighted_buffers.
Test: lsp_style_view_paints_cells_from_semantic_tokens — seeds an
Initialized fake LSP client (using the cfg(test) helper from the
producer arc) on a /tmp/x.cpp buffer with one token, asserts the
expected cells are styled per the theme face and the cell just past
the token range is unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Land the Model Context Protocol (MCP) integration as a transport binding,
not a built-in feature. Six Lua functions plus userdata methods expose
the substance of three MCP feature areas (resources, tools, prompts), a
notification dispatcher, and a non-trivial AI-assistance example package
that meets the architectural ship gate (spec/pmacs-spec.tex:1572): zero
direct calls into the Rust core, zero special-cased MCP handling outside
the public API, source under 2000 lines of Lua.
The M9.5 -> M9.6 -> M9.7 -> M9.8 layered composition validates the claim
"AI is a transport binding, not a feature" -- pmacs-mcp-ai composes with
pmacs-mcp-prompts.render and inherits notification handling transitively
through M9.7's package, demonstrating that the AI domain is a layer
above MCP, not a thread woven through the core.
Subtask shape:
M9.1 stdio transport + initialize handshake + restart policy
M9.2 resources with in-flight + settled cache and per-uri invalidation
M9.3 tools with isError-vs-JSON-RPC-error semantics + cancellation
M9.4 prompts with required-argument validation
M9.5 notification dispatcher (on_notification, off_notification)
M9.6 tools-as-commands fixture package + 12 audit findings disposed
M9.7 prompts-as-result-buffers fixture package + tree-sitter-md grammar
M9.8 AI-assistance fixture package (363+ LoC; 17/17 acceptance tests)
M9.9 formal package audit -- PASS on all three criteria
M9.10 release: TRANSITION-M9.md + MCP-for-package-authors guide
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Levi Neuwirth