Commit Graph

160 Commits

Author SHA1 Message Date
Levi Neuwirth 47ffe5dcff syntax: process bundled locals queries
Compile grammar locals metadata, resolve lexical definitions and references
once per settled layer, and apply local property predicates in both highlight
producers. Restore non-shadowed JavaScript builtins while suppressing local
shadows, with lexical, viewport, render, and edit-freshness regressions.
2026-07-22 12:28:31 -04:00
Levi Neuwirth b45e5ee5ec Merge canonical main into modeline detection
Integrate landed Vterm Stage 2 before the approved modeline merge. Preserve the
active modeline lane in the volatile ledger and record the full integrated gate
results.
2026-07-22 10:56:35 -04:00
Levi Neuwirth 3f0252fb97 Merge canonical main into vterm-tui
Integrate mode-system wiring and handoff updates before PR #130 lands.
Preserve per-frontend terminal dispatch while resolving major-mode keymaps,
and expose mode, terminal, and LSP statusline providers together.
2026-07-22 10:28:56 -04:00
Levi Neuwirth f8d05d2134 feat: detect language from modelines
Parse bounded Emacs and Vim modelines, normalize common aliases, and give
explicit file metadata precedence over inferred language. Pin one fresh-load
language decision for syntax, LSP, pairing, comments, and initial major mode,
while preserving the LSP path guard and explicit mode overrides.

Cover supported forms, rejection boundaries, precedence, unknown modes,
shebang and modeline pinning, reopen behavior, and pathless buffers.
2026-07-22 10:02:20 -04:00
Levi Neuwirth b9a7e40855 fix(vterm): harden view anchors and interactive authority
Clamp anchors into partially evicted wrapped lines, require authenticated
interactive origins, and avoid per-mouse cell snapshots. Restore dispatcher
rationale, named context errors, and focused regressions for the corrected
contracts.

Co-authored-by: OpenAI Codex <codex@openai.com>
2026-07-22 09:09:44 -04:00
Levi Neuwirth 1b4d022fd1 test: preserve mode segments with long paths
Widen the daemon acceptance grid so each split can show a macOS temporary
path and the following mode segment without protected-right clipping.
2026-07-22 08:28:57 -04:00
Levi Neuwirth 8702791de9 fix(vterm): honor terminal escape and view contracts
Require the fixed C-c escape before editor-local terminal bindings, reject context-implicit Lua operations from document windows, and make controller replacement atomic per frontend. Borrow screen rows during view projection instead of deep-cloning retained history, preserve view anchors through zero-area layouts, and remove redundant detach paths.

Add focused child-input coverage for unescaped bound keys and C-c C-c, plus controller, zero-area, context-error, and clone-free projection assertions.

Co-authored-by: OpenAI Codex <codex@openai.com>
2026-07-22 08:28:18 -04:00
Levi Neuwirth 4c382ae797 fix: harden mode acceptance startup
Give the daemon its normal five-second handshake window before switching
the mode-system acceptance client to short frame polling. Document reload
and session-persistence boundaries and correct stale describe-key guidance.
2026-07-22 08:18:18 -04:00
Levi Neuwirth da8f6aeae4 fix(vterm): harden integrated Stage 2 behavior
Resolve post-main integration drift in authenticated routing, terminal view projection, Lua installation, and inherited acceptance callers. Preserve the terminal statusline provider alongside the landed Themes provider and record the final Stage 2 gate evidence.

Co-authored-by: OpenAI Codex <codex@openai.com>
2026-07-21 21:38:08 -04:00
Levi Neuwirth 99cd7ec240 feat: wire major modes through key dispatch
Store a detected major mode on each buffer and expose it through Lua.
Resolve mode-scoped bindings in dispatch, describe-key, and help links,
including exact encoded mode context after entering the help buffer.

Initialize modes once at buffer load, preserve explicit overrides and
clears across switches, and publish the mode through a per-window
statusline provider. Add daemon acceptance for the complete mode lifecycle.
2026-07-21 20:25:48 -04:00
Levi Neuwirth 0ddff24589 Merge canonical main into vterm-tui
Integrate config-registry and handoff updates landed after the Stage 2
framing branch was cut.
2026-07-21 20:18:53 -04:00
Levi Neuwirth dc9225778a test(vterm): prove Stage 2 TUI integration
Add cross-surface Lua, shared-view, clipboard, authenticated routing,
BEL, resize, and real-host PTY acceptance. Ensure terminal-local keymaps
run before raw child transport and document the criterion-to-test map.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-21 20:16:23 -04:00
Levi Neuwirth f86c966090 fix(config): reject wrongly-typed spec fields; make trim-on-save buffer-aware
Review round 1, findings 2-4 plus doc notes. Finding 1 landed in fd80bcb.

Finding 3 --- spec fields meaningless for the declared type are now
rejected. DEFINE_SPEC_FIELDS whitelists all nine keys for every type and
the kind parser only reads its own arm's fields, so
`{ type = "string", choices = {...} }` silently defined a string that
accepts anything (the author meant enum) and `min` on a boolean was
dropped. These are typo-shaped bugs the R50 whitelist structurally
cannot see: the key is spelled correctly, it is on the wrong type.
`check_fields_relevant_to_kind` closes it with a pointed error naming
the misplaced field, and a companion test pins that each field is still
accepted where it belongs, including `min`/`max` on number as well as
integer.

Finding 4 --- the after_buffer_removed purge had no end-to-end test.
Every existing test called ConfigRegistry::remove_buffer directly, so
deleting the three lines wired into mod.rs would have left the whole
suite green. The new acceptance test kills a buffer through
pmacs.buffer.remove (the real remove_buffer_and_fire route) and asserts
the locals are gone; bite-verified by removing the hunk and watching it
fail.

Finding 2 (the half with a natural buffer) --- editing.trim-on-save is
now resolved against the buffer being saved rather than the global
chain. Reading globally meant set_local was accepted, stored, and
reported by describe, then never consulted: a pin the user believes in
that does nothing, which is the shape F1 exists to prevent. Two tests,
one for the override and one for the global fallback the change could
have broken; the override test fails against the old global read.

Both new save tests initially passed VACUOUSLY and were rewritten:
pmacs.editor.save() is the raw save, while buffer.before-save fires
inside the buffer.save COMMAND (default.lua:224), and save() no-ops on
an unmodified buffer --- so the original form asserted on a file that
was never rewritten. They now insert content to dirty the buffer and go
through pmacs.command.invoke("buffer.save").

The other half of finding 2 --- a per-buffer autosave.interval-ms is
semantically meaningless yet still accepted --- is recorded as a named
deferral proposing a define-time `scope = "global"` flag, alongside
deferrals for bound-parse field naming and StartupOnly reset symmetry.
Also recorded: interval_ms(1e30) now raises instead of storing a
nonsense float, an improvement but a real divergence from "the wrapper's
shape stays exactly as it was".

Doc: the module header cited framing revision 2; the shipped doc is
revision 3, whose corrections are what the code implements.

Gates: fmt, clippy -D warnings, --lib (1691), --lib --features crdt
(1865), lua54 backend, config_registry_acceptance (16), editops (72),
autosave (29), PMACS_REQUIRE_GPU=1 pmacs-gpu (109), and the full
workspace sweep (2806 tests, exit 0). git diff --check clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 18:29:18 -04:00
Levi Neuwirth 6844262495 feat(config): typed configuration registry with buffer-local scope
A third registry beside CommandRegistry and HookRegistry, per
docs/config-registry-framing.md. Unblocks the per-buffer auto-pair
toggle, the first of the five backlog items the missing config surface
was gating.

Substrate (src/config_registry.rs):

  * ConfigRegistry keyed by name with definition order preserved, R42
    mandatory descriptions, R50 typo detection, duplicate rejection,
    and SourceLocation provenance -- the command/hook vocabulary.
  * Closed scalar kinds: boolean, integer, number, string, enum. Owned
    Rust values; Lua tables, functions and userdata are never stored.
    Integer exactness is checked by value, never math.type, so the
    luajit and lua54 builds agree.
  * Two scopes. get(name, buf) resolves buffer-local -> global ->
    default; get(name) with no buffer resolves the global chain only
    and never consults an ambient buffer. Buffer-locals live in a
    registry-owned side table purged at after_buffer_removed, beside
    the keymap purge already there.
  * An override is ALWAYS stored, even when equal to the value it
    shadows; only value_epoch and listener dispatch key on effective
    change. Without this a buffer pinned to the current value stores
    nothing and a later global set flips it -- the pin silently never
    existed. equal_valued_local_override_is_still_stored_and_shields_buffer
    fails against the naive reading.

Bindings (src/lua_bindings/config.rs):

  * define/get/set/set_local/reset/is_set/describe/list/on_change.
    Spec tables are read raw, so neither an unknown key nor a
    metatable-provided value can smuggle a field in.
  * Listeners commit inside the borrow, snapshot, drop the borrow, and
    only then re-enter Lua -- verified by holding the borrow and
    watching the test panic with "RefCell already borrowed". A raising
    listener is logged without blocking later ones or rolling back, and
    a depth bound turns an accidental cycle into a pointed error.
    Listeners persist until explicitly disposed; there is no Gc path,
    matching the rest of the codebase.
  * StartupOnly freezes off the existing InitCompleteFlag at write
    time, so this arc adds no editor.rs call at all.

Adopters, each defining its own key so SourceLocation names the owning
module: editing.auto-pair (pair.lua, read per-buffer against the typed
edit's SOURCE buffer), editing.trim-on-save (editops.lua),
autosave.interval-ms (autosave.lua). No public function is removed or
deprecated, and both migration wrappers keep their legacy coercion --
trim_on_save("yes") still enables, interval_ms(1500.7) still floors to
1500 -- coercing before handing the strict registry a conforming value.

M-x describe-setting renders into *help*, modeled on describe-command.

Framing revision 3 records four defects implementation found in the
document itself: acceptance 30 and 31 contradicted each other; the
planned builtin/runtime/config.lua had nothing to hold and would have
broken the source-location contract had it held the one helper it might
have; F5 asked define to police a call it cannot see, moved to
set_local; and list() ordering was underspecified.

No protocol change; SUPPORTED stays [6..18]. No wire surface. Zero
changes to src/editor.rs.

Gates: fmt, clippy -D warnings, --lib (1683), --lib --features crdt
(1857), the new config_registry_acceptance (13) plus auto_pair (45),
editops (72), autosave (29) and m9_6 (25), m4 --skip basedpyright
(114), PMACS_REQUIRE_GPU=1 pmacs-gpu (109), the lua54 backend build,
and the full workspace sweep (2795 tests, exit 0). git diff --check
clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 18:29:18 -04:00
Levi Neuwirth 9797adaa0b fix(vterm): harden terminal cell and input invariants
Reject C0/C1 controls before terminal text reaches screen cells, and preserve
the released button code in SGR mouse reports.

Remove dead screen branches, keep logical-line allocation saturating, and clear
round-trip input state when pruning externally removed terminal buffers.
2026-07-21 16:27:43 -04:00
Levi Neuwirth f0a235f635 fix(vterm): harden Stage 1 terminal contracts
Add typed IND, NEL, and RI operations with exact screen semantics, preserve
application tab stops across resize, and default terminal children to the
supported xterm-256color capability set.

Make shutdown liveness acceptance portable with kill(pid, 0), and document the
public TerminalScreen methods consumed by later stages.
2026-07-21 15:30:06 -04:00
Levi Neuwirth bbc1f33a7c feat(vterm): add Stage 1 terminal core
Add compatibility-preserving full-screen ANSI operations, the bounded terminal
screen and input encoders, and a transactional TerminalManager owning one
read-only identity buffer, PTY process, and screen per session.

Drain terminal-owned process events before process.after-tick, retain exact
final output and PID/outcome annotations, reap killed buffers and shutdown
children safely, and enforce buffer-owned read-only checks across ordinary,
host, undo/redo, and CRDT mutation paths.

Cover split parser and grapheme boundaries, screen/reflow/history invariants,
device responses, lifecycle cleanup, and a real adversarial alternate-screen
PTY. Record the fully gated Stage 1 delivery and downstream TUI/GPU contracts.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-07-21 14:20:28 -04:00
Levi Neuwirth 4b65b9e1e5 feat(statusline): add composable modeline segments at protocol v18
Add the strict pmacs.statusline provider registry, deterministic
borrow-released per-window evaluation, context-scoped failure latches,
and a pure built-in LSP provider.

Preserve the legacy TUI modeline while composing faced custom runs,
and append authoritative complete StatuslineSegments replacements for
semantic frontends. Expand dynamic ThemeFacts, reset producer/frontend
baselines symmetrically, and gate all provider work off protocol v18.

Teach the GPU to atomically validate, resolve, shape, clip, and cache
custom modeline runs without displacing the protected status suffix.
Document the public Lua lifecycle, wire ownership, snapshot semantics,
and the fully gated Arc 4 stage-3 delivery state.

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-07-21 12:01:25 -04:00
Levi Neuwirth ffcb903fc1 docs(json-yaml): refresh final review state
Remove stale transfer-task wording and record the rebased, fully gated
JSON/YAML provider validation in the framing and handoff.
2026-07-21 09:34:54 -04:00
Levi Neuwirth 5c202c54c1 test(json-yaml): verify real YAML provider through pmacs
Drive Red Hat yaml-language-server 1.24.0 through the default YAML
auto-attach path. Disable SchemaStore and the Kubernetes CRD catalog for
network-free determinism, require language-specific initialization and a
real syntax diagnostic, and prove the server remains alive afterward.

Update the framing and runtime commentary with the completed live-provider
evidence. The test passes against the pinned provider and fails against the
pre-JSON/YAML runtime under scripts/bite.

Co-Authored-By: OpenAI Codex <noreply@openai.com>
2026-07-20 17:00:20 -04:00
Levi Neuwirth 19ad5cc8ac fix(json-yaml): checkpoint reviewed LSP configuration fixes
Preserve PR #123's unpushed review fixes on a transfer branch: initial
didChangeConfiguration delivery, explicit JSON validation, the pinned
JSON server provider, corrected YAML configuration sections, and
deterministic plus real-provider acceptance coverage. Record the
observed yaml-language-server 1.24.0 standalone smoke and leave the
real YAML-through-pmacs test, rebase, and full gates explicitly pending
for the destination machine.
2026-07-20 16:49:37 -04:00
Levi Neuwirth 9ce6f1abf3 feat(json-yaml): JSON + YAML grammars and language servers
Add tree-sitter-json (0.24) and tree-sitter-yaml (0.7) to
BUILTIN_LANGUAGES (both ABI-current via tree-sitter-language, verified
compiling under tree-sitter 0.26), each self-contained highlights, no
injections of their own. Extensions json=.json, yaml=.yaml/.yml; root
kinds json `document`, yaml `stream`.

The payoff from the #122 injection engine is free: the markdown block
injection query already sets injection.language "yaml" for `---`
frontmatter (minus_metadata) and "toml" for `+++` (plus_metadata), so
registering yaml lights up YAML frontmatter highlighting with no extra
wiring, and ```json / ```yaml / ```yml fences resolve through the engine
(yml->yaml alias already present). Two acceptance tests pin this synergy.

LSP (builtin/runtime/lsp.lua): pmacs.lsp.config.json uses the maintained
extracted-bundle binary `vscode-json-language-server --stdio` (NOT the
stale standalone vscode-json-languageserver); MIT, no telemetry, remote
$schema fetch left enabled (no handledSchemaProtocols). pmacs.lsp.config
.yaml uses `yaml-language-server --stdio` with Red Hat telemetry
disabled by default. Both ship the exact workspace/configuration sections
each server pulls (json+http; yaml+http+redhat.telemetry) present-not-null
so the servers get defaults rather than erroring — the CMake #117 lesson.
Sections derived from server source/docs (neither binary installed on
this build machine to observe live; verify where present). Filetype
fallback entries added. JSON is the standing prerequisite for the Jupyter
.ipynb arc; handoff §6 updated.

Nine acceptance tests (grammar ABI, highlights compile, detection,
grammar<->LSP-key alignment, the two frontmatter/fence synergy proofs,
and the pinned LSP-config sections).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
2026-07-20 16:49:37 -04:00
Levi Neuwirth 6cd78f0870 test(font): wire acceptance suite + protocol/design docs (items 2-8, 15)
tests/gpu_font_acceptance.rs (items 2-7; the header manifest routes
item 1 to src/protocol.rs pins, item 8 to the src/frontend.rs TUI
drop unit, items 9-14/16-19 to pmacs-gpu's headless suite, and item
15 to the docs):

- 2: a fresh attachment's first frame carries the REAL (None, None)
     default; unchanged ticks are silent; a late joiner receives the
     current preference without post-attach mutation
- 3: a mid-session set_font emits exactly one FontFacts on the next
     frame; an identical re-set advances the epoch without emitting
- 4 (producer half): on_buffer_snapshot_sent re-ships buffer facts
     but never the bufferless FontFacts
- 5: real-daemon probe -- a v17 semantic session receives FontFacts,
     a v16 peer never does (crdt feature)
- 6: the strict Lua contract -- 5.999/72.01/0/-16/NaN/inf/non-number
     sizes error naming `size`; empty/non-string family errors
     naming `family`; unknown keys rejected by name; hostile
     __index/__pairs metatables never invoked and never inject
     values; quantization pins 15.994->1599 / 15.996->1600 and both
     boundaries; set_font {} resets both axes; the getter returns a
     fresh quantized plain table; every rejected shape leaves the
     preference and the wire untouched
- 7: a load_user_config_at fixture's init.lua set_font lands in the
     handle installed before user config, and a pre-attach
     preference ships on the first frame

Bites (mutate, observe the acceptance test fail, restore): the
Option-seeded first-frame send, the payload compare, the snapshot
survival of the font baselines, the v17 gate (BOTH halves widened --
producer for_peer alone leaks nothing because the daemon skip arm
still filters; the wire test only fails when belt AND braces are
cut), the unknown-key rejection, the range-check-original ordering,
and the raw_get metatable isolation.

Docs (item 15): docs/semantic-frontend-protocol.md gains the
FontFacts variant entry (v17 gate, authoritative default, no-pixels
preference relay, fail-closed receiver) and folds FontFacts into the
BufferSnapshot reset-contract paragraph (bufferless facts survive;
the frontend's caret-follow scroll residual is the buffer-scoped
part). docs/pmacs-gpu-design.md's "future customization needs no
wire-protocol changes" claim is corrected in place and points at the
framing as design of record; the Lua-override bullet now names the
landed set_font shape. The framing status bumps to implemented.

Also re-homes the ProcessSupervisor doc comment that the make_font_pref
insertion had orphaned onto the wrong function.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VoiEyuPjoBhvwACf8HAnLB
2026-07-18 11:46:44 +01:00
Levi Neuwirth bc0922bea2 test(m4): keep injection work outside root parse gate 2026-07-15 14:50:45 +01:00
Levi Neuwirth 325dcd553f fix(injections): PR #122 round 2 — sibling precedence, observable cap, docs
Two follow-ups + doc cleanup.

[P2] Same-depth sibling precedence was reversed. The wire priority was
(depth, capture_order), omitting the layer ordinal, so two overlapping
spans from different sibling layers tied — and the active-set insert
then applied the later one first, making the earlier sibling win, the
opposite of the grid's layer-by-layer paint. Priority is now
(layer_index, capture_order): layer_index is the depth-ascending
position in bundle.layers, so a deeper layer AND a later same-depth
sibling both override, matching the grid exactly. New
flatten_same_depth_sibling_later_layer_wins pins it.

[P2] Cap surfacing had no end-to-end test. Added
injection_cap_surfaced_once_and_rearms_via_lua, which drives the real
Lua settle path (syntax.lua tick -> _injection_capped -> pmacs.error)
and asserts surfaced-once, suppressed-on-unchanged-reparse, and
re-armed-after-dropping-below-then-exceeding-the-cap.

Docs:
- Q#IJ6 now states the accurate bound O(n log n + Sum active) for the
  event sweep, not O(boundaries).
- The full-buffer perf test is renamed/narrowed to guard the FLATTENER
  regression; the summary's per-line dominant-style tally is a separate
  pre-existing O(lines x spans) loop, not claimed linear.
- Framing #9 now matches the test: it drives spans_from_segments (the
  extracted replace_style_spans transform) + source_color_at, not a live
  State render.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
2026-07-15 12:07:21 +01:00
Levi Neuwirth 79d75a29e0 fix(injections): PR #122 round 1 — sweep flatten, sync aliases, real multi-range, surfaced cap
Four review findings + a cleanup bundle.

[P1] Wire flattener was O(spans²) and ran over the WHOLE buffer (the
file-style summary uses a whole-buffer viewport, not the visible one).
Replaced the per-interval full scan with an ordered active-set event
sweep (activate on start, expire on end, fold the active set) — linear
in practice. Added full_buffer_summary_scales_on_large_grammar_file
(1500-line rust) as the perf gate.

[P2] _parse_now used the empty alias map from make_request while
_dispatch snapshotted the registry map, so a `py` fence injected async
but not sync. Snapshot aliases on both paths; pinned by
sync_parse_now_resolves_alias.

[P2] The multi-range inline test used a one-line paragraph, whose block
inline node has no named children (link/emphasis are child-grammar
structures) — one range, so it couldn't falsify multi-range. Replaced
with a multi-line blockquote whose inline node carries a named
block_continuation: content_node_ranges now asserts >1 collected range
and emphasis parses on both lines.

[P2] The layer backstop dropped regions silently; the framing requires
a surfaced warning. run_parse now sets ParseTreeBundle::injection_capped;
syntax.lua's settle tick raises it once per buffer via pmacs.error
(_injection_capped). Added injection_layer_cap_surfaces_and_preserves_root
(drives >4096 fences, asserts the flag + bounded count + intact root).

Cleanup:
- The GPU acceptance test now drives the real StyleSpans full-frame
  transform (spans_from_segments, extracted from replace_style_spans)
  instead of a hand-rolled sort.
- content_node_ranges excludes NAMED children (documented as a round-1
  refinement); framing mechanic #3 / Q#IJ5 updated to match.
- parse_duration doc now says root parse; the markdown entry no longer
  describes inline as unhighlighted/future.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
2026-07-15 12:07:21 +01:00
Levi Neuwirth 4282b1c333 feat(injections): multi-language injection layers
Teach the syntax engine that one buffer can hold more than one
language. After the root parse, run the grammar's injections.scm, parse
each embedded region with the injected language, and merge every
layer's highlight spans. First consumer: markdown fenced code + inline
(zero new grammars — the block grammar already ships an injection query
and the injected langs already have grammars from #118).

Engine (src/syntax.rs):
- ParseTreeBundle now holds Vec<Layer> (root layer 0 + injected
  children, depth-ascending); installed atomically so the existing
  Arc::ptr_eq style gate and highlight cache keep working (Q#IJ1).
- run_parse builds layers on the worker: run injections.scm, resolve
  the injected language, compute Vec<Range> (exclude NAMED children,
  intersect the parent's ranges), set_included_ranges cold-parse,
  recurse — bounded by depth (3), a layer backstop (4096), and a
  (lang,ranges) visited guard; any child failure drops that child only
  (Q#IJ3/IJ5). LanguageEntry gains injections_query; markdown_inline is
  registered (retires the M9.7 block-only floor); markdown/rust carry
  injection queries.
- Injected languages resolve off the static BUILTIN_LANGUAGES table
  (Send loaders + query sources), preserving lazy loading. Dynamic
  fence names go through a case-folded alias map seeded with defaults
  and Lua-extensible via pmacs.parse.injection_aliases, snapshotted into
  ParseRequest at dispatch so the worker never touches the Rc registry
  or a Lua table (Q#IJ2/IJ4). Highlight queries are resolved at settle
  (resolve_layer_queries), keeping query compilation main-thread/cached.

Producers:
- SyntaxHighlightView (grid) iterates layers shallow-to-deep so a
  deeper layer's styling wins within its region (Q#IJ6/IJ7).
- scoped_style_spans (wire) flattens all layers into DISJOINT effective
  spans via a boundary sweep, since the GPU re-sorts spans by start
  (replace_style_spans / merge_style_spans) and would otherwise destroy
  producer order. The GPU source_color_at consumer is fixed to fold all
  covering spans (matching semantic_client's effective_style_at) rather
  than returning the first.

Named-children exclusion: content ranges exclude only NAMED children
(matching tree-sitter-md's own inline splitter) — excluding a block
inline node's anonymous text tokens would shred the paragraph into
unparseable fragments.

13 acceptance gates (framing docs/multi-language-injections-framing.md):
layer structure, absolute child offsets, alias resolution (static +
case-folded dynamic + unknown-skip + Lua-async override), multi-range
inline, recursion bounds, wire + grid + GPU producers, incremental edit
/ new fence, many-paragraph settle budget with tail coverage, and the
single-layer regression guard.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
2026-07-15 12:07:21 +01:00
Levi Neuwirth 8ce2e9cb0f
Merge pull request #120 from levineuwirth/theme-faces
feat(themes): named UI faces + ThemeFacts channel (protocol v16) — Arc 4 stage 1
2026-07-15 10:30:31 +00:00
Levi Neuwirth b75d45b1d4 perf(themes): PR #120 round 5 -- O(1) frozen counts via store totals
The round-4 freeze read counted the retained for_uri vector inside
status_facts_msg -- correct, but StatusFacts runs at frame cadence
for every semantic session under the shared store mutex, so a long
stale interval cost O(frames x diagnostics x sessions).
DiagnosticStore now maintains per-URI severity totals alongside
by_uri: set replaces them (an empty publication removes them with
the vector), clear removes them, and mark_stale deliberately
preserves both -- entries exist exactly when by_uri entries do, and
set/clear are the store's only by_uri mutators. status_facts_msg
reads the tuple in O(1), and the all-URI severity_totals sum reuses
the cached tuples.

Store unit (acceptance item 34) pins the invariant: all four totals
replace correctly, survive staleness, and clear with the diagnostic
vector; empty_set_clears_uri asserts the totals drop too. The
rounds 3-4 freeze acceptance passes unchanged -- behavior parity,
so the unit pin is the evidence (no runtime bite exists for a
behavior-preserving refactor). Round-5 implementation
user-authored; this commit folds it with framing revision 9, the
protocol doc's O(1) note, and the acceptance manifest pointer to
the item-34 unit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VoiEyuPjoBhvwACf8HAnLB
2026-07-15 11:19:03 +01:00
Levi Neuwirth 3d6336a912 fix(themes): PR #120 round 4 -- freeze sourced from the store
A session first rendering during diagnostic staleness emitted zero
counts: round 3's frozen_diag_counts cache was per-session and only
seeded from fresh computations, so a late joiner attaching mid-edit
(or a buffer first visited between didChange and fresh diagnostics)
had no entry and fell back to (0, 0), contradicting the documented
"frozen counts, never zeros" contract.

mark_stale (T M11.8) keeps the last published diagnostic vector --
only the positions are invalid -- so status_facts_msg now counts
the retained for_uri entries even while stale: the retained entries
ARE the frozen value. The per-session cache is deleted; sourcing
the freeze from the store means there is no session state to lose
to a snapshot reset and no history needed at attach, so the round-3
reset-survival property holds by construction and its round-trip
test passes unchanged against the new mechanism.

Acceptance item 33 marks a populated store stale BEFORE the
SemanticRenderState exists and asserts the first frame reports the
preserved counts; runtime bite vs pre-fix semantic_render.rs.
Framing revision 8; the protocol doc's freeze sentence now says
store knowledge, including the late-joiner case.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VoiEyuPjoBhvwACf8HAnLB
2026-07-15 11:07:19 +01:00
Levi Neuwirth 2fe6738d68 fix(themes): PR #120 round 3 -- GPU snapshot symmetry, count freeze
Finding 1: the round-2 reset contract was asymmetric. The producer
resets search/menu/status baselines on every snapshot send, but the
GPU's BufferSnapshot arm only cleared spans, decorations,
adornments, summary, and the completion popup -- a menu or search
open at switch time survived the snapshot with no close message
ever coming (the new buffer's first CLOSED state is suppressed
daemon-side), leaving a stale popup that also held
daemon_intercepts_keys true and swallowed pointer events
indefinitely. The arm now clears search_prompt, menu, and
status_facts; the minibuffer is deliberately exempt on both sides
(one global core instance, matching the producer's surviving
last_minibuffer baseline). GPU test opens search + menu + status
via the real wire arms, applies a snapshot, and asserts all three
clear, the intercept gate releases, and the popup pixels vanish --
hand-bitten by disabling the three clears (fix and test share
main.rs).

Finding 2: the round-2 reset broke the diagnostic-count freeze.
last_status was both the peer emission baseline and the
stale-store freeze source, so a snapshot between didChange and
fresh diagnostics re-shipped StatusFacts with zeroed counts. The
freeze source now lives apart: frozen_diag_counts advances on every
fresh count, is read when the store is stale, and survives
on_buffer_snapshot_sent -- which keeps killing the emission
baseline to force the re-send. Acceptance renders (1,1), marks the
store stale, applies the reset, and asserts the re-sent StatusFacts
still carries (1,1); runtime bite vs pre-fix semantic_render.rs
fails exactly as predicted (Some((0,0)) vs Some((1,1))).

Framing revision 7; acceptance items 31-32; the protocol doc's
snapshot-reset paragraph now lists the full frontend drop set and
names the count freeze as daemon knowledge, not peer state.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VoiEyuPjoBhvwACf8HAnLB
2026-07-15 10:49:10 +01:00
Levi Neuwirth d91ff1a9e2 fix(themes): PR #120 round 2 -- snapshot/baseline reset contract
Finding 1: a BufferSnapshot wipes the frontend's buffer-scoped
render state (spans, decorations, adornments, minimap summary,
completion popup), but the producer's per-buffer emission baselines
survived the switch -- on an unchanged A -> B -> A round trip,
last_summary[A]'s key still matched and the daemon emitted nothing,
so the frontend never regained A's themed minimap (or A's
StatusFacts: the band kept B's name) until an edit, republish, or
theme mutation happened to move the key.

The fix is the general contract, not a minimap special case:
SemanticRenderState::on_buffer_snapshot_sent(buffer_id) kills every
buffer-scoped baseline for that buffer (spans + style gate,
decorations, adornments, summary, status, search/menu prompts,
completion popup), called wherever the daemon writes a snapshot --
the active-buffer-follow path and the F29 upgrade broadcast; the
attach bootstrap constructs its session state fresh. Deliberately
surviving: the bufferless ThemeFacts pair, the global minibuffer
baseline, the per-frontend gutter mode, the revision-keyed diag
line cache, and other buffers' baselines.

Evidence: a producer round-trip acceptance test (themed summary and
StatusFacts return at the SAME generation; identical payload), a
real-daemon wire test driving A -> B -> A via dispatched keys
(runtime bite: times out against pre-fix daemon.rs), a Rust unit
pinning the reset's scope, and a GPU test where the re-shipped
summary restores the first visit's pixels exactly (frontend half --
no GPU code change, coverage only). The semantic_render.rs bite is
compile-fail (the hook is absent pre-fix), disclosed as weaker.
The protocol doc's composition section now states the snapshot
reset contract on both sides of the wire.

Finding 2: the acceptance-suite manifest header now lists the true
item split (1-19, 24-26, 28-29 here; 20-23, 27, 30 in the GPU
suite). Framing revision 6 folds the round; acceptance items 28-30.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VoiEyuPjoBhvwACf8HAnLB
2026-07-15 10:13:20 +01:00
Levi Neuwirth 3083458cb0 fix(themes): PR #120 round 1 -- minimap cache, __index holes, v15 face leak
Finding 1: accepting a FileStyleSummary drops the GPU minimap vertex
cache -- theme recolors and diagnostic republishes arrive at an
unchanged generation, and the cache keys only on (generation, dims,
scroll), so stale strokes survived until an edit/resize/scroll. The
daemon payload-suppresses identical summaries, so the invalidation
is precise. GPU test drives two same-generation summaries;
hand-bitten by reverting the single invalidation line (script-bite
is vacuous here: fix and test share main.rs).

Finding 2: lua_to_style propagates every Table::get error -- the
lookups run __index, so a raising metatable previously parsed as an
all-default style and the merge SUCCEEDED, committing valid siblings
against the Q#TH6 all-or-nothing contract. Boolean fields keep Lua
truthiness by design (mlua bool), so only raising lookups fail the
transaction. Acceptance reproduces the reviewer's trap shape;
runtime bite vs pre-fix mod.rs.

Finding 3: SemanticRenderState::for_peer records the negotiated
version; below v16 no ThemeFacts is produced and no ui.diag.* face
folds into the FileStyleSummary marks -- the summary is an ungated
pre-v16 channel, and a v15 peer must not get face-derived minimap
colors while its other severity surfaces stay unthemed. The summary
cache key zeroes its face-epoch component for such peers. Acceptance
drives v15/v16 producers side by side; compile-fail bite disclosed
(the test needs for_peer, absent pre-fix).

Finding 4: framing revision 5 weakens the canonical-severity claim
to what is true -- the daemon-RESOLVED color is canonical, while the
GPU's built-in squiggle/sign/counter defaults are historical bright
RGBs that differ from the minimap's converted Indexed marks, a
pre-existing divergence kept because unset faces must render
byte-identically to before.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VoiEyuPjoBhvwACf8HAnLB
2026-07-14 19:36:31 +01:00
Levi Neuwirth 665fd82860 fix(highlight): PR #118 round 1 — drop locals-predicate captures; stale comments
[P2] The shared JavaScript highlights query guards its builtin captures
(console, require, …) with `#is-not? local`, a PROPERTY predicate
(`Query::property_predicates`) that needs a scope map from the grammar's
LOCALS_QUERY — which pmacs does not run. `compute_highlight_spans` took
every capture, so a locally-shadowed `console`/`require` still surfaced
as `@variable.builtin`/`@function.builtin`; a theme distinguishing
`.builtin` would mis-style the shadowed local.

Full locals processing is substrate work; conservatively fail-closed
instead: drop captures whose pattern carries an `#is?`/`#is-not? local`
property predicate (the identifier falls back to its non-builtin
capture). The text predicates (`#eq?`/`#match?`/`#any-of?`, already
applied by the capture iterator) and `#set!` settings are untouched.
This is a general engine fix — it corrects the same latent mis-styling
for any grammar using the locals predicate, not just JS/TS.

- javascript_shadowed_builtin_is_not_mislabeled: a local `const console`
  produces no `*.builtin` capture (directly observed to fail — two
  `variable.builtin` captures — before the fix).

[P3] Comments this PR invalidated: `lsp.lua` no longer claims Python has
no grammar; `syntax.lua`'s `_has_language` gate comment uses a
still-grammarless example (an init.lua `shebangs.ruby`) instead of
python/javascript; and the rewritten Ruby shebang test's doc no longer
describes it as a Python test.

Gates: fmt; clippy -D warnings; test --lib; --features crdt;
m4_acceptance --skip basedpyright; GPU; full workspace sweep;
git diff --check.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
2026-07-14 17:45:09 +01:00
Levi Neuwirth bcec61e020 feat(highlight): grammars for python/go/typescript/javascript/toml/zig
These five languages already had LSP configs (basedpyright, gopls,
tsserver, taplo, zls) but shipped no tree-sitter grammar, so they
rendered with no lexical color. Fill the gap — 8 BUILTIN_LANGUAGES
entries across 6 crates:

- python (`tree-sitter-python`, root `module`), go (`tree-sitter-go`),
  toml (`tree-sitter-toml-ng`), zig (`tree-sitter-zig`, +`.zon`) — each a
  single self-contained highlights query.
- JavaScript/TypeScript family: `tree-sitter-javascript` parses both
  `.js` and `.jsx`; `tree-sitter-typescript` ships two grammars
  (LANGUAGE_TYPESCRIPT, LANGUAGE_TSX). The four entries — javascript,
  javascriptreact, typescript, typescriptreact — mirror the LSP filetype
  map so tsserver enables the JSX parser. Highlights inherit: the TS
  query is a ~5-capture delta over JavaScript and JSX is a further
  delta, so the entries compose base-first (js → jsx → ts), the same
  pattern as `cuda` over C/C++ (typescript resolves ~22 capture classes,
  typescriptreact ~24).

Each grammar's name equals its existing `pmacs.lsp.config.<name>` key,
so grammar detection (which wins over the filetype map) resolves the id
the server keys off — the file now gets BOTH highlighting and the right
server. No lsp.lua change needed. All crates ride `tree-sitter-language
0.1` with tree-sitter dev-only — no second core in the graph.

Bite-verified acceptance:
- gap_grammars_load_and_parse — each grammar's ABI accepted by the 0.26
  core; a snippet parses without error at its root (covers both TS
  grammars, incl. JSX).
- typescript_highlights_compose_the_javascript_base — the compiled
  typescript/typescriptreact queries resolve >= 15 captures, not just the
  ~5-capture TS delta (the JS base is really composed in).
- builtin_languages_include_gap_grammars /
  gap_grammar_extensions_resolve — entry presence + extension detection
  across all 8 ids.
- m4_gap_grammars_align_with_lsp_configs — through the loaded runtime,
  each path's grammar id matches an existing LSP config. Bite-verified
  against pre-feature src/syntax.rs.

Ripple: two #116 shebang tests used python as their "has-LSP-but-no-
grammar" example, which this PR invalidates. Updated both — the .py +
`#!/bin/sh` precedence test now asserts a python grammar tree (not "no
tree"), and the grammarless-language-is-silent gate test switches to
`ruby` (genuinely grammarless) via a test-local shebang mapping.

Gates: fmt; clippy -D warnings; test --lib; --features crdt;
m4_acceptance --skip basedpyright; GPU; full workspace sweep;
git diff --check.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
2026-07-14 17:29:54 +01:00
Levi Neuwirth ff2ce0f197 fix(lsp): PR #117 round 1 — CMake config via initializationOptions
cmake-language-server does NOT pull a `workspace/configuration` section:
it reads `buildDirectory` from the `initialize` request's
`initializationOptions`, and drives its project model off CMake's File
API under `<buildDirectory>/.cmake/api/` (not `compile_commands.json`).
The `settings = { cmake = {} }` block — and the documented
`settings.cmake.buildDirectory` override — were therefore inert, leaving
conventional out-of-source project data unavailable.

Replace it with `init_options = { buildDirectory = "build" }` (the
conventional out-of-source dir; users override `init_options.buildDirectory`
from init.lua), and correct the comment. The wiring test now asserts
`config.cmake.init_options.buildDirectory == "build"` — bite-verified
against the pre-fix lsp.lua.

Gates: fmt; clippy -D warnings; --features crdt (1720); m4_acceptance
--skip basedpyright (109); GPU (59); full workspace sweep (zero
failures); git diff --check — all green. One `--lib` run flaked on
process::m6_1_pty_mode_lifecycle_started_then_exited (PTY-lifecycle
timing, the m6/m8 daemon-timing family); it passed in the crdt run, the
full sweep, and 4/4 isolated — unrelated to this Lua config change.
Change is Lua config + the acceptance assertion.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
2026-07-14 17:03:05 +01:00
Levi Neuwirth 7646dda583 feat(highlight): filename detection + Dockerfile/Make/CMake grammars
Files identified by their whole basename — Dockerfile, Makefile,
CMakeLists.txt, rc dotfiles — had no detection path (extension-, then
shebang-keyed). Add a filename layer and the three grammars behind it.

- **Grammars** (BUILTIN_LANGUAGES): dockerfile via `tree-sitter-containerfile`
  (the ABI-current grammar; the old `tree-sitter-dockerfile` pins
  `tree-sitter ^0.20` and would fork the graph — containerfile rides
  `tree-sitter-language 0.1`, tree-sitter dev-only, like the others),
  make via `tree-sitter-make`, cmake via `tree-sitter-cmake`. All ship
  self-contained highlights (single fragment). Extensions:
  `.dockerfile`/`.containerfile`, `.mk`/`.make`, `.cmake`.
- **Filename layer**: `pmacs.parse.language_from_filename(name)` backed by
  an extensible `pmacs.parse.filenames` map, wired into the precedence
  chain in both syntax.lua (grammar) and lsp.lua (LSP): grammar-ext →
  filetype map → filename → shebang. A recognized extension still wins;
  the basename map only fires when the extension misses. Seeds the three
  filenames plus shell rc dotfiles (`.bashrc`/`.zshrc`/`PKGBUILD`/… →
  bash) — highlighting them against the grammar shipped in #115.
- **LSP**: `config.dockerfile` (docker-langserver --stdio) and
  `config.cmake` (cmake-language-server). Make has no server, so no
  `config.make` — grammar highlight only. Extension filetype fallbacks
  added for id stability.

Bite-verified acceptance:
- filename_grammars_load_and_parse — each grammar's ABI is accepted by
  the tree-sitter 0.26 core and parses a representative snippet without
  error (dockerfile/cmake root at source_file, make at makefile).
- builtin_languages_include_dockerfile_make_cmake /
  language_for_path_resolves_dockerfile_make_cmake_extensions — entry
  presence and extension detection.
- m4_filename_map_resolves_special_files — the basename map (incl. path
  form and dotfiles→bash), config.dockerfile/cmake commands, and no
  config.make. Bite-verified against pre-feature syntax.lua.
- m4_filename_extensionless_dockerfile_highlights — an extensionless
  `Dockerfile` resolves to dockerfile for LSP and gets a dockerfile parse
  tree; reachable only via the filename map. Bite-verified.

Gates: fmt; clippy -D warnings; test --lib; --features crdt;
m4_acceptance --skip basedpyright; GPU; full workspace sweep;
git diff --check.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
2026-07-14 16:40:50 +01:00
Levi Neuwirth 7975eeda87 feat(themes): named UI faces + ThemeFacts channel (protocol v16)
Arc 4 stage 1 (docs/theme-faces-framing.md, revision 4). Faces are
theme entries under the reserved ui/ui.* namespace -- zero new Lua
API. Theme::face() resolves with the dotted-prefix walk but never
falls back to default_style; each face applies owns-surface within
its stage-1 component mask, identical on both frontends.

Substrate: two monotonic theme mutation counters (syntax/face) with
transactional set/merge/clear/default (parse before locking, commit
all-or-nothing, bump from the prior value); the StyleGate and the
minimap summary key on the counters -- fixing the pre-existing bug
where a mid-session pmacs.theme.set never re-shipped StyleSpans --
with the summary gaining payload-equality suppression that still
advances its key on computation.

Wire: InstanceMessage::ThemeFacts appended after CompletionPopup
(postcard discriminants are ordinal; a byte pin guards placement),
PROTOCOL_VERSION 15 -> 16, daemon-gated >= 16, one authoritative
table per attachment (None-seeded baselines), TUI silent-drop arm.

Grid: paint_frame resolves ui.modeline / ui.statusline /
ui.minibuffer(.candidate) / ui.gutter / ui.selection faces;
SearchView and DiagnosticView take the theme handle through the real
attachment paths (EditorCore injection, install_diag threading); the
canonical severity color resolves ui.diag.* with the Default ->
built-in policy that keeps the minimap presence encoding sound.

GPU: exact-name face table applied per draw with the Q#TH5 Default
mapping (plain text / window bg, reverse swap), local/peer wash
split, candidate-dropdown glyph site, and the status-band
shaping-cache invalidation without which a diag-face recolor with
constant counts kept stale counter colors.

Tests: 18-test acceptance suite (grid, wire, daemon gate, atomicity,
monotonicity, late join), 7 GPU headless tests incl. decoded vertex
colors, units for the face walk / transactional commits / producer
caches; protocol pins for v16 + the CompletionPopup byte pin.
Bites vs 3cbb9de (scripts/bite): semantic_render.rs (8 runtime test
failures), editor.rs (5 runtime), daemon.rs (v15 gate, runtime);
lua_bindings/mod.rs, pmacs-gpu/main.rs, search.rs, diag.rs, and
highlight.rs bite as compile failures (weaker evidence, disclosed).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VoiEyuPjoBhvwACf8HAnLB
2026-07-14 16:11:09 +01:00
Levi Neuwirth 7479213c3f fix(highlight): env -S payload is a full arg list, not a bare interpreter
The attached split-string payload (`-Spython3`, `--split-string=...`) can
itself begin with env options or VAR=value assignments before the
interpreter: `-S-i python3`, `-SFOO=bar python3`,
`--split-string=-u FOO python3`. Rather than taking the payload's first
word as the interpreter, re-inject the attached payload into the token
stream so it flows through the same option / operand / assignment state
machine as a separated payload. Adds the three cases as resolver tests.
2026-07-14 16:00:47 +01:00
Levi Neuwirth 558d00020f fix(highlight): PR #116 round 2 — pin grammar across switch, attached env -S
Two follow-ups from review, both in builtin/runtime/syntax.lua.

1. [P2] Buffer switching bypassed the pinned grammar. after-edit already
   reparsed the pinned language, but the after-switch reattach path
   (attach_for_active_buffer) re-resolved from scratch — so open an
   extensionless `#!/bin/sh` (bash), edit its shebang to lua, switch away
   and back, and the grammar flipped to lua while the LSP side kept its
   bash attachment (lsp.lua's after-switch reuses the existing record).
   attach_for_active_buffer now reuses the language pinned at first attach
   whenever a parse view already exists; only a first-seen buffer
   resolves. A language change still needs a close/reopen, matching both
   the after-edit behavior and how extensions work.

2. [P2] Attached `env -S`/`--split-string` forms failed. The walk skipped
   the whole option token, but for split-string the interpreter rides
   inside it: `-Spython3`, `-vSpython3` (after no-operand short flags
   i/v/0), and `--split-string=python3` all resolved to nil (the last was
   also eaten by the earlier `=` branch). The env walk now extracts the
   interpreter from the attached value (`^-[iv0]*S(.+)$` /
   `^--split-string=(.+)$`); the separated forms (`-S python3`) still work
   by walking on to the next token.

Tests (bite-verified against the round-1 syntax.lua — both fail there;
scripts/bite HEAD builtin/runtime/syntax.lua):
- m4_shebang_edit_keeps_pinned_grammar now adds a switch-away/back cycle
  (via pmacs.window.switch_buffer, which fires after-switch
  synchronously) and asserts the tree stays bash.
- m4_shebang_resolver_maps_interpreters adds the attached split-string
  cases (`-Spython3`, `--split-string=python3`, `-vSpython3`).

Gates: fmt; clippy -D warnings; m4_acceptance --skip basedpyright; GPU;
git diff --check green. Only-known-flake caveat as round 1
(editor::composition_overhead_under_ten_percent perf microbenchmark,
unrelated to this Lua change). Change is Lua-only plus the acceptance
tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
2026-07-14 15:20:22 +01:00
Levi Neuwirth f300b77533 fix(highlight): PR #116 round 1 — shebang precedence, pinned grammar, env operands
Three review findings, all in builtin/runtime/syntax.lua.

1. [P1] Syntax bypassed extension precedence and grammar availability.
   attach_for_active_buffer resolved `language_for_path or shebang`, but
   language_for_path knows only grammar-backed extensions — so a `.py`
   file opening with `#!/bin/sh` fell through to the shebang and got a
   bash parse tree, and an extensionless `#!/usr/bin/env python3` script
   dispatched "python" (no grammar) and raised "unknown language". A new
   resolve_active_language walks the full precedence chain — grammar
   extension -> LSP filetype map -> shebang — consulting the shebang only
   when the extension is unrecognized (a recognized non-grammar extension
   like .py is authoritative). Dispatch is then gated on
   pmacs.parse._has_language(lang), so grammarless languages are skipped
   silently. The extension parts stay keyed on buf:name() (unchanged from
   before), so path-less buffers that resolve a grammar by name — e.g.
   generated markdown buffers — are unaffected.

2. [P2] Editing an open script's shebang left parsing/highlighting stale.
   The after-edit path re-sniffed the mutable shebang: sh -> python
   raised "unknown language" while leaving the old bash tree, and
   sh -> lua swapped the parse tree under a highlight overlay still
   holding the original grammar's query. Reparse now uses the language
   pinned at first attach (parse_lang_by_buffer), never re-resolving —
   a language change needs a close/reopen, as it does for extensions.

3. [P2] `env` options with operands were mistaken for interpreters.
   `#!/usr/bin/env -u FOO python3` skipped `-u` but took `FOO`. The env
   walk now skips the operand of the operand-consuming GNU-env options
   (-u/--unset, -C/--chdir, -a/--argv0) before selecting the interpreter.
   -S/--split-string stays excluded (its string carries the interpreter).

Tests (bite-verified against pre-fix syntax.lua — each fails without its
fix; scripts/bite HEAD builtin/runtime/syntax.lua):
- m4_shebang_does_not_override_extension now also asserts _has_view is
  false (no bash grammar tree for a `.py` + `#!/bin/sh`), not only the
  LSP language.
- m4_shebang_extensionless_grammarless_language_is_silent — extensionless
  python resolves for LSP, gets no grammar view, and records no error.
- m4_shebang_edit_keeps_pinned_grammar — rewriting a `#!/bin/sh` script's
  shebang to lua keeps the bash tree and reports no error.
- m4_shebang_resolver_maps_interpreters — added the env-operand cases
  (`-u FOO`, `-C /tmp`, combined).

Gates: fmt; clippy -D warnings; m4_acceptance --skip basedpyright; GPU;
git diff --check all green. The only sweep failure is the pre-existing
editor::composition_overhead_under_ten_percent render microbenchmark
(ratio hovers at the 1.10 cutoff; flakes ~1/3 even isolated single-
threaded, already asserted-off on macOS) — a pure-Rust render loop this
Lua-only change cannot touch. Change is Lua-only plus the acceptance
tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
2026-07-14 15:02:59 +01:00
Levi Neuwirth 4a60e4c858 feat(highlight): shebang-based language detection for extensionless scripts
Extension detection missed extensionless scripts — `scripts/deploy`, git
hooks, `configure`, and `scripts/bite` itself — so they got neither
highlighting nor an LSP server. Add a first-line shebang fallback.

- New `pmacs.parse.language_from_shebang(buf)` (builtin/runtime/syntax.lua):
  sniffs the first line (capped at 256 bytes), maps the interpreter's
  basename to a language, and resolves the `#!/usr/bin/env python3`
  indirection (skipping env's own `-S`/flags and `VAR=val` assignments).
  Backed by `pmacs.parse.shebangs`, a user-extensible map seeded with the
  interpreters pmacs can act on: sh-family -> bash, python* -> python,
  node -> javascript, lua* -> lua.
- Wired as a strict *fallback* on both resolution paths: syntax.lua's
  grammar attach (`language_for_path or language_from_shebang`) and
  lsp.lua's `buffer_language` (grammar -> filetypes -> shebang). A
  recognized extension always wins, so a `.py`/`.sh` file is never
  re-classified by a stray shebang.
- Cross-language, not shell-only: `#!/usr/bin/env python` /`node` /`lua`
  resolve too. Special filenames (`.bashrc`, `Dockerfile`, `Makefile`)
  are intentionally deferred until there are grammars behind them.

Bite-verified acceptance (tests/m4_acceptance.rs):
- m4_shebang_resolver_maps_interpreters — the mapping incl. env
  indirection and `env -S`; non-shebangs and unmapped interpreters
  (ruby) resolve to nil.
- m4_shebang_extensionless_script_resolves_bash — opening an
  extensionless `#!/bin/sh` script resolves to bash on BOTH paths:
  lsp.lua's `active_buffer_language()` and a settled bash parse tree
  (grammar attach). Reachable only via the shebang, since the file has
  no extension.
- m4_shebang_does_not_override_extension — a `.py` file opening with
  `#!/bin/sh` still resolves to python (extension precedence).

Gates green: fmt; clippy -D warnings; test --lib; --features crdt;
m4_acceptance --skip basedpyright; GPU; full workspace sweep;
git diff --check. Change is Lua-only plus the acceptance tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
2026-07-14 14:33:06 +01:00
Levi Neuwirth 6fd7db81fe feat(highlight): shell/bash tree-sitter grammar for the shell family
Shell scripts already had LSP (bash-language-server + shellcheck/shfmt,
wired in builtin/runtime/lsp.lua), but no tree-sitter grammar, so their
text rendered without lexical color. Fill in the missing half.

- Bundle tree-sitter-bash (0.25) as a BUILTIN_LANGUAGES entry. Unlike
  cuda, bash's highlights.scm is self-contained (no `; inherits:`
  delta), so a single fragment suffices. The crate exports
  LANGUAGE/HIGHLIGHT_QUERY over tree-sitter-language 0.1 — shared ABI
  crate, no second tree-sitter in the graph.
- Extension set is wider than the `.sh`/`.bash` the LSP filetype map
  covered: `.zsh`/`.ksh`/`.ash` are close-enough dialects and `.bats`
  is bash. The grammar's language name is `bash`, matching the
  `pmacs.lsp.config.bash` key, so opening any of these also auto-attaches
  bash-language-server (shellcheck declines zsh, so `.zsh` diagnostics
  may be sparse; highlighting is unaffected). lsp.lua's filetype map is
  extended to the same set as the belt-and-suspenders fallback.
- Extensionless shebang scripts (`#!/bin/sh`) and rc dotfiles
  (`.bashrc`) are intentionally NOT covered: detection is extension-keyed
  and shebang/filename sniffing is a separate, deferred feature.

Bite-verified acceptance:
- bash_grammar_loads_and_parses_script — the 0.25 grammar's ABI is
  accepted by the 0.26 core (set_language succeeds at runtime) and a
  representative script (shebang, set, parameter expansion, function,
  if) parses without error, rooting at `program`.
- builtin_languages_include_bash / language_for_path_resolves_bash_
  extensions — entry presence and detection across the wider set.
- bash_highlights_compile_with_captures — the self-contained query
  compiles against the grammar with real capture classes.
- m4_12_default_bundle_wires_bash — through the loaded runtime,
  config.bash targets bash-language-server and both grammar detection
  and the filetype fallback resolve the new extensions to `bash`.

Gates green: fmt; clippy -D warnings; test --lib; --features crdt;
m4_acceptance --skip basedpyright; GPU; full workspace sweep;
git diff --check.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
2026-07-14 13:50:21 +01:00
Levi Neuwirth 3cbb9dedd0
Merge pull request #114 from levineuwirth/cuda-lsp
feat(lsp): CUDA support — clangd + bundled tree-sitter grammar
2026-07-14 11:04:09 +00:00
Levi Neuwirth ea3641bba2 fix(lsp): PR #114 round 1 — .cuh AST via fallbackFlags, real C/C++ highlights
Two functional gaps from review:

1. Standalone .cuh files got no clangd AST. clangd selects the
   compiler language from the file extension, not the LSP languageId:
   it knows .cu (-> -x cuda) but not .cuh, so a header with no compile
   command fails with fe_expected_compiler_job. config.cuda now sets
   init_options.fallbackFlags = { "-xcuda" }, which supplies -x cuda
   for any file this server opens that lacks a compile_commands.json
   entry (a real compile command still wins). This CUDA server only
   ever serves .cu/.cuh, so the fallback cannot mis-flag C/C++.

2. The CUDA highlights query was only a delta. tree-sitter-cuda's
   HIGHLIGHTS_QUERY opens with `; inherits: cpp` and defines only the
   CUDA-specific captures (launch brackets, __global__/__device__) —
   two capture classes. pmacs does not resolve `inherits:`, so ordinary
   C/C++ syntax went unhighlighted. LanguageEntry.highlights_query is
   now &[&str] (fragments joined base-first); the cuda entry carries
   [c, cpp, cuda], compiling to ~16 capture classes. Fragments are
   newline-joined, never bare-concatenated — a fragment can end mid
   `; comment`, and abutting the next fragment's first token would
   corrupt the query. Existing single-query grammars become one-element
   slices (byte-identical effective query; no behavior change).

Tests:
- cuda_highlights_resolve_c_and_cpp_captures — asserts the COMPILED
  cuda query carries the C base `@variable` capture and >= 8 capture
  classes, not merely a non-empty query (the CUDA delta alone has 2 and
  no `variable`, so this fails without the base prepend).
- builtin_languages_include_cuda — now asserts the entry composes the
  c + cpp + cuda fragments.
- m4_12_default_bundle_wires_cuda — now asserts
  config.cuda.init_options.fallbackFlags[1] == "-xcuda".

Gates green: fmt; clippy -D warnings; test --lib (1515); --features crdt
(1689); m4_acceptance --skip basedpyright (101); GPU (59); full
workspace sweep; git diff --check.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
2026-07-14 11:55:58 +01:00
Levi Neuwirth 11075914f3 feat(lsp): CUDA support — clangd + bundled tree-sitter grammar
Opening a .cu/.cuh file previously resolved to no language, so no
server attached and there was no highlighting. Wire CUDA end to end,
mirroring the existing C/C++ path:

- Bundle tree-sitter-cuda (0.21) as a new BUILTIN_LANGUAGES entry
  claiming .cu/.cuh, with its own HIGHLIGHTS_QUERY. A dedicated grammar
  rather than reusing cpp: the C++ grammar errors on the
  <<<grid, block>>> kernel-launch syntax. The crate rides
  tree-sitter-language 0.1 (its tree-sitter dep is dev-only), so it
  shares the ABI crate with the other grammars — no second tree-sitter
  in the graph.
- pmacs.lsp.config.cuda targets clangd (the same binary that serves
  C/C++; language_id "cuda" so clangd enters its CUDA parse mode), and
  .cu/.cuh filetype fallbacks map to "cuda" to keep the LSP id stable
  if the grammar is ever dropped. LspStyleView layers clangd's CUDA
  semantic tokens on top, exactly as for C/C++.

Bite-verified acceptance:
- cuda_grammar_loads_and_parses_kernel_launch — proves the 0.21
  grammar's ABI is accepted by the 0.26 core (set_language succeeds at
  runtime, which the compile step cannot confirm) and that the entry
  wired the CUDA grammar, not a cpp fallback: the <<<...>>> launch
  parses without error, whereas the cpp grammar reports an error on the
  same source (verified out of band).
- builtin_languages_include_cuda / language_for_path_resolves_cuda_
  extensions — entry presence and .cu/.cuh detection.
- m4_12_default_bundle_wires_cuda — config.cuda targets clangd and the
  filetype + grammar detection resolve to "cuda" through the loaded
  runtime.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
2026-07-14 10:58:36 +01:00
Levi Neuwirth 7562d83198 fix(compile): make overlay teardown atomic 2026-07-14 10:57:14 +01:00
Levi Neuwirth b6e44f21d6 fix(compile): PR #113 round 7 — validated overlay attachment, registry-only dispose
Finding-by-finding (framing revision 13; bites via scripts/bite
against fe04aa4):

1. attach_style_overlay validates the handle. A handle's translator
   follows edits to ITS buffer only, so attaching it to another
   buffer created a render view showing spans nobody maintains —
   rejected now, with the message naming the recorded owner and
   pointing at add_style_overlay for the target buffer. A disposed
   handle's translator is gone, so re-attachment resurrected
   rendering with frozen coordinates — the disposed state is shared
   across handle clones (FromLua clones) via Arc<AtomicBool> and
   attachment after dispose() fails, pointing at add_style_overlay
   for a fresh handle. Bite: r7f1 pins cross-buffer rejection,
   same-buffer acceptance, dispose-then-attach rejection, and both
   message shapes.
2. dispose() detaches the translator through the always-registered
   SharedRegistry; only the window cleanup rides the optional
   SharedCore. Pre-fix all cleanup lived inside the SharedCore
   branch, so an install-only/headless host got success with the
   translator left attached — paying on every edit for the buffer's
   lifetime. Registry-only unit asserts the buffer's view count
   returns to baseline (and stays there on double dispose); the
   acceptance-crate twin r7f2 builds the same install-only host and
   bites via the mod.rs swap (the in-crate unit vanishes with it).

Gates: fmt; clippy workspace all-targets; lib 1535; crdt lib 1709;
compile acceptance 65; crdt acceptance 3; m4 101; m6.4 15; m6.5 11;
m6.8 8; GPU 59; workspace sweep 2526/0; git diff --check.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VoiEyuPjoBhvwACf8HAnLB
2026-07-14 10:48:13 +01:00
Levi Neuwirth fe04aa481b fix(compile): PR #113 round 6 — idempotent split-complete attachment, no-op edit guard, handle disposal
Finding-by-finding (framing revision 12; bites via scripts/bite
against a49adc2):

1. Render-view attachment is idempotent and split-complete. Overlays
   expose overlay_identity (the span store's allocation address);
   Window::ensure_overlay attaches a store-backed render view AT
   MOST once per window — pre-fix every switch into the buffer
   blindly pushed another copy onto EVERY matching window, so
   passive panes accumulated duplicates, each cloning all spans and
   rescanning the buffer per frame. A same-buffer split copies
   clonable overlays to the new pane via clone_for_split (splits
   fire no switch hook and started with an empty overlay list — the
   new compilation pane rendered unstyled). Bites: the acceptance
   test asserts both panes styled with exactly one attachment
   IMMEDIATELY post-split (before any switch could heal the pane
   through the attach-to-all path — the first draft asserted only
   after bouncing and was vacuous against the split fix), then
   re-asserts after three bounce cycles; fails against pre-fix
   editor_core.rs (split half) and pre-fix mod.rs (accumulation
   half) independently. Units pin ensure-once and split-copy/no-copy.
2. The translator ignores pure no-op edits (buffers deliberately
   broadcast empty inserts/deletes for callers that count calls):
   pre-fix each interior no-op split the containing span into two
   adjacent fragments — unbounded list growth for repeated no-ops at
   distinct positions, and a no-op at a UTF-8 continuation byte
   minted a mid-codepoint span boundary. Units now cover genuine
   EditOp::Insert (the round-5 "insertion" unit only replaced) and
   no-ops at five interior positions including the continuation
   byte; the Lua twin (r6f2) bites via the overlay.rs swap — as a
   compile failure, since that file also carries the round-6
   identity machinery (weaker evidence, per the bite script's
   caveat; the in-crate unit pins the behavior directly).
3. StyleOverlayHandleLua retains the buffer and translator ViewId
   and exposes idempotent dispose(): detaches the buffer-attached
   translator (later edits stop paying for it) and removes every
   window render view over the store. Documented lifetime contract:
   one handle per buffer incarnation (the compile/REPL discipline)
   needs no disposal — the buffer's death frees it; repeated
   creation on a long-lived buffer must dispose retired handles.
   Bite: r6f3 (translate → dispose → edit must NOT move the span,
   render views gone, double-dispose safe) fails against pre-fix
   mod.rs.

Gates: fmt; clippy workspace all-targets; lib 1534; crdt lib 1708;
compile acceptance 63; crdt acceptance 3; m4 101; m6.4 15; m6.5 11;
m6.8 8; GPU 59; workspace sweep 2523/0 (one m8-class flake, clean on
rerun); git diff --check.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VoiEyuPjoBhvwACf8HAnLB
2026-07-14 10:17:56 +01:00
Levi Neuwirth a49adc2589 fix(compile): PR #113 round 5 — buffer-level span translation, fragment preservation, tracked line start
Finding-by-finding (framing revision 11; bites via scripts/bite
against 6793edc):

1. Style-span coordinate translation belongs to the BUFFER. A new
   BufferStyleSpanTranslator is attached by
   pmacs.buffer.add_style_overlay and sees every edit exactly once —
   bypass writes, undo/redo, remote CRDT ops — independent of window
   count or visibility; the window-attached BufferStyleOverlay
   copies are render-only (on_edit removed). Pre-fix each attached
   view translated the shared store: start_run's explicit attach
   duplicated the after-switch hook's (switch_buffer fires it
   synchronously), so the normal path shifted later spans TWICE per
   byte-delta rewrite, splits multiplied further, and a hidden
   buffer shifted ZERO times. The redundant attach is removed;
   correctness no longer depends on attachment discipline. Bites:
   per-cell rendered assertions active (red a, blue bc, CR, red é →
   é red, b/c blue) and hidden (run finishes with the buffer in no
   window; switch back renders true colors); three direct units pin
   exactly-once with extra render views attached.
2. Translation preserves the untouched fragments of a partially
   overlapped span: left of the replaced range keeps its styling,
   right of it shifts by the length delta, only the rewritten bytes
   lose theirs (the writer styles what it writes; inserted bytes
   inherit nothing). Pre-fix any overlap dropped the WHOLE span —
   red abc, SGR reset, CR, X left bc unstyled; zero translation
   painted the default X red instead. Bite: exact (glyph, fg) cells
   X=default, b/c=red — any_styled_cell cannot see either failure.
3. The per-CR/BS/erase-line whole-prefix scan is gone:
   slot.line_start is tracked — advanced at every \n (append helper
   + the mid-line newline branch), read O(1) by the rewind paths,
   reset on run start/resync/raw marker appends. Measured on 2 MB of
   output + 3000 CR updates (release): 2.52s pre-fix → 0.67s
   post-fix (remainder is fixture-bound; pre-fix cost grows with
   buffer size). No correctness bite is possible for a pure perf fix
   — the committed test pins the tracked value's behavior across
   multi-line appends, batch-boundary CR, repeated CR, erase-line,
   and recovery paths, and passes on both implementations by design.

Gates: fmt; clippy workspace all-targets; lib 1531; crdt lib 1705;
compile acceptance 60; crdt acceptance 3; m4 101; m6.4 15; m6.5 11;
m6.8 8; GPU 59; workspace sweep 2517/0; git diff --check.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VoiEyuPjoBhvwACf8HAnLB
2026-07-13 22:07:01 +01:00