These five languages already had LSP configs (basedpyright, gopls,
tsserver, taplo, zls) but shipped no tree-sitter grammar, so they
rendered with no lexical color. Fill the gap — 8 BUILTIN_LANGUAGES
entries across 6 crates:
- python (`tree-sitter-python`, root `module`), go (`tree-sitter-go`),
toml (`tree-sitter-toml-ng`), zig (`tree-sitter-zig`, +`.zon`) — each a
single self-contained highlights query.
- JavaScript/TypeScript family: `tree-sitter-javascript` parses both
`.js` and `.jsx`; `tree-sitter-typescript` ships two grammars
(LANGUAGE_TYPESCRIPT, LANGUAGE_TSX). The four entries — javascript,
javascriptreact, typescript, typescriptreact — mirror the LSP filetype
map so tsserver enables the JSX parser. Highlights inherit: the TS
query is a ~5-capture delta over JavaScript and JSX is a further
delta, so the entries compose base-first (js → jsx → ts), the same
pattern as `cuda` over C/C++ (typescript resolves ~22 capture classes,
typescriptreact ~24).
Each grammar's name equals its existing `pmacs.lsp.config.<name>` key,
so grammar detection (which wins over the filetype map) resolves the id
the server keys off — the file now gets BOTH highlighting and the right
server. No lsp.lua change needed. All crates ride `tree-sitter-language
0.1` with tree-sitter dev-only — no second core in the graph.
Bite-verified acceptance:
- gap_grammars_load_and_parse — each grammar's ABI accepted by the 0.26
core; a snippet parses without error at its root (covers both TS
grammars, incl. JSX).
- typescript_highlights_compose_the_javascript_base — the compiled
typescript/typescriptreact queries resolve >= 15 captures, not just the
~5-capture TS delta (the JS base is really composed in).
- builtin_languages_include_gap_grammars /
gap_grammar_extensions_resolve — entry presence + extension detection
across all 8 ids.
- m4_gap_grammars_align_with_lsp_configs — through the loaded runtime,
each path's grammar id matches an existing LSP config. Bite-verified
against pre-feature src/syntax.rs.
Ripple: two #116 shebang tests used python as their "has-LSP-but-no-
grammar" example, which this PR invalidates. Updated both — the .py +
`#!/bin/sh` precedence test now asserts a python grammar tree (not "no
tree"), and the grammarless-language-is-silent gate test switches to
`ruby` (genuinely grammarless) via a test-local shebang mapping.
Gates: fmt; clippy -D warnings; test --lib; --features crdt;
m4_acceptance --skip basedpyright; GPU; full workspace sweep;
git diff --check.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
Files identified by their whole basename — Dockerfile, Makefile,
CMakeLists.txt, rc dotfiles — had no detection path (extension-, then
shebang-keyed). Add a filename layer and the three grammars behind it.
- **Grammars** (BUILTIN_LANGUAGES): dockerfile via `tree-sitter-containerfile`
(the ABI-current grammar; the old `tree-sitter-dockerfile` pins
`tree-sitter ^0.20` and would fork the graph — containerfile rides
`tree-sitter-language 0.1`, tree-sitter dev-only, like the others),
make via `tree-sitter-make`, cmake via `tree-sitter-cmake`. All ship
self-contained highlights (single fragment). Extensions:
`.dockerfile`/`.containerfile`, `.mk`/`.make`, `.cmake`.
- **Filename layer**: `pmacs.parse.language_from_filename(name)` backed by
an extensible `pmacs.parse.filenames` map, wired into the precedence
chain in both syntax.lua (grammar) and lsp.lua (LSP): grammar-ext →
filetype map → filename → shebang. A recognized extension still wins;
the basename map only fires when the extension misses. Seeds the three
filenames plus shell rc dotfiles (`.bashrc`/`.zshrc`/`PKGBUILD`/… →
bash) — highlighting them against the grammar shipped in #115.
- **LSP**: `config.dockerfile` (docker-langserver --stdio) and
`config.cmake` (cmake-language-server). Make has no server, so no
`config.make` — grammar highlight only. Extension filetype fallbacks
added for id stability.
Bite-verified acceptance:
- filename_grammars_load_and_parse — each grammar's ABI is accepted by
the tree-sitter 0.26 core and parses a representative snippet without
error (dockerfile/cmake root at source_file, make at makefile).
- builtin_languages_include_dockerfile_make_cmake /
language_for_path_resolves_dockerfile_make_cmake_extensions — entry
presence and extension detection.
- m4_filename_map_resolves_special_files — the basename map (incl. path
form and dotfiles→bash), config.dockerfile/cmake commands, and no
config.make. Bite-verified against pre-feature syntax.lua.
- m4_filename_extensionless_dockerfile_highlights — an extensionless
`Dockerfile` resolves to dockerfile for LSP and gets a dockerfile parse
tree; reachable only via the filename map. Bite-verified.
Gates: fmt; clippy -D warnings; test --lib; --features crdt;
m4_acceptance --skip basedpyright; GPU; full workspace sweep;
git diff --check.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
Shell scripts already had LSP (bash-language-server + shellcheck/shfmt,
wired in builtin/runtime/lsp.lua), but no tree-sitter grammar, so their
text rendered without lexical color. Fill in the missing half.
- Bundle tree-sitter-bash (0.25) as a BUILTIN_LANGUAGES entry. Unlike
cuda, bash's highlights.scm is self-contained (no `; inherits:`
delta), so a single fragment suffices. The crate exports
LANGUAGE/HIGHLIGHT_QUERY over tree-sitter-language 0.1 — shared ABI
crate, no second tree-sitter in the graph.
- Extension set is wider than the `.sh`/`.bash` the LSP filetype map
covered: `.zsh`/`.ksh`/`.ash` are close-enough dialects and `.bats`
is bash. The grammar's language name is `bash`, matching the
`pmacs.lsp.config.bash` key, so opening any of these also auto-attaches
bash-language-server (shellcheck declines zsh, so `.zsh` diagnostics
may be sparse; highlighting is unaffected). lsp.lua's filetype map is
extended to the same set as the belt-and-suspenders fallback.
- Extensionless shebang scripts (`#!/bin/sh`) and rc dotfiles
(`.bashrc`) are intentionally NOT covered: detection is extension-keyed
and shebang/filename sniffing is a separate, deferred feature.
Bite-verified acceptance:
- bash_grammar_loads_and_parses_script — the 0.25 grammar's ABI is
accepted by the 0.26 core (set_language succeeds at runtime) and a
representative script (shebang, set, parameter expansion, function,
if) parses without error, rooting at `program`.
- builtin_languages_include_bash / language_for_path_resolves_bash_
extensions — entry presence and detection across the wider set.
- bash_highlights_compile_with_captures — the self-contained query
compiles against the grammar with real capture classes.
- m4_12_default_bundle_wires_bash — through the loaded runtime,
config.bash targets bash-language-server and both grammar detection
and the filetype fallback resolve the new extensions to `bash`.
Gates green: fmt; clippy -D warnings; test --lib; --features crdt;
m4_acceptance --skip basedpyright; GPU; full workspace sweep;
git diff --check.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
Opening a .cu/.cuh file previously resolved to no language, so no
server attached and there was no highlighting. Wire CUDA end to end,
mirroring the existing C/C++ path:
- Bundle tree-sitter-cuda (0.21) as a new BUILTIN_LANGUAGES entry
claiming .cu/.cuh, with its own HIGHLIGHTS_QUERY. A dedicated grammar
rather than reusing cpp: the C++ grammar errors on the
<<<grid, block>>> kernel-launch syntax. The crate rides
tree-sitter-language 0.1 (its tree-sitter dep is dev-only), so it
shares the ABI crate with the other grammars — no second tree-sitter
in the graph.
- pmacs.lsp.config.cuda targets clangd (the same binary that serves
C/C++; language_id "cuda" so clangd enters its CUDA parse mode), and
.cu/.cuh filetype fallbacks map to "cuda" to keep the LSP id stable
if the grammar is ever dropped. LspStyleView layers clangd's CUDA
semantic tokens on top, exactly as for C/C++.
Bite-verified acceptance:
- cuda_grammar_loads_and_parses_kernel_launch — proves the 0.21
grammar's ABI is accepted by the 0.26 core (set_language succeeds at
runtime, which the compile step cannot confirm) and that the entry
wired the CUDA grammar, not a cpp fallback: the <<<...>>> launch
parses without error, whereas the cpp grammar reports an error on the
same source (verified out of band).
- builtin_languages_include_cuda / language_for_path_resolves_cuda_
extensions — entry presence and .cu/.cuh detection.
- m4_12_default_bundle_wires_cuda — config.cuda targets clangd and the
filetype + grammar detection resolve to "cuda" through the loaded
runtime.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
The wiring that makes the menu and OS clipboard work end-to-end. The
protocol bump touches every exhaustive match on the wire enums, so the
daemon / frontend / GPU consumers all land together.
Protocol v11 (additive; SUPPORTED = [6..11]):
- `PointerKind::Context` (right-click), `FrontendEvent::MenuPointer`
(GPU->daemon navigation, index-only), `InstanceMessage::MenuPrompt` +
`MenuPromptRow` (daemon->GPU rows + highlight, daemon-gated >= 11).
Dispatch + producer:
- `EditorState`: menu interception in `dispatch_key`/`dispatch_mouse`,
`MenuKey`, `dispatch_menu_key`/`_mouse`, `open_context_menu` (TUI) /
`open_menu_at_byte` + `dispatch_menu_pointer` (GPU), `build_menu_rows`
(calls the Lua resolver), `dispatch_idle` now false while a menu is
open. `dispatch_pointer` gains the `Context` arm.
- daemon: routes `Context` -> open, `MenuPointer` -> navigate; gates
`MenuPrompt` >= 11; drains the clipboard publish as
`InstanceSignal::Clipboard`; honors the previously-dropped
`FrontendEvent::Paste` (so paste works for the first time).
- `semantic_render`: `MenuPrompt` producer with cached-compare.
Frontends:
- TUI (`frontend.rs`): OSC 52 clipboard write; ignores `MenuPrompt`
(the cell overlay renders the menu).
- GPU (`pmacs-gpu`): `arboard` dep; clipboard write/read + Ctrl-V inbound
paste; right-click -> `Context`; `MenuLocal` + `MenuPrompt` handler;
the popup (a second `TextRenderer` over bg quads) at the click pixel;
hover/click -> `MenuPointer`; key intercept while open.
Also folds a pre-existing clippy `unnested_or_patterns` nit in a search
test (`Color::Indexed(11 | 3)`) that newer CI clippy surfaced.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014TXbAwk27agwhrNNrhLi2U
The regex sibling of find_all, on regex::bytes::Regex over the whole
buffer. Returns Option<Vec<ByteRange>>: Some for a valid pattern
(possibly empty), None when it fails to compile — so the caller can
tell an invalid pattern (show [invalid]) from a valid zero-match
search.
Smart-case mirrors the literal path: case-insensitive via a (?i)
prefix unless the pattern carries an uppercase letter. Multi-line is
free — the regex runs over the whole byte slice, so an explicit \n (or
(?s).) spans lines while `.` keeps its default. Zero-width matches
(a*, ^, $) are filtered. The regex crate (already transitive in the
lockfile) is promoted to a direct dependency; its linear-time engine
makes a pathological pattern slow at worst, never catastrophic.
Tests: pattern match, smart-case both ways, \n-spanning + dotall +
default-no-cross, invalid→None vs valid-zero→Some(empty), zero-width
filter.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
pmacs-gpu now has two run modes:
- no args: hello-world (session-2 behavior preserved)
- --attach <socket>: connect to a pmacs daemon, negotiate
semantic_render + crdt_replica, import BufferSnapshot into a local
loro replica, render the rope text. Live CrdtOp updates apply as
they arrive.
Architecture:
- pmacs-gpu/src/attach.rs (new): UnixStream connect + Hello /
AttachRequest handshake on the main thread; spawns a reader thread
that pumps decoded InstanceMessage frames through the winit
EventLoopProxy as AppEvent::Attach(AttachEvent::Message). Clean EOF
or transport errors surface as AttachEvent::Disconnected. Reader
thread holds the read half of the stream; AttachClient retains the
write half (unused yet — session 4 wires FrontendEvents back).
- pmacs-gpu/src/main.rs: ApplicationHandler<AppEvent> with a
user_event handler that dispatches Message variants. BufferSnapshot
builds a fresh LoroDoc, imports the snapshot bytes, extracts text
via doc.get_text('body').to_string(), and re-shapes the glyphon
buffer. CrdtOp passes the op bytes through doc.import (loro
accepts both shapes), re-extracts text, re-shapes. Other
InstanceMessage variants are intentionally ignored at session 3.
- Font size dropped from 48pt to 16pt now that we may render full
files (the hello-world 48pt was fine for one line, awful for code).
- Initial text is '(connecting...)' in attach mode, 'hello, pmacs' in
hello-world; attach failure falls back to '(attach failed; see
stderr)' so the window still opens.
One small finding logged in attach.rs's connect() doc: AttachRequest's
initial_size field is a CellSize (rows × cols), nominally
TUI-shaped. Sent as a placeholder (24×80) — a structural answer
('what does initial size mean for a pixel frontend?') belongs in its
own protocol thread, not session 3. Classified under rule (iii) as
deferred.
Container id for the loro text container ('body') hardcoded to match
pmacs::crdt::CrdtState — second finding worth pre-recording: the
container name is a wire-adjacent convention that isn't carried on
the wire itself. Both ends have to agree out-of-band. Not blocking
for session 3 but a structural smell for the producer arc. Logged
as deferred (rule iii structural; the answer is probably 'thread the
container id through BufferSnapshot' but it's not session-3 scope).
Gates: cargo fmt, cargo clippy --all-targets -D warnings (whole
workspace) clean; lib 1303 + pmacs-protocol 11 = 1314 unchanged;
m4_acceptance 83; m11_5_semantic_acceptance --features crdt 2.
Manual validation pending — agent environment is headless. User
walks through: start a pmacs daemon, run pmacs-gpu --attach <socket>,
confirm the window renders the daemon's file contents.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the pmacs-gpu binary crate to the workspace. wgpu 29.0 + winit
0.30 + glyphon 0.11 (cosmic-text 0.18 via re-export) + pollster +
env_logger; pmacs-protocol in the dep graph but not consumed yet
(session 3 wires the attach loop).
The binary opens an 800x200 window titled 'pmacs-gpu hello-world',
sets up wgpu against its surface, configures glyphon with the bundled
JetBrains Mono Regular, and renders 'hello, pmacs' once per redraw.
Close button or Escape exits. Resize re-configures the surface and
glyphon viewport. Surface acquisition matches wgpu 29's
CurrentSurfaceTexture enum (success/suboptimal render through; lost/
outdated re-configure; timeout/occluded skip the frame).
Bundled assets: pmacs-gpu/fonts/JetBrainsMono-Regular.ttf (268 KB)
and pmacs-gpu/fonts/OFL.txt. Font shipped as required by the SIL
Open Font License 1.1.
One finding surfaced during the move and absorbed under rule (iii)
of the framing pass (small / no structural change): the design doc
recorded JetBrains Mono as Apache 2.0; the actual license has been
OFL since the family's open-source release. Doc corrected in
docs/pmacs-gpu-design.md.
Gates: cargo fmt + cargo clippy --all-targets -D warnings clean for
the whole workspace; cargo test --lib still 1314 (pmacs main crate
untouched); m4_acceptance 83; m11_5_semantic_acceptance --features
crdt 2.
Visual confirmation pending — agent environment is headless, so
'window opens, text renders' is user-side validation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Workspace skeleton: root Cargo.toml becomes a workspace with members
[".", "pmacs-protocol"]; [workspace.dependencies] pins serde,
postcard, thiserror so both crates use byte-identical versions (the
wire format depends on it). pmacs main package keeps its existing
shape (no file moves); it just gains pmacs-protocol as a path
dependency.
Identity types moved: BufferId (from buffer.rs), FrontendId + ByteRange
(from protocol.rs), Position type alias (from rope.rs). All four are
self-contained — no custom-type dependencies — so the first stage of
the move can land atomically without dragging cell/message types along.
src/buffer.rs / src/protocol.rs / src/rope.rs each gain a 'pub use
pmacs_protocol::...' re-export for the moved names, so existing
internal imports (crate::buffer::BufferId, crate::rope::Position, etc.)
continue to resolve unchanged. New consumers (pmacs-gpu, debug tools)
will depend on pmacs-protocol directly.
One visibility change: BufferId::from_raw was pub(crate); promoted to
pub with a doc note that it's not stable API for external consumers.
The (crate) restriction was advisory only — external deserialization
already worked via the derived Deserialize, so making it pub doesn't
widen the actual surface, just makes it honest.
Lib gate: 1314 passed, no regressions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drops the policy-A exclusivity that left grammar-backed languages
without LSP semantic refinement. Adds tree-sitter-c (.c/.h) and
tree-sitter-cpp (.cpp/.cc/.cxx/.hpp/...) to the bundle so the grid
TUI gets lexical highlighting (keywords / strings / operators) on
first open. The Lua attach in builtin/runtime/lsp.lua now pushes
LspStyleView whenever an LSP server is up, regardless of grammar
presence; with both views attached the cell-painter pipeline runs
SyntaxHighlightView first (lexical) then LspStyleView (semantic)
and their styles compose through crate::overlay::merge_styles. The
result is the VSCode / Zed "TextMate + LSP semantic tokens" model
on a terminal grid: keywords colored by tree-sitter, identifiers
refined by clangd's semantic tokens.
`.h` is ambiguous C / C++; the `c` BUILTIN_LANGUAGES entry claims it
to match the LSP filetype map's default. Users who want `.h` parsed
as C++ can override via Lua (extension → language map).
Note the tree-sitter-c / -cpp crates expose `HIGHLIGHT_QUERY`
(singular), matching tree-sitter-md's `HIGHLIGHT_QUERY_BLOCK`
convention; tree-sitter-rust / -lua use `HIGHLIGHTS_QUERY` (plural).
Same bundled highlights.scm either way.
Regression guard: builtin_languages_include_c_and_cpp asserts the
language entries exist and claim their canonical extensions. The
LspStyleView module doc rewritten to reflect dual-authority
composition; the existing headline test's comment updated (the
test fixture still attaches only LspStyleView directly, so its
asserted cells reflect the LSP authority alone — Lua-level
attach_buffer is what exercises composition end-to-end).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Roadmap steps #3–#6 (post-CI-green doc/version work; no source
change — the CI-validated tree at ed78465 is unchanged).
- CHANGELOG: authored the [1.0.0] --- 2026-05-18 body. M7–M10 arc
(third-party packages / fs API + dired-magit-outline / MCP /
multi-frontend CRDT collaboration) over the 0.1.0 M1–M6 preview;
the pulled-forward v0.2-prerequisite public APIs; SSH stderr
Changed + Broken-pipe Fixed carried from Unreleased; Known
limitations (per-frontend undo not persisted across reattach,
Finding 4; macOS m6_5 REPL ctrl-c/exit-marker timing); project
posture (forbid(unsafe_code), 1.95.0 pin, cross-flavor CI).
- Version: 0.1.0 -> 1.0.0 (Cargo.toml + Cargo.lock). Production
version reporting already flows from CARGO_PKG_VERSION; verified
`pmacs --version` -> `pmacs 1.0.0`, version-sensitive tests pass.
- README: Status -> v1.0.0 stable, contributions open; build line
-> the rust-toolchain.toml-pinned 1.95.0.
- MSRV: rust-version 1.85 -> 1.95 to match the validated toolchain
pin (was an unverified floor; pmacs is a pinned-toolchain app, so
MSRV reflects the pinned/validated compiler).
Quiescent audit (#6): doc/version-only delta from CI-green ed78465;
build/version-tests/fmt verified clean on pinned 1.95.0; prose
reviewed accurate. SP-9 (macOS m6_5) logged in the gitignored
V0.2-PREREQUISITES.md.
Not in scope here / remaining: the recorded two-laptop manual
acceptance run (#7, operator) and the v1.0.0 tag (#8, operator).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Land the optimistic local-edit-application layer on top of the M10 CRDT
foundation: frontend-side rope replica with local edit application,
daemon-authoritative broadcast, and bidirectional cursor reconciliation.
Keystrokes feel instantaneous because the local replica answers next-render
queries before the daemon round-trip completes, while the daemon remains
the single source of truth for conflict resolution and broadcast to remote
replicas.
Architecture beats:
- BufferMirror (src/buffer_mirror.rs) holds a per-frontend rope replica
with explicit cursor-staleness tracking. Every event that may move the
active cursor or swap the active buffer marks the mirror stale; the
next CursorByte from the daemon clears it.
- CrdtOpOrigin {OptimisticReplica(FrontendId), DaemonKey} routes broadcast.
OptimisticReplica skips re-application on the originating frontend
(already applied locally); DaemonKey broadcasts to all replicas including
source -- covers Lua-driven and generated-buffer edits that bypass the
optimistic path.
- Generated buffers (*help*, *workers*, *pmacs-instance*, *errors*) funnel
apply_edit output through queue_daemon_origin_crdt_op so post-attach
CRDT upgrades don't drop their edits.
- forbid(unsafe_code) preserved throughout; loro 1.12 added as the CRDT
engine.
Audit posture: M10.10 shipped through six post-audit review rounds with
twenty-eight cumulative findings, most categorized as "incomplete
application of a prior round's mechanism." The audit doc records
grep-driven exhaustiveness as the standing countermeasure.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Land the Model Context Protocol (MCP) integration as a transport binding,
not a built-in feature. Six Lua functions plus userdata methods expose
the substance of three MCP feature areas (resources, tools, prompts), a
notification dispatcher, and a non-trivial AI-assistance example package
that meets the architectural ship gate (spec/pmacs-spec.tex:1572): zero
direct calls into the Rust core, zero special-cased MCP handling outside
the public API, source under 2000 lines of Lua.
The M9.5 -> M9.6 -> M9.7 -> M9.8 layered composition validates the claim
"AI is a transport binding, not a feature" -- pmacs-mcp-ai composes with
pmacs-mcp-prompts.render and inherits notification handling transitively
through M9.7's package, demonstrating that the AI domain is a layer
above MCP, not a thread woven through the core.
Subtask shape:
M9.1 stdio transport + initialize handshake + restart policy
M9.2 resources with in-flight + settled cache and per-uri invalidation
M9.3 tools with isError-vs-JSON-RPC-error semantics + cancellation
M9.4 prompts with required-argument validation
M9.5 notification dispatcher (on_notification, off_notification)
M9.6 tools-as-commands fixture package + 12 audit findings disposed
M9.7 prompts-as-result-buffers fixture package + tree-sitter-md grammar
M9.8 AI-assistance fixture package (363+ LoC; 17/17 acceptance tests)
M9.9 formal package audit -- PASS on all three criteria
M9.10 release: TRANSITION-M9.md + MCP-for-package-authors guide
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Levi Neuwirth