Frame the JSON/YAML grammar-gap addition (tree-sitter-json 0.24 +
tree-sitter-yaml 0.7, both ABI-current) plus their language servers, and
the free frontmatter/fence highlighting that falls out of the #122
injection engine (the markdown block query already injects yaml for `---`
frontmatter and toml for `+++`). Four decisions (Q#JY1-JY4), nine
acceptance tests. JSON is also the standing prerequisite for the Jupyter
.ipynb arc.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
Define the canonical repository by URL and bootstrap a stable local
`githubsucks` alias instead of assigning authority to machine-specific
remote names. Record JSON/YAML checkpoint f99870e and the completed live
YAML provider plus bite evidence, leaving only rebase and full gates.
Keep AGENTS.md and CLAUDE.md synchronized so a fresh agent receives the
same recovery rule on any machine.
Co-Authored-By: OpenAI Codex <noreply@openai.com>
Record the pushed JSON/YAML checkpoint and framing heads so a clean
machine can verify it recovered the intended state, and document the
existing-local-branch worktree variant.
Add synchronized agent bootstraps and a volatile active-work ledger so
another machine can distinguish durable project state from open
branches, local checkpoints, machine-only providers, and incomplete
verification. Record githubsucks/main as the canonical development
line, refresh the #124/protocol-v17 handoff, correct the current
keybinding reference for compile mode, and mark the July roadmap as a
historical snapshot.
Normalize source bytes to representable shaped-cluster cursors and
reflow the code buffer whenever gutter or minimap geometry changes.
Measure alternate monospace advances across complete shaped runs and
load hermetic fixture faces before FontSystem construction. Complete
the rendered geometry, popup, caret, snapshot, and fixture acceptance
coverage, and record the review fixes in framing revision 5.
tests/gpu_font_acceptance.rs (items 2-7; the header manifest routes
item 1 to src/protocol.rs pins, item 8 to the src/frontend.rs TUI
drop unit, items 9-14/16-19 to pmacs-gpu's headless suite, and item
15 to the docs):
- 2: a fresh attachment's first frame carries the REAL (None, None)
default; unchanged ticks are silent; a late joiner receives the
current preference without post-attach mutation
- 3: a mid-session set_font emits exactly one FontFacts on the next
frame; an identical re-set advances the epoch without emitting
- 4 (producer half): on_buffer_snapshot_sent re-ships buffer facts
but never the bufferless FontFacts
- 5: real-daemon probe -- a v17 semantic session receives FontFacts,
a v16 peer never does (crdt feature)
- 6: the strict Lua contract -- 5.999/72.01/0/-16/NaN/inf/non-number
sizes error naming `size`; empty/non-string family errors
naming `family`; unknown keys rejected by name; hostile
__index/__pairs metatables never invoked and never inject
values; quantization pins 15.994->1599 / 15.996->1600 and both
boundaries; set_font {} resets both axes; the getter returns a
fresh quantized plain table; every rejected shape leaves the
preference and the wire untouched
- 7: a load_user_config_at fixture's init.lua set_font lands in the
handle installed before user config, and a pre-attach
preference ships on the first frame
Bites (mutate, observe the acceptance test fail, restore): the
Option-seeded first-frame send, the payload compare, the snapshot
survival of the font baselines, the v17 gate (BOTH halves widened --
producer for_peer alone leaks nothing because the daemon skip arm
still filters; the wire test only fails when belt AND braces are
cut), the unknown-key rejection, the range-check-original ordering,
and the raw_get metatable isolation.
Docs (item 15): docs/semantic-frontend-protocol.md gains the
FontFacts variant entry (v17 gate, authoritative default, no-pixels
preference relay, fail-closed receiver) and folds FontFacts into the
BufferSnapshot reset-contract paragraph (bufferless facts survive;
the frontend's caret-follow scroll residual is the buffer-scoped
part). docs/pmacs-gpu-design.md's "future customization needs no
wire-protocol changes" claim is corrected in place and points at the
framing as design of record; the Lua-override bullet now names the
landed set_font shape. The framing status bumps to implemented.
Also re-homes the ProcessSupervisor doc comment that the make_font_pref
insertion had orphaned onto the wrong function.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VoiEyuPjoBhvwACf8HAnLB
Twenty-one new headless tests covering the GPU-side acceptance items of
docs/gpu-set-font-framing.md (9-14, 16-19):
- 9: size route re-derives metrics; (None, None) reset reproduces
the never-set frame byte-for-byte
- 10: 600/7200 bounds render; minibuffer dropdown windows its rows
above the band at the derived row height; context menu keeps
the clipped route with coherent hit geometry
- 11: wrapped caret survives 16->72->6 px; an optimistic insertion
that wraps a new bottom row follows immediately and its
confirming CursorByte needs no second repair; overscan caret
never snaps the viewport; narrowing resize re-follows via the
coarse + visual-run + fold pipeline; adornment projection
shifts the caret past injected text with left gravity at the
anchor; the CursorByte arm follows into a wrapped run as a
normalized sub-line residual; explicit scrolls clear the
residual (including wheel-up at the clamp edge); the
minibuffer suspends the follow decision
- 4 (GPU half): BufferSnapshot resets the residual while the font
preference/metrics survive -- the replacement buffer wraps
too, so the EOF clamp cannot mask a leaked residual
- 12: unknown + proportional families fall back; the four-style
gate rejects a proportional BOLD sibling the normal-only
query would accept; a valid second monospace family resolves,
changes the ink, and resets cleanly; the parameterized
sanitizer removes exactly the same-family proportional
collision
- 13: the "\0" sentinel defeats the status string-equality gates
- 14: a shrinking visible slice re-declares the scoped viewport
- 16: 0/599/7201/u32::MAX reject the whole message, state untouched
- 17: metrics + drawable dimensions atomic on all seven buffers,
resize symmetric via Buffer::size()
- 18: popup rows never wrap (Wrap::None + unique line_i per run)
- 19: with line numbers and a context menu open over an EMPTY
buffer, the measured probe advance drives gutter reservation
and menu hit width (fixture ratio exactly 1.2 vs the bundled
default); reset restores the exact geometry and frame
Family routing is hermetic: four generated fixture faces under
pmacs-gpu/fonts/test/ (second monospace at 720/1000, a proportional,
and a "Pmacs Test Family" whose NORMAL face is monospaced but whose
BOLD face is proportional), built by the committed generate.py --
provenance and license in LICENSE.txt beside them. fontdb requires a
PostScript name (nameID 6), which the generator sets explicitly.
assemble() now routes construction through sync_buffer_dimensions
too: shape_until_cursor and wrapping must use the painter's clip
from the first frame -- a v16 daemon never sends the FontFacts that
would sync them later (found by the CursorByte-follow test: the
caret parked in the status band at surface-height dims).
Every protection was bitten (temporarily reverted, its test observed
failing, then restored): wire validation, CursorByte visual-run
follow, painted-before gate, projection inversion, Wrap::None,
four-style gate, sanitizer, status sentinel, normalize fold,
scroll_by_lines residual clear, BufferSnapshot residual reset, the
all-seven dimension helper, and the optimistic-completion follow.
The snapshot-reset test was strengthened after its first bite did
NOT fail (a short replacement buffer let cosmic's EOF clamp zero the
residual anyway).
PMACS_REQUIRE_GPU suite 90/90; clippy -D warnings and fmt clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VoiEyuPjoBhvwACf8HAnLB
The stage-2 GPU application half, framing revision 4:
- apply_font_facts transaction: fail-closed wire validation
(600..=7200 centi-px; deserialized protocol input, not the Lua
courtesy check), painted-before decision from the ACTUAL caret rect
intersected with the drawable code clip, four-style monospace
resolution with total fallback to the sanitized default, derived
metrics + measured advance, atomic re-metric/re-size, status
shaping-cache drop, reshape at the retained scroll, drawable-width
settle pass, conditional caret re-follow, minimap cache drop.
- Advance probe: fixed ASCII digits shaped in the resolved family at
the new code metrics; the NORMAL-face advance is authoritative for
gutter geometry (mono_advance no longer trusts an arbitrary code
glyph once measured), and the selected/default ratio scales the
const-based fallbacks -- the default family is ratio 1 by
construction, so never-set/reset stays byte-identical.
- sync_buffer_dimensions: metrics + REAL drawable dimensions change
atomically on all seven buffers via set_metrics_and_size; the code
buffer wraps at the painter's clip width; resize() routes through
the helper (closes the old four-of-seven skew) and applies the same
painted-before follow policy.
- Rows stay rows: Wrap::None on the menu/minibuffer/completion
buffers at assembly and (idempotently) in the transaction.
- Normalized code-buffer Scroll residual: slice-local line == 0
invariant with a vertical pixel residual; horizontal always
discarded (glyphon 0.11 never applies it). normalize_code_scroll
folds line advances into whole-file scroll_top (strictly advancing
origin, EOF clamp) after EVERY final code shape. BufferSnapshot
resets the residual (buffer-scoped view state); explicit
wheel/minimap jumps clear it -- including a wheel-up whose only
remaining motion IS the residual.
- Byte-to-layout projection: code_byte_to_projected inverts the
line_chunk_cache chunk projection (earliest projected boundary for
adornment anchors, the left-gravity caret), then code_byte_px uses
cosmic-text's layout_cursor so wrap boundaries select the same
visual run shape_until_cursor scrolls to. caret_rect and
completion_anchor_px are now visual-run aware instead of scanning
the source line's first run.
- ensure_caret_painted (coarse source-line follow, then
shape_until_cursor, discard Scroll.horizontal, normalize) shared by
the CursorByte arm (under its existing moved gate -- fixes the
pre-existing wrapped-line follow hole), the optimistic edit
completion, the font transaction, and resize.
- Gutter mirrors the code layout's visual runs: continuation blanks
for wrapped runs plus the same normalized vertical scroll, so line
numbers stay row-aligned when wrapping appears.
- assemble() shapes the initial text through reshape() so the
buffer.lines <-> line_chunk_cache invariant holds from
construction (the projection inversion depends on it).
PMACS_REQUIRE_GPU suite 69/69; clippy -D warnings and fmt clean.
Acceptance tests for items 9-11/13-14/16-19 land in the next commit
with the embedded test faces.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VoiEyuPjoBhvwACf8HAnLB
The 13 compile-time metric consts become BASE_* values behind a
FontMetrics { scale, advance_ratio } carried on State: every
surface derives from one knob (size/16.0), menu_char_w and the
empty-gutter advance fallback additionally multiply the measured
selected/default advance ratio, and the Default (1.0, 1.0)
reproduces today's constants bit-for-bit -- all 69 GPU tests pass
unchanged. The compiler enumerated every use site via the BASE_*
rename; fm threads as a parameter through the free helpers
(text_area_bottom, estimated_visible_lines, minimap_height/
band_contains/y_to_line/rects, edge_scroll_direction,
mb_dropdown_window, menu_width_px). The stray Metrics::new(16.0,
22.0) literal in assemble() now derives from the same fields, and
all seven Family::Name("JetBrains Mono") literals route through
State.resolved_family (seeded from the sanitized default; the
borrow checker forced hoisted clones at set-text sites, so the
field itself is the single source rather than an accessor).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VoiEyuPjoBhvwACf8HAnLB
assemble() replaces FontSystem::new() + post-hoc load_font_data
with an explicit fontdb::Database built in today's order -- system
fonts first, the bundled JetBrains Mono second (its fontdb::ID
retained) -- then the parameterized same-family collision filter
(sanitize_font_database removes every NON-monospace face
advertising the default family; the bundled face survives by
construction), cosmic-text's generic-family defaults, and only
then FontSystem::new_with_locale_and_db (sys-locale with the
"en-US" fallback, the same resolution cosmic-text's own
constructor performs; new pmacs-gpu dependency) -- so the internal
monospace-ID set is computed over the final database, bundle
included. FontDefaults { default_family, bundled_id } lands on
State as the total-fallback anchor for the resolution work in the
next commit; query_normal_face is the shared normal-style query
(the same fontdb::Query the base Attrs imply). Debug assertions
pin both anchors present and monospaced at assembly. All 69 GPU
tests pass unchanged -- assembly preserves today's pixels.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VoiEyuPjoBhvwACf8HAnLB
Protocol v16->17: InstanceMessage::FontFacts { family,
size_centi_px } appended after ThemeFacts (integer hundredths of a
logical pixel -- the enum derives Eq, f32 cannot; range 600..=7200
documented on the wire). Pins updated: version 17, ladder accepts
6..=17 rejects 18, FontFacts round-trip (populated + all-None), and
a ThemeFacts byte pin ([23, 0]) guarding the appended placement.
Daemon half: FontPref { family, size_centi_px, epoch } behind a
shared handle on EditorState, installed with the new pmacs.gpu Lua
module BEFORE load_user_config so init.lua set_font lands in the
state the first attachment reads. set_font is strict plain data
(raw_get, unknown raw keys rejected by name, metatables never
consulted, parse/validate/quantize fully before locking -- range-
check the ORIGINAL value so 5.999 errors, then nearest-hundredth
round); pmacs.gpu.font() returns a fresh quantized table.
Producer: font_facts_msg (the theme_facts_msg discipline --
Option-seeded epoch + payload baselines, advance on computation,
one authoritative send per attachment incl (None, None),
bufferless so on_buffer_snapshot_sent never touches it); for_peer
gains peer_knows_font_facts (>= 17); daemon write-loop skip arm;
TUI silent-drop arm + regression test; first-frame count test now
expects 6 messages. GPU application follows in the next commit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VoiEyuPjoBhvwACf8HAnLB
pmacs.gpu.set_font { family?, size? }: one global daemon-side font
preference relayed to GPU peers as the bufferless FontFacts fact at
protocol v17 (size in integer hundredths of a logical pixel -- the
InstanceMessage Eq derive forbids f32), applied mid-session as one
fail-closed GPU transaction: sanitized current-order fontdb
assembly with four-style monospace queries, derived metric fields
scaled from one knob (band height included), atomic
set_metrics_and_size on all seven buffers with real drawable
dimensions, Wrap::None on row-oriented popups, a measured
advance-ratio probe, and visual-run-aware caret preservation (a
normalized code-buffer Scroll residual + shape_until_cursor,
shared with the CursorByte and optimistic-edit paths, gutter
continuation blanks keeping line numbers aligned). Revisions 2-3
fold framing rounds 1-2 (Eq-safe wire size, wire validation
failing closed, caret repair ordered before the final reshape,
vertical-containment acceptance, sanitized default query,
quantization pins); revision 4 is the user's round 3 (visual runs,
horizontal-scroll deferral, atomicity, popup row invariance,
strict Lua raw-data contract). Acceptance items 1-19.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VoiEyuPjoBhvwACf8HAnLB
Post-merge handoff sync for multi-language injections (#122), folded
into the side-quest-backlog PR (rebased onto current main, which now
carries #120 themes stage 1 and #122 injections).
agent-handoff.md:
- §1: main head → 5e73966; add the #122 injection-engine entry (layered
ParseTreeBundle, worker-side layer building off the static grammar
table, settle-time query resolution, the new substrate — injections_query,
the alias snapshot, injection_capped, compute_highlight_spans_for, the
flatten event-sweep, the GPU fold — and the two hard-won findings).
- §6: the "multi-language injections" deferral is retired; replaced with
the real follow-ups (injection.combined, child-tree incrementality,
injectable Lua-registered languages, HTML/CSS/GraphQL/SQL consumer
grammars). Jupyter is now gated on JSON only.
side-quest-backlog.md: injections marked SHIPPED with its remaining
follow-ups; Jupyter gate and north star revised (JSON grammar and
locals-query processing rise to the top now that injections landed).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
Consolidate the non-themes backlog into docs/side-quest-backlog.md: an
exhaustive, deduplicated index compiled from every framing doc's
"Deferred (named)" section, the handoff §6 list, the roadmap arcs, and a
code-level marker sweep. Organized as side quests (one-shot-ish, led by
the highlighting/detection/LSP/editing families that continue the
#114–#118 stack), raid bosses (vterm, folding, DAP, GPU splits, Jupyter),
cross-cutting substrate blockers, and a prioritization north star.
Themes/faces/color items are excluded (they're the Arc 4 main quest) and
listed once so they're visibly accounted for. Handoff §6 gains a pointer
to it.
Docs only. git diff --check clean.
Rebase the injection branch onto main after #120 (themes stage 1,
protocol v16) merged. Conflicts were confined to appended test blocks
in pmacs-gpu/src/main.rs (kept both #120's GPU face tests and the
injection source_color test); highlight.rs, lua_bindings/mod.rs, and
semantic_render.rs auto-merged (my injection code and #120's theme
epochs / resolve_faces summary path / ThemeFacts occupy disjoint
regions). The one API adaptation: #120 gave scoped_file_summary a
`resolve_faces: bool` parameter, so the full-buffer flatten perf gate
passes `false` (it exercises the syntax flatten path).
Gates re-run on the rebased base: fmt, clippy -D warnings, lib (1582),
lib crdt (1756), m4 (110), theme_faces (22, #120's suite unbroken),
PMACS_REQUIRE_GPU gpu (70), full workspace sweep (0 failures),
git diff --check.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
Two follow-ups + doc cleanup.
[P2] Same-depth sibling precedence was reversed. The wire priority was
(depth, capture_order), omitting the layer ordinal, so two overlapping
spans from different sibling layers tied — and the active-set insert
then applied the later one first, making the earlier sibling win, the
opposite of the grid's layer-by-layer paint. Priority is now
(layer_index, capture_order): layer_index is the depth-ascending
position in bundle.layers, so a deeper layer AND a later same-depth
sibling both override, matching the grid exactly. New
flatten_same_depth_sibling_later_layer_wins pins it.
[P2] Cap surfacing had no end-to-end test. Added
injection_cap_surfaced_once_and_rearms_via_lua, which drives the real
Lua settle path (syntax.lua tick -> _injection_capped -> pmacs.error)
and asserts surfaced-once, suppressed-on-unchanged-reparse, and
re-armed-after-dropping-below-then-exceeding-the-cap.
Docs:
- Q#IJ6 now states the accurate bound O(n log n + Sum active) for the
event sweep, not O(boundaries).
- The full-buffer perf test is renamed/narrowed to guard the FLATTENER
regression; the summary's per-line dominant-style tally is a separate
pre-existing O(lines x spans) loop, not claimed linear.
- Framing #9 now matches the test: it drives spans_from_segments (the
extracted replace_style_spans transform) + source_color_at, not a live
State render.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
Four review findings + a cleanup bundle.
[P1] Wire flattener was O(spans²) and ran over the WHOLE buffer (the
file-style summary uses a whole-buffer viewport, not the visible one).
Replaced the per-interval full scan with an ordered active-set event
sweep (activate on start, expire on end, fold the active set) — linear
in practice. Added full_buffer_summary_scales_on_large_grammar_file
(1500-line rust) as the perf gate.
[P2] _parse_now used the empty alias map from make_request while
_dispatch snapshotted the registry map, so a `py` fence injected async
but not sync. Snapshot aliases on both paths; pinned by
sync_parse_now_resolves_alias.
[P2] The multi-range inline test used a one-line paragraph, whose block
inline node has no named children (link/emphasis are child-grammar
structures) — one range, so it couldn't falsify multi-range. Replaced
with a multi-line blockquote whose inline node carries a named
block_continuation: content_node_ranges now asserts >1 collected range
and emphasis parses on both lines.
[P2] The layer backstop dropped regions silently; the framing requires
a surfaced warning. run_parse now sets ParseTreeBundle::injection_capped;
syntax.lua's settle tick raises it once per buffer via pmacs.error
(_injection_capped). Added injection_layer_cap_surfaces_and_preserves_root
(drives >4096 fences, asserts the flag + bounded count + intact root).
Cleanup:
- The GPU acceptance test now drives the real StyleSpans full-frame
transform (spans_from_segments, extracted from replace_style_spans)
instead of a hand-rolled sort.
- content_node_ranges excludes NAMED children (documented as a round-1
refinement); framing mechanic #3 / Q#IJ5 updated to match.
- parse_duration doc now says root parse; the markdown entry no longer
describes inline as unhighlighted/future.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
Teach the syntax engine that one buffer can hold more than one
language. After the root parse, run the grammar's injections.scm, parse
each embedded region with the injected language, and merge every
layer's highlight spans. First consumer: markdown fenced code + inline
(zero new grammars — the block grammar already ships an injection query
and the injected langs already have grammars from #118).
Engine (src/syntax.rs):
- ParseTreeBundle now holds Vec<Layer> (root layer 0 + injected
children, depth-ascending); installed atomically so the existing
Arc::ptr_eq style gate and highlight cache keep working (Q#IJ1).
- run_parse builds layers on the worker: run injections.scm, resolve
the injected language, compute Vec<Range> (exclude NAMED children,
intersect the parent's ranges), set_included_ranges cold-parse,
recurse — bounded by depth (3), a layer backstop (4096), and a
(lang,ranges) visited guard; any child failure drops that child only
(Q#IJ3/IJ5). LanguageEntry gains injections_query; markdown_inline is
registered (retires the M9.7 block-only floor); markdown/rust carry
injection queries.
- Injected languages resolve off the static BUILTIN_LANGUAGES table
(Send loaders + query sources), preserving lazy loading. Dynamic
fence names go through a case-folded alias map seeded with defaults
and Lua-extensible via pmacs.parse.injection_aliases, snapshotted into
ParseRequest at dispatch so the worker never touches the Rc registry
or a Lua table (Q#IJ2/IJ4). Highlight queries are resolved at settle
(resolve_layer_queries), keeping query compilation main-thread/cached.
Producers:
- SyntaxHighlightView (grid) iterates layers shallow-to-deep so a
deeper layer's styling wins within its region (Q#IJ6/IJ7).
- scoped_style_spans (wire) flattens all layers into DISJOINT effective
spans via a boundary sweep, since the GPU re-sorts spans by start
(replace_style_spans / merge_style_spans) and would otherwise destroy
producer order. The GPU source_color_at consumer is fixed to fold all
covering spans (matching semantic_client's effective_style_at) rather
than returning the first.
Named-children exclusion: content ranges exclude only NAMED children
(matching tree-sitter-md's own inline splitter) — excluding a block
inline node's anonymous text tokens would shred the paragraph into
unparseable fragments.
13 acceptance gates (framing docs/multi-language-injections-framing.md):
layer structure, absolute child offsets, alias resolution (static +
case-folded dynamic + unknown-skip + Lua-async override), multi-range
inline, recursion bounds, wire + grid + GPU producers, incremental edit
/ new fence, many-paragraph settle budget with tail coverage, and the
single-layer regression guard.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
Frame the tree-sitter injection-layer engine: after the root parse,
run the grammar's injections.scm, parse each embedded region with the
injected language, and merge every layer's highlight spans. First
consumer is markdown fenced code + inline (zero new grammars).
Eleven numbered decisions (Q#IJ1-IJ11) and thirteen acceptance tests,
approved over three framing rounds. Load-bearing calls: the layer set
lives inside ParseTreeBundle installed atomically (Q#IJ1); a two-stage
worker/settle handoff keeps parsing off the main thread and query
caching in the Rc registry (Q#IJ2); recursion is depth/count/visited
bounded with child-only failure (Q#IJ3); a case-folded alias map
snapshotted into ParseRequest bridges Lua-set aliases to the worker
(Q#IJ4); included ranges are Vec<Range> excluding children and
intersected with the parent (Q#IJ5); the wire producer flattens layers
into disjoint effective spans because the GPU re-sorts by start, and
the GPU first-span consumer is fixed to fold (Q#IJ6).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
Snapshot: main @ 8ce2e9c, protocol v16. Stage-1 summary + the two
substrate contracts the review rounds produced (snapshot/baseline
reset, store-sourced diag-count freeze); NEXT pointer at stage 2
(pmacs.gpu.set_font, protocol v17) and stage 3 (statusline
segments); themes deferrals consolidated into section 6; the
dual-purpose-session-state lesson into section 5.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VoiEyuPjoBhvwACf8HAnLB
The round-4 freeze read counted the retained for_uri vector inside
status_facts_msg -- correct, but StatusFacts runs at frame cadence
for every semantic session under the shared store mutex, so a long
stale interval cost O(frames x diagnostics x sessions).
DiagnosticStore now maintains per-URI severity totals alongside
by_uri: set replaces them (an empty publication removes them with
the vector), clear removes them, and mark_stale deliberately
preserves both -- entries exist exactly when by_uri entries do, and
set/clear are the store's only by_uri mutators. status_facts_msg
reads the tuple in O(1), and the all-URI severity_totals sum reuses
the cached tuples.
Store unit (acceptance item 34) pins the invariant: all four totals
replace correctly, survive staleness, and clear with the diagnostic
vector; empty_set_clears_uri asserts the totals drop too. The
rounds 3-4 freeze acceptance passes unchanged -- behavior parity,
so the unit pin is the evidence (no runtime bite exists for a
behavior-preserving refactor). Round-5 implementation
user-authored; this commit folds it with framing revision 9, the
protocol doc's O(1) note, and the acceptance manifest pointer to
the item-34 unit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VoiEyuPjoBhvwACf8HAnLB
A session first rendering during diagnostic staleness emitted zero
counts: round 3's frozen_diag_counts cache was per-session and only
seeded from fresh computations, so a late joiner attaching mid-edit
(or a buffer first visited between didChange and fresh diagnostics)
had no entry and fell back to (0, 0), contradicting the documented
"frozen counts, never zeros" contract.
mark_stale (T M11.8) keeps the last published diagnostic vector --
only the positions are invalid -- so status_facts_msg now counts
the retained for_uri entries even while stale: the retained entries
ARE the frozen value. The per-session cache is deleted; sourcing
the freeze from the store means there is no session state to lose
to a snapshot reset and no history needed at attach, so the round-3
reset-survival property holds by construction and its round-trip
test passes unchanged against the new mechanism.
Acceptance item 33 marks a populated store stale BEFORE the
SemanticRenderState exists and asserts the first frame reports the
preserved counts; runtime bite vs pre-fix semantic_render.rs.
Framing revision 8; the protocol doc's freeze sentence now says
store knowledge, including the late-joiner case.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VoiEyuPjoBhvwACf8HAnLB
Finding 1: the round-2 reset contract was asymmetric. The producer
resets search/menu/status baselines on every snapshot send, but the
GPU's BufferSnapshot arm only cleared spans, decorations,
adornments, summary, and the completion popup -- a menu or search
open at switch time survived the snapshot with no close message
ever coming (the new buffer's first CLOSED state is suppressed
daemon-side), leaving a stale popup that also held
daemon_intercepts_keys true and swallowed pointer events
indefinitely. The arm now clears search_prompt, menu, and
status_facts; the minibuffer is deliberately exempt on both sides
(one global core instance, matching the producer's surviving
last_minibuffer baseline). GPU test opens search + menu + status
via the real wire arms, applies a snapshot, and asserts all three
clear, the intercept gate releases, and the popup pixels vanish --
hand-bitten by disabling the three clears (fix and test share
main.rs).
Finding 2: the round-2 reset broke the diagnostic-count freeze.
last_status was both the peer emission baseline and the
stale-store freeze source, so a snapshot between didChange and
fresh diagnostics re-shipped StatusFacts with zeroed counts. The
freeze source now lives apart: frozen_diag_counts advances on every
fresh count, is read when the store is stale, and survives
on_buffer_snapshot_sent -- which keeps killing the emission
baseline to force the re-send. Acceptance renders (1,1), marks the
store stale, applies the reset, and asserts the re-sent StatusFacts
still carries (1,1); runtime bite vs pre-fix semantic_render.rs
fails exactly as predicted (Some((0,0)) vs Some((1,1))).
Framing revision 7; acceptance items 31-32; the protocol doc's
snapshot-reset paragraph now lists the full frontend drop set and
names the count freeze as daemon knowledge, not peer state.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VoiEyuPjoBhvwACf8HAnLB
Finding 1: a BufferSnapshot wipes the frontend's buffer-scoped
render state (spans, decorations, adornments, minimap summary,
completion popup), but the producer's per-buffer emission baselines
survived the switch -- on an unchanged A -> B -> A round trip,
last_summary[A]'s key still matched and the daemon emitted nothing,
so the frontend never regained A's themed minimap (or A's
StatusFacts: the band kept B's name) until an edit, republish, or
theme mutation happened to move the key.
The fix is the general contract, not a minimap special case:
SemanticRenderState::on_buffer_snapshot_sent(buffer_id) kills every
buffer-scoped baseline for that buffer (spans + style gate,
decorations, adornments, summary, status, search/menu prompts,
completion popup), called wherever the daemon writes a snapshot --
the active-buffer-follow path and the F29 upgrade broadcast; the
attach bootstrap constructs its session state fresh. Deliberately
surviving: the bufferless ThemeFacts pair, the global minibuffer
baseline, the per-frontend gutter mode, the revision-keyed diag
line cache, and other buffers' baselines.
Evidence: a producer round-trip acceptance test (themed summary and
StatusFacts return at the SAME generation; identical payload), a
real-daemon wire test driving A -> B -> A via dispatched keys
(runtime bite: times out against pre-fix daemon.rs), a Rust unit
pinning the reset's scope, and a GPU test where the re-shipped
summary restores the first visit's pixels exactly (frontend half --
no GPU code change, coverage only). The semantic_render.rs bite is
compile-fail (the hook is absent pre-fix), disclosed as weaker.
The protocol doc's composition section now states the snapshot
reset contract on both sides of the wire.
Finding 2: the acceptance-suite manifest header now lists the true
item split (1-19, 24-26, 28-29 here; 20-23, 27, 30 in the GPU
suite). Framing revision 6 folds the round; acceptance items 28-30.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VoiEyuPjoBhvwACf8HAnLB
Finding 1: accepting a FileStyleSummary drops the GPU minimap vertex
cache -- theme recolors and diagnostic republishes arrive at an
unchanged generation, and the cache keys only on (generation, dims,
scroll), so stale strokes survived until an edit/resize/scroll. The
daemon payload-suppresses identical summaries, so the invalidation
is precise. GPU test drives two same-generation summaries;
hand-bitten by reverting the single invalidation line (script-bite
is vacuous here: fix and test share main.rs).
Finding 2: lua_to_style propagates every Table::get error -- the
lookups run __index, so a raising metatable previously parsed as an
all-default style and the merge SUCCEEDED, committing valid siblings
against the Q#TH6 all-or-nothing contract. Boolean fields keep Lua
truthiness by design (mlua bool), so only raising lookups fail the
transaction. Acceptance reproduces the reviewer's trap shape;
runtime bite vs pre-fix mod.rs.
Finding 3: SemanticRenderState::for_peer records the negotiated
version; below v16 no ThemeFacts is produced and no ui.diag.* face
folds into the FileStyleSummary marks -- the summary is an ungated
pre-v16 channel, and a v15 peer must not get face-derived minimap
colors while its other severity surfaces stay unthemed. The summary
cache key zeroes its face-epoch component for such peers. Acceptance
drives v15/v16 producers side by side; compile-fail bite disclosed
(the test needs for_peer, absent pre-fix).
Finding 4: framing revision 5 weakens the canonical-severity claim
to what is true -- the daemon-RESOLVED color is canonical, while the
GPU's built-in squiggle/sign/counter defaults are historical bright
RGBs that differ from the minimap's converted Indexed marks, a
pre-existing divergence kept because unset faces must render
byte-identically to before.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VoiEyuPjoBhvwACf8HAnLB
Two theme.lock() calls inside one tuple expression self-deadlock:
the first guard lives to the end of the full expression, so the
second lock blocks the same thread forever. Observed as an 84-minute
hung lib suite; one lock per observation via a helper closure.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VoiEyuPjoBhvwACf8HAnLB
- §1: `main` @ 7d1b6d6; new arc bullet covering the grammars (every
LSP-configured language now has one), the extension → filetype →
filename → shebang detection chain and its user-extensible Lua
surfaces, the dockerfile/cmake LSP configs, and the substrate
(`highlights_query: &[&str]` base-first composition; fail-closed
`#is?`/`#is-not? local` handling in `compute_highlight_spans`).
Condensed the merged compile-mode/editops/auto-pairing blocks to
substrate essentials. Fixed the stale "#114 is the USER'S lane" note:
themes (Arc 4) is the user's active `theme-faces` lane.
- §3: folded the confirmed flakes (m6_1_pty, composition_overhead) into
the rerun-isolated note, plus the single-invocation sweep caveat.
- §6: new deferrals — locals-query processing, multi-language
injections (blocks notebooks / markdown code / HTML-in-JS), modeline
detection, JSON/YAML, byte-accurate cursor; a full Jupyter setup is a
real arc, not a one-shot.
Docs only. #118 is approved, merge pending. git diff --check clean.
Non-blocking review nit: the shadowing test only proved the `.builtin`
captures disappeared. Also assert both `console` occurrences retain an
ordinary `@variable` capture, so dropping the locals-predicate pattern
loses only the `.builtin` refinement, not all styling for the token.
Test-only.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
[P2] The shared JavaScript highlights query guards its builtin captures
(console, require, …) with `#is-not? local`, a PROPERTY predicate
(`Query::property_predicates`) that needs a scope map from the grammar's
LOCALS_QUERY — which pmacs does not run. `compute_highlight_spans` took
every capture, so a locally-shadowed `console`/`require` still surfaced
as `@variable.builtin`/`@function.builtin`; a theme distinguishing
`.builtin` would mis-style the shadowed local.
Full locals processing is substrate work; conservatively fail-closed
instead: drop captures whose pattern carries an `#is?`/`#is-not? local`
property predicate (the identifier falls back to its non-builtin
capture). The text predicates (`#eq?`/`#match?`/`#any-of?`, already
applied by the capture iterator) and `#set!` settings are untouched.
This is a general engine fix — it corrects the same latent mis-styling
for any grammar using the locals predicate, not just JS/TS.
- javascript_shadowed_builtin_is_not_mislabeled: a local `const console`
produces no `*.builtin` capture (directly observed to fail — two
`variable.builtin` captures — before the fix).
[P3] Comments this PR invalidated: `lsp.lua` no longer claims Python has
no grammar; `syntax.lua`'s `_has_language` gate comment uses a
still-grammarless example (an init.lua `shebangs.ruby`) instead of
python/javascript; and the rewritten Ruby shebang test's doc no longer
describes it as a Python test.
Gates: fmt; clippy -D warnings; test --lib; --features crdt;
m4_acceptance --skip basedpyright; GPU; full workspace sweep;
git diff --check.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
These five languages already had LSP configs (basedpyright, gopls,
tsserver, taplo, zls) but shipped no tree-sitter grammar, so they
rendered with no lexical color. Fill the gap — 8 BUILTIN_LANGUAGES
entries across 6 crates:
- python (`tree-sitter-python`, root `module`), go (`tree-sitter-go`),
toml (`tree-sitter-toml-ng`), zig (`tree-sitter-zig`, +`.zon`) — each a
single self-contained highlights query.
- JavaScript/TypeScript family: `tree-sitter-javascript` parses both
`.js` and `.jsx`; `tree-sitter-typescript` ships two grammars
(LANGUAGE_TYPESCRIPT, LANGUAGE_TSX). The four entries — javascript,
javascriptreact, typescript, typescriptreact — mirror the LSP filetype
map so tsserver enables the JSX parser. Highlights inherit: the TS
query is a ~5-capture delta over JavaScript and JSX is a further
delta, so the entries compose base-first (js → jsx → ts), the same
pattern as `cuda` over C/C++ (typescript resolves ~22 capture classes,
typescriptreact ~24).
Each grammar's name equals its existing `pmacs.lsp.config.<name>` key,
so grammar detection (which wins over the filetype map) resolves the id
the server keys off — the file now gets BOTH highlighting and the right
server. No lsp.lua change needed. All crates ride `tree-sitter-language
0.1` with tree-sitter dev-only — no second core in the graph.
Bite-verified acceptance:
- gap_grammars_load_and_parse — each grammar's ABI accepted by the 0.26
core; a snippet parses without error at its root (covers both TS
grammars, incl. JSX).
- typescript_highlights_compose_the_javascript_base — the compiled
typescript/typescriptreact queries resolve >= 15 captures, not just the
~5-capture TS delta (the JS base is really composed in).
- builtin_languages_include_gap_grammars /
gap_grammar_extensions_resolve — entry presence + extension detection
across all 8 ids.
- m4_gap_grammars_align_with_lsp_configs — through the loaded runtime,
each path's grammar id matches an existing LSP config. Bite-verified
against pre-feature src/syntax.rs.
Ripple: two #116 shebang tests used python as their "has-LSP-but-no-
grammar" example, which this PR invalidates. Updated both — the .py +
`#!/bin/sh` precedence test now asserts a python grammar tree (not "no
tree"), and the grammarless-language-is-silent gate test switches to
`ruby` (genuinely grammarless) via a test-local shebang mapping.
Gates: fmt; clippy -D warnings; test --lib; --features crdt;
m4_acceptance --skip basedpyright; GPU; full workspace sweep;
git diff --check.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
cmake-language-server does NOT pull a `workspace/configuration` section:
it reads `buildDirectory` from the `initialize` request's
`initializationOptions`, and drives its project model off CMake's File
API under `<buildDirectory>/.cmake/api/` (not `compile_commands.json`).
The `settings = { cmake = {} }` block — and the documented
`settings.cmake.buildDirectory` override — were therefore inert, leaving
conventional out-of-source project data unavailable.
Replace it with `init_options = { buildDirectory = "build" }` (the
conventional out-of-source dir; users override `init_options.buildDirectory`
from init.lua), and correct the comment. The wiring test now asserts
`config.cmake.init_options.buildDirectory == "build"` — bite-verified
against the pre-fix lsp.lua.
Gates: fmt; clippy -D warnings; --features crdt (1720); m4_acceptance
--skip basedpyright (109); GPU (59); full workspace sweep (zero
failures); git diff --check — all green. One `--lib` run flaked on
process::m6_1_pty_mode_lifecycle_started_then_exited (PTY-lifecycle
timing, the m6/m8 daemon-timing family); it passed in the crdt run, the
full sweep, and 4/4 isolated — unrelated to this Lua config change.
Change is Lua config + the acceptance assertion.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
Files identified by their whole basename — Dockerfile, Makefile,
CMakeLists.txt, rc dotfiles — had no detection path (extension-, then
shebang-keyed). Add a filename layer and the three grammars behind it.
- **Grammars** (BUILTIN_LANGUAGES): dockerfile via `tree-sitter-containerfile`
(the ABI-current grammar; the old `tree-sitter-dockerfile` pins
`tree-sitter ^0.20` and would fork the graph — containerfile rides
`tree-sitter-language 0.1`, tree-sitter dev-only, like the others),
make via `tree-sitter-make`, cmake via `tree-sitter-cmake`. All ship
self-contained highlights (single fragment). Extensions:
`.dockerfile`/`.containerfile`, `.mk`/`.make`, `.cmake`.
- **Filename layer**: `pmacs.parse.language_from_filename(name)` backed by
an extensible `pmacs.parse.filenames` map, wired into the precedence
chain in both syntax.lua (grammar) and lsp.lua (LSP): grammar-ext →
filetype map → filename → shebang. A recognized extension still wins;
the basename map only fires when the extension misses. Seeds the three
filenames plus shell rc dotfiles (`.bashrc`/`.zshrc`/`PKGBUILD`/… →
bash) — highlighting them against the grammar shipped in #115.
- **LSP**: `config.dockerfile` (docker-langserver --stdio) and
`config.cmake` (cmake-language-server). Make has no server, so no
`config.make` — grammar highlight only. Extension filetype fallbacks
added for id stability.
Bite-verified acceptance:
- filename_grammars_load_and_parse — each grammar's ABI is accepted by
the tree-sitter 0.26 core and parses a representative snippet without
error (dockerfile/cmake root at source_file, make at makefile).
- builtin_languages_include_dockerfile_make_cmake /
language_for_path_resolves_dockerfile_make_cmake_extensions — entry
presence and extension detection.
- m4_filename_map_resolves_special_files — the basename map (incl. path
form and dotfiles→bash), config.dockerfile/cmake commands, and no
config.make. Bite-verified against pre-feature syntax.lua.
- m4_filename_extensionless_dockerfile_highlights — an extensionless
`Dockerfile` resolves to dockerfile for LSP and gets a dockerfile parse
tree; reachable only via the filename map. Bite-verified.
Gates: fmt; clippy -D warnings; test --lib; --features crdt;
m4_acceptance --skip basedpyright; GPU; full workspace sweep;
git diff --check.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
Syntax.rs theme-handle line refs re-verified; the new shell grammar
introduces no ui-named capture, so the Q#TH2 reservation holds.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VoiEyuPjoBhvwACf8HAnLB
Arc 4 stage 1 (docs/theme-faces-framing.md, revision 4). Faces are
theme entries under the reserved ui/ui.* namespace -- zero new Lua
API. Theme::face() resolves with the dotted-prefix walk but never
falls back to default_style; each face applies owns-surface within
its stage-1 component mask, identical on both frontends.
Substrate: two monotonic theme mutation counters (syntax/face) with
transactional set/merge/clear/default (parse before locking, commit
all-or-nothing, bump from the prior value); the StyleGate and the
minimap summary key on the counters -- fixing the pre-existing bug
where a mid-session pmacs.theme.set never re-shipped StyleSpans --
with the summary gaining payload-equality suppression that still
advances its key on computation.
Wire: InstanceMessage::ThemeFacts appended after CompletionPopup
(postcard discriminants are ordinal; a byte pin guards placement),
PROTOCOL_VERSION 15 -> 16, daemon-gated >= 16, one authoritative
table per attachment (None-seeded baselines), TUI silent-drop arm.
Grid: paint_frame resolves ui.modeline / ui.statusline /
ui.minibuffer(.candidate) / ui.gutter / ui.selection faces;
SearchView and DiagnosticView take the theme handle through the real
attachment paths (EditorCore injection, install_diag threading); the
canonical severity color resolves ui.diag.* with the Default ->
built-in policy that keeps the minimap presence encoding sound.
GPU: exact-name face table applied per draw with the Q#TH5 Default
mapping (plain text / window bg, reverse swap), local/peer wash
split, candidate-dropdown glyph site, and the status-band
shaping-cache invalidation without which a diag-face recolor with
constant counts kept stale counter colors.
Tests: 18-test acceptance suite (grid, wire, daemon gate, atomicity,
monotonicity, late join), 7 GPU headless tests incl. decoded vertex
colors, units for the face walk / transactional commits / producer
caches; protocol pins for v16 + the CompletionPopup byte pin.
Bites vs 3cbb9de (scripts/bite): semantic_render.rs (8 runtime test
failures), editor.rs (5 runtime), daemon.rs (v15 gate, runtime);
lua_bindings/mod.rs, pmacs-gpu/main.rs, search.rs, diag.rs, and
highlight.rs bite as compile failures (weaker evidence, disclosed).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VoiEyuPjoBhvwACf8HAnLB
The attached split-string payload (`-Spython3`, `--split-string=...`) can
itself begin with env options or VAR=value assignments before the
interpreter: `-S-i python3`, `-SFOO=bar python3`,
`--split-string=-u FOO python3`. Rather than taking the payload's first
word as the interpreter, re-inject the attached payload into the token
stream so it flows through the same option / operand / assignment state
machine as a separated payload. Adds the three cases as resolver tests.
Two follow-ups from review, both in builtin/runtime/syntax.lua.
1. [P2] Buffer switching bypassed the pinned grammar. after-edit already
reparsed the pinned language, but the after-switch reattach path
(attach_for_active_buffer) re-resolved from scratch — so open an
extensionless `#!/bin/sh` (bash), edit its shebang to lua, switch away
and back, and the grammar flipped to lua while the LSP side kept its
bash attachment (lsp.lua's after-switch reuses the existing record).
attach_for_active_buffer now reuses the language pinned at first attach
whenever a parse view already exists; only a first-seen buffer
resolves. A language change still needs a close/reopen, matching both
the after-edit behavior and how extensions work.
2. [P2] Attached `env -S`/`--split-string` forms failed. The walk skipped
the whole option token, but for split-string the interpreter rides
inside it: `-Spython3`, `-vSpython3` (after no-operand short flags
i/v/0), and `--split-string=python3` all resolved to nil (the last was
also eaten by the earlier `=` branch). The env walk now extracts the
interpreter from the attached value (`^-[iv0]*S(.+)$` /
`^--split-string=(.+)$`); the separated forms (`-S python3`) still work
by walking on to the next token.
Tests (bite-verified against the round-1 syntax.lua — both fail there;
scripts/bite HEAD builtin/runtime/syntax.lua):
- m4_shebang_edit_keeps_pinned_grammar now adds a switch-away/back cycle
(via pmacs.window.switch_buffer, which fires after-switch
synchronously) and asserts the tree stays bash.
- m4_shebang_resolver_maps_interpreters adds the attached split-string
cases (`-Spython3`, `--split-string=python3`, `-vSpython3`).
Gates: fmt; clippy -D warnings; m4_acceptance --skip basedpyright; GPU;
git diff --check green. Only-known-flake caveat as round 1
(editor::composition_overhead_under_ten_percent perf microbenchmark,
unrelated to this Lua change). Change is Lua-only plus the acceptance
tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
Three review findings, all in builtin/runtime/syntax.lua.
1. [P1] Syntax bypassed extension precedence and grammar availability.
attach_for_active_buffer resolved `language_for_path or shebang`, but
language_for_path knows only grammar-backed extensions — so a `.py`
file opening with `#!/bin/sh` fell through to the shebang and got a
bash parse tree, and an extensionless `#!/usr/bin/env python3` script
dispatched "python" (no grammar) and raised "unknown language". A new
resolve_active_language walks the full precedence chain — grammar
extension -> LSP filetype map -> shebang — consulting the shebang only
when the extension is unrecognized (a recognized non-grammar extension
like .py is authoritative). Dispatch is then gated on
pmacs.parse._has_language(lang), so grammarless languages are skipped
silently. The extension parts stay keyed on buf:name() (unchanged from
before), so path-less buffers that resolve a grammar by name — e.g.
generated markdown buffers — are unaffected.
2. [P2] Editing an open script's shebang left parsing/highlighting stale.
The after-edit path re-sniffed the mutable shebang: sh -> python
raised "unknown language" while leaving the old bash tree, and
sh -> lua swapped the parse tree under a highlight overlay still
holding the original grammar's query. Reparse now uses the language
pinned at first attach (parse_lang_by_buffer), never re-resolving —
a language change needs a close/reopen, as it does for extensions.
3. [P2] `env` options with operands were mistaken for interpreters.
`#!/usr/bin/env -u FOO python3` skipped `-u` but took `FOO`. The env
walk now skips the operand of the operand-consuming GNU-env options
(-u/--unset, -C/--chdir, -a/--argv0) before selecting the interpreter.
-S/--split-string stays excluded (its string carries the interpreter).
Tests (bite-verified against pre-fix syntax.lua — each fails without its
fix; scripts/bite HEAD builtin/runtime/syntax.lua):
- m4_shebang_does_not_override_extension now also asserts _has_view is
false (no bash grammar tree for a `.py` + `#!/bin/sh`), not only the
LSP language.
- m4_shebang_extensionless_grammarless_language_is_silent — extensionless
python resolves for LSP, gets no grammar view, and records no error.
- m4_shebang_edit_keeps_pinned_grammar — rewriting a `#!/bin/sh` script's
shebang to lua keeps the bash tree and reports no error.
- m4_shebang_resolver_maps_interpreters — added the env-operand cases
(`-u FOO`, `-C /tmp`, combined).
Gates: fmt; clippy -D warnings; m4_acceptance --skip basedpyright; GPU;
git diff --check all green. The only sweep failure is the pre-existing
editor::composition_overhead_under_ten_percent render microbenchmark
(ratio hovers at the 1.10 cutoff; flakes ~1/3 even isolated single-
threaded, already asserted-off on macOS) — a pure-Rust render loop this
Lua-only change cannot touch. Change is Lua-only plus the acceptance
tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
Extension detection missed extensionless scripts — `scripts/deploy`, git
hooks, `configure`, and `scripts/bite` itself — so they got neither
highlighting nor an LSP server. Add a first-line shebang fallback.
- New `pmacs.parse.language_from_shebang(buf)` (builtin/runtime/syntax.lua):
sniffs the first line (capped at 256 bytes), maps the interpreter's
basename to a language, and resolves the `#!/usr/bin/env python3`
indirection (skipping env's own `-S`/flags and `VAR=val` assignments).
Backed by `pmacs.parse.shebangs`, a user-extensible map seeded with the
interpreters pmacs can act on: sh-family -> bash, python* -> python,
node -> javascript, lua* -> lua.
- Wired as a strict *fallback* on both resolution paths: syntax.lua's
grammar attach (`language_for_path or language_from_shebang`) and
lsp.lua's `buffer_language` (grammar -> filetypes -> shebang). A
recognized extension always wins, so a `.py`/`.sh` file is never
re-classified by a stray shebang.
- Cross-language, not shell-only: `#!/usr/bin/env python` /`node` /`lua`
resolve too. Special filenames (`.bashrc`, `Dockerfile`, `Makefile`)
are intentionally deferred until there are grammars behind them.
Bite-verified acceptance (tests/m4_acceptance.rs):
- m4_shebang_resolver_maps_interpreters — the mapping incl. env
indirection and `env -S`; non-shebangs and unmapped interpreters
(ruby) resolve to nil.
- m4_shebang_extensionless_script_resolves_bash — opening an
extensionless `#!/bin/sh` script resolves to bash on BOTH paths:
lsp.lua's `active_buffer_language()` and a settled bash parse tree
(grammar attach). Reachable only via the shebang, since the file has
no extension.
- m4_shebang_does_not_override_extension — a `.py` file opening with
`#!/bin/sh` still resolves to python (extension precedence).
Gates green: fmt; clippy -D warnings; test --lib; --features crdt;
m4_acceptance --skip basedpyright; GPU; full workspace sweep;
git diff --check. Change is Lua-only plus the acceptance tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
Levi Neuwirth